I have a working dotnet application that I can run locally, as well, the same code runs in an azure web app. I have been able to containerize it. However, when I run it in the container it fails to read the environment variable:
Code to get/check environment variable in the controller:
public ReportController(ILogger<ReportController> logger, IConfiguration iconfig)
{
_logger = logger;
_config = iconfig;
_storageConnString = Environment.GetEnvironmentVariable("AzureWebJobsStorage");
_containerName = Environment.GetEnvironmentVariable("ReportContainer");
string CredentialConnectionString = Environment.GetEnvironmentVariable("CredentialConnectionString");
if(CredentialConnectionString == null)
{
throw new Exception("Credential connection string is null");
}
}
code in start up:
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
})
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddEnvironmentVariables();
});
```
my docker compose that is setting the variables:
services:
myreports:
image: myreports
build:
context: .
dockerfile: myreports/Dockerfile
ports: [5000:5000]
environment:
- "APPSETTINGS_AzureWebJobsStorage = DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=xxxx+xx/xx==;EndpointSuffix=core.windows.net"
- "APPSETTINGS_HarmonyConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Harmony;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "APPSETTINGS_CredentialConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "CredentialConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "APPSETTINGS_ReportContainer = taxdocuments"
As you can see I'm attempting both the APPSETTINGS_ prefix and not
but when I hit the port in the app the container returns:
myreports-1 | System.Exception: Credential connection string is null
the code works fine the in the app service getting the variables
You don't need to add APPSETTINGS_ in front of the variable names. What's causing the issue is the spaces around the equals sign in your docker-compose file. The quotes are not needed, so I'd remove them.
This should work
services:
myreports:
image: myreports
build:
context: .
dockerfile: myreports/Dockerfile
ports: [5000:5000]
environment:
- AzureWebJobsStorage=DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=xxxx+xx/xx==;EndpointSuffix=core.windows.net
- HarmonyConnectionString=Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Harmony;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass
- CredentialConnectionString=Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass
- ReportContainer=taxdocuments
Related
I have two .NET core web APIs as docker containers. I want with NetMQ one to send messages and other listen to it. But I think I have some problems connectiong those two with tcp connection.
I use Docker compose.
version: '3.4'
services:
gateway:
image: ${DOCKER_REGISTRY-}gateway
build:
context: .
dockerfile: Gateway/Dockerfile
ports:
- 5000:80
pictureperfect:
image: ${DOCKER_REGISTRY-}pictureperfect
build:
context: .
dockerfile: PicturePerfect/Dockerfile
ports:
- 5001:80
- 5002:5002
I want this one to send message
private void Send() {
using (var requester = new RequestSocket("tcp://0.0.0.0:5002")) {
try {
requester.SendFrame("message from pp");
Console.WriteLine(requester.ReceiveFrameString());
} catch (Exception) {
throw;
}
}
}
This is the one that I want to be listening
public static void Main(string[] args) {
CreateHostBuilder(args).Build().Run();
using (var responder = new ResponseSocket()) {
responder.Bind("tcp://pictureperfect:5002");
while (true) {
Console.WriteLine(responder.ReceiveFrameString());
Thread.Sleep(1000);
responder.SendFrame("message from gateway");
}
}
}
My goal is to create seeds of users when the database is created.
I'm using idserver4, with npgsql, docker-compose.
The current behavior creates the database and as well the identityserver user manager tables (AspNetUsers, AspNetUserTokens, AspNetUserRoles, etc..). So I know it's migrating that data to the database. But it skips over the Task of running the User seed because it throws a password exception:
Npgsql.NpgsqlException (0x80004005): No password has been provided but the backend requires one (in MD5)
Here's the code in my Program.cs.
public static void Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
using (var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
try
{
var userManager = services.GetRequiredService<UserManager<User>>();
var roleManager = services.GetRequiredService<RoleManager<IdentityRole>>();
var context = services.GetRequiredService<ApplicationDbContext>();
context.Database.Migrate(); // ERROR HAPPENS HERE
Task.Run(async () => await UserAndRoleSeeder.SeedUsersAndRoles(roleManager, userManager)).Wait(); // I NEED THIS TO RUN
}
catch (Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "Error has occured while migrating to the database.");
}
}
host.Run();
}
Here is the code where it gets the connection string in Startup.cs:
services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseNpgsql(Configuration.GetConnectionString("DefaultConnection"),
b =>
{
b.MigrationsAssembly("GLFManager.App");
});
});
If I use a breakpoint here, it shows that the connection string was obtained along with the user id and password. I verified the password was correct. Or else I don't think it would initially commit the Idserver user manager tables.
Here is my appsettings.json file where the connection string lives:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"AllowedHosts": "*",
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=33010;Database=glfdb;User Id=devdbuser;Password=devdbpassword"
}
}
I'm thinking it's somewhere in the docker-compose file where some configuration is not registering. This is the docker-compose file:
version: '3.4'
services:
glfmanager.api:
image: ${DOCKER_REGISTRY-}glfmanagerapi
container_name: "glfmanager.api"
build:
context: .
dockerfile: GLFManager.Api/Dockerfile
ports:
- "33000:80"
- "33001:443"
environment:
- ConnectionStrings__DefaultConnection=Server=glfmanager.db;Database=glfdb;User Id=devdbuser:password=devdbpassword;
- Identity_Authority=http://glfmanager.auth
volumes:
- .:/usr/src/app
depends_on:
- "glfmanager.db"
glfmanager.auth:
image: ${DOCKER_REGISTRY-}glfmanagerauth
container_name: "glfmanager.auth"
build:
context: .
dockerfile: GLFManager.Auth/Dockerfile
ports:
- "33005:80"
- "33006:443"
environment:
- ConnectionStrings__DefaultConnection=Server=glfmanager.db;Database=glfdb;User Id=devdbuser:password=devdbpassword;
volumes:
- .:/usr/src/app
depends_on:
- "glfmanager.db"
glfmanager.db:
restart: on-failure
image: "mdillon/postgis:11"
container_name: "glfmanager.db"
environment:
- POSTGRES_USER=devdbuser
- POSTGRES_DB=glfdb
- POSTGRES_PASSWORD=devdbpassword
volumes:
- glfmanager-db:/var/lib/postresql/data
ports:
- "33010:5432"
volumes:
glfmanager-db:
I used this code from a class I took on backend developing and the code is Identitcal to the project I've built in that, and it works. So I'm stumped as to why this is giving me that password error.
Found the problem. I used a ':' instead of ';' in my docker file between User Id and password
I am working on a project where we would like to use IdentityServer4 as a token server and have other services authenticated within this token server. I have dev env on Windows using Docker and linux containers. I configured IdentityServer and it's working, I configured Api client and it's working but, when I configured MVC client to authenticate, it's failing to access token server through docker. OK, I realized that Docker works in a way of having external/internal ports, so I configured the api and mvc client this way.
MVC Client
services.AddAuthentication(opts =>
{
opts.DefaultScheme = "Cookies";
opts.DefaultChallengeScheme = "oidc";
})
.AddCookie("Cookies", opts =>
{
opts.SessionStore = new MemoryCacheTicketStore(
configuration.GetValue<int>("AppSettings:SessionTimeout"));
})
.AddOpenIdConnect("oidc", opts =>
{
opts.ResponseType = "code id_token";
opts.SignInScheme = CookieAuthenticationDefaults.AuthenticationScheme;
opts.ClientId = "Mvc.Application";
opts.ClientSecret = "Secret.Mvc.Application";
opts.Authority = "http://authorization.server/";
//opts.Authority = "http://localhost:5001/";
//opts.MetadataAddress = "http://authorization.server/";
opts.UsePkce = true;
opts.SaveTokens = true;
opts.RequireHttpsMetadata = false;
opts.GetClaimsFromUserInfoEndpoint = true;
opts.Scope.Add("offline_access");
opts.Scope.Add("Services.Business");
opts.ClaimActions.MapJsonKey("website", "website");
});
This part is working, because document discovery is working. However it'll fail to access http://authorization.server url, because it's internal container address, not external accessible through web browser. So I tried to set 2 different urls: MetadataAddress from which document from OpenId server should be fetched and Authority, where all Unauthorized requests are redirected. However when I set both MetadataAddress and Authority in OpenIdConnectOptions when calling AddOpenIdConnect, it'll use MetadataAddress instead of Authority. I checked logs, discovery of document is successfull, because I'm hitting http://authorization.server/.well-known..., but it's also initiating request to the IdentityServer to authenticate with the same url http://authorization.server/connect...
Api Client
services.AddAuthorization()
.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddIdentityServerAuthentication(opts =>
{
opts.RequireHttpsMetadata = false;
opts.ApiName = "Api.Services.Business";
opts.ApiSecret = "Secret.Api.Services.Business";
opts.Authority = "http://authorization.server/";
});
This it's working fine using the internal container address.
IdentityServer configuration
services.AddIdentityServer(opt =>
{
opt.IssuerUri = "http://authorization.server/";
})
.AddAspNetIdentity<User>()
.AddSigningCredential(Certificate.Get())
.AddProfileService<IdentityProfileService>()
.AddInMemoryApiResources(Configuration.ApiResources())
.AddInMemoryIdentityResources(Configuration.IdentityResources())
.AddInMemoryClients(Configuration.Clients());
Configuration.cs
public static IEnumerable<Client> Clients(string redirectUri, string allowedCorsOrigins)
{
return new List<Client>
{
new Client
{
ClientId = "Services.Business",
ClientName = "Api Business",
AllowedGrantTypes = GrantTypes.ResourceOwnerPassword,
AllowedScopes =
{
"Services.Business"
},
ClientSecrets =
{
new Secret("Secret.Services.Business".Sha256())
}
},
new Client
{
ClientId = "Mvc.Application",
ClientName = "Mvc Application",
RequireConsent = false,
AllowOfflineAccess = true,
AllowedGrantTypes = GrantTypes.Hybrid,
AllowedScopes =
{
"Services.Business",
IdentityServerConstants.StandardScopes.OpenId,
IdentityServerConstants.StandardScopes.Profile
},
ClientSecrets =
{
new Secret("Secret.Mvc.Application".Sha256())
},
RedirectUris =
{
$"{redirectUri}/signin-oidc"
},
PostLogoutRedirectUris =
{
$"{redirectUri}/signout-callback-oidc"
}
}
};
}
Docker-compose.yml
version: '3.4'
networks:
fmnetwork:
driver: bridge
services:
authorization.server:
image: authorization.server
container_name: svc.authorization.server
build:
context: .
dockerfile: Authorization.Server/Dockerfile
ports:
- "5000:80"
- "5100:443"
environment:
ASPNETCORE_HTTPS_PORT: 5100
ASPNETCORE_ENVIRONMENT: Staging
ASPNETCORE_URLS: "https://+;http://+"
ASPNETCORE_Kestrel__Certificates__Default__Password: "devcertaspnet"
ASPNETCORE_Kestrel__Certificates__Default__Path: /root/.dotnet/https/aspnetapp.pfx
depends_on:
- sql.server
volumes:
- D:\Docker\Data\Fm:/root/.dotnet/https
- D:\Docker\Data\Fm\Logs:/Fm.Logs
networks:
- fmnetwork
services.business:
image: services.business
container_name: api.services.business
build:
context: .
dockerfile: Services.Business/Dockerfile
ports:
- "5001:80"
- "5101:443"
environment:
ASPNETCORE_ENVIRONMENT: Staging
ASPNETCORE_HTTPS_PORT: 5101
ASPNETCORE_URLS: "https://+;http://+"
ASPNETCORE_Kestrel__Certificates__Default__Password: "devcertaspnet"
ASPNETCORE_Kestrel__Certificates__Default__Path: /root/.dotnet/https/aspnetapp.pfx
depends_on:
- sql.server
volumes:
- D:\Docker\Data\Fm:/root/.dotnet/https
- D:\Docker\Data\Fm\Logs:/Fm.Logs
networks:
- fmnetwork
mvc.application:
image: mvc.application
container_name: svc.mvc.application
build:
context: .
dockerfile: Mvc.Application/Dockerfile
ports:
- "5002:80"
- "5102:443"
environment:
ASPNETCORE_ENVIRONMENT: Staging
ASPNETCORE_HTTPS_PORT: 5102
ASPNETCORE_URLS: "https://+;http://+"
ASPNETCORE_Kestrel__Certificates__Default__Password: "devcertaspnet"
ASPNETCORE_Kestrel__Certificates__Default__Path: /root/.dotnet/https/aspnetapp.pfx
volumes:
- D:\Docker\Data\Fm:/root/.dotnet/https
- D:\Docker\Data\Fm\Logs:/Fm.Logs
networks:
- fmnetwork
I just faced this same issue and was able to solve it as follows.
Some things to keep in mind:
This is not an issue with Identity Server itself but with the mismatch between the internal Docker URL (http://authorization.server) that your container sees and the local host URL (http://localhost:5001) that your browser sees.
You should keep using the local URL for Identity Server (http://localhost:5001) and add a special case to handle the container to container communication.
The following fix is only for development when working with Docker (Docker Compose, Kubernetes), so ideally you should check for the environment (IsDevelopment extension method) so the code is not used in production.
IdentityServer configuration
services.AddIdentityServer(opt =>
{
if (Environment.IsDevelopment())
{
// It is not advisable to override this in production
opt.IssuerUri = "http://localhost:5001";
}
})
MVC Client
services.AddAuthentication(... /*Omitted for brevity*/)
.AddOpenIdConnect("oidc", opts =>
{
// Your working, production ready configuration goes here
// It is important this matches the local URL of your identity server, not the Docker internal URL
options.Authority = "http://localhost:5001";
if (Environment.IsDevelopment())
{
// This will allow the container to reach the discovery endpoint
opts.MetadataAddress = "http://authorization.server/.well-known/openid-configuration";
opts.RequireHttpsMetadata = false;
opts.Events.OnRedirectToIdentityProvider = context =>
{
// Intercept the redirection so the browser navigates to the right URL in your host
context.ProtocolMessage.IssuerAddress = "http://localhost:5001/connect/authorize";
return Task.CompletedTask;
};
}
})
You can tweak the code a little bit by passing said URLs via configuration.
My application uses Minio for S3-compatible object storage, and I'd like to use the Minio docker image in my integration tests via Testcontainers.
For some very basic tests, I run a GenericContainer using the minio/minio docker image and no configuration except MINIO_ACCESS_KEY and MINIO_SECRET_KEY. My tests then use Minio's Java Client SDK. These work fine and behave just like expected.
But for other integration tests, I need to set up separate users in Mino. As far as I can see, users can only be added to Minio using the Admin API, for which there is no Java client, only the minio/mc docker image (the mc CLI is not available in the minio/minio docker image used for the server).
On the command line, I can use the Admin API like this:
$ docker run --interactive --tty --detach --entrypoint=/bin/sh --name minio_admin minio/mc
The --interactive --tty is a bit of a hack to keep the container running so I can later run commands like this one:
$ docker exec --interactive --tty minio_admin mc admin user add ...
Using Testcontainers, I try to do the same like this:
public void testAdminApi() throws Exception {
GenericContainer mc = new GenericContainer("minio/mc")
.withCommand("/bin/sh")
.withCreateContainerCmdModifier(new Consumer<CreateContainerCmd>() {
#Override
public void accept(CreateContainerCmd cmd) {
cmd
.withAttachStdin(true)
.withStdinOpen(true)
.withTty(true);
}
});
mc.start();
log.info("mc is running: {}", mc.isRunning());
String command = "mc";
Container.ExecResult result = mc.execInContainer(command);
log.info("Executing command '{}' returned exit code '{}' and stdout '{}'", command, result.getExitCode(), result.getStdout());
assertEquals(0, result.getExitCode());
}
The logs show the container being started, but executing a command against it returns exit code 126 and claims it's in a stopped state:
[minio/mc:latest] - Starting container with ID: 4f96fc7583fe62290925472c4c6b329fbeb7a55b38a3c0ad41ee797db1431841
[minio/mc:latest] - Container minio/mc:latest is starting: 4f96fc7583fe62290925472c4c6b329fbeb7a55b38a3c0ad41ee797db1431841
[minio/mc:latest] - Container minio/mc:latest started
minio.MinioAdminTests - mc is running: true
org.testcontainers.containers.ExecInContainerPattern - /kind_volhard: Running "exec" command: mc
minio.MinioAdminTests - Executing command 'mc' returned exit code '126'
and stdout 'cannot exec in a stopped state: unknown'
java.lang.AssertionError: Expected: 0, Actual: 126
After fiddling around with this for hours, I'm running out of ideas. Can anyone help?
Thanks to #glebsts and #bsideup I was able to get my integration tests to work. Here's a minimal example of how to add a user:
public class MinioIntegrationTest {
private static final String ADMIN_ACCESS_KEY = "admin";
private static final String ADMIN_SECRET_KEY = "12345678";
private static final String USER_ACCESS_KEY = "bob";
private static final String USER_SECRET_KEY = "87654321";
private static GenericContainer minioServer;
private static String minioServerUrl;
#BeforeAll
static void setUp() throws Exception {
int port = 9000;
minioServer = new GenericContainer("minio/minio")
.withEnv("MINIO_ACCESS_KEY", ADMIN_ACCESS_KEY)
.withEnv("MINIO_SECRET_KEY", ADMIN_SECRET_KEY)
.withCommand("server /data")
.withExposedPorts(port)
.waitingFor(new HttpWaitStrategy()
.forPath("/minio/health/ready")
.forPort(port)
.withStartupTimeout(Duration.ofSeconds(10)));
minioServer.start();
Integer mappedPort = minioServer.getFirstMappedPort();
Testcontainers.exposeHostPorts(mappedPort);
minioServerUrl = String.format("http://%s:%s", minioServer.getContainerIpAddress(), mappedPort);
// Minio Java SDK uses s3v4 protocol by default, need to specify explicitly for mc
String cmdTpl = "mc config host add myminio http://host.testcontainers.internal:%s %s %s --api s3v4 && "
+ "mc admin user add myminio %s %s readwrite";
String cmd = String.format(cmdTpl, mappedPort, ADMIN_ACCESS_KEY, ADMIN_SECRET_KEY, USER_ACCESS_KEY, USER_SECRET_KEY);
GenericContainer mcContainer = new GenericContainer<>("minio/mc")
.withStartupCheckStrategy(new OneShotStartupCheckStrategy())
.withCreateContainerCmdModifier(containerCommand -> containerCommand
.withTty(true)
.withEntrypoint("/bin/sh", "-c", cmd));
mcContainer.start();
}
#Test
public void canCreateBucketWithAdminUser() throws Exception {
MinioClient client = new MinioClient(minioServerUrl, ADMIN_ACCESS_KEY, ADMIN_SECRET_KEY);
client.ignoreCertCheck();
String bucketName = "foo";
client.makeBucket(bucketName);
assertTrue(client.bucketExists(bucketName));
}
#Test
public void canCreateBucketWithNonAdminUser() throws Exception {
MinioClient client = new MinioClient(minioServerUrl, USER_ACCESS_KEY, USER_SECRET_KEY);
client.ignoreCertCheck();
String bucketName = "bar";
client.makeBucket(bucketName);
assertTrue(client.bucketExists(bucketName));
}
#AfterAll
static void shutDown() {
if (minioServer.isRunning()) {
minioServer.stop();
}
}
}
You could run an one-off container (use OneShotStartupCheckStrategy) with mc and withCommand("your command"), connected to the same network as the minio server you're running (see Networking).
As #bsideup suggested, you can use one-shot strategy, i.e. as in here.
UPD: added working test. Here is important to know that
When the container is launched, it executes entrypoint + command (this is Docker in general and has nothing to do with Testcontainers). Source from TC github
public class TempTest {
#Rule
public Network network = Network.newNetwork();
private String runMcCommand(String cmd) throws TimeoutException {
GenericContainer container = new GenericContainer<>("minio/mc")
.withCommand(cmd)
.withNetwork(network)
.withStartupCheckStrategy(new OneShotStartupCheckStrategy())
.withCreateContainerCmdModifier(command -> command.withTty(true));
container.start();
WaitingConsumer waitingConsumer = new WaitingConsumer();
ToStringConsumer toStringConsumer = new ToStringConsumer();
Consumer<OutputFrame> composedConsumer = toStringConsumer.andThen(waitingConsumer);
container.followOutput(composedConsumer);
waitingConsumer.waitUntilEnd(4, TimeUnit.SECONDS);
return toStringConsumer.toUtf8String();
}
private void showCommandOutput(String cmd) throws TimeoutException {
String res = runMcCommand(cmd);
System.out.printf("Cmd '%s' result:\n----\n%s\n----%n", cmd, res);
}
#Test
public void testAdminApi() throws Exception {
showCommandOutput("ls");
showCommandOutput("version");
}
}
Another option is to use content of dockerfile of minio/mc, which is small, modify executed command (one-off "mc" by default), and run own container once per test, which, compared to one-off container, will save some time if you need to execute multiple commands:
#Rule
public Network network = Network.newNetwork();
#Rule
public GenericContainer mc = new GenericContainer(new ImageFromDockerfile()
.withDockerfileFromBuilder(builder ->
builder
.from("alpine:3.7")
.run("apk add --no-cache ca-certificates && apk add --no-cache --virtual .build-deps curl && curl https://dl.minio.io/client/mc/release/linux-amd64/mc > /usr/bin/mc && chmod +x /usr/bin/mc && apk del .build-deps")
.cmd("/bin/sh", "-c", "while sleep 3600; do :; done")
.build())
)
.withNetwork(network);
public void myTest() {
mc.execInContainer("mc blah");
mc.execInContainer("mc foo");
}
Basically, it runs image with mc installed, and sleeps for 1h which is enough for your tests. While it runs, you can execute commands etc. After you finish, it is killed.
Your minio container can be in same network.
Minio with docker compose:
For those who are looking for s3 with minio object server integration test.
The current implementation is based on docker-compose.
The current implementation utilising AWS S3 client for CURD opertations
docker-compose file:
version: '3.7'
services:
minio-service:
image: quay.io/minio/minio
command: minio server /data
ports:
- "9000:9000"
environment:
MINIO_ROOT_USER: minio
MINIO_ROOT_PASSWORD: minio123
The actual IntegrationTest class:
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.S3Object;
import org.junit.jupiter.api.*;
import org.testcontainers.containers.DockerComposeContainer;
import java.io.File;
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
class MinioIntegrationTest {
private static final DockerComposeContainer minioContainer = new DockerComposeContainer<>(new File("src/test/resources/docker-compose.yml"))
.withExposedService("minio-service", 9000);
private static final String MINIO_ENDPOINT = "http://localhost:9000";
private static final String ACCESS_KEY = "minio";
private static final String SECRET_KEY = "minio123";
private AmazonS3 s3Client;
#BeforeAll
void setupMinio() {
minioContainer.start();
initializeS3Client();
}
#AfterAll
void closeMinio() {
minioContainer.close();
}
private void initializeS3Client() {
String name = Regions.US_EAST_1.getName();
AwsClientBuilder.EndpointConfiguration endpoint = new AwsClientBuilder.EndpointConfiguration(MINIO_ENDPOINT, name);
s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY)))
.withEndpointConfiguration(endpoint)
.withPathStyleAccessEnabled(true)
.build();
}
#Test
void shouldReturnActualContentBasedOnBucketName() throws Exception{
String bucketName = "test-bucket";
String key = "s3-test";
String content = "Minio Integration test";
s3Client.createBucket(bucketName);
s3Client.putObject(bucketName, key, content);
S3Object object = s3Client.getObject(bucketName, key);
byte[] actualContent = new byte[22];
object.getObjectContent().read(actualContent);
Assertions.assertEquals(content, new String(actualContent));
}
}
I'm quite new to Masstransit/RabbitMq and I encountered a problem cannot deal with.
I have a Rabbitmq server running in docker, also a small microservice in docker container which consumes an event. Beside this I run a windows service on the host machine, which has the task to send the event via the masstransit Request/Response model to the microservice. The interesting thing is that the event arrives to the consumer as supposed but when I try to response the context.RespondAsync from the consume method I get an exception
R-FAULT rabbitmq://autbus/exi_bus 80c60000-eca5-3065-0093-08d62a09d168 HwExi.Extensions.Events.ReservationCreateOrUpdateEvent HwExi.Api.Consumers.ReservationCrateOrUpdateConsumer(00:00:07.8902444) The host was not found for the specified address: rabbitmq://127.0.0.1/bus-SI-GEPE-HwService.Api-oddyyy8cwwagkoscbdmnwncfrg?durable=false&autodelete=true, MassTransit.EndpointNotFoundException: The host was not found for the specified address: rabbitmq://127.0.0.1/bus-SI-GEPE-HwService.Api-oddyyy8cwwagkoscbdmnwncfrg?durable=false&autodelete=true
I'm using this model to messaging between microservices without any problem and its working properly in another queue.
Here is the yaml of microservice / Bus
exiapi:
image: exiapi
build:
context: .
dockerfile: Service/HwExi.Api/Dockerfile
ports:
- "54542:80"
environment:
"BUS_USERNAME": "guest"
"BUS_PASSWORD": "guest"
"BUS_HOST": "rabbitmq://autbus"
"BUS_URL": "exi_bus"
autbus:
image: rabbitmq:3-management
hostname: autbus
ports:
- "15672:15672"
- "5672:5672"
- "5671:5671"
volumes:
- ~/rabbitmq:/var/lib/rabbitmq/mnesia
the config of the windows service:
"Bus": {
"Username": "guest",
"Password": "guest",
"Host": "rabbitmq://127.0.0.1",
"Url": "exi_bus"
},
The windows service connects like this:
var builder = new ContainerBuilder();
builder.Register(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rmq =>
{
var host = rmq.Host(new Uri(options.Value.Bus.Host), "/", h =>
{
h.Username(options.Value.Bus.Username);
h.Password(options.Value.Bus.Password);
});
rmq.ExchangeType = ExchangeType.Fanout;
});
}).As<IBusControl>().As<IBus>().As<IPublishEndpoint>().SingleInstance();
The microservice inside container connects like this
public static class BusExtension
{
public static void InitializeBus(this ContainerBuilder builder, Assembly assembly)
{
builder.Register(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rmq =>
{
var host = rmq.Host(new Uri(Constants.Bus.Host), "/", h =>
{
h.Username(Constants.Bus.UserName);
h.Password(Constants.Bus.Password);
});
rmq.ExchangeType = ExchangeType.Fanout;
rmq.ReceiveEndpoint(host, Constants.Bus.Url, configurator =>
{
configurator.LoadFrom(context);
});
});
}).As<IBusControl>().As<IBus>().As<IPublishEndpoint>().SingleInstance();
builder.RegisterConsumers(assembly);
}
public static void StartBus(this IContainer container, IApplicationLifetime lifeTime)
{
var bus = container.Resolve<IBusControl>();
var busHandler = TaskUtil.Await(() => bus.StartAsync());
lifeTime.ApplicationStopped.Register(() => busHandler.Stop());
}
}
than windows service fires the event like this:
var reservation = ReservationRepository.Get(message.KeyId, message.KeySource);
var operation = await ReservationCreateOrUpdateClient.Request(new ReservationCreateOrUpdateEvent { Reservation = reservation });
if (!operation.Success)
{
Logger.LogError("Fatal error while sending reservation create or update message to exi web service");
return;
}
Finally the microservice catches the event like this.
public class ReservationCrateOrUpdateConsumer : IConsumer<ReservationCreateOrUpdateEvent>
{
public async Task Consume(ConsumeContext<ReservationCreateOrUpdateEvent> context)
{
await context.RespondAsync(new MessageOperationResult<bool>
{
Result = true,
Success = true
});
}
}
I'm using autofac to register the requestclient in windows service:
Timeout = TimeSpan.FromSeconds(20);
ServiceAddress = new Uri($"{Configurarion.Bus.Host}/{Configurarion.Bus.Url}");
builder.Register(c => new MessageRequestClient<ReservationCreateOrUpdateEvent, MessageOperationResult<bool>>(c.Resolve<IBus>(), ServiceAddress, Timeout))
.As<IRequestClient<ReservationCreateOrUpdateEvent, MessageOperationResult<bool>>>().SingleInstance();
Can anybody help debug this out? Also share opinion if this structure is a proper one, maybe I should use https for sending message from the client machine to my microservice environment, and convert it to the bus via a gateway or similar approach more suitable? Thanks