.NET Docker Containers connect each other using NetMQ - docker

I have two .NET core web APIs as docker containers. I want with NetMQ one to send messages and other listen to it. But I think I have some problems connectiong those two with tcp connection.
I use Docker compose.
version: '3.4'
services:
gateway:
image: ${DOCKER_REGISTRY-}gateway
build:
context: .
dockerfile: Gateway/Dockerfile
ports:
- 5000:80
pictureperfect:
image: ${DOCKER_REGISTRY-}pictureperfect
build:
context: .
dockerfile: PicturePerfect/Dockerfile
ports:
- 5001:80
- 5002:5002
I want this one to send message
private void Send() {
using (var requester = new RequestSocket("tcp://0.0.0.0:5002")) {
try {
requester.SendFrame("message from pp");
Console.WriteLine(requester.ReceiveFrameString());
} catch (Exception) {
throw;
}
}
}
This is the one that I want to be listening
public static void Main(string[] args) {
CreateHostBuilder(args).Build().Run();
using (var responder = new ResponseSocket()) {
responder.Bind("tcp://pictureperfect:5002");
while (true) {
Console.WriteLine(responder.ReceiveFrameString());
Thread.Sleep(1000);
responder.SendFrame("message from gateway");
}
}
}

Related

dotnet app in Docker container not seeing environment variables

I have a working dotnet application that I can run locally, as well, the same code runs in an azure web app. I have been able to containerize it. However, when I run it in the container it fails to read the environment variable:
Code to get/check environment variable in the controller:
public ReportController(ILogger<ReportController> logger, IConfiguration iconfig)
{
_logger = logger;
_config = iconfig;
_storageConnString = Environment.GetEnvironmentVariable("AzureWebJobsStorage");
_containerName = Environment.GetEnvironmentVariable("ReportContainer");
string CredentialConnectionString = Environment.GetEnvironmentVariable("CredentialConnectionString");
if(CredentialConnectionString == null)
{
throw new Exception("Credential connection string is null");
}
}
code in start up:
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
})
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddEnvironmentVariables();
});
```
my docker compose that is setting the variables:
services:
myreports:
image: myreports
build:
context: .
dockerfile: myreports/Dockerfile
ports: [5000:5000]
environment:
- "APPSETTINGS_AzureWebJobsStorage = DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=xxxx+xx/xx==;EndpointSuffix=core.windows.net"
- "APPSETTINGS_HarmonyConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Harmony;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "APPSETTINGS_CredentialConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "CredentialConnectionString = Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass"
- "APPSETTINGS_ReportContainer = taxdocuments"
As you can see I'm attempting both the APPSETTINGS_ prefix and not
but when I hit the port in the app the container returns:
myreports-1 | System.Exception: Credential connection string is null
the code works fine the in the app service getting the variables
You don't need to add APPSETTINGS_ in front of the variable names. What's causing the issue is the spaces around the equals sign in your docker-compose file. The quotes are not needed, so I'd remove them.
This should work
services:
myreports:
image: myreports
build:
context: .
dockerfile: myreports/Dockerfile
ports: [5000:5000]
environment:
- AzureWebJobsStorage=DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=xxxx+xx/xx==;EndpointSuffix=core.windows.net
- HarmonyConnectionString=Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Harmony;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass
- CredentialConnectionString=Data Source=mydb.database.windows.net;AttachDbFilename=;Initial Catalog=Credential;Integrated Security=False;Persist Security Info=False;User ID=sqlreporter;Password=mypass
- ReportContainer=taxdocuments

DbContext.Database.Migrate() not providing password to the backend

My goal is to create seeds of users when the database is created.
I'm using idserver4, with npgsql, docker-compose.
The current behavior creates the database and as well the identityserver user manager tables (AspNetUsers, AspNetUserTokens, AspNetUserRoles, etc..). So I know it's migrating that data to the database. But it skips over the Task of running the User seed because it throws a password exception:
Npgsql.NpgsqlException (0x80004005): No password has been provided but the backend requires one (in MD5)
Here's the code in my Program.cs.
public static void Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
using (var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
try
{
var userManager = services.GetRequiredService<UserManager<User>>();
var roleManager = services.GetRequiredService<RoleManager<IdentityRole>>();
var context = services.GetRequiredService<ApplicationDbContext>();
context.Database.Migrate(); // ERROR HAPPENS HERE
Task.Run(async () => await UserAndRoleSeeder.SeedUsersAndRoles(roleManager, userManager)).Wait(); // I NEED THIS TO RUN
}
catch (Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "Error has occured while migrating to the database.");
}
}
host.Run();
}
Here is the code where it gets the connection string in Startup.cs:
services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseNpgsql(Configuration.GetConnectionString("DefaultConnection"),
b =>
{
b.MigrationsAssembly("GLFManager.App");
});
});
If I use a breakpoint here, it shows that the connection string was obtained along with the user id and password. I verified the password was correct. Or else I don't think it would initially commit the Idserver user manager tables.
Here is my appsettings.json file where the connection string lives:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"AllowedHosts": "*",
"ConnectionStrings": {
"DefaultConnection": "Host=localhost;Port=33010;Database=glfdb;User Id=devdbuser;Password=devdbpassword"
}
}
I'm thinking it's somewhere in the docker-compose file where some configuration is not registering. This is the docker-compose file:
version: '3.4'
services:
glfmanager.api:
image: ${DOCKER_REGISTRY-}glfmanagerapi
container_name: "glfmanager.api"
build:
context: .
dockerfile: GLFManager.Api/Dockerfile
ports:
- "33000:80"
- "33001:443"
environment:
- ConnectionStrings__DefaultConnection=Server=glfmanager.db;Database=glfdb;User Id=devdbuser:password=devdbpassword;
- Identity_Authority=http://glfmanager.auth
volumes:
- .:/usr/src/app
depends_on:
- "glfmanager.db"
glfmanager.auth:
image: ${DOCKER_REGISTRY-}glfmanagerauth
container_name: "glfmanager.auth"
build:
context: .
dockerfile: GLFManager.Auth/Dockerfile
ports:
- "33005:80"
- "33006:443"
environment:
- ConnectionStrings__DefaultConnection=Server=glfmanager.db;Database=glfdb;User Id=devdbuser:password=devdbpassword;
volumes:
- .:/usr/src/app
depends_on:
- "glfmanager.db"
glfmanager.db:
restart: on-failure
image: "mdillon/postgis:11"
container_name: "glfmanager.db"
environment:
- POSTGRES_USER=devdbuser
- POSTGRES_DB=glfdb
- POSTGRES_PASSWORD=devdbpassword
volumes:
- glfmanager-db:/var/lib/postresql/data
ports:
- "33010:5432"
volumes:
glfmanager-db:
I used this code from a class I took on backend developing and the code is Identitcal to the project I've built in that, and it works. So I'm stumped as to why this is giving me that password error.
Found the problem. I used a ':' instead of ';' in my docker file between User Id and password

cannot access service from another service from docker-compose

I am having below docker-compose script
version: '3.1'
services:
generator:
image: my-registry:55000/gen:ci-8
ports:
- "8080:80"
mail:
image: mailhog/mailhog
ports:
- "8025:8025"
integration:
image: my-registry:55000/gen:integration-9
build: .
from integration service, I am calling generator service as below
public const string GeneratorApiRoot = "http://generator:80";
var client = new HttpClient();
var sendEmail = new HttpRequestMessage
{
Method = HttpMethod.Post,
RequestUri = new Uri($"{GeneratorApiRoot}/EmailRandomNames")
};
using (var response = await client.SendAsync(sendEmail))
{
response.EnsureSuccessStatusCode();
}
I am getting status code 404 not found error msg. But when I access from host with http://localhost:8080/EmailRandomNames I am getting 200 status code.
What am I doing wrong? Please help.

Accessing chrome devtools protocol in docker grid

My tests are running against a docker grid with selenium docker images for hub and chrome. What I am trying to do is access chrome devtools protocols in the chrome node so that I can access/intercept a request.Any help is appreciated
I was able to get it working without docker in my local. But could not figure out a way to connect the devtools in chrome node of docker grid. Below is my docker-compose and code
docker compose
version: "3.7"
services:
selenium_hub_ix:
container_name: selenium_hub_ix
image: selenium/hub:latest
environment:
SE_OPTS: "-port 4445"
ports:
- 4445:4445
chrome_ix:
image: selenium/node-chrome-debug:latest
container_name: chrome_node_ix
depends_on:
- selenium_hub_ix
ports:
- 5905:5900
- 5903:5555
- 9222:9222
environment:
- no_proxy=localhost
- HUB_PORT_4444_TCP_ADDR=selenium_hub_ix
- HUB_PORT_4444_TCP_PORT=4445
- NODE_MAX_INSTANCES=5
- NODE_MAX_SESSION=5
- TZ=America/Chicago
volumes:
- /dev/shm:/dev/shm
Here is sample code how I got it working in local without grid (chrome driver in my mac)
const CDP = require('chrome-remote-interface');
let webDriver = require("selenium-webdriver");
module.exports = {
async openBrowser() {
this.driver = await new webDriver.Builder().forBrowser("chrome").build();
let session = await this.driver.session_
let debuggerAddress = await session.caps_.map_.get("goog:chromeOptions").debuggerAddress;
let AddressString = debuggerAddress.split(":");
console.log(AddressString)
try {
const protocol = await CDP({
port: AddressString[1]
});
} catch (err) {
console.log(err.message)
const {
Network,
Fetch
} = protocol;
await Fetch.enable({
patterns: [{
urlPattern: "*",
}]
});
}
await Fetch.requestPaused(async ({
interceptionId,
request
}) => {
console.log(request)
})
}
return this.driver;
},
}
When it is grid, I just change the way build the driver to below
this.driver = await new webDriver.Builder().usingServer(process.env.SELENIUM_HUB_IP).withCapabilities(webDriver.Capabilities.chrome()).build();
With that I am getting the port number but could not create a CDP session and I get a connection refused error.

Masstransit in docker using Request/Response model, Request Consumer exception, host not found while responding

I'm quite new to Masstransit/RabbitMq and I encountered a problem cannot deal with.
I have a Rabbitmq server running in docker, also a small microservice in docker container which consumes an event. Beside this I run a windows service on the host machine, which has the task to send the event via the masstransit Request/Response model to the microservice. The interesting thing is that the event arrives to the consumer as supposed but when I try to response the context.RespondAsync from the consume method I get an exception
R-FAULT rabbitmq://autbus/exi_bus 80c60000-eca5-3065-0093-08d62a09d168 HwExi.Extensions.Events.ReservationCreateOrUpdateEvent HwExi.Api.Consumers.ReservationCrateOrUpdateConsumer(00:00:07.8902444) The host was not found for the specified address: rabbitmq://127.0.0.1/bus-SI-GEPE-HwService.Api-oddyyy8cwwagkoscbdmnwncfrg?durable=false&autodelete=true, MassTransit.EndpointNotFoundException: The host was not found for the specified address: rabbitmq://127.0.0.1/bus-SI-GEPE-HwService.Api-oddyyy8cwwagkoscbdmnwncfrg?durable=false&autodelete=true
I'm using this model to messaging between microservices without any problem and its working properly in another queue.
Here is the yaml of microservice / Bus
exiapi:
image: exiapi
build:
context: .
dockerfile: Service/HwExi.Api/Dockerfile
ports:
- "54542:80"
environment:
"BUS_USERNAME": "guest"
"BUS_PASSWORD": "guest"
"BUS_HOST": "rabbitmq://autbus"
"BUS_URL": "exi_bus"
autbus:
image: rabbitmq:3-management
hostname: autbus
ports:
- "15672:15672"
- "5672:5672"
- "5671:5671"
volumes:
- ~/rabbitmq:/var/lib/rabbitmq/mnesia
the config of the windows service:
"Bus": {
"Username": "guest",
"Password": "guest",
"Host": "rabbitmq://127.0.0.1",
"Url": "exi_bus"
},
The windows service connects like this:
var builder = new ContainerBuilder();
builder.Register(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rmq =>
{
var host = rmq.Host(new Uri(options.Value.Bus.Host), "/", h =>
{
h.Username(options.Value.Bus.Username);
h.Password(options.Value.Bus.Password);
});
rmq.ExchangeType = ExchangeType.Fanout;
});
}).As<IBusControl>().As<IBus>().As<IPublishEndpoint>().SingleInstance();
The microservice inside container connects like this
public static class BusExtension
{
public static void InitializeBus(this ContainerBuilder builder, Assembly assembly)
{
builder.Register(context =>
{
return Bus.Factory.CreateUsingRabbitMq(rmq =>
{
var host = rmq.Host(new Uri(Constants.Bus.Host), "/", h =>
{
h.Username(Constants.Bus.UserName);
h.Password(Constants.Bus.Password);
});
rmq.ExchangeType = ExchangeType.Fanout;
rmq.ReceiveEndpoint(host, Constants.Bus.Url, configurator =>
{
configurator.LoadFrom(context);
});
});
}).As<IBusControl>().As<IBus>().As<IPublishEndpoint>().SingleInstance();
builder.RegisterConsumers(assembly);
}
public static void StartBus(this IContainer container, IApplicationLifetime lifeTime)
{
var bus = container.Resolve<IBusControl>();
var busHandler = TaskUtil.Await(() => bus.StartAsync());
lifeTime.ApplicationStopped.Register(() => busHandler.Stop());
}
}
than windows service fires the event like this:
var reservation = ReservationRepository.Get(message.KeyId, message.KeySource);
var operation = await ReservationCreateOrUpdateClient.Request(new ReservationCreateOrUpdateEvent { Reservation = reservation });
if (!operation.Success)
{
Logger.LogError("Fatal error while sending reservation create or update message to exi web service");
return;
}
Finally the microservice catches the event like this.
public class ReservationCrateOrUpdateConsumer : IConsumer<ReservationCreateOrUpdateEvent>
{
public async Task Consume(ConsumeContext<ReservationCreateOrUpdateEvent> context)
{
await context.RespondAsync(new MessageOperationResult<bool>
{
Result = true,
Success = true
});
}
}
I'm using autofac to register the requestclient in windows service:
Timeout = TimeSpan.FromSeconds(20);
ServiceAddress = new Uri($"{Configurarion.Bus.Host}/{Configurarion.Bus.Url}");
builder.Register(c => new MessageRequestClient<ReservationCreateOrUpdateEvent, MessageOperationResult<bool>>(c.Resolve<IBus>(), ServiceAddress, Timeout))
.As<IRequestClient<ReservationCreateOrUpdateEvent, MessageOperationResult<bool>>>().SingleInstance();
Can anybody help debug this out? Also share opinion if this structure is a proper one, maybe I should use https for sending message from the client machine to my microservice environment, and convert it to the bus via a gateway or similar approach more suitable? Thanks

Resources