Getting OAuth settings from env var with full stop in key - docker

I am using docker linux container to run my servicestack application and I need to be able to read the OAuth keys from environment variables defined in my docker-compose.yml.
It appears impossible to do this due to the full stop in the variable name.
For example in OAuthProvider.cs (https://github.com/ServiceStack/ServiceStack/blob/master/src/ServiceStack/Auth/OAuthProvider.cs) line 26:
this.ConsumerKey = appSettings.GetString($"oauth.{oAuthProvider}.{consumerKeyName}");
It is reading for example the key oauth.google.ConsumerKey.
Linux doesn't support full stops in environment variables. Using the debugger I can see that if I put the variables in like:
environment:
- oauth.RedirectUrl=http://example.com
- oauth.CallbackUrl=http://example.com/auth/{0}
- oauth.basecamp.ConsumerKey=fgshghfdhfghgfdhf
- oauth.basecamp.ConsumerSecret=fdghfdghdfghgdfhdfghfgd
Then they are removed. I did some research and this is common issue, if the env var has a full stop then it gets removed. I cannot find any workaround for this.
Do you have any idea how I can pass these hard coded settings with docker environment variable?
Here is representation of my entire dockerfile for reference:
ARG BUILD_MODE=Release
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
RUN apt-get update && apt-get install openssh-server unzip -y
RUN curl -sSL https://aka.ms/getvsdbgsh | /bin/sh /dev/stdin -v latest -l ~/vsdbg
COPY sshd_config /etc/ssh/
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["Project/Project.csproj", "Project/"]
COPY ["Project.ServiceModel/Project.ServiceModel.csproj", "Project.ServiceModel/"]
COPY ["Project.ServiceInterface/Project.ServiceInterface.csproj", "Project.ServiceInterface/"]
COPY ["NuGet.config", "NuGet.config"]
RUN dotnet restore "Project/Project.csproj"
COPY . .
WORKDIR "/src/Project"
RUN dotnet build "Project.csproj" -c "$BUILD_MODE" -o /app/build
FROM build AS publish
RUN dotnet publish "Project.csproj" -c "$BUILD_MODE" -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT service ssh start && echo "root:$SSH_PASSWORD" | chpasswd && dotnet Project.dll
And this is docker-compose:
version: "3.7"
services:
project:
build:
context: E:\project\
dockerfile: E:\project\project\Dockerfile
image: project:local
ports:
- "57008:80"
- "57009:443"
- "57001:2222"
restart: always
depends_on:
- "db"
environment:
- oauth.RedirectUrl=http://example.com
- oauth.CallbackUrl=http://example.com/auth/{0}
- oauth.basecamp.ConsumerKey=fgshghfdhfghgfdhf
- oauth.basecamp.ConsumerSecret=fdghfdghdfghgdfhdfghfgd
db:
image: postgres:10.9
restart: always
environment:
POSTGRES_PASSWORD: fdgdfgdfgdf
POSTGRES_USER: project
ports:
- "5445:5432"

You can provide your own custom AppSettings provider or use the DictionarySettings and populate it with a mapped environment variable that's suitable to use in Docker, e.g:
Use underscore separators in Docker:
environment:
- oauth_RedirectUrl=http://example.com
- oauth_CallbackUrl=http://example.com/auth/{0}
- oauth_basecamp_ConsumerKey=fgshghfdhfghgfdhf
- oauth_basecamp_ConsumerSecret=fdghfdghdfghgdfhdfghfgd
Then create a new Dictionary AppSettings using the keys ServiceStack expects, e.g:
string env(string key) =>
Environment.GetEnvironmentVariable(key.Replace(".","_"));
var envSettings = new DictionarySettings(new Dictionary<string,string> {
["oauth.RedirectUrl"] = env("oauth.RedirectUrl"),
["oauth.CallbackUrl"] = env("oauth.CallbackUrl"),
["oauth.basecamp.ConsumerKey"] = env("oauth.basecamp.ConsumerKey"),
["oauth.basecamp.ConsumerSecret"] = env("oauth.basecamp.ConsumerSecret"),
});
Then use that in your Auth Providers, e.g:
new MyAuthProvider(envSettings)

Related

go/docker : Cannot burn value to binary executable using -X linker flag when using docker compose

I can't seem to figure out how to provide a build time variable using the docker-compose file. I have done quite a bit of research and still can't figure the way out. What I'm trying to do is to burn my latest commit hash as a value to the version variable in my binary executable which is used by my server. I'm using windows as a side note. Here's my dockerfile
# Build Stage
FROM golang:1.18-alpine3.16 AS builder
WORKDIR /app
COPY . .
ARG VERSION
ENV VERSION=$VERSION
RUN GOOS=linux GOARCH=amd64 go build -ldflags='-s -X main.version=$VERSION' -o main
./cmd/api
RUN apk add curl
RUN curl -L https://github.com/golang-
migrate/migrate/releases/download/v4.15.2/migrate.linux-amd64.tar.gz | tar xvz
# Run Stage
FROM alpine:3.16
WORKDIR /app
COPY --from=builder /app/main .
COPY --from=builder /app/migrate ./migrate
COPY app.env .
COPY start.sh .
COPY wait-for.sh .
COPY /migrations ./migration
EXPOSE 3001
CMD [ "/app/main" ]
ENTRYPOINT [ "/app/start.sh" ]
The version variable should be having my latest commit hash. Here's my docker-compose file
version: "3.9"
services:
postgres:
build:
context: .
dockerfile: db.dockerfile
environment:
- POSTGRES_USER=root
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=blogpost
ports:
- "542:5432"
api:
build:
context: .
dockerfile: app.dockerfile
args:
- VERSION=git describe --always --dirty <<< Trying to pass value here
ports:
- "3001:3001"
environment:
- DB_DSN=postgres://root:secret#postgres:5432/blogpost?sslmode=disable
depends_on:
- postgres
entrypoint: [ "/app/wait-for.sh", "postgres:5432", "--", "/app/start.sh" ]
command: [ "/app/main" ]
I have tried this too
args:
- VERSION=$$(shell git describe --always --dirty)
In the end, the value for my version variable ends up being "$VERSION". Please guide me on this

restoring database from dockerfile

I'm trying to restore database from dockerfile, I'm dockerizing api and database, when I run docker-compose up --build I got error
how can I fix this, I'm sending dockerfile and docker-compose.yaml below, If anyone has some idea how to load database please tell me, any help will be welcome. Kind regards and thank you all
DOCKERFILE
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 5192
ENV ASPNETCORE_URLS=http://+:5192
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY . .
FROM build AS publish
RUN dotnet publish "e-Res/e-Res.csproj" -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
COPY ./e-Res/Uploads/Images ./Uploads/Images
COPY ["e-Res/ERes.bak", "var/opt/mssql/data"]
RUN (/opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Starting database restore" && /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'QWElkj132!' -Q "RESTORE FILELISTONLY FROM DISK='/var/opt/mssql/data/ERes.bak';"
DOCKER-COMPOSE.YAML
version: '3'
services:
#mssql docker
eres-sql:
image: mcr.microsoft.com/mssql/server:2017-latest-ubuntu
restart: unless-stopped
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=QWElkj132!
- MSSQL_PID=Developer
ports:
- 1401:1433
expose:
- 1433
networks:
- eresnet2022
eres-api:
restart: unless-stopped
build:
context: .
environment:
- ConnectionStrings:DefaultConnection=Server=eres-sql,1433;Database=eres;User=sa;Password=QWElkj132!;ConnectRetryCount=0
- ASPNETCORE_ENVIRONMENT=Development
ports:
- 5192:5192
networks:
- eresnet2022
links:
- eres-sql
depends_on:
- eres-sql
networks:
eresnet2022:
driver: bridge
ENTRYPOINT ["dotnet", "e-Res.dll"]
You cannot create the Db when building the image but you have to wait until the container has started.
You need to create another SQL Server container with a custom startup, or execute the restore command after the container has started
FROM mcr.microsoft.com/mssql/server:2017-latest-ubuntu
# You can also pass them from docker-compose file
EXPOSE 1433
ENV ACCEPT_EULA y
ENV SA_PASSWORD QWElkj132!
#
COPY ["e-Res/ERes.bak", "var/opt/mssql/data"]
# Init scripts
COPY ./init.sql .
COPY ./entrypoint.sh .
CMD /bin/bash ./entrypoint.sh
On the init.sql you have the Db restore command
RESTORE FILELISTONLY FROM DISK='/var/opt/mssql/data/ERes.bak';
On the entrypoint.sh you have the boot point
#!/bin/bash
# Start the init.sql script (you can also put your command here and remove the init.sql file)
# -l 50 means wait 50 seconds before the timeout (you can adjust this value) this is the time necessary to SQL server to startup (see below)
/opt/mssql-tools/bin/sqlcmd -S localhost -l 50 -U SA -P "QWElkj132!" -i init.sql &
# Start SQL server
/opt/mssql/bin/sqlservr
Remove the last two rows from the Dockerfile
COPY ["e-Res/ERes.bak", "var/opt/mssql/data"]
RUN (/opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Starting database restore" && /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P 'QWElkj132!' -Q "RESTORE FILELISTONLY FROM DISK='/var/opt/mssql/data/ERes.bak';"

.NET6 and docker: Couldn't find a valid ICU package installed on the system

I'm trying to containerize a .NET6 WebApi with a postgrescontainer. I have this Docker file to build the web api image:
#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["MicroserviceTemplateDDD.csproj", "MicroserviceTemplateDDD/"]
RUN dotnet restore "MicroserviceTemplateDDD/MicroserviceTemplateDDD.csproj"
WORKDIR "/src/MicroserviceTemplateDDD"
COPY . .
RUN dotnet build "MicroserviceTemplateDDD.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "MicroserviceTemplateDDD.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
WORKDIR /app/publish
ENTRYPOINT ["dotnet", "MicroserviceTemplateDDD.dll"]
This runs successfully. But when i run docker-compose up to start the postgre and webapi container i get this error:
microservicetemplateddd_service1 | Process terminated. Couldn't find a valid ICU package installed on the system. Please install libicu using your package manager and try again. Alternatively you can set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support. Please see https://aka.ms/dotnet-missing-libicu for more information.
My docker compose:
version: "3.7"
services:
api:
image: microservicetemplateddd_service1
container_name: microservicetemplateddd_service1
restart: always
build:
context: .
dockerfile: Dockerfile
environment:
- ConnectionStrings:Context=Server=MicroserviceTemplateDDD_Service1_database;Database=Database;User Id=pa;Password=P4ssW0rd!;
- DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
depends_on:
- database
networks:
- network
ports:
- 8090:80
database:
image: postgres
container_name: microservicetemplateddd_service1_database
restart: always
environment:
- ACCEPT_EULA=Y
- POSTGRES_PASSWORD=P4ssW0rd!
- POSTGRES_USER=pa
networks:
- network
ports:
- 1433:1433
volumes:
- database:/var/opt/mssql
networks:
network:
volumes:
database:
I already tried to use other ms images like alpine and change the place of setting the env.
Per the docs, you should set the environment variable to 1 to disable globalization. Like this
environment:
- ConnectionStrings:Context=Server=MicroserviceTemplateDDD_Service1_database;Database=Database;User Id=pa;Password=P4ssW0rd!;
- DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
Since you always need to set it to run your container, it'll be a good idea to put the configuration into the image. You can put it at the beginning of the dockerfile and remove it from the docker-compose file
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
WORKDIR /app
EXPOSE 80
EXPOSE 443
That way you can't forget to set it and have the error pop up again.
I had a similar issue and got around it by adding this condition in the csproj
<ItemGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
<RuntimeHostConfigurationOption Include="System.Globalization.AppLocalIcu" Value="68.2" />
</ItemGroup>

Cannot assign requested address (localhost:xxxx) - Docker + Linux Containers

We have a WPF application that will call a Web API called API-1 which is running in a docker container. I also have another API called API-2 which is also running in a docker but a different container.
Whenever the API-1 call happened from the application, the API-1 will do some logic and try to do a post request to API-2.
Now, my problem is, the post request to API-2 always returns Cannot assign requested address (localhost:XXXX).
If I try without docker, it works fine.
Also, a separate request to each of the API works fine (using POSTMAN)
This problem occurs only if the API deployed in the docker. I was using docker-compose to create the containers. I have created a docker network bridge and allocated it to the respective APIs in the docker-compose.yml file.
Here is my docker-compose file as well as the docker file for both of the APIs.
Docker-Compose.yml
version: "3.7"
networks :
localdev:
name: localdev
external: true
services:
api-01:
build:
context: .
dockerfile: api-01/Dockerfile
restart: always
container_name: "api-01"
ports:
- "8082:80"
environment:
ASPNETCORE_ENVIRONMENT: "Development"
networks:
- localdev
api-02:
build:
context: .
dockerfile: api-02/Dockerfile
restart: always
container_name: "api-02"
ports:
- "8083:80"
environment:
ASPNETCORE_ENVIRONMENT: "Development"
networks:
- localdev
DockerFile(API-1)
FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build
WORKDIR /src
COPY ["api-01/api-01.csproj", "api-01/"]
COPY ["CommonEntities/CommonEntities.csproj", "CommonEntities/"]
RUN dotnet restore "api-01/api-01.csproj"
COPY . .
WORKDIR "/src/api-01"
RUN dotnet build "api-01.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "api-01.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "api-01.dll"]
DockerFile(API-2)
FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build
WORKDIR /src
COPY ["api-02/api-02.csproj", "api-02/"]
COPY ["CommonEntities/CommonEntities.csproj", "CommonEntities/"]
RUN dotnet restore "api-02/api-02.csproj"
COPY . .
WORKDIR "/src/api-02"
RUN dotnet build "api-02.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "api-02.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "api-02.dll"]
I'm not sure what I'm missing here!
I have tried most of the solutions from the internet, but nothing works.
It looks like I need to use {DOCKER_IP} instead of "localhost" if I want to have communication between two API in two different containers.
After I changed my post request to hit "http://{My_PC_IP_Address}:8082", the request was successful.
Thanks to David for redirecting me to the right documentation https://docs.docker.com/network/bridge/

Environment variable not being set in Dockerfile

I have a project with 2 Dockerfiles, one for backend and one for database. The dockerfiles are in separate folders. I build the two images with a docker-compose.yml file that looks like this:
version: "3.7"
services:
dotnet-backend:
container_name: dotnet-backend
build: .
env_file: .env
links:
- mssql-db
ports:
- "8000:80"
mssql-db:
container_name: mssql-db
build: ./Database
env_file: .env
volumes:
- ./Database/:/scripts/
ports:
- "1433:1433"
expose:
- "1433"
command:
- /bin/bash
- -c
- |
/opt/mssql/bin/sqlservr &
sleep infinity
As you can see, I use a .env file for environment variables, this file is located in the src folder, same as the Dockerfile for the backend and the rest of the backend code.
This is my Dockerfile for backend:
FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build-env
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# Copy everything else and build
COPY . ./
RUN dotnet publish -c Debug -o out MyProject.csproj
# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0
WORKDIR /app
COPY --from=build-env /app/out .
EXPOSE 80
ENV ASPNETCORE_ENVIRONMENT=$ASPNETCORE_ENVIRONMENT
ENTRYPOINT ["dotnet", "MyProject.dll"]
This is my Dockerfile for the database (located in ./Database):
FROM mcr.microsoft.com/mssql/server:2017-latest
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=$SA_PASSWORD
ENV MSSQL_PID=Developer
ENV MSSQL_TCP_PORT=1433
WORKDIR /src
COPY ./ /scripts/
EXPOSE 1433
RUN (/opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" && sleep 5s && (for foo in /scripts/*.sql;do /opt/mssql-tools/bin/sqlcmd -S127.0.0.1 -Usa -P$SA_PASSWORD -i$foo;done)
And finally my .env file:
SA_PASSWORD=SomePassword
ASPNETCORE_ENVIRONMENT=Development
ConnectionStrings__My_db=Data Source=tcp:mssql-db,1433;Initial Catalog=DevDB;User ID=sa;Password=SomePassword
What's strange is that the ASPNETCORE_ENVIRONMENT variable gets set, as I can tell by testing my API that it is running in development mode, and the connection string from the .env file also gets set as my API can connect to my database when I manually enter the password. But the SA_PASSWORD environment variable does not get set. Here is the output from the docker-compose up --build command:
So why are the other variables and connection strings getting set, but not the password? Everything works fine if I replace ENV SA_PASSWORD=$SA_PASSWORD with ENV SA_PASSWORD=SomePassword in the database Dockerfile.
you don't need to set the env vars in the docker images at all, and you can fix the issue using the changes below
ASPNETCORE_ENVIRONMENT works because the app only check it at runtime and not at build time
database docker file
FROM mcr.microsoft.com/mssql/server:2017-latest
ENV ACCEPT_EULA=Y
ENV MSSQL_PID=Developer
ENV MSSQL_TCP_PORT=1433
WORKDIR /src
COPY ./ /scripts/
EXPOSE 1433
COPY ./docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
docker-entrypoint.sh (chmod a+x)
#!/bin/bash -e
(/opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" && sleep 5s && (for foo in /scripts/*.sql;do /opt/mssql-tools/bin/sqlcmd -S127.0.0.1 -Usa -P$SA_PASSWORD -i$foo;done)
exec "$#"
exit 0
app docker file
FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build-env
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# Copy everything else and build
COPY . ./
RUN dotnet publish -c Debug -o out MyProject.csproj
# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0
WORKDIR /app
COPY --from=build-env /app/out .
EXPOSE 80
ENTRYPOINT ["dotnet", "MyProject.dll"]
you can move also these to the env file and it is recommended to separate the env files for the containers
ENV ACCEPT_EULA=Y
ENV MSSQL_PID=Developer
ENV MSSQL_TCP_PORT=1433
That isn't a valid way to take a var from environment.
The variable $SA_PASSWORD in Dockerfile is actually a var into scope of Dockerfile and build image time.
The var hasn't value because the way of assign it, is through ARG command (https://docs.docker.com/engine/reference/builder/#arg).
Everyway, if you need declare a SA_PASSWORD and read it of environment at container runtime, you must do it trough ENTRYPOINT. ENTRYPOINT is executed when container launch.

Resources