I'm hoping that I've just missed something terribly obvious, but here's the situation I'm faced with.
Problem
Running docker-compose on windows after following docker-compose install steps from the website
docker-compose.yml file works fine on unix systems (have tested on Mac)
Currently fails immediately on Windows when it cannot locate any Dockerfiles for the services defined in the yml file. Here's the error:
NOTE: The image above might be a bit confusing. The environment variable below is called GOPATH, but the folder on my colleague's computer is also called GOPATH. This gives the impression that the env var isn't set correctly, but it is indeed.
version: '3'
services:
renopost:
depends_on:
- reno-cassandra
- reno-kafka
- reno-consul
build:
context: ${GOPATH}/src/renopost
dockerfile: ${GOPATH}/src/renopost/docker/dev/Dockerfile
container_name: renopost
image: renopost
ports:
- "4000:4000"
volumes:
- ${GOPATH}/src/renopost:/go/src/renopost
Above is a snippet of the docker-compose.yml file that is being run. The GOPATH env variable is indeed set and when following the directory path listed, I can confirm the file exists in that location.
Is there some interaction here with the OracleBoxVM that boot2docker uses where it isn't actually finding that file?
Related
I am trying to setup a cluster of 3 RavenDB instances using docker-compose and I am having problems with the RavenDB server not picking up the values in the RAVEN_ environment variables.
At first, I was running a single instance, using this docker-compose file:
version: '3'
services:
ravendb:
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
- "38888:38888"
volumes:
- ../data:/opt/RavenDB/Server/RavenData
With a simple Dockerfile that used the latest raven image and simply copied a settings.json file into the container.
FROM ravendb/ravendb
COPY settings.json /opt/RavenDB/Server/settings.json
{
"License.Eula.Accepted": true,
"License": {/*License here*/},
"Setup.Mode": "Unsecured",
"Security.UnsecuredAccessAllowed": "PublicNetwork",
"ServerUrl": "http://0.0.0.0:8080",
"ServerUrl.Tcp": "tcp://0.0.0.0:38888"
}
Now that I am trying to setup 3 instances, I wanted to avoid this way of creating the containers, since I would have to have different Dockerfiles and a settings.json file for each one.
Therefore, I thought of using a single docker-compose file that creates three containers and configures each one with enviroment variables.
I started with a single instance, to see if any problems would arise:
version: '3'
services:
raven1:
container_name: raven1
image: ravendb/ravendb
ports:
- "8080:8080"
- "38888:38888"
environment:
- RAVEN_Security_UnsecuredAccessAllowed=PublicNetwork
- RAVEN_Setup_Mode=Unsecured
- RAVEN_License_Eula_Accepted=true
- "RAVEN_ServerUrl=http://0.0.0.0:8080"
- "RAVEN_ServerUrl_Tcp=tcp://0.0.0.0:38888"
volumes:
- ../data:/opt/RavenDB/Server/RavenData
And arise they did! Despite the environment variables being set correctly, they are not picked up by the server, and the settings.json file is the default one.
root#8ad95cc439d4:/opt/RavenDB/Server# env
RAVEN_ARGS=
RAVEN_Security_UnsecuredAccessAllowed=PublicNetwork
RAVEN_AUTO_INSTALL_CA=true
RAVEN_ServerUrl=http://0.0.0.0:8080
RAVEN_SETTINGS=
RAVEN_ServerUrl_Tcp=tcp://0.0.0.0:38888
RAVEN_IN_DOCKER=true
RAVEN_Setup_Mode=Unsecured
RAVEN_License_Eula_Accepted=true
RAVEN_DataDir=RavenData
root#8ad95cc439d4:/opt/RavenDB/Server# cat settings.json
{
"Security.UnsecuredAccessAllowed": "PrivateNetwork"
}
Any idea why this might be happening? I can't seem to find any mention of issues regarding this.
Do I understand correctly you expected the configuration from environment variables to be included in the settings.json file in the container?
If that's the case I would like to clarify that passing environment variables does not modify RavenDB's settings.json file. Instead RavenDB loads them up directly from the environment.
Configuration options are loaded in the following order of precedence:
command line arguments
settings.json configuration file
RAVEN_ prefixed environment variables
So if you wanted to override the configuration option Security.UnsecuredAccessAllowed found in settings.json file you would need to either change the file on the container or pass it as a CLI argument --Security.UnsecuredAccessAllowed PublicNetwork.
Both cases are supported by RavenDB docker images:
to clear the default settings.json you can pass RAVEN_SETTINGS={} environment variable to the container.
to pass command line arguments to the RavenDB server binary you can use RAVEN_ARGS environment variable. E.g. RAVEN_ARGS=--Security.UnsecuredAccessAllowed PublicNetwork
I am trying to deploy my django app to app engine using dockerfile and for that after following a few blogs such as these, I created a docker-compose.yml file but when I run the docker compose up command or docker-compose -f docker-compose-deploy.yml run --rm gcloud sh -c "gcloud app deploy" I get an error key cannot contain a space. See below:
For example:
$ docker compose up
key cannot contain a space
$ cat docker-compose.yml
version: '3.7'
services:
app:
build:
context: .
ports: ['8000:8000']
volumes: ['./app:/app']
Can someone please help me to fix this error? I have tried yamllint to validate the yaml file for any space/indentation type of error and it doesn't show any error to me.
EDIT:
Here is the content for file in the longer command:
version: '3.7'
services:
gcloud:
image: google/cloud-sdk:338.0.0
volumes:
- gcp-creds:/creds
- .:/app
working_dir: /app
environment:
- CLOUDSDK_CONFIG=/creds
volumes:
gcp-creds:
Ok this is resolved finally! After beating my head around, I was able to finally resolve this issue by doing the following things:
Unchecked the option to use "Docker Compose v2" from my docker desktop settings. Here is the setting in Docker Desktop
Closed the docker desktop app and restarted it.
Please try these steps in case you face the issue. Thanks!
Just adding another alt answer here that I confirmed worked for me when following the steps above did not. My case is slightly different, but as Google brought me here first I thought I'd leave a note.
Check your env var values for spaces!
This may only be applicable if you are using env_var files (appreciate that OP is not in the minimal example, hence saying this is different).
Unescaped spaces in variables will cause this cryptic error message.
So, given a compose file like this:
version: '3.7'
services:
gcloud:
image: google/cloud-sdk:338.0.0
volumes:
- gcp-creds:/creds
- .:/app
working_dir: /app
env_file:
- some_env_file.env
If some_env_file.env looks like this:
MY_VAR=some string with spaces
then we get the cryptic key cannot contain a space.
If instead we change some_env_file.env to be like this:
MY_VAR="some string with spaces"
then all is well.
The issue has been reported to docker-compose.
Google brought me here first, and when your suggestion sadly didn't work for me, it then took me to this reddit thread, where I found out the above.
Docker Compose (at least since v2) automatically parses .env files before processing the docker-compose.yml file, regardless of any env_file setting within the yaml file. If any of the variables inside your .env file contains spaces, then you will get the error key cannot contain a space.
Two workarounds exist at this time:
Rename your .env file to something else, or
Create an alternate/empty .env file, e.g. .env.docker and then explicitly set the --env-file parameter, i.e. docker compose --env-file .env.docker config.
Track the related issues here:
https://github.com/docker/compose/issues/6741
https://github.com/docker/compose/issues/8736
https://github.com/docker/compose/issues/6951
https://github.com/docker/compose/issues/4642
https://github.com/docker/compose/commit/ed18cefc040f66bb7f5f5c9f7b141cbd3afbbc89
https://docs.docker.com/compose/env-file/
One more thing to be care about - since Compose V2, this error may be raise in case you have inline comments in the env file used by Compose. For example
---
version: "3.7"
services:
backend:
build:
context: .
dockerfile: Dockerfile
env_file: .app.env
and that .app.env is like this
RABBIT_USER=user # RabbitMQ user
the same error may occur. To fix just move comment to its own line
# RabbitMQ user
RABBIT_USER=user
Good morning,
I am currently try to figure out how I create File shares on Ubuntu as the host OS for docker. On Windows and OSX you can set up Filesharing as below:
I require access to the File share in my docker-compose as an example see below:
version: '3.9'
services:
node_gauc:
image: node-g:v1
ports:
- "444:444" # https test port
volumes:
- ./NodeServer/cert/https.crt:/usr/share/node/cert/https.crt
- ./NodeServer/cert/key.pem:/usr/share/node/cert/key.pem
build:
context: .
dockerfile: ./NodeServer/dockerfile
restart: unless-stopped
container_name: node-g
If I don't have access when I build and start the container I get the following issues:
ERROR: for node-g Cannot start service node_g: error while creating mount source path '/usr/share/t/work/6b37be0079afed03/NodeServer/cert/https.crt': mkdir /usr/share/t: read-only file system
ERROR: for node_g Cannot start service node_g: error while creating mount source path '/usr/share/t/work/6b37be0079afed03/NodeServer/cert/https.crt': mkdir /usr/share/t: read-only file system
ERROR: Encountered errors while bringing up the project.
I am still unsure why its trying to create a directory but I suppose that is another matter.
Is it possible to create File share on a Ubuntu host server similar to what you can on OSX(Mac) or Windows OS?
Many thanks for your help
I am relatively new to docker. I have been trying to compose the file below:
version: "3"
services:
postgres:
restart: always
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_DB=test_db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD_FILE=/run/secrets/postgres-passwd
volumes:
- test_db:${PWD}
pgweb:
restart: always
image: sosedoff/pgweb
ports:
- "8081:8081"
environment:
- DATABASE_URL=postgres://postgres:POSTGRES_USER#POSTGRES_PASSWORD_FILE:5432/POSTGRES_DB?sslmode=disable
depends_on:
- postgres
volumes:
test_db:
What I am trying to do is mount the volume test_db to my current working directory by using the environment variable $PWD. When I run docker-compose up in my terminal I get the following warning:
The PWD variable is not set. Defaulting to a blank string.
Now it is important to note that I am currently using Ubuntu running on WSL2 on windows 10. Another thing to note is that I am running ZSH and not BASH.
I followed the exact steps mentioned in the documentation.
I also checked another question which seemed to be similar to mine but not quite the same, as it was possible to replace ${PWD} with ./ which simply does not work in my case.
When using ./ instead of $PWD I get the following error:
for pg_test_postgres_1 Cannot create container for service postgres:\
invalid volume specification: 'pg_test_test_db:.:rw': invalid mount config\
for type "volume": invalid mount path: '.' mount path must be absolute
If you are trying to see what you can an cannot do, this is something you cannot do. Docker does not load an environment variable that would normally be set by the shell, zsh or bash, doesn't matter. And yes, it's the shell that sets $PWD and $OLDPWD. Docker CAN define a variable that will be passed to the distro as an environment variable and also be used by Docker at the time of the container build. Also volumes need to be defined using absolute paths.
Also, like David Maze mentions, your PostgreSQL data folder needs to be specifically in /var/lib/postgresql/data or that folder needs to be symlinked to a different arbitrary folder where PostgreSQL has read-write access. The point of a container is to build it for your needs so under normal circumstances you should know where everything goes and set volumes' paths explicitly.
I want to pass environment variables to the applications containerized in Docker through docker-compose. This is a VS 2017 15.3 solution Tools for Docker.
In my docker.compose.yml file I have:
app.web:
image: app.web
env_file:
- ./path.to.project/config.env
build:
context: ./path.to.project
dockerfile: Dockerfile
In config.env I have:
TEST=Compose
But when I try to read the variables using Environment.GetEnvironmentVariable("TEST"); I always get null.
If I set a non-existent file in env_file it complains when I run it, so I give for granted that is locating the file.
If I set the variable this way:
app.web:
image: app.web
environment:
- TEST=ComposeLiteral
build:
context: ./path.to.project
dockerfile: Dockerfile
I get "ComposeLiteral" when evaluating "TEST".
Which is the correct way of passing a file with environment variables to the application?
The problem is that the config.env file I was using was starting with the UTF8 BOM, making text editors to show me the same content, but causing docker compose to read something else.
When you add a text file through Visual Studio, the BOM is added.
I created an example project with the problem.
Then in the log is possible to see how the wrong parsing is happening:
config.env:
environment:
NUGET_FALLBACK_PACKAGES: /root/.nuget/fallbackpackages
"\uFEFFTEST": Compose
config2.env
environment:
NUGET_FALLBACK_PACKAGES: /root/.nuget/fallbackpackages
TEST: Compose
https://github.com/docker/compose/issues/5220
I am not able to reproduce your problem. If I start with your Dockerfile:
version: "2"
services:
app.web:
image: app.web
env_file:
- ./path.to.project/config.env
build:
context: ./path.to.project
dockerfile: Dockerfile
And in path.to.project I use the following Dockerfile:
FROM alpine
RUN apk add --update python3
COPY dumpenv.py /dumpenv.py
CMD python3 /dumpenv.py
Which runs a simple web server that does nothing but dump environment variables:
import http.server
import os
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.end_headers()
content = []
for k, v in os.environ.items():
content.append('{:20} {}'.format(k, v))
self.wfile.write(bytes('\n'.join(content), 'utf-8'))
return
server = http.server.HTTPServer(('0.0.0.0', 8080), Handler)
server.serve_forever()
If I docker-compose up this environment and then connect to the service, I see:
PWD /
TEST ComposeLiteral
HOSTNAME 5388e6e2717a
SHLVL 1
HOME /root
PATH ...
You can find this complete example online here. If you see different behavior using this same example, could you update your question to include a complete reproducer?