I start to work with a new project using Docker and I can see there is no docker-compose.yml file there, but a few files like docker-compose.myname1.yml, docker-compose.myname2.yml instead. Trying to run docker-compose up I get an error:
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
So I wonder if that's possible for a Docker project to work without the docker-compose.yml file and if it is, what conditions should be met. Maybe that's the matter of some specific version or environment?
https://docs.docker.com/compose/reference/
"Options:
-f, --file FILE Specify an alternate compose file
(default: docker-compose.yml)"
so in your case that would be:
docker-compose -f dockcer-compose.myname1.yml up
docker-compose -f dockcer-compose.myname2.yml up
There are two different issues presented in your question:
The reason for getting an error
When running docker-compose up, you must be in a directory that contains docker-compose.yml (or '.yaml), as indicated by the error you are receiving.
The reason you are getting an error is either because there is no such file in the current directory, or (less likely) the environment variable COMPOSE_FILE is set to something else.
Different names for the docker-compose file
In order to use an alternative file, you can take one of these approaches:
Run docker-compose with the --file argument:
$ docker-compose --help
-f, --file FILE Specify an alternate compose file
(default: docker-compose.yml)
Set the COMPOSE_FILE environment variable prior to running any docker-compose command.
Related
I have a docker-compose.yml file that defines the volumes section like this:
volumes:
seqfs:
driver: azure_file
driver_opts:
share_name: seqtest
storage_account_name: stacctest
storage_account_key: ${STORAGE_ACCOUNT_KEY}
I am trying to pass in STORAGE_ACCOUNT_KEY during the build command:
docker-compose -f docker-compose.yml build --build-arg STORAGE_ACCOUNT_KEY="##########"
But an error is returned:
The STORAGE_ACCOUNT_KEY variable is not set. Defaulting to a blank string.
Please note I do not want to save STORAGE_ACCOUNT_KEY into a file such as .env for security reasons -- I want to pass it from the command line.
How can I pass an argument to the volumes section in my docker-compose.yml?
Regarding the error you get:
docker-compose -f docker-compose.yml build --build-arg STORAGE_ACCOUNT_KEY="##########"
But an error is returned:
The STORAGE_ACCOUNT_KEY variable is not set. Defaulting to a blank string.
the fact is that --build-arg only deals with ARG directives within the Dockerfile.
(BTW it is simpler to run docker-compose up --build
rather than running docker-compose build then docker-compose up)
the only solutions I can think about to achieve what you want are:
either put STORAGE_ACCOUNT_KEY=foobar in an .env file
and run docker-compose up --build
or put STORAGE_ACCOUNT_KEY=foobar in an other.env file
and run docker-compose --env-file=other.env up --build
(Note: the docker/compose syntax contains two different kinds of "env-file", for details on this subtlety, see this answer of mine: Pass variables from .env file to dockerfile through docker-compose.)
or run STORAGE_ACCOUNT_KEY=foobar docker-compose up --build
Concluding remarks:
You said you do not want to save STORAGE_ACCOUNT_KEY into a file such as .env for security reasons, so at first sight you would only accept solution 3. above.
However, note that it is unlikely that 3. is more secure if you assume you don't trust the environment of your docker host. Indeed, .env files may be protected thanks to file permissions (and .gitignore FWIW), while the foobar string above automatically leaks in the name of the running processes (try e.g. ps aux | grep STORAGE_ACCOUNT_KEY).
I have a simple docker-compose file which is used to launch my containers. I wish to have another yaml file which contains additional, optional containers. It can live in a separate directory. My goal is to find a way to force the namespace of the created swarms so they exist within the same network/use space so they can talk to each other.
compose1.yaml
services:
web:
build: .
compose2.yaml
services:
web1:
build: .
So if i run both of these they would be prepended with the folder they exist in, in my case: a, and b respectively.
I wanted to ensure that they flow together, despite not being in the same file hierarchy.
I have been coming over keywords in the docker-compse documents, and was not sure what the best way to do this in the yaml file would be, but noticed in the CLI, might be able up to update various names.
How does one accomplish this?
Note: I have also created a third file under the b directory, a sibling to compose2.yaml. So i can run those separately and they work just fine.
a/
compose.yaml
b/
compose2.yaml
another.yaml
So i have been able to merge them together by doing: cd /b/ && docker-compose -f compose2.yaml -f another.yaml up -d to run 2 files together, and they exist under the B namespace. Likewise, I can also run them sequentially instead of referencing them in 1 command.
So my question is how can I do something like:
docker-compose --namespace test compose.yaml up
docker-compose --namespace test compose2.yaml up
such that I could view items accordingly with docker? It seems that I would need to consider running the command from under the first shared parent folder?
so if a and b existed under test, I could just do:
cd /test
docker-compose -f a/compose.yaml up -d
docker-compose -f b/compose2.yaml up -d
then my services would be listed as: test_web, test_db-box, etc.
So I found out that one person's namespace is another person's project-name.
That being said, after understanding nuances, the project-name ( -p | --project-name ) is the prepend for the compose services.
docker-compose --project-name foo -f a/compose.yaml up
docker-compose --project-name foo -f b/compose2.yaml up
This will create the services: foo_web_1
The format for this is: %{prepend}%{servicename}%{number}
The issue then is, Can we find a way to implement this CLI property to work from within the YAML file, possibly as a config option for the file. The Docker Compose website information states that you can supply an environment variable ( _ COMPOSE_PROJECT_NAME_ ) to change change the project name from the default of the base directory, BUT not from within a Compose YAML.
If i want to then launch multiple compose files, under a particular project what I would want to do is to just encapsulate it with a BASH or SHELL script.
#/bin/bash
export COMPOSE_PROJECT_NAME=ultimate-project
docker-compose -f a/compose.yaml up -d
docker-compose -f b/compose2.yaml up -d
and that would create services :
ultimate-project_web_1
ultimate-project_web2_1
How do you name your docker-compose.yml files? Is there a convention to follow?
With Dockerfiles <purpose>.Dockerfile seems to work the best being instantly recognized by VSCode and PyCharm.
I like the idea of structuring docker/compose files into folders so that default names could be used, but as far as I know docker can't see files up the tree creating different problems.
According to the -f documentation:
If you don’t provide this flag on the command line, Compose [...] looks for a
docker-compose.yml and a docker-compose.override.yml file.
If you don't want to use the -f flag you can use the default name docker-compose.yml and override it with docker-compose.override.yml.
However, if you use -f, since you'll provide the filename, you can use whatever you want.
I think a good way to name them depending on the environment could be docker-compose.{env}.yml and place them at the top level directory:
docker-compose.prod.yml
docker-compose.dev.yml
docker-compose.test.yml
docker-compose.staging.yml
And you can use the default docker-compose.yml to define the base configuration that is common to all environments.
I followed this Kiwi TCMS step, but what is really for me to is to understand how do you mount the common.py(main configuration file) to the working kiwi instance.
I don't see the place of common.py in the kiwi, so I dunno where to mount it? Or do I have to recreate the images every time to get the new settings?
EDIT:
I've tried Kiwi TCMS configuration settings guide and I changed some settings in tcms/settings/common.py
How to implement that setting in the working Kiwi environment?
The config file approach
The common.py file it seems to be located at tcms/settings/common.py as per your second link
All sensible settings are defined in tcms/settings/common.py. You will have to update some of them for your particular production environment.
If you really want to map only this file then from the root of your proeject:
docker run -v ./tcms/settings/common.py:/absolute/container/path/to/tcms/settings/common.py [other-options-here] image-name
Running the docker command with the above volume map will replace the file inside the docker container /absolute/container/path/to/tcms/settings/common.py with the one in the host tcms/settings/common.py, thus the application will run with the settings defined in the host.
If you don't know the full path to tcms/settings/common.py inside the docker container, then you need to add the Dockerfile to your question so that we can help further.
The ENV file approach
If not already existing a .env file in the root of your project create one and add there all env variables in the common.py:
.env example:
KIWI_DB_NAME=my_db_name
KIWI_DB_USER=my_db_user
KIWI_DB_PASSWORD=my_db_password
KIWI_DB_HOST=my_db_host
KIWI_DB_PORT=my_db_port
Add as many environment variables to the .env file as the ones you find in the python code that you want to customize.
Start the docker container from the place where the .env file is with the flag --env-file .env, something like:
docker run --env-file .env [other-options-here] image-name
When using Docker Compose with multiple configuration files (e.g., to allow multiple environments to share a common configuration file), I can't figure out if all commands in the CLI need to have all config files mentioned.
For example, I have docker.compose.yml and docker.compose.dev.yml, and I launch my dev environment as such:
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
If I want to run a command on a service, I can use:
docker-compose web ls
Or I can use:
docker-compose -f docker-compose.yml -f docker-compose.dev.yml run ls
Both commands work, so I wasn't sure if the configuration files get associated with the containers once they're built.
From what I can tell, if you want the overriding configuration file to be applied when the docker-compose CLI command is run, you need to specify all configuration files.
This may not always be apparent: the examples given in the question (docker-compose web ls and docker-compose -f docker-compose.yml -f docker-compose.dev.yml run ls) may both run without an error (really depends on what command you're running), but all settings in docker-compose.dev.yml will not be applied.
Here's an example of when this can be a problem:
Assume docker-compose.yml has all settings defaulted to a production environment where docker-compose.dev.yml sets these to some safe development setup (this is a horrible environment setup, but maybe other setups will have a similar problem)
Assume web is a Django app that uses django-storages to store static files on a CDN;
When running docker-compose run web python manage.py collectstatic, files would be uploaded to the production address, thus possibly causing unexpected production changes.
With all this in mind, I recommend renaming docker-compose.yml to something like docker-compose.base.yml, thus running docker-compose run web python manage.py collectstatic would error about docker-compose.yml being missing.
Note of alternative setup: an alternative setup for the multiple environments, as described on the referenced Compose documentation, is to use the extends keyword in the docker files. Rackspace developed an sample project, available on Github, that provides a great example of using this feature.