How to "docker-compose up" for images from Docker Hub - docker

Is it possible to pull and start all containers defined in docker-compose.yml? I'm trying to execute docker-compose up -d my/image, where my/image is a repo on DockerHub, but it says "Can't find docker-compose.yml". I also tried first to pull the image using docker pull my/image with the same result
UPD: The image https://hub.docker.com/r/gplcart/core/, source - https://github.com/gplcart/docker-core
SOLVED: It seems docker-compose does not work that way I want. I need to create a directory manually, place docker-compose.yml there, then run docker-compose up.
https://docs.docker.com/compose/wordpress/#define-the-project
I expected that running docker-compose up -d repo/image is enough to download and run all defined containers

To pull an image use docker-compose pull <service_name>, where service_name is one of the services listed in your docker-compose.yml file
The docker pull my/image fails, but should fail with a different error than you noted (you posted a compose error)
In your example, my/name is not a valid service name because you can't use a / in the service name. Compose would give you a different error.
It's unclear to me what my/name represents (assuming you replaced it with something locally).
If you post your docker-compose.yml it would help determine what the correct docker and docker-compose commands should be.

Try logging in to Docker Hub so that Docker Compose knows you want to pull images from there.
From the command line ...
docker login
You will be prompted for a username and password. Once authenticated, compose should pull your images from Docker Hub when running docker-compose up.
Also, you need to run docker-compose up from the same directory where your docker-compose.yml file is. Looking at your docker-compose.yml file on Github, it looks like you are missing a few lines. You need to specify the version, and gplcart, db and phpmyadmin should be under services.
version: '3'
services:
gplcart:
build: .
links:
- db
ports:
- 8080:80
db:
image: mariadb:10.2
environment:
MYSQL_DATABASE: test
MYSQL_ROOT_PASSWORD: test
ports:
- 3306:3306
phpmyadmin:
image: phpmyadmin/phpmyadmin:4.7
links:
- db
ports:
- 8181:80

Make sure docker and docker-compose binaries coming from the same package manager. E.g.
$ which -a docker docker-compose
/snap/bin/docker
/snap/bin/docker-compose
In other words, if you've installed Docker from a Snap, docker-compose binary should be already included (ls /snap/docker/current/bin/). When using Apt repository, docker-compose can be installed separately, so don't interchange binaries between Snap with Apt, as well don't mix docker with docker.io package on Apt.

The error Can't find docker-compose.yml indicates that you are not currently in the directory with your docker-compose.yml file, or that you have named the file something different. If you have named the file something different, including a different case or extension, you can either rename the file, or run docker-compose -f your_filename.yml up to pass a different file for docker-compose to parse. If you are not in the directory, make sure to cd into that directory before running docker-compose commands.

docker-compose acts based on your local docker-compose.yml file. Pulling a third-party image with docker-compose is usually useful when, instead of executing separate docker commands (in order to pull an image or deploy your app, etc etc), you want to define your architecture in a more structured way like:
My docker-compose.yml file:
version: '3'
services:
containerA:
image: gplcart/core
containerB:
build: .
# go on defining the rest ...
and deploying with:
docker-compose build && docker-compose up -d

Here is the simplest working example of docker-compose.yml file:
version: '3'
services:
hello-world:
image: hello-world
Which should produce the following results:
$ docker-compose up
Creating network "docker_default" with the default driver
Creating docker_hello-world_1 ... done
Attaching to docker_hello-world_1
hello-world_1 |
hello-world_1 | Hello from Docker!
hello-world_1 | This message shows that your installation appears to be working correctly.
hello-world_1 |
hello-world_1 | To generate this message, Docker took the following steps:
hello-world_1 | 1. The Docker client contacted the Docker daemon.
hello-world_1 | 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
hello-world_1 | (amd64)
hello-world_1 | 3. The Docker daemon created a new container from that image which runs the
hello-world_1 | executable that produces the output you are currently reading.
hello-world_1 | 4. The Docker daemon streamed that output to the Docker client, which sent it
hello-world_1 | to your terminal.
hello-world_1 |
hello-world_1 | To try something more ambitious, you can run an Ubuntu container with:
hello-world_1 | $ docker run -it ubuntu bash
hello-world_1 |
hello-world_1 | Share images, automate workflows, and more with a free Docker ID:
hello-world_1 | https://hub.docker.com/
hello-world_1 |
hello-world_1 | For more examples and ideas, visit:
hello-world_1 | https://docs.docker.com/get-started/
hello-world_1 |
docker_hello-world_1 exited with code 0
In case of problems, use the following commands which can help to track down the problem:
docker-compose config - Validate configuration file.
docker-compose logs - Check the latest logs.
docker info - Check system-wide information.
sudo strace -fe network docker-compose up - Debug the network issues.
journalctl -u docker.service - Check the logs of Docker service.

Related

Docker-compose logs for yml with non-default name

I have 2 docker-compose files.
One of them has a non-default docker-compose file name docker-compose-services.yml.
The second one is docker-compose.yml.
I need to follow logs from time to time for docker-compose-services.yml. Is there an option to follow logs for docker-compose-services.yml or for a project, if I wrap the containers up into one project?
I tried:
docker-compose logs -f - for containers defined in docker-compose.yml.
docker-compose logs --file - does not exist.
docker ps -q | xargs -L 1 docker logs - logs for all running containers. That's not what I need.
To view the logs for non-default docker-compose file (docker-compose-services.yml), you need to specify the file name before the logs command.
Let's say we have a redis container in docker-compose-services.yml
version: "3.3"
services:
redis:
image: "redis:alpine"
We can view the logs using this command
docker-compose -f docker-compose-service.yaml logs -f

First attempt at docker compose, status is "restarting" what have I done wrong?

This is my first attempt at docker composer (and docker since yesterday), however the docker is in a restarting state.
The application is Grafana which I normally run with:
docker volume create grafana-storage
docker run -d -p 3000:3000 --name=grafana -v grafana-storage:/var/lib/grafana grafana/grafana
Today I thought I'd try using docker composer, here is what I have done:Create a folder for the Docker App(s)
sudo mkdir Docker_Applications
cd Docker_Applications
sudo mkdir Grafana
Go into the directory
cd Grafana
sudo nano docker-compose.yml
add
version: '3'
services:
grafana:
image: "grafana/grafana:7.3.7"
volumes:
# Data persistency
# sudo mkdir -p /Docker_Applications/Grafana
- "./database:/var/lib/grafana"
- "./config:/etc/grafana"
ports:
- 3000:3000
restart: always
Then ran it
root#grafana-dev:/Docker_Applications/Grafana$ sudo docker-compose up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Starting grafana_grafana_1 ... done
status
root#grafana-dev:/Docker_Applications/Grafana$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a347b12ae9a3 grafana/grafana:7.3.7 "/run.sh" 18 minutes ago Restarting (1) 4 seconds ago grafana_grafana_1
Hopefully you can see I've tried my best. I wonder if it's to do with the volumes.
Any help would be really appreciated.
I removed the volumes section of the yaml file and it runs now. It looks like permissions or it can't locate my folders, i'm not sure what permissions/commands to try.
grafana-dev:/Docker_Applications/Grafana$ sudo docker-compose up
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating grafana_grafana_1 ... done
Attaching to grafana_grafana_1
grafana_1 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana_1 | GF_PATHS_CONFIG='/etc/grafana/grafana.ini' is not readable.
grafana_1 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana_1 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migration-from-a-previous-version-of-the-docker-container-to-5-1-or-later
Most likely the container exits during startup due to an error and since you've set restart: always in your docker-compose file, the container automatically restarts.
Check the logs or just run docker-compose up non-detached by removing the -d flag to find out what the problem is, fix that and your container will stop restarting itself continuously.

What is the difference between docker run -p and ports in docker-compose.yml?

I would like to use a standard way of running my docker containers. I have have been keeping a docker_run.sh file, but docker-compose.yml looks like a better choice. This seems to work great until I try to access my website running in the container. The ports don't seem to be set up correctly.
Using the following docker_run.sh, I can access the website at localhost. I expected the following docker-compose.yml file to have the same results when I use the docker-compose run web command.
docker_run.sh
docker build -t web .
docker run -it -v /home/<user>/git/www:/var/www -p 80:80/tcp -p 443:443/tcp -p 3316:3306/tcp web
docker-compose.yml
version: '3'
services:
web:
image: web
build: .
ports:
- "80:80"
- "443:443"
- "3316:3306"
volumes:
- "../www:/var/www"
Further analysis
The ports are reported as the same in docker ps and docker-compose ps. Note: these were not up at the same time.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
<id> web "/usr/local/scripts/…" About an hour ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:3307->3306/tcp <name>
$ docker-compose ps
Name Command State Ports
---------------------------------------------------------------------------------------------------------------
web /usr/local/scripts/start_s ... Up 0.0.0.0:3316->3306/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
What am I missing?
As #richyen suggests in a comment, you want docker-compose up instead of docker-compose run.
docker-compose run...
Runs a one-time command against a service.
That is, it's intended to run something like a debugging shell or a migration script, in the overall environment specified by the docker-compose.yml file, but not the standard command specified in the Dockerfile (or the override in the YAML file).
Critically to your question,
...docker-compose run [...] does not create any of the ports specified in the service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports to be created and mapped to the host, specify the --service-ports flag.
Beyond that, the docker run command you show and the docker-compose.yml file should be essentially equivalent.
You don't run docker-compose.yamls the same way that you would run a local docker image that you have either installed or created on your machine. docker-compose files are typically launched running the command docker-compose up -d to run in detached mode. Then when you run docker ps you should see it running. You can also run docker-compose ps as you did above.

Volume data does not fill when running a bamboo container on the server

I am trying to run bamboo on server using docker containers. When i running on local machine work normally and volume save datas successfully. But when i run same docker compose file on server, volume data not save my datas.
docker-compose.yml
version: '3.2'
services:
bamboo:
container_name: bamboo-server_test
image: atlassian/bamboo-server
volumes:
- ./volumes/bamboo_test_vol:/var/atlassian/application-data/bamboo
ports:
- 8085:8085
volumes:
bamboo_test_vol:
Run this compose file on local machine
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
916c98ca1a9d atlassian/bamboo-server "/entrypoint.sh" 24 minutes ago Up 24 minutes 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/bamboo_test_vol/
$ ls
bamboo.cfg.xml logs
localhost:8085
Run this compose file on server
$ ssh <name>#<ip_address>
password for <name>:
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
38b77e1b736f atlassian/bamboo-server "/entrypoint.sh" 12 seconds ago Up 11 seconds 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/
$ cd bamboo_test_vol/
$ ls
$ # VOLUME PATH IS EMPTY
server_ip:8085
I didn't have this problem when I tried the same process for jira-software. Why can't it work through the bamboo server even though I use the exact same compose file?
I had the same problem when I wanted to upgrade my Bamboo server instance with my mounted host volume for the bamboo-home directory.
The following was in my docker-compose file:
version: '2.2'
bamboo-server:
image: atlassian/bamboo-server:${BAMBOO_VERSION}
container_name: bamboo-server
environment:
TZ: 'Europe/Berlin'
restart: always
init: true
volumes:
- ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
- "8085:8085"
- "54663:54663"
When i started with docker-compose up -d bamboo-server, the container never took the files from the host system. So I tried it first without docker-compose, following the instructions of Atlassian Bamboo with the following command:
docker run -v ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
The following error message was displayed:
docker: Error response from daemon: create ./bamboo/bamboo-server/data: "./bamboo/bamboo-server/data" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
So I converted the error message and took the absolute path:
docker run -v /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
After the successful start, I switched to the docker container via SSH and all files were as usual in the docker directory.
I transferred the whole thing to the docker-compose file and took the absolute path in the volumes section. Subsequently it also worked with the docker-compose file.
My docker-compose file then looked like this:
[...]
init: true
volumes:
- /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
[...]
Setting up a containerized Bamboo Server is not supported for these reasons;
Repository-stored Specs (RSS) are no longer processed in Docker by default. Running RSS in Docker was not possible because;
there is no Docker capability added on the Bamboo server by default,
the setup would require running Docker in Docker.

How do I output or log to docker-compose output?

When I run docker-compose up it logs some information to the terminal and I would like to know where this information is coming from and how I might log to it.For example I would like to output each request in a php application within the container. I have tried to look online including the docker docs but have had no luck.
The output in docker-compose is the stdout/stderr from the command the container runs. You see this with docker run if you don't detach, and you can get this from docker logs on a container you're detached from or docker-compose logs from a compose submitted group of containers.
Edit: evidence of this behavior:
$ cat docker-compose.hello-world.yml
version: '2'
services:
hello-world:
image: busybox
command: "echo hello world"
$ docker-compose -f docker-compose.hello-world.yml up
Creating test_hello-world_1
Attaching to test_hello-world_1
hello-world_1 | hello world
test_hello-world_1 exited with code 0

Resources