Is it possible to remove the service name in the log output of docker-compose up?
My logs are currently looking like this:
serviceA | log1
serviceB | log2
serviceB | log3
serviceA | log4
But I want them to look like this:
log1
log2
log3
log4
The documentation states, that you can configure the logging for services like this:
version: "3.7"
services:
some-service:
image: some-service
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "10"
But I couldn't find any option to remove the service name from the logs when running docker-compose up.
Update
As proposed by EchoMike444, I've tried the following command:
docker-compose up --no-color 2>&1 | sed 's/^[^ ]* | //'
with the following docker compose file:
version: "3"
services:
consumer:
image: debian:stretch-slim
command:
[
"bash",
"-c",
"while ( true ) ; do echo $$(date) $$( hostname ) ; sleep $$(( $$RANDOM %4 )) ; done ",
]
ideas:
image: debian:stretch-slim
restart: always
command:
[
"bash",
"-c",
"while ( true ) ; do echo $$(date) $$( hostname ) ; sleep $$(( $$RANDOM %4 )) ; done ",
]
The output looks like this
ideas_1 | Sun Oct 20 05:56:23 UTC 2019 99295bdcbf2c
ideas_1 | Sun Oct 20 05:56:25 UTC 2019 99295bdcbf2c
Sun Oct 20 05:56:25 UTC 2019 dd45c5900159
ideas_1 | Sun Oct 20 05:56:27 UTC 2019 99295bdcbf2c
ideas_1 | Sun Oct 20 05:56:27 UTC 2019 99295bdcbf2c
Sun Oct 20 05:56:27 UTC 2019 dd45c5900159
Sun Oct 20 05:56:29 UTC 2019 dd45c5900159
ideas_1 | Sun Oct 20 05:56:30 UTC 2019 99295bdcbf2c
So it is working for service consumer, but not for ideas.
It seems you can now, this PR was merged:
https://github.com/docker/compose/pull/7435
ex: docker-compose up --no-log-prefix
Currently you can not remove the prefix , I did not see a option in the source code .
docker-compose is wrote in python feel free to patch .
https://github.com/docker/compose/blob/master/compose/cli/log_printer.py
A workaround will :
docker-compose up --no-color 2>&1 | sed 's/^[^ ]* *| //'
or
docker-compose up -d
docker-compose logs --no-color -f 2>&1 | sed 's/^[^ ]* *| //'
I tested MacOS and Linux with this very small docker-compose.yaml
version: '3.7'
services:
serviceA:
image: debian:stretch-slim
command: ["bash","-c","while ( true ) ; do echo $$(date) $$( hostname ) ; sleep $$(( $$RANDOM %4 )) ; done "]
serviceB:
image: debian:stretch-slim
restart: always
command: ["bash","-c","while ( true ) ; do echo $$(date) $$( hostname ) ; sleep $$(( $$RANDOM %4 )) ; done "]
Related
I am attempting to run the docker image of the AWS DynamoDB service using a local volume to store data. When I try to create my first table I am getting the following order repeating until eventually the script fails.
dynamodb-local | Aug 31, 2022 3:32:49 PM com.almworks.sqlite4java.Internal log
dynamodb-local | WARNING: [sqlite] SQLiteQueue[shared-local-instance.db]: stopped abnormally, reincarnating in 3000ms
dynamodb-local | Aug 31, 2022 3:32:52 PM com.almworks.sqlite4java.Internal log
dynamodb-local | WARNING: [sqlite] cannot open DB[5]: com.almworks.sqlite4java.SQLiteException: [14] unable to open database file
dynamodb-local | Aug 31, 2022 3:32:52 PM com.almworks.sqlite4java.Internal log
dynamodb-local | SEVERE: [sqlite] SQLiteQueue[shared-local-instance.db]: error running job queue
dynamodb-local | com.almworks.sqlite4java.SQLiteException: [14] unable to open database file
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open0(SQLiteConnection.java:1480)
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:282)
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:293)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.openConnection(SQLiteQueue.java:464)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:641)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205)
dynamodb-local | at java.lang.Thread.run(Thread.java:750)
My docker-compose file is based on the one from AWS documentation (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html)
version: "3.9" # optional since v1.27.0
services:
server:
build:
context: ./server/
dockerfile: Dockerfile.consumer
ports:
- "5558:5558"
depends_on:
- "dynamodb-local"
environment:
AWS_ACCESS_KEY_ID: 'DUMMYIDEXAMPLE'
AWS_SECRET_ACCESS_KEY: 'DUMMYEXAMPLEKEY'
dynamodb-local:
image: "amazon/dynamodb-local:latest"
ports:
- "8000:8000"
volumes:
- "data:/home/dynamodblocal/data"
command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
container_name: dynamodb-local
working_dir: /home/dynamodblocal
volumes:
logvolume01: {}
data: {}
Below is the table I am creating
import boto3
from pprint import pprint
from botocore.exceptions import ClientError
TABLE_NAME = "check_tbl"
dynamodb = boto3.client(
'dynamodb',
region_name="us-east-1",
aws_access_key_id="dummy_access_key",
aws_secret_access_key="dummy_secret_key",
endpoint_url="http://localhost:8000",
)
try:
response = dynamodb.describe_table(TableName=TABLE_NAME)
except ClientError as ce:
if ce.response['Error']['Code'] == 'ResourceNotFoundException':
result = dynamodb.create_table(
TableName=TABLE_NAME,
KeySchema=[
{
'AttributeName': 'id',
'KeyType': 'HASH' # Partition key
}
],
AttributeDefinitions=[
{
'AttributeName': 'timestamp',
'AttributeType': 'D'
}
],
ProvisionedThroughput={
'ReadCapacityUnits': 10,
'WriteCapacityUnits': 10
}
)
else:
print("Unknown exception occurred while querying for the " + TABLE_NAME + " table. Printing full error:")
pprint(ce.response)
My volumes path (/var/lib/docker/volumes/data) is owned by root, but that doesn't seem likely.
I have been trying to bring up a Prometheus container using docker-compose file. I have looked into the various solutions available online and none of them seem to work. Please go through my prometheus.yaml file and the docker-compose.yml file and let me know, what have I configured wrongly.
My prometheus.yaml file is located at /root/prometheus/prometheus.yaml
Note: I'm trying to run the prometheus in the agent-mode and I'm running the docker-compose file from the /root path.
Error generated:
-bash-5.0# docker-compose up
[...]
prometheus | ts=2022-05-12T14:28:25.350Z caller=main.go:447 level=error msg="Error loading config (--config.file=/etc/prometheus/prometheus.yaml)" file=/etc/prometheus/prometheus.yaml err="open /etc/prometheus/prometheus.yaml: no such file or directory"
prometheus exited with code 2
docker-compose.yml:
version: '3'
volumes:
prometheus_data:
services:
prometheus:
image: prom/prometheus:v2.35.0
container_name: prometheus
volumes:
- ./prometheus:/etc/prometheus
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yaml'
- '--storage.agent.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
- '--web.enable-lifecycle'
- '--enable-feature=agent'
expose:
- 9090
ports:
- "9090:9090"
Update 1: Adding the directory tree structure below
-bash-5.0# pwd
/root
-bash-5.0# tree
.
|-- cadvisor
|-- docker-compose.yml
|-- docker-compose_1.yml
|-- prometheus
| |-- prometheus.yaml
| |-- prometheus.yml
| |-- prometheus_old.yaml
| `-- prometheus_old.yml
|-- prometheus.yaml
`-- prometheus.yml
1 directory, 9 files
-bash-5.0#
Update 2: I did some debugging and found out that the directory is being mounted, whereas the files are being mounted as directory.
Basically what I did was I made changes to the docker-compose.yml file as follows.
version: '3'
services:
prometheus:
image: prom/prometheus:v2.35.0
container_name: prometheus
volumes:
- ./prometheus/prometheus.yml:/etc/prometheus-test/prometheus.yml
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
#- '--storage.agent.path=/prometheus'
- '--web.enable-lifecycle'
#- '--enable-feature=agent'
expose:
- 9090
ports:
- "9090:9090"
volumes:
prometheus_data:
In the above docker file I'm mounting my prometheus.yml file to a different location and let the prometheus configure with default config file present.
Later I logged into the container and checked for the mounted files and this is what was seen.
-bash-5.0# docker container exec -it prometheus sh
/prometheus $ cd /etc
/etc $ ls -ltr
total 68
-rw-r--r-- 1 root root 18774 Feb 10 2019 services
-rw-r--r-- 1 root root 494 Aug 16 2019 nsswitch.conf
-rw-r--r-- 1 root root 118 Mar 22 21:07 localtime
-rw-r--r-- 1 root root 340 Apr 11 21:49 passwd
-rw-rw-r-- 1 root root 306 Apr 11 21:49 group
-rw------- 1 root root 136 Apr 13 00:25 shadow
drwxr-xr-x 6 root root 4096 Apr 13 00:25 network
drwxr-xr-x 3 root root 4096 Apr 15 10:54 ssl
lrwxrwxrwx 1 root root 12 May 13 10:54 mtab -> /proc/mounts
-rw-r--r-- 1 root root 174 May 13 10:54 hosts
-rw-r--r-- 1 root root 13 May 13 10:54 hostname
-rw-r--r-- 1 root root 38 May 13 10:54 resolv.conf
drwxr-xr-x 3 root root 4096 May 13 11:00 prometheus-test
drwxr-xr-x 1 nobody nobody 4096 May 13 11:00 prometheus
/etc $
/etc $ cd prometheus-test/
/etc/prometheus-test $ ls
prometheus.yml
/etc/prometheus-test $ ls -ltr
total 8
drwxr-xr-x 2 root root 4096 May 13 10:54 prometheus.yml
/etc/prometheus-test $
From the above we can observe that the prometheus.yml file is being mounted as a directory instead of a file. Can anyone please let me know about this.
OS: Ubuntu 20.04
This is a vm instance running on the ESXI server
-bash-5.0# docker version
Client: Docker Engine - Community
Version: 19.03.10
API version: 1.40
Go version: go1.13.10
Git commit: 9424aeaee9
Built: Thu May 28 22:16:52 2020
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 633a0ea838f10e000b7c6d6eed1623e6e988b5bb
Built: Sat May 9 16:43:52 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.3.2
GitCommit: ff48f57fc83a8c44cf4ad5d672424a98ba37ded6
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit:
-bash-5.0#
-bash-5.0#
-bash-5.0#
-bash-5.0# docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
-bash-5.0#
I think you may be over-complicating things. Given this docker-compose.yaml:
version: '3'
volumes:
prometheus_data:
services:
prometheus:
image: prom/prometheus:v2.35.0
container_name: prometheus
volumes:
- ./prometheus:/etc/prometheus
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.agent.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
- '--web.enable-lifecycle'
- '--enable-feature=agent'
And this directory structure:
$ tree .
.
├── docker-compose.yaml
└── prometheus
└── prometheus.yml
It Just Works:
$ docker-compose up
Starting prometheus ... done
Attaching to prometheus
prometheus | ts=2022-05-13T12:12:16.206Z caller=main.go:187 level=info msg="Experimental agent mode enabled."
prometheus | ts=2022-05-13T12:12:16.207Z caller=main.go:525 level=info msg="Starting Prometheus" version="(version=2.35.0, branch=HEAD, revision=6656cd29fe6ac92bab91ecec0fe162ef0f187654)"
prometheus | ts=2022-05-13T12:12:16.207Z caller=main.go:530 level=info build_context="(go=go1.18.1, user=root#cf6852b14d68, date=20220421-09:53:42)"
prometheus | ts=2022-05-13T12:12:16.207Z caller=main.go:531 level=info host_details="(Linux 5.17.5-100.fc34.x86_64 #1 SMP PREEMPT Thu Apr 28 16:02:54 UTC 2022 x86_64 02da15afa8e7 (none))"
prometheus | ts=2022-05-13T12:12:16.207Z caller=main.go:532 level=info fd_limits="(soft=1073741816, hard=1073741816)"
prometheus | ts=2022-05-13T12:12:16.207Z caller=main.go:533 level=info vm_limits="(soft=unlimited, hard=unlimited)"
prometheus | ts=2022-05-13T12:12:16.208Z caller=web.go:541 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090
prometheus | ts=2022-05-13T12:12:16.208Z caller=main.go:1013 level=info msg="Starting WAL storage ..."
prometheus | ts=2022-05-13T12:12:16.212Z caller=db.go:332 level=info msg="replaying WAL, this may take a while" dir=/prometheus/wal
prometheus | ts=2022-05-13T12:12:16.213Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false
prometheus | ts=2022-05-13T12:12:16.214Z caller=db.go:383 level=info msg="WAL segment loaded" segment=0 maxSegment=1
prometheus | ts=2022-05-13T12:12:16.214Z caller=db.go:383 level=info msg="WAL segment loaded" segment=1 maxSegment=1
prometheus | ts=2022-05-13T12:12:16.215Z caller=main.go:1034 level=info fs_type=XFS_SUPER_MAGIC
prometheus | ts=2022-05-13T12:12:16.215Z caller=main.go:1037 level=info msg="Agent WAL storage started"
prometheus | ts=2022-05-13T12:12:16.215Z caller=main.go:1162 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
prometheus | ts=2022-05-13T12:12:16.216Z caller=dedupe.go:112 component=remote level=info remote_name=b4f547 url=http://10.120.23.224:9090/api/v1/write msg="Starting WAL watcher" queue=b4f547
prometheus | ts=2022-05-13T12:12:16.216Z caller=dedupe.go:112 component=remote level=info remote_name=b4f547 url=http://10.120.23.224:9090/api/v1/write msg="Starting scraped metadata watcher"
prometheus | ts=2022-05-13T12:12:16.216Z caller=main.go:1199 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=809.925µs db_storage=275ns remote_storage=282.833µs web_handler=394ns query_engine=377ns scrape=211.707µs scrape_sd=43.033µs notify=609ns notify_sd=911ns rules=143ns tracing=1.98µs
prometheus | ts=2022-05-13T12:12:16.216Z caller=main.go:930 level=info msg="Server is ready to receive web requests."
prometheus | ts=2022-05-13T12:12:16.216Z caller=dedupe.go:112 component=remote level=info remote_name=b4f547 url=http://10.120.23.224:9090/api/v1/write msg="Replaying WAL" queue=b4f547
prometheus | ts=2022-05-13T12:12:22.558Z caller=dedupe.go:112 component=remote level=info remote_name=b4f547 url=http://10.120.23.224:9090/api/v1/write msg="Done replaying WAL" duration=6.341973747s
If this fails, the first diagnostic step is probably to remove the command section from docker-compose.yaml and add an entrypoint like this:
version: '3'
volumes:
prometheus_data:
services:
prometheus:
image: prom/prometheus:v2.35.0
container_name: prometheus
volumes:
- ./prometheus:/etc/prometheus
- prometheus_data:/prometheus
entrypoint:
- sleep
- inf
This will come up and just run sleep, allowing you to docker-compose exec into the container and explore the filesystem. If you find that /etc/prometheus is empty, that suggests that the you're not running docker-compose on the same system as the docker daemon (so when it attempts to reference a host path, like ./prometheus, it doesn't find it).
I have a docker-compose.yml file. When I run this, I get the warning "Redis must be restarted after THP is disabled".
Below is my docker-compose.yml:
version: '3'
services:
myapp:
# container_name: myapp
restart: always
build: .
ports:
- '52000:52000'
# - '8080:8080'
# - '4300:4300'
# - '4301:4301'
command: ["./wait-for-it.sh", "redis:6379", "--", "npm", "start"]
links:
- redis
- mongo
mongo:
# container_name: myapp-mongo
image: 'mongo:latest'
ports:
- '28107:28107'
# - '27017:27017'
redis:
# container_name: myapp-redis
# restart: always
image: 'redis:4.0.11'
# command: ["redis-server", "--appendonly", "yes"]
ports:
- '6379:6379'
Below are my logs:
redis_1 | 1:C 24 Sep 10:21:09.224 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 24 Sep 10:21:09.236 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 24 Sep 10:21:09.236 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 24 Sep 10:21:09.239 * Running mode=standalone, port=6379.
redis_1 | 1:M 24 Sep 10:21:09.239 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 24 Sep 10:21:09.239 # Server initialized
mongo_1 | 2019-09-24T10:21:10.304+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
redis_1 | 1:M 24 Sep 10:21:09.239 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 24 Sep 10:21:09.239 * Ready to accept connections
# WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
From the error you could use 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' on docker host to fix your problem.
But, I know you are on windows & cannot access hyper-v MobyLinuxVM, so you should use workaround as next:
docker-compose.yaml:
version: '3'
services:
redis:
image: 'redis:4.0.11'
ports:
- '6379:6379'
depends_on:
- helper
sysctls:
- net.core.somaxconn=511
helper:
image: alpine
command: sh -c "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
privileged: true
Above will first start a helper container which will set never for /sys/kernel/mm/transparent_hugepage/enabled, as all containers will share the same kernel of host, so your redis container which start later will also benefit from this, see next execution log:
PS E:\abc> docker-compose up
Starting abc_helper_1 ... done
Recreating 29ea8bfaeafc_abc_redis_1 ... done
Attaching to abc_helper_1, abc_redis_1
redis_1 | 1:C 25 Sep 15:38:17.822 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 25 Sep 15:38:17.822 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 25 Sep 15:38:17.822 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 25 Sep 15:38:17.822 * Running mode=standalone, port=6379.
redis_1 | 1:M 25 Sep 15:38:17.823 # Server initialized
redis_1 | 1:M 25 Sep 15:38:17.823 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 25 Sep 15:38:17.823 * Ready to accept connections
In my Kubuntu 18.04 I try to run docker for my Laravel application
$ docker --version
Docker version 17.12.1-ce, build 7390fc6
I have 3 files:
.env:
# PATHS
DB_PATH_HOST=./databases
APP_PATH_HOST=./votes
APP_PTH_CONTAINER=/var/www/html/
docker-compose.yml:
version: '3'
services:
web:
build: ./web/Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
/web/Dockerfile.yml:
FROM php:7.2-apache
RUN docker-php-ext-install \
pdo_mysql \
&& a2enmod \
rewrite
When I try to use docker-compose up --build, I get the following:
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose up --build
Building web
Traceback (most recent call last):
File "bin/docker-compose", line 6, in <module>
File "compose/cli/main.py", line 71, in main
File "compose/cli/main.py", line 127, in perform_command
File "compose/cli/main.py", line 1052, in up
File "compose/cli/main.py", line 1048, in up
File "compose/project.py", line 466, in up
File "compose/service.py", line 329, in ensure_image_exists
File "compose/service.py", line 1047, in build
File "site-packages/docker/api/build.py", line 142, in build
TypeError: You must specify a directory to build in path
[6769] Failed to execute script docker-compose
I know that *.py that is python language files, but I do not use python language or work with it, I work with PHP.
Why is an error and how to fix it?
MODIFIED:
$ docker-compose up --build
Building webapp
Step 1/2 : FROM php:7.2-apache
---> a7d68dad7584
Step 2/2 : RUN docker-php-ext-install pdo_mysql && a2enmod rewrite
---> Using cache
---> 519d1b33af81
Successfully built 519d1b33af81
Successfully tagged votes_docker_webapp:latest
Starting votes_docker_adminer_1 ...
Starting votes_docker_composer_1 ...
Starting votes_docker_adminer_1 ... error
votes_docker_db_1 is up-to-date
ERROR: for votes_docker_adminer_1 Cannot start service adminer: driver failed programming external connectivity on endpoint votes_docker_adminer_1 (6e94693ab8b1a990aaa83164df0952e8665f351618a72aStarting votes_docker_composer_1 ... done
ERROR: for adminer Cannot start service adminer: driver failed programming external connectivity on endpoint votes_docker_adminer_1 (6e94693ab8b1a990aaa83164df0952e8665f351618a72a5531f9c3ccc18a2e3d): Bind for 0.0.0.0:8080 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
I tried to check related ports and got:
# sudo netstat -ntpl | grep 8080:8080
# sudo netstat -ntpl | grep 0.0.0.0:8080
# sudo netstat -ntpl | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 7361/docker-proxy
MODIFIED #2:
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose up --build
Creating network "votes_docker_default" with the default driver
Building webapp
Step 1/2 : FROM php:7.2-apache
---> a7d68dad7584
Step 2/2 : RUN docker-php-ext-install pdo_mysql && a2enmod rewrite
---> Using cache
---> 519d1b33af81
Successfully built 519d1b33af81
Successfully tagged votes_docker_webapp:latest
Creating votes_docker_adminer_1 ... done
Creating votes_docker_composer_1 ... done
Creating votes_docker_webapp_1 ... done
Creating votes_docker_db_1 ... done
Attaching to votes_docker_adminer_1, votes_docker_composer_1, votes_docker_webapp_1, votes_docker_db_1
adminer_1 | PHP 7.2.10 Development Server started at Mon Oct 15 10:14:02 2018
composer_1 | Composer could not find a composer.json file in /var/www/html
composer_1 | To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
votes_docker_composer_1 exited with code 1
webapp_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.4. Set the 'ServerName' directive globally to suppress this message
webapp_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.4. Set the 'ServerName' directive globally to suppress this message
webapp_1 | [Mon Oct 15 10:14:05.281793 2018] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.25 (Debian) PHP/7.2.10 configured -- resuming normal operations
webapp_1 | [Mon Oct 15 10:14:05.281843 2018] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
db_1 | 2018-10-15T10:14:06.541323Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
db_1 | 2018-10-15T10:14:06.541484Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.11) starting as process 1
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | 2018-10-15T10:14:07.062202Z 0 [Warning] [MY-011071] [Server] World-writable config file './auto.cnf' is ignored.
db_1 | 2018-10-15T10:14:07.062581Z 0 [Warning] [MY-010107] [Server] World-writable config file './auto.cnf' has been removed.
db_1 | 2018-10-15T10:14:07.063146Z 0 [Warning] [MY-010075] [Server] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 0cd8212e-d063-11e8-8e69-0242ac140005.
db_1 | 2018-10-15T10:14:07.079020Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db_1 | 2018-10-15T10:14:07.091951Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
db_1 | 2018-10-15T10:14:07.103829Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.infoschema#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103896Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103925Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103947Z 0 [Warning] [MY-010315] [Server] 'user' entry 'root#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104006Z 0 [Warning] [MY-010323] [Server] 'db' entry 'performance_schema mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104034Z 0 [Warning] [MY-010323] [Server] 'db' entry 'sys mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104070Z 0 [Warning] [MY-010311] [Server] 'proxies_priv' entry '# root#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.112700Z 0 [Warning] [MY-010330] [Server] 'tables_priv' entry 'user mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.112738Z 0 [Warning] [MY-010330] [Server] 'tables_priv' entry 'sys_config mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.117764Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.11' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
Is it mysql misconfigure?
I run it not as root.
Also as I use LAMP do I need to stop apache and mysql before running docker-compose command?
MODIFIED #3:
After some searching I added mysql version in my config file and added command option:
image: mysql:5.7.23
command: --default-authentication-plugin=mysql_native_password --disable-partition-engine-check
and the above error was fixed.
So:
1. In other console under root I run commands (but I'm still not sure if I need it?)
sudo service apache2 stop
sudo service mysql stop
2. Under nonroot console I run with key to run in background:
docker-compose up -d
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose down
Stopping votes_docker_db_1 ... done
Stopping votes_docker_webapp_1 ... done
Stopping votes_docker_adminer_1 ... done
Removing votes_docker_db_1 ... done
Removing votes_docker_webapp_1 ... done
Removing votes_docker_composer_1 ... done
Removing votes_docker_adminer_1 ... done
Removing network votes_docker_default
docker-compose up -d
Creating network "votes_docker_default" with the default driver
Creating votes_docker_webapp_1 ... done
Creating votes_docker_adminer_1 ... done
Creating votes_docker_db_1 ... done
Creating votes_docker_composer_1 ... done
I have no errors in output, but I expected as a result to have vendor directory in my project, as I have in web/Dockerfile.yml:
FROM php:7.2-apache
RUN docker-php-ext-install \
pdo_mysql \
&& a2enmod \
rewrite
But I do not see this directory...
Was the installation successful or not?
I'm not sure where to move next?
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker info
Containers: 33
Running: 2
Paused: 0
Stopped: 31
Images: 19
Server Version: 17.12.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9b55aab90508bd389d7654c4baf173a981477d55
runc version: 9f9c96235cc97674e935002fc3d78361b696a69e
init version: v0.13.0 (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-36-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.711GiB
Name: serge
ID: BDNU:HFWX:N6YV:IWYW:HJSU:SZ23:URPB:3FR2:7I3E:IFGK:AOLH:YRE5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
votes_docker_webapp latest 519d1b33af81 27 hours ago 378MB
adminer latest 0038b45402de 4 weeks ago 81.7MB
composer 1.6 e28b5b53ab28 4 weeks ago 154MB
php 7.2-apache a7d68dad7584 4 weeks ago 378MB
mysql 5.7.23 563a026a1511 5 weeks ago 372MB
mysql 5.7.22 6bb891430fb6 2 months ago 372MB
test2_php latest 05534d47f926 3 months ago 84.7MB
test1_php latest 05534d47f926 3 months ago 84.7MB
<none> <none> 6060fcf4d103 3 months ago 81MB
php fpm-alpine 601d5b3a95d4 3 months ago 80.6MB
php apache d9faf33e6e40 3 months ago 377MB
mysql latest 8d99edb9fd40 3 months ago 445MB
php 7-fpm 854ffd8dc9d8 3 months ago 367MB
php 7.2 e86d9bb526ef 3 months ago 367MB
ukfx/php apache-stretch 5958cb7c2316 4 months ago 648MB
nginx alpine bc7fdec94612 4 months ago 18MB
hello-world latest e38bc07ac18e 6 months ago 1.85kB
composer/composer latest 5afb0951f2a4 2 years ago 636MB
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f8beea5dceca mysql:5.7.23 "docker-entrypoint.s…" 6 minutes ago Restarting (2) 6 seconds ago votes_docker_db_1
8309b5456dcf adminer "entrypoint.sh docke…" 6 minutes ago Up 6 minutes 0.0.0.0:8081->8080/tcp votes_docker_adminer_1
cc644206931b votes_docker_webapp "docker-php-entrypoi…" 6 minutes ago Up 6 minutes 0.0.0.0:8080->80/tcp votes_docker_webapp_1
How to fix it?
Thanks!
According to docker-compose documentation, build can be specified as a string containing a path to the build context if you have Dockerfile inside it.
You are using Dockerfile.yml file, which is not a default one (Dockerfile), so in this case context and dockerfile should be specified as well:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
The final docker-compose.yaml is:
version: '3'
services:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
Addition to MODIFIED part:
Both web and adminer are configured to be allocated on port 8080 on a host system. That's why you have a conflict here. To resolve above issue you need to bind adminer to another port 8081 for example.
The final docker-compose.yaml is:
version: '3'
services:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8081:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
Addition to MODIFIED3 part:
I see the following error with your composer docker container:
composer_1 | Composer could not find a composer.json file in /var/www/html
composer_1 | To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Is it because you accidentally put ${DB_PATH_HOST} instead of ${APP_PATH_HOST} in the composer config?
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
I am trying to run up a docker container with a mounted local volume using docker-compose. In failure to get it working, I have reduced by docker-compose down to the most basic version:
version: '2'
services:
php:
image: php:5.6-apache
ports:
- "8080:80"
volumes:
- .:/var/www/html
Upon doing this the mount appears in docker inspect:
"Mounts": [
{
"Type": "bind",
"Source": "/home/clark/Projects/apis/contracts",
"Destination": "/var/www/html",
"Mode": "rw",
"RW": true,
"Propagation": ""
}
],
But /var/www/html directory on the container is empty.
I have tried every combination I can think of. Absolute paths, relative paths, ${pwd}, named volumes, mounting to a directory that exists, mounting to a directory that doesn't exist... It's driving me crazy.
PLEASE can someone tell me what I'm doing wrong???
Works here, the only difference is a version 3 docker-compose file.
mkdir /tmp/compose-test && cd /tmp/compose-test
touch iam_a_file
Docker compose file :
---
version: '3'
services:
test:
image: busybox
command: ls -la /var/www/html
volumes:
- .:/var/www/html
Ensure config is ok :
docker-compose config
Up all the stuff
docker-compose up
Creating composetest_test_1
Attaching to composetest_test_1
test_1 | total 8
test_1 | drwxr-xr-x 2 1000 users 80 Apr 26 16:47 .
test_1 | drwxr-xr-x 1 www-data www-data 4096 Apr 26 16:49 ..
test_1 | -rw-r--r-- 1 1000 users 126 Apr 26 16:48 docker-compose.yml
test_1 | -rw-r--r-- 1 1000 users 0 Apr 26 16:46 iam_a_file
Inspect show that when you provide a relative path, docker change it to a full path.
Edit: i also tried with php:5.6-apache, it works.