I am trying to create a MongoDB Docker environment. I also want the data persist between rebuild. So I added a volume in the docker-compose.yml.
Here is my docker-compose.yml
version: '3.1'
services:
mongo:
build:
context: .
dockerfile: ./docker/mongo/Dockerfile
restart: always
ports:
- ${MONGO_PORT_FORWARD}:${MONGO_PORT}
volumes:
- ./data/db:/data/db:Z
And this is my Dockerfile
FROM mongo:latest
ENV MONGO_INITDB_ROOT_USERNAME admin
ENV MONGO_INITDB_ROOT_PASSWORD secret
ENV MONGO_INITDB_DATABASE db
Now I run docker-compose up and it works flawlessly.
But when I run docker-compose build to rebuild the image, I got error below.
Building mongo
Traceback (most recent call last):
File "/usr/bin/docker-compose", line 11, in <module>
load_entry_point('docker-compose==1.23.2', 'console_scripts', 'docker-compose')()
File "/usr/lib/python3.7/site-packages/compose/cli/main.py", line 71, in main
command()
File "/usr/lib/python3.7/site-packages/compose/cli/main.py", line 127, in perform_command
handler(command, command_options)
File "/usr/lib/python3.7/site-packages/compose/cli/main.py", line 287, in build
parallel_build=options.get('--parallel', False),
File "/usr/lib/python3.7/site-packages/compose/project.py", line 384, in build
build_service(service)
File "/usr/lib/python3.7/site-packages/compose/project.py", line 366, in build_service
service.build(no_cache, pull, force_rm, memory, build_args, gzip)
File "/usr/lib/python3.7/site-packages/compose/service.py", line 1082, in build
platform=self.platform,
File "/usr/lib/python3.7/site-packages/docker/api/build.py", line 159, in build
path, exclude=exclude, dockerfile=dockerfile, gzip=gzip
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 30, in tar
files=sorted(exclude_paths(root, exclude, dockerfile=dockerfile[0])),
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 49, in exclude_paths
return set(pm.walk(root))
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 214, in rec_walk
for sub in rec_walk(cur):
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 214, in rec_walk
for sub in rec_walk(cur):
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 214, in rec_walk
for sub in rec_walk(cur):
File "/usr/lib/python3.7/site-packages/docker/utils/build.py", line 184, in rec_walk
for f in os.listdir(current_dir):
PermissionError: [Errno 13] Permission denied: '/home/zendy/Projects/ZendyLim/GTBox/data/db/journal'
I checked the permission of /data/db/ and I found out the it has 999:root as owner.
What I am doing wrong here? What can I do to make this work?
UPDATE 1:
I have tried to set user as mongodb in the docker-compose.yml
version: '3.1'
services:
mongo:
build:
context: .
dockerfile: ./docker/mongo/Dockerfile
restart: always
ports:
- ${MONGO_PORT_FORWARD}:${MONGO_PORT}
volumes:
- ./data/db:/data/db:Z
user: mongodb
And got this error
mongo_1 | about to fork child process, waiting until server is ready for connections.
mongo_1 | forked process: 17
mongo_1 | 2019-03-13T00:58:43.499+0000 I CONTROL [main] ***** SERVER RESTARTED *****
mongo_1 | 2019-03-13T00:58:43.502+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] MongoDB starting : pid=17 port=27017 dbpath=/data/db 64-bit host=1e56c8187004
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] db version v4.0.6
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] git version: caa42a1f75a56c7643d0b68d3880444375ec42e3
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] allocator: tcmalloc
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] modules: none
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] build environment:
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] distmod: ubuntu1604
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] distarch: x86_64
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] target_arch: x86_64
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27017, ssl: { mode: "disabled" } }, processManagement: { fork: true, pidFilePath: "/tmp/docker-entrypoint-temp-mongod.pid" }, systemLog: { destination: "file", logAppend: true, path: "/proc/1/fd/1" } }
mongo_1 | 2019-03-13T00:58:43.504+0000 I STORAGE [initandlisten] exception in initAndListen: IllegalOperation: Attempted to create a lock file on a read-only directory: /data/db, terminating
mongo_1 | 2019-03-13T00:58:43.504+0000 I NETWORK [initandlisten] shutdown: going to close listening sockets...
mongo_1 | 2019-03-13T00:58:43.504+0000 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] now exiting
mongo_1 | 2019-03-13T00:58:43.504+0000 I CONTROL [initandlisten] shutting down with code:100
mongo_1 | ERROR: child process failed, exited with error number 100
mongo_1 | To see additional information in this output, start without the "--fork" option.
gtbox_mongo_1 exited with code 100
Found out the solution
I created a .dockerignore file with data/ in it.
# .dockerignore
data/
# Other files and folder
It manages to rebuild without any problem.
The only downside if I want a clean slate of the database, I need to run sudo rm -rf data to remove the folder.
Related
I am running node-red in docker compose and from the gitlab-cli file I am calling docker/compose image and my pipeline is working and I can see this:
node-red | 11 Nov 11:28:51 - [info]
462node-red |
463node-red | Welcome to Node-RED
464node-red | ===================
465node-red |
466node-red | 11 Nov 11:28:51 - [info] Node-RED version: v3.0.2
467node-red | 11 Nov 11:28:51 - [info] Node.js version: v16.16.0
468node-red | 11 Nov 11:28:51 - [info] Linux 5.15.49-linuxkit x64 LE
469node-red | 11 Nov 11:28:52 - [info] Loading palette nodes
470node-red | 11 Nov 11:28:53 - [info] Settings file : /data/settings.js
471node-red | 11 Nov 11:28:53 - [info] Context store : 'default' [module=memory]
472node-red | 11 Nov 11:28:53 - [info] User directory : /data
473node-red | 11 Nov 11:28:53 - [warn] Projects disabled : editorTheme.projects.enabled=false
474node-red | 11 Nov 11:28:53 - [info] Flows file : /data/flows.json
475node-red | 11 Nov 11:28:53 - [warn]
476node-red |
477node-red | ---------------------------------------------------------------------
478node-red | Your flow credentials file is encrypted using a system-generated key.
479node-red |
480node-red | If the system-generated key is lost for any reason, your credentials
481node-red | file will not be recoverable, you will have to delete it and re-enter
482node-red | your credentials.
483node-red |
484node-red | You should set your own key using the 'credentialSecret' option in
485node-red | your settings file. Node-RED will then re-encrypt your credentials
486node-red | file using your chosen key the next time you deploy a change.
487node-red | ---------------------------------------------------------------------
488node-red |
489node-red | 11 Nov 11:28:53 - [info] Server now running at http://127.0.0.1:1880/
490node-red | 11 Nov 11:28:53 - [warn] Encrypted credentials not found
491node-red | 11 Nov 11:28:53 - [info] Starting flows
492node-red | 11 Nov 11:28:53 - [info] Started flows
but when I am trying to open the localhost server to access the node-red or the dashboard, I am getting the error "Failed to open page"
This is my docker-compose.yml
version: "3.7"
services:
node-red:
image: nodered/node-red:latest
user: '1000'
container_name: node-red
environment:
- TZ=Europe/Amsterdam
ports:
- "1880:1880"
This is my .gitlab-cli.yml
yateen-docker:
stage: build
image:
name: docker/compose
services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
before_script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
script:
- docker-compose up
only:
- main
Any help!
I tried to create the node-red docker via docker-compose not just by running docker run command. Though my node-red image is running but I can't access the server page
I am attempting to run the docker image of the AWS DynamoDB service using a local volume to store data. When I try to create my first table I am getting the following order repeating until eventually the script fails.
dynamodb-local | Aug 31, 2022 3:32:49 PM com.almworks.sqlite4java.Internal log
dynamodb-local | WARNING: [sqlite] SQLiteQueue[shared-local-instance.db]: stopped abnormally, reincarnating in 3000ms
dynamodb-local | Aug 31, 2022 3:32:52 PM com.almworks.sqlite4java.Internal log
dynamodb-local | WARNING: [sqlite] cannot open DB[5]: com.almworks.sqlite4java.SQLiteException: [14] unable to open database file
dynamodb-local | Aug 31, 2022 3:32:52 PM com.almworks.sqlite4java.Internal log
dynamodb-local | SEVERE: [sqlite] SQLiteQueue[shared-local-instance.db]: error running job queue
dynamodb-local | com.almworks.sqlite4java.SQLiteException: [14] unable to open database file
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open0(SQLiteConnection.java:1480)
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:282)
dynamodb-local | at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:293)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.openConnection(SQLiteQueue.java:464)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:641)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77)
dynamodb-local | at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205)
dynamodb-local | at java.lang.Thread.run(Thread.java:750)
My docker-compose file is based on the one from AWS documentation (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html)
version: "3.9" # optional since v1.27.0
services:
server:
build:
context: ./server/
dockerfile: Dockerfile.consumer
ports:
- "5558:5558"
depends_on:
- "dynamodb-local"
environment:
AWS_ACCESS_KEY_ID: 'DUMMYIDEXAMPLE'
AWS_SECRET_ACCESS_KEY: 'DUMMYEXAMPLEKEY'
dynamodb-local:
image: "amazon/dynamodb-local:latest"
ports:
- "8000:8000"
volumes:
- "data:/home/dynamodblocal/data"
command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
container_name: dynamodb-local
working_dir: /home/dynamodblocal
volumes:
logvolume01: {}
data: {}
Below is the table I am creating
import boto3
from pprint import pprint
from botocore.exceptions import ClientError
TABLE_NAME = "check_tbl"
dynamodb = boto3.client(
'dynamodb',
region_name="us-east-1",
aws_access_key_id="dummy_access_key",
aws_secret_access_key="dummy_secret_key",
endpoint_url="http://localhost:8000",
)
try:
response = dynamodb.describe_table(TableName=TABLE_NAME)
except ClientError as ce:
if ce.response['Error']['Code'] == 'ResourceNotFoundException':
result = dynamodb.create_table(
TableName=TABLE_NAME,
KeySchema=[
{
'AttributeName': 'id',
'KeyType': 'HASH' # Partition key
}
],
AttributeDefinitions=[
{
'AttributeName': 'timestamp',
'AttributeType': 'D'
}
],
ProvisionedThroughput={
'ReadCapacityUnits': 10,
'WriteCapacityUnits': 10
}
)
else:
print("Unknown exception occurred while querying for the " + TABLE_NAME + " table. Printing full error:")
pprint(ce.response)
My volumes path (/var/lib/docker/volumes/data) is owned by root, but that doesn't seem likely.
Project structure:
client
nginx
web/
celery_worker.py
project
config.py
api/
I have the following services in my docker-compose:
version: '3.6'
services:
web:
build:
context: ./services/web
dockerfile: Dockerfile-dev
volumes:
- './services/web:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgres://postgres:postgres#web-db:5432/web_dev
- DATABASE_TEST_URL=postgres://postgres:postgres#web-db:5432/web_test
- SECRET_KEY=my_precious
depends_on:
- web-db
- redis
celery:
image: dev3_web
restart: always
volumes:
- ./services/web:/usr/src/app
- ./services/web/logs:/usr/src/app
command: celery worker -A celery_worker.celery --loglevel=INFO -Q cache
environment:
- CELERY_BROKER=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
depends_on:
- web
- redis
links:
- redis:redis
redis:
image: redis:5.0.3-alpine
restart: always
expose:
- '6379'
ports:
- '6379:6379'
monitor:
image: dev3_web
ports:
- 5555:5555
command: flower -A celery_worker.celery --port=5555 --broker=redis://redis:6379/0
depends_on:
- web
- redis
web-db:
build:
context: ./services/web/project/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
nginx:
build:
context: ./services/nginx
dockerfile: Dockerfile-dev
restart: always
ports:
- 80:80
- 8888:8888
depends_on:
- web
- client
- redis
client:
build:
context: ./services/client
dockerfile: Dockerfile-dev
volumes:
- './services/client:/usr/src/app'
- '/usr/src/app/node_modules'
ports:
- 3007:3000
environment:
- NODE_ENV=development
- REACT_APP_WEB_SERVICE_URL=${REACT_APP_WEB_SERVICE_URL}
depends_on:
- web
- redis
CELERY LOG
However, celery is not being able to connect, from this log:
celery_1 | [2019-03-29 03:09:32,111: ERROR/MainProcess] consumer: Cannot connect to redis://localhost:6379/0: Error 99 connecting to localhost:6379. Address not available..
celery_1 | Trying again in 2.00 seconds...
WEB LOG
and so is not web service (running the backend), by the same log:
web_1 | Waiting for postgres...
web_1 | PostgreSQL started
web_1 | * Environment: development
web_1 | * Debug mode: on
web_1 | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
web_1 | * Restarting with stat
web_1 | * Debugger is active!
web_1 | * Debugger PIN: 316-641-271
web_1 | 172.21.0.9 - - [29/Mar/2019 03:03:17] "GET /users HTTP/1.0" 200 -
web_1 | 172.21.0.9 - - [29/Mar/2019 03:03:26] "POST /auth/register HTTP/1.0" 500 -
web_1 | Traceback (most recent call last):
web_1 | File "/usr/lib/python3.6/site-packages/redis/connection.py", line 492, in connect
web_1 | sock = self._connect()
web_1 | File "/usr/lib/python3.6/site-packages/redis/connection.py", line 550, in _connect
web_1 | raise err
web_1 | File "/usr/lib/python3.6/site-packages/redis/connection.py", line 538, in _connect
web_1 | sock.connect(socket_address)
web_1 | OSError: [Errno 99] Address not available
web_1 |
web_1 | During handling of the above exception, another exception occurred:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/lib/python3.6/site-packages/kombu/connection.py", line 431, in _reraise_as_library_errors
web_1 | yield
web_1 | File "/usr/lib/python3.6/site-packages/celery/app/base.py", line 744, in send_task
web_1 | self.backend.on_task_call(P, task_id)
web_1 | File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 265, in on_task_call
web_1 | self.result_consumer.consume_from(task_id)
web_1 | File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 125, in consume_from
web_1 | return self.start(task_id)
web_1 | File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 107, in start
web_1 | self._consume_from(initial_task_id)
web_1 | File "/usr/lib/python3.6/site-packages/celery/backends/redis.py", line 132, in _consume_from
web_1 | self._pubsub.subscribe(key)
web_1 | File "/usr/lib/python3.6/site-packages/redis/client.py", line 3096, in subscribe
web_1 | ret_val = self.execute_command('SUBSCRIBE', *iterkeys(new_channels))
web_1 | File "/usr/lib/python3.6/site-packages/redis/client.py", line 3003, in execute_command
web_1 | self.shard_hint
web_1 | File "/usr/lib/python3.6/site-packages/redis/connection.py", line 994, in get_connection
web_1 | connection.connect()
web_1 | File "/usr/lib/python3.6/site-packages/redis/connection.py", line 497, in connect
web_1 | raise ConnectionError(self._error_message(e))
web_1 | redis.exceptions.ConnectionError: Error 99 connecting to localhost:6379. Address not available.
web_1 |
web_1 | During handling of the above exception, another exception occurred:
web_1 |
web_1 | Traceback (most recent call last):
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
web_1 | return self.wsgi_app(environ, start_response)
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
web_1 | response = self.handle_exception(e)
web_1 | File "/usr/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
web_1 | reraise(exc_type, exc_value, tb)
web_1 | File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
web_1 | raise value
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
web_1 | response = self.full_dispatch_request()
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
web_1 | rv = self.handle_user_exception(e)
web_1 | File "/usr/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
web_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
web_1 | File "/usr/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
REDIS LOG
Redis, however, seems to be working:
redis_1 | 1:C 29 Mar 2019 02:33:32.722 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 29 Mar 2019 02:33:32.722 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 29 Mar 2019 02:33:32.722 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 29 Mar 2019 02:33:32.724 * Running mode=standalone, port=6379.
redis_1 | 1:M 29 Mar 2019 02:33:32.724 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 29 Mar 2019 02:33:32.724 # Server initialized
redis_1 | 1:M 29 Mar 2019 02:33:32.724 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 29 Mar 2019 02:33:32.725 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 29 Mar 2019 02:33:32.725 * Ready to accept connections
config.py
class DevelopmentConfig(BaseConfig):
"""Development configuration"""
DEBUG_TB_ENABLED = True
DEBUG = True
BCRYPT_LOG_ROUNDS = 4
#set key
#sqlalchemy
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL')
#SQLALCHEMY_DATABASE_URI= "sqlite:///models/data/database.db"
# mail
MAIL_SERVER='smtp.gmail.com'
MAIL_PORT = 587
MAIL_USE_TLS = True
MAIL_DEBUG = True
MAIL_USERNAME = 'me#gmail.com'
MAIL_PASSWORD = 'MEfAc6w74WGx'
SEVER_NAME = 'http://127.0.0.1:8080'
# celery broker
REDIS_HOST = "0.0.0.0"
REDIS_PORT = 6379
BROKER_URL = os.environ.get('REDIS_URL', "redis://{host}:{port}/0".format(
host=REDIS_HOST,
port=str(REDIS_PORT)))
INSTALLED_APPS = ['routes']
# celery config
CELERYD_CONCURRENCY = 10
CELERY_BROKER_URL = BROKER_URL
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_IMPORTS = ('project.api.routes.background',)
what am I missing here?
TL;DR change redis://localhost:6379/0 to redis://redis:6379/0
When you run docker-compose, it creates a new network under which all your containers are running. Docker engine also creates an internal routing which allows all the containers to reference each other using their names.
In your case, your web and celery containers were trying to access redis over localhost. But inside the container, localhost means their own localhost. You need to change the configuration to point the hostname to the name of the container.
If you were not using docker, but had different machines for each of your container, localhost would have meant their own server. In order to connect to redis server, you would have passed the IP address of the machine on which redis was running. In docker, instead of IP address, you can just pass the name of the container because of the engine's routing discussed above.
Note that you can still assign static IP addresses to each of your container, and use those IP addresses instead of container_names. For more details, read the networking section of docker documents.
In my Kubuntu 18.04 I try to run docker for my Laravel application
$ docker --version
Docker version 17.12.1-ce, build 7390fc6
I have 3 files:
.env:
# PATHS
DB_PATH_HOST=./databases
APP_PATH_HOST=./votes
APP_PTH_CONTAINER=/var/www/html/
docker-compose.yml:
version: '3'
services:
web:
build: ./web/Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
/web/Dockerfile.yml:
FROM php:7.2-apache
RUN docker-php-ext-install \
pdo_mysql \
&& a2enmod \
rewrite
When I try to use docker-compose up --build, I get the following:
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose up --build
Building web
Traceback (most recent call last):
File "bin/docker-compose", line 6, in <module>
File "compose/cli/main.py", line 71, in main
File "compose/cli/main.py", line 127, in perform_command
File "compose/cli/main.py", line 1052, in up
File "compose/cli/main.py", line 1048, in up
File "compose/project.py", line 466, in up
File "compose/service.py", line 329, in ensure_image_exists
File "compose/service.py", line 1047, in build
File "site-packages/docker/api/build.py", line 142, in build
TypeError: You must specify a directory to build in path
[6769] Failed to execute script docker-compose
I know that *.py that is python language files, but I do not use python language or work with it, I work with PHP.
Why is an error and how to fix it?
MODIFIED:
$ docker-compose up --build
Building webapp
Step 1/2 : FROM php:7.2-apache
---> a7d68dad7584
Step 2/2 : RUN docker-php-ext-install pdo_mysql && a2enmod rewrite
---> Using cache
---> 519d1b33af81
Successfully built 519d1b33af81
Successfully tagged votes_docker_webapp:latest
Starting votes_docker_adminer_1 ...
Starting votes_docker_composer_1 ...
Starting votes_docker_adminer_1 ... error
votes_docker_db_1 is up-to-date
ERROR: for votes_docker_adminer_1 Cannot start service adminer: driver failed programming external connectivity on endpoint votes_docker_adminer_1 (6e94693ab8b1a990aaa83164df0952e8665f351618a72aStarting votes_docker_composer_1 ... done
ERROR: for adminer Cannot start service adminer: driver failed programming external connectivity on endpoint votes_docker_adminer_1 (6e94693ab8b1a990aaa83164df0952e8665f351618a72a5531f9c3ccc18a2e3d): Bind for 0.0.0.0:8080 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
I tried to check related ports and got:
# sudo netstat -ntpl | grep 8080:8080
# sudo netstat -ntpl | grep 0.0.0.0:8080
# sudo netstat -ntpl | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 7361/docker-proxy
MODIFIED #2:
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose up --build
Creating network "votes_docker_default" with the default driver
Building webapp
Step 1/2 : FROM php:7.2-apache
---> a7d68dad7584
Step 2/2 : RUN docker-php-ext-install pdo_mysql && a2enmod rewrite
---> Using cache
---> 519d1b33af81
Successfully built 519d1b33af81
Successfully tagged votes_docker_webapp:latest
Creating votes_docker_adminer_1 ... done
Creating votes_docker_composer_1 ... done
Creating votes_docker_webapp_1 ... done
Creating votes_docker_db_1 ... done
Attaching to votes_docker_adminer_1, votes_docker_composer_1, votes_docker_webapp_1, votes_docker_db_1
adminer_1 | PHP 7.2.10 Development Server started at Mon Oct 15 10:14:02 2018
composer_1 | Composer could not find a composer.json file in /var/www/html
composer_1 | To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
votes_docker_composer_1 exited with code 1
webapp_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.4. Set the 'ServerName' directive globally to suppress this message
webapp_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.20.0.4. Set the 'ServerName' directive globally to suppress this message
webapp_1 | [Mon Oct 15 10:14:05.281793 2018] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.25 (Debian) PHP/7.2.10 configured -- resuming normal operations
webapp_1 | [Mon Oct 15 10:14:05.281843 2018] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
db_1 | 2018-10-15T10:14:06.541323Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
db_1 | 2018-10-15T10:14:06.541484Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.11) starting as process 1
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | mbind: Operation not permitted
db_1 | 2018-10-15T10:14:07.062202Z 0 [Warning] [MY-011071] [Server] World-writable config file './auto.cnf' is ignored.
db_1 | 2018-10-15T10:14:07.062581Z 0 [Warning] [MY-010107] [Server] World-writable config file './auto.cnf' has been removed.
db_1 | 2018-10-15T10:14:07.063146Z 0 [Warning] [MY-010075] [Server] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 0cd8212e-d063-11e8-8e69-0242ac140005.
db_1 | 2018-10-15T10:14:07.079020Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db_1 | 2018-10-15T10:14:07.091951Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
db_1 | 2018-10-15T10:14:07.103829Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.infoschema#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103896Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103925Z 0 [Warning] [MY-010315] [Server] 'user' entry 'mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.103947Z 0 [Warning] [MY-010315] [Server] 'user' entry 'root#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104006Z 0 [Warning] [MY-010323] [Server] 'db' entry 'performance_schema mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104034Z 0 [Warning] [MY-010323] [Server] 'db' entry 'sys mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.104070Z 0 [Warning] [MY-010311] [Server] 'proxies_priv' entry '# root#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.112700Z 0 [Warning] [MY-010330] [Server] 'tables_priv' entry 'user mysql.session#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.112738Z 0 [Warning] [MY-010330] [Server] 'tables_priv' entry 'sys_config mysql.sys#localhost' ignored in --skip-name-resolve mode.
db_1 | 2018-10-15T10:14:07.117764Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.11' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
Is it mysql misconfigure?
I run it not as root.
Also as I use LAMP do I need to stop apache and mysql before running docker-compose command?
MODIFIED #3:
After some searching I added mysql version in my config file and added command option:
image: mysql:5.7.23
command: --default-authentication-plugin=mysql_native_password --disable-partition-engine-check
and the above error was fixed.
So:
1. In other console under root I run commands (but I'm still not sure if I need it?)
sudo service apache2 stop
sudo service mysql stop
2. Under nonroot console I run with key to run in background:
docker-compose up -d
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker-compose down
Stopping votes_docker_db_1 ... done
Stopping votes_docker_webapp_1 ... done
Stopping votes_docker_adminer_1 ... done
Removing votes_docker_db_1 ... done
Removing votes_docker_webapp_1 ... done
Removing votes_docker_composer_1 ... done
Removing votes_docker_adminer_1 ... done
Removing network votes_docker_default
docker-compose up -d
Creating network "votes_docker_default" with the default driver
Creating votes_docker_webapp_1 ... done
Creating votes_docker_adminer_1 ... done
Creating votes_docker_db_1 ... done
Creating votes_docker_composer_1 ... done
I have no errors in output, but I expected as a result to have vendor directory in my project, as I have in web/Dockerfile.yml:
FROM php:7.2-apache
RUN docker-php-ext-install \
pdo_mysql \
&& a2enmod \
rewrite
But I do not see this directory...
Was the installation successful or not?
I'm not sure where to move next?
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker info
Containers: 33
Running: 2
Paused: 0
Stopped: 31
Images: 19
Server Version: 17.12.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9b55aab90508bd389d7654c4baf173a981477d55
runc version: 9f9c96235cc97674e935002fc3d78361b696a69e
init version: v0.13.0 (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.15.0-36-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.711GiB
Name: serge
ID: BDNU:HFWX:N6YV:IWYW:HJSU:SZ23:URPB:3FR2:7I3E:IFGK:AOLH:YRE5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
votes_docker_webapp latest 519d1b33af81 27 hours ago 378MB
adminer latest 0038b45402de 4 weeks ago 81.7MB
composer 1.6 e28b5b53ab28 4 weeks ago 154MB
php 7.2-apache a7d68dad7584 4 weeks ago 378MB
mysql 5.7.23 563a026a1511 5 weeks ago 372MB
mysql 5.7.22 6bb891430fb6 2 months ago 372MB
test2_php latest 05534d47f926 3 months ago 84.7MB
test1_php latest 05534d47f926 3 months ago 84.7MB
<none> <none> 6060fcf4d103 3 months ago 81MB
php fpm-alpine 601d5b3a95d4 3 months ago 80.6MB
php apache d9faf33e6e40 3 months ago 377MB
mysql latest 8d99edb9fd40 3 months ago 445MB
php 7-fpm 854ffd8dc9d8 3 months ago 367MB
php 7.2 e86d9bb526ef 3 months ago 367MB
ukfx/php apache-stretch 5958cb7c2316 4 months ago 648MB
nginx alpine bc7fdec94612 4 months ago 18MB
hello-world latest e38bc07ac18e 6 months ago 1.85kB
composer/composer latest 5afb0951f2a4 2 years ago 636MB
serge#serge:/mnt/_work_sdb8/wwwroot/lar/DockerApps/votes_docker$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f8beea5dceca mysql:5.7.23 "docker-entrypoint.s…" 6 minutes ago Restarting (2) 6 seconds ago votes_docker_db_1
8309b5456dcf adminer "entrypoint.sh docke…" 6 minutes ago Up 6 minutes 0.0.0.0:8081->8080/tcp votes_docker_adminer_1
cc644206931b votes_docker_webapp "docker-php-entrypoi…" 6 minutes ago Up 6 minutes 0.0.0.0:8080->80/tcp votes_docker_webapp_1
How to fix it?
Thanks!
According to docker-compose documentation, build can be specified as a string containing a path to the build context if you have Dockerfile inside it.
You are using Dockerfile.yml file, which is not a default one (Dockerfile), so in this case context and dockerfile should be specified as well:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
The final docker-compose.yaml is:
version: '3'
services:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
Addition to MODIFIED part:
Both web and adminer are configured to be allocated on port 8080 on a host system. That's why you have a conflict here. To resolve above issue you need to bind adminer to another port 8081 for example.
The final docker-compose.yaml is:
version: '3'
services:
web:
build:
context: ./web
dockerfile: Dockerfile.yml
environment:
- APACHE_RUN_USER=www-data
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
ports:
- 8080:80
working_dir: ${APP_PTH_CONTAINER}
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: 1
volumes:
- ${DB_PATH_HOST}:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8081:8080
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
Addition to MODIFIED3 part:
I see the following error with your composer docker container:
composer_1 | Composer could not find a composer.json file in /var/www/html
composer_1 | To initialize a project, please create a composer.json file as described in the https://getcomposer.org/ "Getting Started" section
Is it because you accidentally put ${DB_PATH_HOST} instead of ${APP_PATH_HOST} in the composer config?
composer:
image: composer:1.6
volumes:
- ${DB_PATH_HOST}:${APP_PTH_CONTAINER}
working_dir: ${APP_PTH_CONTAINER}
command: composer install
I am using docker-compose and nodemon for my dev.
my fs looks like this:
├── code
│ ├── images
│ │ ├── api
│ │ ├── db
│ └── topology
│ └── docker-compose.yml
Normally when I run docker-compose up --build, files are copied from my local computer to containers. Since I am in dev mode,
I don't want to run docker-compose up --build every time, that is why I am using volume to share a directory between my local computer and container.
I make some few research and this is what I come out with:
API, Dockerfile:
FROM node:8
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN npm install nodemon -g --save
RUN npm install
CMD [ "nodemon", "app.js" ]
DB, Dockerfile:
FROM mongo:3.2-jessie
docker-compose.yml
version: '2'
services:
api:
build: ../images/api
volumes:
- .:/usr/src/app
ports:
- "7000:7000"
links: ["db"]
db:
build: ../images/db
ports:
- "27017:27017"
The problem is that, when I run docker-compose up --build
I have this error:
---> 327987c38250
Removing intermediate container f7b46029825f
Step 7/7 : CMD nodemon app.js
---> Running in d8430d03bcd2
---> ee5de77d78eb
Removing intermediate container d8430d03bcd2
Successfully built ee5de77d78eb
Recreating topology_db_1
Recreating topology_api_1
Attaching to topology_db_1, topology_api_1
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=5b93871d0f4f
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] db version v3.2.21
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] git version: 1ab1010737145ba3761318508ff65ba74dfe8155
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] allocator: tcmalloc
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] modules: none
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] build environment:
db_1 | 2018-09-22T10:08:41.679+0000 I CONTROL [initandlisten] distmod: debian81
db_1 | 2018-09-22T10:08:41.680+0000 I CONTROL [initandlisten] distarch: x86_64
db_1 | 2018-09-22T10:08:41.680+0000 I CONTROL [initandlisten] target_arch: x86_64
db_1 | 2018-09-22T10:08:41.680+0000 I CONTROL [initandlisten] options: {}
db_1 | 2018-09-22T10:08:41.686+0000 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
db_1 | 2018-09-22T10:08:41.687+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=8G,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),verbose=(recovery_progress),
db_1 | 2018-09-22T10:08:41.905+0000 I STORAGE [initandlisten] WiredTiger [1537610921:904991][1:0x7fdf57debcc0], txn-recover: Main recovery loop: starting at 89/4096
db_1 | 2018-09-22T10:08:41.952+0000 I STORAGE [initandlisten] WiredTiger [1537610921:952261][1:0x7fdf57debcc0], txn-recover: Recovering log 89 through 90
db_1 | 2018-09-22T10:08:41.957+0000 I STORAGE [initandlisten] WiredTiger [1537610921:957000][1:0x7fdf57debcc0], txn-recover: Recovering log 90 through 90
db_1 | 2018-09-22T10:08:42.148+0000 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
db_1 | 2018-09-22T10:08:42.148+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
db_1 | 2018-09-22T10:08:42.148+0000 I NETWORK [initandlisten] waiting for connections on port 27017
api_1 | Usage: nodemon [nodemon options] [script.js] [args]
api_1 |
api_1 | See "nodemon --help" for more.
api_1 |
topology_api_1 exited with code 0
If I comment:
volumes:
-.:/usr/src/app
It compiles and run correctly.
Can someone help find what is what in my approach ?
Thanks
docker-compose.yml and api dockerFile are in different directory?“volume” instruction in compose file overwrite the ”copy” instruction。they have different context。