Manifest not found on docker composer even if the tag exists - docker

I have this YML file
docker-compose-testing.yml
with docker compose configuration:
version: '3'
services:
nginx_testing:
image: MY_SERVER_IP:5000/lens/nginx_testing:${VERSION}
volumes:
- certs:/etc/letsencrypt
- certs-data:/data/letsencrypt
ports:
- 80:80
- 443:443
depends_on:
- ws_server
- translator
- auth
ws_server:
image: MY_SERVER_IP:5000/lens/ws_server:${VERSION}
worker:
image: MY_SERVER_IP:5000/lens/worker:${VERSION}
depends_on:
- ws_server
translator:
image: MY_SERVER_IP:5000/lens/translator:${VERSION}
auth:
image: MY_SERVER_IP:5000/lens/auth:${VERSION}
volumes:
- auth-data:/usr/src/app/data
volumes:
certs:
certs-data:
auth-data:
Normally, I use this command to apply the above configuration:
export VERSION=578d8de && envsubst < docker-compose-testing.yml | docker-compose -f - pull && envsubst < docker-compose-testing.yml | docker-compose -f - -p PROJECT_NAME up -d --no-build --scale worker=5
Now, when I execute this command (above) the console show this error:
Pulling translator (MY_SERVER_IP:5000/lens/translator:578d8de)...
ERROR: manifest for MY_SERVER_IP:5000/lens/translator:578d8de not found
The response for one similar question below says that the tag do not exists:
Error response from daemon: manifest for ibmblockchain/fabric-peer:latest not found
But when I list the images with command:
docker images | grep 578d8de
Console show this output proving that the tag exists:
MY_SERVER_IP:5000/lens/auth 578d8de 8103c4d63870 2 hours ago 195MB
MY_SERVER_IP:5000/lens/nginx_testing 578d8de 578d8dead150 4 hours ago 235MB
MY_SERVER_IP:5000/lens/translator 578d8de e9eb25fa0aef 5 hours ago 185MB
MY_SERVER_IP:5000/lens/ws_server 578d8de 92b1d1a4cee9 5 hours ago 177MB
MY_SERVER_IP:5000/lens/worker 578d8de 22a935deba5c 7 days ago 175MB
Some extra details:
The server (MY_SERVER_IP) has a docker registry listening on 5000 port.
The image with version tag 578d8de was uploaded to the server not with registry, but with the "docker save" and "scp" command on the dev machine, and "docker load" on the server.
Any idea why this error is occurring?

This error is happenning when I send image with scp command to server and load the image to docker.
How I don't use docker push to the registry, the image don't exists in registry.
So when docker-compose pull execute, don't find image on registry and dispatch the error.

Related

Docker Compose - give access to docker.sock before running (Telegraf)

UPDATE:
I've tried implementing the accepted answer from here Telegraf can not connect to Docker sock like this in my docker compose file:
telegraf3:
image: telegraf
user: telegraf:$$(stat -c '%g' /var/run/docker.sock)
volumes:
- ./telegraf/telegraf3.conf:/etc/telegraf/telegraf.conf
- /var/run/docker.sock:/var/run/docker.sock
I am getting this error:
Error response from daemon: unable to find group $(stat -c '%g' /var/run/docker.sock): no matching entries in group file
How can i fix this issue? :)
Background:
I'm trying to run Telegraf (https://github.com/influxdata/telegraf) with the docker input. I'm running Telegraf via Docker Compose, and i've configured it roughly like this:
telegraf3:
image: telegraf
volumes:
- ./telegraf/telegraf3.conf:/etc/telegraf/telegraf.conf
- /var/run/docker.sock:/var/run/docker.sock:rw
env_file:
- ./telegraf/telegraf.env
depends_on:
- influxdb
The telegraf configuration uses a docker input plugin to interact with the docker.sock. It doesn't work, i get a permission related error:
test-grafana-telegraf3-1 | 2022-12-29T12:09:10Z E! [inputs.docker] Error in plugin: Got permission denied while trying to connect to the Docker daemon socket at unix:///docker.sock: Get "http://%2Fdocker.sock/v1.24/info": dial unix /docker.sock: connect: permission denied
Basically, the entrypoint.sh script runs telegraf (the application) with the telegraf user, which can't access the docker.sock
There's a fix for this issue described here: Telegraf can not connect to Docker sock
Issue
As i am using docker compose, i would like this fix to be defined in the compose file, and not be dependent on me starting the container with docker run.
I've tried this:
telegraf3:
image: telegraf
volumes:
- ./telegraf/telegraf3.conf:/etc/telegraf/telegraf.conf
- /var/run/docker.sock:/var/run/docker.sock:rw
env_file:
- ./telegraf/telegraf.env
depends_on:
- influxdb
command: ["bash -c -u telegraf $$(stat -c '%g' /var/run/docker.sock)", "/entrypoint.sh"]
But then i get this error:
test-grafana-telegraf3-1 | setpriv: failed to execute bash -c -u telegraf $(stat -c '%g' /var/run/docker.sock): No such file or directory
test-grafana-telegraf3-1 exited with code 127

GitLab CI, connect from docker dind to Elastic Search service

I have tests which run in a docker container. For this, I use docker-dind service, my .gitlab-ci:
image: "docker:17"
variables:
DOCKER_DRIVER: overlay2
services:
- docker:dind
- name: docker.elastic.co/elasticsearch/elasticsearch:5.5.2
alias: elasticsearch
command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
stages:
- test
before_script:
- apk --update add py2-pip python3 bash zip ansible openssh git docker-py curl
- pip3 install docker-compose
- docker info
- docker-compose --version
# Login to registry.gitlab.com
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
test:
script:
- curl "http://elasticsearch:9200" # this works
- docker-compose docker-compose.test.yml build --pull
- docker-compose docker-compose.test.yml run app
stage: test
My tests use ES for this I added ES service but I can't connect to ES cluster from my container where I run tests.
On my machine with runner when CI works I have:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f5d9a64bbe8e 1ee5a99eba5f "gitlab-runner-build" 1 second ago Up Less than a second runner-b7dcaf73-project-2199705-concurrent-1-predefined-0
c85c49d35946 ca27036dd5e7 "bin/elasticsearch -…" 16 seconds ago Up 15 seconds 9200/tcp, 9300/tcp runner-b7dcaf73-project-2199705-concurrent-1-docker.elastic.co__elasticsearch__elasticsearch-1
57472d0300ad 85e924caedbd "dockerd-entrypoint.…" 17 seconds ago Up 16 seconds 2375/tcp runner-b7dcaf73-project-2199705-concurrent-1-docker-0
598c019aa28c - a container with a runner, I can enter to this container run curl "http://elasticsearch:9200" and it works
57472d0300ad - dind container, right? I can enter to this container but curl "http://elasticsearch:9200" doesn't work, docker ps shows:
/ # docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
252f0588a41d backend_app "make _inside_docker…" 6 seconds ago Up 5 seconds 8080/tcp backend_app_run_116d12907320
cd0ebb2f1d2d postgres:9.6 "docker-entrypoint.s…" 7 seconds ago Up 6 seconds 5432/tcp backend_postgresql_1
How I can connect from my container with tests (252f0588a41d) to container with ES?
Thanks.

Prisma Deploy Docker error "Could not connect to server"

This is steps I have done
prisma init
I set postgresql for database in my local(not exist).
It created 3 files, datamodel.graphql, docker-compose.yml, prisma.yml
docker-compose up -d
I confirmed it running successfully
But if I call prisma deploy, it shows me error
Could not connect to server at http://localhost:4466. Please check if your server is running.
All I have done is standard operation described in manual and there is no customization in
https://www.prisma.io/docs/tutorials/deploy-prisma-servers/local-(docker)-meemaesh3k
And this is docker-compose.yml
version: '3'
services:
prisma:
image: prismagraphql/prisma:1.11
restart: always
ports:
- "4466:4466"
environment:
PRISMA_CONFIG: |
port: 4466
# uncomment the next line and provide the env var PRISMA_MANAGEMENT_API_SECRET=my-secret to activate cluster security
# managementApiSecret: my-secret
databases:
default:
connector: postgres
host: localhost
port: '5432'
database: databasename
schema: public
user: postgres
password: root
migrations: true
What am I missing?
I found this solution to the same problem i was facing
docker-machine ip default
Use this address and replace the "localhost" with the IP with the above command to look something like this in prisma.yml file
endpoint: http://1xx.1xx.xx.xxx:4466
The answer is referred from this Github Link
The documentation mentions:
docker ps
You should see output similar to this:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b799c529e73 prismagraphql/prisma:1.7 "/bin/sh -c /app/sta…" 17 hours ago Up 7 hours 0.0.0.0:4466->4466/tcp myapp_prisma_1
757dfba212f7 mysql:5.7 "docker-entrypoint.s…" 17 hours ago
(Here shown with mysql, but valid with postgresql too)
The point is: there should be two containers running, not one.
Check docker-compose logs to see why the second one (database) did not start.
instead of docker-compose up -d
USE:
docker-compose up
and keep the window running which will keep localhost:4466 alive.
Note : If u want to connect to connect to the database created in docker, you need to map the port in the following way:
docker run --name <ENTER_NAME> -e POSTGRES_PASSWORD=<ENTER_PASSWORD> -d -p 5433:5432 postgres
In the above case PORT(5433) = HOST_PORT and PORT(5432) = CONTAINER_PORT

Connection to docker container failing because of postgis port issue

My docker container is able to successfully build but when I enter the command docker-compose build, the following error is returned:
Starting docker_etl_1 ...
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ...
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ... error
ERROR: for 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland prStarting docker_etl_1 ... done
ERROR: for postgis Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland proxy: Bind for 0.0.0.0:5432 failed: port is already allocated
Here is my docker-compose.yaml
version: '2'
services:
postgis:
build: ./postgis
volumes:
- ../src/main/sql:/sql
ports:
- "5432:5432"
etl:
build: ./etl
volumes:
- ..:/national-voter-file
entrypoint:
- python3
- /national-voter-file/load/loader.py
and here is the Dockerfile:
FROM mdillon/postgis:9.5
ENV POSTGRES_DB VOTER
RUN mkdir /sql
COPY ./dockerResources/z-init-db.sh /docker-entrypoint-initdb.d/
EXPOSE 5432
Docker ps -a returns:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
da74ad97b95c docker_postgis "docker-entrypoint..." About a minute ago Created docker_postgis_1
5872c6e55fe2 docker_etl "python3 /national..." About a minute ago Exited (2) About a minute ago docker_etl_1
However, when I try rm $(docker ps -qa) I get the following error:
rm: da74ad97b95c: No such file or directory
rm: 5872c6e55fe2: No such file or directory
I don't believe I have another container running so I'm confused by the message Bind for 0.0.0.0:5432 failed: port is already allocated
Is it possible that you ran the same docker-compose earlier, which probably failed or at least failed to clean up the services?
Try running docker ps -a to check if any stopped containers exist. It is possible that the stopped containers are hogging the port. If so, just clear them out using docker rm $(docker ps -qa)

Docker compose yml inheritance

There are two tasks: run app container, run almost the same deploy-app container. The differences for them, for example, that deploy container does not have port sharing.
So, I made configs for this tasks...
./dockerfiles/base.yml:
app:
net: docker_internal_net
environment:
APPLICATION_SERVER: "docker"
./dockerfiles/base.run.yml:
app:
container_name: project-app
# set the build context to the project root
build: ..
volumes:
- /var/log/project/nginx:/var/log/nginx
- /var/log/project/php-fpm:/var/log/php5-fpm
- ..:/var/www/project
./dockerfiles/dev/run.yml:
app:
dockerfile: ./dockerfiles/dev/run-app/Dockerfile
ports:
- "80:80"
- "22:22"
environment:
DEV_SSH_PUBKEY: "$SSH_PUBLIC_KEY"
APPLICATION_PLATFORM: "dev"
./dockerfiles/dev/build.yml:
app:
container_name: project-app-deploy
# set the build context to the project root
build: ../..
dockerfile: ./dockerfiles/dev/build-app/Dockerfile
environment:
APPLICATION_PLATFORM: "dev"
volumes:
- ../..:/var/www/project
So I can tun the app container like this:
$ docker-compose -f ./dockerfiles/base.yml -f ./dockerfiles/base.run.yml -f ./dockerfiles/dev/run.yml up -d app
Creating project-app
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dae45f3f2c42 dockerfiles_app "/sbin/my_init" 2 seconds ago Up 1 seconds 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
Everything okay. But if then I trying to run deploy-app container, I will receive this message:
$ docker-compose -f ./dockerfiles/base.yml -f ./dockerfiles/dev/build.yml up -d app
Recreating project-app
WARNING: Service "app" is using volume "/var/www/project" from the previous container. Host mapping ".." has no effect. Remove the existing containers (with `docker-compose rm app`) to use the host volume mapping.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
53059702c09b dockerfiles_app "/sbin/my_init" 6 seconds ago Up 4 seconds 22/tcp, 80/tcp project-app-deploy
This is because both of them are shared one local directory? But why I can run deploy-app container manually without docker-compose?
$ docker run -d --net docker_internal_net -e APPLICATION_SERVER=docker -e APPLICATION_PLATFORM=dev --name project-app-deploy -v ..:/var/www/project mybaseimage
86439874b8df561f529fde0d1e31824d70dc7e2a2377cd529331a2d7fcb00467
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
86439874b8df mybaseimage "/sbin/my_init" 4 seconds ago Up 3 seconds 22/tcp, 80/tcp project-app-deploy
40641f02a09b dockerfiles_app "/sbin/my_init" 2 minutes ago Up 2 minutes 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
I've solved my problem with the extend command in that way:
1) making changes into my ./dockerfiles/dev/build.yml file:
deploy-app:
extends:
file: ../base.yml
service: app
container_name: project-app-deploy
# set the build context to the project root
build: ../..
dockerfile: ./dockerfiles/dev/build-app/Dockerfile
environment:
APPLICATION_PLATFORM: "dev"
volumes:
- ../..:/var/www/project
2) run my deploy app container so:
$ docker-compose -f ./dockerfiles/dev/build.yml up -d deploy-app
Building deploy-app
...
Successfully built 74750fe274c6
Creating lovetime-app-deploy
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9bb2af79ffaa dev_deploy-app "/sbin/my_init" 5 seconds ago Up 4 seconds 22/tcp, 80/tcp project-app-deploy
812b8824f1f0 dockerfiles_app "/sbin/my_init" 3 minutes ago Up 3 minutes 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
$ docker inspect -f '{{ .Mounts }}' project-app-deploy
[{ ...... /var/www/project rw true}]
Update:
According to documentation, this command is not supported in newer compose versions:
The extends keyword is supported in earlier Compose file formats up to Compose file version 2.1 (see extends in v1 and extends in v2), but is not supported in Compose version 3.x.

Resources