Installed Versions
Application
Version
Docker
19.03.6, build 369ce74a3c
Docker Compose
v2.13.0
OS
Ubuntu 18.04.2 LTS
Docker definitions
docker-compose.yml
version: "2.3"
services:
builder:
build:
context: ./
dockerfile: Dockerfile
args:
- NODE_VERSION=${NODE_VERSION:-12.22.7}
image: redacted/node:${NODE_VERSION}
volumes:
- ./:/code
environment:
- BUILDKIT_PROGRESS=plain
.env
CLIENT=${CLIENT_PREFIX:-xx}
PUBLIC_URL=/${CLIENT}-dashboard
REACT_APP_NODEJS_SERVER=${CLIENT}-server
NODE_VERSION=12.22.7
Dockerfile
ARG NODE_VERSION="12.22.7"
FROM node:${NODE_VERSION}
VOLUME ["/code"]
WORKDIR /code
CMD "/code/build_ui.sh"
Issue description
Our project requires multiple versions of node & npm installed. To avoid compatibility issues, we are trying to use docker to stabilize the versions we need.
We use the below command to run the build for our application:
docker-compose run --rm builder
This works on some of our servers, but on some servers, I get either of the below errors:
failed to solve: failed to solve with frontend dockerfile.v0: failed to build llb: failed to load cache key: rpc error: code = unknown desc = error getting credentials - err: exit status 1, out: cannot autolaunch d-bus without x11 $display ubuntu.
Fixed this by following the guide here. However, given that I was trying to pull node and that is a public repo, I don't understand why docker was attempting a login. And why didn't this happen on all servers?
docker run Error response from daemon: No command specified
Fixed this temporarily by manually building the image using docker build command. But I really want the docker-compose to be able to build the image if the image doesn't exist.
I was expecting docker-compose to build the image on first run without issues and run the build_ui.sh on container execution.
When I get the above errors, if I manually build the docker image (and not wait for docker-compose to build it) using the below command, and then use the docker-compose run command it works.
docker build -t redacted/node:12.22.7 .
I am trying to figure out why docker-compose is not building the image correctly when the image doesn't exist.
Related
I get the error:
failed to solve with frontend dockerfile.v0: failed to create LLB definition: no match for platform in manifest
when building the following Dockerfile:
FROM mcr.microsoft.com/dotnet/framework/aspnet:4.8
COPY . /inetpub/wwwroot
The cause was simple. I had my Docker desktop running on Linux containers and the image was build from a Windows image.
Simply switching to Windows containers solved the problem.
The message is clueless, so I hope this save some time for others.
Providing the platform in the Docker file on M1 fixed it for me.
E.g., FROM --platform=linux/amd64 amazonlinux:2018.03
In my case, I was using a Mac with an M1 processor to run a Python image. My docker-compose and Dockerfile looked like this:
docker-compose.yml
version: '3.7'
services:
words_bot:
build: .
restart: unless-stopped
Dockerfile:
FROM python:3-onbuild
COPY . /usr/src/app
CMD ["python", "-m", "bot"]
It seems like the image was expecting an x86 host architecture, so I was getting the error the OP is referring to.
After I added platform: linux/amd64 into docker-compose.yml everything started working as expected:
version: '3.7'
services:
cng_words_bot:
build: .
platform: linux/amd64
restart: unless-stopped
Docker gets confused with some architectures like the ARM architecture(M1 for instance). Make sure to specify the architecture (platform).
services:
service-name:
platform: linux/x86_64. # Specify the architecture here
image: some-image
Update:
If you're using an Apple Silicon Chip machine (arm architecture), you shall
activate on you Docker Desktop the following features for better virtualization:
Use Virtualization framework
Use Rosetta for x86/amd64 emulation on Apple Silicon
It will use the Rosetta 2 emulator instead of the qemu one,
The difference is just outstanding.
(Assuming you are running Docker on a Windows platform) To resolve this issue, switch the container from Linux to Windows by right clicking on the Docker icon in the tray (we see this icon near the system clock after starting the Docker Engine) and select the option "Switch to Windows containers..."
Step 1
Step 2
For me, the Docker image itself was not building. So I had to add --platform linux/x86_64 as parameter for docker build command.
I just stumbled over a similar issue myself when using Docker build on a very simple Dockerfile:
FROM node:lts-alpine
COPY ./ /app/
RUN cd /app && npm ci && npm run build
When running docker build -t foo ., the OP's error with a slightly different cause came up.
However, when running docker pull node:lts-alpine first, then repeating that build command, the build was running just fine.
IMHO, this looks like a hiccup in Docker for Windows. Switching to Windows containers didn't seem like a reasonable option here for the base container is pretty valid for a Linux-based context. I tried to switch anyway, but that was bringing up a different error of Docker for Windows, only.
On macOS with an Intel chip, building a "standard" Docker image, I ran into this.
Restarting the Docker daemon fixed it for me.
In my case. I was using a Mac M1, and it was missing a /, like the following:
How it was:
version: '3.6'
services:
service-name:
build:
# It was without the /, like:
context: .
Fixed:
version: '3.6'
services:
service-name:
build:
# The correct one
context: ./
The error was:
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: unexpected status code [manifests latest]: 401 Unauthorized
And also I needed to change the credsStore value. More information: docker pull gives error: no basic auth credentials #207
I hope that it helps someone, I spent a lot of time to get it.
Searching for a Docker image tag that works for your hardware architecture fixes it. For instance, if you are using Apple silicon (M1 or M2), the architecture is arm64.
In my case I was simply disconnected from the wifi, so double-check that you're online before spending hours trying to debug the issue!
Summary:
I have an application X, I want to deploy multiple instances of the same application (port numbers will be handled by an .env) in the same OS without starting a build for each instance.
What I tried:
So I managed to dynamically (by the user changing .env file), change the container_name of a container. But then we cannot run 5 instances at the same time (even if the ports are different, docker just stops the first re-creates the container for second)
Next I came across COMPOSE_PROJECT_NAME that seems to work BUT starts a new build.
COMPOSE_PROJECT_NAME=hello-01
docker-compose up
Creating network "hello-01_default" with the default driver
Building test
Step 1/2 : FROM ubuntu:latest
---> 113a43faa138
Step 2/2 : RUN echo Hello
---> Using cache
---> ba846acc19e5
Successfully built ba846acc19e5
Successfully tagged hello-01_test:latest
WARNING: Image for service test was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating hello-01_test ... done
Attaching to hello-01_test
hello-01_test exited with code 0
COMPOSE_PROJECT_NAME=hello-2
docker-compose up
Creating network "hello-02_default" with the default driver
Building test
Step 1/2 : FROM ubuntu:latest
---> 113a43faa138
Step 2/2 : RUN echo Hello
---> Using cache
---> ba846acc19e5
Successfully built ba846acc19e5
Successfully tagged hello-02_test:latest
WARNING: Image for service test was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating hello-02_test ... done
Attaching to hello-02_test
hello-02_test exited with code 0
Source files
docker-compose.yml
version: '3'
services:
test:
container_name: "${COMPOSE_PROJECT_NAME}_test"
build: .
.env
COMPOSE_PROJECT_NAME=hello-02
Dockerfile
FROM ubuntu:latest
RUN echo Hello
Ubuntu 18.04.1 LTS
Docker version 18.06.0-ce, build 0ffa825
docker-compose version 1.21.2, build a133471
By changing the container name without providing an image: reference the compose file has no idea that you've already built that image. So if you build that docker image as some local image example/image/local, you can addimage: example/image/localto your docker-compose file and do that to spawndocker-compose up -d` many times by changing the name with an environment variable in your example.
However, it appears that you might want to look into using replicas instead of making this a horrifically manual effort outside of on the one-line full up that you'd get out of docker-compose.
https://docs.docker.com/compose/compose-file/#short-syntax
I have gitlab-runner installed locally.
km#Karls-MBP ~ $ gitlab-runner --version
Version: 10.4.0
Git revision: 857480b6
Git branch: 10-4-stable
GO version: go1.8.5
Built: Mon, 22 Jan 2018 09:47:12 +0000
OS/Arch: darwin/amd64
Docker:
km#Karls-MBP ~ $ docker --version
Docker version 17.12.0-ce, build c97c6d6
.gitlab-ci.yml:
image: docker/compose:1.19.0
before_script:
- echo wtf
test:
script:
- echo test
Results:
km#Karls-MBP ~ $ sudo gitlab-runner exec docker --docker-privileged test
WARNING: Since GitLab Runner 10.0 this command is marked as DEPRECATED and will be removed in one of upcoming releases
WARNING: You most probably have uncommitted changes.
WARNING: These changes will not be tested.
Running with gitlab-runner 10.4.0 (857480b6)
on ()
Using Docker executor with image docker/compose:1.19.0 ...
Using docker image sha256:be4b46f2adbc8534c7f6738279ebedd6106969695f5e596079e89e815d375d9c for predefined container...
Pulling docker image docker/compose:1.19.0 ...
Using docker image docker/compose:1.19.0 ID=sha256:e06b58ce9de2ea3f11634e022ec814984601ea3a5180440c2c28d9217b713b30 for build container...
Running on runner--project-0-concurrent-0 via x.x.x...
Cloning repository...
Cloning into '/builds/project-0'...
done.
Checking out b5a262c9 as km/ref...
Skipping Git submodules setup
No such command: sh
Commands:
build Build or rebuild services
bundle Generate a Docker bundle from the Compose file
config Validate and view the Compose file
create Create services
down Stop and remove containers, networks, images, and volumes
events Receive real time events from containers
exec Execute a command in a running container
help Get help on a command
images List images
kill Kill containers
logs View output from containers
pause Pause services
port Print the public port for a port binding
ps List containers
pull Pull service images
push Push service images
restart Restart services
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
top Display the running processes
unpause Unpause services
up Create and start containers
version Show the Docker-Compose version information
Don't really know what the issue is.
It seems that the docker/compose image is configured with docker-compose as an entrypoint.
You can override the default entrypoint of the docker/compose image in your .gitlab-ci.yml file :
image:
name: docker/compose:1.19.0
entrypoint: [""]
before_script:
- echo wtf
test:
script:
- echo test
The docker/compose image has the command docker-compose as its entrypoint (until version 1.24.x), which enables a usage similar to this (assuming a compatible volume mount):
docker run --rm -t docker/compose -f some-dir/compose-file.yml up
Unfortunately that same feature makes it incompatible with usage within GitLab CI’s Docker Runner. Theoretically you could have a construct like this:
job-name:
image: docker/compose:1.24.1
script:
- up
- --build
- --force-recreate
But the GitLab Docker Runner assumes the entrypoint is /bin/bash - or at least functions likewise (many Docker images thoughtfully use a shell script with "$#" as its final line for the entrypoint) - and from the array elements that you specify for the script, it creates its own temporary shell script on the fly. It starts with statements like set -e and set -o pipeline and will be used in a statement like sh temporary-script.sh as the container command. That’s what causes the unexpected error message you got.
This behaviour was recently documented more clearly:
The Docker executor doesn’t overwrite the ENTRYPOINT of a Docker image.
That means that if your image defines the ENTRYPOINT and doesn’t allow to run scripts with CMD, the image will not work with the Docker executor.
Overriding the entrypoint with [""] will allow usage of docker/docker-compose (before version 1.25.x) with the Docker Runner, but the script that GitLab will create on the fly is not going to run as process 1 and because of that the container will not stop at the end of the script. Example:
job-name:
image:
name: docker/docker-compose
entrypoint: [""]
script:
- docker-compose
- up
- --build
- --force-recreate
At the time I write this the latest version of docker/docker-compose is 1.25.0-rc2. Your mileage may vary, but it suffices for my purposes and entirely resolves both problems.
I recently used the docker image gitlab/gitlab-runner:9.1.0 in conjunction with a gitlab container to have some CI.
An error occurs and similar support requests recommended using a different version, so I tried with :latest and some :1.11 too.
Unfortunately it keeps telling me this error:
Running with gitlab-ci-multi-runner 1.11.4 (5e7ba4a)
on foo (02cdacdc)
Using Docker executor with image pretzlaw/php:7.1-apache ...
Starting service mariadb:latest ...
Pulling docker image mariadb:latest ...
Waiting for services to be up and running...
Pulling docker image pretzlaw/php:7.1-apache ...
Running on runner-02cdacdc-project-7-concurrent-0 via 9d1d33dc9212...
Fetching changes...
HEAD is now at 7580815 QA: CI Lint
From http://idgaf.example.org/foo/bar
7580815..affeede develop -> origin/develop
Checking out affeede as develop...
Skipping Git submodules setup
[: 1: [: Syntax error: end of file unexpected
[: 1: [: Syntax error: end of file unexpected
ERROR: Job failed: exit code 2
Neither do I know how to debug this nor do I see any problem in my container or test script. This is the .gitlab-ci.yml:
before_script:
- composer install
test_7_1:
image: pretzlaw/php:7.1-apache
script: ls
It could be a problem of the container somewhere but I don't get it. Doing this manually (with the recent failed docker container) everything works fine:
docker container exec 68c7b5448a56 ls
bin
builds
...
How do I trace back the problem?
What is it all about?
It's for GitLab 9.1.1-ce.0.
As pointed out by #1550, the problem seems to be coming from the shell detection done in the bash.go file between lines 16-31 which it is injected without newlines and thus creating the syntax error you are yourself experiencing.
Since you have a custom entrypoint in your Dockerfile and it looks like you have not passed quotes to the arguments of the exec here, that's what I think is failing and should be modified.
Change it to
exec "${*}"
As a workaround to the bug mentioned by Balthazar, try overriding the entrypoint in .gitlab-ci.yml as illustrated in the GitLab docs:
For Docker 17.06 and later:
image:
name: super/sql:experimental
entrypoint: [""]
For Docker 17.03 and earlier:
image:
name: super/sql:experimental
entrypoint: ["/bin/sh", "-c"]
I have a docker-compose.yml file, which defines a services and its image.
service:
image: my_image
now, that I run docker-compose up I get the following message:
$ docker-compose up
Pulling service (my_image:latest)...
Pulling repository docker.io/library/my_image
ERROR: Error: image library/my_image:latest not found
It is correct, that my_image in this case is not on the docker hub. But I've created it with docker build -t my_image . (in a different file) before and it is listed in docker images.
Is there anything I miss to tell docker-compose, to not look for the image in the docker.io registry/hub?
[edit] docker client and server version is 1.9.1, docker-compose version is 1.5.2.
I'm running docker-compose (as well as docker) through the HTTP-API on a remote machine, don't know if this makes any difference.
If you have image local or anywhere except docker hub you need to use build and path or url to Dockerfile. So basically when we work OFF dockerhub we change image to path !
ubuntu:
container_name: ubuntu
build: /compose/build/ubuntu
links:
- db:mysql
ports:
- 80:80
In this example am using my own Ubuntu Dockerfile that is places in the build path. The file should be named Dockerfile like normal and you just specify the path to folder where it is.