I recently used the docker image gitlab/gitlab-runner:9.1.0 in conjunction with a gitlab container to have some CI.
An error occurs and similar support requests recommended using a different version, so I tried with :latest and some :1.11 too.
Unfortunately it keeps telling me this error:
Running with gitlab-ci-multi-runner 1.11.4 (5e7ba4a)
on foo (02cdacdc)
Using Docker executor with image pretzlaw/php:7.1-apache ...
Starting service mariadb:latest ...
Pulling docker image mariadb:latest ...
Waiting for services to be up and running...
Pulling docker image pretzlaw/php:7.1-apache ...
Running on runner-02cdacdc-project-7-concurrent-0 via 9d1d33dc9212...
Fetching changes...
HEAD is now at 7580815 QA: CI Lint
From http://idgaf.example.org/foo/bar
7580815..affeede develop -> origin/develop
Checking out affeede as develop...
Skipping Git submodules setup
[: 1: [: Syntax error: end of file unexpected
[: 1: [: Syntax error: end of file unexpected
ERROR: Job failed: exit code 2
Neither do I know how to debug this nor do I see any problem in my container or test script. This is the .gitlab-ci.yml:
before_script:
- composer install
test_7_1:
image: pretzlaw/php:7.1-apache
script: ls
It could be a problem of the container somewhere but I don't get it. Doing this manually (with the recent failed docker container) everything works fine:
docker container exec 68c7b5448a56 ls
bin
builds
...
How do I trace back the problem?
What is it all about?
It's for GitLab 9.1.1-ce.0.
As pointed out by #1550, the problem seems to be coming from the shell detection done in the bash.go file between lines 16-31 which it is injected without newlines and thus creating the syntax error you are yourself experiencing.
Since you have a custom entrypoint in your Dockerfile and it looks like you have not passed quotes to the arguments of the exec here, that's what I think is failing and should be modified.
Change it to
exec "${*}"
As a workaround to the bug mentioned by Balthazar, try overriding the entrypoint in .gitlab-ci.yml as illustrated in the GitLab docs:
For Docker 17.06 and later:
image:
name: super/sql:experimental
entrypoint: [""]
For Docker 17.03 and earlier:
image:
name: super/sql:experimental
entrypoint: ["/bin/sh", "-c"]
Related
Installed Versions
Application
Version
Docker
19.03.6, build 369ce74a3c
Docker Compose
v2.13.0
OS
Ubuntu 18.04.2 LTS
Docker definitions
docker-compose.yml
version: "2.3"
services:
builder:
build:
context: ./
dockerfile: Dockerfile
args:
- NODE_VERSION=${NODE_VERSION:-12.22.7}
image: redacted/node:${NODE_VERSION}
volumes:
- ./:/code
environment:
- BUILDKIT_PROGRESS=plain
.env
CLIENT=${CLIENT_PREFIX:-xx}
PUBLIC_URL=/${CLIENT}-dashboard
REACT_APP_NODEJS_SERVER=${CLIENT}-server
NODE_VERSION=12.22.7
Dockerfile
ARG NODE_VERSION="12.22.7"
FROM node:${NODE_VERSION}
VOLUME ["/code"]
WORKDIR /code
CMD "/code/build_ui.sh"
Issue description
Our project requires multiple versions of node & npm installed. To avoid compatibility issues, we are trying to use docker to stabilize the versions we need.
We use the below command to run the build for our application:
docker-compose run --rm builder
This works on some of our servers, but on some servers, I get either of the below errors:
failed to solve: failed to solve with frontend dockerfile.v0: failed to build llb: failed to load cache key: rpc error: code = unknown desc = error getting credentials - err: exit status 1, out: cannot autolaunch d-bus without x11 $display ubuntu.
Fixed this by following the guide here. However, given that I was trying to pull node and that is a public repo, I don't understand why docker was attempting a login. And why didn't this happen on all servers?
docker run Error response from daemon: No command specified
Fixed this temporarily by manually building the image using docker build command. But I really want the docker-compose to be able to build the image if the image doesn't exist.
I was expecting docker-compose to build the image on first run without issues and run the build_ui.sh on container execution.
When I get the above errors, if I manually build the docker image (and not wait for docker-compose to build it) using the below command, and then use the docker-compose run command it works.
docker build -t redacted/node:12.22.7 .
I am trying to figure out why docker-compose is not building the image correctly when the image doesn't exist.
So I wrote a simple one-page server with node and express. I wrote a dockerfile for this and ran it locally. Then I made a postman collection and tested the endpoints.
I want to do this with gitlab ci using newman so I came up with the following .gitlab-ci.yml:
image: docker:latest
services:
- docker:dind
before_script:
- docker build -t test_img .
- docker run -d -p 3039:3039 test_img
stages:
- test
# test
api-test:
image:
name: postman/newman:alpine
entrypoint: [""]
stage: test
script:
- newman run pdfapitest.postman_collection.json
It fails saying:
docker build -t test_img .
/bin/sh: eval: line 86: docker: not found
ERROR: Job failed: exit code 127
full output: https://pastebin.com/raw/C3mmUXKa
what am I doing wrong here? this seems to me like a very common use case but I haven't found anything useful about this.
The issue is that your api-test job uses the image postman/newman:alpine to run the script.
This means that when GitLab tries to run the before_script section, it has no docker command available.
What you should do is to provide the docker command in the image you're using to run the job. You can do that either by installing docker as the first step of your script, or starting from a custom image which contains the software you're using inside the job plus the docker client itself.
I'm just getting started with docker and continuous integration with Gitlab. I've added the following gitlab-ci.yml file to the root of my repository:
# Official docker image
image: docker:latest
services:
- docker:dind
build-dev:
stage: build
script:
- docker build -t obikerui/project -f app/Dockerfile.dev ./app
test:
stage: test
script:
- docker run obikerui/project npm run test -- --coverage
The build-dev stage runs and passes but the test stage fails with the following error message:
$ docker run obikerui/project npm run test -- --coverage
Unable to find image 'obikerui/project:latest' locally
docker: Error response from daemon: pull access denied for obikerui/project, repository does not exist or may require 'docker login'.
See 'docker run --help'.
ERROR: Job failed: exit code 125
Can anyone explain what's going wrong and suggest a fix? The repository is private, so do I need to provide some extra configuration to accommodate this?
Each job runs in a different container. You build and you tag your image correctly but that stays in that container.
For the test job a new container starts and that one does not have the image build by the previous job.
You should push your image to a registry (after you tag it accordingly) and then the test job should use the image from the repository.
You can use a public registry like the one offered by Docker or you can run a local container based on the image registry:2 provided by docker. In this case you have to make sure that the domain name pointing to the registry is available on your network (it can be an nginx with reverse proxy)
I have my own CI server with gitlab and I'm trying to run docker runner (version 10.6) with this configuration:
image: php:7.1
services:
- mysql:latest
- redis:latest
- elasticsearch:latest
before_script:
- bash ci/install.sh > /dev/null
- php composer install -a
stages:
- test
test:
stage: test
variables:
API_ENVIRONMENT: 'test'
script:
- echo "Running tests"
- php composer app:tests
But everytime when I pull docker container with elastic, I've got error message:
*** WARNING: Service runner-1de473ae-project-225-concurrent-0-elasticsearch-2 probably didn't start properly.
Error response from daemon: Conflict. The container name "/runner-1de473ae-project-225-concurrent-0-elasticsearch-2-wait-for-service" is already in use by container "f26f56b2905e8c3da1977bc7c48e7eba00e943532146b7a8711f91fe67b67c3b". You have to remove (or rename) that container to be able to reuse that name.
*********
I also tried to log into this server and list all containers, but there is only redis one:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5cec961e03b2 811c03fb36bc "gitlab-runner-ser..." 39 hours ago Up 39 hours runner-1de473ae-project-247-concurrent-1-redis-1-wait-for-service
After googling this problem I found this issue: https://gitlab.com/gitlab-org/gitlab-runner/issues/2667 then I update runner to 10.6, but problem persists.
After all, there is no running elastic on my server, then my tests fails on:
FAILED: Battle/BattleDataElasticProviderTest.php method=testGetLocalBattles
Exited with error code 255 (expected 0)
Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster
Is there any way, how to start ES or at least put ES into more verbosive mode?
Thanks!
When a container is stopped, it still exists even though it's now in an exited state. Using command docker ps -a shows you all the running and exited containers.
To start a new container with an already existing name, you need to first manually remove the old container occupying this name by using docker rm.
A convenient way is to use the --rm argument when starting a container, the container will be automatically removed once it stops.
I have gitlab-runner installed locally.
km#Karls-MBP ~ $ gitlab-runner --version
Version: 10.4.0
Git revision: 857480b6
Git branch: 10-4-stable
GO version: go1.8.5
Built: Mon, 22 Jan 2018 09:47:12 +0000
OS/Arch: darwin/amd64
Docker:
km#Karls-MBP ~ $ docker --version
Docker version 17.12.0-ce, build c97c6d6
.gitlab-ci.yml:
image: docker/compose:1.19.0
before_script:
- echo wtf
test:
script:
- echo test
Results:
km#Karls-MBP ~ $ sudo gitlab-runner exec docker --docker-privileged test
WARNING: Since GitLab Runner 10.0 this command is marked as DEPRECATED and will be removed in one of upcoming releases
WARNING: You most probably have uncommitted changes.
WARNING: These changes will not be tested.
Running with gitlab-runner 10.4.0 (857480b6)
on ()
Using Docker executor with image docker/compose:1.19.0 ...
Using docker image sha256:be4b46f2adbc8534c7f6738279ebedd6106969695f5e596079e89e815d375d9c for predefined container...
Pulling docker image docker/compose:1.19.0 ...
Using docker image docker/compose:1.19.0 ID=sha256:e06b58ce9de2ea3f11634e022ec814984601ea3a5180440c2c28d9217b713b30 for build container...
Running on runner--project-0-concurrent-0 via x.x.x...
Cloning repository...
Cloning into '/builds/project-0'...
done.
Checking out b5a262c9 as km/ref...
Skipping Git submodules setup
No such command: sh
Commands:
build Build or rebuild services
bundle Generate a Docker bundle from the Compose file
config Validate and view the Compose file
create Create services
down Stop and remove containers, networks, images, and volumes
events Receive real time events from containers
exec Execute a command in a running container
help Get help on a command
images List images
kill Kill containers
logs View output from containers
pause Pause services
port Print the public port for a port binding
ps List containers
pull Pull service images
push Push service images
restart Restart services
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
top Display the running processes
unpause Unpause services
up Create and start containers
version Show the Docker-Compose version information
Don't really know what the issue is.
It seems that the docker/compose image is configured with docker-compose as an entrypoint.
You can override the default entrypoint of the docker/compose image in your .gitlab-ci.yml file :
image:
name: docker/compose:1.19.0
entrypoint: [""]
before_script:
- echo wtf
test:
script:
- echo test
The docker/compose image has the command docker-compose as its entrypoint (until version 1.24.x), which enables a usage similar to this (assuming a compatible volume mount):
docker run --rm -t docker/compose -f some-dir/compose-file.yml up
Unfortunately that same feature makes it incompatible with usage within GitLab CI’s Docker Runner. Theoretically you could have a construct like this:
job-name:
image: docker/compose:1.24.1
script:
- up
- --build
- --force-recreate
But the GitLab Docker Runner assumes the entrypoint is /bin/bash - or at least functions likewise (many Docker images thoughtfully use a shell script with "$#" as its final line for the entrypoint) - and from the array elements that you specify for the script, it creates its own temporary shell script on the fly. It starts with statements like set -e and set -o pipeline and will be used in a statement like sh temporary-script.sh as the container command. That’s what causes the unexpected error message you got.
This behaviour was recently documented more clearly:
The Docker executor doesn’t overwrite the ENTRYPOINT of a Docker image.
That means that if your image defines the ENTRYPOINT and doesn’t allow to run scripts with CMD, the image will not work with the Docker executor.
Overriding the entrypoint with [""] will allow usage of docker/docker-compose (before version 1.25.x) with the Docker Runner, but the script that GitLab will create on the fly is not going to run as process 1 and because of that the container will not stop at the end of the script. Example:
job-name:
image:
name: docker/docker-compose
entrypoint: [""]
script:
- docker-compose
- up
- --build
- --force-recreate
At the time I write this the latest version of docker/docker-compose is 1.25.0-rc2. Your mileage may vary, but it suffices for my purposes and entirely resolves both problems.