I'm trying to implement a CI&CD server using gitlab.
my gitlab.ci config is something like this:
stages:
- deploy
step-deploy-staging:
stage: deploy
script:
- docker-compose up -d web
if I run my container with -d (detach) I can't understand the result of it because it will run on background. If i run docker-compose without detach mode, our gitlab pipeline never done
Related
I am new to GitLabCI, it seems GitLab CI is docker everywhere.
I was trying to run a Mariadb before run tests. In Github actions, it is very easy, just docker-compose up -d command before my mvn.
When came to GitLab CI.
I was trying to use the following job to archive the purpose.
test:
stage: test
image: maven:3.6.3-openjdk-16
services:
- name: docker
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
- .m2/repository
script: |
docker-compose up -d
sleep 10
mvn clean verify sonar:sonar
But this does not work, docker-compose is not found.
You can make use of docker-dind docker-dind and run the docker commands inside another docker container.
But there is limitation to run docker-compose by default. It is recommended to build a custom image on top of DIND and push it to gitlab image registry. So that can be used across your jobs
I'm a total newbie when it comes to CI/CD, so I'm asking your pardon in advance for not using the right terms/doing stupid things.
I have a docker-compose file which I can use to start my application with sudo docker-compose up -d. Works fine locally, but I also have a remote Virtual Machine, which I want to use to test my application.
I want to run some tests (I will implement them later on) and deploy the app if everything is ok with every push to my repoistory. I looked into the docs, installed gitlab-runner, and tried this for a .gitlab-ci.yml file:
build-job:
stage: build
script:
- echo "Hello, $GITLAB_USER_LOGIN!"
test-job1:
stage: test
script:
- echo "This job tests something"
deploy-prod:
stage: deploy
script:
- echo "Restaring containers.."
- cd /path/too/app/repo
- git pull
- sudo docker-compose down && sudo docker-compose up -d
However, when I tried the docker runner, it gave me an error saying it could not find the path to the directory. I understand that this is caused by the fact that it runs every stage in a separate container. How can I restart my application containers (preferably with compose) on the VM? Is there a better approach to achieve what I want to do?
I'm trying to build a CI Pipeline with Gitlab CI/CD. My project is a really simple API based on Symfony. To create a consistent environment i'm using docker-compose with four very simple containers (nginx,PHP,MySQL & composer). My .gitlab-ci.yaml looks like this:
stages:
- setup
setup:
stage: setup
before_script:
- docker-compose up -d
script:
- sleep 15
- docker-compose exec -T php php bin/console doctrine:schema:create
after_script:
- [...]
- docker-compose down
The problem I'm encountering is that the ci script does not wait till the docker-compose up -d is finished. To bypass this I've added this stupid sleep.
Is there a better way to do this?
To save some time for the people searching it, I implemented the solution commented by #gbrener.
The idea: Wait until getting the log that shows that the container is up, then continue the pipeline.
1 - Get the log to be the checkpoint. I used the last log of my container. Ex: Generated backend app.
2 - Get the container name. Ex: ai-server-dev.
3 - Create a sh script like the below and name it something.sh. Ex:
#!/bin/bash
while ! docker logs ai-server-dev --tail=1 | grep -q "Generated backend app";
do
sleep 10
echo "Waiting for backend to load ..."
done
4 - Replace the 'sleep 15' with 'sh wait_server.sh' as in the question to run the script in your pipeline.
I am getting the following error during the "test_image" step when running tests against docker images in my gitlab CI pipeline. I cannot reproduce it locally, it only occurs on the gitlab runner box. Any ideas?
The container name "/common_run_1" is already in use by container
image: docker:latest
stages:
- build
- test
- release
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN myregistry.gitlab
build_image:
stage: build
script:
- docker build --pull -t $CONTAINER_TEST_IMAGE .
- docker-compose up -d --build
- docker push $CONTAINER_TEST_IMAGE
pylint:
stage: test
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker-compose run common pylint common
test_image:
stage: test
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker-compose run common nosetests common
push_master_image:
stage: release
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker tag $CONTAINER_TEST_IMAGE $CONTAINER_MASTER_IMAGE
- docker push $CONTAINER_MASTER_IMAGE
only:
- master
push_prod_image:
stage: release
script:
- docker pull $CONTAINER_TEST_IMAGE
- docker tag $CONTAINER_TEST_IMAGE $CONTAINER_PROD_IMAGE
- docker push $CONTAINER_PROD_IMAGE
only:
- prod
Update:
There are multiple suggestions to simply use "docker-compose down" or "docker stop". I have done this on my gitlab-runner box (completely cleaned out docker processes, images, volumes, and networks), and re-submitted the pipeline request. In this case, I get the same error in the gitlab pipeline. It makes me think there is a concurrency issue in the "test" stage. Furthermore, if I add a "test2" stage and place the "pylint" script inside of it, the pipeline will succeed, further re-enforcing the idea of a concurrency problem.
Your stage:test is having two docker-compose run and both are running using same container name. You can change this by adding --name test1 in docker-compose run of first test and --name test2 in docker-compose run of second test.
Original Answer
Run docker ps -a and it will list which container names are already being used. This is caused mostly because you have already run the container using docker-compose up and the containers are still up.
Your options are
Run docker-compose down. This should bring down the already running containers. And should most probably solve your error.
If option 1 fails, then you can see which containers are running and stop those containers by running docker stop <container_name>.
I am trying to set up automated deployment using a GitLab CI runner to deploy our 4-container app via docker-compose. I can pull the container images down using docker pull commands, but I'm stuck on how to connect to the Google Compute Engine instance in order to run the full docker-compose script.
Typically, from my local machine, I run something like:
eval $(docker-machine env <machine-instance>)
docker-compose up -d
But my .gitlab-ci.yml script doesn't have docker-machine available.
Do I have to install docker-machine via the script section in my
.gitlab-ci.yml file?
How do I provision the instance without
creating a new one every time? Normally, from my local host, I would
run docker-machine create ... once then just use the eval
command above to reconnect to the instance. But how would this work
with CI?
Here's a sample of my .gitlab-ci.yml:
deploy staging:
image: docker:latest
services:
- docker:dind
environment: staging
stage: deploy
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN my-registry.githost.io
script:
- docker pull my-registry.githost.io/group/project1:develop
- docker pull my-registry.githost.io/group/project2:develop
- docker pull my-registry.githost.io/group/project3:develop
- docker pull my-registry.githost.io/group/project4:develop
- docker-machine ls
Not sure what you need docker-machine for in this case. You might want to get rid of it.
But to go back to your question, the docker image you're using does not come with neither docker-machine, nor docker-compose :
https://github.com/docker-library/docker/blob/36e2107fb879d5d5c3dbb5d8d93aeef0a2d45ac8/1.12/Dockerfile
So you will need to create a new image (or find an existing one) that comes with those two installed.
So in the .gitlab-ci.yml, instead of image: docker:latest, it's going to be something like image: mydocker
You maybe have to install docker-machine in the GitLab CI Runner to use it with GCE
https://docs.docker.com/machine/install-machine/
https://docs.docker.com/machine/drivers/gce/