Travis-ci stages - conditional logic - travis-ci

This is my travis.yml file followed by the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
services:
- docker
stages:
- name: before_deploy
if: branch = master
- name: before_install
if: branch != master
- name: scripts
if: branch != master
before_install:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
scripts:
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
on:
branch: master
However, I'd like to know why my branch != master does not work for the before_install and scripts stages. It runs both of these stages even on master branch after merging my pull request.
(I am aware of the other issues with this travis.yml, I have raised them as separate questions)

Resolved this, set it up differently though. See below. I think the stages have to be set up with the jobs/include object as seen below
see new travis.yml
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
***note I had to add language to avoid a Rakefile error. Best to use generic here as using node_js will prompt travis to look for a package.json and and a "make test" error will occur

Related

travis ci do nothing

I am new on Travis CI and I am trying to deploy my hobby project to AWS by Travis but in my first commit nothing happened here is my .travis.yml:
sudo: required
language: generic
services:
- docker
before_install:
- docker build -t aaa/fakewebappapi -f ./api/Dockerfile.dev ./api
script:
- docker run -e CI=true aaa/fakewebappapi npm run test
after_success:
- docker build -t aaa/fakewebappclient -f ./client/Dockerfile.dev ./client
- docker build -t aaa/fakewebappapi -f ./api/Dockerfile.dev ./api
- docker build -t aaa/fakewebappnginx -f ./nginx/Dockerfile.dev ./nginx
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- docker push aaa/fakewebappclient
- docker push aaa/fakewebappapi
- docker push aaa/fakewebappnginx
Each docker work correctly but when I push the repository to my GitHub nothing changed. Also I checked my GitHub profile and travis connected it succesfully.
Also there is no log in travis-ci.com when I see other travis project on YouTube everyone has log screen in built screen.
Check if you are using tabs instead of spaces.

Docker efficiency of CI/CD pipeline

This is my travis.yml file and this is the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
I'm unsure as to whether this is an efficient setup. Is there a way of caching the docker builds so it doesn't have to rebuild them every time? When I'm updating builds locally it can update just the parts that need to be changed. Is there a way of doing this on Travis? A whole new rebuild seems like a lot of unnecessary work for Travis. What's the most efficient combination of free CI/CD service and free hosting for docker deployments in terms of build efficiency and deployment efficiency? (I know this is subjective, just want to get some idea)
Sorry if these are rookie questions btw I am very new to docker and its corresponding workflows/pipelines... having a bit of trouble finding multi container examples with heroku.

Trouble deploying multi-container docker application to heroku using travis

So really I have several problems here. This is my travis.yml file and this is the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
For some reason I cannot deploy to heroku. The docker push registry.heroku.com/$HEROKU_APP/container_name appears to work along with the echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com sign in but then when I go to release the heroku containers says "Invalid credentials provided" in the terminal and tells me to login. Is there a way of releasing these containers using the docker CLI on Travis?
If not, would the Heroku CLI help?
So I eventually resolved this by simply changing the $HEROKU_API to $HEROKU_API_KEY. This is an environment variable that when present automatically logs you into the Heroku CLI enabling you to run the scripts required to upload to your docker containers to Heroku. This is the travis.yml I eventually ended up with
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API_KEY" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
- docker push harryyy27/aac-client;
- docker push registry.heroku.com/$HEROKU_APP/client;
- docker push harryyy27/aac-src;
- docker push registry.heroku.com/$HEROKU_APP/src;
- docker push harryyy27/aac-nginx;
- docker push registry.heroku.com/$HEROKU_APP/nginx;
deploy:
skip_cleanup: true
provider: script
script:
heroku container:login;
heroku container:release client src nginx --app $HEROKU_APP;
I do now have errors in Heroku though :P

gitlab-ci: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?

I'm installing a personal runner for my projects.
sudo docker run --rm -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \ --non-interactive \ --executor "docker" \ --docker-image docker:stable \ --url "https://gitlab.com" \ --registration-token "my-token" \
--description "docker-runner privileged" \ --tag-list "docker,aws, gara" \ --run-untagged="true" \ --locked="false" \
--access-level="not_protected" \ --docker-privileged
with this runner i can successfully the step test, build and analysis of my maven projet.
For step docker build, I always have:
Cannot connect to the Docker daemon at tcp://docker:2375. Is the
docker daemon running? ERROR: Job failed: exit code 1
Full log:
Running with gitlab-runner 13.1.1 (6fbc7474) on docker-runner
privileged GYDNagVx Preparing the "docker" executor 00:21 Using Docker
executor with image docker:latest ... Starting service docker:dind ...
Pulling docker image docker:dind ... Using docker image
sha256:b3893e48cf281b58a763fb90904fd5d63595f9e5ae5736ee2d892a2dea6a371a
for docker:dind ... Waiting for services to be up and running...
Pulling docker image docker:latest ... Using docker image
sha256:809cc4dba987efb4641d5b6fef38f47abcde1bc5c5e9850616f0ed3552737e8c
for docker:latest ... Preparing environment 00:04 Running on
runner-gydnagvx-project-15477556-concurrent-0 via e59663817b22...
Getting source from Git repository 00:07 Fetching changes with git
depth set to 50... Reinitialized existing Git repository in
/builds/gara-project/back-end-micro-services/msearch/.git/ Checking
out 80fcdd6e as develop... Removing target/ Skipping Git submodules
setup Downloading artifacts 00:18 Downloading artifacts for
maven-build (788252737)... Downloading artifacts from coordinator...
ok id=788252737 responseStatus=200 OK token=jjUorX2D Executing
"step_script" stage of the job script 00:27 $ docker login -u
"${registry_user}" -p "${registry_pass}" "${registry_url}" WARNING!
Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in
/root/.docker/config.json. Configure a credential helper to remove
this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded $ docker build -t
"${registry_url}/${image_name}:${image_tag}" . Cannot connect to the
Docker daemon at tcp://docker:2375. Is the docker daemon running?
ERROR: Job failed: exit code 1
It seems to be a common error but none of solutions found over internet works for me.
My gitla-ci.yml file:
include:
- local: '/templates/.gitlab-ci-common-template.yml'
variables:
SPRING_PROFILES_ACTIVE: test
MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
DOCKER_TLS_CERTDIR: ""
default:
image: maven:3.6.3-jdk-8-slim
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- .m2/repository/
- target/
stages:
- test
- build
- analysis
- docker
- scan
- deploy
- delivery
maven-test:
stage: test
extends:
- .ci_only
- .skip_build
script:
- mvn test
artifacts:
reports:
junit:
- target/surefire-reports/TEST-*.xml
maven-build:
stage: build
extends:
- .ci_only
- .skip_build
script:
- "./mvnw -Pint clean install -DskipTests=true"
artifacts:
paths:
- target/*
expire_in: 1 week
docker-build:
image: docker:latest
services:
- docker:19.03.0-dind
cache: {}
before_script:
- docker login -u "${registry_user}" -p "${registry_pass}" "${registry_url}"
stage: docker
dependencies:
- maven-build
extends:
- .skip_build
- .ci_only
script:
- docker build -t "${registry_url}/${image_name}:${image_tag}" .
- docker run -d --name ${image_tag} "${registry_url}/${image_name}:${image_tag}"
- sleep 60 && docker ps && docker logs ${image_tag}
- if [ $(docker inspect -f '{{.State.Running}}' ${image_tag}) = "true" ]; then echo "Application well started"; else exit 1; fi
- docker rm -f ${image_tag}
- docker push "${registry_url}/${image_name}:${image_tag}"
docker_release_image:
image: docker:latest
stage: delivery
before_script:
- docker login -u "${registry_user}" -p "${registry_pass}" "${registry_url}"
# - docker login -u "${registry_prod_user}" -p "${registry_prod_pass}" "${registry_prod_url}"
services:
- docker:19.03.0-dind
script:
- docker pull "${registry_url}/${image_name}:${image_tag}"
- docker tag "${registry_url}/${image_name}:${image_tag}" "${registry_url}/${image_name}:${CI_COMMIT_TAG/*v/}"
- docker tag "${registry_url}/${image_name}:${image_tag}" "${registry_url}/${image_name}:latest"
# - docker tag "${registry_url}/${image_name}:${image_tag}" "${registry_prod_url}/${image_name}:${CI_COMMIT_TAG/*v/}"
- docker push "${registry_url}/${image_name}:${CI_COMMIT_TAG/*v/}"
- docker push "${registry_url}/${image_name}:latest"
# - docker push "${registry_prod_url}/${image_name}:${CI_COMMIT_TAG/*v/}"
- docker rmi "${registry_url}/${image_name}:${image_tag}"
when: manual
extends:
- .only_tag_on_master
Please how could I correct it ?
Thanks

How to deploy multi container application to heroku via travis ci

Hey I am using travis ci and i dont understand what i have to write inside the travis yml to deploy multi container into heroku
this is my travis.yml
sudo: required
services:
- docker
before_install:
- docker build -t 307949230/client-test -f ./client/Dockerfile.dev ./client
script:
- docker run 307949230/client-test npm test -- --coverage
after_success:
- docker build -t 307949230/mern-client ./client
- docker build -t 307949230/mern-nginx ./nginx
- docker build -t 307949230/mern-server ./server
# Log in to the docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
# Take those images and push them to docker hub
- docker push 307949230/mern-client
- docker push 307949230/mern-nginx
- docker push 307949230/mern-server

Resources