This is my travis.yml file and this is the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
I'm unsure as to whether this is an efficient setup. Is there a way of caching the docker builds so it doesn't have to rebuild them every time? When I'm updating builds locally it can update just the parts that need to be changed. Is there a way of doing this on Travis? A whole new rebuild seems like a lot of unnecessary work for Travis. What's the most efficient combination of free CI/CD service and free hosting for docker deployments in terms of build efficiency and deployment efficiency? (I know this is subjective, just want to get some idea)
Sorry if these are rookie questions btw I am very new to docker and its corresponding workflows/pipelines... having a bit of trouble finding multi container examples with heroku.
Related
.gitlab-ci.yml
workflow:
rules:
- if: $CI_COMMIT_BRANCH != "main" && $CI_PIPELINE_SOURCE != "merge_request_event"
when: never
- when: always
variables:
IMAGE_NAME: $CI_REGISTRY_IMAGE
stages:
- test
- build
run_unit_tests:
image: node:16-alpine3.17
stage: test
tags:
- txy
before_script:
- cd app
- npm install
script:
- npm test
artifacts:
when: always
paths:
- app/junit.xml
reports:
junit: app/junit.xml
build_image:
stage: build
tags:
- remoteone
before_script:
- echo "Docker registry url is $CI_REGISTRY"
- echo "Docker registry username is $CI_REGISTRY_USER"
- echo "Docker registry repo is $CI_REGISTRY_IMAGE"
script:
- docker build -t $IMAGE_NAME .
push_image:
stage: build
needs:
- build_image
tags:
- remoteone
before_script:
- echo "Docker registry url is $CI_REGISTRY"
- echo "Docker registry username is $CI_REGISTRY_USER"
- echo "Docker registry repo is $CI_REGISTRY_IMAGE"
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker push $IMAGE_NAME
enter image description here
correspondence of the above variables
$CI_REGISTRY_IMAGE = docker build -t registry.gitlab.com/komorebi-cqd/mynodeapp-cicd-project .
$CI_REGISTRY = docker push registry.gitlab.com/komorebi-cqd/mynodeapp-cicd-project
Dockerfile
FROM node:16-alpine
RUN apk add docker-cli
WORKDIR /usr/src/app
COPY ./app/package*.json ./
RUN npm install
COPY ./app .
EXPOSE 3000
CMD [ "npm" ,"start"]
cicd pipelines build_image error
Running with gitlab-runner 15.5.1 (7178588d)
on one-docker-runner fAyvKeBR
Preparing the "docker" executor
00:03
Using Docker executor with image alpine:3.14 ...
Pulling docker image alpine:3.14 ...
Using docker image sha256:dd53f409bf0bd55eac632f9e694fd190244fef5854a428bf3ae1e2b636577623 for alpine:3.14 with digest alpine#sha256:4c869a63e1b7c0722fed1e402a6466610327c3b83bdddb94bd94fb71da7f638a ...
Preparing environment
00:01
Running on runner-fayvkebr-project-41046722-concurrent-0 via VM-20-2-ubuntu...
Getting source from Git repository
00:02
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/komorebi-cqd/mynodeapp-cicd-project/.git/
Checking out 023ee3af as main...
Removing app/junit.xml
Skipping Git submodules setup
Downloading artifacts
00:03
Downloading artifacts for run_unit_tests (3502740847)...
Downloading artifacts from coordinator... ok id=3502740847 responseStatus=200 OK token=64_pVtH2
Executing "step_script" stage of the job script
00:01
Using docker image sha256:dd53f409bf0bd55eac632f9e694fd190244fef5854a428bf3ae1e2b636577623 for alpine:3.14 with digest alpine#sha256:4c869a63e1b7c0722fed1e402a6466610327c3b83bdddb94bd94fb71da7f638a ...
$ echo "Docker registry url is $CI_REGISTRY"
Docker registry url is registry.gitlab.com
$ echo "Docker registry username is $CI_REGISTRY_USER"
Docker registry username is gitlab-ci-token
$ echo "Docker registry repo is $CI_REGISTRY_IMAGE"
Docker registry repo is registry.gitlab.com/komorebi-cqd/mynodeapp-cicd-project
$ docker build -t $IMAGE_NAME .
/bin/sh: eval: line 137: docker: not found
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 127
It prompts docker: not found,i have been searching the Internet for a long time, but there is no specific solution
I have a Docker installed on my server, and there is no problem running Docker PS. However, when I use the gitlab registry, I will be prompted that the Docker does not exist. I do not know where the problem is, and there is no corresponding solution online. I hope someone can tell me why the Docker is not found and how to solve it.
GitLab CI uses Docker to run your pipelines (with the Docker executor) but it doesn't mean that Docker will be available from inside the CI containers.
You can see from the CI output that it uses the alpine:3.14 Docker image to run your commands, unfortunately this image has not Docker installed:
Using Docker executor with image alpine:3.14 ...
The GitLab documentation describes 3 ways to build Docker images with their CI using:
the Shell executor
Docker-in-Docker with the Docker or Kubernetes executor
Docker socket binding
I'd say to go with the Docker-in-Docker approach because the other 2 may result in concurrency issues: concurrent pipelines will use the same Docker daemon which could result in naming collision of Docker objects.
You can read more about my personal experience in 2020 about building Docker images in GitLab CI if that helps.
I am new on Travis CI and I am trying to deploy my hobby project to AWS by Travis but in my first commit nothing happened here is my .travis.yml:
sudo: required
language: generic
services:
- docker
before_install:
- docker build -t aaa/fakewebappapi -f ./api/Dockerfile.dev ./api
script:
- docker run -e CI=true aaa/fakewebappapi npm run test
after_success:
- docker build -t aaa/fakewebappclient -f ./client/Dockerfile.dev ./client
- docker build -t aaa/fakewebappapi -f ./api/Dockerfile.dev ./api
- docker build -t aaa/fakewebappnginx -f ./nginx/Dockerfile.dev ./nginx
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- docker push aaa/fakewebappclient
- docker push aaa/fakewebappapi
- docker push aaa/fakewebappnginx
Each docker work correctly but when I push the repository to my GitHub nothing changed. Also I checked my GitHub profile and travis connected it succesfully.
Also there is no log in travis-ci.com when I see other travis project on YouTube everyone has log screen in built screen.
Check if you are using tabs instead of spaces.
This is my travis.yml file followed by the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
services:
- docker
stages:
- name: before_deploy
if: branch = master
- name: before_install
if: branch != master
- name: scripts
if: branch != master
before_install:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
scripts:
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
on:
branch: master
However, I'd like to know why my branch != master does not work for the before_install and scripts stages. It runs both of these stages even on master branch after merging my pull request.
(I am aware of the other issues with this travis.yml, I have raised them as separate questions)
Resolved this, set it up differently though. See below. I think the stages have to be set up with the jobs/include object as seen below
see new travis.yml
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
***note I had to add language to avoid a Rakefile error. Best to use generic here as using node_js will prompt travis to look for a package.json and and a "make test" error will occur
So really I have several problems here. This is my travis.yml file and this is the latest run
https://travis-ci.com/github/harryyy27/allies-art-club:
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
deploy:
skip_cleanup: true
provider: script
script:
docker ps -a;
docker push harryyy27/aac-client;
docker push registry.heroku.com/$HEROKU_APP/client;
docker push harryyy27/aac-src;
docker push registry.heroku.com/$HEROKU_APP/src;
docker push harryyy27/aac-nginx;
docker push registry.heroku.com/$HEROKU_APP/nginx;
heroku container:release client src nginx --app $HEROKU_APP;
For some reason I cannot deploy to heroku. The docker push registry.heroku.com/$HEROKU_APP/container_name appears to work along with the echo "$HEROKU_API" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com sign in but then when I go to release the heroku containers says "Invalid credentials provided" in the terminal and tells me to login. Is there a way of releasing these containers using the docker CLI on Travis?
If not, would the Heroku CLI help?
So I eventually resolved this by simply changing the $HEROKU_API to $HEROKU_API_KEY. This is an environment variable that when present automatically logs you into the Heroku CLI enabling you to run the scripts required to upload to your docker containers to Heroku. This is the travis.yml I eventually ended up with
sudo: required
language: generic
services:
- docker
stages:
- dev
- prod
jobs:
include:
- stage: dev
if: NOT(branch=master)
scripts:
- docker build -t harryyy27/allies_art_club/frontend -f ./client/Dockerfile.dev ./client
- docker build -t harryyy27/allies_art_club/backend -f ./src/Dockerfile.dev ./src
- docker run -e CI=true harryyy27/allies_art_club/frontend npm test
- docker run -e CI=true harryyy27/allies_art_club/backend npm test
- stage: prod
if: branch=master
before_deploy:
- docker build -t harryyy27/aac-client ./client
- docker tag harryyy27/aac-client registry.heroku.com/$HEROKU_APP/client
- docker build -t harryyy27/aac-src ./src
- docker tag harryyy27/aac-src registry.heroku.com/$HEROKU_APP/src
- docker build -t harryyy27/aac-nginx ./nginx
- docker tag harryyy27/aac-nginx registry.heroku.com/$HEROKU_APP/nginx
# Log in to docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
- curl https://cli-assets.heroku.com/install.sh | sh
- echo "$HEROKU_API_KEY" | docker login -u "$HEROKU_USERNAME" --password-stdin registry.heroku.com
- docker push harryyy27/aac-client;
- docker push registry.heroku.com/$HEROKU_APP/client;
- docker push harryyy27/aac-src;
- docker push registry.heroku.com/$HEROKU_APP/src;
- docker push harryyy27/aac-nginx;
- docker push registry.heroku.com/$HEROKU_APP/nginx;
deploy:
skip_cleanup: true
provider: script
script:
heroku container:login;
heroku container:release client src nginx --app $HEROKU_APP;
I do now have errors in Heroku though :P
Hey I am using travis ci and i dont understand what i have to write inside the travis yml to deploy multi container into heroku
this is my travis.yml
sudo: required
services:
- docker
before_install:
- docker build -t 307949230/client-test -f ./client/Dockerfile.dev ./client
script:
- docker run 307949230/client-test npm test -- --coverage
after_success:
- docker build -t 307949230/mern-client ./client
- docker build -t 307949230/mern-nginx ./nginx
- docker build -t 307949230/mern-server ./server
# Log in to the docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
# Take those images and push them to docker hub
- docker push 307949230/mern-client
- docker push 307949230/mern-nginx
- docker push 307949230/mern-server