I have setup a kubernetes cluster on AWS using kops.
I am trying to automate deployment with github actions.
name: Build and Deploy
on:
push:
branches:
- develop
jobs:
build_docker_image:
- uses: actions/checkout#v2
- name: Build the tagged Docker image
run: docker build --target dev -t org/customer-service-backend:la
push_docker_image_to_github-packages:
- uses: docker/build-push-action#v2
with:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: docker.pkg.github.com
repository: org/customer-service-backend:latest
tag_with_ref: true
deploy_to_kubernetes_cluster:
... what to do here?
I am able to built the image and push to gihub packages.
I have created deployment.yml in the root directory of the repository.
How can I deploy to kubernetes cluster?
Also, I am tagging the images with latest. Is it fine or I need to use GITHUB_REF for tagging?
Update
I am able to configure all the things. I only need to get kubeconfig to authenticate to existing cluster.
name: Build and Deploy
on:
push:
branches:
- develop
jobs:
build_docker_image:
- uses: actions/checkout#v2
- name: Build the tagged Docker image
run: docker build --target dev -t org/customer-service-backend:${{ github.sha }}
push_docker_image_to_github_packages:
needs: build_docker_image
- uses: docker/build-push-action#v2
with:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: docker.pkg.github.com
repository: org/customer-service-backend:${{ github.sha }}
tag_with_ref: true
deploy_to_kubernetes_cluster:
needs: push_docker_image_to_github_packages
name: Set Kubernetes Context
uses: azure/k8s-set-context#v1
with:
method: kubeconfig
kubeconfig: ${{ secrets.KUBE_CONFIG }} # Use secret (https://developer.github.com/actions/managing-workflows/storing-secrets/)
run: |
sed -i'' -e 's/IMAGE_LABEL/${{ github.sha }}/g' deployment.yml
kubectl apply -f deployment.yml
By looking your workflow config file, all the jobs running parallelly.
But, probably it's not what you want.
Pushing image needs a built image and deployment job needs an updated built image.
On Access kubernetes cluster, just access into your cluster and do,
cat $HOME/.kube/config
and copy the output.
Now, create a secret in github with KUBE_CONFIG as environment variable.
Notes - this is one method to access kubernetes cluster, there are other methods as well, choose one that suits your need
name: Build and Deploy
on:
push:
branches:
- develop
jobs:
build_docker_image:
name: Build Docker Image
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#v2
- name: Build the tagged Docker image
run: docker build --target dev -t your_org/customer-service-backend:${{ github.sha }} .
push_docker_image_to_github_packages:
name: Push Docker Image to Github Packages
needs: build_docker_image
runs-on: ubuntu-latest
steps:
- name: Push Docker Image
uses: docker/build-push-action#v2
with:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: docker.pkg.github.com
repository: your_org/customer-service-backend:${{ github.sha }}
deploy_to_kubernetes_cluster:
name: Deploy to Kubernetes Cluster
needs: push_docker_image_to_github_packages
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#v2
- name: Set Kubernetes Context
uses: azure/k8s-set-context#v1
with:
method: kubeconfig
kubeconfig: ${{ secrets.KUBE_CONFIG }} # Use secret (https://developer.github.com/actions/managing-workflows/storing-secrets/)
- name: Deploy to Cluster
run: |
sed -i'' -e 's/IMAGE_LABEL/${{ github.sha }}/g' deployment.yml
kubectl apply -f deployment.yml
Related
I have a problem. I am starting to work with Github Actions, and I got a working pipeline where I am trying to:
Build the docker application
Run the tests
Publish to docker repository
But the test running and publishing are 2 separate jobs, so I am building the image twice. Here is the workflow code:
name: Test & Publish Docker Image
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
run_application_tests:
name: Run test suite
runs-on: ubuntu-latest
env:
COMPOSE_FILE: docker-compose.yml
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: Login To Docker Hub
uses: docker/login-action#v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and run docker image
run: docker-compose up -d --build
- name: Run migrations
run: make migrate
- name: Run tests
run: make test
build_and_publish_docker_image:
needs: run_application_tests
name: Build & Publish Docker Images
runs-on: ubuntu-latest
steps:
- name: Checkout The Repo
uses: actions/checkout#v3
- name: Login To Docker Hub
uses: docker/login-action#v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build & Push Docker Image
uses: docker/build-push-action#v3
with:
push: true
tags: |
me/image:latest
me/image:${{ github.sha }}
The docker image is a Ruby-On-Rails application so the bundle install takes very long. Each build takes about 2-3 minutes.
I also tried adding:
cache-from: type=gha
cache-to: type=gha,mode=max
To the docker/build-push-action#v3, but that resulted in:
Error: buildx failed with: ERROR: cache export feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
What can I change to improve this pipeline time based?
in your job run_application_tests you have this code:
- name: Login To Docker Hub
uses: docker/login-action#v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and run docker image
run: docker-compose up -d --build
this code runs docker-compose why?
this is how your code should be if you want to run tests only:
jobs:
run_application_tests:
name: Run test suite
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: Run migrations
run: make migrate
- name: Run tests
run: make test
I'm using the below workflow code (found in the github documentation) to build and publish a docker image to the Github Container Registry.
name: Create and publish a Docker image
on:
push:
branches: ['release']
pull_request:
branches: ['release']
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Log in to the Container registry
uses: docker/login-action#f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action#ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
This works and I now see a public docker image under "Packages" on the github repo. When I click on the image, I am directed to a github page with more information about the image (official docs here):
"Install from the command line:"
docker pull ghcr.io/OWNER/IMAGE_NAME:pr-75
And its Digest sha: sha256:04ea7757e34c4fae527bbe6fb56eb984f54543f2313775572f0817d696ecf48a
I want to add a new job to the same workflow, that pulls the image to a virtual machine using ssh.
deploy:
- name: Deploy to Digital Ocean droplet via SSH action
uses: appleboy/ssh-action#v0.1.4
with:
host: ${{ secrets.DO_HOST }}
username: root
key: ${{ secrets.DO_PRIVATE_SSHKEY }}
port: 22
script: |
docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
This fails with:
err: invalid reference format: repository name must be lowercase (lowercasing it is not enough, read on)
Of course I cannot hard-code docker pull ghcr.io/OWNER/IMAGE_NAME:pr-75 or the Digest sha, because each new branch will increment in its PR number, so the pr-75 tag will change.
How can I deploy the image that was just published? Seems I can either use the tag value or the sha and how can I retrieve those values in real time?
There are two jobs in the above workflow:
"build-and-push-image"
"deploy"
The first one uses the docker/metadata-action to retrieve the tag name ghcr.io/OWNER/IMAGE_NAME:pr-75 which is used in the next step to name the image when docker/build-push-action is used.
I have simply used the docker/metadata-action again in the second job:
deploy:
needs: build-and-push-image
runs-on: ubuntu-latest
steps:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#69f6fc9d46f2f8bf0d5491e4aabe0bb8c6a4678a
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Deploy to Digital Ocean droplet via SSH action
uses: appleboy/ssh-action#v0.1.4
with:
host: ${{ secrets.DO_HOST }}
username: root
key: ${{ secrets.DO_PRIVATE_SSHKEY }}
port: 22
script: |
docker pull ${{ steps.meta.outputs.tags }}
Now I am implement a docker ci in github actions, first step build the docker image and push to remote repo like this:
- name: Build image push to aliyun
uses: docker/build-push-action#v1
with:
registry: ${{ secrets.ALI_DOCKER_HUB_REGISTRY }}
username: ${{ secrets.ALIYUN_DOCKER_REPO_USER_NAME }}
password: ${{ secrets.ALIYUN_DOCKER_REPO_USER_PASSWORD }}
tag_with_sha: true
repository: reddwarf-pro/dolphin-post
path: 'dolphin-post'
after pushed to remote repo, the next step I want to update my kubernetes cluster app using this newest image. I want do it like this:
- name: deploy to cluster
uses: steebchen/kubectl#v2.0.0
with: # defaults to latest kubectl binary version
config: ${{ secrets.KUBE_CONFIG_DATA }}
command: set image --record deployment/my-app container=${{ github.repository }}:${{ github.sha }}
- name: verify deployment
uses: steebchen/kubectl#v2.0.0
with:
config: ${{ secrets.KUBE_CONFIG_DATA }}
version: v1.21.0 # specify kubectl binary version explicitly
command: rollout status deployment/my-app
the github.sha represent the newest image sha code. But now I did not know how to get the github.sha value. Should I set into environment variable? How to set into environment variable? Where and How should I set the github.sha value?
do it like this:
- name: Build image push to aliyun
uses: docker/build-push-action#v1
with:
registry: ${{ secrets.ALI_DOCKER_HUB_REGISTRY }}
username: ${{ secrets.ALIYUN_DOCKER_REPO_USER_NAME }}
password: ${{ secrets.ALIYUN_DOCKER_REPO_USER_PASSWORD }}
tags: ${{ github.sha }}
repository: reddwarf-pro/dolphin-post
path: 'dolphin-post'
- name: deploy to cluster
uses: steebchen/kubectl#v2.0.0
with: # defaults to latest kubectl binary version
config: ${{ secrets.KUBE_CONFIG_DATA }}
command: set image --record deployment/dolphin-post-service dolphin-post-service=registry.cn-hangzhou.aliyuncs.com/reddwarf-pro/dolphin-post:${{ github.sha }} -n reddwarf-pro
I'm new to Docker and trying to perform CI using GitHub Actions.
Here's my .yml file on GitHub.
name: CI to Docker Hub
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Check Out Repo
uses: actions/checkout#v2
- name: Login to Docker Hub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Build and push
id: docker_build
uses: docker/build-push-action#v2
with:
context: .
file: ./Dockerfile
push: true
tags: your-order-backend:latest
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
I have added Secrets in my Github too of Docker Hub.
I'm not sure why, but it is failing at > exporting to image:
It is resloved!
In my case, there was no repo in the Docker Hub created whose tag I have passed here in the yml file.
I created the repo and it worked
for me I fixed it by changing "push: true" to "load: true"
In my case, Id needed to create the repo in dockerhub as in the response of Sagar, but the problem still were there. I'd recognized I was passing the tags incorrectly, fix it and the problem was gone. (You can See the official examples)
Before:
tags: ${{ github.sha }}, latest
After (correctly):
tags: |
henriqueholtz/fullcycle-gitops:${{ github.sha }}
henriqueholtz/fullcycle-gitops:latest
I have created a GitHub action on repo tag creation. I am successfully able to build and push the Docker image to AWS but, I don't know how to tag the image with the same name of the GitHub tag. Below is my git workflow file
name: Build Docker Image and Push to AWS ECR
on:
push:
tags:
- '*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v1
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1
- name: Build, tag, and push image to Amazon ECR
id: build-image
env:
ECR_REGISTRY: ${{ secrets.AWS_REGISTRY }}
ECR_REPOSITORY: repo_name
IMAGE_TAG: latest
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
Please help me in replacing the correct value at IMAGE_TAG in the above code
We decided the use the git commit sha as the image tag, as it always represents the unique state of the code.
- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: reponame
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
If you need or prefer to use the commit tag, you just need to extract it from the ref using something like this:
- name: Extract Git Tag
run: echo "GIT_TAG=${GITHUB_REF/refs\/tags\//}" >> $GITHUB_ENV
- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: reponame
IMAGE_TAG: ${{ env.GIT_TAG }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
Use GITHUB_REF_NAME variable to get latest tag:
name: Bolivia Version - Develop
on:
push:
tags: # <---- only tags, important!!!
- '*'
jobs:
build-version:
runs-on: ubuntu-latest
steps:
- name: Git checkout
uses: actions/checkout#v2
- name: Extract latest tag
run: |
GIT_TAG=$GITHUB_REF_NAME