I want to automatically create an image using github actions and upload it after testing
However, when this build is executed inside the container, Unable to connect to newly launched container. (probably the execution was successful)
Of course it works when I try to do the same thing without using containers (on the machine).
Why the connection was failed when I tried in docker container?
jobs:
build_and_test:
runs-on: my-runner
container:
image: my-image:tag
credentials:
username: ${{ secrets.ID }}
password: ${{ secrets.PW }}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
options: --user root
steps:
- uses: actions/checkout#v2
- name: build_image
run: |
...
# build image
...
- name: bvt_test
run: |
container=$(docker run --rm -itd -p ${port}:1234 ${new_image})
...
# test, but the connection failed
Thanks.
Related
I'm trying to run docker commands in an ssh connection which was made from github actions. There I'm using even mention in the workflow. Here is my workflow
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Node.js CI
on:
pull_request:
branches: ['main']
env:
REGISTRY: 'ghcr.io/testing/api-gateway'
IMAGE_NAME: ${{ format('{0}-{1}', github.event.repository.name, github.sha) }}
USERNAME: ${{ secrets.USERNAME }}
DOCKER_TOKEN: ${{ secrets.DOCKER_TOKEN }}
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm install
- run: npm run build
- name: Build & Push docker
run: docker build -t $REGISTRY:$IMAGE_NAME .
- name: Login to github package repository
run: echo $DOCKER_TOKEN | docker login ghcr.io -u $USERNAME --password-stdin
- name: Push docker image
run: docker push $REGISTRY:$IMAGE_NAME
deploy:
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to Digital Ocean droplet via SSH action
uses: garygrossgarten/github-action-ssh#release
with:
host: ${{ secrets.DO_HOST }}
username: ${{ secrets.DO_USER }}
privateKey: ${{ secrets.DO_KEY }}
passphrase: ${{ secrets.DO_PASSPHRASE }}
command: |
echo $DOCKER_TOKEN | docker login ghcr.io -u $USERNAME --password-stdin
docker pull $REGISTRY:$IMAGE_NAME
docker run -p 3000:3000 $REGISTRY:$IMAGE_NAME
But I'm getting the following error when I run the workflow.
Run garygrossgarten/github-action-ssh#release
Establishing a SSH connection to ***.
using provided private key
(node:1514) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
🤝 Connected to ***.
Executing command: echo $DOCKER_TOKEN | docker login ghcr.io -u $USERNAME --password-stdin
docker pull $REGISTRY:$IMAGE_NAME
docker run -p 3000:3000 $REGISTRY:$IMAGE_NAME
Error: Cannot perform an interactive login from a non-TTY device
invalid reference format
docker: invalid reference format.
See 'docker run --help'.
⚠️ An error happened executing the command echo $DOCKER_TOKEN | docker login ghcr.io -u $USERNAME --password-stdin
docker pull $REGISTRY:$IMAGE_NAME
docker run -p 3000:3000 $REGISTRY:$IMAGE_NAME. Command exited with code 125
Error: Command exited with code 125
1: 0xa1a640 node::Abort() [/home/runner/runners/2.296.0/externals/node12/bin/node]
2: 0xa90649 [/home/runner/runners/2.296.0/externals/node12/bin/node]
3: 0xc06599 [/home/runner/runners/2.296.0/externals/node12/bin/node]
4: 0xc08387 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/home/runner/runners/2.296.0/externals/node12/bin/node]
5: 0x140dd19 [/home/runner/runners/2.296.0/externals/node12/bin/node]
I was looking for a solution and I couldn't find a proper one. what would be the best solution for this?
According to this other thread about the Cannot perform an interactive login from a non-TTY device message:
docker login prints this error message when you use --password-stdin,
but don't actually send a password to the command's stdin.
Therefore, it seems the $DOCKER_TOKEN variable is empty.
You could try using ${{ env.DOCKER_TOKEN}}, or add env: DOCKER_TOKEN: ${{ env.DOCKER_TOKEN }} in the last step of your deploy job.
Something like this:
deploy:
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to Digital Ocean droplet via SSH action
uses: garygrossgarten/github-action-ssh#release
env:
DOCKER_TOKEN: ${{ env.DOCKER_TOKEN }}
with:
host: ${{ secrets.DO_HOST }}
username: ${{ secrets.DO_USER }}
privateKey: ${{ secrets.DO_KEY }}
passphrase: ${{ secrets.DO_PASSPHRASE }}
command: |
echo $DOCKER_TOKEN | docker login ghcr.io -u $USERNAME --password-stdin
# or using directly ${{ env.DOCKER_TOKEN}} instead of $DOCKER_TOKEN
docker pull $REGISTRY:$IMAGE_NAME
docker run -p 3000:3000 $REGISTRY:$IMAGE_NAME
In docs there is described how to run container with additional options: Workflow syntax for GitHub Actions - GitHub Docs
jobs:
build:
name: Container with name
runs-on: ubuntu-latest
container:
image: ghcr.io/repo/image:latest
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
options: --name myContainerName
But snippet above create and start docker without name argument... and of course other argument doesn't work, like --workdir.
Name of container is generated like this: 31ec555a1aav41a5bffe95da6f8b775b_ghcriorepoimage10_c998dc
Updated:
It's funny, because when i read logs, there are twice --name argument:
/usr/bin/docker create --name 31ec555a1aav41a5bffe95da6f8b775b_ghcriorepoimage10_c998dc --label f88420 --network github_network_a6029d55c6024481ba937e113e978280 --name myContainerName...
I am using GitHub Actions to trigger the building of my dockerfile, it is uploading the container to GitHub Container Registry. In the last step i am connecting via SSH to my remote DigitalOcean Droplet and executing a script to pull and install the new image from GHCR. This workflow was good for me as I was only building a single container in the project. Now I am using docker compose as I need NGINX besides by API. I would like to keep the containers on a single dropplet as the project is not demanding in ressources at the moment.
What is the right way to automate deployment with Github Actions and Docker Compose to DigitalOcean on a single VM?
My currently known options are:
Skip building containers on GHCR and fetch the repo via ssh to start building on remote from source by executing a production compose file
Building each container on GHCR, copy the production compose file on remote to pull & install from GHCR
If you know more options, that may be cleaner or more efficient please let me know!
Unfortunatly I have found a docker-compose with Github Actions for CI question for reference.
GitHub Action for single Container
name: Github Container Registry to DigitalOcean Droplet
on:
# Trigger the workflow via push on main branch
push:
branches:
- main
# use only trigger action if the backend folder changed
paths:
- "backend/**"
- ".github/workflows/**"
jobs:
# Builds a Docker Image and pushes it to Github Container Registry
push_to_github_container_registry:
name: Push to GHCR
runs-on: ubuntu-latest
# use the backend folder as the default working directory for the job
defaults:
run:
working-directory: ./backend
steps:
# Checkout the Repository
- name: Checking out the repository
uses: actions/checkout#v2
# Setting up Docker Builder
- name: Set up Docker Builder
uses: docker/setup-buildx-action#v1
# Set Github Access Token with "write:packages & read:packages" scope for Github Container Registry.
# Then go to repository setings and add the copied token as a secret called "CR_PAT"
# https://github.com/settings/tokens/new?scopes=repo,write:packages&description=Github+Container+Registry
# ! While GHCR is in Beta make sure to enable the feature
- name: Logging into GitHub Container Registry
uses: docker/login-action#v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.CR_PAT }}
# Push to Github Container Registry
- name: Pushing Image to Github Container Registry
uses: docker/build-push-action#v2
with:
context: ./backend
version: latest
file: backend/dockerfile
push: true
tags: ghcr.io/${{ github.repository }}:latest
# Connect to existing Droplet via SSH and (re)installs add. runs the image
# ! Ensure you have installed the preconfigured Droplet with Docker
# ! Ensure you have added SSH Key to the Droplet
# ! - its easier to add the SSH Keys bevore createing the droplet
deploy_to_digital_ocean_dropplet:
name: Deploy to Digital Ocean Droplet
runs-on: ubuntu-latest
needs: push_to_github_container_registry
steps:
- name: Deploy to Digital Ocean droplet via SSH action
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.HOST }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.PRIVATE_KEY }}
port: ${{ secrets.PORT }}
script: |
# Stop all running Docker Containers
docker kill $(docker ps -q)
# Free up space
docker system prune -a
# Login to Github Container Registry
docker login https://ghcr.io -u ${{ github.repository_owner }} -p ${{ secrets.CR_PAT }}
# Pull the Docker Image
docker pull ghcr.io/${{ github.repository }}:latest
# Run a new container from a new image
docker run -d -p 80:8080 -p 443:443 -t ghcr.io/${{ github.repository }}:latest
Current Docker-Compose
version: "3"
services:
api:
build:
context: ./backend/api
networks:
api-network:
aliases:
- api-net
nginx:
build:
context: ./backend/nginx
ports:
- "80:80"
- "443:443"
networks:
api-network:
aliases:
- nginx-net
depends_on:
- api
networks:
api-network:
Thought I'd post this as an answer instead of a comment since it was cleaner.
Here's a gist: https://gist.github.com/Aldo111/702f1146fb88f2c14f7b5955bec3d101
name: Server Build & Push
on:
push:
branches: [main]
paths:
- 'server/**'
- 'shared/**'
- docker-compose.prod.yml
- Dockerfile
jobs:
build_and_push:
runs-on: ubuntu-latest
steps:
- name: Checkout the repo
uses: actions/checkout#v2
- name: Create env file
run: |
touch .env
echo "${{ secrets.SERVER_ENV_PROD }}" > .env
cat .env
- name: Build image
run: docker compose -f docker-compose.prod.yml build
- name: Install doctl
uses: digitalocean/action-doctl#v2
with:
token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}
- name: Log in to DO Container Registry
run: doctl registry login --expiry-seconds 600
- name: Push image to DO Container Registry
run: docker compose -f docker-compose.prod.yml push
- name: Deploy Stack
uses: appleboy/ssh-action#master
with:
host: ${{ secrets.GL_SSH_HOST }}
username: ${{ secrets.GL_SSH_USERNAME }}
key: ${{ secrets.GL_SSH_SECRET }}
port: ${{ secrets.GL_SSH_PORT }}
script: |
cd /srv/www/game
./init.sh
In the final step, the directory in my case just contains a .env file and my prod compose file but these things could also be rsyncd/copied/automated as another step in this workflow before actually running things.
My init.sh simply contains:
docker stack deploy -c <(docker-compose -f docker-compose.yml config) game --with-registry-auth
The with-registry-auth part is important since my docker-compose has image:....s that use containers in DigitalOcean's container registry. So on my server, I'd already logged in once when I first setup the directory.
With that, this docker command consumes my docker-compose.yml along with the environment vairables (i.e. docker-compose -f docker-compose.yml config will pre-process the compose file with the .env file in the same directory, since stack deploy doesn't use .env) and registry already authenticated, pulls the relevant images, and restarts things as needed!
This can definitely be cleaned up and made a lot simpler but it's been working pretty well for me in my use case.
I am trying to use GitHub Actions to fire up a Postgres container for my tests. I have a script called build.sh that gets called when npm run build is called via GitHub actions. This script calls upon restore-schema.sh (shown below).
The issue here is when restore-schema.sh gets ran, I keep getting Error: no such container: postgres. GitHub actions is naming the container some arbitrary string. Is there not a way I can run docker exec on an image or somehow name the postgres container that GitHub actions is creating? I've looked through both documentations to no avail.
How should I go about this? I noticed that in the Docker run ps screenshot, it shows command docker-entrypoint.sh. Should I use this instead? Do I specify the Dockerfile inside .github/workflows/?
I tried to include as much relevant information as possible - comment if you need any other information please.
Screenshots from GitHub Actions
Initialize containers
Docker run ps <- docker ps showing name postgres
Run npm run build --if-present <- WHERE THE ISSUE IS OCCURING
build.sh
#!/bin/sh
# Import core db schema
./.deploy/postgres/restore-schema.sh
.deploy/postgres/restore-schema.sh
#!/bin/sh
docker exec -it postgres psql \
--username postgres \
--password dev \
coredb < .deploy/postgres/db-schema.sql
.github/workflows/test-api-gateway.yml
name: API Gateway CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master, develop ]
jobs:
build:
runs-on: ubuntu-latest
services: # Serivce containers to run with `container-job`
# Label used to access the service container
postgres:
# Docker Hub image
image: postgres
# Provide the password for postgres
env:
POSTGRES_USER: postgres
POSTGRES_DB: coredb
POSTGRES_PASSWORD: dev
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
strategy:
matrix:
node-version: [14.x]
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- run: docker ps
- run: chmod +x build.sh .deploy/postgres/restore-schema.sh
- run: npm ci
- run: npm run build --if-present
- run: npm test
Try the --name option
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
--name postgres
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idservices
jobs.<job_id>.services.options: Additional Docker container resource options. For a list of options, see "docker create options."
Another solution I've seen is using last created container
docker exec -it $(docker ps --latest --quiet) bash
I am trying to use the container option in a GitHub Actions workflow to run the entire job in a docker container. How do I specify the login credentials to retrieve this docker image from a private repository on docker hub?
jobs:
build:
runs-on: ubuntu-18.04
container: private_org/test-runner:1.0
I have successfully used the following docker-login "action" to authenticate with docker hub as a "step", but this does not get performed until after the job-level container gets initialized.
jobs:
build:
runs-on: ubuntu-18.04
steps:
- uses: azure/docker-login#v1
with:
username: me
password: ${{ secrets.MY_DOCKERHUB_PASSWORD }}
- name: test docker creds
run: docker pull private_org/test-runner:1.0
This was implemented recently. Use the following workflow definition:
jobs:
build:
container:
image: private_org/test-runner:1.0
credentials:
username: me
password: ${{ secrets.MY_DOCKERHUB_PASSWORD }}
Source:
https://github.blog/changelog/2020-09-24-github-actions-private-registry-support-for-job-and-service-containers/