When I create a GitHub Actions workflow file, the example YAML file contains runs-on: ubuntu-latest. According to the docs, I only have the options between a couple versions of Ubuntu, Windows Server and macOS X.
I thought GitHub Actions runs inside Docker. How do I choose my Docker image?
GitHub actions provision a virtual machine - as you noted, either Ubuntu, Windows or macOS - and run your workflow inside of that. You can then use that virtual machine to run a workflow inside a container.
Use the container specifier to run a step inside a container. Be sure to specify runs-on as the appropriate host environment for your container (ubuntu-latest for Linux containers, windows-latest for Windows containers). For example:
jobs:
vm:
runs-on: ubuntu-latest
steps:
- run: |
echo This job does not specify a container.
echo It runs directly on the virtual machine.
name: Run on VM
container:
runs-on: ubuntu-latest
container: node:10.16-jessie
steps:
- run: |
echo This job does specify a container.
echo It runs in the container instead of the VM.
name: Run in container
A job (as part of a workflow) runs inside a virtual machine. You choose one of the environments provided by them (e.g. ubuntu-latest or windows-2019).
A job consists of one or more steps. A step may be a simple shell command, using run. But it may also be an action, using uses
name: CI
on: [push]
jobs:
myjob:
runs-on: ubuntu-18.04 # linux required if you want to use docker
steps:
# Those steps are executed directly on the VM
- run: ls /
- run: echo $HOME
- name: Add a file
run: touch $HOME/stuff.txt
# Those steps are actions, which may run inside a container
- uses: actions/checkout#v1
- uses: ./.github/actions/my-action
- uses: docker://continuumio/anaconda3:2019.07
run: <COMMAND> executes the command with the shell of the OS
uses: actions/checkout#v1 runs the action from the user / organization actions in the repository checkout (https://github.com/actions/checkout), major release 1
uses: ./.github/actions/my-action runs the action which is defined in your own repository under this path
uses: docker://continuumio/anaconda3:2019.07 runs the anaconda3 image from user / organization continuumio, version 2019.07, from the Docker Hub (https://hub.docker.com/r/continuumio/anaconda3)
Keep in mind that you need to select a linux distribution as the environment if you want to use Docker.
Take a look at the documentation for uses and run for further details.
It should also be noted that there is a container option, allowing you to run any steps that would usually run on the host to be runned inside a container: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idcontainer
Related
How does one pull an image from a github action. Specifically one that requires authentication:
steps:
- name: Pull Docker Image
uses: docker/???
image: image_host.com/image:latest
^^^ Is wrong and I am not sure what the right syntax is.
I want to then run a command inside of the action
- name: Run test
run: |
node index.js # (index.js is inside of the container)
```
In order to use a GitHub workflow with a Docker container, you need a workflow runner which has Docker installed on its system, such as ubuntu-latest. Then use the container directive in order to pick a container.
I would like to run my CI on a Docker image. How should I write my .github/workflow/main.yml?
name: CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
name: build
runs:
using: 'docker'
image: '.devcontainer/Dockerfile'
steps:
- uses: actions/checkout#v2
- name: Build
run: make
I get the error:
The workflow is not valid. .github/workflows/main.yml
(Line: 11, Col: 5): Unexpected value 'runs'
I managed to make it work but with an ugly workaround:
build:
name: Build Project
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v1
- name: Build docker images
run: >
docker build . -t foobar
-f .devcontainer/Dockerfile
- name: Build exam
run: >
docker run -v
$GITHUB_WORKSPACE:/srv
-w/srv foobar make
Side question: where can I find the documentation about this? All I found is how to write actions.
If you want to use a container to run your actions, you can use something like this:
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://{host}/{image}:{tag}
steps:
...
Here is an example.
If you want more details about the jobs.<job_id>.container and its sub-fields, you can check the official documentation.
Note that you can also use docker images at the step level: Example.
I am reposting my answer to another question, in order to be sure to find it while Googling it.
The best solution is to build, publish and re-use a Docker image based on your Dockerfile.
I would advise to create a custom build-and-publish-docker.yml action following the Github documentation: Publishing Docker images.
Assuming your repository is public, you should be able to automatically upload your image to ghcr.io without any required configuration. As an alternative, it's also possible to publish the image to Docker Hub.
Once your image is built and published (based on the on event of the action previously created, which can be triggered manually also), you just need to update your main.yml action so it uses the custom Docker image. Again, here is a pretty good documentation page about the container option: Running jobs in a container.
As an example, I'm sharing what I used in a personal repository:
Dockerfile: the Docker image to be built on CI
docker.yml: the action to build the Docker image
lint.yml: the action using the built Docker image
Hi I'm looking to move our jenkins pipeline build to the Azure Pipeline to build our application.
In Jenkins we are using the groovy script and we are building our application inside of local docker image.
In groovy we are using this:
withDockerContainer(args: '-v /home/jenkins:/home/jenkins ' , image: dockerImage )
From the Jenkins documentation (https://www.jenkins.io/doc/pipeline/steps/docker-workflow/)
Does exist any way to do the same thing in Azure. I would like to be able to specify to run a specific task inside of a specific local docker image
Thanks
You can use container jobs for that:
trigger: none
pr: none
pool:
vmImage: 'ubuntu-18.04'
jobs:
- job: u18
steps:
- bash: |
cat /etc/issue
- job: u20
container: ubuntu:20.04
steps:
- bash: |
cat /etc/issue
Does exist any way to do the same thing in Azure. I would like to be able to specify to run a specific task inside of a specific local docker image
The answer is yes.
If you want to run a task in local docker image, you need create a private agent on the machine where your local docker image exists:
Then you could use following scripts to invoke the local docker image:
pool:
name: YourPrivateAgent
resources:
containers:
- container: pycontainer
image: YourImage
steps:
- task: AnotherTask#1
target: pycontainer
You could check the document Step target for some more details.
I wanted to run django test cases inside container.
I am able to pull private image from docker hub. but when I ran command to test, It is failed to run.
Anyone tried running test cases inside the container.
jobs:
test:
container:
image: abcd
credentials:
username: "<username>"
password: "<password>"
steps:
- uses: actions/checkout#v2
- name: Display Python version
run: |
python -m pip install --upgrade pip
pip install -r requirements/dev.txt
- name: run test
run: |
python3 manage.py test
In my experience, I found out that using GitHub's container instruction causes more confusion than simply running whatever you want on the runner itself, as if you are running it on your own machine.
A big majority of the tests I am running on GitHub actions are running in containers, and some require private DockerHub images.
I always do this:
Create a docker-compose.yml for development use, so I can test things locally.
Usually in CI, you might want slightly different things in your docker-compose (for example, no volume mappings) - if this is the case, I am creating another docker-compose.yml in a .ci subfolder.
My docker-compose.yml contains a test service, that runs whatever test (or test suite) I want.
Here is a sample GitHub actions file I am using:
name: Test
on:
pull_request:
push: { branches: master }
jobs:
test:
name: Run test suite
runs-on: ubuntu-latest
env:
COMPOSE_FILE: .ci/docker-compose.yml
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Login to DockerHub
run: docker login -u $DOCKER_USER -p $DOCKER_PASS
- name: Build docker images
run: docker-compose build
- name: Run tests
run: docker-compose run test
Of course, this entails setting up the two mentioned secrets, but other than that, I found this method to be:
Reliable
Portable (I switched from Travis CI with the same approach easily)
Compatible with dev environment
Easy to understand and reproduce both locally and in CI
I am currently trying to build and deploy a dockerized Go project, pulled from a Git repo in using Concourse.
To give you some background about my current setup:
I got two AWS Lightsail instances set up, both of them using a Docker container to serve Concourse.
One of those instances is serving the web node, the other one is acting as a worker node, which connects to the web node.
My current pipeline looks like this:
resources:
- name: zsu-wasserlabor-api-repo
type: git
webhook_token: TOP_SECRET
source:
uri: git#github.com:lennartschoch/zsu-wasserlabor-api
branch: master
private_key: TOP_SECRET
jobs:
- name: build-api
plan:
- get: zsu-wasserlabor-api-repo
trigger: true
- task: build
config:
platform: linux
image_resource:
type: docker-image
source: {repository: alpine}
inputs:
- name: zsu-wasserlabor-api-repo
run:
path: sh
args:
- -c
- |
cd zsu-wasserlabor-api-repo
docker-compose build
The problem is that docker-compose is not installed.
I am feeling like I am doing something fundamentally wrong. Could anyone give me a hint?
Best,
Lennart
The pipeline described above specifies that it should use the alpine image, which doesn't have docker-compose on it. Thus, you will need to find an image that has docker-compose installed on it, but even then, there are additional steps you will need to take to make it work in Concourse (see this link for more details).
Fortunately, someone has made an image available that takes care of the additional steps, with a sample pipeline that you can find here: https://github.com/meAmidos/dcind
That being said, if you are simply trying to build a Docker image, you can use the docker-image-resource instead and just specify the Dockerfile.