I've got this sample YML file for a GitHub Action:
name: test
on:
push:
branches:
- "**"
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Set up Python
uses: actions/setup-python#v1
with:
python-version: 3.7
- name: Install pytest
run: pip install pytest
It works just fine. However, if I try to run inside of a Docker container:
name: test
on:
push:
branches:
- "**"
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
container:
image: gcc:latest
steps:
- uses: actions/checkout#v3
- name: Set up Python
uses: actions/setup-python#v1
with:
python-version: 3.7
- name: Install pytest
run: pip install pytest
it fails with
/__w/_temp/7f078daf-76ef-4b47-a2d4-d587507806ff.sh: 1: pip: not found
How do I install pytest inside of the container?
Related
I have a GitHub Action that needs to publish a Dockerfile to a specific organization.
The action looks like this:
name: Docker dataeng_github_metrics
# Run workflow on tags starting with v (eg. v2, v1.2.0)
on:
push:
branches: [ "master" ]
paths:
- ./data_pipelines/dataeng_github_metrics/*
pull_request:
branches: [ "master" ]
jobs:
Deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v1
- name: Login to GitHub Container Registry
uses: docker/login-action#v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GHCR_REGISTRY_TOKEN }}
- name: Build and Push Docker Image
uses: docker/build-push-action#v2
with:
file: ./data_pipelines/dataeng_github_metrics/Dockerfile
push: true # Will only build if this is not here
tags: |
ghcr.io/mirantis/dataeng_github_metrics:latest
The problem is that when I run the Dockerfile locally it works, but on this specific action workflow, it does not work. Instead, I get the following:
ERROR: failed to solve: failed to compute cache key: "/go.sum" not found: not found
Error: buildx failed with: ERROR: failed to solve: failed to compute cache key: "/go.sum" not found: not found
And upon inspecting the Dockerfile:
###############
# CACHE IMAGE #
###############
ARG GO_IMAGE=golang:1.17.3-alpine3.14
ARG BASE_IMAGE=alpine:3.14.2
FROM ${GO_IMAGE} AS cache
# Add the keys
ARG GITHUB_ID
ENV GITHUB_ID=$GITHUB_ID
ARG GITHUB_TOKEN
ENV GITHUB_TOKEN=$GITHUB_TOKEN
# Install Git
RUN apk add git
# TODO: ENCRYPT THE GITHUB_ID AND GITHUB_TOKEN
# Make Git Configuration
RUN git config \
--global \
url."https://${GITHUB_ID}:${GITHUB_TOKEN}#github.com/".insteadOf \
"https://github.com/"
WORKDIR /bin
COPY go.mod go.sum /bin/
RUN go mod download
##############
# BASE IMAGE #
##############
FROM cache AS dataeng_github_metrics
COPY . /bin
WORKDIR /bin
# Setup Git Terminal Prompt & Go Build
RUN go build .
###############
# FINAL IMAGE #
###############
FROM ${BASE_IMAGE}
COPY --from=dataeng_github_metrics /bin/dataeng_github_metrics bin/
ENTRYPOINT [ "bin/dataeng_github_metrics" ]
It fails at the following:
COPY go.mod go.sum /bin/
This builds locally so I don't understand what the issue is.
So to get past it, I had to add the full path to my context for the GitHub Action, I have another issue, but it's currently unrelated to the path context and not being able to find the files:
By adding in the following line, I was able to fix it by specifying the context path:
context: ./data_pipelines/dataeng_github_metrics/
name: Docker dataeng_github_metrics
# Run workflow on tags starting with v (eg. v2, v1.2.0)
on:
push:
branches: [ "master" ]
paths:
- ./data_pipelines/dataeng_github_metrics/*
pull_request:
branches: [ "master" ]
jobs:
Deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout#v1
- name: Login to GitHub Container Registry
uses: docker/login-action#v1
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GHCR_REGISTRY_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action#v2
- name: Build and Push Docker Image
uses: docker/build-push-action#v3
with:
context: ./data_pipelines/dataeng_github_metrics/
file: ./data_pipelines/dataeng_github_metrics/Dockerfile
push: true # Will only build if this is not here
tags: |
ghcr.io/mirantis/dataeng_github_metrics:latest
I'm trying to create a docker image using GitHub Actions from a static web application built with npm. However, while running the dockerfile, the /dist folder is not copied into the image as expected.
This is the dockerfile:
FROM nginx:1.21.6-alpine
COPY dist /usr/share/nginx/html
And this is the action:
name: Deploy
on:
push:
tags:
- v*
jobs:
build-homolog:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Setup
uses: actions/setup-node#v3
with:
node-version: '16'
- name: Build
env:
NODE_ENV: homolog
run: npm install; npm run build; docker build -t my-image:1.0.0 .
The result is a working nginx but without content, it just shows its default page. When I run the npm build and the docker build locally on my machine, it works as expected. I think there is a problem with the directory structure on the GitHub Actions machine, but I can't seem to understand it.
I have a GitHub actions CI which builds a my app in the last version of this app .git directory is mandatory for building it the problem is that GitHub action don't remove it in the docker build step of the CI.
Here is my CI template:
on:
push:
pull_request:
jobs:
build:
runs-on: ubuntu-20.04
if: github.ref == 'refs/heads/3.3.5-MV'
steps:
- uses: actions/checkout#v2
-
name: Set up QEMU
uses: docker/setup-qemu-action#v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action#v1
-
name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
id: docker_build
uses: docker/build-push-action#v2
with:
file: ./Dockerfile
tags: foo/bar:latest
push: true
-
name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
The Dockerfile
FROM ubuntu:focal
ENV TZ=Europe/Paris
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update && apt upgrade -y && apt install -y [all my dependencies]
WORKDIR /usr/src/app
COPY . .
WORKDIR /usr/src/app/build
#default path on docker image /usr/local
RUN cmake ../ -DCMAKE_INSTALL_PREFIX=/usr/local/ -DTOOLS=0
RUN make -j $(nproc) install
WORKDIR /usr/local/bin
The issue is in the docker/build-push-action#v2 action, which by default ignores the checkout created using the actions/checkout#v2 action:
By default, this action uses the Git context so you don't need to use the actions/checkout action to check out the repository because this will be done directly by BuildKit.
The git reference will be based on the event that triggered your workflow and will result in the following context: https://github.com/<owner>/<repo>.git#<ref>.
When you pass a git build context to docker build, it won't
include the .git directory.
If you read through the documentation for the docker/build-push-action#v2 action, you'll see that you can override this behavior:
However, you can use the Path context using the context input alongside the actions/checkout action to remove this restriction.
You would need to modify your workflow so that it includes an explicit
context: ., like this:
name: Build and push
id: docker_build
uses: docker/build-push-action#v2
with:
context: .
file: ./Dockerfile
tags: foo/bar:latest
push: true
I've put together an example repository that demonstrates this here; you can see the verification in this build run.
I want to run some NPM scripts, create a docker image and publish it on dockerhub.
I get this error trying to generate the image. It seems the second job doesn't see the build directory.
COPY failed: file not found in build context or excluded by .dockerignore: stat build/: file does not exist
Dockerfile
FROM httpd:2.4-alpine
COPY ./build/ /usr/local/apache2/htdocs/myapp/
EXPOSE 80
this is my workflow
name: CD
on:
push:
branches: [ main ]
jobs:
build:
name: App build
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v2
- name: Npm install
run: npm install
- name: Npm build
run: npm run build
deploy:
name: Docker image in DockerHub repository
runs-on: ubuntu-18.04
needs: build
steps:
- uses: actions/checkout#v2
- name: LS
run: ls -R
- name: Login to dockerhub
run: docker login -u ${{ secrets.DOCKER_HUB_USER }} -p ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Build Docker image
run: docker build -f ./Dockerfile -t myaccount/myapp .
- name: Push Docker image to DockerHub
run: docker push myaccount/myapp:latest
Project structure
| Dockerfile
| package.json
| README.md
| webpack.config.js
+---.github
| \---workflows
| deploy.yml
+---build
+---src
Update: I changed my workflow to ls the whole GITHUB_WORKSPACE.
build dir is actually missing (the other files are there). Yet, the build process (the first job) ends without errors, and if I try to ls -R in the first job the build dir is there. It is missing in the second job.
It seems the state of the workspace at the end of the first job is not available to the second job.
It seems for that you need actions/upload-artifact and actions/download-artifact.
name: CD
on:
push:
branches: [ main ]
jobs:
build:
name: App build
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v2
- name: Npm install
run: npm install
- name: Npm build
run: npm run build
- name: LS
run: ls -R
- name: Temporarily save webpack artifact
uses: actions/upload-artifact#v2
with:
name: webpack-artifact
path: build
retention-days: 1
deploy:
name: Docker image in DockerHub repository
runs-on: ubuntu-18.04
needs: build
steps:
## Build and deploy Docker images to DockerHub
- uses: actions/checkout#v2
- name: Retrieve built package
uses: actions/download-artifact#v2
with:
name: webpack-artifact
path: build
- name: LS
run: ls -R
- name: Login to dockerhub
run: docker login -u ${{ secrets.DOCKER_HUB_USER }} -p ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Build Docker image
run: docker build -f ./Dockerfile -t myaccount/myapp ./
- name: Push Docker image to DockerHub
run: docker push myaccount/myapp:latest
2 jobs in Github Actions are run on 2 separate machines, so the second job cannot see the first one. The solution is to put them into one job.
name: CD
on:
push:
branches: [ main ]
jobs:
deploy:
name: Docker image in DockerHub repository
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v2
- name: Npm install
run: npm install
- name: Npm build
run: npm run build
- name: LS
run: ls -R
- name: Login to dockerhub
run: docker login -u ${{ secrets.DOCKER_HUB_USER }} -p ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Build Docker image
run: docker build -f ./Dockerfile -t myaccount/myapp .
- name: Push Docker image to DockerHub
run: docker push myaccount/myapp:latest
I have a backend API via Node/Express and am just starting to learn CI/CD. I am using CircleCI and have a general template set up but can't seem to figure out how to get CircleCi to create a build folder during the process.
Below is my starting circle.yml file:
version: 2.1
jobs:
build:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "npm installing"
- run: npm install
- run: npm run build
test:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Running test suite (TODO not currently applicable)"
complete:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Completed build & test process (TODO Add linting checks)"
workflows:
version: 2.1
build_test:
jobs:
- build
I have a build script in my package.json with the following: "build": "tsc".
Running this outside of circleci, it compiles the build folder as I need.
Looking at the status on circleci, everything passes. But the branch does not have the actual build folder.
version: 2.1
jobs:
build:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "npm installing"
- run: npm install
- run: npm run build
- persist_to_workspace:
root: ~/project
paths:
- build
test:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Running test suite (TODO not currently applicable)"
complete:
docker:
- image: circleci/node:12.9.1
steps:
- checkout
- run: echo "Completed build & test process (TODO Add linting checks)"
deploy:
docker:
- image: circleci/node:12.9.1
steps:
- attach_workspace:
at: ~/project
- run:
name: Deployment Step
command: |
#!/bin/sh
ls build
workflows:
version: 2.1
build_test:
jobs:
- build
- test:
requires:
- build
- complete:
requires:
- test
- deploy:
requires:
- complete
When the build script runs, creates a folder called build that you can persist in order to use in another job. In the new job called deploy, it calls the folder where the build folder is, so now you can for example send this folder to a server.