Hi every one I have a problem when I executed my ci with github actions I received this problem.
Retrying download gem from https://gems.contribsys.com/ due to error (2/4): Gem::RemoteFetcher::FetchError bad response Unauthorized 401
this is my .yml file to github actions
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:13-alpine
ports:
- "5432:5432"
options:
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
POSTGRES_DB: rails_test
env:
RAILS_ENV: test
DATABASE_URL: "postgres://rails:password#localhost:5432/rails_test"
SIDEKIQ_PRO_USER: ${{ secrets.SIDEKIQ_PRO_USER }}
SIDEKIQ_PRO_PWD: ${{ secrets.SIDEKIQ_PRO_PWD }}
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: Install Ruby and gems
uses: ruby/setup-ruby#8f312efe1262fb463d906e9bf040319394c18d3e # v1.92
with:
bundler-cache: true
# Add or replace any other lints here
- name: Security audit dependencies
run: bin/bundler-audit --update |
bundle config gems.contribsys.com ${secrets.SIDEKIQ_PRO_USER}:${ secrets.SIDEKIQ_PRO_PWD }
- name: Security audit application code
run: bin/brakeman -q -w2
- name: Lint Ruby files
run: bin/rubocop --parallel
in my gemfile I put this
gem 'sidekiq-pro', '~> 5.3', source: 'https://gems.contribsys.com'
For other hands, how I can print my ENV in console of github actions , I am not sure if the variable that I set is filled , any ideas?
Related
I would like to know how I can share service containers between jobs in GitHub Actions. With this workflow currently the containers get destroyed after the build step.
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
RAILS_ENV: test
RACK_ENV: test
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
POSTGRES_PASSWORD: postgres15
POSTGRES_USERNAME: postgres
POSTGRES_HOST: localhost
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Initialize Ruby
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Rails
run: bin/setup
services:
postgres:
image: postgres:15.1-alpine
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_USER: ${{ env.POSTGRES_USERNAME }}
# needed because the postgres container does not provide a healthcheck
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
test:
runs-on: ubuntu-latest
needs: [ build ]
steps:
- name: Lint Ruby
run: bundle exec rubocop
- name: Run tests
run: bin/rails test:all
coverage:
runs-on: ubuntu-latest
needs: [ build, test ]
steps:
- uses: joshmfrankel/simplecov-check-action#main
with:
minimum_suite_coverage: 98
minimum_file_coverage: 90
github_token: ${{ secrets.GITHUB_TOKEN }}
check_job_name: coverage
As per the documentation, you cannot share service containers across jobs.
https://docs.github.com/en/actions/using-containerized-services/about-service-containers
You can configure service containers for each job in a workflow. GitHub creates a fresh Docker container for each service configured in the workflow, and destroys the service container when the job completes. Steps in a job can communicate with all service containers that are part of the same job. However, you cannot create and use service containers inside a composite action.
When running the workflow in GitHub actions, rubocop errors out, but the code that it complains about is not present in my repo. How can I fix this?
Error:
Run bin/rubocop --parallel
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Lint/EndAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Metrics/LineLength has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/ElseAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/SpaceInsideParens has the wrong namespace - should be Layout
Error: The `Lint/HandleExceptions` cop has been renamed to `Lint/SuppressedException`.
(obsolete configuration found in vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop_todo.yml, please update it)
Error: Process completed with exit code 2.
GitHub Actions workflow yml file:
name: Verify
on: [push]
jobs:
linters:
name: Linters
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
yarn install --pure-lockfile
sudo apt-get -yqq install libpq-dev
gem install bundler
bundle install --jobs 4 --retry 3
- name: Run linters
run: |
bin/rubocop --parallel
tests:
name: Tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: myapp
POSTGRES_DB: myapp_test
POSTGRES_PASSWORD: ""
ports: ["5432:5432"]
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
gem install bundler
bundle install --jobs 4 --retry 3
- name: Setup test database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: myapp
run: |
bundle exec rails db:create
bundle exec rails db:migrate
- name: Run tests
run: |
bundle exec rails test
As you already noticed there is not really a benefit in running RuboCop against third-party code and external gems because they are not really under your control and you certainly do not want to "fix" them.
Therefore I suggest excluding folders with external code, for example, gems in the vendor/bundle folder. This can be done by adding the following lines to your project's .rubycop.yml configuration file:
AllCops:
Exclude:
- 'vendor/bundle/**/*'
See the RuboCop docs about excluding files and folders.
Here's my workflow file:
name: Build Pipeline
on: push
env:
NODE_VERSION: 11
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: ${{ env.NODE_VERSION }}
- id: cache-node-modules
uses: actions/cache#v2
with:
path: ${{ github.workspace }}/node_modules
key: node_modules-${{ hashFiles('package-lock.json') }}
restore-keys: node_modules
- uses: actions/cache#v2
with:
path: ${{ github.workspace }}/build
key: build-${{ github.sha }}
restore-keys: build
- if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: npm install
- run: npm run build -- --incremental
npm-scripts:
needs: [build]
runs-on: ubuntu-latest
strategy:
matrix:
script: ['lint:pipeline', 'lint:exports', 'i18n:pipeline', 'schema:validate']
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: ${{ env.NODE_VERSION }}
- id: cache-node-modules
uses: actions/cache#v2
with:
path: ${{ github.workspace }}/node_modules
key: node_modules-${{ hashFiles('package-lock.json') }}
- if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: |
echo 'Expected to have a cache hit for "node_modules", since this job runs after the "build" job, which caches the latest version of "node_modules". Not having a cache hit means probably there is a bug with the workflow file.'
exit 1
- id: cache-build-output
uses: actions/cache#v2
with:
path: ${{ github.workspace }}/build
key: build-${{ github.sha }}
- if: steps.cache-build-output.outputs.cache-hit != 'true'
run: |
echo 'Expected to have a cache hit for the build output folder, since this job runs after the "build" job, which caches the latest version of the "build" folder. Not having a cache hit means probably there is a bug with the workflow file.'
exit 1
- run: npm run ${{ matrix.script }}
jest-tests:
needs: [build]
runs-on: ubuntu-latest
container: node:11
services:
postgres:
image: postgres
env:
POSTGRES_DB: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
redis:
image: redis
steps:
- uses: actions/checkout#v2
- id: cache-node-modules
uses: actions/cache#v2
with:
path: ${{ github.workspace }}/node_modules
key: node_modules-${{ hashFiles('package-lock.json') }}
- if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: |
echo 'Expected to have a cache hit for "node_modules", since this job runs after the "build" job, which caches the latest version of "node_modules". Not having a cache hit means probably there is a bug with the workflow file.'
exit 1
- id: cache-build-output
uses: actions/cache#v2
with:
path: ${{ github.workspace }}/build
key: build-${{ github.sha }}
- if: steps.cache-build-output.outputs.cache-hit != 'true'
run: |
echo 'Expected to have a cache hit for the build output folder, since this job runs after the "build" job, which caches the latest version of the "build" folder. Not having a cache hit means probably there is a bug with the workflow file.'
exit 1
- run: echo
node_modules and build folders are cached in the build job. These caches are able to be restored without a problem in the npm-scripts job. However, they are not able to be restored in the jest-tests job, where it gets a Cache not found for input keys error.
I don't know how this is possible, since the exact same cache keys are able to be restored without a problem in all of the npm-scripts jobs.
When I remove the:
container: node:11
services:
postgres:
image: postgres
env:
POSTGRES_DB: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
redis:
image: redis
part (and hence let the job run on ubuntu-latest, instead of a Docker container), the cache is able to be restored again properly. So not sure what's going on here.
It seems that the #actions/cache job silently fails if there is no zstd binary available in the PATH in the container that you are running in. This may be the case for your Node container.
I found this out by setting ACTIONS_STEP_DEBUG to true in the repository secrets. The debug log shows that the action tries to run zstd and can't, but it is instead reported as a cache miss. Once I figured that out, I found that there is a bug report open for it: https://github.com/actions/cache/issues/580
It is a weird bug. The workaround that I found is not running the jest-tests job in a container. That is, running the jest-tests job in a regular, ubuntu-latest machine, and mapping the service container ports like:
jest-tests:
needs: [build]
runs-on: ubuntu-latest
services:
postgres:
image: postgres
ports:
- 5432:5432
env:
POSTGRES_DB: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
redis:
image: redis
ports:
- 6379:6379
I'm using a custom image and just by adding zstd package to the image, it made the action/cache to work.
I'm trying to use parallel_tests in my github action to run my test suite but I was not able to find a proper solution.
The official docs has one but it is for gitlab:
https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems
Any help would be appreciated thanks!
Here's a sample workflow you can drop into .github/workflows/tests.yml:
name: Rails Tests
on: push
env:
PGHOST: localhost
PGUSER: postgres
RAILS_ENV: test
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
# Set N number of parallel jobs you want to run
# Remember to update ci_node_index below to 0..N-1
ci_node_total: [6]
# set N-1 indexes for parallel jobs
# When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
ci_node_index: [0, 1, 2, 3, 4, 5]
services:
postgres:
image: postgres:11.5
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:5
ports: ["6379:6379"]
steps:
- uses: actions/checkout#v1
- uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Set node version (from .tool-versions)
run: echo "NODE_VERSION=$(cat .tool-versions | grep nodejs | sed 's/^nodejs //')" >> $GITHUB_ENV
- uses: actions/setup-node#v2
with:
node-version: ${{ env.NODE_VERSION }}
- uses: bahmutov/npm-install#v1
- name: Install PostgreSQL client
run: |
sudo apt-get -yqq install libpq-dev postgresql-client
- name: Test Prep
env:
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run: |
bundle exec rake parallel:create["1"] parallel:load_schema["1"]
- name: Run tests
env:
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run : |
bundle exec parallel_test spec/ -n $CI_NODE_TOTAL --only-group $CI_NODE_INDEX --type rspec
This was what I used to get it solved. Note: There are some other omitted parts such as the setup for ruby, postgresql, sqlite, etc.
name: "Lint and Test"
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
services:
redis:
image: redis
ports: ["6379:6379"]
postgres:
ports: ["5432:5432"]
steps:
//omitted setups
- name: Setup Parallel Database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
run: |
cp config/database.yml.example config/database.yml
bundle exec rake parallel:create
bundle exec rake parallel:rake[db:schema:load] || true
- name: Build and test with rspec
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
APP_REDIS_URL: redis://localhost:6379
MINIMUM_COVERAGE: 80
run: |
bundle exec parallel_rspec --verbose spec
I have the following two workflows:
Workflow to run test suite
Workflow to deploy the code, using https://github.com/miloserdow/capistrano-deploy
Now when I push my code, both workflows started. I want deployment Work to only start once Test Suite gets passed.
How can I do this?
Workflow that runs tests:
name: CI
on:
push:
branches: [setup_github]
jobs:
test:
runs-on: ubuntu-18.04
services:
postgres:
image: postgres:10
steps:
- name: Checkout
uses: actions/checkout#v1
- name: Set up Ruby
uses: ruby/setup-ruby#v1
with:
ruby-version: 2.5.3
- uses: borales/actions-yarn#v2.0.2
with:
cmd: install
- name: Install Dependencies
run: |
sudo apt-get -yqq install libpq-dev
- name: Install Gems
run: |
gem install bundler
- name: prepare Database
- name: RSpec
run: |
bundle exec rspec specs
Workflow that deploys:
name: Deploy on server
on:
push:
branches:
- setup_github
jobs:
deploy:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v1
- uses: ruby/setup-ruby#v1
with:
ruby-version: 2.5.3
bundler-cache: true
- uses: miloserdow/capistrano-deploy#master
with:
target: staging
deploy_key: ${{ secrets.DEPLOY_ENC_KEY }}
Your goal should be achievable by ensuring the following are true:
The CI workflow has run
The CI workflow was success
name: Deploy on server
on:
workflow_run:
workflows: [CI]
branches: [setup_github]
types:
- completed
jobs:
deploy:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v1
- uses: ruby/setup-ruby#v1
with:
ruby-version: 2.5.3
bundler-cache: true
- uses: miloserdow/capistrano-deploy#master
with:
target: staging
deploy_key: ${{ secrets.DEPLOY_ENC_KEY }}
This is described in the Github Actions docs on workflow_run.