How to use parallel_tests in github actions - ruby-on-rails

I'm trying to use parallel_tests in my github action to run my test suite but I was not able to find a proper solution.
The official docs has one but it is for gitlab:
https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems
Any help would be appreciated thanks!

Here's a sample workflow you can drop into .github/workflows/tests.yml:
name: Rails Tests
on: push
env:
PGHOST: localhost
PGUSER: postgres
RAILS_ENV: test
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
# Set N number of parallel jobs you want to run
# Remember to update ci_node_index below to 0..N-1
ci_node_total: [6]
# set N-1 indexes for parallel jobs
# When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
ci_node_index: [0, 1, 2, 3, 4, 5]
services:
postgres:
image: postgres:11.5
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:5
ports: ["6379:6379"]
steps:
- uses: actions/checkout#v1
- uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Set node version (from .tool-versions)
run: echo "NODE_VERSION=$(cat .tool-versions | grep nodejs | sed 's/^nodejs //')" >> $GITHUB_ENV
- uses: actions/setup-node#v2
with:
node-version: ${{ env.NODE_VERSION }}
- uses: bahmutov/npm-install#v1
- name: Install PostgreSQL client
run: |
sudo apt-get -yqq install libpq-dev postgresql-client
- name: Test Prep
env:
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run: |
bundle exec rake parallel:create["1"] parallel:load_schema["1"]
- name: Run tests
env:
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run : |
bundle exec parallel_test spec/ -n $CI_NODE_TOTAL --only-group $CI_NODE_INDEX --type rspec

This was what I used to get it solved. Note: There are some other omitted parts such as the setup for ruby, postgresql, sqlite, etc.
name: "Lint and Test"
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
services:
redis:
image: redis
ports: ["6379:6379"]
postgres:
ports: ["5432:5432"]
steps:
//omitted setups
- name: Setup Parallel Database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
run: |
cp config/database.yml.example config/database.yml
bundle exec rake parallel:create
bundle exec rake parallel:rake[db:schema:load] || true
- name: Build and test with rspec
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
APP_REDIS_URL: redis://localhost:6379
MINIMUM_COVERAGE: 80
run: |
bundle exec parallel_rspec --verbose spec

Related

GitHub Actions - share services across jobs

I would like to know how I can share service containers between jobs in GitHub Actions. With this workflow currently the containers get destroyed after the build step.
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
RAILS_ENV: test
RACK_ENV: test
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
POSTGRES_PASSWORD: postgres15
POSTGRES_USERNAME: postgres
POSTGRES_HOST: localhost
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Initialize Ruby
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Rails
run: bin/setup
services:
postgres:
image: postgres:15.1-alpine
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_USER: ${{ env.POSTGRES_USERNAME }}
# needed because the postgres container does not provide a healthcheck
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
test:
runs-on: ubuntu-latest
needs: [ build ]
steps:
- name: Lint Ruby
run: bundle exec rubocop
- name: Run tests
run: bin/rails test:all
coverage:
runs-on: ubuntu-latest
needs: [ build, test ]
steps:
- uses: joshmfrankel/simplecov-check-action#main
with:
minimum_suite_coverage: 98
minimum_file_coverage: 90
github_token: ${{ secrets.GITHUB_TOKEN }}
check_job_name: coverage
As per the documentation, you cannot share service containers across jobs.
https://docs.github.com/en/actions/using-containerized-services/about-service-containers
You can configure service containers for each job in a workflow. GitHub creates a fresh Docker container for each service configured in the workflow, and destroys the service container when the job completes. Steps in a job can communicate with all service containers that are part of the same job. However, you cannot create and use service containers inside a composite action.

Github-actions: Temporary failure in name resolution (PG::ConnectionBad)

I'm trying to configure a ruby.yml with github-actions but I'm getting this error at Setup database
PG::ConnectionBad: could not translate host name "db" to address: Temporary failure in name resolution
my yml file looks like this
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
ports: ['5432:5432']
steps:
- uses: actions/checkout#v2
- name: Setup Ruby
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Install Dependencies
run: |
sudo apt install -yqq libpq-dev
gem install bundler
- name: Install Gems
run: |
bundle install
- name: Setup database
env:
PG_DATABASE: postgres
PG_HOST: localhost
PG_USER: postgres
PG_PASSWORD: password
RAILS_ENV: test
WITH_COVERAGE: true
DISABLE_SPRING: 1
run: |
bundle exec rails db:setup
- name: Run Tests
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
RAILS_ENV: test
run: |
bundle exec rake test
I tried following this tutorial https://dev.to/buildwithallan/how-to-set-up-a-ci-workflow-using-github-actions-4818 but nothing it still doesn’t work. I'm not sure how I can fix this, would appreciate some help
double check this segment:
env:
PG_DATABASE: postgres
PG_HOST: localhost
PG_USER: postgres
PG_PASSWORD: password
RAILS_ENV: test
WITH_COVERAGE: true
DISABLE_SPRING: 1
check db in your config file. seems github action complaints it can not connect to db

Gem::RemoteFetcher::FetchError bad response Unauthorized 401 sidekiqpro github actions

Hi every one I have a problem when I executed my ci with github actions I received this problem.
Retrying download gem from https://gems.contribsys.com/ due to error (2/4): Gem::RemoteFetcher::FetchError bad response Unauthorized 401
this is my .yml file to github actions
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:13-alpine
ports:
- "5432:5432"
options:
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
POSTGRES_DB: rails_test
env:
RAILS_ENV: test
DATABASE_URL: "postgres://rails:password#localhost:5432/rails_test"
SIDEKIQ_PRO_USER: ${{ secrets.SIDEKIQ_PRO_USER }}
SIDEKIQ_PRO_PWD: ${{ secrets.SIDEKIQ_PRO_PWD }}
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: Install Ruby and gems
uses: ruby/setup-ruby#8f312efe1262fb463d906e9bf040319394c18d3e # v1.92
with:
bundler-cache: true
# Add or replace any other lints here
- name: Security audit dependencies
run: bin/bundler-audit --update |
bundle config gems.contribsys.com ${secrets.SIDEKIQ_PRO_USER}:${ secrets.SIDEKIQ_PRO_PWD }
- name: Security audit application code
run: bin/brakeman -q -w2
- name: Lint Ruby files
run: bin/rubocop --parallel
in my gemfile I put this
gem 'sidekiq-pro', '~> 5.3', source: 'https://gems.contribsys.com'
For other hands, how I can print my ENV in console of github actions , I am not sure if the variable that I set is filled , any ideas?

GitHub Actions: rubocop errors out, but the code is not from my repo

When running the workflow in GitHub actions, rubocop errors out, but the code that it complains about is not present in my repo. How can I fix this?
Error:
Run bin/rubocop --parallel
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Lint/EndAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Metrics/LineLength has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/ElseAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/SpaceInsideParens has the wrong namespace - should be Layout
Error: The `Lint/HandleExceptions` cop has been renamed to `Lint/SuppressedException`.
(obsolete configuration found in vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop_todo.yml, please update it)
Error: Process completed with exit code 2.
GitHub Actions workflow yml file:
name: Verify
on: [push]
jobs:
linters:
name: Linters
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
yarn install --pure-lockfile
sudo apt-get -yqq install libpq-dev
gem install bundler
bundle install --jobs 4 --retry 3
- name: Run linters
run: |
bin/rubocop --parallel
tests:
name: Tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: myapp
POSTGRES_DB: myapp_test
POSTGRES_PASSWORD: ""
ports: ["5432:5432"]
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
gem install bundler
bundle install --jobs 4 --retry 3
- name: Setup test database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: myapp
run: |
bundle exec rails db:create
bundle exec rails db:migrate
- name: Run tests
run: |
bundle exec rails test
As you already noticed there is not really a benefit in running RuboCop against third-party code and external gems because they are not really under your control and you certainly do not want to "fix" them.
Therefore I suggest excluding folders with external code, for example, gems in the vendor/bundle folder. This can be done by adding the following lines to your project's .rubycop.yml configuration file:
AllCops:
Exclude:
- 'vendor/bundle/**/*'
See the RuboCop docs about excluding files and folders.

Rails: Deploy multiple databases to github actions

Since multiple databases is still a relatively fresh topic on Rails I wasn't able to find good resources on how to integrate them with my github actions. As long as I had single database in my app those configs we correct:
name: CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: root
POSTGRES_DB: db_test
ports: ['5432:5432']
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout#v1
- name: Set up Ruby 2.6
uses: actions/setup-ruby#v1
with:
ruby-version: 2.6.x
- name: Build and test with Rake
env:
PGHOST: 127.0.0.1
PGUSER: postgres
PGPASSWORD: root
RAILS_ENV: test
run: |
sudo apt-get -yqq install libpq-dev
gem install bundler
bundle install --jobs 4 --retry 3
bundle exec rake db:test:prepare
bundle exec rails test
However after implementing a second db I got this error:
TypeError: no implicit conversion of nil into String
/opt/hostedtoolcache/Ruby/2.6.6/x64/bin/bundle:23:in load' /opt/hostedtoolcache/Ruby/2.6.6/x64/bin/bundle:23:in ' Tasks:
TOP => db:test:load => db:test:purge (See full trace by running task
with --trace) Error: Process completed with exit code 1.
I'm almost 100% positive that this error is related to the lack of credentials for the second db in my github action file. How can I fix it and include both my dbs there?

Resources