I'm trying to use parallel_tests in my github action to run my test suite but I was not able to find a proper solution.
The official docs has one but it is for gitlab:
https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems
Any help would be appreciated thanks!
Here's a sample workflow you can drop into .github/workflows/tests.yml:
name: Rails Tests
on: push
env:
PGHOST: localhost
PGUSER: postgres
RAILS_ENV: test
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
# Set N number of parallel jobs you want to run
# Remember to update ci_node_index below to 0..N-1
ci_node_total: [6]
# set N-1 indexes for parallel jobs
# When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
ci_node_index: [0, 1, 2, 3, 4, 5]
services:
postgres:
image: postgres:11.5
ports: ["5432:5432"]
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
redis:
image: redis:5
ports: ["6379:6379"]
steps:
- uses: actions/checkout#v1
- uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Set node version (from .tool-versions)
run: echo "NODE_VERSION=$(cat .tool-versions | grep nodejs | sed 's/^nodejs //')" >> $GITHUB_ENV
- uses: actions/setup-node#v2
with:
node-version: ${{ env.NODE_VERSION }}
- uses: bahmutov/npm-install#v1
- name: Install PostgreSQL client
run: |
sudo apt-get -yqq install libpq-dev postgresql-client
- name: Test Prep
env:
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run: |
bundle exec rake parallel:create["1"] parallel:load_schema["1"]
- name: Run tests
env:
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
CI_NODE_INDEX: ${{ matrix.ci_node_index }}
run : |
bundle exec parallel_test spec/ -n $CI_NODE_TOTAL --only-group $CI_NODE_INDEX --type rspec
This was what I used to get it solved. Note: There are some other omitted parts such as the setup for ruby, postgresql, sqlite, etc.
name: "Lint and Test"
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
services:
redis:
image: redis
ports: ["6379:6379"]
postgres:
ports: ["5432:5432"]
steps:
//omitted setups
- name: Setup Parallel Database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
run: |
cp config/database.yml.example config/database.yml
bundle exec rake parallel:create
bundle exec rake parallel:rake[db:schema:load] || true
- name: Build and test with rspec
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: postgres
APP_REDIS_URL: redis://localhost:6379
MINIMUM_COVERAGE: 80
run: |
bundle exec parallel_rspec --verbose spec
Related
I would like to know how I can share service containers between jobs in GitHub Actions. With this workflow currently the containers get destroyed after the build step.
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
RAILS_ENV: test
RACK_ENV: test
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
POSTGRES_PASSWORD: postgres15
POSTGRES_USERNAME: postgres
POSTGRES_HOST: localhost
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Initialize Ruby
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Rails
run: bin/setup
services:
postgres:
image: postgres:15.1-alpine
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_USER: ${{ env.POSTGRES_USERNAME }}
# needed because the postgres container does not provide a healthcheck
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
test:
runs-on: ubuntu-latest
needs: [ build ]
steps:
- name: Lint Ruby
run: bundle exec rubocop
- name: Run tests
run: bin/rails test:all
coverage:
runs-on: ubuntu-latest
needs: [ build, test ]
steps:
- uses: joshmfrankel/simplecov-check-action#main
with:
minimum_suite_coverage: 98
minimum_file_coverage: 90
github_token: ${{ secrets.GITHUB_TOKEN }}
check_job_name: coverage
As per the documentation, you cannot share service containers across jobs.
https://docs.github.com/en/actions/using-containerized-services/about-service-containers
You can configure service containers for each job in a workflow. GitHub creates a fresh Docker container for each service configured in the workflow, and destroys the service container when the job completes. Steps in a job can communicate with all service containers that are part of the same job. However, you cannot create and use service containers inside a composite action.
I'm trying to configure a ruby.yml with github-actions but I'm getting this error at Setup database
PG::ConnectionBad: could not translate host name "db" to address: Temporary failure in name resolution
my yml file looks like this
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
ports: ['5432:5432']
steps:
- uses: actions/checkout#v2
- name: Setup Ruby
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Install Dependencies
run: |
sudo apt install -yqq libpq-dev
gem install bundler
- name: Install Gems
run: |
bundle install
- name: Setup database
env:
PG_DATABASE: postgres
PG_HOST: localhost
PG_USER: postgres
PG_PASSWORD: password
RAILS_ENV: test
WITH_COVERAGE: true
DISABLE_SPRING: 1
run: |
bundle exec rails db:setup
- name: Run Tests
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
RAILS_ENV: test
run: |
bundle exec rake test
I tried following this tutorial https://dev.to/buildwithallan/how-to-set-up-a-ci-workflow-using-github-actions-4818 but nothing it still doesn’t work. I'm not sure how I can fix this, would appreciate some help
double check this segment:
env:
PG_DATABASE: postgres
PG_HOST: localhost
PG_USER: postgres
PG_PASSWORD: password
RAILS_ENV: test
WITH_COVERAGE: true
DISABLE_SPRING: 1
check db in your config file. seems github action complaints it can not connect to db
Hi every one I have a problem when I executed my ci with github actions I received this problem.
Retrying download gem from https://gems.contribsys.com/ due to error (2/4): Gem::RemoteFetcher::FetchError bad response Unauthorized 401
this is my .yml file to github actions
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:13-alpine
ports:
- "5432:5432"
options:
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
POSTGRES_DB: rails_test
env:
RAILS_ENV: test
DATABASE_URL: "postgres://rails:password#localhost:5432/rails_test"
SIDEKIQ_PRO_USER: ${{ secrets.SIDEKIQ_PRO_USER }}
SIDEKIQ_PRO_PWD: ${{ secrets.SIDEKIQ_PRO_PWD }}
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v3
- name: Install Ruby and gems
uses: ruby/setup-ruby#8f312efe1262fb463d906e9bf040319394c18d3e # v1.92
with:
bundler-cache: true
# Add or replace any other lints here
- name: Security audit dependencies
run: bin/bundler-audit --update |
bundle config gems.contribsys.com ${secrets.SIDEKIQ_PRO_USER}:${ secrets.SIDEKIQ_PRO_PWD }
- name: Security audit application code
run: bin/brakeman -q -w2
- name: Lint Ruby files
run: bin/rubocop --parallel
in my gemfile I put this
gem 'sidekiq-pro', '~> 5.3', source: 'https://gems.contribsys.com'
For other hands, how I can print my ENV in console of github actions , I am not sure if the variable that I set is filled , any ideas?
When running the workflow in GitHub actions, rubocop errors out, but the code that it complains about is not present in my repo. How can I fix this?
Error:
Run bin/rubocop --parallel
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Lint/EndAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Metrics/LineLength has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/ElseAlignment has the wrong namespace - should be Layout
vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop.yml: Style/SpaceInsideParens has the wrong namespace - should be Layout
Error: The `Lint/HandleExceptions` cop has been renamed to `Lint/SuppressedException`.
(obsolete configuration found in vendor/bundle/ruby/2.7.0/gems/activerecord-import-1.1.0/.rubocop_todo.yml, please update it)
Error: Process completed with exit code 2.
GitHub Actions workflow yml file:
name: Verify
on: [push]
jobs:
linters:
name: Linters
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
yarn install --pure-lockfile
sudo apt-get -yqq install libpq-dev
gem install bundler
bundle install --jobs 4 --retry 3
- name: Run linters
run: |
bin/rubocop --parallel
tests:
name: Tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: myapp
POSTGRES_DB: myapp_test
POSTGRES_PASSWORD: ""
ports: ["5432:5432"]
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Setup Ruby and install gems
uses: ruby/setup-ruby#v1
with:
bundler-cache: true
- name: Setup Node
uses: actions/setup-node#v1
with:
node-version: 10.13.0
- name: Find yarn cache location
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: JS package cache
uses: actions/cache#v1
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install packages
run: |
gem install bundler
bundle install --jobs 4 --retry 3
- name: Setup test database
env:
RAILS_ENV: test
PGHOST: localhost
PGUSER: myapp
run: |
bundle exec rails db:create
bundle exec rails db:migrate
- name: Run tests
run: |
bundle exec rails test
As you already noticed there is not really a benefit in running RuboCop against third-party code and external gems because they are not really under your control and you certainly do not want to "fix" them.
Therefore I suggest excluding folders with external code, for example, gems in the vendor/bundle folder. This can be done by adding the following lines to your project's .rubycop.yml configuration file:
AllCops:
Exclude:
- 'vendor/bundle/**/*'
See the RuboCop docs about excluding files and folders.
Since multiple databases is still a relatively fresh topic on Rails I wasn't able to find good resources on how to integrate them with my github actions. As long as I had single database in my app those configs we correct:
name: CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:11
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: root
POSTGRES_DB: db_test
ports: ['5432:5432']
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout#v1
- name: Set up Ruby 2.6
uses: actions/setup-ruby#v1
with:
ruby-version: 2.6.x
- name: Build and test with Rake
env:
PGHOST: 127.0.0.1
PGUSER: postgres
PGPASSWORD: root
RAILS_ENV: test
run: |
sudo apt-get -yqq install libpq-dev
gem install bundler
bundle install --jobs 4 --retry 3
bundle exec rake db:test:prepare
bundle exec rails test
However after implementing a second db I got this error:
TypeError: no implicit conversion of nil into String
/opt/hostedtoolcache/Ruby/2.6.6/x64/bin/bundle:23:in load' /opt/hostedtoolcache/Ruby/2.6.6/x64/bin/bundle:23:in ' Tasks:
TOP => db:test:load => db:test:purge (See full trace by running task
with --trace) Error: Process completed with exit code 1.
I'm almost 100% positive that this error is related to the lack of credentials for the second db in my github action file. How can I fix it and include both my dbs there?