CircleCI 2.0 - exit code 127, bundle command not found - ruby-on-rails

I'm trying to get a rails app to run on Circle CI. With the following configuration I can't seem to get anything to work at all. When I ssh in, I can't even run sudo.
Here's the error output:
install dependencies
$ #!/bin/bash -eo pipefail
bundler install --jobs=4 --retry=3 --path vendor/bundle
/bin/bash: bundler: command not found
Exited with code 127
I noticed there is nothing but assets/ under the vendor directory.
Here's my configuration file:
version: 2
jobs:
build:
working_directory: ~/repo
docker:
- image: circleci/postgres:9.6-alpine-postgis-ram
- image: circleci/ruby:2.3
environment:
PGHOST: 127.0.0.1
PGUSER: staging
RAILS_ENV: test
- image: circleci/postgres:10.5
environment:
POSTGRES_USER: staging
POSTGRES_DB: test
POSTGRES_PASSWORD: ""
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "Gemfile.lock" }}
# fallback to using the latest cache if no exact match is found
- v1-dependencies-
- run:
name: install dependencies
command: |
bundle install --jobs=4 --retry=3 --path vendor/bundle
- save_cache:
paths:
- ./vendor/bundle
key: v1-dependencies-{{ checksum "Gemfile.lock" }}
# Database setup
- run: bundle exec rake db:create
- run: bundle exec rake db:schema:load
# run tests!
- run:
name: run tests
command: |
mkdir /tmp/test-results
TEST_FILES="$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)"
bundle exec rspec --format progress \
--format RspecJunitFormatter \
--out /tmp/test-results/rspec.xml \
--format progress \
$TEST_FILES
# collect reports
- store_test_results:
path: /tmp/test-results
- store_artifacts:
path: /tmp/test-results
destination: test-results
All help is appreciated, thanks!

Your commands are running on the postgres container because it is listed first. If you make the ruby container first, then you will have access to bundle. And then when you SSH, you will be inside of the ruby container.
https://circleci.com/docs/2.0/configuration-reference/#docker--machine--macosexecutor
The first image listed in the file defines the primary container image where all steps will run.

Related

Problems storing video/screenshots when testing Rails app with Cypress/CircleCI

I am running Cypress to test a Rails app on CircleCI. I have the tests running on CircleCI with the following config. But no video/screenshot assets are created in CircleCI artefacts if tests fail.
version: 2.1
orbs:
ruby: circleci/ruby#1.0.6
node: circleci/node#3.0.1
jobs:
build:
docker:
- image: cimg/ruby:2.7.2-node
steps:
- checkout # pull down our git code.
- ruby/install-deps # use the ruby orb to install dependencies
- node/install-packages:
pkg-manager: yarn
test:
parallelism: 3
docker:
- image: cimg/ruby:2.7.2-node # this is our primary docker image, where step commands run.
- image: circleci/postgres:12.3
environment: # add POSTGRES environment variables.
POSTGRES_USER: user
POSTGRES_PASSWORD: password
POSTGRES_DB: testdb
environment:
BUNDLE_JOBS: "3"
BUNDLE_RETRY: "3"
PGHOST: 127.0.0.1
PGUSER: user
PGPASSWORD: password
RAILS_ENV: test
steps:
- checkout
- ruby/install-deps
- node/install-packages:
pkg-manager: yarn
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: bundle exec rails db:schema:load --trace
- run:
name: Load test data
command: bundle exec rails db:seed --trace
- run:
name: Run Rails Server
background: true
command: CYPRESS=1 bundle exec rails s -p 5017
- run:
name: Wait for server
command: |
until $(curl --retry 10 --output /dev/null --silent --head --fail http://127.0.0.1:5017); do
printf '.'
sleep 5
done
- run: sudo apt-get update
- run: sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb
- run: yarn cypress install --force
- run:
yarn: true
command: yarn run cypress run
- store_test_results:
path: test-reports/

CircleCI AWSCLI SyntaxError: Non-ASCII character '\xc3' error

I am working on a Ruby on Rails project where AWSCLI is used.
Recently, When I push my code to Staging, it tries to import newkeys, PrivateKey, PublicKey from rsa.key and throws an error.
The error message is as follows:
SyntaxError: Non-ASCII character '\xc3' in file /opt/circleci/.pyenv/versions/2.7.12/lib/python2.7/site-packages/rsa/key.py on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
Error message Image
My circleci/config.yml file is as follows:
orbs:
aws-ecr: circleci/aws-ecr#7.0.0
version: 2.1
jobs:
build:
working_directory: ~/mfds
docker:
- image: circleci/ruby:2.3.4-node
environment:
RAILS_ENV: citest
LANG: C.UTF-8
- image: snowhork/mysql_jp:5.7
environment:
MYSQL_ROOT_PASSWORD: ***
MYSQL_DATABASE: myapp
MYSQL_USER: user
MYSQL_PASSWORD: ***
MYSQL_ROOT_HOST: "%"
steps:
- checkout
- restore_cache:
key: gemfiles-{{ checksum "mfds/Gemfile.lock" }}
# Bundle install dependencies
- run: cd mfds/ && bundle install --path vendor/bundle
# Store bundle cache
- save_cache:
key: gemfiles-{{ checksum "mfds/Gemfile.lock" }}
paths:
- /home/circleci/mfds/mfds/vendor/bundle
- run:
name: Wait for db
command: dockerize -wait tcp://localhost:3306 -timeout 1m
# Database setup
- run:
name: db setting
command: |
cd mfds
bundle exec rake db:create
bundle exec rake db:ridgepole:apply[citest]
- run:
name: rspec
command: |
cd mfds
bundle exec rspec
staging_deploy:
machine: true
steps:
- checkout
- run:
name: asset compile
command: |
cd mfds
npm install
./node_modules/.bin/webpack --optimize-minimize
- run:
name: gem install
command: |
gem install aws-sdk-ecs
pip install futures
pip install --upgrade awscli
- run:
name: push image
command: |
export AWS_ACCESS_KEY_ID=$XX_AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$XX_AWS_SECRET_ACCESS_KEY
bin/deployv2/docker-push staging
- run:
name: update service
command: |
export AWS_ACCESS_KEY_ID=$XX_AWS_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$XX_AWS_SECRET_ACCESS_KEY
bin/deployv2/update-task staging
....
I don't understand how to resolve this problem.

Circle CI fails to execute psql command when running rake db:structure:load

So this is my config.yml
version: 2.1
orbs:
queue: eddiewebb/queue#1.5.0
executors:
node_postgres_redis:
docker:
- image: circleci/ruby:2.4.10-node-browsers
environment:
CC_TEST_REPORTER_ID: 7ceff1524bdc09dsd3e232321cf9daa531154170d590823232eddqw2330f3570
PGHOST: 127.0.0.1
RAILS_ENV: test
TEST_REPORT_PATH: "test/reports"
- image: circleci/postgres:9.6.2-alpine
environment:
POSTGRES_USER: circleci
POSTGRES_DB: circleci-jet-test
POSTGRES_PASSWORD: ""
- image: circleci/redis:3.2
ubuntu:
machine:
image: ubuntu-1604:202004-01
commands:
setup_and_run_test:
steps:
- checkout
# Restore bundle cache
- restore_cache:
key: jet-bundle-{{ .Branch }}-{{ checksum "Gemfile.lock" }}
# imagemagic
- run:
name: Setup Image Magick
command: |
sudo apt-get install libmagickwand-dev imagemagick imagemagick-6.q16 libmagickcore-dev
sudo ln -s /usr/lib/x86_64-linux-gnu/ImageMagick-6.9.7/bin-q16/Magick-config /usr/bin/Magick-config
- run:
name: Install Bundler
command: |
gem install bundler -v 1.17.3
# Install gem dependencies
- run: bundle check --path=vendor/bundle || bundle install --path=vendor/bundle
# Store bundle cache
- save_cache:
key: jet-bundle-{{ .Branch }}-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
# Database setup
- run:
name: Database Setup
command: |
bundle exec rake db:create
bundle exec rake db:migrate
bundle exec rake db:structure:load
Everythings runs properly until it tries to execute
bundle exec rake db:structure:load
Then it throws this error which is a bit weird
failed to execute:
psql -q -f /home/circleci/circleci-jet/db/structure.sql *****_test
Please check the output above for any errors and make sure that psql is installed in your PATH and has proper permissions.
I cannot understand this error and what needs to be done on my side.
Anyone with examples, please can show me direction.
Thanks

Circle CI 2.0 Rails Redis Resque - Rspec Redis Server Not Found Error

When our test suite runs we are getting the following issue regarding redis-server. No matter what we have tried, nothing seems to get past this error. We have validated via dockerize that the containers are live by waiting as seen below, but this error still occurs.
Any thoughts would be greatly appreciated!
Resque Initializer
require 'resque'
require 'redis'
require 'yaml'
# Resque Plugins
require 'resque/plugins/retry'
require 'resque-retry'
require 'resque-retry/server'
require 'resque-lock-timeout'
require 'resque-scheduler'
require 'resque/failure/multiple'
require 'resque/failure/redis'
require 'resque-job-stats/server'
require 'resque/rollbar'
if AppUnsecure.settings[:active_db_services].include?('redis')
uri = URI.parse(ENV["REDIS_URL"])
config = {
host: uri.host,
port: uri.port,
password: uri.password
}
Resque::Failure::Multiple.classes = [ Resque::Failure::Redis, Resque::Failure::Rollbar ]
Resque::Failure.backend = Resque::Failure::Multiple
Resque.redis = Redis.new(config)
elsif AppUnsecure.settings[:active_db_services].include?('redis-continous-integration')
Resque::Failure::MultipleWithRetrySuppression.classes = [Resque::Failure::Redis]
Resque.redis = Redis.new(host: 'redis://localhost', port: 6391)
else
Resque::Failure::MultipleWithRetrySuppression.classes = [Resque::Failure::Redis]
Resque.redis = Redis.new
end
Resque.redis.namespace = 'resque:GathrlySmartforms'
# Ignores Resque when processing jobs if activated!
Resque.inline = true if AppUnsecure.settings[:process_redis_inline]
# Setup Scheduler
# https://github.com/resque/resque-scheduler/issues/118
# https://github.com/resque/resque-scheduler/issues/581
Resque::Scheduler.configure do |c|
c.quiet = false
c.verbose = false
c.logfile = File.join(Rails.root, 'log', "#{Rails.env}_resque_scheduler.log")
c.logformat = 'text'
end
Resque::Scheduler.dynamic = true
schedules = {}
global = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
schedules.merge!(global) if global
# http://stackoverflow.com/questions/12158226/how-do-i-skip-loading-of-rails-initializers-when-running-a-rake-task
unless defined?(is_running_migration?) && is_running_migration?
Resque.schedule = schedules if schedules.present?
end
Resque::Server.class_eval do
use Rack::Auth::Basic do |username, password|
[username, password] == [Rails.application.secrets.my_resque_username, Rails.application.secrets.my_resque_password]
end
end
Circle Configuration
version: 2
jobs:
build:
working_directory: ~/DIR_NAME
docker:
- image: circleci/ruby:2.4.1-node
environment:
RAILS_ENV: continous_integration
PGHOST: 127.0.0.1
PGUSER: rails_test_user
- image: circleci/postgres:9.6.3-alpine
environment:
POSTGRES_USER: rails_test_user
POSTGRES_PASSWORD: ""
POSTGRES_DB: continous_integration
- image: redis:4.0.6
steps:
- checkout
- run:
name: Dockerize v0.6.0
command: |
wget https://github.com/jwilder/dockerize/releases/download/v0.6.0/dockerize-linux-amd64-v0.6.0.tar.gz
sudo rm -rf /usr/local/bin/dockerize
sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-v0.6.0.tar.gz
rm dockerize-linux-amd64-v0.6.0.tar.gz
- run:
name: Wait for PG
command: dockerize -wait tcp://localhost:5432 -timeout 2m
- run:
name: Wait for Redis
command: |
dockerize -wait tcp://localhost:6379 -timeout 2m
- restore_cache:
keys:
- DIR_NAME-{{ checksum "Gemfile.lock" }}
- DIR_NAME-
- save_cache:
key: rails-demo-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
- run:
name: Setup Bundler and Gems
command: |
gem install bundler
gem update bundler
gem install brakeman
gem install rubocop
gem install rubocop-rspec
gem install scss_lint
gem install eslint-rails
gem install execjs
bundle config without development:test
bundle check --path=vendor/bundle || bundle install --without development test --path=vendor/bundle --jobs 4 --retry 3
- run:
name: Install Phantom Js
command: |
sudo curl --output /tmp/phantomjs https://s3.amazonaws.com/circle-downloads/phantomjs-2.1.1
sudo chmod ugo+x /tmp/phantomjs
sudo ln -sf /tmp/phantomjs /usr/local/bin/phantomjs
- run:
name: Install Postgres Tools
command: |
sudo apt-get update
sudo apt-get install postgresql-client
- run:
name: Install Redis Tools
command: |
sudo apt-get install redis-tools ; while ! redis-cli ping 2>/dev/null ; do sleep 1 ; done
- run:
name: Build Rails Database Yaml
command: |
cp config/database_example.yml config/database.yml
- run:
name: Setup Rails Database
command: |
bundle exec rake db:drop
bundle exec rake db:setup
- run:
name: Run Rspec
timeout: 60
command: |
RAILS_ENV=continous_integration bundle exec rspec --format RspecJunitFormatter -o /tmp/test-results/rspec.xml
- run:
name: Run Brakeman
command: |
brakeman -z
- run:
name: Run Rubocop
command: |
bundle exec rubocop --format fuubar --require rubocop-rspec --config .rubocop.yml
- run:
name: Run the SCSS Linter
command: |
bundle exec scss-lint --config=config/scsslint.yml
- run:
name: Run the Eslint Linter for JS
command: |
bundle exec rake eslint:run_all
- store_test_results:
path: /tmp/test-results
UPDATE
On various test runs it may occasionally work...however, the error is still the same, this must be the cause...
I don't know what Postgres connects to redis fine means exactly. The postgres user like through redis-cli? Postgres isn't connecting directly is it? (foreign data wrapper -- just read up on this ... I didn't even know you could do that, wow). Well, nevermind, that's sort of a tangent ...
Some minor insight, I can't test this whole thing very easily ... the PID file not being that is probably because it's just dying. The kill: invalid argument Q is probably from something doing kill -QUIT. This could be in redis' scripts or something. I bet the process is already dead and kill is just erroring out poorly. I think kill really wants the PID to exist. Although I just tested this and this is NOT how this works. So maybe Circle has non-GNU coreutils installed or something?
I don't think POSTGRES_USER should be in the redis image. See this example: https://discuss.circleci.com/t/circleci-bug-for-builds-with-redis/13124 Looks like things changed with 2.0.
If the gist has been updated to a completely new config, update the error messages and problem too, unless it's the same.
Based on what we can tell from here, it seems that your configuration is referencing a redis process that does not exist in test.
Also you're bundling without test. Not sure why.
Can you post your Redis configuration file? <- Still need this.
You need to configure REDIS_URL to the docker image.

Circle Ci- Run rspec tests parallely

I added an additional container in circle ci and tried to run tests parallaly.
This is my circle.yml file.
machine:
ruby:
version: 2.3.0
database:
post:
- cp config/sunspot.ci.yml config/sunspot.yml
- bundle exec sunspot-solr start -p 8981
dependencies:
pre:
- sudo apt-get update; sudo apt-get -y install solr-tomcat
test:
override:
- rvm use 2.3.0 && bundle exec rspec --color --format progress:
environment:
RAILS_ENV: test
parallel: true
files:
- "spec/**/*_spec.rb"
but tests don't seem to be running in parallel.
What am I missing? Thanks in advance
For some reason removing rvm use 2.3.0 from test section fixed my problem.
machine:
ruby:
version: 2.3.0
database:
# After default database task finished
post:
# Circle CI already replace our original sunspot.yml, replace it with backup copy `config/sunspot.yml.ci`
- cp config/sunspot.ci.yml config/sunspot.yml
- bundle exec sunspot-solr start -p 8981
dependencies:
pre:
- sudo apt-get update; sudo apt-get -y install solr-tomcat
test:
override:
- bundle exec rspec --tag ~#flaky --color --profile 20 --format progress:
timeout: 240
environment:
RAILS_ENV: test
parallel: true
files:
- my files here

Resources