The project was recently updated from Ruby 3.0.1 and Rails 6.1.4 to Ruby 3.1.1 and Rails 7.0.2.
All tests pass in my local dev environment. I can't get the CI Pipeline to work. I'm using the Cloud version of Gitlab (Shared Gitlab? - I forget what it's called) and so I can't access the pipeline job from a command line. (I don't think).
I cleared my runner caches and made numerous changes to try and get it to work but to no avail.
Here is the CI file:
stages:
- test
- deploy
test-job:
stage: test
image: ruby:3.1
services:
- postgres:12.7
variables:
POSTGRES_USER: depot_postgresql
POSTGRES_PASSWORD: depot_postgresql
DB_USERNAME: depot_postgresql
DB_PASSWORD: depot_postgresql
DB_HOST: postgres
RAILS_ENV: test
# BUNDLE_PATH: vendor/bundle
before_script:
- gem update --system
script:
- apt-get update
- apt-get --assume-yes install sudo
- sudo apt install curl
- curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
- curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
- echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
- sudo apt-get update
- sudo apt-get install nodejs yarn
- node -v
- npm -v
- yarn -v
- gem install bundler -v 2.3.8
- bundle install
- bundle exec ruby -v
- bundle exec bundle version
- bundle exec rails -v
- bundle exec rails db:create db:schema:load --trace
- bundle exec rake assets:precompile
- bundle exec rails test
# - bundle exec rails test:system
The job gets to this point with no previous errors or warnings and then fails.
$ bundle exec ruby -v
ruby 3.1.1p18 (2022-02-18 revision 53f5fc4236) [x86_64-linux]
$ bundle exec bundle version
Bundler version 2.3.8 (2022-02-23 commit 4e812b9ca5)
$ bundle exec rails -v
Rails 7.0.2
$ bundle exec rails db:create db:schema:load --trace
Running via Spring preloader in process 2861
** Invoke db:create (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:load_config
** Execute db:create
Created database 'db/test.sqlite3'
** Invoke db:schema:load (first_time)
** Invoke db:load_config
** Invoke db:check_protected_environments (first_time)
** Invoke db:load_config
** Execute db:check_protected_environments
** Execute db:schema:load
$ bundle exec rake assets:precompile
I, [2022-03-10T19:46:45.438504 #2865] INFO -- : Writing /builds/goandgit/yumbin/public/assets/turbo-89398ded557eeb147879794d5b139c5e68c471e7ae9ee6ed32e01fb3a0798ce1.js
I, [2022-03-10T19:46:45.439116 #2865] INFO -- : Writing /builds/goandgit/yumbin/public/assets/turbo-89398ded557eeb147879794d5b139c5e68c471e7ae9ee6ed32e01fb3a0798ce1.js.gz
and so on. Then...
I, [2022-03-10T19:46:45.509345 #2865] INFO -- : Writing /builds/goandgit/yumbin/public/assets/users-04024382391bb910584145d8113cf35ef376b55d125bb4516cebeb14ce788597.css
I, [2022-03-10T19:46:45.509685 #2865] INFO -- : Writing /builds/goandgit/yumbin/public/assets/users-04024382391bb910584145d8113cf35ef376b55d125bb4516cebeb14ce788597.css.gz
Compiling...
Compilation failed:
yarn run v1.22.17
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command "webpack" not found.
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
If it is going through my public folder sequentially then it is failing in my packs/js/controllers folder perhaps specifically on this first bit of a code:
/******/ function __webpack_require__(moduleId) {
/******/
/******/ // Check if module is in cache
/******/ if(installedModules[moduleId]) {
/******/ return installedModules[moduleId].exports;
/******/ }
/******/ // Create a new module (and put it into the cache)
/******/ var module = installedModules[moduleId] = {
/******/ i: moduleId,
/******/ l: false,
/******/ exports: {}
/******/ };
I'm stumped.
Related
I am using travis for the first time and I don't know how to correctly add webpacker on it?
Here is what I have for now in .travis.yml
language: ruby
rvm:
- 2.4.4
addons:
postgresql: "9.4"
before_install:
- "echo 'gem: --no-document' > ~/.gemrc"
- "echo '--colour' > ~/.rspec"
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
before_script:
- bundle exec rake db:create db:migrate RAILS_ENV=test
- bundle exec rails webpacker:compile
Here is the build error
$ bundle exec rails webpacker:compile
Webpacker is installed 🎉 🍰
Using /home/travis/build/bill/ivan/config/webpacker.yml file for setting up webpack paths
Compiling…
Compilation failed:
/home/travis/build/bill/ivan/vendor/bundle/ruby/2.4.0/gems/webpacker-3.5.5/lib/webpacker/webpack_runner.rb:11:in `exec': No such file or directory - /home/travis/build/bill/ivan/node_modules/.bin/webpack (Errno::ENOENT)
from /home/travis/build/bill/ivan/vendor/bundle/ruby/2.4.0/gems/webpacker-3.5.5/lib/webpacker/webpack_runner.rb:11:in `block in run'
from /home/travis/build/bill/ivan/vendor/bundle/ruby/2.4.0/gems/webpacker-3.5.5/lib/webpacker/webpack_runner.rb:10:in `chdir'
from /home/travis/build/bill/ivan/vendor/bundle/ruby/2.4.0/gems/webpacker-3.5.5/lib/webpacker/webpack_runner.rb:10:in `run'
from /home/travis/build/bill/ivan/vendor/bundle/ruby/2.4.0/gems/webpacker-3.5.5/lib/webpacker/runner.rb:6:in `run'
from ./bin/webpack:15:in `<main>'
The command "bundle exec rails webpacker:compile" failed and exited with 1 during .
Your build has been stopped.
No such file or directory - /home/travis/build/bill/ivan/node_modules/.bin/webpack (Errno::ENOENT)
it's in my rails app tought
well maybe it could help someone else: so here the setup that is finally working
language: ruby
rvm:
- 2.4.4
addons:
postgresql: "9.4"
before_install:
- "echo 'gem: --no-document' > ~/.gemrc"
- "echo '--colour' > ~/.rspec"
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
before_script:
- bundle exec rake db:create db:migrate RAILS_ENV=test
cache:
bundler: true
directories:
- node_modules
yarn: true
install:
- bundle install
- nvm install node
- node -v
- npm i -g yarn
- yarn
script:
- bundle exec rails webpacker:compile
- bundle exec rails spec
I added an additional container in circle ci and tried to run tests parallaly.
This is my circle.yml file.
machine:
ruby:
version: 2.3.0
database:
post:
- cp config/sunspot.ci.yml config/sunspot.yml
- bundle exec sunspot-solr start -p 8981
dependencies:
pre:
- sudo apt-get update; sudo apt-get -y install solr-tomcat
test:
override:
- rvm use 2.3.0 && bundle exec rspec --color --format progress:
environment:
RAILS_ENV: test
parallel: true
files:
- "spec/**/*_spec.rb"
but tests don't seem to be running in parallel.
What am I missing? Thanks in advance
For some reason removing rvm use 2.3.0 from test section fixed my problem.
machine:
ruby:
version: 2.3.0
database:
# After default database task finished
post:
# Circle CI already replace our original sunspot.yml, replace it with backup copy `config/sunspot.yml.ci`
- cp config/sunspot.ci.yml config/sunspot.yml
- bundle exec sunspot-solr start -p 8981
dependencies:
pre:
- sudo apt-get update; sudo apt-get -y install solr-tomcat
test:
override:
- bundle exec rspec --tag ~#flaky --color --profile 20 --format progress:
timeout: 240
environment:
RAILS_ENV: test
parallel: true
files:
- my files here
I have been having this issue for 3 months or so now. When I do a mina deploy no errors will happen and it shows starting/stopping of sidekiq etc.. but sometimes the latest code doesn't actually deploy to the production server. Using the same code I have never had this issue on the staging environment either, which is really confusing.
To fix the issue, I have to manually kill the unicorn pid and then do a re-deploy and this works every time.
I'm not very familiar with any of this and it is mainly copied from articles & SO, but do you see anything in the code that could be causing issues? Should I be stopping unicorn and then starting it back up instead of just restarting it? My thought there is that I might miss requests.
desc "Deploys the current version to the server."
task :deploy => :environment do
deploy do
# stop accepting new workers
invoke :'sidekiq:quiet'
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
to :launch do
invoke :'sidekiq:restart'
invoke :'unicorn:restart'
end
end
Is there anyway I can output statements at certain times inside of the deploy or have it create a log?
-- EDIT: (ADDED TRACE) --
This is a passing trace. Is there some type of error that should show up on a failing trace that I can look for? It doesn't fail all the time, but I can run trace each time to try to catch a failing one.
** Invoke deploy (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke rbenv:load (first_time)
** Execute rbenv:load
** Execute deploy
** Invoke sidekiq:quiet (first_time)
** Invoke environment
** Execute sidekiq:quiet
** Invoke git:clone (first_time)
** Execute git:clone
** Invoke deploy:link_shared_paths (first_time)
** Execute deploy:link_shared_paths
** Invoke bundle:install (first_time)
** Execute bundle:install
** Invoke rails:db_migrate (first_time)
** Execute rails:db_migrate
** Invoke rails:assets_precompile (first_time)
** Execute rails:assets_precompile
** Invoke deploy:cleanup (first_time)
** Execute deploy:cleanup
** Invoke sidekiq:restart (first_time)
** Execute sidekiq:restart
** Invoke sidekiq:stop (first_time)
** Invoke environment
** Execute sidekiq:stop
** Invoke sidekiq:start (first_time)
** Invoke environment
** Execute sidekiq:start
** Invoke unicorn:restart (first_time)
** Invoke environment
** Execute unicorn:restart
-----> Loading environment
$ source ~/.bashrc
-----> Loading rbenv
$ export RBENV_ROOT="$HOME/.rbenv"
$ export PATH="$HOME/.rbenv/bin:$PATH"
$ eval "$(rbenv init -)"
-----> Creating a temporary build path
$ touch "deploy.lock"
$ mkdir -p "$build_path"
$ cd "$build_path"
-----> Quiet sidekiq (stop accepting new work)
$ RAILS_ENV="staging" bundle exec sidekiqctl quiet /home/deploy/PATH $ RAILS_ENV="staging" bundle exec sidekiqctl quiet /home/deploy/PATHNAME//shared/pids/sidekiq.pid
-----> Fetching new git commits
$ (cd "/home/deploy/PATHNAME//scm" && git fetch "git#github.com:k $ (cd "/home/deploy/PATHNAME//scm" && git fetch "git#github.com:kenmazaika/PATHNAME.git" "master:master" --force)
-----> Using git branch 'master'
$ git clone "/home/deploy/PATHNAME//scm" . --recursive --branch " $ git clone "/home/deploy/PATHNAME//scm" . --recursive --branch "master"
Cloning into '.'...
done.
-----> Using this git commit
$ git --no-pager log --format='%aN (%h):%n> %s' -n 1
MYNAME (a0ba681):
> Merge pull request #2040 from repo/issue2039
$ rm -rf .git
-----> Symlinking shared paths
$ mkdir -p "./config"
$ mkdir -p "."
$ rm -rf "./config/database.yml"
$ ln -s "/home/deploy/PATHNAME//shared/config/database.yml" "./co $ ln -s "/home/deploy/PATHNAME//shared/config/database.yml" "./config/database.yml"
$ rm -rf "./log"
$ ln -s "/home/deploy/PATHNAME//shared/log" "./log"
$ rm -rf "./config/secrets.yml"
$ ln -s "/home/deploy/PATHNAME//shared/config/secrets.yml" "./con $ ln -s "/home/deploy/PATHNAME//shared/config/secrets.yml" "./config/secrets.yml"
-----> Installing gem dependencies using Bundler
$ mkdir -p "/home/deploy/PATHNAME//shared/bundle"
$ mkdir -p "./vendor"
$ ln -s "/home/deploy/PATHNAME//shared/bundle" "./vendor/bundle"
$ bundle install --without development:test --path "./vendor/bundle" --de $ bundle install --without development:test --path "./vendor/bundle" --deployment
### USING A BUNCH OF GEMS ###
-----> DB migrations unchanged; skipping DB migration
-----> Skipping asset precompilation
$ cp -R "/home/deploy/PATHNAME//current/public/assets" "./public"
-----> Cleaning up old releases (keeping 5)
$ cd "/home/deploy/PATHNAME//releases" || exit 15
$ count=`ls -1d [0-9]* | sort -rn | wc -l`
$ remove=$((count > 5 ? count - 5 : 0))
$ ls -1d [0-9]* | sort -rn | tail -n $remove | xargs rm -rf {}
-----> Build finished
-----> Moving build to releases/301
$ mv "$build_path" "$release_path"
-----> Updating the current symlink
$ ln -nfs "$release_path" "current"
-----> Launching
$ cd "$release_path"
-----> Stop sidekiq
$ RAILS_ENV="staging" bundle exec sidekiqctl stop /home/deploy/PATHl $ RAILS_ENV="staging" bundle exec sidekiqctl stop /home/deploy/PATHNAME//shared/pids/sidekiq.pid 10
Sidekiq shut down gracefully.
-----> Start sidekiq
$ nohup bundle exec sidekiq -e staging -C /home/deploy/PATHNAME//current/config/sidekiq.yml -i 0 -P /home/deploy/PATHNAME//shared/pids/si $ nohup bundle exec sidekiq -e staging -C /home/deploy/PATHNAME//current/config/sidekiq.yml -i 0 -P /home/deploy/PATHNAME//shared/pids/sidekiq.pid >> /home/deploy/PATHNAME//current/log/sidekiq.log 2>&1 &
-----> Starting Unicorn...
I have a Rails App hosted on gitlab.com, and I am configuring it to deploy to heroku following this guide: http://docs.gitlab.com/ce/ci/examples/test-and-deploy-ruby-application-to-heroku.html. It works fine.
My question is, how can I run migrations every time I deploy to heroku? When deploying via CLI I would usually do:
git push heroku master && heroku run rake db:migrate
but using gitlab-ci.yml I have no clue on how to do this...
If you want to be able to use the full power of the Heroku CLI in your GitLab CI process (including having the build fail if a migration fails for whichever reason), you can also try this approach which will install the Heroku CLI and deliver status codes of your Heroku commands back to GitLab, as well as, of course, the command line output. Using heroku run without credentials on the commandline require the HEROKU_API_KEY environment variable to be set to a key which has access to the app in question.
before_script:
- echo "deb http://toolbelt.heroku.com/ubuntu ./" > /etc/apt/sources.list.d/heroku.list
- wget -O- https://toolbelt.heroku.com/apt/release.key | apt-key add -
- apt-get update
- apt-get install -y heroku-toolbelt
- gem install dpl
stages:
- deploy
test_on_heroku:
type: deploy
script:
- dpl --provider=heroku --app=my_heroku_app --api-key=$HEROKU_API_KEY
- heroku run <your command here> --exit-code --app my_heroku_app
I actually run my tests on an Heroku instance to be sure, the environment is exactly the same. This is where this comes in real handy.
The information in this answer may be out of date. Please see both answers below, and remember to upvote the answers that are up to date to help future visitors.
here is a sample .yml I have that runs my tests then pushed to Heroku stage (for master branch pushes) or production (for tags pushes)
image: "ruby:2.3"
test:
script:
- apt-get update -qy
- apt-get install -y nodejs
- gem install bundler
- bundle install -j $(nproc) --without production
- bundle exec rails db:create RAILS_ENV=test
- bundle exec rails db:migrate RAILS_ENV=test
- bundle exec rails RAILS_ENV=test
staging:
type: deploy
environment: staging
script:
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_STAGING_APP_NAME --api-key=$HEROKU_API_KEY
- "curl -n -X POST https://api.heroku.com/apps/$HEROKU_STAGING_APP_NAME/ps -H \"Accept: application/json\" -H \"Authorization: Bearer ${HEROKU_API_KEY}\" -d \"command=bundle exec rails db:migrate\""
only:
- master
production:
type: deploy
environment: production
script:
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_PRODUCTION_APP_NAME --api-key=$HEROKU_API_KEY
- "curl -n -X POST https://api.heroku.com/apps/$HEROKU_PRODUCTION_APP_NAME/ps -H \"Accept: application/json\" -H \"Authorization: Bearer ${HEROKU_API_KEY}\" -d \"command=bundle exec rails db:migrate\""
only:
- tags
To update #huesforalice's answer, this would also work for the new Heroku CLI, which replaced Heroku Toolbelt in November 2016:
before_script:
- apt-get update
- apt-get install apt-transport-https
- echo "deb https://cli-assets.heroku.com/branches/stable/apt ./" > /etc/apt/sources.list.d/heroku.list
- wget -O- https://cli-assets.heroku.com/apt/release.key | apt-key add -
- apt-get update
- apt-get install -y heroku
- gem install dpl
staging:
type: deploy
variables:
HEROKU_API_KEY: $HEROKU_STAGING_API_KEY
script:
- dpl --provider=heroku --app=$HEROKU_STAGING_APP --api-key=$HEROKU_STAGING_API_KEY
- heroku run rails db:migrate --exit-code --app $HEROKU_STAGING_APP
only:
- master
production:
type: deploy
variables:
HEROKU_API_KEY: $HEROKU_PRODUCTION_API_KEY
script:
- dpl --provider=heroku --app=$HEROKU_PRODUCTION_APP --api-key=$HEROKU_PRODUCTION_API_KEY
- heroku run rails db:migrate --exit-code --app $HEROKU_PRODUCTION_APP
only:
- tags
To further improve #huesforalice and #Jimmy Bosse 's answers - If you want to
avoid putting Heroku CLI installation in the global before_script but only use it in the deployment stages
at the same time, avoid copying and pasting the installation snippet into different stages
You can do something like this using YAML anchors to DRY up
before_script:
# the global before_script
- gem install bundler --no-document
- bundle check || bundle install --jobs $(nproc)
.deployment_before_script: &deployment_before_script
before_script:
- echo "deb http://toolbelt.heroku.com/ubuntu ./" > /etc/apt/sources.list.d/heroku.list
- wget -O- https://toolbelt.heroku.com/apt/release.key | apt-key add -
- apt-get update
- apt-get install -y heroku-toolbelt
- gem install dpl
# other stages...
staging:
stage: deploy
<<: *deployment_before_script
script:
- dpl --provider=heroku --app=$HEROKU_APP_STAGING --api-key=$HEROKU_API_KEY_STAGING
- heroku run bundle exec rails db:migrate --exit-code --app $HEROKU_APP_STAGING
only:
- master
production:
stage: deploy
<<: *deployment_before_script
script:
- dpl --provider=heroku --app=$HEROKU_APP_PRODUCTION --api-key=$HEROKU_API_KEY_STAGING
- heroku run bundle exec rails db:migrate --exit-code --app $HEROKU_APP_PRODUCTION
when: manual
only:
- tags
I have built a ruby on rails application using Ubuntu. My application is on github. Normally I transfer the latest developments from github to my website, www.mywebsite.com, by doing in Terminal:
cap production deploy:migrations
And everything works fine. I see in Terminal the progress of the transfer like:
* executing "rm -rf /var/www/apps/myapp/releases/20130720123042/public/assets &&\\\n mkdir -p /var/www/apps/myapp/releases/20130720123042/public &&\\\n mkdir -p /var/www/apps/myapp/shared/assets &&\\\n ln -s /var/www/apps/myapp/shared/assets /var/www/apps/myapp/releases/20130720123042/public/assets"
servers: ["92.51.243.6"]
[92.51.243.6] executing command
command finished in 626ms
* executing `bundle:install'
* executing "cd /var/www/apps/myapp/releases/20130720123042 && bundle install --gemfile /var/www/apps/myapp/releases/20130720123042/Gemfile --path /var/www/apps/myapp/shared/bundle --deployment --quiet --without development test"
servers: ["92.51.243.6"]
[92.51.243.6] executing command
command finished in 1595ms
* executing "chmod -R g+w /var/www/apps/myapp/releases/20130720123042"
servers: ["92.51.243.6"]
[92.51.243.6] executing command
command finished in 645ms
Now the process doesn't complete as it used to and I get this message:
* executing "cd /var/www/apps/myapp/releases/20130720123042 && bundle exec rake RAILS_ENV=production RAILS_GROUPS=assets assets:precompile"
servers: ["92.51.243.6"]
[92.51.243.6] executing command
** [out :: 92.51.243.6] rake aborted!
** [out :: 92.51.243.6] Received wrong number of arguments. [nil]
** [out :: 92.51.243.6] /var/www/apps/myapp/shared/bundle/ruby/1.8/gems/omniauth-1.1.0/lib/omniauth/strategy.rb:136:in `initialize'
more stuff here like the line above..
failed: "rvm_path=$HOME/.rvm/ $HOME/.rvm/bin/rvm-shell 'ree-1.8.7-2012.02' -c 'cd /var/www/apps/myapp/releases/20130720123042 && bundle exec rake RAILS_ENV=production RAILS_GROUPS=assets assets:precompile'" on 92.51.243.6
mypc#ubuntu:~/myapp$
Any idea how I can fix this problem? Thanks.