I'm deploying to a staging server which is running in environment=development. My deploy script fails with the message Could not find gem 'webmock (>= 0) ruby' in the gems available on this machine. but I cannot see why the script should ever need that gem because my Gemfile only lists it in the test group.
Do you have any idea why my staging server is trying to require webmock during deployment?
My deploy script
Here is the command from my deploy script fails:
executing ["cd /var/www/clu2/staging/releases/20130429170940 && bundle exec whenever --update-crontab staging --set environment=development --roles db"]
One command earlier, the script runs bundle install, omitting the test gems:
executing ["cd /var/www/clu2/staging/current && bundle install --without=test --no-update-sources"]
My usage of webmock is only in test
You can see that webmock is only required in my spec_helper file:
$ grep -r webmock .
./Gemfile: gem "webmock"
./Gemfile.lock: webmock (1.11.0)
./Gemfile.lock: webmock
./spec/spec_helper.rb: require "webmock/rspec"
You can see that webmock is specified as a test gem in my Gemfile:
group :test do
gem 'spork-rails' # pre-load rails environment for faster test launches
gem "webmock"
end
You should explicitly set :bundle_without config option in your staging deployment configuration file (do you use it? If not take a look at capistrano-ext gem which provides multistage features):
set :bundle_without, []
That's because :bundle_without is defined like this in production deployment:
set :bundle_without, [ :development, :test ]
One more thing.
I think that you should run your stage machine with production env. Staging machine is commonly used to emulate how code is working in production.
Related
I've tried everything I could think of, but I can't get deployment to work.
The app needs to deploy to a VPS hosted by Alwaysdata and running ruby 2.6.2, using Capistrano for deployment.
It's a Rails 6.0.2.2 application, using webpack for JS and sprockets for legacy scripts, all images and CSS.
# On local machine (MacOS)
$ bundle exec cap production deploy
00:00 git:wrapper
00:01 git:check
00:03 deploy:check:directories
00:03 deploy:check:linked_dirs
00:03 deploy:check:make_linked_dirs
00:05 git:clone
00:06 git:update
00:08 git:create_release
00:10 deploy:set_current_revision
00:10 deploy:symlink:linked_files
00:12 deploy:symlink:linked_dirs
00:18 deploy:config:bundler
00:20 bundler:install
00:20 deploy:assets:precompile
#<Thread:0x00007fb35ba959f0#/Users/Goulven/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/sshkit-1.21.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
/Users/Goulven/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/sshkit-1.21.0/lib/sshkit/command.rb:97:in `exit_status=': rake exit status: 1 (SSHKit::Command::Failed)
rake stdout: Nothing written
rake stderr: Nothing written
INFO [3def24f1] Running bundle exec rake assets:precompile as vtcontrol#ssh-vtcontrol.alwaysdata.net
DEBUG [3def24f1] Command: cd /home/www/app/releases/20200409174918 && ( export NODE_ENVIRONMENT="production" RAILS_ENV="production" RAILS_GROUPS="" ; bundle exec rake assets:precompile )
Running the last command locally or on the server does not produce any errors, only warnings:
# On production server:
$ cd /home/www/app/releases/20200409174918 && ( export NODE_ENVIRONMENT="production" RAILS_ENV="production" RAILS_GROUPS="" ; bundle exec rake assets:precompile )
yarn install v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents#1.2.12: The platform "linux" is incompatible with this module.
info "fsevents#1.2.12" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > webpack-dev-server#3.10.3" has unmet peer dependency "webpack#^4.0.0 || ^5.0.0".
warning "webpack-dev-server > webpack-dev-middleware#3.7.2" has unmet peer dependency "webpack#^4.0.0".
success Already up-to-date.
Done in 0.62s.
Hypothesis I've thought of:
Running out of RAM. Happens mainly with Digital Ocean, provider excludes that possibility, and anyway it would not compile when sshing to the server.
An issue with node vs nodejs. This used to nag me but it no longer happens, at first because I used sprockets only and added the mini_racer gem, which packages node for ExecJS, then because I included Bootstrap using webpacker instead of Sprockets, thus removing the dependency on autoprefixer-rails which errors out when an old version of nodejs is present alongside node.
Capistrano incorrectly interpreting yarn/webpacker warnings for failures. I extracted Bootstrap into its own app/javascripts/bootstrap.js because webpacker complained that the generated JS was too large, so this can be ruled out. Yarn complains about unmet peer dependencies but these are dev dependencies, I don't think it should matter. This might be the problem, but how can I test that?
Anything else I should try? Could using webpack and sprockets cause conflicts in production for instance? It works fine in development, and the generated assets should not overwrite one another.
Here are the relevant portions of my Gemfile:
# Gemfile
# Use SCSS for stylesheets
gem 'sass-rails', '>= 6'
# Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker
gem 'webpacker', '~> 4.0'
group :production do
# Used to load env vars from an .env file during deploy
gem 'dotenv-rails'
end
group :development do
gem 'capistrano', '~> 3.13.0', require: false
gem 'capistrano-rails', '~> 1.4', require: false
gem 'capistrano-bundler', '~> 1.6', require: false
gem 'capistrano-rails-console', require: false
end
And finally the relevant portion of my Capfile and deploy.rb:
# Capfile
require 'capistrano/bundler'
require 'capistrano/rails'
require 'capistrano/rails/assets'
require 'capistrano/rails/migrations'
require 'capistrano/rails/console'
require 'dotenv'
Dotenv.load
# config/deploy.rb
# Setup bundler
# Unset capistrano/bundler default flags because Bundler has deprecated them
set :bundle_flags, '--quiet'
set :bundle_path, nil
set :bundle_without, nil
before 'bundler:install', 'deploy:config:bundler'
# Remove gems no longer used to reduce disk space used
# This command requires loading capistrano/bundler in Capfile
after 'deploy:published', 'bundler:clean'
# Skip migration if files in db/migrate were not modified
# This command requires loading capistrano/rails in Capfile
set :conditionally_migrate, true
# Rails app server manages the database
set :migration_role, :app
# Defaults to nil (no asset cleanup is performed)
# If you use Rails 4+ and you'd like to clean up old assets after each deploy,
# set this to the number of versions to keep
set :keep_assets, 2
Thanks for your help!
Okay, after days of trying everything I could think of I tried something I'd already tested but with the correct syntax this time, I guess, because it works now.
Here is what I needed to add to Capistrano's config file to ensure Yarn was available when deploying:
# In deploy.rb
# Add or adjust default_env to append .npm-packages to $PATH:
set :default_env, {
PATH: '$HOME/.npm-packages/bin/:$PATH',
NODE_ENVIRONMENT: 'production'
}
Explanation: VPS users are allowed to install binaries like Yarn "globally" (using npm install --global yarn). Under-the-hood the binary is installed in $HOME/.npm-packages/bin, and this folder is added to the $PATH for interactive/login shell sessions. Since Capistrano does its utmost to NOT pick up this, we have to force-feed the updated $PATH in deploy.rb.
Thanks for pointing me in the right direction. I had the same error and it turned out to be a memory issue. For anyone with the same issues, check if you ran out of memory with free -m during compilation. If you ran out you'll see something like -bash: fork: Cannot allocate memory.
I'm using Digitalocean and my solution was to add swap space: https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04
On a GitLab installation from source, I'm having problems running binaries such as rake. I've never seen similar problems with my own Ruby or Rails projects:
# bundle config
Set for your local app (/mnt/data/www/gitlab.bitcetera.com/source/gitlab/.bundle/config): "1"
path
Set for your local app (/mnt/data/www/gitlab.bitcetera.com/source/gitlab/.bundle/config): "vendor/bundle"
without
Set for your local app (/mnt/data/www/gitlab.bitcetera.com/source/gitlab/.bundle/config): "development:test:unicorn:mysql:kerberos:metrics"
disable_shared_gems
Set for your local app (/mnt/data/www/gitlab.bitcetera.com/source/gitlab/.bundle/config): "true"
# bundle install --deployment --without development test unicorn mysql kerberos metrics
# bundle config
The Gemfile's dependencies are satisfied
# bundle exec rake --tasks
rake aborted!
LoadError: cannot load such file -- haml_lint/rake_task
Looking at the relevant section of the Gemfile:
group :development, :test do
[...]
gem 'haml_lint', '~> 0.21.0', require: false
[...]
end
Of course the haml_lint gem is missing, after all, it's in the "development" and "test" groups only which I've excluded by "without".
Any idea what's going on here?
Figured it out, kinda daft reason: RAILS_ENV should have been set automatically yet it wasn't so of course things go awry.
I was trying to run some test because of changes I did...so I ran RAILS_ENV=production bundle exec rspec
which was something very stupid to do since rspec (the tests) were configured to truncate all the tables and this was exactly what happened on PRODUCTION !! and you can imagine the consequences
Is there a way to configure rspec to never run when RAILS_ENV=production so this can never happen to any one.
What other advice or good practices can be applied to avoid this kind of mistakes
UPDATE: I created a ISSUE for the rspec-rails team and they just commited a change that fixes this problem https://github.com/rspec/rspec-rails/pull/1383/files
In your Gemfile, refer to anything test-specific (like rspec-rails or rspec) in the group for environments development and test only, instead of on the toplevel, e.g.:
group :development, :test do
gem 'rspec-rails', '~> 3.0'
end
Then bundling without the development and test gems on your production machine. Without the rspec gem the tests would not have run. Add the --without switch when deploying:
bundle install --without development test
In your spec_helper.rb, before any RAILS_ENV assignment:
raise "Not in test" unless ENV['RAILS_ENV'] == "test"
In my Rails 3.2 app I have a custom rake task that I am trying to run every day at 5pm with a cron job. Right now I am running it on our site's Staging server. The cron job is setup correctly but according to the e-mail output from the cron daemon the rake is being aborted because it's trying to invoke sqlite3.
Here is my cron job:
#crontab
0 17 * * * cd /u/apps/my_app/current && /usr/local/bin/rake my_task
I have reserved sqlite3 for development and test, like so:
#Gemfile
group :development, :test do
gem 'factory_girl_rails'
gem 'letter_opener'
gem 'rspec-rails'
gem 'sqlite3'
gem 'thin'
gem 'pry-rails'
end
I also have set my rake task to load the proper environment like so:
#mytask.rake
task :my_task => :environment do
# my task
end
This is the error I'm getting in the e-mail from crond:
rake aborted!
Please install the sqlite3 adapter: `gem install activerecord-sqlite3-adapter` (sqlite3 is not part of the bundle. Add it to Gemfile.)
If I run the rake tasks directly from the shell I don't get an error. It would seem that this works because of the following line in ~/.bashrc:
alias rake='RAILS_ENV=staging rake'
This, however, doesn't seem to have any effect on the task when run from the cron job. I've tried adding export RAILS_ENV=staging to the .bashrc file as recommended here but it didn't help.
The one thing I've found that works is writing the cron job like this:
#crontab
0 17 * * * cd /u/apps/my_app/current && /usr/local/bin/rake my_task RAILS_ENV=staging
...with the env declaration directly in the cron command. This doesn't seem very elegant but it's okay for now. Is there a better way to go about this?
Υou can set the environment variables in crontab (not in Arch or RedHat though...)
try setting your crontab as follows
#crontab
RAILS_ENV=staging
0 17 * * * cd /u/apps/my_app/current && /usr/local/bin/rake my_task
I set up Capistrano to make the deploy of my app. I made it in steps, so first I set up the code deployment, so I commented all the roles but :app.
I'm using rvm and I had some problems with it. The biggest problem was an error that said /usr/bin/env: ruby: No such file or directory. I solved them using the gem capistrano/rvm and requiring it in the Capfile and adding the following line to the deploy.rb file:
set :default_env, { path: "/usr/local/rvm/gems/ruby-2.0.0-p247#global/bin:$PATH" }
Once the code deploying runned correctly I activated de :db role in order to perform migrations. I get the same error but I can't find the solution this time.
~$ cap production deploy:migrate
…
…
INFO [85d6241d] Running bundle exec rake db:migrate on 10.10.51.10
DEBUG [85d6241d] Command: cd [PROJECT_SRC]/current
&& ( PATH=/usr/local/rvm/gems/ruby-2.0.0-p247#global/bin:$PATH RAILS_ENV=production bundle exec rake db:migrate )
DEBUG [85d6241d] /usr/bin/env: ruby: No such file or directory
cap aborted!
This command runs correctly when I execute it directly in the shell.
Thank you in advance!
Finally it worked using gem 'rvm1-capistrano3', require: false. It seems the best option for my setup (Rails4.0.1, ruby-2.0.0-p247, capistrano3). It didn't require special configuration.
You can find it here.
Hope it helps someone!
I had similar problem and this gem helped me:
https://github.com/wayneeseguin/rvm-capistrano
If you are not using rvm, then the issue is usually you'll have to manually install bundler gem on the server.
gem install bundler
Have you tried offical capistrano gems? That helped me, maybe your use case is similiar.
Gemfile:
...
gem 'capistrano', '~> 3.2.0'
gem 'capistrano-rvm'
gem 'capistrano-rails'
...
Capfile:
require 'capistrano/setup'
require 'capistrano/deploy'
require 'capistrano/rvm'
require 'capistrano/bundler'
require 'capistrano/rails/migrations'
...