I'm using mongoid alongside postgresql in a rails 5 app.
My resque jobs work perfectly with the mongoid models, however, when I try to use one of my postgresql tables inside a job, I get the following error:
PG::UndefinedTable: ERROR: relation "admins" does not exist LINE 1: SELECT "admins".* FROM "admins" ^ : SELECT "admins".* FROM "admins"
This is my lib/tasks/resque.rake file
require 'resque/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
Resque.before_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
Resque.after_fork do
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
end
The mentioned postgres table does exist, and works perfectly with the rails app. It seems like, at least outside of the main rails app, ActiveRecord defaults to using mongoid, so none of my postgresql models are visible inside the worker. Or maybe not.
Am I missing something?
I ended up fixing it by tweaking my Procfile and task file a bit, the issue was due to resque not picking up the right environment (hence the undefined model relationships)
Procfile:
web: bundle exec puma -t 5:5 -p ${PORT:-3000} -e ${RACK_ENV:-development}
worker: bundle exec rake resque:pool
I ended up using lib/tasks/resque.rake just to declare tasks, and moved the initialisation code to lib/tasks/resque-pool.rake. Basically resque:setup gets called before resque:pool:setup, to it preloads the rails environment for the pool manager.
require 'resque/tasks'
require 'resque/pool/tasks'
task "resque:setup" => :environment do
Resque.before_fork = Proc.new do
ActiveRecord::Base.connection.disconnect!
end
Resque.after_fork = Proc.new do
ActiveRecord::Base.establish_connection
end
end
task "resque:pool:setup" do
end
Related
If you are using multiple databases (e.g. to shard tables across databases) for a given environment is it possible to use the built in Rails rake tasks to manipulate the databases beyond the primary one for said environment?
e.g. if I use specific connection details for a set of models, can I use rake db:create to create said database?
If so, how?
I've actually overwritten the built-in Rails rake tasks in order to manipulate a second database. The neat thing is that it totally works with the original rake tasks as well.
This is what I did... credit to Nikolay Sturm, as I based my rake task off of his blog.
I added new database settings in my database.yml, as detailed in the blog I linked.
You also create a base model that has these database settings:
class BaseModel < ActiveRecord::Base
establish_connection "database_connection_name"
self.abstract_class = true #you want to make this an abstract class because you don't have a table for this
end
You then have any model that connects to this secondary database inherit from your BaseModel.
In your rake task, you set it up as follows (keep in mind that this is for a Postgres connection.)
namespace :db do
desc 'create database'
task create: :environment do
connection_config = BaseModel.connection_config #rename BaseModel to name of your model
ActiveRecord::Base.establish_connection connection_config.merge('database => 'postgres') #you don't need this bit if you're not using Postgres
ActiveRecord::Base.connection.create_database connection_config[:database], connection_config
ActiveRecord::Base.establish_connection connection_config
end
desc 'migrate database'
task migrate: :environment do
ActiveRecord::Base.establish_connection BaseModel.connection_config
ActiveRecord::Migrator.migrate('db/migrate/')
end
desc 'drop database'
task drop: :environment do
connection_config = BaseModel.connection_config
ActiveRecord::Base.establish_connection connection_config.merge('database' => 'postgres')
ActiveRecord::Base.connection.drop_database connection_config[:database]
end
Now, if you do a rake -T in your project, you should be able to see the your descriptions for the rake tasks instead of the default. When you run them, it will run the db:create, db:migrate, and db:drop for both your default and your secondary database. Hope this helps!
I want to use Heroku's scheduler to reset my database once every day.
It's recommended to use rake tasks for the scheduler. This is what I've tried:
task :reset_database => :environment do
`heroku pg:reset MY_DB:URL`
`heroku run rake db:migrate db:seed`
# some other ruby commands
end
But how would I do this correctly, because putting the heroku commands within backticks, which with bash normally works, doesn't work here:
No such file or directory - heroku
Try this rake task:
namespace :reset_database do
desc "Destroy all table entries."
task :all => :environment do
ActiveRecord::Base.connection.tables.each do |table|
if table != 'schema_migrations'
table.singularize.camelize.constantize.destroy_all
end
# Use this if you want to use the normal seeds:
# Rails.application.load_seed
# Use this if you want to run another rake task:
Rake::Task["foo:bar"].invoke
end
end
end
I have a problem when I do:
namespace :xaaron do
task :get_roles do
roles = Xaaron::Role.all
puts roles
end
task :get_role, [:name] do |t, args|
role = Xaaron::Role.find(args[:name].parameterize)
puts role
end
end
The first task will work fine. I can even add binding.pry and run Xaaron::Role and get information about Roles back. But the second task fails with:
NameError: uninitialized constant Xaaron::Role
I run each task in my main app because these tasks are inside an engine, using:
bin/rake xaaron:get_roles` and `bin/rake xaaron:get_role
I can run bin/rails c in the main application that uses the engine and run Xaaron::Role and get information about Roles table.
Why is the second one failing but the first one is not? Is there scoping with arguments?
I'm not sure why either works, but if this is Rails and those are Rails models, your tasks should depend on the environment:
task :get_roles => [ :environment ] do
By depending on the :environment task, it first loads Rails.
Also see: What's the 'environment' task in Rake?.
You can also run a Rake task as
bundle exec rake environment xaaron:get_role
This will load the Rails environment first.
I kept getting uninitialized constant errors for a Rake task, even after depending on :environment and running with bundle exec.
The issue was that I was making a Rake::TestTask and, even though the Rake task had access to all constants, the test files themselves did not have access to constants.
The solution was to add this line to the top of my test file:
require_relative '../config/environment'
This is the Rake task:
require "rake/testtask"
Rake::TestTask.new(:test) do |t|
t.libs << "test"
t.libs << "lib"
t.test_files = FileList["test/**/test_*.rb"]
end
To add, as of Ruby 1.9 and above, you can use this hash syntax:
namespace :xaaron do
desc "Rake task to get roles"
task get_roles: :environment do
roles = Xaaron::Role.all
puts roles
end
#####
end
And then you can run the command below to run the Rake task:
rake xaaron:get_roles
or
bundle exec rake xaaron:get_roles
I'am using Capistrano for deployment and Sidekiq for queues. I need to load user ids to set :sidekiq_queues variable in deploy.rb file to perform sidekiq queues. Now I use following code
set :sidekiq_queues, ::User.all.map {|u| "-q parsing_user_#{u.id}"}.join(" ") + " -q parsing_user_0"
but it throws following error
./config/deploy.rb:29:in `load': uninitialized constant User (NameError)
I tried to require 'rubygems' and 'active_record' into deploy.rb, but it didn't help.
In result I should have
sidekiq_queues == "-q parsing_user_1 -q parsing_user_2 -q parsing_user_3 -q parsing_user_4 -q parsing_user_5 -q parsing_user_0".
Hardcoding queue names isn't a solution.
Capistrano executes deploy.rb locally, and it doesn't load your Rails environment in deploy.rb.
It might be more trouble than it is worth. Especially if you want to execute this on your remote server, you might consider doing a Rake task instead. The => :environment in the rake task ensures that your Rails environment is loaded.
# in deploy.rb
namespace :sidekiq do
desc "Do something with queues"
task :queues, :roles => :web do
run "cd #{current_path} ; RAILS_ENV=#{rails_env} bundle exec rake sidekiq:queues"
end
end
# you'll need to decide when to execute this in your deployment process,
# something like this:
after "deploy:update_code", "sidekiq:queues"
# in lib/tasks/sidekiq.rake
namespace :sidekiq do
desc "Do something with queues"
task :queues => :environment do
queues = (User.scoped.pluck(:id) + [0]).map{|id| "-q parsing_user_#{id}"}.join(" ")
# do what you need to do
end
end
The resque jobs I have do not depend on anything in Rails, but I'm having a hard time starting workers without the rails env. I've seen this post, but it didn't help (ruby resque without loading rails environment)
Here is my current rake file:
require "resque/tasks"
task "resque:setup" do
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
#task "resque:setup" => :environment
The commented task would load the Rails env and everything works, but that's not what I want. When running rake resque:work I get this error:
rake aborted!
No such file to load -- application_controller
Tasks: TOP => resque:work => resque:preload
If you've only added a lib/tasks/resque.rake file and haven't modified your Rakefile, you'll still be loading your Rails environment when you call rake resque:work. Try this for Rakefile:
unless ENV['RESQUE_WORKER'] == 'true'
require File.expand_path('../config/application', __FILE__)
My::Application.load_tasks
else
ROOT_PATH = File.expand_path("..", __FILE__)
load File.join(ROOT_PATH, 'lib/tasks/resque.rake')
end
And then this for your resque.rake file:
require "resque/tasks"
task "resque:setup" do
raise "Please set your RESQUE_WORKER variable to true" unless ENV['RESQUE_WORKER'] == "true"
root_path = "#{File.dirname(__FILE__)}/../.."
require "#{root_path}/app/workers/myworker.rb"
end
Then call rake resque:work RESQUE_WORKER=true
I referred the link here It worked perfectly for me:
This error was resolved, by running
$> QUEUE=* rake environment resque:work
a cleaner solution was to define a rake task:
task "resque:setup" => :environment do
ENV['QUEUE'] ||= '*'
#for redistogo on heroku http://stackoverflow.com/questions/2611747/rails-resque-workers-fail-with-pgerror-server-closed-the-connection-unexpectedl
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
and now
rake resque:work
Worked perfectly
Thanks.