I'm setting up a multi tenant Rails application with Postgresql schemas.
How can I scope the db search path for Delayed::Job?
This would work:
initializers/dj_config.rb:
Delayed::Job.class_eval do
connection.schema_search_path = ["#{current_tenant}", "public"].join(",")
end
...but I need a way to pass in the current tenant, which seems hard since the DJ worker is a different process than the one where "current_tenant" is set. Any ideas?
I had a similar problem and ended up creating custom job classes with perform methods that set the current_tenant. I simply passed the current_tenant into the constructor:
Delayed::Job.enqueue CustomJob.new(current_tenant)
Related
In our app, we need to switch to read replica database and read from it for some read-only APIs.
We decided to use the around_action filter for that:
Switch DB to read_replica before the action
Yield
Switching back to master.
We decided to use establish_connection for switching, which did the job but later we noticed that it's not thread-safe i.e it causes our other threads to face "#<ActiveRecord::ConnectionNotEstablished: No connection pool with 'primary' found.>" issue. So this solution would have worked in the case of single-threaded servers.
Later we tried to create a new connection pool, as below which is thread-safe:
databases = Rails.configuration.database_configuration
resolver = ActiveRecord::ConnectionAdapters::ConnectionSpecification::Resolver.new(databases)
spec = resolver.spec(:read_replica)
pool = ActiveRecord::ConnectionAdapters::ConnectionPool.new(spec)
pool.with_connection { |conn|
execute SQL query here.
}
The only problem with the above approach is, we can only execute queries using execute method like conn.execute(sql_query) any AR ORM query we execute inside this with_connection block run on the original DB and not read_replica.
Seems like ActiveRecord do have its default connection and it's using it when we run AR ORM queries.
Not sure how can we execute the AR ORM query inside the with_connection block as User.where(id: 1..10).
Please note:
I am aware that we can do this natively in rails 6, need to skip that for now.
I am also aware of the Octopus gem, again need to skip on that.
Appreciate any help, Thanks.
So I've been looking for the simplest way to send an e-mail when X column of Payments table in the database is == 'condition'. Basically what I want is to add a payment and set a date like 6 months. When 6 months have passed I want to send the mail. I've seen many solutions like using Whenever cron jobs and others but I want to know the absolute simplest way (perhaps using Rails only without relying on outside source) to keep my application light and clean. I was thinking I could use the auto generated created_at to evaluate when x time has passed.
Since you have a column in your db for the time to send email, make it a datetime datatype and you can set the email date as soon as the event payment event is created. Then, you can have a rake task where,
range = Time.now.beginning_of_day..Time.now.end_of_day
Payment.where(your_datetime_custom_column: range).each do |payment|
payment.user.send_email
end
and you can run this task everyday from the scheduler.
The "easiest" way is to use Active Job in conjunction with a state machine:
EmailJob.set(wait: 6.months).perform_later(user.id) if user.X_changed?
The problem with this is that the queue will accumulate jobs since jobs don't get handled right away. This may lead to other performance issues since there are now more jobs to scan and they're taking up more memory.
Cron jobs are well suited for this kind of thing. Depending on your hosting platform, there may be various other ways to handle this; for example, Heroku has Heroku Scheduler.
There are likely other ways to schedule repeating tasks without cron, such as this SO answer.
edit: I did use a gem once called 'fist_of_fury', but it's not currently maintained and I'm not sure how it would perform in a production environment. Below are some snippets for how I used it in a rails project:
in Gemfile
gem 'fist_of_fury'
in config/initializers/fist_of_fury.rb
# Ensure the jobs run only in a web server.
if defined?(Rails::Server)
FistOfFury.attack! do
ObserveAllJob.recurs { minutely(1) }
end
end
in app/jobs/observe_all_job.rb
class ObserveAllJob
include SuckerPunch::Job
include FistOfFury::Recurrent
def perform
::Task.all.each(&:observe)
end
end
I have a case scenario where I need to run multiple record updates in the background(using resque) and I want to give user visual indicator of how the task is running(eg started/running/finished).
One way of achieving this(which I can think of) is saving the current state into a table, then showing the state to user by simple page refresh.
Can anyone suggest a better solution of doing it?I want to avoid creating the whole migration, model, controller for this.
Thanks
As I've commented, resque-status gem could be useful for you. I am not sure if that is an answer but since you said that you do not want to create migration, model and controller for this. Thus, a gem might be the way to go.
From the job id you can get the status you are looking for, for example:
status = Resque::Plugins::Status::Hash.get(job_id)
status.working? #=> true
There is also a front-end called resque-web, check that out too.
You may use ruby's global variable $var_name = 'foo'. However I am not sure about it, because they are considered bad practice in rails, but in this case I see them reasonable, as soon as their name is very unique.
It can be done like (in case of resque):
class UpdateJob
#queue = data
def self.perform
$my_job_name_is_running = true
MyJobName.new.run
$my_job_name_is_running = nil
end
end
then you can access them from anywhere in the app:
while $my_job_name_is_running
puts "job is running..." if $my_job_name_is_running
sleep 3 # important to not overload your processor
end
Ruby global vars are not very popular. Check docs for more info https://ruby-doc.org/docs/ruby-doc-bundle/UsersGuide/rg/globalvars.html
current my application have stat needs and I
make up a background job using rufus-scheduler and runs at 3:00
to batch process these records into CacheStat table. It's just like
any normal application's Weekly/Monthly Stat needs.
And I found out using find_each(say using User.find_each to iterate
all users), which invokes find_in_batches, I checkout the source code
of rails,
while records.any?
records_size = records.size
primary_key_offset = records.last.id
yield records
break if records_size < batch_size
if primary_key_offset
records = relation.where(table[primary_key].gt(primary_key_offset)).to_a
else
raise "Primary key not included in the custom select clause"
end
end
which the implentation is by comparing the primary-key,
my concern is the cocurrency,while I processing the batch,
whatif some records be inserted in-between?
does anybody have this kind of problem?
While I think, this code implementation may be be problemic,
because new records will always have larger PK and later in the
end will be find.
So this is what this kind of needs be implemented? If I want to
implement a batch stat processing by myself(without rails), then I
need to ensure have an integer primary key and using these fields to
compare(better not to use other kind of fields)?
(I was thinking of this because I'm kind of in the middle of switching
from mysql to mongo, so maybe later I need to implement this kind of
functionality by myself).
If I understand correctly, you can ensure correctness here by enforcing transactional isolation, e.g.
User.transaction do
User.find_each do |user|
user
end
end
I have a Rails app that is multi homed.
foo.mysite.com talks to the "foo" db.
bar.mysite.com talks to the "bar" db.
This is accomplished by calling:
ActiveRecord::Base.connection_handler.establish_connection("ActiveRecord::Base", foo_spec)
When requests come in for foo it uses the foo_spec, when requests come in for bar it uses the bar_spec.
Everything is happy and there is peace in the world.
However,
I also use sidekiq, it is heavily multi-threaded.
I was getting weird behavior in sidekiq. Often when I thought I was talking to the foo_db, ActiveRecord::Base.connection was pointed at bar_db.
I dug into the code and found:
def retrieve_connection_pool(klass)
pool = #class_to_pool[klass.name]
return pool if pool
return nil if ActiveRecord::Base == klass
retrieve_connection_pool klass.superclass
end
Turns out the internal design of AR only allows AR::Base to know about a single connection pool.
Is there any way to get thread 1 to talk to db1, and thread 2 to talk to a db2 at the same time, using ActiveRecord::Base.connection ?
I would recommend using Postgres and separate schemas rather than separate databases entirely; that was you can share pools.
Usage would look like: select * from foo.users, select * from bar.users
And you would pass the schema to your background worker as an argument.