I have a shared resource that can only be used by one session at a time, how do I signal to other sessions that the resource is currently in use?
In Java or C I would use a mutex semaphore to coordinate between threads, how can I accomplish that in Rails? Do I define a new environment variable and use it to coordinate between sessions?
A little code snippet along with the answer would be very helpful.
Since your Rails instances can be run in different processes when using Nginx or Apache (no shared memory like in threads), I guess the only solution is using file locks:
lock = File.new("/lock/file")
begin
lock.flock(File::LOCK_EX)
# do your logic here, or share information in your lock file
ensure
lock.flock(File::LOCK_UN)
end
I would consider using Redis for locking the resource.
https://redis.io/topics/distlock
https://github.com/antirez/redlock-rb
This has the advantage of working across multiple servers and not limiting the lock time to live to the lifetime of the current HTTP request.
Ruby has a Mutex class that might do what you want, though it won't work across processes. I apologize though that I don't know enough to give you an example code snippet. Here's what the documentation says: "Mutex implements a simple semaphore that can be used to coordinate access to shared data from multiple concurrent threads."
You can do this with acts_as_lockable_by gem.
Imagine the shared resource is a Patient ActiveRecord class that can only be accessed by a single user (you can replace this with session_id) as follows:
class Patient < ApplicationRecord
acts_as_lockable_by :id, ttl: 30.seconds
end
Then you can do this in your controller:
class PatientsController < ApplicationController
def edit
if patient.lock(current_user.id)
# It will be locked for 30 seconds for the current user
# You will need to renew the lock by calling /patients/:id/renew_lock
else
# Could not lock the patient record which means it is already locked by another user
end
end
def renew_lock
if patient.renew_lock(current_user.id)
# lock renewed return 200
else
# could not renew the lock, it might be already released
end
end
private
def patient
#patient ||= Patient.find(params[:id])
end
end
This is a solution that works with minimum code and across a cluster of RoR machines/servers not just locally on one server (like using file locks) as the gem uses redis as locks/semaphores broker. The lock, unlock and renew_lock methods are all atomic and thread-safe ;)
Related
Currently I am building an application that allows users to place bids on products and admins to approve them. The 'transactions' themselves take place outside of the scope of the application. Currently, users see the price of an asset on the transaction/new page and submit a bid by submitting the form. Admins click a button to approve the bid.
class TransactionsController < ApplicationController
before_action :get_price
def new
#price = get_price
#tansaction = Transaction.new
end
###
def get_price
#price = <<Some External Query>>
end
def approve
t = Transaction.find(params[:id])
t.status = "Approved"
t.update!
end
Obviously this is not ideal. I don't want to query the API every time a user wants to submit a bid. Ideally, I could query this API every 5-10 seconds in the background and use the price in that manner. I have looked at a couple of techniques for running background jobs including delayed_job, sidekiq, and resque. For example in sidekiq I could run something like this:
#app/workers/price_worker.rb
class PriceWorker
include Sidekiq::Worker
def perform(*args)
get_price
end
def get_price
#price = <<Some External Query>>
end
end
#config/initializers/sidekiq.rb
schedule_file = "config/schedule.yml"
if File.exists?(schedule_file) && Sidekiq.server?
Sidekiq::Cron::Job.load_from_hash YAML.load_file(schedule_file)
end
#config/schedule.yml
my_price_job:
cron: "*/10 * * * * * "
class: "PriceWorker"
That code runs. The problem is I am kind of lost on how to handle the price variable and pass it back to the user from the worker. I have watched the Railscasts episodes for both sidekiq and resque. I have written background workers and jobs that queue and run properly, but I cannot figure out how to implement them into my application. This is the first time I have dealt with background jobs so I have a lot to learn. I have spent sometime researching this issue and it seems like more background jobs are used for longer running tasks like updating db indexes rather than constantly recurring jobs (like an API request every 5 seconds).
So to sum up, What is the proper technique for running a constantly recurring task such as querying an external API in Rails? Any feedback on how to do this properly will be greatly appreciated! Thank you.
That is not how background jobs work. You're right, you have a lot of reading up to do. Think of running an asynchronous job processor like Sidekiq as running an entirely separate app. It shares the same code base as your Rails app but it runs completely separately. If you want these two separate apps to talk to each other then you have to design and write that code.
For example, I would define a cache with reader and writer methods, then have the cache populated when necessary:
someone loads product "foo" for the first time on your site
Rails checks the cache and finds it empty
Rails calls the external service
Rails saves the external service response to the cache using its writer method
Rails returns the cached response to the client
The cache would be populated thereafter by Sidekiq:
someone loads product "foo" for the second time on your site
Rails checks the cache and finds the cached value from above
Rails fires a Sidekiq job telling it to refresh the cache
Rails returns the cached response to the client
Continuing from step 3 above:
Sidekiq checks to see when the cache was last refreshed, and if it was more than x seconds ago then continue, else quit
Sidekiq calls the external service
Sidekiq saves the external service response to the cache using its writer method
When the next client loads product "foo", Rails will read the cache that was updated (or not updated) by Sidekiq.
With this type of system, the cache must be an external store of some kind like a relational database (MySQL, Postgres, Sqlite) or a NoSQL database (Redis, memcache). You cannot use the internal Rails cache because the Rails cache exists only within the memory space of the Rails app, and is not readable by Sidekiq. (because Sidekiq runs as a totally separate app)
I guess in this case you should use rails cache. Put something like this in your controller:
#price = Rails.cache.fetch('price') do
<<Some external query>>
end
you can also configure cache expiration date, by setting expires_in argument, see https://apidock.com/rails/ActiveSupport/Cache/Store/fetch for more information.
Regarding using background jobs to update your "price" value, you would need to store retrieved data anyways (use some kind of database) and fetch it in your controller.
I'm transforming my application (single client) in to a multi-tenant application.
I used to store a few settings (rarely changed / very frequently accessed) into a global variable (a hash).
The values for this global variable were pulled out of the DB when the application started.
Obviously, these settings are specific to a client / tenant.
I now see a few options, but none of them seems good:
keep the global hash, but introduce the notion of tenant $global[:client1][:settingX] but this doesn't feel good/scalable/safe
Call the DB everytime, but I fear to take a performance hit (~40 additional calls to the DB )
Is there anything I could do with Memcache? But I don't know where to start with option.
Note: the application is hosted on Heroku
This sounds like a great argument for using memcached or redis. You want a centralized place where these values can live that's accessible to all of your pool of app servers. Your use case is predominantly read with occasional write to update values.
I would build a currency helper that lazy-loads from memcached/redis once per request and then saves the currency rate data for any value calculations in that request.
I don't know your problem domain, but let's assume you're trying to provide local currency pricing to different users in your system and that the rates are different for different users (ie: discounting, etc).
class CurrencyHelper
attr_reader :currency_rates
def initialize(user_id)
#currency_rates = load_or_generate_exchange_rates
end
def load_or_generate_exchange_rates
key = "/currency/rates/#{user_id}"
REDIS.get(key) || begin
rates = generate_exchange_rates
REDIS.put(key, rates)
rates
end
end
def convert_from_usd_to(amount_usd, currency)
round_money( usd * currency_rates[currency] )
end
end
In your controller code:
def currency_helper
#currency_helper ||= CurrencyHelper.new(current_user.id)
end
def show
localized_price = currency_helper.convert_from_usd_to(price_usd, params[:country_code])
end
As Thin/Unicorn are single threaded, how do you handle Thread.current/per-request storage?
Just ran a simple test - set a key in one session, read it from another -- looks like it writes/reads from the same place all the time. Doesn't happen on WEBrick though.
class TestController < ApplicationController
def get
render text: Thread.current[:xxx].inspect
end
def set
Thread.current[:xxx] = 1
render text: "SET to #{Thread.current[:xxx]}"
end
end
Tried adding config.threadsafe! to application.rb, no change.
What's the right way to store per-request data?
How come there are gems (including Rails itself, and tilt) that use Thread.current for storage? How do they overcome this problem?
Could it be that Thread.current is safe per request, but just doesn't clear after request and I need to do that myself?
Tested with Rails 3.2.9
Update
To sum up the discussion below with #skalee and #JesseWolgamott and my findings--
Thread.current depends on the server the app is running on. Though the server might make sure no two requests run at the same time on same Thread.current, the values in this hash might not get cleared between requests, so in case of usage - initial value must be set to override last value.
There are some well known gems who use Thread.current, like Rails, tilt and draper. I guess that if it was forbidden or not safe they wouldn't use it. It also seems like they all set a value before using any key on the hash (and even set it back to the original value after the request has ended).
But overall, Thread.current is not the best practice for per-request storage. For most cases, better design will do, but for some cases, use of env can help. It is available in controllers, but also in middleware, and can be injected to any place in the app.
Update 2 - it seems that as for now, draper is uses Thread.current incorrectly. See https://github.com/drapergem/draper/issues/390
Update 3 - that draper bug was fixed.
You generally want to store stuff in session. And if you want something really short-living, see Rails' flash. It's cleared on each request. Any method which relies on thread will not work consistently on different webservers.
Another option would be to modify env hash:
env['some_number'] = 5
BTW Unicorn is not simply single-threaded, it's forking. The new process is spawned on each request (despite it sounds scary, it's pretty efficient on Linux). So if you set anything in Unicorn, even to global variable, it won't persist to another request.
While people still caution against using Thread.current to store "thread global" data, the possibly correct approach to do it in Rails is by clearing-up the Thread.current object using Rack middleware. Steve Labnik has written the request_store gem to do this easily. The source code of the gem is really, really small and I'd recommend reading it.
The interesting parts are reproduced below.
module RequestStore
def self.store
Thread.current[:request_store] ||= {}
end
def self.clear!
Thread.current[:request_store] = {}
end
end
module RequestStore
class Middleware
def initialize(app)
#app = app
end
def call(env)
RequestStore.clear!
#app.call(env)
end
end
end
Please note, clearing up the entire Thread.current is not a good practice. What request_store is basically doing, is it's keeping track of the keys that your app stashes into Thread.current, and clears it once the request is completed.
One of the caveats of using Thread.current, is that for servers that reuse threads or have thread-pools, it becomes very important to clean up after each request.
That's exactly what the request_store gem provides, a simple API akin to Thread.current which takes care of cleaning up the store data after each request.
RequestStore[:items] = []
Be aware though, the gem uses Thread.current to save the Store, so it won't work properly in a multi-threaded environment where you have more than one thread per request.
To circumvent this problem, I have implemented a store that can be shared between threads for the same request. It's called request_store_rails, and the usage is very similar:
RequestLocals[:items] = []
I'm using the Ruby Money gem in a multi-tenant (SaaS) Rails app, and am looking for a good way to make the Money.default_currency be set to an Account's preference for each request. I have several currency-related models in the app that use the Money class.
I have everything working properly in development, but I'm just looking for some feedback on whether or not the solution with have repercussions in production.
Here's what I did in my ApplicationController (irrelevant code removed for brevity):
class ApplicationController < ActionController::Base
before_filter :set_currency
private
def set_currency
Money.default_currency = Money::Currency.new(current_account.present? && current_account.currency.present? ?
current_account.currency : 'USD')
end
end
So the code above will set the default_currency class variable to the current account's preference, or default back to 'USD' if there isn't one.
By the way, here's the relevant default_currency code in the Money class:
class Money
# Class Methods
class << self
# The default currency, which is used when +Money.new+ is called without an
# explicit currency argument. The default value is Currency.new("USD"). The
# value must be a valid +Money::Currency+ instance.
#
# #return [Money::Currency]
attr_accessor :default_currency
end
end
So, will this work as expected in a multi-user setting? Anything else I need to do?
Most rails apps don't run in multithreaded mode - a given instance is only ever handling one request at a time (this is the default).
If your app was in multithreaded mode this would be dangerous - Money.default_currency could get changed halfway through a request by the new request that has just come in. If you did want to make this thread safe, you could use the Thread.current hash to have per thread values of default_currency
I have my Application Controller called McController which extends ApplicationController, and i set a class variable in McController called ##scheduler_map like below:
class McController < ApplicationController
##scheduler_map = {}
def action
...
end
private
def get_scheduler(host, port)
scheduler = ##scheduler_map[host+"_"+port]
unless scheduler
scheduler = Scheduler.create(host, port)
##scheduler_map[host+"_"+port] = scheduler
end
scheduler
end
end
but i found that from second request start on ##scheduler_map is always an empty hash, i run it in development env, could someone know the reason? is that related to the running env?
Thank you in advance.
You answered your own question :-)
Yes this is caused by the development environment (i tested it) and to be more precise the config option "config.cache_classes = false" in config/environments/development.rb
This flag will cause all classes to be reloaded at every request.
This is done because then you dont have to restart the whole server when you make a small change to your controllers.
You might want to take in consideration that what you are trying can cause HUGE memory leaks when later run in production with a lot of visits.
Every time a user visits your site it will create a new entree in that hash and never gets cleaned.
Imagine what will happen if 10.000 users have visited your site? or what about 1.000.000?
All this data is kept in the systems memory so this can take a lot of space the longer the server is online.
Also, i'm not really sure this solution will work on a production server.
The server will create multiple threats to handle a lot of visitors on the same time.
I think (not sure) that every threat will have his own instances of the classes.
This means that in treat 1 the schedule map for ip xx might exist but in treat 2 it doesn't.
If you give me some more information about what this scheduler is i might be able to give a suggestion for a different solution.