I'm trying to solidify my understanding of what blocking means in terms of requests to a web server and when it's smart to hand off requests to a separate worker (i.e. sidekiq).
Consider the following examples:
Login with Facebook
def sign_in
response = Faraday.get("https://graph.facebook.com/me?fields=id,email&access_token=#{some_token}")
user_info = JSON.parse(response.body)
#user = User.find_by(uid: user_info["id"])
...
end
Send push notification through Google Firebase
def send_push_notification
...
fcm = FCM.new(FCM_KEY)
registration_ids = [recipient_token]
resp = fcm.send(registration_ids, data: {body: notification_params[:body]})
...
end
In both examples, the web requests to a 3rd-party service are synchronous and possibly costly. Intuitively, I would try to handle these cases with a separate worker because they block the main application. But, I am not 100% sure what blocking means. Does it mean that when there are 100 users trying to sign_in and each Faraday.get call takes 1 second, it will take 100 seconds for all the users to sign in?
Does it mean that when there are 100 users trying to sign_in and each
Faraday.get call takes 1 second, it will take 100 seconds for all the
users to sign in?
Simplistic answer: yes.
In a very simple scenario, the 1st user will wait 1 second, the 2nd user will wait 2 seconds and so on.
If your application/web server doesn't abort the user request, the 100th user will wait for 100 seconds.
A bit more detailed: depends.
Today, modern web applications (like Puma) have more than 1 process worker running in your machine. This means that your application is able to handle more than 1 request concurrently.
For example: if you have Puma configured to use 2 workers, your application will handle the requests of 2 users at same time.
Thus, the 1st and 2nd users will wait 1 second, the 3rd and 4th users will wait 2 seconds and the 99th and 100th users will wait 50 seconds.
As each Puma process consumes a considerable amount of CPU and memory, you cannot have infinite workers. That's why is very interesting have a background process to send these push notifications.
In Sidekiq (for example) the cost to delegate a job to a worker is extremely slow and thus the users of your website won't be penalized.
Related
I'm currently trying to optimize and scale an API built on Ruby on Rails behind an AWS ALB that sends traffic to NGINX and then into Puma to our Rails application. Our API has a timeout option of 30 seconds maximum which is when we eventually timeout the request. Currently we have a controller action that queues a Sidekiq worker and then we poll a Redis key every 100ms for the first 1 second and then move to polling every 500ms for the remaining 29 seconds. Many of our requests can be completed in under 1 second, but some of them will take the full 30 seconds before they succeed or timeout, telling the user to retry in a little while.
We're currently trying to load test this API and scale it to 500-1000 RPS and we're running into problems where the slower requests will block up all of our connections. When a slow request is running shouldn't Puma be able to take other requests in during the sleep period of the slow requests?
If this was not an API we could easily just immediately respond after we queue the background worker, but in this case we need to wait for the response and hold the connection for up to 30 seconds for the API request.
My first thought is that you can have multiple redis queues and push specific tasks to certain queues.
If you have a queue for the quick tasks and a queue for the slower tasks, then both can run in parallel without the slow tasks holding everything else up.
I have a situation where I have a worker that makes multiple calls to an external API. The problem is that we have a threshold of many calls we can make to this API per hour.
What I'd like to do is to create a worker which will make these many sequential calls to this external API. If in between these calls we get an error because we've reached the number of connections we're allowed in that hour, the worker would then save the document and schedule a new worker to complete the remaining API calls at a later time (maybe 1, 2 hours later. Ideally this should be configurable e.g.: 10mins, 1hour, etc).
Is there someway I could achieve this?
With SideKiq you can scheduled when a job will be executed with a friendly API :
MyWorker.perform_in(3.hours, 'mike', 1) # Expect a duration
MyWorker.perform_at(3.hours.from_now, 'mike', 1) # Expect a date
Check it out : Scheduled Jobs
You want Sidekiq Enterprise and its Rate Limiting API. The alternative is tracking the rate limit yourself and rescheduling the job manually.
https://github.com/mperham/sidekiq/wiki/Ent-Rate-Limiting
I am building a Rails 4 (postgres) app on the back of a third party API. For now, the third party API allows with 100 requests per min.
The roundtrip for the user takes about 2000 ms so I want to move this into a worker.
I considered using sidekiq, but with each new user and new background thread comes the possibility that I'll exceed my API quota.
What is the best way to control my applications interaction with the third party API? Do I need a single serial queue to control the rate limit effectively?
I assume you'll get an error (like exception) when you are over the 100 requests limit. If all API requests are in a sidekiq worker, the worker will automatically retry on error. Initially the retry will be quite soon, but you can overwrite the retry time with something like:
sidekiq_retry_in do
rand(60..75)
end
In this way each retry will be 60 to 75 seconds after the error.
You can check more about sidekiq's error handling here: https://github.com/mperham/sidekiq/wiki/Error-Handling
I am working on creating a Rails web application with background workers for performing some of the tasks in the background on a set interval. I am using Resque with Redis for queuing the background jobs, and using Resque-scheduler to run it on a set interval for ex., every 30 seconds or so.
The background job needs to be enqueued only when a user visits a particular page, and it should run on a schedule until the user moves away from that page. Basically, I would like to set the schedule dynamically during runtime. My application is deployed in Cloud, and the main Rails app and the background workers run as a separate processes communicating through Redis. How can I set the schedule dynamically, and from where?
resque_schedule.yml
do_my_job:
every: 1m
class: BackgroundJob
description: Runs the perform method in MyJob
queue: backgroundq
inside the controller
def index
Resque.enqueue(BackgroundJob)
end
background_job.rb
class BackgroundJob
#queue = :backgroundq
#job_helper = BackgroundHelper.new
def self.perform
#job_helper.get_job_data
end
end
Resque-scheduler can do what you're describing using dynamic schedules. Every time a user visits your page, you create a schedule with Resque.set_schedule that runs every 30 seconds on that user. When the user leaves, you abort with Resque.remove_schedule. I can imagine a lot of ways the user might leave the page without you noticing (worst case, what if their computer loses power?), so I'd be worried about the schedules being left around. I'm also not sure how happy resque-scheduler remains as you add more and more schedules, so you might run into trouble with lots of users.
You could minimize the number of schedules by having one job that runs every 30 seconds over every user you currently need running, like:
class BackgroundJob
def self.perform
User.active.each { ... }
end
end
If you do something like set User#last_seen_at when the user visits your page and have User.active load everyone seen in the past 10 minutes, then you'll be running every 30 seconds on all your active users, and there's no chance of a user's jobs lasting well after they leave, because it times out in 10 minutes. However, if you have more users than you can do work for in 30 seconds, the scheduled job won't be finished before its next copy starts up, and there will be trouble. You might be able to get around that by having the top-level job enqueue for each user a secondary job that actually does the work. Then as long as you can do that in 30 seconds and have enough resque workers for your job volume, things should be fine.
A third way to approach it is to have the user's job enqueue another copy of itself, like so:
class BackgroundJob
def self.perform(user_id)
do_work(user_id)
Resque.enqueue_in(30.seconds, BackgroundJob, user_id)
end
end
You enqueue the first job when the user visits your page, and then it keeps itself going. This way is nice because every user's job run independently, and you don't need either a separate schedule for each or a top-level schedule managing everything. You'll still need a way to stop the jobs running once the user is gone, perhaps with a timeout or TTL counter.
Finally, I'd question whether Resque is the right tool for your task. If you want something happening every 30 seconds as long as a user is on a page, presumably you want the user seeing it happen, so you should consider implementing it in the browser. For example, you could set up a 30 second JavaScript interval to hit an API endpoint that does some work and returns the value. This avoids any need for Resque at all, and will automatically stop when the user navigates away, even if they lose power and your code doesn't get a chance to clean up.
I have a backend Rails server with Sidekiq, which serves as API server. The app works as follow:
My Rails server receives many requests from incoming API clients at the same time.
For each of these requests, the Rails server will allocate jobs to a Sidekiq server. Sidekiq server makes requests to external APIs (such as Facebook) to get data, and analyze it and return a result to Rails server.
For example, if I receive 10 incoming requests from my API clients, for each request, I need to make 10 requests to external API servers, get data and process it.
My challenge is to make my app responds to incoming requests concurrently. That is, for each incoming request, my app should process in parallel: make calls to external APIs, get data and return result.
Now, I know that Puma can add concurrency to Rails app, while Sidekiq is multi-threaded.
My question is: Do I really need Sidekiq if I already have Puma? What would be the benefit of using both Puma and Sidekiq?
In particular, with Puma, I just invoke my external API calls, data processing etc. from my Rails app, and they will automatically be concurrent.
Yes, you probably do want to use Puma and Sidekiq. There are really two issues at play here.
Concurrency (as it seems you already know) is the number of web requests that can be handled simultaneously. Using an app server like Puma or Unicorn will definitely help you get better concurrency than the default web brick server.
The other issue at play is the length of time that it takes your server to process a web request.
The reason that these two things are related is that number or requests per second that your app can process is a function of both the average processing time for each request and the number of worker processes that are accepting requests. Say your average response time is 100ms. Then a single web worker can process 10 requests per second. If you have 5 workers, then you can handle 50 requests per second. If your average response time is 500ms, then you can handle 2 reqs/sec with a single worker, and 10 reqs/sec with 5 workers.
Interacting with external APIs can be slow at times, and in the worst cases it can be very unreliable with unresponsive servers on the remote end, or network outages or slowdowns. Sidekiq is a great way to insulate your application (and your end users) from the possibility that the remote API is responding slowly. Imagine that the remote API is running slowly for some reason and that the average response time from it has slowed down to 2 seconds per request. In that case you'd only be able to handle 2.5 reqs/sec with 5 workers. With anymore traffic than that your end users might start to have a long wait time before any page on your app could respond, even those that don't make remote API calls, because all of your web workers might be waiting for the slow remote API to respond. As traffic continues to increase your users would start getting connection timeouts.
The idea with using Sidekiq is that you separate the time spent waiting on the external API from your web workers. You'd basically take the request for data from your user, pass it to Sidekiq, and then immediately return a response to the user that basically says "we're processing your request". Sidekiq can then pick up the job and make the external request. After it has the data it can save that data back into your application. Then you can use web sockets to push a notification to the user that the data is ready. Or even push the data directly to them and update the page accordingly. (You could also use polling to have the page continually asking "is it ready yet?", but that gets very inefficient very quickly.)
I hope this makes sense. Let me know if you have any questions.
Sidekiq, like Resque and Delayed Job, is designed to provide asynchronous job processing from a queue.
If you don't need jobs to be queued up and run asynchronously, there's no substantial benefit (or harm) to using Sidekiq.
If the tasks need to run synchronously (which it sounds like you might—it's not clear if clients are waiting for data or just requesting that jobs run), Sidekiq and its relatives are likely the wrong tool for the job. There is no guaranteed processing time when using Sidekiq or other solutions; jobs are pushed onto the end of the stack, however long that may be, and won't be processed until their turn comes up. If clients are waiting for data, they may time out long before your worker pool ever processes their jobs.