I'm making an application that needs to run a job at extremely precise intervals of time (say 30 seconds, maximum acceptable delay is +-1 second).
I'm currently doing so using an external Go application that polls an API endpoint built within my application.
Is there a way that I could run the task on a worker machine (eg a Heroku dyno) with delays less than one second?
I've investigated Sidekiq and delayed_job, but both have significant lag and therefore are unsuitable for my application.
Schedule the job for 60 seconds prior to when you need it run. Pass in the exact time you need the job executed, as a parameter. Then, run sleep until Time.now == exact_time_down_to_the_second?
Related
I want to run a long-running job on cloud run. this task may execute more than 30 minutes and it mostly sends out API requests.
cloud run stops executing after about 20 minutes and from the metrics, it looks like it did not identify that my task is still in the running state. so it probably thinks it is in idling and closing the container. I guess I can run calls to the server while job run to keep the container alive, but is there a way to signal from to container to cloud run that job is still active and not to close the container?
I can tell it is closing the container since the logs just stop. and then, the next call I make to the cloud run endpoint, I can see the "listening" log again from the NodeJS express.
I want to run a long-running job on cloud run.
This is a red herring.
On Cloud Run, there’s no guarantee that the same container will be used. It’s a best effort.
While you don’t process requests, your CPU will be throttled to nearly 0, so what you’re trying to do right now (running a background task and trying to keep container alive by sending it requests) is not a great idea. Most likely your app model is not fit a for Cloud Run, I recommend other compute products that would let you run long-running processes as well.
According to the documentation, Cloud Run will time out after 15 minutes, and that limit can't be increased. Therefore, Cloud Run is not a very good solution for long running tasks. If you have work that needs to run for a long amount of time, consider delegating the work to Compute Engine or some other product that doesn't have time limits.
Yes, You can use.You can create an timer that call your own api after 5 minutes, so no timeout after 15 minutes.Whenever timer executes it will create a dummy request on your server.
Other option you can increase request timeout of container to 1 hour from 5 min, if your backend request gets complete in 1 hour
Just wondering how long can a rails active job be delayed?
Currently I'm delaying some jobs for days and sometimes weeks, is that fine or is there a risk that it won't get executed in the end?
I'm currently using heroku and restarting my server every 6 hours, could that affect my jobs and sidekiq?
It depends which Active Job adapter you're using: some don't support delays at all, some have time limits, some will accept any value but get forgotten if the process restarts, and some (most) use persistent storage so that any delay is fine. Sidekiq is in the latter category: restarts will not cause problems (as long as your server is running -- Heroku stops hobby-size sites when they're inactive, and scheduled jobs don't count).
The only risk to be aware of is that jobs will be scheduled using the parameters you pass when you schedule them, and run using the code that's current when they come due: with longer delays, you need to be more careful about changing the number [or meaning] of parameters.
I should be fine, because it is you redis db that records/keeps the date when to execute the job.
I new to ROR. Wanted to ask something for confirmation. If I run long schedule job. Will it block others schedule job? I have others job running every 5 minutes, Plan to write something that easily run more than 3 hours. Will it block the 5 minutes job?
The whenever gem is basically only a way to configure and handle Cron jobs.
That said: At the given time Cron will just start and run a configured job. Cron will not block other jobs nor it cares if a job fails or if another job is still running.
Limiting factor might be:
Memory/CPU consumption: Each job consumes memory/CPU. If there are too many jobs running at the same time your server might run out of memory or might have a high load. But this doesn't really block other jobs it just slows down the whole server.
Database locks: If your jobs perform tasks that lock database tables other queries might be blocked and need to wait. But this is not Cron specific, this depends on what your code actually does.
We are using DelayedJob to run tasks in the background because they could take a while, and also because if an error is thrown, we still want the web request to succeed.
The issue is, sometimes the job could be really big (changing hundreds or thousands of database rows) and sometimes it could be really small (like 5 db rows). In the case of the small ones, we'd still like to have it run as a delayed job so that the error handling can work the same way, but we'd love to not have to wait roughly 5 seconds for DJ to pick up the job.
Is there a way to queue the job so it runs in the background, but then immediately run it so we don't have to wait for the worker to execute 5 seconds later?
Edit: Yes, this is Ruby on Rails :-)
Delayed Job polls the database for new dj records at a set interval. You can reconfigure this interval in an initializer:
# config/delayed_job.rb
Delayed::Worker.sleep_delay = 2 # or 1 if you're feeling wild.
This will affect DJ globally.
How about
SomeJob.set(
wait: 0,
queue: "queue_name",
).perform_later
I've got an app that could benefit from delayed_job and some background processing. The thing is, I don't really need/want delayed_job workers running all the time.
The app runs in a shared hosting environment and in multiple locations (for different users). Plus, the app doesn't get a large amount of usage.
Is there a way to start and stop processing jobs (either with the script or rake task) from my app only after certain actions/events?
You could call out to system:
system "cd #{Rails.root} && rake delayed_job:start RAILS_ENV=production"
You could just change delayed_job to check less often too. Instead of the 5 second default, set it to 15 minutes or something.
Yes, you can, but I'm not sure what the benefit will be. You say you don't want workers running all the time - what are your concerns? Memory usage? Database connections?
To keep the impact of delayed_job low on your system, I'd run only one worker, and configure it to sleep most of the time.
Delayed::Worker::sleep_delay = 60 * 5 # in your initializer.rb
A single worker will only wake up and check the db for new jobs every 5 minutes. Running this way keeps you from 'customizing' too much.
But if you really want to start a Delayed::Worker programatically, look in that class for work_off, and implement your own script/run_jobs_and_exit script. It should probably look much like script/delayed_job does - 3 lines.
I found this because I was looking for a way to run some background jobs without spending all the money to run them all the time when they weren't needed. Someone made a hack using google app engine to run the background jobs:
http://viatropos.com/blog/how-to-run-background-jobs-on-heroku-for-free/
It's a little outdated though. There is an interesting comment in the thread:
"When I need to send an e-mail, copy a file, etc I basically add it to the queue. At the end of every request it checks if there is anything in the queue. If so then it uses the Heroku API to set the worker to 1. At the end of a worker getting a task done it checks to see if there is anything left in the queue. If not then it sets the workers back to 0. The end result is the background worker will just work for a few seconds here and there. I can do all the background processing that I need and the bill at the end of the month rarely ever reaches 1 hour total worth of work. Even if it does no problem, I'll pay $0.05 for background processing. :)"
If you go to stop a worker, you are given the PID. You can simply kill -9 PID if all else fails.