I have following actions in a Test controller:
def run
while true do
# code
end
end
def stop
# stop the run action
end
how can stop action be implemented to halt the run action?
Because a client will wait for a response from a server, you can't have a loop in an endpoint that waits for another endpoint to be called.
In this case, a client will visit /test/run, but since the server won't return anything until the loop finishes, the client will just keep waiting.
This means (unless you specifically configured your webserver to do so), that another connection can't be made to the server to reach the test/stop endpoint.
If you must have a "job" that runs and be cancel-able by an endpoint, turn it into an actual background task.
Related
How can I send a message when the job finishes successfully? I would like to send the message and show it in a swal in javascript when the work finishes correctly, but I do not know how to do this, any suggestions?
I do not need to do anything other than send a message
class CompileProjectJob < Struct.new(:url)
def perform
end
def success(job)
#send message when the work is successful
end
end
At the end of perform method queue new delayed job for sending the message
class CompileProjectJob < Struct.new(:url)
def perform
# the code here of this job
# queue new job
end
end
the code of the perform method is executed sequentially as any regular code
Update
to send the message to the front end there are two ways (push and pull) more info
- push: using web sockets you push the message from the backend to the front end
- pull: the front end sends requests every certain period to check if the backend has a new data
and you can use any of these techniques to solve the problem
if you used pulling you will make the job update a data store as an example Redis or mysql. the front end will send a request every interval to check for the new data in some scenarios this will be a better solution but i think you are looking for the other technique
pushing:
here you can use something like active cable https://guides.rubyonrails.org/action_cable_overview.html
or a third party like pusher https://www.pusher.com/tutorials/realtime-table-ruby-rails
the main idea here your frontend app will open a websocket connection with your server. this socket will stay opened and listen for any updates from the backend through a channel so when you send the update after finishing the job through this channel it will be received by the front end so you can add code to show the message
I am using watir with headless browser. I would need to perform three steps add location, add vehicle and fetch product from the another site , for the information which I want from a another website.
I am submitting these three details from my server and performing these all three step in one HTTP request with the help of watir and headless.
I just want to breakdown one http request in to three http request on my server. The request will be:
1)add_location: Fire a http request which will open headless browser and select the location.
2)add_vehicle: Fire a http request which will reuse headless browser in which location added and we will select the vehicle.
3)Fetch product: Fire a http request which will reuse headless browser in which location and vehcile added, will fetch the product list.
I am not getting any way to reuse watir and headless session which is already open in the next http request at rails side.
Code Sample:
class TestsController < ApplicationController
def add_location
#headless = Headless.new
#headless.start
#watir = Watir::Browser.new
#watir.goto('www.google.com')
#watir.text_field(id: 'findstore-input')
.wait_until(&:present?).set(params[:zip_code])
#watir.a(id: 'findstore-button').click
#watir.div(class: 'notifier').wait_while(&:present?)
end
def add_vehicle
#need to resuse above #watir object in this action
end
end
The design change from 1 request to three has a big impact on your API, as even this simple part is now stateful, i.e. you need to keep the state between each of the three request.
Once you understand that, you have different possibilities.
Build your information request after request, and only when it is complete, use watir to get the information you need.
This is basically just changing the API and you store the data in a session, cookie, database or whatever.
It doesn't have a big impact on the changes you have to make, but does not bring any advantage.
Already forget this point, but you could pass around a global reference to your object in a session, but it has a HUGE memory impact and you could run into race condition.
NEVER do this, please
In case you really want to split the watir request into three different step (e.g. because it is too slow), you can use a background job to which you can transmit the user's data when it arrives (using dedicated databases, websocket, or whatever), then wait for your job to end (i.e. get a result), e.g. by trying to access it until it's available.
This solution requires a lot more work, but it keeps your HTTP requests with your client lightweight and allow you to do any kind of complex task in the background, which would otherwise probably timeout.
You can make use of the hooks file, to initiate the browser in headless mode and assign to the variable to call within separate def to pass url to the browser.
For example:
in hooks, you can add it as below
#browser = Watir::Browser.new :chrome, options: {args: ['--headless']}
So you can reuse the #browser.goto('www.google.com') in one def and can use the same instance some other call as well.
def example1:
#browser.goto('www.google.com')
end
def example2:
#browser.goto('www.facebook.com')
end
.
.
.
etc
Hope this helps.
I need to do request on remote service after rendering the page
My controller:
after_filter :remote_action, only: :update
def update
#res = MyService.do_action foo, bar
return render json: #res[:json], status: #res[:status] unless #res[:success]
end
def remote_action
# There is remote http request
end
I need to call remote_action method after rendering the page
after_filter is run after the template has been converted into html, but before that html is sent as a response to the client. So, if you're doing something slow like making a remote http request, then that will slow your response down, as it needs to wait for that remote request to finish: in other words, the remote request will block your response.
To avoid blocking, you could fork off a different thread: have a look at
https://github.com/tra/spawnling
Using this, you would just change your code to
def remote_action
Spawnling.new do
# There is remote http request
end
end
The remote call will still be triggered before the response is sent back, but because it's been forked off into a new thread, the response won't wait for the remote request to come back, it will just happen straight away.
You could also look at https://github.com/collectiveidea/delayed_job, which puts jobs into a database table, where a seperate process will pull them out and execute them.
I have a longer running task in the background, and how exactly would I let pull status from my background task or would it better somehow to communicate the task completion to my front end?
Background :
Basically my app uses third party service for processing data, so I want this external web service workload not to block all the incoming requests to my website, so I put this call inside a background job (I use sidekiq). And so when this task is done, I was thinking of sending a webhook to a certain controller which will notify the front end that the task is complete.
How can I do this? Is there a better solution for this?
Update:
My app is hosted on heroku
Update II:
I've done some research on the topic and I found out that I can create a seperate app on heroku which will handle this, found this example :
https://github.com/heroku-examples/ruby-websockets-chat-demo
This long running task will be run per user, on a website with a lot of traffic, is this a good idea?
I would implement this using a pub/sub system such as Faye or Pusher. The idea behind this is that you would publish the status of your long running job to a channel, which would then cause all subscribers of that channel to be notified of the status change.
For example, within your job runner you could notify Faye of a status change with something like:
client = Faye::Client.new('http://localhost:9292/')
client.publish('/jobstatus', {id: jobid, status: 'in_progress'})
And then in your front end you can subscribe to that channel using javascript:
var client = new Faye.Client('http://localhost:9292/');
client.subscribe('/jobstatus', function(message) {
alert('the status of job #' + message.jobid + ' changed to ' + message.status);
});
Using a pub/sub system in this way allows you to scale your realtime page events separately from your main app - you could run Faye on another server. You could also go for a hosted (and paid) solution like Pusher, and let them take care of scaling your infrastructure.
It's also worth mentioning that Faye uses the bayeaux protocol, which means it will utilise websockets where it is available, and long-polling where it is not.
We have this pattern and use two different approaches. In both cases background jobs are run with Resque, but you could likely do something similar with DelayedJob or Sidekiq.
Polling
In the polling approach, we have a javascript object on the page that sets a timeout for polling with a URL passed to it from the rails HTML view.
This causes an Ajax ("script") call to the provided URL, which means Rails looks for the JS template. So we use that to respond with state and fire an event for the object to response to when available or not.
This is somewhat complicated and I wouldn't recommend it at this point.
Sockets
The better solution we found was to use WebSockets (with shims). In our case we use PubNub but there are numerous services to handle this. That keeps the polling/open-connection off your web server and is much more cost effective than running the servers needed to handle these connection.
You've stated you are looking for front-end solutions and you can handle all the front-end with PubNub's client JavaScript library.
Here's a rough idea of how we notify PubNub from the backend.
class BackgroundJob
#queue = :some_queue
def perform
// Do some action
end
def after_perform
publish some_state, client_channel
end
private
def publish some_state, client_channel
Pubnub.new(
publish_key: Settings.pubnub.publish_key,
subscribe_key: Settings.pubnub.subscribe_key,
secret_key: Settings.pubnub.secret_key
).publish(
channel: client_channel,
message: some_state.to_json,
http_sync: true
)
end
end
The simplest approach that I can think of is that you set a flag in your DB when the task is complete, and your front-end (view) sends an ajax request periodically to check the flag state in db. In case the flag is set, you take appropriate action in the view. Below are code samples:
Since you suggested that this long running task needs to run per user, so let's add a boolean to users table - task_complete. When you add the job to sidekiq, you can unset the flag:
# Sidekiq worker: app/workers/task.rb
class Task
include Sidekiq::Worker
def perform(user_id)
user = User.find(user_id)
# Long running task code here, which executes per user
user.task_complete = true
user.save!
end
end
# When adding the task to sidekiq queue
user = User.find(params[:id])
# flag would have been set to true by previous execution
# In case it is false, it means sidekiq already has a job entry. We don't need to add it again
if user.task_complete?
Task.perform_async(user.id)
user.task_complete = false
user.save!
end
In the view you can periodically check whether the flag was set using ajax requests:
<script type="text/javascript">
var complete = false;
(function worker() {
$.ajax({
url: 'task/status/<%= #user.id %>',
success: function(data) {
// update the view based on ajax request response in case you need to
},
complete: function() {
// Schedule the next request when the current one's complete, and in case the global variable 'complete' is set to true, we don't need to fire this ajax request again - task is complete.
if(!complete) {
setTimeout(worker, 5000); //in miliseconds
}
}
});
})();
</script>
# status action which returns the status of task
# GET /task/status/:id
def status
#user = User.find(params[:id])
end
# status.js.erb - add view logic based on what you want to achieve, given whether the task is complete or not
<% if #user.task_complete? %>
$('#success').show();
complete = true;
<% else %>
$('#processing').show();
<% end %>
You can set the timeout based on what the average execution time of your task is. Let's say your task takes 10 minutes on average, so their's no point in checking it at a 5sec frequency.
Also in case your task execution frequency is something complex (and not 1 per day), you may want to add a timestamp task_completed_at and base your logic on a combination of the flag and timestamp.
As for this part:
"This long running task will be run per user, on a website with a lot of traffic, is this a good idea?"
I don't see a problem with this approach, though architectural changes like executing jobs (sidekiq workers) on separate hardware will help. These are lightweight ajax calls, and some intelligence built into your javascript (like the global complete flag) will avoid the unnecessary requests. In case you have huge traffic, and DB reads/writes are a concern then you may want to store that flag directly into redis instead (since you already have it for sidekiq). I believe that will resolve your read/write concerns, and I don't see that it is going to cause problems. This is the simplest and cleanest approach I can think of, though you can try achieving the same via websockets, which are supported by most modern browsers (though can cause problems in older versions).
I have an extremely long operation (~30s on average) that happens in a request cycle. It is not a huge deal because it is ran very rarely and only through an administrative portal, otherwise I would push it to Resque or whatever.
When the request starts and the object is not flagged, the first thing that happens in the controller method is to set a flag:
def create
raise if #foo.flag?
#foo.update_attributes! flag: true
# perform lengthy operation
ensure
#foo.update_attributes! flag: false
end
When I start the first request, it sets flag to true and begins the operation. Then when I start the second request while the first is hanging, it never raises an error even though I can see clearly in the Rails console that flag is true for #foo.
I was under the impression that update_attributes! commits the database transaction, and so subsequent requests should see the changes. Is this not the case?
I am just testing in development with the thin server. Is there any way I can do this?