I'm using the following code to perform a request on the server from within a rake task:
app = ActionDispatch::Integration::Session.new(Rails.application)
app.host!('localhost:3000')
app.get(path)
This works well.
However, if I call app.get(path) again with the same path, the request is not repeated and the previous result is returned.
Is there a way I can force app.get to repeat the call?
Try to reset the session:
app.reset!
Here is how it works when reset,
def reset!
#https = false
#controller = #request = #response = nil
#_mock_session = nil
#request_count = 0
#url_options = nil
self.host = DEFAULT_HOST
self.remote_addr = "127.0.0.1"
self.accept = "text/xml,application/xml,application/xhtml+xml," +
"text/html;q=0.9,text/plain;q=0.8,image/png," +
"*/*;q=0.5"
unless defined? #named_routes_configured
# the helpers are made protected by default--we make them public for
# easier access during testing and troubleshooting.
#named_routes_configured = true
end
end
Otherwise it will just re-use the last response:
# this is a private method in Session, it will called every time you call `get/post, etc`
def process
......
#request_count += 1
#request = ActionDispatch::Request.new(session.last_request.env)
response = _mock_session.last_response
#response = ActionDispatch::TestResponse.new(response.status, response.headers, response.body)
#html_document = nil
....
end
Good luck!
I've worked out what's going on.
Basically the observation "the request is not repeated" is Rails' own caching in action. Which makes sense, app.get is treated as any other request, if caching is enabled, the cache is returned, and if it's not, it will repeat (as #henrikhodne claimed). This explains why a puts in the cached controller will not output the second time.
To verify, add a puts in 2 controller methods, but only set the expires_in in the second. The first one will repeat the output, the second will not.
The way to force the request to repeat is to bust the cache by modifying the URL e.g.
app.get("/") becomes app.get("/?r=123456") as you would if using HTTP. It all seems obvious in hindsight, basically app.get is treated exactly as a client request, and all the same rules apply.
Related
I have a page that needs parameters received by a request from a third-party service. Unfortunately, the request takes a long time and the server crashes with a 504 error.
def show
start_time = Time.now
#project = Project.find(params[:id])
file = File.new(project.rvt_schema, 'rb')
rvt_params = ForgeHandler.instance.get_urn_token_params(file, "#{#project.id.to_s}.rvt")
#urn = rvt_params[:urn]
#token = rvt_params[:token]
end_time = Time.now
end
The most time inside the method is taken by request:
# Translate previously uploaded file to SVF format
def translate_to_svf(object_id,access_token)
base_64_urn = Base64.strict_encode64(object_id)
response = RestClient.post("#{API_URL}/modelderivative/v2/designdata/job",
{
input: {
urn: base_64_urn
},
output: {
formats: [
{
type: "svf",
views: [
"3d"
]
}
]
}
}.to_json,
{ Authorization: "Bearer #{access_token}", content_type:'application/json' })
return response
end
Which status is checked in cycle by another method:
def verify_job_complete(base_64_urn,access_token)
is_complete = false
while(!is_complete)
response = RestClient.get("#{API_URL}/modelderivative/v2/designdata/#{base_64_urn}/manifest",
{ Authorization: "Bearer #{access_token}"} )
json = JSON.parse(response.body)
if(json["progress"]=="complete")
is_complete = true
puts("***** Finished translating your file to SVF - status: #{json['status']}, progress: #{json['progress']} ")
else
puts("***** Haven't finished translating your file to SVF - status: #{json['status']}, progress: #{json['progress']} ")
sleep 5
end
end
I would like to implement asynchronous parameter loading. So I want to load data after losing control of the controller but but initializing the beginning of data loading from remote request in it. Tell me how best to implement this.
Or another way that would remove the error "Gateway timeout".
While this might be more of a question for the ruby-on-rails community, let me answer from the Autodesk Forge standpoint:
First of all, you should never wait for the Model Derivative job to complete when handling a request to your server. If the design file is complex enough, the translation could take up to hours, so this should definitely be handled asynchronously.
One option is to poll the status of the translation by requesting "derivative manifest" using the GET :urn/manifest endpoint.
Another option is to setup a Forge Webhook to get notified when the extraction.finished event is triggered.
It's probably easier to offload asynchronous stuff to a worker and save a reference to the user that needs to know about it. If you couple it with something like StimulusReflex you can render the result once it's finished. Another option might be the Render Async gem.
Let's say I have a method that provides access to an API client in the scope of a user and the API client will automatically update the users OAuth tokens when they expire.
class User < ActiveRecord::Base
def api
ApiClient.new access_token: oauth_access_token,
refresh_token: oauth_refresh_token,
on_oauth_refresh: -> (tokens) {
# This proc will be called by the API client when an
# OAuth refresh occurs
update_attributes({
oauth_access_token: tokens[:access_token],
oauth_refresh_token: tokens[:refresh_token]
})
}
end
end
If I consume this API within a Rails transaction and a refresh occurs and then an error occurs - I can't persist the new OAuth tokens (because the proc above is also treated as part of the transaction):
u = User.first
User.transaction {
local_info = Info.create!
# My tokens are expired so the client automatically
# refreshes them and calls the proc that updates them locally.
external_info = u.api.get_external_info(local_info.id)
# Now when I try to locally save the info returned by the API an exception
# occurs (for example due to validation). This rolls back the entire
# transaction (including the update of the user's new tokens.)
local_info.info = external_info
local_info.save!
}
I'm simplifying the example but basically the consuming of the API and the persistence of data returned by the API need to happen within a transaction. How can I ensure the update to the user's tokens gets committed even if the parent transaction fails.
Have you tried opening a new db connection inside new thread, and in this thread execute the update
u = User.first
User.transaction {
local_info = Info.create!
# My tokens are expired so the client automatically
# refreshes them and calls the proc that updates them locally.
external_info = u.api.get_external_info(local_info.id)
# Now when I try to locally save the info returned by the API an exception
# occurs (for example due to validation). This rolls back the entire
# transaction (including the update of the user's new tokens.)
local_info.info = external_info
local_info.save!
# Now open new thread
# In the new thread open new db connection, separate from the one already opened
# In the new connection execute update only for the tokens
# Close new connection and new thread
Thread.new do
ActiveRecord::Base.connection_pool.with_connection do |connection|
connection.execute("Your SQL statement that will update the user tokens")
end
end.join
}
I hope this helps
Nermin's (the accepted) answer is correct. Here's an update for Rails >= 5.0
Thread.new do
Rails.application.executor.wrap do
record.save
end
# Note: record probably won't be updated here yet since it's async
end
Documented here: Rails guides threading and concurrency
This discussion from a previous question might help you. It looks like you can set a requires_new: true flag and essentially mark the child transaction as a sub transaction.
User.transaction {
User.transaction(requires_new: true) {
u.update_attribute(:name, 'test')
};
u.update_attribute(:name, 'test2');
raise 'boom'
}
In my rails app controller I am posting to the api of the app on the same machine. I have build this out to handle the posting the data to the url:
url = "http://172.16.155.165:3000/api/jobs"
params = {
:input => "original/video.h264",
:output => "new/video.mp4",
:preset => 'h264'
}
jobResults = Net::HTTP.post_form(URI.parse(url), params)
This works great when I run this code through rails console but when I use it in my controller it gives me this error after loading for a minute or so:
Timeout::Error in SeminarsController#create
Timeout::Error
Once the timeout happens the data is actually posted and the api does what it should. It is like it is hanging until it times out then posts the data. The controller never goes beyond this step though. It should write the response body to a file with jobResults.body which would work fine if it didn't time out. If I write this into rails console it outputs the response immediately. The api will never take a whole minute to respond.
Am I doing something to cause this to happen? How can I make it work right?
edit:
This is the code for create in app/controllers/api/jobs_controller.rb:
def create
job = Job.from_api(params, :callback_url => lambda { |job| api_job_url(job) })
if job.valid?
response.headers["X-State-Changes-Location"] = api_state_changes_url(job)
response.headers["X-Notifications-Location"] = api_notifications_url(job)
respond_with job, :location => api_job_url(job) do |format|
format.html { redirect_to jobs_path }
end
else
respond_with job do |format|
format.html { #job = job; render "/jobs/new"}
end
end
end
Yes. Ideally you should remove the long running process (yes this is long running process) into background job. Remember that when many users start updating the videos, this process will show down for many reasons (like bandwidth, API acceptance rate etc). Rake::Timeout always pops out if the process passes the threshold. It is actually designed to abort requests that are taking too long to respond. And, it is not raised in console.
How can I make it work right?
Move it to the background job. Or you can explictly increase the rake timeout interval by doing something like this
# config/initializers/timeout.rb
Rack::Timeout.timeout = 30 # seconds
But i suggest not to do this. This rake-timeout helps in debugging. Mainly people use in heroku with newrelic.
I'm running an application that uses mechanize to fetch some data every so often from an RSS feed.
It runs as a heroku worker and after a day or so I'm receiving the following error:
Errno::EMFILE: Too many open files - socket(2)
I wasn't able to find a "close" method within mechanize, is there anything special I need to be doing in order to close out my browser sessions?
Here is how I create the browser + read information:
def mechanize_browser
#mechanize_browser ||= begin
agent = Mechanize.new
agent.redirect_ok = true
agent.request_headers = {
'Accept-Encoding' => "gzip,deflate,sdch",
'Accept-Language' => "en-US,en;q=0.8",
}
agent
end
end
And actually fetching information:
response = mechanize_browser.get(url)
And then closing after the response:
def close_mechanize_browser
#mechanize_browser = nil
end
Thanks in advance!
Since you manually can't close each instance of Mechanize, you can try invoking Mechanize as a block. According to the docs:
After the block executes, the instance is cleaned up. This includes closing all open connections.
So, rather than abstracting Mechanize.new into a custom function, try running Mechanize via the start class method, which should automatically close all your connections upon completion of the request:
Mechanize.start do |m|
m.get("http://example.com")
end
I ran into this same issue. The Mechanize start example by #zeantsoi is the answer that I ended up following, but there is also a Mechanize.shutdown method if you want to do this manually without their block.
There is also an option that you can add a lambda on post_connect_hooks
Mechanize.new.post_connect_looks << lambda {|agent, url, response, response_body| agent.shutdown }
I have a Rails application that has a Document with the flag available. The document is uploaded to an external server where it is not immediately available (takes time to propogate). What I'd like to do is poll the availability and update the model when available.
I'm looking for the most performant solution for this process (service does not offer callbacks):
Document is uploaded to app
app uploads to external server
app polls url (http://external.server.com/document.pdf) until available
app updates model Document.available = true
I'm stuck on 3. I'm already using sidekiq in my project. Is that an option, or should I use a completely different approach (cron job).
Documents will be uploaded all the time and so it seems relevant to first poll the database/redis to check for Documents which are not available.
See this answer: Making HTTP HEAD request with timeout in Ruby
Basically you set up a HEAD request for the known url and then asynchronously loop until you get a 200 back (with a 5 second delay between iterations, or whatever).
Do this from your controller after the document is uploaded:
Document.delay.poll_for_finished(#document.id)
And then in your document model:
def self.poll_for_finished(document_id)
document = Document.find(document_id)
# make sure the document exists and should be polled for
return unless document.continue_polling?
if document.remote_document_exists?
document.available = true
else
document.poll_attempts += 1 # assumes you care how many times you've checked, could be ignored.
Document.delay_for(5.seconds).poll_for_finished(document.id)
end
document.save
end
def continue_polling?
# this can be more or less sophisticated
return !document.available || document.poll_attempts < 5
end
def remote_document_exists?
Net::HTTP.start('http://external.server.com') do |http|
http.open_timeout = 2
http.read_timeout = 2
return "200" == http.head(document.path).code
end
end
This is still a blocking operation. Opening the Net::HTTP connection will block if the server you're trying to contact is slow or unresponsive. If you're worried about it use Typhoeus. See this answer for details: What is the preferred way of performing non blocking I/O in Ruby?