I am trying to replicate the setup Ryan Bates has in this railscast on Resque, where he queues up a third party service web request and then updates his results page with results.
I am designing an application that will interact with another Rails app, not a browser, and would like to replicate analogous behavior, with key difference being that only JSON output is expected
Currently I have something like this: (my models are Lists and Tasks, a List has_many Tasks and a Task belongs_to a List.
My lists_controller.rb
def show
Resque.enqueue(TaskDataFetcher,params[:id])
# confused if I need to have a render_to below this.
end
In task_data_fetcher.rb
require "net/http"
require "uri"
class TaskDataFetcher
#queue = :tasks_queue
def self.perform(id)
list = List.new(:id => id)
url = "taskservice.com/" + id + ".json"
uri = URI.parse(url)
response = Net::HTTP.get_response(uri)
task = Task.new(:contents => response.body)
task.list = list
# how to return this to the requesting server????
end
end
In the Railscast you see that result doesn't automatically update after the Resque task finishes, he has to reload the page several times, re-making the show request. So if you want to replicate this behaviour you could do something like:
def show
list = List.find(params[:id])
if list
respond_to do |format|
format.json {render :json => list.to_json}
end
else
Resque.enqueue(TaskDataFetcher, params[:id])
render :nothing => true, :status => 202
end
end
Requerement:
So your user is requesting your service to see some tasks. And you have to fetch those from another service taskservice.com. Then i think you have to do this through database for persistency.
Suggestion:
You can have a model like TaskRequest having attributes
`id` # must ;)
`task_list` # might be xml or whatever format suits you best
`is_received` # boolean
In your show method,
You create a TaskRequest entry and render a view which will show a loading type thing and will be requesting for task via ajax. The ajax response should return the task list and the is_received. However, once you get is_received true with a content you should request again.
In parallel, your TaskDataFetcher should receive two ids. One that you are sending now and another is of TaskRequest id. So after fetching the data from the service it will store that in the TaskRequest table and will update the is_recieve to true. Setting it true will eventually turn off requesting for this data anymore.
well the whole explanation might seem a bit hazy. Just let me know if you didnt any part or you need anything else specifically.
Note: It is something like the way SO shows the code formatting while answering a question ;)
Related
In my Rails app, I'm trying to take my working API calls and have them handled by background workers.
I have the following in app/jobs/api_request_job.rb:
class ApiRequestJob
def self.perform(params)
Query.new(params).start
end
end
The Query class is where the HTTParty requests are being executed (there are lots of methods for different query types with the same basic format as the parks method:
require 'ostruct'
class Query
include FourSquare
attr_reader :results,
:first_address,
:second_address,
:queries,
:radius
def initialize(params)
#results = OpenStruct.new
#queries = params["query"]
#first_address = params["first_address"]
#second_address = params["second_address"]
#radius = params["radius"].to_f
end
def start
queries.keys.each do |query|
results[query] = self.send(query)
end
results
end
def parks
category_id = "4bf58dd8d48988d163941735"
first_address_results = FourSquare.send_request(#first_address, radius_to_meters, category_id)["response"]["venues"]
second_address_results = FourSquare.send_request(#second_address, radius_to_meters, category_id)["response"]["venues"]
response = [first_address_results, second_address_results]
end
And, finally, the controller. Before trying to farm this action out to background workers, this line was working fine: #results = Query.new(params).start
class ComparisonsController < ApplicationController
attr_reader :first_address, :second_address
def new
end
def show
#first_address = Address.new(params["first_address"])
#second_address = Address.new(params["second_address"])
if #first_address.invalid?
flash[:notice] = #first_address.errors.full_messages
redirect_to :back
elsif Query.new(params).queries.nil?
flash[:notice] = "You must choose at least one criteria for your comparison."
redirect_to comparisons_new_path(request.params)
else
#queries = params["query"].keys
#results = Resque.enqueue(ApiRequestJob, params) # <-- this is where I'm stuck
end
end
end
I'm running redis, have resque installed, and am running the task/starting the workers. The current value being returned for #results is true instead of the hash of results I was need to get back. Is there a way to have the results of the Resque job persist and return data instead of true? What am I missing about how to get background workers to return the same type of data my regular api calls were returning?
Many thanks in advance!
The true you are receiving means the job was scheduled enqueued successfully. The worker will pick it up and run it on the background asynchronously, which means, not at same time as the thread that enqueued it. So there's no way to retrieve the returned value from the job.
If you need the value from that process, you have to run it from the controller without the worker. Also, you wouldn't gain anything from just pushing the work to be done by another process as the web process would have to wait for the response to then keep going anyway.
If you need that returned value right away and are doing this for performance reasons, then you could look into other forms of concurrency, like having another thread doing the request and then only grabbing the result when you need it on the view like:
class AsyncValue
def initialize(&block)
#thr = Thread.new(&block)
end
def value
#thr.join
end
end
on the controller
#results = AsyncValue.new { Query.new(params).start }
and on the view
<%= #results.value.each .... %>
but you'd still have to work around error handling which can get pretty complicated, but is doable.
Personally, I'd just make the request in place, but you know your domain better than me.
When using the Rails method, find_or_create_by, does it belong in the model or the controller? I am having a hard time understanding how to actually implement this method.
I want my rails application to accept JSON messages from users. The users will be sending data back to the server so it can be saved in the database. That being said, I would assume the user would have to use the 'POST' or 'PATCH method to store or update the data on my server. When I look at my routes the 'POST' method is used by the create action.
I have read the following Rails documentation but it didn't clarify anything to me. http://guides.rubyonrails.org/active_record_querying.html#find-or-create-by
Would I place the find_or_create_by method in my create action like so? Or does it belong somewhere else? It doesn't seem right in the create action...
class WifiNetworksController < ApplicationController
def create
#wifi_network = WifiNetwork.find_or_create_by(bssid: params[:bssid],
ssid: params[:ssid],
channel: params[:channel], etc...)
end
end
Ultimately I want:
Users to save new networks via JSON if it doesn't exist
Users to update existing networks via JSON if certain attributes have improved (like signal strength)
Thank you for your time!
Final Update - Thanks for the great advice everyone! I had to take a bit of everybody's advice to get it to work! Below is what I ended up doing.. Seems to work well with no errors.
def create
respond_to do |format|
if #wifi_network = WifiNetwork.find_by(bssid: wifi_network_params[:bssid])
# Logic for checking whether to update the record or not
#wifi_network.update_attributes(wifi_network_params) if #wifi_network.rssi < params[:rssi]
format.json { render :nothing => true }
else
# Must be a new wifi network, create it
#wifi_network = WifiNetwork.create(wifi_network_params)
format.json { render :nothing => true }
end
end
end
If you use strong params you can do this in your controller:
def create
#wifi_network = WifiNetwork.find_or_create_by(bssid: wifi_network_params[:bssid])
#wifi_network.update_attributes(wifi_network_params)
end
Then when a user makes a curl like:
curl -X POST localhost:3000/wifi_networks -d "wifi_network[bssid]=bssid1&wifi_network[ssid]=ssid1&wifi_network[channel]=channel1"
your create action will look up a WifiNetwork by it's bssid and update the ssid and channel attributes, or if it doesn't exist it will create a WifiNetwork with the bssid param and then update the newly created record with the rest of the atts. Be careful because if the wifi_network_params for the other attrs are empty they will update the params to nil.
I think it may be good to take a step back and really think about the interface of your application. Is there any particular reason why you need to use find_or_create_by and do everything in one controller action?
Why not simplify things and adhere to REST by having separate 'create' and 'update' actions on your WifiNetworksController:
class WifiNetworksController < ApplicationController
def create
#wifi_network = WifiNetwork.new(wifi_network_params)
if #wifi_network.save
# success response
else
# failure response
end
end
def update
# params[:id] won't work here if the client sending the request doesn't know the id of the
# wifi network, so replace it with the attribute you expect to be able to
# uniquely identify a WifiNetwork with.
if #wifi_network = WifiNetwork.find(params[:id])
# Logic for deciding whether to update or not
#wifi_network.update_attributes(wifi_network_params) if #wifi_network.signal_strength < params[:signal_strength]
else
# wifi_network not found, respond accordingly
end
end
private
# strong_parameters for Rails 4
def wifi_network_params
params.require(:wifi_network).permit(:ssid, :channel,...)
end
end
You could then have validations on your WifiNetwork model to ensure that certain attributes are unique, in order to avoid duplicates.
Or, if you really wanted to, you could combine both create and update into a single action, but create probably isn't the best name semantically.
EDIT: After your comment gave some background info, there probably isn't any benefit to using find_or_create_by, since you won't be able to tell if the record returned was 'created' or 'retrieved', which would allow you to avoid redundant update operations on it.
Assuming the bssid attribute is always a unique parameter:
def create
if #wifi_network = WifiNetwork.find(params[:bssid])
# Logic for checking whether to update the record or not
#wifi_network.update_attributes(wifi_network_params) if #wifi_network.signal_strength < params[:signal_strength]
else
# Must be a new wifi network, create it
#wifi_network = WifiNetwork.create(wifi_network_params)
end
end
Is there a way to pre-build a page cache without calling the actual page via a http request?
I looked at solutions like this and this, but these don't generate the cache.
I have a relatively complicated view, and want to cache the entire thing. I want to pre-build this cached version in the application so when a user actually hits it, it will already be there.
Thanks
We had a need to do something similar from a rake task -- we had a partial that would need to display a very long list of entities (~700) which were somewhat context specific and which, due to a series of database structure issues and custom sorting criteria, would easily take > 25 seconds to render the first time before going into cache> This would often time out because our HTTP servers were set to terminate HTTP requests with no response after 30 seconds, and pre-caching this custom list was a solution.
What you need to do is create an instance of ActiveController::Base, or of one of your controllers if you need helper methods or other entities, then pass its lookup_context reference to a new instance of ActionView.Renderer.
In our rake task, we did the following
namespace :rake_for_time_consuming_nonsense do
task :pre_cache_long_list do
PreCacher.pre_fetch_partials
end
end
class PreCacher
def self.pre_fetch_partials
the_controller = ActionController::Base.new
# Set any instance variables required by your partial in the controller,
# they will be passed to the partial with the view_context reference
the_controller.instance_variable_set "#cache_key", cache_key
the_controller.instance_variable_set "#the_object", MyModel.first
view_renderer = ActionView::Renderer.new the_controller.lookup_context
view_renderer.render the_controller.view_context, {partial: 'my_model/the_partial', layout: false}
end
end
This works in Rails 3.2.13.
I think following link should give you a good start.
How do I get the rendered output of a controller's action without visiting the web page?
I try to accomplish the same and as far i can see, your fake request should have the correct host, because the cache-key includes host informations.
I accomplished caching by using ActionController::Integration::Session
ais = ActionController::Integration::Session.new
ais.host = host
ais.xml_http_request(:post, url, params, headers)
I'v got another one:
class FakeRequest
include ActionController::UrlWriter
def initialize(url, params, session, host)
#url = url
#params = params
#session = session
default_url_options[:host] = URI.parse(host).host
end
def post
process(:post)
end
def get
process(:get)
end
def xhr
process(:post, true)
end
def process(method, ajax = false)
uri = URI.parse(url_for(#url))
request = ActionController::TestRequest.new({'HTTP_HOST' => uri.host,'rack.input' => '','rack.url_scheme' => 'http'})
request.query_parameters = #params
request.path = uri.path
request.host = uri.host
request.env['REQUEST_METHOD'] = method.to_s.upcase
if ajax
request.headers['X-Requested-With'] = 'XMLHttpRequest'
end
#session.each_pair do |k,v|
request.session[k] = v
end
response = ActionController::TestResponse.new
controller = ActionController::Routing::Routes.recognize(request).new
return controller.process(request, response)
end
end
This will also return the response object.
My rails app produces XML when I load /reports/generate_report.
On a separate page, I want to read this XML into a variable and save it to the database.
How can I do this? Can I somehow stream the response from the /reports/generate_report.xml URI into a variable? Or is there a better way to do it since the XML is produced by the same web app?
Here is my generate_report action:
class ReportsController < ApplicationController
def generate_report
respond_to do |format|
#products = Product.all
format.xml { render :layout => false }
end
end
end
Here is the action I am trying to write:
class AnotherController < ApplicationController
def archive_current
#output = # get XML output produced by /reports/generate_report
# save #output to the database
respond_to do |format|
format.html # inform the user of success or failure
end
end
end
Solved: My solution (thanks to Mladen Jablanović):
#output = render_to_string(:file => 'reports/generate_report.xml.builder')
I used the following code in a model class to accomplish the same task since render_to_string is (idiotically) a protected method of ActionController::Base:
av = ActionView::Base.new(Rails::Configuration.new.view_path)
#output = av.render(:file => "reports/generate_report.xml.builder")
Perhaps you could extract your XML rendering logic to a separate method within the same controller (probably a private one), which would render the XML to a string using render_to_string, and call it both from generate_report and archive_current actions.
What I typically do in this type of situation is to create a separate module/class/model to generate the report (it could even potentially be right in the Product model). This separate component could be in app/models or it could be in lib. In any case, once you have it extracted you can use it anywhere you need it. The controller can call it directly. You can generate it from the console. You can have a cron job generate it. This is not only more flexible, but it also can help smooth out your request response times if the report becomes slow to generate.
Since you are using a template it's understandable that the controller route is convenient, but even if you have to include some kind of ruby templating system in your auxiliary lib, it's still probably going to be less hassle and more flexible then trying to go through the controller.
#output = Product.all.to_xml
I'm sorry, is you question about Xml or about sessions? I mean is the fact that your action generates Xml material to the question? Or do you just want to save the output of the action for latter use?
You said on a "separate" page - you mean on another request? (like after user approved it?)
Why do you want to save the output? Because it should be saved exactly as rendered? (for example user can get frustrated if he clicked to save one report and you saved another)
Or is this thing expensive to generate?
Or may be, I got it wrong and it's about refactoring?
I am making an activeresource call to a service, and I'd like some custom error messages as feedback. I have some validations that aren't normal model validations, so I can't just return #object.errors.
So, for instance, one of my validations is this. Two objects have a many to many relationship, but I want to restrict one object to only have a limited number (say 2) of relationships to other objects. Here's some code:
In the client:
response = Customer.find(customer_id).put(:add_user, :user_id => user_id)
This puts a request to add a user to the customer. Then in the service I want to check that this addition is valid.
def add_user
#user = User.find(params[:user_id])
#customer = Customer.find(params[:id])
if #customer.users.length > 2
render :xml => "ERR_only_2_users_allowed", :status => :unprocessable_entity
end
end
Here's my problem. In active resource, if the return status is an error, the client side completely fails. I could change the status to 200 and I get back the body err msg fine, but this seems to defeat the purpose of having error reponse codes.
I can put the whole request call from the client in a begin/rescue block
begin
response = Customer.find(customer_id).put(:add_user, :user_id => user_id)
rescue ActiveResource::ResourceInvalid => e
#return error code
end
but when I catch the 422 (unprocessable_entity) response, I get nothing of the body back, so I don't get my custom error message. response = nil
Does anyone know how I can achieve these custom error message with the proper response codes?
This may or may not be your problem, but both of ours seem very close. I'm using a custom put method, but his should work for you too. What's going on is that the code that does this:
rescue ResourceInvalid => error
errors.from_xml(error.response.body)
end
Is only working with the standard save method. If you want errors added when other methods are called it looks like you need to do it yourself.
I had to add it to
vendor/rails/activeresource/lib/active_resource/custom_methods.rb
Here is what my diff from git looks like:
old code:
def put(method_name, options = {}, body = '')
connection.put(custom_method_element_url(method_name, options), body, self.class.headers)
end
new code:
def put(method_name, options = {}, body = '')
begin
connection.put(custom_method_element_url(method_name, options), body, self.class.headers)
rescue ResourceInvalid => error
errors.from_xml(error.response.body)
end
self
end
So look at the stack trace when get the exception thrown for the 422 and see which method it's calling exactly. Then add something like what I have and you should be good to go.
Don't ask me why the activeresource folks thought validations should only work with their save method. the save method does a create or update, but calling 'put or post' is the exact same thing, IMO. If we want validations to work on save we want them to work on put and post...anyway give it a shot.
I'm not sure if i need the self at the end...i may not. I'm not totally done with this as I just figured out how to make it work.
Erik
I think that your problem might be the response isn't an xml document but just a plain string. Try changing your render statement to something like:
render :xml => { :error => "ERR_only_2_users_allowed" }, :status => :unprocessable_entity