With the app I'm building, we're using trumpia to schedule and send sms, and email messages to our users. When they sign up, I'm using an observer to trigger the call to trumpia, and I'm using faraday to actually connect with the trumpia API. My problem is that these calls are taking a lot of time, and I need to put them into the background. I also have a smaller set of commands that I need to run immediately, so I can't just delay the whole after_create.
I'm thinking of using delayed_job to accomplish this, but I have no idea how to do it with something as convoluted as my code is. How can I do this? I think I need to put the Faraday calls into a method, but I don't know how. -- Thanks
Here's my observer. I've left out the code that isn't necessary.
class UserObserver < ActiveRecord::Observer
def after_create(user)
#This first part should not be delayed
#user = user
kevin = User.find(1)
user.increment!(:renewcash, by = 5)
body = "Hi " + #user.name.split(' ', 2).first + ", ..."
kevin.send_message(#user, body, "First message", sanitize_text=true, attachment=nil)
# This whole second part needs to be delayed.
conn = Faraday.new(:url => 'http://api.trumpia.com') do |faraday|
faraday.request :multipart
faraday.response :logger
faraday.adapter Faraday.default_adapter
end
trumpia_body = {:list_name => ... .... }.to_json
response = conn.put do |req|
req.url '/rest/v1/XXXXXXXXX/subscription'
req.headers['X-Apikey'] = 'XXXXXXXXXXXXX'
req.headers['Content-Type'] = 'application/json'
req.body = trumpia_body
end
response2 = conn.get do |req2|
req2.url '/rest/v1/XXXXXXXX/subscription?row_size=1'
req2.headers['X-Apikey'] = 'XXXXXXXXXXX'
req2.headers['Content-Type'] = 'application/json'
end
end
I'm using rails 3.2.8. But I haven't install delayed job yet, and I'm open to a different queing gem.
Related
I have an interactor foo with method perform on it which makes an API call, and records the response through ActiveRecord. It works fine. I have a spec for it that triggers the method with FactoryBot data, then checks the db for the expected records. This also works fine. However, when I wrap the call in a VCR.use_cassette, the call is made (and resulting cassette created), but the db records don't seem to be written. What am I missing?
spec looks like this:
it 'should do a thing' do
bar = FactoryGirl.create(:bar)
VCR.use_cassette('foo/cassette') do
MyInteractor.perform(bar)
end
record = BarRecord.find_by(bar_id: bar.id)
expect(record.status.to_sym).to be(:success)
end
perform method looks roughly like this:
def perform(bar)
uri = URI.parse("<url>")
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth Settings.username, Settings.password
req['Accept'] = 'Application/json'
req.body = post_params.to_json
https = Net::HTTP.new(uri.host, uri.port)
https.use_ssl = true
res = https.request(req)
record = BarRecord.new(bar_id: bar.id)
record.status = JSON.parse(res.body)["status"]
record.save!
record
end
BarRecord should be created by the call to perform. Without VCR it is.
For the curious, there was in fact nothing wrong with VCR. I was storing items with a fixed timestamp, and then reading them with a dynamic one that didn't overlap. Full pebkac.
So, i've generated a rails app using https://github.com/Shopify/shopify_app - and for the most part have the app working as intended - it's goal is to get product quantities from an external stock management API, and then update the variant quantities in Shopify with the latest quantities from that stock management system.
My problem is that the initial POST request to the external API responds with a large number of products - this takes upwards of 15 seconds sometimes. In addition to this, another portion of my app then takes this response, and for every product in the response that also exists in Shopify, it will make a PUT request to Shopify to update the variant quantities. As with the initial request, this also takes upwards of 10-15 seconds.
My problem is that i'm hosting the app on Heroku, and as a result i've hit their 30 second request timeout limit. As a result I need to use a background worker to offset at least one of the requests above (perhaps both) to a worker queue. I've gone with the widely recommended Sidekiq gem - https://github.com/mperham/sidekiq - which is easy enough to set up.
My problem is that I don't know how to get the results from the finished Sidekiq worker job, and then use that again within the Controller - I also don't know if this is best practice (i'm a little new to Rails/App development).
I've included my controller (prior to breaking it down into workers) that currently runs the app below - I guess I just need some advice - am I doing this correctly - should some of this logic be inside a Model, and if so how would that model then communicate with the Controller, and then how would Sidekiq then fit into all of it.
Appreciate any advice or assistance, thanks.
class StockManagementController < ShopifyApp::AuthenticatedController
require 'uri'
require 'net/http'
require 'json'
require 'nokogiri'
require 'open-uri'
require 'rexml/document'
def new
#token = StockManagementController.new
end
def get_token
url = URI('https://external.api.endpoint/api/v1/AuthToken')
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
#HEROKU_ENV_USERNAME = ENV['HEROKU_ENV_USERNAME']
#HEROKU_ENV_PASSWORD = ENV['HEROKU_ENV_PASSWORD']
request = Net::HTTP::Post.new(url)
request['content-type'] = 'application/x-www-form-urlencoded'
request['cache-control'] = 'no-cache'
request.body = 'username=' + #HEROKU_ENV_USERNAME + '&password=' + #HEROKU_ENV_PASSWORD + '&grant_type=password'
response = http.request(request)
responseJSON = JSON.parse(response.read_body)
session[:accessToken] = responseJSON['access_token']
if session[:accessToken]
flash[:notice] = 'StockManagement token generation was successful.'
redirect_to '/StockManagement/product_quantity'
else
flash[:alert] = 'StockManagement token generation was unsuccessful.'
end
end
def product_quantity
REXML::Document.entity_expansion_text_limit = 1_000_000
#theToken = session[:accessToken]
if #theToken
url = URI('https://external.api.endpoint/api/v1/ProductQuantity')
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
request['authorization'] = 'bearer ' + #theToken + ''
request['content-type'] = 'application/xml'
request['cache-control'] = 'no-cache'
response = http.request(request)
responseBody = response.read_body
finalResponse = Hash.from_xml(responseBody).to_json
resultQuantity = JSON.parse finalResponse
#connectionType = resultQuantity['AutomatorResponse']['Type']
#successResponse = resultQuantity['AutomatorResponse']['Success']
#errorResponse = resultQuantity['AutomatorResponse']['ErrorMsg']
productQuantityResponse = resultQuantity['AutomatorResponse']['ResponseString']
xmlResponse = Hash.from_xml(productQuantityResponse).to_json
jsonResponse = JSON.parse xmlResponse
#fullResponse = jsonResponse['StockManagement']['Company']['InventoryQuantitiesByLocation']['InventoryQuantity']
# This hash is used to store the final list of items that we need in order to display the item's we've synced, and to show the number of items we've sycned successfully.
#finalList = Hash.new
# This array is used to contain the available products - this is used later on as a way of only rendering
#availableProducts = Array.new
# Here we get all of the variant data from Shopify.
#variants = ShopifyAPI::Variant.find(:all, params: {})
# For each peace of variant data, we push all of the available SKUs in the store to the #availableProducts Array for use later
#variants.each do |variant|
#availableProducts << variant.sku
end
#Our final list of products which will contain details from both the Stock Management company and Shopify - we will use this list to run api calls against each item
#finalProductList = Array.new
puts "Final product list has #{#fullResponse.length} items."
puts #fullResponse.inspect
# We look through every item in the response from Company
#fullResponse.each_with_index do |p, index|
# We get the Quantity and Product Code
#productQTY = p["QtyOnHand"].to_f.round
#productCode = p["Code"].upcase
# If the product code is found in the list of available products in the Shopify store...
if #availableProducts.include? #productCode
#variants.each do |variant|
if #productCode === variant.sku
if #productQTY != 0
#finalProductList << {
"sku" => variant.sku,
"inventory_quantity" => variant.inventory_quantity,
"old_inventory_quantity" => variant.old_inventory_quantity,
"id" => variant.id,
"company_sku" => #productCode,
"company_qty" => #productQTY
}
end
end
end
end
end
# If we get a successful response from StockManagement, proceed...
if #finalProductList
flash[:notice] = 'StockManagement product quantity check was successful.'
puts "Final product list has #{#finalProductList.length} items."
puts #finalProductList
#finalProductList.each do |item|
#productSKU = item["sku"]
#productInventoryQuantity = item["inventory_quantity"]
#productOldInventoryQuantity = item["old_inventory_quantity"]
#productID = item["id"]
#companySKU = item["company_sku"]
#companyQTY = item["company_qty"]
url = URI("https://example.myshopify.com/admin/variants/#{#productID}.json")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Put.new(url)
request["content-type"] = 'application/json'
request["authorization"] = 'Basic KJSHDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDFKJHSDF'
request["cache-control"] = 'no-cache'
request.body = "{\n\t\"variant\": {\n\t\t\"id\": #{#productID},\n\t\t\"inventory_quantity\": #{#companyQTY},\n\t\t\"old_inventory_quantity\": #{#productOldInventoryQuantity}\n\t}\n}"
# This is the line that actually runs the put request to update the quantity.
response = http.request(request)
# Finally, we populate the finalList has with response information.
#finalList[#companySKU] = ["","You had #{#productOldInventoryQuantity} in stock, now you have #{#companyQTY} in stock."]
end
else
# If the overall sync failed, we flash an alert.
flash[:alert] = 'Quantity synchronisation was unsuccessful.'
end
# Lastly we get the final number of items that were synchronised.
#synchronisedItems = #finalList.length
# We flash this notification, letting the user known how many products were successfully synchronised.
flash[:notice] = "#{#synchronisedItems} product quantities were synchronised successfully."
# We then pretty print this to the console for debugging purposes.
pp #finalList
else
flash[:alert] = #errorResponse
end
end
end
First of all, your product_quantity method is way too long. You should break it into smaller parts. 2nd, http.verify_mode = OpenSSL::SSL::VERIFY_NONE should not be done in production. The example you've provide along with your question are too complex and are therefore difficult to answer. It sounds like you need a basic understanding of design patterns and this is not a specific ruby question.
If your app needs to make realtime API calls inside of a controller this is a poor design. You don't want to keep requests of any kind waiting for more than a couple of seconds at most. You should consider WHY you need to make these requests in the first place. If it's data you need quick access to, you should write background jobs to scrape the data on a schedule and store it in your own database.
If a user of your app makes a request which needs to wait for the API's response, you could write a worker to handle fetching the API data and eventually send a response to the user's browser probably using actioncable.
For your constant definitions you probably should do this in an initializer wihich you would keep in my_app_root/config/initializers/constants.rb which get loaded into your app at runtime. You could just call them where need using te ENV[] syntax but if you prefer simpler constants drop the # since that naming convention in ruby is for instance objects.
#app_root/config/initializers/constants.rb
HEROKU_ENV_USERNAME = ENV['HEROKU_ENV_USERNAME']
HEROKU_ENV_PASSWORD = ENV['HEROKU_ENV_PASSWORD']
I want to update model via PUT request in Rails app. What would be the best way to do that?
Basically I have:
def method
...
#relation = Relation.find(34)
#relation.name = "new_name"
#relation.save
end
This gives me errors in SQLite ("cannot start a transaction within a transaction").
Switching to put/post should I guess save the problem.. What would be the right way to do it?
So after some time, I actually found the solution. Here is the code for the Resque worker, that updates the Relation model via PUT. Using this method I don't get SQLite busy exception errors.
class VideoCollector
def self.perform(rel_id)
#relation = Relation.find_by_id(rel_id)
#url = Rails.application.routes.url_helpers.relation_url(#relation)
#uri = URI(#url)
#body ={"collected" => "true"}.to_json
request = Net::HTTP::Put.new(#uri.path, initheader = {'Content-Type' =>'application/json'})
request.body = #body
response = Net::HTTP.new(#uri.host, #uri.port).start {|http| http.request(request) }
end
end
Maybe that will be useful to someone.
I'm trying to get activeresource (on Rails 3.2) working with the Freebase API and I haven't had much luck yet. How can I debug rails to see what's going on and make sure the request is well formed?
I think the suffixed .json is causing the failure but I don't know how to check what's going on?
Error:
ActiveResource::ResourceNotFound: Failed. Response code = 404. Response message = Not Found.
Code:
class Freebase < ActiveResource::Base
class << self # also tried without this, same result
def element_path(id, prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}/#{id}#{query_string(query_options)}"
end
def collection_path(prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}#{query_string(query_options)}"
end
end
self.site = "https://www.googleapis.com/freebase/v1/"
self.format = :json
#https://www.googleapis.com/freebase/v1/search?query=nirvana&indent=true
#Freebase.get('search', :query => 'nirvana')
end
UPDATE:
Ok so I found two things going wrong...
1) The collection_path I'm trying to supersede doesn't work at all.. it still tacks .json onto every request.
2) https://www.googleapis.com:443/freebase/v1/freebases/search.json?query=nirvana
It tacks freebase on afterwards... any ideas?
I also tried this fix:
Remove .xml extension from ActiveResource request
But it didn't remove the JSON suffix either.
UPDATE UPDATE:
Added the suggest update below which gives the correct PATH but now I'm getting
GET https://www.googleapis.com:443/freebase/v1/search/?query=monkey
--> 200 OK 2732 (693.3ms)
NoMethodError: undefined method `collect!' for #<Hash:0x007f9bde674900>
Add ActiveResource::Base.logger = Logger.new(STDERR) to your config/application.rb ( Rails 3.x ).
You'll get output like :
POST http://localhost:3000/freebase.json
--> 201 Created 0 (15.8ms)
That shows method and response code ...
ActiveResource isn't dead, but relative to ActiveRecord I can understand why you'd think so, it is definitely an unloved and underprivileged stepchild.
You'd probably be better off using something like Faraday_Middleware or HTTPParty. The former is my personal preference. Unless what you're doing is pulling from another Rails app or one that has perfect restful behaviour like Rails (which Freebase doesn't), ActiveResource is usually more trouble than it's worth.
That being said, you can accomplish what you want without overwriting any class methods by doing:
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
To get detail login for ActiveResource have to patch the request method inside the gem(method.
place bellow files inside config/initializers you will get http method, path, request body, request hedaers
response body and header is already there if you need. doc
config/initializers/activeresource_patch.rb
module ActiveResource
class Connection
private
def request(method, path, *arguments)
result = ActiveSupport::Notifications.instrument("request.active_resource") do |payload|
payload[:method] = method
payload[:request_uri] = "#{site.scheme}://#{site.host}:#{site.port}#{path}"
payload[:request_path] = path
payload[:request_body] = arguments[0]
payload[:request_headers] = arguments[1]
payload[:result] = http.send(method, path, *arguments)
end
handle_response(result)
rescue Timeout::Error => e
raise TimeoutError.new(e.message)
rescue OpenSSL::SSL::SSLError => e
raise SSLError.new(e.message)
end
end
end
config/initializers/activeresource_logger.rb
Rails.application.configure do
def activeresource_logger
#activeresource_logger ||= Logger.new("#{Rails.root}/log/activeresource_logger.log")
end
ActiveSupport::Notifications.subscribe('request.active_resource') do |name, start, finish, id, payload|
if Rails.env.development?
activeresource_logger.info("====================== #{start} : #{payload[:method].upcase} ======================")
activeresource_logger.info("PATH: #{payload[:request_path]}")
activeresource_logger.info("BODY: #{payload[:request_body]}")
activeresource_logger.info("HEADERS: #{payload[:request_headers]}")
# activeresource_logger.info("STATUS_CODE: #{payload[:result].code}")
# activeresource_logger.info("RESPONSE_BODY: #{payload[:result].body}")
end
end
end
ActiveResource is going to expect that endpoint to return the data in a very specific format.
For the longest time we've been using ActiveResource at my company for inter-application communication. However more recently we've started leaning towards HTTParty because it performs a lot less voodoo magic, and tends to be a much small exercise in hair-pulling.
Here's an example of how we're using HTTParty now:
module CoreResources
class Job
include HTTParty
base_uri Rails.configuration.core_resource_uri
basic_auth Rails.configuration.core_resource_user, Rails.configuration.core_resource_password
def self.search(entity)
get("/api/v1/jobs.json", :query => {:entity_id => entity.id})
end
def self.find(id)
result = get("/api/v1/jobs/#{id}.json")
raise CoreResources::JobNotFound.new if result.response.code == "404"
raise "Unexpected response from resource job find: #{result.response.code} #{result.response.to_s}" if result.response.code =~ /^(?:4|5)..$/
result
end
end
end
The problem with ActiveResource is that it will take the very specifically-crafted json or xml markup, and instantiate ActiveResource objects and nested objects based on it. It's having issues calling collect because something in the json response was supposed to be formatted like an enumerable, and wasn't (likely the parent node should have been an array or something, not sure), which makes it blow up.
With HTTParty you get a json-parsed collection to work with.
It's easy enough for me to do this:
jobs = CoreResources::Job.search(my_entity)
puts jobs.inspect
# [{
# "id" => 4,
# "created_by_id" => 12,
# "description" => "I like pie"
# },
# {
# "id" => 5",
# "created_by_id" => 12,
# "description" => "Mmm, cake"
# }]
Which let's me access jobs via an easy collection array/hash construct jobs[0].fetch("description"), as opposed to ActiveResource: jobs[0].description. ActiveResource is slower to insantiate those collections, needlessly takes up memory with them, and encourages you to duplicate code that should just be served by the endpoint in your ActiveResource model (Then again, if you're using a third-party API you may have no other choice, but I have never successfully gotten ARes to interface with third-party API's).
We've run into a lot of other ActiveResource problems where it does this nonsensical dynamic creation of class names based on nested resources from your endpoint, but half the time does it incorrectly... It's really just a mess.
Moral of the story: Much more of a fan of HTTParty now. That endpoint is probably just not returning data in the right format, and unless it does ActiveResource will be a hackfest to get it to read it right.
ActiveResource has a fairly narrow use-case. You probably want to use a more generic interface for working with Freebase.
Some code from the LinkTV Platform's FreeBase API might be of some help.
The NoMethodError: undefined method 'collect!' for #<Hash:0x007f9bde674900>
error seems to be an issue on rails https://github.com/rails/rails/issues/2318 .
I had a similar problem but the hacks provided didn't work so I had to tweak them a bit , you can find my answer here .
I am using the following code in my MailChimp Controller to submit simple newsletter data. When It is sent I receive the following error as a "Method is not exported by this server -90" I have attached my controller code below. I am using this controller for a simple newsletter signup form. (Name, Email)
class MailchimpController < ApplicationController
require "net/http"
require "uri"
def subscribe
if request.post?
mailchimp = {}
mailchimp['apikey'] = 'f72328d1de9cc76092casdfsd425e467b6641-us2'
mailchimp['id'] = '8037342dd1874'
mailchimp['email_address'] = "email#gmail.com"
mailchimp['merge_vars[FNAME]'] = "FirstName"
mailchimp['output'] = 'json'
uri = URI.parse("http://us2.api.mailchimp.com/1.3/?method=listSubscribe")
response = Net::HTTP.post_form(uri, mailchimp)
mailchimp = ActiveSupport::JSON.decode(response.body)
if mailchimp['error']
render :text => mailchimp['error'] + "code:" + mailchimp['code'].to_s
elsif mailchimp == 'true'
render :text => 'ok'
else
render :text => 'error'
end
end
end
end
I highly recommend the Hominid gem: https://github.com/tatemae-consultancy/hominid
The problem is that Net::HTTP.post_form is not passing the "method" GET parameter. Not being a big ruby user, I'm not certain what the actual proper way to do that with Net::HTTP is, but this works:
require "net/http"
data="apikey=blahblahblah"
response = nil
Net::HTTP.start('us2.api.mailchimp.com', 80) {|http|
response = http.post('/1.3/?method=lists', data)
}
p response.body
That's the lists() method (for simplicity) and you'd have to build up (and urlencode your values!) your the full POST params rather than simply providing the hash.
Did you take a look at the many gems already available for ruby?
http://apidocs.mailchimp.com/downloads/#ruby
The bigger problem, and main reason I'm replying to this, is that your API Key is not obfuscated nearly well enough. Granted I'm used to working with them, but I was able to guess it very quickly. I would suggest immediately going and disabling that key in your account and then editing the post to actually have completely bogus data rather than anything close to the correct key. The list id on the other hand, doesn't matter at all.
You'll be able to use your hash if you convert it to json before passing it to Net::HTTP. The combined code would look something like:
mailchimp = {}
mailchimp['apikey'] = 'APIKEYAPIKEYAPIKEYAPIKEY'
mailchimp['id'] = '8037342dd1874'
mailchimp['email_address'] = "email#gmail.com"
mailchimp['merge_vars[FNAME]'] = "FirstName"
mailchimp['output'] = 'json'
response = nil
Net::HTTP.start('us2.api.mailchimp.com', 80) {|http|
response = http.post('/1.3/?method=listSubscribe', mailchimp.to_json)
}