I'm trying to load some items from this external API using REST Client and DelayedJob.
But, unlike Ajax check for database change using route, these items don't get saved to the database but rather go straight to the view -- so how would I know when they're done loading so I can load them into the client using Ajax?
Live app which you can run on the fly: http://runnable.com/VW9rQx-KiIFfmpII/ajax-affiliates
app/controllers/main_controller.rb
class MainController < ApplicationController
def index
# Delay fetching
#products = Affiliate.delay.fetch
end
# Pseudo-code from SO `ajax-check-for-database-change-using-route`
#
# def check_items_loaded
# #items_status = Affiliate.find(params[:id]).items_loaded
# respond_to do |wants|
# wants.js
# end
# end
end
config/routes.rb
# Pseudo-code from SO `ajax-check-for-database-change-using-route`
# get '/check_items_loaded/:id', to: 'photos#check_items_loaded', as: :check_items_loaded
app/models/affiliate.rb
require "rest_client"
module Affiliate
def self.fetch
response = RestClient::Request.execute(
:method => :get,
:url => "http://api.shopstyle.com/api/v2/products?pid=uid7849-6112293-28&fts=women&offset=0&limit=10"
)
#products = JSON.parse(response)["products"].map do |product|
product = OpenStruct.new(product)
product
end
end
# More pseudo-code
# Add `items_loaded` boolean to database?
# private
# def set_items_loaded
# self.update_attributes(items_loaded: ???)
# end
end
You could look into a pub-sub architecture using something like PubNub[1] or Pusher[2]
Your front-end would subscribe to those endpoints and then you can have your backend publish events to those endpoints (which are then read from the consumers (your frontend))
[1] PubNub - http://www.pubnub.com/
[2] Pusher - https://pusher.com/
You might be able to hack together a solution using ActionController::Streaming. Do take into account potential session bottlenecks.
An alternate solution might be using Websockets & Redis.
Related
I'm trying to learn processing JSON with Ruby. I went through quite a few tutorial but got confused even more, so I try to learn by doing it.
Here, I'm trying to get user's data from Instagram and display in my View. I have access to the JSON below but how do I reach certain fields and username or loop posts?
# users_controller.rb
require 'httparty'
class UsersController < ApplicationController
include HTTParty
def show
#user = User.find(params[:id])
fetch_instagram("elonofficiall")
end
private
def fetch_instagram(instagram_username)
url = "https://www.instagram.com/#{instagram_username}/?__a=1"
#data = HTTParty.get(url)
return #data
end
end
# show.html.erb
# the code below is just to try if I get anything
<%= #data %>
<% #data.each do |data| %>
<p><%= data %></p>
<% end %>
https://www.instagram.com/elonofficiall/?__a=1
First off don't do HTTP calls straight from your controller.
Instead create a separate class that "talks" to the instagram API:
# app/clients/instagram_client.rb
class InstagramClient
include HTTParty
base_uri 'https://www.instagram.com'
format :json
def initialize(**options)
#options = options
end
def user(username, **opts)
options = #options.reverse_merge(
'__a' => 1 # wtf is this param?
).reverse_merge(opts)
response = self.class.get("/#{username}", options)
if response.success?
extract_user(response)
else
Rails.logger.error("Fetching Instagram feed failed: HTTP #{response.code}")
nil
end
end
private
def extract_user(json)
attributes = response.dig('graphql', 'user')&.slice('id', 'biography')
attributes ? InstagramUser.new(attributes) : nil
end
end
And an class that normalizes the API response for consumption in your application:
# app/models/instagram_user.rb
class InstagramUser
include ActiveModel::Model
include ActiveModel::Attributes
attribute :id
attribute :biography
attribute :username
# etc
end
This is just a straight up Rails model that is not persisted in the database. ActiveModel::Model and ActiveModel::Attributes let you pass a hash of attributes for mass assignment just like you do with ActiveRecord backed models.
This gives you an object you can simply test by:
InstagramClient.new.feed('elonofficiall')
And you would integrate it into your controller like so:
class UsersController < ApplicationController
def show
#user = User.find(params[:id])
fetch_instagram(#user.instagram_username)
end
private
def fetch_instagram(instagram_username)
#instagram_user = InstagramClient.new.user(#user.instagram_username)
end
end
Mixing HTTParty into your controller is a straight up bad idea as controllers are tricky to test and the controller already has the responsibility of responding to client requests and does not need more.
Also processing a "raw" JSON response in your view is not a great practice as it creates a hard coupling between the external API and your application and adds far to much complexity to your views. Normalize the data first in a separate object.
<% if #instagram_user %>
<p><%= #instagram_user.username %></p>
<% end %>
If you want to actually get the media items from the users instagram feed you need to make another HTTP request to GET https://graph.facebook.com/{ig-user-id}/media. See the official API documentation.
Best to take it out of Rails first off. That only convolutes things. Many devs reply Rails === Ruby. It is not.
Write a simple ruby script and test her out:
rec = HTTParty.get(url)
puts rec.headers # content-type is 'text/html' and should be 'application/json'
res = rec.parsed_response # getting error
puts res.class
The issue with Instagram is that I think it's forcing a bad content-header back to the user. It might be a security layer. I can't seem to force the reply into json or a hash either. So that's why httparty is not working for me.
I'm implementing a custom payment gateway in Spree 2.2. It's one of those gateways where you redirect to the gateway's own website to take payment, and then the bank redirects back to you with a bunch of get params.
I'm having an issue where the order's payment_state and shipment_state end up as null in the database, despite the fact that they are not null in the order object itself, if I put a debugger in the code. Calling order.save doesn't seem to help.
I've implemented a dirty hack to workaround it:
# This is a hack - for some reason the payment_state and shipment_state weren't being persisted
# and where being stored in the database as null. Really the spree checkout process
# should take care of this and we shouldn't have to set them manually.
# We must be doing something wrong...
order.update_attribute :payment_state, 'paid'
order.update_attribute :shipment_state, 'ready'
But I'd really like to know what the actual issue is - why aren't those states being persisted? (I should add, before I call the code above, the values for order.payment_state and order.shipment_state respectively are balance_due and pending - but that's another issue. If I can get them to save in any way, that's the main issue.
Any ideas what I'm doing wrong?
Full code for my controller and gateway is below.
class Spree::CommBankController < Spree::StoreController
def secure_payment
order = current_order
#order_info = 'Espionage Online order ' + order.number
payment_params = {
"Title" => 'Espionage Online',
"vpc_AccessCode" => payment_method.preferred_access_code,
"vpc_Amount" => (order.total * 100).to_i, # convert to cents
"vpc_Desc" => #order_info,
"vpc_MerchTxnRef" => order.number,
"vpc_Merchant" => payment_method.preferred_merchant_id_no,
"vpc_OrderInfo" => #order_info,
"vpc_ReturnURL" => secure_payment_callback_url(payment_method_id: payment_method.id),
}
payment_request = ::CommWeb::PaymentRequest.new(payment_params, payment_method.preferred_secure_secret)
redirect_to payment_request.url
end
def secure_payment_callback
# Next line - see http://stackoverflow.com/questions/4116545/how-do-i-get-only-the-query-string-in-a-rails-route
order = current_order
query_params = params.except(*request.path_parameters.keys)
payment_response = ::CommWeb::PaymentResponse.new(query_params, payment_method.preferred_secure_secret)
if !secure_hash_matches?(payment_response)
flash.notice = 'Error with payment - secure hash did not match. Please try again.'
redirect_to checkout_state_path(order.state)
return
end
payment = order.payments.create!({
:source => Spree::CommbankCheckout.create({
:params_hash => payment_response.params.to_s,
:success => payment_response.success?,
:desc => payment_response.description,
:trx_response_code => payment_response.trx_response_code,
:message => payment_response.message,
}),
:amount => order.total,
:payment_method => payment_method,
:response_code => payment_response.trx_response_code,
})
payment.pend
if payment_response.success?
# Set payment to completed after order.next because
# spree expects at least one incomplete payment to process an order to complete
order.next!
payment.complete
debugger
# This is a hack - for some reason the payment_state and shipment_state weren't being persisted
# and where being stored in the database as null. Really the spree checkout process
# should take care of this and we shouldn't have to set them manually.
# We must be doing something wrong...
order.update_attribute :payment_state, 'paid'
order.update_attribute :shipment_state, 'ready'
else
payment.failure
end
if order.complete?
flash.notice = Spree.t(:order_processed_successfully)
session[:order_id] = nil
redirect_to completion_route(order)
else
flash.notice = 'Error: ' + payment_response.message + '. Please try again.'
redirect_to checkout_state_path(order.state)
end
end
def secure_hash_matches? payment_response
payment_response.secure_hash_matches?
end
def payment_method
#payment_method ||= Spree::PaymentMethod.find(params[:payment_method_id])
end
def completion_route(order)
order_path(order)
end
end
and the gateway...
# Partly inspired from https://github.com/spree-contrib/spree-adyen (the hosted payment page class)
module Spree
class Gateway::CommBank < Gateway
preference :merchant_id_no, :string
preference :access_code, :string
preference :secure_secret, :string
def auto_capture?
true
end
# Spree usually grabs these from a Credit Card object but when using
# Commbank's 3 Party where we wouldn't keep the credit card object
# as that's entered outside of the store forms
def actions
%w{capture}
end
# Indicates whether its possible to void the payment.
def can_void?(payment)
!payment.void?
end
# Indicates whether its possible to capture the payment
def can_capture?(payment)
payment.pending? || payment.checkout?
end
def method_type
'commbank'
end
def capture(*args)
ActiveMerchant::Billing::Response.new(true, "", {}, {})
end
def source_required?
false
end
def provider_class
self.class
end
def provider
self
end
def purchase
# This is normally delegated to the payment, but don't do that. Handle it here.
# This is a hack copied from the Spree Better Paypal Express gem.
Class.new do
def success?; true; end
def authorization; nil; end
end.new
end
end
end
Check order.state_changes. Do they show changes to the two states?
I am encountering the same issue while using "spree-adyen". The order.state_changes shows that the payment_state and shipment_state have changed to ready. However, it doesn't persist in the order. This happens randomly with 10% of the orders. I am currently calling order.update! manually on the such order, but would really like to know as well what the issue is.
Also, I am not quite sure if order.update! is a good solution, as it executes a lot of queries and can be very expensive.
Umm. So apparently order.update! will solve my issue. Woops.
Still, a call to order.update! isn't something I've seen in other Spree Payment Gateway gems (https://github.com/spree-contrib/better_spree_paypal_express or https://github.com/spree-contrib/spree-adyen), so I'd be interested to know if I'm doing something really stupid. (I ended up noticing it in the code of https://github.com/coinbase/coinbase-spree/blob/master/app%2Fcontrollers%2Fspree%2Fcoinbase_controller.rb)
I have been trying to setup my first webhook with stripe. I found an article that looks like the right way to do it but 2 years old. I am thinking it is outdated.
Here is my controller so far.
class StripewebhooksController < ApplicationController
# Set your secret key: remember to change this to your live secret key in production
# See your keys here https://manage.stripe.com/account
Stripe.api_key = "mytestapikey"
require 'json'
post '/stripewebhooks' do
data = JSON.parse request.body.read, :symbolize_names => true
p data
puts "Received event with ID: #{data[:id]} Type: #{data[:type]}"
# Retrieving the event from the Stripe API guarantees its authenticity
event = Stripe::Event.retrieve(data[:id])
# This will send receipts on succesful invoices
# You could also send emails on all charge.succeeded events
if event.type == 'invoice.payment_succeeded'
email_invoice_receipt(event.data.object)
end
end
end
Will this work correctly and is this the right way to do it? Here is the stripe documentation.
I'm using Stripe Webhooks in production and this doesn't look quite right. You should first define your webhook URL in your routes like this:
# config/routes.rb
MyApp::Application.routes.draw do
post 'webhook/receive'
end
In this example your webhook url will be at http://yourapp.com/webhook/receive (that's what you give to Stripe). Then you need the appropriate controller and action:
class WebhookController < ApplicationController
# You need this line or you'll get CSRF/token errors from Rails (because this is a post)
skip_before_filter :verify_authenticity_token
def receive
# I like to save all my webhook events (just in case)
# and parse them in the background
# If you want to do that, do this
event = Event.new({raw_body: request.body.read})
event.save
# OR If you'd rather just parse and act
# Do something like this
raw_body = request.body.read
json = JSON.parse raw_body
event_type = json['type'] # You most likely need the event type
customer_id = json['data']['object']['customer'] # Customer ID is the other main bit of info you need
# Do the rest of your business here
# Stripe just needs a 200/ok in return
render nothing: true
end
end
Another thing to note: every webhook you receive has an ID. It's good practice to save and check against this to make sure you're not acting on the same event more than once.
I'm thinking about writing an automatic spam protection system (maybe I will write a public gem) for rails.
My concept is to include a helper method in application_controller f.e.:
class ApplicationController < ActionController::Base
automatic_captcha_redirect(:min_time => 30.seconds :limit => 50)
...
end
Then I want to include automatical a before_filter in every controller, which checks, if the current request is via post, put or delete-method.
If the user's last post-request is smaller than :min_time, then the request should be redirected to an captcha-input-page (the posted user-data resides in hidden html fields).
# before_filter :check_spam
def check_spam
if !request.get? && session[:last_manipulation_at]
&& session[:last_manipulation_at] >= DateTime.now - 30.seconds
redirect_to captcha_path
# (doesn't know yet how to handle the post data to
# display in hidden fields in the spam-captcha-form)
end
end
And in captcha.haml
=form_tag
-request.params.each do |key, value|
=hidden_field_tag key, value
=captcha_image
=submit_button_tag
If the user submits the right captcha-word, his data will be posted to the right action.
Do you think thats realizable?
Any critics or suggestions? Or an idea how to realize this behaviour?
EDIT:
this should not pass through all the ActiveRecord stack; can't it be implemented as a middleware hook (Rails Rack)?
Yes, would be a good idea - but I'm not very familiar with rails rack :/
what about file uploads? (you can not store it in a hidden file)
Hm... maybe a check if there is a file in the post? (How could that be realized?)
what about Ajax posting?
Maybe sending back http-status codes (f.e. 503 Service temporary unavailable)
why only POST and not also PUT and DELETE?
corrected this in my question
EDIT:
First structure of processing (as non rack-app - I dont know how to write rack apps):
0) Settings in environment.rb
auto_recaptcha[:limit] = 10
auto_recaptcha[:min_time] = 1.minute
1) User posts data
Check last_manipulation and max. amount of allowed manipultations in application_controller.rb
class ApplicationController < ActionController::Base
before_filter :automatic_captcha_redirect
def automatic_captcha_redirect
session[:last_manipulation_at][:manipultation] = [] unless session[:last_manipulation_at][:manipultation]
# Checks if requests are falling under the specifications for showing captcha
if !request.get?
&& session[:last_manipulation_at][:date] > DateTime.now - auto_recaptcha[:min_time]
&& session[:last_manipulation_at][:manipultation].count < auto_recaptcha[:limit]
# If user answered captcha, verify it
if !verify_captcha(params)
#url = request.url
#params = request.params
render "layouts/captcha.haml"
else
# Add successfull manipulation to counter
session[:last_manipulation_at][:manipultation] << DateTime.now
session[:last_manipulation_at][:date] = DateTime.now
end
end
end
end
captcha.haml
-form_tag #url do
-request.params.each do |key, value|
=hidden_field_tag key, value
=captcha_image
=submit_button_tag
2)
...
...
...
last) Post userdata to the right location
post(params) => users_path # path "/users" with method: post
First, i would like to say that this is a very good ideea of a feature.
My qs/remarks:
this should not pass through all the ActiveRecord stack; can't it be implemented as a middleware hook (Rails Rack)?
what about file uploads? (you can not store it in a hidden file)
what about Ajax posting?
why only POST and not also PUT and DELETE?
Anyway, i would be more interested to see the number of posts in last 5 mins, for example, that the date of the last request. I believe it is more relevant.
One way this could be put together:
Middleware/rails metal component that
monitors the requests and adds the
information to the rack session.
Controller helpers for before_filters
on things that might need captchas
View helpers for displaying the
captchas
You could make the captcha rate adjustable through the args passing mechanism of use
#config/environment.rb
config.middleware.use 'CaptchaMiddleware',:period=>5.minutes,:limit=>50,:captcha_url=>'/captcha'
Also, this should not rely on hidden form fields because a determined bot writer could just change the value they are posting to your server code.
Simple middleware example code(slightly better than a stab in the dark, but still)
class CaptchaMiddleware
def initialize app,options
#app = app
#options=options
end
def update_stats!
#session based,on account of laziness
session[:reqs] ||= []
session[:reqs].reject!{ |request| request < Time.now - #options[:period]}
session[:reqs] << Time.now
end
def over_limit?
session[:reqs].length > #options[:limit]
end
def call env
#env = env
if #env["REQUEST_METHOD"]!='GET'
update_stats!
if over_limit?
return [302,{"Location: #{options[:captcha_url]}"},'']
end
end
#app.call env
end
def session
#env["rack.session"]
end
end
When generating cache-keys for rendered content in web applications, you have to take into account all variables that might change the result.
In dynamic environments like rails these can be defined in different places: the controller, a model, the session or the server environment. And they can be referenced in the template, in a template rendered within the template or a helper.
Can you think of a way to automatically generate a list of variables that contribute to the content of the rendered template, maybe using ParseTree?
I use a "freshness key" in my cache keys, probably learned from this article:
http://blog.leetsoft.com/2007/5/22/the-secret-to-memcached
This allows me to easily invalidate all cache related to a resource, regardless of the url, despite the fact that memcached does not provide a facility for iterating existing keys.
I typically generate mine using a combination of the request.url and the logged in user id and freshness_key, e.g.
#
# return the freshness_key for caching this particular collection
def get_freshness_key_for(collection_name)
Rails.cache.fetch("#{self.id}:#{collection_name}") { "#{self.send(collection_name).maximum(:updated_at).to_i}:#{Time.now.to_i}" }
end
#
# set the freshness_key for caching this particular collection;
# typically called after_save from an Observer for this collection
#
def set_freshness_key_for(collection_name)
Rails.cache.write("#{self.id}:#{collection_name}", "#{self.send(collection_name).maximum(:updated_at).to_i}:#{Time.now.to_i}")
end
# returns the cache_key for this client, the desired collection, and the
# current url with the latest freshness_key
#
# the url is hashed so as not to overrun memcached's key length limit
def cache_key_for(collection_name, request_url)
freshness_key = self.get_freshness_key_for(collection_name)
"#{self.id}:#{Digest::MD5.hexdigest(request_url)}:#{freshness_key}"
end
I'll use it in a controller:
#posts_cache_key = cache_key_for(:posts)
#posts = cache(#posts_cache_key) do
Post.paginate(
:page => params[:page],
:per_page => params[:pp]
)
end
...and in the view:
<% cache(:key => "#{#posts_cache_key}:posts_list_fragment") do -%>
... html here ...
<% end -%>
Typically I'll have an Observer for the collection model:
class PostObserver < ActiveRecord::Observer
def after_save(post)
post.client.set_freshness_key_for(:posts)
end
def after_destroy(post)
post.client.set_freshness_key_for(:posts)
end
end
Hope this helps