I believe that to have a Shopify webhook integrate with a Rails app, the Rails app needs to disable the default verify_authenticity_token method, and implement its own authentication using the X_SHOPIFY_HMAC_SHA256 header. The Shopify docs say to just use request.body.read. So, I did that:
def create
verify_webhook(request)
# Send back a 200 OK response
head :ok
end
def verify_webhook(request)
header_hmac = request.headers["HTTP_X_SHOPIFY_HMAC_SHA256"]
digest = OpenSSL::Digest.new("sha256")
request.body.rewind
calculated_hmac = Base64.encode64(OpenSSL::HMAC.digest(digest, SHARED_SECRET, request.body.read)).strip
puts "header hmac: #{header_hmac}"
puts "calculated hmac: #{calculated_hmac}"
puts "Verified:#{ActiveSupport::SecurityUtils.secure_compare(calculated_hmac, header_hmac)}"
end
The Shopify webhook is directed to the correct URL and the route gives it to the controller method shown above. But when I send a test notification, the output is not right. The two HMACs are not equal, and so it is not verified. I am fairly sure that the problem is that Shopify is using the entire request as their seed for the authentication hash, not just the POST contents. So, I need the original, untouched HTTP request, unless I am mistaken.
This question seemed like the only promising thing on the Internet after at least an hour of searching. It was exactly what I was asking and it had an accepted answer with 30 upvotes. But his answer... is absurd. It spits out an unintelligible, garbled mess of all kinds of things. Am I missing something glaring?
Furthermore, this article seemed to suggest that what I am looking for is not possible. It seems that Rails is never given the unadulterated request, but it is split into disparate parts by Rack, before it ever gets to Rails. If so, I guess I could maybe attempt to reassemble it, but I would have to even get the order of the headers correct for a hash to work, so I can't imagine that would be possible.
I guess my main question is, am I totally screwed?
The problem was in my SHARED_SECRET. I assumed this was the API secret key, because a few days ago it was called the shared secret in the Shopify admin page. But now I see a tiny paragraph at the bottom of the notifications page that says,
All your webhooks will be signed with ---MY_REAL_SHARED_SECRET--- so
you can verify their integrity.
This is the secret I need to use to verify the webhooks. Why there are two of them, I have no idea.
Have you tried doing it in the order they show in their guides? They have a working sample for ruby.
def create
request.body.rewind
data = request.body.read
header = request.headers["HTTP_X_SHOPIFY_HMAC_SHA256"]
verified = verify_webhook(data, header)
head :ok
end
They say in their guides:
Each Webhook request includes a X-Shopify-Hmac-SHA256 header which is
generated using the app's shared secret, along with the data sent in
the request.
the keywors being "generated using shared secret AND DATA sent in the request" so all of this should be available on your end, both the DATA and the shared secret.
Related
I am using the RedditKit gem and in order to access certain elements, I need to send a request to reddit api to create a "client" object. Below is my current logic:
## application_controller
before_action :redditkit_login
private
def redditkit_login
#client = RedditKit::Client.new ENV["reddit_username"], ENV["reddit_password"]
end
As you can see in my logic here, before EVERY REQUEST, I am subsequently making a new client object and then using that everywhere.
My question is, how do I only make one client object which can be used to serve ALL requests from anywhere?
My motive behind this is speed. For every request to server, I am making a new request to reddit and then responding to the original request. I want to have the client object readily available at all times.
You have a lot of options. A simple one would be to create a config/initializers/reddit_client.rb file and put in there:
RedditClient = RedditKit::Client.new ENV.fetch("reddit_username"), ENV("reddit_password")
(note I switched to ENV.fetch because it will error if the key is not found, which can be helpful).
You could also rename the file as app/models/reddit_client.rb. Although it's not really a model, that folder is also autoloaded so it should work as well.
I am working on an app and up to this point I have been testing just stuff like authentication and request response codes. but it seems like a good idea to test the structure of the payload. ie. if there is embedded resources or sidloaded resources. how do you guys test this. here is a sample of some of the testing I am doing. I am using active model serializers. but seems like a bit of cruft to organize.
describe '#index' do
it 'should return an array of email templates' do
template = EmailTemplate.new(name: 'new email template')
EmailTemplate.stub(:all).and_return([template])
get :index
payload = {:email_templates => [JSON.parse(response.body)["email_templates"][0].symbolize_keys]}
template_as_json_payload = {:email_templates => [ActiveModel::SerializableResource.new(template).as_json[:email_template] ]}
expect(payload).to eq(template_as_json_payload)
end
end
I'm fond of defining a schema for JSON responses and validating that any response seen in a test conforms to it. That alone does not guarantee that the values of the response are correct but it does tell you that the structure of the response matched your expectations.
Those schemas then become part of the documentation of the API which client implementations can reference. By using the schema in tests I get more confidence that the API documentation is not going to fall out of sync with the implementation. Making schema changes in order to get a passing test is also a good prompt for me to consider if a change is safe for existing API clients or if I need to release a new API version.
The folks at Thoughtbot have a nice example of validating schemas using rspec: https://robots.thoughtbot.com/validating-json-schemas-with-an-rspec-matcher
This is one way to do it:
body = JSON.parse(response.body)
assert_equal(body.keys, ["id", "author"])
assert_equal(body["author"].keys, ["id", "name"])
But you should checkout the link that Jonah shared, it's worth reading.
I created a helper module in Rails that creates a hash of parameters that are passed to link_to rails helpers. These utm parameters are being used in emails sent out by the Rails app. The problem is that Google Analytics is not picking up any data when I've been testing. I realize there is a delay in processing and I'm also using the GA debugger to look at the beacons it's actually sending out and I have a staging server and staging Google Analytics that I am testing all of this on. Does anyone see anything with the approach below that would cause GA to not pick up any visits under the "campaigns" report? Does the order of the utm tags actually matter? I realize utm_campaign, utm_source and utm_medium are all required and ensured they are in every link.
For example, this is what one of the links looks like. However, GA is not picking up any data.
http://example.com/?utm_campaign=welcome&utm_medium=email&utm_source=abc
I compare this link that that the link_to method has created with to what the Google UTM Link Builder outputs and the only difference is the order of the parameters. Does the order matter?
http://example.com/?utm_source=abc&utm_medium=email&utm_campaign=welcome
Here is the helper module and an example of it's usage with a link_to.
# mailer_helper.rb
require 'uri'
module MailerHelper
def self.tracking_param_hash(utm_source, utm_medium, utm_campaign, params = {})
{ utm_source: utm_source,
utm_medium: utm_medium,
utm_campaign: utm_campaign
}.merge(params)
end
def self.email_tracking_params(utm_campaign, options = {})
tracking_param_hash("abc", "email", utm_campaign, options)
end
...
end
# example usage in email view
email_params = MailerHelper::email_tracking_params("set-password", reset_password_token: #token)
link_to 'Set my password', edit_password_url(#resource, email_params)
# rendered link in email
http://example.com/users/password/edit?reset_password_token=YEt9PJgQsxb3WpJQ7OEXH3YDT8JZMD&utm_campaign=reset-password&utm_medium=email&utm_source=abc
Data wasn't recording for several reasons. The first one was because one of the link users were clicking on was doing a POST to a user_confirmation_path. Because the urchin was never loads on the POST, no data will be recorded and users are re-directed to the sign in page. The parameters need to persist with the re-direct from the POST, otherwise traffic will be treated as direct. Near the end of this Kissmetrics blog post they outline this problem. To get around this, you can pass parameters into the redirect url. So something like redirect_to user_signin_path(tracking_parameters), where tracking_parameters are the utm tags from the POST URL.
The second was because the utmz cookie was persisting across and wasn't counting the visit as a new session. You need to test everything in a fresh incognito window because of how Google Analytics and chrome treat individual sessions. See this SO post for more details.
Finally, for anyone who reads this, you can confirm that UTM parameters are being set by looking fora cookie called utmz. This cookie is used by Google Analytics and will look something like this 12340657.1412743014.15.1.utmcsr=abc|utmccn=confirm|utmcmd=email and should have the utm parameters in it.
[Edited for clarity and and the various scenarios I was testing]
According to google, the order doesn't matter. AFAICT everything is in-order in your code, so the problem is somewhere else. Cookies?
I'm writing a web site with rails, which can let visitors inputing some domains and check if they had been regiestered.
When user clicked "Submit" button, my web site will try to post some data to another web site, and read the result back. But that website is slow for me, each request need 2 or 3 seconds. So I'm worried about the performance.
For example, if my web server allows 100 processes at most, that there are only 30 or 40 users can visit my website at the same time. This is not acceptable, is there any way to improve the performance?
PS:
At first, I want to use ajax reading that web site, but because of the "cross-domain" problem, it doesn't work. So I have to use this "ajax proxy" solution.
It's a bit more work, but you can use something like DelayedJob to process the requests to the other site in the background.
DelayedJob creates separate worker processes that look at a jobs table for stuff to do. When the user clicks submit, such a job is created, and starts running in one of those workers. This off-loads your Rails workers, and keeps your website snappy.
However, you will have to create some sort of polling mechanism in the browser while the job is running. Perhaps using a refresh or some simple AJAX. That way, the visitor could see a message such as “One moment, please...”, and after a while, the actual results.
Rather than posting some data to the websites, you could use an HTTP HEAD request, which (I believe) should return only the header information for that URL.
I found this code by googling around a bit:
require "net/http"
req = Net::HTTP.new('google.com', 80)
p req.request_head('/')
This will probably be faster than a POST request, and you won't have to wait to receive the entire contents of that resource. You should be able to determine whether the site is in use based on the response code.
Try using typhoeus rather than AJAX to get the body. You can POST the domain names for that site to check using typhoeus and can parse the response fetched. Its extremely fast compared to other solutions. A snippet that i ripped from the wiki page from the github repo http://github.com/pauldix/typhoeus shows that you can run requests in parallel (Which is probably what you want considering that it takes 1 to 2 seconds for an ajax request!!) :
hydra = Typhoeus::Hydra.new
first_request = Typhoeus::Request.new("http://localhost:3000/posts/1.json")
first_request.on_complete do |response|
post = JSON.parse(response.body)
third_request = Typhoeus::Request.new(post.links.first) # get the first url in the post
third_request.on_complete do |response|
# do something with that
end
hydra.queue third_request
return post
end
second_request = Typhoeus::Request.new("http://localhost:3000/users/1.json")
second_request.on_complete do |response|
JSON.parse(response.body)
end
hydra.queue first_request
hydra.queue second_request
hydra.run # this is a blocking call that returns once all requests are complete
first_request.handled_response # the value returned from the on_complete block
second_request.handled_response # the value returned from the on_complete block (parsed JSON)
Also Typhoeus + delayed_job = AWESOME!
I'm playing with the authlogic-example-app and I'm failing to get the email address from the OpenID provider (in my case: Google and Yahoo) when I register a user, resp. I get an empty response instead of an email address (check the comments in code below).
This is how my user model looks like (everything else looks like the "with_openid"-branch of the authlogic-example-app mentioned above). Besides the missing 'email', the openid-authentication-process works as expected:
class User < ActiveRecord::Base
acts_as_authentic do |c|
# not needed because I use OpenID
c.validate_login_field = false
# avoid failed validation before OpenID request
c.validate_email_field = false
# this one sets 'openid.sreg.required=email'
c.required_fields = [:email]
end
private
# overwriting the existing method in '/lib/authlogic_openid/acts_as_authentic.rb'
def map_openid_registration(registration)
# this is my problem: 'registration' is an empty hash
self.email ||= registration[:email] if respond_to?(:email) && !registration[:email].blank?
end
end
Any idea how to solve this? Has anyone here done this before using authlogic? Or even better: Do you have a working example?
Update: I checked the Google Account Authentication API and compared the request submitted by authlogic (using ruby-openid-gem and openid-authentication-plugin) with the example requests on the Google Account Authentication API docs:
Example request to authenticate and fetch email address by Google:
https://www.google.com/accounts/o8/ud
?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0
&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.return_to=http%3A%2F%2Fwww.example.com%2Fcheckauth
&openid.realm=http%3A%2F%2Fwww.example.com%2F
&openid.assoc_handle=ABSmpf6DNMw
&openid.mode=checkid_setup
&openid.ns.ext1=http%3A%2F%2Fopenid.net%2Fsrv%2Fax%2F1.0
&openid.ext1.mode=fetch_request
&openid.ext1.type.email=http%3A%2F%2Faxschema.org%2Fcontact%2Femail
&openid.ext1.required=email
Request submitted by my appliation:
https://www.google.com/accounts/o8/ud
?openid.assoc_handle=AOQobUcdICerEyK6SXJfukaz8ygXiBqF_gKXv68OBtPXmeafBSdZ6576
&openid.ax.mode=fetch_request
&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.mode=checkid_setup
&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0
&openid.ns.ax=http%3A%2F%2Fopenid.net%2Fsrv%2Fax%2F1.0
&openid.ns.sreg=http%3A%2F%2Fopenid.net%2Fextensions%2Fsreg%2F1.1
&openid.realm=http%3A%2F%2Flocalhost%3A3000%2F
&openid.return_to=http%3A%2F%2Flocalhost%3A3000%2Faccount%3Ffor_model%3D1%26_method%3Dpost%26open_id_complete%3D1
&openid.sreg.required=email
While debugging the whole setup, I've found out that the openid-authentication-plugin never receives an email in the response it receives from the openid provider, this at least explains why the registration hash in my user-model is empty...
UPDATE: If you're playing around with authlogic and openid, don't forget to check out the latest railscast on this subject!
As nobody could help me, I helped myself. :-)
The short answer to my question is:
c.required_fields = [:email,"http://axschema.org/contact/email"]
Using this line, the application requests the email-address using sreg and ax (request-type supported by Google).
You can find a more detailed answer and a working implementation of authlogic-openid with the Javascript OpenID-Selector right here:
http://github.com/vazqujav/authlogic_openid_selector_example/
While this pointed me in the right direction, what I needed was:
c.openid_required_fields = [:email,"http://axschema.org/contact/email"]
This pulled in the email and set it.
# fetch email by ax
c.openid_required_fields = [
"http://axschema.org/contact/email",
"http://axschema.org/namePerson/first",
"http://axschema.org/namePerson/last",
"http://axschema.org/contact/country/home",
"http://axschema.org/pref/language"
]
This fetches in multiple values as specified # http://code.google.com/apis/accounts/docs/OpenID.html#Parameters
Though I'm still unable to fetch in the country name... name, email, language works perfectly!
Test against an OpenID server you control, since it'll let you debug every part of the OpenID sequence. There are no guarantees that Google's OpenID provider is doing the right thing. Try checking against Verisign's server, since I'm pretty sure that one at least should do the right thing with the openid.sreg.required=email field.
Your code snippet looks right to me.
the thing is i am able to fetch the parameters from the provider but am not able to extract them from the response...
i have used OpenID::AX::FetchResponse.from_success_response(open_id_response)
as the object to hold the response... what method do i use to extract email,nickname,country,etc...