HTTP request during ActiveRecord creation - ruby-on-rails

Question

I am trying to subscribe to a pubsubhubbub (pshb) feed on superfeedr. The way the pshb protocol works is
you Send POST to the hub requesting to subscribe to a feed and provide a callback
hub sends a GET to your callback to verifying your intent to subscribe
you respond saying yes I do want to subscribe
hub responds with subscription verification
I am running the server locally on my dev machine. I have code that can successfully subscribe to a feed, I test it by executing it in the rails console directly. I have now created a Feed ActiveRecord model and want automactially subscribe every time a new Feed record is created. I have added a ActiveRecord callback in the Feed model as such
after_create :subscribe_feed
Now when I create an active record I see the correct HTTP request go out, then there is a long hang (about 5 seconds where nothing happens in the logs), then the response from the pshb server comes in saying my callback could not be reached.
Here are the logs (numbers added by me for reference points)
(1) (0.1ms) begin transaction
(2) SQL (4.4ms) INSERT INTO "feeds" ("created_at", "updated_at", "url") VALUES (?, ?, ?) [["created_at", Fri, 15 Nov 2013 23:08:39 UTC +00:00], ["updated_at", Fri, 15 Nov 2013 23:08:39 UTC +00:00], ["url", "http://push-pub.appspot.com/feed"]]
(3) pshb subscribe parameters: {:headers=>{"Accept"=>"application/json"}, :body=>{"hub.mode"=>"subscribe", "hub.verify"=>"sync", "hub.callback"=>"http://...myserver.../pub_sub/callback", "hub.topic"=>"http://push-pub.appspot.com/feed", "hub.verify_token"=>"superfeedtest", "format"=>"json"}}
(4) # ABOUT 5 SECOND WAIT
(5) Subscribe Response: #<HTTParty::Response:0x7fd2180cd978 parsed_response="Your callback couldn't be reached.\n", #response=#<Net::HTTPUnprocessableEntity 422 Unprocessable Entity readbody=true>, #headers={"server"=>["nginx/0.8.52"], "date"=>["Fri, 15 Nov 2013 23:08:44 GMT"], "content-type"=>["text/plain; charset=utf-8"], "connection"=>["close"], "status"=>["422 Unprocessable Entity"], "x-runtime"=>["10057"], "content-length"=>["35"], "set-cookie"=>["_superfeedr_session=BAh7BzoMdXNlcl9pZGkC%2BIY6D3Nlc3Npb25faWQiJTBiMDExZGYzNzU4Mjk0MTMxNjc4NmE0OTg3MDhlMjJk--85d29756ae4b5ae9464630741372af7656f3894b; path=/; HttpOnly"], "cache-control"=>["no-cache"], "pubsubhubbub-version"=>["0.3"]}>
(6) (7.2ms) commit transaction
Redirected to http://67.180.177.165/feeds/28
Completed 302 Found in 10695ms (ActiveRecord: 11.6ms)
(7) Started GET "/pub_sub/callback?hub.challenge=437c4b3b47aa1dbf4072a2d8abb5c39a&hub.lease_seconds=315360000&hub.mode=subscribe&hub.topic=http%3A%2F%2Fpush-pub.appspot.com%2Ffeed&hub.verify_token=superfeedtest" for 173.255.193.75 at 2013-11-15 15:08:50 -0800
You can see that after the response the commit transaction happens (6), and then the GET from the phsb server asking to verify comes in (7). So the hub can reach my callback, but for some reason the I don't receive the GET from the hub until after it has timed out?
I am new to rails so not sure how things work, but I'm guessing something is going on with the concurrency of requests in the middle of ActiveRecord creation. The subscription works on its own, but not during an ActiveRecord creation as I have set it up in after_create.
What is the correct way to handle HTTP request to 3rd party servers if you want to do it when an ActiveRecord entry is created?
NOTE** in the superfeedr api you can specify the verification intent to happen asynchronously. When I specify it to be async I do get a success response instead, but the subscription still does not seem to be crated on superfeedr's hub. I will contact superfeedr directly about this issue, but in general I want to know how to handle this situation for API's that do not have an async option.

Answer 1

The problem is a well known issue with Rails in the "devlopment" environment which only allows for one single concurrent HTTP request. Since you're executing your after_create :subscribe_feed during one HTTP request, when Superfeedr tries to verify your intent, it will issue a second request to your input during the first one. The verification will hang until the subscription request is done... triggering some kind of deadlock.
The solution is simple: use the hub.verify=async param when subscribing and Superfeedr will first close the subscription request (with a 202 status) and will then issue the verification of intent.
You could also drop the hub.verify param, as it's async by default.