My app is consuming an OAuth resource and, from time to time, an access token must be refreshed using its refresh token. To this end, I'm doing something like:
record = MyClientModel.find(...)
client = OAuthClient.new(record.access_token)
begin
tries ||= 2
client.do_something
rescue ExpiredOAuthToken => e
new_access_token, new_refresh_token =
client.refresh_token(record.refresh_token)
client.access_token = new_access_token
record.access_token = new_access_token
record.refresh_token = new_refresh_token
record.save
retry unless (tries -= 1).zero?
raise e
end
This code is designed to be run simultaneously in web requests and in worker processes but, obviously, it is not thread-safe, e.g.:
record.access_token expires
Thread A encounters ExpiredOAuthToken
Thread A calls #refresh_token
Thread B encounters ExpiredOAuthToken
Thread B calls #refresh_token but fails because record.refresh_token is now invalid
Thread A persists new token
Thread A continues
I've never really had to think about thread safety before so I'm looking for suggestions on how I might go about improving this code.
I have recently come across this issue - in my case, I am storing my access tokens in an ActiveRecord model, so I can use the ActiveRecord with_lock method (with MySQL / PostgreSQL).
# ...inside Authorization model instance methods
def refresh
# do not allow more than one refresh to occur simultaneously
with_lock do
# if the token was just refreshed, return (there may be threads waiting on this token)
return oauth_token if updated_at > 5.seconds.ago
oauth = OmniAuth::Strategies::Custom.new(nil)
token =
OAuth2::AccessToken.new(
oauth.client,
oauth_token,
refresh_token: oauth_refresh_token
)
new_token = token.refresh!
return unless new_token
update!(
oauth_token: new_token.token,
oauth_refresh_token: new_token.refresh_token,
oauth_expires_at: Time.at(new_token.expires_at)
) && new_token.token
end
end
When my threads encounter an expired token, they will all call refresh, but only the first refresh is executed, the other threads wait, and will immediately get the new token when it is updated.
Related
I have a hobby Python/Flask web application on PythonAnywhere.com to develop functionality that Spotify doesn't natively support. My web app works as expected after manually reload the web app (via the "Web" tab on PythonAnywhere). However, the problem I'm encountering is that after a period of time, when API calls are made via the front-end web pages, an "Internal Server Error" is returned.
I traced the problem and found the calls to the Spotify APIs were returning a 401 error response after about an hour - and then found that Spotify access tokens only last 1 hour.
One troubleshooting theory was that maybe pythonanywhere.com was only running the code to refresh my access tokens once (when reloading the web app), so then they would expire after 60 minutes. So I tried making the access token refresh into a function, and then calling that function from other functions that make the API calls. But this didn't work, but here's below is my code to show what I did.
My only remaining theory now is that the web server only gets the access tokens once (when the web server is refreshed) - but I'm not quite sure how to troubleshoot that.
Any thoughts on that theory, or other avenues I should explore so that I don't need to manually restart the web server to get my app to work?
set variables
AUTH_URL = 'https://accounts.spotify.com/api/token'
BASE_URL = 'https://api.spotify.com/v1/'
Code to get access token and return headers for the API call
auth_response = requests.post(AUTH_URL, {
'grant_type': 'client_credentials',
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET,
})
def setHeaders():
try:
auth_response.raise_for_status()
except requests.exceptions.HTTPError as e:
print("!!!Error setting auth_response. System detail:")
print(e)
raise
# convert the response to JSON
auth_response_data = auth_response.json()
# save the access token
access_token = auth_response_data['access_token']
headers = {
'Authorization': 'Bearer {token}'.format(token=access_token)
}
return headers
function that makes the call to Spotify API that returns all playlist for a user:
#API GET link: https://api.spotify.com/v1/users/{user_id}/playlists
def getRawPlaylistMetadata(username,offset_value=0):
headers = setHeaders()
r_list = []
result_limit = 50
#print(offset_value)
params = {'offset': offset_value,
'limit': result_limit}
r = requests.get(BASE_URL + 'users/' + username + '/playlists', \
headers=headers,params=params)
response_code = str(r.status_code)
sys.stderr.write("\nr response code=" + response_code + "\n\n")
r = r.json()
try:
r_list = r['items']
except KeyError:
print("Getting all of Spotify user's playlist metadata failed.")
if len(r_list) < result_limit:
#print(len(r_list))
return r_list
return r_list + getRawPlaylistMetadata(username,offset_value + len(r_list))
I'm not sure how running it in the terminal does not cause the same issue, as your function doesn't even refresh the session:
# REQUEST EXECUTED HERE
auth_response = requests.post(AUTH_URL, {
'grant_type': 'client_credentials',
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET,
})
def setHeaders():
# Function ran here
# the request is not re-executed
...
So you should probably put it inside the function:
def setHeaders():
auth_response = requests.post(AUTH_URL, {
'grant_type': 'client_credentials',
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET,
})
...
Then, to keep re-authenticating while running the Flask server, I recommend using a Thread:
from time import sleep
def reauth():
while True:
setHeaders()
sleep(30 * 60) # Sleep for half an hour, then re-authenticate
t = Thread(target=reauth)
t.start()
This would keep it running in parallel with the Flask server, and would keep calling it to reauthenticate the client every half an hour (obviously configurable, but should be under the 1 hour limit for Spotify's authentication expiry).
I sell products online through a website I wrote. To manage my fulfilment flow, when a purchase is made I want my app to automatically create a card on a Trello board.
I've managed to do everything okay except that after a few minutes the token that I was using expires, even though I thought I had created a token that would never expire.
I can't manually authenticate every time an order comes in.
Here's the code I've written to generate tokens. (Oauth1).
Step 1 (one time): Get a manually authorized resource owner key, resource owner secret, and verifier.
import requests
from requests_oauthlib import OAuth1Session
oauth = OAuth1Session(CLIENT_KEY, client_secret=CLIENT_SECRET)
fetch_response = oauth.fetch_request_token(REQUEST_TOKEN_URL)
resource_owner_key = fetch_response.get('oauth_token')
resource_owner_secret = fetch_response.get('oauth_token_secret')
print(f'resource_owner_key: {resource_owner_key}')
print(f'resource_owner_secret: {resource_owner_secret}')
auth_url = oauth.authorization_url(AUTHORIZE_TOKEN_URL, scope='read,write', expiration='never') # expiration never
print(auth_url)
# Now manually authenticate in browser using this URL. Record resource owner key, secret and verifier
Step 2 (every time): Use resource owner key, resource owner secret, and verifier to generate a token.
oauth = OAuth1Session(CLIENT_KEY,
client_secret=CLIENT_SECRET,
resource_owner_key=RESOURCE_OWNER_KEY,
resource_owner_secret=RESOURCE_OWNER_SECRET,
verifier=VERIFIER)
oauth_tokens = oauth.fetch_access_token(ACCESS_TOKEN_URL)
token = oauth_tokens.get('oauth_token')
Step 3: Use token in POST request to make card.
This all works fine for a few minutes, then on trying to use it again I get the error:
requests_oauthlib.oauth1_session.TokenRequestDenied: Token request failed with code 500, response was 'token not found'.
I thought that token was last forever? I can still see under my account details on Trello:
read and write access on all your boards
read and write access on all your teams
Approved: today at 6:30 AM
Never Expires
Set expiration long expiration time in token like expire in 2099 something like that
Solved - I was doing everything right, just that Step 2 should only be done once instead of every time. I thought I had to generate a new token for each new request, but the token generated at the 'token = ' line is actually good to save off and use forever.
I have a rails app running on heroku from where I need to create Box.com! folder on an item creation callback using gem for the Box Content API!
The Standard OAuth 2.0 (User Authentication) of box api providing token which last an hour. I need the token alive for all the time so that the app can create box folder anytime from the app.
Recently, implemented box webhook feature as well.
I have tried couple of ways below but nothing help:
Token refresh callback as suggested the boxr gem!
token_refresh_callback = lambda {|access, refresh, identifier|
Setting.box_access_token = access
Setting.box_refresh_token = refresh
}
#client = Boxr::Client.new(
Setting.box_access_token,
refresh_token: Setting.box_refresh_token,
client_id: Setting.box_client_id,
client_secret: Setting.box_client_secret, &token_refresh_callback
)
Called a method before initialisation to update token
unless (Time.now.to_i >= Setting.box_token_expires_in.to_i - 300)
token = Boxr::refresh_tokens(Setting.box_refresh_token, client_id: Setting.box_client_id, client_secret: Setting.box_client_secret)
Setting.box_access_token = token.access_token
Setting.box_refresh_token = token.refresh_token
Setting.box_token_expires_in = Time.now.to_i + token.expires_in.to_i
end
Used scheduler which basically call a method to perform what did in previous step.
Step 2 were working before but sometimes have got refresh token expired exception. suddenly it does not work, require to manually reset token in every hour. Not sure but it might started after implementing box webhook feature.
Would be happy to have suggestion/solution to keep the token alive...
Let's say I have a method that provides access to an API client in the scope of a user and the API client will automatically update the users OAuth tokens when they expire.
class User < ActiveRecord::Base
def api
ApiClient.new access_token: oauth_access_token,
refresh_token: oauth_refresh_token,
on_oauth_refresh: -> (tokens) {
# This proc will be called by the API client when an
# OAuth refresh occurs
update_attributes({
oauth_access_token: tokens[:access_token],
oauth_refresh_token: tokens[:refresh_token]
})
}
end
end
If I consume this API within a Rails transaction and a refresh occurs and then an error occurs - I can't persist the new OAuth tokens (because the proc above is also treated as part of the transaction):
u = User.first
User.transaction {
local_info = Info.create!
# My tokens are expired so the client automatically
# refreshes them and calls the proc that updates them locally.
external_info = u.api.get_external_info(local_info.id)
# Now when I try to locally save the info returned by the API an exception
# occurs (for example due to validation). This rolls back the entire
# transaction (including the update of the user's new tokens.)
local_info.info = external_info
local_info.save!
}
I'm simplifying the example but basically the consuming of the API and the persistence of data returned by the API need to happen within a transaction. How can I ensure the update to the user's tokens gets committed even if the parent transaction fails.
Have you tried opening a new db connection inside new thread, and in this thread execute the update
u = User.first
User.transaction {
local_info = Info.create!
# My tokens are expired so the client automatically
# refreshes them and calls the proc that updates them locally.
external_info = u.api.get_external_info(local_info.id)
# Now when I try to locally save the info returned by the API an exception
# occurs (for example due to validation). This rolls back the entire
# transaction (including the update of the user's new tokens.)
local_info.info = external_info
local_info.save!
# Now open new thread
# In the new thread open new db connection, separate from the one already opened
# In the new connection execute update only for the tokens
# Close new connection and new thread
Thread.new do
ActiveRecord::Base.connection_pool.with_connection do |connection|
connection.execute("Your SQL statement that will update the user tokens")
end
end.join
}
I hope this helps
Nermin's (the accepted) answer is correct. Here's an update for Rails >= 5.0
Thread.new do
Rails.application.executor.wrap do
record.save
end
# Note: record probably won't be updated here yet since it's async
end
Documented here: Rails guides threading and concurrency
This discussion from a previous question might help you. It looks like you can set a requires_new: true flag and essentially mark the child transaction as a sub transaction.
User.transaction {
User.transaction(requires_new: true) {
u.update_attribute(:name, 'test')
};
u.update_attribute(:name, 'test2');
raise 'boom'
}
We have been connecting to Linkedin for awhile now successfully. However, we get some errors from time to time and I'm hoping someone can help shed some light on this. Here's our code:
def linkedin_login
request_token = Linkedin.client.request_token(oauth_callback: "http://#{SITE_URL}/linkedin/auth/")
session[:linkedin_request_token] = request_token.token
session[:linkedin_request_secret] = request_token.secret
redirect_to request_token.authorize_url
end
def linkedin_auth
raise "Don't have proper session or oauth_verifier" if session[:linkedin_request_token].blank? or session[:linkedin_request_secret].blank? or params[:oauth_verifier].blank?
access_token = Linkedin.client.authorize_from_request(session[:linkedin_request_token], session[:linkedin_request_secret], params[:oauth_verifier])
raise "Nil access token" if access_token.blank?
redirect_to linkedin_process_path(token: access_token.first, secret: access_token.second)
end
We're hitting the "raise 'Don't have proper session or oauth_verifier'" more than I would expect. When looking at the ENV for the errors, those people don't have the session values set from the original method. We have before_filters set on the application controller so initialize the session, so I know it's active.
My next thought was whether "request_token" was generating a value request_token, and I've tried many times and they all bring something back. We get many of these a day. After the error, if the user tries again, it works fine, which is why I'm so confused.
any thoughts on what could cause this?
Based on your code, it looks like you're making the request token call every time the user logs into your application. That's not the proper method to authenticate. You really only need to fetch the request token once, then use that to upgrade for an access token (as you're doing in your linkedin_auth method). From there, just save the access token and secret in your DB and fetch it anytime you need to make an API call for that particular user.
Our authentication is described more in detail here: https://developer.linkedin.com/documents/authentication
Also, this is just a personal preference, but I like using the OAuth gem for Rails as opposed to using a LinkedIn wrapper. It's easy to use and light weight.
Just as an example, you could do your auth this way:
require 'oauth'
def auth
api_key = 'XXXXXXXXX'
api_secret = 'XXXXXXXXX'
configuration = { :site => 'https://api.linkedin.com',
:authorize_path => 'https://www.linkedin.com/uas/oauth/authenticate',
:request_token_path => 'https://api.linkedin.com/uas/oauth/requestToken',
:access_token_path => 'https://api.linkedin.com/uas/oauth/accessToken' }
consumer = OAuth::Consumer.new(api_key, api_secret, configuration)
#Request token
request_token = consumer.get_request_token
# Output request URL to console
puts "Please visit this URL: https://api.linkedin.com/uas/oauth/authenticate?oauth_token=" + request_token.token + " in your browser and then input the numerical code you are provided here: "
# Set verifier code
verifier = $stdin.gets.strip
# Retrieve access token object
#access_token = request_token.get_access_token(:oauth_verifier => verifier)
end
You would only need to invoke this method when the user first authorizes your app. Save their access token then use it for subsequent API calls. Note, my example makes use of the console to enter the PIN verifier. In a real world example you'd want to programmatically save the PIN in a session variable or in memory, then use it to get the access token.