I'm trying to send a POST request to a site (https://get.cbord.com/dartmouth/full/login.php) to log in. I think I'm running into issues with the format of the data I'm passing in, but truthfully I'm just getting started so I don't understand this very well. I tried deciphering the POST request using Chrome dev tools and this is what I came up with.
#!/usr/bin/env python
import requests
import sys
EMAIL = 'xxx'
PASSWORD = 'xxx'
LOGIN_URL = 'https://get.cbord.com/dartmouth/full/login.php'
FULL_URL = 'https://get.cbord.com/dartmouth/full/funds_home.php'
def main():
# Start a session so we can have persistant cookies
#session = requests.session(config={'verbose': sys.stderr})
session = requests.session()
# This is the form data that the page sends when logging in
login_data = {
'username': EMAIL,
'password': PASSWORD,
'submit': 'Login',
}
# Authenticate
r = session.post(LOGIN_URL, data=login_data)
# Try accessing a page that requires you to be logged in
r = session.get(FULL_URL)
if __name__ == '__main__':
main()
The majority of the code is directly copy pasted from a response to a similar question. I guess I just don't understand how to figure out what the login_data dictionary should be.
Thanks for the help!
Pat
I examined this page and found that there is one more parameter: formToken.
You can get its value from webpage source. It looks like these
So first you need to get https://get.cbord.com/dartmouth/full/login.php page
login_page = session.get(LOGIN_URL)
login_page_text = login_page.text
then parse the page to retrieve formToken value from it and finally authenticate
login_data = {
'username': EMAIL,
'password': PASSWORD,
'submit': 'Login',
'formToken': formToken_value
}
# Authenticate
r = session.post(LOGIN_URL, data=login_data)
Related
I'm using Sorcery gem with External submodule. For some reason I'm not getting an email back from Facebook and I'm pretty sure I have things configured correctly. I'm trying to troubleshoot this further but I can't figure out how to read what data IS being returned via oauth to verify where things are breaking down. Where can I pry in and read this info? Thanks!
Here is my sorcery config.
Rails.application.config.sorcery.submodules = [:external]
Rails.application.config.sorcery.configure do |config|
config.external_providers = [:facebook, :google]
config.facebook.key = "#{Rails.application.secrets.sorcery_facebook_key}"
config.facebook.secret = "#{Rails.application.secrets.sorcery_facebook_secret}"
config.facebook.callback_url = "#{Rails.application.secrets.sorcery_facebook_callback_url}"
config.facebook.user_info_path = "me?fields=email,first_name,last_name"
config.facebook.user_info_mapping = {:email => "email"}
config.facebook.access_permissions = ["email"]
config.facebook.scope = "email"
config.facebook.display = "popup"
config.facebook.api_version = "v2.5"
config.user_config do |user|
user.authentications_class = Authentication
end
config.user_class = User
end
Well, technically this answers the question of how to find out what is being returned.
Inside your oauth controller if you call access_token.get('me?fields=email') or whatever fields you're wanting you'll get a response with a URL field set. Copy that URL into a browser and you'll get a JSON list of your data. In my case I get nothing with email but I'm able to return first_name, last_name, name. Not quite sure why I still can't get email, but hopefully this helps somebody troubleshoot in the future.
Another way would be to build the URL yourself if you have the access_token available.
https://graph.facebook.com/me?access_token=<access token goes here>&fields=first_name,last_name,email
Access token is retrievable with #access_token.token from oauth controller.
UPDATE
So silly...I had the config correct, but apparently had never logged out of Facebook since I'd made the proper corrections. Logging out and having oauth connect again seems to have fixed things.
Google is deprecating the OpenID endpoint I was using (v1.0 I think, via the django_openid_auth module) and I need to update my app and migrate my users' accounts to use Google OAuth2.
I've changed the app to use python-social-auth and have it authenticating with social.backends.google.GoogleOAuth2 successfully.
I've written a pipeline function to find associated OpenID urls from the old table and this is working for the other backends I care about but Google:
def associate_legacy_user(backend, response, uid=None, user=None,
*args, **kwargs):
if uid and not user:
# Try to associate accounts registered in the old openid table
identity_url = None
if backend.name == 'google-oauth2':
# TODO: this isn't working
identity_url = response.get('open_id')
else:
# for all other backends, see if there is a claimed_id url
# matching the identity_url use identity_url instead of uid
# as uid may be the user's email or username
try:
identity_url = response.identity_url
except AttributeError:
identity_url = uid
if identity_url:
# raw sql as this is no longer an installed app
user_ids = sql_query.dbquery('SELECT user_id '
'FROM django_openid_auth_useropenid '
'WHERE claimed_id = %s',
(identity_url,))
if len(user_ids) == 1:
return {'user': User.objects.get(id=user_ids[0]['user_id'])}
As best I can tell from reading Google's migration guide, I need to add an openid.realm to the request, which I've done as follows in settings.py:
SOCIAL_AUTH_GOOGLE_OAUTH2_AUTH_EXTRA_ARGUMENTS \
= {'openid.realm': 'http://example.com/'}
But this doesn't seem to be returning the open_id value in the response passed into my pipeline function.
I seem to be stuck on Step 3:
I tried sub-classing the backend to change the RESPONSE_TYPE to add id_token but that returned an empty response:
import social.backends.google
class CustomGoogleOAuth2(social.backends.google.GoogleOAuth2):
RESPONSE_TYPE = 'code id_token'
I tried building an additional request to https://www.googleapis.com/oauth2/v3/token similar to this example, but I don't really know how to go about putting that together and debugging it.
Some more details:
My old claimed_ids for Google OpenID users look like: https://www.google.com/accounts/o8/id?id=AItOawmAW18QuHDdn6PZzaiI5BWUb84mZzNB9eo
I'm happy to use social.backends.google.GoogleOpenIdConnect or a similar alternative backend if that's an easier solution. And while it seems to be closer to what the Google docs are talking about, I wasn't able to get it to work when I tried:
I get a 400 Error: invalid_request Parameter not allowed for this message type: nonce
I can get past the nonce error using social.backends.google.GoogleOpenIdConnect by adding id_token to the RESPONSE_TYPE but then I get an AuthMissingParameter error in my /complete/google-openidconnect/ endpoint as the request's GET and POST are empty. (Tried 'code id_token', 'token id_token', 'id_token', ...)
I don't want to use social.backends.google.GooglePlusAuth as that doesn't integrate as nicely with my current login form.
Worst case, I should be able to use social.pipeline.social_auth.associate_by_email, but I only have email addresses for maybe 80% of the users so that leaves quite a few who will have a new account and need support to associate it manually.
Try as I might, I can't find any examples of people doing a similar migration with python-social-auth, but it must be happening to lots of people.
Any ideas?
Solution works for python social auth 0.1.26
In new versions (0.2.*) of python social auth, there is GoogleOpenIdConnect, but it does not work fine (at least I did not succeed). And my project has some legacy, so I can't use new version of social.
I wrote custom GoogleOpenIdConnect backend:
import datetime
from calendar import timegm
from jwt import InvalidTokenError, decode as jwt_decode
from social.backends.google import GoogleOAuth2
from social.exceptions import AuthTokenError
class GoogleOpenIdConnect(GoogleOAuth2):
name = 'google-openidconnect'
ACCESS_TOKEN_URL = 'https://www.googleapis.com/oauth2/v3/token'
DEFAULT_SCOPE = ['openid']
EXTRA_DATA = ['id_token', 'refresh_token', ('sub', 'id')]
ID_TOKEN_ISSUER = "accounts.google.com"
def user_data(self, access_token, *args, **kwargs):
return self.get_json(
'https://www.googleapis.com/plus/v1/people/me/openIdConnect',
params={'access_token': access_token, 'alt': 'json'}
)
def get_user_id(self, details, response):
return response['sub']
def request_access_token(self, *args, **kwargs):
"""
Retrieve the access token. Also, validate the id_token and
store it (temporarily).
"""
response = self.get_json(*args, **kwargs)
response['id_token_parsed'] = self.validate_and_return_id_token(response['id_token'])
return response
def validate_and_return_id_token(self, id_token):
"""
Validates the id_token according to the steps at
http://openid.net/specs/openid-connect-core-1_0.html#IDTokenValidation.
"""
try:
id_token = jwt_decode(id_token, verify=False)
except InvalidTokenError as err:
raise AuthTokenError(self, err)
# Verify the token was issued in the last 10 minutes
utc_timestamp = timegm(datetime.datetime.utcnow().utctimetuple())
if id_token['iat'] < (utc_timestamp - 600):
raise AuthTokenError(self, 'Incorrect id_token: iat')
return id_token
Notes:
get_user_id – An identifier for the user, unique among all Google accounts and never reused.
request_access_token – there is I add id_token_parsed to response, and it will be used in pipeline.
validate_and_return_id_token – validate of jwt is disabled, because in google developers console I have registered Client ID as web application so, I have no certificates for validate this data.
Then I created pipelines:
def social_user_google_backwards(strategy, uid, *args, **kwargs):
"""
Provide find user that was connect with google openID, but is logging with google oauth2
"""
result = social_user(strategy, uid, *args, **kwargs)
provider = strategy.backend.name
user = result.get('user')
if provider != 'google-openidconnect' or user is not None:
return result
openid_id = kwargs.get('response', {}).get('id_token_parsed', {}).get('openid_id')
if openid_id is None:
return result
social = _get_google_openid(strategy, openid_id)
if social is not None:
result.update({
'user': social.user,
'is_new': social.user is None,
'google_openid_social': social
})
return result
def _get_google_openid(strategy, openid_id):
social = strategy.storage.user.get_social_auth('openid', openid_id)
if social:
return social
return None
def associate_user(strategy, uid, user=None, social=None, *args, **kwargs):
result = social_associate_user(strategy, uid, user, social, *args, **kwargs)
google_openid_social = kwargs.pop('google_openid_social', None)
if google_openid_social is not None:
google_openid_social.delete()
return result
And changed my SOCIAL_AUTH_PIPELINE and AUTHENTICATION_BACKENDS settings:
AUTHENTICATION_BACKENDS = (
...
#'social.backends.open_id.OpenIdAuth' remove it
'social_extension.backends.google.GoogleOpenIdConnect', # add it
...
)
and
SOCIAL_AUTH_PIPELINE = (
'social.pipeline.social_auth.social_details',
'social.pipeline.social_auth.social_uid',
'social.pipeline.social_auth.auth_allowed',
# 'social.pipeline.social_auth.social_user', remove it
'social_extension.pipeline.social_user_google_backwards', # add it
'social.pipeline.user.get_username',
...
# 'social.pipeline.social_auth.associate_user', remove it
'social_extension.pipeline.associate_user', # add it
'social.pipeline.social_auth.load_extra_data',
...
)
Users can log in to my Rails app using their LinkedIn account thanks to OAuth. However, I am having trouble displaying the user's profile image. The following URL does not load a picture:
<%= image_tag("http://api.linkedin.com/v1/people/{user-id}/picture-url") %>
How can I get the user's LinkedIn profile image to display in my Rails app?
Thanks!
Try to get the original picture with:
http://api.linkedin.com/v1/people/{user-id}/picture-urls::(original)
Update:
From current docs (recommend to read it):
Using current user (after user logged in):
http://api.linkedin.com/v1/people/~:(picture-url)
Using member_id:
http://api.linkedin.com/v1/people/id=12345:(picture-url)
Public profile:
http://api.linkedin.com/v1/people/url=<public-profile-url>:(picture-url)
Those URLs return xml, so you could parse the xml response to get picture-url string and use it as a param for image_tag. Alternatively, you can retrieve info as a json passing an extra param like:
http://api.linkedin.com/v1/people/~:(picture-url)?format=json
In both cases (xml or json), you need to extract the picture-url from api response for passing it to image_tag.
This gem omniauth-linkedin-oauth2 could probably help you.
This is my solution working perfectly:
def callback(self):
self.validate_oauth2callback()
oauth_session = self.service.get_auth_session(
data={'code': request.args['code'],
'grant_type': 'authorization_code',
'redirect_uri': self.get_callback_url()},
decoder=jsondecoder
)
me = oauth_session.get('people/~:(id,first-name,last-name,public-profile-url,email-address,picture-url,picture-urls::(original))?format=json&oauth2_access_token='+str(oauth_session.access_token), data={'x-li-format': 'json'}, bearer_auth=False).json()
social_id = 'linkedin$' + me['id']
name = me['firstName']
surname = me['lastName']
email = me['emailAddress']
url = me['publicProfileUrl']
image_small = me.get('pictureUrl', None)
image_large = me.get('pictureUrls', {}).get('values', [])[0]
return social_id, name, surname, email, url, image_small, image_large, me
I know how to generate a user key using the pastebin API, but how can I use this userkey to access a raw private paste?
I am using Lua for this.
Obtaining the raw paste bin output is not part of of the Pastebin API:
This option is actually not part of our API, but you might still want to use it. To get the RAW output of a paste you can use our RAW data output URL:
http://pastebin.com/raw.php?i=
Simply add the paste_key at the end of that URL and you will get the RAW output.
Since private pastes can only be seen by the user who created them, my guess is that they use the logincookie for authentication. In that case, you'll need to send it with the HTTP request.
In respect to implementing this in Lua, (since you haven't said which library you're using) I'm gonna go forth and recommend the HTTP module in LuaSocket or the wonderful Luvit (http://luvit.io).
Here is a ready example of the code for you:
local https = require('ssl.https')
https.TIMEOUT= 15
local private_raw_url="https://pastebin.com/raw/YOURPAGE" -- Change raw link
local user_name, user_password = "USER", "PASS" -- and user with password
local request_body = "submit_hidden=submit_hidden&user_name=".. user_name .. "&user_password=" .. user_password .. "&submit=Login"
local resp = {}
local res, code, headers, status = https.request ( {
method = 'POST',
url = "https://pastebin.com/login",
headers = {
Host = "pastebin.com",
["Content-Type"] = "application/x-www-form-urlencoded",
["Content-Length"] = string.len(request_body),
Connection = "keep-alive",
},
source = ltn12.source.string(request_body),
sink = ltn12.sink.table(resp),
protocol = "tlsv1",
verify = "none",
verifyext = {"lsec_continue", "lsec_ignore_purpose"},
options = { "all", "no_sslv2", "no_sslv3" }
} )
if not headers['set-cookie']:find('pastebin_user') then
print('bad login')
return
end
resp={}
local cookie = headers['set-cookie'] or ''
local cookie1, cookie2, cookie3 = cookie:match("(__cfduid=%w+; ).*(PHPSESSID=%w+; ).*(pastebin_user=%w+; )" )
if cookie1 and cookie2 and cookie3 then
cookie = cookie1 .. cookie2 .. cookie3
body, code, headers= https.request{
url = private_raw_url ,
headers = {
--Host = "pastebin.com",
['Cookie'] = cookie,
['Connection'] = 'keep-alive'
},
sink = ltn12.sink.table(resp)
}
if code~=200 then return end
print( table.concat(resp) )
else
print("error match cookies!" )
end
I know that this is a little late to answer the question but I hope this will help someone later on.
If you want to access raw private pastes, you will first need to list the pastes that the user has created. This is a part of the API. This requires the user to be logged in.
With this API you can list all the pastes created by a certain user.
You will need send a valid POST request to the URL below to access the
data:
http://pastebin.com/api/api_post.php
The response that you will get will be an XML response, as follows:
<paste>
<paste_key>0b42rwhf</paste_key>
<paste_date>1297953260</paste_date>
<paste_title>javascript test</paste_title>
<paste_size>15</paste_size>
<paste_expire_date>1297956860</paste_expire_date>
<paste_private>0</paste_private>
<paste_format_long>JavaScript</paste_format_long>
<paste_format_short>javascript</paste_format_short>
<paste_url>http://pastebin.com/0b42rwhf</paste_url>
<paste_hits>15</paste_hits>
</paste>
Once you have that, parse the XML to get the paste_key and the paste_private. You need to check the value of paste_private because you want private pastes only. The documentation says:
We have 3 valid values available which you can use with the
'api_paste_private' parameter:
0 = Public
1 = Unlisted
2 = Private (only allowed in combination with api_user_key, as you have to be logged into your account to access the paste)
So, if your paste has paste_private set to 2, get the paste_key for it.
Once you have the paste_key, use the API call to get the RAW paste. No username or password required once you have the paste key for the private paste.
Have fun!
I have read this thoroughly: https://developers.google.com/google-apps/documents-list/#using_google_apps_administrative_access_to_impersonate_other_domain_users
I have googled this to death.
So far I have been able to:
Authorise with:
clientLogin
OAuth tokens (using my domain key)
retrieve document feeds for all users in the domain (authorised either way in #1)
I am using the "entry" from the feed to Export/Download documents and always get forbidden for other users for documents not shared with admin. The feed query I am using is like:
https://docs.google.com/feeds/userid#mydomain.com/private/full/?v=3
(I have tried with and without the ?v=3)
I have also tried adding the xoauth_requestor_id (which I have also seen in posts as xoauth_requestor), both on the uri, and as a client property: client.xoauth_requestor_id = ...
Code fragments:
Client Login (using administrator credentials):
client.http_client.debug = cfg.get('HTTPDEBUG')
client.ClientLogin( cfg.get('ADMINUSER'), cfg.get('ADMINPASS'), 'HOSTED' )
OAuth:
client.http_client.debug = cfg.get('HTTPDEBUG')
client.SetOAuthInputParameters( gdata.auth.OAuthSignatureMethod.HMAC_SHA1, cfg.get('DOMAIN'), cfg.get('APPS.SECRET') )
oatip = gdata.auth.OAuthInputParams( gdata.auth.OAuthSignatureMethod.HMAC_SHA1, cfg.get('DOMAIN'), cfg.get('APPS.SECRET') )
oat = gdata.auth.OAuthToken( scopes = cfg.get('APPS.%s.SCOPES' % section), oauth_input_params = oatip )
oat.set_token_string( cfg.get('APPS.%s.TOKEN' % section) )
client.current_token = oat
Once the feed is retrieved:
# pathname eg whatever.doc
client.Export(entry, pathname)
# have also tried
client.Export(entry, pathname, extra_params = { 'v': 3 } )
# and tried
client.Export(entry, pathname, extra_params = { 'v': 3, 'xoauth_requestor_id': 'admin#mydomain.com' } )
Any suggestions, or pointers as to what I am missing here?
Thanks
You were very close to having a correct implementation. In your example above, you had:
client.Export(entry, pathname, extra_params = { 'v': 3, 'xoauth_requestor_id': 'admin#mydomain.com' } )
xoauth_requestor_id must be set to the user you're impersonating. Also what you need is to use 2-Legged OAuth 1.0a with the xoauth_requestor_id set either in the token or in the client.
import gdata.docs.client
import gdata.gauth
import tempfile
# Replace with values from your Google Apps domain admin console
CONSUMER_KEY = ''
CONSUMER_SECRET = ''
# Set this to the user you're impersonating, NOT the admin user
username = 'userid#mydomain.com'
destination = tempfile.mkstemp()
token = gdata.gauth.TwoLeggedOAuthHmacToken(
consumer_key, consumer_secret, username)
# Setting xoauth_requestor_id in the DocsClient constructor is not required
# because we set it in the token above, but I'm showing it here in case your
# token is constructed via some other mechanism and you need another way to
# set xoauth_requestor_id.
client = gdata.docs.client.DocsClient(
auth_token=token, xoauth_requestor_id=username)
# Replace this with the resource your application needs
resource = client.GetAllResources()[0]
client.DownloadResource(resource, path)
print 'Downloaded %s to %s' % (resource.title.text, destination)
Here is the reference in the source code to the TwoLeggedOAuthHmacToken class:
http://code.google.com/p/gdata-python-client/source/browse/src/gdata/gauth.py#1062
And here are the references in the source code that provide the xoauth_requestor_id constructor parameter (read these in order):
http://code.google.com/p/gdata-python-client/source/browse/src/atom/client.py#42
http://code.google.com/p/gdata-python-client/source/browse/src/atom/client.py#179
http://code.google.com/p/gdata-python-client/source/browse/src/gdata/client.py#136