I already have an account in Twitter Dev, which allowed me to get a Consumer Key and secret, which I used to generate my token. Now I am trying to send token in request for Twitter API by using the operators "Get Page", "Jason to XML" and "Write Document". However, I do not know which URL to write in the "Get Page" operator. Is it Request token URL Authorize URL or Access token URL ?
In general the request URL contains the REST API call, while you are providing your authorization credentials e.g. via request properties in the HTTP header, OAuth mechanisms and the like. An REST API call is build up on a base URL awaiting a call, in case of twitter https://api.twitter.com/1.1/search/tweets.json, combined with a state and HTTP options forming your request. If your searching for tweets from a given user account (like #twitterapi) your request URL part would look like q=%40twitterapi. Here a key q is given with the value %40twitterapi, where %40 is a representation of the '#' symbol.
Providing key-value-pairs via GET using the HTTP protocol is initialized with a leading '?'. Hence your REST request would look like this:
https://api.twitter.com/1.1/search/tweets.json?q=%40twitterapi. This example is taken from the Twitter documentation.
That's about the URL.
For the RapidMiner site, I'd try to use the Enrich Data by Webservice operator. You can fill in the url parameter with your request URL and set the HTTP header for using an authentication token using the request properties parameter. As a property use Authorization and for the value you use a string starting with Bearer followed by your token. So a potential value could be Bearer 943582034-IH3k6hlskfdj32l4hks.
You can choose betweent different query types (e.g., String Matching, Regular Expression and others) to define the handling of the received payload given as an response to your request.
Before playing around with the operator, you can try using a tool like Postman (Chrome Plugin) or HTTPrequester (Firefox Add-on) to work directly with the HTTP GET request and its response.
Sorry for not including more references, but Stackoverflow doesn't allow for more than two if your reputation is below 10...
I concur with the above - use the Enrich Data via Webservice operator. You'll need to download the Web Mining extension in order to see it. If you re-post on the RapidMiner user community forum, you may get more RapidMiner-specific help as well.
Related
Sources such as this Okta sponsored site (see "Per-Request Customization" section) mention that the redirect_uri parameter of a autorization request SHOULD NEVER have a dynamic query part (ex: for session matching uses).
Quote:
The server should reject any authorization requests with redirect URLs
that are not an exact match of a registered URL.
Our OAuth AZ provider is BIG-IP F5. We are setting it up, and they seem to comply with the above view.
Our client is a web application built elsewhere, and they seem to not follow the above rule.
Here is a complete representation of the Authorization Endpoint (redacted):
https://ourownF5host.ca/f5-oauth2/v1/authorize?client_id=theIDofOurClient&redirect_uri=https%3A%2F%2FourClientAppHostname%2FClientRessource%2FRessource%3FSessionId%3D76eab448-52d1-4adb-8eba-e9ec1b9432a3&state=2HY-MLB0ST34wQUPCyHM-A&scope=RessourceData&response_type=code
They use a redirect_uri with a format similar to (I don't urlencode here, for simplicity's sake) : redirect_uri=https://ourClientAppHostname/ClientRessource/Ressource?SessionId=SOMELONGSESSIONID, with the SOMELONGSESSIONID value being DIFFERENT for each call.
We dug DEEP into RFC6749 (OAuth2), and found this in section 3.1.2.2:
The authorization server SHOULD require the client to provide the
complete redirection URI (the client MAY use the "state" request
parameter to achieve per-request customization). If requiring the
registration of the complete redirection URI is not possible, the
authorization server SHOULD require the registration of the URI
scheme, authority, and path (allowing the client to dynamically vary
only the query component of the redirection URI when requesting
authorization).
What I understand, and would like to validate here, is that the first source, Okta and F5 accept ONLY the first part of the rule above, and require the redirection uri to be COMPLETELY registered without any dynamic part.
Am I right to affirm that they (Okta and F5) DO NOT comply with the second part of the excerpt, citing that they should "allow(ing) the client to dynamically vary
only the query component of the redirection URI when requesting
authorization" ?
OR, is there any kind of official correction/evolution of the RFC6749, that warrants both companies design position ?
TL;DR:
No, the redirect uri must be static for security reasons. If the client needs to keep a state between the authorization request and its asynchronous response, use the OAuth 2.0 state parameter.
Long version :
RFC6749 (the initial OAuth 2.0 specification) has been published in 2012 and OAuth security landscape has evolved a lot since then.
RFC6819, an OAuth 2.0 security review from 2013 already mentioned that refusing dynamically crafted redirect uris was a good way to protect against XSS and client impersonation attacks.
OpenID Connect, from 2014, a commonly used extension of OAuth 2.0 with authentication capabilities, already takes that recommendation into account and mandates exact string matching for all redirect uris.
The current draft recommendation for OAuth 2.0 Best Security Practice confirms that by enforcing redirect_uris preregistration and mandating the use simple string comparison by the AS when validating the redirect_uri passed in the request. So a dynamic redirect_uri must not be used.
Your client definitely makes a wrong move by using the redirect_uri as a "state keeper" between the Authorization request and response, by using a dynamically crafted SessionID attribute inside the redirect_uri. OAuth2.0 has a dedicated authorization request parameter for that purpose, which is "state". The client should use it. The AS will append that state in the parameters of the redirect_uri when it issues the response, so the client will be able to find back this state inside the response.
The proper authorization request would be:
https://youras/authorize?client_id=your_client_id&response_type=code&state=SOMELONGSTATE&redirect_uri=https%3A%2F%2Fsomehost%2Fauthcallback
The response will look like:
https://somehost/authcallback?state=SOMELONGSTATE&code=anazcode
This way the redirect_uri is static, so a simple string comparison is enough to validate that uri on AS side. Any algorithm more complex than simple string comparison would be subject to security flaws.
It appears there's two things mixed up here: the URL you refer to:
https://somehost/authcallback?state=SOMELONGSTATE&scope=someobjecttype&response_type=code
suggests that you mixed up the Redirect URI (aka. callback URL, as suggested by the pathname in the URL) of the Client with the Authorization Endpoint of the Authorization Server.
Only the Authorization Endpoint would take parameters as suggested and may contain e.g. a dynamic state value. The Redirect URI of the Client would not contain e.g. response type.
A redirect_uri parameter can be added to the the authorization request, which would then have to be matched in the way that you described. After confirming the match, the Authorization Server will redirect back to the Redirect URI, adding the code and state parameters.
Update (after the question changed):
OAuth 2.0 RFC6749 allows a dynamic (SessionId) parameter though it is not considered best practice. If however your client is an OpenID Connect Client, this is not allowed since the OpenID Connect spec (a "profile" of OAuth 2.0) locks down the Redirect URI matching to "exact", in which case you'd have to configure all possible SessionIds.
LinkedIn api provides access token to use it in url to post a message which uses oauth2 in Rest client. So by using the url in my application, i can post the message.
Twitter api uses oauth1 to tweet by accepting parameter.But the problem is timestamp,nounce,oauth_signature parameters gets auto generated and keep on changing for every request in Rest client.Here url is not sufficient to tweet, it need parameters but 3 parameters keep on changing on Rest client.
I tried to pass every parameter in url, it throws
error: code 32, could not authenticate you
i am dependent only on url to send a tweet.My application supports parameters too but three parameters are auto generating.I cant auto generate bcz i am hardcoding it in my application.
Is there any solution to tweet by using url?
Is there any way to make those 3 parameters fixed?
It's hard to tell what's going on here. Perhaps you could post more of the code and what you expected to happen?
By definition the nonce is a number only used once, and shouldn't be hardcoded. The timestamp also should reflect the current time on each request.
https://en.wikipedia.org/wiki/Cryptographic_nonce
I am referring to the Figure 4 of the following link to understand the working behind it:
https://tools.ietf.org/id/draft-ietf-oauth-v2-31.html#grant-implicit
I cannot understand why there's a need for Web-Hosted Client Resource? Why doesn't the User-Agent simply pass the Access Token directly to the client?
Firstly, update your reference of OAuth 2.0 to the latest one.
As we know, in implicit grant,
after resource owner grants access, the authorization server redirects the user-agent back to the client using the redirection URI, and access token is in the fragment.
For example: http://www.myapp.com/googleapi/oauth/#access_token=ya29.JACdaU44_m0MQh0AAABLMVzZHm4KnUWyBECHJ9oM_0M2JC9x0xO6UoI9W8YNEw&token_type=Bearer&expires_in=3600
Since the fragment is not returned to the server (hash fragment is only intended for the client), client-side script must parse the fragment and extract the value of the access_token parameter.
Now, here comes to your questions, of course we can write a function in our client to parse the access token from the fragment and use it directly, it's simple and straightforward. Here is a tutorial using this manner.
But, in the standard, there is Web-Hosted Client Resource. Why?
'Web-Hosted Client Resource' is a client Resource, it may include some html pages and JavaScript, and of course it's web-hosted, rather than in the User-Agent. Since authorization server will hit our web application again at the redirect_uri with access token in fragment, our client-side web application server will respond it(parse the hash fragment).Here is a tutorial using this manner
In a word, the difference between these two manners is where you put the parse function.
Put it directly in the client it self, and the function detects if the user-agent hit the redirect URI, and if so, parses the access token from the fragment.
Put it in web-hosted client resource(locates in the redirect URI), when the authorization server hit the redirect URI with access token, the function locates the redirect URI parse it.
The second one is standard, for it makes the most of the redirect URI, also you can use the 1st one as well.
As far as I know, I haven't found any security consideration about the 2nd way.
When an OAuth signed request is made to a Rails 3 app, I can see the OAuth consumer key in amongst the other values in request.headers["HTTP_AUTHORIZATION"]. What is a better way to access it? I'm using the OAuth gem.
According to this section in the OAuth specification, the Authorization header is the preferred way of sending OAuth protocol parameters. The specification does make allowances for sending protocol parameters in a form encoded body or in the request URI, if the request meets certain requirements.
To answer your question: parsing the Authorization header is a must for any OAuth provider. But you may also look for it (and other protocol parameters) amongst "normal" parameters, you will never find them in more than one place,
I want to make sure I have correctly understood the (draft) spec, which states:
The redirection endpoint URI MUST be an absolute URI as defined by
[RFC3986] section 4.3. The endpoint URI MAY include an
"application/x-www-form-urlencoded" formatted
([W3C.REC-html401-19991224]) query component ([RFC3986] section 3.4),
which MUST be retained when adding additional query parameters. The
endpoint URI MUST NOT include a fragment component.
Reason I ask is that neither Google or Facebook appear to preserve any querystrings.
Re-reading the spec it appears that the quoted section of the spec applies not to the OAuth server's handling of URIs but the OAuth client's handling of the original endpoint URI it is given.
In other words it's saying that if I say that my OAuth endpoint which you have to use when redirecting to my server for an OAuth authorization is:
http://example.com/oauth.php?endpoint=token
Then when the client is adding the ?response_type=code&client_id=...&state=...&redirect_uri=... to the URI it is not permitted to discard the "?endpoint=token" in the original endpoint uri and MUST use the URI:
http://example.com/oauth.php?endpoint=token&response_type=code&client_id=...&state=...&redirect_uri=...
So, at least as far as that part of the spec goes there's nothing there saying that Facebook, Google, etc... have to preserve any unknown query arguments besides the 'state' one.
Technically you might be able to use the &state= parameter to pass along custom data in say JSON format. Though that may or may not work. IIRC I noticed that Meetup's implementation of OAuth 2 appears to mangle the state when you use special characters. Something that I believe is against spec.