I'm getting a body hash mismatch when the POST body of an XML web service request contains international characters.
From what I've read, it sounds like international characters in a POST body have to be encoded before calculating the OAuth body hash. UTF-8 for CAFÉ of "CAF%c3%89" doesn't seem to work with the MasterCard Match web service. I'm having trouble with the tool we're using (iWay Service Manager) re-interpreting "CAFÉ" back to "CAFÉ". Before I figure out how to squeeze an encoder in before the OAuth step, I was hoping to confirm with someone who had dealt with this issue. What is the proper encoding to use on a POST body with international characters (or is my problem likely to be something else)?
For calculating MasterCard OAuth body hash, the recommended encoding is UTF-8. Also the Core SDK made available by MasterCard uses UTF-8 encoder to encode oauth_body_hash.
Related
Colleagues from work have created API endpoint which uses language specific characters in url. This api url looks like
http://somedomain.com/someapi/somemethod/zażółć/gęślą/jaźń
Is this OK or is it a bad approach?
Technically, that's not a valid URL but web browsers and other clients finesse it. The script that characters are from is not an issue but structural characters like "/?#" could be. You'll have to consider what to do when they show up in data that you are "pasting" into your URLs.
An HTTP URL is:
an ASCII-encoded scheme (in this case the protocol "http")
a punycode-encoded, ASCII-encoded domain
a %-encoded, ASCII-encoded, server-defined sequence of octets for the path, optional query, and optional hash.
See RFC 3986
The assumption that everyone makes—quite reasonably because it is the predominant practice—is that the path, query, and hash are text. There is no text but encoded text. So, some character encoding is involved. Where %-encoding is needed outside of structural characters, browsers are going to assume UTF-8. If you don't want browsers to do the %-encoding, use valid URLs by doing it yourself with the character encoding that you are using.
As the world is standardizing on UTF-8 (where applicable), the HTML DOM has also with the encodeURIComponent function. Clients using JavaScript in a web browser are likely to use this function, either directly or through some library.
UTF-8 encoded, %-encoded (and, then on the wire, ASCII-encoded) version of your URL that my browser created:
http://somedomain.com/someapi/somemethod/za%C5%BC%C3%B3%C5%82%C4%87/g%C4%99%C5%9Bl%C4%85/ja%C5%BA%C5%84
(You can see this yourself using your browser's dev tools [F12 key, network tab] or a packet sniffer [e.g., Wireshark or Fiddler]. What you gave as a URL is never seen on the wire.)
Your server application probably understands that just fine. In any case, it is your server's rules that the client complies with. If your API uses UTF-8 encoded, %-encoded URLs then just document that. (But phrase it in a way that doesn't confuse people who do that already without knowing.)
I've got Google's oauth2 working on my app, where id_token is a string delimited with periods. I read here that the id_token is delimited into three parts and the second part contains the actual payload. If I split the string apart and decode the second value I get what I expect using my account. What I don't understand though is when I try on my wife's account if I parse that second part to json an "unexpected character" error.
I've tried grabbing the string from the console and running it through an online base64 decoder and do see the json doesn't get evaluated correctly.
{"iss":"accounts.google.com","sub":"111475728886332985448","azp":"74770364428-621332j2r657ish4jh94n9n1k0mplpgd.apps.googleusercontent.com","email":"her.email#gmail.com","at_hash":"lSKFL86HsCeu7TU4tsYBTw","email_verified":true,"aud":"74300369428-621332j2r657ish4jh94n9n1k0mplpgd.apps.googleusercontent.com","iat":1414192526,"exp":191819642���
What could be different between the two accounts that would cause my email to return valid json and hers not to?
An id_token is a JSON Web Token (JWT), in this case using compact serialization. JWT elements are base64url encoded with no padding, which is slightly different from plain base64 encoding as can be seen from: https://datatracker.ietf.org/doc/html/draft-ietf-jose-json-web-signature-38#appendix-C
A nice id_token/JWT decoder can be found here: http://jwt.io/
You may have been lucky for your own id_token so that it did not need padding.
In Rails 3 (Ruby 1.9.2) I send an request
Started GET "/controller/action?path=/41_+"
But the parameter list looks like this:
{"path"=>"/41_ ",
"controller"=>"controller",
"action"=>"action"}
Whats going wrong here? The -, * or . sign works fine, its just the +which will be replaced by a space.
That's normal URL encoding, the plus sign is a shorthand for a space:
Within the query string, the plus sign is reserved as shorthand notation for a space. Therefore, real plus signs must be encoded. This method was used to make query URIs easier to pass in systems which did not allow spaces.
And from the HTML5 standard:
The character is a U+0020 SPACE character
Replace the character with a single U+002B PLUS SIGN character (+).
For POST-requests, (in case that's how some of you stumbled upon this question, like me) one might encounter this problem because one has encoded the data in the wrong way on the client side. Encoding the data as application/x-www-form-urlencoded will tell rails to decode the data as it decodes a URL, and hence replace + signs with whitespace, according to the standard RFC1738 as explained by #mu is too short
The solution is to encode the data on the client side as multipart/form-data.
In PHP, using cURL, this is done by taking into consideration the following gotcha:
Passing an array to CURLOPT_POSTFIELDS will encode the data as
multipart/form-data, while passing a URL-encoded string will encode
the data as application/x-www-form-urlencoded. http://php.net/manual/en/function.curl-setopt.php
You might wonder why I was using PHP on the client side (that's because the client in my example was another webserver, since I'm working on an API connection.)
I'm encoding a query parameter using GWT's com.google.gwt.http.client.URL.encode() method, but have found I can't use URL.decode() on the server to decode it because the implementation isn't available (I suspect it uses the javascript client side implementation). I get...
java.lang.UnsatisfiedLinkError: com.google.gwt.http.client.URL.decodeImpl(Ljava/lang/String;)Ljava/lang/String;
Can someone suggest what I'm supposed to use server side to decode the encoded string?
I solved my problem this way: on the client side, I encode the parameters using com.google.gwt.http.client.URL.encodeQueryString(), like:
URL.encodeQueryString(param)
On the server side, I get the parameters using the ServletRequest methods, like:
String myParam = req.getParameter("myparam");
PS I initially +1'd Riley Lark's answer, but then I got some problems with some characters too... Letting the ServletRequest do the job will handle all character's encoding for you.
See Decoding international chars in AppEngine
java.net.URLDecoder is implemented on AppEngine and works perfectly with com.google.gwt.http.client.URL.encode().
If you're not willing to use gwt-rpc you can encode/decode with Base64. Check this link for a gwt implementation of the Base64 encoder/decoder. Then all you have to do is Base64.encode(yourParameterValue) before sending the request to the server and Base64.decode(request.getParameter(yourParameterName)) on the backend right after receiving the request.
cheers!
How many characters long can an oauth access token and oauth access secret be and what are the allowed characters? I need to store them in a database.
I am not sure there are any explicit limits. The spec doesn't have any.
That said, OAuth tokens are often passed as url parameters and so have some of the same limitations. ie need to be properly encoded, etc.
OAuth doesn't specify the format or content of a token. We simply use encrypted name-value pairs as token. You can use any characters in token but it's much easier to handle if the token is URL-safe. We achieve this by encoding the ciphertext with an URL-safe Base64.
As most people already pointed out. The OAuth specification doesn't give you exact directions but they do say...
cited from: https://datatracker.ietf.org/doc/html/draft-hammer-oauth-10#section-4.9
"Servers should be careful to assign
shared-secrets which are long enough,
and random enough, to resist such
attacks for at least the length of
time that the shared-secrets are
valid."
"Of course, servers are urged to err
on the side of caution, and use the
longest secrets reasonable."
on the other hand, you should consider the maximum URL length of browsers:
see: http://www.boutell.com/newfaq/misc/urllength.html
If you read the spec, it says,
The authorization server issues the registered client a client
identifier - a unique string representing the registration
information provided by the client. The client identifier is not a
secret; it is exposed to the resource owner, and MUST NOT be used
alone for client authentication. The client identifier is unique to
the authorization server.
The client identifier string size is left undefined by this
specification. The client should avoid making assumptions about the
identifier size. The authorization server SHOULD document the size
of any identifier it issues.
Second, Access Token should be sent as header, not as a URL param.
Authorization: Bearer < token>.
An OAuth token is conceptually an arbitrary-sized sequence of bytes, not characters. In URLs, it gets encoded using standard URL escaping mechanisms:
unreserved = ALPHA, DIGIT, '-', '.', '_', '~'
Everything not unreserved gets %-encoded.
I'm not sure whether you just talk about the oauth_token parameter that gets passed around. Usually, additional parameters need to be stored and transmitted as well, such as oauth_token_secret, oauth_signature, etc. Some of them have different data types, for example, oauth_timestamp is an integer representing seconds since 1970 (encoded in decimal ASCII digits).
Valid chars for OAuth token are limited by HTTP header value restrictions as OAuth token is frequently sent in HTTP header "Authorization".
Valid chars for HTTP headers are specified by https://www.rfc-editor.org/rfc/rfc7230#section-3.2.6. Alternatively you may check HTTP header validating code of some popular HTTP client libs, for example see Headers.checkNameAndValue() util of OkHttp framework: https://github.com/square/okhttp/blob/master/okhttp/src/main/java/okhttp3/Headers.java
And this is not all. I wouldn't include HTTP header separator (; and many others) and whitespace symbols (' ' and '\t') and double quote (") (see https://www.rfc-editor.org/rfc/rfc7230#section-3.2.6) as it would require to escape OAuth token before using in HTTP header. Frequently tokens are used by humans in curl test requests, and so good token generators don't add such characters. But you should check what characters may produce Oauth token generator with which your service is working before making any assumptions.
To be specific, even if Oauth spec doesn't say anything, if you are using java and mysql then it will be 16 characters as we generally generate the tokens using UUID and store it as BINARY(16) in the database. I know these details as I have recently done the development using OAuth.