I am passing Base64 encoded String to API call, but when we receive in API it shows as space .
Eg.) Passing like this "kv+lluLOKRkGK6v+BqNPAPsx" But in API response
receive "kv lluLOKRkGK6v BqNPAPsx" like this.
Can anyone explain why swift not sending "+" symbol in API.
And Please tell how to solve this.
Use the proper Base64 variant that does not have the + character for REST calls: RFC 4648 base64url.
If you can't change the encoding, simply transcode the Base64 you receive.
Related
Currently we are using SendGrid Inbound Parse to receive emails.
We handle the Inbound Parse webhook request by Azure HttpTrigger function implmented in C# (.NET 6).
When the received email is in UTF-8 encoding, everything's okay.
However, when we tried to receive email in shift_jis encoding, headers are okay,
but japanese characters in text and html are garbled.
From Inbound Parse request, we got the charsets as below:
subject: UTF-8
to: UTF-8
from: UTF-8
cc: UTF-8
html: shift_jis
text: shift_jis
And the string we got directly from request.form["text"] (or "html") was already garbled like "�e�L�X�gshiftJis-007"
(should be "テキストshiftJis-007"), so we cannot use string in request directly.
Then we tried to convert (System.Text.Encoding.Convert method) it from charset encoding (shift_jis) to utf-8,
and the result was different from original string but still unreadable "?e?L?X?gshiftJis-007".
Our questions are:
When using C# HttpTrigger Azure function to handle Inbound Parse webhook request
(request data is passed through AspNetCore.)
What encoding is in html/text string in Inbound Parse webhook request
when the email is send in encoding other than UTF-8?
How to read text and html in shift_jis encoding (or other encodings excluding UTF-8)
correctlyfrom an Inbound Parse webhook request?
Twilio Developer Evangelist here. I would recommend reaching out to the support team because it requires to investigate the payload to figure out what is going on.
I also tried to replicate the issue on my end with using send_raw option. Here's the payload, and it does contain shift_jis characters. You may be able to process the payload manually.
(stripped X-Mailer info)
'Content-Type: text/plain; charset="shift_jis"\n' +
'X-Mailer: \n' +
'Content-Transfer-Encoding: quoted-printable\n' +
'\n' +
'\n' +
'=83e=83L=83X=83gshiftJis-007\n'
I am trying to follow the guide to generate Oauth authentication tokens for YAHOO DSP API.
Base64 encoding is a way of encoding binary data into text so that it can be easily transmitted across a network without error.
In this step, you will take the client ID and client secret that the YDN console generated for you and encode them using the base64 protocol. You can use an online encoding service like base64encode.org.
No matter which service you use, ensure that no spaces are appended to the CLIENT_ID and CLIENT_SECRET keys and separate the CLIENT_ID and CLIENT_SECRET with a colon, i.e. CLIENT_ID:CLIENT_SECRET.
The generated value will now be referenced as ENCODED(CLIENT_ID:CLIENT_SECRET) in this guide.
An example is given:
CLIENT_ID = dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA–
CLIENT_SECRET= a7e13ea3740b933496d88755ff341bfb824805a6
AUTHORIZATION = ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5BLS06YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
Using the recommended website I get the wrong AUTHORIZATION.
I have tried both encoding the whole thing at once ie. encode(CLIENT_ID:CLIENT_SECRET), and each element individually encode(CLIENT_ID):encode(CLIENT_SECRET).
Attempt encoding whole thing:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5B4oCTOiBhN2UxM2VhMzc0MGI5MzM0OTZkODg3NTVmZjM0MWJmYjgyNDgwNWE2
Attempt encoding each element:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5B4oCT:YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
Expected result:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5BLS06YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
The difference between 'each element' and the expected result is only a few characters corresponding to the end of client_ID and the colon.
B4oCT: should be BLS06.
Links to full documentation:
https://developer.yahoo.com/dsp/api/docs/authentication/tokens.html
https://developer.yahoo.com/dsp/api/docs/traffic/info/sandbox.html
Update:
The final character of Client_ID is '–' . This is some sort of non-standard character that is interpreted as two dashes i.e.'--' in utf-8 and windows 1258.
One different, TO NOTE is, that when you decrypt the expected output you will get your client id as
dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA--
instead of
dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA–
NOTE, there are two "-" at the end.
OAuth client auth token is always generated using Base64 encoding with following format
Base64_Encoding(CLIENT_ID:CLIENT_SECRET)
Most of the usage perform this Base64 encoding with encoding type as "UTF-8".
It looks like, Yahoo requires this token with different encoding. On "https://www.base64encode.org/" if you try to encode your "CLIENT_ID:CLIENT_SECRET" with "Windows-1254" as destination charset, you will receive the expected result. So, it looks like both encoding and decoding here is done keeping "Windows-1254" charset in place.
How to prevent a server from returning an error 400 code error when the URL contains % symbol using NGINX server?
Nginx configuration for my website:
....
rewrite ^/download/(.+)$ /download.php?id=$1 last;
....
When I tried to get access to this URL:
http://mywebsite.net/download/some-string-100%-for-example
I got this error:
400 Bad Request
With this url :
http://mywebsite.net/download/some-string-%25-for-example
it's work fine !
It's because it needs to be URL encoded first.
This will explain:
http://www.w3schools.com/tags/ref_urlencode.asp
URLs can only be sent over the Internet using the ASCII character-set.
Since URLs often contain characters outside the ASCII set, the URL has to be converted into a valid ASCII format.
URL encoding replaces unsafe ASCII characters with a "%" followed by two hexadecimal digits.
URLs cannot contain spaces. URL encoding normally replaces a space with a plus (+) sign or with %20.
The URL interpreter is confused to see a % without hexadecimals after it.
Why would you think of solving by changing Nginx configuration???
It's impossible to solve from the server side. It's a problem from the client side.
https://headteacherofgreenfield.wordpress.com/2016/03/23/100-celebrations/
In that URL, the title is 100% Celebrations! but the permalink is autogenerated to 100-celebrations. It's because they know putting 100% will cause a URL encode problem.
If even Wordpress doesn't do it your way, then why should you do it?
I am trying to fetch product related data from Amazon Product API using this URL mentioned here:
http://docs.aws.amazon.com/AWSECommerceService/latest/DG/ItemLookup.html
Item Lookup URL:
http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[AWS_Access_Key_ID]&Operation=ItemLookup&ItemId=B00008OE6I&Timestamp=[YYYY-MM-DDThh:mm:ssZ]&Signature=[Request_Signature]
I have entered required details/parameters in URL but I am unable to understand where to get this Signature from or how to generate this signature? Any help would be appreciated.
The easiest and fastest way would be using one of AWS SDKs. You can find the full list here: https://aws.amazon.com/tools/
When you use the SDK you just supply your access key and secret key. The libraries handle preparing the signature.
Well, there's no point in doing this by yourself in Ruby, once it has a SDK, but an explanation about signature process may be useful for some people.
Signature is the way AWS authenticates API requests. It is passed as a query string parameter on your HTTP request and is validated against your own request content and you credentials on server-side.
There are 2 versions of it: 2 and 4, explained here and here. Version 2 is not supported on newer regions (China and Frankfurt).
In version 2, which I'm more familiar with, the signature is generated based on a string with this format:
HTTPVerb + "\n" + ValueOfHostHeaderInLowercase + "\n" + HTTPRequestURI
+ "\n" + CanonicalizedQueryString
The CanonicalizedQueryString is your query string, with parameters sorted (UTF-8 byte order) by paramater names and with parameter values url-encoded.
Then you calculate an HMAC of that string, using your AWS secret key, and SHA256 or SHA1 as the hash algorithm (depending on which one you have specified on query string parameter SignatureMethod), and encode it to base64. The result of all this will be the value of the Signature parameter.
I will try to walk you through a CloudWatch request, generated manually with bash tools.
Let's say you want to list you CloudWatch metrics. The request (without the signature) would be like the following.
http://monitoring.amazonaws.com/?AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
So, your string to sign would be:
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
Well, let's put it in a variable and do all the magic (hmac + base64)
#!/bin/bash
# your secret key
key='ry+SyjvXLvi4lYie/QfZYnxYOBQDggf01DTUxFSK'
# using 'read' to assign the signing string
# to a variable for readability reasons
read -r -d '' payload <<EOF
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG\
&Action=ListMetrics\
&SignatureMethod=HmacSHA1\
&SignatureVersion=2&\
Timestamp=2015-08-20T22%3A40%3A00.000Z\
&Version=2010-08-01
EOF
echo -n "$payload" | openssl dgst -sha256 -hmac $key -binary | openssl enc -base64
Then, after all this, you will end up with something like this:
xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM=
This is your signature, and finally, you just need to url-encoded it and append it to your request as one more query-string parameter.
&Signature=xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM%3D
Hopefully, if you did everything correctly (putting the sign string together, generating the hash and encoding it), your request will be accepted and, if you have enough permission, executed.
So, as you can see, it's something not worth to do by yourself. Use the SDK. ;)
Thank You guys for your response. But I have finally figured out my problem and my solution was in this gem:
https://github.com/jugend/amazon-ecs
This is what I was actually looking for and it finally helped me achieve what I was trying to. Posting it over here so people can find a better solution.
I have a service which returns ' 6"_x_5" ' as part of its json response, but active resource interprets this as ' 6���_x_4��� '.
When i curl against the same url the response looks fine, any idea how to fix this?
this is an encoding issue. what encoding is the app set to use? you may need to encode the value or escape them.
e.g.
'6\"_x_5\"'