I have a string in my Redis DB containing encoded chars with accents:
hum... probl\xc3\xa0me :(
This is string is correctly read from DB (using node-redis)
hum... problème :(
But when it is sent to client (iOS app), utf8 encoding appears.
UPDATE
After some hours of struggle, I figure this out.
The retrieval from DB is ok, then the transport to the client leads to those encoded chars to be added. On client side I then correct the values , based on this SOP answer
Related
I am building a Web API for which the plan is to do OAuth Authentication using Azure AD. The API code is using Microsoft's Owin library and Identity.Web to validate the token once received in HTTP Request header. The challenge is that when I generate client secret from Azure AD and provide that in Web API code to validate received token, there is an error -
The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
Have tried generate client secret multiple times but all trial has resulted in invalid base64 secret. Is there a way to generate base64 compatible client secret from Azure AD because MS Own library needs it that way? Can Azure AD be used for OAuth authentication?
Thanking all in advance.
The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
The above error occurs that when your token is not a valid Base64 encoded string and also you should verify that the base64string length is a multiple of 4 and that it is valid.
In order to correct this, you may have to try to pad it and change - to + and to / as follows:
public static byte[] DecodeUrlBase64(string s)
{
s = s.Replace('-', '+').Replace('_', '/').PadRight(4*((s.Length+3)/4), '=');
return Convert.FromBase64String(s);
}
Your code required encodedbase64 format but token is decoded base64 this may cause the error so try the above code.
Reference:
c# - The input is not a valid Base-64 string as it contains a non-base 64 character - Stack Overflow
Currently we are using SendGrid Inbound Parse to receive emails.
We handle the Inbound Parse webhook request by Azure HttpTrigger function implmented in C# (.NET 6).
When the received email is in UTF-8 encoding, everything's okay.
However, when we tried to receive email in shift_jis encoding, headers are okay,
but japanese characters in text and html are garbled.
From Inbound Parse request, we got the charsets as below:
subject: UTF-8
to: UTF-8
from: UTF-8
cc: UTF-8
html: shift_jis
text: shift_jis
And the string we got directly from request.form["text"] (or "html") was already garbled like "�e�L�X�gshiftJis-007"
(should be "テキストshiftJis-007"), so we cannot use string in request directly.
Then we tried to convert (System.Text.Encoding.Convert method) it from charset encoding (shift_jis) to utf-8,
and the result was different from original string but still unreadable "?e?L?X?gshiftJis-007".
Our questions are:
When using C# HttpTrigger Azure function to handle Inbound Parse webhook request
(request data is passed through AspNetCore.)
What encoding is in html/text string in Inbound Parse webhook request
when the email is send in encoding other than UTF-8?
How to read text and html in shift_jis encoding (or other encodings excluding UTF-8)
correctlyfrom an Inbound Parse webhook request?
Twilio Developer Evangelist here. I would recommend reaching out to the support team because it requires to investigate the payload to figure out what is going on.
I also tried to replicate the issue on my end with using send_raw option. Here's the payload, and it does contain shift_jis characters. You may be able to process the payload manually.
(stripped X-Mailer info)
'Content-Type: text/plain; charset="shift_jis"\n' +
'X-Mailer: \n' +
'Content-Transfer-Encoding: quoted-printable\n' +
'\n' +
'\n' +
'=83e=83L=83X=83gshiftJis-007\n'
I am trying to follow the guide to generate Oauth authentication tokens for YAHOO DSP API.
Base64 encoding is a way of encoding binary data into text so that it can be easily transmitted across a network without error.
In this step, you will take the client ID and client secret that the YDN console generated for you and encode them using the base64 protocol. You can use an online encoding service like base64encode.org.
No matter which service you use, ensure that no spaces are appended to the CLIENT_ID and CLIENT_SECRET keys and separate the CLIENT_ID and CLIENT_SECRET with a colon, i.e. CLIENT_ID:CLIENT_SECRET.
The generated value will now be referenced as ENCODED(CLIENT_ID:CLIENT_SECRET) in this guide.
An example is given:
CLIENT_ID = dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA–
CLIENT_SECRET= a7e13ea3740b933496d88755ff341bfb824805a6
AUTHORIZATION = ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5BLS06YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
Using the recommended website I get the wrong AUTHORIZATION.
I have tried both encoding the whole thing at once ie. encode(CLIENT_ID:CLIENT_SECRET), and each element individually encode(CLIENT_ID):encode(CLIENT_SECRET).
Attempt encoding whole thing:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5B4oCTOiBhN2UxM2VhMzc0MGI5MzM0OTZkODg3NTVmZjM0MWJmYjgyNDgwNWE2
Attempt encoding each element:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5B4oCT:YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
Expected result:
ZGoweUptazlOMnBJYXpsc1prMWlUekl4Sm1ROVdWZHJPV1ZFVW1wVk1GcFdUWHBSYldOSGJ6bE5RUzB0Sm5NOVkyOXVjM1Z0WlhKelpXTnlaWFFtZUQwME5BLS06YTdlMTNlYTM3NDBiOTMzNDk2ZDg4NzU1ZmYzNDFiZmI4MjQ4MDVhNg==
The difference between 'each element' and the expected result is only a few characters corresponding to the end of client_ID and the colon.
B4oCT: should be BLS06.
Links to full documentation:
https://developer.yahoo.com/dsp/api/docs/authentication/tokens.html
https://developer.yahoo.com/dsp/api/docs/traffic/info/sandbox.html
Update:
The final character of Client_ID is '–' . This is some sort of non-standard character that is interpreted as two dashes i.e.'--' in utf-8 and windows 1258.
One different, TO NOTE is, that when you decrypt the expected output you will get your client id as
dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA--
instead of
dj0yJmk9N2pIazlsZk1iTzIxJmQ9WVdrOWVEUmpVMFpWTXpRbWNHbzlNQS0tJnM9Y29uc3VtZXJzZWNyZXQmeD00NA–
NOTE, there are two "-" at the end.
OAuth client auth token is always generated using Base64 encoding with following format
Base64_Encoding(CLIENT_ID:CLIENT_SECRET)
Most of the usage perform this Base64 encoding with encoding type as "UTF-8".
It looks like, Yahoo requires this token with different encoding. On "https://www.base64encode.org/" if you try to encode your "CLIENT_ID:CLIENT_SECRET" with "Windows-1254" as destination charset, you will receive the expected result. So, it looks like both encoding and decoding here is done keeping "Windows-1254" charset in place.
How to prevent a server from returning an error 400 code error when the URL contains % symbol using NGINX server?
Nginx configuration for my website:
....
rewrite ^/download/(.+)$ /download.php?id=$1 last;
....
When I tried to get access to this URL:
http://mywebsite.net/download/some-string-100%-for-example
I got this error:
400 Bad Request
With this url :
http://mywebsite.net/download/some-string-%25-for-example
it's work fine !
It's because it needs to be URL encoded first.
This will explain:
http://www.w3schools.com/tags/ref_urlencode.asp
URLs can only be sent over the Internet using the ASCII character-set.
Since URLs often contain characters outside the ASCII set, the URL has to be converted into a valid ASCII format.
URL encoding replaces unsafe ASCII characters with a "%" followed by two hexadecimal digits.
URLs cannot contain spaces. URL encoding normally replaces a space with a plus (+) sign or with %20.
The URL interpreter is confused to see a % without hexadecimals after it.
Why would you think of solving by changing Nginx configuration???
It's impossible to solve from the server side. It's a problem from the client side.
https://headteacherofgreenfield.wordpress.com/2016/03/23/100-celebrations/
In that URL, the title is 100% Celebrations! but the permalink is autogenerated to 100-celebrations. It's because they know putting 100% will cause a URL encode problem.
If even Wordpress doesn't do it your way, then why should you do it?
I am dealing with strings in my project where in i send some data (string) to a service and the service responses me XML with some data (String).Earlier i used to face probs while sending special charcters like &,",',<,>,$,%,(,) etc in the string.
Then i encoded the string before sending as below:
NSString *str=#"£&#)(;:/-.,?!'"[]{}#%^*+=Â¥$â¬><~||_~<.,?!'m"";
NSString *stringToSend=[str stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
then data sent is as below:
"%C2%A3%26%40%29%28%3B%3A%2F-.%2C%3F%21%27%22%5B%5D%7B%7D%23%25%5E%2A%2B%3D%C2%A5%24%E2%82%AC%3E%3C~%7C%7C_~%3C.%2C%3F%21%27m%22"
then the server handles the data as required.
While receving the data from server i do decoding as below:
NSString *strReceived=#"%C2%A3%26%40%29%28%3B%3A%2F-.%2C%3F%21%27%22%5B%5D%7B%7D%23%25%5E%2A%2B%3D%C2%A5%24%E2%82%AC%3E%3C~%7C%7C_~%3C.%2C%3F%21%27m%22";
NSString *str=[str stringByReplacingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
then data received is :£&#)(;:/-.,?!'"[]{}#%^*+=Â¥$â¬><~||_~<.,?!'m" as expected.
PROBLEM/ISSUE
Now if my string contains norwegian characters like å,ø,Ø,æ,etc.
the string in response contains (%E5 for å) (%E6 for æ)and (%D8 for Ø)and (%F8 for ø).
I get the string as invalid after ececution of "stringByReplacingPercentEscapesUsingEncoding:NSUTF8StringEncoding".
Can some one suggest how to handle all the charcters received in the xml and parse them perfectly.Plz help with ur inputs...
Tnx in advance.