I have a service which returns ' 6"_x_5" ' as part of its json response, but active resource interprets this as ' 6���_x_4��� '.
When i curl against the same url the response looks fine, any idea how to fix this?
this is an encoding issue. what encoding is the app set to use? you may need to encode the value or escape them.
e.g.
'6\"_x_5\"'
Related
I am passing Base64 encoded String to API call, but when we receive in API it shows as space .
Eg.) Passing like this "kv+lluLOKRkGK6v+BqNPAPsx" But in API response
receive "kv lluLOKRkGK6v BqNPAPsx" like this.
Can anyone explain why swift not sending "+" symbol in API.
And Please tell how to solve this.
Use the proper Base64 variant that does not have the + character for REST calls: RFC 4648 base64url.
If you can't change the encoding, simply transcode the Base64 you receive.
How to prevent a server from returning an error 400 code error when the URL contains % symbol using NGINX server?
Nginx configuration for my website:
....
rewrite ^/download/(.+)$ /download.php?id=$1 last;
....
When I tried to get access to this URL:
http://mywebsite.net/download/some-string-100%-for-example
I got this error:
400 Bad Request
With this url :
http://mywebsite.net/download/some-string-%25-for-example
it's work fine !
It's because it needs to be URL encoded first.
This will explain:
http://www.w3schools.com/tags/ref_urlencode.asp
URLs can only be sent over the Internet using the ASCII character-set.
Since URLs often contain characters outside the ASCII set, the URL has to be converted into a valid ASCII format.
URL encoding replaces unsafe ASCII characters with a "%" followed by two hexadecimal digits.
URLs cannot contain spaces. URL encoding normally replaces a space with a plus (+) sign or with %20.
The URL interpreter is confused to see a % without hexadecimals after it.
Why would you think of solving by changing Nginx configuration???
It's impossible to solve from the server side. It's a problem from the client side.
https://headteacherofgreenfield.wordpress.com/2016/03/23/100-celebrations/
In that URL, the title is 100% Celebrations! but the permalink is autogenerated to 100-celebrations. It's because they know putting 100% will cause a URL encode problem.
If even Wordpress doesn't do it your way, then why should you do it?
I am trying to fetch product related data from Amazon Product API using this URL mentioned here:
http://docs.aws.amazon.com/AWSECommerceService/latest/DG/ItemLookup.html
Item Lookup URL:
http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[AWS_Access_Key_ID]&Operation=ItemLookup&ItemId=B00008OE6I&Timestamp=[YYYY-MM-DDThh:mm:ssZ]&Signature=[Request_Signature]
I have entered required details/parameters in URL but I am unable to understand where to get this Signature from or how to generate this signature? Any help would be appreciated.
The easiest and fastest way would be using one of AWS SDKs. You can find the full list here: https://aws.amazon.com/tools/
When you use the SDK you just supply your access key and secret key. The libraries handle preparing the signature.
Well, there's no point in doing this by yourself in Ruby, once it has a SDK, but an explanation about signature process may be useful for some people.
Signature is the way AWS authenticates API requests. It is passed as a query string parameter on your HTTP request and is validated against your own request content and you credentials on server-side.
There are 2 versions of it: 2 and 4, explained here and here. Version 2 is not supported on newer regions (China and Frankfurt).
In version 2, which I'm more familiar with, the signature is generated based on a string with this format:
HTTPVerb + "\n" + ValueOfHostHeaderInLowercase + "\n" + HTTPRequestURI
+ "\n" + CanonicalizedQueryString
The CanonicalizedQueryString is your query string, with parameters sorted (UTF-8 byte order) by paramater names and with parameter values url-encoded.
Then you calculate an HMAC of that string, using your AWS secret key, and SHA256 or SHA1 as the hash algorithm (depending on which one you have specified on query string parameter SignatureMethod), and encode it to base64. The result of all this will be the value of the Signature parameter.
I will try to walk you through a CloudWatch request, generated manually with bash tools.
Let's say you want to list you CloudWatch metrics. The request (without the signature) would be like the following.
http://monitoring.amazonaws.com/?AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
So, your string to sign would be:
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
Well, let's put it in a variable and do all the magic (hmac + base64)
#!/bin/bash
# your secret key
key='ry+SyjvXLvi4lYie/QfZYnxYOBQDggf01DTUxFSK'
# using 'read' to assign the signing string
# to a variable for readability reasons
read -r -d '' payload <<EOF
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG\
&Action=ListMetrics\
&SignatureMethod=HmacSHA1\
&SignatureVersion=2&\
Timestamp=2015-08-20T22%3A40%3A00.000Z\
&Version=2010-08-01
EOF
echo -n "$payload" | openssl dgst -sha256 -hmac $key -binary | openssl enc -base64
Then, after all this, you will end up with something like this:
xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM=
This is your signature, and finally, you just need to url-encoded it and append it to your request as one more query-string parameter.
&Signature=xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM%3D
Hopefully, if you did everything correctly (putting the sign string together, generating the hash and encoding it), your request will be accepted and, if you have enough permission, executed.
So, as you can see, it's something not worth to do by yourself. Use the SDK. ;)
Thank You guys for your response. But I have finally figured out my problem and my solution was in this gem:
https://github.com/jugend/amazon-ecs
This is what I was actually looking for and it finally helped me achieve what I was trying to. Posting it over here so people can find a better solution.
I am getting following error whenever Jmeter gets a | (pipe) symbol in the URL since the "pipe" symbol is not allowed in URL. Is there any way to convert the |(pipes) in URL to %7C automatically?
Response code: Non HTTP response code: java.net.URISyntaxException
Assuming the pipe is in an existing parameter try the following:
Check "Encode?" box in "Send Parameters with the Request" input of HTTP Request
Wrap your variable in __urlEncode function as ${__urlencode(queryTerm)}
Use Beanshell Pre Processor as a child of HTTP Request with the following code:
vars.put("queryTerm", URLEncoder.encode(vars.get("queryTerm")));
This should convert the pipes automatically.
Got the instructions from here: Jmeter - Configure Keywords with spaces using CSV File Configuration
Rubyists,
something's wrong with my HTTP AUTH params that are coming into my Rails 3 app. The password has some whitespace at the end. I was debugging my client app and it looks like it is sending it correctly.
I am doing this in my app:
params[:auth_username], params[:auth_password] = user_name_and_password(request)
Then I am sending this into Warden.
I would like to see the raw data to see if the whitespace is there. How to do that?
Edit: I have debugged the wire between httpd and thin process and I am pretty sure the data are coming correctly. It must be something wrong in my Rails 3.0.10. I was able to decode the base64 string that is coming in the headers and it did not contain any whitespace.
This really looks like BASE64 decoder issue. Maybe a padding problem. My string is:
Qmxvb21iZXJnOnRjbG1lU1JT
which decodes to
Bloomberg:tclmeSRS
correctly using non-Ruby base64 decoders. Even in Ruby:
>> Base64.decode64 "Qmxvb21iZXJnOnRjbG1lU1JT"
=> "Bloomberg:tclmeSRS"
I don't get it. Searching for a bugreport in Rails or something like that.
Edit: So it turns out our Apache httpd proxy adds something to the header:
Authorization: Basic Qmxvb21iZXJnOnRjbG1lU1JT, Basic
This leads to the incorrect characters at the end of the password, because:
>> Base64.decode64('Basic Qmxvb21iZXJnOnRjbG1lU1JT, Basic'.split(' ', 2).last || '')
=> "Bloomberg:tclmeSRS\005\253\""
The question is now - is this correct? Is it a bug in httpd or rails?
Rails user_name_and_password method makes a call to decode_credentials that performs the following, then splits using ":" :
::Base64.decode64(request.authorization.split(' ', 2).last || '')
Applied to your data :
::Base64.decode64("Qmxvb21iZXJnOnRjbG1lU1JT".split(' ', 2).last || '').split(/:/, 2)
=> ["Bloomberg", "tclmeSRS"]
Everything seems to be ok, the problem sits elsewhere IMO. To dump the authorization data from your controller :
render :text => "Authorization: #{request.authorization}"