I already read the following: Send request to page with windows-1251 encoding from python
Yet I cannot print a correct string. I have tried nearly every combination of u''.join, encode and decode I could think about...
import requests
url = 'http://www.multitran.ru/c/m.exe'
payload = {'CL': '1', 's': 'foo', 'l1':'1'}
r = requests.post(url, data=payload)
# ????????? print(u''.join(r.text))
I'd be very glad to get a solution and an explanation, because I really do not get it!
Related
I want my app to response with body utf-8 and iso-8859-1 encoded
per requests with Accept-Charset="utf-8" or Accept-Charset="iso-8859-1".
The response body is always JSON.
In my controller, when I doing this
render(json: data, status: :created)
It response with Content-Type="application/json; charset=utf-8" as well.
But how to make a response with body iso-8859-1 encoded when request Accept-Charset="iso-8859-1"?
In order to do this, you can use the method force_encoding and encoding for example
data = {'name'=>'raghav'}.to_json
data.encoding #This would return what encoding the value as #<Encoding:UTF-8>
new_data = data.force_encoding('ISO-8859-1') #This would force the encoding
new_data.encoding #<Encoding:ISO-8859-1>
Also to do this on the specific case you can always read the request.headers hash to determine the encoding.
There is also another method called encode the main difference between these are force_encoding changes the way string is being read from bytes, and encode changes the way string is written without changing the output (if possible)
On the client side I am reading an image file and encoding it in base64, sending it to as an URL param.
img = open("file.png", "rb").read()
print len(img)
img = img.encode("base64")
print len(img)
print len(img.decode("base64"))
Prints 252235, 340742 and 252235.
On server side decoding the received str couldn't yield the same result. I am posting the encoded base64 as "http://url.com/test?image=img_str".
img = flask.request.args["image"]
print len(img)
img = img.decode("base64")
print len(img)
Prints 340742 which is perfectly fine and 248176 which should actually be the original length. Is image param modifying during the post request? How to do this without using files param in requests or any other solution.
So, I figured this out!
While sending the encoded string as an URL parameter, "+" in the string are converting into " ". So, had to encoded_base64.replace(" ", "+") before decoding. And it worked!
I'm trying to get an Application Only Auth token following the steps of this link:
https://dev.twitter.com/docs/auth/application-only-auth
I'm using Ruby on Rails and Rest Client to make the POST request needed and I'm setting the headers (I think) properly.
The step-by-step says:
URL encode the consumer key and the consumer secret according to RFC
1738. Note that at the time of writing, this will not actually change the consumer key and secret, but this step should still be performed
in case the format of those values changes in the future.
Concatenate the encoded consumer key, a colon character ":", and the
encoded consumer secret into a single string.
Base64 encode the string from the previous step.
And my code is:
require 'rest_client'
key = URI::encode('app_key')
secret = URI::encode('app_secret')
encoded = Base64.encode64("#{key}:#{secret}")
res = RestClient::Resource.new "https://api.twitter.com/oauth2/token/"
response = ''
options = {}
options['Authorization'] = "Basic #{encoded}"
options['Content-Type'] = 'application/x-www-form-urlencoded;charset=UTF-8'
res.post('grant_type=client_credentials', options) do |response, request, result|
response << "#{CGI::escapeHTML(response.inspect)}<br /><br />"
response << "#{CGI::escapeHTML(request.inspect)}<br /><br />"
response << "#{CGI::escapeHTML(result.inspect)}<br />"
end
render :text => txt
And I print out this:
"{\"errors\":[{\"label\":\"authenticity_token_error\",\"code\":99,\"message\":\"Unable to verify your credentials\"}]}"
#<RestClient::Request:0x9ece5d8 #method=:post, #headers={"Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8"}, #url="https://api.twitter.com/oauth2/token/", #cookies={}, #payload="", #user=nil, #password=nil, #timeout=nil, #open_timeout=nil, #block_response=nil, #raw_response=false, #verify_ssl=false, #ssl_client_cert=nil, #ssl_client_key=nil, #ssl_ca_file=nil, #tf=nil, #max_redirects=10, #processed_headers={"Accept"=>"*/*; q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate", "Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8", "Content-Length"=>"29"}, #args={:method=>:post, :url=>"https://api.twitter.com/oauth2/token/", :payload=>"grant_type=client_credentials", :headers=>{"Authorization"=>"Basic bXlfa2V5Om15X3NlY3JldA==\n", "Content-Type"=>"application/x-www-form-urlencoded;charset=UTF-8"}}>
#<Net::HTTPForbidden 403 Forbidden readbody=true>
My key and secret are valid.
Am I missing something?
Thanks!
EDIT:
Updating with the solution I've found.
The problem was on the Base64 convertion and string encoding.
I had to add a forced encoding parameter to the key+secret combination, for UTF-8 convertion:
encoded = Base64.encode64("#{key}:#{secret}".force_encoding('UTF-8'))
The Rails Base64.encode64 inserts a line break every 60 encoded characters.
The workaround was:
For Ruby 1.9+ (strict_ was included in Ruby 1.9)
Base64.strict_encode64(string)
For Ruby 1.9-
Base64.encode64(string).gsub('/\n/') # To remove the line break
Are you trying to implement Authorization with Tweeter (as OAuth Provider). Instead of writing it from the scratch following the API documentation, I would suggest to use OmniAuth. The setup & boilerplate code is fairly easy to use.
Read more about it at http://www.omniauth.org/ & https://github.com/intridea/omniauth/wiki
Let us know, if that helped you or not.
In order to use a third-party API, I need to encode the Net::HTTP::Post request as an MD5 hex digest, which is then used as part of the signature. However, when I try to simply Digest::MD5.hexdigest(req), it throws a "Cannot convert to string error", and when I explicitly req.to_s, it just gives the MD5 of #<Net::HTTP::Post:0x112a0eef8>
I'm simply:
request = Net::HTTP::Post.new(url.path)
request.body = {
"key" => "val"
}.to_json
# later...
hexDigest = Digest::MD5.hexdigest(request)
which is the documented spec, I think: "[with the] JSON body containing the new information."
This is the relevant sample Java code they supply:
ByteArrayOutputStream requestOutputStream = new ByteArrayOutputStream();
httpMethod.getEntity().writeTo(requestOutputStream);
DigestUtils.md5Hex(requestOutputStream.toByteArray()).toLowerCase();
Any ideas?
Thanks!
Try to call 'to_s' method explicitly, it should help:
hexDigest = Digest::MD5.hexdigest(request.to_s)
The equivalent ruby code for those lines is:
OpenSSL::Digest::MD5.hexdigest(request.body)
httpMethod.getEntity() will return the json defined as the request body.
requestOutputStream.toByteArray() will return the array of bytes corresponding to the request body.
We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.