Apply monkey patch for JSON parsing in rails - ruby-on-rails

I'm trying to handle edge-case issues with the BigCommerce API returning invalid JSON objects, resulting in un-paired octects as discussed here: A JSON text must at least contain two octets
I believe the solution is to deploy the code (from that thread):
module JSON
def self.parse_nil(json)
JSON.parse(json) if json && json.length >= 2
end
end
parsed = JSON.parse_nil(json)
Where do I deploy this code to apply the patch?

Patches like these are typically added in an otherwise empty .rb file in the /config/initializers directory (see here for an example). Note that the patch should not include the last line:
parsed = JSON.parse_nil(json)
That was added for demonstrative purposes.

Related

Reading JavaScript variable in Ruby through curl request

I need to read the GBP rate from this javascript file: http://cdn.shopify.com/s/javascripts/currencies.js. I want to be able to get the js variable as JSON so that I can easily access the variable I need with its index. I tried a couple of ways as follows with eventually no success.
Way 1
Source: https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html
My code:
uri = URI('http://cdn.shopify.com/s/javascripts/currencies.js')
#response = Net::HTTP.get(uri) # => String
Result: I get the result as a string and reading the GBP rate from the string is difficult and probably not the correct way.
Way 2
Source: curl request in ruby
My Code:
url = 'http://cdn.shopify.com/s/javascripts/currencies.js'
mykey = 'demo'
uri = URI(url)
request = Net::HTTP::Get.new(uri.path)
request['Content-Type'] = 'application/xml'
request['Accept'] = 'application/xml'
request['X-OFFERSDB-API-KEY'] = mykey
#response = Net::HTTP.new(uri.host,uri.port) do |http|
http.request(request)
end
Result: This returns me Net::HTTP:0x007f2480874050 which looks like a memory address, definitely not what I want.
In addition, I've included require 'net/http', require 'json' in my controller in either case.
I am very new to Ruby and I don't know how to figure this out. So looking for someone who can help.
This is a bit of a weird request, IMO, but Rails can do it. Rails comes with a library called execjs automatically, which lets you run javascript from ruby. So, you have some javascript you want to run in that file, but you also want to return specific key from that javascript, so something like this should do it:
# Expanding upon 'Way 1', which got you the javascript as a string
uri = URI('http://cdn.shopify.com/s/javascripts/currencies.js')
response = Net::HTTP.get(uri)
gbp_rate = ExecJS.exec "#{response}; return Currency.rates.GBP;"
p gbp_rate # => 1.40045
I just want to reiterate (from their FAQ in the README) though:
Can ExecJS be used to sandbox scripts?
No, ExecJS shouldn't be used for any security related sandboxing. Since runtimes are automatically detected, each runtime has different sandboxing properties. You shouldn't use ExecJS.eval on any inputs you wouldn't feel comfortable Ruby eval()ing.
This file looks safe, but just keep it in mind, you are actually executing this javascript.
Personally, I would look to see if there's an API somewhere that can give you this value more easily, or if it doesn't change often (I have never used Shopify so don't know how much this changes) just hardcode it in the app as a config value and update it manually. Just feels cleaner, to me.

MultiJson::DecodeError unexpected token at {"email":"foo#bar.com"}

I'm developing a new Ruby on Rails 3.2 application.
This application will receive a periodic json callback with statistics.
The callback i receive is not entirely valid. The json rules are separated by newlines.
The callback POSTs have a content-type header of application/json, and contain exactly one JSON string per line, with each line representing one event. Please note that currently the POST headers define this post as application/json, though it’s not; each line is a valid JSON string, but the overall POST body is not. For example:
This is a example of the callback:
{"email":"foo#bar.com","timestamp":1322000095,"unique_arg":"my unique arg","event":"delivered"}
{"email":"foo#bar.com","timestamp":1322000097,"unique_arg":"my unique arg","event":"click"}
{"email":"foo#bar.com","timestamp":1322000096,"unique_arg":"my unique arg","event":"open"}
When i receive this callback my Rails application crashes with a "MultiJson::DecodeError"
743: unexpected token at '{"email":"foo#bar.com","timestamp":1322000096,"unique_arg":"my unique arg","event":"open"}'
I think the application detects the application/json header and try automatic to parse it.
How can i convert this to a valid JSON object, so i can use this in my controller?
Thanks.
Put
gem 'yajl-ruby' in Gemfile and
require 'yajl/json_gem' config/application.rb
In gem file declare
gem 'yajl-ruby'
It is used to Parse and encode multiple JSON objects to and from streams or strings continuous.
And require 'yajl/json_gem' in application.rb
Put Elements in Brackets:
[{"email":"foo#bar.com","timestamp":1322000095,"unique_arg":"my unique arg","event":"delivered"},
{"email":"foo#bar.com","timestamp":1322000097,"unique_arg":"my unique arg","event":"click"},
{"email":"foo#bar.com","timestamp":1322000096,"unique_arg":"my unique arg","event":"open"}]`
Unable to reproduce your problem by copy-and-pasting the error message token using both ActiveSupport::JSON.decode and MultiJson.decode in a Rails shell. Rails 3.2.1
Any other clues?
This issue is likely due to response body encoded in ISO-8859-1 instead of UTF-8. Possible fix on client side is be convert it back to UTF-8 using ruby 'iconv' encoding methods.

Rails 3 application/octet-stream

I am working on an application where I use paperclip for uploading images, then the image is manipulated in a flash app and returned to my application using application/octet-stream. The problem is that the parameters from flash are not available using params. I have seen examples where something like
File.open(..,..) {|f| f.write(request.body) }
but when I do this, the file is damaged some how.
How can I handle this in rails 3?
After you make sure that the request parameters have hit the Rails application, you may want to ensure that there were no parsing problems. Try to add these lines in you controller's action:
def update # (or whatever)
logger.debug "params: #{params.inspect}"
# I hope you do not test this using very large files ;)
logger.debug "request.raw_post: #{request.raw_post.inspect}"
# ...
end
Maybe the variable names got changed somehow? Maybe something escaped the parameter string one time too much?
Also, you have said that the file into which you want to save the request body is damaged. How exactly?
The request.body object does not need to be String. It may be a StringIO, for example, so you may want to type this:
File.open(..,..) {|f| f.write(request.body.read) }

Rails API functional tests with XML body

I've written functional tests for API endpoints built in Rails using shoulda testing framework.
An example looks like the following:
setup do
authenticated_xml_request('xml-file-name')
post :new
end
should respond_with :success
authenticated_xml_request is a test helper method that sets
#request.env['RAW_POST_DATA'] with XML content.
After upgrading the app from rails 2.3.3 to rails 2.3.8, the functional tests are failing because the XML content received is not merged in the params hash.
I'm setting the request with the correct mime type via #request.accept =
"text/xml"
I'm able to inspect the content of the request using request.raw_post but i'd like to keep the current setup working.
Also, running a test from the terminal using cURL or any other library (rest_http) in development mode, the API works perfectly well.
Any tips or help is much appreciated.
Now it's simpler:
post "/api/v1/users.xml", user.to_xml, "CONTENT_TYPE" => 'application/xml'
Note that you have to specify appropriate "CONTENT_TYPE". In other case your request will go as 'application/x-www-form-urlencoded' and xml won't be parsed properly.
I solved the issue by adding a custom patch to rails (test_process.rb file) to convert incoming xml to hash then merge into parameters hash.
on line 439:
parameters ||= {}
parameters.merge!(Hash.from_xml(#request.env['RAW_POST_DATA'])) if #request.env['RAW_POST_DATA'] && #request.env['CONTENT_TYPE']=='application/xml'

Filtering parts or all of request URL from rails logs

Rails provides filter_parameter_logging to filter sensitive parameters from the rails log.
If you have a a JSONP API, some sensitive information could be present in the URL. Is there a way to filter request URLS from the log also?
Note: The answer here was the way to get it work on Rails 2.x ~> 3.0. Starting from Rails 3.1, if you set config.filter_parameters, Rails will filter out the sensitive parameter in the query string as well. See this commit for more detail.
I think in that case, you need to override complete_request_uri in ActionController::Base, since ActionController::Benchmarking calls that method and prints the line that looks like:
Completed in 171ms (View: 35, DB: 7) | 200 OK [http://localhost:3000/]
I think you can put this in initializer to override this method
class ActionController::Base
private
def complete_request_uri
"#{request.protocol}#{request.host}#{request.request_uri.gsub(/secret=([a-z0-9]+)/i, "secret=[FILTERTED]")}"
end
end
Note that you need to play a bit with regular expression to make it substitute the portion you wanted.
Sadly this no longer works in Rails 3. The path (including query parameters) comes from ActionDispatch::Request, which inherits from Rack::Request. Here's the relevant monkeypatch that you can throw into an initializer:
class ActionDispatch::Request
def fullpath
#fullpath ||= super.gsub(/secret=[^&]+/, 'secret=[FILTERED]')
end
end
I switched to using [^&] in the regex since the parameter could easily have characters that aren't letters or numbers in it.

Resources