Can I reduce duplication in the serialization of a ruby hash? - ruby-on-rails

I have a hash of the format
{com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}
which I am sending to a remote server. I am using the HTTParty gem to do this. The code looks like this
class Api
include HTTParty
attr_accessor :headers
def initialize
#headers = { 'Content-Type' => 'application/json' }
end
def post_com(hsh)
response = self.class.post('some_url', query: hsh, headers: headers, format: :plain)
end
end
When I do
api = Api.new.post_com({com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}
at the remote server, the hash is being sent in the following format
POST "/some_url?com=1234&users[][nid]=3&users[][sets][]=1&users[][sets][]=2&users[][sets][]=3&users[][sets][]=4&users[][nid]=4&users[][sets][]=5&users[][sets][]=6&users[][sets][]=7&users[][sets][]=8
This means for every entry in set, duplicate characeters users[][sets][] are being sent. In operation, there can be many entries in set, and the result is the server rejects the post as having too many characters.
Is there anyway I can have the hash serialized with far less duplication. For instance if I just do
{com: 1234, users: [{nid: 3, sets: [1,2,3,4]}, {nid: 4, sets: [5,6,7,8]}]}.to_json
I receive
"{\"com\":1234,\"users\":[{\"nid\":3,\"sets\":[1,2,3,4]},{\"nid\":4,\"sets\":[5,6,7,8]}]}"
which has far fewer characters.

HTTParty, by default, converts the :query hash into what it calls 'rails style query parameters':
For a query:
get '/', query: {selected_ids: [1,2,3]}
The default query string looks like this:
/?selected_ids[]=1&selected_ids[]=2&selected_ids[]=3
Since you are doing a POST, it is possible/preferable to send your hash in the body of the request rather than in the query string.
def post_com(hsh)
self.class.post('some_url', body: hsh.to_json, headers: headers, format: :plain)
end
This has the advantage that it doesn't do any transformation of the payload and the query string length limit doesn't apply anyway.
For the record, you can disable the default 'rails style' encoding like this:
class Api
include HTTParty
disable_rails_query_string_format
...
end
You can also roll your own custom query string normalizer by passing a Proc to query_string_normalizer.

Related

How to extract JSON array fields using HTTParty?

I am using FOOD2FORK api to extract data using HTTParty in ruby.
My code :
require 'httparty' #Note : This file is in the models >> filename.rb
#Don't need to require when using bundler
#Restful Web Services
#1. Base URI 2. Support XML or JSON 3. Support HTTP operations (GET, POST)
#Think of web at MVC : website you can get resources in any format
#HTTParty parses XML or JSON for you (like your browser - it's a client). Parses into a Ruby hash or array
class Recipe
include HTTParty
ENV["FOOD2FORK_KEY"] = "5b6b74c6cc0fa9dc23871a7ae753f6c7"
base_uri "https://food2fork.com/api" #Same across most requests
default_params key: ENV["FOOD2FORK_KEY"], fields: "image_url" #defaults, like API developer key #fields: "image_url, source_url, title",
format :json #Tell it which format data comes in
#q:"search" request parameter
def self.for (keyword)
#class method called for which takes in a term
get("/search", query: {q: keyword})["recipes"] #get is method provided by HTTParty
#returns array where each element in the array is a hash
end
pp Recipe.for "chocolate"
end
It's returning me
[
{
"publisher"=>"BBC Good Food",
"f2f_url"=>"http://food2fork.com/view/9089e3",
"title"=>"Cookie Monster cupcakes",
"source_url"=>"http://www.bbcgoodfood.com/recipes/873655/cookie-monster-cupcakes",
"recipe_id"=>"9089e3",
"image_url"=>"http://static.food2fork.com/604133_mediumd392.jpg",
"social_rank"=>100.0,
"publisher_url"=>"http://www.bbcgoodfood.com"
}
]
But I only want to extract it's image_url even on using field it's extracting the whole data set any idea how to extract only image_url ?
P.S You can check the format of JSON here -
http://food2fork.com/api/search?key=65987bf1f4b22007c365c243f5670f35&q=shredded%20chicken
Below code should work
response = your response from API
response_data = JSON.parse(response.body).with_indifferent_access
response_data['recipes'].each do |recipe|
puts recipe['image_url'] # image_url of block
end

Unknown Attribute error when uploading a CSV file

Here is what my CSV file that I am trying to upload looks like:
1,Order,”{‘customer_name’:’Jack’,’customer_address’:’Trade St.’,’status’:’unpaid’}”
2,Order,”{‘customer_name’:’Sam’,’customer_address’:’Gecko St.’,’status’:’unpaid’}”
1,Product,”{‘name’:’Laptop’,’price’:2100,’stock_levels’:29}"
1,Order,”{‘status’:’paid’,’ship_date’:’2017-01-18’,’shipping_provider’:’DHL’}”
2,Product,”{‘name’:’Microphones’,’price’:160,’stock_levels’:1500}"
1,Invoice,”{‘order_id’:7,’product_ids’:[1,5,3],’status’:’unpaid’,’total’:2500}"
1,Invoice,”{‘status’:’paid’}”
But I'm getting this error: ActiveModel::UnknownAttributeError in CustomersController#import
And these errors in my console:
app/models/customer.rb:4:in `block in import'
app/models/customer.rb:3:in `import'
app/controllers/customers_controller.rb:65:in `import'
Here is my customer.rb model:
class Customer < ApplicationRecord
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
Customer.create! row.to_hash
end
end
end
To be perfectly honest, part of the problem I'm having here stems from not totally understanding what row.to_hash is doing here. If this function does not iterate through a row, then we want to convert it to a hash. I feel like it may be causing other problems here that I may not be aware of though.
Here is the import function I have as well:
def import
Customer.import(params[:file])
redirect_to customer_path, notice: "Customer Added Successfully"
end
The error, ActiveModel::UnknownAttributeError happens when you pass an attribute that the model doesn't have to new or create, for instance:
User.create(not_a_real_attribute: 'some value')
# => ActiveModel::UnknownAttributeError: unknown attribute 'not_a_real_attribute' for User.
The reason you are getting this is because you are using a header-less CSV as if it had headers. In ruby, if a CSV has headers, it can convert the row to a hash using the header of a column as a key and the column of the current row as a value. So, by telling the CSV library you have headers (via headers: true in the foreach), it treats the first row as a header row and you end up with weird results:
enum = CSV.foreach("./customers.csv", headers: true)
enum.first.to_hash
# => {"1"=>"2", "Order"=>"Order", "”{‘customer_name’:’Jack’"=>"”{‘customer_name’:’Sam’", "’customer_address’:’Trade St.’"=>"’customer_address’:’Gecko St.’", "’status’:’unpaid’}”"=>"’status’:’unpaid’}”"}
Then you are passing that hash to Customer.create!.If you don't expect headers, you need to treat each row as an array:
enum = CSV.foreach("./customers.csv")
enum.first
# => ["1", "Order", "”{‘customer_name’:’Jack’", "’customer_address’:’Trade St.’", "’status’:’unpaid’}”"]
Or, if you want to use a hash, you can insert a better first row, corresponding to the attributes of your model:
"Maybe an ID?","Some kind of Class?","Customer Data?"
1,Order,”{‘customer_name’:’Jack’,’customer_address’:’Trade St.’,’status’:’unpaid’}”
# ... the rest of your file ...
enum = CSV.foreach("./customers.csv", headers: true)
enum.first.to_hash
# => {"Maybe an ID?"=>"1", "Some kind of Class?"=>"Order", "Customer Data?"=>"”{‘customer_name’:’Jack’", nil=>"’status’:’unpaid’}”"}
You'll also notice throughout these examples, those special quotes in your file weren't being handled properly, the results you get if you change those to normal quotes are:
# => {"Maybe an ID?"=>"1", "Some kind of Class?"=>"Order", "Customer Data?"=>"{'customer_name':'Jack','customer_address':'Trade St.','status':'unpaid'}"}
Also, if that last column is the customer data you want to create with, you'll need to pull that column out and parse it into a ruby hash on your own. Looks like maybe YAML?:
YAML.load enum.first.to_hash['Customer Data?']
# => {"customer_name"=>"Jack", "customer_address"=>"Trade St.", "status"=>"unpaid"}

Pass array as get parameters in RoR using HTTParty

I have a rails api endpoint that takes an array as one of it's parameters. In rspec I have the following test
it 'should get a list of cars' do
car_ids = [1,2]
get :index, {car_ids: car_ids, format: :json}
end
and in the controller I have verified that
params[:car_ids] == [1,2]
However, when I use HTTParty to hit this endpoint, the ids in the arrays are Strings, not Integers.
prams[:car_ids] == ["1", "2"]
Here is my HTTParty call
HTTParty.get('api_endpoint', query: {car_ids: [1,2]})
I have tried a bunch of variations of this request, such as adding {format: :json}, and setting the content-type and accept headers but to no avail.
Anyone know how to solve this?

How to construct URI with query arguments from hash in Ruby

How to construct URI object with query arguments by passing hash?
I can generate query with:
URI::HTTPS.build(host: 'example.com', query: "a=#{hash[:a]}, b=#{[hash:b]}")
which generates
https://example.com?a=argument1&b=argument2
however I think constructing query string for many arguments would be unreadable and hard to maintain. I would like to construct query string by passing hash. Like in example below:
hash = {
a: 'argument1',
b: 'argument2'
#... dozen more arguments
}
URI::HTTPS.build(host: 'example.com', query: hash)
which raises
NoMethodError: undefined method `to_str' for {:a=>"argument1", :b=>"argument2"}:Hash
Is it possible to construct query string based on hash using URI api? I don't want to monkey patch hash object...
For those not using Rails or Active Support, the solution using the Ruby standard library is
hash = {
a: 'argument1',
b: 'argument2'
}
URI::HTTPS.build(host: 'example.com', query: URI.encode_www_form(hash))
=> #<URI::HTTPS https://example.com?a=argument1&b=argument2>
If you have ActiveSupport, just call '#to_query' on hash.
hash = {
a: 'argument1',
b: 'argument2'
#... dozen more arguments
}
URI::HTTPS.build(host: 'example.com', query: hash.to_query)
=> https://example.com?a=argument1&b=argument2
If you are not using rails remember to require 'uri'

rails params hash to String:String hash

I am doing an http get using the url http://localhost/add?add_key[0][key]=1234&add_key[0][id]=1.
I have a rails app which gives me a neat params hash {"add_key"=>{"0"=>{"key"=>"1234", "id"=>"1"}}. However when I try to post this to a different server using
new_uri = URI.parse("http://10.10.12.1/test")
res = Net::HTTP.post_form new_uri,params
The server handling the post is seeing this parameter in the request
{"add_key"=>"0key1234id1"}
Looks like post_form requires a String to String hash. So how do I convert the params hash to
{"add_key[0][key]" => "1234", add_key[0][id]" => "1"}
From the fine manual:
post_form(url, params)
Posts HTML form data to the specified URI object. The form data must be provided as a Hash mapping from String to String.
So you're right about what params needs to be.
You could grab the parsed params in your controller:
{"add_key"=>{"0"=>{"key"=>"1234", "id"=>"1"}}
and then recursively pack that back to the flattened format that post_form expects but that would be a lot of pointless busy work. An easy way to do this would be to grab the raw URL and parse it yourself with URI.parse and CGI.parse, something like this in your controller:
u = URI.parse(request.url)
p = CGI.parse(u.query)
That will leave you with {"add_key[0][key]" => "1234", "add_key[0][id]" => "1"} in p and then you can hand that p to Net::HTTP.post_form.

Resources