I seem to be having the same problem as this chap here
I want to encode some parameters (for the import.io api). Effectively:
params = {
:input => "webpage/url:http://www.example.com",
:input => "keywords:some+keywords"
}
But that won't work, so I think this is the right approach:
params = { :input => ["webpage/url:http://www.example.com", "keywords:some+keywords"] }
and I want it to output
params.to_query
=> "input=webpage%2Furl%3Ahttp%3A%2F%2Fwww.example.com%2Fsome-id&input=keywords%3Asome%2Bkeywords"
unfortunately, I get
"input%5B%5D=webpage%2Furl%3Ahttp%3A%2F%2Fwww.example.com%2Fsome-id&input%5B%5D=keywords%3Asome%2Bkeywords"
It's adding [] after the input, which I believe is standard behaviour. How can I stop it doing it?
To clarify, what is the ruby or 'rails way' of dealing with url parameters that require duplicate keys?
Ran into a similar issue, there's a helpful post here Ruby Hash with duplicate keys? but briefly
params = {}.compare_by_identity
params['input'] = "webpage/url:http://www.example.com"
params['input'.dup] = "keywords:some+keywords"
then
params.to_query
returns
"input=keywords%3Asome%2Bkeywords&input=webpage%2Furl%3Ahttp%3A%2F%2Fwww.example.com"
Some characters in a url have special importance to the processing of the url: they are reserved, like keywords in a programming language. See Which characters make a URL invalid?
If you try to use these as the name or value of a parameter, it will break the uri and you'll get hard to predict results like you're seeing.
The answer is to URI escape the string, which will replace special characters with their encoded version. Rails will automatically unescape them when it gets the the request, so you don't need to worry about it.
You can escape them manually, but the best way, if you have them as a hash already, is to call .to_param on the hash.
params = { :input => ["webpage/url:http://www.example.com", "keywords:some+keywords"] }
=> {:input=>["webpage/url:http://www.example.com", "keywords:some+keywords"]}
params.to_param
=> "input%5B%5D=webpage%2Furl%3Ahttp%3A%2F%2Fwww.example.com&input%5B%5D=keywords%3Asome%2Bkeywords"
Related
i want to pass a query to url like
http:/localhost:3000/saldo/;1010917745800000015?1
in my routes i have:
get 'saldo/:nomor' => 'kartus#show_saldo' , as: :show_saldo
and controller:
def show_saldo
#kartu = Kartu.find_by_nomor(params[:nomor])
end
but instead i get this params
Parameters {"1"=> nil,"nomor"=>";1010917745800000015"}
how can i get my param as {"nomor"=>";1010917745800000015?1"}
<%= link_to 'xyz' show_saldo_path(:nomor => 'nomor', :def => 'def'......) %>
In get everything you passed other than url parameter will become your query parameter. def will become your url parameter. More information here.
? is a special character in urls. If you want to include it in the value of a parameter then you should Uri Encode, eg with CGI.escape(), the parameter before submitting it: this will convert "?" to "%3F", and will similarly convert any other special characters (spaces, brackets etc). So, the parameter that is actually submitted will become "%3B1010917745800000015%3F1".
At the server side, rails will call CGI.unescape on the params, so it should show up in your controller as ";1010917745800000015?1" again.
This should happen automatically with form inputs - ie, if someone writes ;1010917745800000015?1 into a text field then it should actually be sent through as "%3B1010917745800000015%3F1"
If you want people to diagnose why this isn't happening then you should include the html (of the form or link which is submitting this value) to your question.
I am trying to parse an email in Ruby on Rails that is an attachment? I am not to worried about the regular expression, but more so the method I use to get the parsed output. I am looking to do this without any mail parsing gems. The code below appears to work, is this the correct way?
model.rb
def parse_delivered_to
str = File.read("public/emails/email.txt").to_s
delivered_to = str.match(/(Delivered-To: )[\w+\-.]+#[a-z\d\-.]+\.+[a-z]+[a-z]+[a-z]/i)
end
show.html.erb
<%= #email.parse_delivered_to %><br>
Analysis
The email specifications allow for multiline headers, which your current expression won't match. In addition, I don't think your regular expression allows for all of the permissible address characters.
Solution
Using a variation of procmail's ^TO_ syntax should allow you to match multiline address patterns more liberally. For example:
header.scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im ).flatten.map(&:strip)
Some Tests and Examples
header = "Delivered-To:\n Foo <foo#example.com>"
header.scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im ).flatten.map(&:strip)
header.scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im ).flatten.map(&:strip)
=> ["Foo <foo#example.com>"]
header.scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im).
flatten.map(&:strip).to_s.scan(/[\w#.+_-]+/).grep(/#/).first.to_s
=> "foo#example.com"
'Delivered-To: foo.bar+extension#example.com'.
scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im).
flatten.map(&:strip).to_s.scan(/[\w.+_-]+/)
=> ["foo.bar+extension"]
'Delivered-To: foo.bar-extension#example.com'.
scan( /^Delivered-To:(.*[^-a-zA-Z0-9_.])?/im).
flatten.map(&:strip).to_s.scan(/[\w.+_-]+/)
=> ["foo.bar-extension"]
I have a Facebook batch request that looks like this:
https://graph.facebook.com/?access_token=ACCESS_TOKEN&batch=[{"method": "GET", "relative_url": "search?q=EMAIL#ADDRESS.COM&type=user"}]
Sending this across the wire returns:
{"error"=>0, "error_description"=>"batch parameter must be a JSON array"}
If I remove the &type=user, it works fine (sends back an empty data array). I am absolutely certain that Facebook is not parsing the & character correctly. I read online somewhere that I could try encoding the & symbol to %26, however using that replacement seems to instead do a query for "EMAIL#ADDRESS.COM%26type=user". If you reverse the order of the parameters, you will see what I mean.
Any ideas how I can get the batch request parser on Facebook to recognize the & symbol without filing a bug report that will never be fixed?
EDIT:
I am using URI.encode. Here is the exact code:
queries = email_array.map { |email| { :method => "GET", :relative_url => "search?q=#{email}&type=user" } }
route = "https://graph.facebook.com/?access_token=#{token}&batch=#{URI.encode(queries.to_json)}"
res = HTTParty.post(route)
After actually playing around with this some more, I managed to reproduce the same behavior, even with a careful check and double-check that I was following the api specs correctly. This looks like a bug in facebook's batch method -- it doesn't understand ampersands in param values correctly.
Don't use a string literal to construct the json. Use to_json, like below. (Also, as an aside, don't use {} notation across more than one line, use do/end).
queries = []
email_array.each do |email|
queries << {:method => 'GET', :relative_url => "search?q=#{email}&type=user"}
end
route = "https://graph.facebook.com/?access_token=#{token}&batch=#{URI.encode(queries.to_json)}"
res = HTTParty.post(route)
Also, you can use Array#map to simply the code, like this:
queries = email_array.map { |email| {:method => 'GET', :relative_url => "search?q=#{email}&type=user"} }
route = "https://graph.facebook.com/?access_token=#{token}&batch=#{URI.encode(queries.to_json)}"
res = HTTParty.post(route)
EDIT: below is my original answer before the question was edited, for reference.
Try properly url encoding the whole parameter:
https://graph.facebook.com/?access_token=ACCESS_TOKEN&batch=[%7B%22method%22:%20%22GET%22,%20%22relative_url%22:%20%22search?q=EMAIL#ADDRESS.COM&type=user%22%7D]
In practice, you'd use URI.encode from the uri library to do this. Example:
irb(main):001:0> require 'uri'
=> true
irb(main):002:0> URI.encode('[{"method": "GET", "relative_url": "search?q=EMAIL#ADDRESS.COM&type=user"}]')
=> "[%7B%22method%22:%20%22GET%22,%20%22relative_url%22:%20%22search?q=EMAIL#ADDRESS.COM&type=user%22%7D]"
Or even better, use to_json to create your json string in the first place. Example:
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'json'
=> true
irb(main):003:0> require 'uri'
=> true
irb(main):004:0> URI.encode([{:method => 'GET', :relative_url => 'search?q=EMAIL#ADDRESS.COM&type=user'}].to_json)
=> "[%7B%22method%22:%22GET%22,%22relative_url%22:%22search?q=EMAIL#ADDRESS.COM&type=user%22%7D]"
If this helps anyone, when my AdSet batch update failed because there was an "&" in one of the interests name:
{u'id': u'6003531450398', u'name': u'Dolce & Gabbana'}
I learned that the name can be anything, and as long as the id is correct, FB will populate the name itself.
How would I go about manually filtering a hash using my application's parameter filter?
I imagine it'd go like this:
Rails.application.filter :password => 'pass1234'
# => {:password => '[FILTERED]'}
EDIT (clarification): I'm aware that Rails filters the params hash when writing to the logs. What I want to do is apply that same filter to a different hash at my prerogative before writing it to the logs with something like Rails.logger.info. I'm calling a remote HTTP query as a part of my application (since most of the backend operates through a remote API), and I'm logging the URL and parameters passed. I want to have the logs but also ensure that none of the sensitive params show up there.
After a few minutes of shotgunning it, I figured out this was the way to do it:
filters = Rails.application.config.filter_parameters
f = ActionDispatch::Http::ParameterFilter.new filters
f.filter :password => 'haha' # => {:password=>"[FILTERED]"}
See the config/application.rb file, towards the end there is a line:
config.filter_parameters += [:password]
This way the "password" param will not be shown in logs, but you can still access the value normally.
Edit
It seem that have misunderstood your meaning of "filter" originally. As for the clarified issue, I have no idea on how to handle it the truly Rails way.
Here is a brute force approach:
Parse the query with CGI::parse(URI.parse(my_url_address_with_params).query) to get a hash of param/values (note: values are actually stored as an array; here is the discussion).
Locate the parameters you want to filter out and replace values with literal *filtered*.
Call Rails.logger.info (or debug) directly to log.
Here is what you should dig into when relying on Rails magical classes and methods:
In Rails 3 the code that does the trick seems to live in ActionDispatch::Http (ParameterFilter in particular, method `filtered_parameters'). The documentation is available at API Dock (or, to be honest, very little documentation). You can examine the sources to get an idea of how this works.
My knowledge of Rails internals is not good enough to suggest anything else. I believe that someone with a better understanding of it might be of more help.
Building on Steven Xu's answer above, I made this initializer in my rails app:
class ActionController::Parameters
def filtered
ActionDispatch::Http::ParameterFilter.new(Rails.application.config.filter_parameters).filter(self)
end
end
Which let's me call params.filtered
[1] pry(#<LessonsController>)> params.filtered
{
"controller" => "lessons",
"action" => "search",
"locale" => "en"
}
[2] pry(#<LessonsController>)> params[:password] = "bob"
"bob"
[3] pry(#<LessonsController>)> params.filtered
{
"controller" => "lessons",
"action" => "search",
"locale" => "en",
"password" => "[FILTERED]"
}
Basicaly I just want to insert this + "?direction=desc" in helper method.
But once it parses it comes out like this..
/organizations/search?order_by=contactable%3Fdirection%3Ddesc
Anyone know a way around this?
My Helper Method:
def search_sort(name, sort_by, order = 'asc')
link_to(name, url_for(:overwrite_params => { :order_by => sort_by + "?direction=desc" :page => nil }), :class => 'selected save_pushstate')
...
I know what you're thinking. Just add :order into it. The problem being is that I 'm using an AJAX history saver from #175 of railscasts.
$(".save_pushstate").live("click", function() {
$.setFragment({"order_by" : $.queryString($(this).attr('href')).order_by});
//$.setFragment({"direction" : $.queryString($(this).attr('href')).direction});
return false;
});
And it rewrites my url to just one "fragment". I can't have two! So I decided that if I can just add the direction param in the href hard-coded, it could deal with this whole mess.
Try:
+ "?direction=desc".html_safe
Edit:
Since you're using rails 2.3.5, try this:
def search_sort(name, sort_by, order = 'asc')
link_to(name, url_for(:overwrite_params => { :order_by => sort_by + "?direction=desc" :page => nil }, :escape => false), :class => 'selected save_pushstate')
...
Note the ":escape => false" in url_for.
Edit2:
After reading this:
http://www.ruby-forum.com/topic/80381
Specifically this excerpt:
I think this is where the confusion is
arising. There are two different kinds
of escaping going on.
It sounds like you're talking about
the URL encoding that uses '%xx' to
represent special characters.
However, the html_escape function does
something completely different. It
takes a string and turns '&' into
'&' and '<' into '<', etc., so
that it can go into HTML without being
interpreted as literal '&'s and '<'s.
Escaping special characters in URLs
using the '%xx' scheme is mandatory,
otherwise they are not valid URLs.
I've realized that the 'escaping' that you see happening is url encoding, and it shouldn't affect your query/sorting, etc. You can test it out by taking the encoded url and typing it into your browser.
:escape => false disable html escaping, which means dangerous characters get converted to display codes, such as '&' into '&' and '<' into '<', etc.,
And the "?" in your append should be "&":
+ "&direction=desc"
Hope this helps. =)