I'm doing some custom logging in my Rails application and I want to automatically sensor some parameters. I know that we have fitler_parameter_logging.rb which does this for the params object. How can I achieve something like this for my custom hash.
Let's say I'm logging something like this:
Rails.logger.info {name: 'me', secret: '1231234'}.inspect
So my secret key should be sensored in the logs.
I know I can personally delete the key before logging, but it adds noise to my application.
The question title talks about removing the parameters, but your question refers to censoring the parameters similar to how Rails.application.config.filter_parameters works. If it's the latter, it looks like that's already been answered in Manually filter parameters in Rails. If it's the former, assuming a filter list, and a hash:
FILTER_LIST = [:password, :secret]
hash = {'password' => 123, :secret => 321, :ok => "this isn't going anywhere"}
then you could do this:
hash.reject { |k,v| FILTER_LIST.include?(k.to_sym) }
That'll cope with both string and symbol key matching, assuming the filter list is always symbols. Additionally, you could always use the same list as config.filter_parameters if they are going to be the same and you don't need a separate filter list:
hash.reject { |k,v| Rails.application.config.filter_parameters.include?(k.to_sym) }
And if you wanted to make this easier to use within your own logging, you could consider monkey patching the Hash class:
class Hash
def filter_like_parameters
self.reject { |k,v| Rails.application.config.filter_parameters.include?(k.to_sym) }
end
end
Then your logging code would become:
Rails.logger.info {name: 'me', secret: '1231234'}.filter_like_parameters.inspect
If you do monkey patch custom functionality to core classes like that though for calls you're going to be making a lot, it's always best to use a quite obtuse method name to reduce the likelihood of a clash with any other library that might share the same method names.
Hope that helps!
Related
In my application, the session hash can contain the keys sort and ratings (in addition to _csrf_token and session_id), depending on what action a user takes. That is, it can contain both of them, either one of them, or neither, depending on what a user does.
Now, I wish to call redirect_to in my application and, at the same time, restore any session information (sort or ratings) the user may have provided.
To do this, I want to insert whatever key-value session has currently got stored (out of sort and ratings) as query parameters in my call to redirect_to. So, the path might look something like /movies?sort=...&ratings=....
I don't know how to write the logic for this. How can I do this? And how do I go about selectively inserting query parameters while calling redirect_to? Is it even possible to do this?
Any help is greatly appreciated. Thank you in advance.
First just compose a hash containing the parameters you want - for example:
opts = session.slice(:sort, :ratings)
.merge(params.slice(:sort, :ratings))
.compact_blank
This example would contain the keys :sort, :ratings with the same keys from the parameters merged on top (taking priority).
You can then pass the hash to the desired path helper:
redirect_to foos_path(**opts)
You can either just pass a trailing hash option or use the params option to explitly set the query string:
irb(main):007:0> app.root_path(**{ sort: 'backwards' })
=> "/?sort=backwards"
irb(main):008:0> app.root_path(params: { ratings: 'XX' })
=> "/?ratings=XX"
irb(main):009:0> app.root_path(params: { })
=> "/"
An empty hash will be ignored.
If your calling redirect_to with a hash instead of a string you can add query string parameters with the params: key:
redirect_to { action: :foo, params: opts }
If you're working with an arbitrary given URL/path and want to manipulate the query string parameters you can use the URI module together with the utilities provided by Rack and ActiveSupport for converting query strings to hashes and vice versa:
uri = URI.parse('/foo?bar=1&baz=2&boo=3')
parsed_query = Rack::Utils.parse_nested_query(uri.query)
uri.query = parsed_query.except("baz").merge(x: 5).to_query
puts uri.to_s # => "/foo?bar=1&boo=3&x=5"
I want to filter certain user information so that sensitive info is not captured in the logs. However, in one case, I want to log the format instead of the default, "[FILTERED]".
For example, I'd like to make this transformation:
# user input value to log
"e6132 DG71" => "a9999 AA99"
This would help us troubleshoot users' issues.
Can I use a custom filter (maybe a proc?) for Rails' filter_parameters method?
Append a lambda filter
You can append a lambda filter to Rails.application.config.filter_parameters. Only parameters that aren't caught by the typical filtering will go on to the custom filtering.
# Typical filters - will show as [FILTERED]
Rails.application.config.filter_parameters += [
:password
]
# Custom filter - this only sees what the previous list didn't catch
Rails.application.config.filter_parameters << lambda do |param_name, value|
if %w[foo_param bar_param].include?(param_name) && value.respond_to?(:gsub!)
# Alter the string in place because we don't have access to
# the hash to update the key's value
value.gsub!(/[a-z]/, "a")
value.gsub!(/[A-Z]/, "A")
value.gsub!(/[0-9]/, "9")
# Stick this at the beginning to make it easy to see in logs
value.gsub!(/\A/, "[FORMAT FILTERED]")
end
end
Note that the default filters match as regexes; :password is treated as /password/. You could do likewise for the custom filter:
if [/foo/, /bar/].detect {|r| r.match(param_name) }
# ...
end
Reading the ActionDispatch::Http::FilterParameters docs for Rails 4.2, there is an option to pass a block to parameter_filter to customize the filter behaviour. I assume you can do something like this:
env["action_dispatch.parameter_filter"] = lambda do |k,v|
your_transformation(v) if k =~ /secret/i
end
where your_transformation knows how to deal with the user input labeled secret to transform it into a9999 AA99. I also guess you'll have to handle your already present filtered_attributes to keep the [FILTERED] behavior.
I'm migrating our app from 3.0 to 3.2.x. Earlier the streaming was done by the assigning the response_body a proc. Like so:
self.response_body = proc do |response, output|
target_obj = StreamingOutputWrapper.new(output)
lib_obj.xml_generator(target_obj)
end
As you can imagine, the StreamingOutputWrapper responds to <<.
This way is deprecated in Rails 3.2.x. The suggested way is to assign an object that responds to each.
The problem I'm facing now is in making the lib_obj.xml_generator each-aware.
The current version of it looks like this:
def xml_generator(target, conditions = [])
builder = Builder::XmlMarkup.new(:target => target)
builder.root do
builder.elementA do
Model1.find_each(:conditions => conditions) { |model1| target << model1.xml_chunk_string }
end
end
end
where target is a StreamingOutputWrapper object.
The question is, how do I modify the code - the xml_generator, and the controller code, to make the response xml stream properly.
Important stuff: Building the xml in memory is not an option as the model records are huge. The typical size of the xml response is around 150MB.
What you are looking for is SAX Parsing. SAX reads files "chunks" at a time instead of loading the whole file into DOM. This is super convenient and fortunately there are a lot of people before you who have wanted to do the same thing. Nokogiri offers XML::SAX methods, but it can get really confusing in the disastrous documentation and syntactically, it's a mess. I would suggest looking into something that sits on top of Nokogiri and makes getting your job done, a lot more simple.
Here are a few options -
SAX_stream:
Mapping out objects in sax_stream is super simple:
require 'sax_stream/mapper'
class Product
include SaxStream::Mapper
node 'product'
map :id, :to => '#id'
map :status, :to => '#status'
map :name_confirmed, :to => 'name/#confirmed'
map :name, :to => 'name'
end
and calling the parser in is also simple:
require 'sax_stream/parser'
require 'sax_stream/collectors/naive_collector'
collector = SaxStream::Collectors::NaiveCollector.new
parser = SaxStream::Parser.new(collector, [Product])
parser.parse_stream(File.open('products.xml'))
However, working with the collectors (or writing your own) and end up being slightly confusing, so I would actually go with:
Saxerator:
Saxerator gets the job doen and has some really handy methods for traversing into nodes that can be a little less complex than sax_stream. Saxerator also has a few really great configuration options that are well documented. Simple Saxerator example below:
parser = Saxerator.parser(File.new("rss.xml"))
parser.for_tag(:item).each do |item|
# where the xml contains <item><title>...</title><author>...</author></item>
# item will look like {'title' => '...', 'author' => '...'}
puts "#{item['title']}: #{item['author']}"
end
# a String is returned here since the given element contains only character data
puts "First title: #{parser.for_tag(:title).first}"
If you end up having to pull the XML from an external source (or it is getting updated frequently and do you don't want to have to update the version on your server manually, check out THIS QUESTION and the accepted answer, it works great.
You could always monkey-patch the response object:
response.stream.instance_eval do
alias :<< :write
end
builder = Builder::XmlMarkup.new(:target => response.stream)
...
I think the best way for me to explain this question is with example. Here is a simple method with a hash:
def getValues(table)
allColumns = {
'User' => ['first_name', 'last_name'],
'Vehicle' => ['make', 'model', 'id'],
}
end
I am trying to pass in the table and based on that table return a range of values. I would like to know what would be (performance-wise) the best way to accomplish this. Is it using a switch statement, if/else, some sort of loop? If you come up with an answer, please be as kind to include an example so that I may understand better.
I suggest you to rename the parameter first, maybe to table_name or something more descriptive.
Second it is kind of a convention in ruby to use method names separated by _, and avoid using camelCase as another languages.
Third, i would put the list on a constant variable or something, and just reference it inside the method.
Then you can look up the values of some hash based on a key just by using hash[key].
LIST = {
'User' => ['first_name', 'last_name'],
'Vehicle' => ['make', 'model', 'id'],
}
def get_values(table_name)
LIST[table_name]
end
Hash lookup by key is probably one of the most performant operations you could do with a collection, so there is no need to worry about it.
In Python, I can do this:
>>> import urlparse, urllib
>>> q = urlparse.parse_qsl("a=b&a=c&d=e")
>>> urllib.urlencode(q)
'a=b&a=c&d=e'
In Ruby[+Rails] I can't figure out how to do the same thing without "rolling my own," which seems odd. The Rails way doesn't work for me -- it adds square brackets to the names of the query parameters, which the server on the other end may or may not support:
>> q = CGI.parse("a=b&a=c&d=e")
=> {"a"=>["b", "c"], "d"=>["e"]}
>> q.to_params
=> "a[]=b&a[]=c&d[]=e"
My use case is simply that I wish to muck with the values of some of the values in the query-string portion of the URL. It seemed natural to lean on the standard library and/or Rails, and write something like this:
uri = URI.parse("http://example.com/foo?a=b&a=c&d=e")
q = CGI.parse(uri.query)
q.delete("d")
q["a"] << "d"
uri.query = q.to_params # should be to_param or to_query instead?
puts Net::HTTP.get_response(uri)
but only if the resulting URI is in fact http://example.com/foo?a=b&a=c&a=d, and not http://example.com/foo?a[]=b&a[]=c&a[]=d. Is there a correct or better way to do this?
In modern ruby this is simply:
require 'uri'
URI.encode_www_form(hash)
Quick Hash to a URL Query Trick :
"http://www.example.com?" + { language: "ruby", status: "awesome" }.to_query
# => "http://www.example.com?language=ruby&status=awesome"
Want to do it in reverse? Use CGI.parse:
require 'cgi'
# Only needed for IRB, Rails already has this loaded
CGI::parse "language=ruby&status=awesome"
# => {"language"=>["ruby"], "status"=>["awesome"]}
Here's a quick function to turn your hash into query parameters:
require 'uri'
def hash_to_query(hash)
return URI.encode(hash.map{|k,v| "#{k}=#{v}"}.join("&"))
end
The way rails handles query strings of that type means you have to roll your own solution, as you have. It is somewhat unfortunate if you're dealing with non-rails apps, but makes sense if you're passing information to and from rails apps.
As a simple plain Ruby solution (or RubyMotion, in my case), just use this:
class Hash
def to_param
self.to_a.map { |x| "#{x[0]}=#{x[1]}" }.join("&")
end
end
{ fruit: "Apple", vegetable: "Carrot" }.to_param # => "fruit=Apple&vegetable=Carrot"
It only handles simple hashes, though.