I am using Ruby on Rails 4.1, Ransack and attr_encrypted. I have sensitive data being stored in my database and I want to protect it using the gem attr_encrypted.
As I expected, I got zero results when searching encrypted test data with Ransack.
I tried the following solution to but it didn't seem to work for me. I was under the impression that the load function was used to return the decrypted value.
ReportsController
def index
#report_list = Report.all.load
#q = #report_list.search(params[:q])
#reports = #q.result(distinct: true).order('created_at DESC')
end
Has anyone had any experience searching across encrypted data and could help me generate a working solution?
load will cause the Active Record Collection to execute a query and retrieve the results matching your query (in addition to running after_create call backs which I believe is where the decrypt you were expecting is happening).
def index
#returns all records in DB
#report_list = Report.all.load
#I'm surprised these aren't throwing undefined method search of Array (or something similar)
#q = #report_list.search(params[:q])
#reports = #q.result(distinct: true).order('created_at DESC')
end
I would like to precede this with, I normally do this thing manually and am not familiar with attr_encrypted or Ransack, but I believe these concepts are general enough they could be applied to any setup. So, as to your question, 2 possibilities.
If your ok searching for exact values:
Model.where(encrypted_field: encrypt(params[:value])).first
where encrypt is a method that encrypts and returns the passed string.
Secondly (and painfully)
Model.all.delete_if{|m| !m.encrypted_field.include?(params[:value]) }
This will literally pull, decrypt, and scan every entry in your database.
I would highly recommend not doing this, but you need to do what you need to do.
If you absolutely need to have the information encrypted but still need to be able to do searches like this. I would highly recommend adding tags of some sort to your model. This would allow you to remove sensitive information but still search by some attributes.
Related
I'm not sure if this is just a lacking of the Rails language, or if I am searching all the wrong things here on Stack Overflow, but I cannot find out how to add an attribute to each record in an array.
Here is an example of what I'm trying to do:
#news_stories.each do |individual_news_story|
#user_for_record = User.where(:id => individual_news_story[:user_id]).pluck('name', 'profile_image_url');
individual_news_story.attributes(:author_name) = #user_for_record[0][0]
individual_news_story.attributes(:author_avatar) = #user_for_record[0][1]
end
Any ideas?
If the NewsStory model (or whatever its name is) has a belongs_to relationship to User, then you don't have to do any of this. You can access the attributes of the associated User directly:
#news_stories.each do |news_story|
news_story.user.name # gives you the name of the associated user
news_story.user.profile_image_url # same for the avatar
end
To avoid an N+1 query, you can preload the associated user record for every news story at once by using includes in the NewsStory query:
NewsStory.includes(:user)... # rest of the query
If you do this, you won't need the #user_for_record query — Rails will do the heavy lifting for you, and you could even see a performance improvement, thanks to not issuing a separate pluck query for every single news story in the collection.
If you need to have those extra attributes there regardless:
You can select them as extra attributes in your NewsStory query:
NewsStory.
includes(:user).
joins(:user).
select([
NewsStory.arel_table[Arel.star],
User.arel_table[:name].as("author_name"),
User.arel_table[:profile_image_url].as("author_avatar"),
]).
where(...) # rest of the query
It looks like you're trying to cache the name and avatar of the user on the NewsStory model, in which case, what you want is this:
#news_stories.each do |individual_news_story|
user_for_record = User.find(individual_news_story.user_id)
individual_news_story.author_name = user_for_record.name
individual_news_story.author_avatar = user_for_record.profile_image_url
end
A couple of notes.
I've used find instead of where. find returns a single record identified by it's primary key (id); where returns an array of records. There are definitely more efficient ways to do this -- eager-loading, for one -- but since you're just starting out, I think it's more important to learn the basics before you dig into the advanced stuff to make things more performant.
I've gotten rid of the pluck call, because here again, you're just learning and pluck is a performance optimization useful when you're working with large amounts of data, and if that's what you're doing then activerecord has a batch api you should look into.
I've changed #user_for_record to user_for_record. The # denote instance variables in ruby. Instance variables are shared and accessible from any instance method in an instance of a class. In this case, all you need is a local variable.
I am using ruby 1.8.7 and rails 2.3.2
The following code is prone to sql injection
params[:id] = "1) OR 1=1--"
User.delete_all("id = #{params[:id]}")
My question is by doing the following will be the best solution to avoid sql injection or not. If not then what is the best way to do so?
User.delete_all("id = #{params[:id].to_i}")
What about:
User.where(id: params[:id]).delete_all
Ok sorry for Rails 2.x its:
User.delete_all(["id = ?", params[:id]])
Check doc
Btw, be sure you want to use delete_all instead of destroy_all, the former doesn't trigger callbacks.
You can use this also
User.delete(params[:id])
The other answers answer this well for Rails and it'll work fine if you follow their suggestions. In a more generic setting when you have to handle this yourself you can typically use a regular expression to extract a value that's in an expected format. This is really simple with an integer id. Think of it like this:
if params[:id] =~ /(\d+)/
safe_id = $1.to_i
# do something with safe_id now
end
That gets a little more complicated when you're handling strings and arbitrary data. If you have to handle such data then you can use the quoting methods available for the database adapters. In Rails this is ultimately rolled into a consistent interface:
safe_string = ActiveRecord::Base.connection.quote(unsafe_string)
For most database systems this will handle single quotes and backslashes in a special manner.
If you're outside of Rails you will have to use the quoting methods specific to your database adapter, but usage is quite similar.
The takeaway:
If your data has a particular format, enforce the format with a regular expression
Otherwise, use your database adapter's quoting function to make the data "safe" for use in a query
Rails will handle most of this for you if you properly use the various methods and "conditions"
Use the rails methods to pass your where options. You can always hardcode them, as in the example that you give, but the usual way would be something like:
User.where(:id => params[:id]).delete_all
User.where("id = ?", params[:id]).delete_all
User.where("id = :id", :id => params[:id]).delete_all
They are well tested and in case a new vulnerability is detected, an update will fix the problem and your code will not need to be changed.
By the way, if you just want to delete 1 record based on its id, what I would do is:
User.find(params[:id]).destroy
I'm working on an audit trail of sorts for an app so that the user can see what is being changed throughout the system.
I have a hash of changes from ActiveRecord Dirty, like follows:
{"ingredient_type_id"=>[nil, 199575006], "name"=>[nil, "asdfg"], "amount"=>[nil, 3.0], "unit"=>[nil, "x"], "notes"=>[nil, "asdf"]}
This works great and I can parse what I need to output and create database records with the info.
I just have one question - How can I get associated objects from this? In this case, the ingredient_type? I actually want to output something like:
"Ingredient type was changed to #{IngredientType.find(199575006).name}."
But I'm not sure how I would parse that hash on a dynamic basis to do that.
Pretty much the way you've suggested I'd have thought, But you don't need to parse the hash for the changes, Dirty gives you much more than that
if ingredient_type_id_changed?
unless ingredient_type_id.blank?
ingredient_name = IngredientType.find(ingredient_type_id).name
else
ingredient_name = 'blank'
end
end
You might even be able to do ingredient_type.name, Not sure at that point if active record dirty will let you go through the association. If you test it (or if anyone else knows) let me know
I have a controller method which currently looks like:
#people = Person.where(conditions).includes(eager_fetch).all
I'm now trying to make the controller cache-aware. Since the eager fetch is rather expensive, I want to avoid loading as much data as possible. If it's relevant, the output is XML from an RPC style endpoint. I've arrived at:
#people = Person.where(conditions).all
#fragments = {}
#people.dup.each do |person|
cache_key = "fragment-for-#{person.id}-#{person.updated_at.to_i}"
fragment = Rails.cache.fetch(cache_key)
unless fragment.nil?
#fragments[person.id] = fragment
#people.delete person
end
end
#people = Person.where(:id => #people.collect(&:id)).includes(eager_fetch).all
There's another possibility, which is very much the same, except instead of re-querying on the last line,
Person.send :preload_associations, #people, eager_fetch
Am I missing an important piece of API for handling this correctly? Currently on Rails 3.0.12, but will be upgrading to 3.2.x, so a solution that only works with 3.2.x would be fine. Neither of my solutions seem elegant to me.
(I've anonymized and simplified this code, apologies if I've left out anything important)
Don't rely on ActiveRecord's eager loading. It will load everything that isn't in the ActiveRecord per-request query cache.
Instead query for your primary object, and then use your own crafty method to fetch the cached things and query the slower datastore for the missed ID's.
I have the following:
#users = User.all
User has several fields including email.
What I would like to be able to do is get a list of all the #users emails.
I tried:
#users.email.all but that errors w undefined
Ideas? Thanks
(by popular demand, posting as a real answer)
What I don't like about fl00r's solution is that it instantiates a new User object per record in the DB; which just doesn't scale. It's great for a table with just 10 emails in it, but once you start getting into the thousands you're going to run into problems, mostly with the memory consumption of Ruby.
One can get around this little problem by using connection.select_values on a model, and a little bit of ARel goodness:
User.connection.select_values(User.select("email").to_sql)
This will give you the straight strings of the email addresses from the database. No faffing about with user objects and will scale better than a straight User.select("email") query, but I wouldn't say it's the "best scale". There's probably better ways to do this that I am not aware of yet.
The point is: a String object will use way less memory than a User object and so you can have more of them. It's also a quicker query and doesn't go the long way about it (running the query, then mapping the values). Oh, and map would also take longer too.
If you're using Rails 2.3...
Then you'll have to construct the SQL manually, I'm sorry to say.
User.connection.select_values("SELECT email FROM users")
Just provides another example of the helpers that Rails 3 provides.
I still find the connection.select_values to be a valid way to go about this, but I recently found a default AR method that's built into Rails that will do this for you: pluck.
In your example, all that you would need to do is run:
User.pluck(:email)
The select_values approach can be faster on extremely large datasets, but that's because it doesn't typecast the returned values. E.g., boolean values will be returned how they are stored in the database (as 1's and 0's) and not as true | false.
The pluck method works with ARel, so you can daisy chain things:
User.order('created_at desc').limit(5).pluck(:email)
User.select(:email).map(&:email)
Just use:
User.select("email")
While I visit SO frequently, I only registered today. Unfortunately that means that I don't have enough of a reputation to leave comments on other people's answers.
Piggybacking on Ryan's answer above, you can extend ActiveRecord::Base to create a method that will allow you to use this throughout your code in a cleaner way.
Create a file in config/initializers (e.g., config/initializers/active_record.rb):
class ActiveRecord::Base
def self.selected_to_array
connection.select_values(self.scoped)
end
end
You can then chain this method at the end of your ARel declarations:
User.select('email').selected_to_array
User.select('email').where('id > ?', 5).limit(4).selected_to_array
Use this to get an array of all the e-mails:
#users.collect { |user| user.email }
# => ["test#example.com", "test2#example.com", ...]
Or a shorthand version:
#users.collect(&:email)
You should avoid using User.all.map(&:email) as it will create a lot of ActiveRecord objects which consume large amounts of memory, a good chunk of which will not be collected by Ruby's garbage collector. It's also CPU intensive.
If you simply want to collect only a few attributes from your database without sacrificing performance, high memory usage and cpu cycles, consider using Valium.
https://github.com/ernie/valium
Here's an example for getting all the emails from all the users in your database.
User.all[:email]
Or only for users that subscribed or whatever.
User.where(:subscribed => true)[:email].each do |email|
puts "Do something with #{email}"
end
Using User.all.map(&:email) is considered bad practice for the reasons mentioned above.