I'm trying to investigate if nosql injection is possible on Ruby on Rails with mongo and mongoid gems.
I did Mongodb's requests using Mongo::Client collections and models with Mongoid::Document inclusion.
I tried to pass some command characters like ' " \ ; { }, but is was sanitaized. Passing GET search?title[$ne]=foo was traited like {"title"=>"{\"$ne\"=>\"foo\"}"}, so it don't seems like any problem here.
Is any nosql injection possible if I use ordinary methods of this technology stack?
Common operations including queries and inserts/updates in Mongoid sanitize their inputs, thus most times one does not need to worry about "nosql injection".
However, there are methods that pass commands directly to the database, and in those cases it is important to carefully consider whether unsanitized user input can end up as a database command. For example, if Post is a Mongoid model, one can run the following command to create an infinite loop in a MongoDB server:
Post.all.map_reduce('function(){while(true);}','function(){}').out(inline:1).count
Another example is Database#command method provided by the driver to run arbitrary database commands: http://api.mongodb.com/ruby/current/Mongo/Database.html#command-instance_method. If an application places user input into parameters given to this method, this creates potential for "nosql injection".
Note also that it is not necessary to pass an unexpected command to the database - sometimes unexpected data is sufficient. See, for example, https://www.owasp.org/index.php/Regular_expression_Denial_of_Service_-_ReDoS. Assuming the Post model has a body field, passing an arbitrary regular expression from the user could be problematic:
# This might take a while
Post.where('body' => {'$regex' => '((a|a?|a*)*)*$'}).count
This is from the mongdodb docs
As a client program assembles a query in MongoDB, it builds a BSON
object, not a string. Thus traditional SQL injection attacks are not a
problem. More details and some nuances are covered below.
MongoDB represents queries as BSON objects. Typically client libraries
provide a convenient, injection free, process to build these objects.
Consider the following C++ example:
https://docs.mongodb.com/manual/faq/fundamentals/#how-does-mongodb-address-sql-or-query-injection
Possible, and easily overlooked.
In fact, you are pretty close.
Untrust sources are not just from GET parameters.
Mongoid won't help you with anything; in your example, what prevents the succeeded exploitation is the fact that, in RoR, you can't pass a Hash as a GET parameter.
An unsanitized parameter can come from JSON, like this one for example.
posts = PrivatePost.where({ owner_id: json_params[:owner_id] }).each.to_a
Where json_params[:owner_id] could contains { '$ne': 'the owner' },
which can leak posts to someone else.
Or mess with the operation scope via POST-based API:
Post.where({ _id: json_params[:id] }).delete_all
Related
I am new to RoR and I have a basic question.
A colleague told me that the following code can cause a NoSQL Injection:
User.find_by(username: param[:username])
I know that I am not supposed to read directly from param but I still think that the code cannot generate any NoSQL injections. Am I wrong?
Your colleague is wrong, and should produce a working exploit if he believes this is true.
Providing a hash of key/value pairs to Rail's find_by or where methods is the intended, secure, and correct way of using those methods. The keys and values are always made safe before being used in a query.
There are certain problems that Rails will not protect you from, for example, the value of params[:username] could be a gigabyte of text, a File object, a nested hash of key/value pairs, etc. but this isn't an "injection" per se. Any special characters that could cause the meaning of the query to be changed will have no effect.
What I'm going for is URL's very similar to Basecamp's:
https://3.basecamp.com/4000818/buckets/7452203/message_boards/1039416768
I have already achieved this functionality by following this guide, but I am unsatisfied with the process of needing to run potentially millions of .exists? lookups to find an open number and fear this will very quickly hamper performance of my app.
def set_hash_id
hash_id = nil
loop do
hash_id = SecureRandom.urlsafe_base64(9).gsub(/-|_/,('a'..'z').to_a[rand(26)])
break unless self.class.name.constantize.where(:hash_id => hash_id).exists?
end
self.hash_id = hash_id
end
I find it hard to believe that Basecamp is relying on something so inefficient on every record save and I'm looking to find out how they do it or to find a setup that will look the same but without the overhead of the linked tutorial.
I'd appreciate any input on methods to generate a non-sequential record ID. I am not interested in UUID's as I can't stand the non-pleasing URL's they generate. Also, they must be integers. Basically, exactly like the Basecamp URL but without the overhead of the exists? checks. Is it possible they are doing some kind of combination of numbers with an encoded timestamp or something to ensure there is no collisions? I have explored the hashids.org method but this does not generate integer-only hashes.
I am using Postgres as my database, in case this is helpful.
Efficiency-wise I think you should be fine. GitLab also uses something similar for unique token generation.
There's another issue though that's worth considering:
Your method does not guarantee to generate a unique key, as the operation is not atomic (neither is GitLab's). Between checking for uniqueness and writing the record to the database the same key could have been generated.
You have at least 2 options to handle this. Both solution should also be more efficient (which is your main concern).
Catch the DB's unique key constrain violation on save
def save
begin
self.hash_id = generate_hash_id
super
rescue ActiveRecord::RecordNotUnique => e
# 1. you may need to check the message for the specific constraint
# 2. you may need to implement an abort condition to prevent infinite retries
retry
end
end
You could can also do this in an ActiveRecord callback.
Have the DB generate the key
An alternative solution would be to have your database generate the unique key on create. A function like the one described in this blogpost A Better ID Generator For PostgreSQL may serve your purpose better.
This solution has the advantage that your application code does not need to be concerned about generating or catching collisions. The drawback is though that this solution is DB specific.
If we have a small table which contains relatively static data, is it possible to have Active Record load this in on startup of the app and never have to hit the database for this data?
Note, that ideally I would like this data to be join-able from other Models which have relationships to it.
An example might be a list of countries with their telephone number prefix - this list is unlikely to change, and if it did it would be changed by an admin. Other tables might have relationships with this (eg. given a User who has a reference to the country, we might want to lookup the country telephone prefix).
I saw a similar question here, but it's 6 years old and refers to Rails 2, while I am using Rails 5 and maybe something has been introduced since then.
Preferred solutions would be:
Built-in Rails / ActiveRecord functionality to load a table once on startup and if other records are subsequently loaded in which have relationships with the cached table, then link to the cached objects automatically (ie. manually caching MyModel.all somewhere is not sufficient, as relationships would still be loaded by querying the database).
Maintained library which does the above.
If neither are available, I suppose an alternative method would be to define the static dataset as an in-memory enum/hash or similar, and persist the hash key on records which have a relationship to this data, and define methods on those Models to lookup using the object in the hash using the key persisted in the database. This seems quite manual though...
[EDIT]
One other thing to consider with potential solutions - the manual solution (3) would also require custom controllers and routes for such data to be accessible over an API. Ideally it would be nice to have a solution where such data could be offered up via a RESTful API (read only - just GET) if desired using standard rails mechanisms like Scaffolding without too much manual intervention.
I think you may be discounting the "easy" / "manual" approach too quickly.
Writing the data to a ruby hash / array isn't that bad an idea.
And if you want to use a CRUD scaffold, why not just use the standard Rails model / controller generator? Is it really so bad to store some static data in the database?
A third option would be to store your data to a file in some serialized format and then when your app loads read this and construct ActiveRecord objects. Let me show an example:
data.yml
---
- a: "1"
b: "1"
- a: "2"
b: "2"
This is a YAML file containing an array of hashes; you can construct such a file with:
require 'yaml'
File.open("path.yml", "w") do |f|
data = [
{ "a" => "1", "b" => 1 },
{ "a" => "2", "b" => 2 }
]
f.write(YAML.dump(data))
end
Then to load the data, you might create a file in config/initializers/ (everything here will be autoloaded by rails):
config/initializers/static_data.rb
require 'yaml'
# define a constant that can be used by the rest of the app
StaticData = YAML.load(File.read("data.yml")).map do |object|
MyObjectClass.new(object)
end
To avoid having to write database migrations for MyObjectClass (when it's not actually being stored in the db) you can use attr_accessor definitions for your attributes:
class MyObjectClass < ActiveRecord::Base
# say these are your two columns
attr_accessor :a, :b
end
just make sure not to run stuff like save, delete, or update on this model (unless you monkeypatch these methods).
If you want to have REST / CRUD endpoints, you'd need to write them from scratch because the way to change data is different now.
You'd basically need to do any update in a 3 step process:
load the data from YAML into a Ruby object list
change the Ruby object list
serialize everything to YAML and save it.
So you can see you're not really doing incremental updates here. You could use JSON instead of YAML and you'd have the same problem. With Ruby's built in storage system PStore you would be able to update objects on an individual basis, but using SQL for a production web app is a much better idea and will honestly make things more simple.
Moving beyond these "serialized data" options there are key-val storage servers store data in memory. Stuff like Memcached and Redis.
But to go back to my earlier point, unless you have a good reason not to use SQL you're only making things more difficult.
It sounds like FrozenRecord would be a good match for what you are looking for.
Active Record-like interface for read only access to static data files of reasonable size.
Let's say I need to implement a search algorithm for a product catalog database. This would include multiple joins across the products table, manufacturers table, inventory table, etc. etc.
In .NET / MSSQL, I would isolate such logic in a DB stored procedure, then write a wrapper method in my data access layer of my .NET app to simply call this stored procedure.
How does something like this work in RoR? From my basic understanding, RoR uses its ORM by default. Does this mean, I have to move my search logic into the application layer, and write it using its ORM? The SQL stored proc is pretty intense... For performance, it needs to be in the stored procedure.
How does this work in RoR?
Edit: From the first two responses, I gather that ActiveRecord is the way to do things in Ruby. Does this mean that applications that require large complex queries with lots of joins, filtering and even dynamic SQL can (should) be re-written using ActiveRecord classes?
Thanks!
While it is possible to run raw SQL statements in Rails, using the execute method on a connection object, by doing so you will forfeit all the benefits of ActiveRecord. If you still want to go down this path, you can use it like so:
ActiveRecord::Base.connection.execute("call stored_procedure_name")
Another option to explore might be to create a "query object" to encapsulate your query logic. Inside, you could still use ActiveRecord query methods. ActiveRecord has become fairly proficient in optimizing your SQL queries, and there is still some manual tweaking you could do.
Below is a simple scaffoold for such an object:
# app/queries/search_products.rb
class SearchProducts
def initialize(params)
#search = search
end
def call
Product.where(...) # Plus additional search logic
end
end
The third option would be to go with something like elasticsearch-rails or sunspot. This will require some additional setup time and added complexity, but might pay off down the line, if your search requirements change.
Stored procedure is one of way to make the apps can be faster sometimes but it will need high costs and time to debug code for developer. Rails is using ActiveRecord ORM so if you want to use stored procedure that will lead to the main function ActiveRecord unused well.
There are some explains about rails
stored-procedures-in-ruby-on-rails and using stored procedure
I am looking for a Ruby/Rails tool that will help me accomplish the following:
I would like to store the following string, and ones similar to it, in my database. When an object is created, updated, deleted, etc., I want to run through all the strings, check to see if the CRUD event matches the conditions of the string, and if so, run the actions specified.
When a new ticket is created and it's category=6 then notify user 1234 via email
I am planning to create an interface that builds these strings, so it doesn't need to be a human-readable string. If a JSONish structure is better, or a tool has an existing language, that would be fantastic. I'm kinda thinking something along the lines of:
{
object_types: ['ticket'],
events: ['created', 'updated'],
conditions:'ticket.category=6',
actions: 'notify user',
parameters: {
user:1234,
type:'email'
}
}
So basically, I need the following:
Monitor CRUD events - It would be nice if the tool had a way to do this, but Ican use Rails' ModelObservers here if the tool doesn't natively provide it
Find all matching "rules" - This is my major unknown...
Execute the requested method/parameters - Ideally, this would be defined in my Ruby code as classes/methods
Are there any existing tools that I should investigate?
Edit:
Thanks for the responses so far guys! I really appreciate you pointing me down the right paths.
The use case here is that we have many different clients, with many different business rules. For the rules that apply to all clients, I can easily create those in code (using something like Ruleby), but for all of the client-specific ones, I'd like to store them in the database. Ideally, the rule could be written once, stored either in the code, or in the DB, and then run (using something Resque for performance).
At this point, it looks like I'm going to have to roll my own, so any thoughts as to the best way to do that, or any tools I should investigate, would be greatly appreciated.
Thanks again!
I don't think it would be a major thing to write something yourself to do this, I don't know of any gems which would do this (but it would be good if someone wrote one!)
I would tackle the project in the following way, the way I am thinking is that you don't want to do the rule matching at the point the user saves as it may take a while and could interrupt the user experience and/or slow up the server, so...
Use observers to store a record each time a CRUD event happens, or to make things simpler use the Acts as Audited gem which does this for you.
1.5. Use a rake task, running from your crontab to run through the latest changes, perhaps every minute, or you could use Resque which does a good job of handling lots of jobs
Create a set of tables which define the possible rules a user could select from, perhaps something like
Table: Rule
Name
ForEvent (eg. CRUD)
TableInQuestion
FieldOneName
FieldOneCondition etc.
MethodToExecute
You can use a bit of metaprogramming to execute your method and since your method knows your table name and record id then this can be picked up.
Additional Notes
The best way to get going with this is to start simple then work upwards. To get the simple version working first I'd do the following ...
Install acts as audited
Add an additional field to the created audit table, :when_processed
Create yourself a module in your /lib folder called something like processrules which roughly does this
3.1 Grabs all unprocessed audit entries
3.2 Marks them as processed (perhaps make another small audit table at this point to record events happening)
Now create a rules table which simply has a name and condition statement, perhaps add a few sample ones to get going
Name: First | Rule Statement: 'SELECT 1 WHERE table.value = something'
Adapt your new processrules method to execute that sql for each changed entry (perhaps you want to restrict it to just the tables you are working with)
If the rule matched, add it to your log file.
From here you can extrapolate out the additional functionality you need and perhaps ask another question about the metaprogramaming side of dynamically calling methods as this question is quite broad, am more than happy to help further.
I tend to think the best way to go about task processing is to setup the process nicely first so it will work with any server load and situation then plug in the custom bits.
You could make this abstract enough so that you can specify arbitrary conditions and rules, but then you'd be developing a framework/engine as opposed to solving the specific problems of your app.
There's a good chance that using ActiveRecord::Observer will solve your needs, since you can hardcode all the different types of conditions you expect, and then only put the unknowns in the database. For example, say you know that you'll have people watching categories, then create an association like category_watchers, and use the following Observer:
class TicketObserver < ActiveRecord::Observer
# observe :ticket # not needed here, since it's inferred by the class name
def after_create(ticket)
ticket.category.watchers.each{ |user| notify_user(ticket, user) }
end
# def after_update ... (similar)
private
def notify_user(ticket, user)
# lookup the user's stored email preferences
# send an email if appropriate
end
end
If you want to store the email preference along with the fact that the user is watching the category, then use a join model with a flag indicating that.
If you then want to abstract it a step further, I'd suggest using something like treetop to generate the observers themselves, but I'm not convinced that this adds more value than abstracting similar observers in code.
There's a Ruby & Rules Engines SO post that might have some info that you might find useful. There's another Ruby-based rules engine that you may want to explore that as well - Ruleby.
Hope that this helps you start your investigation.