How to query MongoDB directly from Ruby instead of using Mongoid? - ruby-on-rails

I am writing a migration for a Rails application that uses MongoDB and Mongoid. My migration currently uses my models that use Mongoid to query and update records, but the performance is sub-par. I am essentially updating all records in a large collection and making n+20 queries. I killed the migration after taking an hour to run locally (and didn't finish). I would like to be able to run raw queries to mongo without too much effort. I'm assuming there is some way to access a mongo driver from Mongoid since Mongoid has already loaded a connection to the database. How can I access the database to run my update queries direcly?

If you're using Mongoid 3, it provides easy access to its MongoDB driver: Moped. Here's an example of accessing some raw data without using Models to access the data:
db = Mongoid::Sessions.default
# inserting a new document
collection = db[:collection_name]
collection.insert(name: 'my new document')
# finding a document
doc = collection.find(name: 'my new document').first
# iterating over all documents in a collection
collection.find.each do |document|
puts document.inspect
end

For Mongoid 5:
db = Mongoid::Clients.default
collection = db[:collection_name]
Now we can perform queries on the collection

Here how you do it (This would work for 2+ and 3+ as well)
1) All your Model exhibit this behavior you have include Mongoid::Document inside all your model so technically each document is mapped in monogodb thru moped or mongodb-ruby driver via mongoid
so If you have model Like
class PerformerSource
include Mongoid::Document
## Definition
end
Now you can run Mongo Query using the driver (Moped or Mongodb-ruby driver) like this
PerformerSource.collection.insert("something")
## where something is json document you want to insert
This would give u the moped (if using mongoid 3) connection for that document
2) You can also do it something like this
Mongoid::Sessions.default.collections.find { |document| document.name == "performer_sources"}.insert("something")
How to more on mongo query and how mongoid map those using moped u can follow this section of querying where it describe how query is acheived internally via moped
Hope this help

For Mongoid 6:
db = Mongoid::default_client
collection = db[:collection_name]

The short answer is Moped. This is the lower-level API that Mongoid is built upon and will be available if you already use Mongoid. The Moped API is a thin wrapper around the raw MongoDB operations. The documentation here: http://mongoid.org/en/moped/docs/driver.html should be useful.

Like anyone has mentioned here, your answer is Moped.
Here is my example for a ruby script (simple file test.rb)
Define a mongoid.yml (in this case, at localhost)
development:
sessions:
default:
database: test_development
hosts:
- localhost:27017
options:
Set load config and test collection
#!/usr/bin/env ruby
require 'mongoid'
Mongoid.load!("path/to/file/mongoid.yml",:development) # :development corresponds to mongoid.yml first line environment
db = Mongoid::Sessions.default
puts "Collection documents count :> #{db[:collection].find.count}"

If you using mongoid 5(five) I would recommend using this.
Item.collection.update_one({_id: BSON::ObjectId('55512b7070722d22d3050000')}, '$set' => { 'category_name': 'Test' })
The trick to this is the BSON::ObjectID. This is like in the mongo query if you want to search for a single id.
db.items.update({ '_id': ObjectId("55512b7070722d22d3050000") }, { $set: {'category_name': 'Test' } })
Above is the mongo version of the query. I found translating ruby code to mongo code is the hard part as there are a few pieces that can be a bit hard to find in the documentation.
http://www.rubydoc.info/gems/mongo/Mongo%2FCollection%3Aupdate_one

Related

Collections not being read correctly after mongomapper is replaced by mongoid and mongo driver is updated

MongoDB V3.2
Upgraded the following the gems:
Ruby Mongo Driver from 1.11.1 to 2.10.4 + dependencies
Replaced MongoMapper 0.13.1 with Mongoid 5.4.1 + dependencies
After these changes, I noticed immediately that any collections that were placed in an additional module (FolderModuleName::ClassName) that could display data in MongoMapper would no longer display any data.
The only collections that would display data would be those without any modules for example a class that looked like this class DataClass.
I was able to figure out the issue by using the rails console and connecting to my database using the ruby mongo driver. (https://docs.mongodb.com/ruby-driver/master/quick-start/)
client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'dbname')
db = client.database
db.collections # returns a list of collection objects
db.collection_names # returns a list of collection names
Using db.collection_names in the rails console I was able to see that any collection with a module was saved like this:
module_name.collection_name
After my upgrade the only collections names with modules I could read were:
module_name_collection_name
With this information, I added the following code to the ruby models affected:
store_in collection: 'module_name.collection_name'
This fixed my issue.
The reason collections without modules could be read without using the code above is because the collection names were simply stored as:
collection_name
Adding 'store in' in that case would just be redundant.

How can I edit existing activerecord objects in a sql database using ruby scripts?

How can activerecord objects be edited externally with scripts? I'm currently using the activerecord-import gem to insert objects into the database. Is it possible to edit existing records using this gem or using another tool or gem?
That's a bit unspecific. There are quite a few methods. Here's the best:
You can run rails on the console. Try irb in your project's directory, and you're on a REPL that allows you to run code live. Just try something like:
that_guy = User.find(4)
that_guy.name = "John"
that_guy.save!
(Replace User, 4 and name with a model you have, a record's id and some string attribute).
And of course you can also just run a file of ruby against your project: rails runner your_script.rb
And, if I misunderstood you: you can of course access the database through any other means/languages/libraries.

Ruby on rails -elasticsearch tire impoting existing db

I have a Ruby on rails 3.2 application. I want to enable text based search on a model that has a lot of data already populated in it. Suppose the name of the model class is Post. I am planning on using elasticsearch since I heard it is one of the best real-time search engines around and I am using tire gem so that my application can interact with elasticsearch.
As I am new to elasticsearch I am having trouble creating indices for the existing data for the model. I am using mongodb as the backend database. Can anyone tell me how to import the indices.
I have already tried
Tire.index "posts" do
import Post.all
end
The error that I got was :
BSON::InvalidObjectId: illegal ObjectId format: Career Guidance
from /Users/anirvan/.rvm/gems/ruby-1.9.3-p125/gems/bson-1.5.1/lib/bson/types/object_id.rb:126:in `from_string'
Can anyone help me out here ?
I use Mysql and this code in bash(from railscasts.com):
rake environment tire:import CLASS=Post FORCE=true
http://www.elasticsearch.org/guide/appendix/clients.html
tire: Ruby API & DSL, with full Rails ActiveModel compatibility and mongoid integration through mebla.
Try it, maybe help you.

How to get list of mongodb databases and collections list from a ruby on rails app

I am using Rails 3 and Mongoid gem. But I need to fill a combobox with the list of mongodb databases. In mongodb shell we can list databases with "show dbs" command. Also there is getDBNameList() and db.getCollectionNames() commands in mongodb drivers. But I could not figure out how to use these commands from a ruby on rails app.
Also I wonder; if I can get databases and collections list with using mongoid gem. Because I am sure that I had read that mongoid supports using more than one database, but I think it was model dependent.
So what do you think; is there any solution or I have to use mongo-ruby-driver gem, not mongoid.
In mongoid 3
Mongoid.default_session.collections # returns the collections
I usually extract the names as follows:
Mongoid.default_session.collections.map(&:name).sort
You can do the following using the mongo ruby driver:
require 'rubygems'
require 'mongo'
connection = Mongo::Connection.new("localhost")
connection.database_names.each do |name|
db = connection.db(name)
db.collections.each do |collection|
puts "#{name} - #{collection.name}"
end
end
It would be easier to get the Mongo::DB out of the Mongoid config:
db = Mongoid::Config.master
db.collection_names
A short version.
db = Mongoid.master
db.collection_names

Which MongoDB DSL should I learn?

Im using MongoDB and Ruby.
I have noticed there are different DSL:s.
The Javascript DSL used with the MongoDB client (mongo):
show dbs
use my_db
db.person.find({first_name: "Syd"})
The Ruby DSL used with the Ruby driver for MongoDB:
connection = Mongo::Connection.new
connection.database_names.each { |name| puts name }
connection.database_info.each { |info| puts info.inspect}
person.find({"hello" => "world"})
Then the MongoID/MongoMapper DSL for MongoDB:
Person.desc(:last_name).asc(:first_name)
Person.descending(:last_name).ascending(:first_name)
Person.all(:conditions => { :first_name => "Syd" })
Questions:
Is it correct MongoID/MongoMapper is build on top of the Ruby DSL that is built on top of MongoDB client's DSL?
Should I learn all three DSL:s or just make my pick depending on the level of abstraction I want?
Are there any reasons I would like to learn/use the MongoDB client DSL? Can I use it in a script or is it just interactive with it's client (mongo)?
Thanks!
Learn all three.
The first one is going to be heavily used when you want to test query or find data etc, especially when you are in production. You would want to use the mongo client to do this kind of stuff.
The second one is used when the driver DSL does not support the features on the mongo. e.g:
At some stage you can not use the $or operator with MongoMapper when it was already supported on mongo 1.5
The last time I used mongoid and mongomapper does not support mapping to GridFS so you would use the driver API for this
And the last time I used, mongoid and mongomapper does not support map-reduce again you have to use the driver API for this situation
MongoMapper and Mongoid is used to map your domain object to mongo document, at some stage where the ODM is lack of you have to have the fallback plan, which is to use the mongo driver API.
Hope that helps.

Resources