Ruby on Rails: Concatenate results of Mongoid criterias and paging - ruby-on-rails

I'm pretty sure that I'm doing something wrong. Consider the following code:
criteria1 = Model.where(...)
criteria2 = Model.where(...)
results = (criteria1.to_a + criteria2.to_a)[offset..(offset + count_per_page - 1)]
This code concatenates results of two different criterias and get a certain number of results with a given offset (paging).
The problem in this code is implicit. The to_a method call actually loads all results of a criteria to the memory as an array.
Now consider a really huge collection... The to_a call slows all things down dramatically.
What I wish to do is something like this:
criteria1 = Model.where(...)
criteria2 = Model.where(...)
# A criteria, which returns results of the first criteria concatenated with results of the second criteria
criteria = criteria1 + criteria2
results = criteria.offset(offset).limit(count_per_page)
The important thing is that results of the second criteria goes after results of the first criteria.
Any clues how is it possible to achieve with Mongoid?
Thanks!
UPDATE
Gergo Erdosi suggested to use merge method. I've tried to use this and it is not what I'm looking for. The problem here is the following:
criteria1 = Model.where(:name => "John", :age => "23")
criteria2 = Model.where(:name => "Bob", :gender => "male")
criteria = criteria1.merge(criteria2)
p criteria.selector
# prints: { "name" => "Bob", :age => 23, :gender => "male" }
So here are two problems:
merge doesn't produce OR, it overrides common keys of the first query with the second;
Even if we use Model.or({ :name => "John" }, { :name => "Bob" }) or Model.in(:name => ["John", "Bob"]) results won't have the right order. I wish results of the first criteria go first and then results of the second criteria go after.
It is possible that I don't understand something and Gergo's answer is right. Do you have any other ideas? Thanks.
UPDATE 2
Thank you Gergo for helping me out here. Let's try a simple example in Mongo shell:
// Fill out test db with some simple documents.
for (var i = 0; i < 10; ++i) { db.users.insert({ name: i % 2 ? "John" : "Bob", age: Math.round(Math.random() * 100) }); }
// These queries give me the same order of documents.
db.users.find({ name: { $in: ["Bob", "John"] } });
db.users.find({ $or: [{ name: "Bob" }, { name: "John" }] });
// Like this:
{ "_id" : ObjectId("53732076b110ab9be7619a8e"), "name" : "Bob", "age" : 69 }
{ "_id" : ObjectId("53732076b110ab9be7619a8f"), "name" : "John", "age" : 63 }
{ "_id" : ObjectId("53732076b110ab9be7619a90"), "name" : "Bob", "age" : 25 }
{ "_id" : ObjectId("53732076b110ab9be7619a91"), "name" : "John", "age" : 72 }
// ...
// But I wish to get concatenated results of these queries:
db.users.find({ name: "Bob" });
db.users.find({ name: "John" });
// Like this (results of the first criteria go first):
{ "_id" : ObjectId("53732076b110ab9be7619a8e"), "name" : "Bob", "age" : 69 }
{ "_id" : ObjectId("53732076b110ab9be7619a90"), "name" : "Bob", "age" : 25 }
// ...
{ "_id" : ObjectId("53732076b110ab9be7619a8f"), "name" : "John", "age" : 63 }
{ "_id" : ObjectId("53732076b110ab9be7619a91"), "name" : "John", "age" : 72 }
// ...
Notice, that I cannot use a simple sorting here, because the data in the real application is more complex. In the real application criterias look like these:
// query variable is a string
exact_match_results = Model.where(:name => query)
inexact_match_results = Model.where(:name => /#{query}/i)
So we cannot just sort alphabetically here.

Use the merge method:
criteria = criteria1.merge(criteria2)
results = criteria.offset(offset).limit(count_per_page)
You can see the details in the method description.
Edit: As pointed out, merge doesn't produce an OR query.
irb(main):010:0> Model.where(name: 'John').merge(Model.where(name: 'Bob'))
=> #<Mongoid::Criteria
selector: {"name"=>"Bob"}
options: {}
class: Model
embedded: false>
Which is not the expected behavior in this case. The reason is that merge uses Hash.merge which behaves this way. The relevant code from Criteria.merge:
selector.merge!(criteria.selector)
This can be illustrated as:
irb(main):011:0> {name: 'John'}.merge({name: 'Bob'})
=> {:name=>"Bob"}
Because of this, it's not easy to give a general advice on how to merge two criteria in a way that the result is an OR query. But with a little change in the criteria, it's possible. For example:
criteria1 = Model.any_of(name: 'John').where(age: '23')
criteria2 = Model.any_of(name: 'Bob').where(gender: 'male')
The result of the merge is an OR query which contains both names:
irb(main):014:0> criteria1.merge(criteria2)
=> #<Mongoid::Criteria
selector: {"$or"=>[{"name"=>"John"}, {"name"=>"Bob"}], "age"=>"23", "gender"=>"male"}
options: {}
class: Model
embedded: false>

Related

how to iterate through json result in ruby on rails?

I have a function in ruby on rails, where I make a call and get the following json data. now i want to add another “key” : "value" to each item in the json data and return the new or modified json object from the function. what is the best way to do this in ruby?
def get_results
results = getJsonData();
end
[
{ "name" : "Harry Potter", "rating" : 1, },
{ "name" : "Lord of the rings", "rating" : 2, },
{ "name" : "game of thrones", "rating" : 3, },
]
You can use map to transform every element of a collection
def get_results
results = getJsonData();
results.map { |item| item.merge("year" => 2000) }
end
In this case I'm adding a key "year" to your hashes with value 2000, it depends what you want to do

mongodb: best way to get specific documents and then the rest

lets say I have 1000 documents where each one has:
user_id
text
Now, I would like to pull all those documents but first pull the documents from a few specific users (given an array of user ids) and then all the rest.
I was thinking to use map reduce to create a new weight inline attribute if the user_id exists in the specific users array (using scope to pass the array) and then to sort that new attribute. But from what I could understand, you can not sort after map reduce.
Any one has a good suggestion how to pull this off? Any suggestion will be welcome.
Thanks!
Well there isn't a lot of detail here, but I can give a sample case for consideration. Consider the following set of documents:
{ "user" : "fred", "color" : "black" }
{ "user" : "bill", "color" : "blue" }
{ "user" : "ted", "color" : "red" }
{ "user" : "ted", "color" : "black" }
{ "user" : "fred", "color" : "blue" }
{ "user" : "bill", "color" : "red" }
{ "user" : "bill", "color" : "orange" }
{ "user" : "fred", "color" : "orange" }
{ "user" : "ted", "color" : "orange" }
{ "user" : "ally", "color" : "orange" }
{ "user" : "alice", "color" : "orange" }
{ "user" : "alice", "color" : "red" }
{ "user" : "bill", "color" : "purple" }
So suppose you want to bubble the items for the users "bill" and "ted" to the top of your results, then everything else sorted by the user and the color. What you can do is run the documents through a $project stage in aggregate, as follows:
db.bubble.aggregate([
// Project selects the fields to show, and we add a weight value
{$project: {
_id: 0,
"user": 1,
"color": 1,
"weight": {$cond:[
{$or: [
{$eq: ["$user","bill"]},
{$eq: ["$user","ted"]}
]},
1,
0
]}
}},
// Then sort the results with the `weight` first, then `user` and `color`
{$sort: { weight: -1, user: 1, color: 1 }}
])
So what that does is conditionally assign a value to weight based on whether the user was matched to one of the required values. Documents that do not match are simply given a 0 value.
When we move this modified document on to the $sort phase, the new weight key can be used to order the results so the "weighted" documents are on top, and anything else will then follow.
There a quite a few things you can do to $project a weight in this way. See the operator reference for more information:
http://docs.mongodb.org/manual/reference/operator/aggregation/

Rails Model Syntax Confusion

I came across this code in Rails app using mongodb:
"""
Folder format:
{
name: <folder name>,
stocks: [
{
name: <stock name>,
id: <stock id>,
qty: <stock quantity>
}
]
}
"""
def format_with_folders(stocks)
fmap = stock_folder_map
res = stocks.group_by {|s| fmap[s["id"]] }.collect {|fname, ss|
{
"name" => fname,
"stocks" => ss
}
}
new(folders: res)
end
def stock_folder_map
res = {}
folders.each { |ff|
ff.stocks.each { |s|
res[s["id"]] = ff["name"]
}
}
return res
end
end
The doubts are:
1) What does the code inside triple quote signify? Is is a commented code?
2)What would be the right format to use this code inside a ruby script?
First of all, the triple quoted string is often used as a comment, and that is the case here.
To get this to work outside of the class, you would need create a folders method that returns an array of folders in the correct structure. You could do something like this:
Folder = Struct.new(:name, :stocks)
def folders
[
Folder.new(
"Folder 1",
[
{ "name" => "stock name", "id" => "stock id", "qty" => 3 },
{ "name" => "stock name", "id" => "stock id", "qty" => 5 }
]
),
Folder.new(
"Folder 2",
[
{ "name" => "stock name", "id" => "stock id", "qty" => 2 },
{ "name" => "stock name", "id" => "stock id", "qty" => 1 }
]
)
]
end
def format_with_folders(stocks)
# ...
end
def stock_folder_map
# ...
end
The folders method returns an array of Folder objects, which both have a name and stocks attribute. Stocks are an array of hashes.
In Ruby, if you have multiple string literals next to each other, they get concatenated at parse time:
'foo' "bar"
# => 'foobar'
This is a feature inspired by C.
So, what you have there is three string literals next to each other. The first string literal is the empty string:
""
Then comes another string literal:
"
Folder format:
{
name: <folder name>,
stocks: [
{
name: <stock name>,
id: <stock id>,
qty: <stock quantity>
}
]
}
"
And lastly, there is a third string literal which is again empty:
""
At parse time, this will be concatenated into a single string literal:
"
Folder format:
{
name: <folder name>,
stocks: [
{
name: <stock name>,
id: <stock id>,
qty: <stock quantity>
}
]
}
"
And since this string object isn't referenced by anything, isn't assigned to any variable, isn't returned from any method or block, it will just get immediately garbage collected.
In other words: the entire thing is a no-op, it's dead code. A sufficiently smart Ruby compiler (such as JRuby or Rubinius) will probably completely eliminate it, compile it into nothing.

Rails to_json - exact order of existing columns

I have a task to form JSON data for jqGrid. It requires a special format:
{
total: 50,
page:"1",
records: "1500",
rows: [
{ 20, "{2ae39c44-ca9d-4565-9e05-bbd875c1579c}", "Description 1"},
{ 23, "{e1aaf69d-1040-4afa-8995-fd15c3a591b3}", "Description 2"},
{ 25, "{e3df29c7-ef34-46ba-bf66-7838aca7c137}", "Description 3"},
{ 29, "{768ec164-28e5-4614-a259-63257b79e8e0}", "Description 4"}
]
}
So the basic rules for "rows" are: do not generate root object name, list fields without their names, list fields in exact order to bind to corresponding columns.
Can I force to_json method to modify output as I need?
Currently the to_json produces:
myobjs : [
myobj : { id: 20, uuid: "{2ae39c44-ca9d-4565-9e05-bbd875c1579c}", name: "Description 1"},
myobj : { id: 20, uuid: "{e1aaf69d-1040-4afa-8995-fd15c3a591b3}", name: "Description 2"},
myobj : { id: 20, uuid: "{e3df29c7-ef34-46ba-bf66-7838aca7c137}", name: "Description 3"},
myobj : { id: 20, uuid: "{768ec164-28e5-4614-a259-63257b79e8e0}", name: "Description 4"}
]
You can't do it with a model-level to_json call, you'll need to build an intermediary data representation as #Paul said. Something like:
class MyObj
def to_json
[id, uuid, name]
end
end
And then in the controller:
class MyController < ApplicationController
def grid_data
objs = MyObj.all
json_data = {
:total => objs.count,
:page => 1,
:records => 1500,
:rows => objs.collect {|o| o.to_json}
}
... send json as usual ...
end
end
Note that I set your model up to generate an array, not a hash as you specified, as I think you copied that wrong - your JSON example above is not valid. { 20, 'foo', 'bar' } is not valid JSON as "{...}" represents a hash, which must be keyed, and is not ordered.

How to join query in mongodb?

I have user document collection like this:
User {
id:"001"
name:"John",
age:30,
friends:["userId1","userId2","userId3"....]
}
A user has many friends, I have the following query in SQL:
select * from user where in (select friends from user where id=?) order by age
I would like to have something similar in MongoDB.
To have everything with just one query using the $lookup feature of the aggregation framework, try this :
db.User.aggregate(
[
// First step is to extract the "friends" field to work with the values
{
$unwind: "$friends"
},
// Lookup all the linked friends from the User collection
{
$lookup:
{
from: "User",
localField: "friends",
foreignField: "_id",
as: "friendsData"
}
},
// Sort the results by age
{
$sort: { 'friendsData.age': 1 }
},
// Get the results into a single array
{
$unwind: "$friendsData"
},
// Group the friends by user id
{
$group:
{
_id: "$_id",
friends: { $push: "$friends" },
friendsData: { $push: "$friendsData" }
}
}
]
)
Let's say the content of your User collection is the following:
{
"_id" : ObjectId("573b09e6322304d5e7c6256e"),
"name" : "John",
"age" : 30,
"friends" : [
"userId1",
"userId2",
"userId3"
]
}
{ "_id" : "userId1", "name" : "Derek", "age" : 34 }
{ "_id" : "userId2", "name" : "Homer", "age" : 44 }
{ "_id" : "userId3", "name" : "Bobby", "age" : 12 }
The result of the query will be:
{
"_id" : ObjectId("573b09e6322304d5e7c6256e"),
"friends" : [
"userId3",
"userId1",
"userId2"
],
"friendsData" : [
{
"_id" : "userId3",
"name" : "Bobby",
"age" : 12
},
{
"_id" : "userId1",
"name" : "Derek",
"age" : 34
},
{
"_id" : "userId2",
"name" : "Homer",
"age" : 44
}
]
}
Edit: this answer only applies to versions of MongoDb prior to v3.2.
You can't do what you want in just one query. You would have to first retrieve the list of friend user ids, then pass those ids to the second query to retrieve the documents and sort them by age.
var user = db.user.findOne({"id" : "001"}, {"friends": 1})
db.user.find( {"id" : {$in : user.friends }}).sort("age" : 1);
https://docs.mongodb.org/manual/reference/operator/aggregation/lookup/
This is the doc for join query in mongodb , this is new feature from version 3.2.
So this will be helpful.
You can use in Moongoose JS .populate() and { populate : { path : 'field' } }.
Example:
Models:
mongoose.model('users', new Schema({
name:String,
status: true,
friends: [{type: Schema.Types.ObjectId, ref:'users'}],
posts: [{type: Schema.Types.ObjectId, ref:'posts'}],
}));
mongoose.model('posts', new Schema({
description: String,
comments: [{type: Schema.Types.ObjectId, ref:'comments'}],
}));
mongoose.model('comments', new Schema({
comment:String,
status: true
}));
If you want to see your friends' posts, you can use this.
Users.find(). //Collection 1
populate({path:'friends', //Collection 2
populate:{path:'posts' //Collection 3
}})
.exec();
If you want to see your friends' posts and also bring all the comments, you can use this and too, you can indentify the collection if this not find and the query is wrong.
Users.find(). //Collection 1
populate({path:'friends', //Collection 2
populate:{path:'posts', //Collection 3
populate:{path:'commets, model:Collection'//Collection 4 and more
}}})
.exec();
And to finish, if you want get only some fields of some Collection, you can use the propiertie select Example:
Users.find().
populate({path:'friends', select:'name status friends'
populate:{path:'comments'
}})
.exec();
MongoDB doesn't have joins, but in your case you can do:
db.coll.find({friends: userId}).sort({age: -1})
one kind of join a query in mongoDB, is ask at one collection for id that match , put ids in a list (idlist) , and do find using on other (or same) collection with $in : idlist
u = db.friends.find({"friends": ? }).toArray()
idlist= []
u.forEach(function(myDoc) { idlist.push(myDoc.id ); } )
db.friends.find({"id": {$in : idlist} } )
Only populate array friends.
User.findOne({ _id: "userId"})
.populate('friends')
.exec((err, user) => {
//do something
});
Result is same like this:
{
"_id" : "userId",
"name" : "John",
"age" : 30,
"friends" : [
{ "_id" : "userId1", "name" : "Derek", "age" : 34 }
{ "_id" : "userId2", "name" : "Homer", "age" : 44 }
{ "_id" : "userId3", "name" : "Bobby", "age" : 12 }
]
}
Same this: Mongoose - using Populate on an array of ObjectId
You can use playOrm to do what you want in one Query(with S-SQL Scalable SQL).
var p = db.sample1.find().limit(2) ,
h = [];
for (var i = 0; i < p.length(); i++)
{
h.push(p[i]['name']);
}
db.sample2.find( { 'doc_name': { $in : h } } );
it works for me.
You can do it in one go using mongo-join-query. Here is how it would look like:
const joinQuery = require("mongo-join-query");
joinQuery(
mongoose.models.User,
{
find: {},
populate: ["friends"],
sort: { age: 1 },
},
(err, res) => (err ? console.log("Error:", err) : console.log("Success:", res.results))
);
The result will have your users ordered by age and all of the friends objects embedded.
How does it work?
Behind the scenes mongo-join-query will use your Mongoose schema to determine which models to join and will create an aggregation pipeline that will perform the join and the query.

Resources