I am trying to implement this solution in rails, using the collection aggregate method, to clone an entire collection within the same database.
In mongo shell, this works perfectly, and a cloned collection is created successfully:
db.source_collection.aggregate([ { $match: {} }, { $out: "target_collection" } ])
The rails-mongoid alternate, according to my research, should be this, which runs without errors:
SourceCollection.collection.aggregate({"$match" => {}, "$out" => "target_collection"})
#<Mongo::Collection::View::Aggregation:0x000000055bced0 #view=#<Mongo::Collection::View:0x44951600 namespace='DB_dev.source_collection' #filter={} #options={}>, #pipeline={"$match"=>{}, "$out"=>"target_collection"}, #options={}>
I also tried with an array
SourceCollection.collection.aggregate([{"$match" => {}}, {"$out" => "target_collection"}])
#<Mongo::Collection::View::Aggregation:0x000000054936d0 #view=#<Mongo::Collection::View:0x44342320 namespace='DB_dev.source_collection' #filter={} #options={}>, #pipeline=[{"$match"=>{}}, {"$out"=>"target_collection"}], #options={}>
UPDATE
This simplest syntax also works in Mongo console:
db.source_collection.aggregate( { $out: "target_collection" } )
But the respective syntax does not seem to work in Ruby:
SourceCollection.collection.aggregate({"$out" => "target_collection"})
Unfortunately, although there are no errors, the collection is not created.
Any clues as to the way I can make this happen?
Mongo gem version 2.5.3
Update2
Apparently $out is not considered in the pipeline, thus rendering the aggregation invalid.
This can be fixed with code... I am looking for a module/class/method override, as contacting mongodb issue tracking system for a change request might not be as quick..
UPDATE - FINAL
This issue has been solved, by help of Thomas R. Koll (thank you).
I add an update to post the response I got from the ticketing service of MongoDB, which pretty much describes Thomas's solution.
The reason you're not seeing the results without count is that the
aggregate method returns a lazy cursor; that is, the query does not
execute until the return value of aggregate is iterated over.
Calling count is one way to do this. This is the same behavior
you'll see if you call find or if you call aggregate without
specifying $out; the difference is that $out has an side-effect
beyond just returning the results, so it's more obvious when exactly
it occurs.
Found the solution, and I have to explain a few thigs:
This returns a Mongo::Collection::View::Aggregation object, it won't send a query to the database
User.collection.aggregate({"$out": "target_collection"})
Only when you call a method like count or to_a on the aggregation object it will be sent to the server, but if you pass a hash you'll get an error, so the pipeline has to be an array of hashes to work
User.collection.aggregate([{"$out": "target_collection"}]).count
Related
I'm having trouble with a little Ruby on Rails I'm building and need some help.
I have a Table with 20+ Columns and a corresponding XML File which can be parsed as some sort of hash with a gem. Every key would be mapped to a column and every value would be a data record in said column.
The way I access a specific value in the already parsed XML file is:
filename["crs","inputkeyhere"]
which returns the value, for example "52" or whatever.
What I am trying to do is upload the file, parse it with the gem and give each column the corresponding value.
My table (or model) is called "Attributeset" and I already know how I can access every column:
#attributeset = Attributeset.new
#attributeset.attributes.keys
So my thought process was:
Iterate over all the keys
Pass every key into a block called |a|
Use the rails possibilty to set attributes by calling the corresponding #attributeset.
Set colum attribute to the corresponding xml key
So my code would go something like this:
#attributeset.attributes.keys.each do |a|
#attributeset.a=filename["crs",a]
end
But my problem is, that ruby thinks ".a" is a method and apparently does not evaluate "a" to the block parameter.
I've read through lambdas and procs and whatnot but didn't really understand how they could work for my specific situation.
Coming from bash scripting maybe my thinking might be wrong but I thought that the .a might get evaluated.
I know I can run the block with yield, but this only works in methods as far as I know..
Any help is appreciated.
Thanks and stay healthy,
Alex
Thanks for the input!
I wanted to make it as clean as possible, and not using any temporary hashes to pass arguments.
I've found the method
write_attribute
which can be used like this:
#attributeset.write_attribute(a, xmp["crs",a])
worked perfectly for me.
You can use []= method to set values dynamically:
#attributeset.attribute_names.each do |attribute|
#attributeset[attribute] = filename["crs", attribute]
end
I have an EmberJS filter like this:
/app/routes/trails/new.js
model: function (filterCurrentEmployees) {
return Ember.RSVP.hash({
trail: this.store.createRecord('trail'),
employees: this.store.query('employee', { status: '1,2'}).then(
function(data) {return data})
})
},
I was hoping that status: '1,2' would end up as a normal Rails param so I could do params[:status] and then filter the returned employees (you know so the db would get a query like 'where status IN ['1','2']')
But when this filter query is sent to Rails API I get this error:
ActiveModelSerializers::Adapter::JsonApi::Deserialization::InvalidDocument (Invalid payload ({:data=>"Expected hash"}): {"status"=>"1,2", "controller"=>"employees", "action"=>"index"}):
which occurs here in the controller:
api/app/controllers/employees.rb
def employee_params
ActiveModelSerializers::Deserialization.jsonapi_parse!(params)
end
but that is needed for AMS to work with Ember.
So I presume something is wrong with the way I am sending/creating the query in EmberJS?
It seems to be trying to make a GET collection request. Not sure really and this explains sort-of what is going on at the Rails end, but I don't know how to get Ember to create the filter properly so that AMS is happy to accept it.
EDIT - JSON-API Spec Reference
I was under the impression that AMS and Ember 2.7 with JSON-API 'just work out of the box'. I was actually expecting the Ember filter to comply with the spec and send
/employees?filter=status[1,2]
but it seems to not be doing that. AMS says the same thing here.
UPDATE
So with some pointers from the comments (thank you) I learned that the structure of the query hash might be incorrect. I also tried changing 'query' to filter but that then raises an actual Ember error:
The filter API has been moved to a plugin. To enable store.filter using an environment flag, or to use an alternative, you can visit the ember-data-filter addon page.
Now I don't know what the difference is between an Ember query and an Ember filter, perhaps a filter only runs on the client? Who knows, the docs are so sparse and so many questions on this are 1 or 2 years old.
So after much digging around for Ember Gold Nuggets, I found out that Controllers are STILL needed for...you guessed it...Query Parameters.
I've never used a Controller, and thought I never needed one.
So it seems I was basing my attempt at using query parameters on outdated information and code examples.
For those who run into this, the gold nuggets are here and here.
Also, it seems Javascript Object parameters as per JSON-API spec are not yet supported in Ember, according to this, although it's nearly a year old so do not know if that is still true.
UPDATE
So, after further gold mining, it seems I was confused and that Controllers are needed for Query Parameters in Ember on the client side, to assist the frontend application when it transitions from one route to another (and that is when you need a Controller to set them up).
But I want to just send a query parameter from within the Route code directly to the API, and examples of doing that are very hard to find.
So in the Rails Log I would expect to see BEFORE deserialization by ASM:
Rails.logger.info "Params are: #{params.to_json}"
something like this:
Params are:
{"data":{"filter":{"status["1,2"]}},
"controller":"employees","action":"index"}
ASM expects 'data' to be the root element of the hash, and then inside that I can place my filter. For example, from the Ember Route:
model: function () {
let myFilter = {};
myFilter.data = { filter: {status: ['1,2']}};
return Ember.RSVP.hash({
trail: this.store.createRecord('trail'),
employees: this.store.query('employee', myFilter).then(function(data) {return data})
})
},
I've been messing around with the Youtube_It gem for the past couple of days as I'm trying to integrate it into a Rails 4 app I'm currently working on.
When I run a search query with it, it outputs an array with a ton of results. As long as I'm in the Rails console, there's no problem with manipulating it. A brief example :
require 'youtube_it'
client = YouTubeIt::Client.new(:dev_key => "dev_key")
data = client.videos_by(:query => "penguin")
data.videos.each { |v| puts v.unique_id }
This outputs a nice, tidy list of all the unique id's that were returned from the query.
When I try to do something similar within a view (I know it shouldn't really be in the view, it's simply there for testing at this point) it just outputs a huge jumbled list of what appears to be XML. This only seems to happen when I try to iterate using .each.
When I do something like :
<% data = client.videos_by(:query => "penguin") %>
<%= data.videos[1].unique_id %>
This returns exactly what I was expecting, one unique_id from the index that I chose. Great! But how do I get ALL the unique id's?
That's where I'm stuck and would really appreciate some help! :)
OK, two reasons (working from the gist you gave me on IRC, located here):
1) You are not actually using the same code as in the console. Your console uses puts unique_id which will print the value of the unique ID, but you are just using unique_id which will get the ID... and then do nothing with it. What you want is probably something like data.videos.map(&:unique_id) (or data.videos.map { |v| v.unique_id } in long form) which will return you an array of the IDs.
2) You are using <%=, which means 'evaluate this ruby line and output the return value onto the page'. The return value of an each statement is the object you called each on - ie. data.videos, so that is what is getting printed out.
I found the following method to be a HEAVY memory user on Ruby 1.8.7 and return absolutely no results (when there should be lots). The method also works like a charm on Ruby 1.9.2, returning all the wanted results while consuming no memory at all (or so!). I guess that's because a local variable has the same name as the containing method, but anyone have a clear answer for that?
def contact_of
contact_of = Circle.joins(:ties).where('ties.contact_id' => self.guid).map { |circle| circle.owner } || []
return contact_of.uniq!
end
By the way, I'm running Rails 3.1.1.
Thanks!
UPDATE : There's a part of the question that is erroneous. The fact that no contacts are returned when there should be is my misunderstading of 'uniq!' instead of 'uniq'. The first one does return 'nil' when no duplicates are found.
Still trying to figure out the memory problem...
Yeah, contact_of.uniq! would make a recursive call to the same function. I'm surprised it works in Ruby 1.9, actually.
Also, your DB query is terrible, because it retrieves a lot of unnecessary records and then does further select logic on the Ruby side. You probably want to start the find from Owner, not Circle.
I'm trying to update one of my objects in my rails app and the changes just don't stick. There are no errors, and stepping through with the debugger just reveals that it thinks everything is updating.
Anyway, here is the code in question...
qm = QuestionMembership.find(:first, :conditions => ["question_id = ? AND form_id = ?", q_id, form_id])
qm.position = x
qm.save
For reference sake, QuestionMembership has question_id, form_id, and position fields. All are integers, and have no db constraints.
That is basically my join table between Forms and Questions.
Stepping through the code, qm gets a valid object, the position of the object does get changed to the value of x, and save returns 'true'.
However, after the method exits, the object in the db is unchanged.
What am I missing?
You may not be finding the object that you think you are. Some experimenting in irb might be enlightening.
Also, as a general rule when changing only one attribute, it's better to write
qm.update_attribute(:position, x)
instead of setting and saving. Rails will then update only that column instead of the entire row. And you also get the benefit of the data being scrubbed.
Is there an after_save?
Is the correct SQL being emitted?
In development log, you can actually see the sql that is generated.
For something like this:
qm = QuestionMembership.find(:first, :conditions => ["question_id = ? AND form_id = ?", q_id, form_id])
qm.position = x
qm.save
You should see something to the effect of:
SELECT * FROM question_memberships WHERE question_id=2 AND form_id=6 LIMIT 1
UPDATE question_memberships SET position = x WHERE id = 5
Can you output what sql you are actually seeing so we can compare?
Either update the attribute or call:
qm.reload
after the qm.save
What is the result of qm.save? True or false? And what about qm.errors, does that provide anything that makes sense to you? And what does the development.log say?
I have run into this problem rather frequently. (I was about to say consistently, but I cannot, as that would imply that I would know when it was about to happen.)
While I have no solution to the underlying issue, I have found that it seems to happen to me only when I am trying to update mysql text fields. My workaround has been to set the field to do something like:
qm.position = ""
qm.save
qm.position = x
qm.save
And to answer everyone else... when I run qm.save! I get no errors. I have not tried qm.save?
When I run through my code in the rails console everything works perfectly as evidenced by re-finding the object using the same query brings the expected results.
I have the same issue when using qm.update_attribute(... as well
My workaround has gotten me limping this far, but hopefully someone on this thread will be able to help.
Try changing qm.save to qm.save! and see if you get an exception message.
Edit: What happens when you watch the log on the call to .save!? Does it generate the expected SQL?
Use ./script/console and run this script.. step by step..
see if the position field for the object is update or not when you run line 2
then hit qm.save or qm.save!... to test
see what happens. Also as mentioned by Tim .. check the logs
Check your QuestionMembership class and verify that position does not have something like
attr_readonly :position
Best way to debug this is to do
tail -f log/development.log
And then open another console and do the code executing the save statement. Verify that the actual SQL Update statement is executed.
Check to make sure your database settings are correct. If you're working with multiple databases (or haven't changed the default sqlite3 database to MySQL) you may be working with the wrong database.
Run the commands in ./script/console to see if you see the same behavior.
Verify that a similar object (say a Form or Question) saves.
If the Form or Question saves, find the difference between the QuestionMembership and Form or Question object.
Turns out that it was emitting the wrong SQL. Basically it was looking for the QuestionMembeship object by the id column which doesn't exist.
I was under the impression that that column was unnecessary with has_many_through relationships, although it seems I was misguided.
To fix, I simply added the id column to the table as a primary key. Thanks for all the pointers.