I have an elasticsearch index 'events' - within that index there's a type 'event'.
event objects have a 'venue' which has various properties, including a 'name' - so the simplified structure is:
event {
venue {
name: "foo"
}
}
Now, i'm using elasticsearch-rails - everything works fine for listing the events, searching etc using the query dsl - but what if i want to list all the events at a venue with a particular name?
I'm assuming something like this should be possible:
Event.search "{ 'query': { 'match': { 'venue.name': '#{params[:v]}' }}}
but i get the following error:
Elasticsearch::Transport::Transport::Errors::BadRequest
followed by a substantial stack trace which contains a lot of this sort of thing:
Was expecting one of:\n \"]\" ...\n \"}\" ...\n ];
ParseExceptions suggesting malformed json - but i'm not sure why.
The simple search
Event.search '{"query" : { "match_all" : {} }}'
works fine, so i'm guessing it's just the structure of the query that's wrong.
I've tried switching single/double quotes around, tried following more closely the example on this page:
https://www.elastic.co/guide/en/elasticsearch/guide/current/denormalization.html
all to no avail, wondered if anyone else had encountered this situation and could suggest how to work this in ruby.
The Json you are trying to pass to the search function is not a valid Json. You can try passing a hash instead of Json to the search function. Try the following:
query_hash = {query: {match: {'venue.name' => params[:v] }}}
Event.search query_hash
Elasticsearch's json parser won't the use of single quotes to delimit strings - while some later parser's may, this isn't part of the standard.
You can of course escape them, although this makes things somewhat less legible, so using an alternative form of quoting may be preferable:
%< {"query": { "match": { "venue.name": "#{params[:v]}"}}} >
However it's much better to represent the query as a ruby hash and then convert that to json (for example the snippet above doesn't correctly escape special characters in the submitted value)
Related
I have multiple YAML (localization) files. I parse them and convert to hash in Ruby.
For example this is one of them:
hello: Hallo
messages:
alerts:
yay: Da!
no: Nein
deep:
nested:
another:
level:
hi: Hi!
test: Test!
Basically, this is look like a locale file in Rails App using YAML.
What I want to do is iterate this Hash recursively and get key and value. So that i can translate values one-by-one from API Endpoint like Google Translate. I want to keep nested hashes in same schema so that Rails can find by keys.
I know i can use nested loops but there is no guarantee that nested hashes is a known number of. How can I iterate this hash recursively so i can manipulate values (translate/replace)?
Expected Result: (after used translation service from API call)
hello: Hello
messages:
alerts:
yay: Yup!
no: No
deep:
nested:
another:
level:
hi: Hi!
test: Test!
What I've tried so far:
hash = YAML.load('de.yml') # parse source Deutsch locale
new_hash = {}
hash.each |key, value| do
new_hash[key] = translate_func(value) # here... translate value then assign very same key including parents.
# Do more loops....
end
# Now write this new_hash to yaml file...
But this only manipulate hello only. To get work with others I have to make a loop. But how many keys are nested is unknown.
How can I iterate over all values of locale hash and keep the schema intact?
And if possible but not mandatory, I would be very happy if we can keep the order of keys on final result. That would be awesome to find missing keys later when manually reviewed.
I am very new to ruby.
I am using Ruby 2.7.2
Conclusion / Resolve
All answers are correct and I love all of them. However, I would like to be able to control both keys and values. Not just transform by values. Therefore, I accepted an answer that fits to my needs. I was able to do my intention with selected answer.
You can use deep_transform_values! on your hash object to change the values recursively. (Or its non-destructive version deep_transform_values which returns a new hash instead of changing the original hash.)
hash.deep_transform_values! { |value| translate_func(value) }
Note: deep_transform_values! is a Rails method. See the source code here for inspiration if you're not using Rails.
If you want a simple non-rails solution then you can just create a recursive method:
def recurse(hash)
hash.transform_values do |v|
case v
when String
v.reverse # Just for the sake of the example
when Hash
recurse(v)
else
v
end
end
end
Output:
{"hello"=>"ollaH", "messages"=>{"alerts"=>{"yay"=>"!aD", false=>"nieN"}, "deep"=>{"nested"=>{"another"=>{"level"=>{"hi"=>"!iH"}}}}}, "test"=>"!tseT"}
However this might be a case of reinventing the wheel - you can use the i18n gem for translations and i18n-tasks to prefill your YAML files with translations from the Google Translate API.
So, you want to parse recursively until there are no more levels to parse into.
It’s super common in software and referred to as “recursion”. Have a search around to learn more about it - it’ll come up again and again in your journey. Welcome to ruby btw!
As for your actual current problem. Have a read of https://mrxpalmeiras.wordpress.com/2017/03/30/how-to-parse-a-nested-yaml-config-file-in-python-and-ruby/
But also, consider the i18n gem. See this answer https://stackoverflow.com/a/51216931/1777331 and the docs for the gem https://github.com/ruby-i18n/i18n This might fix your problem of handling internationalisation without you having to get into the details of handling yaml files.
the following ruby slice command runs as expected
#points.map{ |a| a.slice('point', 'point_name') }
returning and array of keys and values.
However, before dumping the array of hashes off to json, the goal is to transform the key 'point_name' to 'title'. Attempting a rails helper, as such
#points.map{ |a| a.slice('point', 'point_name' as: 'title') }
fails. What is the proper syntax?
There is no such syntax in ruby. Key rename can be achieved like this:
#points.map do |a|
a['title'] = a.delete('point_name')
a.slice('point', 'title')
end
You may need json serializer (as you mentioned Rails), consider using FastJsonApi.
I am trying to implement this solution in rails, using the collection aggregate method, to clone an entire collection within the same database.
In mongo shell, this works perfectly, and a cloned collection is created successfully:
db.source_collection.aggregate([ { $match: {} }, { $out: "target_collection" } ])
The rails-mongoid alternate, according to my research, should be this, which runs without errors:
SourceCollection.collection.aggregate({"$match" => {}, "$out" => "target_collection"})
#<Mongo::Collection::View::Aggregation:0x000000055bced0 #view=#<Mongo::Collection::View:0x44951600 namespace='DB_dev.source_collection' #filter={} #options={}>, #pipeline={"$match"=>{}, "$out"=>"target_collection"}, #options={}>
I also tried with an array
SourceCollection.collection.aggregate([{"$match" => {}}, {"$out" => "target_collection"}])
#<Mongo::Collection::View::Aggregation:0x000000054936d0 #view=#<Mongo::Collection::View:0x44342320 namespace='DB_dev.source_collection' #filter={} #options={}>, #pipeline=[{"$match"=>{}}, {"$out"=>"target_collection"}], #options={}>
UPDATE
This simplest syntax also works in Mongo console:
db.source_collection.aggregate( { $out: "target_collection" } )
But the respective syntax does not seem to work in Ruby:
SourceCollection.collection.aggregate({"$out" => "target_collection"})
Unfortunately, although there are no errors, the collection is not created.
Any clues as to the way I can make this happen?
Mongo gem version 2.5.3
Update2
Apparently $out is not considered in the pipeline, thus rendering the aggregation invalid.
This can be fixed with code... I am looking for a module/class/method override, as contacting mongodb issue tracking system for a change request might not be as quick..
UPDATE - FINAL
This issue has been solved, by help of Thomas R. Koll (thank you).
I add an update to post the response I got from the ticketing service of MongoDB, which pretty much describes Thomas's solution.
The reason you're not seeing the results without count is that the
aggregate method returns a lazy cursor; that is, the query does not
execute until the return value of aggregate is iterated over.
Calling count is one way to do this. This is the same behavior
you'll see if you call find or if you call aggregate without
specifying $out; the difference is that $out has an side-effect
beyond just returning the results, so it's more obvious when exactly
it occurs.
Found the solution, and I have to explain a few thigs:
This returns a Mongo::Collection::View::Aggregation object, it won't send a query to the database
User.collection.aggregate({"$out": "target_collection"})
Only when you call a method like count or to_a on the aggregation object it will be sent to the server, but if you pass a hash you'll get an error, so the pipeline has to be an array of hashes to work
User.collection.aggregate([{"$out": "target_collection"}]).count
I am using ahoy gem for analytics. In the ahoy_events table, I have properties column of json data type. I want to find specific data based on that column.
Suppose I have
{"tag":"a","class":"bigyapan-6","page":"/client/dashboard","text":"","href":"http://www.google.com"}
this as data and I want to find_by class.
In the rails c I ran Ahoy::Event.find_by(properties[:class]: "bigyapan-6") and it gave me an err
Ahoy::Event.find_by(properties["class"]: "bigyapan-6")
SyntaxError: unexpected ')', expecting end-of-input
This is a syntax error since properties[:class] is not a valid hash key in Ruby. To query Postgres JSON columns you need to provide the query as a string:
Ahoy::Event.find_by("properties ->> 'class' = 'bigyapan-6'")
ActiveRecord does not take a nested hash in this case like it would for an association. I doubt that ActiveRecord will ever support this since its very RBDMS specific and the type coercion thing (-> vs ->>) would make it really complex.
# this won't work.
Ahoy::Event.find_by(properties: { class: 'bigyapan-6' })
See:
http://edgeguides.rubyonrails.org/active_record_postgresql.html#json
http://robertbeene.com/rails-4-2-and-postgresql-9-4/
Try the following:
Ahoy::Event.find_by properties["class"].to_sym 'bigyapan-6'
I think it is getting confused with string followed by :
I'm accessing the Amazon AWS API using the ruby-aaws gem, but without going to much into details of the API or the gem, I think my problem is more of a general nature.
When I query the API I will end up with "object array", let's call it item, containing the API response.
I can easily access the data in the array, e.g. puts item.item_attributes.artist.to_s
Now the API returns attributes whose identifier are reserved words in Rails, e.g. format or binding.
So doing this:
puts item.item_attributes.format.to_s will return method not found
while
puts item.item_attributes.binding.to_s will return some object hash like #<Binding:0xb70478e4>.
I can see that there are values under that name when doing
puts item.item_attributes.to_yaml
Snippet from the resulting yaml show artist and binding:
--- !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::ItemAttributes
__val__:
artist: !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::Artist
__val__: Summerbirds in the Cellar
binding: !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::Binding
__val__: Vinyl
This was probably a very detailed explanation with a very simple solution, but I can't seem to find the solution.
edit
Finally found it. I guess it is because it is an array of objects, duh...
puts item.item_attributes[0].binding.to_s
You may be able to access the individual attributes by using [] instead of the method name (which is probably provided using method_missing anyway).
So, item.item_attributes[:artist].to_s may return what you want. If it doesn't it would be worth trying 'artist' as the key instead.
Finally found it. I guess it is because it is an array of objects, duh...
puts item.item_attributes[0].binding.to_s