Doing a query with aggregate functions directly on ActiveRecord with Postgres seems to be working ok.
ActiveRecord::Base.connection.execute("
SELECT created_at::date as date,
sum(item1_count) as sum_item1,
sum(item2_count) as sum_item2,
sum(item3) as sum_item3 from items
GROUP by
created_at::date ORDER BY date desc").to_a
And returns something like this which is ok.
[
{
"date" => "2014-01-23",
"sum_item1" => "3239",
"sum_item2" => "90",
"sum_item3" => "0.00000"
},
{
"date" => "2014-01-22",
"sum_item1" => "1981",
"sum_item2" => "19",
"sum_item3" => "0.00000"
}
]
The problem is when trying to do the same using scopes, for instance.
class Item < ActiveRecord::Base
scope :myscope, -> {
select("created_at::date as date, sum(item1_count) as sum_item1,
sum(item2_count) as sum_item2,
sum(item3) as sum_item3")
.group("created_at::date")
.order("date desc") }
end
The result here is different. When running user.items.myscope.to_a I get the following result missing the aggregate values and adding an id field that should not be there.
[
#<Item:0x00000103cc3d38> {
:id => nil,
:date => Thu, 23 Jan 2014
},
#<Item:0x00000103cc39a0> {
:id => nil,
:date => Wed, 22 Jan 2014
}
]
How it would be possible to pass the aggregate functions to the scope?
Related
I have integrated elasticsearch in a ROR application.
I am using elastic search to find products named 'xyz' and filtered on basis of star count = 1
This is my mapping:
{
:product => {
:dynamic => false,
:properties => {
:name => {
:analyzer => "english",
:type => "text"
},
:description => {
:analyzer => "english",
:type => "text"
},
:available => {
:type => :boolean
},
:last_updated => {
:type => :date
},
:categories_names => {
:type => "text"
},
:stars => {
:type => :integer
}
}
}
}
I am using the following query:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "xyz",
"fields": ["name", "description"],
"type": "most_fields"
}
},
{
"match": {
"available": true
}
}
],
"filter": {
"term": {
"stars": 1
}
}
}
}
}
This gives me no match but I have product records with stars having value 1.
Also, range query is not giving any results with field stars.
This is the record that should match the above query:
#<Product> {
"id" => 90,
"name" => "xyz",
"last_updated" => Tue, 22 May 2018 15:04:43 UTC +00:00,
"stars" => 1,
"description" => "training",
"available" => true,
"created_at" => Thu, 21 Jun 2018 11:56:15 UTC +00:00,
"updated_at" => Fri, 29 Jun 2018 10:30:06 UTC +00:00,
}
try adding doc_value: true while indexing the field
indexes :stars, type: 'integer', doc_values: true
Most fields are indexed by default, which makes them searchable. The inverted index allows queries to look up the search term in unique sorted list of terms, and from that immediately have access to the list of documents that contain the term.
Sorting, aggregations, and access to field values in scripts require a different data access pattern. Instead of looking up the term and finding documents, we need to be able to look up the document and find the terms that it has in a field.
Doc values are the on-disk data structure, built at document index time, which makes this data access pattern possible. They store the same values as the _source but in a column-oriented fashion that is way more efficient for sorting and aggregations. Doc values are supported on almost all field types, with the notable exception of analyzed string fields.
Though default doc_values is true but explicitly adding it solved my purpose
Mapping and query looks good, maybe the issue in within the data (or maybe the mapping in your code is not the same in your cluster?).
Also may I suggest you move the "available: true" match as a "term" filter in the filter part? Filter is an array too, like must.
I have one model call DataIndicator, it contains daily data,
And It has the following column.
:id => :integer,
:date => :datetime,
:dau => :integer,
:login_count => :integer
It had many data, but now I need to change some of it.
How do I massive update its value by date?
EX:
The original
{ "id" => 1, "date" => 2017-01-01 00:00:00 UTC, "dau" => 5 , "login_count" => 150 },
{ "id" => 2, "date" => 2017-01-02 00:00:00 UTC, "dau" => 5 , "login_count" => 140 },
{ "id" => 3, "date" => 2017-01-03 00:00:00 UTC, "dau" => 5 , "login_count" => 300 }
Now I have a hash value, which would be referred to modify the original data.
Like this
update_date = {
"2017-01-01" => {
"dau" => 5,
"login_count" => 5,
},
"2017-01-02" => {},
"2017-01-03" => {
"dau" => 5,
},
...
}
As you can see, the update_date will not contain all attributes, it may only have one or even zero new data.
What is the best way to update this value?
I can only think about the bad one.
Like this
update_date.each do |k, v|
data_by_date = DataIndicator.where(date: DateTime.parse(k)).first
next if data_by_date.nil?
data_by_date.update(v)
end
I think I misread your question.
Wouldn't it be easier if you just picked all DataIndicator records and then checked if the hash contained data for it?
DataIndicator.all.each do |di|
date = di.date.strftime("%Y-%m-%d")
to_update = update_date[date]
next if to_update.blank?
to_update.each do |field,value|
di.send("#{field}=".to_sym, value)
end
di.save!
end
This works for me.
For Rails 4+
DataIndicator.where(["date(date) = ?"], "2017-01-01").update_all(dau: 5, login_count: 5)
I have a huge array full of a bunch of hashes. What I need to do is single out one index hash from the array that meets a specific criteria. (doing this due to an rspec test, but having trouble singling out one of them)
My array is like this
[
{
"name" => "jon doe",
"team" => "team2",
"price" => 2000,
"eligibility_settings" => {}
},
{
"name" => "jonny doe",
"team" => "team1",
"value" => 2000,
"eligibility_settings" => {
"player_gender" => male,
"player_max_age" => 26,
"player_min_age" => 23,
"established_union_only" => true
}
},
{
"name" => "jonni doe",
"team" => "team3",
"price" => 2000,
"eligibility_settings" => {}
},
]
I need to single out the second one, based on its eligibility settings. I just took three of them from my array, have lots more, so simple active record methods like (hash.second) won't work in this instance.
I've tried things like
players.team.map(&:hash).find{ |x| x[ 'eligibility_settings?' ] == true}
However when I try this, I get a nil response. (which is odd)
I've also looked into using the ruby detect method, which hasn't gotten me anywhere either
Players.team.map(&:hash).['hash.seligibiltiy_settings'].detect { true }
Would anybody have any idea what to do with this one?
Notes
players.team.map(&:hash).find{ |x| x[ 'eligibility_settings?' ] == true}
Players.team.map(&:hash).['hash.seligibiltiy_settings'].detect { true }
Is is players or Players ?
Why is it plural?
If you can call map on team, it probably should be plural
Why do you convert to a hash?
eligibility_settings? isn't a key in your hash. eligibility_settings is
eligibility_settings can be a hash, but it cannot be true
If you want to check if it isn't empty, use !h['eligibility_settings'].empty?
Possible solution
You could use :
data = [
{
'name' => 'jon doe',
'team' => 'team2',
'price' => 2000,
'eligibility_settings' => {}
},
{
'name' => 'jonny doe',
'team' => 'team1',
'value' => 2000,
'eligibility_settings' => {
'player_gender' => 'male',
'player_max_age' => 26,
'player_min_age' => 23,
'established_union_only' => true
}
},
{
'name' => 'jonni doe',
'team' => 'team3',
'price' => 2000,
'eligibility_settings' => {}
}
]
p data.find { |h| !h['eligibility_settings'].empty? }
# {"name"=>"jonny doe", "team"=>"team1", "value"=>2000, "eligibility_settings"=>{"player_gender"=>"male", "player_max_age"=>26, "player_min_age"=>23, "established_union_only"=>true}}
If h['eligibility_settings'] can be nil, you can use :
data.find { |h| !h['eligibility_settings'].blank? }
or
data.find { |h| h['eligibility_settings'].present? }
I have a model for languages and i want to get all the languages as json but the json output looks as follows
[{"language":{"created_at":null,"id":1,"language":"English","updated_at":null}},{"language":{"created_at":null,"id":2,"language":"Swedish","updated_at":null}},{"language":{"created_at":null,"id":3,"language":"German","updated_at":null}},{"language":{"created_at":null,"id":4,"language":"French","updated_at":null}},{"language":{"created_at":null,"id":5,"language":"spanish","updated_at":null}},{"language":{"created_at":null,"id":6,"language":"dutch","updated_at":null}},{"language":{"created_at":"2012-12-03T05:01:18Z","id":7,"language":"Tamil","updated_at":"2012-12-03T05:01:18Z"}}]
but i want to make this as
{"language":[{"created_at":null,"id":1,"language":"English","updated_at":null},{"created_at":null,"id":2,"language":"Swedish","updated_at":null},{"created_at":null,"id":3,"language":"German","updated_at":null},{"created_at":null,"id":4,"language":"French","updated_at":null},{"created_at":null,"id":5,"language":"spanish","updated_at":null},{"created_at":null,"id":6,"language":"dutch","updated_at":null},{"created_at":null,"id":7,"language":"Tamil","updated_at":null} ] }
Update
def index
#languages = Language.all
respond_to do |format|
format.json { render json: #languages}
end
end
update 2
class Language < ActiveRecord::Base
ActiveRecord::Base.include_root_in_json = false
has_and_belongs_to_many :users
end
I believe this should work:
format.json { render json: { "language" => #languages.as_json(:root => false) }.to_json }
What this does it to convert the #languages array into an array of JSON-formatted hash models with no root keys (using as_json), then wraps the result in a hash with a root key "language", and convert that hash into a JSON-formatted string with to_json. (See the docs for details on including or not including a root node using as_json.)
For example, with a model Post:
posts = Post.all
#=> [#<Post id: 1, name: "foo", title: "jkl", content: "some content", created_at: "2012-11-22 01:05:46", updated_at: "2012-11-22 01:05:46">]
# convert to array of hashes with no root keys
posts.as_json(root: false)
#=> [{"content"=>"some content", "created_at"=>Thu, 22 Nov 2012 01:05:46 UTC +00:00, "id"=>1, "name"=>"foo", "title"=>"jkl", "updated_at"=>Thu, 22 Nov 2012 01:05:46 UTC +00:00}]
# add root back to collection:
{ "post" => posts.as_json(root: false) }
#=> {"post"=>[{"content"=>"some content", "created_at"=>Thu, 22 Nov 2012 01:05:46 UTC +00:00, "id"=>1, "name"=>"foo", "title"=>"jkl", "updated_at"=>Mon, 03 Dec 2012 09:41:42 UTC +00:00}]}
# convert to JSON-formatted string
{ "post" => posts.as_json(root: false) }.to_json
#=> "{\"post\":[{\"content\":\"some content\",\"created_at\":\"2012-11-22T01:05:46Z\",\"id\":1,\"name\":\"foo\",\"title\":\"jkl\",\"updated_at\":\"2012-12-03T09:43:37Z\"}]}"
override the as_json on the Model you want to customize
def as_json options={}
{
id: id,
login: login,
name: custom.value, #for custom name
...
}
end
==> or
def as_json(options={})
super(:only => [:id, :login, :name ....])
end
from : here
Other link: here
I suggest you to use rabl gem (https://github.com/nesquena/rabl) to format your data.
Override as_json method in your model, to include associations, hide columns and why not? calling custom methods as they were attributes
def as_json(options={})
super(:except => [:created_at,:updated_at],
:include => {
:members => {
:only => [:role, :account],
:include => {
:account => {
:only => [:name, :subdomain]
}
}
}
},
:methods => [:jwt_token]
)
end
This will output something like this:
{
"id": 2,
"name": "Test Teacher",
"email": "teacher#testing.io",
"jwt_token":"eyJhbGciOiJIUzI1NiJ9.eyJpZCI6MiwiZXhwIjoxNTY2NzQ0OTQzfQ.HDGu7JiJEQEEpGo7inuXtOZBVQOfTaFquy8dr-QH5jY",
"members": [{
"role": "instructor",
"account": {
"name": "Testing",
"subdomain": "test"
}
}],
}
The easiest way of adding custom json output when you render json is by using gem that provide many json templates-
https://github.com/fabrik42/acts_as_api
I have a donations table where I'm trying to calculate the total amount for each month. For months without without any donations, I'd like the result to return 0.
Here's my current query:
Donation.calculate(:sum, :amount, :conditions => {
:created_at => (Time.now.prev_year.all_year) },
:order => "EXTRACT(month FROM created_at)",
:group => ["EXTRACT(month FROM created_at)"])
which returns:
{7=>220392, 8=>334210, 9=>475188, 10=>323661, 11=>307689, 12=>439889}
Any ideas how to grab the empty months?
Normally you'd left join to a calendar table (or generate_series in PostgreSQL) to get the missing months but the easiest thing with Rails would be to merge your results into a Hash of zeroes; something like this:
class Donation
def self.by_month
h = Donation.calculate(:sum, :amount, :conditions => {
:created_at => (Time.now.prev_year.all_year) },
:order => "EXTRACT(month FROM created_at)",
:group => ["EXTRACT(month FROM created_at)"])
Hash[(1..12).map { |month| [ month, 0 ] }].merge(h)
end
end
then just call the class method, h = Donation.by_month, to get your results.
In addition to mu is too short answer, in Rails 3.2.12 did not work for me, ActiveRecord returns the keys as strings:
h = Donation.calculate(:sum, :amount, :conditions => {
:created_at => (Time.now.prev_year.all_year) },
:order => "EXTRACT(month FROM created_at)",
:group => ["EXTRACT(month FROM created_at)"])
Which returns:
{"7"=>220392, "8"=>334210, "9"=>475188, "10"=>323661, "11"=>307689, "12"=>439889}
So when I merge the hash with zeros:
{1=>0, 2=>0, 3=>0, 4=>0, 5=>0, 6=>0, 7=>0, 8=>0, 9=>0, 10=>0, 11=>0, 12=>0, "7"=>220392, "8"=>334210, "9"=>475188, "10"=>323661, "11"=>307689, "12"=>439889}
The little fix (to_s):
Hash[(1..12).map { |month| [ month.to_s, 0 ] }].merge(h)