I am building a Rails 5 app and in this app I got four models.
User, Report, Trip and Expense.
User has_many Reports, trips and expenses
Reports has_many trips and expenses.
I want to get a JSON response with all the trips and expenses that a user has done this month, grouped by Report.
I can do this but I need only the Id and Title of the report (when grouping) and now I get only the object name.
I use this method (located in the User model):
def grouped_reports
trips = self.trips
expenses = self.expenses
items = trips + expenses
items.group_by(&:report)
end
The output in JSON is this:
{
"#<Report:0x007fa225163ba8>": [{
"id": 12,
"account_id": 20,
"user_id": 92,
"vehicle_id": null,
"ttype": null,
"description": "saf",
"start_latitude": 57.4874919,
"start_longitude": 12.0761927999999,
"end_latitude": 59.3293235,
"end_longitude": 18.0685808000001,
"start_address": "Chicago",
"end_address": "New york",
"distance": 490,
"time": null,
"status": "pending",
"registered_at": "2018-08-24T02:00:00.000+02:00",
"approvals_count": 0,
"rejections_count": 0,
"created_at": "2018-08-24T22:39:22.637+02:00",
"updated_at": "2018-08-24T22:39:22.637+02:00",
"report_id": 79,
"return_trip": null,
"triptype_id": 10
},
{
"id": 13,
"account_id": 20,
"user_id": 92,
"cost": 100,
"etype": null,
"description": "sdsd",
"start_address": null,
"end_address": null,
"distance": null,
"reimbursable": false,
"status": "pending",
"registered_at": "2018-08-08T00:00:00.000+02:00",
"created_at": "2018-08-24T22:39:40.343+02:00",
"updated_at": "2018-08-24T22:39:40.343+02:00",
"approvals_count": 0,
"rejections_count": 0,
"report_id": 79,
"expensetype_id": 15
}
]
}
There is two things I need to improve.
Display id and title of report not .
Get only this months reports.
Update, this works but is it the right way in terms of performance?
And not only performance, is it actually using the title of the report as a grouping factor? Which is not good since may reports can share the same title. I want to group by report_id but display report title.
def grouped_reports
trips = self.trips
expenses = self.expenses
items = trips + expenses
items.group_by{ |t| [t.report.id, t.report.title] }
end
Code would differ depending on what JSON format you want to output.
I usually use json generator such jbuilder, but this time I suggest a array and hash structure.
class User < ApplicationRecord
has_many :reports
has_many :expenses
has_many :trips
def grouped_reports
start_date = Time.current.beginning_of_month
monthly_trips = trips.where('created_at >= ?', start_date)
monthly_expenses = expenses.where('created_at >= ?', start_date)
report_ids = (monthly_trips.map(&:report_id) + monthly_expenses.map(&:report_id)).uniq
reports = Report.where(id: report_ids)
reports.map do |report|
{
id: report.id,
title: report.title,
trips: trips.select {|t| t.report_id == report.id},
expenses: expenses.select {|e| e.report_id == report.id}
}
end
end
end
You are grouping with trips and expenses joined to one array, but it is not preferable to put different types in same arrays for JSON. It would be safe to have a hash format and separate key for trip and expense.
To extract records for this month, use where to filter.
It is possible to fetch trips by using includes for reports and expenses, but from a performance point of view it is better to get the related trips at once.
If you want to further improve performance, narrow down only columns used when outputting JSON by using select method. This would be a huge improvement if a lot of records are outputted.
Related
Sample post request to controller:
{
"card": 1,
"cardPot": 6,
"purchases": [
{
"description": "groceries",
"amount": "40.60",
"transaction_date": "17/02/2022",
"statement_reference": "10076608",
"statement_description": "SAINSBURYS ",
"reimbursements": [
{
"amount": "19.00",
"pot": 3
},
{
"amount": "21.60",
"pot": 7
},
],
"spread": false
},
....
]
}
Purchase has many reimbursements, Reimbursement belongs to purchase...
In my controller I'm doing:
def import
card_id = params[:card]
card = CreditCard.find_by(id: card_id)
pay_to_pot = params[:cardPot]
purchases = params[:purchases]
purchases.each do |p|
purchase_params = p.permit(:description, :amount, :transaction_date, :statement_reference, :statement_description).merge({user_id: 1, credit_card_id: card_id})
new_purchase = Purchase.create(purchase_params)
p[:reimbursements].each do |r|
start_date = p[:start_date] || Date.today
instalments = p[:spread] ? card.free_months_remaining : 1
Reimbursement.create({purchase_id: new_purchase.id, instalments: instalments, user_id: 1, pay_to_pot_id: pay_to_pot, pay_from_pot_id: r[:pot], total_amount: p[:amount], start_date: start_date })
end
end
end
This gets the desired effect but it feels like I could be doing this more efficiently, and/or that it's not the standard convention. I could create all of the purchase entities without needing to do an each, but then I would need some way to create the reimbursements and link them to the purchase ids
Note: obviously user_id: 1 isn't going to be in production! Just a convenience while getting started.
You can make use Active Record Nested Attributes, which should create associated relation.
Also for the custom params which are not present in request, you can add a before save callback on the respective model and set the value.
I have a performance issue in my application. I would like to gather some ideas on what I can do to improve it. The application is very easy: I need to add values inside a nested table to get the total an user wants to pay out of all the pending payments. The user chooses a number of payments and I calculate how much it is they will pay.
This is what I have:
jsonstr = "{ "name": "John",
"surname": "Doe",
"pending_payments": [
{
"month": "january",
"amount": 50,
},
{
"month": "february",
"amount": 40,
},
{
"month": "march",
"amount": 45,
},
]
}"
local lunajson = require 'lunajson'
local t = lunajson.decode(jsonstr)
local limit -- I get this from the user
local total = 0;
for i=1, limit, 1 do
total = total + t.pending_payments[i].amount;
end;
It works. At the end I get what I need. However, I notice that it takes ages to do the calculation. Each JSON has only twelve pending payments (one per month). It is taking between two to three seconds to come up with a result!. I tried in different machines and LUA 5.1, 5.2., 5.3. and the result is the same.
Can anyone please suggest how I can implement this better?
Thank you!
For this simple string, try the test code below, which extracts the amounts directly from the string, without a json parser:
jsonstr = [[{ "name": "John",
"surname": "Doe",
"pending_payments": [
{
"month": "january",
"amount": 50,
},
{
"month": "february",
"amount": 40,
},
{
"month": "march",
"amount": 45,
},
]
}]]
for limit=0,4 do
local total=0
local n=0
for a in jsonstr:gmatch('"amount":%s*(%d+),') do
n=n+1
if n>limit then break end
total=total+tonumber(a)
end
print(limit,total)
end
I found the delay had nothing to do with the calculation in LUA. It was related with a configurable delay in the retrieval of the limit variable.
I have nothing to share here related to the question asked since the problem was actually in an external element.
Thank #lfh for your replies.
I'm writing a controller index method that returns a sorted array of ActiveRecord Contact objects. I need to be able to sort the objects by attributes or by the output of an instance method. For example, I need to be able to sort by contact.email as well as contact.photos_uploaded, which is an instance method that returns the number of photos a contact has.
I can't use ActiveRecord's native order or reorder method because that only works with attributes that are columns in the database. I know from reading that normally array#sort_by is much faster than array#sort for complex objects.
My question is, how can I improve the performance of this block of code in my controller method? The code currently
contacts = company.contacts.order(last_name: :asc)
if params[:order].present? && params[:order_by].present? && (Contact::READ_ONLY_METHOD.include?(params[:order_by].to_sym) || Contact::ATTRIBUTES.include?(params[:order_by].to_sym))
contacts = contacts.sort_by do |contact|
if params[:order_by] == 'engagement'
contact.engagement.to_i
else
contact.method(params[:order_by].to_sym).call
end
end
contacts.reverse! if params[:order] == 'desc'
end
The root problem here (I think) is that I'm calling sort_by on contacts, which is an ActiveRecord::Relation that could have several hundred contacts in it. Ultimately I paginate the results before returning them to the client, however they need to be sorted before they can be paginated. When I run the block of code above with 200 contacts, it takes an average of 900ms to execute, which could be a problem in a production environment if a user has thousands of contacts.
Here's my Contact model showing some relevant methods. The reason I have a special if clause for engagement is because that method returns a string that needs to be turned into an integer for sorting. I'll probably refactor that before I commit any of this to return an integer. Generally all the methods I might sort on return an integer representing the number of associated objects (e.g. number of photos, stories, etc that a contact has). There are many others, so for brevity I'm just showing a few.
class Contact < ActiveRecord::Base
has_many :invites
has_many :responses, through: :invites
has_many :photos
has_many :requests
belongs_to :company
ATTRIBUTES = self.attribute_names.map(&:to_sym)
READ_ONLY_METHOD = [:engagement, :stories_requested, :stories_submitted, :stories_published]
def engagement
invites = self.invites.present? ? self.invites.count : 1
responses = self.responses.present? ? self.responses.count : 0
engagement = ((responses.to_f / invites).round(2) * 100).to_i.to_s + '%'
end
def stories_requested
self.invites.count
end
def stories_submitted
self.responses.count
end
def stories_published
self.responses.where(published: true).count
end
end
When I run a query to get a bunch of contacts and then serialize it to get the values for all these methods, it only takes ~80ms for 200 contacts. The vast majority of the slowdown seems to be happening in the sort_by block.
The output of the controller method should look like this after I iterate over contacts to build a custom data structure, using this line of code:
#contacts = Hash[contacts.map { |contact| [contact.id, ContactSerializer.new(contact)] }]
I've already benchmarked that last line of code so I know that it's not a major source of slowdown. More on that here.
{
contacts: {
79: {
id: 79,
first_name: "Foo",
last_name: "Bar",
email: "t#t.co",
engagement: "0%",
company_id: 94,
created_at: " 9:41AM Jan 30, 2016",
updated_at: "10:57AM Feb 23, 2016",
published_response_count: 0,
groups: {
test: true,
test23: false,
Test222: false,
Last: false
},
stories_requested: 1,
stories_submitted: 0,
stories_published: 0,
amplify_requested: 1,
amplify_completed: 1,
photos_uploaded: 0,
invites: [
{
id: 112,
email: "t#t.co",
status: "Requested",
created_at: "Jan 30, 2016, 8:48 PM",
date_submitted: null,
response: null
}
],
responses: [ ],
promotions: [
{
id: 26,
company_id: 94,
key: "e5cb3bc80b58c29df8a61231d0",
updated_at: "Feb 11, 2016, 2:45 PM",
read: null,
social_media_posts: [ ]
}
]
}
}
}
if params[:order_by] == 'stories_submitted'
contact_ids = company.contact_ids
# count all invites that have the relevant contact ids
invites=Invite.where(contact_id:contact_ids).group('contact_id').count
invites_contact_ids = invites.map(&:first)
# Add contacts with 0 invites
contact_ids.each{|c| invites.push([c, 0]) unless invites_contact_ids.include?(c)}
# Sort all invites by id (add .reverse to the end of this for sort DESC)
contact_id_counts=invites.sort_by{|r| r.last}.map(&:first)
# The [0, 10] limits you to the lowest 10 results
contacts=Contact.where(id: contact_id_counts[0, 10])
contacts.sort_by!{|c| contact_id_counts.index(c.id)}
end
I'm running into a strange issue with nested rails models related to the includes method. I'm attempting to simply move an item from one object to its parent like so:
Current:
[
{
"created_on": "2014-09-11T15:52:34-04:00",
"id": 8,
"mail_notification": false,
"project_id": 2,
"user_id": 15,
"member_roles": [
{
"id": 10,
"inherited_from": null,
"member_id": 8,
"role_id": 3
}
]
}
]
Needed:
[
{
"created_on": "2014-09-11T15:52:34-04:00",
"id": 8,
"mail_notification": false,
"project_id": 2,
"user_id": 15,
"role_id": 3
}
]
For some reason, when I loop through the current object, It strips out the :member_roles. Case in point:
members = Member.includes(:member_roles).find_all_by_project_id(#project)
# Contains :member_roles
puts members.to_json(include: [:member_roles])
#=> [{"created_on":"2014-09-11T15:52:34-04:00","id":8,"mail_notification":false,"project_id":2,"user_id":15,"member_roles":[{"id":10,"inherited_from":null,"member_id":8,"role_id":3}]}]
# Does not contain :member_roles
puts members.first.attributes
#=> {"id"=>8, "user_id"=>15, "project_id"=>2, "created_on"=>Thu, 11 Sep 2014 15:52:34 EDT -04:00, "mail_notification"=>false}
Why does the :member_roles object disappear?
you cannot do what you expect.
Member.includes(:member_roles) is eager loading your relation (ie it fetchs all the member_roles instances required by the member collection when you actually use on of this object the first time)
to_json(include: [:member_roles]) is including the json reprensation of the related model in the parent member model json.
What you describe is called method delegation (Module.delegate), but since you have a one to many relation between your 2 models you cannot do it
After creating a customer successfully, I can inspect the object with:
Rails.logger.debug("single card object has: #{customer.cards.data.card.inspect}")
which returns a json like this:
#<Stripe: : Customer: 0x2801284>JSON: {
"id": "cus_2WXxmvhBJgSmNY",
"object": "customer",
"cards": {
"object": "list",
"data": [
{
"id": "card_2WXxcCsdY0Jjav",
"object": "card",
"last4": "4242",
"type": "Visa",
"exp_month": 1,
"exp_year": 2014,
}
]
},
"default_card": "card_2WXxcCsdY0Jjav"
}
But I will do Customer.cards.data.last4 it gives a NOMethodError.
If I remove the last4 and just call Customer.cards.data, it gives
#<Stripe: : Card: 0x1ed7dc0>JSON: {
"id": "card_2Wmd80yQ76XZKH",
"object": "card",
"last4": "4242",
"type": "Visa",
"exp_month": 1,
"exp_year": 2015,
}
Now I seem to have the direct card object but if I do
card = Customer.cards.data
self.last4 = card.last4
I still get a noMethodError
Here is shortened version of my model:
class Payment < ActiveRecord::Base
def create_customer_in_stripe(params)
if self.user.stripe_card_token.blank?
user_email = self.user.email
customer = Stripe::Customer.create(email: user_email, card: params[:token])
card = customer.cards.data
self.card_last4 = card.last4
self.card_type = card.type
self.card_exp_month = card.exp_month
self.card_exp_year = card.exp_year
self.user.save
end
self.save!
end
end
customer.cards, as the name implies, returns multiple cards in an array.
You can't call card accessor methods because you don't have a Stripe::Card object; you have an array of Stripe::Card objects. You need to either call customer.cards.first (the most likely answer) or iterate over the array for a specific card you're looking for.
Once you have a Stripe::Card object, all the accessor methods will work correctly.
cself.card_last4 = card.last4 should be self.card_last4 = card["last4"] as the gem itself doesn't have a last4 method when searching on github. I know i have to use Hash syntax.
I have a feeling that all of your methods on card will need this syntax.
EDit:
So it sounds like your model's last4 column is an integer, do card["last4"].to_i or change the migration to a string column in the DB.
card = Customer.cards.data
self.last4 = card[0].last4
how do you get the default active card in
rails "default_card": "card_2WXxcCsdY0Jjav" in the list of customer.cards?
is there a better way rather than to loop thru customer.cards to get it or even easier way?
Any pointers?
Hope this help someone who is wondering as well :- )
default_active_card = customer.cards.data.detect{|obj| obj[:id] == customer.default_card}