In requests to my application i often get ids of the objects, which should be associated. However I have to perform a check to see if they are.
Example scenario:
Class A and B are associated:
A {
static hasMany = [bs: B]
}
In my request I will get aid and bid.
What I usually do is:
def a = A.get(aid)
def b = a.bs.find {it.id == bid}
What would be a better way to make this check? From performance point of view?
Thanks
If B has a belongsTo = [ a : A ] defined in it, then you can do this:
def a = A.get(aid)
def b = B.find("from B as b where b.id=:id and b.a = :a", [id:bid, a:a])
if (b) {
// b exists -- do something with it here
} else {
// uh oh! b isn't within a
}
This won't do an iteration over all the sets elements like your code. Essentially, it's the same as erturne's solution, but this actually loads the object.
My first inclination is to drop to HQL and use count() to see if it exists. There may be more elegant ways to achieve the same thing (perhaps using withCriteria) but my first crack at it looks like:
def result = A.executeQuery("select count(*) from A as a inner join a.bs as b where a.id=:aid and b.id=:bid", [aid: aid, bid: bid] )
def associated = result[0] > 0
I think that would be pretty efficient, although anytime you're looking at improving performance it's best to take measurements of different implementations so you can compare them for yourself. I'll leave that as an exercise for the reader. ;-)
EDIT: I think it's efficient because I've left the heavy lifting in the database rather than pulling data for instances of A and B across the network, creating instances in memory, or iterating over results.
Related
I'm modding a game. I'd like to optimize my code if possible for a frequently called function. The function will look into a dictionary table (consisting of estimated 10-100 entries). I'm considering 2 patterns a) direct reference and b) lookup with ipairs:
PATTERN A
tableA = { ["moduleName.propertyName"] = { some stuff } } -- the key is a string with dot inside, hence the quotation marks
result = tableA["moduleName.propertyName"]
PATTERN B
function lookup(type)
local result
for i, obj in ipairs(tableB) do
if obj.type == "moduleName.propertyName" then
result = obj
break
end
end
return result
end
***
tableB = {
[1] = {
type = "moduleName.propertyName",
... some stuff ...
}
}
result = lookup("moduleName.propertyName")
Which pattern should be faster on average? I'd expect the 'native' referencing to be faster (it is certainly much neater), but maybe this is a silly assumption? I'm able to sort (to some extent) tableB in a order of frequency of the lookups whereas (as I understand it) tableA will have in Lua random internal order by default even if I declare the keys in proper order.
A lookup table will always be faster than searching a table every time.
For 100 elements that's one indexing operation compared to up to 100 loop cycles, iterator calls, conditional statements...
It is questionable though if you would experience a difference in your application with so little elements.
So if you build that data structure for this purpose only, go with a look-up table right away.
If you already have this data structure for other purposes and you just want to look something up once, traverse the table with a loop.
If you have this structure already and you need to look values up more than once, build a look up table for that purpose.
I've been making a game with the LOVE2D game engine, and I've stumbled across an issue. I want to access a variable inside a nested table, but I don't know how.
Here's my code right now:
local roomNum = 1
local rooms = { r1 = { complete = false, name = "Room 1" }
if rooms[roomNum].complete == true then --problematic line
--do stuff
end
If I replace rooms[roomNum].complete with rooms.r1.complete then it works.
Any help would be appreciated!
'http://lua-users.org/wiki/TablesTutorial'
The provided link gives easy to understand examples on tables in Lua, so it may prove a useful resource in the future.
As for the why the replacement code worked, a dictionary is just sets of key/value pairs (kvp) . In examples from other languages, these pairs are normally shown as something like KeyValuePair.
In your case, you are using a variation on how dictionaries are used. As you have seen, you can use numbered indexes like room[1], or you can use a string like room["kitchen"]. It gets interesting when you provide a set of data to initialize the dictionary.
Building off of the provided data, you have the following:
local rooms = { r1 = { complete = false, name = "Room 1" }
r1 is equivalent to using rooms["r1"] without the dataset. In providing the dataset, any "named" Key can be referenced like it is a property of the dictionary (think of classes with public getter/setter). For the named keys of a dataset, you can provide a key as numbers as well.
local rooms = { [1] = { complete = false, name = "Room 1" }
This indexing fits the direction you were headed on providing a room index. So, you could either swap the dataset to use integers instead of r1, r2 and so on, or you could concatenate r and the index numbering. That is pretty much up to you. Keep in mind as you go further down nesting the same rules apply. So, complete could look like rooms[1].complete, rooms["r1" ].complete, or rooms.r1.complete.
I'm trying to display a table that counts webhooks and arranges the various counts into cells by date_sent, sending_ip, and esp (email service provider). Within each cell, the controller needs to count the webhooks that are labelled with the "opened" event, and the "sent" event. Our database currently includes several million webhooks, and adds at least 100k per day. Already this process takes so long that running this index method is practically useless.
I was hoping that Rails could break down the enormous model into smaller lists using a line like this:
#today_hooks = #m_webhooks.where(:date_sent => this_date)
I thought that the queries after this line would only look at the partial list, instead of the full model. Unfortunately, running this index method generates hundreds of SQL statements, and they all look like this:
SELECT COUNT(*) FROM "m_webhooks" WHERE "m_webhooks"."date_sent" = $1 AND "m_webhooks"."sending_ip" = $2 AND (m_webhooks.esp LIKE 'hotmail') AND (m_webhooks.event LIKE 'sent')
This appears that the "date_sent" attribute is included in all of the queries, which implies that the SQL is searching through all 1M records with every single query.
I've read over a dozen articles about increasing performance in Rails queries, but none of the tips that I've found there have reduced the time it takes to complete this method. Thank you in advance for any insight.
m_webhooks.controller.rb
def index
def set_sub_count_hash(thip) {
gmail_hooks: {opened: a = thip.gmail.send(#event).size, total_sent: b = thip.gmail.sent.size, perc_opened: find_perc(a, b)},
hotmail_hooks: {opened: a = thip.hotmail.send(#event).size, total_sent: b = thip.hotmail.sent.size, perc_opened: find_perc(a, b)},
yahoo_hooks: {opened: a = thip.yahoo.send(#event).size, total_sent: b = thip.yahoo.sent.size, perc_opened: find_perc(a, b)},
other_hooks: {opened: a = thip.other.send(#event).size, total_sent: b = thip.other.sent.size, perc_opened: find_perc(a, b)},
}
end
#m_webhooks = MWebhook.select("date_sent", "sending_ip", "esp", "event", "email").all
#event = params[:event] || "unique_opened"
#m_list_of_ips = [#List of three ip addresses]
end_date = Date.today
start_date = Date.today - 10.days
date_range = (end_date - start_date).to_i
#count_array = []
date_range.times do |n|
this_date = end_date - n.days
#today_hooks = #m_webhooks.where(:date_sent => this_date)
#count_array[n] = {:this_date => this_date}
#m_list_of_ips.each_with_index do |ip, index|
thip = #today_hooks.where(:sending_ip => ip) #Stands for "Today Hooks ip"
#count_array[n][index] = set_sub_count_hash(thip)
end
end
Well, your problem is very simple, actually. You gotta remember that when you use where(condition), the query is not straight executed in the DB.
Rails is smart enough to detect when you need a concrete result (a list, an object, or a count or #size like in your case) and chain your queries while you don't need one. In your code, you keep chaining conditions to the main query inside a loop (date_range). And it gets worse, you start another loop inside this one adding conditions to each query created in the first loop.
Then you pass the query (not concrete yet, it was not yet executed and does not have results!) to the method set_sub_count_hash which goes on to call the same query many times.
Therefore you have something like:
10(date_range) * 3(ip list) * 8 # (times the query is materialized in the #set_sub_count method)
and then you have a problem.
What you want to do is to do the whole query at once and group it by date, ip and email. You should have a hash structure after that, which you would pass to the #set_sub_count method and do some ruby gymnastics to get the counts you're looking for.
I imagine the query something like:
main_query = #m_webhooks.where('date_sent > ?', 10.days.ago.to_date)
.where(sending_ip:#m_list_of_ips)
Ok, now you have one query, which is nice, but I think you should separate the query in 4 (gmail, hotmail, yahoo and other), which gives you 4 queries (the first one, the main_query, will not be executed until you call for materialized results, don forget it). Still, like 100 times faster.
I think this is the result that should be grouped, mapped and passed to #set_sub_count instead of passing the raw query and calling methods on it every time and many times. It will be a little work to do the grouping, mapping and counting for sure, but hey, it's faster. =)
In case this helps anybody else, I learned how to fill a hash with counts in a much simpler way. More importantly, this approach runs a single query (as opposed to the 240 queries that I was running before).
#count_array[esp_index][j] = MWebhook.where('date_sent > ?', start_date.to_date)
.group('date_sent', 'sending_ip', 'event', 'esp').count
Say I have many different classes that inherit from Tree and each of them implements a method called grow! but with a slightly different ActiveRecord implementation. Say each method begins with an ActiveRecord query to find the right trees to grow with something like:
trees = Tree
.joins(:fruits)
.where(land_id: land.id)
.where(fruits: { sweet: true })
.where(fruits: { season_id: season.id })
Say the part we want to swap out from query to query is this part:
.where(fruits: { sweet: true })
Say we want to then build a WinterTree class and its own grow method but it only grows non sweet fruits and so we want to return trees that only grow non-sweet fruits. Is there anyway to not have to rewrite the rest of the query and only swap out that one piece of the query and maybe write the rest of the query in the parent Tree class? Is there anyway to call AR segments of queries dynamically?
I found it easy to build dynamic queries using where statements in sql such as: Tree.joins(:fruits).where("land_id = ?", land.id ) etc. Below is what I did yesterday to give you some idea of what I'm talking about but you'll need to extrapolate it to fit your needs:
query = ''
counter = 1
sets_of_data_ill_query.each do |set|
if counter == 1
query += "district = '#{set[0]}' AND second_district = '#{set[1]}'"
else
query += " OR district = '#{set[0].to_s}' AND second_district = '#{set[1]}'"
end
end
voters = Voter.where(query)
NOTE: I knew the data I was querying was safe so I just used the raw info but you'll want to do it as I showed in the first paragraph with ?escaping values if it's data that will be entered by users. Also, since you're chaining where statements you would want to use an "AND" instead of where I used "OR" if you need to loop through sets etc.
I am not sure if I am going about this the best way, but I will try to explain what I am trying to do.
I have the following domain classes
class User {
static hasMany = [goals: Goal]
}
So each User has a list of Goal objects. I want to be able to take an instance of User and return 5 Users with the highest number of matching Goal objects (with the instance) in their goals list.
Can someone kindly explain how I might go about doing this?
The easiest and most efficient way to achieve this is using plain SQL. Assuming you have these tables
users [id]
goals [id, description]
user_goals [user_id, goal_id]
You can have the following query to do what you need:
set #userId=123;
select user_id, count(*) as matched from user_goals
where user_id!=#userId
and goal_id in (select ug.goal_id from user_goals ug where ug.user_id=#userId)
group by user_id order by matched desc limit 5;
This takes a user id and returns a list of other users with matching goals, sorted by the number of matches. Wrap it up in a GoalService and you're done!
class GoalService {
def findUsersWithSimilarGoals(user) {
// ...
}
}
It may also be possible to do this with criteria or HQL, but with queries like this it's usually easier to use SQL.
If you're looking for a simple match, perhaps the easiest way would be to do a findAll for each Goal and then count the number of results that each other User appears in:
Map user2Count = [:]
for (goal in myUser.goals){
for (u in User.findAllByGoal(goal)){
def count = user2Count.containsKey(u) ? user2Count.get(u) : 0
count++
user2Count.put(u, count)
}
}
// get the top 5 users
def topUsers = user2Count.entrySet().sort({ it.value }).reverse()[0..5]
This may be too slow, depending on your needs, but it is simple. If many users share the same goals then you could cache the results of findAllByGoal.