I rank API's by upvotes on this site.
I'm calculating the score on every page load. This is expensive and slow and the page load is taking ~5 seconds.
Is there a more efficient way to do this? Every time a new post is created, do I need to rerank everything using a background job? How about every time a vote is submitted and every time a post grows older by an hour?
This seems like a common problem to have on any site that is ranking dependent on time + votes.
If you can give some info on your models and ranking system I can refine this answer, but here's my suggestion based on the info you've provided thus far:
Create a column on your AR model called count.
When a new Api object is created, set the count column to 0. When it gets upvoted, do this:
def upvote
#api = Api.find_by_id(params[:id])
new_count = #api.count + 1
#api.update_attributes(:count => new_count)
end
You can do the opposite for downvotes.
Then in your view, just do this in the loop:
<% #apis.each do |api| %>
<%= api.count %>
<% end %>
That will stop counting on page loads, and will simply pull a value from the database. Hope this helps, and cool idea for a site. If you want to use a gem that I've liked pretty well so far, Thumbs Up is a good option.
Related
I have a hash of hashes that I use to create a stacked column chart. Along the x axis, I have the days of the week and the y axis is the total points for the day for each goal, stacked to the total points for the day overall.
Here is the method in my controller:
def goal_xp_by_day
# Create an array of goals to iterate through only if they have activities
# recorded in the last 7 days.
days_goals = current_user.goals.select { |g| g.empty_days}
# Iterate through goals and for each goal, create a hash of days and points
goals = days_goals.map do |goal|
days = {}
(Date.today-6..Date.today).each do |day|
# Pass the day to a method which uses the paramater
# to calculate the points for that period
days[day.strftime("%a")] = goal.daily_goal_xp(day)
end
{
name: goal.name,
data: days
}
end
render json: goals
end
As the days pass, I know the results for each day (excluding the current) will not change but the current day's result could change at any time so caching the entire chart makes no sense. I'd like to find a away to cache the data for the previous 6 days to prevent excessive querying but I'm not sure how to go about breaking up this method to query the last 6 days, cache it and the run the queries to calculate the results for today then finally merge it all together to create the updated graph.
I was thinking of passing a parameter like the date of the day into the cache_key and possibly using some form of Russian doll caching for each hash. Is this something that can be done in the controller? I know usually they're used in the view files but in this case, the results are returned in the controller.
If there's a way to improve the efficiency of these queries, I would really appreciate some advice/pointers, even if caching isn't an option.
Yes, something like this could be done at the controller level. You could wrap the entire method contents with a general cache call and then wrap each infrequently changing section of the code with another key.
Something like this might get you pointed down the right road:
(Date.today-6..Date.today).each do |day|
days[day.strftime("%a")] = Rails.cache.fetch("day-goal", day) do
goal.daily_goal_xp(day)
end
end
This will result in a cache key for a given day which won't change. Don't forget to update your cache keys with any model instances which might cause the cache to expire.
I have a 10,000+ row table that I am trying to query in Rails 4. Let's call it events and there are many, many years of past events still in the database. I need to grab only the events with a date of today or in the future. I could do something like:
#events = Event.where('date >= ?', todays_date)
but this still takes a long time as it has to go through every record. Is there a simple way to speed this up? I've even considered that starting from the most recent rows, moving backwards, and limiting to say 1,000 total would be "good enough", although it could potentially leave out an event created a long time ago but with a date in the near future. I'm not even sure how to go about starting from the last row, however. And I feel like there has to be a better solution anyhow.
Any ideas? Thank you in advance!
One thing you can do to speed up the search is to add index to the created_at field (or to the custom date field you are using). Index makes the searching process faster in the database.
You can refer this answer to add index.
It sounds like adding an index to your date column could help with efficiency here.
Migration code below:
class AddIndexToEvents < ActiveRecord::Migration
def change
add_index :events, :date
end
end
I am implementing a score system to rank trending in a rails application. To score the items I'm using a basic number of likes x age.
#post.score = #post.likers(User).length * age
I have a field in the posts database called score. My question is where do I call the above code so that the score is constantly getting updated as it gets older or when someone new likes the post.
Thanks for any help.
As it's an ever changing value, and not really a static field, I'd suggest consider putting this in a method, not in the database. Unless for performance reasons that is needed. So, in your Post model just have:
def score
#score algorithm here
end
This way, whenever you call post.score, it will be calculated at that time and shown.
Alternatively if age is a daily value, you could use some kind of scheduled task (cron/whenever) to update this on a daily basis.
The code you posted should work. Just make sure you call #post.save on the next line. Can we see your codebase?
List item
each day I want to find the "most popular" post on the website and feature it on the home page.
For each post, I'm keeping track of how many times it has been "liked", "disliked", "favorited" and "viewed".
I would like to run a daily cron job where I do something like:
post = Post.order("popularity_score DESC").first
post.feature!
My question is, how should I compute the value of popularity_score?
Is there a formula that takes into consideration "statistical significance"? Meaning, a post which has 1 "like" vote and nothing else, although having a 100% approval rating, it shouldn't mean much because only one person voted on it.
In general I have these loose ideas off the top of my head:
a post with 10 likes and no other votes is more popular than a
post with 1 like vote.
a post post with more "dislikes" than
"likes" should have a lower score than a post with more "likes" than
"dislikes"
a post with 20 views and no other votes is more
popular than a post with 3 views.
I've punched in some arbitrary formulas to try to satisfy this goal, but there are exactly that, arbitrary and I don't really know if there is a better way to go about this?
Suggestions?
Maybe you could just take the SO approach? it seems rather decent.
+ gives 10 points
- substracts 2 points
view add a low number, like 0.01 point
comment add 2 points
One suggestion is to not reset your counter each day (that leaves the "most popular" open to a single vote).
Instead, weight the votes by their age -- newer votes count more than older votes. This will give you gradual and meaningful rerankings over time.
I have a current implementation of will_paginate that uses the paginate_by_sql method to build the collection to be paginated. We have a custom query for total_entries that's very complicated and puts a large load on our DB. Therefore we would like to cut total_entries from the pagination altogether.
In other words, instead of the typical pagination display of 'previous 1 [2] 3 4 5 next', we would simply like a 'next - previous' button only. But we need to know a few things.
Do we display the previous link? This would only occur of course if records existing prior to the ones displayed in the current selection
Do we display the next link? This would not be displayed if the last record in the collection is being displayed
From the docs
A query for counting rows will
automatically be generated if you
don’t supply :total_entries. If you
experience problems with this
generated SQL, you might want to
perform the count manually in your
application.
So ultimately the ideal situation is the following.
Remove the total_entries count because it's causing too much load on the database
Display 50 records at a time with semi-pagination using only next/previous buttons to navigate and not needing to display all page numbers available
Only display the next button and previous button accordingly
Has anyone worked with a similar issue or have thoughts on a resolution?
There are many occasions where will_paginate does a really awful job of calculating the number of entries, especially if there are joins involved that confuse the count SQL generator.
If all you need is a simple prev/next method, then all you need to do is attempt to retrieve N+1 entries from the database, and if you only get N or less than you're on the last page.
For example:
per_page = 10
page = 2
#entries = Thing.with_some_scope.find(:all, :limit => per_page + 1, :offset => (page - 1) * per_page)
#next_page = #entries.slice!(per_page, 1)
#prev_page = page > 1
You can easily encapsulate this in some module that can be included in the various models that require it, or make a controller extension.
I've found that this works significantly better than the default will_paginate method.
The only performance issue is a limitation of MySQL that may be a problem depending on the size of your tables.
For whatever reason, the amount of time it takes to perform a query with a small LIMIT in MySQL is proportional to the OFFSET. In effect, the database engine reads through all rows leading up to the particular offset value, then returns the next LIMIT number rows, not skipping ahead as you'd expect.
For large data-sets, where you're having OFFSET values in the 100,000 plus range, you may find performance degrades significantly. How this will manifest is that loading page 1 is very fast, page 1000 is somewhat slow, but page 2000 is extremely slow.