I've built, maintain and continually improve a website for an insanely complex fantasy football league I'm in where GMs have $130 of salary cap a year, players all have a starting salary value and may be signed to longterm deals using that initial salary amount as a starting point for subsequent years.
There's a lot more to it, but that's enough for what I need info on. Each GM has a team page that shows all the players he has on his team/signed to longterm deals along with some other roster breakdowns, historical payout information, draft rosters (basically a drag-and-drop sortable "shopping list" of players to keep an eye on), pending/completed/rejected trade requests, and private messages along with a lot of other information:
As you might imagine, this is an INSANE amount of information to load for each team. Upgrading to Ruby 2.1.2 alleviated a LOT of issues (page load times dropped from ~7 seconds to ~3 seconds on average), but there's still more that could be done.
Is there any way to make my 93-line teams#show action load lazily? So, like, for instance, the Roster and all associated variables would load when going to a team page since that's the first/default tab. But then the Payouts, Draft Rosters, Trades and Messages variables wouldn't fire their queries to set the variables until each of their tabs was activated? I've been trying to find anything I can about Rails lazy loading but I've had no luck at all finding anything even remotely close to what I want to do.
You can create one controller action per tab.
So you have something like this. The main page would be the users#show action. That would render the main layout and the tabpanel (with the roster tab).
Then create one action per tab. So you would have:users#payouts, users#draft_rosters, users#trades users#mailbox
Those actions can be loaded via ajax only when the tab is clicked for the first time: http://jqueryui.com/tabs/#ajax
Each tab would have it's own html.erb file. This file is the content of the page.
This way each tab has it's own instance variables that are only loaded when the tab is clicked.
Related
I am currently working on an application in Rails (though language/framework shouldn't matter for this question since it is more of a theoretical one). I'm working on wrapping my head around this problem:
Say I am tracking millions of blogs online and am plugged into their RSS feeds. My app pings these feeds every few few minutes to see if there has been any new activity across any of these millions of blogs. If there is any new activity, I want to alert users of my application who have signed up to receive alerts for specific blogs that there has been an alert.
Does it make sense to have a user_blog_alerts table (where a user can specify custom keywords to be alerted about) and continuously check this table against every new entry that comes in from my feed? And when there is a match, to add them to a queue (using Redis)?
What is the best, most efficient way to build and model this alerting system? Am I even thinking about this in the right way? Are there any good examples or tutorials on this when working with such large amounts of data?
I'm not sure what the right way to do this is, but the thought of continuously scanning a table over and over sounds exhausting (ie. unscalable).
Off the top of my head, what if you created a LIST for every blog in Redis. The values would be the user IDs of those who wanted an alert. The key name would contain the blog id (ex: "user_blog_alerts:12345").
Then when you got a new post for blog 12345 it's a simple lookup to see if that key exists. If it does, then fire off alerts for each user in the list.
On the side of each page on my site I list the other Pages in that Section. I also mark if the User visited or completed each page. Every time a page loads, Rails has to check the database to see if the User visited or completed each page in the Section, which makes displaying a page take longer. Is there a way I can speed this up?
Example: user visits page 2 and the sidebar loads. He then visits page 3 and the sidebar loads again, but likely with a different mark next to Page 2.
Page 2
------
- ✓ page 1 | Main Content
- > page 2 | The quick brown fox...
- page 3 |
...
There are a few areas you could improve performance, both in database lookup and in rendering.
For rendering, you should look at fragment cacheing the sidebar for visited links, although your scenario is a bit out of the ordinary in terms of keeping track of visited links in your app.
For each link in the side bar you could save a fragment that represents visited and unvisited, then the outer container could represent any permutation of visited and unvisited since it doesn't really matter which user it is when you display it. You will end up with a lot of various side bars for each of those scenarios.
For the database you could look at something like identity_cache and hold the list of users page visits in the cache to avoid unneeded lookups, invalidating when they hit a new page.
This is as good as I can get for such a broad question in relation to Rails and speed ups in general, which seems to be what the question is asking. There are numerous other platforms specific speedups that you can get by playing with different caching back-ends and servers as well.
I assume you are keeping track of the pages they have visited and/or completed in a database table because this data needs to remain stateful.
If so, first off, make sure your table is properly indexed on the columns it needs to look up.
Then, cache the results in a sessions variable, so that you can retrieve it from there and only go to the database when necessary (e.g. something changes the state or a certain period of time lapses)
I am trying to convert my asp.net mvc4 app, which had fairly heavy use of SessionState, into a stateless app. I understand that I can store this information in the DB, and intend to do so.
My question, though, is about my particular architecture. My app has a main 'page' consisting of a number of partial view panels, which each have actions in them that can affect the other panels. What i've been doing up to now is storing the entire state of the viewModel (lots of inter-related EF list collections and 'record' objects) in the session, and its been working great. Except when the session just randomly dies.
So, I need to get this data out of the session, and into the DB where I can rebuild the thing at need. My concern is that, if I store the info in the database, every single action done on screen might affect 3-5 different panels, each with their own State updates, thats a minimum of 10 round trips to the DB for every interaction!
What are some strategies I can use to make this idea more scalable?
EXTRA INFO
The view in question here is a sort of POS shopping cart system. There are panels for selecting events, selecting/adding items to the cart, editing cart items, selecting contacts, editing contacts, displaying the cart items, displaying the cart 'subtotals', and finally, a panel with a [checkout] button.
Selecting a new event will change the list of available items. Selecting an item to add to the cart will change the cart item list, subtotals, as well as the checkout panel. Same for editing a cart item.
The main concern is how to recover from a lost session, as I've found the built-in asp.net session code too unreliable. My testers have encountered issues with sessions timing out, and then my app not having any kind of recovery process. When its installed on 1500 sites, each with an average of 10 users, its going to be a plague of lost session issues, and I need to combat that before it becomes a real problem.
I agree that I'm not going stateless...wrong choice of words used in a rush. I'm just trying to move that state into a form that I can rely on past the session failure. My main idea presently is to continue using the session as the local cache for the viewModel data, but to have a fallback operation that can rebuild the viewModel from DB if the session one is lost somehow.
You shouldn't necessarily be using a database to store (what sounds like) data that only needs to be persisted in the short term.
If these changes to the other partials are only relevant in the context of the current "master view," then I would suggest using jQuery AJAX to send off the requests, parse the response JSON and update the other views. Tutorials on jQuery AJAX and ASP.NET MVC are easy to find, if you don't already have the knowledge:
http://www.codeproject.com/Articles/41828/JQuery-AJAX-with-ASP-NET-MVC
This way, you don't need to make a bunch of round trips. If the changes need to be persisted beyond the context of the current view, make ONE round trip to the database to perform the update and then simply update all of the other partials from the in-memory response from the AJAX call.
You don't need to read from secondary storage multiple times when you already have all of the information you need in-memory. Just do the reading and writing once.
I decided to go with a hybrid approach. I'm still using session, but I'm building out a DB 'recovery' option, so that if the session portion is lost, the DB will be able to provide the values needed to rebuild the session seamlessly.
Seems to be working well, so far.
I'm once again looking into the world of tabbed browsing and Sessions. Looking over a few google searches it seems that there isn't a nice way of supporting this.
Does anyone know of a method that allows Bookmarking without stealing a session (cookieless) (and this doesn't work in MVC2 for dataannotations).
Supporting tabs in such a way that it's per use case (like Windows Workflow), going through two workflows at once.
I'm thinking a url in the query string might support this, but I'm wondering if anybody else has done a similar implementation.
[Edit] Use Case: Say I'm writing an application that uses something like Windows Workflow. Each UI workflow may do an action such as collect settings of a page and execute some external process. I may wish to do two of these workflows at once (not necessarily the same UI workflow). As such if I saved in session I would get:
a) Different tabs interfering with the workflow
b) Previous/Next buttons would be extremely difficult to work out, due to a).
I would like it so either, a user cannot open another tab to a url (don't think there is a 100% method of preventing this), or allow a user to use a UI workflow in isolation without one affecting another (much like running two workflows in two different browsers).
Hopefully that gives an indication of what I'm attempting to do.
Regards,
J
It sounds like you might be trying to do the following:
For example, let's say you have a two page questionaire, the first page has first name on it and the second page has last name on it. You desire that the user can open two tabs, and be at different pages in the questionaire while entering different data in the questionaire in each tab.
So in Tab A, you have entered Mark as the first name and submitted and you are at page two now in Tab A. You decide you are going to do a questionairre for your friend also, so you open up a new Tab, Tab B. In Tab B you enter Tom and submit the page.
Currently in the browser you have Tab A, which is at page 2 of the questionaire with firstname = "Mark" and Tab B which is at page 2 of the questionaire with first name = "Tom". Assuming you wanted to maintain both of these in session on the server here is an approach that i think will work for you.
When a web browser requests page 1 of your form, on a GET request(no posted questionaire data to the server), you supply a hidden field in the the response html and generate a random number to store in that field. When this form is submitted you do the following on the server:
Look in session using the random number as a key "var questionaire = session[Request.Form["questionaire_rnumber"]]
if the questionaire is not in session you create a new questionaire and update it's properties and stick it in session
var questionaire = new Questionaire();
questionaire.FirstName = Request.Form["firstName"]
session[Request.Form["questionaire_rnumber"]] = questionaire;
if the questionaire was in the session you simply update the object, and display the next page, however when you display the next page you will want to supply the hidden random number field in the html again, using the same random number you used on page 1.
This way you can hold any number of questionaires in a single session. With MVC.NET it should be straight forward for you to add the random number field to your view model and add the logic for looking in session for an existing questionaire or creating a new one and I think you'll be good to go.
You should keep in mind the possible issues with the approach also, like back button issues, security issues, and performance issues.
One example of a security and a performance issue would be that an attacker realizes your application works like this and the attacker requests page 1 of your form 10,000 times and submit the page 1 each time. You would have 10,000 questionaire objects in that one user session. If the attacker deleted his session ID cookie 10,000 times and for each session id cookie he created 10,000 requests for page 1 and submitted the page 1 form, you would have 100,000 questionaire objects cumulatively across 10,000 sessions on your server. So you should put some constraints on it also to protect your application, for example:
Any individual session can only have X questionaires in session
Any individual IP address can only have Y concurrent sessions (this you would probably need to track in the Application object)
ADDITIONAL RESPONSE TO ADDED USE CASE
Thanks for the use case. My solution should still work for you. You have two options.
If you want to ensure there is only one tab working with your workflow, then when the random number is passed to the server from a new tab you will be able to detect that there is another workflow in progress and that the random number from the new tab does not match the random number from the first, so you will throw an exception and show the user some messaging that says they can't start a new workflow until they finish the first one, and ask if they want to cancel the first. You have to ask if they want to cancel it because if they close their browser on the first workflow they started they will be stuck until their session expires. Which won't happen if they keep trying to start a new workflow.
Secondly, you could allow them to do multiple, but segment the context of each workflow by the random number, as suggested in the first answer. The whole point is that you are making little mini-sessions in your session, but keyed of a value that is only stored in the client. So since each tab has a different random number when the form posts to the server, it's easy to correlate that random number with an entry in your session that has all the information about the workflow initiated from that tab.
Hope this helps.
You need to store wizard state information in the client in some way, via query string or form values. As you've intuited, Session will not work. Nor will anything else that relies solely on what is on the server.
I have a rails application on a shared server that also has a decently sized database, which is still growing, behind it. The application takes a long time to start/load the homepage, about 20-30 seconds for me, although some people report waiting up to several minutes.
Is there a way to flash a notice that informs people that the database may take several minutes to load while they are waiting?
It's hard to say based on your question, since we don't know exactly what your home page is showing or how it's displaying it, but assuming you are referring to an AJAX (based on the tag) call that is retrieving something from the database to be displayed on your homepage, there are a few things you can try:
Paginate the items. Is whatever you're loading a long list of items? If so, only retrieve a few at once, and let the user decide if they want to see more.
Load the rest of the page (header, footer, navigation bar, etc), and then place a loading gif spinner in the area where the content is to be loaded. If you use a javascript library like jQuery this is pretty trivial, and there are a ton of tutorials out there for it. Here is a good site for free loading indicators: http://ajaxload.info/. What you'll want to do is make the AJAX call and use your javascript library to set the loading image. Then, in the success callback for your ajax call, hide the spinner and show the content.
Load one item at a time. Make a separate ajax call for each item you're going to load, so that the user sees them coming in. This will probably end up taking longer overall (you're hitting the database more often), but the visual may be a nice psychological hack.
Look at how your database queries are set up. Are you getting everything you need in one find? That's the best way to do it, as every time you have to make another trip to the database you're increasing the wait time.
Other than that the best thing you can do is get better hardware if possible, maybe look into a VPS like linode.com.