I'm working on web app (Rails 3 based). And I really don't like the time it takes to generate the page - depending on the displayed data it takes up to 2.5 and even 4 seconds.
So I just was wondering what is the average reasonable time for generating page in your apps. Saying you check the generation time, e.g. it's 750ms and think "Ok, that should be fine even without caching". Or when you see 1.5sec you think "Oh my God, the user won't wait so long and leave the site"
There's a huge amount of research data regarding the time from query to rendering and user's experience. I'd recommend reading this useit.com article. After all Google integrated page speed in its results for a reason ;)
The 3 response-time limits are the
same today as when I wrote about them
in 1993 (based on 40-year-old research
by human factors pioneers):
0.1 seconds gives the feeling of instantaneous response — that is, the
outcome feels like it was caused by
the user, not the computer. This level
of responsiveness is essential to
support the feeling of direct
manipulation (direct manipulation is
one of the key GUI techniques to
increase user engagement and control —
for more about it, see our Principles
of Interface Design seminar).
1 second keeps the user's flow of thought seamless. Users can sense a
delay, and thus know the computer is
generating the outcome, but they still
feel in control of the overall
experience and that they're moving
freely rather than waiting on the
computer. This degree of
responsiveness is needed for good
navigation.
10 seconds keeps the user's attention. From 1–10 seconds, users
definitely feel at the mercy of the
computer and wish it was faster, but
they can handle it. After 10 seconds,
they start thinking about other
things, making it harder to get their
brains back on track once the computer
finally does respond.
A 10-second delay will often make
users leave a site immediately. And
even if they stay, it's harder for
them to understand what's going on,
making it less likely that they'll
succeed in any difficult tasks.
As a rule of thumb, think that you always should aim for a balance of optimization time vs time gained. Don't spend days optimizing the hell out of one routine when your images aren't compressed correctly, or your scripts/css not combined. Yes, faster is better, but a 90% gain in generating the page by setting up a smart cache beats a 10% gain after one week tweaking the algorithm.
Also don't look too much into the first-render-time when the framework has to load everything, but use stress-testing, cached or not, to simulate various situations.
Now, some data; some of the latest sites i worked on used DotNetNuke, a huge open-source CMS, and Asp.Net MVC where you nearer to the metal. Average page time with average db queries was 600-700 milliseconds for DotNetNuke. For Asp.net MVC, it's 70-100 milliseconds... Users really like the second one :)
There's no 'right' answer to this - the faster the better. Personally I normally aim for < 200ms, although I know from experience that it can be quite difficult to achieve this in Rails on anything but simple apps. Try and figure out where your bottlenecks are and cache what you can.
Edit: There seems to be some confusion between page generation time and page render time. Obviously a quick page render is the goal, and on most sites doing things like reducing HTTP requests, gzipping CSS/JS are where you can get most of your quick wins. But if the page itself can take 4-5 seconds to generate, then you're probably right that your app is where you should start.
It depends on whether nothing is displayed for 2.5-4 seconds, or that the user already sees (a part of) the page from the start, and it finishes loading completely after 2.5-4 seconds. In that case the user doesn't experience a 2.5-4 second load. Take the http://www.nytimes.com/ website; I see most of it right away, but according to the Web Inspector it takes 1.94 seconds for it to be loaded completely.
And keep in mind that the speed will also depend on the browser, computer, internet connection. What's fast for you might be slower for others.
Measure your apdex score and see how it is performing. That will give you a rough indiciation. From there, you can decide how you want to increase performance.
It also depends on what your site is; an system application for a business or software as a service (SaaS)? If it's a system application, the users are forced to use it to performance can be negotiated. If it is a SaaS, then the higher your apdex score, the more chance you have of losing your user's interest.
There are a few gems out there that measure performance and report on what your apdex is.
Here's a little more info: http://apdex.org/blog/?p=630
My personal rule - no page should take more than 0.05 seconds, or you are in troubles.
As long as you write proper code, you don't need to spend much time on optimization to stay under 0.05.
If you stick to giant frameworks, then you are out of luck.
Related
I'm using Glimpse to debug some perfomance problems in my website, and it seems that the server/framework sits iddle for too long between method calls.
This picture shows 320 ms of server time;
This second picture reveals that 125.29 ms are used by ViewResult.ExecuteResult (I understand that as "rendering", which seems pretty slow to me, considering that my views are pre-compiled - more on that below);
But the really odd thing here is that more than 100 ms are pretty much wasted with iddle time, as you can verify in this picture.
Those little blocks representing server work sometimes account for 0 ms! But then there's a lapse of about 15 ms before the next block.
Is it really iddle time? Do you have any tips for where to look next, or how to optimize this?
Disclaimer: I've been looking into this for a week or so, and I have already found and applied those general performance recommendations, like:
Only one View Engine is active (RazorViewEngine);
Run in Release mode;
Specify full view paths, like "~/Views/Folder/ActionName.cshtml".
Besides that, Donut Caching is active, views are pre-compiled with Razor Generator, and I'm using Glimpse for diagnostics. Anyways, I've tried disabling these things to ensure that they were not the offenders, and I verified that they're actually improving the times.
Thanks in advance.
Caching is something that I kind of ignored for a long time, as projects that I worked on were on local intranets with very little activity. I'm working on a much larger Rails 3 personal project now, and I'm trying to work out what and when I should cache things.
How do people generally determine this?
If I know a site is going to be relatively low-activity, should I just cache every single page?
If I have a page that calls several partials, is it better to do fragment caching in those partials, or page caching on those partials?
The Ruby on Rails guides did a fine job of explaining how caching in Rails 3 works, but I'm having trouble understanding the decision-making process associated with it.
Don't ever cache for the sake of it, cache because there's a need (with the exception of something like the homepage, which you know is going to be super popular.) Launch the site, and either parse your logs or use something like NewRelic to see what's slow. From there, you can work out what's worth caching.
Generally though, if something takes 500ms to complete, you should cache, and if it's over 1 second, you're probably doing too much in the request, and you should farm whatever you're doing to a background process…for example, fetching a Twitter feed, or manipulating images.
EDIT: See apneadiving's answer too, he links to some great screencasts (albeit based on Rails 2, but the theory is the same.)
You'll want to think about caching several kinds of things:
Requests that are hit a lot, and seldom change
Requests that are "expensive" to draw, lots of database calls, etc. Also hopefully these seldom change.
The other side of caching that shouldn't go without mention, is expiration. Its also often the harder part. You have to know when a cache is no longer good, and clear it out so fresh content will be generated. Sweepers, or Observers, depending on how you implement your cache can help you with this. You could also do it just based on a time value, allow caches to have a max-age and clear them after that no matter what.
As for fragment vs full page caching, think of it in terms of how often those parts are updated. If 3 partials of a page are never updated, and one is, maybe you want to cache those 3, and allow that 1 to be fetched live for so you can have up to the second accuracy. Or if the different partials of a page should have different caching rules: maybe a "timeline" section is cached, but has a cache-age of 1 minute. While the "friends" partial is cached for 12 hours.
Hope this helps!
If the site is relatively low activity you shouldn't cache any page. You cache because of performance problems, and performance problems come about because you have too much data to query, too many users, or worse, both of those situations at the same time.
Before you even think about caching, the first thing you do is look through your application for the requests that are taking up the most time. Not the slowest requests, but the requests your application spends the most aggregate time performing. That is if you have a request A that runs 10 times at 1500ms and request B that runs 5000 times at 250ms you work on optimizing B first.
It's actually pretty easy to grep through your production.log and extract rendering times and URLs to combine them into a simple report. You can even do that in real-time if you want.
Once you've identified a problematic request, you go about picking apart what it's doing to service the request. The first thing is to look for any queries that can be combined by using eager loading or by looking ahead a bit more to anticipate what you'll need. The next thing is to ensure you're not loading data that isn't used.
So many times you'll see code to list users and it's loading 50KB per person of biographical data, their Facebook and Twitter handles, literally everything about them, and all you use is their name.
Fetch as little as you need, and fetch it in the most efficient way you can. Use connection.select_rows when you don't need models.
The next step is to look at what kind of queries you're running, and how they're under-performing. Ensure your indexes are all set properly and are being used. Check that you're not doing complicated JOIN operations that could be resolved by a bit of tactical de-normalization.
Have a look at what data you are storing in your application, and try and find things that can be removed from your production database and warehoused somewhere else. Cycle your data out regularly when it's no longer relevant, preserve it in a separate database if you need to.
Then go over and have a look at how your database server is tuned. Does it have sufficiently large buffers? Is it on hardware that could be upgraded with more memory at a nominal cost? Too many people are running a completely un-tuned database server and with a few simple settings they can get ten-fold performance increases.
If, and only if, you still have a performance problem at this point then you might want to consider caching.
You know why you don't cache first? It's because once you cache something, that cached data is immediately stale. If parts of your application use this data under the assumption it's always up to date, you will have problems. If you don't expire this cache when the data does change, you will have problems. If you cache the data and never use it again, you're just clogging up your cache and you will have problems. Basically you'll have lots of problems when you use caching, so it's often a last resort.
I need to implement a draft application for a fantasy sports website. Each users will have 1m30 to choose a player on its team and if that time has elapsed it will be selected automatically. Our planned implementation will use Juggernaut to push the turn changes to each user participating in the draft. But I'm still not sure about how to handle latency.
The main issue here is if a user got a higher latency than the others, he will receive the turn changes a little bit later and his timer won't be synchronized. Say someone receive a turn change after choosing a player himself while on his side he think he still got 2 seconds left, how can we handle that case? Is it better to try to measure each user latency and adjust the client-side timer to minimize that issue? If so, how could we implement that?
This is a tricky issue, but there are some good solutions out there. Look into what time.gov does, and how it does it; essentially, as I understand it, they use Java to perform multiple repeated requests to the server, to attempt to get an idea of the latency involved in the communication, then they generate a measure of latency that they use to skew the returned time data. You could use the same process for your application, with even more accuracy; keeping track of what the latency is and how it varies over time lets you make some statistical inferences about how reliable your latency numbers are, etc. It can be a bit complex, but it can definitely allow you to smooth out your performance. My understanding is that this is what most MMOs do as well, to manage lag.
I've been building an application in Ruby on Rails 3, and I'm starting to worry about performance optimization. Now I hope that my question is not too subjective for this site, but I'm interested in facts, not a discussion, so here goes:
While I'm trying to get my views to render faster, there is one thing I simply do not know: What should I aim for? Given a reasonably complex page, what load time is realistic? I simply don't have any reference.
What I'm typically seeing for my application is something like this:
Completed 200 OK in 397ms (Views: 341.1ms | ActiveRecord: 17.7ms)
This is on my production server, running Apache/Passenger.
I am the only one (!) making requests on that server, it's a root server (not virtual), running Ubuntu, AMD Athlon 64 X2 5600+, 4 GB RAM
That is, for most of my more complicated actions (not unusually complicated, just assume it's a paginated listing of 20 objects with 5 computed properties each or something) the ActiveRecord times are almost always fine (<20-30ms), but the "views" number is usually >200 ms.
Now, to my question: When I started using RoR my expectation (maybe unrealistic) was that for most consumer-oriented applications with average complexity (let's say something like Facebook, Twitter, etc. WITHOUT the millions of users) I would get < 20 ms load times as long as I was the only one making requests, and that for a single server load times would only approach 100ms or more if there were lots of people making requests at the same time.
My expectation was also that database requests would be the major bottleneck, since all the rest is just relatively simple computations without any real complexity. I thought that it might take 10ms to get all the objects from the database, and then maybe another 5 ms to run the controller code, build the view, etc.
Since I've never been in charge of any production app, I don't know if this expectation was in any way realistic. So I would like somebody with experience point out to me what my realistic expectation should be.
(e.g. "pretty much everything but really nasty stuff should render in 50 ms tops as long as you are the only one making requests")
or ("actually 300 ms is not unusual for RoR applications, even if you're the only user")
or ("Are you kidding? I get < 10 ms with 150 concurrent requests on a smaller server than yours. There must be something very wrong with your app)
Again, I hope this is not too subjective, but I'm not really interested in an opinion of whether or not RoR is fast, I want facts from someone with more experience on what numbers are average and to be expected from production RoR applications. Otherwise I simply have no clue at what point I should stop optimizing and just accept that I'll never get 10 ms load times.
Gosh, I'm not sure I'm the one to answer this, but since I've been around these waters enough times, I may have an incomplete idea of things to look at.
First of all, the response times is pretty subjective. Meaning, it's good enough if it's good enough for you. From my experience, pages resembling your description seem to take about as much time as what you're describing. So, you're not orders of magnitude off in either direction.
If you want to optimize your view renders with your current architecture, your next step is here, I think. Greg Pollack does a great job breaking this stuff down for you and will make sure you're on track. You'll be sure to get your assets cached and your stack fine-tuned. That'll be your most practical general advice.
If you're willing to look at your deployment architecture, Ilya Grigorik raises some great questions in this article and then answers them with Goliath. If your bottlenecks are speeding up your server-client round trip, that's probably the approach to do.
I try to pay attention to anything Aaron Patterson says about performance, like in this talk. He's going to teach general optimization ideas, most of them for your server-side code. You may catch a few things that relate to your current problem.
I was pulled aside by a former co-worker at MWRC this year and told that I'm absolutely nuts if I'm not building with JRuby these days. It's a bit of a commitment, and I've resisted making major changes like that until I have truly painful response times, which I don't, and it doesn't sound like you're having either. However, JRuby's a very mainstream thing to do now, and you and I will likely embrace this for some projects at some point in the future.
So, bottom line, I think you're in the realm of a spry app as you are. I think I'd work down these resources in the order I presented them.
Not knowing what you're rendering, it's hard to comment on the performance, but I would venture to say that 200ms is very high. Don't forget that the debug information in your logs can be a little misleading: if you're querying your DB or some external resource from within a view, as opposed to preloading that data in your controller, then that time will be attributed to view rendering.
Common culprits: you load Model X in your model, but then access an association in your view which triggers a bunch of selects under the hood. The time to fetch Model x is low, but the associated records will show up as "view time".
In other words, dig into the logs and if its actually your view code, then bring up a profiler.
I'm getting view times < 20ms on a $20/month linode server. That's well-optimized code, for a request of medium complexity, running on JRuby. You haven't hit Rails' performance limits by any means. Time to use a profiler and see what's taking so long.
I don't think your 200 ms view time is abnormal, or even high in any way.
However, you have room for improvement. You say " (not unusually complicated, just assume it's a paginated listing of 20 objects with 5 computed properties each or something)"
To me, that's 100 operations that could be pre-calculated, and would speed up your view rendering time.
Finally -- Rendering time doesn't usually have a direct correlation to number of users. Under most deployments, as a request comes in, it is handled by a process and then responded to. Other requests wait until the first is completed before they are processed.
Use static content where possible. Outside of that, use caching where possible, at the highest possible level, preferably at the page level. When content can't be cached, try to get -something- static or cacheable back to the user quickly. You might, for instance, serve up a static page with the basic layout, and an animated busy-image where the content belongs, and then use JavaScript to load the dynamic content.
For security reasons I have a feeling that that testing should be done server side. Nonetheless, that would be rather taxing on the server, right? Given the gear and buffs a player is wearing they will have a higher movement speed, so each time they move I would need to calculate that new constant and see if their movement is legitimate (using TCP so don't need to worry about lost, unordered packets). I realize I could instead just save the last movement speed and only recalculate it if they've changed something affecting their speed, but even then that's another check.
Another idea I had is that the server randomly picks data that the client is sending it and verifies it and gives each client a trust rating. A low enough trust rating would mean every message from the client would be inspected and all of their actions would be logged in a more verbose manner. I would then know they're hacking by inspecting the logs and could ban/suspend them as well as undo any benefits they may have spread around through hacking.
Any advice is appreciated, thank you.
Edit: I just realized there's also the case where a hacker could send tiny movements (within the capability of their regular speed) in a very high succession. Each individual movement alone would be legite, but the cumulative effect would be speed hacking. What are some ways around this?
The standard way to deal with this problem is to have the server calculate all movement. The only thing that the clients should send to the server are commands, e.g. "move left" and the server should then calculate how fast the player moves etc., then finally send the updated position back to the client.
If you leave any calculation at all on the client, the chances are that sooner or later someone will find a way to cheat.
[...] testing should be done server side. Nonetheless, that would be rather taxing on the server, right?
Nope. This is the way to do it. It's the only way to do it. All talk of checking trust or whatever is inherently flawed, one way or another.
If you're letting the player send positions:
Check where someone claims they are.
Compare that to where they were a short while ago. Allow a tiny bit of deviation to account for network lag.
If they're moving too quickly, reposition them somewhere more reasonable. Small errors may be due to long periods of lag, so clients should use interpolation to smooth out these corrections.
If they're moving far too quickly, disconnect them. And check for bugs in your code.
Don't forget to handle legitimate traversals over long distance, eg. teleports.
The way around this is that all action is done on the server. Never trust any information that comes from the client. If anybody actually plays your game, somebody will reverse-engineer the communication to the server and figure out how to take advantage of it.
You can't assign a random trust rating, because cautious cheaters will cheat only when they really need to. That gives them a considerable advantage with a low chance of being spotted cheating.
And, yes, this means you can't get by with a low-grade server, but there's really no other method of preventing client-side cheating.
If you are developing in a language that has access to Windows API function calls, I have found from my own studies in speed hacking, that you can easily identify a speed hacker by calling two functions and comparing results.
TimeGetTime
and...
GetTickCount
Both functions will return the number of seconds since the system started. However, TimeGetTime is much more accurate than GetTickCount, whereas TimeGetTime is accurate up to ~1ms vs. GetTickCount, which is accurate at around ~50ms
Even though there is a small lag between these two functions, if you turn on a speed hacking application (pick your poison), you should see a very large difference between the two result sets, sometimes even up to a couple of seconds. The difference is very noticable.
Write a simple application that returns the GetTickCount and TimeGetTime results, without the speed hacking application running, and leave it running. Compare the results and display the difference -- you should see a very small difference between the two. Then, with your application still running, turn on the speed hacking application and you will see the very large difference in the two result sets.
The trick is figuring out what threshold will constitue suspicious activity.