Well, not really reinventing, however, we have a large content-based website which handles load (after we fixed the SQL pooling issue) to a certain point, then we just run out of steam. A lot of this is due to bad code we are trying to fix up, but a lot is just due to the level of requests etc.
We were considering page caching, because well, it's damn quick (yep... :D) but that doesn't work because we have certain fragments within the page which are specific to the logged in user. But, not all hope is lost...
I was wondering if it would be ideal to do the following:
Page level caching, with sweepers to clean out the pages when content is updated.
Replace the user-specific fragments with placeholders (and perhaps generic content like... 'View your Account, or Sign Up Here')
When the users page loads fire off an async request (AJAX, or, some would call it AJAH), which requests the 'dynamic' fragment, then place the content placeholders with this fragment..
The main issue I can see with this is that users with JS turned off wouldn't see the content, but I honestly don't think we would be too affected by this, and IMHO people who disable javascript, for the most part, are idiots (yes, I said it!).
I'd also be interested to know if I'm (no doubt) reinventing something, and if anyone can point me to a site which is already doing something like this it would be appreciated.
Thanks awesome SO community!
Ryan Bates covered this technique in Railscast 169: Dynamic Page Caching. It's worth taking a look at.
Have you thought about server-side fragment caching? I've used it extensively, and it couldn't be more awesome. You could simply cache the content 'fragments' and render normally whatever depends on the logged in user.
There are many good resources for fragment caching, I'd start at the documentation:
http://api.rubyonrails.org/classes/ActionController/Caching/Fragments.html
Also very good from the Scaling Rails series:
http://railslab.newrelic.com/2009/02/09/episode-7-fragment-caching
When serving static content or cached content starts to slow down serving the real working processes put a reverse proxy as frontend to your application. It will free processes to do real work and reduce slowdowns due to file system caches becoming inefficient. And it will help you to make "client side caching" shareable to multiple visitors. Have a look at the fantastic screen cast series about caching and scaling by NewRelic: http://railslab.newrelic.com/scaling-rails
Related
I am creating applications where the menu structure is kept in a SQL database. Each user has a different menu depending on their permissions. The menu items as well as the permissions to them change very rarely.
It is enough that the query that creates the menu structure is executed only once when logging into the application.
How and where to store this menu so as not to unnecessarily generate a lot of menu queries. Probably some kind of cache. Just where.
Please advise what topics will be helpful in solving the problem. Fragment caching, page caching, Redis?
Best regards
Like Dave says there are a lot of ways to do that, fragment caching can work and is the easiest, but I think it does not scale well, in this case, if you have a lot of users.
I would recommend a hotwire approach. Build your menu with hotwire and data-turbo-permanent attribute, this post can help you with more ideas: https://discuss.hotwired.dev/t/sidebar-or-navigation-bar-with-hotwire-turbo/1740
I am taking over development on a rails 3.2 application and am looking for the best way to improve page loading time. The site itself is more of a large dynamic website than an actual web application (the site is http://korduroy.tv/, a surf lifestyle community site), and while there is a couple of small pieces that differ from user to user, most of the site is the same experience for everyone.
Page loading time is fairly slow, and from looking at the server logs, it seems to be because each page is loading so much dynamic content (for example, most pages are loading resources from 10+ models). While I hope to go through and refactor what I can, I am looking for some basic performance wins. Knowing that most of the site is the same for every user, is their a way to aggressively cache the content on the server or even serve static content that has been generated through some kind of background job?
My initial thought was to create a job that uses a static site generator, maybe something like Jekyl, and basically creates a static copy of the site, which could then be served on a cdn. My gut is telling me this is probably not the way to do it, plus there are some pages (such as the user profile page), that need to be served dynamically.
Any recommendations would be great. Disclaimer, I come from front end land and have very little knowledge of best practices when it comes to server side optimizations. Thanks!
From what you write, I believe your biggest gain will be in implementing a fragment cache using a memcache store. See http://guides.rubyonrails.org/caching_with_rails.html as the definitive guide on rails caching.
You might be able to get away with a page cache or action caches for some of the content that doesn't depend on the user (like the homepage), but unless you're serving up millions of requests a day, I'm not sure this is necessary.
I notice that while the javascript and css seems to be compiled according to the rails asset pipeline, the images are missing the sha1 hashes that allow aggressive browser caching of the resources (since you don't have to worry about the contents changing, as they get new hashes when you change the images). The key here is enabling the asset pipeline, making sure you compile your assets as part of deployment (rake assets:precompile) and using the image_tag and asset_path helpers (or image-url sass helper). Also make sure that nginx is responding with code 304 (not modified) to your browser when you're refreshing a page. This won't affect the load on the rails server (unless you have both nginx and rails running on the same server), but will reduce the average page load time.
You can look into more advanced techniques like caching your sql queries, or optimizing these, but this can lead to increased complexity making it harder to maintain your codebase. Try the view caching first, and see if that gets the load time to an acceptable level.
Most of the time, whenever I hit a website that looks "bubbly" in nature, and all prettified in those pastel-like colors, I think to myself, "This was probably done with Rails." And, lo and behold, after some digging into the site's information pages I discover this is actually true. So, I pose the question, not knowing much about Rails but enough about Django to understand how the database stuff works:
Does RoR have any display-specific qualities that affect how a web page looks? Or do all RoR devs naturally use the same Adobe tools to make everything look so ubiquitous?
Ruby on Rails is a server side technology, so it doesn't lend any specific quality to the user visible design. That said, it is a "trendy" technology so people who are likely to write their back-end code with RoR are likely to choose a particular "Web 2.0" style for their views.
As a Ruby on Rails developer, I can tell you that most Ruby on Rails developers are passionate about their work and we pay a lot of attention to details when building websites as much backend as front end. Its not just a trend, its a way of thinking and working.
No, it hasn't any display-specific qualities.
The theory is that RoR makes that backend stuff easier, so more time can, and apparently is, spent on the front end stuff.
Its all done with Mirrors. And CSS. :)
Rails is a very popular Web framework, it's just be coincidence that all the ones you've looked at have been rails apps.
What kind of sites have you been looking at to draw this hypothesis?
that's a funny question with a funny description :) ... bubbly!
As a madman, I develop with RoR, it's kind of rule in our area. We learn madness from the beginning, as a result of http://railsforzombies.org...
May wise men follow a wise path!
Short Answer: NO
However...
As a Rails developer I can say that due to the Agile nature of Rails and the speed in which you can develop web applications with Rails I do find myself having more time freed up on a project to spend polishing the user interface. I believe this may be a reason you often see more polished looking Rails sites.
So in my mind I believe your choice of framework can have a direct correlation to the end product that is produced!
Rails does add some stuff to the front end. Like to every html form, it will add a hidden input element authenticity_token.
You can also tell because rails URLs and form actions will never end with suffixes like .aspx or .php or .html or .jsp, and they won't usually append ?query=book&encoding=utf8 like you see on google. And they won't usually have superlong crufties like you see on amazon (eg http://www.amazon.com/Agile-Web-Development-Rails-Ruby/dp/1934356549/ref=sr_1_1?ie=UTF8&qid=1297922135&sr=8-1). Instead Rails prefers simple routing URLs. If amazon were written in rails, you might instead expect amazon.com/books/Agile-Web-Development-Rails-Ruby
So there are ways to spot a Rails app. I expect other web frameworks, especially the ones that emulate rails, would duplicate some or all of these features, so this isn't a sure-fire method, but it helps.
I have a Rails app where I load up a base HTML layout and I fill in the main content with rows of divs from JSON. This works in 2 steps:
Render the HTML
Ajax call to get the JSON
This has the benefit of being able to cache the HTML layout which doesn't change much, but it seems to have more drawbacks:
2 HTTP requests
HTML isn't that complex, the generated html is where all the work is done, so I'm not saving that much on time probably.
Each request in my specific case requires that we check the current user, their roles, and some things related to that user, so those 2 calls are somewhat involved.
Granted, memcached will probably solve a lot of this, I am wondering if there are some best practices here. I'm thinking I could do this:
Render the first page of JSON inline, in a script block, along with the HTML. This would cut out those 2 server calls requiring user authentication. And, assuming 80% of the time you don't need to make the second ajax call (pagination/sorting in this case), that seems like a fairly good solution.
What are your thoughts on how to approach this?
There are advantages and disadvantages to doing stuff like this. In general I'd say it's only a good idea, if whatever you're delaying via an ajax call would delay the page load enough to annoy the end user for most of the use cases on your page.
A good example of this is browsing a repository on github. 90% of the time all you want is to navigate the files, so they use an ajax load to fill in the commit messages per file after the page load.
It sounds like you're trying to do this to speed up or do something fancy for your users, but I think you should consider instead, what part is slow, and what speed of page load (and maybe for what information on that page) on your users are expecting. As you say, using memcached or fragment caching might well give you the improvements you're looking for.
Are you using some kind of monitoring tool? I'm using the free version of New Relic RPM on Heroku. It gives a lot of data on request times for individual controller actions. Data like that could help you focus your optimization process.
I'd love to use page caching on a Rails site I run. The information on each page is mostly constant, but the queries that need to be run to collect the information are complicated and can be slow in some cases.The only obstacle to using page caching is that the administrative interface is built into the main site so that admin operations can be performed without leaving the page of interest.
I'm using Apache+mod_rails (Passenger). Is there a way to indicate to Apache that .html files should be ignored when the current user either has a session variable or a cookie named 'admin'*? The session variable need not be evaluated by Apache for validity (since it will be evaluated by Rails in this case).
Is there a way to indicate to Apache that .html files should be ignored when the current user either has a session variable or a cookie named 'admin'*?
I believe it is not really possible. Even if it is, I guess should be very tricky.
Instead you can use Action Caching. A quote from docs:
One of the issues with page caching is
that you cannot use it for pages that
require checking code to determine
whether the user should be permitted
access.
This sounds like exactly your case.
But if you still really need Page Caching via web server, I think you better implement separate pages for admin and non-admin.
This is because of one reason. When you enable Page Caching rails doesn't process the request at all and thus there is no way to know if user is authenticated or not.
You probably can overcome this using Dynamic Page Caching. The idea is basically to add the "admin" part from the JavaScript. I don't personally like this a lot though.
One more update: quick search brought me to this article.
The idea is to cache page conditionally and plug mod_rewrite to serve admin pages.
Will work for you but is pretty dirty solution.