In my Rails 4 app, I am using the metainspector gem to allow users to display meta data from a URL they post to a form.
Since I installed this gem, each time I try to visit the page of my own app where metadata is pulled from another website, load time increases significantly.
The load time increase goes from an imperceptible delay for small and local websites, to almost freezing the app for larger and foreign websites.
To give you an idea, a regular page usually loads under 400 ms: when we pull data with metainspector, it can go beyond a 30,000 ms (I measured these load times with rack-mini-profiler).
I did not find much about similar issues online.
Here is what I am trying to figure out:
Does this sound normal or did I setup something in the wrong way?
Is there a way to speed up load time with metainspector? For instance by caching responses?
If there is no way to speed up load time, should I implement a timeout limit and display an error message?
That is perfectly normal, to be exact it's not metainspector what is slowing your app down, it's the fact that you're requesting external URLs.
You should try to cache the responses using the built-in caching mechanism in metainspector, but also if possible move this to an asynchronous job using a background queue, and save or cache the result.
Related
We have a rails app that does some user-driven/filtered data representation over a large dataset. So we're calculating things on the fly and it takes longer than the 15s Unicorn gives us!
What's the best option here? I was thinking of using a pub/sub model (like a Node/Faye setup) to allow the rails app to send data that the browser could then render.
I guess another option is to try to pre-generate the data, but as we have a lot of clients and very few would be looking at the data it seems like we'd be wasting a lot of time on preparing data that would never be used.
You're on the right track with pre-generating the data.
If you're concerned about needless number crunching and want to do it on-demand, you could kick off a background job to process the data, and poll periodically to see if the background generation is done or not.
If you're looking for a library to do this for you:
Alternatively, if you're using ActionCable already, get_schwifty was built for this very purpose (shameless plug, I'm the author).
render_async is another option if you're not using ActionCable, however, I beleive it still does the processing in a Unicorn process instead of a background job.
I'm generating json of 65,000 users to populate a typeahead. The query is quick, turns out building the json was the bottleneck. I'm trying to cache the result but what happens when the cache expires, does it rebuild it automatically or is it going to wait until someone triggers the call, resulting in a 9-second page load once every 12-hours?
def user_json
Rails.cache.fetch("users", expires_in: 12.hours) do
User.all.to_json
end
end
If you did not want to hit the database each time then you could look in to a solution such as elastic search or sphinx which are designed to perform quick searching like your describing.
I was listening to javascript jabber this morning and they where saying that the average web page is now a shade under 2mb including images and CSS. Your request doubles that size. While that's fine for north Americans your page is likely to feel much slower in internet backwaters such as Australia.
It's also worth noting that older browsers such as IE don't handle iteration in javascript too well. I would suggest that your application would crash in any IE pre version 9.
Because of these reasons I would avoid pushing JSON that contains 65,000 rows over the wire and in to the browser. If the query is quick why not do a trip back to the server each time the user changes the input. Many trips back to the server based on input would be quicker than sending all 65,000 records and in the process removes the entire class of problems I have described above. Your original problem also goes away as you don't have to cache any responses any more.
When I upload an image (on Amazon S3 servers) to a Heroku app from a camera, where the photos have let's say more than 2.5MB, sometimes the app is not processed within 30 seconds and I see on the screen the warning message about Application Error.
This is not very user-friendly behavior, how to avoid this acting? I now I can buy an additional dyno, but I am not sure that this is a solution. For file upload I use Paperclip gem.
How do you solve this situation, when users uploads let's say images bigger than 3MB?
There's a couple things you could do: (in order from best to worst bet)
If you have a need to do a lot of post-procesing on images (like resizing them) you can have all of that processing done on a worker dyno using "Delayed Jobs." This way you get a response back much faster, but your alternate or resized versions of the image aren't immediately available, only :original. There's a tutorial on it here: http://madeofcode.com/posts/42-paperclip-s3-delayed-job-in-rails
You could use Unicorn or one of it's cousins. While it likely won't fix the image upload issue by itself, it allows you to adjust how long it takes for a request to timeout. As an added bonus it should also speed up the rest of your app.
You could try using Carrier Wave with CarrierWaveDirect instead of paperclip. This is kind of a shot in the dark as I've never personally used it, but it's supposedly 'better,' which could mean faster? maybe? It sounds like it works in a similar way as Paperclip with delayed jobs.
I am using Newrelic on my Ruby on Rails app? In the "Browser page load time" section, I can see a large portion of the loading time falls into the "DOM processing"(about 5 sec). I just want to know if this is normal? Should I be worrying about this and optimizing this more? There a lots of JS code in my app and many DOM are dynamically created, I think that is why its taking the most time to load. But the Firebug shows the load time is 6.18s (onload: 5.16s), seems to be pretty fast loading to me, and many js load a at the bottom of the page.
Thanks
If you want to improve client load times, then yes, you should care :) If not, don't worry about it.
https://newrelic.com/docs/features/how-does-real-user-monitoring-work
I'm working on an application that works like a search engine, and all the time it has workers in the background searching the web and adding results to the Results table.
While everything works perfectly, lately I started getting huge response times while trying to browse, edit or delete the results. My guess is that the Results table is being constantly locked by the workers who keep adding new data, which means web requests must wait until the table is freed.
However, I can't figure out a way to lower that load on the Results table and get faster respose times for my web requests. Has anyone had to deal with something like that?
The search bots are constantly reading and adding new stuff, it adds new results as it finds them. I was wondering if maybe by only adding the bulk of the results to the database after the search would help, or if it would make things worse since it would take longer.
Anyway, I'm at a loss here and would appreciate any help or ideas.
I'm using RoR 2.3.8 and hosting my app on Heroku with PostgreSQL
PostgreSQL doesn't lock tables for reads nor writes. Start logging your queries and try to find out what is going on. Guessing doesn't help, you have to dig into it.
To check the current activity:
SELECT * FROM pg_stat_activity;
Try the NOWAIT command. Since you're only adding new stuff with your background workers, I'd assume there would be no lock conflicts when browsing/editing/deleting.
You might want to put a cache in front of the database. On Heroku you can use memcached as a cache store very easily.
This'll take some load off your db reads. You could even have your search bots update the cache when they add new stuff so that you can use a very long expiration time and your frontend Rails app will very rarely (if ever) hit the database directly for simple reads.