I am currently working on an MVC4 application that accesses a set of wcf services which delivers content.
I have a page that lists products. This page has a pagination feature called infinite scrolling, so as you scroll down the page, products are loaded.
I am wondering what is the best way to achieve such a pagination feature. The data source is sql server. Options as I see it are:
Paginate at the sql server 2012 layer - returning only the required recordset and feed it back up the stack through wcf and in to the MVC application to display
As option 1 but also include caching at the WCF layer so that the recordset is cached long term. This will mean though that a number recordsets will be stored in cache instead of one large one
Cache all the data, and paginate the cached items, returning the subset from the WCF services cached data
Note: I am using asp.net for caching.
So I am looking for feedback as to the best practice for this.
How big is your possible dataset? That I think would be the concern with caching, if it's feasible to hold it all in memory, then do option 3, I don't see the point of option 2, as if you cache for long term, you will most likely get to caching everything anyway. If you want to implement #2, I would cache for a short period of time (the timeframe would depend on how busy the site is).
When i tried this for test purposes i followed this guide: http://www.gavindraper.co.uk/2012/05/10/infinite-scroll-with-asp-net-mvc-4/
I dont know if it's the best way of implementing infinite scroll but it's at least a proof of concept.
Related
Looking at this question:
SO question
The accepted answer by Darin Dimitrov looks appealing (.NET 4.5 version). I am just wondering how this compares performance wise with client side solutions (e.g. using knockout/angular/jquery) to assemble the HTML given some JSON from the web api endpoint. Did someone ever do some perfromance tests on this. What are the pros and cons of the 'client side solution' vs the 'razor server side' solution?
You should have to define performance.
However there is a very big difference between the two options:
if you do it client-side (with ko/ng/jQuery) the server only executes the API controller action and returns the formatted data.
if you do it server side, apart from execution the API action, the server has to execute the MVC controller action, so, undoubtedly, the server does more work.
The only conclusion is that the server has less work to do in the first case. And, usually, the network traffic is reduced (a JSON object is usually lighter than a rendered partial view).
If we're speaking about the user experience, in general client side technologies (jQuery, ko, ng) offer a much better user experience becasue the page is much more responsive: it's easy to show/hide elements, set the focus, make trivial calculations, remote validations... And if we use modern libraries, we can go further on improving the interface resposiveness. For example breeze.js allows to cache data in the client side to avoid making extra ajax calls to the server, giving a much more responsive experience, specially if you anticipate what data can be needed and cached it before hand. You could even persist data in HTML5 storage, so that it's available for other sessions.
Then, from the user viewpoint, I think it's much better the second option. And the server has less work to do, which can make it also more resposive in high-traffic sites.
Even so, I don't know what is "more performant" or even what it is "to be performant".
Whatever it is, using client side technologies is a much better option. But it takes some time to master the associated technologies.
At the company that I work we have some problems with performance loading the data of database.
Our ERP works with a big database and we need to make some complex queries.
We are using C# following the DDD pattern and in the front-end we are using ASP.NET MVC.
The example that I'm talking is that:
We have alot of lists (grids) in the system where the data is loaded by ajax. But everytime that a user enter in some page like that (an html page), we need to make a query in the database.
Its important to say that this data aren't only for "query". This data is constantly changed by the users.
My doubt is if you have any sugestion to how we can minimize this problem. I already have searching some solutions of cache in the server side, but I don't have any experience working with cache in high complex scenarios.
Very thanks,
Renan Cunha.
I have an existing web-forms website that runs on a web-farm (multiple web servers, each request is NON-sticky).
One of the tasks is to retrieve a lot of data from a 3rd party webservice. This is an expensive process (in terms of time taken to respond). The optimal solution has been to initially grab all the data and stick it in the page's ViewState (as a List<Product>. We then have a grid that allows us to page through this list of products. For each request for the next page we don't have to re-visit the slow web service because we've already cached the data in the ViewState.
So, how would I accomplish this using MVC? If I were using classic ASP, I would serialize the list and hold it in a hidden field in the form.
But what is the preferred approach when using MVC? As mentioned, I'm using non-sticky sessions so can't rely upon caching on the server.
If I am to hold it in a hidden-field, then is it sensible to compress the data first (zip) to reduce the size of the page? Again, what's "best practice" here?
Many thanks for any/all advice
Griff
PS - I know there are similar posts out there (e.g. ASP.NET MVC and ViewState), but they don't quite provide the detail I require.
Caching, in my humble opinion, is the best way to deal with this; hit the webservice, cache the data, and use that for each subsequent request.
It is possible to share a cache across many web servers. The default behaviour is to hold it InProcess, but this is by no means fixed and can be configured to store it in a remote InProc cache, a database, or any other method of semi-persistant storage. AzureAppFabric caching comes to mind.
Second to that, as you mentioned is to dump all the data in a hidden field, but I dont like this idea for a number of reasons
Page bloat - you're submitting this data every time the page is changed
Lost data - you must submit a form for every navigation, forgetting to do this means loosing your data
To name 2.
Your strategy should depend on how volatile the data retrieved from the 3rd party service is going to be and how expensive it is to call it.
Volatile and expensive. I would go down the road of caching the data in a distributed cache such as Appfabric Velocity, or Memcached.
Non-Volatile and expensive. Cache a copy of it in memory on each server node.
Cheap. Hit the call every time you need it. Don't bother caching.
If the data set returned from the service is large, I would not pass this up and down every time you page through your grid data. Only retrieve the data for the current page and render it in your grid. Telerik have a good implementation for this or you could try and roll your own.
We are happy users of the ASP.NET MVC framework and SQL Server, currently using LINQ-to-SQL. It serves our needs well with a consumer-facing application with about 1.4 million users and 2+ million active uniques per month.
We are long overdue to start logging all user actions (views of articles, searches on our site, etc.) and we're trying to scope out the right architecture to do so.
We'd like the archiving system to be its own entity, and not part of the main SQL cluster that stores the production articles and search engine. We'd like it to be its own SQL cluster, starting out with just one box initially.
To simplify the problem, let's say we just want to log the search terms that these millions of users enter into our site for the month, and we want to do so in the least cycle-intensive-way possible.
My questions:
(1) Is there an asynchronous way to dump the search terms to a remote box? Does LINQ support async for this?
(2) Would you recommend building up a cache of say 1,000 (userId, searchTerm, date) logging items in a RAM cache, and then flushing those at intervals to the database? I assume this method would cut down on open/close connections.
Or am I thinking about this entirely wrong? We'd like to strike a balance between ease of implementation and robustness.
1)Sure you can, there are different solution to achieve it. Linq is not the instrument you need.
2)There should not be any major improvement by doing it, the "logging" will be triggered only when a search is performed. You will end up with two calls instead of one, not a big deal.
A suggestion is to use AOP
You can create a clean and separate layer for logging using Postsharp (there are other alternatives though). You will then decorate your actions with the required logging attribute only when you need to trace what is passed to the action.
Main advantages with this approch are :
Logging logic doesn't reside inside your code (you don't need to change your methods code) but is executed before/after your method.
Clean separation of the Aspect from the target method.
You can easily switch on/off the aspects
AOP is a common practice specially when it comes to behavior that can be added to more than one method, like logging, authentication and so on. And yes it can be used in an async way.
1)I would suggest you to create an HttpModule that "catch" all the search terms used by the users. How and where you will dump those information(you said you will use a SQL box) it's another matter which is outside the scope of the module which should just catch the Search tems.
Then you can create a component that contains the login to store those information using Async call(or even a third part component like Log4Net )
2) if you want create a kind of batch insert caching all the information you need to store and at some point dump them on SQL I would use MSMQ or any other technology that support the Reliability: I think you want loose all those information in the case of a system-crash,etc
I have a heavy page that is cached. This is okay for anonymous users. They all see the same page.
The problem is for logged in users. They should have minor parts of the page re-rendered on every request (like personal notes on content in the page, etc.)
But still all the rest of the page should be cached (it does tons of SQL and calcuations when rendered).
As a workaround I put placeholders in page templates (like #var1#, #var2#,..).
Then I make controller method to render View into string, where I do string.Replace #var1# and other into real values.
Any cleaner way to do such kind of partial "non-caching"?
This is called donut caching.
The ASP.Net MVC framework doesn't currently support it, but it's planned for version 3.
To start things off, it might make sense to go through the page and see if there is anything about it that you can do to streamline or reduce the weight. Depending upon how bad things are, investing some time here might pay off in the long run.
That said, in regards to trying to server the content to anonymous as well as logged in users, one option is to have two versions of the page: one for anonymous users and one for logged in users. This may not be the best approach though as it means that you now have two versions of the same page to maintain.
Given the lack of support doughnut caching mentioned by SLaks what I would likely do is try and cache the results of the calculations that are being done for the page (e.g. if you are querying a database for a table of data, cache a DataTable that you can check for before running the operation) and seeing what that does for the performance. It may not be the most elegant solution in the world, but it may solve the problems that you are having.