I am developing an application which includes a WCF service and its ASP.NET MVC client. The ASP.NET MVC website must display a grid of objects - say, products. These products are stored in database which is accessible through the WCF service. So somewhere inside an MVC controller I call WCF service's method that returns me an array of products that I need to display.
So what is my question? I want to implement a pager functionality for my products grid, because it is possible that there will be a lot of products. So there are several ways to do that:
My controller can get the whole list of products and just do in-memory paging
WCF can select all the products and store them somewhere in its cache, then pass to the controller only part of them, according to the requested page number.
WCF can select only part of the products from the database, according to the requested page number.
WCF can return IQueryable to the controller, and then the controller will select whatever he wants whenever he wants.
As far as I understand (and correct me if it is not true), the first option is useless, so I must choose between the others.
The second option wastes my server's memory.
The third option is OK, but it seems a little bit ugly to implement paging on the WCF side.
And the fourth option sounds confusing. I actually pass some kind of query to the client, and then he queries my database by himself, through the WCF service. I can't figure out how to implement this correctly.
So can you please help me to choose the correct way to implement this?
What is your back-end database layer look like? If you're using LINQ (-to-SQL or -to-Entities), you could implement paging through WCF by specifying the page size and the page number you want, and then use LINQ's "Skip" and "Take" operators to fetch the page requested - something roughly like:
[ServiceContract]
public interface IFetchData
{
[OperationContract]
public List<Data> GetData(int pageSize, int pageNumber)
}
and then implement it something like this (simplified):
public class FetchDataService : IFetchData
{
public List<Data> GetData(int pageSize, int pageNumber)
{
var query = yourContext.DataTable
.Skip((pageNumber - 1) * pageSize)
.Take(pageSize);
return query.ToList();
}
}
Would that be helpful for you??
Marc
Related
I am building a mvc based website with ef6 code first type db. Right now i have a web api controller named Categories Controller that works nicely and shells out json data to my js requests.
// GET: api/Categories
public IQueryable<Category> GetCategories()
{
return db.Categories;
}
Now I need to wire up same for products, materials and some other entities. What I would like is something like :
//GET: api/Data/Categories
//GET: api/Data/Products
etc. all wired into one DataController. Is there a way to accomplish this?
For example one Data Controller, with separate region of code for all category specific api actions, product specific api actions and so on. Then I could do /api/Data/Categories/Create or api/Data/Products/Create
Instead of bringing in a new technology, you could handle this by creating a new class that contains both. I am assuming there is no relation between them, such as a Product has Categories, but rather you want both exclusively.
// GET: api/Data/ProductCategories
public IQueryable<ProductsCategory> GetProductsCategories()
{
return GetProductsCategories();
}
...
public class ProductsCategory
{
public IEnumerable<Category> Categories { get; set;}
public IEnumerable<Product> Products{ get; set;}
}
...
public ProductsCategory GetProductsCategories()
{
var products = db.Products.ToList();
var categories = db.Categories.ToList();
var productCategories = new ProductsCategory()
{
Products = products,
Categories = categories
};
return productCategories;
}
Or something to that degree.
Also don't return IQueryable directly, it's redundant and ill advised unless the caller of that API is going to somehow be executing some Query against what has been returned which is unlikey seeing as its WebAPI and not some method. Instead return a List or IEnumerable.
And if you are looking to improve what you have a little bit as well, be sure to wrap that repository that you have in some sort of service, so you can say something along the lines of:
productsService.GetProductsCategories()
Rather than accessing the context directly in your API.
Extended Reading for Repositories and Service Layers
https://codereview.stackexchange.com/questions/33109/repository-service-design-pattern
Difference between Repository and Service Layer?
The easiest way to do what you want is to implement an OData controller, or a Breeze controller. They will do all the heavylifting to expose your EF model to in Web APi endpoints:
OData
Breeze
They're wuite easy to setup and OData is a recognized standard for this kind of task. They both had prebuilt support for oldering, filtering, paging, including related conllections and so. There are Nuget packages to use both of them.
Breeze also has feature-rich clients for JS and C#, and a lot of extra functionality.
I am supposed to create an internal statistics mechanism for our ASP.NET MVC 4 web application. We are not going to use external ones like Google Analytics or even Glimpse. Because I'm not sure if I can extract needed data from their API.
What we expect this mechanism is very like to Google Analytics including page hit count, referer, keyword, etc. But just for part of pages not all. We want to use these data in our own pages and reports.
Now I have 2 questions. Is it correct to ignore Google Analytics or Glimpse and implement my own? If yes, it is reasonable to save a record in database per each website visit and then use theses record to extract statistics?
Any help is highly appreciated
I think you can implement both this satistic. Its difficult to say without understanding business logic you need. But if you need more detailed information about every request (visited user roles, retrive controller/action name for some specific statistic, log access to specific resources etc.) you can easily implement this by using action filter.
public class StatisticFilter : ActionFilterAttribute
{
public override void OnActionExecuted(ActionExecutedContext filterContext)
{
base.OnActionExecuted(filterContext);
if (filterContext.IsChildAction) //if action call was from view like #Html.Action do nothing
return;
var CurrentUser = filterContext.RequestContext.HttpContext.User;
if (CurrentUser.IsInRole("some_role"))
return; //write logic for this role
string controllerName = filterContext.ActionDescriptor.ControllerDescriptor.ControllerName;
string actionNaem = filterContext.ActionDescriptor.ActionName;
//here the id of the accessed resource - document, sale, page etc.
string id = filterContext.RequestContext.RouteData.Values["id"].ToString();
}
}
Thats all. You can extend this by any logic you need.
In my project i have the statistic table with filds:
Date - timestamp,
Controller - string,
Action - string,
id - bigint
method - string(POST, GET... if post - submited)
user_id - bigint
And insert record for every request executed. So i have most important information about request for any statistic.
In the examples for ServiceStack I don't see a single application that is ASP.NET MVC website first and then made ServiceStack service second.
Let's take a very simple ASP.NET MVC web application that renders products through Views. It uses controllers, views, models and viewmodels.
Let's say we have a model of Product which gets persisted into a document DB. Let's assume we have a viewmodel of ProductViewModel which gets mapped from Product and display within MVC Razor View/PartialView.
so this is a web side of things..now let's assume we want to add a service returning products to various clients like the Windows 8 applications.
Should the request/response classes be completely disconnected from what we already have? Our ProductViewModel might already contain everything we want to return from the service.
Since we already have Product (model class) we can't have another Product class in the API namespace..well we could but that makes things unclear and I'd like to avoid that.
So, should we introduce standalone ProductRequest class and ProductRequestResponse (inherits ProductViewModel) class in the API namespace?
Like so ProductRequestResponse : ProductViewModel?
What i'm saying is, we already have the Model and ViewModel classes and to construct Request and Response classes for the SS service we would have to create another two files, mostly by copying everything from the classes we already have. This doesn't look DRY to me, it might follow the separation of concerns guidelines but DRY is important too, actually more than separating everything (separating everything leads to duplication of code).
What I would like to see is a case where a web application has already been made, it currently features Models and ViewModels and returns the appropriate Views for display on the Web but can be extended into a fully functional service to support programmatic clients? Like AJAX clients etc...with what we already have.
Another thing:
If you take a look at this example https://github.com/ServiceStack/ServiceStack.Examples/blob/master/src/ServiceStack.MovieRest/MovieService.cs
you will see there is Movie Request class and Movies Request class (one for single movie request, the other one for a list of movies). As such, there are also two services, MovieService and MoviesService, one dealing with requests for a single movie, the other one for a genre of movies.
Now, while I like SS approach to services and I think it is the right one, I don't like this sort of separation merely because of the type of request. What if I wanted movies by director? Would I be inventing yet another request class having a Director property and yet another service (MoviesByDirector) for it?
I think the samples should be oriented towards one service. Everything that has to deal with movies need to be under one roof. How does one achieve that with ServiceStack?
public class ProductsService : Service
{
private readonly IDocumentSession _session;
private readonly ProductsHelperService _productsHelperService;
private readonly ProductCategorizationHelperService _productCategorization;
public class ProductRequest : IReturn<ProductRequestResponse>
{
public int Id { get; set; }
}
// Does this make sense?
// Please note, we use ProductViewModel in our Views and it holds everything we'd want in service response also
public class ProductRequestResponse : ProductViewModel
{
}
public ProductRequestResponse GetProducts(ProductRequest request)
{
ProductRequestResponse response = null;
if (request.Id >= 0)
{
var product = _session.Load<Product>(request.Id);
response.InjectFrom(product);
}
return response;
}
}
The Service Layer is your most important Contract
The most important interface that you can ever create in your entire system is your external facing service contract, this is what consumers of your service or application will bind to, i.e. the existing call-sites that often won't get updated along with your code-base - every other model is secondary.
DTOs are Best practices for remote services
In following of Martin Fowler's recommendation for using DTOs (Data Transfer Objects) for remote services (MSDN), ServiceStack encourages the use of clean, untainted POCOs to define a well-defined contract with that should kept in a largely implementation and dependency-free .dll. The benefits of this allows you to be able to re-use typed DTOs used to define your services with, as-is, in your C#/.NET clients - providing an end-to-end typed API without the use of any code-gen or other artificial machinery.
DRY vs Intent
Keeping things DRY should not be confused with clearly stating of intent, which you should avoid trying to DRY or hide behind inheritance, magic properties or any other mechanism. Having clean, well-defined DTOs provides a single source of reference that anyone can look at to see what each service accepts and returns, it allows your client and server developers to start their work straight away and bind to the external service models without the implementation having been written.
Keeping the DTOs separated also gives you the freedom to re-factor the implementation from within without breaking external clients, i.e. your service starts to cache responses or leverages a NoSQL solution to populate your responses with.
It's also provides the authoritative source (that's not leaked or coupled inside your app logic) that's used to create the auto-generated metadata pages, example responses, Swagger support, XSDs, WSDLs, etc.
Using ServiceStack's Built-in auto-mapping
Whilst we encourage keeping separate DTO models, you don't need to maintain your own manual mapping as you can use a mapper like AutoMapper or using ServiceStack's built-in Auto Mapping support, e.g:
Create a new DTO instance, populated with matching properties on viewModel:
var dto = viewModel.ConvertTo<MyDto>();
Initialize DTO and populate it with matching properties on a view model:
var dto = new MyDto { A = 1, B = 2 }.PopulateWith(viewModel);
Initialize DTO and populate it with non-default matching properties on a view model:
var dto = new MyDto { A = 1, B = 2 }.PopulateWithNonDefaultValues(viewModel);
Initialize DTO and populate it with matching properties that are annotated with the Attr Attribute on a view model:
var dto = new MyDto { A=1 }.PopulateFromPropertiesWithAttribute<Attr>(viewModel);
When mapping logic becomes more complicated we like to use extension methods to keep code DRY and maintain the mapping in one place that's easily consumable from within your application, e.g:
public static class MappingExtensions
{
public static MyDto ToDto(this MyViewModel viewModel)
{
var dto = viewModel.ConvertTo<MyDto>();
dto.Items = viewModel.Items.ConvertAll(x => x.ToDto());
dto.CalculatedProperty = Calculate(viewModel.Seed);
return dto;
}
}
Which is now easily consumable with just:
var dto = viewModel.ToDto();
If you are not tied specifically to ServiceStack and just want "fully functional service to support programmatic clients ... with what we already have", you could try the following: Have your controllers return either a ViewResult or a JsonResult based on the request's accept header - Request.AcceptTypes.Contains("text/html") or Request.AcceptTypes.Contains("application/json").
Both ViewResult and JsonResult are ActionResult, so the signature of actions remains the same, and both View() and Json() accept a ViewModel. Furthermore, if you have a ControllerBase you can make a base method (for example protected ActionResult RespondWith(Object viewModel)) which calls either View() or Json() so the change to existing code is minimal.
Of course, if your ViewModels are not pure (i.e. have some html-specific stuff or you rely on some ViewBag magic) then it's a little more work. And you won't get SOAP or other binding types provided by ServiceStack, but if your goal is to support a JSON data interface with minimal code changes to the existing MVC app then this could be a solution.
Lp
Say i want to add pagination support.
My app is separated in Web, Services and repositories.
The Controller only speaks to the Service and the Service gets Data and does business logic.
Say i want pagination support.. I have this method in my ContributorService
public IQueryable<Contributor> GetContributors(int page, int pageSize)
{
return _repository.GetAll().OrderBy(c=>c.ACC_CREATEDATE).Skip((page)*pageSize).Take(pageSize);//solo temporalmente
}
is that ok?? or should the OrderBy, Skip, Take be done in the repository?
which currently only does this
public IQueryable<Contributor> GetAll()
{
return db.Contributors;
}
I would add the query to the business object ( I think you dont have one, do you), there you may have a base version of it, and another one for the paged data. And would expect to service to execute that query by calling the ToList, I find it dangerous to return a query object to the controller.
Your repository might have a GetPartial Method, with the sort, from, to, and a filter params. If you have a generic service, you might also implement this in that generic service.
public List<Contributor> GetPartial<TSortBy>(Expression<Func<Contributor, TSortBy>> sortByExpr, Expression<Func<Contributor, bool>> filterExpr, int pageNo, int pageSize)
{
var query = db.Contributors;
if (filterExpr != null)
query.Where(filterExpr);
query.orderBy(sortByExpr).Skip (...).Take(...).ToList();
}
if you have a Repository class, you can add this method.
By the way, I am using Dynamic LINQ, which makes it easier to pass order by expression as plain text (just like sql)
I was reading Steven Sanderson's book Pro ASP.NET MVC Framework and he suggests using a repository pattern:
public interface IProductsRepository
{
IQueryable<Product> Products { get; }
void SaveProduct(Product product);
}
He accesses the products repository directly from his Controllers, but since I will have both a web page and web service, I wanted to have add a "Service Layer" that would be called by the Controllers and the web services:
public class ProductService
{
private IProductsRepository productsRepsitory;
public ProductService(IProductsRepository productsRepository)
{
this.productsRepsitory = productsRepository;
}
public Product GetProductById(int id)
{
return (from p in productsRepsitory.Products
where p.ProductID == id
select p).First();
}
// more methods
}
This seems all fine, but my problem is that I can't use his SaveProduct(Product product) because:
1) I want to only allow certain fields to be changed in the Product table
2) I want to keep an audit log of each change made to each field of the Product table, so I would have to have methods for each field that I allow to be updated.
My initial plan was to have a method in ProductService like this:
public void ChangeProductName(Product product, string newProductName);
Which then calls IProductsRepository.SaveProduct(Product)
But there are a few problems I see with this:
1) Isn't it not very "OO" to pass in the Product object like this? However, I can't see how this code could go in the Product class since it should just be a dumb data object. I could see adding validation to a partial class, but not this.
2) How do I ensure that no one changed any other fields other than Product before I persist the change?
I'm basically torn because I can't put the auditing/update code in Product and the ProductService class' update methods just seem unnatural (However, GetProductById seems perfectly natural to me).
I think I'd still have these problems even if I didn't have the auditing requirement. Either way I want to limit what fields can be changed in one class rather than duplicating the logic in both the web site and the web services.
Is my design pattern just bad in the first place or can I somehow make this work in a clean way?
Any insight would be greatly appreciated.
I split the repository into two interfaces, one for reading and one for writing.
The reading implements IDisposeable, and reuses the same data-context for its lifetime. It returns the entity objects produced by linq to SQL. For example, it might look like:
interface Reader : IDisposeable
{
IQueryable<Product> Products;
IQueryable<Order> Orders;
IQueryable<Customer> Customers;
}
The iQueryable is important so I get the delayed evaluation goodness of linq2sql. This is easy to implement with a DataContext, and easy enough to fake. Note that when I use this interface I never use the autogenerated fields for related rows (ie, no fair using order.Products directly, calls must join on the appropriate ID columns). This is a limitation I don't mind living with considering how much easier it makes faking read repository for unit tests.
The writing one uses a separate datacontext per write operation, so it does not implement IDisposeable. It does NOT take entity objects as input or out- it takes the specific fields needed for each write operation.
When I write test code, I can substitute the readable interface with a fake implementation that uses a bunch of List<>s which I populate manually. I use mocks for the write interface. This has worked like a charm so far.
Don't get in a habit of passing the entity objects around, they're bound to the datacontext's lifetime and it leads to unfortunate coupling between your repository and its clients.
To address your need for the auditing/logging of changes, just today I put the finishing touches on a system I'll suggest for your consideration. The idea is to serialize (easily done if you are using LTS entity objects and through the magic of the DataContractSerializer) the "before" and "after" state of your object, then save these to a logging table.
My logging table has columns for the date, username, a foreign key to the affected entity, and title/quick summary of the action, such as "Product was updated". There is also a single column for storing the change itself, which is a general-purpose field for storing a mini-XML representation of the "before and after" state. For example, here's what I'm logging:
<ProductUpdated>
<Deleted><Product ... /></Deleted>
<Inserted><Product ... /></Inserted>
</ProductUpdated>
Here is the general purpose "serializer" I used:
public string SerializeObject(object obj)
{
// See http://msdn.microsoft.com/en-us/library/bb546184.aspx :
Type t = obj.GetType();
DataContractSerializer dcs = new DataContractSerializer(t);
StringBuilder sb = new StringBuilder();
XmlWriterSettings settings = new XmlWriterSettings();
settings.OmitXmlDeclaration = true;
XmlWriter writer = XmlWriter.Create(sb, settings);
dcs.WriteObject(writer, obj);
writer.Close();
string xml = sb.ToString();
return xml;
}
Then, when updating (can also be used for logging inserts/deletes), grab the state before you do your model-binding, then again afterwards. Shove into an XML wrapper and log it! (or I suppose you could use two columns in your logging table for these, although my XML approach allows me to attach any other information that might be helpful).
Furthermore, if you want to only allow certain fields to be updated, you'll be able to do this with either a "whitelist/blacklist" in your controller's action method, or you could create a "ViewModel" to hand in to your controller, which could have the restrictions placed upon it that you desire. You could also look into the many partial methods and hooks that your LTS entity classes should have on them, which would allow you to detect changes to fields that you don't want.
Good luck! -Mike
Update:
For kicks, here is how I deserialize an entity (as I mentioned in my comment), for viewing its state at some later point in history: (After I've extracted it from the log entry's wrapper)
public Account DeserializeAccount(string xmlString)
{
MemoryStream s = new MemoryStream(Encoding.Unicode.GetBytes(xmlString));
DataContractSerializer dcs = new DataContractSerializer(typeof(Product));
Product product = (Product)dcs.ReadObject(s);
return product;
}
I would also recommend reading Chapter 13, "LINQ in every layer" in the book "LINQ in Action". It pretty much addresses exactly what I've been struggling with -- how to work LINQ into a 3-tier design. I'm leaning towards not using LINQ at all now after reading that chapter.