I am currently developing a proof-of-concept web service in Java using Google Cloud Endpoints and Objectify. Currently I want to implement/define partial responses to client queries to minimize GAE datastore ops.
Here are my researches and observations so far:
On the GAE datastore layer I know there exists the concept of projection queries, which happens during the entity fetch phase, explained here. (optimization of datastore ops possible)
On the Google Cloud Endpoints layer I know there exists the concept of field masking, which happens after the entity was fetched from the GAE datastore, explained here. (optimization of datastore ops not possible)
From the YouTube-API I know there is this concept of partial resources, which seems to come close to the thing I want to achieve. (optimization of datastore ops already implemented)
Now my questions:
1.) Is there a "simple way" to implement partial responses like it is done in the YouTube-API e.g. using certain configurations or annotations?
2.) If there is no "simple way" to implement partial responses would it be a "preferred way" to decompose entities and build relational entities with different property groups? Then these relational entities could be composed to a partial response entity which is returned to the client. As far as I know the downside of this approach would be that every response entity needs to be saved first before it can be returned to client.
3.) Are there any other preferred solutions to this problem?
Related
I'm quite new to the Odata topic and try to understand what is the best practice scenario when working with OData service.
Sceanrio 1:
I have an complex application with several EntitySets coming from an remote Odata model, which is loaded from SAP Backend. I can read data and bind it to UI controls, thats not the problem, but what I am confused about is how I can/should write back data to the backend.
First assumption Odata is One-Way Binding:
The user manipulates inputFields , dropdowns ,tables and so on, and all data is writen to the Odata Model with createEntry() or setProperty(). Right? Or should i use another JSONModel and collect all user changes ?
Question : How do i transfer now this changes made on the Odata model to backend ? What is the best practive I have read something about batchprocessing or having an own service and trigger this one with the create() function ? Can someone just give some hints or some kind of a recipe.
Sceanrio 2:
Odata in Two-Way Binding ?
How does that work ? Which prerequisite must the backend provide in the OdataServices ? I read something that it's experimental.
YOu see I'm quite a little bit confused.
It's important to know what you will be getting if you use one-way or two-way binding. None of these binding actually involve writing data back to the back-end OData service.
In short:
One-way binding means that the model (e.g. ODataModel) only keeps your UI controls in sync. Changes made to the model, will also be cascaded to the UI controls bound to the model. However, when you change values in your UI controls, the updated value will not automatically be written back to the model.
Two-way binding means that the model keeps your UI in sync (similar to one-way binding), but on top of that, changes in your UI controls will also cascade back to the model. Two-way binding
In the one-way model, you would indeed need to programmatically update the model using createEntry and setProperty methods. Using two-way binding, this will be done automatically for you.
If you want changes to your model to be written back to your OData service on the server, you could run the 'submitChanges' method. This method will look at all changes made in the ODataModel and will send corresponding OData requests to the server to synchronise the changes with the back-end.
To make sure this is done in a consistent fashion, the ODataModel will wrap the required changes into a so-called change-set. The back-end then knows which requests belong together and will be able to roll-back all changes in a change-set whenever one of the changes fails. In ABAP you would call this a logical unit of work (LUW).
Because it may be necessary to send multiple requests to the server (e.g. if the change set change multiple entities), the ODataModel (v2) groups as many requests as possible in one batch. When this is switched on (which is the default), only one request is sent to the server instead of multiple requests, which increases performance. It would be advisable to only switch batching off for debugging purposes.
Please note that two way binding in sap.ui.model.odata.ODataModel used to be experimental, but please don't use that class anymore as it's old. Use sap.ui.model.odata.v2.ODataModel instead, as it is way better and supports lots more OData features (such as batches and two-way binding).
That's actually multiple answers in one, but I hope it clarifies some of the confusion.
I have a business requirement where I need to expose set of custom properties defined by user and since this is user configuration, I cannot go away by creating classes. Therefore I need to opt for open types feature in Odata.
Q1. Is there any sample implementation out there on how I can persist the data to database and also support the querying capabilities on open types?
Q2. One issue I noticed is currently client library is not correctly handling open types and can only be achieved by partial classes that means user has to know the custom properties up front so that they can hand craft partial classes which is not what I want to do. Instead better approach would have been to support open types on client side by dynamic properties. Any pointers on how the client side experience can be optimized.
About query capabilities on open types, order and filter is supported in v5.5(will be released by the end of this month), query the value of dynamic properties, you can follow this pull request, part of this is in master branch now.
About persist the data to database, I think you can consider non-relational database, which can be a good choice for your open type data.
About Q2, achieve by dynamic properties is not implement in client library, maybe you can open an issue in github for us.
I have an EF model with about 200 tables, 75 of which I'd like to expose via REST in an MVC app. I started by adding a WCF Data Service (WCF-DS), pointed it to the EF context, and bam, I had the entire database mapped to REST URI's with full OData syntax support in about 2 minutes.
Next I tried to create the same REST URI space with WebAPI. When I tried to add a WebAPI OData Controller the first thing it asks for was a Model Class and when I was done creating the controller (and copying all the required ODataConventionModelBuilder code into the WebApiConfig) I only had one REST endpoint! My impression now is that WebAPI is not well suited to expose entire EF models with a lot of brute force.
So my questions:
Am I missing a way to map a bunch of WebAPI endpoints to a EF model in one fell swoop?
(Maybe T4 templates that build all the WebAPI code when I generate my EF model??)
Are there any compelling reasons to consider WebAPI vs WCF-DS to expose large URI domains?
(Some say that the benefits of WebAPI are to have fine grained control over each and every MVC/HTTP request but that seems counter-productive if the goal is to conform to the OData spec. I'm not sure I want to have 75 controllers and 1000's of lines of code that would tempt my dev colleagues to change one entity's behavior that would result in different behavior from other entities.)
(For cross cutting concerns such as security, caching, or performance throttling WCF-DS seems to have sufficient configure-ability with Interceptors and its DataServiceConfiguration class. Are there any features of WebAPI that would do better here?)
Thanks.
Update: I found this article by Julie Lermon that helps a bit: http://msdn.microsoft.com/en-us/magazine/dn201742.aspx
Since I have only exposed EF model using WCF DS, I can't comment on Web API questions. But we never really had a reason to replace WCF DS with Web API for our model because as you also noticed, EF and WCF DS play so nicely together that you basically get an OData feed for free. On a client side the situation is different: we started with WCF Data Services Client that is trying to mimic Linq to Entity Framework, but is has so many limitations that I ended up writing my own OData client (you can read about what made us unhappy with WCF DS client part here).
Coming back to server side: our domain was large, we had 80+ tables with almost 1000 columns. And we even supported all CRUD operations using batch updates (OData analog of transactions). While I would recommend to think twice before exposing database record update operations over OData protocol due to design principles, we haven't had any technical issues with that approach.
It is my opinion that Web API + OData extensions is highly overrated for a large majority or use-cases, and my argument is that OData is fundamentally data-oriented while Web API has come to become a great fit for general-purpose APIs, which include service-oriented APIs.
Your use-case is, I believe, a prime example of a very data-oriented layer since you don't seem to want to add much domain logic on this tier (server-side of this HTTP API). And WCF-DS works great for that, especially if you're merely wrapping an EF model which does 90% of the work for you (as you've already observed).
Of course it would be a different story if you were modeling more intricate processes at that layer (in that tier), so exploring both options like you did is always a very good idea. Normally the obvious choice should come naturally, either you'll be writing a lot of redundant code with Web API (go with WCF-DS) or you'll be fighting with WCF-DS's very rigid framework by playing with odd entities and not-very-RESTful OData actions (go with Web API alone).
Web API with its OData extensions stands somewhere in the middle, although it's not always clear what advantages it provides over custom WCF-DS providers. I guess it's nice for people who already know Web API or ASP.NET MVC, and may be a requirement if you want open source. I personally wouldn't debate on this technology with technical arguments, except for the few gotchas one should know about (but which have nothing to do with its design). I've ranted about all of this a while ago, should you want more, but I stress again that there are no hard truths in any of this -- discussing architecture and design is in no-trivial proportions a matter of opinions.
Update: WCF-DS was killed.
I'm very new with breezejs and having a few questions.
I think that breezejs has very nice features so I can replace my own datacontext. However, I don't want breezejs to interact directly with the dbcontext layer. In fact, in my application, the Service layer only exposes ViewModels - not even the real Business models - to the Controllers . So i'm not so sure whether I can use Breeze or not, since in few Breeze's examples, I only saw Breeze interact directly with DBContext.
Thanks.
=========================================
Thanks Ward for the answer,
About the features that I like from Breeze is that it will help to reduce a lot of time to build my own client-side view models. And to build a SPA, maintaining client-side view models is really painful for me, especially my application have desktop app client and other hand-held device's apps as well. Also, to handle the mapping from a JSon object to Knockout - which means with each view models, I will need a mapper as well.
Currently, my architecture is like this:
Server-side:
Repository layer <=> Service layer <=> Controllers (with the Web API that exposes to Client-side)
Controllers only can get the data (in format of a View Model) by sending request through Service.
So, my question is whether it is possible to make use of Breeze to query and also its integration with knockout.
Breeze never works directly with your DbContext; it works with the service model that you expose through endpoints on your service (e.g., Web API controller methods). But you certainly get the most value from Breeze when the client can query and save entities that are structurally the same as entities on the server.
You can retrieve ViewModels with Breeze - you can call almost any HTTP service method with Breeze. It's not clear to me how Breeze would help you manage those ViewModels on the client once you had retrieved them.
What features of Breeze seem "very nice" to you? Your answer to that question will help you determine if Breeze can be helpful with your preferred architectural style.
Querying data through Breeze without API controllers using DBContext directly should be no problem, saving might be harder but still manageable. I think the most complicated part is to get metadata to the client.
According to this SO answer, samples for exposing metadata from other sources that DBContext directly should be out in a week or so.
Meanwhile, check BreezeJS spa-template sample as there is repository pattern involved on the server side which makes it similar to your data access setup.
I wanted to know which would be best suitable - EDM or reflection provider for a project.
Definitely EDM is much simpler to develop over the database.But the problem is that,if we already have a data access layer over the database then we may have to change the existing architecture if chosen with EDM.So i wanted to know if there were any specific differences in using between entity data model or reflection provider to expose the data as Odata feed.
Reflection provider uses reflection to build model based on your class hierarchy. EDM provider uses directly the model created by the Entity Framework. If you have already a data access layer then I think you actually have three options:
- try using Reflection provider if it works great. Unfortunately I doubt it will work - it will change your objects but I don't think it will send queries to the database. In addition to be able to update data you will need to implement IUpdatable interface
- with EDM/EF provider you would probably need to move all you data access layer to EF. This means you would probably have to get rid of the access layer you have. The benefit is that once done it should work pretty much out of the box (queries updated etc)
- finally you can implement a custom provider. There is a few interfaces you would have to implement which would act as a bridge between the WCF Data Service and your access layer. Note that it is quite a lot of work. Here is the first post in the series that describes how to do it: http://blogs.msdn.com/b/alexj/archive/2010/01/07/data-service-providers-getting-started.aspx