My app talk to webServer. At login, I pull down JSON and make up CoreData with 4 Entities (about 1000 rows each). The data changes on Server, So with every login, I have to update my existing CoreData.
What is the best approach to find out if records exist and insert new ones if need be?
To be smart on update (not blindly update every time), you need some intelligence on the server side.
One idea I would do.
Server has master table that records the timestamp of modified date of the 4 entities. It also has API to expose the master table. Every time change occurs to one of the 4 entities, master table's corresponding entry has to be updated as well.
You create the same copy of master table in application side as well.
On application launch, you query API in 1. and compare with the value in 2. to see if the timestamp has updated on the server side.
If YES, then download and replace the corresponding entity.
Another one which allows finer control.
Add timestamp column to the 4 entities on the server side. Every time entry is added/updated, the timestamp is updated.
Prepare an API for each entity that filters only newer items than the specified timestamp
On application launch, you query API in 2. and update.
The hole of the second approach is that it cannot handle deletion on the server side. Maybe you can combine something like the first approach to support this.
Related
I am new to Dynamics FnO, and recently followed the articles to access data through oData, and was successful.
What I see missing in the data objects that I normally receive in integrations out of the Microsoft World is the created/updated timestamps.
I am trying to put a synchronous data flow from FnO to my NodeJs application, so that my app keeps polling data from FnO whenever there is a change. This can be achieved easily if there were timestamps with the data that flows in.
Is there a way to setup those timestamps somewhere?
You have to make sure that the underlying table that you are querying has the fields added on it, and also that the data entity you are accessing through odata has the fields setup up on it as well.
Make sure this is setup on the table:
And then you have to drag and drop the field(s) from the datasource field list to the exposed field list in the data entity:
After this, you will have these fields
I am looking for solution of logging data changes for public API.
There is a need to tell client app which tables form db has changed and need to be synchronised since the app synchronised last time and also need to be for specific brand and country.
Current Solution:
Version table with class_names of models which is touched from every model on create, delete, touch and save action.
When we are touching version for specific model we also look at the reflected associations and touch them too.
Version model is scoped to brand and country
REST API is responding to a request that includes last_sync_at:timestamp, brand and country
Rails look at Version with given attributes and return class_names of models which were changed since lans_sync_at timestamp.
This solution works but the problem is performance and is also hard to maintenance.
UPDATE 1:
Maybe the simple question is.
What is the best practice how to find out and tell frontend apps when and what needs to be synchronized. In terms of whole concept.
Conditions:
Front end apps needs to download only their own content changes not whole dataset.
Does not invoked synchronization when application from different country or brand needs to be synchronized.
Thank you.
I think that the best solution would be to use redis (or some other key-value store) and save your information there. Writing to redis is much faster than any sql db. You can write some service class that would save the data like:
RegisterTableUpdate.set(table_name, country_id, brand_id, timestamp)
Such call would save given timestamp under key that could look like i.e. table-update-1-1-users, where first number is country id, second number is brand id, followed by table name (or you could use country and brand names if needed). If you would like to find out which tables have changed you would just need to find redis keys with query "table-update-1-1-*", iterate through them and check which are newer than timestamp sent through api.
It is worth to rmember that redis is not as reliable as sql databases. Its reliability depends on configuration so you might want to read redis guidelines and decide if you would like to go for it.
You can take advantage of the fact that ActiveModel automatically logs every time it updates a table row (the 'Updated at' column)
When checking what needs to be updated, select the objects you are interested in and compare their 'Updated at' with the timestamp from the client app
The advantage of this approach is that you don't need to keep an additional table that lists all the updates on models, which should speed things up for the API users and be easier to maintain.
The disadvantage is that you cannot see the changes in data over time, you only know that a change occurred and you can access the latest version. If you need to track changes in data over time efficiently, than I'm afraid you'll have to rework things from the top.
(read last part - this is what you are interested in)
I would recommend that you use the decorator design pattern for changing the client queries. So the client sends a query of what he wants and the server decides what to give him based on the client's last update.
so:
the client sends a query that includes the time it last synched
the server sees the query and takes into account the client's nature (device-country)
the server decorates (changes accordingly) the query to request from the DB only the relevant data, and if that is not possible:
after the data are returned from the database manager they are trimmed to be relevant to where they are going
returns to the client all the new stuff that the client cares about.
I assume that you have a time entered field on your DB entries.
In that case the "decoration" of the query (abstractly) would be just to add something like a "WHERE" clause in your query and state you want data entered after the last update.
Finally, if you want that to be done for many devices/locales/whatever implement a decorator for the query and the result of the query and serve them to your clients as they should be served. (Keep in mind that in contrast with a subclassing approach you will only have to implement one decorator for each device/locale/whatever - not for all combinations!
Hope this helped!
One of my application is using Parse.com as its backend service. There is one table called Product which is queried through cachePolicy as kPFCachePolicyCacheElseNetwork. The problem is the client side always got the cached data even after I modified some of the fields. The reason I don't always get data through network is that I'm trying to save data traffic as much as possible.
My question is if there is a way to expire the cache in server side so that I'll get new data in client side as soon as I modify the data? Thanks (My only solution by far is to delete the client app and reinstall it.This obviously is not an ideal solution.)
You need to decide some time limit on the cache validity, generally on the client, and either call clearCachedResult on the query instance or clearAllCachedResults on PFQuery when the limit is exceeded.
You could create a cloud function which returns a minimal amount of data and informs the app about changes so it can decide how / when / which caches to remove. For example, you pass a list of class names and last requested dates and the cloud function returns the names of classes which have new data since those dates.
I need some help with a CouchDB iOS project.
I'm using Apache CouchDB Server and the couchbase-lite iOS Framework.
On my CouchDB I have a template document.
- CouchDB Server
- database
- template
- document 1
- document 2
- ...
My goal is to only synchronise my iPad with this template document to get the latest data which my application needs.
But when I enter some data on my iPad, I want that this data should be pushed only to couchBase Server.
How can I "tell" my application to synchronise only one file and not the entire database with my server and at the end how can I "tell" my application to only push the data that is input from user side ?
More importantly, Do I need two databases on my server? One for the template and a second one for user input data?
If YES, then I just need to know how I can only push my data.
Guidance needed. Thanks.
This is how I solve this:
I tend to add a 'last update' date to all my documents, and store this in a format that means they'll be sorted in time order (epoch or yyymmddhhmmss) both do.
Create a view that uses the update time as a date.
On your client, store the time since you last updated.
When you update, access the view with a startkey parameter set to the last update date.
You can then either use 'include-docs=true' to get the documents as you query the view.
I tend to use 'include-docs=false' though as it means when a lot of documents have been updated I transfer less data in a single query. I then just directly access each document id that the view returns.
I have a database of many users,which i want to store on both locally and on my server.Whenever user updates any information,i successfully updates it in local database using core database. But how to change this information into the server?? I am not getting this please help?
I thinking of sqlite file to server every time user updates his information. but how to send data of sqlite file to server?
Add a column to your local DB that is the last time updated. (I think there may be a way to get SQLite to fill this in semi-automatically for you, but even doing it "manually" is no big deal.) Then, when you want to upload updates, query for rows updated since the last upload. Ship to the server as JSON records.
You can also keep a separate table that tracks updates, but that's for more complex scenarios.
You have to use some tactics for this. Here is a short explanation of this work.
Database Structure
Web service
You have to design database at local and server side and manage flag(Bool) and update time.
Like when you launch app then you have to check your local data and take last update date and send request to server what's new update after this date. If there is any update then you can send data as a result of that webservice and handle that response at local device.
When you do some changes at local device then you have to manage flag, update time and created date. flag will show it has update on server or not. If updated then Y otherwise N. And you have to send created and updated date with this.
During this request you have to manage in a same timezone. You can use standard UTC timezone because there may be chances that User can switch in different timezones so handle this.
If you need more clarification then you can ask at our chat room https://chat.stackoverflow.com/rooms/43424/coders-diary
This approach definitely work for you.