I'd like some advice on how to implement the following in Objective C. The actual application in related to an AB testing framework I'm working on, but that context shouldn't matter too much.
I have an IOS application, and when a certain event happens I'd like to send a log message to a HTTP Service endpoint.
I don't want to send the message every time the event happens. Instead I'd prefer to aggregate them, and when it gets to some (configurable) number, I'd like to send them off async.
I'm thinking to wrap up a static NSMutableArray in a class with an add method. That method can check to see if we have reached the configurable max number, if we have, aggregate and send async.
Does objective-c offer any better constructs to store this data? Perhaps one that helps handle concurrency issues? Maybe some kind of message queue?
I've also seen some solutions with dispatching that I'm still trying to get my head around (I'm new).
If the log messages are important, keeping them in memory (array) might not suffice. If the app quits or crashes the NSArray will not persist on subsequent execution.
Instead, you should save them to a database with an 'sync' flag. You can trigger the sync module on every insert to check if the entries with sync flag set to false has reached a threshold and trigger the upload and set sync flag to true for all uploaded records of simply delete the synced records. This also helps you separate your logging module and syncing module and both of them work independently.
You will get a lot of help for syncing SQLite db or CoreData online. Check these links or simply google iOS database sync. If your requirements are not very complex, and you don't mind using third party or open source code, it is always better to go for a readily available solution instead of reinventing the wheel.
Related
I am currently implementing a web application in .net core(C#) using entity framework. While working on the project, I actually encountered quite a few challenges but I will start with the one which I think are most important. My questions are as follows:
Instead of frequent loading data from the database, I am having a set of static objects which is a mirror of the data in the database. However, it is tedious and error prone when I want to ensure any changes, i.e., adding/deleting/modifying of objects are being saved to the database at real time. Is there any good example or advice that I can refer to improve my approach to do this?
Another thing is that value of some objects' properties will be changed on the fly according to the value of some other objects' properties. Something like a spreadsheet where a cell's value will be changed automatically if the value in the cell that the formula is referring to changes. I do not have a solution to do this yet. Appreciate if anyone has any example that I can refer to. But this will add another layer of complexity to sync the changes of the objects in memory to database.
At the moment, I am unsure if there is any better approach. Appreciate if anyone can help. Thanks!
Basically, you're facing a problem that's called eventual consistency. Something changes and two or more systems need to be aware at the same time. The problem here is that both changes need to be applied in order to consider the operation successful. If either one fails, you need to know.
In your case, I would use the Azure Service Bus. You can create queues and put messages on a queue. An Azure Function would handle these queue messages. You would create two queues, one for database updates, and one for the in-memory update (I think changing this to a cache service may be something to think off). Now the advantage of these queues is that you can easily drop messages on these queues from anywhere. Because you mentioned the object is going to evolve, you may need to update these objects either in the database or in memory (cache).
Once you've done that, I'd create a topic, with two subscriptions. One forwarding messages to Queue 1, and the other to Queue 2. This will solve your primary problem. In case an object changes, just send it to the topic. Both changes (database and memory) will be executed automagically.
The only problem you have now, it that you mentioned you wanted to update the database in real-time. With this scenario, you're going to have to leave that.
Also, you need to make sure you have proper alerts in place for the queues so in case you did miss a message, or your functions didn't handle it well enough, you'll receive an alert to check & correct errors.
I'm totally agree with #nineedm's and answer, but there are also other solutions.
If you introduce cache, you will always face cache revalidation problem - you have to mark cache as invalid when data were changed. Sometimes it is easy, depending on nature of cached data and how often data are changed.
If you have just single application, MemoryCache can be enough with proper specified expiration options.
If there is a cluster - you have to look at Distributed Cache solutions, for example Redis. There is MS article about that Distributed caching in ASP.NET Core
I have an Firebase-backed app with the following database structure:
posts
/uid
/postId
Originally, i'd load data from the posts/uid node using ObserveEventOfType with .childAdded. This would load stale data frequently (~5 times a day) for all users of my app simutaneously. When attempting to update the data by making a new post, Firebase would still return stale data.
As a result, I decided to try keepSynced. Now, if my reference looked like this:
reference = Database().database.reference.child("posts").child(uid)
keepSynced would load all of the data at that node, which could amount to very large downloads if there are many children in that node. So, I decided to change the reference/query to:
reference = Database().database.reference.child("posts").child(uid).queryLimited(toLast: 25)
When turning keepSynced on for this node, it syncs for the last 25 children in the node successfully. However, I still am facing the issue of receiving stale data rather frequently. So here are my questions:
When adding the keepSynced mode on the limited query, does it only sync from the initial node you added it to, or does it always just sync the 25 latest children under that node?
Where is the best place to add the keepSynced(true) line in code? Before we load the reference, in viewWillAppear, or inside of the actual download callback?
Similarly, where is the best place to use keepSynced(false)?
Do the keepSynced listeners delete when the app fades into the background?
Why does keepSynced sometimes not address for child updates?
I currently use keepSynced(true) inside of the function I use to load posts which is called on viewDidLoad.
Thanks in advance.
As its name implies keepSynced(true) keeps whatever query or reference you call it on synchronized in the local cache. It quite literally just attaches an empty observer to that query/reference. So in your Database().database.reference.child("posts").child(uid).queryLimited(toLast: 25) it will sync the last 25 child nodes, and keep synchronizing those (removing previous ones as new ones are added).
Firebase Realtime Database caching mechanism works most reliably if you repeatedly listen for the exact same data. Specifically, a .value listener attached to Database().database.reference.child("posts").child(uid) may not see data that was cached through Database().database.reference.child("posts").child(uid).queryLimited(toLast: 25). This is because the Firebase client guarantees to never fire events for partial updates, and in this example it can't guarantee that it has all data from the first reference.
For your questions:
See above...
It's most common to add them in viewWillAppear.
I'm not sure why you'd want to call keepSynced false, so can't recommend anything there.
Not sure if this is what you mean, but keepSynced(true) is not persisted between runs of the app. So you have to call keepSynced(true) each time your app/view starts.
See above...
In general you seem to try and work around the way the API works, by calling the API in different ways. I typically don't see great results from that. If you want your app to behave differently than the API does, consider creating a custom wrapper, and caching the data there.
This question already has answers here:
Firebase Offline Capabilities and addListenerForSingleValueEvent
(4 answers)
Closed 4 years ago.
[Disclaimer] I have personally posted and answered this question after having struggled with it myself and, relevantly, noticed that many people still do
Context
I am developing an iOS mobile application and - for this particular project - decided to use the Firebase Realtime Database as my backend infrastructure.
Problem
When querying data at a specific node using the .observeSingleEvent(of:with:) method, I always find myself retrieving either cached or old data rather than the newly updated one.
In some cases, calling the method twice in a row retrieves the desired server data.
Attempts
Used .keepSynced(true) at the relevant node which, according to the Firebase documentation
automatically downloads the data at these locations and keeps it in sync even if the reference has no active listeners
Overview
Going through the documentation, you notice that there are two primary ways of querying data from the Firebase Realtime Database into your iOS mobile application
The .observe(_:with:) method which, according to the Firebase Documentation, continuously listens for changes at a particular node and triggers the callback every time the data changes at the latter.
This method is triggered once when the listener is attached and again every time the data, including any children, changes. The event callback is passed a snapshot containing all data at that location, including child data. If there is no data, the snapshot will return false when you call exists() and nil when you read its value property.
The .observeSingleEvent(of:with:) method which, according to the Firebase Documentation, is called exactly once.
In some cases you may want a callback to be called once and then immediately removed, such as when initializing a UI element that you don't expect to change. You can use the observeSingleEventOfType method to simplify this scenario, [in which] the event callback [is triggered] once and then does not trigger again.
Problem
After going through the different possible methods of querying your data, you've realized that the .observeSingleEvent(of:with:) method suits better your current database-reading needs. However, implementing it in your application keeps on retrieving cached and old data no matter how many times you modify your database. You've called the .keepSynced(true) at the relevant database reference, yet in vain. You've opted for the .observe(_:with:) method instead, and everything starts to work perfectly fine.
So what might be the issue?
Solution
The reason you might be going through this problem is perfectly logical if you have invalid database security rules. These can easily prevent you from retrieving your desired data and synchronizing your realtime database.
Let's assume you are trying to synchronize the myRef database reference. You need to set the correct rules that allow reading from this database reference - something along the lines of ".read" = true".
[Warning] Please be careful with these database security rules. Incorrect rules can lead to drastically undesired behaviors, such as people illegally reading and/or writing from/into your database. A good video on how to set flawless security rules is The key to firebase security - Google I/O 2016
I'm currently using the dataAPI to keep the dataitems synchronized between handheld and wearable.
Still I want to make sure that every data is stored and there is no data lost in the process.
I'm currently reading GPS parameters when the wear is not connected to the handheld and when they connect, they sync the dataitems.
How reliable is DataAPI?
Is my idea of creating a local file doubling my effort?
How can I create a local file on my wear device and then access it?
Syncing data using DataApi is reliable and I recommend using that; if you come across a scenario that sync is not happening reliably, that should be considered a bug and needs to be reported as such. One issue that folks run into is that they create the same data item and they don't get the onDataChanged() callback but that is by design, if the very same data is being added multiple times, there is no change, hence no callback triggers.
Another factor you might want to consider is whether the data you create on one node is for consumption by all other nodes or only a targeted one; DataApi syncs data across all connected nodes so if I create a data item on watch1 and want to sync that with my phone and if there is a watch2 in the picture as well, watch2 also gets the same data.
If you end up using the DataApi, I strongly recommend to make sure to put in place a policy that removes the data once it is synced and consumed otherwise data will be accumulated with no supervision and you'll finally run out of space.
To answer your questions:
I don't know how reliable it effectively is, but we had problems where data updates didn't trigger the appropriate listeners on the watch side. So I'm not sure. Maybe someone has an official statement for this?
I think it depends on the amount of data you want to store. So I suggest you first become clear about the amount and then choose the format. Keep in mind that there is also the possibility to store data in the Shared Preferences.
These guys here tried to save an image on the watch, but that makes no difference wheter it is an image file or text or whatever file.
I am trying to build an offline synchronization capability into my iOS App and would like to get some feedback/advice from the community on the strategy and best practice to be followed to do the same. The app details are as follows:
The app shows a digital catalog to users and allows them to perform actions like creating and placing orders, among others.
Currently the app only works when online, and we have APIs for all actions like viewing the catalog, creating/placing orders which return JSON data.
We would like to provide offline/synchronization capability to users, through which users can view the catalog and create/place orders while offline, and when they come online the order details will be synchronized and updated to our server.
We would also like to pull the latest data from the server, and have the app keep itself up to date in case of catalog changes or order changes that happened at the Server while the app was offline.
Can you guys help me to come with the best design and approach for handling this kind of functionality?
I have done something similar just in the beginning of this year. After I read about NSOperationQueue and NSOperation I did a straight forward approach:
Whenever an object is changed/added/... in my local database, I add a new "sync"-operation to the queue and I do not care about, if the app is online or offline (I added a reachability observer which either suspended the queue or takes it back working; of course, I do re-queueing if an error occurs (lost network during sync)). The operation itself reads/writes the database and does the networking stuff. My ViewController use a NSFetchedResultsController (with delegate=self) to get callbacks on changes. In some cases I needed some extra local data (it is about counting objects), where I have used NSManagedObjectContextObjectsDidChangeNotification.
Furthermore, I have used Multi-Context CoreData which sounded quite reasonable to use (I have only two contexts).
To get notified about changes from your server, I believe that iOS 7 has something new for you.
On the server side, you should read a little for the actual approach you want to go for: i.e. Data Synchronization by Dan Grover or Developing Android REST Client Applications (of course there are many more good articles out there).
Caution: you might be disappointed when you expect an easy solution. Your requirement is not unusual, but the solution might become more complex than you expect - depending on the "business rules" and other reasonable requirements. If you intelligently restrict your requirements you may find a solution which you can implement yourself, otherwise you may also consider to use a commercial product.
I could imagine, that if you design the business logic such that it takes an offline state into account and exposes this explicitly in the business logic, you may find a solution which you can implement yourself with moderate effort. What I mean by this is for example, when a user creates an order, it is initially in "not committed" stated. The order will only be committed when there is access to the server and if the server gives the "OK" that this order can actually be placed by this user. The server may also deny the order, sending corresponding messages to the user.
There are probably quite a few subtle issues that may arise due to the requirement of eventual consistency.
See also this question which contains pointers to solutions from commercial products, and if you visit their web sites give valuable information about the complexity of the problem and how this can be solved.