Periodically in my app, I need to perform large writes to Realm, anywhere between 100 to 10,000 objects. Obviously this is a large write, so I'm attempting to perform this write in the background so that the user can perform other operations and not even notice the write. Unfortunately, even though I thought my write was being performed on a background thread, the main UI still gets blocked. Here is the jist of the method that I call to perform the writes to realm. This method is called repeatedly on single objects from an array I'm looping through. Does it look like I'm doing anything blatantly wrong? Any help would be greatly appreciated.
func writeCustomerToRealm(inputCustomer:Customer) {
let qualityOfServiceClass = QOS_CLASS_BACKGROUND
let backgroundQueue = dispatch_get_global_queue(qualityOfServiceClass, 0)
dispatch_async(backgroundQueue, {
let realm = try! Realm()
realm.beginWrite()
realm.add(self.swapCustomerForRealmCustomer(inputCustomer))
try! realm.commitWrite()
})
}
Your provided code is the correct way to perform an asynchronous, background write using Realm. It should not block the main thread unless the main thread itself attempts to perform a write transaction while the background write is in progress.
The instruments trace you shared indicates that the majority of the non-idle time spent on the main thread is spent processing web socket messages. There's no indication of any work related to Realm occurring on the main thread. The web socket message processing on the main thread accounts for 45% of the total time period of the trace, and is likely the cause of the blocked UI you're experiencing. The majority of the web socket processing time is spent performing work related to NSDateFormatter. In particular, the NSDateFormatter appears to be frequently regenerating its underlying CFDateFormatter. This suggests that properties of the NSDateFormatter are being frequently updated, or that new NSDateFormatter instances are being used for each call. Both of those practices should be avoided.
Related
This might sound quite basic and stupid but it has been bothering me for a while. How can print be classified in terms of operation - main or background ?
As a small test, on putting print in a background task - web service call :
Webservice().loadHeadlinesForSource(source: source) { headlines in
print("background print")
self.headlineViewModels = headlines.map(HeadlineViewModel.init)
DispatchQueue.main.async {
print("main thread print")
completion()
}
}
Both the print statements get printed. From previous experience, if print was a main thread task, Xcode would have given me a warning saying that I need to put that in main thread. This is an evidence that print is not a main thread operation. Note that I am not saying print is a background task.
However, I have this understanding that since print displays output on Console, it is not a background operation. As a matter of fact all logging operations are not.
How would one justify the classification ?
It seems what you consider to be a main thread operation is a call that needs to be performed on the main thread. From that perspective you are correct and have found an evidence of this call not being a main thread operation.
But does this have anything to do with anything else? Internally if needed this method may still execute its real operation on the main thread or any other thread for what we care. So in this sense a main thread operation is a restriction that call needs to be performed on main thread but has nothing to do with its execution or multithreading.
Without looking into what print does in terms of coding we can see that it works across multiple "computers". You can run your app on your device (iPhone) while plugged and Xcode on your computer will print out logs. This makes a suspicion that print is much like call to the remote server in which case the server is responsible for serializing the events so it makes no difference what thread the client is on. There are other possibilities such as dropping logs into file and then sending it which really makes little difference.
So How can print be classified in terms of operation - main or background? The answer is probably none. The call is not restricted to any thread so it is not main. It will probably lock whatever thread it is on until the operation is complete so it is not background either. Think of it like Data(contentsOf: <#T##URL#>) which will block the thread until data from given URL is retrieved (or exception is thrown).
I'm starting using Realm recently, I'm not sure if my use case is valid:
Normally, when reading a lot of data from DB, I want to put it in a background queue so it will async get the data and later use it on main thread.
For example, I want to fetch several results based on city:
private var results: [Results<SomeObject>?] = []
autoreleasepool {
DispatchQueue(label: "background").async {
[unowned self] in
do
{
let realm = try Realm()
for i in 1...City.count
{
self.results.append(realm.objects(SomeObject.self).filter("city=\(i)"))
}
}
catch
{
NSLog("Failed to open Realm instance on background qeueue")
}
}
}
And later use results to update my chart:
cell.setChartData(ChartDataFactory.createCombinedData(from: results[0]))
However if I apply this model for Realm, I'm getting error like
Terminating app due to uncaught exception 'RLMException', reason: 'Realm accessed from incorrect thread.
I understand I must use realm for each thread, and I can do this by reading realm on main thread, but I don't want the realm query block my main thread.
is there any way I can achieve my goal? e.g. reading realm in a background queue and access the results from another thread, while keeping the auto-refresh feature.
Thanks.
Realm has built-in functionality for running a query on a background thread and delivering the results to the main thread by using Results.observe().
If you specifically need to perform expensive filtering logic that can't be expressed as a Realm query, you can manually pass an array of objects between threads using ThreadSafeReference.
As of 5.0, you can now construct the query on a background thread and receive notifications on the main thread using the on: parameter to observe():
DispatchQueue.global().async {
let realm = try! Realm()
let results = realm.objects(ObjectType.self).filter("property in %#", expensiveFunction(realm))
self.token = results.observe(on: .main) { change in
// do stuff with the results on the main thread
}
}
Realm objects are only accessible through the realm from which they are fetched or created. Realm instances cannot be shared between threads (which you are aware of), and sharing an object from a specific realm instance to another thread, implicitly has the same effects as sharing a realm instance between threads. This is due to the tight coupling between the objects and the realm instance.
As mentioned in this GitHub issue https://github.com/realm/realm-cocoa/issues/946, the recommended practice is to share the primary keys (if your realm object overrides the primaryKey method of RealmObject (Objective-C) / Object (Swift)).
You're trying to directly access 'results' property from a different queue and that will crash. You should instead use ThreadSafeReference as indicated on the answer of Thomas.
Make sure to create a ThreadSafeReference for results and call realm.resolve() on your background queue before fetching from your Realm database.
I solved it like this. I see an overall performance improvement, but I couldn't find any implementation example for querying on a background thread. Might be there is a solution with even better performance.
self.results = self.realm.objects(Object.self).filter(predicate).sorted(by: sortProperties)
self.notificationToken = self.results.observe({ (notification) in
self.tableview.reloadData()
})
This is the results on an iPhone X with a database of ~171k items. Durations are in seconds.
Search on UI tread:
UI thread blocked 0.730504035949707
Search with the code from above:
UI thread blocked 0.28138411045074463
background search duration 0.5073530673980713
I am working with an external device that I receive data from. I want to handle its data read/write queue asynchronously, in a thread.
I've got it mostly working: There is a class that simply manages the two streams, using the NSStreamDelegate to respond to incoming data, as well as responding to NSStreamEventHasSpaceAvailable for sending out data that's waiting in a buffer after having failed to be sent earlier.
This class, let's call it SerialIOStream, does not know about threads or GCD queues. Instead, its user, let's call it DeviceCommunicator, uses a GCD queue in which it initializes the SerialIOStream class (which in turn creates and opens the streams, scheduling them in the current runloop):
ioQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT,0);
dispatch_async(ioQueue, ^{
ioStreams = [[SerialIOStream alloc] initWithPath:[#"/dev/tty.mydevice"]];
[[NSRunLoop currentRunLoop] run];
});
That way, the SerialIOStreams stream:handleEvent: method runs in that GCD queue, apparently.
However, this causes some problems. I believe I run into concurrency issues, up to getting crashes, mainly at the point of feeding pending data to the output stream. There's a critical part in the code where I pass the buffered output data to the stream, then see how much data was actually accepted into the stream, and then removing that part from my buffer:
NSInteger n = self.dataToWrite.length;
if (n > 0 && stream.hasSpaceAvailable) {
NSInteger bytesWritten = [stream write:self.dataToWrite.bytes maxLength:n];
if (bytesWritten > 0) {
[self.dataToWrite replaceBytesInRange:NSMakeRange(0, bytesWritten) withBytes:NULL length:0];
}
}
The above code can get called from two places:
From the user (DeviceCommunicator)
From the local stream:handleEvent: method, after being told that there's space in the output stream.
Those may be (well, surely are) running in separate thread, and therefore I need to make sure they do not run concurrently this code.
I thought I'd solve this by using the following code in DeviceCommunicator when sending new data out:
dispatch_async (ioQueue, ^{
[ioStreams writeData:data];
});
(writeData adds the data to dataToWrite, see above, and then runs the above code that sends it to the stream.)
However, that doesn't work, apparently because ioQueue is a concurrent queue, which may decide to use any available thread, and therefore lead to a race condition when writeData get called by the DeviceCommunicator while there's also a call to it from stream:handleEvent:, on separate threads.
So, I guess I am mixing expectations of threads (which I'm a bit more familiar with) into my apparent misunderstandings with GCD queues.
How do I solve this properly?
I could add an NSLock, protecting the writeData method with it, and I believe that would solve the issue in that place. But I am not so sure that that's how GCD is supposed to be used - I get the impression that'd be a cludge.
Shall I rather make a separate class, using its own serial queue, for accessing and modifying the dataToWrite buffer, perhaps?
I am still trying to grasp the patterns that are involved with this. Somehow, it looks like a classic producer / consumer pattern, but on two levels, and I'm not doing this right.
Long story, short: Don't cross the streams! (haha)
NSStream is a RunLoop-based abstraction (which is to say that it intends to do its work cooperatively on an NSRunLoop, an approach which pre-dates GCD). If you're primarily using GCD to support concurrency in the rest of your code, then NSStream is not an ideal choice for doing I/O. GCD provides its own API for managing I/O. See the section entitled "Managing Dispatch I/O" on this page.
If you want to continue to use NSStream, you can either do so by scheduling your NSStreams on the main thread RunLoop or you can start a dedicated background thread, schedule it on a RunLoop over there, and then marshal your data back and forth between that thread and your GCD queues. (...but don't do that; just bite the bullet and use dispatch_io.)
I want to know As we all know how asynchronous task are necessary for concurrency but Wanted to know why we need the synchronous tasks. while we can achieve the same with the normal usage of function.
Thanks & regards
Rohit
When you calls something synchronously, it means that 'the thread that initiated that operation will wait for the task to finish before
continuing'. Asynchronous means that it will not wait for finish the task.
synchronous calls stops your current action and returns when the call returned. with asynchronous calls you can continue.
synchronous is the opposite of asynchronous code, and therefore is ordinary code.
At the end, if asynchronous is totally out of scope then you will not emphasize the word synchronous.
It helps to synchronise threads, as the name suggests.
consider a typical usage of GCD async and sync (pseudo)
async background_thread {
//1 call webservice or other long task that would block the main thread
sync main_thread {
//2 update UI with results from 1
}
//3 do something else that relies on 2
}
now if 2 was in an async and you needed to do something at 3 that relies on the updates at 2 to have happened, then you are not guaranteed (and most likely wont) get the behaviour you are expecting. instead, you use a sync to make sure that the task is completed before continuing the execution in the background thread.
If you are asking now, why not just take out the sync/async around 2 so it executes in order anyway? the problem is, the UI must not be updated on a background thread otherwise the behaviour is undefined (which usually means the UI lags a lot). So in essence what happens is the background thread waits at 2's sync until the main thread gets round to executing that block, then it will continue with the rest of the execution on the background thread.
If you were dealing with a task that doesnt require the main thread (or some other thread) to execute properly, then yes you may as well take out the sync at 2.
This is just one example of how a sync is useful, there are others if you are doing advanced threading in your app.
Hope this helps
Typically it's because you want to do an operation on a specific different thread but you need the result of that operation. You cannot do the operation asynchronously because your code will proceed before the operation on the other thread completes.
Apple has a very nice example:
func asset() -> AVAsset? {
var theAsset : AVAsset!
self.assetQueue.sync {
theAsset = self.getAssetInternal().copy() as! AVAsset
}
return theAsset
}
Any thread might call the asset method; but to avoid problems with shared data, we require that only functions that are executed from a particular queue (self.assetQueue) may touch an AVAsset, so when we call getAssetInternal we do it on self.assetQueue. But we also need the result returned by our call to getAssetInternal; hence the call to sync rather than async.
The handler code (in this case, collecting accelerometer data) is being executed asynchronously when my accelerometer moves.
That means if I try to save data.x, data.y, data.z in a variable, even if the variable is declared outside of the handler, the variable will be nil if I attempt to print it anywhere.
How do I save this data to access in other parts of my code?
(Or does everything have to happen in my handler, best-practices wise?)
if motionManager.accelerometerAvailable{
let motionQueue = NSOperationQueue.mainQueue()
motionManager.startDeviceMotionUpdatesToQueue(motionQueue,
withHandler: gravityUpdated)
}
func gravityUpdated(motion: CMDeviceMotion!, error: NSError!) {
let grav : CMAcceleration = motion.gravity;
println(grav.x)
}
The main thing to be wary of is that these events can come in more quickly than the main thread can process them. As the documentation says:
Because the processed events might arrive at a high rate, using the main operation queue is not recommended.
Hence, you should your own background queue to handle these events.
Regarding how you then use this updated information on the main thread, there are two considerations:
To ensure your code is thread-safe, any variables that you use both from this background thread and other threads must be synchronized.
Make sure you don't just dispatch updates back to the main thread. I would generally create a dispatch source of DISPATCH_SOURCE_TYPE_DATA_OR or DISPATCH_SOURCE_TYPE_DATA_ADD, put a handler for that source on the main queue, and then your motionQueue can then then perform a dispatch_source_merge_data on this source.
GCD will then coalesce these data changes, notifying the main thread when there were updates, but not backlogging the main thread in the process.
By the way, you may also want to review Table 4-1 of the Event Handling Guide, which outlines common update intervals for acceleration events (measured in Hz), depending upon the intended usage:
10–20: Suitable for determining a device’s current orientation vector.
30–60: Suitable for games and other apps that use the accelerometer for real-time user input.
70–100: Suitable for apps that need to detect high-frequency motion. For example, you might use this interval to detect the user hitting the device or shaking it very quickly.
You might want to choose a deviceMotionUpdateInterval commensurate with your application's needs.