I am still trying to wrap my mind around on how to perform Realm queries with GDC.
I have this code in one of my classes:
class func placeNameForChatChannel(chatChannel: String, withCompletion handler: (String?)->()) {
dispatch_async(realmQueue) {
var channelEnvPredicate = NSPredicate(format: "channelName = %#", chatChannel)
var channelEnvs = PSTChannelEnvironment.objectsInRealm(realmdb, withPredicate: channelEnvPredicate)
if channelEnvs.count > 0 {
var channelEnvironment = channelEnvs[0] as! PSTChannelEnvironment
let placeName = channelEnvironment.placeName
handler(placeName)
} else {
handler(nil)
}
}
}
These two are declared as globals in my Application Delegate
var realmdb: RLMRealm {
return RLMRealm.defaultRealm()
}
var realmQueue = dispatch_queue_create("com.myapp.realmdb", DISPATCH_QUEUE_SERIAL)
I am getting the now infamous RLMException, reason: 'Realm accessed from incorrect thread when the handler callback is getting called.
What am I doing wrong?
You need to make sure that you recreate the RLMRealm on every dispatch to a GCD queue. If, instead of using that realmdb, you use RLMRealm.defaultRealm(), do things work?
Related
I'm trying to import a large data set of around 80k objects. I'm trying to follow Apple example
I have two issues:
In the code example there is a comment:
// taskContext.performAndWait runs on the URLSession's delegate queue
// so it won’t block the main thread.
But in my case I'm not using URLSession to fetch the JSON. The file is bundled with the app. In this case how to make sure the import won’t block the main thread. Should I create a custom queue ? Any example ?
In the example it's just importing an array of entities. But in my case I need to import just one entity that has 70k object in a relation to many.
So what I want to achieve is:
If there is a ContactBook don't import anything because we have already imported the JSON.
If there is no ContactBook create one and import all the 70k Contact object to the contacts relation of the ContactBook. This should happen in batches like in the example and should not block the UI.
What I have tried:
private func insertContactbookIfNeeded() {
let fetch: NSFetchRequest<Contactbook> = ContactBook.fetchRequest()
let contactBookCount = (try? context.count(for: fetch)) ?? 0
if contactBookCount > 0 {
return
}
let contacts = Bundle.main.decode([ContactJSON].self, from: "contacts.json")
// Process records in batches to avoid a high memory footprint.
let batchSize = 256
let count = contacts.count
// Determine the total number of batches.
var numBatches = count / batchSize
numBatches += count % batchSize > 0 ? 1 : 0
for batchNumber in 0 ..< numBatches {
// Determine the range for this batch.
let batchStart = batchNumber * batchSize
let batchEnd = batchStart + min(batchSize, count - batchNumber * batchSize)
let range = batchStart..<batchEnd
// Create a batch for this range from the decoded JSON.
let contactsBatch = Array(contacts[range])
// Stop the entire import if any batch is unsuccessful.
if !importOneBatch(contactsBatch) {
assertionFailure("Could not import batch number \(batchNumber) range \(range)")
return
}
}
}
private func importOneBatch(_ contactsBatch: [ContactJSON]) -> Bool {
var success = false
// Create a private queue context.
let taskContext = container.newBackgroundContext()
taskContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
// NOT TRUE IN MY CASE: (Any suggestion ??)
// taskContext.performAndWait runs on the URLSession's delegate queue
// so it won’t block the main thread.
print("isMainThread: \(Thread.isMainThread)") // prints true
taskContext.performAndWait {
let fetchRequest: NSFetchRequest<ContactBook> = ContactBook.fetchRequest()
fetchRequest.returnsObjectsAsFaults = true
fetchRequest.includesSubentities = false
let contactBookCount = (try? taskContext.count(for: fetchRequest)) ?? 0
var contactBook: ContactBook?
if contactBookCount > 0 {
do {
contactBook = try taskContext.fetch(fetchRequest).first
} catch let error as NSError {
assertionFailure("can't fetch the contactBook \(error)")
}
} else {
contactBook = ContactBook(context: taskContext)
}
guard let book = contactBook else {
assertionFailure("Could not fetch the contactBook")
return
}
// Create a new record for each contact in the batch.
for contactJSON in contactsBatch {
// Create a Contact managed object on the private queue context.
let contact = Contact(context: taskContext)
// Populate the Contact's properties using the raw data.
contact.name = contactJSON.name
contact.subContacts = NSSet(array: contactJSON.subContacts { subC -> Contact in
let contact = Contact(context: taskContext)
contact.name = subC.name
})
book.addToContacts(contact)
}
// Save all insertions and deletions from the context to the store.
if taskContext.hasChanges {
do {
try taskContext.save()
} catch {
print("Error: \(error)\nCould not save Core Data context.")
return
}
// Reset the taskContext to free the cache and lower the memory footprint.
taskContext.reset()
}
success = true
}
return success
}
The problem is that this is very slow because in each batch I fetch the workbook (which is getting bigger in each iteration) to be able to insert new batch of contacts in the contact book. Is there a efficient way to avoid fetching the workbook in each batch ? also any suggestion to make this is faster ? increase the batch size ? create a background queue ?
Update:
I have tried to create a ContactBook once in insertWordbookIfNeeded and pass it to importOneBatch with each iteration but I get:
Thread 1: Exception: "Illegal attempt to establish a relationship
'contactBook' between objects in different contexts
I have been following the following Realm tutorial:
Tutorial: Build iOS App from Scratch
The app is running, the ROS instance as well, but when I add a new task, it doesn't get saved. I have zeroed-in and found that in add(), the write portion of the code doesn't get executed:
let items = self.items
try! items.realm?.write {
items.insert(Task(value: ["text": text]), at: items.filter("completed = false").count)
}
self.realm looks like this:
(lldb) p self.realm
(RealmSwift.Realm?) $R0 = 0x000061800022ff60 {
rlmRealm = 0x00006000000bdd00 {
ObjectiveC.NSObject = {
isa = RLMRealm
}
}
}
items looks like this:
(lldb) p items
(RealmSwift.List<RealmDemo.Task>) $R1 = 0x0000600000026fa0 {
RealmSwift.ListBase = {
Realm.RLMListBase = {
baseNSObject#0 = {
isa = 0x0000600000026fa0
}
__rlmArray = 0x000061000004ee20
}
}
}
items.realm looks like this:
(lldb) p items.realm
(RealmSwift.Realm?) $R2 = nil
That explains why the write op doesn't get executed. What am I missing? I have verified the code and looks the same as in the tutorial (except for the credentials, of course).
Any ideas?
I need a read\write lock for my application. I've read https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock
and wrote my own class, cause there are no read/write lock in swift
class ReadWriteLock {
var logging = true
var b = 0
let r = "vdsbsdbs" // string1 for locking
let g = "VSDBVSDBSDBNSDN" // string2 for locking
func waitAndStartWriting() {
log("wait Writing")
objc_sync_enter(g)
log("enter writing")
}
func finishWriting() {
objc_sync_exit(g)
log("exit writing")
}
// ждет пока все чтение завершится чтобы начать чтение
// и захватить мютекс
func waitAndStartReading() {
log("wait reading")
objc_sync_enter(r)
log("enter reading")
b++
if b == 1 {
objc_sync_enter(g)
log("read lock writing")
}
print("b = \(b)")
objc_sync_exit(r)
}
func finishReading() {
objc_sync_enter(r)
b--
if b == 0 {
objc_sync_exit(g)
log("read unlock writing")
}
print("b = \(b)")
objc_sync_exit(r)
}
private func log(s: String) {
if logging {
print(s)
}
}
}
It works good, until i try to use it from GCD threads.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0)
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)
When i try to use this class from different async blocks at some moment it allows to write when write is locked
here is sample log:
wait reading
enter reading
read lock writing
b = 1
wait reading
enter reading
b = 2
wait reading
enter reading
b = 3
wait reading
enter reading
b = 4
wait reading
enter reading
b = 5
wait reading
enter reading
b = 6
wait reading
enter reading
b = 7
wait reading
enter reading
b = 8
wait reading
enter reading
b = 9
b = 8
b = 7
b = 6
b = 5
wait Writing
enter writing
exit writing
wait Writing
enter writing
So, as you can see g was locked, but objc_sync_enter(g) allows to continue.
Why could this happen ?
BTW i checked how many times ReadWriteLock constructed, and it's 1.
Why objc_sync_exit not working and allowing to objc_sync_enter(g) when it's not freed ?
PS Readwirtelock defined as
class UserData {
static let lock = ReadWriteLock()
Thanks.
objc_sync_enter is an extremely low-level primitive, and isn't intended to be used directly. It's an implementation detail of the old #synchronized system in ObjC. Even that is extremely out-dated and should generally be avoided.
Synchronized access in Cocoa is best achieved with GCD queues. For example, this is a common approach that achieves a reader/writer lock (concurrent reading, exclusive writing).
public class UserData {
private let myPropertyQueue = dispatch_queue_create("com.example.mygreatapp.property", DISPATCH_QUEUE_CONCURRENT)
private var _myProperty = "" // Backing storage
public var myProperty: String {
get {
var result = ""
dispatch_sync(myPropertyQueue) {
result = self._myProperty
}
return result
}
set {
dispatch_barrier_async(myPropertyQueue) {
self._myProperty = newValue
}
}
}
}
All your concurrent properties can share a single queue, or you can give each property its own queue. It depends on how much contention you expect (a writer will lock the entire queue).
The "barrier" in "dispatch_barrier_async" means that it is the only thing allowed to run on the queue at that time, so all previous reads will have completed, and all future reads will be prevented until it completes. This scheme means that you can have as many concurrent readers as you want without starving writers (since writers will always be serviced), and writes are never blocking. On reads are blocking, and only if there is actual contention. In the normal, uncontested case, this is extremely very fast.
Are you 100% sure your blocks are actually executing on different threads?
objc_sync_enter() / objc_sync_exit() are guarding you only from object being accessed from different threads. They use a recursive mutex under the hood, so they won't either deadlock or prevent you from repeatedly accessing object from the same thread.
So if you lock in one async block and unlock in another one, the third block executed in-between can have access to the guarded object.
This is one of those very subtle nuances that is easy to miss.
Locks in Swift
You have to really careful what you use as a Lock. In Swift, String is a struct, meaning it's pass-by-value.
Whenever you call objc_sync_enter(g), you are not giving it g, but a copy of g. So each thread is essentially creating its own lock, which in effect, is like having no locking at all.
Use NSObject
Instead of using a String or Int, use a plain NSObject.
let lock = NSObject()
func waitAndStartWriting() {
log("wait Writing")
objc_sync_enter(lock)
log("enter writing")
}
func finishWriting() {
objc_sync_exit(lock)
log("exit writing")
}
That should take care of it!
In addition to #rob-napier's solution. I've updated this to Swift 5.1, added generic typing and a couple of convenient append methods. Note that only methods that access resultArray via get/set or append are thread safe, so I added a concurrent append also for my practical use case where the result data is updated over many result calls from instances of Operation.
public class ConcurrentResultData<E> {
private let resultPropertyQueue = dispatch_queue_concurrent_t.init(label: UUID().uuidString)
private var _resultArray = [E]() // Backing storage
public var resultArray: [E] {
get {
var result = [E]()
resultPropertyQueue.sync {
result = self._resultArray
}
return result
}
set {
resultPropertyQueue.async(group: nil, qos: .default, flags: .barrier) {
self._resultArray = newValue
}
}
}
public func append(element : E) {
resultPropertyQueue.async(group: nil, qos: .default, flags: .barrier) {
self._resultArray.append(element)
}
}
public func appendAll(array : [E]) {
resultPropertyQueue.async(group: nil, qos: .default, flags: .barrier) {
self._resultArray.append(contentsOf: array)
}
}
}
For an example running in a playground add this
//MARK:- helpers
var count:Int = 0
let numberOfOperations = 50
func operationCompleted(d:ConcurrentResultData<Dictionary<AnyHashable, AnyObject>>) {
if count + 1 < numberOfOperations {
count += 1
}
else {
print("All operations complete \(d.resultArray.count)")
print(d.resultArray)
}
}
func runOperationAndAddResult(queue:OperationQueue, result:ConcurrentResultData<Dictionary<AnyHashable, AnyObject>> ) {
queue.addOperation {
let id = UUID().uuidString
print("\(id) running")
let delay:Int = Int(arc4random_uniform(2) + 1)
for _ in 0..<delay {
sleep(1)
}
let dict:[Dictionary<AnyHashable, AnyObject>] = [[ "uuid" : NSString(string: id), "delay" : NSString(string:"\(delay)") ]]
result.appendAll(array:dict)
DispatchQueue.main.async {
print("\(id) complete")
operationCompleted(d:result)
}
}
}
let q = OperationQueue()
let d = ConcurrentResultData<Dictionary<AnyHashable, AnyObject>>()
for _ in 0..<10 {
runOperationAndAddResult(queue: q, result: d)
}
I had the same problem using queues in background. The synchronization is not working all the time in queues with "background" (low) priority.
One fix I found was to use semaphores instead of "obj_sync":
static private var syncSemaphores: [String: DispatchSemaphore] = [:]
static func synced(_ lock: String, closure: () -> ()) {
//get the semaphore or create it
var semaphore = syncSemaphores[lock]
if semaphore == nil {
semaphore = DispatchSemaphore(value: 1)
syncSemaphores[lock] = semaphore
}
//lock semaphore
semaphore!.wait()
//execute closure
closure()
//unlock semaphore
semaphore!.signal()
}
The function idea comes from What is the Swift equivalent to Objective-C's "#synchronized"?, an answer of #bryan-mclemore.
How would I tell the for-loop to move on to the next item in the array in Swift?
for (key, value) in commentIds {
NetworkHelper.makeNetworkCall(key, completionHandler:{
//Tell current thread the for-loop is being called, to move on to the next item in the for loop?
}
}
I'm planning on having the for-loop in a dispatch_async thread, so it doesn't block the main thread.. Am I designing this wrong..? Should I have a flag?
edit
Flags in the main thread seem to work. Once the network call is done, flip the flag.
I would make the assumption that if the makeNetworkCall method takes a completionHandler parameter that it will be doing its work asynchronously. Given that, the for loop will call makeNetworkCall for every key in commentIds before any of them complete.
If you want to process them in series and not start the next one until the prior has finished, you will need to create an array of the keys and write some code like this:
var commentIdKeys: Array<KeyType>()?
var keyIndex: Int?
func startRequests() {
commentIdKeys = Array(commentIds.keys)
keyIndex = 0
processNextKey()
}
func processNextKey() {
guard let keys = commentIdKeys, let index = keyIndex else {
print("processNextKey() invoked without values set")
return
}
guard keyIndex < keys.count else {
commentIdKeys = nil
keyIndex = nil
print("Finished processing keys")
return
}
NetworkHelper.makeNetworkCall(key, completionHandler:{
// Do some work
self.keyIndex = ++index
self.processNextKey()
}
}
I've created the Class StartConnection to handle my NSURL requests. It gets called from my AppDelegate twice and that works well. It's called by one other class as well and that also works well. This is the implementation of StartConnection:
#import "StartConnection.h"
#import "BlaBlaBlog-swift.h"
#implementation StartConnection
{
BOOL startedForBlog;
}
- (void)getRssFileWithUrl: (NSString*)rssUrlString forBlog:(BOOL)forBlog
{
startedForBlog = forBlog;
NSURL *url = [NSURL URLWithString:rssUrlString];
NSURLRequest *rssRequest = [NSURLRequest requestWithURL:url];
NSURLConnection *connection = [[NSURLConnection alloc]initWithRequest:rssRequest delegate:self];
[connection start];
}
-(void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
dataSize = [response expectedContentLength];
}
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
if (receivedData==nil )
{
receivedData = [[NSMutableData alloc]init];
}
// Append the new data to the instance variable you declared
[receivedData appendData:data];
}
-(void)connectionDidFinishLoading:(NSURLConnection *)connection
{
self.receivedDataComplete = receivedData;
if (startedForBlog){
[self.delegate performSelector: #selector(receiveDataCompleted)];
}
else
[self.delegate performSelector: #selector(receivePodCastDataCompleted)];
}
To get some hands on experience with SWIFT I've added an experimental SWIFT class to my code that also uses StartConnection.h. In the debugger I can see an instance of StartConnection being created and the getFileWithUrl methode seems to be kicked of normally. But that's all that happens. None of the delegate methods is called.
This is SWIFT class:
import UIKit
class CheckActuality: NSObject, GetReceivedDataProtocol, WordPressParserDelegate {
var retrievePostData = StartConnection()
var parseCompleted: Bool=false
var result: Bool = true
lazy var wpParser = WordPressParser()
lazy var defaults = NSUserDefaults.standardUserDefaults()
func isActual () -> Bool {
var url = "http://blablablog.nl/new_api.php?function=get_recent_posts&count=1"
self.retrievePostData.delegate=self
self.retrievePostData.getRssFileWithUrl(url, forBlog:true)
while !self.parseCompleted
{
// wait till wpparser has completed
}
if self.wpParser.arrayWithPostDictionaries.count == 1
// Some info has been retrieved
{
var posts: NSArray = wpParser.arrayWithPostDictionaries
var post: NSDictionary = posts.objectAtIndex(0) as NSDictionary
var latestPostUrl: String = post.objectForKey("postUrl") as String
var currentLatestPostUrl = defaults.stringForKey("ttt")
if latestPostUrl != currentLatestPostUrl {
result = false
}
else {
result = true
}
}
return result
}
func receiveDataCompleted () {
if self.retrievePostData.receivedDataComplete != nil
{
self.wpParser.delegate=self
self.wpParser.parseData(retrievePostData.receivedDataComplete)
}
else
{
// warning no internet
}
}
func wpParseCompleted () {
self.parseCompleted=true
}
}
And to be complete, the call in my AppDelegate look like this:
//
// retrieving PostData. Create a connection, set delegate to self en start with created url
//
retrievePostData = [[StartConnection alloc]init];
retrievePostData.delegate = self;
NSString *url = [[wordPressUrl stringByAppendingString:apiString] stringByAppendingString:pageCountString];
[retrievePostData getRssFileWithUrl:url forBlog:(BOOL)true];
//
// retrieving PodcastData. Create a connection, set delegate to self en start with created url
//
retrievePodCastData = [[StartConnection alloc]init];
retrievePodCastData.delegate = self;
[retrievePodCastData getRssFileWithUrl:podcastUrl forBlog:(BOOL)false];
I'm breaking my head for almost a day. Hope some of you far more experienced guys can help this starter out.
The while !parseCompleted loop is blocking the main thread until the download and parsing is done. But the processing of the received data happens on the main thread, too, so if that thread is blocked, your app will be deadlocked.
I would eliminate that while loop and put all of the post processing inside your receivedDataComplete method.
By the way, implicit in this change is the fact that isActual must be an asynchronous method. Thus, rather than returning a Bool value, it should be a Void return type but you should employ the completionHandler pattern (and I'd change it to also return an error object, too):
class CheckActuality: NSObject, GetReceivedDataProtocol, WordPressParserDelegate {
let errorDomain = "com.domain.app.CheckActuality"
lazy var retrievePostData = StartConnection()
lazy var wpParser = WordPressParser()
lazy var defaults = NSUserDefaults.standardUserDefaults()
var completionHandler: ((latestPost: Bool!, error: NSError?)->())!
func isActual(completionHandler: (latestPost: Bool!, error: NSError?)->()) {
// save the completionHandler, which will be called by `receiveDataCompleted` or `wpParseCompleted`
self.completionHandler = completionHandler
// now perform query
let url = "http://blablablog.nl/new_api.php?function=get_recent_posts&count=1" // as an aside, use `let` here
retrievePostData.delegate = self // also note that use of `self` is redundant here
retrievePostData.getRssFileWithUrl(url, forBlog:true)
}
func receiveDataCompleted () {
if retrievePostData.receivedDataComplete != nil {
wpParser.delegate = self
wpParser.parseData(retrievePostData.receivedDataComplete)
} else {
// frankly, I'd rather see you change this `receiveDataCompleted` return the `NSError` from the connection, but in the absence of that, let's send our own error
let error = NSError(domain: errorDomain, code: 1, userInfo: nil)
completionHandler(latestPost: nil, error: error)
}
}
func wpParseCompleted () {
if wpParser.arrayWithPostDictionaries.count == 1 { // Some info has been retrieved
let posts: NSArray = wpParser.arrayWithPostDictionaries
let post: NSDictionary = posts.objectAtIndex(0) as NSDictionary
let latestPost: String = post.objectForKey("postUrl") as String
let currentlatestPost = defaults.stringForKey("ttt")
completionHandler(latestPost: (latestPost != currentlatestPost), error: nil)
}
// again, I'd rather see you return a meaningful error returned by the WordPressParser, but I'll make up an error object for now
let error = NSError(domain: errorDomain, code: 2, userInfo: nil)
completionHandler(latestPost: nil, error: error)
}
}
Now, I don't know if latestPost is the appropriate name for the value you were trying to return, so change that to whatever makes sense for your routine. Also, the name isActual doesn't really make sense, but I'll let you change that to whatever you want.
Anyway, when you use it, you'd use the trailing closure syntax to specify the completionHandler block that will be performed asynchronously:
let checkActuality = CheckActuality()
func someFunc() {
checkActuality.isActual() { latestPost, error in
if error != nil {
// do whatever error handling you want
println(error)
} else if latestPost {
// yes, latest post
} else {
// nope
}
}
// note, do not try to check `latestPost` here because the
// above closure runs asynchronously
}
Needless to say, this is a completionHandler pattern, but looking at your code, you seem to favor delegate patterns. If you wanted to implement this using the delegate pattern, you can. But the idea is the same: This isActual method (whatever you end up renaming it to) runs asynchronously, so you have to inform the caller when it is complete.