I am performing multiple read operations on the same resource stored on disk.
Sometimes the read operation itself takes longer than the time between requests for that same resource. In those cases it would make sense to batch the read operations into one read request from the disk and then return the same result to the various requester.
Initially I tried caching the result from the initial fetch resource request - but this didn't work because the time it took to read the resource was too long, and new requests came in - meaning that they would try to fetch the resource as well.
Is it possible to "append" the additional requests to the ones that are already in progress?
The code I have now follows this basic structure (which isn't good enough):
-(void)fileForKey:(NSString *)key completion:(void(^)(NSData *data) {
NSData *data = [self.cache threadSafeObjectForKey:key];
if (data) {
// resource is cached - so return it - no need to read from the disk
completion(data);
return;
}
// need to read the resource from disk
dispatch_async(self.resourceFetchQueue, ^{
// this could happen multiple times for the same key - because it could take a long time to fetch the resource - all the completion handlers should wait for the resource that is fetched the first time
NSData *fetchedData = [self fetchResourceForKey:key];
[self.cache threadSafeSetObject:fetchedData forKey:key];
dispatch_async(self.completionQueue, ^{
completion(fetchedData);
return;
});
});
}
I think you want to introduce a helper object
#interface CacheHelper{
#property (nonatomic, copy) NSData *data;
#property (nonatomic, assign) dispatch_semaphore_t dataReadSemaphore;
}
Your reader method now becomes something like
CacheHelper *cacheHelper = [self.cache threadSafeObjectForKey:key]
if (cacheHelper && cacheHelper.data)
{
completion(cacheHelper.data);
return;
}
if (cacheHelper)
{
dispatch_semaphore_wait(cacheHelper.dataReadSemaphore, DISPATCH_TIME_FOREVER);
dispatch_semaphore_signal(cacheHelper.dataReadSemaphore);
completion(cacheHelper.data)
return;
}
cacheHelper = [[CacheHelper alloc] init]
cacheHelper.dataReadSemaphore = dispatch_semaphore_create(0);
cacheHelper.data = [self fetchResourceForKey:key];
[self.cache threadSafeSetObject:cacheHelper forKey:key];
dispatch_semaphore_signal(cacheHelper.dataReadSemaphore);
completion(cacheHelper.data)
This is uncompiled code, so please check the spelling and logic, but I hope it explains the idea. I like this post if you want an introduction to semaphores.
Related
I'm getting occasional crashes in my GCDWebServer handlers, which access mutable dictionaries. The GCDWebServer ReadMe says the handlers "are executed on arbitrary threads within GCD so special attention must be paid to thread-safety and re-entrancy," and I think that's my problem. Is there a best practice or recommended pattern for accessing mutable properties of the parent object from the handlers?
I don't know if I can synchronize between threads since I'm not creating the handler threads. Also, I imagine I could use an asynchronous handler, then call a method on the main thread from there, then do my work in that method, then send the response, but that seems more complicated and less efficient than necessary.
Here's a simplified version of my code:
#property (nonatomic, strong) NSMutableDictionary *data;
#property (nonatomic, strong) GCDWebServer *webServer;
- (void)setup {
self.data = [NSMutableDictionary dictionary];
[self.data setObject:#"1" forKey:#"status"];
self.webServer = [[GCDWebServer alloc] init];
[self.webServer addHandlerForMethod:#"GET" path:#"/getStatus.txt" requestClass:[GCDWebServerRequest class] processBlock:^(GCDWebServerRequest *request) {
return [self handleStatusRequest:request];
}];
}
- (GCDWebServerDataResponse *)handleStatusRequest:(GCDWebServerRequest *)request {
NSString *status = [self.data objectForKey:#"status"]; // crash here
return [GCDWebServerDataResponse responseWithText:status];
}
Are you mutating your data dictionary after creating it? If so that would explain the crashes.
You must prevent concurrent access to your data dictionary by using locks. The easiest way is through GCD e.g.
#property dispatch_queue_t lock;
__block NSString* status;
dispatch_sync(self.lock, ^{
status = [self.data objectForKey:#"status"];
});
NSString* status = #"Hello";
dispatch_async(self.lock, ^{
[self.data setObject:status forKey:#"status"];
}); // Use dispatch_sync() or dispatch_async() here depending on your needs
While building my app, Marco Polo (getmarcopolo.com), I found that one of the most challenging parts of the app was pulling data from the server without slowing down the interface and without it crashing. I've got it settled now, and wanted to share my knowledge with any other developers having the same issue.
When pulling data from a server, there are a number of factors that need to be taken into consideration:
Data integrity - No data is ever missed from the server
Data persistence - Data is cached and can be accessed even when offline
Lack of interference with the interface (main thread) - Achieved using multithreading
Speed - Achieved using thread concurrency
Lack of thread collisions - Achieved using serial thread queues
So the question is, how do you achieve all 5?
I've answered this below, but would love to hear feedback on how to improve the process (with this example), as I feel it is not very easy to find in one place right now.
I'll be using the example of refreshing the marco's in the notification feed. I'll also be referring to Apple's GCD library (see https://developer.apple.com/library/mac/documentation/Performance/Reference/GCD_libdispatch_Ref/Reference/reference.html). First, we create a singleton (see http://www.galloway.me.uk/tutorials/singleton-classes/):
#implementation MPOMarcoPoloManager
+ (MPOMarcoPoloManager *)instance {
static MPOMarcoPoloManager *_instance = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
_instance = [[self alloc] init];
});
return _instance;
}
#end
This allows for us to call [MPOMarcoPoloManager instance] at any time, from any file, and access the properties in the the singleton. It also ensures that there is always only one instance of the marco polos. 'static dispatch_once_t onceToken; dispatch_once(&onceToken, ^{' ensures thread stability.
The next step is to add the data structure we will be accessing publicly. In this case, add an NSArray for the marcos to the header file, as well as a public declaration of 'instance':
#interface MPOMarcoPoloManager : NSObject
+ (MPOMarcoPoloManager *)instance;
#property (strong, nonatomic) NSArray *marcoPolos;
#end
Now that the array and the instance are accessible publicly, it's time to ensure data persistence. We will achieve this by adding the ability to cache the data. The following code will
1. Initializes our serverQueue to the global queue, which allows multiple threads to run concurrently
2. Initializes our localQueue to a serial queue, which allows only one thread to be run at a time. All local data manipulation should be done on this thread to ensure no thread collisions
3. Gives us a method to call for caching our NSArray, with objects that conform to NSCoding (see http://nshipster.com/nscoding/)
4. Attempts to pull the data structure from the cache, and initializes a new one if it cannot
#interface MPOMarcoPoloManager()
#property dispatch_queue_t serverQueue;
#property dispatch_queue_t localQueue;
#end
#implementation MPOMarcoPoloManager
+ (MPOMarcoPoloManager *)instance {
static MPOMarcoPoloManager *_instance = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
_instance = [[self alloc] init];
});
return _instance;
}
- (id)init {
self = [super init];
if (self) {
_marcoPolos = [NSKeyedUnarchiver unarchiveObjectWithFile:self.marcoPolosArchivePath];
if(!self.marcoPolos) {
_marcoPolos = [NSArray array];
}
//serial queue
_localQueue = dispatch_queue_create([[NSBundle mainBundle] bundleIdentifier].UTF8String, NULL);
//Parallel queue
_serverQueue = dispatch_queue_create(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), NULL);
}
return self;
}
- (NSString *)marcoPolosArchivePath {
NSArray *cacheDirectories = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES);
NSString *cacheDirectory = [cacheDirectories objectAtIndex:0];
return [cacheDirectory stringByAppendingFormat:#"marcoPolos.archive"];
}
- (BOOL)saveChanges {
BOOL success = [NSKeyedArchiver archiveRootObject:self.marcoPolos toFile:[self marcoPolosArchivePath]];
return success;
}
#end
Now that we have the structure of the singleton, It's time to add the ability to refresh our marco's. Add the declarations of refreshMarcoPolosInBackgroundWithCallback:((^)(NSArray *result, NSError *error)) to the header file:
...
- (void)refreshMarcoPolosInBackground:((^)(NSArray *result, NSError *error))callback;
...
Now it's time to implement the refresh. Notice that all server calls are performed on the serverQueue (which is parallel), and any data manipulation is done on the localQueue (which is serial). When the method is completed, we use what is called a C block (see https://developer.apple.com/library/ios/documentation/cocoa/Conceptual/Blocks/Articles/00_Introduction.html) to callback the result to the main thread. Any task that acts on a background thread should have a callback to the main thread to inform the interface that the refresh has completed (whether it be successful or not).
...
- (void)refreshMarcoPolosInBackground:((^)(NSArray *result, NSError *error))callback {
//error checking ommitted
//Run the server call on the global parallel queue
dispatch_async(_serverQueue, ^{
NSArray *objects = nil;
NSError *error = nil;
//This can be any method with the declaration "- (NSArray *)fetchMarcoPolo:(NSError **)callbackError" that connects to a server and returns objects
objects = [self fetchMarcoPolo:&error];
//If something goes wrong, callback the error on the main thread and stop
if(error) {
dispatch_async(dispatch_get_main_queue(), ^{
callback(nil, error);
});
return;
}
//Since the server call was successful, manipulate the data on the serial queue to ensure no thread collisions
dispatch_async(_localQueue, ^{
//Create a mutable copy of our public array to manipulate
NSMutableArray *mutableMarcoPolos = [NSMutableArray arrayWithArray:_marcoPolos];
//PFObject is a class from Parse.com
for(PFObject *parseMarcoPoloObject in objects) {
BOOL shouldAdd = YES;
MPOMarcoPolo *marcoPolo = [[MPOMarcoPolo alloc] initWithParseMarcoPolo:parseMarcoPoloObject];
for(int i = 0; i < _marcoPolos.count; i++) {
MPOMarcoPolo *localMP = _marcoPolos[i];
if([marcoPolo.objectId isEqualToString:localMP.objectId]) {
//Only update the local model if the object pulled from the server was updated more recently than the local object
if((localMP.updatedAt && [marcoPolo.updatedAt timeIntervalSinceDate:localMP.updatedAt] > 0)||
(!localMP.updatedAt)) {
mutableMarcoPolos[i] = marcoPolo;
} else {
NSLog(#"THERE'S NO NEED TO UPDATE THIS MARCO POLO");
}
shouldAdd = NO;
break;
}
}
if(shouldAdd) {
[mutableMarcoPolos addObject:marcoPolo];
}
}
//Perform any sorting on mutableMarcoPolos if needed
//Assign an immutable copy of mutableMarcoPolos to the public data structure
_marcoPolos = [NSArray arrayWithArray:mutableMarcoPolos];
dispatch_async(dispatch_get_main_queue(), ^{
callback(marcoPolos, nil);
});
});
});
}
...
You may be wondering why we would manipulate the data on a queue for something like this, but lets add a method where we can mark the marco as viewed. We don't want to have to wait for the server to update, but we also don't want to manipulate the local object in a manor that can cause a thread collision. So let's add this declaration to the header file:
...
- (void)setMarcoPoloAsViewed:(MPOMarcoPolo *)marcoPolo inBackgroundWithlocalCallback:((^)())localCallback
serverCallback:((^)(NSError *error))serverCallback;
...
Now it's time to implement the method. Notice that the local manipulation is done on the serial queue, then immediately calls back to the main thread, allowing the interface to update without waiting for a server connection. It then updates the server, and calls back to the main thread on a separate callback to inform the interface that the server save was completed.
- (void)setMarcoPoloAsViewed:(MPOMarcoPolo *)marcoPolo inBackgroundWithlocalCallback:(MPOOrderedSetCallback)localCallback
serverCallback:(MPOErrorCallback)serverCallback {
//error checking ommitted
dispatch_async(_localQueue, ^{
//error checking ommitted
//Update local marcoPolo object
for(MPOMarcoPolo *mp in self.marcoPolos) {
if([mp.objectId isEqualToString:marcoPolo.objectId]) {
mp.updatedAt = [NSDate date];
//MPOMarcoPolo objcts have an array viewedUsers that contains all users that have viewed this marco. I use parse, so I'm going to add a MPOUser object that is created from [PFUser currentUser] but this can be any sort of local model manipulation you need
[mp.viewedUsers addObject:[[MPOUser alloc] initWithParseUser:[PFUser currentUser]]];
//callback on the localCallback, so that the interface can update
dispatch_async(dispatch_get_main_queue(), ^{
//code to be executed on the main thread when background task is finished
localCallback(self.marcoPolos, nil);
});
break;
}
}
});
//Update the server on the global parallel queue
dispatch_async(_serverQueue, ^{
NSError *error = nil;
PFObject *marcoPoloParseObject = [marcoPolo parsePointer];
[marcoPoloParseObject addUniqueObject:[PFUser currentUser] forKey:#"viewedUsers"];
//Update marcoPolo object on server
[marcoPoloParseObject save:&error];
if(!error) {
//Marco Polo has been marked as viewed on server. Inform the interface
dispatch_async(dispatch_get_main_queue(), ^{
serverCallback(nil);
});
} else {
//This is a Parse feature that your server's API may not support. If it does not, just callback the error.
[marcoPoloParseObject saveEventually];
NSLog(#"Error: %#", error);
dispatch_async(dispatch_get_main_queue(), ^{
serverCallback(error);
});
}
});
}
With this setup, a refresh can be occuring the background, while setting a marco as viewed at the same time, while ensuring that the local model is not manipulated at the same time. While the necessity of the localQueue may not be obvious with only two methods, when having many different types of manipulation available, it becomes critical.
I use a dataManager that contains two sub managers, core data fetch manager and restkit manager, which maps to core data.
for example:
anywhereInApp.m
[dataManager getData: someSearchPrecate withCompletionBlock: someBlock];
dataManager.m
- (void) getData: somePredicate withCompletionBlock: someblock{
[self.coreDataManager fetchData: somePredicate withCompletionBlock: some block];
[self.restkitManager fetchData: somePredicate withCompletionBlock: some block];
}
and then core data manger runs on a thread to fetch data and executes completion block.
and reskitmanager runs a thread and executes completion block when http request and object mapping complete.
usually the completion block updates the data shown in a collection view.
only need to worry about old data getting removed from core data, but that's another story and can involve comparing the results from the two different calls and taking appropriate action. I try to picture a venn diagram of result sets and it all makes sense or I am too tired & drinking too good of beer.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have 100 URLs, I want to download all these images and save in Documents. For saving I have done, also I can do lazy loading, but I am unable to download all with minimum time and GUI should not hang.
What is suitable method to do so?
Thanks
Use SDWebImage. You can download it from below url
https://github.com/rs/SDWebImage
For load 100 images using Asynchronous request
for(int i=0; i<99; i++)
{
strImage=[[res valueForKey:#"result"] objectAtIndex:i];
if ([[strImage lowercaseString] hasSuffix:#".jpg"] || [[strImage lowercaseString] hasSuffix:#".png"])
{
//[HUD show:YES];
NSURL *url=[[NSURL alloc]initWithString:strImage];
NSURLRequest *urlRequest = [NSURLRequest requestWithURL:url];
[NSURLConnection sendAsynchronousRequest:urlRequest queue:[NSOperationQueue mainQueue] completionHandler:^(NSURLResponse *response, NSData *data, NSError *error)
{
thumbnail.image=[UIImage imageWithData:data];
}];
}
}
A sophisticated solution may become quite complex. Downloading and saving 100 URLs is not as easy as 100 times downloading one image.
The most vexing issue that will arise in ambitious but less smart implementations are memory pressures. A third party network library will not automagically solve this problem for you.
One simple - yet viable - approach which tries to avoid memory problems at the cost of a bit performance, is executing the download and save to disk operation all sequentially. This ensures, that only one image will be handled at any time. Thus, you can make safe assumptions about the maximum memory required for this approach.
A solution may look as below:
Suppose, you have an asynchronous method which loads the image data from a given URL:
typedef void (^load_completion_t)(NSData* data, NSError* error);
- (void) loadURL:(NSURL*)url completion:(load_completion_t)completionHandler;
This method will load the whole image data into memory. This isn't really the best way, but IFF we can assume that one image always fits into memory, it becomes a simple solution.
Furthermore, suppose, there is a synchronous method which saves the data to disk:
- (void) saveData:(NSData*)data;
Now, given an array of URLs you can sequentially load and save a number of images as follows:
typedef void(^completion_t)(id result, NSError* error);
- (void) saveImagesWithURLs:(NSMutableArray*)urls
completion:(completion_t)completionHandler
{
if ([urls count] > 0) {
NSURL* url = [urls firstObject];
[urls removeObjectAtIndex:0];
[self loadURL:url completion:^(NSData* imageData, NSError*error){
if (imageData) {
[self saveData:imageData];
[self saveImagesWithURLs:urls completion:completionHandler];
}
else {
// handle error
}
}];
}
else {
// finished
if (completionHandler) {
completionHandler(#"Images saved", nil);
}
}
}
The method above is an "asynchronous loop":
The completion handler of loadURL:completion will call saveImagesWithURLs:completion:, much like a recursive invocation. However, this is NOT a recursive method: when the completion handler of method saveImagesWithURLs:completion:gets executed, saveImagesWithURLs:completion: already returned.
Given a propert which is an array of URLs:
#property (nonatomic, strong) NSArray* imageURLs;
you can invoke the asynchronous loop as shown below:
[self saveImagesWithURLs:[self.imageURLs mutableCopy]
completion:^(id result, NSError*error){
if (error) {
// handle error
}
else {
// result equals #"Images saved"
}
}];
You can call this method from the main thread. It will NOT block, because it is asynchronous. We also assume, that the completion handlers will be invoked on a private queue, and not on the main thread.
Better user AFNetworking, it will help you to download all the images asynchronously (no GUI hang)
https://github.com/AFNetworking/AFNetworking
You can use AFHTTPClient to enqueueBatchOperations and this has a completionBlock which is called when all operations are finished. Should be exactly what you're looking for.
I'm working on an application that create contents and send it to an existing backend. Content is a title, a picture and location. Nothing fancy.
The backend is a bit complicated so here is what I have to do :
Let the user take a picture, enter a title and authorize the map to use its location
Generate a unique identifier for the post
Create the post on the backend
Upload the picture
Refresh the UI
I've used a couple of NSOperation subclasses to make this work but I'm not proud of my code, here is a sample.
NSOperation *process = [NSBlockOperation blockOperationWithBlock:^{
// Process image before upload
}];
NSOperation *filename = [[NSInvocationOperation alloc] initWithTarget: self selector: #selector(generateFilename) object: nil];
NSOperation *generateEntry = [[NSInvocationOperation alloc] initWithTarget: self selector: #selector(createEntry) object: nil];
NSOperation *uploadImage = [[NSInvocationOperation alloc] initWithTarget: self selector: #selector(uploadImageToCreatedEntry) object: nil];
NSOperation *refresh = [NSBlockOperation blockOperationWithBlock:^{
// Update UI
[SVProgressHUD showSuccessWithStatus: NSLocalizedString(#"Success!", #"Success HUD message")];
}];
[refresh addDependency: uploadImage];
[uploadImage addDependency: generateEntry];
[generateEntry addDependency: filename];
[generateEntry addDependency: process];
[[NSOperationQueue mainQueue] addOperation: refresh];
[_queue addOperations: #[uploadImage, generateEntry, filename, process] waitUntilFinished: NO];
Here are the things I don't like :
in my createEntry: for example, I'm storing the generated filename in a property, which mees with the global scope of my class
in the uploadImageToCreatedEntry: method, I'm using dispatch_async + dispatch_get_main_queue() to update the message in my HUD
etc.
How would you manage such workflow ? I'd like to avoid embedding multiple completion blocks and I feel like NSOperation really is the way to go but I also feel there is a better implementation somewhere.
Thanks!
You can use ReactiveCocoa to
accomplish this pretty easily. One of its big goals is to make this kind of
composition trivial.
If you haven't heard of ReactiveCocoa before, or are unfamiliar with it, check
out the Introduction
for a quick explanation.
I'll avoid duplicating an entire framework overview here, but suffice it to say that
RAC actually offers a superset of promises/futures. It allows you to compose and
transform events of completely different origins (UI, network, database, KVO,
notifications, etc.), which is incredibly powerful.
To get started RACifying this code, the first and easiest thing we can do is put
these separate operations into methods, and ensure that each one returns
a RACSignal. This isn't strictly necessary (they could all be defined within
one scope), but it makes the code more modular and readable.
For example, let's create a couple signals corresponding to process and
generateFilename:
- (RACSignal *)processImage:(UIImage *)image {
return [RACSignal startEagerlyWithScheduler:[RACScheduler scheduler] block:^(id<RACSubscriber> subscriber) {
// Process image before upload
UIImage *processedImage = …;
[subscriber sendNext:processedImage];
[subscriber sendCompleted];
}];
}
- (RACSignal *)generateFilename {
return [RACSignal startEagerlyWithScheduler:[RACScheduler scheduler] block:^(id<RACSubscriber> subscriber) {
NSString *filename = [self generateFilename];
[subscriber sendNext:filename];
[subscriber sendCompleted];
}];
}
The other operations (createEntry and uploadImageToCreatedEntry) would be very similar.
Once we have these in place, it's very easy to compose them and express their
dependencies (though the comments make it look a bit dense):
[[[[[[self
generateFilename]
flattenMap:^(NSString *filename) {
// Returns a signal representing the entry creation.
// We assume that this will eventually send an `Entry` object.
return [self createEntryWithFilename:filename];
}]
// Combine the value with that returned by `-processImage:`.
zipWith:[self processImage:startingImage]]
flattenMap:^(RACTuple *entryAndImage) {
// Here, we unpack the zipped values then return a single object,
// which is just a signal representing the upload.
return [self uploadImage:entryAndImage[1] toCreatedEntry:entryAndImage[0]];
}]
// Make sure that the next code runs on the main thread.
deliverOn:RACScheduler.mainThreadScheduler]
subscribeError:^(NSError *error) {
// Any errors will trickle down into this block, where we can
// display them.
[self presentError:error];
} completed:^{
// Update UI
[SVProgressHUD showSuccessWithStatus: NSLocalizedString(#"Success!", #"Success HUD message")];
}];
Note that I renamed some of your methods so that they can accept inputs from
their dependencies, giving us a more natural way to feed values from one
operation to the next.
There are huge advantages here:
You can read it top-down, so it's very easy to understand the order that
things happen in, and where the dependencies lie.
It's extremely easy to move work between different threads, as evidenced by
the use of -deliverOn:.
Any errors sent by any of those methods will automatically cancel all the
rest of the work, and eventually reach the subscribeError: block for easy
handling.
You can also compose this with other streams of events (i.e., not just
operations). For example, you could set this up to trigger only when a UI
signal (like a button click) fires.
ReactiveCocoa is a huge framework, and it's unfortunately hard to distill the
advantages down into a small code sample. I'd highly recommend checking out the
examples for when to use
ReactiveCocoa
to learn more about how it can help.
A couple of thoughts:
I would be inclined to avail myself of completion blocks because you probably only want to initiate the next operation if the previous one succeeded. You want to make sure that you properly handle errors and can easily break out of your chain of operations if one fails.
If I wanted to pass data from operation to another and didn't want to use some property of the caller's class, I would probably define my own completion block as a property of my custom operation that had a parameter which included the field that I wanted to pass from one operation to another. This assumes, though, that you're doing NSOperation subclassing.
For example, I might have a FilenameOperation.h that defines an interface for my operation subclass:
#import <Foundation/Foundation.h>
typedef void (^FilenameOperationSuccessFailureBlock)(NSString *filename, NSError *error);
#interface FilenameOperation : NSOperation
#property (nonatomic, copy) FilenameOperationSuccessFailureBlock successFailureBlock;
#end
and if it wasn't a concurrent operation, the implementation might look like:
#import "FilenameOperation.h"
#implementation FilenameOperation
- (void)main
{
if (self.isCancelled)
return;
NSString *filename = ...;
BOOL failure = ...
if (failure)
{
NSError *error = [NSError errorWithDomain:... code:... userInfo:...];
if (self.successFailureBlock)
self.successFailureBlock(nil, error);
}
else
{
if (self.successFailureBlock)
self.successFailureBlock(filename, nil);
}
}
#end
Clearly, if you have a concurrent operation, you'll implement all of the standard isConcurrent, isFinished and isExecuting logic, but the idea is the same. As an aside, sometimes people will dispatch those success or failures back to the main queue, so you can do that if you want, too.
Regardless, this illustrates the idea of a custom property with my own completion block that passes the appropriate data. You can repeat this process for each of the relevant types of operations, you can then chain them all together, with something like:
FilenameOperation *filenameOperation = [[FilenameOperation alloc] init];
GenerateOperation *generateOperation = [[GenerateOperation alloc] init];
UploadOperation *uploadOperation = [[UploadOperation alloc] init];
filenameOperation.successFailureBlock = ^(NSString *filename, NSError *error) {
if (error)
{
// handle error
NSLog(#"%s: error: %#", __FUNCTION__, error);
}
else
{
generateOperation.filename = filename;
[queue addOperation:generateOperation];
}
};
generateOperation.successFailureBlock = ^(NSString *filename, NSData *data, NSError *error) {
if (error)
{
// handle error
NSLog(#"%s: error: %#", __FUNCTION__, error);
}
else
{
uploadOperation.filename = filename;
uploadOperation.data = data;
[queue addOperation:uploadOperation];
}
};
uploadOperation.successFailureBlock = ^(NSString *result, NSError *error) {
if (error)
{
// handle error
NSLog(#"%s: error: %#", __FUNCTION__, error);
}
else
{
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// update UI here
NSLog(#"%#", result);
}];
}
};
[queue addOperation:filenameOperation];
Another approach in more complicated scenarios is to have your NSOperation subclass employ a technique analogous to how the standard addDependency method works, in which NSOperation sets the isReady state based upon KVO on isFinished on the other operation. This not only allows you to not only establish more complicated dependencies between operations, but also to pass database between them. This is probably beyond the scope of this question (and I'm already suffering from tl:dr), but let me know if you need more here.
I wouldn't be too concerned that uploadImageToCreatedEntry is dispatching back to the main thread. In complicated designs, you might have all sorts of different queues dedicated for particular types of operations, and the fact that UI updates are added to the main queue is perfectly consistent with this mode. But instead of dispatch_async, I might be inclined to use the NSOperationQueue equivalent:
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// do my UI update here
}];
I wonder if you need all of these operations. For example, I have a hard time imagining that filename is sufficiently complicated to justify its own operation (but if you're getting the filename from some remote source, then a separate operation makes perfect sense). I'll assume that you're doing something sufficiently complicated that justifies it, but the names of those operations make me wonder, though.
If you want, you might want to take a look at couchdeveloper's RXPromise class which uses promises to (a) control the logical relationship between separate operations; and (b) simplify the passing of data from one to the next. Mike Ash has a old MAFuture class which does the same thing.
I'm not sure either of those are mature enough that I'd contemplate using them in my own code, but it's an interesting idea.
I'm probably totally, biased - but for a particular reason - I like #Rob's approach #6 ;)
Assuming you created appropriate wrappers for your asynchronous methods and operations which return a Promise instead of signaling the completion with a completion block, the solution looks like this:
RXPromise* finalResult = [RXPromise all:#[[self filename], [self process]]]
.then(^id(id filenameAndProcessResult){
return [self generateEntry];
}, nil)
.then(^id(id generateEntryResult){
return [self uploadImage];
}, nil)
.thenOn(dispatch_get_main_queue() , ^id(id uploadImageResult){
[self refreshWithResult:uploadImageResult];
return nil;
}, nil)
.then(nil, ^id(NSError*error){
// Something went wrong in any of the operations. Log the error:
NSLog(#"Error: %#", error);
});
And, if you want to cancel the whole asynchronous sequence at any tine, anywhere and no matter how far it has been proceeded:
[finalResult.root cancel];
(A small note: property root is not yet available in the current version of RXPromise, but its basically very simple to implement).
If you still want to use NSOperation, you can rely on ProcedureKit and use the injection properties of the Procedure class.
For each operation, specify which type it produces and inject it to the next dependent operation. You can also at the end wrap the whole process inside a GroupProcedure class.
i'm using the amazing FMDB project in my app in development, i have a NSOperation like this:
- (void)main
{
#autoreleasepool {
FMDatabaseQueue *queue = [FMDatabaseQueue databaseQueueWithPath:[[NSUserDefaults standardUserDefaults] valueForKey:#"pathDB"]];
[queue inDatabase:^(FMDatabase *db) {
FMResultSet *toQuery;
if (self._id == nil) {
toQuery = [db executeQuery:#"SELECT id,language,update_time FROM task"];
while ([toQuery next]) {
[myarray addObject:[toQuery resultDictionary]];
}
}];
for (int i = 0; i<[myarray count]; i++){
...Do Something
[queue inDatabase:^(FMDatabase *db) {
FMResultSet *checkImgQuery = [db executeQuery:#"SELECT img_url,img_path FROM task WHERE id = ? AND language = ?",myTask.id ,myTask.lang];
while ([checkImgQuery next]) {
if (![[checkImgQuery stringForColumn:#"img_url"] isEqualToString:myTask.img]) {
NSData *my_img = [NSData dataWithContentsOfURL:[NSURL URLWithString:myTask.img]];
if (my_img != nil) {
NSError *er;
[my_img writeToFile:[checkImgQuery stringForColumn:#"img_path"] options:NSDataWritingAtomic error:&er];
//In the line under here the code block, the app still running, but this operation doesn't
//go over this task
[db executeUpdate:#"UPDATE task SET img_url = ? WHERE id = ? AND language = ?",myTask.img,[NSNumber numberWithInt:myTask.id],[NSNumber numberWithInt:myTask.language];
NSLog(#"%#",[db lastErrorMessage]);
}
...Do Something
}
}
}
}];
}
}
The problem is in [db executeUpdate:...] that sometime works with no problem and sometime freeze and doesn't go over that line, the NSLog i have put there doesn't print anything, the app doesn't crash and continue working, but the thread is stuck there, if i shutdown the run of the app, and i restart it again the thread doesn't stop on the same task, but random on another, with no criteria, some time one works, and some time doesn't...anyone can help?
There are a couple issues that leap out at me, one or more of which may be contributing to your problem:
I notice that you're creating a FMDatabaseQueue object locally. You should only have one FMDatabaseQueue object shared for the entire app (I put it in a singleton). The purpose of the database queue is to coordinate database interactions, and it can't reasonably do that if you're creating new FMDatabaseQueue objects all over the place.
I'd advise against having an inDatabase block in which you're synchronously downloading a bunch of images from the network.
When you submit an inDatabase task, any inDatabase calls on other threads using the same FMDatabaseQueue (and they should use the same queue, or else you're defeating the purpose in having a queue in the first place) will not proceed until the one running in your operation does (or vice versa).
When doing database interaction from multiple threads, coordinated by the FMDatabaseQueue serial queue, you really want to make sure that you "get in and get out" as quickly as possible. Don't embed potentially slow network calls in the middle of the inDatabase block, or else all other database interaction will be blocked until it finishes.
So, do an inDatabase to identify the images that need to be downloaded, but that's it. Then outside of the inDatabase call, retrieve your images, and if you need to update image paths or the like, separate inDatabase call do to do that. But don't include anything slow and synchronous inside the inDatabase block.
I also notice that you're doing a SELECT on task table, keeping that FMRecordSet open, and then trying to update the same record. You want to open your record set, retrieve what you need, and close that recordset before you try to update the same record you retrieved in your recordset.
Always close the FMResultSet before you try to do the executeUpdate that updates the same record.
A bit unrelated, but I might suggest you consider including img_url and img_path in your original SELECT statement, that way your array of dictionary entries will already have everything you need and it saves you from have to do that second SELECT at all.
If you're wondering what the FMDatabaseQueue singleton might look like, you might have a DatabaseManager singleton whose interface looks like:
// DatabaseManager.h
#import Foundation;
#import SQLite3;
#import "FMDB.h"
NS_ASSUME_NONNULL_BEGIN
#interface DatabaseManager : NSObject
#property (nonatomic, strong, readonly) FMDatabaseQueue *queue;
#property (class, readonly, strong) DatabaseManager *sharedManager;
- (id)init __attribute__((unavailable("Use +[DatabaseManager sharedManager] instead")));
+ (id)new __attribute__((unavailable("Use +[DatabaseManager sharedManager] instead")));
#end
NS_ASSUME_NONNULL_END
and the implementation might look like:
// DatabaseManager.m
#import "DatabaseManager.h"
#implementation DatabaseManager
+ (DatabaseManager *)sharedManager {
static id sharedMyManager = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
sharedMyManager = [[self alloc] init];
});
return sharedMyManager;
}
- (id)init {
if ((self = [super init])) {
_queue = [[FMDatabaseQueue alloc] initWithPath:[[NSUserDefaults standardUserDefaults] valueForKey:#"pathDB"]];
}
return self;
}
#end
Then, any code that needs to interact with the database can retrieve the queue like so:
FMDatabaseQueue *queue = DatabaseManager.sharedManager.queue;