I have a cache of files in which I need to write/read images.
All the work with the file system I need to perform in background.
For this purposes for saving files I use:
dispatch_barrier_async([AxPanoramaDataManager sharedInstance].dataManagerQueue, ^{
[data writeToFile:tileFilePathName atomically:YES];
});
And for reading:
__block UIImage *tileImage = nil;
dispatch_sync([AxPanoramaDataManager sharedInstance].dataManagerQueue, ^{
tileImage = [[UIImage imageWithContentsOfFile:tileFilePathName] retain];
dispatch_async(dispatch_get_main_queue(), ^{
completion (tileCoordValue, side, tileImage, error);
[tileImage release];
});
});
Everything works well, there is a great amount of files in cash folder, but I sometimes I need to clean the cash. I need to cancel all the queue blocks and perform a block with cleaning the folder.
The first my realization of the method looks like this:
+ (void) cleanCash
{
NSString *folderPath = [[NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0] stringByAppendingString:#"cash"];
dispatch_sync([AxPanoramaDataManager sharedInstance].dataManagerQueue, ^{
NSError *error = nil;
[[NSFileManager defaultManager] removeItemAtPath:folderPath error:&error];
NSParameterAssert(error == nil);
}
});
}
But I have a numerous problems with it because of it is not cancel the all the waiting operations in queue. I try to look solutions in SO but can't implement them unfortunately. Can anyone help me with this?
You can use NSOperationQueue with maxConcurrentOperationCount = 1 (it will make a serial queue) for run NSOperation that will load your images;
NSOperation is a cancelable object. Also you can make dependencies between operations.
P.S.
You can view SDWebImage, it contains SDImageCache class. It class is very good for manage cache of images.
Related
I am curious if it is possible to easily assign and retain __block objects inside asynchronous blocks in MRC and ARC? I obtained the following code while rewriting a piece of code with minimal changes. I got stuck when trying to return the image from an asynchronous block. A concern is that the code would one day be converted to ARC. I do not want to have hidden memory crash after the conversion. My options were
Using a GCD object holder if such a thing exists
Using a homebrew object holder, an array or others
Directly adding the image to the array (which I am using)
Rewriting the code to another structure
Basically the code load multiple images in a background thread. Searching // image deallocated :( will give you the location.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
ALAssetsLibrary *assetsLibrary = nil;
for (int i = 0; i < sources.count; ++i) {
__block UIImage *image = nil;
id source = sources[i];
if ([source isKindOfClass:[NSString class]]) {
image = something
} else if ([source isKindOfClass:[NSURL class]]) {
NSURL *url = source;
if ([url.scheme isEqualToString:#"assets-library"]) {
dispatch_group_t group = dispatch_group_create();
if (!assetsLibrary)
assetsLibrary = [[[ALAssetsLibrary alloc] init] autorelease];
dispatch_group_enter(group);
[assetsLibrary assetForURL:url resultBlock:^(ALAsset *asset) {
image = something
dispatch_group_leave(group);
} failureBlock:^(NSError *error) {
dispatch_group_leave(group);
}];
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
// image deallocated :(
} else if (url.isFileURL) {
image = something
}
}
// add image to an array
}
dispatch_async(dispatch_get_main_queue(), ^{
// notify
});
});
I think moving the image variable declaration would be the first right step to think about this. You can't expect to have one image variable for multiple concurrent operations.
so to get started, let's do this-
ALAssetsLibrary *assetsLibrary = nil;
for (int i = 0; i < sources.count; ++i) {
__block UIImage *image = nil;
After that, you are currently keeping UIImage instances in an array? This is not recommended. Please find out a way to write these images to documents directory and later read from there when needed. Keeping UIImage instances in memory will lead to severe memory overuse issues.
You need to come up with a naming convention to write your images into documents directory. Whenever an image load completes, just save it to your local folder and later read from there.
You could also call the notify completion part as soon as one image is loaded and keep notifying about others as you get them. Not sure how your design is working.
Also, I noticed you are creating multiple groups. One new group with each iteration inside for loop here-
dispatch_group_t group = dispatch_group_create();
Moving it outside for loop would be a good call.
Hope this helps.
I created an overload method for NSLog in order to write all logs in an HTML file.
All logs are written in the HTML file using GCD.
The problem is that lines are sometimes truncated...
Here my code :
Write in log file function :
+(void)writeInLogFile:(NSString *)strLog inFolder:(NSString *)folder fileName:(NSString *)fileName extension:(NSString *)extension{
//Create Directory
NSString *path;
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
path = [[paths objectAtIndex:0] stringByAppendingPathComponent:folder];
NSError *error;
if (![[NSFileManager defaultManager] fileExistsAtPath:path]) //Does directory already exists?
{
if (![[NSFileManager defaultManager] createDirectoryAtPath:path withIntermediateDirectories:YES attributes:nil error:&error])
{
NSLog(#"%d||Create log directory error: %#",LOG_SEVERITY_HIGH, error);
}
}
NSString* filePath = [NSString stringWithFormat:#"%#/%#.%#",path,fileName,extension];
int nbOfLogFiles = [[[NSFileManager defaultManager] contentsOfDirectoryAtPath:path error:nil] count];
if(nbOfLogFiles <= NB_OF_LOG_FILES_BEFORE_PURGE +1){
[HandleString createLogFile:filePath andStrLog:strLog];
}
else{
[HandleString purgeLogDirectory:path];
[HandleString createLogFile:filePath andStrLog:strLog];
}
}
That is the result :
And the call (called each time an NSLog is executed):
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
[HandleString writeInLogFile:message];
});
That is the result :
As you can see, the last line is truncated...
This appens throughout the file.
I tried to run the process on the main thread and it works well without problems.
Another interesting thing when i change the QOS the result isn't the same, for exemple whith priority high, i have more truncated lines.
Edit : The code to write in file :
+(void)createLogFile:(NSString*)filePath andStrLog:(NSString*)strLog{
if(![[NSFileManager defaultManager] fileExistsAtPath:filePath]){
[[NSFileManager defaultManager] createFileAtPath:filePath contents:nil attributes:nil];
}
NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath];
[fileHandle seekToEndOfFile];
[fileHandle writeData:[strLog dataUsingEncoding:NSUTF8StringEncoding]];
[fileHandle closeFile];
}
The issue is that you're writing from multiple threads simultaneously. Consider this sequence of events:
Thread A seeks to end of file
Thread B seeks to end of file
Thread B writes its data
Thread A writes its data
Thread A's data will overwrite Thread B's data. If Thread A's data is longer, then no trace of Thread B's data will be left. If Thread A's data is shorter, then the part of Thread B's data beyond that length will remain.
Swapping the order of the last two steps has a similar problem. And there are, of course, more complicated scenarios with more threads.
One solution is to serialize all accesses to the file, as you've done in your self-answer. Another is to open the file in append-only mode. In this mode, all writes are done at the end of the file. This is enforced by the OS. There's no way for two simultaneous writes to overwrite each other. The current file position of the file handle is irrelevant, so there's no need to seek.
NSFileHandle doesn't directly support opening a file in append-only mode. You have to use open() to open a file descriptor and then hand that file descriptor off to NSFileHandle. (You could also just use write() and close(), but it's a bit messier.) For example:
int fd = open(filePath.fileSystemRepresentation, O_WRONLY | O_APPEND | O_CREAT, S_IRUSR | S_IWUSR);
if (fd < 0)
{
// handle error
}
NSFileHandle* fileHandle = [[NSFileHandle alloc] initWithFileDescriptor:fd closeOnDealloc:YES];
[fileHandle writeData:[strLog dataUsingEncoding:NSUTF8StringEncoding]];
[fileHandle closeFile];
[fileHandle release]; // Only needed if not using ARC
Note that you don't have to explicitly create the log file in this case. The open() call will create it if it doesn't exist, because of the O_CREAT flag.
Given the phrase "I tried to run the process on the main thread and it works well without problems." I would say that your truncation problem comes from a buffer being closed before completely empty. That does not explain the disparity you might have observed while using high priority (remember that the extra lines might be coincidental until proven otherwise...).
Now, I would recommend you to try a small sleep before ending the thread scope...
I found a solution a lil bit tricky :
I removed the dispatch_asynch :
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
[HandleString writeInLogFile:message];
});
And in my write in log file function i did :
static dispatch_queue_t asyncQueue;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
asyncQueue = dispatch_queue_create("asyncQueue", NULL);
});
if(nbOfLogFiles <= NB_OF_LOG_FILES_BEFORE_PURGE +1){
dispatch_async(asyncQueue, ^{
[HandleString createLogFile:filePath andStrLog:strLog];
});
}
else{
dispatch_async(asyncQueue, ^{
[HandleString purgeLogDirectory:path];
[HandleString createLogFile:filePath andStrLog:strLog];
});
}
It could may be help someone.
But i have to know if according to you, it is a good solution?
John
This is the function to download images , in this function i have to update the UI, the UILabels that shows total images and counter value (downloaded images).
thats why i am calling a thread.
-(void) DownloadImages
{
NSLog(#"Images count to download %lu",(unsigned long)[productID count]);
if ([productID count] > 0)
{
// Document Directory
NSString *myDirectory = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
for (i = 0; i < [productID count]; i++)
{
[NSThread detachNewThreadSelector: #selector(UpdateLabels) toTarget:self withObject:nil];
// [self UpdateLabels]; this does not work….
NSLog(#"image no %d", i);
//[self Status];
NSString *imageURL = [NSString stringWithFormat:#"http://serverPath/%#/OnlineSale/images/products/%#img.jpg",clientSite,[productID objectAtIndex:i]];
UIImage *image = [[UIImage alloc] initWithData:[NSData dataWithContentsOfURL:[NSURL URLWithString:imageURL]]];
// Reference path
NSString *imagePath = [NSString stringWithFormat:#"%#/%#-1-lrg.jpg",myDirectory,[productID objectAtIndex:i]];
NSData *imageData = [NSData dataWithData:UIImageJPEGRepresentation(image, 1.0f)];
[imageData writeToFile:imagePath atomically:YES];
}
NSLog(#"Downloading images completed...");
}
}
Problem:
- here for every new image i have to call every time a new NSThread, there may be thousands of images and it looks bad that there will be thousands threads.
in simulator i have tested it for 3000 images, but in my iPad when testing it gives the error,,,
App is Exited and terminated due to memory pressure.
i guess the error is due to this calling approach of new thread every time.
what i am doing wrong here.?????????????
i have searched there are two different options but i don't know which one is right option,
Options are:
- [self performSelectorInBackground:#selector(UpdateLabels) withObject:nil];
- [NSThread detachNewThreadSelector: #selector(UpdateLabels) toTarget:self withObject:nil];
and i also want to know that if i use
[NSThread cancelPreviousPerformRequestsWithTarget:self];
just after the call of selector, is it right approach or not..???
My suggestion is to use NSOperationQueue . you could then set the maxConcurrentOperationCount property to whatever you like.
In addition in your case is the best approach.
Make some research on what are your maxConcurrentOperationCount with how many concurrent thread your iphone could run without any issue.
NSOperationQueue will manage this for you run new thread with this limitation.
It should look something like this:
NSOperationQueue *myQueue = [[NSOperationQueue alloc] init];
[myQueue setMaxConcurrentOperationCount:500];
for (i = 0; i < [productID count]; i++) {
[myQueue addOperationWithBlock:^{
// you image download here
}];
}
but check apple reference first.
I'm essentially loading images from the server every time a user taps the refresh button. However, this loading mechanism freezes the main thread, so I decided to try and use GCD to load the images in a different thread. I've looked at tutorials, and it seems that I'm setting it up right, but I'm still encountering the same issue.
My code:
-(IBAction)refreshButton:(id)sender{
dispatch_queue_t myQueue = dispatch_queue_create("My Queue",NULL);
dispatch_async(myQueue, ^{
[[API sharedInstance] commandWithParams:[NSMutableDictionary dictionaryWithObjectsAndKeys:
#"stream",#"command",#"0",#"IdImage",_contentFilter,#"contentFilter",
nil]
onCompletion:^(NSDictionary *json){
NSMutableArray *tempArray = [json objectForKey:#"result"];
NSMutableArray *tempDataArray = [[NSMutableArray alloc] initWithCapacity:[_imagesTemporaryHolder count]];
NSLog(#"loaded temporary image holder");
for (int i=0;i<[_images count];i++) {
NSLog(#" populating _imagesData");
NSString *imageID = [[tempArray objectAtIndex:i] objectForKey:#"IdImage"];
NSString *imageURL = [NSString stringWithFormat:#"http://swiphtapp.com/Swipht/upload/%#.jpg",imageID];
NSData *tempImageData = [[NSData alloc] initWithContentsOfURL: [NSURL URLWithString:imageURL]];
[tempDataArray insertObject:tempImageData atIndex:i];
}
}];
dispatch_async(dispatch_get_main_queue(), ^{
// Update the UI
});
});
}
Could it be that I need to go to the API class and change something there? I'm using AFNNETWORKING by the way.
Thank you so much for the help!
UPDATE 1:
It looks like the API's "onCompletion" block is what's causing the problem...I did the NSLog(isMainThread) check in it, and it returns YES meaning that it's running in the main thread although it shouldn't be as I've encapsulated it in the GCD. Thoughts on how to solve this?
ANSWERED THANKS TO AWESOME PEOPLE
#eric
The completion block of the API call is meant to be performed on the main thread, because it's where you would normally update the views based on the data received. Try placing your GCD stuff INSIDE the completion block instead of encapsulating the whole thing in it. It's taking so long on the main thread because you are getting the NSData from the url returned from your original API call, which can take a long time (relatively speaking)
Basically the GCD needed to be in the API call's completion block.
Thanks again!
I believe this is the answer we came to in the comments under the initial post:
-(IBAction)refreshButton:(id)sender
{
[[API sharedInstance] commandWithParams:[NSMutableDictionary dictionaryWithObjectsAndKeys:
#"stream",#"command",#"0",#"IdImage",_contentFilter,#"contentFilter",
nil]
onCompletion:^(NSDictionary *json){
dispatch_queue_t myQueue = dispatch_queue_create("My Queue",NULL);
dispatch_async(myQueue, ^{
NSMutableArray *tempArray = [json objectForKey:#"result"];
NSMutableArray *tempDataArray = [[NSMutableArray alloc] initWithCapacity:[_imagesTemporaryHolder count]];
NSLog(#"loaded temporary image holder");
for (int i=0;i<[_images count];i++) {
NSLog(#" populating _imagesData");
NSString *imageID = [[tempArray objectAtIndex:i] objectForKey:#"IdImage"];
NSString *imageURL = [NSString stringWithFormat:#"http://swiphtapp.com/Swipht/upload/%#.jpg",imageID];
NSData *tempImageData = [[NSData alloc] initWithContentsOfURL: [NSURL URLWithString:imageURL]];
[tempDataArray insertObject:tempImageData atIndex:i];
}
dispatch_async(dispatch_get_main_queue(), ^{
// Update the UI
});
});
}];
}
I am working on an iOS app which dispatch quite a number of tasks to my serial queue. The task is to download images from my web server, save it to disk, and later displayed on UIImageView. However, [NSURLConnection sendAsynchrousRequest] will keep eating up more and more memory until iOS kill my process.
The downloader method looks like this:
// dispatch_queue_t is created once by: m_pRequestQueue = dispatch_queue_create( "mynamespace.app", DISPATCH_QUEUE_SERIAL);
- (void) downloadImageInBackgroundWithURL:(NSString*) szUrl {
__block typeof(self) bSelf = self;
__block typeof(m_pUrlRequestQueue) bpUrlRequestQueue = m_pRequestQueue;
dispatch_async( m_pRequestQueue, ^{
NSAutoreleasePool *pAutoreleasePool = [[NSAutoreleasePool alloc] init];
NSURLRequest *pRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:szUrl]
cachePolicy:NSURLRequestReloadIgnoringCacheData
timeoutInterval:URL_REQUEST_TIMEOUT];
[NSURLConnection sendAsynchronousRequest:pRequest queue:bpUrlRequestQueue completionHandler:^(NSURLResponse *pResponse, NSData *pData, NSError *pError) {
NSAutoreleasePool *pPool = [[NSAutoreleasePool alloc] init];
if ( pError != nil ) {
} else {
// convert image to png format
UIImage *pImg = [UIImage imageWithData:pData];
NSData *pDataPng = UIImagePNGRepresentation(pImg);
bool bSaved = [[NSFileManager defaultManager] createFileAtPath:szCacheFile contents:pDataPng attributes:nil];
}
__block typeof(pDataPng) bpDataPng = pDataPng;
__block typeof(pError) bpError = pError;
dispatch_sync( dispatch_get_main_queue(), ^ {
NSAutoreleasePool *autoreleasepool = [[NSAutoreleasePool alloc] init];
UIImage *pImage = [[UIImage alloc] initWithData:bpDataPng];
// display the image
[pImage release];
// NSLog( #"image retain count: %d", [pImage retainCount] ); // 0, bad access
[autoreleasepool drain];
});
}
[pPool drain];
}]; // end sendAsynchronousRequest
[pAutoreleasePool drain];
}); // end dispatch_async
} // end downloadImageInBackgroundWithURL
I am quite sure it is something inside [NSURLConnection sendAsynchronousRequest] as the profiler is showing that the function is the one eating up all the memory...
However, I am also not very sure about the dispatch_*** and block things, I've always used C and C++ code with pthread before, but after reading from Apple's documentation on migrating away from thread, I decided to give GCD a try, objective-c is so troublesome and I'm not sure how to release the NSData *pData and NSURLResponse *pResponse as it crash whenever I do it.
Please advice... really need help to learn and appreciate objective-c...
ADDITIONAL EDIT:
Thanks to #robhayward, I put the pImg and pDataPng outside as __block variable, use his RHCacheImageView way of downloading data ( NSData initWithContentOfURL )
Thanks as well to #JorisKluivers, the first UIImage can actually be reused to display as UIImageView recognized both jpg and png format, just my later processing requires png format and I am reading from the disk later just when required
I would firstly put it down to the image and data objects that you are creating:
UIImage *pImg = [UIImage imageWithData:pData];
NSData *pDataPng = UIImagePNGRepresentation(pImg);
Which might be hanging around too long, perhaps put them outside the block, as they are probably being created/released on different threads:
__block UIImage *pImg = nil;
__block NSData *pDataPng = nil;
[NSURLConnection sendAsynchronousRequest..
(Also consider using ARC if you can)
I have some code on Github that does a similar job without this issue, feel free to check it out:
https://github.com/robinhayward/RHCache/blob/master/RHCache/RHCache/Helpers/UIImageView/RHCacheImageView.m
First of all try simplifying your code. Things I did:
Remove the outer dispatch_async. This is not needed, your sendAsynchronousRequest is async already. This also removes the need another __block variable on the queue.
You create an image named pImg from the received pData, then convert that back to NSData of type png, and later create another image pImage from that again. Instead of converting over and over, just reuse the first image. You could even write the original pData to disk (unless you really want the png format on disk).
I didn't compile the code below myself, so it might contain a few mistakes. But it is a simpler version that might help solve the leak.
- (void) downloadImageInBackgroundWithURL:(NSString*)szUrl
{
__block typeof(self) bSelf = self;
NSAutoreleasePool *pAutoreleasePool = [[NSAutoreleasePool alloc] init];
NSURLRequest *pRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:szUrl]
cachePolicy:NSURLRequestReloadIgnoringCacheData
timeoutInterval:URL_REQUEST_TIMEOUT];
[NSURLConnection sendAsynchronousRequest:pRequest queue:m_pRequestQueue completionHandler:^(NSURLResponse *pResponse, NSData *pData, NSError *pError) {
NSAutoreleasePool *pPool = [[NSAutoreleasePool alloc] init];
if (pError) {
// TODO: handle error
return;
}
// convert image to png format
__block UIImage *pImg = [UIImage imageWithData:pData];
// possibly just write pData to disk
NSData *pDataPng = UIImagePNGRepresentation(pImg);
bool bSaved = [[NSFileManager defaultManager] createFileAtPath:szCacheFile contents:pDataPng attributes:nil];
dispatch_sync( dispatch_get_main_queue(), ^ {
// display the image in var pImg
});
}];
[pAutoreleasePool drain];
}