I'm relatively new to the Metal, and I have a pretty straight-forward question. I simply can not init MTLBuffer with MTLStorageModePrivate option:
id<MTLBuffer> privateBuff = [device newBufferWithLength:dataLength options:MTLStorageModePrivate];
Compiler gives me an assert with that text:
-[MTLDebugDevice validateResourceOptions:isTexture:isIOSurface:]:437: failed assertion `options 0x2 conveys invalid cpuCacheMode of 0x2'
And it doesn't make much sense. I'm creating a buffer which can be accessed only from GPU, so I need no cpu cache modes whatsoever for this particular entity. I suppose I need to turn off that cpu cache mode, but how?
I looked through MTLCPUCacheMode, but it has nothing regarding turning cpu cache mode completely off.
Interesting note: I absolutely can create MTLHeap with MTLStorageModePrivate, but not MTLBuffer.
Any help would be appreciated. Thanks in advance!
UPDATE: I can create MTLBuffer with MTLStorageModePrivate by using MTLHeap. It looks something like this:
MTLHeapDescriptor *heapDescriptor = [MTLHeapDescriptor new];
heapDescriptor.storageMode = MTLStorageModePrivate;
heapDescriptor.size = 0;
MTLSizeAndAlign sizeAndAlign = [device heapBufferSizeAndAlignWithLength:lutSharedBuffer.length options:MTLResourceStorageModePrivate];
sizeAndAlign.size += (sizeAndAlign.size & (sizeAndAlign.align - 1)) + sizeAndAlign.align;
heapDescriptor.size += sizeAndAlign.size;
privateHeap = [device newHeapWithDescriptor:heapDescriptor];
privateBuff = [privateHeap newBufferWithLength:lutSharedBuffer.length options:MTLResourceStorageModePrivate]; //now it works!
But it's still not possible to do without the heap.
The issue here seems to be that you're using the incorrect enum to specify your resource options. In your first snippet, you use MTLStorageModePrivate, but you should be using MTLResourceStorageModePrivate, which contains a bit shift to place the storage mode in the correct bits.
MTLResourceStorageModePrivate = MTLStorageModePrivate << MTLResourceStorageModeShift
In Swift, this would have caused a compile-time error.
Related
I have one (or possibly two) CVPixelBufferRef objects I am processing on the CPU, and then placing the results onto a final CVPixelBufferRef. I would like to do this processing on the GPU using GLSL instead because the CPU can barely keep up (these are frames of live video). I know this is possible "directly" (ie writing my own open gl code), but from the (absolutely impenetrable) sample code I've looked at it's an insane amount of work.
Two options seem to be:
1) GPUImage: This is an awesome library, but I'm a little unclear if I can do what I want easily. First thing I tried was requesting OpenGLES compatible pixel buffers using this code:
#{ (NSString *)kCVPixelBufferPixelFormatTypeKey : [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA],
(NSString*)kCVPixelBufferOpenGLESCompatibilityKey : [NSNumber numberWithBool:YES]};
Then transferring data from the CVPixelBufferRef to GPUImageRawDataInput as follows:
// setup:
_foreground = [[GPUImageRawDataInput alloc] initWithBytes:nil size:CGSizeMake(0,0)pixelFormat:GPUPixelFormatBGRA type:GPUPixelTypeUByte];
// call for each frame:
[_foreground updateDataFromBytes:CVPixelBufferGetBaseAddress(foregroundPixelBuffer)
size:CGSizeMake(CVPixelBufferGetWidth(foregroundPixelBuffer), CVPixelBufferGetHeight(foregroundPixelBuffer))];
However, my CPU usage goes from 7% to 27% on an iPhone 5S just with that line (no processing or anything). This suggests there's some copying going on on the CPU, or something else is wrong. Am I missing something?
2) OpenFrameworks: OF is commonly used for this type of thing, and OF projects can be easily setup to use GLSL. However, two questions remain about this solution: 1. can I use openframeworks as a library, or do I have to rejigger my whole app just to use the OpenGL features? I don't see any tutorials or docs that show how I might do this without actually starting from scratch and creating an OF app. 2. is it possible to use CVPixelBufferRef as a texture.
I am targeting iOS 7+.
I was able to get this to work using the GPUImageMovie class. If you look inside this class, you'll see that there's a private method called:
- (void)processMovieFrame:(CVPixelBufferRef)movieFrame withSampleTime:(CMTime)currentSampleTime
This method takes a CVPixelBufferRef as input.
To access this method, declare a class extension that exposes it inside your class
#interface GPUImageMovie ()
-(void) processMovieFrame:(CVPixelBufferRef)movieFrame withSampleTime:(CMTime)currentSampleTime;
#end
Then initialize the class, set up the filter, and pass it your video frame:
GPUImageMovie *gpuMovie = [[GPUImageMovie alloc] initWithAsset:nil]; // <- call initWithAsset even though there's no asset
// to initialize internal data structures
// connect filters...
// Call the method we exposed
[gpuMovie processMovieFrame:myCVPixelBufferRef withSampleTime:kCMTimeZero];
One thing: you need to request your pixel buffers with kCVPixelFormatType_420YpCbCr8BiPlanarFullRange in order to match what the library expects.
Was testing some code and found an error with the following lines:
NSString *stringA = #"C99";
NSString *stringB = (__bridge id)malloc(sizeof (stringA));
It is not necessary to alloc a NSString this way, of course, and I am not required to do that. Again I was just testing on something else and I happened to stumble upon this.
The error reads:
Thread 1: EXC_BAD_ACCESS (code=1, address=0x20)
In the console:
(lldb)
To generalize, perhaps I should ask:
Could we alloc Objective-C objects through the use of malloc?
Has someone encountered this before (which I doubt, because I don't think anyone who uses Objective-C would alloc a NSString this way), but rather than shoving it aside and call it a day, I thought I would ask and see if someone knows what the exact cause of this is and why.
It is possible to use custom allocators for Objective-C objects. The problems with your code include:
NSString is a class cluster superclass (similar to an "abstract class") and cannot be instantiated on its own. You would need to use some concrete subclass of NSString. Note that the OS API does not provide any such class.
sizeof(stringA) is the size of the pointer variable, 4 or 8 bytes, which is too small to hold an NSString instance. You would need to use class_getInstanceSize() to compute the size.
+alloc performs work other than the allocation itself which is not present here. You would need to erase the memory and call objc_constructInstance().
ARC forbids the use of the low-level runtime functions that are needed to accomplish the above tasks.
well as far as I found the closest example of allocating NSSTring Clike is like this:
NSString* s4 = (NSString*)
CFStringCreateWithFormat(kCFAllocatorDefault, 0,
(CFStringRef) __builtin___CFStringMakeConstantString("%# %# (%#)"), s1, s2, s3);
ofcourse if you want to go lower and lower levels of this allocations , you should watch the CFStringRef class for its lower allocation .
but I hope this answer will satisfy you
found here, also there is more interesting things
http://www.opensource.apple.com/source/clang/clang-318.0.45/src/tools/clang/test/Analysis/NSString.m
I think the question you should be asking is what purpose that code serves.
Note that sizeof doesn't return the number of bytes in stringA, it simply returns the size of the pointer that is stringA. Who knows what lives in that little block of memory that has been allocated to stringB. Maybe it's a string, maybe not. Life is full of mystery.
This query is regarding the Portaudio framework. A little background before I ask the question:I am working on an application in PortAudio to output audio through a multichannel(=8) device. However, the device I am using does not expose itself as a single 8-channel device but instead shows up in my device-list as 4 stereo devices. On searching for an approach to handle this, I got to know that WinMME in PortAudio supports multiple devices.
Now, I went through the appropriate header file("pa_win_wmme.h") and followed the suggestions present. But I get the 'Invalid device' error (error number -9996) after calling PA_OpenStream(). In the above mentioned header file, they have in fact specified the right parameter(s) to use when configuring multiple devices to avoid this error, but in-spite of following them, I still get the error.
So I wanted to know if anybody has faced a similar issue and whether I have missed/wrongly configured anything.
I am pasting the required snippets of code below for reference:
PaStreamParameters outputParameters;
PaWinMmeStreamInfo wmmeStreamInfo;
PaWinMmeDeviceAndChannelCount wmmeDeviceAndNumChannels;**
...
...
outputParameters.device = paUseHostApiSpecificDeviceSpecification;
outputParameters.channelCount = 8;
outputParameters.sampleFormat = paFloat32; /* 32 bit floating point processing */
outputParameters.hostApiSpecificStreamInfo = NULL;
wmmeStreamInfo.size = sizeof(PaWinMmeStreamInfo);
wmmeStreamInfo.hostApiType = paMME;
wmmeStreamInfo.version = 1;
wmmeStreamInfo.flags = paWinMmeUseMultipleDevices;
wmmeDeviceAndNumChannels.channelCount = 2;
wmmeDeviceAndNumChannels.device = 3;
wmmeStreamInfo.devices = &wmmeDeviceAndNumChannels;
wmmeStreamInfo.deviceCount = 4;
outputParameters.hostApiSpecificStreamInfo = &wmmeStreamInfo;
The device id = 3 was obtained through
Pa_GetHostApiInfo( Pa_HostApiTypeIdToHostApiIndex( paMME ) )->defaultOutputDevice
I hope I have made the query clear enough. Will be happy to provide more details if required.
Thanks.
I finally figured out the mistake :-)
The configuration for multiple devices must be made as an array. For instance, in the above case
wmmeDeviceAndNumChannels must be an array of 4, with each individual device field containing the corresponding device index of each of the 4 stereo devices. The channelCount remains 2. The outputParameters.channelCount still has to be the aggregate number of channels, i.e. 8. With this I was able to run the application with a single stream, and of course, without any errors related to invalid device or invalid number of channels.:-)
Thanks.
Based on the code pasted above, it looks like you are trying to call open on a single 8-channel device. Instead you will have to get the Pa index of all four devices and call open 4 times. Once for each stereo device. You will then have 4 interleaved stereo streams to maintain. My guess is that changing channelCount = 8 to channelCount = 2 will allow the first stream to open.
I want to clear sensitive data from memory in my iOS app.
In Windows I used to use SecureZeroMemory. Now, in iOS, I use plain old memset, but I'm a little worried the compiler might optimize it:
https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/coding/771-BSI.html
code snippet:
NSData *someSensitiveData;
memset((void *)someSensitiveData.bytes, 0, someSensitiveData.length);
Paraphrasing 771-BSI (link see OP):
A way to avoid having the memset call optimized out by the compiler is to access the buffer again after the memset call in a way that would force the compiler not to optimize the location. This can be achieved by
*(volatile char*)buffer = *(volatile char*)buffer;
after the memset() call.
In fact, you could write a secure_memset() function
void* secure_memset(void *v, int c, size_t n) {
volatile char *p = v;
while (n--) *p++ = c;
return v;
}
(Code taken from 771-BSI. Thanks to Daniel Trebbien for pointing out for a possible defect of the previous code proposal.)
Why does volatile prevent optimization? See https://stackoverflow.com/a/3604588/220060
UPDATE Please also read Sensitive Data In Memory because if you have an adversary on your iOS system, your are already more or less screwed even before he tries to read that memory. In a summary SecureZeroMemory() or secure_memset() do not really help.
The problem is NSData is immutable and you do not have control over what happens. If the buffer is controlled by you, you could use dataWithBytesNoCopy:length: and NSData will act as a wrapper. When finished you could memset your buffer.
I am using the following pseudocode to generate a PDF document:
CGContextRef context = CGPDFContextCreateWithURL(url, &rect, NULL);
for (int i = 1; i <= N; i++)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
CGContextBeginPage(context, &mediaBox);
// drawing code
CGContextEndPage(context);
[pool release];
}
CGContextRelease(context);
It works very well with small documents (N < 100 pages), but it uses too much
memory and crashes if the document has more than about 400 pages (it received
two memory warnings before crashing.) I have made sure there were no leaks using
Instruments. What's your advice on creating large PDF documents on iOS? Thanks a lot.
edit: The pdf creation is done in a background thread.
Since you're creating a single document via CGPDFContextCreateWithURL the entire thing has to be held in memory and appended to, something that commonly (though I can't say for certain with iOS and CGPDFContextCreateWithURL) requires a full before and after copy of the document to be kept. No need for a leak to create a problem, even without the before-and-after issue.
If you aren't trying to capture a bunch of existing UIKit-drawn stuff -- and in your sample it seems that you're not -- use the OS's printing methods instead, which offer built-in support for printing to a PDF. UIGraphicsBeginPDFContextToFile writes the pages out to disk as they're added so the whole thing doesn't have to be held in memory at once. You should be able to generate a huge PDF that way.
Probably not the answer you want to hear, but looking at it from another perspective.
Could you consider it as a limitation of the device?... First check the number of pages in the PDF and if it is too large, give a warning to the user. Therefore handling it gracefully.
You could then expand on this....
You could construct small PDF's on the iDevice and if the PDF is too large, construct it server-side the next time the iDevice has a net connection.
If you allocate too much memory, your app will crash. Why is generating an unusually large PDF a goal? What are you actually trying to accomplish?
What about using a memory mapped file to back your CG data consumer? Then it doesn't necessarily have to fit in RAM all at once.
I created an example here: https://gist.github.com/3748250
Use it like this:
NSURL * url = [ NSURL fileURLWithPath:#"pdf.pdf"] ;
MemoryMappedDataConsumer * consumer = [ [ MemoryMappedDataConsumer alloc ] initWithURL:url ] ;
CGDataConsumerRef cgDataConsumer = [ consumer CGDataConsumer ] ;
CGContextRef c = CGPDFContextCreate( cgDataConsumer, NULL, NULL ) ;
CGDataConsumerRelease( cgDataConsumer ) ;
// write your PDF to context `c`
CGPDFContextClose( c ) ;
CGContextRelease( c ) ;
return 0;