I'm trying to create an IOSurface in Swift and then create a CIImage from it. The IOSurfaceRef looks alright, but CIImage returns nil:
textureImageWidth = 1024
textureImageHeight = 1024
let macPixelFormatString = "ARGB"
var macPixelFormat: UInt32 = 0
for c in macPixelFormatString.utf8 {
macPixelFormat *= 256
macPixelFormat += UInt32(c)
}
let ioSurface = IOSurfaceCreate([kIOSurfaceWidth: textureImageWidth,
kIOSurfaceHeight: textureImageHeight,
kIOSurfaceBytesPerElement: 4,
kIOSurfaceBytesPerRow: textureImageWidth * 4,
kIOSurfaceAllocSize: textureImageWidth * textureImageHeight * 4,
kIOSurfacePixelFormat: macPixelFormat] as CFDictionary)!
IOSurfaceLock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
let test = CIImage(ioSurface: ioSurface)
IOSurfaceUnlock(ioSurface, IOSurfaceLockOptions.readOnly, nil)
Any suggestions please? I'm guessing that the CIImage initialiser needs a bit more metadata to do its job but I've no idea what.
I had the pixel format byte order the wrong way around. It should be:
for c in macPixelFormatString.utf8.reversed()
Leaving the question up as I don't think there are any other examples of using IOSurfaceCreate in Swift on stackoverflow.
Related
Here was my workable code in the previous version of Swift:
let imageOptionsDictKeys = [ kCVPixelBufferPixelFormatTypeKey, kCVPixelBufferWidthKey, kCVPixelBufferHeightKey, kCVPixelBufferOpenGLESCompatibilityKey, kCVPixelBufferIOSurfacePropertiesKey]
let imageOptionsDictValues = [ cvPixelFormatType, frameW, frameH, boolYES]
var keyCallbacks = kCFTypeDictionaryKeyCallBacks
var valueCallbacks = kCFTypeDictionaryValueCallBacks
let imageOptions = CFDictionaryCreate(kCFAllocatorDefault, UnsafeMutablePointer(imageOptionsDictKeys), UnsafeMutablePointer(imageOptionsDictValues), 4, &keyCallbacks, &valueCallbacks)
After changes in the Swift 3.0 I have to convert my keys and values arrays into UnsafeMutablePointer<UnsafeRawPointer?> for creating CFDictionary.
This way:
let imageOptionsDictKeysPointer = UnsafeMutablePointer<UnsafeRawPointer?>.allocate(capacity: 1)
imageOptionsDictKeysPointer.initialize(to: imageOptionsDictKeys)
gives an Bad Access error.
And after reading documentation I am trying to compile this code:
let imageOptionsDictKeys = [kCVPixelBufferPixelFormatTypeKey, kCVPixelBufferWidthKey, kCVPixelBufferHeightKey, kCVPixelBufferOpenGLESCompatibilityKey]
let imageOptionsDictKeysRawPointer = Unmanaged.passUnretained(imageOptionsDictKeys).toOpaque()
let imageOptionsDictKeysPointer = UnsafeMutablePointer<UnsafeRawPointer?>.allocate(capacity: 1)
imageOptionsDictKeysPointer.initialize(to: imageOptionsDictKeysRawPointer)
let imageOptionsDictValues = [ cvPixelFormatType, frameW, frameH, boolYES]
let imageOptionsDictValuesRawPointer = Unmanaged.passUnretained(imageOptionsDictValues).toOpaque()
let imageOptionsDictValuesPointer = UnsafeMutablePointer<UnsafeRawPointer?>.allocate(capacity: 1)
imageOptionsDictValuesPointer.initialize(to: imageOptionsDictValuesRawPointer)
let imageOptions = CFDictionaryCreate(kCFAllocatorDefault, imageOptionsDictKeysPointer, imageOptionsDictValuesPointer, 4, &keyCallbacks, &valueCallbacks)
but error Generic parameter 'Instance' could not be inferred appears in the lines Unmanaged.passUnretained(array).toOpaque()
I have no idea how to create CFDictionary programmatically now.
I just solved a similar issue converting arrays to UnsafeMutablePointer< UnsafeMutablePointer<T>> which you can find here:
Swift 3 UnsafeMutablePointer initialization for C type float**
To convert swift arrays using the same scheme, use UnsafeMuTablePointer as suggested here: http://technology.meronapps.com/2016/09/27/swift-3-0-unsafe-world-2/
I'm trying to extract the native [Float] array out of a rendered AudioBufferList where the audio unit's stream is set up for 32-bit floating point. In Swift 1 & 2 I did this:
var monoSamples = [Float]()
for i in 0..<Int(inNumberFrames) {
withUnsafePointer(to: &bufferList.mBuffers.mData, {
let ptr = $0
let newPtr = ptr + i
let sample = unsafeBitCast(newPtr.pointee, to: Float.self)
monoSamples.append(sample)
})
}
Undoubtedly not the fastest method, but it worked. In Swift 3 this compiles without error but at runtime I get a crash:
fatal error: can't unsafeBitCast between types of different sizes
That's a bit surprising; Swift's Float is 32-bit and the stream's data is 32-bit.
What I'd like to do is simply:
withUnsafeMutablePointer(to: &bufferList.mBuffers.mData, {
let monoSamples = [Float](UnsafeBufferPointer(start: $0, count:Int(inNumberFrames)))
})
But there I get a compile error:
Expression type '[Float]' is ambiguous without more context
What's the right way to do this?
If you un-converted your code to Swift 2, it does not work. You may have modified too much when converting your code to Swift 3.
Try this:
var monoSamples = [Float]()
let ptr = bufferList.mBuffers.mData?.assumingMemoryBound(to: Float.self)
monoSamples.append(contentsOf: UnsafeBufferPointer(start: ptr, count: Int(inNumberFrames)))
Try this may be?
Yeah you are correct, You have to input your $0 as UnsafePointer. Choose any type of UnsafePointer initializer and pass that value here, for the explanation purpose I have used bitPattern here.
UnsafeBufferPointer<Float>(start: UnsafePointer(bitPattern: i), count: Int(inNumberFrames))
I'm attempting to set up a video stream viewer in a Swift based project.
I've reviewed the following (Objective C) which has been very helpful: How AVSampleBufferDisplayLayer displays H.264
In a Swift context, I am having difficulty with the fact that the CMTimebaseCreateWithMasterClock requires the CMTimebase related element to be of type UnsafeMutablePointer. Would someone be able to explain how to convert to this and back to address the issue in the following code section.
var controlTimebase : CMTimebase
var myAllocator : CFAllocator!
CMTimebaseCreateWithMasterClock( myAllocator, CMClockGetHostTimeClock(), CMTimebase)
// Problem is here...below is the expected format.
//CMTimebaseCreateWithMasterClock(allocator: CFAllocator!, masterClock: CMClock!, timebaseOut: UnsafeMutablePointer < Unmanaged < CMTimebase > ? >)
videoLayer.controlTimebase = controlTimebase
Spotted the UnsafeMutablePointer syntax needed in a different context here:
CVPixelBufferPool Error ( kCVReturnInvalidArgument/-6661)
Using that the following seems to compile happily :-)
var _CMTimebasePointer = UnsafeMutablePointer<Unmanaged<CMTimebase>?>.alloc(1)
CMTimebaseCreateWithMasterClock( kCFAllocatorDefault, CMClockGetHostTimeClock(), _CMTimebasePointer )
videoLayer.controlTimebase = _CMTimebasePointer.memory?.takeUnretainedValue()
let cmTimebasePointer = UnsafeMutablePointer<CMTimebase?>.allocate(capacity: 1)
let status = CMTimebaseCreateWithMasterClock(allocator: kCFAllocatorDefault, masterClock: CMClockGetHostTimeClock(), timebaseOut: cmTimebasePointer)
videoLayer.controlTimebase = cmTimebasePointer.pointee
if let controlTimeBase = videoLayer.controlTimebase, status == noErr {
CMTimebaseSetTime(controlTimeBase, time: CMTime.zero)
CMTimebaseSetRate(controlTimeBase, rate: 1)
}
Currently I am using this method to loop through every pixel, and insert a value into a 3D array based upon RGB values. I need this array for other parts of my program, however it is extraordinarily slow. When run on a 50 x 50 picture, it is almost instant, but as soon as you start getting into the hundreds x hundreds it takes a long time to the point where the app is useless. Anyone have any ideas on how to speed up my method?
#IBAction func convertImage(sender: AnyObject) {
if let image = myImageView.image {
var pixelData = CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage))
var data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let height = Int(image.size.height)
let width = Int(image.size.width)
var zArry = [Int](count:3, repeatedValue: 0)
var yArry = [[Int]](count:width, repeatedValue: zArry)
var xArry = [[[Int]]](count:height, repeatedValue: yArry)
for (var h = 0; h < height; h++) {
for (var w = 0; w < width; w++) {
var pixelInfo: Int = ((Int(image.size.width) * Int(h)) + Int(w)) * 4
var rgb = 0
xArry[h][w][rgb] = Int(data[pixelInfo])
rgb++
xArry[h][w][rgb] = Int(data[pixelInfo+1])
rgb++
xArry[h][w][rgb] = Int(data[pixelInfo+2])
}
}
println(xArry[20][20][1])
}
}
Maybe there is a way to convert the UIImage to a different type of image and create an array of pixels. I am open to all suggestions. Thanks!
GOAL: The goal is to use the array to modify the RGB values of all pixels, and create a new image with the modified pixels. I tried simply looping through all of the pixels without storing them, and modifying them into a new array to create an image, but got the same performance issues.
Update:
After countless tries I realized I was making my tests on debug configuration.
Switched to release, and now it's so much faster.
Swift seems to be many times slower on the debug configuration.
The difference now between your code and my optimized version is several times faster.
It seems as you have a big slowdown from using image.size.width instead of the local variable width.
Original
I tried to optimize it a bit and come up with this:
#IBAction func convertImage () {
if let image = UIImage(named: "test") {
let pixelData = CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage))
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let height = Int(image.size.height)
let width = Int(image.size.width)
let zArry = [Int](count:3, repeatedValue: 0)
let yArry = [[Int]](count:width, repeatedValue: zArry)
let xArry = [[[Int]]](count:height, repeatedValue: yArry)
for (index, value) in xArry.enumerate() {
for (index1, value1) in value.enumerate() {
for (index2, var value2) in value1.enumerate() {
let pixelInfo: Int = ((width * index) + index1) * 4 + index2
value2 = Int(data[pixelInfo])
}
}
}
}
}
However in my tests this is barely 15% faster. What you need is orders of magnitude faster.
Another ideea is use the data object directly when you need it without creating the array like this:
let image = UIImage(named: "test")!
let pixelData = CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage))
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let width = Int(image.size.width)
// value for [x][y][z]
let value = Int(data[((width * x) + y) * 4 + z])
You didn't say how you use this array in your app, but I feel that even if you find a way to get this array created much faster, you would get another problem when you try to use it, as it would take a long time too..
I'm on beta 3. Consider the following Objective-C line:
const uint8_t *reportData = [data bytes];
where data is a NSData object.
How would this line be re-written in Swift?
data.bytes is of type ConstUnsafePointer<()>, and while there's plenty of documentation on how to create a pointer type in Swift, there isn't much info on how to work with them.
edit:
To add some context, I'm trying to port Apple's HeartRateMonitor sample code to Swift. This code interacts with BLE heart rate monitors. This code I'm working on translates the data received by the Bluetooth system into an int for use in the UI. The data received from BT is expected to be an array of uints, element 0 is used to check for a flag and element 1 contains the value.
Here's the same Objective-C line in context:
const uint8_t *reportData = [data bytes];
uint16_t bpm = 0;
if ((reportData[0] & 0x01) == 0)
{
/* uint8 bpm */
bpm = reportData[1];
}
What you were looking for was how to convert NSData to an array of UInt8. Here's how.
import Foundation
let path = "/etc/csh.cshrc" // something existent
let data = NSData(contentsOfFile: path)
var aofb = [UInt8](count:data.length, repeatedValue:0)
data.getBytes(&aofb, length:data.length)
for c in aofb {
let s = UnicodeScalar(Int(c)).escape(asASCII:true)
println("\(c):\(s)")
}
Just built following code (Note code below works on Beta 3, ConstUnsafePointer<()> needs to be changed to COpaquePointer in order to work on Beta 2, please see edit history for more information)
var dataPath = NSBundle.mainBundle().pathForResource("TestData", ofType: "") // What I have in TestData is "GREETINGS WORLD"
var originalData = NSData(contentsOfFile: dataPath)
var dataLength = originalData.length
println("original data: \(originalData)") // Output original data
// Data to bytes
var reportBytes: ConstUnsafePointer<()> = originalData.bytes
var bytesToString = NSString(bytes: reportBytes, length: dataLength, encoding: NSUTF8StringEncoding)
println("string from bytes: \(bytesToString)")
// Bytes to data
var bytesToData = NSData(bytes: reportBytes, length: dataLength)
println("data from bytes: \(bytesToData)")
Console log
original data: <47524545 54494e47 5320574f 524c44>
string from bytes: GREETINGS WORLD
data from bytes: <47524545 54494e47 5320574f 524c44>
Also found this may help
ConstUnsafePointer<T>
/// This type stores a pointer to an object of type T. It provides no
/// automated memory management, and therefore the user must take care
/// to allocate and free memory appropriately.
Hope this shed light.
Looking at handling bluetooth heart rate monitors in Swift now I found the simplest way to get the NSData byte values to UInt8 format:
let bytes = UnsafePointer<UInt8>(data.bytes)
if bytes[0] & 0x01 == 0 {
NSLog("BPM \(bytes[1]")
}