I'm using the Swift compiler's Bridging Header feature to call a C function that allocates memory using malloc(). It then returns a pointer to that memory. The function prototype is something like:
char *the_function(const char *);
In Swift, I use it like this:
var ret = the_function(("something" as NSString).UTF8String)
let val = String.fromCString(ret)!
Forgive my ignorance concerning Swift but normally in C, if the_function() is malloc'ing memory and returning it, somebody else needs to free() it at some point.
Is this being handled by Swift somehow or am I leaking memory in this example?
Thanks in advance.
Swift does not manage memory that is allocated with malloc(), you have to free the memory eventually:
let ret = the_function("something") // returns pointer to malloc'ed memory
let str = String.fromCString(ret)! // creates Swift String by *copying* the data
free(ret) // releases the memory
println(str) // `str` is still valid (managed by Swift)
Note that a Swift String is automatically converted to a UTF-8
string when passed to a C function taking a const char * parameter
as described in String value to UnsafePointer<UInt8> function parameter behavior.
That's why
let ret = the_function(("something" as NSString).UTF8String)
can be simplified to
let ret = the_function("something")
Related
This question already has answers here:
UnsafeMutablePointer Warning with Swift 5
(3 answers)
Warning: Initialization of 'UnsafeBufferPointer<T>' results in a dangling buffer pointer
(6 answers)
Initialization of 'UnsafeMutableRawPointer' results in a dangling pointer
(1 answer)
Closed 2 years ago.
So I have some code to create H264ParameterSets like:
var formatDesc: CMVideoFormatDescription?
func createH264FormatDescription(SPS: Array<UInt8>, PPS: Array<UInt8>) -> OSStatus {
if formatDesc != nil { formatDesc = nil }
let paramSet = [UnsafePointer<UInt8>(SPS), UnsafePointer<UInt8>(PPS)]
let paramPointers = UnsafePointer<UnsafePointer<UInt8>>(paramSet)
let paramSizes = UnsafePointer<Int>([SPS.count, PPS.count])
let status = CMVideoFormatDescriptionCreateFromH264ParameterSets(allocator: kCFAllocatorDefault, parameterSetCount: 2, parameterSetPointers: paramPointers, parameterSetSizes: paramSizes, nalUnitHeaderLength: 4, formatDescriptionOut: &formatDesc)
return status
}
Starting on Xcode 11.4 I got warnings for those UnsafePointer(), which seems not happen before:
Initialization of UnsafePointer<UInt8> results in a dangling pointer
Initialization of UnsafePointer<UnsafePointer<UInt8>> results in a dangling pointer
Initialization of UnsafePointer<Int> results in a dangling pointer
I'm not sure why we see this? and how can I remove the warning? Thank in advance.
The easiest way to explain this warning is to look at one of the cases causing it. So lets start with your use of SPS.
It is an Array<UInt8> so it is backed by a buffer of UInt8 just like in C. When you pass SPS with UnsafePointer<UInt8>(SPS) it creates a valid pointer to the buffer for that moment. The issue is that you could then mutate SPS say by appending another value to it. This would mean that the buffer backing the Array is potentially moved to another place in memory. This would mean that your pointer that is now part of paramSet is invalid.
The other issue is that if you pass this pointer to something, like you do in this case, the other function could try to hold onto it and then it has an invalid pointer. So if you expect the other function to hold onto the pointer you need to manually manage memory with UnsafePointers and Unmanaged yourself. If CMVideoFormatDescriptionCreateFromH264ParameterSets() doesn't hold onto the pointers then the code I'll share is correct, if it does you will need to adjust it to create/destory the memory as needed.
Also it is worth noting that in this case, you can't mutate any of the Arrays you have because they are constants but in general the principle is still the same. This means that in theory it could never be mutated but the Swift compiler prefers to help us write code that is always safe and correct whenever possible, even with UnsafePointer types.
So how can you fix this? You will need to be able to call withUnsafeBufferPointer and then access the pointer through the UnsafeBufferPointer like this:
var formatDesc: CMVideoFormatDescription?
func createH264FormatDescription(SPS: Array<UInt8>, PPS: Array<UInt8>) -> OSStatus {
if formatDesc != nil { formatDesc = nil }
let status = SPS.withUnsafeBufferPointer { SPS in
PPS.withUnsafeBufferPointer { PPS in
let paramSet = [SPS.baseAddress!, PPS.baseAddress!]
let paramSizes = [SPS.count, PPS.count]
return paramSet.withUnsafeBufferPointer { paramSet in
paramSizes.withUnsafeBufferPointer { paramSizes in
CMVideoFormatDescriptionCreateFromH264ParameterSets(allocator: kCFAllocatorDefault, parameterSetCount: 2, parameterSetPointers: paramSet.baseAddress!, parameterSetSizes: paramSizes.baseAddress!, nalUnitHeaderLength: 4, formatDescriptionOut: &formatDesc)
}
}
}
}
return status
}
The reason this approach works is that for the scope of withUnsafeBufferPointer the Law of Exclusivity is protecting the arrays so they can't be mutated.
If you are worried about the baseAddress! usage you can check that it isn't nil but it is guaranteed to not be nil when count > 0 according the the compiler engineers (they have stated this on either Twitter or the Swift forums I forget...).
I had this:
let alphaPtr = UnsafeMutablePointer<vImagePixelCount>(mutating: alpha) as UnsafeMutablePointer<vImagePixelCount>?
Which now I get the warning:
Initialization of 'UnsafeMutablePointer' (aka
'UnsafeMutablePointer') results in a dangling pointer
Detailed warning consists of:
Implicit argument conversion from '[vImagePixelCount]' (aka 'Array') to 'UnsafePointer' (aka
'UnsafePointer') produces a pointer valid only for the duration
of the call to 'init(mutating:)'
Use the 'withUnsafeBufferPointer' method on Array in order to explicitly convert argument to buffer pointer valid for a defined
scope
Is there a way around this?
Try this
var bytes = [UInt8]()
let uint8Pointer = UnsafeMutablePointer<UInt8>.allocate(capacity: bytes.count)
uint8Pointer.initialize(from: &bytes, count: bytes.count)
It was never safe to do this, and the compiler now is warning you more aggressively.
let alphaPtr = UnsafeMutablePointer ...
At the end of this line, alphaPtr is already invalid. There is no promise that what it points to is still allocated memory.
Instead, you need to nest whatever usage you need into a withUnsafeMutablePointer() (or withUnsafePointer()) block. If you cannot nest it into a block (for example, if you were storing the pointer or returning it), there is no way to make that correct. You'll have to redesign your data management to not require that.
Do you need use the withUnsafeBufferPointer method from Array as
var alphaPtr: UnsafeBufferPointer = alpha.withUnsafeBufferPointer { $0 }
that's command produce a pointer optional if you need working with a specific type could you you use bindMemory(to:) or other function that match with you requirements.
Sometimes use a &alpha if you need a UnsafeRawPointer as a function parameter.
I am working on a project that was written in Objective-C and needs to be updated to Swift. We use a C file for transferring data.
Here is the code I was given in Objective-C:
- (NSData *)prepareEndPacket {
UInt8 *buff_data;
buff_data = (uint8_t *)malloc(sizeof(uint8_t)*(PACKET_SIZE+5));
// Call to C File
PrepareEndPacket(buff_data);
NSData *data_first = [NSData dataWithBytes:buff_data length:sizeof(uint8_t)*(PACKET_SIZE+5)];
return data_first;
}
In the C .h file I have this for reference:
#define PACKET_SIZE ((uint32_t)128)
I can not seem to find a good way of converting this to Swift. Any help would be appreciated.
malloc and free actually work fine in Swift; however, the UnsafeMutablePointer API is more "native". I'd probably use Data's bytesNoCopy for better performance. If you want, you can use Data(bytes:count:), but that will make a copy of the data (and then you need to make sure to deallocate the pointer after making the copy, or you'll leak memory, which is actually a problem in the Objective-C code above since it fails to free the buffer).
So, something like:
func prepareEndPacket() -> Data {
let count = PACKET_SIZE + 5
let buf = UnsafeMutablePointer<UInt8>.allocate(capacity: count)
PrepareEndPacket(buf)
return Data(bytesNoCopy: buf, count: count, deallocator: .custom { ptr, _ in
ptr.deallocate()
})
}
By using bytesNoCopy, the Data object returned is basically a wrapper around the original pointer, which will be freed by the deallocator when the Data object is destroyed.
Alternatively, you can create the Data object from scratch and get a pointer to its contents to pass to PrepareEndPacket():
func prepareEndPacket() -> Data {
var data = Data(count: PACKET_SIZE + 5)
data.withUnsafeMutableBytes { (ptr: UnsafeMutablePointer<UInt8>) in
PrepareEndPacket(ptr)
}
return data
}
This is slightly less efficient, since the Data(count:) initializer will initialize all the Data's bytes to zero (similar to using calloc instead of malloc), but in many cases, that may not make enough of a difference to matter.
After I convert from swift 2 to swift 3, there is an error pop up for the below metioned line
let value = UnsafePointer<UInt32>(array1).pointee
'init' is unavailable: use 'withMemoryRebound(to:capacity:_)' to temporarily view memory as another layout-compatible type.
in swift2 it is like
let value = UnsafePointer<UInt32>(array1).memory
Can someone explain please?
Sorry I'm quite new to swift3
After i have make the changes to
let abc = UnsafePointer<UInt32>(array1).withMemoryRebound(to: <#T##T.Type#>, capacity: <#T##Int#>, <#T##body: (UnsafeMutablePointer<T>) throws -> Result##(UnsafeMutablePointer<T>) throws -> Result#>)
but still what value should go in to the variable? Sorry, i have search around but too bad i can't find a solution
You can try this:
let rawPointer = UnsafeRawPointer(array1)
let pointer = rawPointer.assumingMemoryBound(to: UInt32.self)
let value = pointer.pointee
Raw pointer is a pointer for accessing untype data.
assumingMemoryBound(to:) can convert from an UnsafeRawPointer to UnsafePointer<T>.
Reference :Swift 3.0 Unsafe World
If array is an Array, your best bet is to use withUnsafeBufferPointer:
array.withUnsafeBufferPointer { buffer in
// do something with 'buffer'
// (if you need an UnsafePointer rather than an UnsafeBufferPointer,
// you can access that via the buffer's .baseAddress property)
}
Make sure you don't let the buffer pointer escape from the closure, because it will not be valid outside it.
I'd like to createa Metal buffers with the newBufferWithBytesNoCopy function for letting CPU and GPU share memory and practicing zero-copy data transfer.
The newBufferWithBytesNoCopy function takes a UnsafeMutablePointer-type pointer, and the pointer needs to be aligned to 16K(16384) bytes.
Could anyone provide advice on how to create a aligned memory to a certain size in Swift?
I believe this should work for you:
var memory:UnsafeMutablePointer<Void> = nil
var alignment:UInt = 0x4000 // 16K aligned
var size:UInt = bufferSize // bufferSize == your buffer size
posix_memalign(&memory, alignment, size)
For reference:
http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_memalign.html
Swift PageAligned Array (just works solution)
PageAlignedArray is a project that handles the problem of memory allocation for you when memory that will be used with Metal.
More detail
4096 byte alignment
I'm not seeing your 16k byte requirement. The debugger says 4k. Maybe you can provide a reference? It should be based upon the system's page size.
Using posix_memalign
The original idea came from memkite I believe and goes like this:
private func setupSharedMemoryWithSize(byteCount: Int) ->
(pointer: UnsafeMutablePointer<Void>,
memoryWrapper: COpaquePointer)
{
let memoryAlignment = 0x1000 // 4096 bytes, 0x4000 would be 16k
var memory: UnsafeMutablePointer<Void> = nil
posix_memalign(&memory, memoryAlignment, byteSizeWithAlignment(memoryAlignment, size: byteCount))
let memoryWrapper = COpaquePointer(memory)
return (memory, memoryWrapper)
}
Calculating the memory size
You'll want to calculate the correct amount of memory to allocate so that it lands on the byte boundary. Which means you'll probably need to allocate a bit more than you desired. Here you pass the size you need and alignment and it will return the amount you should allocate.
private func byteSizeWithAlignment(alignment: Int, size: Int) -> Int
{
return Int(ceil(Float(size) / Float(alignment))) * alignment
}
Using the functions
let (pointer, memoryWrapper) = setupSharedMemoryWithSize(byteCount)
var unsafeVoidPointer: UnsafeMutablePointer<Void> = pointer
// Assuming your underlying data is unsigned 8-bit integers.
let unsafeMutablePointer = UnsafeMutablePointer<UInt8>(memoryWrapper)
let unsafeMutableBufferPointer = UnsafeMutableBufferPointer(start: unsafeMutablePointer, count: byteCount)
Don't forget to free the memory you allocated.
Allocating shared memory without posix_memalign.
These days you can allocate the memory without posix_memalign by just specifying .StorageModeShared.
let byteCount = 1000 * sizeof(Float)
let sharedMetalBuffer = self.metalDevice.newBufferWithLength(byteCount, options: .StorageModeShared)
After experiencing some annoyance with this problem, I've decided to go ahead and create a simple solution that should make this a lot easier.
I've created a Swift array implementation called PageAlignedArray that matches the interface and functionality of the built-in Swift array, but always resides on page-aligned memory, and so can be very easily made into an MTLBuffer. I've also added a convenience method to directly convert PageAlignedArray into a Metal buffer.
Of course, you can continue to mutate your array afterwards and your updates will be automatically available to the GPU courtesy of the shared-memory architecture. However, keep in mind that you must regenerate your MTLBuffer object whenever the array's length changes.
Here's a quick code sample:
var alignedArray : PageAlignedContiguousArray<matrix_double4x4> = [matrixTest, matrixTest]
alignedArray.append(item)
alignedArray.removeFirst() // Behaves just like a built-in array, with all convenience methods
// When it's time to generate a Metal buffer:
let device = MTLCreateSystemDefaultDevice()
let testMetalBuffer = device?.makeBufferWithPageAlignedArray(alignedArray)
The sample uses matrix_double4x4, but the array should work for any Swift value types. Please note that if you use a reference type (such as any kind of class), the array will contain pointers to your elements and so won't be usable from your GPU code.
Please grab PageAlignedArray here: https://github.com/eldoogy/PageAlignedArray