I am currently using the Swift release of Tensorflow in my iOS app.
My model is working fine, but I am having trouble copying the data into the first Tensor so I can use the neural net to detect stuff.
I consulted the testsuite inside the repository, and their code is working as follows:
They are using some extensions:
extension Array {
/// Creates a new array from the bytes of the given unsafe data.
///
/// - Note: Returns `nil` if `unsafeData.count` is not a multiple of
/// `MemoryLayout<Element>.stride`.
/// - Parameter unsafeData: The data containing the bytes to turn into an array.
init?(unsafeData: Data) {
guard unsafeData.count % MemoryLayout<Element>.stride == 0 else { return nil }
let elements = unsafeData.withUnsafeBytes {
UnsafeBufferPointer<Element>(
start: $0,
count: unsafeData.count / MemoryLayout<Element>.stride
)
}
self.init(elements)
}
}
extension Data {
/// Creates a new buffer by copying the buffer pointer of the given array.
///
/// - Warning: The given array's element type `T` must be trivial in that it can be copied bit
/// for bit with no indirection or reference-counting operations; otherwise, reinterpreting
/// data from the resulting buffer has undefined behavior.
/// - Parameter array: An array with elements of type `T`.
init<T>(copyingBufferOf array: [T]) {
self = array.withUnsafeBufferPointer(Data.init)
}
}
to create the array containing the data, and a Data object from that:
static let inputData = Data(copyingBufferOf: [Float32(1.0), Float32(3.0)])
Afterwards, they copy the inputData into the neural net.
I've tried to modify their code to load an image into a [1,28,28,1] Tensor.
The image is looking something like this:
[[[[Float32(254.0)],
[Float32(255.0)],
[Float32(254.0)],
[Float32(250.0)],
[Float32(252.0)],
[Float32(255.0)],
[Float32(255.0)],
[Float32(255.0)],
[Float32(255.0)],
[Float32(254.0)],
[Float32(214.0)],
[Float32(160.0)],
[Float32(130.0)],
[Float32(124.0)],
[Float32(129.0)],
...
you get the point.
But if I try to cast that to Data / init Data with the image data I somehow only get 8 bytes:
private func createTestData() -> Data {
return Data(copyingBufferOf:
[[[[Float32(254.0)],
[Float32(255.0)],
[Float32(254.0)],
...
Same goes for the code in the tests, but for them, it is fine (2*Float32 = 8 bytes).
For me, that is considerably too small (should be 28*28*4 = 3136 bytes)!
Is there something I am missing (have I overlooked something)?
What do I need to do to get my images into the correct arrays/data types?
A Swift Array is a fixed-sized structure with (opaque) pointers to the actual element storage. The withUnsafeBufferPointer() method calls the given closure with a buffer pointer to that element storage. In the case of a [Float] array, that is a pointer to the memory address of the floating point values. That's why
array.withUnsafeBufferPointer(Data.init)
works to get a Data value representing the floating point numbers.
If you pass a nested array (e.g. of type [[Float]]) to the withUnsafeBufferPointer() method then the closure is called with a pointer to the Array structures of the inner arrays. So the element type now is not Float but [Float] – and not a “trivial type” in the sense of the warning
/// - Warning: The given array's element type `T` must be trivial in that it can be copied bit
/// for bit with no indirection or reference-counting operations; otherwise, reinterpreting
/// data from the resulting buffer has undefined behavior.
What you need to do is to flatten the nested array to a simple array, and then create a Data value from the simple array.
Related
I had this:
let alphaPtr = UnsafeMutablePointer<vImagePixelCount>(mutating: alpha) as UnsafeMutablePointer<vImagePixelCount>?
Which now I get the warning:
Initialization of 'UnsafeMutablePointer' (aka
'UnsafeMutablePointer') results in a dangling pointer
Detailed warning consists of:
Implicit argument conversion from '[vImagePixelCount]' (aka 'Array') to 'UnsafePointer' (aka
'UnsafePointer') produces a pointer valid only for the duration
of the call to 'init(mutating:)'
Use the 'withUnsafeBufferPointer' method on Array in order to explicitly convert argument to buffer pointer valid for a defined
scope
Is there a way around this?
Try this
var bytes = [UInt8]()
let uint8Pointer = UnsafeMutablePointer<UInt8>.allocate(capacity: bytes.count)
uint8Pointer.initialize(from: &bytes, count: bytes.count)
It was never safe to do this, and the compiler now is warning you more aggressively.
let alphaPtr = UnsafeMutablePointer ...
At the end of this line, alphaPtr is already invalid. There is no promise that what it points to is still allocated memory.
Instead, you need to nest whatever usage you need into a withUnsafeMutablePointer() (or withUnsafePointer()) block. If you cannot nest it into a block (for example, if you were storing the pointer or returning it), there is no way to make that correct. You'll have to redesign your data management to not require that.
Do you need use the withUnsafeBufferPointer method from Array as
var alphaPtr: UnsafeBufferPointer = alpha.withUnsafeBufferPointer { $0 }
that's command produce a pointer optional if you need working with a specific type could you you use bindMemory(to:) or other function that match with you requirements.
Sometimes use a &alpha if you need a UnsafeRawPointer as a function parameter.
I have this static dictionary created as so:
static var pictures = Dictionary<Int, Array<UIImage>>()
I want to populate it with images. At the moment when I am creating it I don't know how many key/value pairs I need to create. I have to fetch from the internet the data, but after that I am doing this to populate, but still my dictionary is empty:
for i in 0...Fetching.numberOfAliveListings - 1 {
for _ in 0...AdsCollectionView.listings[i].photos.count - 1 {
AdsCollectionView.pictures[i]?.append(UIImage(named: "noimage")!)
}
}
pictures is initially empty. So any attempt to access a value for a given key will result in a nil value. Since the value (the array) is nil, the optional chaining skips the call to append.
One solution is to provide a default array when looking up the value for a given Int.
AdsCollectionView.pictures[i, default: []].append(UIImage(named: "noimage")!)
You may also wish to consider alternate syntax when declaring pictures:
static var pictures = [Int: [UIImage]]()
This is a fairly simple issue, but one I would like to solve, as it MAY help with performance.
I want to find out if Swift has a way to create a Dictionary, specifying ONLY keys, and maybe no values, or a single value that is set in each entry.
In other words, I want to create a Dictionary object, and "preload" its keys. Since this is Swift, the values could be 0 or nil (or whatever is a default empty).
The reason for this, is so that I can avoid two loops, where I go through once, filling a Dictionary with keys and empty values, and a second one, where I then set those values (There's a practical reason for wanting this, which is a bit out of the scope of this question).
Here's sort of what I'm thinking:
func gimme_a_new_dictionary(_ inKeyArray:[Int]) -> [Int:Int] {
var ret:[Int:Int] = [:]
for key in inKeyArray {
ret[key] = 0
}
return ret
}
let test1 = gimme_a_new_dictionary([4,6,1,3,0,1000])
But I'm wondering if there's a quicker way to do the same thing (as in "language construct" way -I could probably figure out a faster way to do this in a function).
UPDATE: The first solution ALMOST works. It works fine in Mac/iOS. However, the Linux version of Swift 3 doesn't seem to have the uniqueKeysWithValues initializer, which is annoying.
func gimme_a_new_dictionary(_ inKeyArray:[Int]) -> [Int:Int] {
return Dictionary<Int,Int>(uniqueKeysWithValues: inKeyArray.map {($0, 0)})
}
let test1 = gimme_a_new_dictionary([4,6,1,3,0,1000])
For Swift 4, you can use the dictionary constructor that takes a sequence and use map to create the sequence of tuples from your array of keys:
let dict = Dictionary(uniqueKeysWithValues: [4,6,1,3,0,1000].map {($0, 0)})
I presume you could optimize your code in terms of allocation by specifying the minimum capacity during the initialization. However, one liner may be the above answer, it's essentially allocation and looping to add 0 in each position.
func gimme_a_new_dictionary(_ inKeyArray:[Int], minCapacity: Int) -> [Int:Int] {
var ret = Dictionray<Int, Int>(minimumCapacity: minCapacity)
for key in inKeyArray {
ret[key] = 0
}
return ret
}
let test1 = gimme_a_new_dictionary([4,6,1,3,0,1000])
Take a look at this official documentation:
/// Use this initializer to avoid intermediate reallocations when you know
/// how many key-value pairs you are adding to a dictionary. The actual
/// capacity of the created dictionary is the smallest power of 2 that
/// is greater than or equal to `minimumCapacity`.
///
/// - Parameter minimumCapacity: The minimum number of key-value pairs to
/// allocate buffer for in the new dictionary.
public init(minimumCapacity: Int)
I wonder, what is the use case for Collection underestimateCount? Documentation says that it has the same complexity as standard Collection count.
/// Returns a value less than or equal to the number of elements in
/// `self`, *nondestructively*.
///
/// - Complexity: O(N).
public func underestimateCount() -> Int
But it doesn't describe when it should be used and for what reason.
underestimatedCount is actually a requirement of the Sequence protocol, and has a default implementation that just returns 0:
public var underestimatedCount: Int {
return 0
}
However, for sequences that provide their own implementation of underestimatedCount, this can be useful for logic that needs a lower bound of how long the sequence is, without having to iterate through it (remember that Sequence gives no guarantee of non-destructive iteration).
For example, the map(_:) method on Sequence (see its implementation here) uses underestimateCount in order to reserve an initial capacity for the resultant array:
public func map<T>(
_ transform: (Iterator.Element) throws -> T
) rethrows -> [T] {
let initialCapacity = underestimatedCount
var result = ContiguousArray<T>()
result.reserveCapacity(initialCapacity)
// ...
This allows map(_:) to minimise the cost of repeatedly appending to the result, as an initial block of memory has (possibly) already been allocated for it (although its worth noting in any case that ContiguousArray has an exponential growth strategy that amortises the cost of appending).
However, in the case of a Collection, the default implementation of underestimateCount actually just returns the collection's count:
public var underestimatedCount: Int {
// TODO: swift-3-indexing-model - review the following
return numericCast(count)
}
Which will be an O(1) operation for collections that conform to RandomAccessCollection, O(n) otherwise.
Therefore, because of this default implementation, using a Collection's underestimatedCount directly is definitely less common than using a Sequence's, as Collection guarantees non-destructive iteration, and in most cases underestimatedCount will just return the count.
Although, of course, custom collection types could provide their own implementation of underestimatedCount – giving a lower bound of how many elements they contain, in a possibly more efficient way than their count implementation, which could potentially be useful.
(Since the duplicate target I've suggested is somewhat outdated)
In Swift 3, the method underestimateCount() has been replaced by the computed property underestimatedCount. We can have a look at the source code for the implementation of the latter for Collection:
/// A value less than or equal to the number of elements in the collection.
///
/// - Complexity: O(1) if the collection conforms to
/// `RandomAccessCollection`; otherwise, O(*n*), where *n* is the length
/// of the collection.
public var underestimatedCount: Int {
// TODO: swift-3-indexing-model - review the following
return numericCast(count)
}
/// The number of elements in the collection.
///
/// - Complexity: O(1) if the collection conforms to
/// `RandomAccessCollection`; otherwise, O(*n*), where *n* is the length
/// of the collection.
public var count: IndexDistance {
return distance(from: startIndex, to: endIndex)
}
Its apparent that underestimatedCount simply makes use of count for types conforming to Collection (unless these types implements their own version of underestimatedCount).
I am writing an iOS application using Apple's new Metal framework. I have an array of Matrix4 objects (see Ray Wenderlich's tutorial) that I need to pass in to a shader via the MTLDevice.newBufferWithLength() method. The Matrix4 object is leveraging Apple's GLKit (it contains a GLKMatrix4 object).
I'm leveraging instancing with the GPU calls.
I will later change this to a struct which includes more data per instance (beyond just the Matrix4 object.
How can I efficiently copy the array of [Matrix4] objects into this buffer?
Is there a better way to do this? Again, I'll expand this to use a struct with more data in the future.
Below is a subset of my code:
let sizeofMatrix4 = sizeof(Float) * Matrix4.numberofElements()
// This returns an array of [Matrix4] objects.
let boxArray = createBoxArray(parentModelViewMatrix)
let sizeOfUniformBuffer = boxArray.count * sizeOfMatrix4
var uniformBuffer = device.newBufferWithLength(sizeofUniformBuffer, options: .CPUCacheModeDefaultCache)
let bufferPointer = uniformBuffer?.contents()
// Ouch - way too slow. How can I optimize?
for i in 0..<boxArray.count
{
memcpy(bufferPointer! + (i * sizeOfMatrix4), boxArray[i].raw(), sizeOfMatrix4)
}
renderEncoder.setVertexBuffer(uniformBuffer, offset: 0, atIndex: 2)
Note:
The boxArray[i].raw() method is defined as this in the Objective-C code:
- (void *)raw {
return glkMatrix.m;
}
You can see I'm looping through each array object and then doing a memcpy. I did this since I was experiencing problems treating the array as a contiguous set of memory.
Thanks!
A Swift Array is promised to be contiguous memory, but you need to make sure it's really a Swift Array and not secretly an NSArray. If you want to be completely certain, use a ContiguousArray. That will ensure contiguous memory even if the objects in it are bridgeable to ObjC. If you want even more control over the memory, look at ManagedBuffer.
With that, you should be using newBufferWithBytesNoCopy(length:options:deallocator) to create a MTL buffer around your existing memory.
I've done this with an array of particles that I pass to a compute shader.
In a nutshell, I define some constants and declare a handful of mutable pointers and a mutable buffer pointer:
let particleCount: Int = 1048576
var particlesMemory:UnsafeMutablePointer<Void> = nil
let alignment:UInt = 0x4000
let particlesMemoryByteSize:UInt = UInt(1048576) * UInt(sizeof(Particle))
var particlesVoidPtr: COpaquePointer!
var particlesParticlePtr: UnsafeMutablePointer<Particle>!
var particlesParticleBufferPtr: UnsafeMutableBufferPointer<Particle>!
When I set up the particles, I populate the pointers and use posix_memalign() to allocate the memory:
posix_memalign(&particlesMemory, alignment, particlesMemoryByteSize)
particlesVoidPtr = COpaquePointer(particlesMemory)
particlesParticlePtr = UnsafeMutablePointer<Particle>(particlesVoidPtr)
particlesParticleBufferPtr = UnsafeMutableBufferPointer(start: particlesParticlePtr, count: particleCount)
The loop to populate the particles is slightly different - I now loop over the buffer pointer:
for index in particlesParticleBufferPtr.startIndex ..< particlesParticleBufferPtr.endIndex
{
[...]
let particle = Particle(positionX: positionX, positionY: positionY, velocityX: velocityX, velocityY: velocityY)
particlesParticleBufferPtr[index] = particle
}
Inside the applyShader() function, I create a copy of the memory which is used as both the input and output buffer:
let particlesBufferNoCopy = device.newBufferWithBytesNoCopy(particlesMemory, length: Int(particlesMemoryByteSize),
options: nil, deallocator: nil)
commandEncoder.setBuffer(particlesBufferNoCopy, offset: 0, atIndex: 0)
commandEncoder.setBuffer(particlesBufferNoCopy, offset: 0, atIndex: 1)
...and after the shader has run, I put the shared memory (particlesMemory) back into the buffer pointer:
particlesVoidPtr = COpaquePointer(particlesMemory)
particlesParticlePtr = UnsafeMutablePointer(particlesVoidPtr)
particlesParticleBufferPtr = UnsafeMutableBufferPointer(start: particlesParticlePtr, count: particleCount)
There's an up to date Swift 2.0 version of this at my GitHub repo here
Obviously the point of using shared memory and MTLDevice.makeBuffer(bytesNoCopy:...) is to avoid redundant memory copies. Therefore, ideally we look for a design that allows us to easily manipulate the data after it's already been loaded into the MTLBuffer object.
After researching this for a while, I've decided to try and create a semi-generic solution to allow for simplified allocation of page-aligned memory, loading your content into that memory, and subsequently manipulating your items in that shared memory block.
I've created a Swift array implementation called PageAlignedArray that matches the interface and functionality of the built-in Swift array, but always resides on page-aligned memory, and so can be very easily made into an MTLBuffer. I've also added a convenience method to directly convert PageAlignedArray into a Metal buffer.
Of course, you can continue to mutate your array afterwards and your updates will be automatically available to the GPU courtesy of the shared-memory architecture. However, keep in mind that you must regenerate your MTLBuffer object whenever the array's length changes.
Here's a quick code sample:
var alignedArray : PageAlignedContiguousArray<matrix_double4x4> = [matrixTest, matrixTest]
alignedArray.append(item)
alignedArray.removeFirst() // Behaves just like a built-in array, with all convenience methods
// When it's time to generate a Metal buffer:
let testMetalBuffer = device?.makeBufferWithPageAlignedArray(alignedArray)
The sample uses matrix_double4x4, but the array should work for any Swift value types. Please note that if you use a reference type (such as any kind of class), the array will contain pointers to your elements and so won't be usable from your GPU code.