I wanted to append Different byte storage types like Int8, Int16, Int32, .. into data. It has become very complex as every time I have to convert one by one to data and append the same.
How to create extension for Data which will accept Int8, Int16, Int16 Int32 in any order and return bytes
Typically the below code should work. I tried but. not able to finish the extension method
let value1: Int8 = 0x01
let value2: Int16 = 0x0001
let value3: Int32 = 0x00000001
let data = Data.getBytes(value1, value2, value3)
extension Data {
static func getBytes(...) -> Data {
// How to proceed?
}
}
First, you need a common type that will be accepted by getBytes. Unfortunately we cannot use FixedWidthInteger directly, because it contains a Self constraint. We cannot use FixedWidthInteger as a generic parameter either, because then we would not be able to mix Int8 and Int16 and so on in a single call.
Therefore, instead of using Any, I would suggest to introduce a new protocol MyIntType and let all the types you want to support implement this:
protocol MyIntType {
// ...
}
extension Int8: MyIntType { }
extension Int16: MyIntType { }
extension Int32: MyIntType { }
extension Data {
static func getBytes(_ values : MyIntType...) -> Data { /* ... */ }
}
In getBytes, you need to access the internal bytes from the concrete Int-Type, e.g. call something like Data(bytes: &intValue, count: MemoryLayout<Int8>.size). Fortunately, Int8 etc. support something like bitWidth (which is similar to the MemoryLayout) because the all conform to the FixedWidthInteger protocol.
Nevertheless, we cannot use FixedWidthInteger directly in the getBytes function, because it contains a Self constraint.
As a work-around, we just need to add the bitWidth property to our MyIntType protocol:
protocol MyIntType {
var bitWidth:Int { get }
}
All those Intx-Types already implement it, so we do not need an extension for them, and can simply use it in getBytes:
extension Data {
static func getBytes (_ values:MyIntType...) -> Data {
var d = Data()
for var v in values {
let data = Data(bytes:&v, count:v.bitWidth / 8)
d.append(contentsOf:data)
}
return d
}
}
Now we can check it:
let value1: Int8 = 0x01
let value2: Int16 = 0x0010
let value3: Int32 = 0x00000100
let bytes = Data.getBytes(value1, value2, value3)
print (bytes)
for b in bytes {
print (b)
}
Related
Within my app, I have a MTLBuffer which is being instantiated using a generic type. In one particular case, the buffer will hold values as related to particles in a point cloud, and is defined as such;
struct ParticleUniforms {
simd_float3 position;
simd_float3 color;
float confidence;
};
I am instantiating my MTLBuffer like so;
guard let buffer = device.makeBuffer(length: MemoryLayout<Element>.stride * count, options: options) else {
fatalError("Failed to create MTLBuffer.")
}
Where I am struggling, however, is to understand how to read the contents of the buffer. More-so, I am looking to copy one element of each item in the buffer to an array on the CPU, which I will use at a later time.
Effectively, the buffer holds a collection of ParticleUniforms, and I would like to access the position value of each item, saving that position to a separate array.
All of the examples I've seen here on Stack Overflow seem to show the MTLBuffer as holding a collection of Floats, though I've not seen any that use a generic type.
It seems what you are looking to achieve can only be done with C structures which hold each member in a contiguous block (arrays of C structs are not necessarily contiguous, but MemoryLayout<Type>.stride will account for any potential padding). Swift structure properties may not be contiguous, so the below method for accessing member values would not work in a practical manner. Unfortunately, when working with void* you need to know what the data describes, which isn't particularly suited for Swift generic types. However, I will offer a potential solution.
C file:
#ifndef Test_h
#define Test_h
#include <simd/simd.h>
typedef struct {
vector_float3 testA;
vector_float3 testB;
} CustomC;
#endif /* Test_h */
Swift file (bridging header assumed)
import Metal
// MARK: Convenience
typealias MTLCStructMemberFormat = MTLVertexFormat
#_functionBuilder
struct ArrayLayout { static func buildBlock<T>(_ arr: T...) -> [T] { arr } }
extension MTLCStructMemberFormat {
var stride: Int {
switch self {
case .float2: return MemoryLayout<simd_float2>.stride
case .float3: return MemoryLayout<simd_float3>.stride
default: fatalError("Case unaccounted for")
}
}
}
// MARK: Custom Protocol
protocol CMetalStruct {
/// Returns the type of the `ith` member
static var memoryLayouts: [MTLCStructMemberFormat] { get }
}
// Custom Allocator
class CustomBufferAllocator<Element> where Element: CMetalStruct {
var buffer: MTLBuffer!
var count: Int
init(bytes: UnsafeMutableRawPointer, count: Int, options: MTLResourceOptions = []) {
guard let buffer = device.makeBuffer(bytes: bytes, length: count * MemoryLayout<Element>.stride, options: options) else {
fatalError("Failed to create MTLBuffer.")
}
self.buffer = buffer
self.count = count
}
func readBufferContents<T>(element_position_in_array n: Int, memberID: Int, expectedType type: T.Type = T.self)
-> T {
let pointerAddition = n * MemoryLayout<Element>.stride
let valueToIncrement = Element.memoryLayouts[0..<memberID].reduce(0) { $0 + $1.stride }
return buffer.contents().advanced(by: pointerAddition + valueToIncrement).bindMemory(to: T.self, capacity: 1).pointee
}
func extractMembers<T>(memberID: Int, expectedType type: T.Type = T.self) -> [T] {
var array: [T] = []
for n in 0..<count {
let pointerAddition = n * MemoryLayout<Element>.stride
let valueToIncrement = Element.memoryLayouts[0..<memberID].reduce(0) { $0 + $1.stride }
let contents = buffer.contents().advanced(by: pointerAddition + valueToIncrement).bindMemory(to: T.self, capacity: 1).pointee
array.append(contents)
}
return array
}
}
// Example
// First extend the custom struct to conform to out type
extension CustomC: CMetalStruct {
#ArrayLayout static var memoryLayouts: [MTLCStructMemberFormat] {
MTLCStructMemberFormat.float3
MTLCStructMemberFormat.float3
}
}
let device = MTLCreateSystemDefaultDevice()!
var CTypes = [CustomC(testA: .init(59, 99, 0), testB: .init(102, 111, 52)), CustomC(testA: .init(10, 11, 5), testB: .one), CustomC(testA: .zero, testB: .init(5, 5, 5))]
let allocator = CustomBufferAllocator<CustomC>(bytes: &CTypes, count: 3)
let value = allocator.readBufferContents(element_position_in_array: 1, memberID: 0, expectedType: simd_float3.self)
print(value)
// Prints SIMD3<Float>(10.0, 11.0, 5.0)
let group = allocator.extractMembers(memberID: 1, expectedType: simd_float3.self)
print(group)
// Prints [SIMD3<Float>(102.0, 111.0, 52.0), SIMD3<Float>(1.0, 1.0, 1.0), SIMD3<Float>(5.0, 5.0, 5.0)]
This is similar to a MTLVertexDescriptor, except the memory is accessed manually and not via the [[stage_in]] attribute and the argument table passed to each instance of a vertex of fragment shader. You could even extend the allocator to accept a string parameter with the name of the property and hold some dictionary which maps to member IDs.
With Swift 3 leaning towards Data instead of [UInt8], I'm trying to ferret out what the most efficient/idiomatic way to encode/decode swifts various number types (UInt8, Double, Float, Int64, etc) as Data objects.
There's this answer for using [UInt8], but it seems to be using various pointer APIs that I can't find on Data.
I'd like to basically some custom extensions that look something like:
let input = 42.13 // implicit Double
let bytes = input.data
let roundtrip = bytes.to(Double) // --> 42.13
The part that really eludes me, I've looked through a bunch of the docs, is how I can get some sort of pointer thing (OpaquePointer or BufferPointer or UnsafePointer?) from any basic struct (which all of the numbers are). In C, I would just slap an ampersand in front of it, and there ya go.
Note: The code has been updated for Swift 5 (Xcode 10.2) now. (Swift 3 and Swift 4.2 versions can be found in the edit history.) Also possibly unaligned data is now correctly handled.
How to create Data from a value
As of Swift 4.2, data can be created from a value simply with
let value = 42.13
let data = withUnsafeBytes(of: value) { Data($0) }
print(data as NSData) // <713d0ad7 a3104540>
Explanation:
withUnsafeBytes(of: value)
invokes the closure with a buffer pointer covering the raw bytes of the value.
A raw buffer pointer is a sequence of bytes, therefore Data($0) can be used to create the data.
How to retrieve a value from Data
As of Swift 5, the withUnsafeBytes(_:) of Data invokes the closure with an “untyped” UnsafeMutableRawBufferPointer to the bytes. The load(fromByteOffset:as:) method the reads the value from the memory:
let data = Data([0x71, 0x3d, 0x0a, 0xd7, 0xa3, 0x10, 0x45, 0x40])
let value = data.withUnsafeBytes {
$0.load(as: Double.self)
}
print(value) // 42.13
There is one problem with this approach: It requires that the memory is property aligned for the type (here: aligned to a 8-byte address). But that is not guaranteed, e.g. if the data was obtained as a slice of another Data value.
It is therefore safer to copy the bytes to the value:
let data = Data([0x71, 0x3d, 0x0a, 0xd7, 0xa3, 0x10, 0x45, 0x40])
var value = 0.0
let bytesCopied = withUnsafeMutableBytes(of: &value, { data.copyBytes(to: $0)} )
assert(bytesCopied == MemoryLayout.size(ofValue: value))
print(value) // 42.13
Explanation:
withUnsafeMutableBytes(of:_:) invokes the closure with a mutable buffer pointer covering the raw bytes of the value.
The copyBytes(to:) method of DataProtocol (to which Data conforms) copies bytes from the data to that buffer.
The return value of copyBytes() is the number of bytes copied. It is equal to the size of the destination buffer, or less if the data does not contain enough bytes.
Generic solution #1
The above conversions can now easily be implemented as generic methods of struct Data:
extension Data {
init<T>(from value: T) {
self = Swift.withUnsafeBytes(of: value) { Data($0) }
}
func to<T>(type: T.Type) -> T? where T: ExpressibleByIntegerLiteral {
var value: T = 0
guard count >= MemoryLayout.size(ofValue: value) else { return nil }
_ = Swift.withUnsafeMutableBytes(of: &value, { copyBytes(to: $0)} )
return value
}
}
The constraint T: ExpressibleByIntegerLiteral is added here so that we can easily initialize the value to “zero” – that is not really a restriction because this method can be used with “trival” (integer and floating point) types anyway, see below.
Example:
let value = 42.13 // implicit Double
let data = Data(from: value)
print(data as NSData) // <713d0ad7 a3104540>
if let roundtrip = data.to(type: Double.self) {
print(roundtrip) // 42.13
} else {
print("not enough data")
}
Similarly, you can convert arrays to Data and back:
extension Data {
init<T>(fromArray values: [T]) {
self = values.withUnsafeBytes { Data($0) }
}
func toArray<T>(type: T.Type) -> [T] where T: ExpressibleByIntegerLiteral {
var array = Array<T>(repeating: 0, count: self.count/MemoryLayout<T>.stride)
_ = array.withUnsafeMutableBytes { copyBytes(to: $0) }
return array
}
}
Example:
let value: [Int16] = [1, Int16.max, Int16.min]
let data = Data(fromArray: value)
print(data as NSData) // <0100ff7f 0080>
let roundtrip = data.toArray(type: Int16.self)
print(roundtrip) // [1, 32767, -32768]
Generic solution #2
The above approach has one disadvantage: It actually works only with "trivial"
types like integers and floating point types. "Complex" types like Array
and String have (hidden) pointers to the underlying storage and cannot be
passed around by just copying the struct itself. It also would not work with
reference types which are just pointers to the real object storage.
So solve that problem, one can
Define a protocol which defines the methods for converting to Data and back:
protocol DataConvertible {
init?(data: Data)
var data: Data { get }
}
Implement the conversions as default methods in a protocol extension:
extension DataConvertible where Self: ExpressibleByIntegerLiteral{
init?(data: Data) {
var value: Self = 0
guard data.count == MemoryLayout.size(ofValue: value) else { return nil }
_ = withUnsafeMutableBytes(of: &value, { data.copyBytes(to: $0)} )
self = value
}
var data: Data {
return withUnsafeBytes(of: self) { Data($0) }
}
}
I have chosen a failable initializer here which checks that the number of bytes provided
matches the size of the type.
And finally declare conformance to all types which can safely be converted to Data and back:
extension Int : DataConvertible { }
extension Float : DataConvertible { }
extension Double : DataConvertible { }
// add more types here ...
This makes the conversion even more elegant:
let value = 42.13
let data = value.data
print(data as NSData) // <713d0ad7 a3104540>
if let roundtrip = Double(data: data) {
print(roundtrip) // 42.13
}
The advantage of the second approach is that you cannot inadvertently do unsafe conversions. The disadvantage is that you have to list all "safe" types explicitly.
You could also implement the protocol for other types which require a non-trivial conversion, such as:
extension String: DataConvertible {
init?(data: Data) {
self.init(data: data, encoding: .utf8)
}
var data: Data {
// Note: a conversion to UTF-8 cannot fail.
return Data(self.utf8)
}
}
or implement the conversion methods in your own types to do whatever is
necessary so serialize and deserialize a value.
Byte order
No byte order conversion is done in the above methods, the data is always in
the host byte order. For a platform independent representation (e.g.
“big endian” aka “network” byte order), use the corresponding integer
properties resp. initializers. For example:
let value = 1000
let data = value.bigEndian.data
print(data as NSData) // <00000000 000003e8>
if let roundtrip = Int(data: data) {
print(Int(bigEndian: roundtrip)) // 1000
}
Of course this conversion can also be done generally, in the generic
conversion method.
You can get an unsafe pointer to mutable objects by using withUnsafePointer:
withUnsafePointer(&input) { /* $0 is your pointer */ }
I don't know of a way to get one for immutable objects, because the inout operator only works on mutable objects.
This is demonstrated in the answer that you've linked to.
In my case, Martin R's answer helped but the result was inverted. So I did a small change in his code:
extension UInt16 : DataConvertible {
init?(data: Data) {
guard data.count == MemoryLayout<UInt16>.size else {
return nil
}
self = data.withUnsafeBytes { $0.pointee }
}
var data: Data {
var value = CFSwapInt16HostToBig(self)//Acho que o padrao do IOS 'e LittleEndian, pois os bytes estavao ao contrario
return Data(buffer: UnsafeBufferPointer(start: &value, count: 1))
}
}
The problem is related with LittleEndian and BigEndian.
I am trying to create an enum of a struct that I would like to initialize:
struct CustomStruct {
var variable1: String
var variable2: AnyClass
var variable3: Int
init (variable1: String, variable2: AnyClass, variable3: Int) {
self.variable1 = variable1
self.variable2 = variable2
self.variable3 = variable3
}
}
enum AllStructs: CustomStruct {
case getData
case addNewData
func getAPI() -> CustomStruct {
switch self {
case getData:
return CustomStruct(variable1:"data1", variable2: SomeObject.class, variable3: POST)
case addNewData:
// Same to same
default:
return nil
}
}
}
I get the following errors:
Type AllStructs does not conform to protocol 'RawRepresentable'
I am assuming that enums cannot be used this way. We must use primitives.
It should be:
struct CustomStruct {
var apiUrl: String
var responseType: AnyObject
var httpType: Int
init (variable1: String, variable2: AnyObject, variable3: Int) {
self.apiUrl = variable1
self.responseType = variable2
self.httpType = variable3
}
}
enum MyEnum {
case getData
case addNewData
func getAPI() -> CustomStruct {
switch self {
case .getData:
return CustomStruct(variable1: "URL_TO_GET_DATA", variable2: 11 as AnyObject, variable3: 101)
case .addNewData:
return CustomStruct(variable1: "URL_TO_ADD_NEW_DATA", variable2: 12 as AnyObject, variable3: 102)
}
}
}
Usage:
let data = MyEnum.getData
let myObject = data.getAPI()
// this should logs: "URL_TO_GET_DATA 11 101"
print(myObject.apiUrl, myObject.responseType, myObject.httpType)
Note that upon Naming Conventions, struct should named as CustomStruct and enum named as MyEnum.
In fact, I'm not pretty sure of the need of letting CustomStruct to be the parent of MyEnum to achieve what are you trying to; As mentioned above in the snippets, you can return an instance of the struct based on what is the value of the referred enum.
I'm not commenting on the choice to use an enum here, but just explaining why you got that error and how to declare an enum that has a custom object as parent.
The error shows you the problem, CustomStruct must implement RawRepresentable to be used as base class of that enum.
Here is a simplified example that shows you what you need to do:
struct CustomStruct : ExpressibleByIntegerLiteral, Equatable {
var rawValue: Int = 0
init(integerLiteral value: Int){
self.rawValue = value
}
static func == (lhs: CustomStruct, rhs: CustomStruct) -> Bool {
return
lhs.rawValue == rhs.rawValue
}
}
enum AllStructs: CustomStruct {
case ONE = 1
case TWO = 2
}
A few important things that we can see in this snippet:
The cases like ONE and TWO must be representable with a Swift literal, check this Swift 2 post for a list of available literals (int,string,array,dictionary,etc...). But please note that in Swift 3, the LiteralConvertible protocols are now called ExpressibleByXLiteral after the Big Swift Rename.
The requirement to implement RawRepresentable is covered implementing one of the Expressible protocols (init?(rawValue:) will leverage the initializer we wrote to support literals).
Enums must also be Equatable , so you'll have to implement the equality operator for your CustomStruct base type.
Did you try conforming to RawRepresentable like the error is asking?
Using JSON representation should work for variable1 and variable3. Some extra work may be required for variable2.
struct CustomStruct: RawRepresentable {
var variable1: String
var variable2: AnyClass
var variable3: Int
init?(rawValue: String) {
guard let data = rawValue.data(using: .utf8) else {
return nil
}
guard let json = try? JSONSerialization.jsonObject(with: data, options: []) as? [String: Any] else {
return nil
}
self.variable1 = (json["variable1"] as? String) ?? ""
self.variable2 = (json["variable2"] as? AnyClass) ?? AnyClass()
self.variable3 = (json["variable3"] as? Int) ?? 0
}
var rawValue: String {
let json = ["variable1": self.variable1,
"variable2": self.variable2,
"variable3": self.variable3
]
guard let data = try? JSONSerialization.data(withJSONObject: json, options: []) else {
return ""
}
return String(data: data, encoding: .utf8) ?? ""
}
}
According to the documentation:
If a value (known as a “raw” value) is provided for each enumeration case, the value can be a string, a character, or a value of any integer or floating-point type.
So yes, you cannot set a struct type to be enum's raw value.
In your case I would suggest using string as the enum raw value and some dictionary mapping these strings to CUSTOM_STRUCT type.
A bit late on the party but maybe useful for someone else. I would consider to simply use computed variables instead of a struct.
enum MyEnum {
case getData
case addNewData
var variable1: String {
switch self {
case .getData: return "data1"
case .addNewData: return "data2"
}
}
var variable2: Int {
switch self {
case .getData: return 1
case .addNewData: return 2
}
}
// ....
}
Usage:
let data = MyEnum.getData
print (data.variable1) // "data1"
I'm trying to gain a deeper understanding of how Swift copies value types:
The behavior you see in your code will always be as if a copy took
place. However, Swift only performs an actual copy behind the scenes
when it is absolutely necessary to do so.
To advance my understanding, I'd like to get the memory address of a value type. I tried unsafeAddressOf(), but this doesn't work with structs and it seems to cast Swift's standard library types to reference types (e.g. String is cast to NSString).
How can I get the memory address of a value type, like an instance of Int, or a custom struct in Swift?
According to Martin R' s answer
addressOf() cannot be used with struct variables. String is a struct, however, it is automatically bridged to NSString when passed to a function expecting an object.
According to nschum's answer, you can get the (stack) address of a struct, build-in type or object reference like this:
import UIKit
func address(o: UnsafePointer<Void>) -> Int {
return unsafeBitCast(o, Int.self)
}
func addressHeap<T: AnyObject>(o: T) -> Int {
return unsafeBitCast(o, Int.self)
}
struct myStruct {
var a: Int
}
class myClas {
}
//struct
var struct1 = myStruct(a: 5)
var struct2 = struct1
print(NSString(format: "%p", address(&struct1))) // -> "0x10f1fd430\n"
print(NSString(format: "%p", address(&struct2))) // -> "0x10f1fd438\n"
//String
var s = "A String"
var aa = s
print(NSString(format: "%p", address(&s))) // -> "0x10f43a430\n"
print(NSString(format: "%p", address(&aa))) // -> "0x10f43a448\n"
//Class
var class1 = myClas()
var class2 = class1
print(NSString(format: "%p", addressHeap(class1))) // -> 0x7fd5c8700970
print(NSString(format: "%p", addressHeap(class2))) // -> 0x7fd5c8700970
unsafeAddressOf(class1) //"UnsafePointer(0x7FD95AE272E0)"
unsafeAddressOf(class2) //"UnsafePointer(0x7FD95AE272E0)"
//Int
var num1 = 55
var num2 = num1
print(NSString(format: "%p", address(&num1))) // -> "0x10f1fd480\n"
print(NSString(format: "%p", address(&num2))) // -> "0x10f1fd488\n"
One thing I found is, if myStruct has no value, the address will be retain same:
struct myStruct {
}
var struct1 = myStruct()
var struct2 = struct1
print(NSString(format: "%p", address(&struct1))) // -> ""0xa000000000070252\n""
print(NSString(format: "%p", address(&struct2))) // -> ""0xa000000000070252\n""
Swift 2.0 :
You can use this unsafeAddressOf(someObject)
or in Swift 3.0:
use withUnsafePointer(to: someObejct) { print("\($0)") }
I'm not sure if there's a "recommended" way to do that, but one method is to use withUnsafePointer(_:_:), like this:
var s: String = "foo"
withUnsafePointer(&s) { NSLog("\($0)") }
This printed 0x00007ffff52a011c8 on my machine.
In my app I am doing something like this:
struct Record {
var exampleData : String
}
class ExampleClass : UIViewController {
let records = [Record]()
override func viewDidLoad() {
super.viewDidLoad()
let data = NSKeyedArchiver.archivedDataWithRootObject(self.records) as! NSData
}
...
}
But in the last line of viewDidLoad() I got this error:
Argument type '[Record]' does not conform to expected type 'AnyObject'
How can I fix this? Thanks.
If you want to keep struct, you can encode data using withUnsafePointer(). Here's an example, which I adapted from this Gist:
import UIKit
enum EncodingStructError: ErrorType {
case InvalidSize
}
func encode<T>(var value: T) -> NSData {
return withUnsafePointer(&value) { p in
NSData(bytes: p, length: sizeofValue(value))
}
}
func decode<T>(data: NSData) throws -> T {
guard data.length == sizeof(T) else {
throw EncodingStructError.InvalidSize
}
let pointer = UnsafeMutablePointer<T>.alloc(1)
data.getBytes(pointer, length: data.length)
return pointer.move()
}
enum Result<T> {
case Success(T)
case Failure
}
I added some error handling and marked the method as throws. Here's one way you can use it, in a do…catch block:
var res: Result<String> = .Success("yeah")
var data = encode(res)
do {
var decoded: Result<String> = try decode(data)
switch decoded {
case .Failure:
"failure"
case .Success(let v):
"success: \(v)" // => "success: yeah"
}
} catch {
print(error)
}
The error handling I added will not decode if the NSData length doesn't match the type size. This can commonly happen if you write the data to disk, the user updates to a newer version of the app with a different-sized version of the same type, and then the data is read in.
Also note that sizeof() and sizeofValue() may return different values on different devices, so this isn't a great solution for sending data between devices (NSJSONSerialization might be better for that).
AnyObject means any reference type object, primarily a class. A struct is a value type and cannot be passed to a function needing an AnyObject. Any can be used to accept value types as well as reference types. To fix your code above, change struct Record to class Record. But I have a feeling you may want to use a struct for other reasons. You can create a class wrapper around Record that you can convert to and from to use for functions that need an AnyObject.
I did a similar thing:
static func encode<T>(value: T) -> NSData {
var val = value
return withUnsafePointer(to: &val) { pointer in
NSData(bytes: pointer, length: MemoryLayout.size(ofValue: val))
}
}
static func decode<T>(data: NSData) -> T {
guard data.length == MemoryLayout<T>.size.self else {
fatalError("[Credential] fatal unarchiving error.")
}
let pointer = UnsafeMutablePointer<T>.allocate(capacity: 1)
data.getBytes(pointer, length: data.length)
return pointer.move()
}