Swift: best way to send large arrays of numbers ([Double]) over HTTP - ios

Question

I thought I'd ask after hours of inconclusive research and tests:
Introduction
I'm trying to send very large arrays of Doubles from an app to a server, naturally, I want to compress this as much as possible.
Specifically, these array contain CMDeviceMotion components (acceleration, x, y, z, gyroscope, etc...), but this question should apply to any large array of numbers (over 100K or a million values)
What I've tried and found by researching options
Say I have a large array of Double (There are many others) :
var CMX = CM.map({$0.userAcceleration.x})
here, CMX is of type [Double] and CM is [CMDeviceMotion]
I've tried making POST requests to my server by sending CMX in different ways, then calculating the total size after I receive it on the server :
First, as a single comma separated string :
{"AX":"-0.0441827848553658,-0.103976868093014,-0.117475733160973,-0.206566318869591,-0.266509801149368,-0.282151937484741,-0.260240525007248,-0.266505032777786,-0.315020948648453,-0.305839896202087,0.0255246963351965,0.0783950537443161,0.0749507397413254,0.0760494321584702,-0.0101579604670405,0.106710642576218,0.131824940443039,0.0630970001220703,0.21177926659584,0.27022996544838,0.222621202468872,0.234281644225121,0.288497060537338,0.176655143499374,0.193904414772987,0.169417425990105,0.150193274021149,0.00871349219232798,-0.0270088445395231,-0.0 ....
Size 153 Kb.
It makes sense that this is larger than sending as binary data, since a single number here is 64 bits (8 bytes), and becomes 17 bytes long (one byte per character) +1 = 18 (added a character for the comma).
With this reasoning, sending the array as binary data should be smaller.
Base 64 encoding
Here, I convert the array to a Data object using NSKeyedArchiver and base 64 encode the data before sending it :
["AX":NSKeyedArchiver.archivedData(withRootObject:CM.map({$0.userAcceleration.x})).base64EncodedString()]
This made the file size 206 Kb
Sending the data as a JSON array
By just sending :
["AX": CM.map({$0.userAcceleration.x})]
It turned out that this array of numbers was practically converted to a comma separated string, the size ended up being the same as in trial 1 (160Kb)
Sending as Data without base 64 encoding
Doing this:
["AX":NSKeyedArchiver.archivedData(withRootObject:CM.map({$0.userAcceleration.x}))
made the application crash at runtime, so I can't send a Data object as a value in a JSON
Question
How can I send these array in a more condensed way in a JSON object ?
Note that I already have downsampling in mind, and using 32 bit floats to reduce the size.

Answer 1

Simple way would be to do this:
let data: Data = CMX.withUnsafeBufferPointer { pointer in
return Data(buffer: pointer)
}
And you have binary buffer with all your Doubles/Floats combined.
But because HTTP is text-based protocol you will have to convert this data to base64 string:
let base64String = data.base64EncodedString()
And this base64String should be passed for AX parameter of your POST(?) HTTP request.
EDIT:
To convert it back you may use code like this:
extension Array {
init?(data: Data) {
// This check should be more complex, but here we just check if total byte count divides to one element size in bytes
guard data.count % MemoryLayout<Element>.size == 0 else { return nil }
let elementCount = data.count / MemoryLayout<Element>.size
let buffer = UnsafeMutableBufferPointer<Element>.allocate(capacity: elementCount)
data.copyBytes(to: buffer)
self = buffer.map({$0})
buffer.deallocate()
}
// Wrapped here code above
var data: Data {
return self.withUnsafeBufferPointer { pointer in
return Data(buffer: pointer)
}
}
}
let converted: [Double]? = Array(data: CMX.data) // converted now should be equal to CMX