String returns only numbers after separatedBy - ios

Question

I´m trying to separate a string like the following:
let path = "/Users/user/Downloads/history.csv"
do {
let contents = try NSString(contentsOfFile: path, encoding: String.Encoding.utf8.rawValue )
let rows = contents.components(separatedBy: "\n")
print("contents: \(contents)")
print("rows: \(rows)")
}
catch {
}
I have two files, which are looking almost identical.
From the first file the output is like this:
Output File1:
contents: 2017-07-31 16:29:53,0.10109999,9.74414271,0.98513273,0.15%,42302999779,-0.98513273,9.72952650
2017-07-31 16:29:53,0.10109999,0.25585729,0.02586716,0.25%,42302999779,-0.02586716,0.25521765
rows: ["2017-07-31 16:29:53,0.10109999,9.74414271,0.98513273,0.15%,42302999779,-0.98513273,9.72952650", "2017-07-31 16:29:53,0.10109999,0.25585729,0.02586716,0.25%,42302999779,-0.02586716,0.25521765", "", ""]
Output File2:
contents: 40.75013313,0.00064825,5/18/2017 7:17:01 PM
19.04004820,0.00059900,5/19/2017 9:17:03 PM
rows: ["4\00\0.\07\05\00\01\03\03\01\03\0,\00\0.\00\00\00\06\04\08\02\05\0,\05\0/\01\08\0/\02\00\01\07\0 \07\0:\01\07\0:\00\01\0 \0P\0M\0", "\0", "1\09\0.\00\04\00\00\04\08\02\00\0,\00\0.\00\00\00\05\09\09\00\00\0,\0\05\0/\01\09\0/\02\00\01\07\0 \09\0:\01\07\0:\00\03\0 \0P\0M\0", "\0", "\0", "\0"]
So both files are readable as String because the print(content) is working.
But as soon as the string gets separated, the second file is not readable anymore.
I tried different encodings, but nothing worked. Has anyone an idea, how to force the string to the second file, to remain a readable string?

Answer 1

Your file is apparently UTF-16 (little-endian) encoded:
$ hexdump fullorders4.csv
0000000 4f 00 72 00 64 00 65 00 72 00 55 00 75 00 69 00
0000010 64 00 2c 00 45 00 78 00 63 00 68 00 61 00 6e 00
0000020 67 00 65 00 2c 00 54 00 79 00 70 00 65 00 2c 00
0000030 51 00 75 00 61 00 6e 00 74 00 69 00 74 00 79 00
...
For ASCII characters, the first byte of the UTF-16 encoding is the
ASCII code, and the second byte is zero.
If the file is read as UTF-8 then the zeros are converted to an
ASCII NUL character, that is what you see as \0 in the output.
Therefore specifying the encoding as utf16LittleEndian works
in your case:
let contents = try NSString(contentsOfFile: path, encoding: String.Encoding.utf16LittleEndian.rawValue)
// or:
let contents = try String(contentsOfFile: path, encoding: .utf16LittleEndian)
There is also a method which tries to detect the used encoding
(compare iOS: What's the best way to detect a file's encoding). In Swift that would be
var enc: UInt = 0
let contents = try NSString(contentsOfFile: path, usedEncoding: &enc)
// or:
var enc = String.Encoding.ascii
let contents = try String(contentsOfFile: path, usedEncoding: &enc)
However, in your particular case, that would read the file as UTF-8
again because it is valid UTF-8. Prepending a byte order mark (BOM)
to the file (FF FE for UTF-16 little-endian) would solve that
problem reliably.