I try to add bytes with bytes in order to get space that will be occupied by photos which i retrieve from internet.
I have following code, it gets sizes in bytes for each id in array of id
var diskSpace:Int64 = 0
for var i = 0; i < array.count; i++ {
let id = array[i]
let urlString = "urlToFetchData"
if let url = NSURL(string: urlString) {
if let data = try? NSData(contentsOfURL: url, options: []) {
let json = JSON(data: data)
let size = Int64(json["size"].stringValue)
diskSpace = diskSpace + size!
}
}
}
var diskSpaceInMb = diskSpace / 1024 / 1024
print("diskSpaceInMb is \(diskSpaceInMb)")
for example, I try to get size of three elements, which have following size in bytes (these sizes in bytes I receive in json)
3223653
5855382
8948976
when the code above is executed i receive result of
diskSpaceInMb is 8
which is obviously not try
How to convert bytes to megabytes correctly ?
let fileSizeWithUnit = ByteCountFormatter.string(fromByteCount: diskSpace, countStyle: .file)
print("File Size: \(fileSizeWithUnit)")
The problem is obviously in the for loop. Maybe the JSON is not as you expect.
Another reason why this might fail is the you use the try? keyword, which in this context it means that it gives you a value if it succeeds, but otherwise it returns nil. In your case, it may silently fail. If you want to check if it fails, you could add an else branch.
Related
I am writing a point cloud file and need to keep updating the file header with the total number of points in a file: vertexCount. I don't know when the points will stop coming, so I can not just keep accumulating the values and waiting to write it to file.
The vertexCount value is kept on line 3 of ascii file, which is newline terminated.
I only see examples and functions that append data to the end of the file using write(to: URL, options: .atomic)
How can I use FileHandle to replace a specific line in a file, or overwrite the entire header?
ply
format ascii 1.0
element vertex \(vertexCount)
I see this question about replacing file contents using an array. Due to the file having at least 400 thousand lines, I do not want to separate it into individual lines. I was thinking of separating it on the end_header keyword, and then generating a new header, but am not sure how efficient this is.
Well the issue you will face is that when the numbers of digits increase it will overwrite the characters after it. You will need to use a fixed numbers of digits to be able to write them exactly over it (something like 0000000001). The number of lines doesn't really matter because it will replace any new line character after the last digit.
extension FixedWidthInteger where Self: CVarArg {
func strZero(maxLength: Int) -> String {
String(format: "%0*d", maxLength, self)
}
func write(toPLYFile atURL: URL) throws {
let fileHandle = try FileHandle(forUpdating: atURL)
try fileHandle.seek(toOffset: 36)
try fileHandle.write(contentsOf: Data(strZero(maxLength: 10).utf8))
fileHandle.closeFile()
}
}
var vertexCount = 1
let text = """
ply
format ascii 1.0
element vertex \(vertexCount.strZero(maxLength: 10))
abcdefghijklmnopqrstuvwxyz
1234567890
"""
print(text)
print("=========")
let fileURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
.appendingPathComponent("file.txt")
try Data(text.utf8).write(to: fileURL, options: .atomic)
let fileHandle = try FileHandle(forUpdating: fileURL)
try fileHandle.seek(toOffset: 36)
vertexCount = 12345
try fileHandle.write(contentsOf: Data(vertexCount.strZero(maxLength: 10).utf8))
fileHandle.closeFile()
let stringLoaded = try String(contentsOf: fileURL)
print(stringLoaded)
This will print
ply
format ascii 1.0
element vertex 0000000001
abcdefghijklmnopqrstuvwxyz
1234567890
=========
ply
format ascii 1.0
element vertex 0000012345
abcdefghijklmnopqrstuvwxyz
1234567890
Updated use:
do {
// filepath to PLY file that is being updated
let url = URL(fileURLWithPath: path)
let totalVertexCount = 12345
try totalVertexCount.write(toPLYFile: url)
} catch {
print("Error writing PLY! \(error)")
return
}
This is just something I've written really quick, definitely look into making it look and work a bit better - but should be enough to demonstrate the concept. Needs some testing to see if this is actually more efficient than separating line by line.
What it's essentially doing is it's reading the file byte by byte into an array until it finds the beginning of the third line. Then, it copies our new string into the buffer. After that, it's looking for the beginning of the fourth line and copying the rest of the file into the buffer.
I'm also calculating the total byte size so that I can trim the buffer at the end.
var finalFileByteLength = 0;
if let d = NSData(contentsOfFile: filePath) {
let newLine = "element vertex \(vertexCount)\n".data(using: .ascii)! as NSData
var buffer = [UInt8](repeating: 0, count: d.length + newLine.length)
var bytePosition = 0
var lineCount = 0
while(true) {
//Read one Byte
d.getBytes(&buffer+bytePosition, range: NSMakeRange(bytePosition, 1))
//If it's a new line character
if(buffer[bytePosition] == 10) {
lineCount += 1
//If it found the end of the second line, copy our new line
if lineCount == 2 {
newLine.getBytes(&buffer+(bytePosition+1), length: newLine.length)
bytePosition += 1
break
}
}
bytePosition += 1
finalFileByteLength+=1
}
var oldLine3Length = 0
finalFileByteLength+=newLine.length
//Find the start of the fourth line in the initial file
while(true) {
//Read one Byte
var char = UInt8()
d.getBytes(&char, range: NSMakeRange(bytePosition, 1))
//If it's a new line character
if(char == 10) {
//If it found the end of the third line, break so we have the start of the fourth line
bytePosition += 1
oldLine3Length += 1
break
}
bytePosition += 1
oldLine3Length += 1
}
//Header is now modified, copy the rest of the file
d.getBytes(&buffer+(bytePosition+newLine.length-oldLine3Length), range: NSMakeRange(bytePosition, d.length - bytePosition))
finalFileByteLength+=d.length - bytePosition + 1
let finalFileData = NSData(bytes: &buffer, length: finalFileByteLength)
//Print the result - this is probably where you'll write the entire String to a file
print(String(data: finalFileData as Data, encoding: .ascii))
}
EDIT: Managed to reduce to this:
if let d = NSData(contentsOfFile: filePath) {
let newLine = "element vertex \(vertexCount)".data(using: .ascii)! as NSData
var newLineBuffer = [UInt8](repeating: 0, count: newLine.length)
newLine.getBytes(&newLineBuffer, length: newLine.length)
var buffer = [UInt8](repeating: 0, count: d.length)
d.getBytes(&buffer, length: d.length)
var thirdIndex = buffer.firstIndex(of: 10)
thirdIndex = buffer[buffer.index(after: thirdIndex!)...].firstIndex(of: 10)
thirdIndex = buffer[buffer.index(after: thirdIndex!)...].firstIndex(of: 10)
var fourthIndex = buffer[buffer.index(after: thirdIndex!)...].firstIndex(of: 10)
buffer.removeSubrange(thirdIndex!+1..<fourthIndex!)
buffer.insert(contentsOf: newLineBuffer, at: thirdIndex!+1)
let finalFileData = NSData(bytes: buffer, length: buffer.count) as Data
print(String(data:finalFileData, encoding: .ascii))
}
I am reading data from an MFi external device into a buffer using a 3rd party SDK "sessionController". See below:
let handle: UInt64 = self.sessionController.openFile(file.path, mode: openMode)
if handle == 0 {
//Error
return
}
let c: UInt64 = file.size
var bytesArray: [UInt8] = [UInt8](fileData)
let bufferPointer: UnsafeMutablePointer<UInt8> = UnsafeMutablePointer<UInt8>.allocate(capacity: Int(c))
bufferPointer.initialize(repeating: 0, count: Int(c))
defer {
bufferPointer.deinitialize(count: Int(c))
bufferPointer.deallocate()
}
var sum: UInt32 = 0
let singleSize: UInt32 = 8 << 20
while sum < c {
let read = self.sessionController.readFile(handle, data: bufferPointer, len: singleSize)
if read == 0 {
//There was an error
return
}
sum += read
}
let newPointer : UnsafeRawPointer = UnsafeRawPointer(bufferPointer)
fileURL = try! FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: false).appendingPathComponent("test.MOV")
fileData = Data(bytes: newPointer, count: Int(c))
try! fileData.write(to: fileURL)
//Now use this fileURL to watch video in an AVPlayer...
//AVPlayer(init: fileURL)
For some reason the data stored at the fileURL becomes corrupted (I think) and I am unable to play the video file. I think I am not doing something correctly with Unsafe Swift but I am not sure what. How can I make sure that I have properly read the data from the device into memory, and then taken that data from memory and stored it on the hard drive at the fileURL? What am I doing wrong here? The video will not play in AVPlayer given the fileURL.
The main error is here:
let read = self.sessionController.readFile(handle, data: bufferPointer, len: singleSize)
If you read in multiple chunks then the second and all subsequent reads will overwrite the data read previously. So that should probably be
let read = self.sessionController.readFile(handle, data: bufferPointer + sum, len: singleSize)
Note also that the file size is defined as UInt64, but the variable sum (which holds the total number of bytes read so far) is an UInt32. This will lead to problems if there is more than 4GB data.
But generally I would avoid to read the complete data into a memory buffer. You already read in chunks, so you can write the data immediately to the destination file. Here is how that could look like:
// Output file:
let fileURL = ...
let fileHandle = try FileHandle(forWritingTo: fileURL)
defer { fileHandle.closeFile() }
// Buffer:
let bufferSize = 1024 * 1024 // Choose some buffer size
var buffer = Data(count: bufferSize)
// Read/write loop:
let fileSize: UInt64 = file.size
var remainingToRead = fileSize
while remainingToRead > 0 {
let read = buffer.withUnsafeMutableBytes { bufferPointer in
self.sessionController.readFile(handle, data: bufferPointer, len: UInt32(min(remainingToRead, UInt64(bufferSize))))
}
if read == 0 {
return // Read error
}
remainingToRead -= UInt64(read)
fileHandle.write(buffer)
}
Note also that the data is read directly into a Data value, instead of reading it into allocated memory and then copying it to another Data.
So I have some code I've been using to upload files in my app, along the lines of this:
var mutableURLRequest = URLRequest(url: url)
var uploadData = try! Data(contentsOf: dataUrl)
session.uploadTask(with: mutableURLRequest, from: uploadData).resume()
There's a little more to it than that, but those are the relevant parts. However I've noticed for some large video files Data(contentsOf: dataUrl) fails since the file is to big to load into memory. I want to restructure this so that I'm able to stream piece by piece to the server without ever having to load the whole file into memory.
I already have this figured out from my server, the only piece I haven't figured out is how to get a chunkSize piece from the data in a URL, without putting it into a data object. I essentially want this construct:
let chunkSize = 1024 * 1024
let offset = 0
let chunk = //get data from dataUrl of size chunkSize offset by offset
//Upload each chunk and increment offset
NSInputStream seemed promising in being able to do this, but I wasn't able to figure out the minimum set up in order to pull bytes from a file on disk in this fashion. What code can I use above to fill in the let chunk = line to do such a task?
I have a working solution, might need a little tweaking, but seems to work for big files I've tried:
public func getNextChunk() -> Data?{
if _inputStream == nil {
_inputStream = InputStream(url: _url)
_inputStream?.open()
}
var buffer = [UInt8](repeating: 0, count: CHUNK_SIZE)
var len = _inputStream?.read(&buffer, maxLength: CHUNK_SIZE)
if len == 0 {
return nil
}
return Data(buffer)
}
I also call _inputStream?.close() on deinit of my class that manages the chunking of a file on disk.
I'm trying to concatenate two NSData objects into one NSMutableData, and than get them back. For now i'm trying to do it in such way:
Get length of first object.
Write into NSMutableData in such order: first object length, first object, second object.
Code looks like:
let firstString = "first_string";
let secondString = "secondSting";
let firstData = firstString.dataUsingEncoding(NSUTF8StringEncoding, allowLossyConversion: false)!
let secondData = secondString.dataUsingEncoding(NSUTF8StringEncoding, allowLossyConversion: false)!
let mutableData = NSMutableData()
var length = firstData.length
mutableData.appendBytes(&length, length: sizeof(Int))
mutableData.appendData(firstData)
mutableData.appendData(secondData)
Then I want to get datas back. So I suppose to read first data length and then get two datas.
var length = 0
mutableData.getBytes(&length, length: sizeof(Int))
But when I'm trying to get data I'm getting crash instead:
var data = NSData()
mutableData.getBytes(&data, range: NSMakeRange(sizeof(Int), length))
Maybe somebody know where is my problem or how to get datas?
You can extract the data using subdataWithRange():
let firstData1 = mutableData.subdataWithRange(NSMakeRange(sizeof(Int), length))
if let firstString1 = NSString(data: firstData1, encoding: NSUTF8StringEncoding) as? String {
println(firstString1)
} else {
// bad encoding
}
Your solution
var data = NSData()
mutableData.getBytes(&data, range: NSMakeRange(sizeof(Int), length))
does not work and crashes because NSData is a reference type and
data a pointer to the object. You are overwriting this pointer
and the following bytes in memory.
This works perfectly without a crash in my storyboard. I just omitted the second var before length in order to avoid redefining it.
Here is the output for each line:
"first_string"
"secondSting"
<66697273 745f7374 72696e67> // let firstData = ...
<7365636f 6e645374 696e67> // let secondData = ...
<> // let mutableData = ...
12 // var length = ...
// appending data
<0c000000 00000000>
<0c000000 00000000 66697273 745f7374 72696e67>
<0c000000 00000000 66697273 745f7374 72696e67 7365636f 6e645374 696e67>
0 // length = 0
<0c000000 00000000 66697273 745f7374 72696e67 7365636f 6e645374 696e67>
12 // length
This means you probably have an error somewhere else. You did not redefine length, right?
I keep getting this error :
fatal error: unexpectedly found nil while unwrapping an Optional value
and cannot figure out how to debug it!
Here's my code :
func readCSV() -> Array<String> {
// Creates a new array of strings
var csvArray : Array<String> = Array<String>()
if let url: NSURL = NSURL(string : "URLFROMCSV" ) {
// Creates an Input Stream that will load the datas from our URL
let data :NSData! = NSData(contentsOfURL: url)!
let stream : NSInputStream! = NSInputStream(data: data)
// Opens the receiving stream
stream.open()
// Sets a buffer with a given size size
let bufferSize = 1024
var buffer = Array <UInt8>(count: bufferSize, repeatedValue: 0)
// String variable initialization
var csvFullString : String = ""
// While the stream receives datas, parses datas, convert them into strings and then concatenate them into one big string
while (stream.hasBytesAvailable) {
let readSize = stream.read(&buffer, maxLength: bufferSize)
let csvRaw = NSString (bytes: &buffer, length: readSize, encoding: NSUTF8StringEncoding)
let csvString = csvRaw as String!
csvFullString = csvFullString + csvString
}
// Fills the array with each strings. Separation between strings is made when a Θ character is parsed
csvArray = csvFullString.componentsSeparatedByString("Θ")
// Delete each null string
for(var i = 0 ; i < csvArray.count; i++) {
if(csvArray[i] == "") {
csvArray.removeAtIndex(i)
}
}
}
return csvArray
}
After searching on the web, I'm pretty sure it has something to do with unwrapping elements but the fact is when I debug it, i don't get any nil value anywhere.
PS: Would like to upload a screen but can't because i don't have 10 reputation, so bad!
Thanks in advance!
EDIT : Line let data :NSData! = NSData(contentsOfURL: url)! got the error.
Terry
You're probably creating the error in one of these two lines (though it may show up later):
let data :NSData! = NSData(contentsOfURL: url)!
let stream : NSInputStream! = NSInputStream(data: data)
You're assigning an optional value to an implicitlyUnwrappedOptional type and then using it without checking if you have a valid value.
This is why if let exists. It's a little funny that you've started to indent as if you're using if let but aren't.
Try this instead:
if let url = NSURL(string : "http://gorillaapplications.com/etablissements.csv" ) {
// Creates an Input Stream that will load the datas from our URL
if let data = NSData(contentsOfURL: url) {
let stream = NSInputStream(data: data)
stream.open()
// rest of your code here
}
else {
println("Didn't get a data object")
}
}
else {
println("Didn't get a URL object")
}
You really need to grasp Optionals for Swift. I'd recommend reading my Optionals chapter in this iBook: https://itunes.apple.com/us/book/swift-optionals-generics-for/id943445214?mt=11&uo=4&at=11lMGu
Update:
Since you added a bit more in your comments above, you're saying you get the error on this line: let data: NSData! = NSData(contentsOfURL: url)!. This is because of the ! at the end, which tells Swift you're sure that this function will return a valid value, so just use it, without checking if it's nil first. In your case, the function is returning nil and so your app crashes. Using the sample code I've provided above, you'll see that you'll no longer get a crash, but your code will execute the "Didn't get a data object" line. You'll need to correctly handle the case where you can't load data from that URL.