I have a text file with 8000+ lines of information. I want to parse it to JSON for using it in iOS application. Because parsing that kind of txt file takes about 35 seconds, which is not good for user experience.
Text file goes like this:
TURKIYE;ADANA;ADANA;36,9914194;35,3308285;03:00:00
TURKIYE;ADANA;ALADAĞ;37,545348;35,394608;03:00:00
TURKIYE;ADANA;CEYHAN;37,028765;35,8124309;03:00:00
TURKIYE;ADANA;FEKE;37,814467;35,910391;03:00:00
TURKIYE;ADANA;İMAMOĞLU;37,2984443;35,6095474;03:00:00
TURKIYE;ADANA;KARAISALI;37,2758825;35,1268781;03:00:00
TURKIYE;ADANA;KARATAŞ;36,6649776;35,2587964;03:00:00
I just want the "TURKIYE", "ADANA", "ALADAĞ", "Latitude" and "Longitude" parts and don't need the last "03:00:00" part.
Edit:
Forget the mention about something. Text files some lines doesn't includes third column. For example;
AVUSTURYA;HORBRANZ;47,5557073;9,7525947;01:00:00
AVUSTURYA;HORN;48,66607;15,65716;01:00:00
AVUSTURYA;IMST;47,24013;10,73954;01:00:00
AVUSTURYA;IMSTERAU;47,21018;10,70901;01:00:00
I want to parse them like "if third column doesn't exist print nil value there."
My txt parsing code is:
let textFilePath = Bundle.main.path(forResource: "LatLongData", ofType: "txt")
let fileManager = FileManager.default
if fileManager.fileExists(atPath: textFilePath!) {
do {
let fullText = try String(contentsOfFile: textFilePath!, encoding: String.Encoding.utf8)
let lines = fullText.components(separatedBy: "\n") as [String]
for line in lines {
let data = line.components(separatedBy: ";")
let locationData = LocationData()
if data.first == "TURKIYE" || data.first == "ABD" {
locationData.country = data[0]
locationData.city = data[1]
locationData.district = data[2]
locationData.latitude = data[3]
locationData.longitude = data[4]
locationData.compoundKey = "\(data[0])-\(data[1])-\(data[3])"
} else {
locationData.country = data[0]
locationData.city = data[1]
locationData.latitude = data[2]
locationData.longitude = data[3]
locationData.compoundKey = "\(data[0])-\(data[1])-\(data[3])"
}
locationData.writeToRealmDB()
}
} catch let error as NSError {
print(error.localizedDescription)
}
How can I convert this text to JSON for using in iOS app?
Thanks in advance.
import pandas as pd
with open('filename.txt', 'r') as f:
lines = f.readlines()
for line in lines:
if line.count(';') == 4:
index = line.index(';', 2)
line = line[:index] + ';' + line[index:]
with open('filename2.txt', 'a') as f2:
f2.write(line)
df = pd.read_csv('filename2.txt', header=None, sep=';')
df = df.iloc[:, :-1]
df.columns = ['col1', 'col2', 'col3', 'Latitude', 'Longitude']
df.to_json('filename.json')
http://pandas.pydata.org/pandas-docs/stable/io.html
Related
I've txt file with content:
("All our dreams can come true, if we have the courage to pursue them.","Walt Disney")
("The secret of getting ahead is getting started","Mark Twain")
I want to get array of tuples from it with type [(String, String)]. I try to use code:
do {
if let path = Bundle.main.path(forResource: "quotes", ofType: "txt"){
let data = try String(contentsOfFile: path, encoding: .utf8)
let arrayOfStrings = data.components(separatedBy: "\n")
print(arrayOfStrings[0])
}
} catch let err as NSError {
// do something with Error
print(err)
}
But with it I cannot get tuple values. How I can get array of tuples from txt file with Swift?
As already mentioned in comments by Larme it would be better to properly format your text. If you can't change the text format you woill need to manually parse its contents:
let data = """
("All our dreams can come true, if we have the courage to pursue them.","Walt Disney")
("The secret of getting ahead is getting started","Mark Twain")
"""
let tuples = data.split(whereSeparator: \.isNewline)
.compactMap { line -> (Substring,Substring)? in
let comps = line.components(separatedBy: #"",""#)
guard comps.count == 2,
let lhs = comps.first?.dropFirst(2),
let rhs = comps.last?.dropLast(2) else { return nil }
return (lhs,rhs)
}
for tuple in tuples {
print(tuple.0)
print(tuple.1)
}
This will print:
All our dreams can come true, if we have the courage to pursue them.
Walt Disney
The secret of getting ahead is getting started
Mark Twain
There is an array like:
let datas =
[["#A1CCE4","", "+0.0%", "+0.0%"],
["#4C3C2F","G", "+1.0%", "+0.2%"],
["#4C3C2F","G", "+3.1%", "+0.6%"],
["#C07155","S", "+0.3%", "+0.1%"],
["#C07155","G", "+2.0%", "+0.4%"],
["#C07155","P", "+1.8%", "+0.3%"],
["#AEB0B3","R", "+2.0%", "+2.0%"]]
How to convert it to string like:
"""let data = [["#A1CCE4", "", "+0.0%", "+0.0%"], ["#4C3C2F", "G", "+1.0%", "+0.2%"], ["#4C3C2F", "G", "+3.1%", "+0.6%"], ["#C07155", "S", "+0.3%", "+0.1%"], ["#C07155", "G", "+2.0%", "+0.4%"], ["#C07155", "P", "+1.8%", "+0.3%"], ["#AEB0B3", "R", "+2.0%", "+2.0%"]]"""
(with "let data = " or not is fine)
EDIT:
"\(datas)" should be the answer.
or
"\(String(describing: datas))"
or
datas.description
Why do I ask this question?
The array actually from the plist, and the number values are from different places.
The number values need to be load in the js code in WKWebView. And acutually the js code is just string.
In my js code I have actually the same code datas variable code.
You actually want to create a JSON string, it's pretty easy with JSONEncoder because [String] conforms to Encodable by default.
let datas =
[["#A1CCE4","", "+0.0%", "+0.0%"],
["#4C3C2F","G", "+1.0%", "+0.2%"],
["#4C3C2F","G", "+3.1%", "+0.6%"],
["#C07155","S", "+0.3%", "+0.1%"],
["#C07155","G", "+2.0%", "+0.4%"],
["#C07155","P", "+1.8%", "+0.3%"],
["#AEB0B3","R", "+2.0%", "+2.0%"]]
do {
let jsonData = try JSONEncoder().encode(datas)
let result = String(data: jsonData, encoding: .utf8)!
print(result)
} catch { print(error) }
It's more reliable than just calling .description on the array.
And only an array literal is directly interchangeable with JSON, a dictionary is not.
If you really need the let datas = prefix just concatenate the strings
let resultWithPrefix = "let datas = " + result
Given your datas array, you can achieve your goal this way:
let datas =
[["#A1CCE4","", "+0.0%", "+0.0%"],
["#4C3C2F","G", "+1.0%", "+0.2%"],
["#4C3C2F","G", "+3.1%", "+0.6%"],
["#C07155","S", "+0.3%", "+0.1%"],
["#C07155","G", "+2.0%", "+0.4%"],
["#C07155","P", "+1.8%", "+0.3%"],
["#AEB0B3","R", "+2.0%", "+2.0%"]]
var str = " let datas = ["
for data in datas{
str += "\(data)"
}
str += "]"
EDIT:
You can use this code, with a loop for each String array in your data, to get a string with comma
var str = " let datas = ["
for data in datas{
str += "["
for s in data{
str += " \" \(s) \" , "
}
str += "],"
}
str += "]"
Another good solution is to use JSONEncoder, as suggested in other answer, I just report it here to complete
let datas = [...]
var myString = "let datas ="
do{
let datasToJson = JSONEncoder().encode(datas)
myString += String(data: datasToJson, encoding: .utf8)!
}catch{
print(error)
}
How about something like this:
let stringStart = "let datas = "
let stringDatas = String(datas)
let finalString = stringStart + stringDatas
More answers here
I am trying to parse a CSV but i am getting some issues. Below is the code i used for parsing CSV:
let fileURL = Bundle.main.url(forResource: "test_application_data - Sheet 1", withExtension: "csv")
let content = try String(contentsOf: fileURL!, encoding: String.Encoding.utf8)
let parsedCSV: [[String]] = content.components(separatedBy: "\n").map{ $0.components(separatedBy: ",")}
And this is the data in the CSV i am parsing :
Item 9,Description 9,image url
"Item 10 Extra line 1 Extra line 2 Extra line 3",Description 10,image url
So by using above code i get correct response for first row i.e Item 9 but i am getting malformed response for Item 10
How can i correctly parse both rows?
The RFC for CSV: Common Format and MIME Type for Comma-Separated Values (CSV) Files(RFC-4180)
Not all CSV data or CSV processors conform to all descriptions of this RFC, but generally, fields enclosed within double-quotes can contain:
newlines
commas
escaped double-quotes ("" represents a single double-quote)
This code is a little bit simplified than RFC-4180, but handles all three cases above:
UPDATE This old code does not handle CRLF well. (Which is a valid newline in RFC-4180.) I added a new code at the bottom, please check it.
Thanks to Jay.
import Foundation
let csvText = """
Item 9,Description 9,image url
"Item 10
Extra line 1
Extra line 2
Extra line 3",Description 10,image url
"Item 11
Csv item can contain ""double quote"" and comma(,)", Description 11 ,image url
"""
let pattern = "[ \r\t]*(?:\"((?:[^\"]|\"\")*)\"|([^,\"\\n]*))[ \t]*([,\\n]|$)"
let regex = try! NSRegularExpression(pattern: pattern)
var result: [[String]] = []
var record: [String] = []
let offset: Int = 0
regex.enumerateMatches(in: csvText, options: .anchored, range: NSRange(0..<csvText.utf16.count)) {match, flags, stop in
guard let match = match else {fatalError()}
if match.range(at: 1).location != NSNotFound {
let field = csvText[Range(match.range(at: 1), in: csvText)!].replacingOccurrences(of: "\"\"", with: "\"")
record.append(field)
} else if match.range(at: 2).location != NSNotFound {
let field = csvText[Range(match.range(at: 2), in: csvText)!].trimmingCharacters(in: .whitespaces)
record.append(field)
}
let separator = csvText[Range(match.range(at: 3), in: csvText)!]
switch separator {
case "\n": //newline
result.append(record)
record = []
case "": //end of text
//Ignoring empty last line...
if record.count > 1 || (record.count == 1 && !record[0].isEmpty) {
result.append(record)
}
stop.pointee = true
default: //comma
break
}
}
print(result)
(Intended to test in a Playground.)
New code, CRLF ready.
import Foundation
let csvText = "Field0,Field1\r\n"
let pattern = "[ \t]*(?:\"((?:[^\"]|\"\")*)\"|([^,\"\r\\n]*))[ \t]*(,|\r\\n?|\\n|$)"
let regex = try! NSRegularExpression(pattern: pattern)
var result: [[String]] = []
var record: [String] = []
let offset: Int = 0
regex.enumerateMatches(in: csvText, options: .anchored, range: NSRange(0..<csvText.utf16.count)) {match, flags, stop in
guard let match = match else {fatalError()}
if let quotedRange = Range(match.range(at: 1), in: csvText) {
let field = csvText[quotedRange].replacingOccurrences(of: "\"\"", with: "\"")
record.append(field)
} else if let range = Range(match.range(at: 2), in: csvText) {
let field = csvText[range].trimmingCharacters(in: .whitespaces)
record.append(field)
}
let separator = csvText[Range(match.range(at: 3), in: csvText)!]
switch separator {
case "": //end of text
//Ignoring empty last line...
if record.count > 1 || (record.count == 1 && !record[0].isEmpty) {
result.append(record)
}
stop.pointee = true
case ",": //comma
break
default: //newline
result.append(record)
record = []
}
}
print(result) //->[["Field0", "Field1"]]
The problem is with this line of code:
content.components(separatedBy: "\n")
It separates your csv file into rows based on the newline character. There are newline characters in your "Item 10 Extra line 1 Extra line 2 Extra line 3" String so each extra line is getting treated as a different row, so in the end you get the wrong result.
I'd suggest escaping the newline characters in your multiline text column or getting rid of them altogether. You can also modyfy the input file so the newline delimeter isn't \n at the end of each row but something custom (a string that won't appear elsewhere in the csv file).
I'm looking for a fast, optimized way to trim log files on iOS. I want to specify that my log files have a maximum number of lines (e.g., 10,000). Appending new lines to the end of a text file seems relatively simple. However, I haven't yet found a fast way to trim lines at the beginning of the file. Here's the (slow) code I came up with.
guard let fileURL = self.fileURL else {
return
}
guard let path = fileURL.path else {
return
}
guard let fileHandle = NSFileHandle(forUpdatingAtPath: path) else {
return
}
fileHandle.seekToEndOfFile()
fileHandle.writeData(message.dataUsingEncoding(NSUTF8StringEncoding)!)
fileHandle.writeData("\n".dataUsingEncoding(NSUTF8StringEncoding)!)
currentLineCount += 1
// TODO: This could probably use some major optimization
if currentLineCount >= maxLineCount {
if let fileString = try? NSString(contentsOfURL: fileURL, encoding: NSUTF8StringEncoding) {
var lines = fileString.componentsSeparatedByCharactersInSet(NSCharacterSet.newlineCharacterSet())
lines.removeFirst()
let newData = lines.joinWithSeparator("\n")
fileHandle.seekToFileOffset(0)
fileHandle.writeData(newData.dataUsingEncoding(NSUTF8StringEncoding)!)
}
}
fileHandle.closeFile()
There are two aspects of your question. First, your code removes a single
line from the log file. Therefore, once the limit is reached, every new
log message causes the entire file to be read, shortened, and be re-written.
It would be more effective to use a "high-water mark" and a "low-water mark". For example, if you want the last 10.000 lines to be preserved,
let the log file grow until it has 15.000 lines, and only then truncate
it to 10.000 lines. This reduces the number of "trim actions"
considerably.
The second part is about the truncating itself. Your code loads the
file into an NSString, which requires the conversion of UTF-8 data
to Unicode characters
(and fails if there is a single invalid byte in the log file).
Then the string is split into an array, one array element removed,
the array concatenated to a string again, and then written back
to the file, which converts the Unicode characters to UTF-8.
I haven't done performance tests, but I can imagine that it could be
faster to operate on binary data only, without the conversions to
NSString, Array and back. Here is a possible implementation
which removes a given number of lines from the start of a file:
func removeLinesFromFile(fileURL: NSURL, numLines: Int) {
do {
let data = try NSData(contentsOfURL: fileURL, options: .DataReadingMappedIfSafe)
let nl = "\n".dataUsingEncoding(NSUTF8StringEncoding)!
var lineNo = 0
var pos = 0
while lineNo < numLines {
// Find next newline character:
let range = data.rangeOfData(nl, options: [], range: NSMakeRange(pos, data.length - pos))
if range.location == NSNotFound {
return // File has less than `numLines` lines.
}
lineNo++
pos = range.location + range.length
}
// Now `pos` is the position where line number `numLines` begins.
let trimmedData = data.subdataWithRange(NSMakeRange(pos, data.length - pos))
trimmedData.writeToURL(fileURL, atomically: true)
} catch let error as NSError {
print(error.localizedDescription)
}
}
I have updated Martin R answer to Swift 3, and I also changed it so that we can pass the number of lines to keep instead of the number of lines to remove:
func removeLinesFromFile(fileURL: URL, linesToKeep numLines: Int) {
do {
let data = try Data(contentsOf: fileURL, options: .dataReadingMapped)
let nl = "\n".data(using: String.Encoding.utf8)!
var lineNo = 0
var pos = data.count-1
while lineNo <= numLines {
// Find next newline character:
guard let range = data.range(of: nl, options: [ .backwards ], in: 0..<pos) else {
return // File has less than `numLines` lines.
}
lineNo += 1
pos = range.lowerBound
}
let trimmedData = data.subdata(in: pos..<data.count)
try trimmedData.write(to: fileURL)
} catch let error as NSError {
print(error.localizedDescription)
}
}
Instead of writing the new line to the log, and processing the appended content afterwards, do both write and trimming in one step:
let fileString = (try? NSString(contentsOfURL: fileURL, encoding: NSUTF8StringEncoding)) as NSString? ?? ""
var lines = fileString.characters.split("\n").map{String($0)}
lines.append(message)
// this also more generic as it will remove any number of extra lines
lines.removeFirst(max(currentLineCount - maxLineCount), 0))
let newLogContents = lines.joinWithSeparator("\n")
(newLogContents as NSString).writeToURL(fileURL, atomically: true, encoding: NSUTF8StringEncoding)
I am parsing data from csv file to dictionary with the help of github.
After parsing I am getting this type of dictionary :-
{
"" = "";
"\"barred_date\"" = "\"\"";
"\"company_id\"" = "\"1\"";
"\"company_name\"" = "\"\"";
"\"contact_no\"" = "\"1234567890\"";
"\"created_date\"" = "\"2015-06-01 12:43:11\"";
"\"current_project\"" = "\"111\"";
"\"designation\"" = "\"Developer\"";
"\"doj\"" = "\"2015-06-01 00:00:00\"";
"\"fin_no\"" = "\"ABC001\"";
"\"first_name\"" = "\"sssd\"";
"\"last_name\"" = "\"dd\"";
"\"project_name\"" = "\"Project 1\"";
"\"qr_code\"" = "\"12345678\"";
"\"resignation_date\"" = "\"\"";
"\"status\"" = "\"1\"";
"\"work_permit_no\"" = "\"ssdda11\"";
"\"worker_id\"" = "\"1\"";
"\"worker_image\"" = "\"assets/uploads/workers/eb49364ca5c5d22f11db2e3c84ebfce6.jpeg\"";
"\"worker_image_thumb\"" = "\"assets/uploads/workers/thumbs/eb49364ca5c5d22f11db2e3c84ebfce6.jpeg\"";}
How can I convert this to simple dictionary. I need data like this "company_id" = "1"
Thanks
I recommend using CSVImporter – it takes care of things like quoted text (following RFC 4180) for you and even handles very large files without problems.
Compared to other solutions it works both asynchronously (prevents delays) and reads your CSV file line by line instead of loading the entire String into memory (prevents memory issues). On top of that it is easy to use and provides beautiful callbacks for indicating failure, progress, completion and even data mapping if you desire to.
You can use it like this to get an array of Strings per line:
let path = "path/to/your/CSV/file"
let importer = CSVImporter<[String]>(path: path)
importer.startImportingRecords { $0 }.onFinish { importedRecords in
for record in importedRecords {
// record is of type [String] and contains all data in a line
}
}
Take advantage of more sophisticated features like header structure support like this:
// given this CSV file content
firstName,lastName
Harry,Potter
Hermione,Granger
Ron,Weasley
// you can import data in Dictionary format
let path = "path/to/Hogwarts/students"
let importer = CSVImporter<[String: String]>(path: path)
importer.startImportingRecords(structure: { (headerValues) -> Void in
// use the header values CSVImporter has found if needed
print(headerValues) // => ["firstName", "lastName"]
}) { $0 }.onFinish { importedRecords in
for record in importedRecords {
// a record is now a Dictionary with the header values as keys
print(record) // => e.g. ["firstName": "Harry", "lastName": "Potter"]
print(record["firstName"]) // prints "Harry" on first, "Hermione" on second run
print(record["lastName"]) // prints "Potter" on first, "Granger" on second run
}
}
Use the CSwiftV parser instead: https://github.com/Daniel1of1/CSwiftV
It actually handles quoted text, and therefore it handles both line breaks and commas in text. SwiftCSV cost me some time in that it doesn't handle that. But I did learn about the CSV format and parsing it ;)
Parse CSV to two-dimension array of Strings (rows and columns)
func parseCsv(_ data: String) -> [[String]] {
// data: String = contents of a CSV file.
// Returns: [[String]] = two-dimension array [rows][columns].
// Data minimum two characters or fail.
if data.count < 2 {
return []
}
var a: [String] = [] // Array of columns.
var index: String.Index = data.startIndex
let maxIndex: String.Index = data.index(before: data.endIndex)
var q: Bool = false // "Are we in quotes?"
var result: [[String]] = []
var v: String = "" // Column value.
while index < data.endIndex {
if q { // In quotes.
if (data[index] == "\"") {
// Found quote; look ahead for another.
if index < maxIndex && data[data.index(after: index)] == "\"" {
// Found another quote means escaped.
// Increment and add to column value.
data.formIndex(after: &index)
v += String(data[index])
} else {
// Next character not a quote; last quote not escaped.
q = !q // Toggle "Are we in quotes?"
}
} else {
// Add character to column value.
v += String(data[index])
}
} else { // Not in quotes.
if data[index] == "\"" {
// Found quote.
q = !q // Toggle "Are we in quotes?"
} else if data[index] == "\r" || data[index] == "\r\n" {
// Reached end of line.
// Column and row complete.
a.append(v)
v = ""
result.append(a)
a = []
} else if data[index] == "," {
// Found comma; column complete.
a.append(v)
v = ""
} else {
// Add character to column value.
v += String(data[index])
}
}
if index == maxIndex {
// Reached end of data; flush.
if v.count > 0 || data[data.index(before: index)] == "," {
a.append(v)
}
if a.count > 0 {
result.append(a)
}
break
}
data.formIndex(after: &index) // Increment.
}
return result
}
Call above with the CSV data
let dataArray: [[String]] = parseCsv(yourStringOfCsvData)
Then extract the header row
let dataHeader = dataArray.removeFirst()
I assume you want an array of dictionaries (most spreadsheet data includes mulitple rows, not just one). The next loop is for that. But if you only need a single row (and header for keys) into a single dictionary, you can study below and get the idea of how to get there.
var da: [Dictionary<String, String>] = [] // Array of dictionaries.
for row in dataArray {
for (index, column) in row.enumerated() {
var d: Dictionary<String, String> = Dictionary()
d.updateValue(column, forKey: dataHeader[index])
da.append(d)
}
}