Here is a sample of my CSV
10820 0 0 0 0
10900 2 4 4 4
11000 21 50 54 58
11100 23 54 59 63
11200 25 59 63 68
11300 27 63 68 73
11400 29 68 73 78
11500 31 72 78 83
11600 32 76 82 88
11700 34 81 87 93
I'm looking to create to use xcode to retreive one line of code from this very lengthy CSV based on the first line.
For example:
if the user enters "10900", the second line columns will be returned.
If the user returns 11650, the 11600 line columns will be returned...always taking the lower line when the input value is less then the following line.
Any help would be appreciated. I've seen code to parse an entire CSV file, but I'm thinking this may be a big memory drain, right now my CSV has 2000 lines of values, which are all in ascending order based on the first column.
You have to load a file into memory anyways to find correct value.
With such a big CSV file I would recommend to turn CSV file into binary file (plist file for example) and put it as binary into your application - instead of parsing it each time in RunTime. It has much better performance and it's easier to work with that since you are working directly with NSDictonaries an NSArrays.
If you don't want to do it for some reason, the next solution is to use something like CHCSVParser:
https://github.com/davedelong/CHCSVParser
It provides optimization for loading only part of file at a time - which is the optimization you might be looking for.
Related
I'm looking for the exact specifications of this file format. Anyone got a link? Or want to comment?
I have spent the better part of the day searching, yet I keep getting directed back to the GIMP online user-manual. It says "look at a .gpl file and you will see it is easy" to build manually with a text editor. I don't actually have GIMP, but I see examples online. Yep, easy. • EXCEPT:
• What meaning do the color names ultimately have? Are they purely semantic, or does a program rely on them? If the latter, then what if there are two (2) or more colors with the same name?
• What does the "Columns" line do?
I've seen examples that have no "Columns" line.
I've seen examples that have values of 0, 4, and 16; yet this does not in any way that I can see correspond to the color data. I see 3 columns of decimal-sRGB values, and an optional 4th column with the color-name; seems I remember the example with "Columns 4" had no color names, only the 3 RGB columns.
• Do columns of RGB values need to "line up"? Or will the following example from my output algorithm work? (from the Crayola palette):
159 129 112 Beaver
253 124 110 Bittersweet
0 0 0 Black
172 229 238 Blizzard Blue
31 117 254 Blue
162 162 208 Blue Bell
102 153 204 Blue Gray
13 152 186 Blue Green
• Does this format accept sRGBA colors? And if so, how is the "A" value defined (0-1, 0%-100%, 0-127, 0-255, etc.?) (seems I remember when creating .png files with PHP, the "A" value was 7-bit)?
• How exactly do you add comments / metadata?
Today I see an example that says lines that begin with # are comments, or anything after a # on a line is a comment. Yesterday I thought (maybe I'm confused) I saw an example that said that comment lines begin with ;
• Is any other data-format supported?
Originally I thought the text-line just before the color-data that I see in every example indicated the format: "#" signifying decimal-sRGB; until today when I see that is just a blank-line comment.
• What line ending character(s) can / must I use?
\n
\r
• What character-encodings can I use? ASCII only? ¿UTF-8 ☺ with extended ♪♫ charset (¡hopefully!)?
• Anything I'm missing? Any other options available?
Here is an example from http://gimpchat.com/viewtopic.php?f=8&t=3375#
GIMP Palette
Name: bugslife_final.png-10
Columns: 16
#
191 180 180 Index 0
163 158 157 Index 1
145 136 132 Index 2
130 125 112 Index 3
… … …
56 50 49 Index 29
41 38 38 Index 30
23 23 23 Index 31
242 245 213 Index 32
227 232 181 Index 33
210 217 147 Index 34
195 204 118 Index 35
… … …
0 0 0 Index 251
0 0 0 Index 252
0 0 0 Index 253
0 0 0 Index 254
0 0 0 Index 255
Aloha!
Looking at the source code:
Columns is just an indication for display in the palette editor
Comments must start with a #. In non-empty lines that don't, the first three tokens are parsed as numbers
There is no alpha support
I have two files with thousands of lines:
file1:
COL22A1 LCT 1 12 0.149667616334 2.16226378401
GPRIN2 TP53 12 170 0.0455368539793 44.2359753827
MUC3A TP53 12 170 0.0455368539793 44.2359753827
file2:
COL22A1 LCT 12 41 23 0.0296296296296 0.101234567901 0.0567901234568 2.36563
MEGF10 SORCS1 10 21 39 0.0246913580247 0.0518518518519 0.0962962962963 2.30599
I want to compare first two columns of these files and if they match I want to print whole line of second file and last column of first file:
output:
COL22A1 LCT 12 41 23 0.0296296296296 0.101234567901 0.0567901234568 2.36563 2.16226378401
I tried awk, grep, join but it always gives me output of just one file
Could you please try following and let us know then.
awk 'FNR==NR{a[$1,$2]=$NF;next} a[$1,$2]{print $0,a[$1,$2]}' Input_file1 Input_file2
The powerball schema and separators are not consistent which makes it an unusual file to read. (http://www.powerball.com/powerball/winnums-text.txt)
Sample:
Draw Date WB1 WB2 WB3 WB4 WB5 PB PP
09/24/2016 15 07 29 41 20 22 2
09/21/2016 63 67 01 69 28 17 4
09/17/2016 51 19 09 62 55 14 4
Any suggestions?
This looks like a "fixed column width" file rather than an ordinary CSV (meaning that the columns are not separated by any single character, but instead have fixed number of characters, with padding spaces).
There is some early work on supporting this in F# Data in the pull request here. We'd welcome any help getting this tested - but you'd need to get the soruce code and build F# Data from source (which is just a matter of running the build script though!)
Alternatively, you could probably do some simple pre-processing on the file before reading it as an ordinary CSV file. Looking at the sample file, using a regular expression to replace 1 or more spaces with a comma would produce regular CSV that the CSV provider can consume.
With a Postscript driver (Xerox, Canon, HP, all), when I create a PS file, for example when I print the test page in the printer properties, I get :
OK :
The view of the result is correct (with GSview for example)
Not OK :
The file size is to big, more than 4 MB.
When I edit the file, I have one big image (doNimage). I think is the reason of the big size file.
The example file : https://drive.google.com/open?id=0B9bet657DEU5alV6WFZZdDFjMmc
I'm on Windows 10, similar problem with Windows server 2012 r2.
I let the configuration of the driver by default.
Anyone has an idea ?
Thanks a lot.
Regards.
I don't understand your problem, the file you posted a link to contains text. Here's an example:
360 4485 M <202530360E0F1102381030100D100B0824152D30103102020C302A1E19181B1E1730132E28301530132D3B02230B2A2E22081308>[46 16 28 70 18 42 44 44 54 32 28 32 36 32 25 39 65 40 40 28 32 44 44 44 18 28 53 45 20 47 38 45
40 28 34 40 40 28 40 28 34 40 18 44 44 25 53 40 16 39 34 0]xS
M is a moveto and xS uses the xshow operator to draw the glyphs represented by the character codes in the hexstring, using the values in the array to modify the width of each glyph.
If you were expecting to see ASCII character codes you are going to be sadly disappointed, the files uses an incrementally downloaded subset TrueType font, so the character codes are defined as they are encountered, that is the first glyph used will be given character code 1, the second will be character code 2 and so on.
Even without that, using ASCII would limit the languages that could be supported. Back in the 1980s that maybe didn't seem like a problem, but its a long time since that was considered acceptable.
If you were expecting to be able to modify the text by editing it in a text editor, forget it. PostScript is a programming language, and the output of a PostScript printer driver is a machine-generated program. Its a lengthy process for a skilled user of the language to decipher what the program is doing. The program is not amenable to alteration, if there's a fault in the output, correct the original document and recreate the PostScript program from the original.
PostScript is not an editable format.
Thanks all for your response. I see I was not very clear in my question.
Here is the state :
With the PS driver, on a windows server 2008, I get this file :
http://expirebox.com/download/0bb511565377e8b74eead67641fe7f68.html
Inside the file I can see the text "Page de test d\222imprimante"
On a Windows server 2012 R2 :
http://expirebox.com/download/60fa957cba97c82bbcd5c0e975825b52.html
I can't see any text. It's a printer page test too.
I need to see text because I'll print document with code inside. Code for a printer to identify page type. (for example a white page for the tray n° 1, yellow page for tray 2)
KenS : I understand your point. But why the same driver give different file.
I checked if it's really the same. The only difference I see is the OS, one x86, the other x64.
Thanks.
Regards.
I would like to parse files with several sequences of data (same number of column, same content, ...) with Haskell.
My data sequences will be delimited by keywords before and after.
BEGIN
1 882
2 809
3 435
4 197
5 229
6 425
...
END
BEGIN
1 235 623 684
2 871 699 557
3 918 686 49
4 53 564 906
5 246 344 501
6 929 138 474
...
END
My problem is that after several tests with Parsec, I have the impression that Parsec is rather made to parse a file line by line and not the whole file.
Is Parsec the right way to make what I want or should I consider an other tool like Happy or Alex ?
Is there a website (or other ressource) providing examples of parsing complex text files with Parsec ?
Note : The example I give is a very simple one. Things would be more tricky in my files with many more keywords and combinations.
The format as you've described wouldn't be hard at all to handle in parsec.
As for learning how to use it: your first step should be to avoid whatever guide gave you the impression that parsec worked line-by-line. I recommend Chapter 16 of Real World Haskell as a good place to get started, and once you're comfortable with the basics the reference material at http://hackage.haskell.org/package/parsec is actually very clear.