How can I open or review a large txt file - data-warehouse

I have a few large txt files I'm trying to load into a data warehouse, I do get an error message with the offending row/line number but cannot open the txt file to review it as says it's 2 large 2,413,060KB. Someone suggested using the cmd option to do this but unsure how.

You can either use HJSplit in order to split the data or using another vim-based program such as gVim. Also it may be good to free up your PC's RAM as much as possible.

Related

Identify old dos system file type and decode it into text file

I have pick up an old dos system from my friend, and I need to import the data into SQL, but before importing the data, i need to decode it into a readable text file, but I failed to do so. I have try several stuff:
file command in ubuntu terminal, it said "data"
Use online trid and it said macbin(MacBinary 1)
Tried bin2hex, but couldn't unhex it
Tried some online macbin to hex, no luck as well
Tried to open in macOS, but it keep extracting files
bin2hex said, nothing here
stuffitexpander.... Doesn't recognize...
This is the file that i need to decode
https://gofile.io/?c=wdbs6A
Please let me know if you need the original program.
I think they are just some database files.
Use this site for explanations. they even have a file analyzer - showing you the data inside.
You will need to rename the files to .db extension instead of .ocm.

SPSS indicates that the .sav file is empty. Is there any way to recover the file from the Temp folder?

I was just finishing my work. Had been working on this for 1 month, more than 100 questionnaires' data to enter on the database, there were only 3 left. My computer crashed, I turned it off and after I turned it on again, I opened SPSS and coudln't open the .sav file because of an error saying the file is empty (although its size is 68kb) and to try to open another file. I tried to recover the data through the journal file, through syntaxe, but I can't. Is there any way to recover the file from the Temp folder? I noticed that in the Temp folder there are some folders named like this: "pasw-cfe-1324076806484081348-tmp" which contain executable jar files inside. There are also some LCK files with the same names as those folders. After searching about PASW, I got the idea that it may be related to SPSS. Is there any way to open these folders/files? Also, I can send the file to anyone, in case this is needed.
Any help you can offer me will be truly appreciated. Thanks
Unfortunately, no there is no way to convert those temporary scratch file back into an SPSS Statistics system file (.sav). If you do not have backups of the file, perhaps in another format (.sav, *.xlsx, *.csv, etc.) or stored as a table in a database, your only recourse will be to start over.
I have an idea, you can do "system Restore"

Copy only new added files from one folder to another, without moving the existing files from source folder

I am doing file integration using mirth. There is one software which generate the HL7 files. I want to read data from that files, without moving them to another destination. Next time when I want to read data, at that time it'll ignore the files from which the data are already read (i.e.Just read the new files data which are generated after last data read).
I had done this but I'll achieve it when I modify the original filename, if I am not modifying the filename then it'll read the duplicate data.
Is there is any solution for this problem, so we can read data from the files which are generated new. I am using mirth 3.5.1 version and HL7 v2 messages.
Thanks in advance.
Thanks #daveloyall, I am posting your comment as a answer here.
When you rename a file at the time you process it, for example, to add a .DONE suffix to the filename, you are adding information that can be used later. The part of the channel that reads files could be configured to skip files that have the .DONE suffix. You also add information if you move the files. Or store the filenames in some database table. I don't know if Mirth has an internal feature that tracks which HL7 messages it already processed, but if such a feature exists, the keyword 'deduplication' might be associated with it.

"Fastest way to unzip many files on iOS" or "How else can I download many files quickly into my iOS app"

In my app i want the user to be able to download offline map content.
So I (compressed) moved all my tiles into a zip file. (I used 0 compression)
The structure is like that: {z/x/y.jpg}
+0
+-0
+--0.jpg
+1
+-1
+--0.jpg
+2
+-2
+--1.jpg
So basically there are going to be many many files for zoom level 0-15. (about 120.000 tiles for my test-region).
I am using https://github.com/mattconnolly/ZipArchive now but also tried out https://github.com/soffes/ssziparchive before and both are pretty slow. It takes about 5!! minutes on my iPhone 5S for the files to unzip.
Is there any way I can speed things up? What other possibilities rather than downloading the tiles in one big zip file would there be?
Edit:
How can i download the content of the whole folder quickly to my iPhone without the need of unzipping something?
Any help is appreciated!
JPGs rarely compress at all with zip - they are by definition already compressed. What you should do is create your own binary file format, and put whatever metadata you need into it along with the images (which you should encode with a really low quality number, to get their size down).
When you download those files, you can open then, quickly read them into memory, and extract out data or images as needed.
This will be really fast and have virtually no overhead if your extra data is binary (not text).
PS: I just tripped on a PHP Plist class
If anyone is wondering how I was ending up:
For my use-case (MapTiles) I am using MBTiles now instead of zipped images. It's one big database file and super easy to read if using FMDB. No unpacking whatsoever needed...
Even if I was placing the Images all in one binary file without any compression, the "extracting" still took forever!

how can I open a sqlite file in ios for reading without copying to documents?

I got an app I'm working on that uses static data from a sqlite database to do various things, While I only need read only access to the database, depending on the episode they pick from the first screen I want it to use a different database file and I want the list of available episodes to be updateable on the fly. and I got help to get the list of available episodes updated, and the proper content downloaded and stored in separate folders, So I know I could when the episode is selected delete the sql file in the documents folder and copy in the new one each time and that would work well enough for what I'm trying to do. but it seems like a bit much extra work to have to check for file, delete file, copy in new one. then open it from there each time the user wants to pick a different episode. and I don't want to put all the sql files together as that will be a bigger hassle then the first route especially if this app stays around long enough to have a long list of episodes.
so my question here is: can I get at least read-only access to an sql file that I've downloaded (or one in the bundle for testing) with out having to first copy it to the documents? and if so how would i open the file?
Can I get at least read-only access to an SQL file that I've downloaded (or one in the bundle for testing) without having to first copy it to the documents directory?
Yes. Files in the app bundle are readable (if they weren't, there would be no point in storing files in the bundle).
And if so, how would I open the file?
It's not clear what you're asking here - if you want to perform SQL queries on the file, you should use the sqlite3 library which is available on iOS.

Resources