I have a path library which has a zip function in it, and while writing the unit test, I tried to find the best way to verify that the zip function works correctly.
Can someone please show me the best way to verify that the zip function works correctly ?
The few ways I can think of are:
Comparing the md5 of the resultant zip file against a sample zip file
Listing out the contents of the zip file and ensuring the content are correct
However, both ways seems a little long winded and I am guessing not exactly idiomatic Elixir. Can someone please show me a better way ?
I would create a directory of test files to zip up in your unit test, zip it up using a trusted utility and get the resulting md5. Then for your unit test, perform the zip function, take the md5, and compare with your verified md5.
If I am correct, there are some parameters that can affect the result of zip processing (like block size). So, unless your purpose is to check exhaustively that your zip process reproduces all or a subset of the zip features, I find more interesting to validate that various files from your system (or/and some random files) can be compresses with your function, and uncompressed outside using different unzip tools, and finally compare that the files are identical to the original. (of course it takes more time, but it can be easily automated).
Concerning the usage of MD5 rather than file comparison, I think that on one hand file comparison is easy to write and in the other hand you need to read the whole file to calculate the MD5, so I don't think it is worth to use this trick to accelerate the comparison - (especially if files are different :o).
Related
I've been trying to edit the lua files for the game victor vran (made by developers of the tropico series) but when i tried to decompile the files i'd get back errors. So i opened up some of the files in both notepad++ and luaedit, and noticed the first string of every file was lz4 which I automatically associated with compression. After a quick google I found the lua rocks lz4 module. With that i tried to decompress some files but couldn't really wrap my head arround it.
If anybody could give me some tips to point me in the right direction/tell me where i'm going wrong, or even take a quick look at the files i'll link to below, to give me advice on the matter I'd be greatly appreciatave. I will include 2 types of lua files from the game, ones with the lz4 string and ones without. thank you so much in advance, ifanybody needs more information on the files or the steps i took ask away
.lua files with "LZ4" as first string https://drive.google.com/open?id=0B042ggDJxSmJaTJ2Z1l4RTkzUHc
.lua files without lz4 as first string https://drive.google.com/open?id=0B042ggDJxSmJa2VNcjQxVC1TZ1k
I have a corrupted 7-zip archive that I am extracting manually using the method outlined by Igor Pavlov at this link. An intermediate result is a large file that is a bunch of files cat'ed together that must be separated manually. I understand that some file formats will need to be extracted manually by a human using discretion (text files, etc.) but many file formats encode the size of the file as part of the file itself (e.g. .zip). Furthermore, some files can be parsed and their size can be deduced with just a little information about the file format (e.g. .pdf). Let's say the large file consists of the following files concatenated together:
Key: <filename>(<contents>)
badfile(aaaaaaaaaaabbbbbbbbbcccccccdddddddd) -> zip1.zip(aaaaaaaaaaa)
badfile2(bbbbbbbbbcccccccdddddddd)
I am looking for a program that I can run on a large file (call it badfile) that can determine the type and size of the first logical file (let's say it's a .zip file) contained within and create a new file to hold the contents (e.g. zip1.zip since filenames are lost) and chop the file off the front of badfile. This would allow me to run the program in a loop to extract files with known types and/or pause and let the user handle the difficult cases. Does such a program exist? I know that the *nix command file(1) will do a lot of the work here, but there would be a lot of effort in encoding rules for sizing files (e.g. .pdf) that I would prefer to not duplicate.
I believe this question should be closed due to being off topic as it asks to find existing programs to solve the problem, but open bounty prevents close vote. However.
Does such a program exist?
Yes they exist is and are called data carving tools.
Some commom ones include scalpel and foremost and PhotoRec
A list of other tools is avaliable here
I have two files inside a zip file.
Now, imagine these two files are big... REALLY big... so big that I can't uncompress them into my old, poor, tiny hard disk.
However, they are simple txt files, so the zipped version is quite small.
I need to JOIN the two files into ONE single file.
As they're too big to extract, I need to do this INSIDE the zip.
Is there a way to do this?
Example:
"compressed.zip" contains "part_1.txt" and "part_2.txt".
I want "compressed.zip" to contain one file, called "part_1_and_2.txt".
(If it's not possible with zip, I can pick another compressor... but the idea is the same: each uncompressed file is bigger than total capacity of my hard disk)
Tnx!
It seems like you just need to ensure that the storage requirements are low; I don't think the operation needs to occur "within the zip file" per se. You can do this with command-line tools (in Linux or with similar tools via Cygwin) in the following way:
Start with a tarred, gzipped file with your input files in it. Let that be compressed.tar.gz. Then you can extract the contents of the gzipped tar archive to standard output and pipe it back to gzip:
tar xzf compressed.tar.gz -O | gzip > part_1_and_2.txt.gz
The resultant compressed file is the text of part_1.txt and part_2.txt concatenated (though I suppose it is not the same as having a tar archive that contains one file, but perhaps this will be sufficient).
If you need to do this within a program, I would guess that libtar and zlib can perform this functionality programmatically, or you can run a script from your program.
You can use libzip (which in turn uses zlib) to read uncompressed data from the input files in turn and write a new output zip file. You would not need to store all of an uncompressed file on the mass storage or in memory. You can read and write a small chunk at a time, as you would without compression. I presume that you have room on your mass storage for all three of those zip files.
Those who know C++ may know what I mean by 'unity build':
*.cpp files of a project are all effectively #include-ed into a single supermassive source file following #include directives specified in *.cpp and *.h files
this source file is fed into the compiler
finish! You get the output binary!
Doing things this way means that there are that there are fewer intermediate files (*.o), fewer file reads and disk IO overheads, fewer invocations of the compiler, leading to a better build performance.
My question is, is this possible for Latex at all? I want it because there is a slow post-processing pass that I would like to run over .tex files before building my final .pdf using pdflatex. Currently, it takes around 7 seconds to process my growing list of .tex files. I believe that running this pass over one file is significantly faster. This motivates my question!
To summarize, I want to
'merge' all the .tex files into a supermassive .tex source file by following the \input{} and \include{} macros in each .tex file
feed the supermassive .tex source file into the slow post-processing pass (actually the Ott tex-filter, fyi)
pipe the output straight into pdflatex
finish! I get the output PDF file!
The first step is the problem here. Any ideas welcome. It's best if I don't need to write my own script to do this step!
Many thanks!
A good tool that can handle this is rubber, with the help of its combine module. It will gather all dependencies, and produce a single file ready for consumption.
I'm developing a BlackBerry application in which I need to unpack a zip file compressed with PKZIP. The package could have one file in it, or it could have 10; it will vary in each case. I know that the BlackBerry API has native support for GZip and Zlib, although I'm pretty sure that these methods aren't going to be helpful in my case. It doesn't look as if I can extract the individual files using these calls.
I've tried JZlib (http://www.jcraft.com/jzlib/), which compiled fine, but again it doesn't look as if the methods contained therein are going to allow me to pull the individual files.
It appears as if this is possible, as there's an application called Ziplorer (http://www.s4bb.com/software/ziplorer/) that claims to do perform this exact procedure. How they're doing it, however, escapes me.
So here I am after hours of Googling. I'm welcoming any insight into my problem with open arms.
"zip" algorithms are typically offshoots of the Lempel-Ziv-Welch algorithm. They are a fairly efficient stream compression algorithms, but because of how they function, you can't start to decompress at random points in the file, you have to start from the start.
Any product that claims to be able to decompress one file from a zip still has to decompress everything before it in the zip file in order to know how to decrypt the given file, or even, for that matter, where the file is in the archive.
If you can tolerate GPL code in your application, then this library http://jazzme.sourceforge.net/ that might work. However the project (and its parent project http://sourceforge.net/projects/jazzlib/) don't look like they're being developed.