I've searched many forums and blogs before posting the question.
I've found samples in python and VB which are using ZLib.
but I can't get it to work in Delphi.
I have the stream from a pdf that is encoded with FlateDecode.
Here is the stream saved as a simple file named "compressed_stream.pdf" (in fact it's not pdf - it's only the stream, but I just left the .pdf file extension)
https://files.fm/u/epka2hxz
Here is my code:
Execution goes to System.Zlib.ZDecompressStream(streamIn, streamOut); and just sleeps... no errors, no crashes, nothing - just sleeps until I break the execution.
Any idea?
var
fs: TFileStream;
streamIn, streamOut: TMemoryStream;
begin
fs := TFileStream.Create(sDocumentFolder + 'compressed_stream.pdf', fmOpenRead);
streamIn := TMemoryStream.Create();
streamOut := TMemoryStream.Create();
streamIn.CopyFrom(fs, 0);
streamIn.Position := 0;
System.Zlib.ZDecompressStream(streamIn, streamOut);
end;
Thanks to Dima I quickly found a sample for TZDecomoressionStream:
https://forum.lazarus.freepascal.org/index.php?topic=33009.0
function ZDecompressString(aText: string): string;
var
strInput,
strOutput: TStringStream;
Unzipper: TZDecompressionStream;
begin
Result:= '';
strInput:= TStringStream.Create(aText);
strOutput:= TStringStream.Create;
try
Unzipper:= TZDecompressionStream.Create(strInput);
try
strOutput.CopyFrom(Unzipper, Unzipper.Size);
finally
Unzipper.Free;
end;
Result:= strOutput.DataString;
finally
strInput.Free;
strOutput.Free;
end;
end;
Related
I'm modifying a program that is written in Delphi 6.0
I have a table in Oracle with a BLOB column named FILE_CONTENT.
I have already managed to upload an XML File that is about 100 KB. I have verified that the file content was correctly uploaded using SQL Developer.
The problem I have is when I try to download back the file content from DB to a file. This is an example code I'm using to donwload it:
procedure TfrmDownload.Save();
var
fileStream: TFileStream;
bField: TBlobField;
begin
dmDigital.qrGetData.Open;
dmDigital.RequestLive := True;
bField := TBlobField(dmDigital.qrGetData.FieldByName('FILE_CONTENT'));
fileStream := TFileStream.Create('FILE.XML', fmCreate);
bField.SaveToStream(fileStream);
FlushFileBuffers(fileStream.Handle);
fileStream.Free;
dmDigital.qrGetData.Close;
end;
The previous code already downloads the file content to FILE.XML. I'm using RequestLive:=True to be able to download a large BLOB (otherwise the file content is truncated to 32K max)
The resulting file is the same size as the original file. However, when I compare the downloaded file with the original one there are some differences (for example the last character is missing and other characters are also changed), therefore it seems to be a problem while downloading the content.
Do you know what cuould be wrong?
The problem seems to be related to Delphi code because I already tried with C# and the file content is downloaded correctly.
Don't use TBlobField.SaveToStream() directly, use TDataSet.CreateBlobStream() instead (which is what TBlobField.SaveToStream() uses internally anyway):
procedure TfrmDownload.Save;
var
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
fileStream.CopyFrom(bStream, 0);
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;
TDataSet.CreateBlobStream() allows the DataSet to decide the best way to access the BLOB data. If the returned TStream is not delivering the data correctly, then either the TStream class implementation that CreateBlobStream() uses is broken, or the underlying DB driver is buggy. Try taking CopyFrom() out of the equation so you can verify the data as it is being retrieved:
procedure TfrmDownload.Save;
const
MaxBufSize = $F000;
var
Buffer: array of Byte;
N: Integer;
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
//fileStream.CopyFrom(bStream, 0);
SetLength(Buffer, MaxBufSize);
repeat
N := bStream.Read(PByte(Buffer)^, MaxBufSize);
if N < 1 then Break;
// verify data here...
fileStream.WriteBuffer(PByte(Buffer)^, N);
until False;
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;
I want to use a subtitle API. It requires a md5 hash of first and last 64kb of the video file. I know how to do the md5 part just want to know how will I achieve to get the 128kb of data.
Here is the solution to the problem in Java which I am unable to implement in Delphi. How to read first and last 64kb of a video file in Java?
My Delphi code so far:
function TSubdbApi.GetHashFromFile(const AFilename: string): string;
var
Md5: TIdHashMessageDigest5;
Filestream: TFileStream;
Buffer: TByteArray;
begin
Md5 := TIdHashMessageDigest5.Create;
Filestream := TFileStream.Create(AFilename, fmOpenRead, fmShareDenyWrite);
try
if Filestream.Size > 0 then begin
Filestream.Read(Buffer, 1024 * 64);
Filestream.Seek(64, soFromEnd);
Filestream.Read(Buffer, 1024 * 64);
Result := Md5.HashStreamAsHex(Filestream);
end;
finally
Md5.Free;
Filestream.Free;
end;
end;
I am not getting the accurate md5 hash as stated by the official API.API url here. I am using Delphi XE8.
The hash function used by that API is described as:
Our hash is composed by taking the first and the last 64kb of the
video file, putting all together and generating a md5 of the resulting
data (128kb).
I can see a few problems in your code. You are hashing the file stream, not your Buffer array. Except that you were overwriting that array by subsequent reading from the file stream. And you were trying to seek only 64 bytes, and beyond the end of the stream (you need to use a negative value to seek from the end of the stream). Try something like this instead:
type
ESubDBException = class(Exception);
function TSubdbApi.GetHashFromFile(const AFileName: string): string;
const
KiloByte = 1024;
DataSize = 64 * KiloByte;
var
Digest: TIdHashMessageDigest5;
FileStream: TFileStream;
HashStream: TMemoryStream;
begin
FileStream := TFileStream.Create(AFileName, fmOpenRead, fmShareDenyWrite);
try
if FileStream.Size < DataSize then
raise ESubDBException.Create('File is smaller than the minimum required for ' +
'calculating API hash.');
HashStream := TMemoryStream.Create;
try
HashStream.CopyFrom(FileStream, DataSize);
FileStream.Seek(-DataSize, soEnd);
HashStream.CopyFrom(FileStream, DataSize);
Digest := TIdHashMessageDigest5.Create;
try
HashStream.Position := 0;
Result := Digest.HashStreamAsHex(HashStream);
finally
Digest.Free;
end;
finally
HashStream.Free;
end;
finally
FileStream.Free;
end;
end;
okay, so I (VERY) recently started playing with lazaruz/free pascal, and I'm a little stuck with reading files with TMemoryStream and it's streaming kin.
I'm trying to write a simple base64 encoder, that can encode strings of text, or files (like images and WAVs) to then be used in html and javascript.
The following code compiles great but I get EReadError Illegal stream image when trying to load a file. I'll include the working string only procedure for reference:
procedure TForm1.TextStringChange(Sender: TObject);
begin
Memo1.Lines.Text := EncodeStringBase64(TextString.Text);
end;
procedure TForm1.FormCreate(Sender: TObject);
begin
Memo1.Lines.Text := '';
Form1.BorderIcons := [biSystemMenu,biMinimize];
end;
procedure TForm1.BitBtn1Click(Sender: TObject);
begin
if OpenDialog1.Execute then
begin
filename := OpenDialog1.Filename;
stream := TMemoryStream.Create;
try
StrStream := TStringStream.Create(s);
try
stream.LoadFromFile(filename);
stream.Seek(0, soFromBeginning);
ObjectBinaryToText(stream, StrStream);
StrStream.Seek(0, soFromBeginning);
Memo1.Lines.Text := EncodeStringBase64(StrStream.DataString);
finally
StrStream.Free;
end;
finally
stream.Free;
end;
end;
end;
Can anyone help me out?
You get the "illegal stream image" exception because the file you're loading probably isn't a binary DFM file. That's what ObjectBinaryToText is meant to process. It's not for arbitrary data. So get rid of that command.
You can skip the TMemoryStream, too. TStringStream already has a LoadFromFile method, so you can call it directly instead of involving another buffer.
StrStream.LoadFromFile(filename);
But a string isn't really the right data structure to store your file in prior to base64-encoding it. The input to base64 encoding is binary data; the output is text. Using a text data structure as an intermediate format means you may introduce errors into your data because of difficulties in encoding certain data as valid characters. The right interface for your encoding function is this:
function Base64Encode(Data: TStream): string;
You don't need to load the entire file into memory prior to encoding it. Just open the file with a TFileStream and pass it to your encoding function. Read a few bytes from it at a time with the stream's Read method, encode them as base64, and append them to the result string. (If you find that you need them, you can use an intermediate TStringBuilder for collecting the result, and you can add different buffering around the file reads. Don't worry about those right away, though; get your program working correctly first.)
Use it something like this:
procedure TForm1.BitBtn1Click(Sender: TObject);
var
filename: string;
stream: TStream;
begin
if OpenDialog1.Execute then begin
filename := OpenDialog1.Filename;
stream := TFileStream.Create(filename, fmOpenRead);
try
Memo1.Lines.Text := Base64Encode(stream);
finally
stream.Free;
end;
end;
end;
I never heard before about ObjectBinaryToText(), but looks like funky one. Also, what is EncodeStringBase64() function?
At first place, you shouldn't convert binary stream to text to encode it, instead you should directly B64 encode binary data. B64 algorithm is intended to work on array of bytes.
Since Delphi 6, there is EncdDecd.pas unit, which implements B64 encoding methods. I'm not sure if Lazarus/FPC have this, but if they do, your code to B64 encode file should look like this (add EncdDecd to uses list):
procedure TForm1.Button1Click(Sender: TObject);
var
instream : TFileStream;
outstream: TStringStream;
begin
if OpenDialog1.Execute then
begin
instream := TFileStream.Create(OpenDialog1.FileName, fmOpenRead or fmShareDenyNone);
try
outstream := TStringStream.Create;
try
EncodeStream(instream, outstream);
Memo1.Lines.Text := outstream.DataString;
finally
outstream.Free;
end;
finally
instream.Free;
end;
end;
end;
I am downloading an EXE file from internet using Indy (idHTTP), and I can use memorystream or filestream to save it to disk, but I really do not know if there is any difference between them (maybe in the result structure of the file?). I could't find yet an answer for this.
Where, here are 2 simple functions to simulate what I am doing:
Function DownloadMS(FUrl, Dest: String): Boolean;
var
Http: TIdHTTP;
Strm: TMemoryStream;
Begin
Result := False;
Http := TIdHTTP.Create;
Strm := TMemoryStream.Create;
With Http, Strm Do
Try
Try
Get(FUrl, Strm);
If (Size > 0) Then
Begin
Position := 0;
SaveToFile(Dest);
Result := True;
end;
Except
end;
Finally
Strm.Free;
Http.Free;
end;
end;
Function DownloadFS(FUrl, Dest: String): Boolean;
var
Http: TIdHTTP;
Strm: TFileStream;
Begin
Result := False;
Http := TIdHTTP.Create;
Strm := TFileStream.Create(Dest, fmCreate);
With Http, Strm Do
Try
Try
Get(FUrl, Strm);
Result := (Size > 0);
Except
end;
Finally
Strm.Free;
Http.Free;
end;
end;
What you experts think about using one or other type (memorystream or filestream)? Is there any difference in the structure of the EXE file when using one or other type? What type is recommended?
Thank you! Have a nice weekend!
There is no difference between TMemoryStream or TFileStream from the stream point of view.
They are both streams and hold a stream of bytes and are both derived from TStream.
You can implement your function generalized like this
function DownloadToStream( const AUrl : String; ADest : TStream ): Boolean;
var
LHttp: TIdHTTP;
begin
LHttp := TIdHTTP.Create;
try
LHttp.Get( AUrl, ADest );
Result := ADest.Size > 0;
finally
LHttp.Free;
end;
end;
and call it with a TFileStream
var
LStream : TStream;
begin
LStream := TFileStream.Create( 'MyFile.exe', fmCreate );
if DownloadToStream( '', LStream ) then
...
end;
or TMemoryStream or whatever stream instance you like
In many cases there will be no point in putting an intermediate memory stream in between the download and the file. All that will do is consume memory because you have to put the entire file in memory before you can put it to disk. Using a file stream directly avoids that issue.
The main situation where the file stream option has problems is if you want to be sure that you've downloaded the entire file successfully before saving to disk. For example, if you are overwriting a previous version of a file, you may want to download it, check a hash signature, and only then overwrite the original file. In that scenario you need to put the file to some temporary location before over-writing. You could use a memory stream, or you could use a file stream using a temporary file name.
How can I deny access (only to write) to a file for other processes? I will read\write a file all time.
I use
FileOpen('c:\1.txt', fmOpenReadWrite or fmShareDenyWrite)
but after (starting to load the file to StringList) I get error
Cannot open file C:\1.txt. The process cannot access the file because it is being used by other process."
Only I open the file.
Here, the error message is actually slightly misleading. The reason you can't load into the stringlist is because you already opened the file in read/write.
if you check the implementation of TStrings.LoadfromFile:
procedure TStrings.LoadFromFile(const FileName: string);
var
Stream: TStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
try
LoadFromStream(Stream);
finally
Stream.Free;
end;
end;
You see that it tries to open the file with a "DenyWrite" condition, but you already opened the file in write mode. That is the reason why it fails.
You can work around that by using LoadFromStream instead.
procedure TForm1.Button1Click(Sender: TObject);
var
Stream: TStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone);
try
Stringlist.LoadFromStream(Stream);
finally
Stream.Free;
end;
end;
Note that you will need to use fmShareDenyNone for this to work in that situation. Then again, you could probably reuse the Read/Write handle you got from your OpenFile, probably do something like this:
procedure TForm1.Button1Click(Sender: TObject);
var
Stream: TStream;
iPosition : Int64;
begin
Stream := THandleStream.Create(FHandle); //FHandle is the read/write handle returned by OpenFile
try
iPosition := Stream.Position;
Stream.Seek(0, soFromBeginning);
Stringlist.LoadFromStream(Stream);
Stream.Position := iPosition;
//Restore stream position.
finally
Stream.Free;
end;
end;
But be advised that these approach might have a few "gotchas" I'm unaware of.