Delphi TStreamReader - How to read in a shareable mode? - delphi

I have written a routine in Delphi Tokyo which takes multiple files (such as CSV) and merges them together, giving the user the option to ignore the first line on all files except the first one (as CSV files often have header lines/column name lines, when merging the files, I only want one copy of the header). The issue I am having is that even though I am only reading the various input files, if the file is open in another process, (specifically Excel), my app gives an error: "Cannot open file . The process cannot access the file because it is being used by another process."
I am using TStreamReader. How do I tell TStreamReader that it should open the file in read-only...and continue even if the file is open elsewhere?
Code below:
procedure glib_MergeTextFiles(const InFileNames: array of string; const OutFileName: string;
HasHeader: Boolean = True;
KeepHeader: Boolean = True);
var
I: Integer;
InStream: TStreamReader;
OutStream: TStreamWriter;
Line: string;
IsFirstLine: Boolean;
begin
// Create our output stream
OutStream := TStreamWriter.Create(OutFileName, False, TEncoding.UTF8);
try
for I := 0 to high(InFileNames) do
begin
InStream := TStreamReader.Create(InFileNames[I], TEncoding.UTF8);
IsFirstLine := True;
try
while not InStream.EndOfStream do
begin
Line := InStream.ReadLine;
if IsFirstLine then { First Line }
begin
if HasHeader = False then
begin
OutStream.WriteLine(Line);
end
else
begin
// Is First Line, Has Header
if I = 0 then {is first file}
OutStream.WriteLine(Line);
end;
end
else
begin
OutStream.WriteLine(Line);
end;
IsFirstLine := False;
end;
finally
InStream.Free;
end;
end;
finally
OutStream.Free;
end;
end;

The problem is with the sharing mode. By default, the stream reader creates a file stream for reading only, but specifies no sharing mode, so it opens the file for exclusive access. However, to open a file for reading when it is already opened elsewhere, the file must have been previously opened to share reading access using the FILE_SHARE_READ flag:
FILE_SHARE_READ
0x00000001
Enables subsequent open operations on a file or device to request read access.
Otherwise, other processes cannot open the file or device if they request read access.
If this flag is not specified, but the file or device has been opened for read access, the function fails.
You can pass your own file stream to the stream reader, opened with the mode you like:
var
I: Integer;
FileStream: TFileStream;
InStream: TStreamReader;
..
begin
...
FileStream := TFileStream.Create(InFileNames[I], fmOpenRead or fmShareDenyNone);
try
InStream := TStreamReader.Create(FileStream, TEncoding.UTF8);
try
..
Again, this requires Excel doing the same while opening the file, but with my simple test it looks like it does.

Related

Saved data from TBlobField is corrupted for lengths >= 100KB

I'm modifying a program that is written in Delphi 6.0
I have a table in Oracle with a BLOB column named FILE_CONTENT.
I have already managed to upload an XML File that is about 100 KB. I have verified that the file content was correctly uploaded using SQL Developer.
The problem I have is when I try to download back the file content from DB to a file. This is an example code I'm using to donwload it:
procedure TfrmDownload.Save();
var
fileStream: TFileStream;
bField: TBlobField;
begin
dmDigital.qrGetData.Open;
dmDigital.RequestLive := True;
bField := TBlobField(dmDigital.qrGetData.FieldByName('FILE_CONTENT'));
fileStream := TFileStream.Create('FILE.XML', fmCreate);
bField.SaveToStream(fileStream);
FlushFileBuffers(fileStream.Handle);
fileStream.Free;
dmDigital.qrGetData.Close;
end;
The previous code already downloads the file content to FILE.XML. I'm using RequestLive:=True to be able to download a large BLOB (otherwise the file content is truncated to 32K max)
The resulting file is the same size as the original file. However, when I compare the downloaded file with the original one there are some differences (for example the last character is missing and other characters are also changed), therefore it seems to be a problem while downloading the content.
Do you know what cuould be wrong?
The problem seems to be related to Delphi code because I already tried with C# and the file content is downloaded correctly.
Don't use TBlobField.SaveToStream() directly, use TDataSet.CreateBlobStream() instead (which is what TBlobField.SaveToStream() uses internally anyway):
procedure TfrmDownload.Save;
var
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
fileStream.CopyFrom(bStream, 0);
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;
TDataSet.CreateBlobStream() allows the DataSet to decide the best way to access the BLOB data. If the returned TStream is not delivering the data correctly, then either the TStream class implementation that CreateBlobStream() uses is broken, or the underlying DB driver is buggy. Try taking CopyFrom() out of the equation so you can verify the data as it is being retrieved:
procedure TfrmDownload.Save;
const
MaxBufSize = $F000;
var
Buffer: array of Byte;
N: Integer;
fileStream: TFileStream;
bField: TField;
bStream: TStream;
begin
dmDigital.qrGetData.Open;
try
dmDigital.RequestLive := True;
bField := dmDigital.qrGetData.FieldByName('FILE_CONTENT');
bStream := bField.DataSet.CreateBlobStream(bField, bmRead);
try
fileStream := TFileStream.Create('FILE.XML', fmCreate);
try
//fileStream.CopyFrom(bStream, 0);
SetLength(Buffer, MaxBufSize);
repeat
N := bStream.Read(PByte(Buffer)^, MaxBufSize);
if N < 1 then Break;
// verify data here...
fileStream.WriteBuffer(PByte(Buffer)^, N);
until False;
FlushFileBuffers(fileStream.Handle);
finally
fileStream.Free;
end;
finally
bStream.Free;
end;
finally
dmDigital.qrGetData.Close;
end;
end;

How to set text dfm value to checked on all dfm files/Count number of lines in .dfm binary file

with below example i am counting number of lines in .dfm file and the count is coming wrong because .dfm is saved in binary format.
if i open .dfm file and do right click and set text dfm to checked and the count is coming correctly. Below is the code
function TForm1.FindNumberOfLinesInFile(FileName: String): Integer;
var
contents : TStringList;
filestream : TFileStream;
outStream : TMemoryStream;
begin
try
try
Result := 0;
contents := TStringList.Create;
if edtFileToSearch.Text = '.dfm' then
begin
contents.LoadFromFile(FileName);
//i am binary
if pos('OBJECT', Uppercase(contents[0])) = 0 then // Count is coming wrong with this
begin
contents.Clear;
fileStream := TFileStream.Create(FileName, fmShareDenyNone);
outStream := TMemoryStream.Create;
try
ObjectResourceToText(filestream,outStream);
outStream.Position := 0;
Contents.LoadFromStream(outStream);
finally
FreeAndNil(outStream);
end;
end
else
begin
fileStream := TFileStream.Create(FileName, fmShareDenyNone);
Contents.LoadFromStream(fileStream);
end;
end
else
begin
fileStream := TFileStream.Create(FileName, fmShareDenyNone);
Contents.LoadFromStream(filestream);
end;
Result := contents.Count;
finally
FreeAndNil(fileStream);
FreeAndNil(contents);
end;
except
on e: Exception do Result := -1;
end;
end;
i have 2 questions
1)how to set text dfm value to checked in all dfm files(i have around 1000 dfm files)?
2)how load binary file correctly and count number of lines?
Delphi comes with a command line tool to do this, named convert. Open up a command prompt and ensure that your Delphi bin directory is in the PATH. Then type:
C:\projects\myprocject> convert
The output will be something like this:
Delphi Form Conversion Utility Version 5.0
Copyright (c) 1995,99 Inprise Corporation
Usage: convert.exe [-i] [-s] [-t | -b]
-i Convert files in-place (output overwrites input)
-s Recurse subdirectories
-t Convert to text
-b Convert to binary
So, you should be able to write:
C:\projects\myprocject> convert -i -s -t *.dfm
to effect the change required.
David's answer addresses the first of your questions: You can convert all of your existing binary DFM's to text using the command line tool provided with Delphi.
As well as addressing your immediate problem this is also highly recommended as it will make it much easier (i.e. possible at all!) to visually diff changes to your DFM files in version control.
As for the second part, if for some reason you still want or need to handle binary DFM files in your code is to use the TestStreamFormat() function to determine whether a stream is a valid resource stream and whether it is binary or text format, before calling ObjectResourceToText() function only if required.
This helper function to return the contents of a specified filename (of a DFM) into a supplied TStrings (e.g. a TStringlist) demonstrates this and might simplify things for you:
procedure GetDfmIntoStrings(aFilename: String; aStrings: TStrings);
var
istrm, ostrm: TStream;
begin
ostrm := NIL;
istrm := TFileStream.Create(aFilename, fmOpenRead or fmShareDenyNone);
try
case TestStreamFormat(istrm) of
sofBinary : begin
ostrm := TStringStream.Create('');
ObjectResourceToText(istrm, ostrm)
end;
sofText : ostrm := istrm;
else
raise EFilerError.Create(aFilename + ' is not a valid resource stream (DFM)');
end;
ostrm.Position := 0;
aStrings.LoadFromStream(ostrm);
finally
if ostrm <> istrm then
ostrm.Free;
istrm.Free;
end;
end;

How to correctly download csv without 'save as' dialog' using TWebbrowser and Delphi?

I subscribe to a secure https web page containing a button that downloads some data as csv. I am trying to automate the download without the 'save as' dialog appearing but always seem to get an empty file downloaded. I suspect it has something to do with file type I'm using with IdHttp as most of my code works correctly.
Please can anyone help with my use of IdHttp or see where else I am going wrong?
The download button on the site calls some javascript to perform the download as follows
<a class="dlCSV" href="javascript:void(0);" onclick="dl_module.DownloadCsv();return false;">Download in CSV format…</a>
In Delphi I use a TWeb browser to log on securely and navigate to the page.
Clicking the download button in the TwebBrowser by hand shows the 'save as' dialog and then correctly downloads the csv data, defaulting to the filename 'data.csv'.
Automating clicking the button using execScript (below) also works, again showing the 'save as' dialog and correctly downloading the data with the same default filename.
procedure TForm1.BtnClickDownloadbuttonClick(Sender: TObject);
var TheDocument : IHTMLDocument2; // current HTML document
HTMLWindow: IHTMLWindow2; // parent window of current HTML document
begin
TheDocument := WebBrowser1.Document as IHTMLDocument2; // Get reference to current document
if not Assigned(TheDocument) then
Exit;
HTMLWindow := TheDocument.parentWindow; // Get parent window of current document
if Assigned(HTMLWindow) then
try
HTMLWindow.execScript('dl_module.DownloadCsv()', 'JavaScript'); // execute JS function to do download
except
on E : Exception do
begin
showmessage ('Exception class name = '+E.ClassName+ slinebreak
+ 'Exception message = '+E.Message);
end //on E
end;
end;
Then I added TLama's code from here How do I keep an embedded browser from prompting where to save a downloaded file? to use IDownloadManager to intercept the download and prevent the 'save as' dialog. This is where it seems to go wrong as I then get an empty file downloaded, and not with the name data.csv.
My code for function TWebBrowser.Download, TWebBrowser.InvokeEvent, function TWebBrowser.QueryService and TForm1.FormCreate are identical to that provided by TLama in the link above.
My procedure TForm1.Button1Click is the same except that I changed the download function being called to the one on my page by changing the line
HTMLWindow.execScript('SRT_stocFund.Export()', 'JavaScript');
to
HTMLWindow.execScript('dl_module.DownloadCsv()', 'JavaScript');
and my procedure TForm1.BeforeFileDownload is identical except that because I'm on a secure site I added the variable
var
LHandler: TIdSSLIOHandlerSocketOpenSSL; //<< on a https site
and after creating the Filestream I added the lines
LHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
IdHTTP.IOHandler := LHandler;
The issue seems to be in procedure TForm1.BeforeFileDownload where I note that the value of FileSource is
https://www.the_web_site_name/Ashx/GenericCSV.ashx.
There is a short delay while IdHTTP.Get(FileSource, FileStream); executes and then a file is created on my hard disc but called 'GenericCSV.ashx' (not data.csv) and the file is zero bytes long and completely empty.
Any ideas why its not downloading the file called data.csv (Do I somehow have to execute GenericCSV.ashx as well? if so how?)
For info here is my version of procedure TForm1.BeforeFileDownload
procedure TForm1.BeforeFileDownload(Sender: TObject; const FileSource: WideString; var Allowed: Boolean);
var
IdHTTP: TIdHTTP;
FileTarget: string;
FileStream: TMemoryStream;
LHandler: TIdSSLIOHandlerSocketOpenSSL; // added as its a https site
begin
FileSourceEdit.Text := FileSource;
Allowed := ShowDialogCheckBox.Checked;
if not Allowed then
try
IdHTTP := TIdHTTP.Create(nil);
try
FileStream := TMemoryStream.Create;
LHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil); //<<< added as its a https site
IdHTTP.IOHandler := LHandler; //<<< added as its a https site
try
IdHTTP.HandleRedirects := True;
IdHTTP.Get(FileSource, FileStream);
FileTarget := IdHTTP.URL.Document;
if FileTarget = '' then
FileTarget := 'File';
FileTarget := ExtractFilePath(ParamStr(0)) + FileTarget;
FileStream.SaveToFile(FileTarget);
finally
FileStream.Free;
end;
finally
IdHTTP.Free;
end;
ShowMessage('Downloading finished! File has been saved as:' + sLineBreak +
FileTarget);
except
on E: Exception do
ShowMessage(E.Message);
end;
end;
After you login, you can use this code to retrieve cookies from TWebBrowser
procedure GetHttpOnlyCookie(const AUrl: string; var ACookies: string);
const
INTERNET_COOKIE_HTTPONLY = 8192;
var
i: Integer;
hModule: THandle;
InternetGetCookieEx: function(lpszUrl, lpszCookieName, lpszCookieData
: PAnsiChar; var lpdwSize: DWORD; dwFlags: DWORD; lpReserved: pointer)
: BOOL; stdCall;
CookieSize: DWORD;
CookieData: PAnsiChar;
begin
LoadLibrary('wininet.dll');
hModule := GetModuleHandle('wininet.dll');
if (hModule <> 0) then
begin
#InternetGetCookieEx := GetProcAddress(hModule, 'InternetGetCookieExA');
if (#InternetGetCookieEx <> nil) then
begin
CookieSize := 1024;
Cookiedata := AllocMem(CookieSize);
try
if InternetGetCookieEx(PAnsiChar(AUrl), nil, Cookiedata, CookieSize, INTERNET_COOKIE_HTTPONLY, nil) then
begin
ACookies:=CookieData;
end;
finally
FreeMem(Cookiedata);
end;
end;
end;
end;
Then you just parse your cookies and add them (you have to create CookieManager in IdHTTP first)
IdHTTP1.CookieManager.AddServerCookie();
Then you start your download and it should work if you passed all parameters correctly (unfortunately, it is not possible to find out what your site requires).
Thank you smooty86 but I think its time I gave up trying to doing it this way and simply parse the page I can see.
I don't mind trying to understand code and adapting it to my needs but its so much harder trying to follow hints and suggestions when I'm working in the dark and especially don't know what parameters are needed everywhere. (I'm not daft, I've been programming for nearly 30 years and have spent over 4 years developing this particular data processing application but rarely touch web stuff)
However, the progress so far is...
Running your GetHttpOnlyCookie code after a successful login using automated filling in of the fields and clicking the login button returned an empty string so I used this code instead that at least seemed to return something that looked a little similar to your cookie string, ie seveveral strings separated by semicolons, most being name=value. (IdCookieManager1 is connected to IdHttp)
CookieList := Tstringlist.Create ;
try
CookieList.Delimiter := ';' ;
document := WebBrowser1.Document as IHTMLDocument2;
CookieList.DelimitedText := document.cookie;
for i := 0 to CookieList.Count-1 do
IdCookieManager1.AddCookie(CookieList[i],LOGIN_URL)
finally
CookieList.Free;
end;
Then in my original procedure BeforeFileDownload I try to log IdHttp into the site as well using code I adapted from here Log in to website from Delphi and the the cookies held in the cookie manager.
Displaying the string returned showed lots of HTML that appeared to represent the oringinal log in page and not the page you see after log in
procedure TFrmInportGrades.BeforeFileDownload(Sender: TObject; const FileSource: WideString; var Allowed: Boolean);
var
FileTarget: string;
FileStream: TMemoryStream;
request : Tstringlist;
s : string;
begin
FileSourceEdit.Text := FileSource;
Allowed := ShowDialogCheckBox.Checked;
if not Allowed then
begin
try
FileStream := TMemoryStream.Create;
IdHTTP.CookieManager := IdCookieManager1;
s := LogInIdHttp; //<<<< log in the IdHttp
showmessage(s); //<<<< debug
IdHTTP.Get(FileSource, FileStream);
FileTarget := IdHTTP.URL.Document;
if FileTarget = '' then
FileTarget := 'File';
FileTarget := ExtractFilePath(ParamStr(0)) + FileTarget;
FileStream.SaveToFile(FileTarget);
finally
FileStream.Free;
end;
ShowMessage('Downloading finished! File has been saved as:' + sLineBreak +
FileTarget);
end;
end;
The login code I used is below but I don't really know what I am doing here or what needs to be put into the Request.Add() parameters. I used 'Inspect element' from firefox to get the name of the user and password boxes and put the correct users name and password after the '=' sign in lines {3} and {4}. In lines {2},{6} and {7} I put the url of the log in site. I've no idea what lines {1}, {2}, {5} do or even if they are correct or necessary
function TFrmInportGrades.LogInIdHttp: string;
var
Request: TStringList;
Response: TMemoryStream;
LHandler: TIdSSLIOHandlerSocketOpenSSL; // added as its a https site
begin
Result := '';
try
Response := TMemoryStream.Create;
try
Request := TStringList.Create;
try
{1} Request.Add('op=login');
{2} Request.Add('redirect=https://www.thewebsite.com/Login.aspx' );
{3} Request.Add('ctl00$ctl00$Body$Body$loginManager$ctl00$loginEmailInput=usernme');
{4} Request.Add('ctl00$ctl00$Body$Body$loginManager$ctl01$passwordInput=password'});
LHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil); //<<< added as its a https site
IdHTTP.IOHandler := LHandler; //<<< added as its a https site
IdHTTP.AllowCookies := True;
IdHTTP.HandleRedirects := True;
{5} IdHTTP.Request.ContentType := 'application/x-www-form-urlencoded';
{6} IdHTTP.Post('https://www.thewebsite.com/Login.aspx', Request, Response);
{7} Result := IdHTTP.Get('https://www.thewebsite.com/Login.aspx');
finally
Request.Free;
end;
finally
Response.Free;
end;
except
on E: Exception do
ShowMessage(E.Message);
end;
end;
The net result of all this is that I don't get a file created at all now, not even a zero byte one. This all seems very overcomplicated simply to avoid or automate the 'Save As' dialog and is requiring lots of code that I won't be able to maintan afterwards. Unless somebody has a simpler solution I'll just parse what I can see (BTW I tried TEmbeddedWebBrowser but there is so little documentation for it I couldn't see how to make it download correctly. Might try again later.) Thank you for trying to help!

Use wave file from project

I currently can only playback my background sound from having my wave file next to my compiled exe. But I actually want to have a single static executable with the wave file inside. Is this possible in Delphi XE2?
This is my code:
SndPlaySound('.\Raw.wav', SND_ASYNC or SND_LOOP);
#This will play the Raw.wav that is next to my program.
You can add the SND_MEMORY flag, and pass a TResourceStream.Memory pointer as the first parameter.
First, use XE2's Project->Resources and Images menu item to add a new resource. Give it the path and filename of your .wav file, a resource type of RC_DATA (it's not in the drop down list, but you can manually type it in), and a resource name you can use at runtime to refer to it. (In my example, I'm using C:\Microsoft Office\Office12\MEDIA\APPLAUSE.WAV, and giving it a resource name of APPLAUSE.)
procedure TForm2.Button1Click(Sender: TObject);
var
Res: TResourceStream;
begin
Res := TResourceStream.Create(HInstance, 'APPLAUSE', 'RC_DATA');
try
Res.Position := 0;
SndPlaySound(Res.Memory, SND_MEMORY or SND_ASYNC or SND_LOOP);
finally
Res.Free;
end;
end;
If you use PlaySound() instead of sndPlaySound(), you can utilize the SND_RESOURCE flag to play the wave sound directly from its resource without having to load it into memory first.
type "WAVE" as Resource type when importing the wav file in the Resource Editor (Delphi 10, Project, Resources and Images)
and simply use
PlaySound(resourceIndentifierName, 0, SND_RESOURCE or SND_ASYNC);
P.S. uppercase no longer required
Just tested and it works on mine:
var
hFind, hRes: THandle;
Song : PChar;
begin
hFind := FindResource(HInstance, 'BATTERY', 'WAV');
if (hFind <> 0) then
begin
hRes := LoadResource(HInstance, hFind);
if (hRes <> 0) then
begin
Song := LockResource(hRes);
if Assigned(Song) then
begin
SndPlaySound(Song, snd_ASync or snd_Memory);
end;
UnlockResource(hRes);
end;
FreeResource(hFind);
end;

Delphi: deny access to file for other processes

How can I deny access (only to write) to a file for other processes? I will read\write a file all time.
I use
FileOpen('c:\1.txt', fmOpenReadWrite or fmShareDenyWrite)
but after (starting to load the file to StringList) I get error
Cannot open file C:\1.txt. The process cannot access the file because it is being used by other process."
Only I open the file.
Here, the error message is actually slightly misleading. The reason you can't load into the stringlist is because you already opened the file in read/write.
if you check the implementation of TStrings.LoadfromFile:
procedure TStrings.LoadFromFile(const FileName: string);
var
Stream: TStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
try
LoadFromStream(Stream);
finally
Stream.Free;
end;
end;
You see that it tries to open the file with a "DenyWrite" condition, but you already opened the file in write mode. That is the reason why it fails.
You can work around that by using LoadFromStream instead.
procedure TForm1.Button1Click(Sender: TObject);
var
Stream: TStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyNone);
try
Stringlist.LoadFromStream(Stream);
finally
Stream.Free;
end;
end;
Note that you will need to use fmShareDenyNone for this to work in that situation. Then again, you could probably reuse the Read/Write handle you got from your OpenFile, probably do something like this:
procedure TForm1.Button1Click(Sender: TObject);
var
Stream: TStream;
iPosition : Int64;
begin
Stream := THandleStream.Create(FHandle); //FHandle is the read/write handle returned by OpenFile
try
iPosition := Stream.Position;
Stream.Seek(0, soFromBeginning);
Stringlist.LoadFromStream(Stream);
Stream.Position := iPosition;
//Restore stream position.
finally
Stream.Free;
end;
end;
But be advised that these approach might have a few "gotchas" I'm unaware of.

Resources