i am working on a little byte patching program but i encountered an error.
copying the file before modifying fails with no error, (no copied output is seen) but the file patches successfully.
Here is the Patch Code
procedure DoMyPatch();
var
i: integer;
FileName: string;
input: TFileStream;
FileByteArray, ExtractedByteArray: array of Byte;
begin
FileName := 'Cute1.res';
try
input := TFileStream.Create(FileName, fmOpenReadWrite);
except
begin
ShowMessage('Error Opening file');
Exit;
end
end;
input.Position := 0;
SetLength(FileByteArray, input.size);
input.Read(FileByteArray[0], Length(FileByteArray));
for i := 0 to Length(FileByteArray) do
begin
SetLength(ExtractedByteArray, Length(OriginalByte));
ExtractedByteArray := Copy(FileByteArray, i, Length(OriginalByte));
// function that compares my array of bytes
if CompareByteArrays(ExtractedByteArray, OriginalByte) = True then
begin
// Begin Patching
CopyFile(PChar(FileName), PChar(ChangeFileExt(FileName, '.BAK')),
true); =======>>> fails at this point, no copied output is seen.
input.Seek(i, SoFromBeginning);
input.Write(BytetoWrite[0], Length(BytetoWrite)); =====>>> patches successfully
input.Free;
ShowMessage('Patch Success');
Exit;
end;
end;
if Assigned(input) then
begin
input.Free;
end;
ShowMessage('Patch Failed');
end;
sidenote : it copies fine if i close the filestream before attempting copy.
by the way, i have tested it on Delphi 7 and XE7.
Thanks
You cannot copy the file because you locked it exclusively when you opened it for the file stream, which is why CopyFile fails.
You should close the file before attempting to call CopyFile. Which would require you to reopen the file to patch it. Or perhaps open the file with a different sharing mode.
Some other comments:
The exception handling is badly implemented. Don't catch exceptions here. Let them float up to the high level.
Lifetime management is fluffed. You can easily leak as it stands. You need to learn about try/finally.
You overrun buffers. Valid indices for a dynamic array are 0 to Length(arr)-1 inclusive. Or use low() and high().
You don't check the value returned by CopyFile. Wrap it with a call to Win32Check.
The Copy function returns a new array. So you make a spurious call to SetLength. To copy the entire array use the one parameter overload of Copy.
Showing messages in this function is probably a mistake. Better to let the caller provide user feedback.
There are loads of other oddities in the code and I've run out of energy to point them all out. I think I got the main ones.
Related
I am experiencing a strange error which I have narrowed down to a small piece of code (see below). I am reading a binary file into a TMemoryStream (variable ms1), through a TFileStream object (variable fs). Then I want to copy the binary data to another TMemoryStream object (ms2). This is what gives me the "stream read error" exception. Strange thing is, that if I don't load up the ms1 object with the file contents, things work fine, i.e. ms2.CopyFrom does not give me an exception..
Any help is greatly appreciated....
procedure TForm5.BitBtn1Click(Sender: TObject);
var
ms1: TMemoryStream;
fs: TFileStream;
ms2 : TMemoryStream;
FilePath: string;
begin
FilePath := 'C:\weekcpdf_tech6.bin';
ms1 := TMemoryStream.Create;
fs := nil;
try
ms1 := TMemoryStream.Create;
fs := TFileStream.Create(FilePath, fmOpenRead);
ms1.CopyFrom(fs, fs.Size);
ms2 := TMemoryStream.Create;
ms2.CopyFrom(ms1, ms1.Size);
finally
FreeAndNil(fs);
FreeAndNil(ms1);
end;
end;
First, you have a memory leak, since you create two memory stream objects that you assign to ms1. Remove the second one.
Second, after ms1.CopyFrom(fs, fs.Size); you must set ms1.Position := 0 so that you copy from the start of ms1 -- and not from the end of it.
This is actually fairly well documented:
CopyFrom copies Count bytes from the stream specified by Source into the stream. It then moves the current position by Count bytes and returns the number of bytes copied.
Hence, after ms1.CopyFrom(fs, fs.Size); you are at the end of ms1. Further,
If Count is greater than or less than 0, CopyFrom reads from the current position in Source.
Therefore, ms2.CopyFrom(ms1, ms1.Size); will read from the current position (= the end!) of ms1. Hence, you will likely try to read a lot of bytes that do not exist. And what happens then?
Because the CopyFrom method uses ReadBuffer and WriteBuffer to do the effective copying, if the Count is greater than the SourceStream size, ReadBuffer throws an exception stating that a stream read error has occured [sic!].
Always read the docs! :)
(Although I must admit that the last quote isn't quite perfect in this situation.)
I'm trying to create a simple example of using IFileOperation to delete the files in a
given directory, to include in the answer to another q for comparison with other methods.
Below is the code of my MRE. It
successfully creates 1000 files in a subdirectory off C:\Temp and then attempts to delete
them in the DeleteFiles method. This supposedly "easy" task fails but I'm not sure
exactly where it comes off the rails. The comments in the code show what I'm expecting
and the actual results. On one occasion, instead of the exception noted, I got a pop-up
asking for confirmation to delete an item with an odd name which was evidently an array of
numbers referring to a shell item, but my attempt to capture it using Ctrl-C failed;
I'm fairly sure I'm either missing a step or two, misusing the interfaces involved
or both. My q is, could anybody please show the necessary corrections to the code to get IFileOperation.DeleteItems() to delete the files in question, as I am completely out of my depth with this stuff? I am not interested in alternative methods of deleting these files, using the shell interfaces or otherwise.
procedure TForm2.DeleteFiles;
var
iFileOp: IFileOperation;
iIDList : ItemIDList;
iItemArray : IShellItemArray;
iArray : Array[0..1] of ItemIDList;
Count : DWord;
begin
iFileOp := CreateComObject(CLSID_FileOperation) as IFileOperation;
iIDList := ILCreateFromPath(sPath)^;
// IFileOperation.DeleteItems seems to require am IShellItemArray, so the following attempts
// to create one
// The definition of SHCreateShellItemArrayFromIDLists
// seems to require a a zero-terminated array of ItemIDLists so the next steps
// attempt to create one
ZeroMemory(#iArray, SizeOf(iArray));
iArray[0] := iIDList;
OleCheck(SHCreateShellItemArrayFromIDLists(1, #iArray, iItemArray));
// Next test the number of items in iItemArray, which I'm expecting to be 1000
// seeing as the CreateFiles routine creats that many
OleCheck(iItemArray.GetCount(Count));
Caption := IntToStr(Count); // Duh, this shows Count to be 1, not the expected 1000
OleCheck(iFileOp.DeleteItems(iItemArray));
OleCheck( iFileOp.PerformOperations );
// Returns Exception 'No object for moniker'
end;
procedure TForm2.Button1Click(Sender: TObject);
begin
DeleteFiles;
end;
procedure CreateFiles;
var
i : Integer;
SL : TStringList;
FileName,
FileContent : String;
begin
SL := TStringList.Create;
try
if not (DirectoryExists(sPath)) then
MkDir(sPath);
SL.BeginUpdate;
for i := 0 to 999 do begin
FileName := Format('File%d.Txt', [i]);
FileContent := Format('content of file %s', [FileName]);
SL.Text := FileContent;
SL.SaveToFile(sPath + '\' + FileName);
end;
SL.EndUpdate;
finally
SL.Free;
end;
end;
procedure TForm2.FormCreate(Sender: TObject);
begin
CreateFiles;
end;
You are leaking the memory returned by ILCreateFromPath(), you need to call ILFree() when you are done using the returned PItemIDList.
Also, you should not be dereferencing the PItemIDList. SHCreateShellItemArrayFromIDLists() expects an array of PItemIDList pointers, but you are giving it an array of ItemIDList instances.
Try this instead:
procedure TForm2.DeleteFiles;
var
iFileOp: IFileOperation;
iIDList : PItemIDList;
iItemArray : IShellItemArray;
Count : DWord;
begin
iFileOp := CreateComObject(CLSID_FileOperation) as IFileOperation;
iIDList := ILCreateFromPath(sPath);
try
OleCheck(SHCreateShellItemArrayFromIDLists(1, #iIDList, iItemArray));
finally
ILFree(iIDList);
end;
// Next test the number of items in iItemArray, which I'm expecting to be 1000
// seeing as the CreateFiles routine creates that many
OleCheck(iItemArray.GetCount(Count));
Caption := IntToStr(Count); // Duh, this shows Count to be 1, not the expected 1000
OleCheck(iFileOp.DeleteItems(iItemArray));
OleCheck( iFileOp.PerformOperations );
// Returns Exception 'No object for moniker'
end;
That being said, even if this were working correctly, you are not creating an IShellItemArray containing 1000 IShellItems for the individual files. You are creating an IShellItemArray containing 1 IShellItem for the C:\Temp subdirectory itself.
Which is fine if your goal is to delete the whole folder. But in that case, I would suggest using SHCreateItemFromIDList() or SHCreateItemFromParsingName() instead, and then pass that IShellItem to IFileOperation.DeleteItem().
But, if your goal is to delete the individual files without deleting the subdirectory as well, then you will have to either:
get the IShellFolder interface for the subdirectory, then enumerate the relative PIDLs of its files using IShellFolder.EnumObjects(), and then pass the PIDLs in an array to SHCreateShellItemArray().
get the IShellFolder interface of the subdirectory, then query it for an IDataObject interface using IShellFolder.GetUIObjectOf(), and then use SHCreateShellItemArrayFromDataObject(), or just give the IDataObject directly to IFileOperation.DeleteItems().
get an IShellItem interface for the subdirectory, then query its IEnumShellItems interface using IShellItem.BindToHandler(), and then pass that directly to IFileOperation.DeleteItems().
I feel like this should be easy, but google is totally failing me at the moment. I want to open a file, or create it if it doesn't exist, and write to it.
The following
AssignFile(logFile, 'Test.txt');
Append(logFile);
throws an error on the second line when the file doesn't exist yet, which I assume is expected. But I'm really failing at finding out how to a) test if the file exists and b) create it when needed.
FYI, working in Delphi XE.
You can use the FileExists function and then use Append if exist or Rewrite if not.
AssignFile(logFile, 'Test.txt');
if FileExists('test.txt') then
Append(logFile)
else
Rewrite(logFile);
//do your stuff
CloseFile(logFile);
Any solution that uses FileExists to choose how to open the file has a race condition. If the file's existence changes between the time you test it and the time you attempt to open the file, your program will fail. Delphi doesn't provide any way to solve that problem with its native file I/O routines.
If your Delphi version is new enough to offer it, you can use the TFile.Open with the fmOpenOrCreate open mode, which does exactly what you want; it returns a TFileStream.
Otherwise, you can use the Windows API function CreateFile to open your file instead. Set the dwCreationDisposition parameter to OPEN_ALWAYS, which tells it to create the file if it doesn't already exist.
You should be using TFileStream instead. Here's a sample that will create a file if it doesn't exist, or write to it if it does:
var
FS: TFileStream;
sOut: string;
i: Integer;
Flags: Word;
begin
Flags := fmOpenReadWrite;
if not FileExists('D:\Temp\Junkfile.txt') then
Flags := Flags or fmCreate;
FS := TFileStream.Create('D:\Temp\Junkfile.txt', Flags);
try
FS.Position := FS.Size; // Will be 0 if file created, end of text if not
sOut := 'This is test line %d'#13#10;
for i := 1 to 10 do
begin
sOut := Format(sOut, [i]);
FS.Write(sOut[1], Length(sOut) * SizeOf(Char));
end;
finally
FS.Free;
end;
end;
If you are just doing something simple, the IOUtils Unit is a lot easier. It has a lot of utilities for writing to files.
e.g.
procedure WriteAllText(const Path: string; const Contents: string);
overload; static;
Creates a new file, writes the specified string to the file, and then
closes the file. If the target file already exists, it is overwritten.
You can also use the load/save feature in a TStringList to solve your problem.
This might be a bad solution, because the whole file will be loaded into memory, modified in memory and then saved to back to disk. (As opposed to your solution where you just write directly to the file). It's obviously a bad solution for multiuser situations.
But this approach is OK for smaller files, and it is easy to work with and easy understand.
const
FileName = 'test.txt';
var
strList: TStringList;
begin
strList := TStringList.Create;
try
if FileExists(FileName) then
strList.LoadFromFile(FileName);
strList.Add('My new line');
strList.SaveToFile(FileName);
finally
strList.Free;
end;
end;
I had some code before I moved to Unicode and Delphi 2009 that appended some text to a log file a line at a time:
procedure AppendToLogFile(S: string);
// this function adds our log line to our shared log file
// Doing it this way allows Wordpad to open it at the same time.
var F, C1 : dword;
begin
if LogFileName <> '' then begin
F := CreateFileA(Pchar(LogFileName), GENERIC_READ or GENERIC_WRITE, 0, nil, OPEN_ALWAYS, 0, 0);
if F <> 0 then begin
SetFilePointer(F, 0, nil, FILE_END);
S := S + #13#10;
WriteFile(F, Pchar(S)^, Length(S), C1, nil);
CloseHandle(F);
end;
end;
end;
But CreateFileA and WriteFile are binary file handlers and are not appropriate for Unicode.
I need to get something to do the equivalent under Delphi 2009 and be able to handle Unicode.
The reason why I'm opening and writing and then closing the file for each line is simply so that other programs (such as WordPad) can open the file and read it while the log is being written.
I have been experimenting with TFileStream and TextWriter but there is very little documentation on them and few examples.
Specifically, I'm not sure if they're appropriate for this constant opening and closing of the file. Also I'm not sure if they can make the file available for reading while they have it opened for writing.
Does anyone know of a how I can do this in Delphi 2009 or later?
Conclusion:
Ryan's answer was the simplest and the one that led me to my solution. With his solution, you also have to write the BOM and convert the string to UTF8 (as in my comment to his answer) and then that worked just fine.
But then I went one step further and investigated TStreamWriter. That is the equivalent of the .NET function of the same name. It understands Unicode and provides very clean code.
My final code is:
procedure AppendToLogFile(S: string);
// this function adds our log line to our shared log file
// Doing it this way allows Wordpad to open it at the same time.
var F: TStreamWriter;
begin
if LogFileName <> '' then begin
F := TStreamWriter.Create(LogFileName, true, TEncoding.UTF8);
try
F.WriteLine(S);
finally
F.Free;
end;
end;
Finally, the other aspect I discovered is if you are appending a lot of lines (e.g. 1000 or more), then the appending to the file takes longer and longer and it becomes quite inefficient.
So I ended up not recreating and freeing the LogFile each time. Instead I keep it open and then it is very fast. The only thing I can't seem to do is allow viewing of the file with notepad while it is being created.
For logging purposes why use Streams at all?
Why not use TextFiles? Here is a very simple example of one of my logging routines.
procedure LogToFile(Data:string);
var
wLogFile: TextFile;
begin
AssignFile(wLogFile, 'C:\MyTextFile.Log');
{$I-}
if FileExists('C:\MyTextFile.Log') then
Append(wLogFile)
else
ReWrite(wLogFile);
WriteLn(wLogfile, S);
CloseFile(wLogFile);
{$I+}
IOResult; //Used to clear any possible remaining I/O errors
end;
I actually have a fairly extensive logging unit that uses critical sections for thread safety, can optionally be used for internal logging via the OutputDebugString command as well as logging specified sections of code through the use of sectional identifiers.
If anyone is interested I'll gladly share the code unit here.
Char and string are Wide since D2009. Thus you should use CreateFile instead of CreateFileA!
If you werite the string you shoudl use Length( s ) * sizeof( Char ) as the byte length and not only Length( s ). because of the widechar issue. If you want to write ansi chars, you should define s as AnsiString or UTF8String and use sizeof( AnsiChar ) as a multiplier.
Why are you using the Windows API function instead of TFileStream defined in classes.pas?
Try this little function I whipped up just for you.
procedure AppendToLog(filename,line:String);
var
fs:TFileStream;
ansiline:AnsiString;
amode:Integer;
begin
if not FileExists(filename) then
amode := fmCreate
else
amode := fmOpenReadWrite;
fs := TFileStream.Create(filename,{mode}amode);
try
if (amode<>fmCreate) then
fs.Seek(fs.Size,0); {go to the end, append}
ansiline := AnsiString(line)+AnsiChar(#13)+AnsiChar(#10);
fs.WriteBuffer(PAnsiChar(ansiline)^,Length(ansiline));
finally
fs.Free;
end;
Also, try this UTF8 version:
procedure AppendToLogUTF8(filename, line: UnicodeString);
var
fs: TFileStream;
preamble:TBytes;
outpututf8: RawByteString;
amode: Integer;
begin
if not FileExists(filename) then
amode := fmCreate
else
amode := fmOpenReadWrite;
fs := TFileStream.Create(filename, { mode } amode, fmShareDenyWrite);
{ sharing mode allows read during our writes }
try
{internal Char (UTF16) codepoint, to UTF8 encoding conversion:}
outpututf8 := Utf8Encode(line); // this converts UnicodeString to WideString, sadly.
if (amode = fmCreate) then
begin
preamble := TEncoding.UTF8.GetPreamble;
fs.WriteBuffer( PAnsiChar(preamble)^, Length(preamble));
end
else
begin
fs.Seek(fs.Size, 0); { go to the end, append }
end;
outpututf8 := outpututf8 + AnsiChar(#13) + AnsiChar(#10);
fs.WriteBuffer(PAnsiChar(outpututf8)^, Length(outpututf8));
finally
fs.Free;
end;
end;
If you try to use text file or Object Pascal typed/untyped files in a multithreaded application you gonna have a bad time.
No kidding - the (Object) Pascal standard file I/O uses global variables to set file mode and sharing. If your application runs in more than one thread (or fiber if anyone still use them) using standard file operations could result in access violations and unpredictable behavior.
Since one of the main purposes of logging is debugging a multithreaded application, consider using other means of file I/O: Streams and Windows API.
(And yes, I know it is not really an answer to the original question, but I do not wish to log in - therefor I do not have the reputation score to comment on Ryan J. Mills's practically wrong answer.)
I am loading a file into a array in binary form this seems to take a while
is there a better faster more efficent way to do this.
i am using a similar method for writing back to the file.
procedure openfile(fname:string);
var
myfile: file;
filesizevalue,i:integer;
begin
assignfile(myfile,fname);
filesizevalue:=GetFileSize(fname); //my method
SetLength(dataarray, filesizevalue);
i:=0;
Reset(myFile, 1);
while not Eof(myFile) do
begin
BlockRead(myfile,dataarray[i], 1);
i:=i+1;
end;
CloseFile(myfile);
end;
If your really want to read a binary file fast, let windows worry about buffering ;-) by using Memory Mapped Files. Using this you can simple map a file to a memory location an read like it's an array.
Your function would become:
procedure openfile(fname:string);
var
InputFile: TMappedFile;
begin
InputFile := TMappedFile.Create;
try
InputFile.MapFile(fname);
SetLength(dataarray, InputFile.Size);
Move(PByteArray(InputFile.Content)[0], Result[0], InputFile.Size);
finally
InputFile.Free;
end;
end;
But I would suggest not using the global variable dataarray, but either pass it as a var in the parameter, or use a function which returns the resulting array.
procedure ReadBytesFromFile(const AFileName : String; var ADestination : TByteArray);
var
InputFile : TMappedFile;
begin
InputFile := TMappedFile.Create;
try
InputFile.MapFile(AFileName);
SetLength(ADestination, InputFile.Size);
Move(PByteArray(InputFile.Content)[0], ADestination[0], InputFile.Size);
finally
InputFile.Free;
end;
end;
The TMappedFile is from my article Fast reading of files using Memory Mapping, this article also contains an example of how to use it for more "advanced" binary files.
You generally shouldn't read files byte for byte. Use BlockRead with a larger value (512 or 1024 often are best) and use its return value to find out how many bytes were read.
If the size isn't too large (and your use of SetLength seems to support this), you can also use one BlockRead call reading the complete file at once. So, modifying your approach, this would be:
AssignFile(myfile,fname);
filesizevalue := GetFileSize(fname);
Reset(myFile, 1);
SetLength(dataarray, filesizevalue);
BlockRead(myFile, dataarray[0], filesizevalue);
CloseFile(myfile);
Perhaps you could also change the procedure to a boolean function named OpenAndReadFile and return false if the file couldn't be opened or read.
It depends on the file format. If it consists of several identical records, you can decide to create a file of that record type.
For example:
type
TMyRecord = record
fieldA: integer;
..
end;
TMyFile = file of TMyRecord;
const
cBufLen = 100 * sizeof(TMyRecord);
var
file: TMyFile;
i : Integer;
begin
AssignFile(file, filename);
Reset(file);
i := 0;
try
while not Eof(file) do begin
BlockRead(file, dataarray[i], cBufLen);
Inc(i, cBufLen);
end;
finally
CloseFile(file);
end;
end;
If it's a long enough file that reading it this way takes a noticeable amount of time, I'd use a stream instead. The block read will be a lot faster, and there's no loops to worry about. Something like this:
procedure openfile(fname:string);
var
myfile: TFileStream;
filesizevalue:integer;
begin
filesizevalue:=GetFileSize(fname); //my method
SetLength(dataarray, filesizevalue);
myFile := TFileStream.Create(fname);
try
myFile.seek(0, soFromBeginning);
myFile.ReadBuffer(dataarray[0], filesizevalue);
finally
myFile.free;
end;
end;
It appears from your code that your record size is 1 byte long. If not, then change the read line to:
myFile.ReadBuffer(dataarray[0], filesizevalue * SIZE);
or something similar.
Look for a buffered TStream descendant. It will make your code a lot faster as the disk read is done fast, but you can loop through the buffer easily. There are various about, or you can write your own.
If you're feeling very bitheaded, you can bypass Win32 altogether and call the NT Native API function ZwOpenFile() which in my informal testing does shave a tiny bit off. Otherwise, I'd use Davy's Memory Mapped File solution above.