This question might or might not solve my problem - but I hope to learn how Delphi/Windows can behave in a way which can cause this.
I have an application which uses a 3rd party component to load an Outlook .msg file.
In some cases (specific mails) the application freezes when calling SetLength (inside of the component, I have the source code).
This happens sometimes when setLength is called inside of a procedure which loads the properties from the file (stream). It happens the exact same place on the same mail - and can be reproduced every time.
Obviously the component does a lot of stuff and it is probably a sideeffect of some of this. However, the mails contains confidential data which I cannot send to the developer of the 3rd party component, so I cannot send it to him to debug it.
The program is running under windows XP on a domain.
The curious thing is that it only happens when the user running the program is not set to be administrator on the local machine.
ms := TMemoryStream.Create;
try
WriteStorageToStream(SubStorage, ms);
ApplyValue(ms, ms.Size)
finally
ms.Free;
end;
procedure ApplyValue(Stream: TStream; brLen: Integer);
var
s: AnsiString;
begin
SetLength(s, brLen); // this freezes it all. brLen=3512
FillChar(s[1], brLen, #0);
Stream.Read(s[1], brLen);
Value := s;
end;
What WriteStorageToStream does exactly is unknown to me, but since we are not manipulating the stream and brLen has an integer value, I assume it's irrelevant.
I'd say it was simple memory overwrite, causing a failure of the memory manager when the SetLength is called which then tries to use the memory management structures. The problem is in WriteStorageToStream(SubStorage, ms);
To find it, use the FastMM debug version with the memory overwrite detection options turned on.
There's absolutely no reason SetLength would freeze on a AnsiString that's only 3512 characters long. How are you sure that it's freezing there and not somewhere earlier (like in WriteStorageToSteam)? Presumably you're stepping through this in the debugger. Does the CPU spike to 100% on that process thread when it's frozen? The fact that it freezes only on certain emails indicates to me that something in the contents of those emails is causing the freeze. The call to SetLength has nothing to do with the contents; it only cares about the length.
Related
In a simple Delphi VCL application with one button on the form and the following OnButton event code:
procedure TForm1.Button1Click(Sender: TObject);
var
OpenDialog : TFileOpenDialog;
begin
OpenDialog := TFileOpenDialog.Create(nil);
try
OpenDialog.Options := OpenDialog.Options + [fdoPickFolders];
if not OpenDialog.Execute then
exit;
finally
OpenDialog.Free;
end;
end;
When I Execute the dialog my application memory usage more then doubles but after I OpenDialog.Free that memory is not released.(I'm using ProcessExplorer to see how much memory my application is using)
How can I make it so that after I Free the object my memory usage returns back to what it was before I called the dialog?
This is normal operation. Memory managers typically don't return memory to the system and instead cache it later re-use. Further, the modules that are loaded the first time a file dialog is shown remain loaded in your process.
It's entirely possible that the system caches other resources to improve performance for subsequent uses of file dialogs.
This behaviour leads to better performance. Were you to be able to force the memory to be returned to the system, your program would perform more slowly.
Your code is correct. There is no leak. There is no problem for you to solve.
I'm developing as small diabetes program using Delphi 5 and ADO. I do a little query like this:
function GetLowestGlucoseLevel(StartDate:string;EndDate:string): Integer;
var
Q:TADOQuery;
begin
try
Q:=TADOQuery.Create(Application); //Separate unit, owner set to App
Q.Connection:=dtMod.ADOCon;
Q.DisableControls;
Q.Close;
Q.SQL.Clear;
Q.SQL.Add('SELECT Min(qGlucose.Glucose) AS MinOfGlucose from qGlucose');
Q.Parameters[0].Value:=StartDate;
Q.Parameters[1].Value:=EndDate;
Q.Open;
Result:=Q.FieldByName('MinOfGlucose').AsInteger;
Q.Close;
finally
Q:=nil;
Q.Free;
end;
end;
The query runs OK and returns the result as expected. However, when I checked Windows Task Manager, memory usage keeps on rising rather than decreasing after the query.
How to fix this?
Thanks!
You are leaking the TADOQuery by first setting it to nil, and then calling Free on the nil variable (which does nothing)
Did you install Delphi 5 updates? The
RTM ADO implementation is known to
have issues.
Use FastMM4, it should work with
Delphi 5 as well, and tell you more
about where the leaks are.
The Delphi way:
function GetLowestGlucoseLevel(const StartDate:string; const EndDate:string): Integer;
var
Q:TADOQuery;
begin
Q:=TADOQuery.Create(nil); //(nil) because local use only. Placed before try\finally block
//because if it fails to .create then there would be no object to
//.Free
try
Q.Connection := dtMod.ADOCon;
//------can erase these------
//Q.DisableControls; //No controls attached so unnecessary
//Q.Close; //Q is local and was never opened so no need to close
//Q.SQL.Clear; //Q is local and was never filled so no need to clear
Q.SQL.Add('SELECT Min(qGlucose.Glucose) AS MinOfGlucose from qGlucose');
Q.Parameters[0].Value:=StartDate;
Q.Parameters[1].Value:=EndDate;
Q.Open;
Result := Q.FieldByName('MinOfGlucose').AsInteger;
Q.Close;
finally
Q.Free;
//Q := nil //not needed because Q's scope is local
end;
end;
Quote:
finally
Q:=nil;
Q.Free;
end;
You're kidding, right? First nil the variable, then free it? You're a genius! :-)
Use:
finally
Q.Free;
Q:=nil;
end;
Or don't even bother assigning nil to it, since Q is a local variable...
But rereading your code, I notice you use Application as owner. As a result, it will not really be a leak, since it will be freed when the application is freed. If you use a form, it would be freed when the owner form gets freed. What you should try is to call this query about 100.000 times to check if it keeps reserving memory or if it's just increasing memory until a certain size has been reached. The latter is more likely, since the memory is reserved for future ADO calls.
As others pointed out, the finally section should have the 2 statements reversed, like so:
finally
Q.Free;
Q:=nil; // <- not even necessary since this is a local var
end;
Or you can call SysUtils.FreeAndNil(Q) (if that is available in Delphi 5, not sure).
Beside that, the TaskManager is an awful instrument to determine memory usage anyway. You might release the memory for Q, but that does not automatically mean the Delphi memory manager releases the memory to the OS.
Aside inversing the lines as Arjan, jasonpenny and WorkShop Alex said, you can use Process Explorer to see the real consumption of memory (Private Bytes) of the process. Task Manager is not really suited to this task, as it only shows the working set of the process.
I have been trying to get to the bottom of this problem off and on for the past 2 days and I'm really stuck. Hopefully some smart folks can help me out.
The issue is that I have a function that I call in a thread that downloads a file (using Synapse libraries) from a website that is passed to it. However, I've found that every once in a while there are sites where it will not pull down a file, but wget or Firefox/IE will download it without issue.
Digging into it, I've found some curious things. Here is the relevant code:
uses
//[..]
HTTPSend,
blcksock;
//[..]
type
TMyThread = class(TThread)
protected
procedure Execute; override;
private
{ Private declarations }
fTheUrl: string;
procedure GetFile(const TheUrl: string);
public
property thrd_TheUrl: string read fTheUrl write fTheUrl;
end;
implementation
[..]
procedure TMyThread.GetFile(const TheUrl: string);
var
HTTP: THTTPSend;
success: boolean;
sLocalUrl: string;
IsSame : boolean;
begin
HTTP := THTTPSend.Create;
try
HTTP.UserAgent :=
'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727)';
HTTP.ProxyHost := 'MYPROXY.COM';
HTTP.ProxyPort := '80';
sLocalUrl :=
'http://web.archive.org/web/20071212205017/energizer.com/usbcharger/download/UsbCharger_setup_V1_1_1.exe';
IsSame := SameText(sLocalUrl, sTheUrl); //this equals True when I debug
///
///
/// THIS IS WHERE THE ISSUE BEGINS
/// I will comment out 1 of the following when debugging
///
HTTP.HTTPMethod('GET', sLocalUrl); // ----this works and WILL download the file
HTTP.HTTPMethod('GET', sTheUrl); // --- this always fails, and HTTP.ResultString contains "Not Found"
success := SysUtils.UpperCase(HTTP.ResultString) = 'OK';
if HTTP.ResultCode > 0 then
success := True; //this is here just to keep the value around while debugging
finally
HTTP.Free;
end;
end;
procedure TMyThread.Execute
begin
//fTheURL contains this value: http://web.archive.org/web/20071212205017/energizer.com/usbcharger/download/UsbCharger_setup_V1_1_1.exe
GetFile(fTheUrl);
end;
The problem is that when I assign a local variable to the function and give it the URL directly, everything works. However, when passing the variable into the function, it fails. Anyone have any ideas?
HTTP.HTTPMethod('GET', sLocalUrl); // ----this works and WILL download the file
HTTP.HTTPMethod('GET', sTheUrl); // --- this always fails, and HTTP.ResultString contains "Not Found"
I'm using the latest version of Synapse from their SVN repository (version from 2 days ago).
NOTE: The file I am attempting to download is known to have a virus, the program I am writing is meant to download malicious files for analysis. So, don't execute the file once you download it.
However, I'm using this URL b/c this is the one I can reproduce the issue with.
Your code is missing the crucial detail how you use TMyThread class. However, you write
every once in a while there are sites where it will not pull down a file, but wget or Firefox/IE will download it without issue.
which sounds like a timing issue.
Using the local variable works every time. Using the function parameter works only some of the time. That may be caused by the function parameter not containing the correct URL some of the time.
You need to be aware that creating a non-suspended thread may result in it starting to execute immediately (and possibly even to complete), before the next line after the construction call has even started to execute. Setting any property of the thread object after the thread has been created may therefore not work, as the thread execution may be past the point where the property is read. The fTheUrl field of the thread object will be an empty string initially, so whether the thread downloads the file will depend on it being set before.
Your fTheUrl field isn't even protected by a synchronization primitive. Both the thread proc and the code in the main thread can access it concurrently. Sharing data in this way between threads is an unsafe thing to do and may result in any thing from wrong behaviour to actual crashes.
If your thread is really used to download a single file you should remove the write access to the property, and write a custom constructor with a parameter for the URL. That will properly initialize the field before the thread starts.
If you are downloading several files in your program you should really not create a thread for each. Use a pool of threads (may be only one even) that will be assigned the files to download. For that a thread property is the right solution, but then it will need to be implemented with synchronization, and the thread should block when no file is to be downloaded, and unblock when the property is set. The download thread (or threads) would be consumer(s) in a producer-consumer implementation. Stack Overflow has questions and answers regarding this in the Delphi tag, in particular in the questions where alternatives to Suspend() and Resume() are discussed.
One last thing: Don't let unhandled exceptions escape the Execute() method. I'm not sure about whether Delphi 2010 handles these exceptions in the VCL, but unhandled exceptions in a thread may lead to problems like app crashes or freezes.
Well, I'm almost embarrassed to report it, but I owe it to those that took the time to respond.
The issue had nothing to do with Synapse, or TThread, but instead had everything to do with the fact that the URL is case-sensitive!
In my full application, I had a helper function that lowercased the URL (for some reason). I removed that and everything started working again...
Please update last Synapse Revision 127.
I am trying to extend a 3rd party application so that it can be invoked via command line in addition to using the windows form GUI (mixed mode is desired). It's a fairly simple program which basically loads a file and then you click a button it starts sending UDP network packets.
I need to invoke the application from another and would like to pass in an argument and need to be able to return the ExitCode to the calling app. From what i've read, in order to do so you need to add the compiler directive {APPTYPE CONSOLE}.
I did this and my application worked as I wanted it to except sending the network packets slowed down to a crawl. I found that whenever I moved my mouse around on the form. That the network transfer rate increased significantly. I suspect there is some type of Windows Message queue problem and moving mouse is causing interrupts which in turn is causing the message queue to be processed?
I have googled around and tried calling Application.ProcessMessages and PeekMessages in a Timer with a 1ms interval and that didn't help at all. I found in this user manual for some other application it says that Indy 10 is supported in both APPTYPE CONSOLE and GUI types. Quite frankly this just confuses me as I would have assumed that all network library would work in both modes... but like I said I'm not familiar with Delphi.
I am positive that the issue is isolated to a single line in my application and that is whether or not {APPTYPE CONSOLE} is included or not.
Anyone have any ideas?
Version Info:
Delphi 7 Personal (Build 4.453)
Indy 9.0.4
If you add {APPTYPE CONSOLE} to your application even though you desire mixed mode execution, then you will have to live with a console even when the application is in GUI mode. You can of course close the console, but this will cause some flicker and feels a bit hackish to me.
You should be able to do what you want without a console program. A small test program proves that the exit code can be read from a GUI program:
procedure TForm1.Timer1Timer(Sender: TObject);
begin
Close;
end;
procedure TForm1.FormCreate(Sender: TObject);
begin
ExitCode := 42;
Timer1.Interval := 1000;
Timer1.Enabled := TRUE;
end;
If this is executed with the following cmd file:
#echo off
start /WAIT project1.exe
echo %ERRORLEVEL%
the program shows its main form for 1 second, closes, and the script prints 42 to the console window.
Now for capturing the output - doing this from a GUI program is actually easier than doing it from a console program, if you allow for the use of a temporary file. You need to start the program with a command line parameter anyway, so why not give it the name of a temporary file, wait for the application to finish, read in the file and delete it afterwards?
If you want an application to return an "error" code there is no need to make it a console application. You only need to set the ExitCode, e.g.
ExitCode := 10;
in a batch file
#Echo off
project1
echo %errorlevel%
Will display the application, then display 10 when.
Note: It is also possible to create a console window dynamically from the windows API using AllocConsole or to attach using AttachConsole.
I created an object wrapper for this once, but no longer have the code available. From memory it didn't support redirection (because I didn't need it).
If I understand you correctly, then you want your app to have two modes:
If no argument is passed, run in GUI mode
Run in non-GUI mode otherwise
The easiest is if you can centralize your logic so it can be called from one method (CoreLogic in my example).
The below app then should work fine.
Two tricks:
Application.ShowMainForm := False; that will not make the MainForm show at all.
ExitCode := 327; which will set your return code (like mghie and Gerry already mentioned).
A few notes:
because the CoreLogic does not process any windows messages, anything in your application that depends on Windows messages being processed will stall.
if you need windows message processing, then just all Application.ProcessMessages() inside your CoreLogic
if you need your form to be visible, then you change the logic inside your MainForm to test for the commandline parameters, and exit when it's work as been done (by calling Application.Terminate()). The best place to put that logic in is the event method for the MainForm.OnShow event.
Hope this helps :-)
program VCLAppThatDoesNotShowMainForm;
uses
Forms,
MainFormUnit in 'MainFormUnit.pas' {MainForm},
Windows;
{$R *.res}
procedure CoreLogic;
begin
Sleep(1000);
ExitCode := 327;
end;
procedure TestParams;
begin
if ParamCount > 0 then
begin
MessageBox(0, CmdLine, PChar(Application.Title), MB_ICONINFORMATION or MB_OK);
CoreLogic();
Application.ShowMainForm := False;
end;
end;
begin
Application.Initialize();
Application.MainFormOnTaskbar := True;
TestParams();
Application.CreateForm(TMainForm, MainForm);
Application.Run();
end.
A timer with 1ms will only fire about every 40 ms (due to Windows limitations), so it won't help. I have seen effects like you describe with mixed console and GUI apps, another is that they don't minimize properly.
Instead of enabling the console in the project, you could probably use the CreateConsole API call (Not sure whether the name is correct) to create one after the programm was started. I have seen no adverse effects in the one (!) program I have done this.
But this is only necessary if you want to write to the console. If you only want to process command line parameters and return an exit code, you do not need a console. Just evaluate the ParamCount/ParamStr functions for the parameters and set ExitCode for the return value.
If some threads in your console application call Synchronize (and I guess the Indy stuff is actually doing that), you have to make some preparations:
Assign a method to the WakeMainThread variable. This method must have the signature of TNotifyEvent.
Inside this method call CheckSynchronize.
For additional information see the Delphi help for these two items.
I get an EInOutError with message 'Too many open files' when executing this code block repeatedly for some time from a number of client threads:
var InputFile : Text;
...
Assign (InputFile, FileName);
Reset (InputFile)
try
// do some stuff
finally
CloseFile (InputFile);
end;
The number of client threads is approximately 10, so only 10 files can be open at any time. Is there any possibility that Delphi refuses to close files right away? Can I ensure that it does? Or am I making a mistake here? This is the only place where I open files and the try..finally block should guarantee that opened files get closed, shouldn't it?
REEDIT: forget the edit
I can only advise you to use the more "modern" facilities for dealing with files. I don't know whether there is a limit of open files using the Windows API, but I just tested and could easily open 1000 streams in parallel:
procedure TForm1.Button1Click(Sender: TObject);
var
Strs: TList;
i: integer;
begin
Strs := TList.Create;
try
for i := 1 to 1000 do begin
Strs.Add(TFileStream.Create('D:\foo.txt', fmOpenRead or fmShareDenyWrite));
end;
finally
FreeObjectList(Strs);
end;
end;
I have never understood why people still use untyped files instead of TStream and its descendants in new code.
Edit: In your comment you write that you only want to read plain text files - if so just create a TStringList and use its LoadFromFile() method.
You aren't running this on an older Windows 9x based computer, are you? If so, you might be running into a DOS filehandle problem.
Delphi closes immidiately in the CloseFile. Your example code seems to be correct.
Try again without anything between try and finally.
There IS a thread safety issue here although I can't see how it could cause the problem.
The problem is Reset uses the global FileMode variable.
As for client threads--are you sure they aren't leaking away on broken connections or something?
Might be useful to put some debug output alongside the Reset and the Close so you can see how long each thread has the file open for.
Do you really need threads? It sounds like they are causing you problems. Your code would be easier to debug without them.
This code should work just fine. There are no known problems related to using files from threaded code (as far as I know). We use such idioms fairly regularly and everything works fine.
I would suggest adding some logging code (before Assign and CloseFile) to see if a) close is executed and b) you really have only 10 threads running. Maybe your thread terminating logic is faulty and CloseFile never executes.