Is there a way to prevent Indy from raising exceptions on any/all HTTP statuses?
My understanding of the IgnoreReplies array is that it'll prevent those statuses from ever coming back, but that's not what I want. I want ALL statuses coming back, and none of them to raise an exception. There are plenty of REST services that return 404 for example, and that's considered a perfectly "valid" response.
I really don't want 1/2 of my code in exception handlers, so is there way to get the current Indy to just return everything?
Sorry, but you have to use the IgnoreReplies parameter to tell TIdHTTP which specific HTTP reply codes to not raise an exception for. There is currently no way to tell TIdHTTP to not raise an exception for all possible replies generically. If you want to receive a non-success reply, like 404, without raising an exception then you have to tell TIdHTTP that, eg:
IdHTTP.Get('http://host/path', [404]);
Note that this syntax is only available for GET requests, not other requests, like POST (unless you call the protected TIdHTTP.DoRequest() method directly).
Indy is specifically designed around exception handling. You have to embrace exceptions, not avoid them, if you want to use Indy effectively. What is so troublesome about wrapping TIdHTTP.Get() or TIdHTTP.Post() in a small try/except block? If you don't want to handle all possible exceptions, then you can have the except block handle EIdHTTPProtocolException by itself.
try
IdHTTP.Get('http://host/path');
except
on E: EIdHTTPProtocolException do
begin
if E.ErrorCode <> 404 then Raise;
end;
end;
Update: thinking about it more, it might be worth adding a new flag to the TIdHTTP.HTTPOptions property to disable the exception for all status codes. I'm considering it.
Update: a new hoNoProtocolErrorException flag has now been added to the TIdHTTP.HTTPOptions property:
IdHTTP.HTTPOptions := IdHTTP.HTTPOptions + [hoNoProtocolErrorException];
IdHTTP.Get('http://host/path');
// use IdHTTP.ResponseCode as needed...
Update: in addition, there is also a hoWantProtocolErrorContent flag as well. When hoNoProtocolErrorException is enabled and an HTTP error occurs, if hoWantProtocolErrorContent is enabled then the error content will be returned to the caller (either in a String return value or in an AResponseContent output stream, depending on which version of Get()/Post() is being called), otherwise the content will be discarded.
Related
I've got a Delphi 6 client-server application where the server is essentially:
initialize;
repeat
try
getInput;
putOutput;
except
on e: exception do writeln(e.message);
end
until false;
During development, bugs (such as a pointer exception) were rare but the server would be able to keep running. Now that it's in production the server seems debugged and it's tempting to remove the try...except because it adds substantial execution overhead to the server, but then if an exception were to occur it would stop the server.
One solution would be to have a parent process simply restart the entire server, and bypassing initialization on restart would effectively reproduce what try...except accomplished, but there wouldn't be any record of what caused the exception, and restarting with or without initialization actually creates a lot of unwanted "state" issues, especially with client connections.
Is there some way to override or amend exception handling so that when the exception happens it's recorded somewhere (such as in the system event log or %errorlevel%) before the programs quits and is restarted or (better:) recovers, but without the overhead of an all-encompassing try...except?
The ideal solution would record the exception but resume execution at the level of the repeat...until. Perhaps by doing something like saving the address of the repeat in some accessible global, then make any exception unwind the call stack and resume at that saved address? (Which I guess is what try..except essentially does, but is there a way to avoid the overhead it imposes when there aren't exceptions?)
Expanding on Serg's suggestion:
repeat
try
repeat
getInput;
putOutput;
until false;
except
on e: exception do writeln(e.message);
end
until false;
Has the same behaviour but moves the try .. except outside the innermost loop.
However, I'd be amazed if this had any discernible impact on performance.
Expanding on David and Sergs suggestions...
If the complexity of your loop is as low as your posted code suggests (or could be reduced to such a level of simplicity) then it may be practical to provide both behaviours, subject to a command line switch to either enable or disable runtime errors via exception handling.
Parse the command line at startup to determine the mode of operation and set an indicator variable accordingly. e.g using an enum:
type
TServerMode = (smHighThroughput, smDiagnostic);
var
ServerMode: TServerMode;
if FindCmdLineSwitch('d') then
ServerMode := smDiagnostic
else
ServerMode := smHighThroughput;
You could of course use a setting in a configuration file or database table if a command line parameter is not desirable or practical for whatever reason.
However you go about it, you provide the ability to run the server in "high throughput" mode or "recoverable diagnostic" mode with a minimum of duplicated syntax:
try
case ServerMode of
smDiagnostic : repeat
try
getInput;
putOutput;
except
on e: Exception do .... // etc
end;
until FALSE;
smHighThroughput : repeat
getInput;
putOutput;
until FALSE;
end;
except
on e: Exception do .... // etc
end;
Note that even in HighThroughput mode the above example captures information about any exception that causes server termination. The Diagnostic mode logs all exceptions but attempts to keep the server running.
You can then default to HighThroughput mode with the option of enabling Diagnostic mode in the event that you need to diagnose a server that seems unusually unstable.
Having said all that, I would try to definitively quantify the impact of the try..except on the loop before going to great lengths to engineer around that impact.
Drop an ApplicationEvents on a central place in your application.
Implement the OnException Event.
Delete the local try...except to remove overhead.
In the exception event you can do anything including log the exception.
If you want to add something special to the exception it is possible to still have a local try..except and modify the E.Message and then call raise. Then the global exception handler will be called with the modified message.
I have situation when I know how much bytes I expect to get from client (in this code it is 100). If client does't gives me 100 I don't need this packet at all, because it is corrupted.
I have situation when client gives less than 100 bytes and disconnects. In this case I'm catching exception and everything would be fine except that after this exception my ServerTCPExecute is called for many times uncontinuously. There are no any other clients connected. Why it is so? If I do TIdYarnOfThread(AContext.Yarn).Thread.Terminate; everything goes fine. But I'm not sure it is good solution.
procedure Form1.ServerTCPExecute(AContext: TIdContext);
begin
try
AContext.Connection.IOHandler.ReadBytes(b, 100, False);
except
//TIdYarnOfThread(AContext.Yarn).Thread.Terminate;
end;
end;
Solution: do not use try ... except to catch and ignore all exceptions within OnExecute.
This will swallow the Indy exception which happens when the client disconnects, the server will execute the next iteration of OnExecute if the IOHandler.InputBuffer still has data in it, and ReadBytes raises another exception because the peer disconnected.
If any exception is raised and leaves the method, the Indy TCP server will close and clean up the connection.
Indy uses exceptions for error handling and notifications in OnExecute, so do not suppress them with an "empty" exception handler. If you need to catch exceptions, re-raise any EIdException-derived exceptions you catch and let the server handle them.
p.s.: the posted code seems to use a non-local variable b. Because TIdTCPServer is a multi-threaded component, non-local variables must always be accessed in a thread-safe way, for example using a critical section, or putting the variable inside of the TIdContext object.
I am using the Internet Direct TIdTCPClient component to communicate with a remote service to retrieve a message that is normally about 5k is size. During a typical operation I will send about 400 requests to the service, each one taking about 1 second to complete. Most of the time everything works perfectly. However, about one percent of the time the request takes 189 seconds and I receive no data at all. For ease of discussion, I'll call this a failure.
I am particularly interested in understanding exactly what is happening when the failure occurs so that I can take my evidence to the publisher of the service. First of all, the failure is not reproducible. If I re-send a failed request there is a very high probability (maybe 99 percent) that it will work.
I also capture the request that I send when a failure occurs, so I am able to confirm that the request is well formed.
I am assuming that during a failure that I am getting some data, just not all of it. Here is why. My IdTCPClient has a 30 second timeout (I've even set it to 5 seconds, but that didn't make a difference). When the failure occurs, it always fails after 189 seconds (plus about 500 milliseconds).
As a result, I am thinking that during a failure my component is receiving a trickle of data, which is why my client is not timing out. And, I am assuming that the disconnection is happening at the service since none of my timeout values are ever set to 189 seconds. On the other hand, reading IOHandler.AllData does not raise an exception (not even an EIdConnClosedGracefully exception). Am I interpreting this evidence correctly?
What I want to do is to confirm that I am getting some data, just not all of it, before the service terminates the connection. Furthermore, I want to know what that partial data looks like as I believe it can help identify the source of the failure.
Currently, my request is similar to the following:
//ExceptionName is a temporary global variable
//that I am using while trying to solve this issue
ExceptionName = 'no exception';
try
s := GetRequest(id);
IdTcpClient1.Host := Host;
IdTcpClient1.Port := StrToInt(Port);
IdTcpClient1.ReadTimeout := ReadTimeout;
try
IdTcpClient1.Connect;
except
on e: exception do
begin
ExceptionName := e.ClassName;
raise EConnectionFailure.Create('Connection refused: ' + e.Message)
end;
end;
IdTcpClient1.IOHandler.Writeln(s);
try
Result := IdTcpClient1.IOHandler.AllData;
except
on E: EIdConnClosedGracefully do
begin
ExceptionName := e.ClassName;
//eat this exception
end;
on e: Exception do
begin
ExceptionName := e.ClassName;
raise;
end;
end;
finally
if IdTcpClient1.Connected then
IdTcpClient1.Disconnect;
end;
Using IOHandler.AllData to read the data is very convenient, but I cannot retrieve any data following a failure (AllData returns an empty string). I've tested IOHandler.InputBufferIsEmpty after a failure, and it returns True.
I've also tried other methods to read the data, such as IOHandler.ReadStream (this produced the same result as reading AllData). I also used IOHandler.ReadBytes and IOHandler.ReadByte (in conjunction with IOHandler.CheckForDataOnSource). Nothing has worked.
Am I wrong about partial data transmission? If so, why would I see a consistent 189.nnnn seconds delay before the failure.
If partial data transmission is a possibility, what approach should I take to capture every byte of data received before the failure.
I am using Delphi 2009 for this project and Indy 10, but I don't think the version has anything to do with it. I don't think this is an Indy issue.
Edit: I have inspected the communication between my Indy client and the server using WireShark. When one of these failures occur, after sending my request the server sent two [ACK] packets followed by silence for just over 189 seconds. After that delay, the response included [FIN, PSH, ACK] but no application data.
When the communication worked normally, the two ACK packets returned by the server in response to my request was followed by an application data packet.
Edit: Have reported the issue to the publisher of the Web service, and am waiting for a reply.
Edit: Ok, the publisher of the Web service has responded. They acknowledged problems on their end and have addressed some of those. We are no longer getting timeouts. Most responses are received in about 2 seconds, with a few taking slightly longer. The publisher is working to fix the remaining issues.
Thank you everyone for your input.
You need to run Fiddler2 and watch the traffic. It inserts itself as a proxy and sniffs evertying that uses the WinInet stack.
Then you know whether you've gotten any data or not, and exactly what you're sending and receiving.
http://www.fiddler2.com/fiddler2/
Solution found, see my comment below
D5, odbc to mysql database
This code:
with QryCmdPerf do begin
Close;
ParamByName('ACCTID').AsInteger:= AcctId;
ParamByName('FROMDT').AsString:= MySQLDate(FromDt);
ParamByName('TODT').AsString:= MySQLDate(ToDt);
Open;
first;
try
edit;
FieldByName('PnL').AsFloat:= 97979;
ApplyUpdates;
except
close;
end;
end; // with
(specifically the "ApplyUpdates") causes a popup to appear with the text "Update Failed" if the PnL field already has the value 97979, evidently because of this code:
procedure TUpdateSQL.ExecSQL(UpdateKind: TUpdateKind);
begin
with Query[UpdateKind] do
begin
Prepare;
ExecSQL;
if RowsAffected <> 1 then DatabaseError(SUpdateFailed);
end;
end;
in DBTables.pas. Anyway, I want to be able to issue ApplyUpdates, and not have to worry about a popup if it doesn't do any updating. But if "try...except" doesn't work, what will?
TIA
You're confusing the dialog displayed by the debugger with a dialog displayed by your program. Please see this article I wrote a few years ago:
Why do I continue getting error messages even after I have written an exception handler?
It describes several ways to avoid the debugger interfering:
Use "advanced breakpoints" to temporarily disable the debugger around the code that throws exceptions.
Configure the debugger to ignore certain exception types. (Read the debugger's message more carefully to see exactly what exception class you're dealing with.)
Configure the debugger not to interrupt on any exceptions.
Turn off integrated debugger entirely.
The short answer is, you have to set up an eventhandler for OnUpdateError or no amount of "try...except" blocks will block the popup. The long answer is it appears to be a bug with odbc. The repro is here: http://www.codeupload.com/3919 for anyone who wants to take a look at it. You can skip the MySQL stuff, any odbc database will do.
There are two things that can be going wrong here.
Option 1
First, some "very bad code" may be short-circuiting the unwinding of the call stack on exceptions. E.g. ApplyUpdates or one of its child routines may also have a try...except block that calls Application.HandleException directly.
To test this, if you put a breakpoint on QryCmdPerf.Close, do you reach it?
If not, then Application.HandleException (or worse Application.ShowException) has been called directly.
Solving this requires a custom exception handler hooked to the Application.OnException event. You may have to set temporary state to know when this particular exception can be ignored.
Yes it's messy, that why calling Application.HandleException directly, is "very bad code".
Option 2
If you do reach the breakpoint, but the exception is being raised again, then it should be a lot simpler to solve.
The Close method is probably attempting to save any pending changes, so is effectively applying the updates again. Rather than simply closing the data set, call CancelChanges or equivalent.
I'm writing a program to perform checks on websites to test they're current status, e.g. 200 OK. I'm using Indy HTTP shipped with Delphi 2010 for this task and have the current code.
procedure TCheckSiteIndividual.Execute;
var
http : TIdhttp;
begin
try
try
http := Tidhttp.create(nil); //Create indy
http.Get(CSiteURL,nil); //Send Request To Check Site
except
on E: EIdHTTPProtocolException do
status := http.ResponseText; // or: code := E.ErrorCode;
end;
status := http.ResponseText; //Return Status of Site
synchronize(updateform); //Update data to form
finally
http.Free;
end;
end;
The CSiteURL variable is set in the Thread Constructor, and the status variable is a variable to the entire thread.
The try, except parts of this code came from Remy Lebeau - TeamB here which was a question I asked when initially fiddling around with this idea. The problem I'm having is that this code only works if the site returns a 200 OK code. Anything else causes an exception, including if I disable the internet. This also causes exceptions on sites that redirect e.g. www.google.com causes an exception during debugging I presume causes it's redirecting to www.google.co.uk for me, however if I continue past the exception then it returns a 301 FOUND code with no problems.
The final result I'm looking for it for it to give me a return based on whether the site is online or having error (e.g. HTML RETURN CODE) additionally giving feedback when the site doesn't exist (e.g. a non-registered URL) or just can't be contacted.
The other peices of advice I'm looking for are into how to get some more information back from the request. Specifically at the moment I'm looking to also get the IP of the site that was checked.
It would also be nice to have it work without needing to have http:// before any URL, so for example www.google.com will work which at the moment it doesn't because of unknown protocol.
Basically it would be great if someone had a link to some nice documentation for IndyHTTP because it seems to be sparse,
Yes, of course it causes an exception. That's how Indy works, and that's why Remy advised you to catch exceptions.
What you're confused by now is that even though you have code to catch exceptions, the Delphi debugger still intercepts them and pauses execution of your program to notify you that an exception occurred. I've written about that phenomenon before.
Since you're apparently not interested in knowing about non-status-200 exceptions while you're debugging — because your program is designed to handle all those errors itself — your best option is probably to add EIdHTTPProtocolException to the debugger's list of ignored exceptions. The easiest way to do that is to choose "Ignore this exception type" the next time the debugger notifies you.