I'm writing a test Delphi 10.4 Win32 application that uses TIdHTTPServer and TIdServerIOHandlerSSLOpenSSL from Indy 10.6.2.0 installed by Delphi 10.4, using a port on localhost for the purposes of communicating between a C# (.NET 5) client application and this Win32 Delphi VCL (server) application.
I know very little about TLS, SSL, and certificates but have been learning as I attempt to build this; from the examples I have seen, I had expected this to be simple.
The "general" page for the .pfx certificate properties says "Enable all purposes for this certificate".
I started partly by reviewing examples and also Remy Lebeau's postings here:
https://stackoverflow.com/a/34422109/17371832
https://stackoverflow.com/a/63082235/17371832
Using Indy 10 IdHTTP with TLS 1.2
I have provided it with OpenSSL 1.0.2.17 DLLs:
libeay32.dll
ssleay32.dll
These DLLs load successfully.
The problem is that clients can't connect to it. .NET says "The SSL connection could not be established" (more about the error inside Indy later).
WireShark shows "Client Hello" (TLS 1.2 with about 20 cipher types) but no "Server Hello", alerts, or errors are sent. In fact, only one packet marked as TLS 1.2 is sent (when the client said "Client Hello").
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:(and many others)
I had tried providing the full list (as formatted above) to the TIdHTTPServer's CipherList property, but got an error (I think it was "SetCipherList failed").
If the cipher list is left empty, it seems to internally use "AES:ALL:!aNULL:!eNULL:+RC4:#STRENGTH". I left it that way.
Here are the components involved:
object srv: TIdHTTPServer
Bindings = <
item
IP = '127.0.0.1'
Port = 5500
end>
DefaultPort = 5500
IOHandler = iohSSL
MaxConnections = 25
AutoStartSession = True
KeepAlive = True
SessionState = True
OnCreatePostStream = srvCreatePostStream
OnDoneWithPostStream = srvDoneWithPostStream
OnCommandGet = srvCommandGet
Left = 183
Top = 118
end
object iohSSL: TIdServerIOHandlerSSLOpenSSL
SSLOptions.CertFile = 'C:\Users\Mike\AppData\Local\Somewhere\cert.pfx'
SSLOptions.KeyFile = 'C:\Users\Mike\AppData\Local\Somewhere\cert.pfx'
SSLOptions.Method = sslvTLSv1_2
SSLOptions.SSLVersions = [sslvTLSv1_2]
SSLOptions.Mode = sslmServer
OnGetPasswordEx = iohSSLGetPasswordEx
Left = 256
Top = 120
end
The code isn't interesting:
procedure TdmCloud.DataModuleCreate(Sender: TObject);
begin
iohSSL.SSLOptions.CertFile:='.\cert.pfx';
//iohSSL.SSLOptions.RootCertFile:='.\root2.pem';
//iohSSL.SSLOptions.KeyFile:='.\key.pem';
srv.Active:=true;
end;
procedure TdmCloud.iohSSLGetPasswordEx(ASender: TObject; var VPassword: string;
const AIsWrite: Boolean);
begin
VPassword := 'password';
end;
procedure TdmCloud.srvCommandGet(AContext: TIdContext;
ARequestInfo: TIdHTTPRequestInfo;
AResponseInfo: TIdHTTPResponseInfo);
begin
AResponseInfo.ResponseNo := HTTP_OK;
AResponseInfo.ContentType := 'text/html';
AResponseInfo.ContentText := '<html><head><title>Title</title></head><body>HELLO</body></html>';
end;
procedure TdmCloud.srvCreatePostStream(AContext: TIdContext;
AHeaders: TIdHeaderList; var VPostStream: TStream);
begin
VPostStream := TMemoryStream.Create;
end;
procedure TdmCloud.srvDoneWithPostStream(AContext: TIdContext;
ARequestInfo: TIdHTTPRequestInfo; var VCanFree: Boolean);
begin
VCanFree := true;
end;
I have seen in the debugger that the certificate file is loaded successfully with the password provided by the event handler. The listener thread runs ok until it receives a request.
TIdHTTPServer's OnCommandGet event doesn't fire.
I've seen in the debugger that InternalReadLn() returns a string that looks encrypted; it's definitely binary. Anyway, below, I get "error in parsing command"; judging from the data it's looking at, I'm not surprised.
var
i: integer;
s, LInputLine, LRawHTTPCommand, LCmd, LContentType, LAuthType: String;
LURI: TIdURI;
LContinueProcessing, LCloseConnection: Boolean;
LConn: TIdTCPConnection;
LEncoding: IIdTextEncoding;
begin
LContinueProcessing := True;
Result := False;
LCloseConnection := not KeepAlive;
try
try
LConn := AContext.Connection;
repeat
// InternalReadLn( ) returns a string that looks encrypted; it's definitely binary
LInputLine := InternalReadLn(LConn.IOHandler);
i := RPos(' ', LInputLine, -1); {Do not Localize}
if i = 0 then begin // true
raise EIdHTTPErrorParsingCommand.Create(RSHTTPErrorParsingCommand);
end;
As it happens, InternalReadLn() calls TIdServerIOHandlerSSLOpenSSL's Readln() method, which has provided this binary data.
Remy Lebeau said this about the error in an online posting:
The error is actually coming from several lines further down:
i := RPos(' ', LInputLine, -1); {Do not Localize}
if i = 0 then begin
raise EIdHTTPErrorParsingCommand.Create(RSHTTPErrorParsingCommand);
end;
Which means your client is sending a malformed HTTP request. The first
line in EVERY request must end with an HTTP version number, ie: GET /login.html HTTP/1.0 Hence the call to RPos(). The error means there
is no space character anywhere in the line.
Although it could certainly be a malformed request, I'm told that the application's requests have been tested and found to be valid/working.
I've tried a browser directed to https://localhost:5500/ with similar results.
That brings me back to the conclusion that TLS negotiation didn't start at "Client hello". I have no idea why. I don't know where that process is handled in Indy.
Here's a typical, working TLS negotiation: https://tls.ulfheim.net/
Key areas of suspicion for me; I've made some effort to explore each possibility:
might the .pfx certificate missing something important (even though I'm told it's a good certificate)? It has the "Server Authentication" option checked in the properties.
which of CertFile, RootCertFile, KeyFile, and CipherList do I need to provide for TLS 1.2 to work? (and how do I choose what to use?)
is Delphi 10.4 update 2's Indy SSL broken or incompatible with OpenSSL 1.0.2q?
Unhelpfully, all of documentation links at https://www.indyproject.org/documentation/ are broken! My Delphi install didn't include any Indy help either.
Help being absent, I'm uncertain which to provide of RootCertFile, CertFile, and KeyFile. I've tried the .pfx which was accepted for CertFile and KeyFile but later saw https://stackoverflow.com/a/15416234/17371832 so I followed the instructions and was able to make a .pem file (root file was not accepted by Indy). This provided no improvement.
Is Indy dying? Is there something else that's better? Haven't looked into that at all.
Why won't it connect?
The standard TCP port for HTTPS is 443, but you are running your TIdHTTPServer on port 5500 instead. As such, you must use the TIdHTTPServer.OnQuerySSLPort event to explicitly tell TIdHTTPServer that you want to use SSL/TLS on that port, eg:
procedure TdmCloud.srvQuerySSLPort(APort: TIdPort; var VUseSSL: Boolean);
begin
VUseSSL := APort = 5500;
end;
This event allows HTTP and HTTPS to co-exist on a single server (via multiple TIdHTTPServer.Bindings items) even when using non-standard ports. Without this event handler assigned, this is why your server is not handling the client's SSL/TLS handshake correctly.
Related
How i can set ConnectTimeout/ReadTimeout in Indy when using SSL ?
MCVE:
program mcve;
uses
{$IFDEF UNIX}{$IFDEF UseCThreads}
cthreads,
{$ENDIF}{$ENDIF}SysUtils, IdHTTP, IdSSLOpenSSL, DateUtils;
var
HTTP : TIdHTTP;
SSL : TIdSSLIOHandlerSocketOpenSSL;
Started : TDateTime;
begin
HTTP := TIdHTTP.Create();
try
HTTP.ReadTimeout := 1000;
HTTP.ConnectTimeout := 2000;
SSL := TIdSSLIOHandlerSocketOpenSSL.Create(HTTP);
SSL.ConnectTimeout := HTTP.ConnectTimeout;
SSL.ReadTimeout := HTTP.ReadTimeout;
SSL.SSLOptions.SSLVersions := [sslvTLSv1, sslvTLSv1_1, sslvTLSv1_2];
HTTP.IOHandler := SSL;
Started := Now;
try
HTTP.Get(ParamStr(1));
except
On E: Exception do WriteLn(E.Message);
end;
Writeln(FormatDateTime('hh:nn:ss', SecondsBetween(Started, Now) / SecsPerDay));
finally
HTTP.Free;
end;
end.
When using http ConnectTimeout/ReadTimeout work fine the issue only when using https see below:
:~$ ./mcve http://x.x.x.x
Read timed out.
00:00:01 <-- Correct.
:~$ ./mcve https://x.x.x.x
Socket Error # 0
00:03:38 <-- NOT Correct / More than SSL.ReadTimeout value.
Lazarus 2.0.6 Indy installed from OPM version 10.6.2.5494.
Note: On Windows same code using Delphi with shipped Indy 10.6.2.5366, The results works as expected
You don't need to manually set the ConnectTimeout and ReadTimeout on the IOHandler itself, only on the client component (in this case, TIdHTTP) . TIdTCPClient.Connect() will assign the values to the IOHandler for you.
The ConnectTimeout applies when the underlying socket is being connected to the server, before any SSL/TLS session is created, so it operates the same whether you use SSL/TLS or not.
The ReadTimeout applies when Indy attempts to read bytes from the IOHandler's internal connection. When not using SSL/TLS, that means it goes straight to the socket, and thus times out when no bytes arrive on the socket. But when using SSL/TLS, Indy uses OpenSSL's legacy SSL_...() APIs, not its newer BIO_...() APIs, which means OpenSSL is doing its own socket reading and buffering on Indy's behalf, and thus Indy times out when OpenSSL does not provide any decrypted application bytes.
One difference in how TIdSSLIOHandlerSocketOpenSSL operates on Windows vs other platforms is that on Windows Vista+ only, TIdSSLIOHandlerSocketOpenSSL does apply the ReadTimeout to the underlying socket's SO_RCVTIMEO and SO_SNDTIMEO timeouts via the IOHandler's Binding.SetSockOpt() method, as a workaround to an OpenSSL bug on Windows. For other platforms, Indy does not currently set those two socket timeouts.
A good place to set those timeouts manually would be in the IOHandler's OnBeforeConnect event, which is fired after the socket is connected to the server and before any SSL/TLS session is created.
The following code works as shown, but not with the commented out URL.
Can you see my error?
var
IdHTTP1: TIdHTTP;
sl: TStringList;
Src : string;
LHandler: TIdSSLIOHandlerSocketOpenSSL;
begin
try
IdHTTP1 := TIdHTTP.Create(nil);
try
LHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
try
IdHTTP1.IOHandler := LHandler;
Src := IdHTTP1.Get(TIdURI.URLEncode(
//'https://geocoding.geo.census.gov/geocoder/locations/onelineaddress?address=4600+Silver+Hill+Rd%2C+Suitland%2C+MD+20746&benchmark=9&format=json'
'https://tools.usps.com/go/ZipLookupResultsAction!input.action?resultMode=0&companyName=&address1=1600+PENNSYLVANIA+AVE+NW&address2=&city=&state=Select&urbanCode=&postalCode=&zip=20500'
));
ShowMessage(Copy(src, 1, 100));
finally
LHandler.Free;
end;
finally
IdHTTP1.Free;
end;
except
on E: Exception do
ShowMessage('e');
end;
ShowMessage('done');
end;
The commented out URL raises an exception:
EIdOSSLUnderlyingCryptoError: Error connecting with SSL. Error connecting with SSL. error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
The two URLs you have shown are already url-encoded, so you should not be passing them to TIdURI.URLEncode() at all, just pass them as-is to TIdHTTP.Get(). That being said, make sure you are using an up-to-date version of Indy so that you have this bug fix to avoid double-encoding any sequences that are already percent-encoded.
In any case, the problem you are having is not related to the URL encoding. The cause of the SSL alert error is that TIdSSLIOHandlerSocketOpenSSL enables only TLS 1.0 by default, but many modern web servers are slowly moving away from TLS 1.0 and now require TLS 1.1+. geocoding.geo.census.gov won't work with anything less than TLS 1.2, whereas tools.usps.com still allows TLS 1.0. Modern web browsers support TLS 1.0+.
So the fix is to enable TLS 1.1 and TLS 1.2 in the TIdSSLIOHandlerSocketOpenSSL.SSLOptions.SSLVersions property 1.
1: I have opened a ticket in Indy's issue tracker to enable TLS 1.1+ by default in a future release.
How i can limit the connections per ip? i've tried this code how to block unknown clients in indy (Delphi) but if my server is flooded i can't connect. the code from Remy just prevent CPU to use 100% and use more ram. but the connection from flood still alive on tcpserver and i can't connect to server. so my question is, how i can limit the connection before onconnect, something on accept using tcpserver? maybe hook accept function and try limit the connection ?
How i can limit the connections per ip?
You already know the answer to that, as that is exactly what the other code is doing.
if my server is flooded i can't connect.
The purpose of the other code is simply to limit a client IP address to a max of 10 simultaneous connections, not to prevent flooding or lower CPU/RAM usage. You can't stop unwanted clients from connecting to your server, unless you deactivate the server, or set its MaxConnections property. Outside of that, about all you can do is disconnect unwanted clients as soon as possible, which you can do in the server's OnConnect event. But if you are getting flooded, that is going to take time to process, especially if you are continuously locking and unlocking the server's Contexts list, which will end up serializing the server's internal threading.
Flood management really needs to be handled by a firewall or router/load balancer, not in the server app itself. If this is not acceptable to you, then at least on Windows only, an option might be to write a custom TIdServerIOHandlerStack-derived component that overrides the virtual Accept() method to call WinSock's WSAAccept() function, which offers a callback you can use to reject connections before they leave the accept queue, and thus they will not be seen by TIdTCPServer. For example:
type
TMyServerIOHandler = class(TIdServerIOHandlerStack)
public
function Accept(ASocket: TIdSocketHandle; AListenerThread: TIdThread; AYarn: TIdYarn): TIdIOHandler; override;
end;
function MyConditionFunc(lpCallerId, lpCallerData: LPWSABUF; lpSQOS, lpGQOS: LPQOS; lpCalleeId, lpCalleeData: LPWSABUF; g: PGROUP dwCallbackData: DWORD_PTR): Integer; stdcall;
begin
if (the address stored in lpCallerId is blocked) then
Result := CF_REJECT
else
Result := CF_ACCEPT;
end;
type
TIdSocketHandleAccess = class(TIdSocketHandle)
end;
function TMyServerIOHandler.Accept(ASocket: TIdSocketHandle; AListenerThread: TIdThread; AYarn: TIdYarn): TIdIOHandler;
var
LIOHandler: TIdIOHandlerSocket;
LBinding: TIdSocketHandle;
LAcceptedSocket: TIdStackSocketHandle;
begin
Result := nil;
LIOHandler := TIdIOHandlerStack.Create(nil);
try
LIOHandler.Open;
while not AListenerThread.Stopped do
begin
if ASocket.Select(250) then
begin
LBinding := LIOHandler.Binding;
LBinding.Reset;
LAcceptedSocket := WSAAccept(ASocket.Handle, nil, nil, #MyConditionFunc, 0);
if LAcceptedSocket <> Id_INVALID_SOCKET then
begin
TIdSocketHandleAccess(LBinding).SetHandle(LAcceptedSocket);
LBinding.UpdateBindingLocal;
LBinding.UpdateBindingPeer;
LIOHandler.AfterAccept;
Result := LIOHandler;
LIOHandler := nil;
Break;
end;
end;
end;
finally
FreeAndNil(LIOHandler);
end;
end;
Then you can assign an instance of TMyServerIOHandler to the TIdTCPServer.IOHandler property before activating the server.
I thought I would try something simple to try to understand Web client/server applications since I know nothing about it but I need to learn. I downloaded the "Zip Code look up" example from the web (both client and server). Compiled them and everything worked as advertised.
Then I dropped an SSL IO Handler on each form and specified them as the handlers with "sslvTLSv1" selected in both. Now it doesn't work. Every time I try to look up zip codes in the client app, I get, "error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number." I uninstalled the original Indy library and installed the latest version (10.6.2.5264). Just to be sure, I got the most recent SSL libraries and put them in both program folders.
I have looked at numerous questions and answers about SSL in Indy and it seems like it should work. What am I missing?
This is the relevant section from the Server DFM:
object IdTCPServer1: TIdTCPServer
Active = True
Bindings = <
item
IP = '127.0.0.1'
Port = 6000
end>
DefaultPort = 6000
IOHandler = Handler
OnConnect = IdTCPServer1Connect
OnExecute = IdTCPServer1Execute
Left = 32
Top = 24
end
object Handler: TIdServerIOHandlerSSLOpenSSL
SSLOptions.Mode = sslmServer
SSLOptions.VerifyMode = []
SSLOptions.VerifyDepth = 0
Left = 100
Top = 32
end
And this is the relevant section of the Client DFM.
object Client: TIdTCPClient
IOHandler = Handler
ConnectTimeout = 0
Host = '127.0.0.1'
IPVersion = Id_IPv4
Port = 6000
ReadTimeout = -1
Left = 209
Top = 16
end
object Handler: TIdSSLIOHandlerSocketOpenSSL
Destination = '127.0.0.1:6000'
Host = '127.0.0.1'
MaxLineAction = maException
Port = 6000
DefaultPort = 0
SSLOptions.Mode = sslmClient
SSLOptions.VerifyMode = []
SSLOptions.VerifyDepth = 0
Left = 296
Top = 32
end
After trying Remy's suggestion
I added code to set the PassThrough to false and now I get a different error.
"error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure."
For those who want to look at the source code, it is on the web at http://www.atozedsoftware.com/indy/demos/10/index.EN.aspx (I'm using the OnExecute version of the server). The only difference now being the line in each unit setting the PassThrough mode.
In the sever code:
procedure TformMain.IdTCPServer1Connect(AContext: TIdContext);
begin
TIdSSLIOHandlerSocketBase(AContext.Connection.IOHandler).PassThrough := False;
AContext.Connection.IOHandler.WriteLn('200 Zip Code Server Ready.');
AContext.Data := TUserData.Create;
end;
In the client code:
procedure TformMain.butnLookupClick(Sender: TObject);
var
i: integer;
begin
butnLookup.Enabled := False; try
lboxResults.Clear;
Handler.PassThrough := False;
You did not show any of your code, only your DFMs. That is not enough. But I suspect what is likely happening is that you are setting the IOHandler.PassThrough property to false on the client side, but are not doing the same on the server side.
When a client connects to TIdTCPServer, SSL/TLS is not activated by the server by default so the server can analyze the client, maybe even communicate unsecurely with it, before deciding whether SSL/TLS is needed. This is to facilitate the server listening on multiple ports where not all ports use SSL/TLS at all (think HTTP vs HTTPS), and to handle STARTTLS-based protocols.
So, if you are not already doing so, make sure to set the IOHandler.PassThrough to false on both ends of the connection, eg:
Handler.PassThrough := False;
IdTCPClient1.Connect;
procedure TForm1.IdTCPServer1Connect(AContext: TIdContext);
begin
TIdSSLIOHandlerSocketBase(AContext.Connection.IOHandler).PassThrough := False;
end;
If you are indeed doing so on both ends, but are still getting the same error, than make sure the IOHandler.SSLOptions.SSLMethod/IOHandler.SSLOptions.TLSVersions property really is configured correctly on both ends and they are using compatible settings. You say you are using TLSv1 on both ends, so you should not be getting the error unless the server is really not using TLSv1.
So, in the end I had to do a little bit more than drop a couple of controls on the form.
First I had to add code to set the PassThrough property for each of the SSL controls to false (maybe they should be published properties?).
Then I had to install a valid certificate and set the CertFile, KeyFile and RootCertFile properties on the server side.
Now it works!
Thanks to Remy for his help and, BTW this was using OpenSSL v1.0.2 and Indy 10.5.5. in Win32 (Vista) with Delphi 2010.
I have a small client-server application project using Indy 9. My server application is using 'command handler' to handle client's command.
In this case my client application using writeLn procedure to send command and data, which is text based. For example:
IdTcpClient1.WriteLn('cmd1'+'#'+'hello server');
note: 'cmd1' is a command, '#' is command delimiter, and 'hello server' is the data
In order to handle this command (cmd1), my server application has a procedure as follows:
procedure TForm1.IdTcpServer1cmd1Command(Asender:TIdCommand);
var
s: string;
begin
if ( Assigned(ASender.Params) ) then
begin
s:= Asender.Param[0];
...............
...............
end;
end;
So far everything is fine. The problem is that I want to add a feature so that my server application is able to request and receive a JPEG_image from client. If client send this image using: WriteStream procedure, for example:
IdTcpClient1.WriteStream(MyImageStream);
How then the the server handle this event considering that there is no specific command to it (such as 'cmd1' in this example)?
You would simply call ReadStream() in the command handler, eg:
IdTcpClient1.WriteLn('JPEG');
IdTcpClient1.WriteStream(MyImageStream, True, True); // AWriteByteCount=True
procedure TForm1.IdTcpServer1JPEGCommand(ASender: TIdCommand);
var
Strm: TMemoryStream;
begin
...
Strm := TMemoryStream.Create;
try
ASender.Thread.Connection.ReadStream(Strm); // expects AWriteByteCount=True by default
...
finally
Strm.Free;
end;
...
end;
Alternatively:
IdTcpClient1.WriteLn('JPEG#' + IntToStr(MyImageStream.Size));
IdTcpClient1.WriteStream(MyImageStream); // AWriteByteCount=False
procedure TForm1.IdTcpServer1JPEGCommand(ASender: TIdCommand);
var
Size: Integer;
Strm: TMemoryStream;
begin
...
Size := StrToInt(ASender.Params[0]);
Strm := TMemoryStream.Create;
try
if Size > 0 then
ASender.Thread.Connection.ReadStream(Strm, Size, False); // AWriteByteCount=True not expected
...
finally
Strm.Free;
end;
...
end;
How then the the server handle this event
So what is you actually want to learn ? What is your actual question ?
If you want to know how your server would behave - then just run your program and see what happens.
If you want to know how to design your server and client, so client could upload a picture to server - then ask just that.
I think that the proper thing to do would be adding the command "picture-upload" with two parameters: an integer token and a jpeg stream.
When the server would reply to cmd1 command, if it would need, it would generate and attach a unique integer token, asking the client to upload the screenshot.
When client would receive this reply, it would parse it, and if the token would be found - would issue one more command, a specially designed "picture-upload#12345" (where 12345 replaced with actual token value) followed by the picture itself. The server would then use the token value to know which client and why did the upload.
PS. though personally I think you'd better just use standardized HTTP REST rather than making your own incompatible protocol.