I've written a small application using delphi to update my twitter status.
I use Indy 10 and OpenSSL and everything works fine, which means that i can both authenticate my app and update my status.
The thing is that if i use a program like "http analyzer" i can see the request's headers and so i can see sensitive information like the consumer_key.
Is that normal or is it a sign that i have not set properly the iohandler (TIdSSLIOHandlerSocketOpenSSL)?
mSslIoHandler.SSLOptions.Method := sslvSSLv3;
mSslIoHandler.SSLOptions.Mode := sslmBoth;
mSslIoHandler.SSLOptions.VerifyMode := [];
mSslIoHandler.SSLOptions.VerifyDepth := 0;
By providing their own SSL certificates, HTTP analyzers are able to monitor the HTTP traffic as if it was unencrypted. I guess you have to set the analyzer IP address and port as proxy only, and leave the destination server address and port unchanged in the client. Then the analyzer will be able to decrypt your client data with its own key, and forwards it to the destination server. (This is the same way a 'man in the middle'-attack would work)
So yes, it is normall with this type of HTTP analyzers (such as Fiddler)
Related
Previously the URL I needed to check was http and I used TidTCPClient to emulate the http protocol to determine the file at the web server existed. Now they have switched to https and I don't know how to do it.
Can I use TIdHTTP with the TIdSSLIOHandlerSocketOpenSSL hander, and then some Indy function to only determine the resource exists? I have OpenSSL installed.
Your existing TIdTCPClient code should continue to work if you simply assign a TIdSSLIOHandlerSocketOpenSSL to the client's IOHandler, change the client's Port to 443, and set the IOHandler's PassThrough to False. HTTPS is just HTTP over an SSL/TLS connection, it doesn't change HTTP itself.
But to answer your question, yes, you can switch to TIdHTTP if you want to. You can use the HTTP HEAD command (the TIdHTTP.Head() method) to check if a URL is valid without having to download a full resource.
Our Delphi application is trying to connect to our website via IdFTP on a client machine using a proxy server, and I always get a 'read timed out' message. I don't know how to fix it.
My code:
IdFTP1.Host :=Website_address;
IdFTP1.Username :=Website_user;
IdFTP1.Password :=Website_password;
IdFTP1.TransferType:=ftBinary;
IdFTP1.ProxySettings.ProxyType:=fpcmNone;
IdFTP1.ProxySettings.Host :=Proxy_server;
IdFTP1.ProxySettings.Port :=Proxy_port;
IdFTP1.ProxySettings.Username :=Proxy_username;
IdFTP1.ProxySettings.Password :=Proxy_password;
IdFTP1.Connect;
...which returns a 'read timed out' exception.
Having looked on the web for possible solutions, I have tried various combinations of the following with no joy (although there might be a combination that might work, I just don't really know what I'm doing):
IdFTP1.IOHandler :=TIdSSLIOHandlerSocketOpenSSL.Create
(IdFTP1);
IdFTP1.UseTLS :=utUseExplicitTLS;
IdFTP1.NATKeepAlive.UseKeepAlive:=True;
IdFTP1.NATKeepAlive.IdleTimeMS :=100000;
IdFTP1.DataPortProtection :=ftpdpsPrivate;
The frustrating thing is I can't test it on my machine, I have to compile a new version of the application, copy it to their machine, and then see if it works.
There is nothing complex about the installation on the client side as far as I can tell. They have given me the proxy host address, and the proxy username and password are blank.
Other bits of the application connect to the same website via HTTP and the proxy server, and this works perfectly. So my logic is that it can't be firewalls or anything like that.
IdHTTP1.HandleRedirects :=True;
IdHTTP1.ProxyParams.BasicAuthentication:=True;
IdHTTP1.ProxyParams.ProxyServer :=Proxy_server;
IdHTTP1.ProxyParams.ProxyPort :=Proxy_port;
IdHTTP1.ProxyParams.ProxyUsername :=Proxy_username;
IdHTTP1.ProxyParams.ProxyPassword :=Proxy_password;
I am using Delphi XE8.
They have created a virtual server for our testing, it runs Windows 7 64 bit.
Update
Remy, is this the right idea? One problem I am having is the TIdConnectThroughHttpProxy component, what must be in the uses clause for this? Delphi is not recognizing it.
var
TempIO : TIdIOHandlerStack;
TempProxy : TIdConnectThroughHttpProxy;
......
TempIO :=TIdIOHandlerStack.Create;
TempProxy :=TIdConnectThroughHttpProxy.Create;
TempProxy.Host :=Proxy_host;
TempIO.TransparentProxy:=TempProxy;
IdFTP1.IOHandler :=TempIO;
IdFTP1.Connect;
Update 2
A point of clarity: the HTTP request that is successfully reaching the web server through the proxy server goes to a different web address than the FTP request. In other words, they both go through the same proxy server, but the destination addresses are different. Just in case this is of use.
I have now tried using Fiddler to find the problem, not sure if this is a great idea? My understanding is is that Fiddler acts as a proxy server, so I thought I would see if I encountered the same problem. Sure enough, can't connect.
To be clear about my steps:
Run Fiddler, and check the box that says 'Capture FTP requests'.
Update my IdHTTP component:
IdHTTP1.ProxyParams.ProxyServer:='127.0.0.1';
IdHTTP1.ProxyParams.ProxyPort :=Fiddler port;
Update my IdFTP component:
IdFTP1.ProxySettings.Host:='127.0.0.1';
IdFTP1.ProxySettings.Port:=Fiddler port;
So now I have removed the client setup completely, I am mirroring the problem from my local machine using Fiddler as far as I can tell. If I don't use Fiddler, everything works great. If I use Fiddler as described above, then the HTTP request works correctly, but the FTP request can't connect.
Any ideas as to what I can do to try and solve this? I'm sure it is something really stupid that I'm doing wrong.
IdFTP1.ProxySettings.ProxyType:=fpcmNone
This tells TIdFTP not to communicate with an FTP-aware proxy. If you want to use the TIdFTP.ProxySettings properties, you need to set the ProxyType so TIdFTP.Connect() will connect to the ProxySettings.Host and TIdFTP.Login() will know what kind of commands it needs to send to login to the proxy and request a connection to the next host.
Note that TIdFTP.ProxySettings only works with FTP proxies. If you need to connect to a different type of proxy, before you call TIdFTP.Connect() you will have to assign a TIdIOHandler-derived component to the TIdFTP.IOHandler property, and then assign a TIdCustomTransparentProxy-derived component to the TIdIOHandler.TransparentProxy property. To connect to an HTTP proxy (which it sounds like you need, since that is what TIdHTTP.ProxyParams works with), use TIdConnectThroughHttpProxy. To connect to a SOCKS proxy, use TIdSocksInfo.
I'm writing an HTTP API wrapper to integrate with particular IP Cameras. Part of the process requires login credentials in the URI, for example:
http://user:pass_with_#_sign#address:port/somespecificpath/
I'm using Indy's TIdHTTP.Get to send the request to the device. However, the password consists of a '#' character. If this URI is placed in any browser with the plain password, the # character throws it off. Therefore, I need to encode the password's # to %23...
http://user:pass_with_%23_sign#address:port/somespecificpath/
When I paste this URI into any browser, it successfully logs in and does what it needs. However, when I pass the exact same URI into TIdHTTP.Get, it does not successfully log in, and therefore I cannot do anything as long as the password contains # (or %23). Changing the password to not include a # is far too sloppy of a solution. Indy must be messing something up with this URI/password.
Is this a bug in Indy, or is there something else I need to do to make Indy accept such an encoded password?
UPDATE
I added a new account on one of the cameras with username and password without any special characters which need encoding, and authentication still does not work. It seems as if Indy is stripping out the login credentials completely from the URI, and doesn't even send these credentials. Next thing I need to do is monitor the URI which is actually sent.
UPDATE 2
I did a packet capture via WireShark and have verified that the credentials ...user:pass#... are not even sent - they're stripped from the URI that Indy actually sends.
UPDATE 3
TLama suggested that I get a capture of what's sent when using a browser. I tried this, and sure enough even when using a browser the capture doesn't show these login credentials either... even though it works from a browser. So I have to figure out another way to identify whether or not these credentials are sent.
UPDATE 4
I tried (before seeing Remy's answer) to provide these credentials in Request.Username and Request.Password instead of in the URI, and I still have no success. I keep getting "Unauthorized" back from the device.
UPDATE 5
The documentation for this API mentions nothing relevant to how users are authenticated other than this paragraph:
Grandstream Video Surveillance API (Application Programming Interface) supports HTTP 1.0 protocol (RFC1945). This document explains in detail the parameter of functions in client side, via the supported GET/POST method. Users will require administrator privilege to retrieve or set the parameters.
And on that note, I did switch the TIdHTTP protocol version to 1.0.
UPDATE 6
Hopefully the last update needed... I did another comparison with the packet captures between a browser (Chrome) and TIdHTTP. Chrome actually sends two requests, the first one does not have any credentials, but in the second request there's a node in the header Authorization: Basic... and Credentials: User:Pass, whereas using TIdHTTP only sends 1 single request without these credentials.
UPDATE 7
7 is a lucky number :-) I just realized, the very first request I make to the device returns "Unauthorized", but all following requests I make (using the same TIdHTTP instance) are successful! So going back to my prior update, just like I see in the browser capture, it takes that second repetitive request for it to work.
# is an illegal character in a URL prior to the fragment portion of the URL, that is why it has to be encoded as %23 when used in other areas of the URL.
A username/password is not actually part of a real URL, that is why they get stripped off when TIdHTTP sends the URL to a server (monitor the traffic of any web browser and you will see the same thing happen).
To use HTTP authentication with TIdHTTP, you need to use the TIdHTTP.Request.Username and TIdHTTP.Request.Password properties instead (and you do not need to URL encode the values), eg:
IdHTTP1.Request.Username := 'user';
IdHTTP1.Request.Password := 'pass_with_#_sign';
IdHTTP1.Get('http://address:port/somespecificpath/');
If you pass a URL that has an encoded username/password in it, TIdHTTP will strip off the values and move them to the Request.Username and Request.Password properties for you, but they will remain in their original encoded format, eg:
IdHTTP1.Get('http://user:pass_with_%23_sign#address:port/somespecificpath/')
// this will set Request.Username to 'user',
// Request.Password to 'pass_with_%23_sign', and
// send a request for 'http://address:port/somespecificpath/'
If you are being given an encoded URL to start with, you can use the TIdURI class to manually decode it prior to then calling TIdHTTP.Get(), eg:
var
RequestUrl: string;
Uri: TIdURI;
begin
RequestUrl := 'http://user:pass_with_%23_sign#address:port/somespecificpath/';
...
Uri := TIdURI.Create(RequestURL);
try
IdHTTP1.Request.Username := TIdURI.URLDecode(Uri.UserName);
IdHTTP1.Request.Password := TIdURI.URLDecode(Uri.Password);
RequestURL := Uri.URI;
finally
Uri.Free;
end;
IdHTTP1.Get(RequestUrl);
...
end;
Update: either way, make sure you have appropriate IdAuthentication... units, or the IdAllAuthentications unit, in your uses clause to enable Indy's HTTP authentication classes for TIdHTTP to use.
I have solved the issue by sending two sequential Get requests. After observing the packet captures between a browser and Indy, I noticed browsers would always send one request without credentials, and then another identical request with credentials. So it only sends the credentials when it needs to. Indy was only sending one request, but if I send another request right afterward, I have success.
So, the request now looks like...
FWeb.Get(U);
FWeb.Get(U, R);
Of course it really should be in the order of "If the first request is unauthorized, then send another request with credentials".
I use MySQL in a shared server account and my worry is if a hacker monitors the connection between my application and that MySQL server.
Is that connection ciphered anyhow or is it raw data passing through (including at the connection time when Zeos TZConnection component informs the server what is the data base name, user name and password)?
If it's raw, how could I add some protection to it?
Actually it is not your TZConnection which speaks with the server. Instead, it communicates with libmysql.dll which sends and receives data from the Mysql server.
To secure your connection to the server you can use SSL. You will need 3 certificates:
CA-cert.pem
client-cert.pem
client-key.pem
You can find information on how to generate them in this MySQL link.
Once you have them you need to setup TZConnection to indicate SSLshould be used as follows:
Connection.Properties.Values['MYSQL_SSL'] := 'TRUE';
Connection.Properties.Values['MYSQL_SSL_CA'] := 'c:/MyPath/CA-cert.pem';
Connection.Properties.Values['MYSQL_SSL_CERT'] := 'c:/MyPath/client-cert.pem';
Connection.Properties.Values['MYSQL_SSL_KEY'] := 'c:/MyPath/client-key.pem';
More information about MySql and SSL can be found in this discussion in Zeos forums.
Good day,
Please help with an example of how to use idWHOIS via a web proxy?
I want to make whois calls anonymously using free web proxies.
Why is it so simple with idHTTP component that have ProxyParams but with idWHOIS not?
I am really clueless :)
See my attempted code snipped:
procedure TForm2.Button2Click(Sender: TObject);
var
ProxyHTTP :TIdConnectThroughHttpProxy;
Proxy :TIdCustomTransparentProxy;
begin
ProxyHTTP := TIdConnectThroughHttpProxy.Create(Self);
Proxy := TIdCustomTransparentProxy.Create(Self);
Try
{ idIOHandler.TransparentProxy := Proxy;
Proxy.Host := 'whois.ausregistry.net.au';
Proxy.Port := StrToInt('43');
Proxy.IPVersion := ID_DEFAULT_IP_VERSION;
// Proxy.Connect(idIOHandler, edtProxy.Text, StrToInt(edtPort.Text), ID_DEFAULT_IP_VERSION);
// Proxy.Enabled;
Proxy.ChainedProxy := ProxyHTTP;
}
idIOHandler.TransparentProxy := ProxyHTTP;
ProxyHTTP.Host := edtProxy.Text;
ProxyHTTP.Port := StrToInt(edtPort.Text);
ProxyHTTP.IPVersion := ID_DEFAULT_IP_VERSION;
ProxyHTTP.OnStatus := ProxyHTTPOnStatus;
ProxyHTTP.Enabled;
ProxyHTTP.Connect(idIOHandler, edtProxy.Text, StrToInt(edtPort.Text), ID_DEFAULT_IP_VERSION);
Memo1.Clear;
Memo1.Update;
Memo1.Lines.Text := idWhois.WhoIs(edtDomain.Text + '.com.au');
Finally
idIOHandler.Close;
idWhois.Disconnect;
Proxy.Free;
ProxyHTTP := TIdConnectThroughHttpProxy.Create(Self);
End;
end;
1st of all - do you use whois or http protocol ?
there are native whois services working via their native protocol.
and there are WWW front-ends, that let you make whois requests and vew results in WWW browsers like MSIE (Microsoft Internet Explorer).
Obviously you should be able to connect to latter via HTTP-proxy.
WWW here is monicker for several related technologies used together: HTTP + SSL/TLS + WebDAV + HTML + CSS + JS, etc. So, HTTP proxy may be considered subset of HTTP which is subset of WWW
And not be able to connect to former, for whois is not part of WWW blob.
Same thing for example is with e-mail, which is out of WWW realm as well:
there are mail programs, connecting directly by IMAP/POP/SMTP protocols and corresponding ports,
and there are WWW front-ends like Yahoo and GMail that allow to view mails via MSIE. WWW access works via HTTP proxy, direct access is not HTTP and does not work.
Exception: some proxies allow pin-holling. That is treated as security hole and bad configuration, but nonetheless it is sometimes possbile via HTTP/SSL support command CONNECT. But usually it is not possible to connect to every ports but only to :80 and :443 ports. And it is extremely rare that whois protocol would be on that ports, only HTTP or HTTPS protocols. Some servers, like many messengers like Jabber, ICQ, GTalk, or like Skype super-nodes, intentionally misuse those ports :80 and :443 to provide their non-HTTP protocols, thus helping those program to cheat on HTTP Proxy and bypass NAT isolation. But i think you'd hardly find many whois servers doing this
Whoever, if you find poorly configured proxy or such specially-configured whois server, you would have chance to use it.
Otherwise you would only be ably to use idHTTP over existing WWW-front-ends to whois servers. Most DNS name registrars do provide them, so users could check the data using no special tools but only WWW browsers.
Google for "http proxy ssl connect pinholling" for more details about such use of HTTP proxy.
Google for "NAT Traversal" and "Proxy Tunelling" for more general concept.
Sometimes VPN techniques are [mis-]used to tunnel outside.
PS. this does not answer "how can i do it" - there is no 100% reliable way - but hopefully answers "Why is it so simple with idHTTP component that have ProxyParams but with idWHOIS not?".
I guess you'd meet the very same obstacles, if to us idSMTP or other e-mail components through HTTP proxy.
PPS. there are more generic kind of proxies - Socks Proxy - that should allow any TCP-class protocol to be forwarded. But they are very rare thing.