Is TIdHTTPServer Compatible with Microsoft BITS - delphi

We are trying to write an update server for our software using the TIdHTTPServer component. Currently we are serving an XML file that lists the available updates and their file versions etc.., when the client program finds a updated version it should start to download it using BITS.
Now this is where we have a problem, our programs are requesting the XML file and seeing there is an update available. It then creates a BITS job to download it, however BITS keeps reporting that the download failed. We can download the file using the same URL and IE/Firefox/Chrome.
so my question:
Is TIdHTTPServer compatible with BITS?
I ask this as I have discovered that there are these download requirements for bits to work.
HTTP Requirements for BITS Downloads
BITS supports HTTP and HTTPS downloads and uploads and requires that the server supports the HTTP/1.1 protocol. For downloads, the HTTP server's Head method must return the file size and its Get method must support the Content-Range and Content-Length headers. As a result, BITS only transfers static file content and generates an error if you try to transfer dynamic content, unless the ASP, ISAPI, or CGI script supports the Content-Range and Content-Length headers.
BITS can use an HTTP/1.0 server as long as it meets the Head and Get method requirements.
To support downloading ranges of a file, the server must support the following requirements:
Allow MIME headers to include the standard Content-Range and Content-Type headers, plus a maximum of 180 bytes of other headers.
Allow a maximum of two CR/LFs between the HTTP headers and the first boundary string.

Just found a bug in indy that prevents transfer of files over 2.1GB when using range requests.
here it is
IdHTTPHeaderInfo.pas aprox line 770
procedure TIdEntityRange.SetText(const AValue: String);
var
LValue, S: String;
begin
LValue := Trim(AValue);
if LValue <> '' then
begin
S := Fetch(LValue, '-'); {do not localize}
if S <> '' then begin
FStartPos := StrToIntDef(S, -1);
FEndPos := StrToIntDef(Fetch(LValue), -1);
FSuffixLength := -1;
end else begin
FStartPos := -1;
FEndPos := -1;
FSuffixLength := StrToIntDef(Fetch(LValue), -1);
end;
end else begin
FStartPos := -1;
FEndPos := -1;
FSuffixLength := -1;
end;
end;
This should be
procedure TIdEntityRange.SetText(const AValue: String);
var
LValue, S: String;
begin
LValue := Trim(AValue);
if LValue <> '' then
begin
S := Fetch(LValue, '-'); {do not localize}
if S <> '' then begin
FStartPos := StrToInt64Def(S, -1);
FEndPos := StrToInt64Def(Fetch(LValue), -1);
FSuffixLength := -1;
end else begin
FStartPos := -1;
FEndPos := -1;
FSuffixLength := StrToInt64Def(Fetch(LValue), -1);
end;
end else begin
FStartPos := -1;
FEndPos := -1;
FSuffixLength := -1;
end;
end;
One for Remy to fix

When you handle the OnCommandGet event, you are given a TIdRequestHeaderInfo, which descends from TIdEntityHeaderInfo; that contains all the headers the request contained, and it even parses out some header values to read as properties, including ContentRangeStart, ContentRangeEnd, and ContentLength.
You can use those properties to populate the stream that you assign to the TIdHTTPResponseInfo.ContentStream property. The entire stream will get sent.
It's your job to differentiate between GET and HEAD requests; OnCommandGet will get triggered either way. Check the IdHTTPRequestInfo.CommandType property.
So, although Indy may not support BITS, it provides all the tools you need to write a program that does support BITS.

So the answer to this question is:
Yes TIdHTTPServer is Bits Compatible.
But only if you are prepared to do the work yourself.
As suggested by #Rob Kennedy and Myself it is possible to read the headers and send the data back using the requested ranges, one chunk at a time.
Here is an example of what I am doing in the OnCommandGet event
procedure TForm3.IdHTTPServer1CommandGet(AContext: TIdContext;
ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
Ranges : TIdEntityRanges;
DataChunk: TMemoryStream;
ReqFile: TFileStream;
ChunkLength: Int64;
Directory, FileName: string;
begin
Directory := 'H:';
case ARequestInfo.Ranges.Count of
0:
begin
//serve file normally
end;
1:
begin
//serve range of bytes specified for file
filename := Directory + ARequestInfo.Document;
if FileExists(FileName) then
begin
ReqFile := TFileStream.Create(FileName, fmOpenRead);
try
ChunkLength := Succ(ARequestInfo.Ranges.Ranges[0].EndPos - ARequestInfo.Ranges.Ranges[0].StartPos);
if ChunkLength > ReqFile.Size then
ChunkLength := ReqFile.Size;
DataChunk := TMemoryStream.Create;
DataChunk.Posistion := ARequestInfo.Ranges.Ranges[0].StartPos;
DataChunk.CopyFrom(ReqFile, ChunkLength);
AResponseInfo.ContentStream := DataChunk;
AResponseInfo.ContentType := IdHTTPServer1.MIMETable.GetFileMIMEType(FileName);
AResponseInfo.ContentRangeUnits := ARequestInfo.Ranges.Units;
AResponseInfo.ContentRangeStart := ARequestInfo.Ranges.Ranges[0].StartPos;
AResponseInfo.ContentRangeEnd := ARequestInfo.Ranges.Ranges[0].StartPos + Pred(ChunkLength);
AResponseInfo.ContentRangeInstanceLength := ReqFile.Size;
AResponseInfo.ResponseNo := 206;
finally
ReqFile.Free;
end;
end
else
AResponseInfo.ResponseNo := 404;
end
else
begin
//serve the file as multipart/byteranges
end;
end;
end;
This is by no means finished but it shows the basics of responding to the range requests from BITS. Most importantly it works.
Any comments on the code would be appreciated, constructive criticism always welcome.

Related

Error connecting with SSL using IdHttp Indy

I have a problem in IdHttp using Indy (Delphi).
I try to using IdHttp to post XML in web service SOAP, but dont work. Return "Error connecting with SSL." in IdSSLOpenSSL.Connect#1437 from indy.
My code is simple:
procedure TForm1.Button1Click(Sender: TObject);
var
vRequest : TStringStream;
s : String;
begin
vRequest := TStringStream.Create((Memo1.Lines.Text));
try
IdHTTP1.Host := edHost.Text;
IdHTTP1.Request.ContentLength := length(Memo1.Lines.Text);
IdHTTP1.Request.ContentType := edContentType.Text;
IdHTTP1.Request.CustomHeaders.Text := 'SOAPAction: "removed for safe"'#13#10;
IdHTTP1.request.CacheControl := 'no-cache';
IdHTTP1.Request.AcceptEncoding := edAccept.Text;
IdHTTP1.HTTPOptions := [hoKeepOrigProtocol];
IdHTTP1.ProtocolVersion := pv1_1;
Memo2.Clear;
try
s := IdHTTP1.post(Edit1.Text, vRequest);
Memo2.Lines.Text := s;
except
on e: EIdHTTPProtocolException do begin
Label1.Caption := e.Message;
MEMO2.LINES.TEXT := e.Message;
end;
on e:Exception do begin
Label1.Caption := e.Message;
MEMO2.LINES.TEXT := e.Message;
end;
end;
requestHeaders.Lines.Text := IdHTTP1.Request.RawHeaders.Text;
responseHeaders.Lines.Text := IdHTTP1.Response.RawHeaders.Text;
finally
vRequest.Free;
end;
end;
In exe folder contain libeay32.dll and ssleay32.dll. Any sugestion?
I Used delphi 5, but in delphi 7 is the same problem.
By default, the SSLVersions property of TIdSSLIOHandlerSockOpenSSL is set to enable only TLS 1.0 1. But many websites are starting to phase out TLS 1.0 and only accept TLS 1.1+. So try enabling TLS 1.1 and TLS 1.2 in the SSLVersions property and see if it helps.
1: there is an open ticket in Indy's issue tracker to also enable TLS 1.1 and TLS 1.2 by default.
On a side note, there are some further tweaks you should make to your code:
do not assign any values to TIdHTTP's Host or ContentLength properties. They are populated automatically by TIdHTTP.
if you set the AcceptEncoding property manually, make sure NOT to include deflate or gzip unless you have a Compressor assigned to TIdHTTP, otherwise it will fail to decode a compressed response. You really should not assign anything to AcceptEncoding unless you are prepared to handle custom encodings. The Compressor handles deflate/gzip and TIdHTTP will update AcceptEncoding accordingly if a Compressor is assigned and ready for use.
use the CustomHeaders.Values property to set individual headers, not the CustomHeaders.Text property.
you do not need to catch EIdHTTPProtocolException explicitly, since that exception handler is not doing anything extra that the more generic Exception handler is not doing.
the RawHeaders property is a TStringList descendant, so it is more efficient to use Lines.Assign(RawHeaders) instead of Lines.Text := RawHeaders.Text.
Try this:
procedure TForm1.Button1Click(Sender: TObject);
var
vRequest : TStringStream;
s : String;
begin
IdHTTP1.Request.ContentType := edContentType.Text;
IdHTTP1.Request.CustomHeaders.Values['SOAPAction'] := 'removed for safe';
IdHTTP1.Request.CacheControl := 'no-cache';
IdHTTP1.HTTPOptions := [hoKeepOrigProtocol];
IdHTTP1.ProtocolVersion := pv1_1;
Memo2.Clear;
try
vRequest := TStringStream.Create(Memo1.Lines.Text);
try
s := IdHTTP1.Post(Edit1.Text, vRequest);
finally
vRequest.Free;
end;
Memo2.Lines.Text := s;
except
on e: Exception do begin
Label1.Caption := e.Message;
Memo2.Lines.Text := e.Message;
end;
end;
RequestHeaders.Lines.Assign(IdHTTP1.Request.RawHeaders);
ResponseHeaders.Lines.Assign(IdHTTP1.Response.RawHeaders);
end;

Saved Sqlite Image Appears Totally Black On Uploaded Server Database

I have an interesting problem. I have to take a signature image that is saved in Sqlite3 database.
When you view the saved signature on the Sqlite3 database it is a signature. The database is uploaded to a server that has MSSQL Server which then reads the signature image. Unfortunately this image appears totally black on the server and it comes back the same. The db administrator says that the data appears to be garbage (includes unicode characters).
Example of the image data is:
Sent Data: Tried to include but unfortunately it will not show unicode. It is thousands of bytes long.
Returned Data: *System.Byte[]* <<and that is it - 26 bytes long.
My guess is that the unicode is responsible. Not sure how to resolve this issue.
The data connection component is TSQLConnection. The query component is TSQLQuery. I am using XE5 building a Firemonkey iOS mobile application.
Here is my code. Appreciate any help.
function SaveSig: boolean;
var
fStream: TMemoryStream;
begin
Result := False;
fStream := TMemoryStream.Create;
try
try
fStream.Seek(0, soFromBeginning);
fStream.Position := 0;
if Assigned(imgSig) then
begin
imgSig.Bitmap.SaveToStream(fStream);
Result := SqlInsertSig(fStream);
end;
except
on e: Exception do
ShowMessage(ERROR_BITMAP + e.Message);
end;
finally
if Assigned(fStream) then
FreeAndNil(fStream);
end;
end;
function SqlInsertSig(const ms: TMemoryStream): boolean;
begin
Result := False;
try
try
sq.Active := False;
sq.CommandText := 'Insert Into Signatures (Id, Sig) Values (' +
QuotedStr(IntToStr(Id)) + ', :sig)';
sq.Params.ParamByName('sig').LoadFromStream(ms, ftBlob);
Result := (sq.ExecSQL > 0);
except
on e: Exception do
MessageDlg(e.Message, TMsgDlgType.mtError, [TMsgDlgBtn.mbOK], 0);
end;
finally
sq.Active := False;
end;
end;

How to get body texts of all e-mail messages from a certain IMAP mailbox?

How to get body texts of all e-mail messages from a certain IMAP mailbox in Delphi ? For example, from the INBOX mailbox ?
There are many ways to retrieve all body texts of all messages from the selected mailbox. I've used the one, where you iterate the mailbox and Retrieve every single message from the mailbox one by one. This way allows you to modify the code, so that you'll be able to break the loop when you need or e.g. replace Retrieve by RetrievePeek which won't mark the message as read on server like the first mentioned does. When the message is retrieved from server, all its parts are iterated and when it's the text part, its body is appended to a local S variable. After the iteration the S variable is added to the output BodyTexts string list. So, as the result you'll get string list collection where each item consists from the concatenated message's text part bodies and where each item means one message.
uses
IdIMAP4, IdSSLOpenSSL, IdText, IdMessage, IdExplicitTLSClientServerBase;
procedure GetGmailBodyTextParts(const UserName, Password: string;
BodyTexts: TStrings);
var
S: string;
MsgIndex: Integer;
MsgObject: TIdMessage;
PartIndex: Integer;
IMAPClient: TIdIMAP4;
OpenSSLHandler: TIdSSLIOHandlerSocketOpenSSL;
begin
BodyTexts.Clear;
IMAPClient := TIdIMAP4.Create(nil);
try
OpenSSLHandler := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
try
OpenSSLHandler.SSLOptions.Method := sslvSSLv3;
IMAPClient.IOHandler := OpenSSLHandler;
IMAPClient.Host := 'imap.gmail.com';
IMAPClient.Port := 993;
IMAPClient.UseTLS := utUseImplicitTLS;
IMAPClient.Username := UserName;
IMAPClient.Password := Password;
IMAPClient.Connect;
try
if IMAPClient.SelectMailBox('INBOX') then
begin
BodyTexts.BeginUpdate;
try
for MsgIndex := 1 to IMAPClient.MailBox.TotalMsgs do
begin
MsgObject := TIdMessage.Create(nil);
try
S := '';
IMAPClient.Retrieve(MsgIndex, MsgObject);
MsgObject.MessageParts.CountParts;
if MsgObject.MessageParts.TextPartCount > 0 then
begin
for PartIndex := 0 to MsgObject.MessageParts.Count - 1 do
if MsgObject.MessageParts[PartIndex] is TIdText then
S := S + TIdText(MsgObject.MessageParts[PartIndex]).Body.Text;
BodyTexts.Add(S);
end
else
BodyTexts.Add(MsgObject.Body.Text);
finally
MsgObject.Free;
end;
end;
finally
BodyTexts.EndUpdate;
end;
end;
finally
IMAPClient.Disconnect;
end;
finally
OpenSSLHandler.Free;
end;
finally
IMAPClient.Free;
end;
end;
This code requires OpenSSL, so don't forget to put the libeay32.dll and ssleay32.dll libraries to a path visible to your project; you can download OpenSSL libraries for Indy in different versions and platforms from here.

Download CSV in Delphi 5 with Indy

I know there's alot of Indy threads but I can't get one to match my case.
I have been given a URL with a username and password form. this then actions to a URL/reports.php on which there are multiple hyperlinks.
Each of these links will direct to a page with URL variables e.g. reports.php?report=variablename where a download will immediately start.
My thinking so far:
procedure TForm1.PostData(Sender: TObject);
var
paramList:TStringList;
url,text:string;
// IdHTTP1: TIdHTTP;
IdSSLIOHandlerSocket1: TIdSSLIOHandlerSocket;
idLogFile1 : TidLogFile;
begin
idLogFile1 := TidLogFile.Create(nil);
with idLogFile1 do
begin
idLogFile1.Filename := 'C:\HTTPSlogfile.txt';
idLogFile1.active := True;
end;
IdHTTP1 := TIdHTTP.Create(nil);
IdSSLIOHandlerSocket1 := TIdSSLIOHandlerSocket.Create(nil);
IdSSLIOHandlerSocket1.SSLOptions.Method := sslvSSLv23;
IdHTTP1.IOHandler := IdSSLIOHandlerSocket1;
IdHTTP1.HandleRedirects := true;
IdHTTP1.ReadTimeout := 5000;
IdHTTP1.Intercept := idLogFile1;
paramList:=TStringList.create;
paramList.Clear;
paramList.Add('loguser=testuser');
paramList.Add('logpass=duke7aunt');
paramList.Add('logclub=8005');
url := 'https://www.dfcdata.co.uk/integration/reports.php?report=live';
try
IdHTTP1.Post(url,paramList);
except
on E:Exception do
begin
showMessage('failed to post to: '+url);
ShowMessage('Exception message = '+E.Message);
end;
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
var
reportType : String;
begin
PostData(Self);
reportType := 'live';
GetUrlToFile('',reportType+'.csv');
end;
procedure TForm1.GetUrlToFile(AURL, AFile : String);
var
Output : TMemoryStream;
success : Boolean;
begin
success := True;
Output := TMemoryStream.Create;
try
try
IdHTTP1.Get(AURL, Output);
IdHTTP1.Disconnect;
except
on E : Exception do
begin
ShowMessage('Get failed to GET from '+IdHTTP1.GetNamePath +'. Exception message = '+E.Message);
success := False;
end;
end;
if success = True then
begin
showMessage('Filed saved');
Output.SaveToFile(AFile);
end;
finally
Output.Free;
end;
end;
On each try I get "IOHandler is not valid" error. Obviously I'm not posting correctly to the initial page but can anyone advise me on what I'm missing? Also can I simply then hit the download URL after login or will I have to use cookies?
Thanks
There are several bugs in your code:
1) PostData() is requesting an HTTPS URL, but it is not assigning an SSL-enabled IOHandler to the TIdHTTP.IOHandler property. You need to do so.
2) Button1Click() is passing a URL to GetUrlToFile() that does not specify any protocol, so TIdHTTP will end up treating that URL as relative to its existing URL, and thus try to download from https://www.testurl.com/test/testurl.com/test/reports.phpinstead of https://testurl.com/test/reports.php. If you want to request a relative URL, don't include the hostname (or even the path in this case, since you are sending multiple requests to the same path, just different documents).
3) you are leaking the TIdHTTP object.
Issue 1) has now been resolved in another post:
Delphi 5 Indy/ics SSL workaround?
However I would greatly appreciate help on the rest, as follows.
Would I need to make a GET call with the same IdHTTP object and additional URL variable? or should I create a new IdHTTP object?
Would I need to record the session using cookies or can all of this be done with the same call?
Is the GET call above actually what I need to save a csv to file? I may also choose to handle it directly as the data will need importing anyway.
Currently the code gets the error: EIdHTTPProtocolException

How to receive emails using indy 10 and delphi 7 with the file attachment?

How to receive emails using indy 10 and delphi 7 with the file attachment?
This is working Indy 10 code. 'Files' is a stringlist which holds a list of attachments which have been downloaded - I'm interested in the attachments, not the letters themselves.
with IdPop31 do
begin
ConnectTimeout := 5000;
Connect;
try
files.Clear;
for i := 1 to checkmessages do
begin
msg.clear;
flag := false;
if retrieve (i, msg) then
begin
for j := 0 to msg.MessageParts.Count-1 do
begin
if msg.MessageParts[j] is TIdAttachment then
begin
with TIdAttachment(msg.MessageParts[j]) do
begin
s := IncludeTrailingPathDelimiter(mydir) + ExtractFileName(FileName);
log ('Downloaded ' + s);
if not FileExists(s) then
begin
SaveToFile(s);
files.Add(s);
end;
end;
end;
flag := true;
end;
end;
end;
if flag then Delete(i); // remove the email from the server
end;
finally
Disconnect;
end
end;
Attachments are stored as TIdAttachment objects in the TIdMessage.MessageParts collection.
Your code is working fine, but need correction in "begin-end" section where "s" is defining. If "FileName" is empty program has to skip saving. Probably you cut this line and "end" is hanging.

Resources