Serving files with IdHTTPServer when the files are being written - delphi

Question

I'm working with a TIdHTTPServer to serve files to clients, using the ResponseInfo->ServeFile function. This works fine for files that are "static": not being written by some other process. As far as I can see from the code, the ServeFile function internally uses a TIdReadFileExclusiveStream, which disallows me from reading a file being written, but I need to be able to send also files that are being written by some other process.
So, I moved to create a FileStream myself and use the ContentStream property to return it to the client, but I get a 0 bytes file in the client (for any file, being written or not), and I can't see what I'm missing or doing wrong. Here is the code I'm using on the OnCommandGet event handler:
AResponseInfo->ContentStream = new TFileStream(path, fmOpenRead | fmShareDenyNone);
AResponseInfo->ContentStream->Position = 0;
AResponseInfo->ContentLength = AResponseInfo->ContentStream->Size;
AResponseInfo->ResponseNo = 200;
AResponseInfo->WriteHeader();
AResponseInfo->WriteContent();
The ContentLength property at this point has a valid value (i.e., the file size when calling ContentStream->Size), and that's what I would like to send to the client, even if the file changes in between.
I have tried removing the WriteContent() function, the WriteHeader(), but the results are the same. I searched for some examples but the few I found are more or less the same than this code, so I don't know what's wrong. Most examples don't include the WriteContent() call, that's why I have tried removing them, but there doesn't seem to be any difference.
As a side note: the files being written take 24 hours to finish writing, but that's to be expected from the client side: I just need the bytes already written at the time of the request (even somewhat less is valid). The files will never get deleted: they will just keep getting bigger.
Any ideas?
Update
Using Fiddler, I get some warnings on protocol violations, that would be related to this. I get, for instance:
Content-Length mismatch: Response Header indicated 111,628,288 bytes, but server sent 41 bytes
The content length is correct, it's the file size, but I don't know what I'm doing wrong that makes the app sent just 41 bytes.

Answer 1

WriteHeader() and WriteContent() expect the ContentStream to be complete and unchanging at the time they are called. WriteHeader() creates a Content-Length header using the current ContentStream->Size value if the AResponseInfo->ContentLength property is -1 (you are actually setting the value yourself), and WriteContent() sends only as many bytes as the current ContentStream->Size value says. So your client is receiving 0 bytes because the file Size is still 0 at the time you are calling WriteHeader() and WriteContent().
Neither ServeFile() nor ContentStream are suitable for your needs. Since the file is being written live, you do not know the final file size when the HTTP headers are created and sent to the client. So you must use HTTP 1.1's chunked transfer coding to send the file data. That will allow you to send the file data in chunks as the file is being written, and then signal the client when the file is finished.
However, TIdHTTPServer does not natively support sending chunked responses, so you will have to implement it manually, eg:
TFileStream *fs = new TFileStream(path, fmOpenRead | fmShareDenyNone);
try
{
AResponseInfo->ResponseNo = 200;
AResponseInfo->TransferEncoding = "chunked";
AResponseInfo->WriteHeader();
TIdBytes buffer;
buffer.Length = 1024;
do
{
int NumRead = fs->Read(&buffer[0], 1024);
if (NumRead == -1) RaiseLastOSError();
if (NumRead == 0)
{
// check for EOF, unless you have another way to detect it...
Sleep(1000);
NumRead = fs->Read(&buffer[0], 1024);
if (NumRead <= 0) break;
}
// send the current chunk
AContext->Connection->IOHandler->WriteLn(IntToHex(NumRead));
AContext->Connection->IOHandler->Write(buffer, NumRead);
AContext->Connection->IOHandler->WriteLn();
}
while (true);
// send the last chunk to signal EOF
AContext->Connection->IOHandler->WriteLn("0");
// send any trailer headers you need, if any...
// finish the transfer encoding
AContext->Connection->IOHandler->WriteLn();
}
__finally
{
delete fs;
}

Answer 2

The final working code is:
std::unique_ptr< TFileStream >fs(new TFileStream(path, fmOpenRead | fmShareDenyNone));
fs->Position = 0;
__int64 size = fs->Size;
AResponseInfo->ContentLength = size;
AResponseInfo->ResponseNo = 200;
AResponseInfo->WriteHeader();
AContext->Connection->IOHandler->Write(fs.get(), size);
This allows the client to receive up to size bytes of the original file, even if the file is being written to at the same time.
For some reason passing the ContentStream did not return any content to the client, but doing the IOHandler->Write directly (which is what the ServeFile ends doing internally) works fine.