I've written a DataSnap server method that returns a TStream object to transfer a file. The client application calls the method and reads the stream to download the file. The server method is very simple :
function TServerMethods.DownloadFile(sFilePath: string): TStream;
var
strFileStream: TFileStream;
begin
strFileStream := TFileStream.Create(sFilePath, fmOpenRead);
Result := strFileStream;
end;
It works fine downloading many file types (PDF, GIF, BMP, ZIP, EXE) but it doesn't work when downloading JPG files. On the client side the stream object returned from the method call is always 0 in size with JPGs. I can successfully stream JPG files locally on my PC, so it must be something to do with DataSnap. I've done some research which suggests DataSnap converts the stream to JSON behind the scenes and there could be a problem with this when it comes to JPG files - can anybody confirm this? On the client side I'm using the TDSRESTConnection to call the server method. I realise I could ZIP the JPG files before streaming, but would rather not have to do this.
Thought I'd update the thread on my attempts to resolve this. I never found a way to transfer a JPEG file over DataSnap using TStream, but have done it by converting the stream to a TJSONArray and passing this back instead. So my server method now looks as follows:
function TServerMethods.DownloadJPEGFile(sFilePath: string): TJSONArray;
var
strFileStream: TFileStream;
begin
strFileStream := TFileStream.Create(sFilePath, fmOpenRead);
Result := TDBXJSONTools.StreamToJSON(strFileStream, 0, strFileStream.Size);
end;
then at the client end I convert back to a TStream with:
strFileStream := TDBXJSONTools.JSONToStream(JSONArray);
I have created this as a new server method call purely for downloading JPEGs, as I've found transferring the files using TJSONArray instead of TStream is as much as 4 times slower, so I use my original method for all other file types.
Just as an update - after further research I've found this is related to the system locale in use on the PC. I'm using 'English (United Kingdom)' but if I change this to for example 'Japan (Japanese)' then the errors disappear and the file transfer works fine. I've logged this as a QC report with Embarcadero.
Embarcadero have now come back with a fix to this problem (which also affects .DOC files) :
1.Copy '...\RAD Studio\9.0\source\data\datasnap\Datasnap.DSClientRest.pas' to your DataSnap Client project folder
2.Add the .pas file to the project
3.Modify Line#1288 as below
// LResponseJSON := TJSONObject.ParseJSONValue(BytesOf(LResponseText.StringValue), 0);
LResponseJSON := TJSONObject.ParseJSONValue(BytesOf(UTF8String(LResponseText.StringValue)), 0);
4.Rebuild DataSnap REST Client project
5.Run it with REST Server
This fixes the problem.
Add this line to your DownloadFile method:
GetInvocationMetadata.ResponseContentType := 'image/jpeg';
Related
Using Delphi XE2 and the native TZipFile I attempt to extract the contents of a downloaded zip file (which contains 2 zipped XML files) and it always extracts zero byte files.
The file is being compressed by C# code like this:
var zipFile = new ZipFile();
foreach (Tuple<string, string> t in filesMeta) {
zipFile.AddFile(string.Format("{0}{1}", StaticVariables.WebServerFileStorage, t.Item2), "").FileName = t.Item1 + ".xml";
}
response.Clear();
response.ContentType = "application/zip";
zipFile.Save(response.OutputStream);
response.End();
The Delphi extraction code is this:
zipFile := TZipFile.Create;
try
filename := 'C:\test\57f52480-ec87-4169-a820-0a65bc4ad952.zip';
if zipFile.IsValid(filename) then begin
zipFile.Open(filename, zmRead);
zipFile.ExtractAll('C:\test\');
end;
finally
zipFile.Free;
end;
I even tried using a TStream as the source instead of a file on disk. That's actually what I want to do since the zip file is downloaded from a web server into a TStream. I tried to extract the data using the overloaded open method of TZipFile to open the stream.
That got me zero byte files so I saved the zip file to disk and tried to open the file from disk and extract. Same zero byte files are extracted.
I even tried using the class method to extract the files from the zip file on disk:
System.Zip.TZipFile.ExtractZipFile(filename, 'C:\Test\');
Same zero byte files extracted.
The zip file is valid and the 2 zipped XML files can be extracted properly by both Windows 7 native file handling and 7-Zip.
Now here is something nutty...
In desperation I tried to see what the ExtractToFile() procedure
David Heffernan came up with in this question about extracting a zip to a stream would do so I tried using it like this:
var x : integer;
var fileCount : integer;
var fileNames : TArray<string>;
begin
zipFile := TZipFile.Create;
try
filename := 'C:\test\57f52480-ec87-4169-a820-0a65bc4ad952.zip';
if zipFile.IsValid(filename) then begin
zipFile.Open(filename, zmRead);
fileCount := zipFile.FileCount;
fileNames := copy(zipFile.FileNames, 0, MaxInt);
zipFile.Close;
for x := 0 to fileCount-1 do begin
// Use David Heffernan's stream procedure
ExtractToFile(filename, x, 'C:\test\' + fileNames[x]);
end;
end;
finally
zipFile.Free;
end;
end;
And David's procedure extracts the files to disk as expected! WTF???
I am really confused why a convoluted method of extraction would work and the simple extraction method would not work. I'll use David's example if I have to but I'd prefer to get the normal extract working if possible.
Any ideas are appreciated.
Cheers!
TJ
I had this same problem.
The source code to TZipFile shows that the TStream passed into the Read function returns the entire zip file with position set to the start of the filename you're wanting. So don't rewind. Just copyfrom or do what you want with the TStream for the uncompressed length given in the TZipHeader.
ZipStream := TStream.Create;
ZipFile.Read(MyFileName, ZipStream, ZipHeader);
//leave ZipStream pointer where it is!!!
SomethingElse.LoadFromStream(ZipStream, ZipHeader.UncompressedSize);
ZipStream.Free;
In my opinion, TZipFile should really load the ZipStream with only what is requested. The way this is implemented is not intuitive without first going through the TZipFile source code.
TL;DR: The solution to my problem was an external component. Zip Forge (or Abbrevia)
Read on for details.
Nothing I tried except for the roundabout way of saving the file and re-opening it using David's function worked. While that would have worked, it was not optimal as it required me to save the downloaded file to disk first and reopen it for extract and then delete the zip file. My goal was to just open the downloaded stream and extract the files directly to disk. One write to disk and no temporary file.
We even tried two different C# libraries to zip the files and both gave the same results on the streamed data. The Delphi TZipFile component could not handle it.
It turns out we have a license to ZipForge which I had forgotten about since I had not used it in ages it and that handles the download stream from the C# web server and extracts the files successfully.
For reference, I also tried the Abbrevia component version 5.2 and that also successfully extracted the files from the stream.
Hopefully this will help someone else.
All the suggestions by David and Uwe were appreciated.
Cheers!
TJ
I use this code to download small files:
Var
ms:TMemoryStream;
begin
ms:=TMemoryStream.Create;
Idhttp1.get('http://mydomain.com/myfile.zip',ms);
ms.SaveToFile('myfile.zip');
ms.Free;
end;
But file is saved in RAM before storing to disk, so it may be difficult to download files >1Gb, for example. Is there a way to download a file by its parts? Or do I need to use the WinInet? Thanks in advance!
TMemoryStream provides an in-memory buffer, so if you download into one, you need to have enough memory to hold everything you receive. It's not the only kind of stream, though. You can pass the Get method any kind of stream you want, including one that writes its contents to disk as it receives it. Use TFileStream, for example.
var
s: TStream;
s := TFileStream.Create('myfile.zip', fmCreate);
try
IdHttp1.Get(..., s);
finally
s.Free;
end;
Anywhere you call LoadFromFile or SaveToFile on a TMemoryStream, it's possible that TFileStream is a better choice.
Trying to use the below mentioned approach to get more details about the locked file.
Is file in use
function GetFileInUseInfo(const FileName : WideString) : IFileIsInUse;
var
ROT : IRunningObjectTable;
mFile, enumIndex, Prefix : IMoniker;
enumMoniker : IEnumMoniker;
MonikerType : LongInt;
unkInt : IInterface;
begin
result := nil;
OleCheck(GetRunningObjectTable(0, ROT));
OleCheck(CreateFileMoniker(PWideChar(FileName), mFile));
OleCheck(ROT.EnumRunning(enumMoniker));
while (enumMoniker.Next(1, enumIndex, nil) = S_OK) do
begin
OleCheck(enumIndex.IsSystemMoniker(MonikerType));
if MonikerType = MKSYS_FILEMONIKER then
begin
if Succeeded(mFile.CommonPrefixWith(enumIndex, Prefix)) and
(mFile.IsEqual(Prefix) = S_OK) then
begin
if Succeeded(ROT.GetObject(enumIndex, unkInt)) then
begin
if Succeeded(unkInt.QueryInterface(IID_IFileIsInUse, result)) then
begin
result := unkInt as IFileIsInUse;
exit;
end;
end;
end;
end;
end;
end;
But the call to
unkInt.QueryInterface(IID_IFileIsInUse, result)
always returns E_NOINTERFACE.
Platform: Windows 7 32 bit-OS, opening word files and .msg files.
Checked opening files from the explorer and trying to delete. It shows proper details about the application in which the file is opened. In my application, I am try to display the information about application in which the file is opened. But when trying to cast the pointer to IFileIsInUse interface, QueryInterface calls fails with return code E_NOINTERFACE which means the object in ROT does not implement IFileIsInUse. AFASIK, MS Office files implements IFileIsInUse
Any idea what is wrong here?
In fact your code works fine. The problem is that the programs you are testing against really do not implement IFileIsInUse. When the system returns E_NOINTERFACE it is accurate. The interface is not implemented.
I tested this with the File Is In Use Sample from the SDK. Files that are added to the ROT by that application, which does implement IFileIsInUse, were picked up by your code. On the other hand, files opened by Acrobat 8 and Word 2010 were not.
The conclusion that I draw from this is that IFileIsInUse is a fine idea in principle, but not much use if applications don't support it. And it appears that there are major applications that do not.
It is clear that you will need to use one or more of the other mechanisms to detect which application has a file locked when you find that IFileIsInUse is not implemented.
SysInternals Process Explorer worked for me to delete a locked .msg file that was causing system problems like locking up the desktop.
Run Process Explorer, use the Find menu,
enter the full path file name,
hit Search.
For deleting a locked file, I opened a cmd window and tried to del the locked file, but the delete hung on the lock.
Then I used Process Explorer to restart the process holding the lock - Explorer.exe.
The del then completed successfully.
I want to load a url directly into a string without any data stream,what is the best way, internet open url works but it seems not clear.
I don't want to use any component for reading some short messages
Delphi 6 and later ship with Indy, which has a TIdHTTP client component, eg:
uses
..., IdHTTP;
var
Reply: String;
begin
Reply := IdHTTP1.Get('http://test.com/postaccepter?=msg1=3444&msg2=test');
...
end;
Or:
uses
..., IdHTTP;
var
Reply: TStream;
begin
Reply := TMemoryStream.Create;
try
IdHTTP1.Get('http://test.com/postaccepter?=msg1=3444&msg2=test', Reply);
Reply.Position := 0;
...
finally
Reply.Free;
end;
end;
Depending on your needs.
You can use Synapse, a very light weight library that has a simple function call to get just what your asking for:
uses
classes, httpsend;
var
Response : TStringlist;
begin
if HttpGetText(URL,Response) then
DoSomethingWithResponse(Response.Text);
end;
I would suggest getting the latest copy from SVN, which is more current and contains support for the latest versions of Delphi. There are also simple functions for posting form data, or retrieving binary resources. These are implemented as simple functions and are a great template if you want to do something extra, or that is not directly supported.
You can use our SynCrtSock unit, which is even lighter than Synapse.
See http://synopse.info/fossil/finfo?name=SynCrtSock.pas
It is a self-contained unit (only one dependency with the WinSock unit), and it works from Delphi 6 up to Delphi XE.
You have these two functions available to get your data in one line of code:
/// retrieve the content of a web page, using the HTTP/1.1 protocol and GET method
function HttpGet(const server, port: AnsiString; const url: TSockData): TSockData;
/// send some data to a remote web server, using the HTTP/1.1 protocol and POST method
function HttpPost(const server, port: AnsiString; const url, Data, DataType: TSockData): boolean;
Or you can use textfile-based commands (like readln or writeln) to receive or save data.
TSockData is just a wrapper of RawByteString (under Delphi 2009/2010/XE) or AnsiString (up to Delphi 2007).
If you need also to write a server, you have dedicates classes at hand, resulting in fast processing and low resource consummation (it uses a Thread pool, and is implemented over I/O Completion Ports).
If I'm already using XML in an application (and the MSXML2_TLB), I generally use IXmlHttpRequest to perform http operations. If you open and send the request, you can either use the response data as XML DOM using the ResponseXML, as text using ResponseText or as a data-stream using ResponseStream, see here for an example how to use this in Delphi: http://yoy.be/item.asp?i142
For some reason, lately the *.UDL files on many of my client systems are no longer compatible as they were once saved as ANSI files, which is no longer compatible with the expected UNICODE file format. The end result is an error dialog which states "the file is not a valid compound file".
What is the easiest way to programatically open these files and save as a unicode file? I know I can do this by opening each one in notepad and then saving as the same file but with the "unicode" selected in the encoding section of the save as dialog, but I need to do this in the program to cut down on support calls.
This problem is very easy to duplicate, just create a *.txt file in a directory, rename it to *.UDL, then edit it using the microsoft editor. Then open it in notepad and save as the file as an ANSI encoded file. Try to open the udl from the udl editor and it will tell you its corrupt. then save it (using notepad) as a Unicode encoded file and it will open again properly.
Ok, using delphi 2009, I was able to come up with the following code which appears to work, but is it the proper way of doing this conversion?
var
sl : TStrings;
FileName : string;
begin
FileName := fServerDir+'configuration\hdconfig4.udl';
sl := TStringList.Create;
try
sl.LoadFromFile(FileName, TEncoding.Default);
sl.SaveToFile(FileName, TEncoding.Unicode);
finally
sl.Free;
end;
end;
This is very simple to do with my TGpTextFile unit. I'll put together a short sample and post it here.
It should also be very simple with the new Delphi 2009 - are you maybe using it?
EDIT: This his how you can do it using my stuff in pre-2009 Delphis.
var
strAnsi : TGpTextFile;
strUnicode: TGpTextFile;
begin
strAnsi := TGpTextFile.Create('c:\0\test.udl');
try
strAnsi.Reset; // you can also specify non-default 8-bit codepage here
strUnicode := TGpTextFile.Create('c:\0\test-out.udl');
try
strUnicode.Rewrite([cfUnicode]);
while not strAnsi.Eof do
strUnicode.Writeln(strAnsi.Readln);
finally FreeAndNil(strUnicode); end;
finally FreeAndNil(strAnsi); end;
end;
License: The code fragment above belongs to public domain. Use it anyway you like.