All,
I am working on a new datasnap project based on the example project located in C:\Users\Public\Documents\Embarcadero\Studio\18.0\Samples\Object Pascal\DataSnap\FireDAC_DBX.
I am trying to transfer a large stream (1,606,408 bytes) from datasnap server to client. I am running into what appears to be a common issue and that is that the entire stream does not make it to the client.
Here is my server code:
//Returns Customer Info
function TServerMethods.GetBPInfo(CardCode : String): TStringStream;
begin
Result := TStringStream.Create;
try
qBPInfo.Close;
if CardCode.Trim = '' then
qBPInfo.ParamByName('ParmCardCode').AsString := '%%'
else
qBPInfo.ParamByName('ParmCardCode').AsString := '%' + CardCode + '%';
qBPInfo.Open;
FDSchemaAdapterBPInfo.SaveToStream(Result, TFDStorageFormat.sfBinary);
Result.Position := 0;
// Result.SaveToFile('output.adb');
except
raise;
end;
end;
Here is my client code:
procedure TdmDataSnap.GetBPInfo(CardCode : String);
var
LStringStream : TStringStream;
begin
dmDataSnap.FDStoredProcBPInfo.ParamByName('CardCode').AsString := CardCode;
FDStoredProcBPInfo.ExecProc;
LStringStream := TStringStream.Create(FDStoredProcBPInfo.ParamByName('ReturnValue').asBlob);
//LStringStream.Clear;
//LStringStream.LoadFromFile('Output.adb');
try
if LStringStream <> nil then
begin
LStringStream.Position := 0;
try
DataModuleFDClient.FDSchemaAdapterBP.LoadFromStream(LStringStream, TFDStorageFormat.sfBinary);
except
on E : Exception do
showmessage(e.Message);
end;
end;
finally
LStringStream.Free;
end;
end;
You will see the stream save and load code; that is how I determined that the server was getting the entire result set into the stream, and that the client could handle the entire result set and display it properly.
So smaller streams transfer just fine, but this big one, when examined in the ide debugger, does not start with the 65,66,68,83 characters and the load fails with the error, '[FireDAC][Stan]-710. Invalid binary storage format'.
I know from extended Googling that there are work-arounds for this, but I do not understand how to apply the workarounds to my case, with the use of Tfdstoredproc and TfdSchemaAdaptor components. I'm trying to stay with this coding scheme.
How do I adapt this code to correctly receive large streams?
Update 1:
Ok, I tried strings and Base64 encoding. It didn't work.
Client Code:
procedure TdmDataSnap.GetBPInfo(CardCode : String);
var
LStringStream : TStringStream;
TempStream : TStringStream;
begin
dmDataSnap.FDStoredProcBPInfo.ParamByName('CardCode').AsString := CardCode;
FDStoredProcBPInfo.ExecProc;
try
TempStream := TStringStream.Create;
TIdDecoderMIME.DecodeStream(FDStoredProcBPInfo.ParamByName('ReturnValue').asString,TempStream);
if TempStream <> nil then
begin
TempStream.Position := 0;
try
DataModuleFDClient.FDSchemaAdapterBP.LoadFromStream(TempStream, TFDStorageFormat.sfBinary);
except
on E : Exception do
showmessage(e.Message);
end;
end;
finally
TempStream.Free;
end;
end;
Here is my server code:
//Returns Customer Info
function TServerMethods.GetBPInfo(CardCode : String): String;
var
TempStream : TMemoryStream;
OutputStr : String;
begin
Result := '';
TempStream := TMemoryStream.Create;
try
try
qBPInfo.Close;
if CardCode.Trim = '' then
qBPInfo.ParamByName('ParmCardCode').AsString := '%%'
else
qBPInfo.ParamByName('ParmCardCode').AsString := '%' + CardCode + '%';
qBPInfo.Open;
FDSchemaAdapterBPInfo.SaveToStream(TempStream, TFDStorageFormat.sfBinary);
TempStream.Position := 0;
OutputStr := IdEncoderMIMEBPInfo.EncodeStream(TempStream);
Result := OutputStr
except
raise;
end;
finally
TempStream.Free;
end;
end;
The result is the same.
Related
I am in need of passing a couple things using Data Snap in Delphi XE6 (TStream & TClientdataSet). Lets start with a TStream - maybe what I learn here, I can figure out the TClientDataSet.
Here is my attempt, but it throws an error:
Remote error: Access violaion at address 0040801C in module DSServer.exe
Client demo (DSClient.exe)
//RunReportObj is the real object I will be passing to the server method
Procedure TForm8.Button1Click(Sender: TObject);
var
RunReportObj: TRunReportObject;
S: TStream;
FS: TFileStream;
begin
RunReportObj:= TRunReportObject.Create;
RunReportObj.ID:= '10101';
RunReportObj.ReportName:= 'Test';
RunReportObj.ExportType:= 'PDF';
S:= TStream.Create;
try
S:= ClientModule1.ServerMethods1Client.GetReport(RunReportObj);
S.Seek(0,soFromBeginning);
FS:= TFileStream.Create(RunReportObj.ReportName + '.' + RunReportObj.ExportType, fmOpenWrite);;
try
FS.CopyFrom(S, S.Size);
finally
FS.Free;
end;
finally
S.Free;
end;
end;
ClientClassesUnit1.pas
function TServerMethods1Client.GetReport(RunReportObj: TRunReportObject): TStream;
begin
if FGetReportCommand = nil then
begin
FGetReportCommand := FDBXConnection.CreateCommand;
FGetReportCommand.CommandType := TDBXCommandTypes.DSServerMethod;
FGetReportCommand.Text := 'TServerMethods1.GetReport';
FGetReportCommand.Prepare;
end;
if not Assigned(RunReportObj) then
FGetReportCommand.Parameters[0].Value.SetNull
else
begin
FMarshal := TDBXClientCommand(FGetReportCommand.Parameters[0].ConnectionHandler).GetJSONMarshaler;
try
FGetReportCommand.Parameters[0].Value.SetJSONValue(FMarshal.Marshal(RunReportObj), True);
if FInstanceOwner then
RunReportObj.Free
finally
FreeAndNil(FMarshal)
end
end;
FGetReportCommand.ExecuteUpdate;
Result := FGetReportCommand.Parameters[1].Value.GetStream(FInstanceOwner);
end;
Server demo (DSServer.exe)
//not really doing anything with the RunReportObj yet,
// just trying to test whether or not I can pass a TStream back first
function TServerMethods1.GetReport(RunReportObj: TRunReportObject): TStream;
var
Stream: TMemoryStream;
Writer: TBinaryWriter;
Bytes: TBytes;
begin
result := TMemoryStream.Create;
try
Writer := TBinaryWriter.Create(result);
try
Writer.Write(TEncoding.UTF8.GetBytes('Hello World' + sLineBreak));
finally
Writer.Free;
end;
finally
Stream.Free;
end;
end;
I'm sure I did something foolish :)
You have to take care who is responsible for freeing objects sent with DataSnap. TServerMethods1.GetReport() should not free the Result, as it has to be sent to the client first. On the other side, the client should not free the TStream it gets from TServerMethods1Client.GetReport(), as long as FInstanceOwner is true (which it is by default).
The first condition is fulfilled more by accident, although as David pointed out, you are freeing the uninitialized local variable Stream.
Without being able to actually test this in the moment, the correct code for the client should look like:
Procedure TForm8.Button1Click(Sender: TObject);
var
RunReportObj: TRunReportObject;
S: TStream;
FS: TFileStream;
begin
RunReportObj:= TRunReportObject.Create;
RunReportObj.ID:= '10101';
RunReportObj.ReportName:= 'Test';
RunReportObj.ExportType:= 'PDF';
S:= ClientModule1.ServerMethods1Client.GetReport(RunReportObj);
S.Seek(0,soFromBeginning);
FS:= TFileStream.Create(RunReportObj.ReportName + '.' + RunReportObj.ExportType, fmOpenWrite);;
try
FS.CopyFrom(S, S.Size);
finally
FS.Free;
end;
end;
And for the server side:
function TServerMethods1.GetReport(RunReportObj: TRunReportObject): TStream;
var
Writer: TBinaryWriter;
Bytes: TBytes;
begin
result := TMemoryStream.Create;
Writer := TBinaryWriter.Create(result);
try
Writer.Write(TEncoding.UTF8.GetBytes('Hello World' + sLineBreak));
finally
Writer.Free;
end;
end;
This program raises an I/O 104 error on EoF when first entering the while loop.
The purpose of the program is to look up if a username is already taken. The existing usernames are stored in a text file.
procedure TForm1.btnRegisterClick(Sender: TObject);
begin
sCUser := edtUserName.Text;
AssignFile(tNames, 'Names.txt');
begin
try
Reset(tNames);
except
ShowMessage('File not found');
Exit;
end;
end;
rewrite(tNames);
while not EoF(tNames) do // I get a I/O 104 Error here `
begin
Readln(tNames, sLine);
iPosComme := Pos(',', sLine);
sUser := Copy(sLine, 1, iPosComme - 1);
Delete(sLine, 1, iPosComme - 1);
if sCUser = sUser then begin
ShowMessage('Username taken');
end
else
begin
rewrite(tNames);
Writeln(tNames, sCUser + ',' + '0');
CloseFile(tNames);
end;
end;
end;
Remove the call to Rewrite()before Eof(). Even if you were not getting an IO error, your code would still fail because Rewrite() closes the file you opened with Reset() and then it creates a new bank file, so Eof() would always be True.
Update: error 104 is file not open for input, which means Reset() is not opening the file but is not raising an exception (which sounds like an RTL bug if Eof() is raising an exception, indicating that {I+} is active).
In any case, using AssignFile() and related routines is the old way to do file I/O. You should use newer techniques, like FileOpen() with FileRead(), TFileStream with TStreamReader, TStringList, etc...
Update: your loop logic is wrong. You are comparing only the first line. If it does not match the user, you are wiping out the file, writing the user to a new file, closing the file, and then continuing the loop. EoF() will then fail at that point. You need to rewrite your loop to the following:
procedure TForm1.btnRegisterClick(Sender: TObject
var
SCUser, sUser: String;
tNames: TextFile;
iPosComme: Integer;
Found: Boolean;
begin
sCUser := edtUserName.Text;
AssignFile(tNames,'Names.txt');
try
Reset(tNames);
except
ShowMessage('File not found');
Exit;
end;
try
Found := False;
while not EoF(tNames) do
begin
Readln(tNames,sLine);
iPosComme := Pos(',', sLine);
sUser := Copy(sLine ,1,iPosComme -1);
if sCUser = sUser then
begin
ShowMessage('Username taken') ;
Found := True;
Break;
end;
end;
if not Found then
Writeln(tNames,sCUser + ',0');
finally
CloseFile(tNames);
end;
end;
For the sake of completeness, this Version works for me, but it is hard to guess what the code is intended to do. Especially the while loop seems a bit displaced, since the file will contain exactly one line after the rewrite-case has ben hit once.
program wtf;
{$APPTYPE CONSOLE}
{$I+}
uses
SysUtils;
procedure Sample( sCUser : string);
var sUser, sLine : string;
iPosComme : Integer;
tnames : textfile;
begin
AssignFile(tNames,'Names.txt');
try
Reset(tNames);
except
Writeln('File not found');
Exit;
end;
while not EoF(tNames) do
begin
Readln(tNames,sLine);
iPosComme := Pos(',', sLine);
sUser := Copy(sLine ,1,iPosComme -1);
Delete( sLine,1, iPosComme -1);
if sCuser = sUser then begin
Writeln('Username taken') ;
end
else begin
Rewrite(tNames);
Writeln(tNames,sCUser + ',' + '0');
CloseFile(tNames);
Break; // file has been overwritten and closed
end;
end;
end;
begin
try
Sample('foobar');
except
on E: Exception do Writeln(E.ClassName, ': ', E.Message);
end;
end.
I wrote a version of this method that uses the newer TStreamReader and TStreamWriter classes.
This won't work with Delphi 7 of course, it's just to show how this could be done in newer versions of Delphi.
The code was heavily inspired by Remys answer.
procedure TForm1.btnRegisterClick(Sender: TObject);
var
Stream: TStream;
Reader: TStreamReader;
Writer: TStreamWriter;
Columns: TStringList;
UserName: string;
Found: Boolean;
FileName: string;
Encoding: TEncoding;
begin
FileName := ExpandFileName('Names.txt'); // An absolute path would be even better
UserName := edtUsername.Text;
Found := False;
Encoding := TEncoding.Default; // or another encoding, e.g. TEncoding.Unicode for Unicode
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
try
Reader := TStreamReader.Create(Stream, Encoding);
try
Columns := TStringList.Create;
try
Columns.Delimiter := ',';
Columns.StrictDelimiter := True; // or False, depending on the file format
while not Reader.EndOfStream do
begin
Columns.DelimitedText := Reader.ReadLine;
if Columns.Count > 0 then
begin
if AnsiSameStr(Columns[0], UserName) then // or AnsiSameText if UserName is not case-sensitive
begin
ShowMessage('Username taken') ;
Found := True;
Break;
end;
end;
end;
finally
Columns.Free;
end;
finally
Reader.Free;
end;
finally
Stream.Free;
end;
if not Found then
begin
Writer := TStreamWriter.Create(FileName, True, Encoding);
try
// Warning: This will cause problems when the file does not end with a new line
Writer.WriteLine(UserName + ',0');
finally
Writer.Free;
end;
end;
end;
If performance and memory usage are not a concern:
procedure TForm1.btnRegisterClick(Sender: TObject);
var
Rows: TStringList;
Columns: TStringList;
UserName: string;
Found: Boolean;
FileName: string;
Encoding: TEncoding;
Row: string;
begin
FileName := ExpandFileName('Names.txt'); // An absolute path would be even better
UserName := edtUsername.Text;
Found := False;
Encoding := TEncoding.Default; // or another encoding, e.g. TEncoding.Unicode for Unicode
Rows := TStringList.Create;
try
Rows.LoadFromFile(FileName, Encoding);
Columns := TStringList.Create;
try
Columns.Delimiter := ',';
Columns.StrictDelimiter := True; // or False, depending on the file format
for Row in Rows do
begin
Columns.DelimitedText := Row;
if Columns.Count > 0 then
begin
if AnsiSameStr(Columns[0], UserName) then // or AnsiSameText if UserName is not case-sensitive
begin
ShowMessage('Username taken') ;
Found := True;
Break;
end;
end;
end;
finally
Columns.Free;
end;
if not Found then
begin
Rows.Add(UserName + ',0');
Rows.SaveToFile(FileName, Encoding);
end;
finally
Rows.Free;
end;
end;
This solution can be adapted to Delphi 7 by removing the Encoding variable.
If it's part of a bigger database it should be stored in a real database management system rather than a text file.
I have a function declare like this :
function execProc(ProcName,InValues:PChar;out OutValues:PChar):integer; //The "OutValues" is a out parameter.
And I call this function like this:
procedure TForm1.Button6Click(Sender: TObject);
var
v:integer;
s:pchar;
begin
Memo1.Clear;
v := execProc(pchar('PROC_TEST'),pchar('aaa'),s);
showmessage(inttostr(v)); //mark line
Memo1.Lines.Add(strpas(s));
end;
when i delete the mark line(showmessage(inttostr(v))),i will have a correct result display in the Memo1,but if i keep use the showmessage(), the memo1 will dispaly an error string : "Messag" ,Why?
Thanks for any help!
function execProc(ProcName,InValues:PChar;out OutValues:PChar):integer;
var
str: TStrings;
InValue,OutValue: string;
i,j,scount: integer;
begin
Result := -100;
i := 0;
j := 0;
str := TStringList.Create;
try
sCount := ExtractStrings(['|'], [], InValues, str);
with kbmMWClientStoredProc1 do
begin
Close;
Params.Clear;
StoredProcName := StrPas(ProcName);
FieldDefs.Updated := False;
FieldDefs.Update;
for i := 0 to Params.Count - 1 do
begin
if (Params[i].ParamType = ptUnknown) or
(Params[i].ParamType = ptInput) or
(Params[i].ParamType = ptInputOutput) then
begin
inc(j);
InValue := str[j-1];
Params[i].Value := InValue;
end;
end;
try
ExecProc;
for i := 0 to Params.Count - 1 do
begin
if (Params[i].ParamType = ptOutput) or
(Params[i].ParamType = ptInputOutput) then
OutValue := OutValue + '|' + Params[i].AsString;
end;
OutValues := PChar(Copy(OutValue,2,Length(OutValue)-1));
Result := 0;
except
on E:Exception do
begin
if E.Message = 'Connection lost.' then Result := -101;//服务器连接失败
if E.Message = 'Authorization failed.' then Result := -102;//身份验证失败
Writelog(E.Message);
end;
end;
end;
finally
str.Free;
end;
end;
The problem is in the design of your interface and the use of PChar.
OutValues := PChar(Copy(OutValue,2,Length(OutValue)-1));
This is implemented by making an implicit, hidden, local string variable which holds the value
Copy(OutValue,2,Length(OutValue)-1)
When the function returns, that string variable is destroyed and so OutValues points at deallocated memory. Sometimes your program appears to work but that's really just down to chance. Any small change can disturb that, as you have observed.
The problem is easy enough to fix. Simply use string parameters rather than PChar. This will make the code easier to read as well as making it work correctly.
function execProc(ProcName, InValues: string; out OutValues: string): integer;
How is it possible to write to Delphi files, oem encoding?
How do you set the encoding? as string encoding is set by
setCodePage(RawBytes;Word;boolean);
You need the Windows API function CharToOemBuff().
EDIT
Inspired by #Free Consulting, the above API is what you would use in an older non-Unicode Delphi.
As #Free Consulting correctly points out, the new versions of Delphi offer extensive code page translation services. As a more modern variant of the old-style Pascal I/O, you could use a TStringList saved with a specified encoding.
Encoding := TEncoding.GetEncoding(GetOEMCP);
Try
StringList.SaveToFile('test.txt', Encoding);
Finally
Encoding.Free;
End;
I have a wrote a function that does that. It is no pretty, but it works.
function SetFileContent(aFileName: string; aFileContent: string; out aErrorMsg: string; aEncoding: TEncoding = nil; aRecreateFile: Boolean = True): Boolean;
var
vStream: TFileStream;
vCurEncoding: TEncoding;
vPreamble, vContent: TBytes;
vOffSet: Integer;
procedure _SetFileContent(aNewFile: Boolean);
begin
if aNewFile then begin
vStream := TFileStream.Create(aFileName, fmCreate);
try
vPreamble := aEncoding.GetPreamble;
If Length(vPreamble) > 0 then begin
vStream.WriteBuffer(Pointer(vPreamble)^, Length(vPreamble));
end;
vStream.WriteBuffer(Pointer(vContent)^, Length(vContent));
finally
vStream.Free;
End;
end
else begin
vStream := TFileStream.Create(aFileName, fmOpenWrite);
try
vStream.Position := vStream.Size;
vStream.WriteBuffer(Pointer(vContent)^, Length(vContent));
finally
vStream.Free;
end;
end;
end;
begin
Result := True;
try
vContent := BytesOf(aFileContent);
vCurEncoding := nil;
if aEncoding = nil then begin
aEncoding := TEncoding.Default;
end;
vOffSet := TEncoding.GetBufferEncoding(vContent, vCurEncoding);
if (vCurEncoding <> aEncoding) and aRecreateFile then begin
vContent := TEncoding.Convert(vCurEncoding, aEncoding, vContent, vOffSet, Length(vContent) - vOffSet);
end;
if FileExists(aFileName) then begin
if aRecreateFile then begin
_SetFileContent(True);
end
else begin
_SetFileContent(False);
end;
end
else begin
_SetFileContent(True);
end;
except
on E: Exception do begin
Result := False;
aErrorMsg := 'There was an error while trying to write the string ' + aFileContent + ' in the file ' + aFileName + '. Error: ' + E.Message;
end;
end;
end;
TStringList.SaveToFile() has a TEncoding parameter, you can use TEncoding.GetEncoding() to get an encoding object for any installed codepage, so you can specify the return value of GetOEMCP() for that. Or use TFileStream or FileWrite() to write to a file manually, and then use TEncoding.GetBytes() to encode String values during your writing.
With ease. It depends on file I/O method to be used. Example of one:
procedure TForm11.FormCreate(Sender: TObject);
var
S: string;
begin
DefaultSystemCodePage := GetOEMCP();
S := 'Если используешь мой код, то ты должен мне почку';
AssignFile(Output, 'license.txt');
Rewrite(Output);
Write(Output, S); // converts to single byte here
CloseFile(Output);
Application.Terminate;
end;
NO bum rap about Pascal I/O, please :-P
I'm making requests to the webaddress to get XML files throught the HTTPS connection. But this connection works like 50%. In most cases it fails. Usual error is "socket error #10060". Or "Error connecting with SSL. EOF was observed that violates the protocol". What I'm doing wrong?
function SendRequest(parameters: string): IXMLDocument;
var
sPostData: TStringList;
sHttpSocket: TIdHTTP;
sshSocketHandler: TIdSSLIOHandlerSocketOpenSSL;
resStream: TStringStream;
xDoc: IXMLDocument;
begin
sPostData := TStringList.Create;
try
sPostData.Add('add some parameter to post' + '&');
sPostData.Add('add some parameter to post' + '&');
sPostData.Add('add some parameter to post' + '&');
sPostData.Add(parameters);
sHttpSocket := TIdHTTP.Create;
sshSocketHandler := TIdSSLIOHandlerSocketOpenSSL.Create;
sHttpSocket.IOHandler := sshSocketHandler;
sHttpSocket.Request.ContentType := 'application/x-www-form-urlencoded';
sHttpSocket.Request.Method := 'POST';
resStream := TStringStream.Create;
sHttpSocket.Post(Self.sUrl, sPostData, resStream);
xDoc := CreateXMLDoc;
xDoc.LoadFromStream(resStream);
Result := xDoc;
resStream.Free;
sHttpSocket.Free;
sshSocketHandler.Free;
sPostData.Free;
except on E: Exception do
begin
TCommon.ErrorLog('errorLog.txt', DateTimeToStr(Now) + ' ' + E.Message);
end
end;
end;
Maybe I can do this in another way, that works like 100%, when internet connection is available?
Regards,
evilone
An "EOF" error suggests you are connnecting to a server that is not actually using SSL to begin with, or the SSL data may be corrupted.
Besides that, why are you including explicit '&' characters between your post data parameters? Don't do that, Indy will just encode them and send its own '&' characters. Also, consider using TMemoryStream instead of TStringStream to ensure IXMLDocumect.LoadFromStream() is loading the server's original raw XML data as-is, and not an altered version that the RTL/VCL produces due to Unicode handling (TStringStream is TEncoding-enabled).
Edit: Given the URL you provided, an example of calling verifyUser() would look like this:
const
ERPLYAccountCode = '...';
function verifyUser(const user, pass: string; const sessionLength: Integer = 3600): IXMLDocument;
var
sPostData: TStringList;
sHttpSocket: TIdHTTP;
sshSocketHandler: TIdSSLIOHandlerSocketOpenSSL;
resStream: TMemoryStream;
xDoc: IXMLDocument;
begin
Result := nil;
try
resStream := TMemoryStream.Create;
try
sPostData := TStringList.Create;
try
sPostData.Add('clientCode=' + ERPLYAccountCode);
sPostData.Add('request=verifyUser');
sPostData.Add('version=1.0');
sPostData.Add('responseType=XML');
sPostData.Add('responseMode=normal');
sPostData.Add('username=' + user);
sPostData.Add('password=' + pass);
sPostData.Add('sessionLength=' + IntToStr(sessionLength));
sHttpSocket := TIdHTTP.Create;
try
sshSocketHandler := TIdSSLIOHandlerSocketOpenSSL.Create(sHttpSocket);
sHttpSocket.IOHandler := sshSocketHandler;
sHttpSocket.Request.ContentType := 'application/x-www-form-urlencoded';
sHttpSocket.Post('https://www.erply.net/api/', sPostData, resStream);
finally
sHttpSocket.Free;
end;
finally
sPostData.Free;
end;
resStream.Position := 0;
xDoc := CreateXMLDoc;
xDoc.LoadFromStream(resStream);
Result := xDoc;
finally
resStream.Free;
end;
except
on E: Exception do
begin
TCommon.ErrorLog('errorLog.txt', DateTimeToStr(Now) + ' ' + E.Message);
end;
end;
end;