I've built a DataSnap REST server using Delphi XE2 and I've added a server method for uploading files via TStream :
function TServerMethods.updateUploadFile(sFilename: string; UploadStream: TStream): string;
I want to be able to call this from a number of different clients (Android, iOS etc) and I've been testing the method using various REST clients such as Postman (Chrome plugin). However so far I cannot get it to accept the content for the HTTP POST body. Whenever I send the POST command I always get the following response :
{"error":"Message content is not a valid JSON value."}
I've tried using various different 'Content-Type' settings but nothing seems to work. It looks as though DataSnap is insisting on the content being JSON? I've tried pasting some valid JSON into the content area but this also gave the same error response.
Has anybody successfully used TStream as an input parameter for a DataSnap server method? Should I be doing it another way? I've used TStream as an output parameter for downloading files many times and it works well, this is my first attempt at uploading.
UPDATE
I made a quick Delphi DataSnap client to test the uploadFile server method and this all works great. I then used Fiddler to examine the POST command the Delphi client uses to send the TStream in the content body, and noticed it is a JSON array of integers (bytes) e.g. [37,80,68,70,45,49,46,51,13,10]. So I can see that I could modify my Android/iOS clients to convert the binary data to this byte array format before POSTing, but this is an overhead I could do without. If DataSnap streams raw binary when TStream is a return parameter, why can't it stream raw binary as an input parameter?
It seems when adding content data to the request body in a POST command, DataSnap server insists that this data is always JSON. This is why when using TStream as an input parameter, the stream data is converted to a JSON array of integers (bytes) by the Delphi DataSnap client. This format is very size inefficient as with upto 3 digits per byte, plus the comma, the size of the data being uploaded can grow by as much as 4 times.
What I have therefore done instead is to encode the data to upload in Base64. My server method now looks like this :
function TServerMethods.updateUploadFile(sFilename: string; Base64Data: TJSONObject): string;
Notice I'm wrapping the Base64 string in a TJSONObject. This is because if you just specify a String type, the Delphi DataSnap client will call the method with a GET and attempt to put the whole Base64 string in the URL path, causing a 'Connection Closed Gracefully' error. Using a TJSONObject forces DataSnap to use a POST and put the data in the content body. The JSON object passed is a single pair object :
{"UploadedData":"JVBERi0xLjMNCiXi48B5SiWGTK3PaY.........."}
This way the size of the data uploaded is much smaller and faster to transfer. I'd still prefer to be able to stream the raw data in the content body but DataSnap doesn't allow this.
Related
first of all I'd like to thank the team for this amazing project, it is indeed exiting to be able to start writing server-side software in Swift.
I'm successfully running a POC using PerfectServer on an Ubuntu VM and working on the API to interact with the mobile client.
There is one aspect I didn't quite understand yet, and that is accessing the request body data from my PerfectServer Handler.
Here is the workflow I have in mind:
The client submits a POST request to PerfectServer including some
JSON encoded body data
Once that hits the "valuesForResponse:" of
my server side Handler, I retrieve the WebRequest representation of
my request successfully
The request object does expose a many
properties of the HTTP request, including headers and the url-like
formatted query parameters.
Unfortunately, I cannot see a way to retrieve the underlying request body data. I would expect that to be some kind of public properties exposing the raw data that my handle can retrieve and decode in order to process the request.
The only example provided in the Examples workspace that comes with the project and sends a POST request that includes a body is in the project Authenticator. Here the HTTP body part takes the form os a UTF-8 encoded string where the values are query-params-like formatted.
name=Matteo&password=mypassword
This gets somehow exposed on the server handler by the WebRequest "param" property, that in the inner implementation of HTTPServer seems to expect an "&" separated string of key-values:
What I would expect is to have a way to provide body data in whatever form / encoding needed, in my case a JSON form:
{"name":"Matteo", "password":"psw"}
and be able to access that data from the WebRequest in my handler, decode it and use it to serve the request.
To summarise, I assume you could say that a WebRequest.bodyData public property is what I am after here :).
Is there something I am missing here?
Thanks in advance for any clarification!
With IdFTP, the server i'm connecting to is not using UTF-8, but ANSI. There's nothing special about my code, i simply set Host, Username, Password and Connect to server. Then i call List method with no parameters. Iterating through DirectoryListing gives me incorrect results for file names. My sample directory name encoded in local code page (CP-1250) is:
aąęsśćńółżźz
I thought i'll be able to "fix" file name field by converting it to AnsiString and setting code page but it seems to be already broken - memory dump of DirectoryListing[I].FileName:
a ? ? s ? ? ? ?? ?? z
6100 FDFF FDFF 7300 FDFF FDFF FDFF 8FDB DFDF 7A00
Manipulating with GIdDefaultAnsiEncoding or IOHandler.DefStringEncoding (after Connect, before List) makes no difference. I don't want to mess in IdFTP or IdGlobal code because i'm using it with other projects that involve Unicode and these works perfectly. Delphi XE2 or XE7.
As you can see FData contains raw file name in a 2 bytes per char string:
Even if i set IOHandler.DefStringEncoding to any TIdTextEncoding that is FIsSingleByte = True, FMaxCharSize = 1. However it looks promising because #$009F is "ź" in CP-1250, but i'm not looking for a per server, temporary solution. I expected Indy to handle this correctly after setting IOHandler.DefStringEncoding and GIdDefaultAnsiEncoding based on server capabilities (UTF-8 or ANSI with specified encoding).
Total Commander connection log:
Your server supports the MLSD command. Total Commander is sending the MLSD command and not the older LIST command. This is good, because MLSD has a standardized format (see RFC 3659), which includes support for embedded charset information. If no charset is explicitly stated, UTF-8 must be used.
You did not show the command/response log for TIdFTP, but the fact that the TIdFTPListItem.Data property is showing MLSD formatted output data means TIdFTP.List() is also using the MLSD command (by calling TIdFTP.ExtListDir() internally). The output shown does not include an explicit charset attribute, so TIdFTP will decode the filename as UTF-8.
However, the raw filename data that is shown in the TIdFTPListItem.Data property is NOT the correct UTF-8 encoded form of the directory name you have shown (even when stored as a raw 8-bit encoded UnicodeString - which is what TIdFTP.ExtListDir() does internally before parsing it). So the problem is either:
your FTP server is not converting the directory name from CP-1250 to UTF-8 correctly in the first place. Considering that Total Commander appears to be able to handle the listing correctly, this is not likely.
TIdFTP is not storing the raw UTF-8 octet data correctly before parsing it. This is more likely.
Hard to say which is actually the case since you did not show the raw listing data that is actually being transmitted. And you did not specify which exact version of Delphi and Indy you are using, either. Assuming the server is transmitting UTF-8 correctly, you might simply be using an older Indy version that does not handle the UTF-8 transmission correctly. AFAIK, the current version available (10.6.2.5270 at the time of this writing) should be able to handle it, as long as you are using Delphi 2009 or later. If you can provide a Wireshark capture of the raw listing data, I can check if there are any logic issues in TIdFTP that need to be fixed or not.
My team was looking for quick solution that i had to provide. My solution is based on this post: http://forums2.atozed.com/viewtopic.php?p=32301#p32301 and this question: Converting UnicodeString to AnsiString
Once FTP listing is finished i do overwrite FileName property via function that extracts file name from Data, and then convert String to RawByteString with correct code page. Fix is applied only if server doesn't support UTF-8. This way i'm able to move around FTP - ChangeDir, Get, Put etc. without problems.
there exist some sample code for an Http Server in the Dart:io section.
Now I will distribute images with this server. To achieve this, I read the requested image file and send its content to the client via request.response.write().
The problem is the format of the read data:
Either I read the image file as 16bit-String or as Byte Array. Neither of them is compatible to a raw 8-bit array, which I have to send to the client.
May someone help me?
There exist several kinds of write-methods in the response class.
write
writeCharCode
add
While "write" writes the data 'as seen', "writeCharCode" transforms the data back to raw-format. However, writeCharCode prepends some "magic byte" (C2) at the beginning, so it corrupts the data.
Another function, called add( List < int > ) processes the readAsBytes-result as desired.
Best regards,
Alex
On my code I got a image from camera but I want to upload it on the server, I convert the image to byte array I sent it to the url of php server can any one tell me Which type of code I have to write.
One problem is that the byte array data is 11 character in length my PM told me that the byte you got is too small
I got the byte array as follows
[B#f359616f
when i run this code at php side the imagecreatefromstring($images); not create the image
Any code plz help me
It looks like you're calling 'toString()' on a byte array. This means you are calling the default implementation of toString provided by the VM, which is to format the type name, then '#' then some object specific int, likely the object identity hash code.
Typically, StackOverflow questions include source code. In this case, you would want to include the function that is uploading the image data. People helping you would be interested in the datatypes involved, and how you are formatting and delivering that data to the server's php script.
in my application ,i am sending data from my application to database
i am getting some odd characters in my database like this
i am sending my data like this
var
w:widestring;
u:utf8string;
begin
w:=data //data is function to get some info(string)
u:=utf8encode(w);
sendfn(u);
end;
i am using utf8_decode(my get data) in my php code before adding to my database.
and my database and tables collation is utf8_general_ci
can anyone help me in this issue
It's an educated guess, but does the Data function return an UTF-8 string instead of a WideString? I think the error could be in this Data function that you're calling, which returns the data in the wrong string format.
The php function utf8_decode converts characters from utf-8 to ISO-8859-1. If the path your data take (the browser or whatever component you use to send your data to the web server (http request, your php installation, your webpage and your database connection) from your delphi app to your database that is behind you webpage are able to support and configured to use utf-8 data you don't need the utf8_decode function, you can just insert your data the way it comes.
If you haven't already configured php to work with UTF-8, be aware that it is difficult and never works 100% (for me at least it never did), so maybe it would be better for you to use data in your locale encoding.