I am trying to create the upload PUT request for the OneDrive API. It's the large file "resumable upload" version which requires the createUploadSession.
I have read the Microsoft docs here: As a warning the docs are VERY inaccurate and full of factual errors...
The docs simply say:
PUT
https://sn3302.up.1drv.com/up/fe6987415ace7X4e1eF866337Content-Length:
26Content-Range: bytes 0-25/128 <bytes 0-25 of the
file>
I am authenticated and have the upload session created, however when I pass the JSON body containing my binary file I receive this error:
{ "error": {
"code": "BadRequest",
"message": "Property file in payload has a value that does not match schema.", .....
Can anyone point me at the schema definition? Or explain how the JSON should be constructed?
As a side question, am I right in using "application/json" for this at all? What format should the request use?
Just to confirm, I am able to see the temp file created ready and waiting on OneDrive for the upload, so I know I'm close.
Thanks for any help!
If you're uploading the entire file in a single request then why do you use upload session when you can use the simple PUT request?
url = https://graph.microsoft.com/v1.0/{user_id}/items/{parent_folder_ref_id}:/{filename}:/content
and "Content-Type": "text/plain" header and in body simply put the file bytes.
If for some reason I don't understand you have to use single-chunk upload session then:
Create upload session (you didn't specified any problems here so i'm not elaborating)
Get uploadUrl from createUploadSession response and send PUT request with the following headers:
2.1 "Content-Length": str(file_size_in_bytes)
2.2 "Content-Range": "bytes 0-{file_size_in_bytes - 1}/{file_size_in_bytes}"
2.3 "Content-Type": "text/plain"
Pass the file bytes in body.
Note that in the PUT request the body is not json but simply bytes (as specified by the content-type header.
Also note that max chuck size is 4MB so if your file is larger than that, you will have to split into more than one chunks.
Goodlcuk
Related
The official documentation does not provide an example for any SDK's (including the Java SDK): https://learn.microsoft.com/en-us/graph/api/user-sendmail?view=graph-rest-1.0&tabs=java#example-4-send-a-new-message-using-mime-format. As there is no example, I have tried in vain to send the MIME content using the SDK (microsoft-graph 5.0.0):
Message sending = new Message();
ItemBody body = new ItemBody();
final String mimeMessageRFC822 = input.getMimeMessageRFC822();
body.content = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
sending.body = body;
GraphServiceClient service = getService(acHost, configuration);
service
.me()
.sendMail(UserSendMailParameterSet.newBuilder().withMessage(sending).withSaveToSentItems(true).build())
.buildRequest(new HeaderOption("Content-Type", "text/plain"))
.post();
The above code sets the request's content-type to text/plain, however the request body that is being sent is JSON (xxxxxx below is a placeholder for a valid Base64 encoded MIME content string).
{
"message":
{
"body":
{
"content": xxxxxx
}
},
"saveToSentItems": true
}
The response is a 404, stating:
GraphServiceException: Error code: ErrorMimeContentInvalidBase64String
Error message: Invalid base64 string for MIME content.
I can understand why it is responding with this error as the graph endpoint is parsing the text/plain content as base64 encoded MIME but finds the JSON structure instead. I have been on a video call with a Microsoft Graph support agent, and they have seen that my MIME content is valid. Sadly, they are not able to help with the Microsoft Graph Java SDK even though it is developed by Microsoft!
This suggests that we are not supposed to use the Java SDK at all for sending MIME formatted emails. Is this correct? Surely it can't be otherwise what is the point of a library that can receive MIME formatted emails but can't send them? Does anyone have a solution?
For now at least the solution is to send a CustomRequest with MIME content instead of using the fluent API provided by the Graph client.
final String encodedContent = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
CustomRequest<String> request = new CustomRequest<>(requestUrl, service, List.of(new HeaderOption("Content-Type", "text/plain")), String.class);
request.post(encodedContent);
Here is the code which is working using Net::HTTP::Post
request = Net::HTTP::Post.new(url)
...
form_data = [
['attachments[]', File.open('file1.txt')],
['attachments[]', File.open('file2.txt')]
]
request.set_form form_data, 'multipart/form-data'
http.request(request)
Now, I am trying to use httparty like below but it is not working.
body = { attachments: [ File.open('file1.txt'), File.open('file2.txt') ] }
HTTParty.post(url, body: body)
The response I am getting from web service call is below:
#<HTTParty::Response:0x557d7b549f90 parsed_response={"error"=>true, "error_code"=>"invalid_attachment", "error_message"=>"Attachmen
t(s) not found or invalid."}, #response=#<Net::HTTPBadRequest 400 Bad Request readbody=true>, #headers={"server"=>["nginx"], "date"=>[
"Mon, 20 May 2019 07:41:50 GMT"], "content-type"=>["application/json"], "content-length"=>["102"], "connection"=>["close"], "vary"=>["
Authorization"], "set-cookie"=>["c18664e1c22ce71c0c91742fbeaaa863=uv425hihrbdatsql1udrlbs9as; path=/"], "expires"=>["Thu, 19 Nov 1981
08:52:00 GMT", "-1"], "cache-control"=>["no-store, no-cache, must-revalidate", "private, must-revalidate"], "pragma"=>["no-cache", "no
-cache"], "x-ratelimit-limit"=>["60"], "x-ratelimit-remaining"=>["59"], "strict-transport-security"=>["max-age=63072000; includeSubdom
ains;"]}>
It looks like it is not able to read the contents of files. Does HTTParty support this or I need to use some other gem?
Something like this should work, I just tested it, worked for me no problem.
HTTParty.post(url,
body: { attachments: [
File.read('foo.txt'),
File.read('bar.txt')] })
With HTTParty you can pass IO/Files as parameters the same way (multipart is automatically set to true if there's a file in parameters).
But keep in mind that files should be closed after upload, otherwise you may run out of file descriptors before GC collects them:
files = ['file1.txt', 'file2.txt'].map{|fname| File.open(fname) }
begin
HTTParty.post(url, body: { attachments: files })
ensure
files.each(&:close)
end
That should work for you if net/http variant does (and is actually the same as your code).
Other thing to look at is content type detection by filename - because file upload consists of filename, content type and data itself.
Error 400 with "invalid_attachment" you're getting suggests that more probably it's related to content type or other validation on server side (so make sure you're testing with the same files and nothing else changes other than http lib), also check httparty to be a recent version
I've written a test program which sends the same multipart request using both Net::HTTP and HTTParty. Then it compares and prints the request strings so that we can compare them. The only substantive difference between the two requests is that HTTParty attempts to guess and set the Content-Type header (e.g. text/plain for a file named file1.txt), whereas Net::HTTP always uses application/octet-stream.
HTTParty definitely does read the files and send them in the request. So, I suggest you investigate if the server is returning an error due to the Content-Type (maybe the content type in your particular request is not supported).
For your reference, here is the test program and specific results.
I am trying to send a csv file from snaplogic to an api endpoint using through a REST POST snap but it doesn't seem to actually send the file as the response says file not found
when I do the post through postman it just fine and use the same information I use in the snap
my settings in the POST snap are as follows:
Label*= REST Post Service URL*=http://(ip:port)/v1/innovazones HTTP
entity=(blank) Batch size=(blank) Show all headers(selected) Single
file upload: File=…/shared/TEMPcsv.csv Single file upload: File
key=file Single file upload: Filename to be used=innovaZones_csv_file
Upload transfer request type=calculate content length Upload body
type=Multipart form data Single file upload: Multipart Content-Type=text/csv
HTTP header:
Key=Content-Type;Value=application/x-www-form-urlencoded
key=Authorization Value="Bearer " + account.access_token
Trust all certificates*(checked) Follow redirects*(checked)
I should be getting this message:
{
"status": "success",
"message": "Data transmitted successfully",
"payload": {}
}
but I get this in the response instead:
"error": "REST API service endpoint returned error result: status code = 400, reason phrase = Bad Request"
"error_entity": {
"status":"fail",
"message":"File not found",
"payload":{
"message":"File not found",
"statusCode":400,
"status":"fail"
}
}
"original": {filename:../shared/TEMPcsv.csv, result:overwritten, original:{, ...}}
"filename": "../shared/TEMPcsv.csv"
"result": "overwritten"
You have not added the actual files to upload. You have only set the name of the file.
As per the SnapLogic documentation -
Single file Upload: Filename to be used
This property can be configured if you are uploading a file, the value entered in this field will be treated as the name for the file being uploaded. You can retain the original file name by leaving this property blank.
What you also need to do is point to the actual file you want to upload in the Upload file(s) section.
Again, from SnapLogic documentation -
Upload File(s)
Lists out the file keys and file names of the files that you want to upload. By default, the Settings popup displays the fields required to upload a single file. Click the + button to add new rows to this fieldset. For each file that you want to upload, you need to specify the upload-file key, the location and name, the name of the file in the destination directory, and the multi-part content type associated with the file that you want to upload.
Please refer to - SnapLogic Docs - REST POST
Now i'm working with GoogleDrive API for iOS ( i don't use the Objective-C client library ) and I can update file's data only by "media" uploadType, file's data and/or file's meta-data by "multipart" uploadType, but I can't update anything with "resumable" uploadType.
By google's API references, request update file uses method PUT, but request with "resumable" uploadType uses method POST in the first request to send meta-data of file only, the second request with PUT method send data of file, this make me confused so much. Anyone has any suggestion for me T_T ?
Finally I did it:
Request 1:
Header:
PUT https://www.googleapis.com/upload/drive/v2/files?uploadType=resumable
Content-Type: application/json; charset=UTF-8
X-Upload-Content-Type: video/mp4
X-Upload-Content-Length: 7421411
Body:
{
Meta-data of files to upload
}
After received upload_id from Location header of response 1, call the 2nd request:
Header:
PUT https://www.googleapis.com/upload/drive/v2/files?uploadType=resumable&upload_id=adZc...
Content-Length: 7421411
Content-Type: video/mp4
Body:
{
bytes of file to upload
}
Are you using XMLHTTP request or using the Google Drive SDK API. request = gapi.client.request(); and request.exeute().
Most important question is are you able to send the next chunk. What is the point of doing an resumable upload with the full set of bytes. The point for resumable upload is to use chunking.
Upload is resumable from the next chunk, i.e technically on the next chunk, the filesize should increase. I don't mind if the server doesn't give me number of bytes received. I need to know if the file is increasing. If the file size is increasing, I can simply offset the bytes to upload to the diff of filesize and the data I send.
If by definition of what you are doing, resumable upload is supported by default. All you have to do is use:
'path': '/upload/drive/v2/files/'+ NewFileID,
the NewFileID can by any file ID old or new. It will still update the file with new bytes.
So, I am not sure what is the advantage of you are doing? Did you figure out how to send the next chunk (256KB default) and confirm that the filesize on the google drive change to a value:
First ChunkSize + + New ChunkSize +
First ChunkSize : your initial chunk that you put. Example 256KB
: This is space allocated by Google drive for storing its metadata.
New ChunkSize: You second Chunck size (which should be default 256KB again)
: This is again metadata stored by Google drive for storing the next chunk.
So total size by the time both are processed should be > 512KB
I'm trying out http requests to download a pdf file from google docs using google document list API and OAuth 1.0. I'm not using any external api for oauth or google docs.
Following the documentation, I obtained download URL for the pdf which works fine when placed in a browser.
According to documentation I should send a request that looks like this:
GET https://doc-04-20-docs.googleusercontent.com/docs/secure/m7an0emtau/WJm12345/YzI2Y2ExYWVm?h=16655626&e=download&gd=true
However, the download URL has something funny going on with the paremeters, it looks like this:
https://doc-00-00-docs.googleusercontent.com/docs/securesc/5ud8e...tMzQ?h=15287211447292764666&\;e=download&\;gd=true
(in the url '&\;' is actually without '\' but I put it here in the post to avoid escaping it as '&').
So what is the case here; do I have 3 parameters h,e,gd or do I have one parameter h with value 15287211447292764666&ae=download&gd=true, or maybe I have the following 3 param-value pairs: h = 15287211447292764666, amp;e = download, amp;gd = true (which I think is the case and it seems like a bug)?
In order to form a proper http request I need to know exectly what are the parameters names and values, however the download URL I have is confusing. Moreover, if the params names are h,amp;e and amp;gd, is the request containing those params valid for obtaining file content (if not it seems like a bug).
I didn't have problems downloading and uploading documents (msword docs) and my scope for downloading a file is correct.
I experimented with different requests a lot. When I treat the 3 parameters (h,e,gd) separetaly I get Unauthorized 401. If I assume that I have only one parameter - h with value 15287211447292764666&ae=download&gd=true I get 500 Internal Server Error (google api states: 'An unexpected error has occurred in the API.','If the problem persists, please post in the forum.').
If I don't put any paremeters at all or I put 3 parameters -h,amp;e,amp;gd, I get 302 Found. I tried following the redirections sending more requests but I still couldn't get the actual pdf content. I also experimented in OAuth Playground and it seems it's not working as it's supposed to neither. Sending get request in OAuth with the download URL responds with 302 Found instead of responding with the PDF content.
What is going on here? How can I obtain the pdf content in a response? Please help.
I have experimented same issue with oAuth2 (error 401).
Solved by inserting the oAuth2 token in request header and not in URL.
I have replaced &access_token=<token> in the URL by setRequestHeader("Authorization", "Bearer <token>" )