I am trying to POST files containing arbitrary binary data to a WebAPI 2 controller as multipart/form-data.
When trying to read the files using the following:
var provider = new MultipartFormDataStreamProvider(uploadDirectoryPath);
await Request.Content.ReadAsMultipartAsync(provider);
I get:
System.InvalidOperationException: The character set provided in ContentType is invalid.
Cannot read content as string using an invalid character set.
---> System.ArgumentException: '"utf-8"' is not a supported encoding name.
For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.
Parameter name: name
I understand that the problem is the quotes around utf-8 and it is this bug in HttpContent.ReadAsStringAsync() used internally as explained here.
The request is being sent from a Unity3D app and I am unable to change the charset in the request body.
Here is the code I am using in the Unity3D script to create the POST request.
WWW chunkFile = new WWW("file:///" + Path.Combine(tempZipFolderPath, chunkFilenames[j]));
yield return chunkFile;
if (chunkFile.error != null)
throw new Exception(string.Format("{0} error opening file {1}", chunkFile.error, chunkFilenames[j]));
//BUGFIX: Not using IMultipartFormSection because of https://fogbugz.unity3d.com/default.asp?826626_htlchp13nh8th2to
var form = new WWWForm();
form.AddField("chunkNumber", chunkNumber);
form.AddField("totalChunks", totalChunks);
form.AddField("identifier", tempZipFileName);
form.AddBinaryData("filename", chunkFile.bytes, chunkFilenames[j]);
var scanUploadPostRequest = UnityWebRequest.Post(WebApiScanUploadUrl, form);
yield return scanUploadPostRequest.Send();
//BUGBUG: charset UTF-8 wrapped in quotes causes error
Is there a workaround to this?
Related
I have developed a REST API using feathers.js (https://feathersjs.com/).
When trying to do a HTTP 'read' request in Flutter using package:http/http.dart I have encountered an error. The http.dart library is unable to correctly parse the query params I pass to the URI.
The error I receive through the Android Studio debug console is ;
FormatException: Invalid character (at character 84) E/flutter
(11338): ...lk.com/weatherobs?last=true&location[$in]=Bellambi&location[$in]=Nowra ....
The error is indicating the square brackets and possibly the $ sign ('[$in]' ) are the issue.
_getDemoRequest() {
String url = r"http://demoapi.ap-southeast-2.elasticbeanstalk.com/weatherobs?last=true&location[$in]=Bellambi&location[$in]=Nowra&location[$in]=Sydney Airport&location[$in]=Thredbo Top Station&location[$in]=Hobart&location[$in]=Coolangatta";
http.read(url).then(print);
}
In the URL I have tried prefixing the String with and without 'r' for a raw string to no avail.
I have also tried using httpClient with params with no success and the exact same error on the square brackets eg '[$in]'
String httpbaseUri = "http://xxxx.ap-southeast-2.elasticbeanstalk.com";
String qParams = r"?last=true&location[$in]=Bellambi&location[$in]=Nowra";
String path = "/weatherobs";
var _uri = new Uri.http(baseUri, path, qParams);
await httpClient.read(_uri, headers: {"Accept": "application/json"});
As a person with approximately 3 weeks of Flutter/Dart experience I believe its an elementary problem, but one in which several hours of research has uncovered no solution.
The ways the URI query parameters are structured (with the square brackets ie [$in]) are dictated by the feathers.js framework.
Any help would be appreciated.
It has been brought to my attention in another thread https://stackoverflow.com/questions/40568/are-square-brackets-permitted-in-urls :
That the URL Specification RFC 3986 generally does not permit square brackets in an URL.
My question was triggered as the get request works as intended in Postman, Chrome Browser and also javascript applications using axios.js, but not in an application developed in Flutter/Dart using standard http.read methods.
It doesn't look like [] are supported in the URL (except for the host IP for IPv6). See Are square brackets permitted in URLs?.
Please check if the API accepts them when they are encoded like:
void main() {
var url = r'http://demoapi.ap-southeast-2.elasticbeanstalk.com/weatherobs';
var locationKey = Uri.encodeQueryComponent(r'location[$in]');
var qParams = 'last=true&$locationKey=Bellambi&$locationKey=Nowra&$locationKey=Sydney Airport&$locationKey=Thredbo Top Station&$locationKey=Hobart&$locationKey=Coolangatta';
try {
print(Uri.parse(url).replace(query: qParams));
} catch(e) {
print(e);
}
}
DartPad example
See also api.dartlang.org/stable/1.24.3/dart-core/Uri/…
You can use this flutter package which allow you to communicate with your feathers js server from flutter app as said in: https://stackoverflow.com/a/65538226/12461921
I'm trying to delete entries in my lightswitch database, from a external app.
I'm using restsharp, and the code looks like the following:
var request = new RestRequest("/FilesSet/{Id}", Method.DELETE);
request.AddParameter("Id", 8);
var resp = client.Execute(request);
Here is the full error message:
The request URI is not valid. Since the segment 'FilesSet' refers to a collection, this
must be the last segment in the request URI. All intermediate segments must refer to a
single resource.
It seems like you're trying to locate an entity and delete it. In the OData URL convention, the canonical URL for accessing an entity in a collection is as follows:
~/FilesSet({Id})
Thus, you need to modify your code as follows:
var request = new RestRequest("/FilesSet({Id})", Method.DELETE);
request.AddParameter("Id", 8);
var resp = client.Execute(request);
Reference: 4.3.1 Canonical URL
I've had success creating objects with POST and Content-Type application/xml
I've also had success querying using Content-Type application/x-www-form-urlencoded with a blank request body which returns all of the object type depending on which URI I specify.
I can also get the same to work with something like PageNum=1&ResultsPerPage=1 in the request body and I have figured out how to incorporate that into the signature so I get a valid response.
However no matter how I format it, I cannot get anything other than a 401 response when I try to use a filter (something basic like Filter=FAMILYNAME :EQUALS: Doe). I've read over the OAuth Core 1.0 Revision A specifications on how all parameter names and values are escaped using the [RFC3986] percent-encoding. However I feel like I'm missing a step or formatting incorrectly. I've seen inconsistent information in my searching through Intuit's forums on what exactly is the proper format.
Any help on this would be greatly appreciated. I've been struggling with this for a good week now.
The response I get when trying to use a filter is:
HTTP Status 401 - message=Exception authenticating OAuth; errorCode=003200; statusCode=401
----Update----
I'm am seeing the same error when I try to use filters with the New IPP Developer Tools - IPP API Explorer. I'm using the IDS V2 QBO API Explorer. I'm able to use that tool to do a retrieve all Post and the response shows all of my customers, but when I try to use a filter I get :
Server Error
401 - Unauthorized: Access is denied due to invalid credentials.
You do not have permission to view this directory or page using the credentials that you supplied.
Any Ideas? If I'm getting the same error from the API Explorer tool, it makes me think the problem is something else entirely.
----Final Update----
I have finally had success with filters and I believe I have figure out what my problem was. I was always suspicious that I was able to get queries with pagination like "PageNum=1&ResultsPerPage=1" to work, but could not get something like "Filter=FAMILYNAME :EQUALS: Doe". I suspected there problem was with the white space in the filter format. What threw me off tracking this down earlier was that I could not get the filters to work in the IDS V2 QBO API Explorer. That made me suspect there was something else going on. I decided to ignore the API Explorer all together and focus on why I could get it to work the one way but no the other.
I believe my problem came down to improper encoding of the Filter's value in the signature. That explains the 401 invalid signature errors I was getting.
"Filter=Name :EQUALS: Doe" becomes "Filter=Name%20%3AEQUALS%20%3ADoe" after normalization.
Percent-Encoding that should give "Filter%3DName%2520%253AEQUALS%2520%253ADoe".
In essence you have to "double" encode the blank space and the colons, but not the equal sign. I tried many permutations of doing the encoding, but believe my mistake was that I was either not "double" encoding, or when I was double encoding I was including the '=' sign. Either way breaks your signature. Thanks for everyone's input.
I believe my problem came down to improper encoding of the Filter's value in the signature. That explains the 401 invalid signature errors I was getting.
I used an online tool to take me through the steps in properly signing an Oauth request. While going through those steps I realized my problem was with the steps where you normalize the request parameters and then percent-encode them. I was including the '=' of the filter in the normalization step, which breaks your signature. The tool I used can be found at:
http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/
Thanks for everyone's input.
Do you get a 401 with the same request in the API Explorer?
http://ippblog.intuit.com/blog/2013/01/new-ipp-developer-tool-api-explorer.html
Also, are you using the static base URL or retrieving it at runtime?
https://ipp.developer.intuit.com/0010_Intuit_Partner_Platform/0050_Data_Services/0400_QuickBooks_Online/0100_Calling_Data_Services/0010_Getting_the_Base_URL
If you are using the static base URL, try switching to the runtime base URL to see if you still get the error.
peterl answered one of my questions on here that may also answer yours. I had been trying to put the Filters in the body when they should have gone into the header. Here was peterl's code sample for getting all unpaid invoices (open balance greater than 0.00) for a particular customer.
http://pastebin.com/raw.php?i=7VUB6whp
public List<Intuit.Ipp.Data.Qbo.Invoice> GetQboUnpaidInvoices(DataServices dataServices, int startPage, int resultsPerPage, IdType CustomerId)
{
StringBuilder requestXML = new StringBuilder();
StringBuilder responseXML = new StringBuilder();
var requestBody = String.Format("PageNum={0}&ResultsPerPage={1}&Filter=OpenBalance :GreaterThan: 0.00 :AND: CustomerId :EQUALS: {2}", startPage, resultsPerPage, CustomerId.Value);
HttpWebRequest httpWebRequest = WebRequest.Create(dataServices.ServiceContext.BaseUrl + "invoices/v2/" + dataServices.ServiceContext.RealmId) as HttpWebRequest;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.Headers.Add("Authorization", GetDevDefinedOAuthHeader(httpWebRequest, requestBody));
requestXML.Append(requestBody);
UTF8Encoding encoding = new UTF8Encoding();
byte[] content = encoding.GetBytes(requestXML.ToString());
using (var stream = httpWebRequest.GetRequestStream())
{
stream.Write(content, 0, content.Length);
}
HttpWebResponse httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse;
using (Stream data = httpWebResponse.GetResponseStream())
{
Intuit.Ipp.Data.Qbo.SearchResults searchResults = (Intuit.Ipp.Data.Qbo.SearchResults)dataServices.ServiceContext.Serializer.Deserialize<Intuit.Ipp.Data.Qbo.SearchResults>(new StreamReader(data).ReadToEnd());
return ((Intuit.Ipp.Data.Qbo.Invoices)searchResults.CdmCollections).Invoice.ToList();
}
}
protected string GetDevDefinedOAuthHeader(HttpWebRequest webRequest, string requestBody)
{
OAuthConsumerContext consumerContext = new OAuthConsumerContext
{
ConsumerKey = consumerKey,
ConsumerSecret = consumerSecret,
SignatureMethod = SignatureMethod.HmacSha1,
UseHeaderForOAuthParameters = true
};
consumerContext.UseHeaderForOAuthParameters = true;
//URIs not used - we already have Oauth tokens
OAuthSession oSession = new OAuthSession(consumerContext, "https://www.example.com",
"https://www.example.com",
"https://www.example.com");
oSession.AccessToken = new TokenBase
{
Token = accessToken,
ConsumerKey = consumerKey,
TokenSecret = accessTokenSecret
};
IConsumerRequest consumerRequest = oSession.Request();
consumerRequest = ConsumerRequestExtensions.ForMethod(consumerRequest, webRequest.Method);
consumerRequest = ConsumerRequestExtensions.ForUri(consumerRequest, webRequest.RequestUri);
if (webRequest.Headers.Count > 0)
{
ConsumerRequestExtensions.AlterContext(consumerRequest, context => context.Headers = webRequest.Headers);
if (webRequest.Headers[HttpRequestHeader.ContentType] == "application/x-www-form-urlencoded")
{
Dictionary<string, string> formParameters = new Dictionary<string, string>();
foreach (string formParameter in requestBody.Split('&'))
{
formParameters.Add(formParameter.Split('=')[0], formParameter.Split('=')[1]);
}
consumerRequest = consumerRequest.WithFormParameters(formParameters);
}
}
consumerRequest = consumerRequest.SignWithToken();
return consumerRequest.Context.GenerateOAuthParametersForHeader();
}
You can also see my original Question Here on StackOverflow: Query for All Invoices With Open Balances using QuickBooks Online (QBO) Intuit Partner Platform (IPP) DevKit.
Following the tutorial found on ASP.NET, implemented a Web API controller method for doing asynchronous file uploads that looks like this:
public Task<HttpResponseMessage> PostFormData()
{
// Check if the request contains multipart/form-data.
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
// Read the form data and return an async task.
var task = Request.Content.ReadAsMultipartAsync(provider).
ContinueWith<HttpResponseMessage>(t =>
{
if (t.IsFaulted || t.IsCanceled)
{
Request.CreateErrorResponse(HttpStatusCode.InternalServerError, t.Exception);
}
return Request.CreateResponse(HttpStatusCode.OK);
});
return task;
}
Uploading a file via a standard multipart HTML form works perfectly. However, when another developer attempts to upload a file via multipart form constructed by Flex's FileReference class, an error is thrown:
Unexpected end of MIME multipart stream. MIME multipart message is not complete.
I have no idea if the problem lies in Web API or Flex. I've found some sort of related fixes that had no affect (Multipart form POST using ASP.Net Web API), and more recently this one ("MIME multipart stream. MIME multipart message is not complete" error on webapi upload). If the second link holds true, does anyone know if it's out in the current release of Web API available via Nuget? The discussion was in May, the most recent release from Nuget was August, so I assume this fix was deployed already, and is not the root cause of my issue.
I had the same problem with MVC4, but Will is correct, add a name to your input.....
<input type="file" id="fileInput" name="fileInput"/>
and all the magic is back up and working!
I had the same problem with flex. And below is the code that solved it. Basically I used a custom stream to append the newline that asp.net web api is expecting.
Stream reqStream = Request.Content.ReadAsStreamAsync().Result;
MemoryStream tempStream = new MemoryStream();
reqStream.CopyTo(tempStream);
tempStream.Seek(0, SeekOrigin.End);
StreamWriter writer = new StreamWriter(tempStream);
writer.WriteLine();
writer.Flush();
tempStream.Position = 0;
StreamContent streamContent = new StreamContent(tempStream);
foreach(var header in Request.Content.Headers)
{
streamContent.Headers.Add(header.Key, header.Value);
}
// Read the form data and return an async task.
await streamContent.ReadAsMultipartAsync(provider);
Hope this helps.
Reading through your existing research and following through to the codeplex issue reported it looks like someone else confirmed this issue to still exist in September.
They believe that MVC 4 fails to parse uploads without a terminating "\r\n".
The issue is really simple but extremely hard to fix. The problem is that Uploadify does > not add an "\r\n" at the end of the MultiPartForm message
http://aspnetwebstack.codeplex.com/discussions/354215
It may be worth checking that the Flex upload adds the "\r\n"
For those landing here googling:
Unexpected end of MIME multipart stream. MIME multipart message is not complete.
Reading the request stream more than once will also cause this exception. I struggled with it for hours until I found a source explaining that the request stream only could be read once.
In my case, I combined trying to read the request stream using a MultipartMemoryStreamProvider and at the same time letting ASP.NET do some magic for me by specifying parameters (coming from the request body) for my api method.
Make sure the virtual directory ("~/App_Data" directory as below example) where the image files are first uploaded are physically existance. When you publish the project, it may not be in the output files.
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
I just removed my headers I was setting on my post method which ended up solving this issue.
The problem is this line:
string root = HttpContext.Current.Server.MapPath("~/App_Data");
It will only work in localhost, you can use HostingEnvironment.MapPath instead in any context where System.Web objects like HttpContext.Current are not available (e.g also from a static method).
var mappedPath = System.Web.Hosting.HostingEnvironment.MapPath("~/SomePath");
See also What is the difference between Server.MapPath and HostingEnvironment.MapPath?
Reference to this answer How to do a Server Map Path.
I'm working on twitter client for win8, using RestSharp ( http://restsharp.org/ ) and I have such problem:
When I'm posting new tweet with RestClient
var timeLine = new RestRequest("/1/statuses/update.json", Method.POST);
var txt = "Hello world";
timeLine.AddParameter("status", txt);
everything works excelent, but if I add more complex status like:
var txt = "Hello, World! What a nice day! #1May";
timeLine.AddParameter("status", txt);
I recieve 401 error. In debugger I saw, that status parameter in Signature Base String is incorrect. I have:
status%3DHello%2C%2520World%21%2520What%2520a%2520nice%2520day%21%2520%231May
and right string (from dev.twitter.com):
status%3DHello%252C%2520World%2521%2520What%2520a%2520nice%2520day%2521%2520%25231May
You can see, that punctuation marks ,!# and other encodes incorrect. How can I fix it?
Signature base generation and Encoding are in /Authenticators/OAuth/OAuthTools.cs
I have the same problem when I display twitter feed in website. Hence, I used this code to convert the text.
Regex.Replace(str, "#(.*?):", #"#http://twitter.com/#!/$1>$1:");