This is the REST API used to connect with Visual Studio Team Services (was Visual Studio Online), and on-premise TFS. I would like to set the headers so I can compress my requests, but the API documentation does not specify that gzip is supported. I'm hoping somebody might have experience.
using (var wc = new WebClient())
{
wc.Credentials = TfsCredentials;
wc.Headers[HttpRequestHeader.ContentEncoding] = "gzip";
wc.Headers[HttpRequestHeader.ContentType] = "application/json";
var gzipByteArray = GZipBytes(serializedJson);
var uploadResponse = wc.UploadData(repoUri, gzipByteArray);
return Encoding.UTF8.GetString(uploadResponse);
}
Response is a 400, with the following error message:
The body of the request contains invalid Json. Parameter name: contentStream
I cannot find any documentation about this either. But I tested it with and without gzip compress from curl. The size of response is indeed compressed with gzip and the response can be decompressed correctly. So it should be supported.
Related
The official documentation does not provide an example for any SDK's (including the Java SDK): https://learn.microsoft.com/en-us/graph/api/user-sendmail?view=graph-rest-1.0&tabs=java#example-4-send-a-new-message-using-mime-format. As there is no example, I have tried in vain to send the MIME content using the SDK (microsoft-graph 5.0.0):
Message sending = new Message();
ItemBody body = new ItemBody();
final String mimeMessageRFC822 = input.getMimeMessageRFC822();
body.content = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
sending.body = body;
GraphServiceClient service = getService(acHost, configuration);
service
.me()
.sendMail(UserSendMailParameterSet.newBuilder().withMessage(sending).withSaveToSentItems(true).build())
.buildRequest(new HeaderOption("Content-Type", "text/plain"))
.post();
The above code sets the request's content-type to text/plain, however the request body that is being sent is JSON (xxxxxx below is a placeholder for a valid Base64 encoded MIME content string).
{
"message":
{
"body":
{
"content": xxxxxx
}
},
"saveToSentItems": true
}
The response is a 404, stating:
GraphServiceException: Error code: ErrorMimeContentInvalidBase64String
Error message: Invalid base64 string for MIME content.
I can understand why it is responding with this error as the graph endpoint is parsing the text/plain content as base64 encoded MIME but finds the JSON structure instead. I have been on a video call with a Microsoft Graph support agent, and they have seen that my MIME content is valid. Sadly, they are not able to help with the Microsoft Graph Java SDK even though it is developed by Microsoft!
This suggests that we are not supposed to use the Java SDK at all for sending MIME formatted emails. Is this correct? Surely it can't be otherwise what is the point of a library that can receive MIME formatted emails but can't send them? Does anyone have a solution?
For now at least the solution is to send a CustomRequest with MIME content instead of using the fluent API provided by the Graph client.
final String encodedContent = Base64.getMimeEncoder().encodeToString(mimeMessageRFC822.getBytes());
CustomRequest<String> request = new CustomRequest<>(requestUrl, service, List.of(new HeaderOption("Content-Type", "text/plain")), String.class);
request.post(encodedContent);
I'm currently using BizTalk Server 2013 R2 to exchange EDI as well as non-EDI documents using AS2 with a number of different trading partners. I recently added a new trading partner and after receiving a number of documents successfully I started seeing this error occur every now and then:
An output message of the component "Microsoft.BizTalk.EdiInt.PipelineComponents" in receive pipeline "Microsoft.BizTalk.EdiInt.DefaultPipelines.AS2Receive, Microsoft.BizTalk.Edi.EdiIntPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" is suspended due to the following error: The content transfer encoding quoted-printable is not supported..
The sequence number of the suspended message is 2.
After some investigation I found that the AS2 platform of the trading partner in question will sometimes set the Content-Transfer-Encoding of the MIME body part to quoted-printable when the enclosed XML payload contains non-ASCII characters. When this happens the message is suspended (non-resumable) with the error above.
Messages received from this trading partner are encrypted and signed, but not compressed - and received using a HTTP request-response (two-way) port configured with the out-of-the-box AS2Receive pipeline. I've tried using a custom pipeline with the AS Decoder, S/MIME decoder and AS2 disassembler components, but this does not seem to have any effect - the error stays the same.
I've also tried receiving unencrypted messages from the trading partner (by mutual agreement) but seem to be doing something wrong here as well as the message passed to the Message Box then ends up not being disassembled properly (the MIME part boundaries and AS2 signature is still visible in the actual message payload). Since the trading partner won't allow sending of unencrypted messages in a production environment anyway, I need to get this working with encryption. They also cannot change their platform's behavior as this will reportedly affect all of their other trading partners.
Here are the unfolded HTTP headers (ellipses denotes redacted values) of the encrypted and signed AS2 message received at the point of being suspended:
Date: Mon, 20 Jan 2020 17:30:53 GMT
Content-Length: 8014
Content-Type: application/pkcs7-mime; name="smime.p7m"; smime-type=enveloped-data
From: ...
Host: ...
User-Agent: Jakarta Commons-HttpClient/3.1
AS2-To: ...
Subject: AS2 Message from ... to ...
Message-Id: <1C20200120-173053-740219#xxx.xxx.130.163>
Disposition-Notification-To: <mailto:...> ...
Disposition-Notification-Options: signed-receipt-protocol=optional, pkcs7-signature; signed-receipt-micalg=optional, sha1
AS2-From: ...
AS2-Version: 1.1
content-disposition: attachment; filename="smime.p7m"
X-Original-URL: /as2
Here is the unencrypted (ellipses denotes redacted content) payload when exact same message is sent from source party without encryption:
------=_Part_16155_1587439544.1579506174880
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
...
------=_Part_16155_1587439544.1579506174880
Content-Type: application/pkcs7-signature; name=smime.p7s; smime-type=signed-data
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
...
------=_Part_16155_1587439544.1579506174880--
Question: does BizTalk Server support the quoted-printable encoding method? If it does, what am I doing wrong? If it does not, what are my options in terms of a workaround?
For anyone else that may encounter this same issue, I thought I'd share the solution I ended up with.
Since the error was encountered during AS2 receive pipeline processing, naturally my solution was focussed around creating a custom receive pipeline component that does more or less the same than the out-of-the-box AS2 decoder component, but with support for the quoted-printable encoding method:
1. Decode and decrypt the CMS/PKCS#7 data envelope
This is actually the easiest step with only 5 lines of code:
EnvelopedCms envelopedCms = new EnvelopedCms();
envelopedCms.Decode(encryptedData);
envelopedCms.Decrypt();
byte[] decryptedData = envelopedCms.Encode();
string decryptedMessageString = Encoding.ASCII.GetString(decryptedData);
-encryptedData is a byte-array instantiated from the body-part data stream of the AS2 message received bythe HTTP adapter.
-The Decrypt method automatically searches the user and computer certificate stores for the appropriate certificate private key and uses this to decrypt the AS2 payload. For more information on the `EnvelopedCms' class follow this link.
2. Convert any quoted-printable content in the payload to normal UTF-8 text
First we have to get the MIME boundary name from the content type string at the beginning of the decrypted payload:
int firstBlankLineInMessage = decryptedMessageString.IndexOf(Environment.NewLine + Environment.NewLine);
string contentType = decryptedMessageString.Substring(0, firstBlankLineInMessage);
Regex boundaryRegex = new Regex("boundary=\"(?<boundary>.*)\"");
Match boundaryMatch = boundaryRegex.Match(contentType);
if (!boundaryMatch.Success)
throw new Exception("Failed to get boundary name from content type");
string boundary = "--" + boundaryMatch.Groups["boundary"].Value;
Then we split the envelope and re-merge without the content-type header part:
string[] messageParts = decryptedMessageString.Split(new string[] {boundary}, StringSplitOptions.RemoveEmptyEntries);
string signedMessageString = boundary + messageParts[1] + boundary + messageParts[2] + boundary + "--\r\n";
Next we get the `Content-Transfer-Encoding' value in the MIME body-part header:
int firstBlankLineInBodyPart = messageParts[1].IndexOf(Environment.NewLine + Environment.NewLine);
string partHeaders = messageParts[1].Substring(0, firstBlankLineInBodyPart);
Regex cteRegex = new Regex("Content-Transfer-Encoding: (?<cte>.*)");
Match cteMatch = cteRegex.Match(partHeaders);
if (!cteMatch.Success)
throw new Exception("Failed to get CTE from body part headers");
string cte = cteMatch.Groups["cte"].Value;
string payload = messageParts[1].Substring(firstBlankLineInBodyPart).Trim();
And finally we check the CTE and decode if neccessary:
string payload = messageParts[1].Substring(firstBlankLineInBodyPart).Trim();
if (cte == "quoted-printable")
{
// Get charset
Regex charsetRegex = new Regex("Content-Type: .*charset=(?<charset>.*)");
Match charsetMatch = charsetRegex.Match(partHeaders);
if (!charsetMatch.Success)
throw new Exception("Failed to get charset from body part headers");
string charset = charsetMatch.Groups["charset"].Value;
QuotedPrintableDecode(payload, charset);
}
Note: There are many different implementations out there for decoding QP, including a .NET implementation that has (reportedly) been found buggy by some users. I decided to use this implementation shared by Gonzalo.
3. Update the Content-Type HTTP header and BizTalk message body-part stream
string httpHeaders = objHttpHeaders.ToString().Replace("Content-Type: application/pkcs7-mime; name=\"smime.p7m\"; smime-type=enveloped-data", "Content-Type: application/xml");
inMessage.Context.Write("InboundHttpHeaders", "http://schemas.microsoft.com/BizTalk/2003/http-properties", httpHeaders);
MemoryStream payloadStream = new MemoryStream(Encoding.UTF8.GetBytes(payload));
payloadStream.Seek(0, SeekOrigin.Begin);
pipelineContext.ResourceTracker.AddResource(payloadStream);
inMessage.BodyPart.Data = payloadStream;
-pipelineContext is the IPipelineContext variable passed to the Execute method of the custom pipeline component
-inMessage is the IBaseMessage variable passed to the Execute method
Last Thoughts
The code above can still be improved in a number of ways:
Checking HTTP headers for encryption before attempting to decrypt
Re-encrypting payload before passing message to AS2 disassembler component (if required by BizTalk party configuration)
Adding support for compression
If you'd like a copy of the source code drop me a message and I'll see about upping it to an online repo.
I had ticket opened with Microsoft BizTalk tech support on the issue. Their response is that
The quoted-printable encoding is not supported by MS BizTalk Server 2013R2" and most likely is not supported by MS BizTalk Server 2020
I am using Azure Mobile Service as a backend. Data structure is implemented and I tested CRUD calls using Fiddler. Everything seems right.
Then I implemented offline data synchronization with a client iOS device. Btw I followed this tutorial: https://azure.microsoft.com/en-gb/documentation/articles/app-service-mobile-ios-get-started-offline-data-preview/
My problem is when I try to synchronize data using the pullWithQuery function. I get this errror:
2015-08-19 11:36:12.525 Hykso[1820:330268] Logged in as Facebook:10155931659265500
2015-08-19 11:36:30.285 Hykso[1820:330522] ERROR Error Domain=com.Microsoft.WindowsAzureMobileServices.ErrorDomain Code=-1302 "{"message":"An error has occurred."}" UserInfo=0x14671c30 {NSLocalizedDescription={"message":"An error has occurred."}, com.Microsoft.WindowsAzureMobileServices.ErrorResponseKey=<NSHTTPURLResponse: 0x14584de0> { URL: https://hyksomobileservice.azure-mobile.net/tables/Athlete?$top=50&__includeDeleted=true&$skip=0&__systemProperties=__createdAt%2C__updatedAt%2C__deleted%2C__version } { status code: 500, headers {
"Cache-Control" = "no-cache";
"Content-Length" = 36;
"Content-Type" = "application/json; charset=utf-8";
Date = "Wed, 19 Aug 2015 15:36:28 GMT";
Expires = 0;
Pragma = "no-cache";
Server = "Microsoft-IIS/8.0";
"X-Powered-By" = "ASP.NET";
} }, com.Microsoft.WindowsAzureMobileServices.ErrorRequestKey=<NSMutableURLRequest: 0x14657830> { URL: https://hyksomobileservice.azure-mobile.net/tables/Athlete?$top=50&__includeDeleted=true&$skip=0&__systemProperties=__createdAt%2C__updatedAt%2C__deleted%2C__version }}
I just tested the same get call using Fiddler and I get this message:
exceptionMessage=The specified type member 'Version' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
Does anyone have some advice to give me in order to debug this? Thank you!
(First off, make sure you use the Mobile Services topic, not Mobile Apps topic. For this article they are very similar, so that's not your issue.)
The error looks really strange, and it points to something wrong with your server setup. Your client is sending a query asking for the system properties __createdAt, __updatedAt, __deleted, and __version, but somehow the "version" part is being translated into an invalid LINQ query. If you remove the ms_version column from your Core Data model, then version won't be requested, but then you can't do any conflict handling.
Based on the EF error message, you must be using the .NET backend. What happens when you test with the default TodoItem type that's in the server project? Your data class needs to extend EntityData to ensure everything is mapped correctly. (You can use ITableData, but that's more advanced.)
In terms of debugging, you can run the server project locally on IIS express and set a breakpoint after hitting the endpoint via Fiddler. You can also use remote debugging to go to your actual remote service.
For tutorials on this, see:
https://azure.microsoft.com/en-us/documentation/articles/mobile-services-dotnet-backend-how-to-troubleshoot/#runtime-debugging
https://azure.microsoft.com/en-us/documentation/articles/web-sites-dotnet-troubleshoot-visual-studio/
Are assigning any value to the Version column in your data? I developed my app with C# and had a similar problem. There were two constraints about Version. First, don't touch it on the client just define it in your model and never assign to it. Second, in C# we need to define it this way
[Version]
public byte[] Version { get; set; }
I don't know the equivalent of [Version] in Objective-c
Problem
I have added support for http compression in our self-hosted OWIN/Katana Web API OData 4 service but I do not see how to support compression in the .NET client. I'm using OData libraries v6.5.0 and I need to support compression/decompression in the client (OData v4 Client Code Generator). I am using Deflate encoding for the compression via an ActionFilter. Everything compresses correctly on the server as confirmed via Fiddler but I do not know how to configure the client to support this now that the OData client uses the Request and Response Pipelines instead of the now defunct WritingRequest and RecievingResponse events that once supported this very scenario.
Attempts
By experimentation I found that I can hook into the ReceivingResponse event on my DataServiceContext and then call ReceivingResponseEventArgs.ResponseMessage.GetStream() but I don't know what to do to overwrite the message content correctly. If I CopyTo() on the stream, I get a null reference exception at Microsoft.OData.Core.ODataMessageReader.DetectPayloadKind(). I presume this is because the stream was read to the end and the position needs to be set back to zero but I cannot do that because the stream also throws an exception when setting the position back because it says it does not support seeking. I presume this is simply due to the stream being read-only. Even if I could copy the stream to decompress it successfully, how do I modify the response message content with the decompressed content? I don't see any hooks for this at all in the RequestPipeline or ResponsePipeline. To clarify, I want to decompress the response message content and then set it for the materialization that occurs soon after, how might I do that? Extra credit for how to also send compressed requests to the OData service. Thanks!
OData client use the HTTPWebRequest and HTTPWebReponse, which supports the compression well. Try setting the AutomaticDecompression of HTTPWebRequest to Deflate or GZip, in SendingRequest2 pipeline event, like this:
private void OnSendingRequest_(object sender, SendingRequest2EventArgs args)
{
if (!args.IsBatchPart) // The request message is not HttpWebRequestMessage in batch part.
{
HTTPWebRequest request = ((HttpWebRequestMessage)args.RequestMessage).HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
}
}
Then in response, HTTPWebResponse will decompress the stream automatically, before the materialization work.
Am currently communicating to a Mobile device using Windows Compact Framework 3.5. The message sent to the device is built is as thus,
HttpResponseMessage result;
var response = Encoding.UTF8.GetBytes("<?xml version=\"1.0\" encoding=\"windows-1252\"?><message type=\"response\"><header><datetime>2013-04-03T09:49:35</datetime><sender version=\"1.1.4.1138\"><userid>Connect Server</userid></sender><commandlist><module>ADMIN</module><command1>VALIDATE</command1></commandlist><result type=\"ok\"/></header></message>");
result = Request.CreateResponse(HttpStatusCode.OK, response);
The device then retrieves the message and then uses
Encoding.UTF8.GetString(responseContent);
After decoding the message is:
<base64Binary xmlns="http://schemas.microsoft.com/2003/10/Serialization/">PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0id2luZG93cy0xMjUyIj8+PG1lc3NhZ2UgdHlwZT0icmVzcG9uc2UiPjxoZWFkZXI+PGRhdGV0aW1lPjIwMTMtMDQtMDNUMDk6NDk6MzU8L2RhdGV0aW1lPjxzZW5kZXIgdmVyc2lvbj0iMS4xLjQuMTEzOCI+PHVzZXJpZD5Db25uZWN0IFNlcnZlcjwvdXNlcmlkPjwvc2VuZGVyPjxjb21tYW5kbGlzdD48bW9kdWxlPkFETUlOPC9tb2R1bGU+PGNvbW1hbmQxPlZBTElEQVRFPC9jb21tYW5kMT48L2NvbW1hbmRsaXN0PjxyZXN1bHQgdHlwZT0ib2siLz48L2hlYWRlcj48L21lc3NhZ2U+</base64Binary>
Tried decoding the message on the server before sending it off and it's fine. Unsure what could be going wrong.
Any help would be greatly appreciated.
Request.CreateResponse() uses ObjectContent. For this scenario, you don't want that. You should use either StringContent or StreamContent to return the XML. See this question for details https://stackoverflow.com/a/15372410/6819
You are encoding your XML as binary. You are then returning a byte array. Then your client requests XML in the Accept: application/xml header. The Web API serializes the binary into XML. That's what you're seeing.
Just return the XML as a string and you should have no problems, unless you've tried that already?
See here for similar question.