Bad Request "ErrorExecuteSearchStaleData" in Microsoft Graph API - Search for Mails - microsoft-graph-api

We are using Microsoft Graph Search API to search through our O365 emails. Since the search only allows 25 results per request for mails. (see https://learn.microsoft.com/en-us/graph/api/resources/search-api-overview?view=graph-rest-beta#page-search-results)
We figured to work around this by batching our search request like this:
POST https://graph.microsoft.com/beta/$batch
{
"requests": [
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-0",
"body":
{
"requests": [
{
"entityTypes": ["message"],
"query":
{
"query_string":
{
"query": "some search text"
}
},
"from": 0,
"size": 25
}
]
},
"headers":
{
"Content-Type": "application/json"
}
},
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-1",
"body":
{
"requests": [
{
"entityTypes": ["message"],
"query":
{
"query_string":
{
"query": "some search text"
}
},
"from": 25,
"size": 25
}
]
},
"headers":
{
"Content-Type": "application/json"
}
},
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-2",
"body":
{
"requests": [
{
"entityTypes": ["message"],
"query":
{
"query_string":
{
"query": "some search text"
}
},
"from": 50,
"size": 25
}
]
},
"headers":
{
"Content-Type": "application/json"
}
},
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-3",
"body":
{
"requests": [
{
"entityTypes": ["message"],
"query":
{
"query_string":
{
"query": "some search text"
}
},
"from": 75,
"size": 25
}
]
},
"headers":
{
"Content-Type": "application/json"
}
}
]
}
This works usually well, but in some cases the API returns something like this:
{
"id": "MyMails-2",
"status": 400,
"headers":
{
"Link": "<https://developer.microsoft-tst.com/en-us/graph/changes?$filterby=beta,RemoveDeprecatedUnderscoreProperty&from=2021-12-01&to=2022-01-01>;rel=\"deprecation\";type=\"text/html\",<https://developer.microsoft-tst.com/en-us/graph/changes?$filterby=beta,RemoveDeprecatedUnderscoreProperty&from=2021-12-01&to=2022-01-01>;rel=\"deprecation\";type=\"text/html\",<https://developer.microsoft-tst.com/en-us/graph/changes?$filterby=beta,RemoveDeprecatedUnderscoreProperty&from=2021-12-01&to=2022-01-01>;rel=\"deprecation\";type=\"text/html\"",
"Deprecation": "Tue, 14 Dec 2021 23:59:59 GMT",
"Sunset": "Sat, 31 Dec 2022 23:59:59 GMT",
"Cache-Control": "no-cache",
"Content-Type": "application/json"
},
"body":
{
"error":
{
"code": "BadRequest",
"message": "\r\n An exception occurred\r\n Lss call failed with status code 400. \"Exchange service returned error ErrorExecuteSearchStaleData: Please reissue the query with rowOffset = 0. The specified rowoffset is '50', but the results are stale.\".",
"innerError":
{
"date": "2022-01-25T09:58:53",
"request-id": "75def95f-a857-427d-a8b4-ee2792329e87",
"client-request-id": "75def95f-a857-427d-a8b4-ee2792329e87"
}
}
}
}
I noticed the deprecation note in the header, however I can't find anything about it. Neither about the actual exception.
What am I missing?
[workaround]
Thanks for the hint #user2250152.
This heavily reduced the issue by splitting the request into two requests, while the second is able to request more than 25 mails at a time:
POST https://graph.microsoft.com/beta/$batch
{
"requests": [
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-0",
"body": {
"requests": [
{
"entityTypes": [
"message"
],
"query": {
"query_string": {
"query": "some search text"
}
},
"from": 0,
"size": 25
}
]
},
"headers": {
"Content-Type": "application/json"
}
},
{
"url": "/search/query",
"method": "POST",
"id": "MyMails-1",
"body": {
"requests": [
{
"entityTypes": [
"message"
],
"query": {
"query_string": {
"query": "some search text"
}
},
"from": 25,
"size": 200
}
]
},
"headers": {
"Content-Type": "application/json"
}
}
]
}

The error with stale results can happen time to time.
You can decrease the number of batch requests to reduce a chance that the error with stale results will occur.
For the first page "from": 0 the max size is 25. But for the next page "from": 25 you can increase the page size to 200.
I've tested the search query with "from": 25 and "size": 200 and it returns 200 results.
Resources:
Page search results

Related

Paypal sandbox cancel webhook not working

The webhook for a subscription activation BILLING.SUBSCRIPTION.ACTIVATED in the Paypal's sandbox works very well.
Nevertheless, when I'm trying to cancel the subscription from a customer sandbox account, no webhook post is received.
It should be BILLING.SUBSCRIPTION.CANCELLED.
In fact, the status is "pending" from the sandbox admin account:
Does it wait until the subscription's end date in order to send the cancellation?
Or does it wait a few hours for safety reasons?
Here are the details from the paypal sandbox web site:
{
"id": "WH-3BV96304FY002321K-38Y64104YL460680C",
"create_time": "2022-07-04T17:51:34.874Z",
"resource_type": "subscription",
"event_type": "BILLING.SUBSCRIPTION.CANCELLED",
"summary": "Subscription cancelled",
"resource": {
"quantity": "1",
"subscriber": {
"email_address": "valid_adress#yahoo.fr",
"payer_id": "X3FWGFE2SAXUL",
"name": {
"given_name": "Anyname",
"surname": "Walker"
},
"shipping_address": {
"address": {
"address_line_1": "Great st 25",
"admin_area_2": "New York",
"admin_area_1": "NY",
"postal_code": "10002",
"country_code": "US"
}
}
},
"create_time": "2022-07-04T17:49:00Z",
"plan_overridden": false,
"shipping_amount": {
"currency_code": "USD",
"value": "0.0"
},
"start_time": "2022-07-04T17:46:51Z",
"update_time": "2022-07-04T17:51:25Z",
"billing_info": {
"outstanding_balance": {
"currency_code": "USD",
"value": "0.0"
},
"cycle_executions": [
{
"tenure_type": "REGULAR",
"sequence": 1,
"cycles_completed": 1,
"cycles_remaining": 0,
"current_pricing_scheme_version": 1,
"total_cycles": 0
}
],
"last_payment": {
"amount": {
"currency_code": "USD",
"value": "3.99"
},
"time": "2022-07-04T17:49:01Z"
},
"failed_payments_count": 0
},
"links": [
{
"href": "https://api.sandbox.paypal.com/v1/billing/subscriptions/I-6BR7NSX98KLF",
"rel": "self",
"method": "GET",
"encType": "application/json"
}
],
"id": "I-6BR7NSX98KLF",
"plan_id": "P-5XR3459593FMK2CYGA",
"status": "CANCELLED",
"status_update_time": "2022-07-04T17:51:25Z"
},
"status": "PENDING",
"transmissions": [
{
"webhook_url": "https://www.myapp.com/webhook",
"http_status": 440,
"reason_phrase": "HTTP/1.1 200 Connection established",
"response_headers": {
"Server": "gunicorn",
"Connection": "keep-alive",
"Content-Length": "5",
"Date": "Mon, 04 Jul 2022 18:15:52 GMT",
"Content-Type": "text/html; charset=utf-8",
"Via": "1.1 google"
},
"transmission_id": "02cba350-fbc2-9abc-d537a3be4540",
"status": "PENDING",
"timestamp": "2022-07-04T17:52:03Z"
}
],
"links": [
{
"href": "https://api.sandbox.paypal.com/v1/notifications/webhooks-events/WH-3BV96304FY002321K-38Y64104YL460680C",
"rel": "self",
"method": "GET",
"encType": "application/json"
},
{
"href": "https://api.sandbox.paypal.com/v1/notifications/webhooks-events/WH-3BV96304FY002321K-38Y64104YL460680C/resend",
"rel": "resend",
"method": "POST",
"encType": "application/json"
}
],
"event_version": "1.0",
"resource_version": "2.0"
}
Pending means pending delivery to the subscribed webhook URL. The event ocurred, and delivery to the URL was attempted, but that URL did not respond with an HTTP 200 success message for the webhook to be marked as successfully delivered. In fact, according to the details logged in your question:
"webhook_url": "https://www.myapp.com/webhook",
"http_status": 440,
That URL instead responded with an HTTP 440 status
Assuming the URL is correct and that you changed it to "myapp" for this question, the issue is that you need to debug the endpoint/code at that URL so that it responds with a 200 success status when webhook deliveries are posted to it. Then their status will stop being Pending.

Mountebank - How do I assert message body is JSON format in a POST request in mountebank

I want to stub and check the message body in POST request in mountebank,
{
"port": "22001",
"protocol": "http",
"name": "login_user",
"stubs": [
{
"responses": [
{
"is": {
"statusCode": 201,
"headers": {
"Content-Type": "application/json"
},
"body": {}
},
"_behaviors": {
"wait": 100
}
}
],
"predicates": [
{
"equals": {
"path": "/login_user",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": {
"name": "Tony",
"age": "20"
}
}
}
]
}
]
}
if a message body in JSON format. expected response status Code 200.
for Example
{
"body": {
"name": "Tony",
"age": "20"
}
}
if a message body in JSON format but JSON string. expected response status Code 400.
for Example
{
"body": "{\"name\": \"Tony\", \"age\": \"20\"}"
}
You could achieve this with a 'matches' predicate containing a (pretty crude in this example) regex to capture any string input and return a 400:
"stubs": [
{
"responses": [
{
"is": {
"statusCode": 400,
"headers": {
"Content-Type": "application/json"
},
"body": {}
},
"_behaviors": {
"wait": 100
}
}
],
"predicates": [
{
"matches": {
"path": "/login_user",
"method": "POST",
"body": ".*\\\\\\\"name.*"
}
}
]
},
{
"responses": [
{
"is": {
"statusCode": 201,
"headers": {
"Content-Type": "application/json"
},
"body": {}
},
"_behaviors": {
"wait": 100
}
}
],
"predicates": [
{
"equals": {
"path": "/login_user",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": {
"name": "Tony",
"age": "20"
}
}
}
]
}
]
Alternatively, you can also specify a default response code of 400 in the imposter declaration, so that anything that doesn't match a specific predicate will return a 400 response by default:
{
"port": "22001",
"protocol": "http",
"name": "login_user",
"defaultResponse": {
"statusCode": 400
},
"stubs": [
{
.... snip ....

Call MS Graph API from MS Flow with Binary Body

Have an MS Flow trying to call the Graph API to update a user's profile photo. It takes a username and jpg photo file and then calls an HTTP connection secured using Azure AD (with https://graph.microsoft.com as the resource URI). Unfortunately, I get the error The file you chose isn't an image. Please choose a different file. which seems to indicate that I am not posting the file encoded in a way that MS Graph recognizes.
I've tried all variations I can think of with the body (using the #binary, #base64ToBinary, etc):
"body": "#{triggerBody()?['file']?['contentBytes']}"
to try to get it to treat "correctly", but with no luck.
This is the full flow definition:
{
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
},
"$authentication": {
"defaultValue": {},
"type": "SecureObject"
}
},
"triggers": {
"manual": {
"type": "Request",
"kind": "Button",
"inputs": {
"schema": {
"type": "object",
"properties": {
"file": {
"title": "Photo",
"type": "object",
"x-ms-dynamically-added": true,
"description": "Please select file or image",
"x-ms-content-hint": "FILE",
"properties": {
"name": {
"type": "string"
},
"contentBytes": {
"type": "string",
"format": "byte"
}
}
},
"text": {
"title": "Username",
"type": "string",
"x-ms-dynamically-added": true,
"description": "user#domain.com",
"x-ms-content-hint": "TEXT"
}
},
"required": [
"text"
]
}
}
}
},
"actions": {
"Invoke_an_HTTP_request": {
"runAfter": {},
"metadata": {
"flowSystemMetadata": {
"swaggerOperationId": "InvokeHttp"
}
},
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#json(decodeBase64(triggerOutputs().headers['X-MS-APIM-Tokens']))['$connections']['webcontents_1']['connectionId']"
}
},
"method": "post",
"body": {
"method": "PUT",
"url": "https://graph.microsoft.com/v1.0/users/#{triggerBody()['text']}/photo/$value",
"headers": {
"Content-Type": "image/jpeg"
},
"body": "#{triggerBody()?['file']?['contentBytes']}"
},
"path": "/codeless/InvokeHttp",
"authentication": {
"type": "Raw",
"value": "#json(decodeBase64(triggerOutputs().headers['X-MS-APIM-Tokens']))['$ConnectionKey']"
}
}
}
}
}
and the full error:
{
"error": {
"code": 502,
"message": "{\r\n \"error\": {\r\n \"code\": 502,\r\n \"source\": \"unitedstates-002.azure-apim.net\",\r\n \"clientRequestId\": \"e105d9f2-af13-4a5b-8fbd-c84c49cadc8f\",\r\n \"message\": \"BadGateway\",\r\n \"innerError\": {\r\n \"error\": {\r\n \"code\": \"ErrorInternalServerError\",\r\n \"message\": \"An internal server error occurred. The operation failed., The file you chose isn't an image. Please choose a different file.\",\r\n \"innerError\": {\r\n \"request-id\": \"e3214f20-8592-4fb0-9d34-677322b231ae\",\r\n \"date\": \"2019-07-30T21:42:10\"\r\n }\r\n }\r\n }\r\n }\r\n}"
}
}

JSON Batching not working with some Microsoft Teams API

We are trying the JSON batching of Microsoft Graph using this request:
{
"requests": [
{
"id": "1",
"method": "GET",
"url": "/me"
},
{
"id": "2",
"method": "GET",
"url": "/me/joinedTeams"
}
]
}
We are able to get a response from /me, but not for /me/joinedTeams:
{
"responses": [
{
"id": "1",
"status": 200,
"headers": {
"Cache-Control": "no-cache",
"OData-Version": "4.0",
"Content-Type": "application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8"
},
"body": {
// valid response here
}
},
{
"id": "2",
"status": 400,
"body": {
"error": {
"code": "BadRequest",
"message": "Unsupported segment type. ODataQuery: users/35036c48-1e5a-4ca4-89f0-2d11c4cf8937/joinedTeams",
"innerError": {
"request-id": "a075c4f6-362a-469f-945b-5b46d96784a0",
"date": "2018-09-12T10:09:40"
}
}
}
}
]
}
Does batching not support the Teams APIs?
Teams Graph APIs are still in Beta but you're using the /1.0 endpoint. It should work fine on the /beta endpoint.
POST https://graph.microsoft.com/beta/$batch
Accept: application/json
Content-Type: application/json
{
"requests": [
{
"id": "1",
"method": "GET",
"url": "/me"
},
{
"id": "2",
"method": "GET",
"url": "/me/joinedTeams"
}
]
}

How do I get browsermob-proxy to record ALL response bodies

I'm having trouble with browsermob-proxy and its har exporting feature. Some response bodies are not logged (the entire "text" field is missing)
My setup (using browsermob-proxy 2.1.4)
curl -X POST http://localhost:8080/proxy?port=9091
curl -X PUT "http://localhost:8080/proxy/9091/har?captureHeaders=true&captureCookies=true&captureContent=true"
Some responses are fine:
"response": {
"status": 201,
"statusText": "Created",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [{
"name": "Cache-Control",
"value": "max-age=0, no-cache, no-store"
}, {
"name": "Content-Type",
"value": "application/json"
}, {
"name": "Date",
"value": "Thu, 15 Feb 2018 13:07:39 GMT"
}, {
"name": "Location",
"value": ...
}, {
"name": "Pragma",
"value": "no-cache"
}, {
"name": "Render-Time",
"value": "8"
}, {
"name": "Server",
"value": "openresty"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=31536000; includeSubDomains"
}, {
"name": "transfer-encoding",
"value": "chunked"
}, {
"name": "Connection",
"value": "keep-alive"
}],
"content": {
"size": 8607,
"mimeType": "application/json",
"text": "{ <actual json body> }",
"comment": ""
}, ...
But some are not (maybe it is the special content/mime type? or maybe it is the gzip content encoding?)
"response": {
"status": 200,
"statusText": "OK",
"httpVersion": "HTTP/1.1",
"cookies": [],
"headers": [{
"name": "Cache-Control",
"value": "max-age=0, no-cache, no-store"
}, {
"name": "Content-Encoding",
"value": "gzip"
}, {
"name": "Content-Type",
"value": "application/some.custom.type-v1+json"
}, {
"name": "Date",
"value": "Thu, 15 Feb 2018 13:07:39 GMT"
}, {
"name": "Pragma",
"value": "no-cache"
}, {
"name": "Render-Time",
"value": "92"
}, {
"name": "Server",
"value": "openresty"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=31536000; includeSubDomains"
}, {
"name": "Vary",
"value": "Accept-Encoding"
}, {
"name": "Content-Length",
"value": "1978"
}, {
"name": "Connection",
"value": "keep-alive"
}],
"content": {
"size": 7429,
"mimeType": "application/some.custom.type-v1+json",
"comment": ""
< there's nothing else here!! >
},
"redirectURL": "",
"headersSize": 444,
"bodySize": 1978,
"comment": ""
}, ...
Oh yea, both requests are over https, using MITM.
I think I found the answer to my own question
In BrowserMobHttpClient.java I found the following:
private boolean hasTextualContent(String contentType) {
return contentType != null && contentType.startsWith("text/") ||
contentType.startsWith("application/x-javascript") ||
contentType.startsWith("application/javascript") ||
contentType.startsWith("application/json") ||
contentType.startsWith("application/xml") ||
contentType.startsWith("application/xhtml+xml");
}
Looks like I'll have go make a custom build of browsermob-proxy to make it work.

Resources