For the Discovery REST api, the argument/parameter "return" controls which fields are returned.
So if I pass these arguments to the API
{
"query": named_sector,
"count": "10",
"filter": filter_dates,
"aggregation" : "term(docSentiment.type,count:3)"
}
my_query = discovery.query(my_disc_environment_id, my_disc_collection_id, qopts)
print(json.dumps(my_query, indent=2))
I get the following:
{
"matching_results": 14779,
"aggregations": [
{
"type": "term",
"field": "docSentiment.type",
"count": 3,
"results": [
{
"key": "positive",
"matching_results": 4212
},
{
"key": "negative",
"matching_results": 3259
},
{
"key": "neutral",
"matching_results": 152
}
]
}
],
"results": [
{
"id": "6389715fe7e7f711e0bc09d4f1236639",
"score": 1.3689895,
"yyyymm": "201704",
"url": "https://seekingalpha.com/article/4060446-valuation-dashboard-consumer-discretionary-update",
"enrichedTitle": null,
"host": "seekingalpha.com",
"text": "Valuation Dashboard: Consumer Discretionary - Update\n\nSummary\n\nValuation metrics in Consumer Discretionary.\n\nEvolution since last month.\n\nA list of stocks loo ....
and thousands of more lines. How do I restrict the output to the aggregations section? Is this an issue of me better handling the JSON structure that is returned?
thanks
If you change the count argument to 0, the returned JSON will only contain the aggregations.
Also, if you're using the Discovery web tooling, you can enter 0 for the "Number of results to return (Count)" field.
More details and an example can be found here: https://www.ibm.com/watson/developercloud/doc/discovery/using.html#building-aggregations
Related
Following huobi API, I subscribe to the trade.detail channel, and among one day data, I got one entry like this:
{
"ch": "market.BTC-USDT.trade.detail",
"ts": 1666158766936,
"tick": {
"id": 118671341528,
"ts": 1666158766920,
"data": [
{
"amount": 2,
"quantity": 0.002,
"trade_turnover": 38.4776,
"ts": 1666158766920,
"id": 1186713415280000,
"price": 19238.8,
"direction": ""
}
]
}
}
the direction field is empty string, don't know which side it is.
We have an Microsoft Search instance for crawling one custom app : https://learn.microsoft.com/en-us/microsoftsearch/connectors-overview
Query & display is working as expected but aggregation provides wrong results
query JSON : https://graph.microsoft.com/v1.0/search/query
select title + submitter and aggregation on submitter
"fields": [
"title",
"submitter"
],
"aggregations": [
{
"field": "submitter",
"size": 1,
"bucketDefinition": {
"sortBy": "keyAsString",
"isDescending": true,
"minimumCount": 0
}
}
]
JSON response
submitter property is correctly returned with Firstname Lastname on row 0 but aggregate is lowercase and middle space trimmed firstnamelastname
"hitsContainers": [
{
"total": 1,
"moreResultsAvailable": false,
"hits": [
{
"hitId": "xxxx",
"contentSource": "ConnectionId",
"rank": 1,
"summary": "New service / <c0>business</c0> <c0>model</c0> <c0>design</c0> <ddd/>",
"resource": {
"#odata.type": "#microsoft.graph.externalConnectors.externalItem",
"properties": {
"title": "New service / business model design",
"submitter": "Firstname Lastname"
}
}
}
],
"aggregations": [
{
"field": "submitter",
"buckets": [
{
"key": "firstnamelastname",
"count": 1,
"aggregationFilterToken": "\"ǂǂ696c736573706f656c73747261\""
}
]
}
]
}
]
reproducible in Microsoft Graph Explorer (a bit obfuscated)
result with space
aggregation concatenated in lowercase
Rootcause has been identified as submitter property wasn't created with flag refinable
{
"name": "submitter",
"type": "String",
"isSearchable": "true",
"isQueryable": "true",
"isRetrievable": "true"
"isRefinable": "false"
}
as a consequence, output was incorrect.
testing with refinable = true provides correct aggregation value (1 = non refinable, 2 = refinable).
small note : refinable properties can't be searchable
I'm trying to fetch all users from a specific group via an HTTP connector, a registered app, and Microsoft Graph.
The registered app has Directory.Read.All permissions.
My idea is that I'm calling the nextLink as long as it's there while appending all of the fetched users' userPrincipalName to an array eventually filling the array with all users of the group.
My Logic App looks like this:
Unfortunately, I'm just 1 reputation short of posting images, please forgive. The 3 links should provide an overview of the structure of my app.
First, nextLink is initialized to the first Graph API endpoint. This variable is set to the current nextLink through each iteration of the until loop.
Second, For the purpose of this exercise, I only get the top 5. I know there are only 9 users:
Lastly, I call the union method on the "users" array that I initialized earlier and the "value" array from the HTTP get method, to get one single array consisting of all users:
The issue is that the HTTP action always returns the same top 5 users. I've checked that the nextLink provided in the first HTTP GET call to Graph, is correct by copying it from the Runs history and pasting it into Microsoft Graph Explorer and there the next 4 users are correctly returned.
I also made sure that, for each iteration in the until loop, I call the Graph API with the nextLink from the previous iteration as expected.
The nextLink returned inside of the Logic App is exactly the same when I test it in Graph Explorer, but the same nextLink returns 2 different results when called from Graph Explorer and inside my Logic App.
Why is the result always the same top 5 users and not the next 4 users as expected?
If not sure about the reason why you will get this issue, but based on your requirement, I did a sample below:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Initialize_variable": {
"inputs": {
"variables": [
{
"name": "GetGroupUrl",
"type": "string",
"value": "https://graph.microsoft.com/v1.0/groups/<your group id>/members?$select=userPrincipalName&$top=5"
}
]
},
"runAfter": {},
"type": "InitializeVariable"
},
"Initialize_variable_2": {
"inputs": {
"variables": [
{
"name": "users",
"type": "array"
}
]
},
"runAfter": {
"Initialize_variable": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Until": {
"actions": {
"Compose": {
"inputs": "#union(variables('users'),body('HTTP')['value'])",
"runAfter": {
"HTTP": [
"Succeeded"
]
},
"type": "Compose"
},
"HTTP": {
"inputs": {
"authentication": {
"audience": "https://graph.microsoft.com",
"clientId": "<app id>",
"secret": "<app secret>",
"tenant": "<your secret>",
"type": "ActiveDirectoryOAuth"
},
"method": "GET",
"uri": "#variables('GetGroupUrl')"
},
"runAfter": {},
"type": "Http"
},
"Set_variable": {
"inputs": {
"name": "GetGroupUrl",
"value": "#{if(equals(body('HTTP')?['#odata.nextLink'], null),null,body('HTTP')['#odata.nextLink'])}"
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "SetVariable"
}
},
"expression": "#equals(variables('GetGroupUrl'), '')",
"limit": {
"count": 60,
"timeout": "PT1H"
},
"runAfter": {
"Initialize_variable_2": [
"Succeeded"
]
},
"type": "Until"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"method": "GET",
"schema": {
"properties": {
"text": {
"type": "string"
}
},
"type": "object"
}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {}
}
You can just replace the params with your own and paste it into your logic app code view and test it .
It works for me, as you can see , each request results are different :
Hope it helps .
This issue solved by OP self, this issue is due to queries in request URL , copy OP's comment as an answer :
After fiddling a bit more around with what each of you providing I
found a solution. It seems that when the query arguments are passed to
the HTTP GET outside of the endpoint itself (meaning in the "queries"
field inside of the block) it seems to keep overriding the nextLink.
When writing the endpoint URL out entirely with the odata parameters,
it works as intended.
I am trying to build up a response from a variety of schema components using OpenAPI 3. There are basically three parts to the response:
A shared component that other endpoints use (i.e. success/failure flags). - #/components/schemas/core_response_schema inside allOf.
Properties that all responses on this endpoint use (i.e., user_id) - the properties component of the below.
One of several schemas that will vary depending on the type of user. - the oneOf component.
I've determined that I have to use allOf to be able to mix properties (item 2) and the core response (item 1), though this feels wrong as there's only one item. I tried a $ref, but it didn't work.
The below successfully passes three different OpenAPI linting tools, but in the example it builds, Swagger UI does not show the item 2 things (properties), and does show all of the item 3 things (should be oneOf).
"responses": {
"200": {
"description": "Operation successfully executed.",
"content": {
"application/json": {
"schema": {
"properties": {
"user_id": {
"$ref": "#/components/schemas/user_id"
},
"results": {
"type": "array",
"items": {
"$ref": "#/components/schemas/result_user_by_id"
}
}
},
"type": "object",
"allOf": [
{
"$ref": "#/components/schemas/core_response_schema"
}
],
"oneOf": [
{
"$ref": "#/components/schemas/user_type_a"
},
{
"$ref": "#/components/schemas/user_type_b"
},
{
"$ref": "#/components/schemas/user_type_c"
}
]
}
}
}
}
},
"components": {
"schemas": {
"core_response_schema": {
"properties": {
"success": {
"description": "A flag indicating whether the request was successfully completed or not.",
"type": "boolean"
},
"num_results": {
"description": "The number of results for this request",
"type": "integer"
}
},
"type": "object"
},
"user_id": {
"description": "Unique 10 character `user_id`.",
"type": "string",
"maxLength": 10,
"minLength": 10,
"example": "a1b2c3d4e5"
},
}
}
And example payloads for two users. Type A and B (it's a contrived example).
User Type A:
{
"success": true,
"num_results": 1,
"user_id": "c1b00cb714",
"results": [{
"user_type": "a",
"group_id": "e7a99e3769",
"name": null,
"title": null,
... (and so on until we get to the stuff that's unique to this type of user) ...
"favourite_artworks": [
"sunflowers",
"landscapes"
],
"artwork_urls": [
"http://sunflowers.example"
]
}
]
}
User Type B:
{
"success": true,
"num_results": 1,
"user_id": "c1b00cb715",
"results": [{
"user_type": "B",
"group_id": "e7a99e3769",
"name": null,
"title": null,
... (and so on until we get to the stuff that's unique to this type of user) ...
"supported_charities": [
"UN Foundations"
],
"charity_urls": [
"http://www.un.int"
],
}
]
}
What's the correct way to merge together different schemas and properties in OpenAPI? Is this right and Swagger UI just can't handle it?
And how do you mix a schema with properties without having to use allOf?
This suggests it's possible: Swagger Schema: oneOf, anyOf, allOf valid at the same time?
After further investigation, I've determined this is a bug in swagger-ui - https://github.com/swagger-api/swagger-ui/issues/3803 - they simply don't support oneOf (or anyOf) currently.
As far as at least three different linting tools are concerned, a mixture of anyOf, oneOf, and allOf can be used together in the same schema.
Redoc appears to have similar problems - https://github.com/Rebilly/ReDoc/issues/641
I am trying to store a list of changes made to a Vertex in the Vertex itself. Ideally I would want something like this:
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"log": [{
"user": "user#user.dk",
"action": "update",
"timestamp": "22-03-2017",
"field": "firstName",
"oldValue": "Marco"
}
]
}
Using this method chain I am able to a achieve the following structure
graph.addV('person')
.property('firstName', 'Thomas')
.property(list, 'log', '22-03-2017')
.properties('log')
.hasValue('22-03-2017', '21-03-2017')
.property('user','user#user.dk')
.property('action', 'update')
.property('field', 'firstName')
.property('oldValue', 'Marco')
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"properties": {
"firstName": [{
"id": "f23482a9-48bc-44e0-b783-3b74a2439a11",
"value": "Thomas"
}
],
"log": [{
"id": "5cfa35e1-e453-42e2-99b1-eb64cd853f22",
"value": "22-03-2017",
"properties": {
"user": "user#user.dk",
"action": "update",
"field": "firstName",
"oldValue": "Marco"
}
}
]
}
}
However this seems overly complex, as I will have to store a value and add properties to it.
Is it possible to add anonymous objects (i.e. without id and value) with the above mentioned data?
Not an actual solution to storing proper objects in a history log, but if you just use it as a log and don't have to access or query it by its properties, you could just put the serialised JSON in the value?
Something like along these lines should approximate the structure you're requesting:
dynamic entry = new JObject();
entry.user = "user#user.dk";
entry.action = "update";
entry.timestamp = "22-03-2017 12:34:56";
entry.field = "firstName";
entry.oldValue = "Marco";
graph.addV('person')
.property('firstName', 'Thomas')
.property(list, 'log', entry.ToString());
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"properties": {
"firstName": [{
"id": "f23482a9-48bc-44e0-b783-3b74a2439a11",
"value": "Thomas"
}
],
"log": [{
"id": "5cfa35e1-e453-42e2-99b1-eb64cd853f22",
"value": "{\"user\":\"user#user.dk\",\"action\":\"update\",\"timestamp\":\"22-03-2017\",\"field\":\"firstName\",\"oldValue\":\"Marco\"}"
}
]
}
}
These log entries can easily be read, deserialised, used, and presented, but will not do much for queriability.