i tried to poll tasks with hearts opt_field. In the documentation the specified json structure was
hearts [ { id: 1245, name: "Mittens" }, ... ]
but when when polled we get,
"hearts": [
{
"id": 17818qwerty545640,
"user": {
"id": 104764787853157345,
"name": "Gurdy Code"
}
},
{
"id": 17827xxxxuyyyy34676,
"user": {
"id": 1zxxcv672489,
"name": "Boumer Weckhunt"
}
}
]
does the id in the outer brackets have any usage?
The outer ids are the IDs of the hearts themselves - they are in a sense resources in their own right. They have a user and a created_at, which is the time when the user hearted the resource in question. You're absolutely correct that the documentation needs to be updated.
Related
I am looking to convert JSON to Avro without altering the shape of the data
A nested field in the JSON contains a variable number of keys, which are never known in advance. The record that branches off of each of these unknown nodes however is of known, well-defined shape.
An example of this input data is as shown below
{
"customers": {
"paul_ince": {
"name": "Mr Paul Ince",
"age": 54
},
"kim_kardashian": {
"name": "Ms Kim Kardashian",
"age": 41
},
"elon_musk": {
"name": "Elon Musk, Technoking of Tesla",
"age": 50
}
}
Now it would be more avro friendly of course to have customers lead to an array of the form
{
"customers": [
{
"customer_name": "paul_ince",
"name": "Mr Paul Ince",
"age": 54
},
{
...
}
]
}
But this would violate my constraint that the shape of the input data be unchanged.
This problem seems to manifest itself frequently if I rely on data from external sources or scrapes, or preexisting data that was never created for Avro.
In my head, the schema would look something like the below,
{
"fields": [
{
"name": "customers",
"type": {
"type": "record",
"name": "customers",
"fields": [
{
"name": $customer_name,
"type": {
"type": "record",
"name": $customer_name,
"fields": [
{
"name": "name",
"type": "string",
},
{
"name": "age",
"type": "int"
}
]
}
}
]
}
}
]
}
where $customer_name is an assignment value, defined on read. Just asking the question it feels like this violates fundamental avro but I must use Avro and I strongly desire to maintain the input shape of the data. It would be highly impractical to modify this, not least given how frequently this problem appears and how large and varied the data I need to transfer from JSON to Avro is
I'm trying to fetch all users from a specific group via an HTTP connector, a registered app, and Microsoft Graph.
The registered app has Directory.Read.All permissions.
My idea is that I'm calling the nextLink as long as it's there while appending all of the fetched users' userPrincipalName to an array eventually filling the array with all users of the group.
My Logic App looks like this:
Unfortunately, I'm just 1 reputation short of posting images, please forgive. The 3 links should provide an overview of the structure of my app.
First, nextLink is initialized to the first Graph API endpoint. This variable is set to the current nextLink through each iteration of the until loop.
Second, For the purpose of this exercise, I only get the top 5. I know there are only 9 users:
Lastly, I call the union method on the "users" array that I initialized earlier and the "value" array from the HTTP get method, to get one single array consisting of all users:
The issue is that the HTTP action always returns the same top 5 users. I've checked that the nextLink provided in the first HTTP GET call to Graph, is correct by copying it from the Runs history and pasting it into Microsoft Graph Explorer and there the next 4 users are correctly returned.
I also made sure that, for each iteration in the until loop, I call the Graph API with the nextLink from the previous iteration as expected.
The nextLink returned inside of the Logic App is exactly the same when I test it in Graph Explorer, but the same nextLink returns 2 different results when called from Graph Explorer and inside my Logic App.
Why is the result always the same top 5 users and not the next 4 users as expected?
If not sure about the reason why you will get this issue, but based on your requirement, I did a sample below:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Initialize_variable": {
"inputs": {
"variables": [
{
"name": "GetGroupUrl",
"type": "string",
"value": "https://graph.microsoft.com/v1.0/groups/<your group id>/members?$select=userPrincipalName&$top=5"
}
]
},
"runAfter": {},
"type": "InitializeVariable"
},
"Initialize_variable_2": {
"inputs": {
"variables": [
{
"name": "users",
"type": "array"
}
]
},
"runAfter": {
"Initialize_variable": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Until": {
"actions": {
"Compose": {
"inputs": "#union(variables('users'),body('HTTP')['value'])",
"runAfter": {
"HTTP": [
"Succeeded"
]
},
"type": "Compose"
},
"HTTP": {
"inputs": {
"authentication": {
"audience": "https://graph.microsoft.com",
"clientId": "<app id>",
"secret": "<app secret>",
"tenant": "<your secret>",
"type": "ActiveDirectoryOAuth"
},
"method": "GET",
"uri": "#variables('GetGroupUrl')"
},
"runAfter": {},
"type": "Http"
},
"Set_variable": {
"inputs": {
"name": "GetGroupUrl",
"value": "#{if(equals(body('HTTP')?['#odata.nextLink'], null),null,body('HTTP')['#odata.nextLink'])}"
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "SetVariable"
}
},
"expression": "#equals(variables('GetGroupUrl'), '')",
"limit": {
"count": 60,
"timeout": "PT1H"
},
"runAfter": {
"Initialize_variable_2": [
"Succeeded"
]
},
"type": "Until"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"method": "GET",
"schema": {
"properties": {
"text": {
"type": "string"
}
},
"type": "object"
}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {}
}
You can just replace the params with your own and paste it into your logic app code view and test it .
It works for me, as you can see , each request results are different :
Hope it helps .
This issue solved by OP self, this issue is due to queries in request URL , copy OP's comment as an answer :
After fiddling a bit more around with what each of you providing I
found a solution. It seems that when the query arguments are passed to
the HTTP GET outside of the endpoint itself (meaning in the "queries"
field inside of the block) it seems to keep overriding the nextLink.
When writing the endpoint URL out entirely with the odata parameters,
it works as intended.
For the Discovery REST api, the argument/parameter "return" controls which fields are returned.
So if I pass these arguments to the API
{
"query": named_sector,
"count": "10",
"filter": filter_dates,
"aggregation" : "term(docSentiment.type,count:3)"
}
my_query = discovery.query(my_disc_environment_id, my_disc_collection_id, qopts)
print(json.dumps(my_query, indent=2))
I get the following:
{
"matching_results": 14779,
"aggregations": [
{
"type": "term",
"field": "docSentiment.type",
"count": 3,
"results": [
{
"key": "positive",
"matching_results": 4212
},
{
"key": "negative",
"matching_results": 3259
},
{
"key": "neutral",
"matching_results": 152
}
]
}
],
"results": [
{
"id": "6389715fe7e7f711e0bc09d4f1236639",
"score": 1.3689895,
"yyyymm": "201704",
"url": "https://seekingalpha.com/article/4060446-valuation-dashboard-consumer-discretionary-update",
"enrichedTitle": null,
"host": "seekingalpha.com",
"text": "Valuation Dashboard: Consumer Discretionary - Update\n\nSummary\n\nValuation metrics in Consumer Discretionary.\n\nEvolution since last month.\n\nA list of stocks loo ....
and thousands of more lines. How do I restrict the output to the aggregations section? Is this an issue of me better handling the JSON structure that is returned?
thanks
If you change the count argument to 0, the returned JSON will only contain the aggregations.
Also, if you're using the Discovery web tooling, you can enter 0 for the "Number of results to return (Count)" field.
More details and an example can be found here: https://www.ibm.com/watson/developercloud/doc/discovery/using.html#building-aggregations
I am reading this before building an API endpoints. I read this quote about compound documents:
To reduce the number of HTTP requests, servers MAY allow responses
that include related resources along with the requested primary
resources. Such responses are called "compound documents".
Here is a sample JSON response using the JSON API specification:
{
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!"
},
"links": {
"self": "http://example.com/articles/1"
},
"relationships": {
"author": {
"links": {
"self": "http://example.com/articles/1/relationships/author",
"related": "http://example.com/articles/1/author"
},
"data": { "type": "people", "id": "9" }
},
"comments": {
"links": {
"self": "http://example.com/articles/1/relationships/comments",
"related": "http://example.com/articles/1/comments"
},
"data": [
{ "type": "comments", "id": "5" },
{ "type": "comments", "id": "12" }
]
}
}
}],
"included": [{
"type": "people",
"id": "9",
"attributes": {
"first-name": "Dan",
"last-name": "Gebhardt",
"twitter": "dgeb"
},
"links": {
"self": "http://example.com/people/9"
}
}, {
"type": "comments",
"id": "5",
"attributes": {
"body": "First!"
},
"relationships": {
"author": {
"data": { "type": "people", "id": "2" }
}
},
"links": {
"self": "http://example.com/comments/5"
}
}, {
"type": "comments",
"id": "12",
"attributes": {
"body": "I like XML better"
},
"relationships": {
"author": {
"data": { "type": "people", "id": "9" }
}
},
"links": {
"self": "http://example.com/comments/12"
}
}]
}
So from what I can see, the relationships sections give basic/sparse information about the associations between the articles table and other tables. It looks like an article belongs_to an author and has_many comments.
What will the links be used for? Will the API have to use the link in order to receive more detailed JSON about the relationship? Doesn't this require an additional API call? Is this efficient?
The "included" section seems like it contains more detailed information about the relationships/associations?
Are both "included" and "relationships" necessary? What's the intuition behind needing both of these sections?
The idea is that a relationship in a resource simply gives linkage data (that is basic data to uniquely identify the related resource – these data are the id and the type), in order to keep it to a minimum.
On the other hand, the included section is here in case you want to send along detailed information about some related resources (for instance to minimise the number of HTTP requests). Note that the included section is expected to contain only resources that are related to either a primary resource (i.e. within the data section), or an included resource (this constraint is called full linkage in the spec).
To put it simply, the relationships section of a resource tell you which resources are related to a given resource, and the included section tells you what those resources are.
As far as links are concerned, they may come in handy when you have a has_many relationship, for which the linkage data itself might contain several thousands of id/type records, thus making your response document quite big. In case those are not necessarily needed by your client when they request the base resource, you might decide to make them available through a link.
I'm wondering if a json can be partially paginate.
For example
{
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!",
"body": "The shortest article. Ever."
}
}],
"included": [
{
"type": "people",
"id": 42,
"attributes": {
"name": "John"
}
},
{
...annnd 80000 others
}
}
]
}
Where included have soo many elements (80.000 for examples) than maybe we need pagination?
But if it's paginate and we go on the next page only included elements will change, the json will still return the data.articles.
Is it a correct behavior ?
First proposal :
{
"data": [{
"type": "articles",
"id": "1",
"attributes": {
"title": "JSON API paints my bikeshed!",
"body": "The shortest article. Ever."
},
"relationships": {
"users": {
"link": "https://website.com/api/v1/articles/1/users.json"
}
}
}]
}
To be compliant with the JSON API spec, your compound document must obey the full linkage requirement. Any included resources MUST be identified via relationship data.
In your example, you could fulfill this by adding a data member under the users relationship. You could then link to every included person.
If the relationship data is a partial set, you can use pagination links within the relationship object.