Below JSON contains 4 items in an array. If you look at each item it is some what shows incomplete keys. I am unable to figure out how to represent the consistent data in iOS (of any UI design patterns). By looking at the below information somewhat they are interlinked to each other for example parent key value information is same as company key value and also in employees one value is same as name key value. This seems to be very typical.
{
"arays": [
{
"company": "Microchip",
"parent": "File",
"employees": [
"John",
"Mike"
]
},
{
"company": "Apple",
"mobiles": [
"111-111-1111",
"121-121-1212"
]
},
{
"company": "File",
"parent": "Apple",
"addresses": [
"2600 space center blvd",
"2700 university dr"
]
},
{
"name": "John",
"mobiles": [
"222-222-2222"
],
"addresses": [
"Time Square, NY"
]
}
]
}
Related
I'm retrieving data from Microsoft workbook/excel file's table.
Workbook's table has date-of-birth column which has date format.
While retrieving values using Microsoft graph's table api it returns dates as some integer value.
API i'm using:
https://graph.microsoft.com/v1.0/me/drive/root:/book.xlsx:/workbook/tables/table4/columns
Microsoft graph API reference : https://learn.microsoft.com/en-us/graph/api/table-list-columns?view=graph-rest-1.0&tabs=http
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('<user_id>')/drive/root/workbook/tables('table4')/columns",
"value": [
{
"#odata.id": "/users('<user_id>')/drive/root/workbook/tables(%27%7BF44C7573-9BBE-46F3-B0BE-C5607762D2AB%7D%27)/columns(%271%27)",
"values": [
[
"Name"
],
[
"Student 1"
],
[
"Student 2"
]
],
"id": "1",
"index": 0,
"name": "Name"
},
{
"#odata.id": "/users('<user_id>')/drive/root/workbook/tables(%27%7BF44C7573-9BBE-46F3-B0BE-C5607762D2AB%7D%27)/columns(%273%27)",
"values": [
[
"DOB"
],
[
35339
],
[
34249
]
],
"id": "3",
"index": 1,
"name": "DOB"
}
]
}
Original values are
Is there any way to get date values in its original format
The endpoint you are using
GET /me/drive/root:/book.xlsx:/workbook/tables/table4/columns
returns workbookTableColumn resource type which has only information about raw values without information about data type and formatting.
You can use id or name of workbookTableColumn and call dataBodyRange endpoint
GET /me/drive/root:/{item-path}:/workbook/tables/{id|name}/columns/{columnId|columnName}/dataBodyRange
GET /me/drive/root:/book.xlsx:/workbook/tables/{id|name}/columns/{columnId|columnName}/dataBodyRange
which returns range resource type.
Range resource type has property text which represents text values of the specified range.
...
"text": [
[
"09.06.2022"
],
[
""
]
],
...
I am looking to convert JSON to Avro without altering the shape of the data
A nested field in the JSON contains a variable number of keys, which are never known in advance. The record that branches off of each of these unknown nodes however is of known, well-defined shape.
An example of this input data is as shown below
{
"customers": {
"paul_ince": {
"name": "Mr Paul Ince",
"age": 54
},
"kim_kardashian": {
"name": "Ms Kim Kardashian",
"age": 41
},
"elon_musk": {
"name": "Elon Musk, Technoking of Tesla",
"age": 50
}
}
Now it would be more avro friendly of course to have customers lead to an array of the form
{
"customers": [
{
"customer_name": "paul_ince",
"name": "Mr Paul Ince",
"age": 54
},
{
...
}
]
}
But this would violate my constraint that the shape of the input data be unchanged.
This problem seems to manifest itself frequently if I rely on data from external sources or scrapes, or preexisting data that was never created for Avro.
In my head, the schema would look something like the below,
{
"fields": [
{
"name": "customers",
"type": {
"type": "record",
"name": "customers",
"fields": [
{
"name": $customer_name,
"type": {
"type": "record",
"name": $customer_name,
"fields": [
{
"name": "name",
"type": "string",
},
{
"name": "age",
"type": "int"
}
]
}
}
]
}
}
]
}
where $customer_name is an assignment value, defined on read. Just asking the question it feels like this violates fundamental avro but I must use Avro and I strongly desire to maintain the input shape of the data. It would be highly impractical to modify this, not least given how frequently this problem appears and how large and varied the data I need to transfer from JSON to Avro is
We have an Microsoft Search instance for crawling one custom app : https://learn.microsoft.com/en-us/microsoftsearch/connectors-overview
Query & display is working as expected but aggregation provides wrong results
query JSON : https://graph.microsoft.com/v1.0/search/query
select title + submitter and aggregation on submitter
"fields": [
"title",
"submitter"
],
"aggregations": [
{
"field": "submitter",
"size": 1,
"bucketDefinition": {
"sortBy": "keyAsString",
"isDescending": true,
"minimumCount": 0
}
}
]
JSON response
submitter property is correctly returned with Firstname Lastname on row 0 but aggregate is lowercase and middle space trimmed firstnamelastname
"hitsContainers": [
{
"total": 1,
"moreResultsAvailable": false,
"hits": [
{
"hitId": "xxxx",
"contentSource": "ConnectionId",
"rank": 1,
"summary": "New service / <c0>business</c0> <c0>model</c0> <c0>design</c0> <ddd/>",
"resource": {
"#odata.type": "#microsoft.graph.externalConnectors.externalItem",
"properties": {
"title": "New service / business model design",
"submitter": "Firstname Lastname"
}
}
}
],
"aggregations": [
{
"field": "submitter",
"buckets": [
{
"key": "firstnamelastname",
"count": 1,
"aggregationFilterToken": "\"ǂǂ696c736573706f656c73747261\""
}
]
}
]
}
]
reproducible in Microsoft Graph Explorer (a bit obfuscated)
result with space
aggregation concatenated in lowercase
Rootcause has been identified as submitter property wasn't created with flag refinable
{
"name": "submitter",
"type": "String",
"isSearchable": "true",
"isQueryable": "true",
"isRetrievable": "true"
"isRefinable": "false"
}
as a consequence, output was incorrect.
testing with refinable = true provides correct aggregation value (1 = non refinable, 2 = refinable).
small note : refinable properties can't be searchable
For the Discovery REST api, the argument/parameter "return" controls which fields are returned.
So if I pass these arguments to the API
{
"query": named_sector,
"count": "10",
"filter": filter_dates,
"aggregation" : "term(docSentiment.type,count:3)"
}
my_query = discovery.query(my_disc_environment_id, my_disc_collection_id, qopts)
print(json.dumps(my_query, indent=2))
I get the following:
{
"matching_results": 14779,
"aggregations": [
{
"type": "term",
"field": "docSentiment.type",
"count": 3,
"results": [
{
"key": "positive",
"matching_results": 4212
},
{
"key": "negative",
"matching_results": 3259
},
{
"key": "neutral",
"matching_results": 152
}
]
}
],
"results": [
{
"id": "6389715fe7e7f711e0bc09d4f1236639",
"score": 1.3689895,
"yyyymm": "201704",
"url": "https://seekingalpha.com/article/4060446-valuation-dashboard-consumer-discretionary-update",
"enrichedTitle": null,
"host": "seekingalpha.com",
"text": "Valuation Dashboard: Consumer Discretionary - Update\n\nSummary\n\nValuation metrics in Consumer Discretionary.\n\nEvolution since last month.\n\nA list of stocks loo ....
and thousands of more lines. How do I restrict the output to the aggregations section? Is this an issue of me better handling the JSON structure that is returned?
thanks
If you change the count argument to 0, the returned JSON will only contain the aggregations.
Also, if you're using the Discovery web tooling, you can enter 0 for the "Number of results to return (Count)" field.
More details and an example can be found here: https://www.ibm.com/watson/developercloud/doc/discovery/using.html#building-aggregations
I have a simple "rss" (ApplicationRecord) table indexed by an id. I would like to have a structured JSON that group each user from a family in an array structure. And then each family in a global array. How can I do that ?
my current plain code to put my data in a json file is :
json.rss #rss do |rs|
json.id rs.id
json.name rs.name
json.family rs.family
json.lastdate rs.lastdate
json.last rs.last
json.s1w rs.s1w
json.s2w rs.s2w
end
But the target file that I want is this one :
{
"rss": [
{
"familyname": "Smith",
"children": [
{
"id": "1",
"name": "bob",
"lastdate": "2010-09-23",
"last": "0.88",
"s1w": "0.83",
"s2w": "0.88"
},
{
"id": 2,
"name": "Mary",
"lastdate": "2011-09-23",
"last": "0.89",
"s1w": "0.83",
"s2w": "0.87"
}
]
},
{
"familyname": "Wesson",
"children": [
{
"id": "1",
"name": "john",
"lastdate": "2001-09-23",
"last": "0.88",
"s1w": "0.83",
"s2w": "0.88"
},
{
"id": 2,
"name": "Bruce",
"lastdate": "2000-09-23",
"last": "0.89",
"s1w": "0.83",
"s2w": "0.87"
}
]
}
]
}
The grouping you are trying to achieve can be done in Ruby with:
#rss.group_by(&:family).values
This is assuming #rss is an array-like collection of objects that have a .family method. The result: is an array of arrays of objects grouped by family.
Now it will be up to use to use Jbuilder's array! method to build the desired JSON output.