OPA masking a dynamic array field - open-policy-agent

I'm trying to apply masking on an input and result field that is part of an array. And the size of the array is dynamic. Based on the documentation, it is instructed to provide absolute array index which is not possible in this use case. Do we have any alternative?
Eg. If one needs to mask the age field of all the students from the input document?
Input:
"students" : [
{
"name": "Student 1",
"major": "Math",
"age": "18"
},
{
"name": "Student 2",
"major": "Science",
"age": "20"
},
{
"name": "Student 3",
"major": "Entrepreneurship",
"age": "25"
}
]

If you want to just generate a copy of input that has a field (or set of fields) removed from the input, you can use json.remove. The trick is to use a comprehension to compute the list of paths to remove. For example:
paths_to_remove := [sprintf("/students/%v/age", [x]) | some x; input.students[x]]
result := json.remove(input, paths_to_remove)
If you are trying to mask fields from the input document in the decision log using the Decision Log Masking feature then you would write something like:
package system.log
mask[x] {
some i
input.input.students[i]
x := sprintf("/input/students/%v/age", [i])
}

Related

How to get only matched data from nested class using query builder in elastic search?

I am trying to get only the matched data from nested array of elastic search class. but I am not able to get it..the whole nested array data is being returned as output.
this is my Query:-
QueryBuilders.nestedQuery("questions",
QueryBuilders.boolQuery()
.must(QueryBuilders.matchQuery("questions.questionTypeId", quesTypeId)), ScoreMode.None)
.innerHit(new InnerHitBuilder());
I am using querybuilders to get data from nested class.Its working fine but not able to get only the matched data.
Request Body :
{
"questionTypeId" : "MCMC"
}
when questionTypeId = "MCMC"
this is the output i am getting..Here I want to exclude the output for which the questionTypeId = "SCMC".
output :
{
"id": "46",
"subjectId": 1,
"topicId": 1,
"subtopicId": 1,
"languageId": 1,
"difficultyId": 4,
"isConceptual": false,
"examCatId": 3,
"examId": 1,
"usedIn": 1,
"questions": [
{
"id": "46_31",
"pid": 31,
"questionId": "QID41336691",
"childId": "CID1",
"questionTypeId": "MCMC",
"instruction": "This is a single correct multiple choice question.",
"question": "Who holds the most english premier league titles?",
"solution": "Manchester United",
"status": 1000,
"questionTranslation": []
},
{
"id": "46_33",
"pid": 33,
"questionId": "QID41336677",
"childId": "CID1",
"questionTypeId": "SCMC",
"instruction": "This is a single correct multiple choice question.",
"question": "Who holds the most english premier league titles?",
"solution": "Manchester United",
"status": 1000,
"questionTranslation": []
}
]
}
As you have tagged this with spring-data-elasticsearch:
Support to return inner hits was recently added to version 4.1.M1 and so will be included in the next released version. Then in a SearchHit you will get the complete top level document, but in the innerHits property only the matching inner hits will be returned.

After adding arrays together, is there a way to know which elements came from which parent array?

Here's the JSON i'm working with:
{
"featured": [
{
"name": "Featured Show number 1",
"id": "123",
"slug": "featured-show-number-one",
"description": "This is an item description for show number 1"
},
{
"name": "Featured Show number 2",
"id": "456",
"slug": "featured-show-nubmer-tow",
"description": "This is an item description for show number 2"
}
],
"nonfeatured": [
{
"name": "Show number 3",
"id": "789",
"slug": "show-number-three",
"description": "This is an item description for show number 3"
},
{
"name": "Show number 4",
"id": "135",
"slug": "show-number-four",
"description": "This is an item description for show number 4"
}
]
}
What I am trying to figure out is after I parse this JSON using two data models, one for "Featured" and one for "Nonfeatured", looping through each show and adding it to an array, I need to add the arrays of shows together to create one array containing all the shows. However, I need to keep track of which shows are featured and which ones are non featured from the single array. Is there a way to do this?
The short answer to your specific question here is 'no'. The result of adding array A and array B (where both contain the same types) is A + B; there is no metadata providing any kind of source information.
But that is not to say that you couldn't accomplish the same thing by changing the model slightly. One option would be to add an extra boolean flag to the model called isFeatured or similar. Or you could 'future-proof' any work by using an enumeration of source lists containing featured, non-featured plus anything else you may require later.
To take the first example, an option would be to add the boolean field and then call code similar to below prior to 'summing' the arrays.
arrayA.forEach { $0.isFeatured = true }
arrayB.forEach { $0.isFeatured = false }
let arrayC = arrayA + arrayB
Then each element in the summed array will tell you its source list.

How to create Neo4J relationship between nodes yelp dataset

I am new to Neo4j. I am trying to populate Yelp dataset in Neo4j. Basically, I am interested in three json file provided by them i.e.
user.json
{
"user_id": "-lGwMGHMC_XihFJNKCJNRg",
"name": "Gabe",
"review_count": 277,
"yelping_since": "2014-10-31",
"friends": ["Oa84FFGBw1axX8O6uDkmqg", "SRcWERSl4rhm-Bz9zN_J8g", "VMVGukgapRtx3MIydAibkQ", "8sLNQ3dAV35VBCnPaMh1Lw", "87LhHHXbQYWr5wlo5W7_QQ"],
"useful": 45,
"funny": 4,
"cool": 55,
"fans": 17,
"elite": [],
"average_stars": 4.72,
"compliment_hot": 5,
"compliment_more": 1,
"compliment_profile": 0,
"compliment_cute": 1,
"compliment_list": 0,
"compliment_note": 11,
"compliment_plain": 20,
"compliment_cool": 15,
"compliment_funny": 15,
"compliment_writer": 1,
"compliment_photos": 8
}
I have omitted several entries from friends array to make output readable
business.json
{
"business_id": "YDf95gJZaq05wvo7hTQbbQ",
"name": "Richmond Town Square",
"neighborhood": "",
"address": "691 Richmond Rd",
"city": "Richmond Heights",
"state": "OH",
"postal_code": "44143",
"latitude": 41.5417162,
"longitude": -81.4931165,
"stars": 2.0,
"review_count": 17,
"is_open": 1,
"attributes": {
"RestaurantsPriceRange2": 2,
"BusinessParking": {
"garage": false,
"street": false,
"validated": false,
"lot": true,
"valet": false
},
"BikeParking": true,
"WheelchairAccessible": true
},
"categories": ["Shopping", "Shopping Centers"],
"hours": {
"Monday": "10:00-21:00",
"Tuesday": "10:00-21:00",
"Friday": "10:00-21:00",
"Wednesday": "10:00-21:00",
"Thursday": "10:00-21:00",
"Sunday": "11:00-18:00",
"Saturday": "10:00-21:00"
}
}
review.json
{
"review_id": "VfBHSwC5Vz_pbFluy07i9Q",
"user_id": "-lGwMGHMC_XihFJNKCJNRg",
"business_id": "YDf95gJZaq05wvo7hTQbbQ",
"stars": 5,
"date": "2016-07-12",
"text": "My girlfriend and I stayed here for 3 nights and loved it.",
"useful": 0,
"funny": 0,
"cool": 0
}
As we can see in the sample files that relationship between user and business is associated via the review.json file. How can I create a relationship edge between user and business using the review.json file.
I have also seen Mark Needham tutorial where he has shown StackOverflow data population but in that case, relationship file was already present with sample data. Do I need to build a similar file? If yes, how should I approach this problem? or is there any other way to build relationship between user & business?
It very much depends on your model as to what you want to do, but you could do 3 imports:
//Create Users - does assume the data is unique
CALL apoc.load.json('file:///c://temp//SO//user.json') YIELD value AS user
CREATE (u:User)
SET u = user
then add the businesses:
CALL apoc.load.json('file:///c://temp//SO//business.json') YIELD value AS business
CREATE (b:Business {
business_id : business.business_id,
name : business.name,
neighborhood : business.neighborhood,
address : business.address,
city : business.city,
state : business.state,
postal_code : business.postal_code,
latitude : business.latitude,
longitude : business.longitude,
stars : business.stars,
review_count : business.review_count,
is_open : business.is_open,
categories : business.categories
})
For the businesses, we can't just do the SET b = business because the JSON has nested maps. So you might want to decide if you want them, and might have to go down a different route.
Lastly, the reviews, which is where we join it all up.
CALL apoc.load.json('file:///c://temp//SO//review.json') YIELD value AS review
CREATE (r:Review)
SET r = review
WITH r
//Match user to a review
MATCH (u:User {user_id: r.user_id})
CREATE (u)-[:HAS_REVIEW]->(r)
WITH r, u
//Match business to a review, and a user to a business
MATCH (b:Business {business_id: r.business_id})
//Merge here in case of multiple reviews
MERGE (u)-[:HAS_REVIEWED]->(b)
CREATE (b)-[:HAS_REVIEW]->(r)
Obviously - change labels/relationships to types you want, and it might need tuning depending on the size of data etc, so you might need to use apoc.periodic.iterate to work it.
Apoc is here if you need it (and you should use it!)

Swift Filter same location coordinates contained inside an array of objects

I am receiving from Server, a list of Stores information in an array of objects. Below is a sample -
"stores": [
{
"name": “Store 1”,
"number": "5381",
"country": "BELGIE",
"latLng": [
50.730614,
4.231847
]
},
{
"name": "Store 2”,
"number": "5220",
"country": "BELGIE",
"latLng": [
50.730614,
4.231847
]
},
{
"name": "Store 3”,
"number": "3982”,
"country": "BELGIE",
"latLng": [
50.7315706,
4.2303477
]
},
{
"name": "Store 4”,
"number": "4179",
"country": "BELGIE",
"latLng": [
50.7262577,
4.245589
]
}]
What am I trying?:
I need to filter out the stores in the array that has same latLng values.
Why?
I need to identify these 'same latLng' values and add an offset of some value like 0.001 to the latitude value so that when I show these stores on a map, the stores on same location appear side by side.
I found this (answer by Rob B) as reference for this approach.
What I need?
1. How can I filter with values inside of an object in the array?
I tried something like this inside a for loop -
print("\(allStoresInfo.filter({ $0.latLng![0] == $0.latLng![0] }).count)")
This value always returns 4. I know I am missing some basic sense here but need know what it is :-(
After I filter and add the offset for same values, how do I update my array with these updated values?
The following method modifies in place the latitude of each store that matches the latitude of another store:
allStoresInfo.map{ currentStore in allStoresInfo.filter{$0.latLng![0] == currentStore.latLng![0]}.enumerated().forEach{ index, matchingStore in
matchingStore.latLng![0] += Double(index)*0.001
}
}
Just a small piece of advice: don't store lat-long values in an array. Either create a struct/class for them or use a tuple to store them.

JSON-LD normalization - ignore JSON nesting

I'm working on JSON-LD serialization, and ideally I would like to have a #context which I can add to the existing GeoJSON output (together with some #ids and #types), so that both the Turtle output and the JSON-LD output will normalize to the same triples.
Data is organized as follows: each object/feature has an ID and a name, and data on one or more layers. Per layer, there is a data field, which contains a JSON object.
Example GeoJSON output:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"id": "admr.nl.appingedam",
"name": "Appingedam",
"layers": {
"cbs": {
"data": {
"name": "Appingedam",
"population": 1092
}
},
"admr": {
"data": {
"name": "Appingedam",
"gme_code": 4654,
"admn_level": 3
}
}
}
},
"geometry": {…}
}
]
}
Example Turtle output:
<admr.nl.appingedam>
a :Node ;
dc:title "Appingedam" ;
:createdOnLayer <layer/admr> ;
:layerData <admr.nl.appingedam/admr> ;
:layerData <admr.nl.appingedam/cbs> .
<admr.nl.appingedam/admr>
a :LayerData ;
:definedOnLayer <layer/admr> ;
<layer/admr/name> "Appingedam" ;
<layer/admr/gme_code> "4654" .
<layer/admr/admn_level> "3" .
<admr.nl.appingedam/cbs>
a :LayerData ;
:definedOnLayer <layer/cbs> ;
<layer/cbs/name> "Appingedam" ;
<layer/cbs/population> "1092" ;
The properties object does not have its own URI. Is there a way to create a JSON-LD context which takes the contents of the properties into account, but further 'ignores' its precence?
Answered by Gregg Kellogg on JSON-LD mailing list:
This is something that keeps coming up: having a transparent layer,
that basically folds properties up a level. This was discussed during
the development of JSON-LD, but ultimately it was rejected.
I don't see any prospects for doing something in the short-term, but
it could be revisited in a possible future WG chartered with revising
the spec. Feedback like this is quite useful.
In the mean time, you can play with different JSON-LD encodings that
match your RDF though tools like http://json-ld.org/playground and my
own http://rdf.greggkellogg.net/distiller.
Gregg

Resources