Loop through a nested json object in OPA - open-policy-agent

I am very new to OPA and trying to loop through the below input data
input =
"data":{
"list":[
{"Name": "abc", "Status": "Done"},
{"Name": "def", "Status": "Done"},
{"Name": "ghi", "Status": "pending"},
{"Name": "jkl", "Status": ""},
{"Name": "mno", "Status": null},
]
}
and return two lists based on that input, one list that would return the names of all objects that has the status as 'Done' and another list with status not equal to 'Done', here is the rego code I was trying, I used somewhat python syntax to convey the code as I was not sure how the opa syntax would look like
package play
default done_list := []
default other_list := []
done_list {
some i
input.data.list[i].Status == "Done"
done_list.append(input.data.list[i].Name) #python syntax and looking for something similar in opa
}
other_list{
some j
input.data.list[j].Status != "Done"
other_list.append(input.data.list[j].Name)
}
#or maybe use a list comprehension like this?
#not even sure if this makes sense but ...
done_list = [x.Name | x := input.data.list[_]; x.Status = "Done"]
other_list = [x.Name | x := input.data.list[_]; x.Status != "Done"]
test_return{
"done_list": done_list,
"other_list": other_list
}
and the output I'm looking for is something like
{
"done_list": ["abc", "def"],
"other_list": ["ghi","jkl","mno"]
}
Any help is greatly appreciated. Thanks in advance.

I think you're best to use a list comprehension as you guessed :)
package play
doneStatus := "Done"
result := {
"done_list": [e |
item := input.data.list[_]
item.Status == doneStatus
e := item.Name
],
"other_list": [e |
item := input.data.list[_]
item.Status != doneStatus
e := item.Name
],
}
I've created a Rego playground which shows this: https://play.openpolicyagent.org/p/dBKUXFO3v2

Related

Open Policy Agent reduce array when element in array

I have been trying to achieve something with Open Policy Agent that I am sure should be possible. Just struggling with the on-ramp to the Rego language (I think). I am using the playground to get a feel for how to achieve the following.
I have the following data input.
{
"tags": [
{
"key": "test"
},
{
"key": "test2"
}
]
}
Rego code
package play
minimum_tags = {"test","test2"}
deny[msg] {
tags := input.tags[_][key]
# At this point I have an array and a set. I can convert the minimum_tags to Array
# But I can't really figure out how to do an iteration to check each tags value is in minimum_tags. Or reduce the minimum tags until its empty
}
I can only see the ability to reduce a set using the a1 - a2 built in. Doesn't seem to be a way to effect change on an Array
I think the idiomatic approach would be to convert tags to a set as well, so that you can use set operations (like you suggest) to check that all tags from minimum_tags are included in tags:
deny[msg] {
tags := {tag | tag := input.tags[_].key}
missing_tags := minimum_tags - tags
count(missing_tags) > 0
msg := sprintf("Missing tags: %v", [concat(", ", missing_tags)])
}
If you really want to have tags as an array, you could do something like this:
deny[msg] {
tags := [tag | tag := input.tags[_].key]
required_tag := minimum_tags[_]
not in_array(required_tag, tags)
msg := sprintf("Missing tag: %v", [required_tag])
}
in_array(item, arr) {
item == arr[_]
}

OPA/Rego execute function for each element of an array

I am new at OPA/Rego and I am trying to write a policy to check if an Azure Network Security Group contains all the rules that I define on an array
package sample
default compliant = false
toSet(arr) = {x | x := arr[_]}
checkProperty(rule, index, propertySingular, propertyPlural) = true
{
object.get(input.properties.securityRules[index].properties, propertySingular, "") == object.get(rule, propertySingular, "")
count(toSet(object.get(input.properties.securityRules[index].properties, propertyPlural, [])) - toSet(object.get(rule, propertyPlural, []))) == 0
}
existRule(rule) = true
{
input.properties.securityRules[i].name == rule.name
input.properties.securityRules[i].properties.provisioningState == rule.provisioningState
input.properties.securityRules[i].properties.description == rule.description
input.properties.securityRules[i].properties.protocol == rule.protocol
checkProperty(rule, i, "sourcePortRange", "sourcePortRanges")
checkProperty(rule, i, "destinationPortRange", "destinationPortRanges")
checkProperty(rule, i, "sourceAddressPrefix", "sourceAddressPrefixes")
checkProperty(rule, i, "destinationAddressPrefix", "destinationAddressPrefixes")
input.properties.securityRules[i].properties.access == rule.access
input.properties.securityRules[i].properties.priority == rule.priority
input.properties.securityRules[i].properties.direction == rule.direction
}
compliant
{
rules := [
{
"name": "name1",
"provisioningState": "Succeeded",
"description": "description1",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "53",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 1,
"direction": "Inbound",
"sourceAddressPrefixes":
[
"xx.xx.xx.xx",
"xx.xx.xx.xx",
"xx.xx.xx.xx"
],
},
{
"name": "name2",
"provisioningState": "Succeeded",
"description": "description2",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "54",
"sourceAddressPrefix": "*",
"access": "Allow",
"priority": 2,
"direction": "Outbound",
"destinationAddressPrefixes":
[
"xx.xx.xx.xx",
"xx.xx.xx.xx",
"xx.xx.xx.xx"
]
}
]
#checks
existRule(rules[i])
}
The issue seem to be that when execute existRule(rules[i]) if one of the rules match it returns true, don't mather if other rules doesn't
If I replace existRule(rules[i]) with existRule(rules[0]) or existRule(rules[1]), it return true or false depending on if the rule on that position matchs.
Is there any way to get the result of the execution of existRule(rules[i]) for all the elements of the array?
I already tried result := [existRule(rules[i])] but it only return one element with true
Sure! Use a list comprehension and call the function inside of it, then compare the size of the result to what you had before. Given your example, you would replace existRule(rules[i]) with something like this:
compliantRules := [rule | rule := rules[_]
existRule(rule)]
count(compliantRules) == count(rules)

Open Policy Agent - Improve performance of a grouping comprehension

I have a id and role mapping in below format
{
"ra": [
{
"id": 168,
"code": "AFAP"
},
{
"id": 180,
"code": "ABC"
},
{
"id": 180,
"code": "ABCMND"
}
]
}
I need the output to be like below
{
"roleactions": {
"168": [
"AFAP"
],
"180": [
"ABC",
"ABCMND",
"DCRMP"
]
}
}
So i wrote the below code
roleactions = r_map {
r := data.ra
r_map := {id: list |
some i
id := r[i].id
list := [obj |
some j
r[j].id == id
obj := r[j].code
]
}
}
But when I run this for it almost takes 5-6 seconds
Found 1 result in 5682526.465 µs.
Can someone guide on how to make write this policy map to improve performance?
OPA can evaluate comprehensions like this in linear-time: https://www.openpolicyagent.org/docs/latest/policy-performance/#comprehension-indexing. The problem in this case is that the local variable r is not safe when considering the comprehensions in isolation.
If you refactor the comprehension like below, the runtime should be linear:
roleactions := r_map {
r_map := {id: list |
some i
id := data.ra[i].id
list := [obj |
some j
data.ra[j].id == id
obj := data.ra[j].code
]
}
}

Mapping items in an array

Hoping this is a nice easy one, but I just can't see how to do it.
I am wanting to with rego map items in an array to a cleaner version. For example from the data below
data = [
{
"some": "value",
"another": "mvalue",
"dont": "want"
},
{
"some": "value1",
"another": "mvalue1",
"dont": "want1"
},
{
"some": "value2",
"another": "mvalue2",
"dont": "want2"
}
]
I want to turn data into
result = [
{
"some": "value",
"another": "mvalue"
},
{
"some": "value1",
"another": "mvalue1"
},
{
"some": "value2",
"another": "mvalue2"
}
]
The two closest I think I've got is
result1 = cleaned {
cleaned := {d |
d := {
"some": data[_].some,
"another": data[_].another
}
}
}
result2 = cleaned {
d := data[_]
cleaned := {
"some": p.some,
"another": p.another
}
}
TLDR; If the fields are static and you can easily enumerate them, both of your solutions is almost correct (see below for explanation of why they are incorrect.) Here's the right way to do that:
result = [
mapped |
original := data[_]
mapped := {"some": original["some"], "another": original.another}
]
A slightly more elegant option is to define the fields to include or exclude like in #eephillip's example. For instance:
result = [
mapped |
original := data[_]
mapped := {k: v |
some k
v := original[k]
not exclude[k]}
]
exclude = {"dont", "dont2"} # defines a SET of keys to exclude
Of course you could generalize it even more by making the inner comprehension invoke a function that implements other filters.
Here's an interactive example: https://play.openpolicyagent.org/p/P6wPd3rudJ
Two notes about the original solution.
1. result1 does not iterate over data correctly
{d |
d := {
"some": data[_].some, # problem: _ is a different variable in each expression
"another": data[_].another
}
}
Conceptually each occurrence of _ is a unique variable. If you explicitly declare the variables, the problem is more obvious:
# note: this is still wrong
{d |
some i, j
d := {
"some": data[i]["some"],
"another": data[j].another
}
}
If you run this, you'll discover that it produces a cross-product (which is not what you want). You want the "some" and "another" fields to be selected from the same object like this:
{d |
some i
d := {
"some": data[i]["some"],
"another": data[i].another
}
}
Of course, coming up with unique variable names can be a pain, so you can use _. Just do not mistake multiple _ variables as referring to the same value. We can rewrite the statement to use _ as follows:
{d |
obj := data[_]
d := {
"some": obj["some"],
"another": obj.another
}
}
result2 is close but may assign multiple values (which should be avoided)
result2 = cleaned {
d := data[_]
cleaned := { # problem: there could be multiple values for 'cleaned'
"some": d["some"],
"another": d.another
}
}
Rules of the form NAME = VALUE { BODY } assign VALUE to NAME if the statements in BODY are satisfied. If you omit BODY, i.e., you write NAME = VALUE, then BODY defaults to true (which is always satisfied.)
In your above example:
NAME is result2
VALUE is cleaned
BODY is d := data[_]; cleaned := {...}
In Rego, we call these rules "complete rules". Complete rules are just IF-THEN statements that assign a single value to a variable. The "IF" portion is the rule body and the "THEN" portion is the assignment. You should avoid writing rules that may assign MULTIPLE values to the same variable because that may result in an evaluation time error. For example:
# do not do this
result = v {
v := data[_] # if 'data' is [1,2,3] then what is the value of 'result'? Hint: There is more than one answer.
}
If you want to assign MULTIPLE values to a variable then you can define a "partial rule" For example:
result[v] { # result is a SET.
v := data[_]
}
What about performing a rejection of the key name during the comprehensions?
Probably a more elegant way to do this, but might be helpful.
package play
reject(key) = result {
remove := [ "dont", "someotherthing" ]
result := key == remove[_]
}
result = d {
d := [ obj |
val := input.data[_];
obj := { k: v |
v := val[k]
not reject(k)
}
]
}
https://play.openpolicyagent.org/p/1A3DNLiNfj

XQuery: best way to convert node sequence to array

I am using Saxon, version 9.8.0.6 with this input document:
<simple>
<hello>Hello World!</hello>
<number>42</number>
<keyword>abc</keyword>
<keyword>def</keyword>
<keyword>ghi</keyword>
</simple>
And this query
xquery version "3.1";
fn:serialize(map{
'greeting': data(/simple/hello),
'number': number(/simple/number),
'keywords': array{ for $k in /simple/keyword return data($k) }
}, map{'method':'json', 'indent':true()})
Output is (as expected):
{
"number":42,
"keywords": [
"abc",
"def",
"ghi"
],
"greeting":"Hello World!"
}
Question:
'keywords': array{ for $k in /simple/keyword return data($k) } seems a little clumsy for me. Is this the way to do it? Any suggestions for improving?
You could reduce
array{ for $k in /simple/keyword return data($k) }
to
array{data(/simple/keyword)}

Resources