OPA REGO: How to find all not matching items in another dictionary? - open-policy-agent

Given input as follow:
{
"source": "serverA",
"destination": "serverB",
"rules": {
"tcp": {
"ssh": [
22
],
"https": [
8443
]
},
"udp": [
53
]
}
}
and data source:
{
"allowedProtocolTypeToPortMapping": {
"tcp": {
"ssh": [22],
"https": [443]
},
"udp": {
"dns": [53]
},
"icmp": {
"type": [0]
}
}
}
I want to create a policy that checks all rules and shows the ones that are not compliant to data source. In this example that would be port 8443 with protocol type https that is not compliant (allowed only 443). What is the best way to achieve it using rego language?

Here's one way of doing it.
package play
violations[msg] {
# Traverse input object stopping only at array values
walk(input.rules, [path, value])
is_array(value)
# Get corresponding list of allowed ports from matching path
allowedPorts := select(data.allowedProtocolTypeToPortMapping, path)
# At this point, we have one array of ports from the input (value)
# and one array of allowed ports. Converting both to sets would allow
# us to compare the two using simple set intersection - i.e. checking
# that all ports in the input are also in the allowed ports.
ip := {port | port := value[_]}
ap := {port | port := allowedPorts[_]}
# Unless all ports in the input array are in both the input and the
# allowed ports, it's a violation
count(ip) != count(ip & ap)
msg := concat(".", path)
}
select(o, path) = value {
walk(o, [path, value])
}
Rego Playground example available here.

Related

Loop through a nested json object in OPA

I am very new to OPA and trying to loop through the below input data
input =
"data":{
"list":[
{"Name": "abc", "Status": "Done"},
{"Name": "def", "Status": "Done"},
{"Name": "ghi", "Status": "pending"},
{"Name": "jkl", "Status": ""},
{"Name": "mno", "Status": null},
]
}
and return two lists based on that input, one list that would return the names of all objects that has the status as 'Done' and another list with status not equal to 'Done', here is the rego code I was trying, I used somewhat python syntax to convey the code as I was not sure how the opa syntax would look like
package play
default done_list := []
default other_list := []
done_list {
some i
input.data.list[i].Status == "Done"
done_list.append(input.data.list[i].Name) #python syntax and looking for something similar in opa
}
other_list{
some j
input.data.list[j].Status != "Done"
other_list.append(input.data.list[j].Name)
}
#or maybe use a list comprehension like this?
#not even sure if this makes sense but ...
done_list = [x.Name | x := input.data.list[_]; x.Status = "Done"]
other_list = [x.Name | x := input.data.list[_]; x.Status != "Done"]
test_return{
"done_list": done_list,
"other_list": other_list
}
and the output I'm looking for is something like
{
"done_list": ["abc", "def"],
"other_list": ["ghi","jkl","mno"]
}
Any help is greatly appreciated. Thanks in advance.
I think you're best to use a list comprehension as you guessed :)
package play
doneStatus := "Done"
result := {
"done_list": [e |
item := input.data.list[_]
item.Status == doneStatus
e := item.Name
],
"other_list": [e |
item := input.data.list[_]
item.Status != doneStatus
e := item.Name
],
}
I've created a Rego playground which shows this: https://play.openpolicyagent.org/p/dBKUXFO3v2

Open Policy Agent reduce array when element in array

I have been trying to achieve something with Open Policy Agent that I am sure should be possible. Just struggling with the on-ramp to the Rego language (I think). I am using the playground to get a feel for how to achieve the following.
I have the following data input.
{
"tags": [
{
"key": "test"
},
{
"key": "test2"
}
]
}
Rego code
package play
minimum_tags = {"test","test2"}
deny[msg] {
tags := input.tags[_][key]
# At this point I have an array and a set. I can convert the minimum_tags to Array
# But I can't really figure out how to do an iteration to check each tags value is in minimum_tags. Or reduce the minimum tags until its empty
}
I can only see the ability to reduce a set using the a1 - a2 built in. Doesn't seem to be a way to effect change on an Array
I think the idiomatic approach would be to convert tags to a set as well, so that you can use set operations (like you suggest) to check that all tags from minimum_tags are included in tags:
deny[msg] {
tags := {tag | tag := input.tags[_].key}
missing_tags := minimum_tags - tags
count(missing_tags) > 0
msg := sprintf("Missing tags: %v", [concat(", ", missing_tags)])
}
If you really want to have tags as an array, you could do something like this:
deny[msg] {
tags := [tag | tag := input.tags[_].key]
required_tag := minimum_tags[_]
not in_array(required_tag, tags)
msg := sprintf("Missing tag: %v", [required_tag])
}
in_array(item, arr) {
item == arr[_]
}

Open Policy Agent - Improve performance of a grouping comprehension

I have a id and role mapping in below format
{
"ra": [
{
"id": 168,
"code": "AFAP"
},
{
"id": 180,
"code": "ABC"
},
{
"id": 180,
"code": "ABCMND"
}
]
}
I need the output to be like below
{
"roleactions": {
"168": [
"AFAP"
],
"180": [
"ABC",
"ABCMND",
"DCRMP"
]
}
}
So i wrote the below code
roleactions = r_map {
r := data.ra
r_map := {id: list |
some i
id := r[i].id
list := [obj |
some j
r[j].id == id
obj := r[j].code
]
}
}
But when I run this for it almost takes 5-6 seconds
Found 1 result in 5682526.465 µs.
Can someone guide on how to make write this policy map to improve performance?
OPA can evaluate comprehensions like this in linear-time: https://www.openpolicyagent.org/docs/latest/policy-performance/#comprehension-indexing. The problem in this case is that the local variable r is not safe when considering the comprehensions in isolation.
If you refactor the comprehension like below, the runtime should be linear:
roleactions := r_map {
r_map := {id: list |
some i
id := data.ra[i].id
list := [obj |
some j
data.ra[j].id == id
obj := data.ra[j].code
]
}
}

parsing JSON file using telegraf input plugin : unexpected Output

I’m new to telegraf and influxdb, and currently looking forward to exploring telegraf, but unfortunetly, I have some difficulty getting started, I will try to explain my problem below:
Objectif: parsing JSON file using telegraf input plugin.
Input : https://wetransfer.com/downloads/0abf7c609d000a7c9300dc20ee0f565120200624164841/ab22bf ( JSON file used )
The input json file is a repetition of the same structure that starts from params and ends at it.
you find below the main part of the input file :
{
"events":[
{
"params":[
{
"name":"element_type",
"value":"Home_Menu"
},
{
"name":"element_id",
"value":""
},
{
"name":"uuid",
"value":"981CD435-E6BC-01E6-4FDC-B57B5CFA9824"
},
{
"name":"section_label",
"value":"HOME"
},
{
"name":"element_label",
"value":""
}
],
"libVersion":"4.2.5",
"context":{
"locale":"ro-RO",
"country":"RO",
"application_name":"spresso",
"application_version":"2.1.8",
"model":"iPhone11,8",
"os_version":"13.5",
"platform":"iOS",
"application_lang_market":"ro_RO",
"platform_width":"320",
"device_type":"mobile",
"platform_height":"480"
},
"date":"2020-05-31T09:38:55.087+03:00",
"ssid":"spresso",
"type":"MOBILEPAGELOAD",
"user":{
"anonymousid":"6BC6DC89-EEDA-4EB6-B6AD-A213A65941AF",
"userid":"2398839"
},
"reception_date":"2020-06-01T03:02:49.613Z",
"event_version":"v1"
}
Issue : Following the documentation, I tried to define a simple telegraf.conf file as below:
[[outputs.influxdb_v2]]
…
[[inputs.file]]
files = ["/home/mouhcine/json/file.json"]
json_name_key = "My_json"
#... Listing all the string fields in the json.(I put only these for simplicity reason).
json_string_fields = ["ssid","type","userid","name","value","country","model"]
data_format = "json"
json_query= "events"
Basically declaring string fields in the telegraf.conf file would do it, but I couldn’t get all the fields that are subset in the json file, like for example what’s inside ( params or context ).
So finally, I get to parse fields with the same level of hierarchy as ssid, type, libVersion, but not the ones inside ( params, context, user).
Output : Screen2 ( attachment ).
OUTPUT
By curiosity, I tried to test the documentation’s example, in order to verify whether I get the same expected result, and the answer is no :/, I don’t get to parse the string field in the subset of the file.
The doc’s example below:
Input :
{
"a": 5,
"b": {
"c": 6,
"my_field": "description"
},
"my_tag_1": "foo",
"name": "my_json"
}
telegraf.conf
[[outputs.influxdb_v2]]
…
[[inputs.file]]
files = ["/home/mouhcine/json/influx.json"]
json_name_key = "name"
tag_keys = ["my_tag_1"]
json_string_fields = ["my_field"]
data_format = "json"
Expected Output : my_json,my_tag_1=foo a=5,b_c=6,my_field="description"
The Result I get : "my_field" is missing.
Output: Screen 1 ( attachement ).
OUTPUT
By the way, I use the influxdb cloud 2, and I apologize for the long description of this little problem, I would appreciate some help please :), Thank you so much in advance.

Handling groups of key, value pairs while parsing lxc container file with python

I have lxc configuration file looking like ini/prop file but it contains duplicate key-value pairs that i need to group, I wish to convert in dict and json, here is the sample:
lxc.tty = 4
lxc.pts = 1024
lxc.rootfs = /opt/mail0/rootfs/
lxc.network.type = veth
lxc.network.name = eth7
lxc.network.link = br7
lxc.network.ipv4 = 192.168.144.215/24
lxc.network.type = veth
lxc.network.name = eth9
lxc.network.link = br9
lxc.network.ipv4 = 10.10.9.215/24
here is desired python data, that also converts keys from cfg to key paths
data = {
"lxc":{ "tty": 4, "pts": 1024 , "rootfs": "/opt/mail0/rootfs",
"network": [
{"type": "veth", "name": "eth7", "link": "br7", "ipv4":"192.168.144.215/24"},
{"type": "veth", "name": "eth9", "link": "br9", "ipv4":"10.10.9.215/24"}
]
}
}
Can you share/suggest your method & rules for handling such key, value pairs groups. Thank you in advance!

Resources