Mapping items in an array - open-policy-agent

Question

Hoping this is a nice easy one, but I just can't see how to do it.
I am wanting to with rego map items in an array to a cleaner version. For example from the data below
data = [
{
"some": "value",
"another": "mvalue",
"dont": "want"
},
{
"some": "value1",
"another": "mvalue1",
"dont": "want1"
},
{
"some": "value2",
"another": "mvalue2",
"dont": "want2"
}
]
I want to turn data into
result = [
{
"some": "value",
"another": "mvalue"
},
{
"some": "value1",
"another": "mvalue1"
},
{
"some": "value2",
"another": "mvalue2"
}
]
The two closest I think I've got is
result1 = cleaned {
cleaned := {d |
d := {
"some": data[_].some,
"another": data[_].another
}
}
}
result2 = cleaned {
d := data[_]
cleaned := {
"some": p.some,
"another": p.another
}
}

Answer 1

TLDR; If the fields are static and you can easily enumerate them, both of your solutions is almost correct (see below for explanation of why they are incorrect.) Here's the right way to do that:
result = [
mapped |
original := data[_]
mapped := {"some": original["some"], "another": original.another}
]
A slightly more elegant option is to define the fields to include or exclude like in #eephillip's example. For instance:
result = [
mapped |
original := data[_]
mapped := {k: v |
some k
v := original[k]
not exclude[k]}
]
exclude = {"dont", "dont2"} # defines a SET of keys to exclude
Of course you could generalize it even more by making the inner comprehension invoke a function that implements other filters.
Here's an interactive example: https://play.openpolicyagent.org/p/P6wPd3rudJ
Two notes about the original solution.
1. result1 does not iterate over data correctly
{d |
d := {
"some": data[_].some, # problem: _ is a different variable in each expression
"another": data[_].another
}
}
Conceptually each occurrence of _ is a unique variable. If you explicitly declare the variables, the problem is more obvious:
# note: this is still wrong
{d |
some i, j
d := {
"some": data[i]["some"],
"another": data[j].another
}
}
If you run this, you'll discover that it produces a cross-product (which is not what you want). You want the "some" and "another" fields to be selected from the same object like this:
{d |
some i
d := {
"some": data[i]["some"],
"another": data[i].another
}
}
Of course, coming up with unique variable names can be a pain, so you can use _. Just do not mistake multiple _ variables as referring to the same value. We can rewrite the statement to use _ as follows:
{d |
obj := data[_]
d := {
"some": obj["some"],
"another": obj.another
}
}
result2 is close but may assign multiple values (which should be avoided)
result2 = cleaned {
d := data[_]
cleaned := { # problem: there could be multiple values for 'cleaned'
"some": d["some"],
"another": d.another
}
}
Rules of the form NAME = VALUE { BODY } assign VALUE to NAME if the statements in BODY are satisfied. If you omit BODY, i.e., you write NAME = VALUE, then BODY defaults to true (which is always satisfied.)
In your above example:
NAME is result2
VALUE is cleaned
BODY is d := data[_]; cleaned := {...}
In Rego, we call these rules "complete rules". Complete rules are just IF-THEN statements that assign a single value to a variable. The "IF" portion is the rule body and the "THEN" portion is the assignment. You should avoid writing rules that may assign MULTIPLE values to the same variable because that may result in an evaluation time error. For example:
# do not do this
result = v {
v := data[_] # if 'data' is [1,2,3] then what is the value of 'result'? Hint: There is more than one answer.
}
If you want to assign MULTIPLE values to a variable then you can define a "partial rule" For example:
result[v] { # result is a SET.
v := data[_]
}

Answer 2

What about performing a rejection of the key name during the comprehensions?
Probably a more elegant way to do this, but might be helpful.
package play
reject(key) = result {
remove := [ "dont", "someotherthing" ]
result := key == remove[_]
}
result = d {
d := [ obj |
val := input.data[_];
obj := { k: v |
v := val[k]
not reject(k)
}
]
}
https://play.openpolicyagent.org/p/1A3DNLiNfj