Rego - how to mimic set generation using functions - open-policy-agent

I have a rule that I expect to be reused by a variety of modules. I figured, let's turn that into a function, have the modules pass their input into a function and use a set comprehension like approach, but I'm running into the "functions must not produce multiple outputs for same inputs" error.
Here's a contrived example of what I want to accomplish. I'm thinking I'm going about this the wrong way and there's another approach to this type of problem in Rego.
Classic generator:
arr = [1,2,3,4]
result[entry] {
itm := arr[i]
r := itm % 2
r == 0
entry := { "type": "even", "val": itm }
}
result[entry] {
itm := arr[i]
r := itm % 2
r == 1
entry := { "type": "odd", "val": itm }
}
This works as expected.
"result": [
{
"type": "even",
"val": 2
},
{
"type": "even",
"val": 4
},
{
"type": "odd",
"val": 1
},
{
"type": "odd",
"val": 3
}
]
Here's the function approach that will trigger this error. I am passing that t_label variable to give the function some argument, but it's not really important.
f(t_label) := q {
q := [ entry | itm := arr[i]
r := itm % 2
r == 0
entry := { t_label: "even", "val": itm }
]
}
f(t_label) := q {
q := [ entry | itm := arr[i]
r := itm % 2
r == 1
entry := { t_label: "odd", "val": itm }
]
}
Is this a thing that is done? How is this problem generally approached using Rego?

You're right — unlike rules, functions can't be partial. If you really need something like that for a function you could either have a function to aggregate the result of two (or more) other function calls:
even(t_label) := {entry |
itm := arr[_]
itm % 2 == 0
entry := {t_label: "even", "val": itm}
}
odd(t_label) := {entry |
itm := arr[_]
itm % 2 == 1
entry := {t_label: "odd", "val": itm}
}
f(t_label) := even(t_label) | odd(t_label)
both := f("foo")
For your particlar example though, I think we'd be excused using a little "or hack" using map based branching:
f(t_label) := {entry |
itm := arr[_]
entry := {t_label: {0: "even", 1: "odd"}[itm % 2], "val": itm}
}

Related

Rego Object matching and comparison

I'm trying to match the domain key ("domain":"example.com") from the given input, and return an error if the values are not identical. This is what I've thought of so far, but I can't seem to match the domain key, thus the tests are failing. Any advice would be appreciated.
OPA Policy:
package main
import data.config
default warning_mode = []
warning_mode = config.warn_mode { config.warn_mode }
array_contains(arr, elem) {
arr[_] = elem
}
exception[rules] {
rules := config.exceptions.rules
}
deny_missing_config[msg] {
not config
msg := "Missing configuration file"
}
## Main
aws_ses_dkim[a] {
a := input.resource_changes[_]
a.type == "aws_ses_domain_dkim"
}
aws_ses_domain[e] {
e := input.resource_changes[_]
e.type == "aws_ses_domain_identity"
}
ses_missing_dkim[msg] {
a := aws_ses_dkim[_]
e := aws_ses_domain[_]
walk(a, [["values", "domain"], x])
walk(e, [["values", "domain"], y])
err := x - y
not err == set()
msg := sprintf("Placeholder error", [err, a.address, e.address])
}
## Test Cases
deny_ses_missing_dkim[msg]{
not array_contains(warning_mode, "ses_missing_dkim")
ses_missing_dkim[_] != []
msg := ses_missing_dkim[_]
}
warn_ses_missing_dkim[msg]{
array_contains(warning_mode, "ses_missing_dkim")
ses_missing_dkim[_] != []
msg := ses_missing_dkim[_]
}
test_ses_missing_dkim_invalid {
i := data.mock.invalid_ses_dkim
r1 := warn_ses_missing_dkim with input as i with data.config.warn_mode as []
count(r1) == 0
r2 := warn_ses_missing_dkim with input as i with data.config.warn_mode as ["ses_missing_dkim"]
count(r2) == 1
r3 := deny_ses_missing_dkim with input as i with data.config.warn_mode as []
count(r3) == 1
r4 := deny_ses_missing_dkim with input as i with data.config.warn_mode as ["ses_missing_dkim"]
count(r4) == 0
count(r1) + count(r2) == 1
count(r3) + count(r4) == 1
}
test_ses_missing_dkim_valid {
i := data.mock.ses_dkim
r1 := warn_ses_missing_dkim with input as i with data.config.warn_mode as []
r2 := warn_ses_missing_dkim with input as i with data.config.warn_mode as ["ses_missing_dkim"]
r3 := deny_ses_missing_dkim with input as i with data.config.warn_mode as []
r4 := deny_ses_missing_dkim with input as i with data.config.warn_mode as ["ses_missing_dkim"]
count(r1) + count(r2) + count(r3) + count(r4) == 0
}
Input (Terraform JSON):
"resource_changes":[
{
"address":"aws_ses_domain_dkim.example",
"mode":"managed",
"type":"aws_ses_domain_dkim",
"name":"example",
"provider_name":"registry.terraform.io/hashicorp/aws",
"schema_version":0,
"values":{
"domain":"example.com"
},
"sensitive_values":{
"dkim_tokens":[
]
}
},
{
"address":"aws_ses_domain_identity.example",
"mode":"managed",
"type":"aws_ses_domain_identity",
"name":"example",
"provider_name":"registry.terraform.io/hashicorp/aws",
"schema_version":0,
"values":{
"domain":"example.com"
},
"sensitive_values":{
}
}
]
The x and y values retrieved by the walk function will be strings, so err := x - s isn't going to work. If you want a set of values, you could wrap the walk call in a set comprehension to get a set of all values:
ses_missing_dkim[msg] {
a := aws_ses_dkim[_]
e := aws_ses_domain[_]
xs := {x | walk(a, [["values", "domain"], x])}
ys := {y | walk(e, [["values", "domain"], y])}
err := xs - ys
not err == set()
msg := sprintf("Placeholder error", [err, a.address, e.address])
}
You probably don't need walk here though, as the values are always at a known path.

Unsure why this Dafny verification fails

function method abs(m: int): nat
{ if m>0 then m else -m }
method CalcTerm(m: int, n: nat) returns (res: int)
ensures res == 5*m-3*n;
{
var m1: nat := abs(m);
var n1: nat := n;
res := 0;
while (m1!=0)
invariant m1>=0
invariant 0<=res
invariant res <=5*abs(m)
decreases m1
{
res := res+5;
m1 := m1-1;
}
if (m<0) { res := -res; }
while (n1!=0)
invariant n1>=0
decreases n1
{
res := res-3;
n1 := n1-1;
}
}
I have tried to increase the invariance in the loops. To the first loop, I added the condition, res<=5*abs(m) but Dafny complains that "This loop invariant might not be maintained by the loop." I don't understand how it is not.
What could I be doing wrong?
If you make your loop invariant stronger by stating exactly what res is equal to after each iteration, Dafny will be able to verify it.
So in the first while loop, instead of invariant res <= 5*abs(m), use invariant res == 5*abs(m) - 5*m1. When the loop terminates, m1 is equal to zero, so res will be 5*abs(m).
Similarly, for the second while loop, define the invariant res == 5*m - 3*n + 3*n1. Now when this loop terminates, n1 is equal to zero, so res will be 5*m - 3*n and Dafny will be able to prove that the post-condition of the method holds.
P.S. I usually use > 0 instead of != 0 as a loop condition.
After making these changes, you would have:
function method abs(m: int): nat
{
if m > 0 then m else -m
}
method CalcTerm(m: int, n: nat) returns (res: int)
ensures res == 5*m - 3*n;
{
var m1: nat := abs(m);
var n1: nat := n;
res := 0;
while (m1 > 0)
invariant m1 >= 0;
invariant 0 <= res;
invariant res == 5*abs(m) - 5*m1;
decreases m1;
{
res := res + 5;
m1 := m1 - 1;
}
if (m < 0)
{
res := -res;
}
while (n1 > 0)
invariant n1 >= 0;
invariant res == 5*m - 3*n + 3*n1;
decreases n1;
{
res := res - 3;
n1 := n1 - 1;
}
}
which verifies in Dafny.

Dafny linear search

When working on a basic linear search I encountered an error with my Valid() predicate. It seems to only work when I uncomment the additional ensures statements on the constructor and the data method. That is, when I am very explicit about the contents.
I'm also having trouble with the postcondition of my search when the item isn't found.
Any suggestions on how to resolve these?
class Search{
ghost var Contents: set<int>;
var a : array<int>;
predicate Valid()
reads this, a;
{
a != null &&
a.Length > 0 &&
Contents == set x | 0 <= x < a.Length :: a[x]
}
constructor ()
ensures a != null;
ensures a.Length == 4;
//ensures a[0] == 0;
ensures Valid();
{
a := new int[4](_ => 0);
Contents := {0};
new;
}
method data()
modifies this, a;
requires Valid();
requires a != null;
requires a.Length == 4;
ensures a != null;
ensures a.Length == 4;
// ensures a[0] == 0;
// ensures a[1] == 1;
// ensures a[2] == 2;
// ensures a[3] == 3;
ensures Valid();
{
a[0] := 0;
a[1] := 1;
a[2] := 2;
a[3] := 3;
Contents := {0, 1, 2, 3};
}
method search(e: int) returns (r: bool)
modifies this, a;
requires Valid();
ensures Valid();
ensures r == (e in Contents)
ensures r == exists i: int :: 0 <= i < a.Length && a[i] == e
{
var length := a.Length - 1;
while (length >= 0)
decreases length;
{
var removed := a[length];
if (e == removed)
{
return true;
}
length := length - 1;
}
return false;
}
}
method Main()
{
var s := new Search();
s.data();
}
There are several orthogonal issues going on here.
First, you have noticed that Dafny is reluctant to reason about the part of Valid that describes Contents. This is a common problem when reasoning about sets in Dafny. Essentially, the only way Dafny will ever "notice" that something is a member of the set set x | 0 <= x < a.Length :: a[x] is if it already has the expression a[x] lying around somewhere. Your solution of including extra postconditions works because it mentions a lot of expressions of the form a[x]. Another solution is to include those facts as assertions instead of postconditions:
// in data()
assert a[0] == 0;
assert a[1] == 1;
assert a[2] == 2;
assert a[3] == 3;
Second, Dafny cannot show your search procedure satisfies its postcondition. You need a loop invariant to keep track of the progress of the search. See the guide for more information about how to design loop invariants.
Third, Dafny reports a problem with your Main about modifies clauses. You can fix this by adding a postcondition fresh(a) to the constructor. The problem here is that the data method claims to modify a, but Dafny can't tell if a is visible from the caller.

Dafny generic type array error

In trying to verify a generic FIFO queue backed by an array I ran into a confusing error. The queue was found in this paper, authored by the creator of Dafny.
The error in question is:
unless an initializer is provided for the array elements, a new array of 'Data' must have empty size
which relates to both lines allocating an array via new Data[whatever] in the constructor and the enqueue method.
Dafny version: Dafny 2.0.0.00922 technical preview 0
Full code for reference.
class {:autocontracts} SimpleQueue<Data>
{
ghost var Contents: seq<Data>;
var a: array<Data>;
var m: int, n: int;
predicate Valid() {
a != null && a.Length != 0 && 0 <= m <= n <= a.Length && Contents == a[m..n]
}
constructor ()
ensures Contents == [];
{
a := new Data[10];
m := 0;
n := 0;
Contents := [];
}
method Enqueue(d: Data)
ensures Contents == old(Contents) + [d];
{
if n == a.Length {
var b := a;
if m == 0 {
b := new Data[2 * a.Length];
}
forall (i | 0 <= i < n - m) {
b[i] := a[m + i];
}
a, m, n := b, 0, n - m;
}
a[n], n, Contents := d, n + 1, Contents + [d];
}
method Dequeue() returns (d: Data)
requires Contents != [];
ensures d == old(Contents)[0] && Contents == old(Contents)[1..];
{
assert a[m] == a[m..n][0];
d, m, Contents := a[m], m + 1, Contents[1..];
}
}
method Main()
{
var q := new SimpleQueue();
q.Enqueue(5); q.Enqueue(12);
var x := q.Dequeue();
assert x == 5;
}
Since the time of writing that paper, Dafny's type system has been generalized to support types that are not "default initializable". This has led to some backwards incompatibilities.
The easiest fix is to change
class SimpleQueue<Data>
to
class SimpleQueue<Data(0)>
which means that the type variable Data can only be instantiated with default-initializable types.
Another fix is to change the constructor to accept a default value for type Data as an argument. Then you can allocate an array using an initializer function, as in
new Data[10] (_ => d)

Dafny syntax error in function

I am struggling with dafny syntax.
searchAndReplace receives three arrays of chars. Let's imagine line is [s][n][a][k][e]; pat is [n][a] and dst is [h][i]. I want to search for all the occurrences of pat in line and replace it with dst resulting in [s][h][i][k][e]
Method findwill return the indice of the first letter in line that is equal to pat.
Method deletewill remove pat from line at the variable at returned at find, and move all the others elements after at+p to the left in order to fill the null spaces.
Method insertwill make space in order to dst be added to lineat atby moving all the characters between atand at + p ppositions to the right.
I created an auxiliar function which will compare patand dst in order to verify that they aren't equal(if they were it would be replacing infinitely time dstin line in case patexisted in line)
For now i'm receiving the error "then expected" on the following section of code inside function checkIfEqual:
if(pat.Length != dst.Length) {
return false;
}
The full code:
method searchAndReplace(line:array<char>, l:int,
pat:array<char>, p:int,
dst:array<char>, n:int)returns(nl:int)
requires line != null && pat!=null && dst!=null;
requires !checkIfEqual(pat, dst);
requires 0<=l<line.Length;
requires 0<=p<pat.Length;
requires 0<=n<dst.Length;
modifies line;
{
var at:int := 0;
var p:int := n;
while(at != -1 )
invariant -1<=at<=l;
{
at := find(line, l, dst, n);
delete(line, l, at, p);
insert(line, l, pat, p, at);
}
var length:int := line.Length;
return length;
}
function checkIfEqual(pat:array<char>, dst:array<char>):bool
requires pat!=null && dst!=null;
reads pat;
reads dst;
{
var i:int := 0;
if(pat.Length != dst.Length) {
return false;
}
while(i<dst.Length) {
if(pat[i] != dst[i]){
return false;
}
i := i + 1;
}
return true;
}
method insert(line:array<char>, l:int, nl:array<char>, p:int, at:int)
requires line != null && nl != null;
requires 0 <= l+p <= line.Length && 0 <= p <= nl.Length ;
requires 0 <= at <= l;
modifies line;
ensures forall i :: (0<=i<p) ==> line[at+i] == nl[i]; // error
{
var i:int := 0;
var positionAt:int := at;
while(i<l && positionAt < l)
invariant 0<=i<l+1;
invariant at<=positionAt<=l;
{
line[positionAt+p] := line[positionAt];
line[positionAt] := ' ';
positionAt := positionAt + 1;
i := i + 1;
}
positionAt := at;
i := 0;
while(i<p && positionAt < l)
invariant 0<=i<=p;
invariant at<=positionAt<=l;
{
line[positionAt] := nl[i];
positionAt := positionAt + 1;
i := i + 1;
}
}
method find(line:array<char>, l:int, pat:array<char>, p:int) returns (pos:int)
requires line!=null && pat!=null
requires 0 <= l < line.Length
requires 0 <= p < pat.Length
ensures 0 <= pos < l || pos == -1
{
var iline:int := 0;
var ipat:int := 0;
pos := -1;
while(iline<l && ipat<pat.Length)
invariant 0<=iline<=l
invariant 0<=ipat<=pat.Length
invariant -1 <= pos < iline
{
if(line[iline]==pat[ipat] && (line[iline]!=' ' && pat[ipat]!=' ')){
if(pos==-1){
pos := iline;
}
ipat:= ipat + 1;
} else {
if(ipat>0){
if(line[iline] == pat[ipat-1]){
pos := pos + 1;
}
}
ipat:=0;
pos := -1;
}
if(ipat==p) {
return;
}
iline := iline + 1;
}
return;
}
method delete(line:array<char>, l:nat, at:nat, p:nat)
requires line!=null
requires l <= line.Length
requires at+p <= l
modifies line
ensures line[..at] == old(line[..at])
ensures line[at..l-p] == old(line[at+p..l])
{
var i:nat := 0;
while(i < l-(at+p))
invariant i <= l-(at+p)
invariant at+p+i >= at+i
invariant line[..at] == old(line[..at])
invariant line[at..at+i] == old(line[at+p..at+p+i])
invariant line[at+i..l] == old(line[at+i..l]) // future is untouched
{
line[at+i] := line[at+p+i];
i := i+1;
}
}
functions in Dafny are pure, inductively defined and use a different syntax to the imperative methods. You cannot use imperative language features inside a function. In this case you are not allowed to use:
The conditional statement if cond { s1* } else { s2* }
The loop statement while cond { s1* }
Instead the body of the function must be an expression:
predicate checkIfEqual(pat:array<char>, dst:array<char>)
requires pat!=null && dst!=null;
reads pat;
reads dst;
{
pat.Length == dst.Length
&& forall i:nat :: i < pat.Length ==> pat[i] == dst[i]
}
Although it is not needed here, Dafny does have a conditional expression (ite):
predicate checkIfEqual(pat:array<char>, dst:array<char>)
requires pat!=null && dst!=null;
reads pat;
reads dst;
{
if pat.Length != dst.Length then false
else forall i:nat :: i < pat.Length ==> pat[i] == dst[i]
}
Note that:
You cannot put a loop in the body of a function, but you may use recursion
predicate is a shorthand for a function that returns bool

Resources