I am new to rego and trying to write a policy to validate elements of a list against another list. Here is the problem I am trying to solve:
I have an approved list of security groups against which I need to check the security groups from my input. If the SG's in my input are not from the approved list, I need to output a message saying "Invalid SG's are being used".
I couldn't find any documentation which explains how I could compare one list with another.
Here is the policy in rego playground: https://play.openpolicyagent.org/p/m09f2K1g0h
Policy:
check_list = ["sg-1001a1d199tx8866a","sg-2002b2e200in9966b", "sg-3003c3f201dc0011c","sg-4004d4e202nj1122d", "sg-5003c3f201dc0011e","sg-6004d4e202nj1122f"]
aws_valid_sgs := {i: Reason |
doc = input[i]; i="aws_launch_configuration.ecs-cluster-lc"
key_ids := [k | doc[k]; startswith(k, "security_groups.")]
resource := {
doc[k] : doc[replace(k, ".key", ".value")] | key_ids[_] == k
}
sg_ids := [m | resource[m]; startswith(m, "sg-")]
list := {x| x:=check_list[_]}
not(list[sg_ids[i]])
Reason := sprintf("Resource %v is not using an approved SG ", [doc.name])
}
Input:
{
"aws_launch_configuration.ecs-cluster-lc": {
"associate_public_ip_address": "false",
"destroy": false,
"destroy_tainted": false,
"ebs_block_device.#": "",
"ebs_optimized": "",
"enable_monitoring": "true",
"id": "",
"instance_type": "t3.large",
"key_name": "",
"name": "test_r1-us-east-1",
"security_groups.1122334455": "sg-1001a1d199tx8866a",
"security_groups.2233445566": "sg-2002b2e200in9966b",
"security_groups.3344556677": "sg-3003c3f201dc0011c",
"security_groups.4455667788": "sg-4004d4e202nj1122d"
},
"destroy": false
}
Any help is appreciated. Thanks.
TLDR; This is best done using set operations, e.g., {"foo", "bar", "baz"} - {"bar"} == {"foo", "baz"}). Here's a complete example:
# Define a SET of valid SGs to compare against.
valid_sgs := {"sg-1001a1d199tx8866a","sg-2002b2e200in9966b", "sg-3003c3f201dc0011c","sg-4004d4e202nj1122d", "sg-5003c3f201dc0011e","sg-6004d4e202nj1122f"}
# Compute the SET of SGs from the launch configuration in the input.
launch_config_sgs := {sg |
some key
launch_config := input["aws_launch_configuration.ecs-cluster-lc"]
sg := launch_config[key]
startswith(key, "security_groups.")
}
# Compute the SET of invalid SGs by taking the difference of the other two sets.
invalid_launch_config_sgs := launch_config_sgs - valid_sgs
# Generate a 'deny' message if there are any invalid SGs.
deny[msg] {
launch_config := input["aws_launch_configuration.ecs-cluster-lc"]
count(invalid_launch_config_sgs) > 0
msg := sprintf("Resource %v is not using an approved SG", [launch_config.name])
}
Playground link
There were a few things the posters version that made the problem harder than necessary:
check_list is defined as an array (not a set). Arrays are indexed by position. To check if a value is contained in the list, you need to iterate. It's often easier to just construct sets and test for membership/intersection/difference/etc.
The aws_valid_sgs rule constructs a bunch of unnecessary intermediate values from the input (e.g., doc, key_ids, resource, etc.) All that's needed is the set of SGs from the input which can be computed in 2 statements: find fields in the launch config where the key starts with security_group. and construct a set of field values for those keys.
Related
Jmeter ForEach Controller failing to write variables in original order correctly
I am executing a http request retrieving a json payload with an array of employees. For each record (employee) I need to parse the record for specific fields e.g. firstName, lastName, PersonId and write to a single csv file, incrementing a new row per record.
Unfortunately, the file created has two issues. The PersonId never gets written and secondly the sequence of the values is not consistent with the returned original values. Sometimes I get the same record for lastName with the wrong firstName and vice versa. Not sure if the two issues are related, I suspect my regular expression extract is wrong for a number.
Jmeter setup. (5.2.1)
jmeter setUp
Thread group
+ HTTP Request
++ JSON JMESPath Extractor
+ ForEach Controller
++ Regular Expression Extractor: PersonId
++ Regular Expression Extractor: firstName
++ Regular Expression Extractor: lastName
++ BeanShell PostProcessor
getWorker returns the following payload
jsonPayload
JSON JMESPath Extractor to handle the payload.
{
"items" : [
{
"PersonId" : 398378,
"firstName" : "Sam",
"lastName" : "Shed"
},
{
"PersonId" : 398379,
"firstName" : "Bob",
"lastName" : "House"
}
],
"count" : 2,
"hasMore" : true,
"limit" : 2,
"offset" : 0,
"links" : [
{
"rel" : "self",
"href" : "https://a.site.on.the.internet.com/employees",
"name" : "employees",
"kind" : "collection"
}
]
}
JSON JMESPath Extractor Configuration
Name of created variables: items
JMESPath expressions: items
Match No. -1
Default Values: Not Found
ForEach Controller
ForEach Controller Configuration
Input variable prefix: items
Start Index: Empty
End Index: Empty
Output variable name: items
Add "_"? Checked
Each of the Regular Expression Extracts follow the same pattern as below.
Extract PersonId with Regular Expression
Apply to: Main Sample Only
Field to check: Body
Name of created variable: PersonId
Regular Expression: "PersonId":"(.+?)"
Template: $1$
Match No. Empty
Default Value: PersonId
The final step in the thread is where I write out the parsed results.
BeanShell PostProcessor
PersonNumber = vars.get("PersonNumber");
DisplayName = vars.get("DisplayName");
f = new FileOutputStream("/Applications/apache-jmeter-5.2.1/bin/scripts/getWorker/responses/myText.csv", true);
p = new PrintStream(f);
this.interpreter.setOut(p);
print(PersonId+", "+ PersonNumber+ ", " + DisplayName);
f.close();
I am new to this and looking either for someone to tell me where I screwed up or direct me to a place I can read up on the appropriate topics. (Both are fine). Thank you.
For Each Controller doesn't know the structure of items variable since it is in JSON format. It is capable of just understanding an array and traverses through them. I would suggest to move away from For Each Controller in your case and use the JSON extractor itself for all the values like below
Person ID
First Name
Last Name
Beanshell Sampler Code
import java.io.FileWriter; // Import the FileWriter class
int matchNr = Integer.parseInt(vars.get("personId_C_matchNr"));
log.info("Match number is "+matchNr);
f = new FileOutputStream("myText.csv", true);
p = new PrintStream(f);
for (int i=1; i<=matchNr; i++){
PersonId = vars.get("personId_C_"+i);
FirstName = vars.get("firstName_C_"+i);
LastName = vars.get("lastName_C_"+i);
log.info("Iteration is "+i);
log.info("Person ID is "+PersonId);
log.info("First Name is "+FirstName);
log.info("Last Name is "+LastName);
p.println(PersonId+", "+FirstName+", "+LastName);
}
p.close();
f.close();
Output File
HOW THE ABOVE ACTUALLY WORKS
When you extract values using the matchNr, it goes in a sequential order in which the response has arrived. For example, in your case, Sam & Shed appear as first occurrences and Bob & House appear as subsequent occurrences. Hence JMeter captures them with the corresponding match and stores them as 1st First Name = Sam, 2nd First Name = Bob and so on.
GENERIC STUFF
The regex expression for capturing Person ID which you have used seems to be inaccurate. The appropriate one would be
"PersonId" :(.+?),
and not
"PersonId":"(.+?)"
Move to JSR223 processors instead of Beanshell as they are more performant. Source: Which one is efficient : Java Request, JSR223 or BeanShell Sampler for my script. The migration is pretty simple. Just copy the code that you have in Beanshell and paste it in JSR223.
Close any stream or writer that is open appropriately else it might cause issues when other users are trying to write to the file during load test
In case you are planning to use this file as a subsequent input within JMeter, please note that there is a space between comma and the next element. For example, it is "Sam, Shed" and not "Sam,Shed".JMeter by default does not trim any spaces and will use the value just like that. Hence you might want to take a judicious call regarding that space
Hope this helps!
Since JMeter 3.1 you shouldn't be using Beanshell, go for JSR223 Test Elements and Groovy language for scripting.
Given Groovy has built-in JSON support you shouldn't need any extractors, you can write the data into a file in a single shot like:
new groovy.json.JsonSlurper().parse(prev.getResponseData()).items.each { item ->
new File('myText.csv') << item.get('PersonId') << ',' << item.get('firstName') << ',' << item.get('lastName') << System.getProperty('line.separator')
}
More information: Apache Groovy - Why and How You Should Use It
Trying to insert simple xml file with one row in IIB with simple message flow into Oracle XE DB. Message flow works fine and inserts data into database, but data written in db is different from starting data. For example, as I'm trying to insert my name "Dino" I'd get Korean/Japanese/Chinese signs in return.
I've tried changing XML formats thinking there might be problem, but I suppose it has to do with encoding.
Input:
Output in DB:
This is how my compute node looks like:
CREATE COMPUTE MODULE SimpleDB_mf_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
CALL CopyMessageHeaders();
-- CALL CopyEntireMessage();
INSERT INTO Database.dkralj.emp VALUES(InputRoot.XMLNSC.emp.name);
SET OutputRoot.XMLNSC.DBINSERT.STATUS='SUCCESS';
RETURN TRUE;
END;
CREATE PROCEDURE CopyMessageHeaders() BEGIN
DECLARE I INTEGER 1;
DECLARE J INTEGER;
SET J = CARDINALITY(InputRoot.*[]);
WHILE I < J DO
SET OutputRoot.*[I] = InputRoot.*[I];
SET I = I + 1;
END WHILE;
END;
CREATE PROCEDURE CopyEntireMessage() BEGIN
SET OutputRoot = InputRoot;
END;
END MODULE;
Looking at the IBM documentation for the INSERT statement in ESQL it might be worth trying.
INSERT INTO Database.dkralj(NAME) VALUES(InputRoot.XMLNSC.emp.name);
If weird things are still happening then I'd try a string constant to avoid any issues with character coding in the input message.
INSERT INTO Database.dkralj(NAME) VALUES('TheEmpValue');
Before this statement in your code
SET OutputRoot.XMLNSC.DBINSERT.STATUS='SUCCESS';
You should check for success or otherwise by using the inbuilt SQLSTATE, SQLCODE, SQLERRORTEXT to check the result of your call.
IF NOT ((SQLCODE = 0) OR (SQLSTATE = '01000' AND SQLNATIVEERROR = 8153)) THEN
-- Do something about the error.
-- The check of SQLSTATE and SQLNATIVEERROR covers warnings
-- The 8153 is for Microsoft SQL Server other databases may use a different value
END IF;
Also check the codepages aka CodedCharSetId of the source system data, the message in IIB and the default codepage of the database.
Use mqsicvp MYBROKER -n ODBC_DB_NAME to get other details about the connection you need to use -n to get the details.
Use something like DBeaver to add some data. Have a look at the datatype specified for the field.
As per your comment below and my response here is an example of a PASSTHRU statement. Note the use of the ? to avoid SQL Injection.
PASSTHRU('SELECT RTRIM(A.EMPLID) AS EMPLID,
RTRIM(A.ADDRESS_TYPE) AS ADDRESS_TYPE,
RTRIM(A.ADDR_TYPE_DESCR) AS ADDR_TYPE_DESCR,
CAST(RTRIM(A.EFFDT) AS DATE) AS EFFDT,
RTRIM(A.EFF_STATUS) AS EFF_STATUS,
RTRIM(A.ADDRESS1) AS ADDRESS1,
RTRIM(A.ADDRESS2) AS ADDRESS2,
RTRIM(A.ADDRESS3) AS ADDRESS3,
RTRIM(A.ADDRESS4) AS ADDRESS4,
RTRIM(A.CITY) AS CITY,
RTRIM(A.STATE) AS STATE,
RTRIM(A.POSTAL) AS POSTAL
FROM ADDRESS_VW AS A
WHERE UPPER(A.EMPLID) = ?') VALUES(AggrRef.EmployeeID)
Delphi XE6 - I have a set. I would like a simple way to turn ALL elements off. i.e. instead of Exclude, something like ExcludeALL. I have tried to loop through all elements, but I get an error.
Code
type
TSearchParametersType =
(smDUNSAvailable = 1,
smDUNSHit,
smDUNSMiss,
smDUNSAbsent,
smRegistryAvailable,
smRegistryHit,
smRegistryAbsent,
smRegistryMiss,
smNameAvailable,
smNameHitExact,
smNameHitWords,
smNameMiss
);
// Now create a set type, where we can have a variable that has all the values of TSearchParametersType
type
TSearchParametersSet = set of TSearchParametersType;
...
var
i : Integer;
sSearchStatus: TSearchParametersSet;
begin
for i := smDUNSAvailable to smNameMiss do
Exclude(sSearchStatus, i);
The error I get is "Incompatible Type: 'Integer' and TSearchParametersType. "
Is there a simple way to Exclude ALL, other than MANUALLY going through every element?
Thanks
From the documentation:
Every set type can hold the empty set, denoted by [].
So you can assign the empty set to your variable like this:
sSearchStatus := [];
FWIW, your code fails because smDUNSAvailable and smNameMiss are of type TSearchParametersType and so not compatible with the variable i which is of type Integer. In order to make your code work you would need to change the loop variable to be of type TSearchParametersType.
Let me start by saying that David's answer is the correct one.
I'll just post another one to show how you could do it manually. This code might come in handy some other time:
var
sSearchStatus: TSearchParametersSet;
SearchParametersType : TSearchParametersType;
begin
sSearchStatus := [smDUNSHit, smDUNSMiss, smDUNSAbsent, smRegistryAvailable, smRegistryHit];
for SearchParametersType := low(TSearchParametersType) to high(TSearchParametersType) do
Exclude(sSearchStatus, SearchParametersType);
end;
In local security policy (PC-Control panel-Administration-local security policy) there is a parameter "Minimum length of the password" and a parameter "Password must meet complexity requirements" (true/false). How can I read them in Delphi (for WinXpSp3-Win2003-Vista-Win7-Win2008(+r2))?
I'm looking for something like:
Function DetectSystemMinPassLength:integer;
begin
//?
end;
Function DetectSystemPassComplexity:boolean;
begin
//?
end;
Additional question:
Does there exist in Delphi (or WinApi) function which can check if a given password conforms to system policies (or set)?
For example:
Function MyCheckPassComplexity (Password:string):boolean;
begin
// ???
end;
use
MyCheckPassComplexity (' 12345 '); //result False
MyCheckPassComplexity (' MyCoolPassword9999 '); //result True
Usually to read a local or group policy setting you must use the Group Policy Settings Reference for Windows and Windows Server which basically is a set of excel files which contains the windows registry keys where is stored such info. unfortunately in this case if you check such reference for these Account policies (Enforce password history,
Maximum password age, Minimum password age, Minimum password length) you will find this message:
Password Policy security settings are not registry keys.
Exist a set of WMI classes in the root\RSOP\Computer namespace like RSOP_SecuritySettingBoolean, RSOP_SecuritySettingNumeric , RSOP_SecuritySettings to access the an account policy but these classes only works (I mean retrieve information) on systems which is are in a domain, but it does not work in a workgroup.
For the moment I think which you best option is export the local policies to a ini file using this command (and then parse the result using a TIniFile class)
secedit.exe /export /cfg C:\Output\Policy.Ini
This command will create a file like this
[Unicode]
Unicode=yes
[System Access]
MinimumPasswordAge = 0
MaximumPasswordAge = 42
MinimumPasswordLength = 0
PasswordComplexity = 0
PasswordHistorySize = 0
About your second question to validate a password you can use the NetValidatePasswordPolicy WinAPI function.
I have an accuracy problem with a reporting tool that I'm using. The issue is that it is reading in data from a TADODataset on the fly but converting the ftBCD fields into doubles.
This causes rounding errors when performing operations at the reporting level (sums, etc).
I'd like to specify my currency fields as ftCurrency, not ftBCD to stop this from hapening.
If I create a field as:
TFieldDef.Create( ADODataset.FieldDefs,
'test',
ftCurrency,
0,
True,
0
);
ADODataset.CreateDataSet;
Using the watch, I can now see that ADODataset.Fields[0].DataType has a value of ftBCD.
Is there a way to explicitly specify a ftCurrency field within a TADODataset so it's not assigned as a ftBCD field?
Try this:
ADODataSet.FieldByName('AField').Currency := True;