In QAF (Gerkin), I want to pass map data or Array List from one step to another.
I have tried following:
Step 1 (inside Step Def):
List<Object> tdData = new ArrayList<>(data.values());
store(tdData, "data_values");
Step 2 (inside Step Def):
System.out.println(getBundle().getObject("data_values")); // Printing the list
ArrayList<Object> tmpValues = getBundle().getObject("data_values"); // Throwing error
System.out.println(tmpValues.get(0));
Able to print the list as string with "System.out.println(getBundle().getObject("data_values"));" but not able to assign the list to ArrayList. Throwing below error.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project qaf-blank-project-maven: Compilation failure
[ERROR] /Users/XXXXXXX/Downloads/60b76ff32243f233355d048b/src/test/java/com/sample/steps/steps.java:[125,57] incompatible types: java.lang.Object cannot be converted to java.util.ArrayList<java.lang.Object>
You need to cast object to list.
List<Object> tmpValues = (List<Object>)getBundle().getObject("data_values");
System.out.println(tmpValues.get(0));
Related
I've added a Lambda Step as the first step in my Sagemaker Pipeline. It processes some data and creates 2 files as part of the output like so:
from sagemaker.workflow.lambda_step import LambdaStep, Lambda, LambdaOutput, LambdaOutputTypeEnum
# lamb_preprocess = LambdaStep(func_arn="")
output_param_1 = LambdaOutput(output_name="status", output_type=LambdaOutputTypeEnum.Integer)
output_param_2 = LambdaOutput(output_name="file_name_a_c_drop", output_type=LambdaOutputTypeEnum.String)
output_param_3 = LambdaOutput(output_name="file_name_q_c_drop", output_type=LambdaOutputTypeEnum.String)
step_lambda = LambdaStep(
name="ProcessingLambda",
lambda_func=Lambda(
function_arn="arn:aws:lambda:us-east-1:xxxxxxxx:function:xxxxx"
),
inputs={
"input_data": input_data,
"input_file": trigger_file,
"input_bucket": trigger_bucket
},
outputs = [
output_param_1, output_param_2, output_param_3
]
)
In my next step, I want to trigger a Processing Job for which I need to pass in the above Lambda function's outputs as it's inputs. I'm trying to do it like so:
inputs = [
ProcessingInput(source=step_lambda.properties.Outputs["file_name_q_c_drop"], destination="/opt/ml/processing/input"),
ProcessingInput(source=step_lambda.properties.Outputs["file_name_a_c_drop"], destination="/opt/ml/processing/input"),
]
However, when the processing step is trying to get created, I get a validation message saying
Object of type Properties is not JSON serializable
I followed the data dependency docs here: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_building_pipeline.html#lambdastep and tried accessing step_lambda.OutputParameters["file_name_a_c_drop"] too but it errored out saying 'LambdaStep' object has no attribute 'OutputParameters'
How do I properly access the return value of a LambdaStep in a Sagemaker pipeline ?
You can access the output as follows - step_lambda.OutputParameters["output1"]. You don't need to add .properties
To access a LambdaStep output in another step you can do this:
step_lambda.properties.Outputs["file_name_a_c_drop"]
Try this
steplambda.properties.ProcessingOutputConfig.Outputs["file_name_q_c_drop"].S3Output.S3Uri
I have a file called apps.txt which has three app names in it
frontendapp
authorizationservice
connectorservice*
In jenkins pipeline i want to perform some operation on them one by one so i am trying to get them in groovy list using this code -
list = readFile.readFileLineByLine("${workspace}/apps.txt").collect {it}
for (item in list) {
println "I need to perform some operations on files"
}
But getting groovy.lang.MissingPropertyException.
If i use file class like this - list = new File("${workspace}/apps.txt").collect {it} then it search for a file on Jenkins master node only and i get fileNotFoundException.
If i use list = readFile("${workspace}/apps.txt").collect {it} then list gets values character by character. How i can get app names from apps.txt inorder to perform operation on each app.
Your attempts are close, but mix things up.
Those are working ways:
def list = new File("${workspace}/apps.txt").text.readLines()
Note the .text call inbetween.
def list = readFile("${workspace}/apps.txt").readLines()
Or with the helper Jenkins provides.
Side note: .collect{it} is just .collect() - and usually is only
needed to copy a list. Since the read lines from the file are already
eagerly read, those are copies.
This work for me in a windows worker
script {
def list = readFile("filesChanged").readLines()
for(item in list){
print item
bat('type '+item)
}
}
I am retrieving an XML file from a remote host and parsing it using XmlParser. The content of the file is as follows:
<?xml version="1.0" encoding="utf-8"?><Metrics> <Safety> <score>81.00</score> <Percentrules>98.00</Percentrules> </Safety> </Metrics>
I am able to retrieve the score value in the following way when I execute the script outside the Groovy sandbox.
def report = readFile(file: 'Qualitycheck.xml')
def metrics = new XmlParser().parseText(report)
println metrics
double score = Double.parseDouble(metrics.Safety.score[0].value()[0])
However, when I execute the script using SCM I get the following:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: No such field found: field groovy.util.Node
The issue persist even though I have installed the Permissive-Script-Security-Plugin and enabled the plugin using the -Dpermissive-script-security.enabled=no_securityJVM option. Is there something different about this method? No other method is causing issues. Why?
Edit
I decided to use XmlSlurper(), and retrieved the value 81.00. However the result was type groovy.util.slurpersupport.NodeChildren
def metrics2 = new XmlSlurper().parseText(report)
def score = metrics2.Safety.score
print score
print score.getClass()
=> 81.0098.00
=> groovy.util.slurpersupport.NodeChildren
How do I use XmlSlurper to extract the value 81.00 and cast it as double? Will that be a good alternative?
There seems to be some issues with the script sandbox with Node and NodeList field access. You can work around this like the following, its not nice but works at least.
node() {
def xml = readFile "${env.WORKSPACE}/Qualitycheck.xml"
def rootNode = new XmlParser().parseText(xml)
print Double.parseDouble(rootNode.value()[0].value()[0].value()[0])
// Next line if position isnt fixed, can return an array
// if theres more than 1 with structure "Safety.score", [0] at the end takes the first.
print Double.parseDouble(rootNode.find{it.name() == "Safety"}.value().find{it.name() == "score"}.value()[0])
}
You also need to approve following signatures in the In-process Script Approval section in Manage Jenkins menu.
method groovy.util.Node name
method groovy.util.Node value
method groovy.util.XmlParser parseText java.lang.String
new groovy.util.XmlParser
staticMethod java.lang.Double parseDouble java.lang.String
staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods find java.lang.Object groovy.lang.Closure
I am trying to create a dynamic choice parameter that will be populated by the resulting values of a groovy script.
The following code works and lists the contents of a dir:
new File("/tmp/testing/source/").eachFile() { file->
println file.getName()
}
I created a new jenkins project, and entered the menu 'Configure' where I selected This project is parameterized
When I save and try to build with parameters nothing has been parsed from the groovy script
Solved with the following code:
list = []
def process = "ls /tmp/testing/source".execute()
process.text.eachLine {list.add it}
return list
I am trying to make changes to the database using the changelog. Since I cannot guarantee that the values currently exist for the specific code, but could exist, I need to be able to check for them in order to either do an insert or an update.
Here is what I have been testing, which doesn't appear to do anything. Any words of advice are welcome.
databaseChangeLog = {
changeSet(author:'kmert', id:'tubecap-insert-update-1') {
preConditions(onFail="WARN",onFailMessage:"Tube cap does not exist,skipping because it cannot be updated."){
sqlCheck(expectedResult='1', 'SELECT * FROM [ltc2_tube_cap] WHERE code=11')
}
grailsChange {
change {
sql.execute("""
UPDATE [ltc2_tube_cap]
SET [name] = 'White'
WHERE [code] = 11;
""")
}
rollback {
}
}
}
}
UPDATE: I got the changelog script running, but I am now getting this error. I found the code from an online source. I cannot find a lot of documentation on preconditions...
| Starting dbm-update for database hapi_app_user # jdbc:jtds:sqlserver://localhost;databaseName=LabTraffic;MVCC=TRUE;LOCK_TIMEOUT=10000
problem parsing TubeCapUpdate.groovy: No signature of method: grails.plugin.databasemigration.DslBuilder.sqlCheck() is applicable for argument types: (java.lang.String, java.lang.String) values: [1, SELECT * FROM ltc2_tube_cap WHERE code=11] (re-run with --verbose to see the stacktrace)
problem parsing changelog.groovy: No signature of method: grails.plugin.databasemigration.DslBuilder.sqlCheck() is applicable for argument types: (java.lang.String, java.lang.String) values: [1, SELECT * FROM ltc2_tube_cap WHERE code=11] (re-run with --verbose to see the stacktrace)
groovy.lang.MissingMethodException: No signature of method: grails.plugin.databasemigration.DslBuilder.sqlCheck() is applicable for argument types: (java.lang.String, java.lang.String) values: [1, SELECT * FROM ltc2_tube_cap WHERE code=11]
at grails.plugin.databasemigration.DslBuilder.invokeMethod(DslBuilder.groovy:117)
at Script1$_run_closure1_closure2_closure3.doCall(Script1.groovy:13)
at grails.plugin.databasemigration.DslBuilder.invokeMethod(DslBuilder.groovy:117)
at Script1$_run_closure1_closure2.doCall(Script1.groovy:12)
at grails.plugin.databasemigration.DslBuilder.invokeMethod(DslBuilder.groovy:117)
at Script1$_run_closure1.doCall(Script1.groovy:11)
at grails.plugin.databasemigration.GrailsChangeLogParser.parse(GrailsChangeLogParser.groovy:84)
at grails.plugin.databasemigration.DslBuilder.handleIncludedChangeLog(DslBuilder.groovy:747)
at grails.plugin.databasemigration.DslBuilder.createNode(DslBuilder.groovy:139)
at grails.plugin.databasemigration.DslBuilder.createNode(DslBuilder.groovy:590)
at grails.plugin.databasemigration.DslBuilder.invokeMethod(DslBuilder.groovy:117)
at Script1$_run_closure1.doCall(Script1.groovy:6)
at grails.plugin.databasemigration.GrailsChangeLogParser.parse(GrailsChangeLogParser.groovy:84)
at liquibase.Liquibase.update(Liquibase.java:107)
at DbmUpdate$_run_closure1_closure2.doCall(DbmUpdate:26)
at _DatabaseMigrationCommon_groovy$_run_closure2_closure11.doCall(_DatabaseMigrationCommon_groovy:59)
at grails.plugin.databasemigration.MigrationUtils.executeInSession(MigrationUtils.groovy:133)
at _DatabaseMigrationCommon_groovy$_run_closure2.doCall(_DatabaseMigrationCommon_groovy:51)
at DbmUpdate$_run_closure1.doCall(DbmUpdate:25)
Your syntax is incorrect for the sqlCheck preCondition.
sqlCheck(expectedResult:'1', 'SELECT * FROM [ltc2_tube_cap] WHERE code=11')
Notice in your code the first argument is an assignment statement expectedResult=1 and it should be a map entry expectedResult:1.
I found the answer buried on this Jira page. https://jira.grails.org/browse/GPDATABASEMIGRATION-40 which ironically is about adding lots of examples to the DB migration DSL to the documentation.
Make sure the following
grails dbm-gorm-diff add-your-file-forupdate.groovy -add
then inside your-file-forupdate.groovy is expected to see
databaseChangeLog = {
changeSet(author:'kmert', id:'tubecap-insert-update-1') {
.
.
.
}
}
Then ,the Big deal is either you have included this as migration script file to be executed as follow:
just manually add a line like the following to the end of grails-app/migrations/changelog.groovy:
include file: 'your-file-forupdate.groovy'
The changelog.groovy is always run from beginning to end, so make sure that you always add newly created migrations to the end.
Cheers! for more info see this