I have a monorepo and i am trying to make them run in parallel
def abc = findJenkinsfileToRun(modifiedFiles)
parallel {
for (file in abc) {
println("Building ${file.toString()}")
load "${file.toString()}/Jenkinsfile"
}
}
This results in the following
java.lang.IllegalArgumentException: Expected named arguments but got org.jenkinsci.plugins.workflow.cps.CpsClosure2#b7ccdc
can anyone help how to resolve this?
You are not using the parallel keyword correctly, it should receive a map of branch names as keys and execution code (closure) as values. See the Documentation.
So in your case you should use something like:
def abc = findJenkinsfileToRun(modifiedFiles)
parallel abc.collectEntries { file ->
["Building ${file.toString()}" : {
// The code to run in parallel
println("Building ${file.toString()}")
load "${file.toString()}/Jenkinsfile"
}]
}
Related
I have jenkins shared library which contains internal method (let's call it A())
Currently the A() method run sequentially between 10-100 times.
I want to run A() in parallel with pre define bulk number (10)
I didn't find a way to implement it via groovy, some of the method Jenkins require approval.
What is the way to do it via groovy ( not parallel steps of jenkis) without require any jenkins approval?
I don't know if you need permissions for this, but just try it:
def A = {
println "hello from A($it)"
Thread.sleep(Math.random() * 2000 as long)
println "hello from A($it)"
}
def threads = (0..5).collect { counter ->
Thread.start { A(counter) }
}
threads.each { it.join() }
I put some println and some sleep to demonstrate the methods run in parallel. You can replace the body of A{} as you wish.
I'm currently trying to automate the versioning of my maven project with a Jenkins (version 2.190.3) job and following the SemVer. So I have my Jenkinsfile and I'm doing something like that:
script {
def version = "1.2.4"
def pattern = ~/(\d{1,3})\.(\d{1,3})\.\d{1,4}$/
def newVersion = version.replaceFirst(pattern) { _,major,minor -> "${major}.${(minor as int) + 1}.0"}
}
The expectation is to have 1.3.0 in newVersion.
The code seems OK, working on web console but when I'm using Jenkins I have the following error:
java.lang.NullPointerException: Cannot execute null+1
Am I doing something wrong ?
Almost the exact answer to your question is given in this post: Jenkins groovy regex match string : Error: java.io.NotSerializableException: java.util.regex.Matcher
What it comes down to is that the script executed by Jenkins is kind of groovy, but not exactly executed as it is. It is transformed first to a serializable state (can be suspended, saved to file, transported, restored, resumed).
This doesn't work with certain objects that have state, but aren't serializable, including java.util.regex.Matcher, which is working under the hood of your regular expression. If you put this code in a method marked #NonCPS, the code is not transformed, and (more of less) executed as plain groovy.
#NonCPS
def foo() {
def version = "1.2.4"
def pattern = ~/(\d{1,3})\.(\d{1,3})\.\d{1,4}$/
def newVersion = version.replaceFirst(pattern) { _,major,minor -> "${major}.${(minor as int) + 1}.0"}
println "Version ${version} -> new ${newVersion}"
}
script {
foo()
}
what is wrong with below code, comparing 2 strings in groovy
I am trying do the comparison between the 2 parameters in a single line to make it look tidier
if (params.dirname == ((params.path =~ ~/${params.dirname}/).with { matches() ? it[0] : null })) {
print success
}
Throwing Exception -
java.lang.NoSuchMethodError: No such DSL method 'matches' found among steps
There is no need to over-complicate your use case. According to:
params.dirname = hde, params.path = /usr/tmp/jenkins/hde/filename.txt or /usr/hde/jenkins/ing/filename.txt or any random path which has hde in it
you are trying to find if given string a contains substring b. It can be done using Java's method String.contains(String substring). Alternatively you can use regular expression for that, but String.contains() just looks a few times simpler to understand what is your intention. Consider following Groovy script:
def params = [
dirname: 'hde',
path: '/usr/tmp/jenkins/hde/filename.txt'
]
// Using String.contains()
if (params.path.contains(params.dirname)) {
println "Path '${params.path}' contains '${params.dirname}'"
}
// Using regular expression
if (params.path ==~ /(.*)${params.dirname}(.*)/) {
println "Path '${params.path}' contains '${params.dirname}'"
}
When you run it both if statements evaluates to true:
Path '/usr/tmp/jenkins/hde/filename.txt' contains 'hde'
Path '/usr/tmp/jenkins/hde/filename.txt' contains 'hde'
I have a requirement to run a set of tasks for a build in parallel, The tasks for the build are dynamic it may change. I need some help in the implementation of that below are the details of it.
I tasks details for a build will be generated dynamically in an xml which will have information of which tasks has to be executed in parallel/serial
example:
say there is a build A.
Which had below task and the order of execution , first task 1 has to be executed next task2 and task3 will be executed in parallel and next is task 4
task1
task2,task3
task4
These details will be in an xml dynamically generated , how can i parse that xml and schedule task accordingly using pipeline plugin. I need some idea to start of with.
You can use Groovy to read the file from the workspace (readFile) and then generate the map containing the different closures, similar to the following:
parallel(
task2: {
node {
unstash('my-workspace')
sh('...')
}
},
task3: {
node {
unstash('my-workspace')
sh('...')
}
}
}
In order to generate such data structure, you simply iterate over the task data read using XML parsing in Groovy over the file contents you read previously.
By occasion, I gave a talk about pipelines yesterday and included very similar example (presentation, slide 34ff.). In contrast, I read the list of "tasks" from another command output. The complete code can be found here (I avoid pasting all of this here and instead refer to this off-site resource).
The kind of magic bit is the following:
def parallelConverge(ArrayList<String> instanceNames) {
def parallelNodes = [:]
for (int i = 0; i < instanceNames.size(); i++) {
def instanceName = instanceNames.get(i)
parallelNodes[instanceName] = this.getNodeForInstance(instanceName)
}
parallel parallelNodes
}
def Closure getNodeForInstance(String instanceName) {
return {
// this node (one per instance) is later executed in parallel
node {
// restore workspace
unstash('my-workspace')
sh('kitchen test --destroy always ' + instanceName)
}
}
}
I have a very long workflow for building and testing our application. So long, in fact, that when we try to load the main workflow script, we get this exception:
java.lang.ClassFormatError: Invalid method Code length 67768 in class file WorkflowScript
I am not proud of this. I'm tying to split the workflow into smaller scripts that we load from the main workflow script, but are running into an issue with variable scoping. For example:
def a = 'foo' //some variable referenced in multiple workflow stages
node {
echo a
}
//... and then a whole bunch of other stages
might become
def a = 'foo' //some variable referenced in multiple workflow stages
node {
git: ...
load 'flowPartA.groovy'
}()
where flowPartA.groovy looks like:
{ ->
node {
echo a
}
}
Based on my understanding of the documentation, where flowPartA.groovy is interpreted as a closure, I expect the variable 'a' would remain in scope, but instead, I get an exception to the contrary.
groovy.lang.MissingPropertyException: No such property: a for class: groovy.lang.Binding
Am I missing something about the way workflow interprets the flow scripts? Is there a good way to take a huge workflow that uses many, many parameters and split it into smaller chunks?
You have to define a function in the external groovy and call it passing all required parameters:
def a = 'foo'
node('slave') {
git '…'
def flow = load 'flowPartA.groovy'
flow.echoFromA(a)
}
And flowPartA.groovy contains:
def echoFromA(String a) {
echo a
}
return this
See the documentation for more information.