I don't even know where to begin. JUnit plugin is installed and is running in some scripts (I couldn't find those, it's a large project, but the statistic page indicates 28 usages). But, the pipeline code junit "foo.xml" fails with
java.lang.NoSuchMethodError: No such DSL method 'junit' found among steps
Followed by 100,500 lines of usual nonsense.
Is there anything special I need to do to "enable" this plugin? Jenkins wiki lists it as "required" (whatever that means). The example Jenkinfile that illustrates the usage of this plugin never imports anything. Nonexistent debugging and ninja-style documentation don't really help getting to the culprit of this problem.
When I tried to replace junit 'foo.xml' with step([$class: 'JUnitResultArchiver', testResults: './foo.xml']). This "step" produced the following output:
Recording test results
And failed the build without any further messages. Neither in console nor in the logs collected by the pipeline.
Related
From time to time our Jenkins pipeline is marked as unstable, after researching it I found that it originates from the Junit plugin, which is publishing test results.
The weird thing is that all the tests are successfully passing (logs and also the whole pipeline proceeding), however the exported test results show that there are some errors.
Can anyone explain this to me?
All tests are passing (logs):
The test results exported by junit are showing some failures:
The whole build is marked as unstable (yellow):
I'm trying to wrap my head around how this declarative Jenkinsfile is Groovy. I want to write supporting code to execute this outside the Jenkins environment, in pure Groovy, if that's possible. I've been writing example groovy code but still am unsure what "pipeline", "agent", and "stages" are.
Any tips to understand this structure is appreciated
EDIT: I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins
pipeline {
stages {
// extra code here
}
}
No, you can't run Jenkinsfile as a standalone Groovy script. In short, Jenkins executes the pipeline code inside a pre-configured GroovyShell that knows how to evaluate things like pipeline, agent, stages, and so forth. However, there is a way to execute Jenkinsfie without the Jenkins server - you can use JenkinsPipelineUnit test library to write JUnit/Spock unit tests that will evaluate your Jenkinsfile and display the call stack tree. It uses mocks, so you can treat it as interaction-based testing, to see if a specific part of your pipeline gets executed. Plus, you can catch some code errors prior to running the pipeline on the server.
A simple unit test for the declarative pipeline can look like this:
import com.lesfurets.jenkins.unit.declarative.*
class TestExampleDeclarativeJob extends DeclarativePipelineTest {
#Test
void should_execute_without_errors() throws Exception {
def script = runScript("Jenkinsfile")
assertJobStatusSuccess()
printCallStack()
}
}
You can find more examples in the official README.md - https://github.com/jenkinsci/JenkinsPipelineUnit
Alternatively, you can try Jenkinsfile Runner command-line tool that can execute your Jenkinsfile outside of the Jenkins server - https://github.com/jenkinsci/jenkinsfile-runner
UPDATE
I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins.
Your pipeline code example looks like a valid Jenkinsfile, but you can't turned it into a Groovy code that can be run e.g. from the command-line as a regular Groovy script:
$ groovy Jenkinsfile
This won't work, because Groovy is not aware of the Jenkins Pipeline syntax. The syntax is added as a DSL via the Jenkins plugin, and it uses a dedicated GroovyShell that is pre-configured to interpret the pipeline syntax correctly.
If you are interested in checking if the syntax of the Jenkins Pipeline is correct, there are a few different options:
npm-groovy-lint (https://github.com/nvuillam/npm-groovy-lint) can validate (and even auto-fix) the syntax of your Jenkinsfile without connecting to the Jenkins server,
Command-Line Pipeline Linter (https://www.jenkins.io/doc/book/pipeline/development/#linter) can send your pipeline code to the Jenkins server and validate its syntax.
These are a few tools that can help you with catching up the syntax errors before you run the pipeline. But that's just a nice addon to your toolbox. The first step, as always, is to understand what the syntax means, and the official documentation (https://www.jenkins.io/doc/book/pipeline/syntax) is the best place to start.
Within my freestyle Jenkins-job I´m executing unit-tests via the "execute Windows batch-command"-step:
call "C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" MyAssembly.dll
call SomeOtherProcess
As there are tests that fail, I´d expected the build to fail as well. However the test-publishing-step for NUnit markes the build as unstable:
Build step 'Publish NUnit test result report' changed build result to UNSTABLE
If I´d remove the SomeOtherProcess-line from my batch-script, everything works fine and the errors produced by nunit are reported as error in the build-process.
I read a similar issue for the JUnit-test-reporter (Jenkins JUnit Plugin reports a build as unstable even if test fails). Obviously that reporter does not even support failing the build. I´m not sure if the same applies to the NUnit-reporter as well.
The plugin set the result to UNSTABLE because the option, by default, failedTestsFailBuild is set to false.
You can control the behavior applies of NUnit, setting failedTestsFailBuild to true. When you call from a scripted or declarative pipeline.
The issue is the GUI doesn't reflect all the options available for this plugin. There is a PR opened to include this option inside the freestyle pipeline, you can vote up or ask the status of this PR.
To change to an error you need to catch the unstable result and set it to failure using a plugin or a scripted or declarative pipeline.
I'm using the Jenkins JUnit Plugin to collect my tests result and according to the documentation:
Allow empty results: If checked, the default behavior of failing a
build on missing test result files or empty test results is changed to
not affect the status of the build. Please note that this setting make
it harder to spot misconfigured jobs or build failures where the test
tool does not exit with an error code when not producing test report
files.
This is how I use it:
junit testResults: '**/reports/junit*.xml', testDataPublishers: [[$class: 'StabilityTestDataPublisher']]
So the behavior should fail if the test file is empty but I get "UNSTABLE" result with the message
Test report file /opt/workspace/integrations_develop/reports/junit_integrations.xml was length 0
How can I make it fail if the test file is empty?
I need to overload jenkins functions to debug pipeline script in IDE.
I'm new to java/groovy etc. I'm going to write a several hundred lines scripted pipeline. Groovy is based on java. As I'm new I prefer the function name completion/suggestion and a debugging feature would be awesome where I can walk through the lines step by step and see what is in the vars.
I set up a eclipse Luna with the groovy-plugin. Which is actually working =)! (for newest eclipse the plugin is not yet ready). Also debugging is quiet cool!
But special jenkins expressions will still throw errors.
node(MasterName){ ... }
sh
...
Is there a chance to overload those functions?
Just killing the error, not performing any actions. Maybe converting it to a print like "I'm executing script XYZ" or "Switching to node BLUBB"?
The outcome should be a copy paste script, for checking in and running with jenkins without major changes.
Is there any better way?
In the end it turned out, that even if you omit the pipeline specific expressions, you will still have trouble. In some more complex situations e.g. calling a constructor with super in a extended class and executing a function afterwards, it does not behave the same then in my local python interpreter.
So what I did was an error to assume, my Ubuntu system default groovy interpreter would work the same like the jenkins interpreter. It would be nice to run a debugger inside the jenkins environment, or going step by step through the pipeline script and see how it is actually working without a print in every second line.