Can jenkins pipeline scripts be tested using groovysh or groovy scriptname to run tests for validation without using the Jenkins UI
For example for a simple script
pipeline {
stages {
stage ('test') {
steps {
sh '''
env
'''
}
}
}
}
running a test like this, depending on the subset of scripting gives:
No signature of method: * is applicable for argument types
groovysh_evaluate.pipeline()
or for
stage('test'){
sh '''
env
'''
}
reports:
No signature of method: groovysh_evaluate.stages()
or simply
sh '''
env
'''
reports:
No signature of method: groovysh_evaluate.sh()
The question may be which imports are required and how to install them outside of a jenkins installation?
Why would anyone want to do this?
Simplify and shorten iterating over test cases, validation of library versions without modifying jenkins installations and other unit and functional test scenarios.
JenkinsPipelineUnit is what you're looking for.
This testing framework lets you write unit tests on the configuration and conditional logic of the pipeline code, by providing a mock execution of the pipeline. You can mock built-in Jenkins commands, job configurations, see the stacktrace of the whole execution and even track regressions.
Related
I'm trying to wrap my head around how this declarative Jenkinsfile is Groovy. I want to write supporting code to execute this outside the Jenkins environment, in pure Groovy, if that's possible. I've been writing example groovy code but still am unsure what "pipeline", "agent", and "stages" are.
Any tips to understand this structure is appreciated
EDIT: I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins
pipeline {
stages {
// extra code here
}
}
No, you can't run Jenkinsfile as a standalone Groovy script. In short, Jenkins executes the pipeline code inside a pre-configured GroovyShell that knows how to evaluate things like pipeline, agent, stages, and so forth. However, there is a way to execute Jenkinsfie without the Jenkins server - you can use JenkinsPipelineUnit test library to write JUnit/Spock unit tests that will evaluate your Jenkinsfile and display the call stack tree. It uses mocks, so you can treat it as interaction-based testing, to see if a specific part of your pipeline gets executed. Plus, you can catch some code errors prior to running the pipeline on the server.
A simple unit test for the declarative pipeline can look like this:
import com.lesfurets.jenkins.unit.declarative.*
class TestExampleDeclarativeJob extends DeclarativePipelineTest {
#Test
void should_execute_without_errors() throws Exception {
def script = runScript("Jenkinsfile")
assertJobStatusSuccess()
printCallStack()
}
}
You can find more examples in the official README.md - https://github.com/jenkinsci/JenkinsPipelineUnit
Alternatively, you can try Jenkinsfile Runner command-line tool that can execute your Jenkinsfile outside of the Jenkins server - https://github.com/jenkinsci/jenkinsfile-runner
UPDATE
I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins.
Your pipeline code example looks like a valid Jenkinsfile, but you can't turned it into a Groovy code that can be run e.g. from the command-line as a regular Groovy script:
$ groovy Jenkinsfile
This won't work, because Groovy is not aware of the Jenkins Pipeline syntax. The syntax is added as a DSL via the Jenkins plugin, and it uses a dedicated GroovyShell that is pre-configured to interpret the pipeline syntax correctly.
If you are interested in checking if the syntax of the Jenkins Pipeline is correct, there are a few different options:
npm-groovy-lint (https://github.com/nvuillam/npm-groovy-lint) can validate (and even auto-fix) the syntax of your Jenkinsfile without connecting to the Jenkins server,
Command-Line Pipeline Linter (https://www.jenkins.io/doc/book/pipeline/development/#linter) can send your pipeline code to the Jenkins server and validate its syntax.
These are a few tools that can help you with catching up the syntax errors before you run the pipeline. But that's just a nice addon to your toolbox. The first step, as always, is to understand what the syntax means, and the official documentation (https://www.jenkins.io/doc/book/pipeline/syntax) is the best place to start.
My whole scripts are in one branch of repo and I have multiple jenkins pipeline job.
1. smoke
2. Regression
3. Epic wise Execution
each have a different pipeline script. So is it possible to have multiple jenkins file with custom name ?
pipeline {
node('Slave-Machine-1') {
env.NODE_HOME="${tool '8.9.4'}"
env.PATH="${env.NODE_HOME}/bin:${env.PATH}"
def AUTO = ''
stage("Install Dependency") {
sshagent(['agent-id']) {
sh 'npm install'
sh 'npm run webdriver-install'
}
}
stage("smoke") {
sh 'npm run smoke-test'
}
}
}
This is my sample pipeline script. similarly i have multiple pipeline scripts
You can name your pipeline scripts random_joe or anything you like as long as:
You do not use multibranch or organization pipeline projects, which specifically look for the filename Jenkinsfile to automatically create new jobs
You do not mind your text editor not syntax highlighting the pipeline scripts until you add the extension .groovy to them
It is advisable to follow conventions wherever not impracticable though.
Background
After a lot of hard work we finally got a Jenkins CI pulling code from out GitHub repositories and are now doing Continuous Integration as well as Deployment.
We get the code and only deploy it if all the tests pass, as usual.
Now I have checked that there are a number of plugins for Java that besides running the tests, also do test coverage, like Cobertura.
But we don't use Java. We use Elixir.
In the Elixir world, we have excoveralls, which is a facade for the coveralls API. The coveralls API supports jenkins so it stands to reason I would find a Coveralls Plugin for Jenkins.
I was wrong. There is nothing.
Questions
So now I have a test coverage metric that is basically useless because I can't integrate it with Jenkins.
Are there any Erlang/Elixir plugins one can use with Jenkins for code coverage?
I also created a Issue in the projects ( which seems to be abandoned ... ) https://github.com/parroty/excoveralls/issues/167
I have a stage to publish the coverage on my Jenkinsfile. I'm not sure if that is the metric that you want but...
stage('Publish Coverage') {
when{
branch 'master'
}
steps {
publishHTML target: [
allowMissing: true,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'cover',
reportFiles: 'excoveralls.html',
reportName: 'Coverage Report'
]
}
}
I have found 2 ways of doing this:
Using Hex package JUnit formatter together with junit post pipeline step
Using covertool together with Cobertura Jenkins pluing
Option 1
This solution works and is quite nice. It forces me to change the test_helper.exs but that is a minor inconvenience overall. It is nice but it only offers the most basic of reports and for me this is where it fails.
Option 2
The option I decided to go with. Yes, making the Jenkinsfile work for Cobertura was a nightmare, specially because in previous versions it was not even possible and because there is contradictory information scattered all over the place.
However, once you get that Jenkinsfile going, you get to rip those sweet reports from Cobertura. Cobertura was made with Java in mind, there is no two ways about it. In the reports you see things like Class coverage and such, but you can easily translate that do modules. The interface offers a lot more information and tracks coverage over time, which is something I actually want.
For future notice, here is my Jenkinsfile:
pipeline {
agent any
environment {
SOME_VAR = "/home/deployer"
}
stages {
stage("Build") {
steps {
sh "MIX_ENV=test mix do deps.get, deps.compile"
}
}
stage("Test") {
steps {
sh "mix test --cover"
}
}
stage("Credo"){
steps{
sh "mix credo --strict"
}
}
stage("Deploy"){
when{
expression{
env.BRANCH_NAME == "master"
}
}
steps{
sh '''
echo "Deploy with AWS or GCP or whatever"
'''
}
}
}
post{
always{
cobertura coberturaReportFile: "coverage.xml"
}
}
}
Of notice:
1. I am extremely Nazi with my code, so I also use Credo. You can further configure it as to not blow the entire pipeline because you missed a new line at the end of file but as I said, I am quite Nazi with my code.
2. The Deploy stage only runs if the pushed branch is Master. There are other ways of doing this, but I found it that having this way for a small project was good enough.
Overall I like covertools for now but I don't know if the first solution has the same potential. At least I didn't see it.
Hope this post helps!
Original thread:
https://elixirforum.com/t/excoveralls-plugin-for-jenkins-ci/18842
Another way to post coverage from Jenkins for Elixir project is using ExCoveralls option mix coveralls.post. This allows you to post the coverage from any host, including your Jenkins server. Based on the example on this Jenkins tutorial page, you can write in Jenkinsfile like this:
pipeline {
agent any
stages {
// Assuming all environment variables are set beforehand
stage('run unit test') {
steps {
sh 'echo "Run Unit Test and Post coverage"'
sh '''
MIX_ENV=test mix coveralls.post --token $COVERALLS_REPO_TOKEN --sha $GIT_COMMIT --branch $GIT_BRANCH --name "jenkins" --message $GIT_COMMIT_MSG
'''
}
}
}
}
I need to get GitVersion.exe variables in my Jenkins pipeline.
The GitVersion documentation gives a hint on how to do that. Essentially call gitversion /output buildserver.
This call does add the variables to the current step and they are lost once the step completes. I can show this call executes when combining a set command in the same bat execution. The second set shows the variables are gone from the environment.
bat 'nuget install GitVersion.CommandLine -OutputDirectory c:/packages -Version 3.6.5'
bat 'c:/packages/GitVersion.CommandLine.3.6.5/tools/GitVersion.exe /output buildserver && set'
bat 'set'
The documentation of GitVersion is aware of that and suggests to use EnvInject.
Installing the plugin and executing the same pipeline did not change the result. I read that the Plugin is not made for pipelines so that may have something to do with it.
Pipelines support a syntax for environment.
Following that syntax I can set static variables at the top of my pipeline like this:
environment {
ASuperVariable = 'MySuperVariable'
}
What I need is combining those calls so that I can add run time variables to the Jenkinsfile pupeline.
environment {
bat 'gitversion /output buildserver'
}
Now obviously the above call is not even syntax correct. Is there a way to mark a section so that the contained environment changes are available for other steps?
EDIT:
This is still unsolved. At the moment I need to create a batch script and pass the tool into it as an argument. Inside the batch I can call the tool to add to the environment of the batch script and use that wile the batch is running. A Multi line batch in the Jenkins file could be a solution if the process remains the same over all the multiple lines.
Not sure whether you would be able to use scripted pipeline or at least a script block inside declarative. It'd be quite easy doing so:
withEnv(['ASuperVariable=MySuperVariable']) {
echo env.ASuperVariable
}
Or when calling a windows cmd script:
node('win') {
withEnv(['ASuperVariable=MySuperVariable']) {
bat 'echo %ASuperVariable%'
}
}
Is it possible to somehow interact with the Jenkins Pipeline from the script (bat or sh)? E.g. to initiate new stage?
echo This is a batch file in no stage
pipeline.stage(build)
echo This is a batch file in build stage
I have a pretty compact build job written in PowerShell. My team is now experimenting with the Jenkins Pipeline feature and it would be great to split our ps build code to stages (build core, build modules, test, coverage and so on). We could easily do it by creating function for every stage but it would be inefficient (loading ps modules...)
I would propose you another way:
you may define your different steps as CmdLet in your Powershell script:
function step_1()
{
[CmdletBinding(SupportsShouldProcess=$true)]
param ()
Write-Verbose "step 1"
}
function step_2()
{
[CmdletBinding(SupportsShouldProcess=$true)]
param ()
Write-Verbose "step 2"
}
Then, you can define the Powershell method, like I describe hier: To call a PowerScript from the Groovy-Script
In your Groovy pipeline-script:
node ('MyWindowsSlave') {
stage ('Stage 1') {
PowerShell(". '.\\all-stages.ps1'; stage1 -Verbose")
}
stage ('Stage 2') {
PowerShell(". '.\\all-stages.ps1'; stage2 -Verbose")
}
}
You may also consider to split your Groovy Script in several files, and let the powershell developer using they own Groovy sub-script.
Regards.
You can use the input step (https://jenkins.io/doc/pipeline/steps/pipeline-input-step/) to wait a user response to continue the pipeline:
A simple 'yes' or 'no', a multiple choosen, a password confirmation...
You can control who can response to this input event using ldap groups for example.