I'm trying to create my first Groovy script for Jenkins:
After looking here https://jenkins.io/doc/book/pipeline/, I created this:
node {
stages {
stage('HelloWorld') {
echo 'Hello World'
}
stage('git clone') {
git clone "ssh://git#mywebsite.example/myrepo.git"
}
}
}
However, I'm getting:
java.lang.NoSuchMethodError: No such DSL method "stages" found among steps
What am I missing?
Also, how can I pass my credentials to the Git Repository without writing the password in plain text?
You are confusing and mixing Scripted Pipeline with Declarative Pipeline, for complete difference see here. But the short story:
declarative pipelines is a new extension of the pipeline DSL (it is basically a pipeline script with only one step, a pipeline step with arguments (called directives), these directives should follow a specific syntax. The point of this new format is that it is more strict and therefor should be easier for those new to pipelines, allow for graphical editing and much more.
scripted pipelines is the fallback for advanced requirements.
So, if we look at your script, you first open a node step, which is from scripted pipelines. Then you use stages which is one of the directives of the pipeline step defined in declarative pipeline. So you can for example write:
pipeline {
...
stages {
stage('HelloWorld') {
steps {
echo 'Hello World'
}
}
stage('git clone') {
steps {
git clone "ssh://git#mywebsite.example/myrepo.git"
}
}
}
}
So if you want to use declarative pipeline that is the way to go.
If you want to scripted pipeline, then you write:
node {
stage('HelloWorld') {
echo 'Hello World'
}
stage('git clone') {
git clone "ssh://git#mywebsite.example/myrepo.git"
}
}
E.g.: skip the stages block.
A Jenkinsfile can be written using two types of syntax - Declarative and Scripted.
Declarative and Scripted Pipelines are constructed fundamentally differently. Declarative Pipeline is a more recent feature of Jenkins Pipeline which:
provides richer syntactical features over Scripted Pipeline syntax, and
is designed to make writing and reading Pipeline code easier.
Many of the individual syntactical components (or "steps") written into a Jenkinsfile, however, are common to both Declarative and Scripted Pipeline.
Example:
Declarative Pipeline fundamentals
In Declarative Pipeline syntax, the pipeline block defines all the work done throughout your entire Pipeline.
Jenkinsfile (Declarative Pipeline):
pipeline {
agent any 1
stages {
stage('Build') { 2
steps {
// 3
}
}
stage('Test') { 4
steps {
// 5
}
}
stage('Deploy') { 6
steps {
// 7
}
}
}
}
Execute this Pipeline or any of its stages, on any available agent.
Defines the "Build" stage.
Perform some steps related to the "Build" stage.
Defines the "Test" stage.
Perform some steps related to the "Test" stage.
Defines the "Deploy" stage.
Perform some steps related to the "Deploy" stage.
Scripted Pipeline fundamentals
In Scripted Pipeline syntax, one or more node blocks do the core work throughout the entire Pipeline. Although this is not a mandatory requirement of Scripted Pipeline syntax, confining your Pipeline's work inside of a node block does two things:
Schedules the steps contained within the block to run by adding an item to the Jenkins queue. As soon as an executor is free on a node, the steps will run.
Creates a workspace (a directory specific to that particular Pipeline) where work can be done on files checked out from source control.
Caution: Depending on your Jenkins configuration, some workspaces may not get automatically cleaned up after a period of inactivity. See tickets and discussion linked from JENKINS-2111 for more information.
Jenkinsfile (Scripted Pipeline):
node { 1
stage('Build') { 2
// 3
}
stage('Test') { 4
// 5
}
stage('Deploy') { 6
// 7
}
}
Execute this Pipeline or any of its stages, on any available agent.
Defines the "Build" stage. stage blocks are optional in Scripted Pipeline syntax. However, implementing stage blocks in a Scripted Pipeline provides clearer visualization of each `stage's subset of tasks/steps in the Jenkins UI.
Perform some steps related to the "Build" stage.
Defines the "Test" stage. 5
Perform some steps related to the "Test" stage.
Defines the "Deploy" stage.
Perform some steps related to the "Deploy" stage.
Pipeline example
Here is an example of a Jenkinsfile using Declarative and it's equivalent scriptive Pipeline syntax:
Jenkinsfile (Declarative Pipeline):
pipeline {
agent any
options {
skipStagesAfterUnstable()
}
stages {
stage('Build') {
steps {
sh 'make'
}
}
stage('Test'){
steps {
sh 'make check'
junit 'reports/**/*.xml'
}
}
stage('Deploy') {
steps {
sh 'make publish'
}
}
}
}
Jenkinsfile (Scripted Pipeline):
node {
stage('Build') {
sh 'make'
}
stage('Test') {
sh 'make check'
junit 'reports/**/*.xml'
}
if (currentBuild.currentResult == 'SUCCESS') {
stage('Deploy') {
sh 'make publish'
}
}
}
Related
I'm trying to configure a pipeline with these steps:
Build
Copy artifacts
Integration test
I want them to be executed in this specific order on the same agent.
My setup is the server and one agent.
If I do this configuration
pipeline {
agent ???
options {
copyArtifactPermission(...);
}
stages
{
stage('BuildStage')
{
agent {label 'agent1'}
steps
{
build( [...] )
}
}
stage('Copy artifacts')
{
agent {label 'agent1'}
steps
{
copyArtifacts ...
}
}
stage('Integration Test')
{
agent {label 'agent1'}
steps
{
build( [...] )
}
}
}
}
It seems that the steps are not scheduled because the master node is busy executing the pipeline.
I'm thinking that I should force the master to execute the pipeline (hence the agent ???) and each stage on the agent?
I can get it to work by adding 'wait:false' to each step, but then I'm not sure the order is kept?
I have also set number of executors to 2, but it doesn't seem to solve the problem.
I'm coming from TeamCity, so maybe I've just misunderstood something basic about Jenkins?
If all the steps are meant to be run in the same agent, you can set it only once:
pipeline {
agent { label 'agent1' }
...
// Remove all other 'agent'
}
Don`t worry about the stages order (quote from https://www.jenkins.io/doc/book/pipeline/syntax/#sequential-stages)
Stages in Declarative Pipeline may have a stages section containing a
list of nested stages to be run in sequential order.
I am trying to build a Jenkins Declarative pipeline with a Jenkinsfile. The Jenkinsfile would be present on the repo of the project.
The Jenkinsfile would be something like the following:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
} }
However, I would like to enforce some stages in Jenkins regardless of the file. So as an example the pipeline would run Build -> Test -> Deploy stages from the file and an additional stage predefined on Jenkins like if it was a scripted pipeline.
Do you know if this is possible? How can I do it?
You could use a shared library to reuse code. It won't be as smooth as you probably liked, but you might use global variables and/or custom steps to encapsulate common functionality.
This would require some glue code (#Library(), script { } etc.), but this approach is very explicit and powerful - you can use library in any place in your pipeline.
Maybe you can try out Jenkins Templating Engine.
It gives you comprehensive Jenkins pipeline governance and templating capability.
I am trying to automate my build using Jenkins. My build process needs to execute three different shell scripts. The first script sets some environment variables which is used by the second and the third scripts.
I am trying with a pipeline job in Jenkins where each script is executed stage by stage. However I am unable to get the environment variables from the first script to the next one.
NB: There is a set of variables that are being set.So I don't feel like using a simple variable will do.
Please help
You are probably confusing declarative pipeline with scripted pipeline
Jenkinsfile (Declarative Pipeline)
pipeline {
agent any
environment {
DISABLE_AUTH = 'true'
DB_ENGINE = 'sqlite'
}
stages {
stage('Build') {
steps {
sh 'printenv'
}
}
}
}
Jenkinsfile (Scripted Pipeline)
node {
withEnv(['DISABLE_AUTH=true',
'DB_ENGINE=sqlite']) {
stage('Build') {
sh 'printenv'
}
}
}
I have two jobs running on two different OS, all the build steps are the same, it is the tools (jdk and maven), the meta data that are different.
I want to make a job that include both jobs on two agents depending on the OS.
I'm using Jenkins Pipeline Declarative Syntax, the problem is that I couldn't find a way to declare tools for a specific agent.
In Jenkins Pipeline, we can declare tools inside the entire pipeline or inside a specific stage and that's it.
PS: I need to use the declarative Syntax: no use of node {}
If I do so:
stage('Environment Set Up Linux') {
agent {
label "linux"
}
tools {
jdk 'oracle-jdk-1.8'
}
steps {
echo "Environment tools have been configured"
}
}
stage('Environment Set Up Solaris') {
agent {
label "solaris-64"
}
tools {
jdk 'oracle-jdk-1.7'
}
steps {
echo "Environment tools have been configured"
}
}
The tools will be used only for those stages not all stages and making tools in every stage would be stupid.
Define the common tools which are available on every slave in the entire pipeline and the specific ones in the stage section:
pipeline {
agent any
tools {
maven 'Maven 3.3.9'
}
stages {
stage('test'){
tools {
maven 'Maven 2.2.1'
}
steps {
sh 'mvn --version'
}
}
stage('random'){
steps {
sh 'mvn --version'
}
}
}
}
In this case the output in the stage 'test' is 2.2.1 because I define my tools in the stage section which overwrites the global pipeline. In the stage random I define no tools inside the stage so the tools which are defined in the global pipeline are used and 3.3.9 is printed. I hope this is what you meant.
In your case it could be all agents contain jdk1.8 and you want to use it in nearly ever stage (define it in the pipeline), if there is one stage in which you want to use jdk 1.7, just define the tools in the stage section which will overwrite the global config.
We use the jenkins pipeline dsl for our job descriptions. Now we have something like that:
pipeline {
agent none
options {
timestamps()
}
environment {
//SOME ENV VARS
}
stages {
// more stages
stage('stage1'){
when {
expression { env.BRANCH_NAME == 'master' }
}
agent { label 'slave1' }
steps{
//doing stuff
}
}
}
A stage in the build process that should only run when the master branch is build, you can think of a deploy job or something in that direction. The problem is, our resources of agents with that particular label are limited. When we build other branches the job still invoke a slave1 agent and than skips the stage after checking the that the branch is not master. This is bad, because when all slave1 agents are working on master branch jobs, the other jobs will have to wait for a slave1 agent becoming available just to check that it does need to run that stage.
Is there any way with the jenkins pipeline DSL to skip that stage without waiting for the slave1 agent to determine the branch?
The approach provided in the accepted answer works because you are not declaring an agent in the stage('stage1'). Instead you explicitly create a node within the steps and so the agent is not created when you check the condition.
This approach will work but it adds unnecessary complexity to your Jenkinsfile. As of Jenkins pipeline plugin version 1.2.6, the correct way to achieve this would be:
pipeline {
agent none
stages {
// more stages
stage('stage1'){
when {
beforeAgent true
branch 'master'
}
agent { label 'slave1' }
steps {
// do stuff
}
}
}
}
Check the syntax and available options available in the when tag documentation.
I found a solution that worked for me. Although I'm not quite sure why.
Introducing a parallel section and using nodes however fixed the issue and the stages are skipped without invoking the agent first. See the modified pipeline:
pipeline {
agent none
options {
timestamps()
}
environment {
//SOME ENV VARS
}
stages {
// more stages
stage('stage1'){
when { branch 'master' }
steps{
parallel(
'Job1': {
node( 'slave1' ){
//doing stuff
}
}
)
}
}
}