I'm trying to write a JenkinsFile that automatically will reach to a git repo via ssh and perform some actions but I want to make the repo and ssh key use variables with the ssh id stored in Jenkins but I seem to be missing the Jenkins documentation for how to pass down variables to Jenkins Files as I'm not able to pass values down into the credentials key. The variables being passed down to the sh commands resolve perfectly fine though...
Example Pipeline Below:
pipeline {
parameters {
string(name: 'SSH_priv', defaultValue: 'd4f19e34-7828-4215-8304-a2d1f87a2fba', description: 'SSH Credential with the private key added to Jenkins and the public key to the username stored in Git Server, this id can be found in the credential section of Jenkins post its creation.')
string(name: 'REPO', defaultValue: 'git#--------------------')
}
stages {
stage ('Output Variables'){
// checks I can get these variables
steps{
sh("echo ${params.SSH_priv}")
sh("echo ${params.REPO}")
}
}
stage('Do Something') {
steps {
// this below commented line, does not work.
// sshagent (credentials: ['${params.SSH_priv}']){
// this line does work
sshagent (credentials: ['d4f19e34-7828-4215-8304-a2d1f87a2fba']){
sh("git clone --mirror ${params.REPO} temp")
dir("temp"){
// start doing fancy stuff ...
....
....
}
}
}
}
The aim is a Pipeline that my fellow developers could call and will work with their own repos and own ssh id's that I'm not using. When I try to run this with the SSH_priv parameter passing down the value I get the below failure in Jenkins.
The JenkinsFile works perfectly fine with the credential id hard-coded- as shown below:
So after testing different things a friend solved this in sub 5 minutes. Quotation mark types matter in Groovy Script
Changing
sshagent (credentials: ['${params.SSH_lower}']){
To
sshagent (credentials: ["${params.SSH_lower}"]){
Solved the issue.
Better to use environment step in pipeline.
pipeline {
agent any
environment {
AN_ACCESS_KEY = credentials('an_access_key_id')
}
stages {
stage('Example') {
steps {
sh 'printenv'
}
}
}
}
And credentials should exist in jenkins with id an_access_key_id
Take a look at official documentation here
Related
I've looked at this Pass Artifact or String to upstream job in Jenkins Pipeline and this Pass variables between Jenkins stages and this How do I pass variables between stages in a declarative Jenkins pipeline?, but none of these questions seem to deal with my specific problem.
Basically I have a pipeline consisting of multiple stages, each run in its own agent.
In the first stage I run a shell script. Here two variables are generated. I would like to use these variables in the next stage. The methods I've seen so far seem to only work when passing variables within the same agent.
pipeline {
stages {
stage("stage 1") {
agent {
docker {
image 'my_image:latest'
}
}
steps {
sh ("""
export VAR1=foo
export VAR2=bar
""")
}
}
stage("stage 2") {
agent {
docker {
image 'my_other_image:latest'
}
}
steps {
sh ("echo "$VAR1 $VAR2")
//expecting to see "foo bar" printed here
}
}
I have two Jenkins Pipelines :
Pipeline A : In a stage, I defined an environment variable called MAVEN_PROFILE (the user can choose a value from a list)
Pipeline B : I need to get the MAVEN_PROFILE environment variable value that was set in Pipeline A
I need two pipelines because I can't do it in a single Pipeline for process reason.
I saw there was some answers on how to share variable between stages within a single Pipeline but this not my case.
I want to share environment variable value between different Pipelines.
Pipeline A
pipeline {
agent any
...
stages {
stage('Profile Selection'){
steps {
script {
env.MAVEN_PROFILE = input message: 'Choose the profile :',
parameters: [choice(name: 'MAVEN_PROFILE',
choices: 'all\nserver\nclient', description: 'Profiles')]
}
}
}
stage(...){
steps {
script {
bat "mvn deploy -P ${env.MAVEN_PROFILE}"
}
}
}
... other stages
}
}
Pipeline B
pipeline {
agent any
...
stages {
... other stages
stage(...){
steps {
script {
bat "mvn release ... -P ${env.environmentVariableValueFromPipelineA}"
}
}
}
}
}
They're not running in the same environment, so they can't directly share environment variables. The easiest is probably to write these values to a file in the workspace in pipeline A, and read them back in in pipeline B. Something like this:
Pipeline A:
sh "echo ${MAVEN_PROFILE} > .MAVEN_PROFILE"
Pipeline B:
def MAVEN_PROFILE = sh(script: 'cat .MAVEN_PROFILE', returnStdout: true).trim()
I have a project which has multiple build pipelines to allow for different types of builds against it (no, I don't have the ability to make one build out of it; that is outside my control).
Each of these pipelines is represented by a Jenkinsfile in the project repo, and each one must use the same build agent label (they need to share other pieces of configuration as well, but it's the build agent label which is the current problem). I'm trying to put the label into some sort of a configuration file in the project repo, so that all the Jenkinsfiles can read it.
I expected this to be simple, as you don't need this config data until you have already checked out a copy of the sources to read the Jenkinsfile. As far as I can tell, it is impossible.
It seems to me that a Jenkinsfile cannot read files from SCM until the project has done its SCM step. However, that's too late: the argument to agent{label} is read before any stages get run.
Here's a minimal case:
final def config
pipeline {
agent none
stages {
stage('Configure') {
agent {
label 'master'
}
steps {
checkout scm // we don't need all the submodules here
echo "Reading configuration JSON"
script { config = readJSON file: 'buildjobs/buildjob-config.json' }
echo "Read configuration JSON"
}
}
stage('Build and Deploy') {
agent {
label config.agent_label
}
steps {
echo 'Got into Stage 2'
}
}
}
}
When I run this, I get:
java.lang.NullPointerException: Cannot get property 'agent_label' on null object I don't get either of the echoes from the 'Configure' stage.
If I change the label for the 'Build and Deploy' stage to 'master', the build succeeds and prints out all three echo statements.
Is there any way to read a file from the Git workspace before the agent labels need to be set?
Please see https://stackoverflow.com/a/52807254/7983309. I think you are running into this issue. label is unable to resolve config.agent_label to its updated value. Whatever is set in the first line is being sent to your second stage.
EDIT1:
env.agentName = ''
pipeline {
agent none
stages {
stage('Configure') {
agent {
label 'master'
}
steps {
script {
env.agentName = 'slave'
echo env.agentName
}
}
}
stage('Finish') {
steps {
node (agentName as String) { println env.agentName }
script {
echo agentName
}
}
}
}
}
Source - In a declarative jenkins pipeline - can I set the agent label dynamically?
I'm new to using Jenkins....
I'm trying to automate the production of an image (to be stashed in a repo) using a declarative Jenkinsfile. I find the documentation to be confusing (at best). Simply put, how can I convert the following scripted example (from the docs)
node {
checkout scm
def customImage = docker.build("my-image:${env.BUILD_ID}")
customImage.push()
}
to a declarative Jenkinsfile....
You can use scripted pipeline blocks in a declarative pipeline as a workaround
pipeline {
agent any
stages {
stage('Build image') {
steps {
echo 'Starting to build docker image'
script {
def customImage = docker.build("my-image:${env.BUILD_ID}")
customImage.push()
}
}
}
}
}
I'm using following approach:
steps {
withDockerRegistry([ credentialsId: "<CREDENTIALS_ID>", url: "<PRIVATE_REGISTRY_URL>" ]) {
// following commands will be executed within logged docker registry
sh 'docker push <image>'
}
}
Where:
CREDENTIALS_ID stands for key in Jenkis under which you store credentials to your docker registry.
PRIVATE_REGISTRY_URL stands for url of your private docker registry. If you are using docker hub then it should be empty.
I cannot recommend the declarative syntax for building a Docker image bcos it seems that every important step requires falling back to the old scripting syntax. But if you must, a hybrid approach seems to work.
First a detail about the scm step: when I defined the Jenkins "Pipeline script from SCM" project that fetches my Jenkinsfile with a declarative pipline from git, Jenkins cloned the repo as the first step in the pipeline even tho I did not define a scm step.
For the build and push steps, I can only find solutions that are a hybrid of old-style scripted pipeline steps inside the new-style declarative syntax. For example see gustavoapolinario's work at Medium:
https://medium.com/#gustavo.guss/jenkins-building-docker-image-and-sending-to-registry-64b84ea45ee9
which has this hybrid pipeline definition:
pipeline {
environment {
registry = "gustavoapolinario/docker-test"
registryCredential = 'dockerhub'
dockerImage = ''
}
agent any
stages {
stage('Cloning Git') {
steps {
git 'https://github.com/gustavoapolinario/microservices-node-example-todo-frontend.git'
}
}
stage('Building image') {
steps{
script {
dockerImage = docker.build registry + ":$BUILD_NUMBER"
}
}
}
stage('Deploy Image') {
steps{
script {
docker.withRegistry( '', registryCredential ) {
dockerImage.push()
}
}
}
}
stage('Remove Unused docker image') {
steps{
sh "docker rmi $registry:$BUILD_NUMBER"
}
}
}
}
Because the first step here is a clone, I think he built this example as a standalone pipeline project in Jenkins (not a Pipeline script from SCM project).
I am trying to parameterize a Jenkins pipeline. The only input parameter will be GITHUB_URL. I have a Jenkinsfile as a part of the repo. I want to use this variable (defined as parameter) in my pipeline configuration as "Repository URL". How can I access the parameter ?
I have tried $GITHUB_URL, ${GITHUB_URL} and ${params.GITHUB_URL}. No luck
Any other suggestions?
Because you are telling you have a jenkinsfile inside your git repo I suppose you do not mean that you want to call a Jenkinsfile using parameters from a shared library.
It's also not sure if you are using a declarative or scripted pipeline.
I will explain the "recommended" declarative pipeline:
pipeline {
agent any
parameters {
string(defaultValue: "https://github.com", description: 'Whats the github URL?', name: 'URL')
}
stages {
stage('Checkout Git repository') {
steps {
git branch: 'master', url: "${params.URL}"
}
}
stage('echo') {
steps {
echo "${params.URL}"
}
}
}
}
In this pipeline you will add a string parameter to which you can add a URL. When you run the build it will ask for the parameter:
To use this parameter use "${params.URL}":
This pipeline will clone the github repo in the first stage and print the URL in the next (echo) stage:
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (echo)
[Pipeline] echo
https://github.com/lvthillo/docker-ghost-mysql.git
[Pipeline] }