I am looking to use a database username/password in my config.ini file. I have the following withCredentials line in my Jenkinsfile:
withCredentials([usernamePassword(credentialsId: 'database', usernameVariable: 'DATABASE_USER', passwordVariable: 'DATABASE_PASSWORD')])
I don't explicitly call this config.ini file in my Jenkinsfile, however I do use a bash script to:
export CONFIG_FILE='config.ini'
Is there any way to set these accordingly in my config.ini:
DB_USERNAME = {DATABASE_USER}
DB_PASSWORD = {DATABASE_PASSWORD}
Bash can do this for you. You have two options:
Use envsubst. You'll need to install it on all of your nodes (it's usually part of the gettext package).
Use evil eval
Full example:
pipeline {
agent {
label 'linux' // make sure we're running on Linux
}
environment {
USER = 'theuser'
PASSWORD = 'thepassword'
}
stages {
stage('Write Config') {
steps {
sh 'echo -n "user=$USER\npassword=$PASSWORD" > config.ini'
}
}
stage('Envsubst') {
steps {
sh 'cat config.ini | envsubst > config_envsubst.ini'
sh 'cat config_envsubst.ini'
}
}
stage('Eval') {
steps {
sh 'eval "echo \"$(cat config.ini)\"" > config_eval.ini'
sh 'cat config_eval.ini'
}
}
}
}
This this Stackexchange question for more options.
Related
I have a Jenkins file that sits in a directory on a github repository.
I am basically trying to automate the build, test, and publication of all docker images that are build within the sub-directories of the root directory where the Jenkinsfile sits.
As I know right now, there is an issue with the dir block under the stage('Build'), the following is my Jenkinsfile:
#!groovy
// Iterate over data_pipeline directory
// Store all directory pipelines if Dockerfile found in variable map
def dirMap = [
"docker_hub_image_metadata",
"github_metrics",
"google_sheets_sync",
"jira_syn",
"sfdc_get_license_keys_from_clouds",
"sfdc_sync",
"sfdc_sync_reports_to_snowflake",
"snowflake_oppty_history",
"snowflake_telemetry_tasks",
"support_analytics",
"sync_ee_license_from_s3"
]
def builds = [:]
dirMap.each { dir ->
builds << [
"${dir}": { ->
node {
stage('Build') {
sh "echo 'Building Images for ${dir}...'"
sh "cd ${dir}"
sh "echo 'Switched into ${dir}'"
sh "pwd"
//dir("${dir}") {
sh "pwd"
// def image = docker.build("mirantiseng/${dir}")
//}
}
stage('Test') {
sh "echo 'Testing Images for ${dir}...'"
}
if (currentBuild.currentResult == 'SUCCESS') {
stage('Push') {
sh "echo 'Puhing Images for ${dir}...'"
}
}
}
}
]
}
parallel(builds)
As you can see I have my sub directories hardcoded as a list ad then that list get injected onto the map builds. However, I have an issue when I try to run the follwing inside my stage('Build'):
dir("${dir}") {
sh "pwd"
def image = docker.build("mirantiseng/${dir}")
}
The last know build that succeeded did not include the above block.
To run this, to something similar you can just mimic it by placing the file into a directory with other directories and hardcode them to the list. The file I am trying to get working with Jenkins looks like this:
#!groovy
// Iterate over data_pipeline directory
// Store all directory pipelines if Dockerfile found in the variable map
def dirMap = [
"docker_hub_image_metadata",
"github_metrics",
"google_sheets_sync",
"jira_syn",
"sfdc_get_license_keys_from_clouds",
"sfdc_sync",
"sfdc_sync_reports_to_snowflake",
"snowflake_oppty_history",
"snowflake_telemetry_tasks",
"support_analytics",
"sync_ee_license_from_s3"
]
def builds = [:]
dirMap.each { dir ->
builds << [
"${dir}": { ->
node {
stage('Build') {
sh "echo 'Building Images for ${dir}...'"
dir("${dir}") {
def image = docker.build("mirantiseng/${dir}")
}
}
stage('Test') {
sh "echo 'Testing Images for ${dir}...'"
}
if (currentBuild.currentResult == 'SUCCESS') {
stage('Push') {
sh "echo 'Puhing Images for ${dir}...'"
}
}
}
}
]
}
parallel(builds)
The error I got with the above Jenkinsfile showed this:
+ echo 'Building Images for docker_hub_image_metadata...'
Building Images for docker_hub_image_metadata...
groovy.lang.MissingMethodException: No signature of method: java.lang.String.call() is applicable for argument types: (org.codehaus.groovy.runtime.GStringImpl, org.jenkinsci.plugins.workflow.cps.CpsClosure2) values: [docker_hub_image_metadata, org.jenkinsci.plugins.workflow.cps.CpsClosure2#dde112]
Possible solutions: wait(), any(), trim(), split(), collect(), grep()
I have a below jenkins pipeline and it is working fine
pipeline {
agent
{
node
{
label 'test'
}
}
environment{
ansible_pass = 'credentials('ans-pass')'
}
stages {
stage('Load Vars'){
steps{
script{
configFileProvider([configFile(fileId: "${ENV_CONFIG_ID}", targetLocation: "${ENV_CONFIG_FILE}")]) {
load "${ENV_CONFIG_FILE}"
}
}
}
}
stage('svc install') {
steps {
sshagent(["${SSH_KEY_ID}"])
{
sh '''
ansible-playbook main.yaml -i hosts.yaml -b --vault-password-file $ansible_pass
'''
}
}
}
}
}
Now i want to pass the global environment variable id from shell instead of hartcoding
ansible_pass = 'credentials('ans-pass')'===>>>>
this ansible-pass1 should come from managed files(config provider)
I have already below from managed files
env.ARTI_TOKEN_ID='art-token'
env.PLAYBOOK_REPO='dep.stg'
env.SSH_KEY_ID = 'test_key'
Now how to add this credential id in this file?.Tried like below
env.ansible_pass = 'ansible-pass1'
and in jenkins pipeline refered the same as below
environment{
ansible_pass = 'credentials($ansible_pass)'
}
But it didn't worked.Could you please advice
As you are using secrets in config file it is better to use secret type 'secret file' in jenkins. Follow the link to read about different types of credentials.
Also correct way of setting credentials is:
environment{
ansible_pass = credentials('credentials-id-here')
}
I have a spring boot application on centos server and use a shell file to restart it.
jenkins version: docker run -dp 8080:8080 --name jenkins jenkinsci/blueocean
start-service.sh
#!/bin/bash
sudo systemctl restart sb
In my Jenkinsfile i upload jar file to server and execute the start-service.sh, but jenkins seem dosen't know my java application restart success or fail.
Jenkinsfile
pipeline {
agent none
stages {
stage('Build') {
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
}
steps {
sh 'mvn -B -DskipTests clean package'
sh 'mvn help:evaluate -Dexpression=project.name | grep "^[^\\[]" > project-name'
sh 'mvn help:evaluate -Dexpression=project.version | grep "^[^\\[]" > project-ver'
}
}
stage('Deploy') {
agent any
environment {
HOST = "${HEHU_HOST}"
USER = "yunwei"
DIR = "/www/java/sb-demo"
VERSION_FILE = "${DIR}/version"
CMD_SERVICE = "${DIR}/start-service.sh"
}
steps {
sshagent (credentials: ['hehu']) {
sh '''
name=$(cat project-name)
ver=$(cat project-ver)
jarFile=${name}-${ver}.jar
scp target/${jarFile} ${USER}#${HOST}:${DIR}/${jarFile}
scp project-ver ${USER}#${HOST}:${VERSION_FILE}
ssh -o StrictHostKeyChecking=no -l ${USER} ${HOST} -a ${CMD_SERVICE}
'''
}
}
}
}
}
I deliberately let Java application go wrong, and systemctl restart is fail but jenkins stage is success.
#SpringBootApplication
#RestController
public class Application {
public static void main(String[] args) {
throw new RuntimeException("Test error");
// SpringApplication.run(Application.class, args);
}
#GetMapping("/test")
String test() {
return "furukawa nagisa\n";
}
}
Try Daniel Taub solution get syntax error.
pipeline {
agent none
stages {
stage('Hello') {
agent any
steps {
sshagent (credentials: ['hehu']) {
SH_SUCCESS = sh(
script: '''
ssh -o StrictHostKeyChecking=no -l yunwei ${HEHU_HOST} -a /www/java/sb-demo/start-service.sh
''',
returnStatus: true
) == 0
echo '${SH_SUCCESS}'
}
}
}
}
}
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 9: Expected a step # line 9, column 21.
SH_SUCCESS = sh(
You can get the sh returned exit status code and fail the build manually if its error status.
For checking your exit code, Jenkins support it that way
returnStatus (optional)
Normally, a script which exits with a nonzero status code will cause the step to fail with an exception. If this option is checked, the return value of the step will instead be the status code. You may then compare it to zero, for example.
script {
SH_SUCCESS = sh (
script: "your command",
returnStatus: true
) == 0
}
To manually fail the build you have couple of options:
error('Fail my build!')
Or alternatively
currentBuild.result = 'FAILURE'
return
I have a jenkins pipeline where I am executing different scripts at different stages. However in one stage I want to get the output of the stage to a variable where I want to pass that variable as an input to next stage . Here is my code in Jenkinsfile
timestamps
{
node('cf_slave')
{
checkout scm
stage('Download HA image from GSA')
{
withCredentials(usernamePassword(credentialsId: 'ssc4icp_GSA', usernameVariable: 'GSA_USERNAME', passwordVariable: 'GSA_PASSWORD')
{
environment {
script {
OUTPUT = """${sh(
returnStdout: true,
script: 'bash jenkins/try_install.sh $VAR_ABC'
)}"""
echo $OUTPUT
}
}
}
}
}
}
Here i am getting syntax error. I want to get the OUTPUT in OUTPUT variable and pass that to next stage. Please help me how to do that in a correct way
When referencing variable outside of a string you should not us a dollar sign ($). The code should be (including changes suggested by Matt):
timestamps
{
node('cf_slave')
{
checkout scm
stage('Download HA image from GSA')
{
withCredentials(usernamePassword(credentialsId: 'ssc4icp_GSA', usernameVariable: 'GSA_USERNAME', passwordVariable: 'GSA_PASSWORD'))
{
environment {
script {
OUTPUT = sh returnStdout: true,
script: "bash jenkins/try_install.sh $VAR_ABC"
echo OUTPUT
}
}
}
}
}
}
I have defined global variable in Jenkins pipeline
def BUILDNRO = '0'
pipeline { ...
Then i manipulate variable with shell script to enable running builds parallel by using job build number as identifier so we don't mix different docker swarms.
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
# Check if BUILD first character is 0
if [[ $Build:0:1 == "0" ]]; then
# replace BUILD first character from 0 to 5
Build=`echo $Build | sed s/./5/1`
fi
echo $Build
''',returnStdout: true).trim()
}
}
}
i get value out from previos stage and trying to get global variable on next stage
stage('DOCKER: Init docker swarm') {
steps {
echo "BUILDNRO is: ${BUILDNRO}" --> Value is here.
sh '''#!/bin/bash
echo Buildnro is: ${BUILDNRO} --> This is empty.
...
}
}
This will out give global variable empty. why? in previous stage there was value in it.
EDIT 1.
Modified code blocks to reflect current status.
I managed to figure it out. Here is solution how i managed to did it.
BUILDNRO is groovy variable and if wanting to used in bash variable it have to pass using withEnv. BUILD_NUMBER in first stage is bash variable hence it can be used directly script in first stage.
def BUILDNRO = '0'
pipeline {
....
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
''',returnStdout: true).trim()
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
dir("prose_env/prose_api_dev_env") {
withEnv(["MYNRO=${BUILDNRO}"]) {
sh(returnStdout: false, script: '''#!/bin/bash
echo Buildnro is: ${MYNRO}`
'''.stripIndent())
}
}
}
}
}
}
If you are using single quotes(```) in the shell module, Jenkins treats every variable as a bash variable. The solution is using double quotes(""") but then if you made bash variable you have to escape it. Below an example with working your use case and escaped bash variable
pipeline {
agent any
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh(script: 'pwd', returnStdout: true).trim()
echo "BUILDNRO is: ${BUILDNRO}"
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
sh """#!/bin/bash
echo Buildnro is: ${BUILDNRO}
variable=world
echo "hello \${variable}"
sh """
}
}
}
}
output of the second stage:
Buildnro is: /var/lib/jenkins/workspace/stack1
hello world