I would like to activate a specific set of tools via SDKMAN in my scripted Jenkinsfile pipeline.
Basically I would like to do
node("sdkman && mvn386") {
sh "sdk use maven 3.8.6"
sh "sdk current" // << back to whatever was before
sh "mvn --version"
sh "mvn clean install"
}
But since sdk ... is executed in a subshell the use is not active in the next like.
(It would work with sdk default ... but that is out of the question, because I do not want to destroy other running pipelines)
Semantically it would be something like this I need (which is not supported, I think):
node("sdkman && mvn386") {
sh ("sdk use maven 3.8.6") { // << unsupported syntax
sh "sdk current"
sh "mvn --version"
sh "mvn clean install"
}
}
How can I keep the changes that sdk makes to the shell env to the next lines? Is there another tool to use instead of sh? Something like withEnv?
Side problem: Because sdk is just a shell function I needed to create a wrapper script sdh.sh (containing sdk $*). That of course defies the "chaning the environment", too. sh "sdk ..." just does not seem to work with shell functions. Any way around that?
Something I do not want to do is putting the script lines into one block, because that would not work well with our more complicated pipelines.
sh '''
sdk use maven 3.8.6
sdk current
''' // << I dont want to do this in one shell block
Related
I need to get GitVersion.exe variables in my Jenkins pipeline.
The GitVersion documentation gives a hint on how to do that. Essentially call gitversion /output buildserver.
This call does add the variables to the current step and they are lost once the step completes. I can show this call executes when combining a set command in the same bat execution. The second set shows the variables are gone from the environment.
bat 'nuget install GitVersion.CommandLine -OutputDirectory c:/packages -Version 3.6.5'
bat 'c:/packages/GitVersion.CommandLine.3.6.5/tools/GitVersion.exe /output buildserver && set'
bat 'set'
The documentation of GitVersion is aware of that and suggests to use EnvInject.
Installing the plugin and executing the same pipeline did not change the result. I read that the Plugin is not made for pipelines so that may have something to do with it.
Pipelines support a syntax for environment.
Following that syntax I can set static variables at the top of my pipeline like this:
environment {
ASuperVariable = 'MySuperVariable'
}
What I need is combining those calls so that I can add run time variables to the Jenkinsfile pupeline.
environment {
bat 'gitversion /output buildserver'
}
Now obviously the above call is not even syntax correct. Is there a way to mark a section so that the contained environment changes are available for other steps?
EDIT:
This is still unsolved. At the moment I need to create a batch script and pass the tool into it as an argument. Inside the batch I can call the tool to add to the environment of the batch script and use that wile the batch is running. A Multi line batch in the Jenkins file could be a solution if the process remains the same over all the multiple lines.
Not sure whether you would be able to use scripted pipeline or at least a script block inside declarative. It'd be quite easy doing so:
withEnv(['ASuperVariable=MySuperVariable']) {
echo env.ASuperVariable
}
Or when calling a windows cmd script:
node('win') {
withEnv(['ASuperVariable=MySuperVariable']) {
bat 'echo %ASuperVariable%'
}
}
I need to ssh to a server from a simple jenkin pipeline and make a deploy which is simply moving to a directory and do a git fetch and some other comands (nmp install among others). Thing is that when jenkin job ssh to the remote server it connects ok but then It gets stucked, I have to stop it. I just now modify the script to simply do a "ssh to server " and a "pwd command" to go to the easiest but it connects to it and it get stuck untill I abort. What Am I missing? here is the simpe pipeline script and the output on an screenshot
pipeline {
agent any
stages {
stage('Connect to server') {
steps {
sh "ssh -t -t jenkins#10.x.x.xx"
sh "pwd"
}
}
stage('branch status') {
steps {
sh "git status"
}
}
}
}
Jenkins executes each "sh" step as a separate shell script. Content is written to a temporary file on Jenkins node and only then executed. Each command is executed in separate session and is not aware of previous one. So neither ssh session or changes in environment variable will persist between the two.
More importantly though, you are forcing pseudo-terminal allocation with -t flag. This is pretty much opposite to what you want to achieve, i.e. run shell commands non-interactively. Simply
sh "ssh jenkins#10.x.x.xx pwd"
is enough for your example to work. Placing the commands on separate lines would not work with regular shell script, regardless of Jenkins. However you still need to have private key available on node, otherwise the job will hang, waiting for you to provide password interactively. Normally, you will want to use SSH Agent Plugin to provide private key at runtime.
script {
sshagent(["your-ssh-credentals"]) {
sh "..."
}
}
For execution on longer commands see What is the cleanest way to ssh and run multiple commands in Bash?
I've set up a home based CI server for working with a personal project. Below you can see what happens for the branch "staging". It works fine, however the problems with such a pipeline config are:
1) The only way to stop the instance seem to be to abort the build in jenkins whiсh leads to the exit code 143 and build marked as red instead of green
2) If the machine reboots I have to trigger build manually
3) I suppose there should be a better way of handling this?
Thanks
stage('Staging') {
when {
branch 'staging'
}
environment {
NODE_ENV = 'production'
}
steps {
sh 'docker-compose -f docker-compose/staging.yml build'
sh 'docker-compose -f docker-compose/staging.yml up --abort-on-container-exit'
}
post {
always {
sh 'docker-compose -f docker-compose/staging.yml rm -f -s'
sh 'docker-compose -f docker-compose/staging.yml down --rmi local --remove-orphans'
}
}
}
So, what's the goal here? Are you trying to deploy to staging? If so, what do you mean by that? If jenkins is to launch a long running process (say a docker container running a webserver) then the shell command line must be able to start and then have its exit status tell jenkins pipeline if the start was successful.
One option is to wrap the docker compose in a script that executes, checks and exits with the appropriate exit code. Another is to use yet another automation tool to help (e.g. ansible)
The first question remains, what are you trying to get jenkins to do and how on the commandline will that work. If you can model the command line then you can encapsulate in a script file and have jenkins start it.
Jenkins pipeline code looks like groovy and is much like groovy. This can make us believe that adding complex logic to the pipeline is a good idea, but this turns jenkins into our IDE and that's hard to debug and a trap into which I've fallen several times.
A somewhat easier approach is to have some other tool allow you to easily test on the commandline and then have jenkins build the environment in which to run that command line process. Jenkins handles what it is good at:
scheduling jobs
determining on which nodes jobs run
running steps in parallel
making the output pretty or easily understood by we carbon based life forms.
I am using parallel stages.
Here is a minimum example:
pipeline {
agent any
options {
parallelsAlwaysFailFast() // https://stackoverflow.com/q/54698697/4480139
}
stages {
stage('Parallel') {
parallel {
stage('docker-compose up') {
steps {
sh 'docker-compose up'
}
}
stage('test') {
steps {
sh 'sleep 10'
sh 'docker-compose down --remove-orphans'
}
}
}
}
}
post {
always {
sh 'docker-compose down --remove-orphans'
}
}
}
I am using jenkins to run pipeline groovy scripts. One of the first step is a checkout via the checkout plugin. The checkout happens into the <workspace>/source-repo folder.
Now, when I do lsof (it is a linux machine) I get a lot of open file handlers like this:
java 16932 1000 567r REG 202,80 91 7996215 <workspace>/source-repo#tmp/durable-a06b8b8d/output.txt (deleted)
They are building up over time ... Why? And what can i do?
I found the problem, it seems to be related to sh in combination with returnStdout: true. So I replaced calls like this:
def ret = sh script: "command", returnStdout: true
with
sh "command > output.txt"
def ret = readFile "output.txt"
sh "rm output.txt"
Feels a bit hacky, but now I am fine.
Seems to be fixed in the Durable Task Plugin since Version 1.14 (Jun 15 2017)
https://issues.jenkins-ci.org/browse/JENKINS-43639
I am currently trying to run a groovy script from my pipeline as one of my nodes, but I ran into this error:
[CompanyName] Running shell script
+ ./ideainspect.groovy
env: groovy: No such file or directory
Also, I tried installing the plugin for groovy, but for some reason, it won't install. Whenever I refresh the page for tools, the installer goes away. Am I installing groovy wrong? Please help!
Edit: Relevant Data
stage 'Static Analysis'
node {
dir("Android/btMobileApp") {
sh "./ideainspect.groovy"
sh "./gradlew checkstyle lintDebug"
}
}
And the ideainspect.groovy file is an executable with the shebang #!/usr/bin/env groovy, which could be the problem.
Does your jenkins user have groovy on its path? If so ditch the shebang and try
sh 'groovy ideainspect.groovy'
If it's not on the path, you can try
sh '/usr/bin/groovy ideainspect.groovy'
(or wherever groovy is installed)
Or alternatively add it to the path environment variable using Manage Jenkins -> Configure Systems. This may not be appropriate if you have lots of projects using different versions of groovy