According to Jenkins Pipeline Docs, I should be able to use pipeline steps while working with Docker. However, "archiveArtifacts" does not appear to work:
def container = docker.image("some_image")
container.inside {
sh 'date > /tmp/test.txt'
sh 'cat /tmp/test.txt' //works, shows file
def fileContents = readFile '/tmp/test.txt' //works
echo "Contents: ${fileContents}" //works, shows file
archiveArtifacts '/tmp/*.txt' //FAILS
}
"ERROR: No artifacts found that match the file pattern "/tmp/*.txt". Configuration error?".
Things I've tried:
Adding sync and sleep(5) before the archive step in case this is a file sync issue.
Trying to archive '/*' and '*' in case it's running on the host (same error).
Any suggestions on archiving files generated in a Docker container?
PS: I opened a bug report... It looks like archiveArtifacts will only work on files in $WORKSPACE in docker containers.
You seem to have found the solution as reported in the same Jira ticket so I'll post here for everyone:
this works fine:
def image = docker.image("alpine")
image.inside {
sh 'date > /tmp/test.txt'
sh "cp /tmp/test.txt ${WORKSPACE}"
archiveArtifacts 'test.txt'
}
Related
In my Jenkinsfile I want to dynamically find the unity version using a python script like so:
environment {
UNITY_EDITOR = bat(script: "py $WORKSPACE/get_versions.py --unity", returnStdout: true).trim()
UNITY_BASE = "C:/Program Files/Unity/Hub/Editor/$UNITY_EDITOR/Editor/Unity.exe"
UNITY_WRAPPER = "UnityBatchWrapper -silent-crashes -no-dialogs -batchmode -quit -unityPath \"$UNITY_BASE\""
}
post {
always {
script {
echo "Returning license"
licenseReturnStatus = bat (
script: "$UNITY_WRAPPER -returnlicense",
returnStatus: true
) == 0
}
}
From other stackoverflow answers this seems like it should work, but instead my Jenkins job errors out during the post-build step because $UNITY_WRAPPER isn't defined:
groovy.lang.MissingPropertyException: No such property: UNITY_WRAPPER for class: groovy.lang.Binding
I'm thinking the batch step is what's failing, even though Jenkins doesn't complain about it. I've also tried using $env.WORKSPACE and %WORKSPACE% and that doesn't work either.
I'm beginning to think $WORKSPACE doesn't exist til after the environments step...
Turns out I didn't have Python installed since it was an ephemeral GCP builder and I hadn't updated the node label yet.
For anyone reading this that has trouble with bat commands - be sure to put an # sign in front of your command like "#py ..." or else the command will be echoed in the output. Also trim your output so it doesn't have CRLF in it.
While running, my pipeline is duplicating the binaries located in a BitBucket workspace into the build workspace, then needs to add in the build workspace some secret files from the credential store, and then start to build the docker image.
But the pipeline is failing when copying the files.
I searched and applied different solutions found here but still have the same error.
Running the following command :
stage('push credential in jenkins workspace') {
steps {
script {
withCredentials([
file(credentialsId: 'saremediation', variable: 'SA_KEY_PATH')]){
sh "ls -al"
sh "mkdir ${CERTIFDIR}"
sh "cp ${SA_KEY_PATH} ${CERTIFDIR}/credent.json"
}
}
}
}
failed with the following error :
[Pipeline] sh
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [SA_KEY_PATH]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ cp **** server/src/configuration/certificats/credent.json
cp: target 'server/src/configuration/certificats/credent.json' is not a directory
the CERTIFDIR folder is well created, because when I add sh "ls -al ${CERTIFDIR}", I cans see that the folder is created and empty.
fix the problem by applyong this syntax in the cp command
sh "cp \"${SA_KEY_PATH}\" \"${CERTIFDIR}\""
As part of our pipeline I need to rename a file before it gets pushed up to GitHub. Previously this worked when running the Jenkins job on a master node, but now we run them on agents
def rename_build_file() {
print "Append Version Number to File"
// File without version
String myFile = "${WORKSPACE_PATH}/release-pipeline/project/dist/myFile.js
// File with version
String myFileNew = "${WORKSPACE_PATH}/release-pipeline/project/dist/myfile-1.0.js"
// Rename File
new File(myFile).renameTo(new File(myFileNew));
}
Within our JenkinsFile we call helper.rename_build_file() and this usually works
When i sshd onto the agent I found that I had to run sudo to manually change a filename (did not have to enter a password), am i to assume that when the Jenkins job is running it's not running as sudo
And if that's the case how could i do this running the job?
Thanks
When working with files across multiple agents, you should use pipeline's workflow steps like fileExists, readFile, and writeFile. You can use a combination of these steps to create a new file with the desired name in the current workspace.
def sourceFile = "release-pipeline/project/dist/myFile.js"
if (fileExists(file: sourceFile)) {
def newFile = "release-pipeline/project/dist/myFile-1.0.js"
writeFile(file: newFile, encoding: "UTF-8", text: readFile(file: sourceFile, encoding: "UTF-8"))
}
This can be done with the File Operations plugin:
pipeline {
agent any
stages {
stage('Rename') {
steps {
cleanWs()
fileOperations([fileCreateOperation(fileName: 'foo', fileContent: '')])
fileOperations([fileRenameOperation(destination: 'bar', source: 'foo')])
sh "ls -l"
}
}
}
}
The plugin has quite a list of supported file operations.
I am stuck in trying to get a Jenkinsfile to work. It keeps failing on sh step and gives the following error
process apparently never started in /home/jenkins/workspace
...
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
I have tried adding
withEnv(['PATH+EXTRA=/usr/sbin:/usr/bin:/sbin:/bin'])
before sh step in groovy file
also tried to add
/bin/sh
in Manage Jenkins -> Configure System in the shell section
I have also tried replacing the sh line in Jenkinsfile with the following:
sh "docker ps;"
sh "echo 'hello';"
sh ./build.sh;"
sh ```
#!/bin/sh
echo hello
```
This is the part of Jenkinsfile which i am stuck on
node {
stage('Build') {
echo 'this works'
sh 'echo "this does not work"'
}
}
expected output is "this does not work" but it just hangs and returns the error above.
what am I missing?
It turns out that the default workingDir value for default jnlp k8s slave nodes is now set to /home/jenkins/agent and I was using the old value /home/jenkins
here is the config that worked for me
containerTemplate(name: 'jnlp', image: 'lachlanevenson/jnlp-slave:3.10-1-alpine', args: '${computer.jnlpmac} ${computer.name}', workingDir: '/home/jenkins/agent')
It is possible to get the same trouble with the malformed PATH environment variable. This prevents the sh() method of the Pipeline plugin to call the shell executable. You can reproduce it on a simple pipeline like this:
node('myNode') {
stage('Test') {
withEnv(['PATH=/something_invalid']) {
/* it hangs and fails later with "process apparently never started" */
sh('echo Hello!')
}
}
}
There is variety of ways to mangle PATH. For example you use withEnv(getEnv()) { sh(...) } where getEnv() is your own method which evaluates the list of environment variables depending on the OS and other conditions. If you make a mistake in the getEnv() method and PATH gets overwritten you get it reproduced.
Yesterday, I wrote and ran a Katalon test suite, and today, I'm trying to integrate Katalon with Jenkins. I successfully setup Jenkins, created a new job for the Katalon testing, as per these instructions, but when I went to build it, I get failing builds.
In particular, this is the error message I keep getting :
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE
I went ahead and copied the Reports folder structure from the project directory that I specified to the Jenkins workspace. Upon later inspection, I found that, when Jenkins was running the Katalon tests, the JUnit_Report.xml file was actually getting created in the project's Reports folder, instead of at %JENKINS_HOME%\workspace\[project name]\Reports. I explicitly told it to generate test reports to : Reports/LoginSuite/*/JUnit_Report.xml.
NOTE: I'm on a Windows machine.
How can I fix this so that I can display test results from Jenkins?!
UPDATE : I have revised my Windows shell code to the following
C:
cd C:\Katalon
katalon -runMode=console -projectPath="C:\Users\mwarren\Katalon Studio\TestProject" -reportFolder="../../.jenkins/workspace/Katalon Studio Tests/Reports" -reportFileName="report" -retry=0 -testSuitePath="Test Suites/LoginSuite" -browserType="Chrome"
and it's still giving me the same error, even though now the tests are being created there.
Report folder is generated in the jenkins job folder.
Reports/**/JUnit_Report.xml
PEBKAC, apparently. I should have, from the getgo, listened to Jenkins and set my Test Report XMLs as */JUnit_Report.xml
Copy all reports to a temp folder, rename each xml with test case name and then copy it back to junit folder.
testCasesTxt = sh (
script: 'sudo find $WORKSPACE -name "*.ts*" -type f -printf "%f\n"',
returnStdout: true
).trim()
testCasesTxt = testCasesTxt.replace(".ts", "")
testCases = testCasesTxt.split("\n")
for (int i = 0; i < testCases.size(); i++) {
script {
try {
wrap([$class: 'Xvfb']) {
sh """
cd /opt/katalon
./katalon -noSplash -consoleLog -runMode=console -projectPath=$WORKSPACE/"katalon-project.prj" -reportFolder="Reports" -reportFileName="report" -retry=0 -testSuitePath="Test Suites/${testCases[i]}" -executionProfile="qa" -browserType="Chrome"
"""
}
} catch (any) {
currentBuild.result = 'FAILURE'
throw any //rethrow exception to prevent the build from proceeding
} finally {
sh """
cd /home/environment/tmp/
cd Reports
mkdir ${testCases[i]}
cd $WORKSPACE
cp -r Reports/ /home/environment/tmp/Reports/${testCases[i]}
cd /home/environment/tmp/Reports/${testCases[i]}/Reports
mv JUnit_Report.xml JUnit_Report_${testCases[i]}.xml
cd $WORKSPACE
cp -r "Data Files/" "/home/environment/tmp/"
"""
//
}
}
}
According to this thread, this can happen when trying to execute two Katalon instances at the same moment.
If that's the case, try changing Jenkins number of executors 1.
If your report is at \Reports\20200611_172240\TestSuite1\20200611_172240/JUnit_Report.xml location in your project folder, then configure the folder path as below - /Reports///*/JUnit_Report.xml
as the 3 folders after report folder name are always going to change after each execution.
Please use the Test report XMLs path as like this
**/target/surefire-reports/*.xml