I have added many plugins to Jenkins. How can I list the plugins and dependencies? Which plugins depend on which ones? Which ones are orphaned or unused, etc.
Ideally, explain how to make a graph (graphviz/dot...) ?
Copy-paste this groovy snippet to get a list of plugins (this snippet based on this exemple from zendesk.com):
Note: the groovy must be pasted in _Manage Jenkins >> Script Console
def plugins = jenkins.model.Jenkins.instance.getPluginManager().getPlugins()
plugins.each {
println "${it.getShortName()} (${it.getVersion()}) => ${it.getDependencies()}"
}
To produce a graph, execute this snippet to generate a DOT graph (graphviz) file...
def plugins = jenkins.model.Jenkins.instance.getPluginManager().getPlugins()
println "digraph test {"
plugins.each {
def plugin = it.getShortName()
println "\"${plugin}\";"
def deps = it.getDependencies()
deps.each {
def s = it.shortName
println "\"${plugin}\" -> \"${s}\";"
}
}
println "}"
Then use graphviz to generate an image from the output above:
dot -Tsvg plugins.txt > plugins.svg
dot -Tpng plugins.txt > plugins.png
Or copy-paste the output in one of the Graphviz: Online tool capable of accepting larger files
Related
I'm trying to list folders containing a certain file on jenkins and use later this array.
I read about findFiles but I can't find a way to use it in this situation.
The finality is that I need to cd to those folders in a loop and perform some actions.
I have only one jenkins where everything is running
Use case:
I have a workspace in which I have packages. I need to run some commands in some folders, I can't do it from the root of y workspace. They may be in subfolders or subsubfolders. The way I can identify a package is when it contains a package.xml (on ROS). Also I don't have any command to list their paths
If nothing else is working then you can try running a normal linux command like:
folders = sh(
script: "locate myfile",
returnStdout: true
)
Then split this to form an array and use the value like :
folders.split("\n")[1]
def packageDirs = findFiles(glob: '**/package.xml')
.findAll { f -> !f.directory }
.collect{ f -> f.path.replace('/', '\\') - ~/\\[^\\]+$/ }
packageDir.each { d ->
dir(d) {
// Process each package here
}
}
I have a Jenkins declarative pipeline I have been running on the Jenkins master and it works fine. However, now that I have moved to trying to execute this on a slave node, the groovy scripts which are called in the pipeline can not access the files in the workspace.
My jenkinsfile looks like this...
pipeline {
agent {
label {
label "windows"
customWorkspace "WS-${env.BRANCH_NAME}"
}
}
stages {
stage('InitialSetup') {
steps {
"${env.WORKSPACE}/JenkinsScripts/myScript.groovy"
}
}
}
I can see on the slave that it is creating the workspace, doing the checkout from git, and executing the script correctly. However, if something in the script try's to interact with the files in the workspace it fails.
If I have something simple like this...
def updateFile(String filename) {
echo env.NODE_NAME
filename = "${env.WORKSPACE}/path/to/file"
def myFile = new File(filename)
<do other things with the file>
}
...it says it can not find the file specified. It gives me the path it is looking for and I can confirm the file exists, and that the code runs when just building on the master.
Why can the script not find the files this way when in can just running on the master node? I added the "echo env.NODE_NAME" command into my groovy file and it says the script is executing on the correct node.
Thanks.
Turns out Groovy File commands are considered insecure, and although they will run on the master, they will not run on the slave. If you call them from a script that has the agent set to another node, it will still execute the command just fine, just on the master node, not the agent. Here's an excerpt of an article post https://support.cloudbees.com/hc/en-us/articles/230922508-Pipeline-Files-manipulation
The operation with File class are run on master, so only works if build is run on master, in this example I create a file and check if I can access it on a node with method exists, it does not exist because the new File(file) is executed on master, to check this I search for folder Users that exist on my master but not in the node.
stage 'file move wrong way'
//it only works on master
node('slave') {
def ws = pwd()
def context = ws + "/testArtifact"
def file = ws + '/file'
sh 'touch ' + file
sh 'ls ' + ws
echo 'File on node : ' + new File(file).exists()
echo 'Users : ' + new File('/Users').exists()
sh 'mv ' + file + ' ' + context
sh 'ls ' + ws
}
To execute file manipulation command we recommend to use native commands.
This is a simple example of operations in shell
stage 'Create file'
sh 'touch test.txt'
stage 'download file'
def out='$(pwd)/download/maven.tgz'
sh 'mkdir -p ./download'
sh 'curl -L http://ftp.cixug.es/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -o ' + out
stage 'move/rename'
def newName = 'mvn.tgz'
sh 'mkdir -p $(pwd)/other'
sh 'mv ' + out + ' ' + newName
sh 'cp ' + newName + ' ' + out
}
I run into this same issue recently. I had a python file that runs and writes the results to a JSON file. I was trying to access the JSON file to retrieve the data from there. Here is the code I was using inside a stage block of a declarative pipeline:
script {
def jsonSlurper = new JsonSlurper()
def fileParsed = new File("parameters.json")
def dataJSON = jsonSlurper.parse(fileParsed)
}
As everyone stated already, the above was failing with FileNotFoundException because anything inside script{} will only run on master and not the agent.
To work around the issue, I have used the Pipeline Utility Steps plugin (reference: https://plugins.jenkins.io/pipeline-utility-steps/ -- How to use: https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#writejson-write-json-to-a-file-in-the-workspace)
The plugin will allow you to do any read/write operation on multiple file formats.
Here is an example of the code I used after installing the plugin:
script {
def props = readJSON file: 'parameters.json'
println("just read it..")
println(props)
}
Note: I was using jenkins 2.249.1
I have implemented the code which automatically installs Groovy on slave (for scripted pipeline). Perhaps this solution is a little bit cumbersome, but pipelines don't offer any other way to achieve the same functionality as "Execute Groovy Script" stuff from the old Jenkins, because the plugin https://wiki.jenkins.io/display/JENKINS/Groovy+plugin is not supported yet for pipeline.
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.tools.ToolDescriptor;
import hudson.tools.ToolInstallation;
import hudson.tools.ToolInstaller;
import hudson.util.DescribableList;
import hudson.plugins.groovy.GroovyInstaller;
import hudson.plugins.groovy.GroovyInstallation;
/*
Installs Groovy on the node.
The idea was taken from: https://devops.lv/2016/12/05/jenkins-groovy-auto-installer/
and https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/configMavenAutoInstaller.groovy
COMMENT 1: If we use this code directly (not as a separate method) then we get
java.io.NotSerializableException: hudson.plugins.groovy.GroovyInstaller
COMMENT 2: For some reason inst.getExecutable(channel) returns null. I use inst.forNode(node, null).getExecutable(channel) instead.
TODO: Check if https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.step.MultiJobStepContext.groovyCommand
works better.
*/
#NonCPS
def installGroovyOnSlave(String version) {
if ((version == null) || (version == "")) {
version = "2.4.7" // some default should be
}
/* Set up properties for our new Groovy installation */
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def proplist = new DescribableList<ToolProperty<?>, ToolPropertyDescriptor>()
def installers = new ArrayList<GroovyInstaller>()
def autoInstaller = new GroovyInstaller(version)
installers.add(autoInstaller)
def InstallSourceProperty isp = new InstallSourceProperty(installers)
proplist.add(isp)
def inst = new GroovyInstallation("Groovy", "", proplist)
/* Download and install */
autoInstaller.performInstallation(inst, node, null)
/* Define and add our Groovy installation to Jenkins */
def descriptor = Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy")
descriptor.setInstallations(inst)
descriptor.save()
/* Output the current Groovy installation's path, to verify that it is ready for use */
def groovyInstPath = getGroovyExecutable(version)
println("Groovy " + version + " is installed in the node " + node.getDisplayName())
}
/* Returns the groovy executable path on the current node
If version is specified tries to find the specified version of groovy,
otherwise returns the first groovy installation that was found.
*/
#NonCPS
def getGroovyExecutable(String version=null) {
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def channel = node.getComputer().getChannel()
for (ToolInstallation tInstallation : Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy").getInstallations()) {
if (tInstallation instanceof GroovyInstallation) {
if ((version == null) || (version == "")) {
// any version is appropriate for us
return tInstallation.forNode(node, null).getExecutable(channel)
}
// otherwise check for version
for (ToolProperty prop in tInstallation.getProperties()) {
if (prop instanceof InstallSourceProperty) {
for (ToolInstaller tInstaller: prop.installers) {
if (
(tInstaller instanceof GroovyInstaller) &&
(tInstaller.id.equals(version))
)
return tInstallation.forNode(node, null).getExecutable(channel)
}
}
}
}
}
return null
}
/* Wrapper function. Returns the groovy executable path as getGroovyExecutable()
but additionally tries to install if the groovy installation was not found.
*/
def getGroovy(String version=null) {
def installedGroovy = getGroovyExecutable(version)
if (installedGroovy != null) {
return installedGroovy
} else {
installGroovyOnSlave(version)
}
return getGroovyExecutable(version)
}
Just put these 3 methods to your pipeline script and you will be able to get the Groovy executable path with the help of the method getGroovy(). If it is not installed yet then the installation will be done automatically. You can test this code with the simple pipeline, like this:
// Main
parallel(
'Unix' : {
node ('build-unix') {
sh(getGroovy() + ' --version')
}
},
'Windows' : {
node ('build-win') {
bat(getGroovy() + ' --version')
}
}
)
For me the output was:
[build-unix] Groovy Version: 2.4.7 JVM: 1.8.0_222 Vendor: Private Build OS: Linux
[build-win] Groovy Version: 2.4.7 JVM: 11.0.1 Vendor: Oracle Corporation OS: Windows 10
To work with files on the slave workspace use the readFile, writeFile, findFiles etc steps.
Or if they are large as FloatingCoder said use native tooling; which may be running a groovy script.
A workaround could be load the library via sh command in Jenkinsfile.
So, if you use in Jenkinsfile:
sh 'groovy libraryName.groovy'
You can load the lib locally and in this way you can store File on the slave node.
Even without pipelines, there is no option to restrict a job based on slave agent label. So, I think, pipelines are only for master node execution.
Starting from release 2.4 of the Groovy plugin there is withGroovy step available which sets up the environment on the agent so that you can do sh 'groovy yourscript.groovy' with expected environments. It also enables limited interaction between Pipeline and groovy script.
See https://www.jenkins.io/doc/pipeline/steps/groovy/ for some details about the step.
I am using the grails-cdn-asset-pipline plugin. I've gone through the installation and configuration steps on GitHub and I reach the usage section which says
Add this command to your build process (usually before war generation and deployment).
// If all the settings are defined in your Config.groovy
grails asset-cdn-push
// Or
grails asset-cdn-push --provider=S3 --directory=my-bucket --gzip=true --storage-path=some-prefix --expires=365 --region=eu-west-1 --access-key=$MY_S3_ACCESS_KEY --secret-key=$MY_S3_SECRET_KEY
Where in my project do I put this command?
Is it something that I can do within the context of my project, or do I need to keep it separate in another build process and run it in an environment like Jenkins?
In _Events.groovy, I tried to invoke the script in the eventCreateWarStart, but I am having no luck there. (Code taken from this question)
eventCreateWarStart = { warName, stagingDir ->
def pluginManager = PluginManagerHolder.pluginManager
def plugin = pluginManager.getGrailsPlugin("cdn-asset-pipline")
def pluginDir = plugin.descriptor.file.parentFile
Map<String, String> env = System.getenv()
final processBuilder = new ProcessBuilder()
processBuilder.directory(new File("${cdnAssetPipelinePluginDir}/scripts"))
processBuilder.command([env['GRAILS_HOME']+"/bin/grails","cdn-asset-push"])
println processBuilder.directory()
Process proc = processBuilder.start()
proc.consumeProcessOutput(out, err)
proc.waitFor()
}
This link explains the run-script functionality which was merged into Grails 1.3.6. But I ran into the same problem of not knowing where to run it automatically.
Imagine that we have a grails plugin (that we're developing) and also a test app that we're using to test the grails plugin. I'm trying to test the _Install.groovy script of the plugin, but it's quite a process each time.
Can anyone please recommend way using gant or bash to automate the process of:
Increment the version of the plugin
Package the plugin
Check if the plugin is already installed in the test app and if it is, remove it.
Re-install the plugin.
Much appreciated,
The answer for me was to create a groovy/GANT script and execute it via the command line.
In my plugin, I have a scripts directory and in there I created a script called Deploy.groovy.
Running the command
grails deploy
Automatically executes this script. The script packages the plugin using built in grails targets, then executes grails commands via the commandline.
plugin-home/scripts/Deploy.groovy
includeTargets << grailsScript("PackagePlugin_")
includeTool << gant.tools.Execute
target(main: "This is the script that tries to test what's going on...") {
println ("1. Package the plugin")
packagePlugin()
println ("2. Confirm the directory")
execute.shell("cd ~/quirk-projects/admin-cms-plugin-test && pwd")
println ("3. Remove the plugin ")
execute.shell("cd ~/quirk-projects/admin-cms-plugin-test && grails uninstall-plugin grails-admin-cms-plugin")
println ("4. Install the plugin ")
execute.shell("cd ~/quirk-projects/admin-cms-plugin-test && grails install-plugin ../admin-cms-plugin/grails-admin-cms-plugin-0.1.zip")
println ("5. Run Application")
execute.shell("cd ~/quirk-projects/admin-cms-plugin-test && grails run-app")
println ("6. Your plugin is ready for testing...")
}
setDefaultTarget(main)
To increment, I wrote a script that simply edits the source file and increments the version number:
plugin-home/scripts/_Events.groovy
eventCompileStart = { kind ->
println("Incrementing Version Number for next time")
def version = metadata.'version'
if (!version) {
version = '1'
} else {
version = version.toInteger() + 1
}
metadata.'version' = version.toString()
metadata.persist()
def file = new File("${basedir}/AdminCmsPluginGrailsPlugin.groovy")
def script = file.text
def array = script.split("\n")
for (int i = 0 ; i < array.length; i++) {
if (array[i].indexOf("def version") > -1) {
array[i] = " def version = 0." + version
}
}
def newScript = array.join(System.getProperty("line.separator"))
file.text = newScript
}
I recently updated the configuration of one of my hudson builds. The build history is out of sync. Is there a way to clear my build history?
Please and thank you
Use the script console (Manage Jenkins > Script Console) and something like this script to bulk delete a job's build history https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/bulkDeleteBuilds.groovy
That script assumes you want to only delete a range of builds. To delete all builds for a given job, use this (tested):
// change this variable to match the name of the job whose builds you want to delete
def jobName = "Your Job Name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
// uncomment these lines to reset the build number to 1:
//job.nextBuildNumber = 1
//job.save()
This answer is for Jenkins
Go to your Jenkins home page → Manage Jenkins → Script Console
Run the following script there. Change copy_folder to your project name
Code:
def jobName = "copy_folder"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
My post
If you click Manage Hudson / Reload Configuration From Disk, Hudson will reload all the build history data.
If the data on disk is messed up, you'll need to go to your %HUDSON_HOME%\jobs\<projectname> directory and restore the build directories as they're supposed to be. Then reload config data.
If you're simply asking how to remove all build history, you can just delete the builds one by one via the UI if there are just a few, or go to the %HUDSON_HOME%\jobs\<projectname> directory and delete all the subdirectories there -- they correspond to the builds.
Afterwards restart the service for the changes to take effect.
Here is another option: delete the builds with cURL.
$ curl -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The above deletes build #1 to #56 for job myJob.
If authentication is enabled on the Jenkins instance, a user name and API token must be provided like this:
$ curl -u userName:apiToken -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The API token must be fetched from the /me/configure page in Jenkins. Just click on the "Show API Token..." button to display both the user name and the API token.
Edit: one might have to replace doDeleteAll by doDelete in the URLs above to make this work, depending on the configuration or the version of Jenkins used.
Here is how to delete ALL BUILDS FOR ALL JOBS...... using the Jenkins Scripting.
def jobs = Jenkins.instance.projects.collect { it }
jobs.each { job -> job.getBuilds().each { it.delete() }}
You could modify the project configuration temporarily to save only the last 1 build, reload the configuration (which should trash the old builds), then change the configuration setting again to your desired value.
If you want to clear the build history of MultiBranchProject (e.g. pipeline),
go to your Jenkins home page → Manage Jenkins → Script Console and run the following script:
def projectName = "ProjectName"
def project = Jenkins.instance.getItem(projectName)
def jobs = project.getItems().each {
def job = it
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
}
This one is the best option available.
Jenkins.instance.getAllItems(AbstractProject.class).each {it -> Jenkins.instance.getItemByFullName(it.fullName).builds.findAll { it.number > 0 }.each { it.delete() } }
This code will delete all Jenkins Job build history.
Using Script Console.
In case the jobs are grouped it's possible to either give it a full name with forward slashes:
getItemByFullName("folder_name/job_name")
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
or traverse the hierarchy like this:
def folder = Jenkins.instance.getItem("folder_name")
def job = folder.getItem("job_name")
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
Deleting directly from file system is not safe. You can run the below script to delete all builds from all jobs ( recursively ).
def numberOfBuildsToKeep = 10
Jenkins.instance.getAllItems(AbstractItem.class).each {
if( it.class.toString() != "class com.cloudbees.hudson.plugins.folder.Folder" && it.class.toString() != "class org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject") {
println it.name
builds = it.getBuilds()
for(int i = numberOfBuildsToKeep; i < builds.size(); i++) {
builds.get(i).delete()
println "Deleted" + builds.get(i)
}
}
}
Go to "Manage Jenkins" > "Script Console"
Run below:
def jobName = "build_name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.save()
Another easy way to clean builds is by adding the Discard Old Plugin at the end of your jobs. Set a maximum number of builds to save and then run the job again:
https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Go to the %HUDSON_HOME%\jobs\<projectname> remove builds dir and remove lastStable, lastSuccessful links, and remove nextBuildNumber file.
After doing above steps go to below link from UI
Jenkins-> Manage Jenkins -> Reload Configuration from Disk
It will do as you need
If using the Script Console method then try using the following instead to take into account if jobs are being grouped into folder containers.
def jobName = "Your Job Name"
def job = Jenkins.instance.getItemByFullName(jobName)
or
def jobName = "My Folder/Your Job Name
def job = Jenkins.instance.getItemByFullName(jobName)
Navigate to: %JENKINS_HOME%\jobs\jobName
Open the file "nextBuildNumber" and change the number. After that reload Jenkins configuration. Note: "nextBuildNumber" file contains the next build no that will be used by Jenkins.
Tested on jenkins 2.293 over linux. It will remove all the build logs but not the corellative build number
cd /var/lib/jenkins/jobs
find . -name "builds" -exec rm -rf {} \;
Be careful with this command because it executes a rm -rf on each find result. You could exec this first to validate if the result are only the builds folder of you jobs
find . -name "builds"
If you are looking for a solution where you have job inside a Folder you can use getItemByFullName function. It also supports white space in folder and job name.
def jobName = "folder_name/job_name"
def job = Jenkins.instance.getItemByFullName(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()