I am using Job DSL in Jenkins. There is a seed job that generates some files that should be shared across other jobs that could run on different nodes. If the files were not generated, the config files provider plugin could be used for this task. However I need the files to be dynamic so that no Jenkins UI interaction is needed.
Is it possible to add a file to the plugin with a groovy script?
The only other option I could think of was to record the UI interaction and let a script replay it with modified data. In case of a more secured Jenkins this would also require to get authentication and CSRF tokens right.
You can use Job DSL to create config files that are managed by the Config File Provider plugin:
configFiles {
customConfig {
id('one')
name('Config 1')
comment('lorem')
content('ipsum')
providerId('???')
}
}
See https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands#config-file
When you are using job-dsl you can read in data from anywhere that the Groovy runtime can access.
You could store shared config in a hard coded variable in your script itself.
You could inject the data via a Jenkins parameter to your seed job.
You could retrieve the data from a file in the git repo where your store your seed job.
You could retrieve the data from a database, REST API.
etc etc.
Related
I am new to Jenkins.
I have written a groovy script, which loads a token secret from an adjacent config.properties file.
What's the best way to execute this script in a groovy pipeline ?
What I think I need to do is:
download that script from SCM
change the token in config.properties (retrieved from the stored jenkins credentials)
execute that script
I've seen various options :
loading the script with load(), but this seems to be more dedicated to loading helper functions, whereas I only need to execute the whole script
using shared libraries, but this seems to be more dedicated to code snippets reused across multiple jobs, which is not my case
using withGroovy {}
using sh groovy
using a docker image that contains a groovy sdk
I tried these, and managed to get none of them to work (in part due to the extra-difficulty of retrieving the credential), so before I go any further I'd like to know what's the best option to proceed. For example, instead of trying to change the file config.properties in my pipeline to set the correct token, should I rather try to change my groovy script so that it takes the token from an environment variable ?
Thanks
I am trying the access the values set on a job's configuration page from within my pipeline. These values are not made available as params, nor are they injected as envvars.
Setup
Jenkins, v2.263.1
GitLab Branch Source plugin, v1.5.3 (link)
Multibranch pipeline job which is pointed to a Gitlab repo
Remote Jenkinsfile Provider, v1.13 (link)
Problem
Ordinarily, one would have a Jenkinsfile in the root of the repo and therefore the scm would be associated with the repo we want to checkout and build. However, in my case the code I want to build is in a different repo to the Jenkinsfile (hence the Remote Jenkinsfile Provider plugin).
This means that I need to checkout the code I wish to build as an explicit step in the pipeline, and to do that I need to know the repo. This repo is, however, already defined in the job config.
The Branch Source plugin does export things like the branch name or merge request number/branch/target into appropriate envvars, but NOT the actual repo.
As this is a multibranch pipeline, I cannot use something like envInject either (multibranch jobs do not provide the option to 'Prepare an environment for the run' as with other jobs)
Goal
I would like to be able to access the server, owner and project fields set in the job config page. Ultimately I could manage with just the project's ssh/http address even.
Is there some clever way of accessing a job's config from within the pipeline?
Thanks for any suggestions!
Reference images
Within the gitlab branch source plugin (and the documentation) you have a lot more information, than just with the normal branch source plugin. there are environment variables for the project like GITLAB_PROJECT_GIT_SSH_URL/GITLAB_PROJECT_GIT_HTTPS_URL for the git source and many more. So far i did not see one for the server, but that would be parse-able our of the URLs.
Within this information, it should be fairly easy to checkout the repository and build it.
As through the process it came clear, that it is needed to also trigger the pipeline manually, and this is normally also possible with variables (not sure about the Remote File plugin). I assume your Jenkinsfile is a groovy script, which opens up a lot of possibilities. You can define variables and use some logic to determine if the env variable or the parameter is used.
pipeline {
parameters {
string(name: 'projectUrl', defaultValue: "")
}
stages {
stage('Prepare') {
steps {
def projectUrl = env.GITLAB_PROJECT_GIT_SSH_URL ?: params.projectUrl
// DO Checkout with projectUrl
}
}
}
}
The only critical thing you have to take into account, is that the multibranch pipeline, has to run once, for each branch or mr - so they detect the variables. Afterwards you can easily trigger it, manually by providing your values.
This allows you, to utilize webhooks for automatic actions, and also allows you to trigger the build manually when ever you like.
Sidenote: if you use the centralized jenkinsfile, for reducing duplication, you might also want to checkout Shared libraries for jenkins.
For completeness, here is a list of all current environment variables added by the jenkins gitlab branch source plugin version 1.5.3 (and only for Push Events - but they are pretty similar in the other event types too)
GITLAB_OBJECT_KIND
GITLAB_AFTER
GITLAB_BEFORE
GITLAB_REF
GITLAB_CHECKOUT_SHA
GITLAB_USER_ID
GITLAB_USER_NAME
GITLAB_USER_EMAIL
GITLAB_PROJECT_ID
GITLAB_PROJECT_ID_2
GITLAB_PROJECT_NAME
GITLAB_PROJECT_DESCRIPTION
GITLAB_PROJECT_WEB_URL
GITLAB_PROJECT_AVATAR_URL
GITLAB_PROJECT_GIT_SSH_URL
GITLAB_PROJECT_GIT_HTTP_URL
GITLAB_PROJECT_NAMESPACE
GITLAB_PROJECT_VISIBILITY_LEVEL
GITLAB_PROJECT_PATH_NAMESPACE
GITLAB_PROJECT_CI_CONFIG_PATH
GITLAB_PROJECT_DEFAULT_BRANCH
GITLAB_PROJECT_HOMEPAGE
GITLAB_PROJECT_URL
GITLAB_PROJECT_SSH_URL
GITLAB_PROJECT_HTTP_URL
GITLAB_REPO_NAME
GITLAB_REPO_URL
GITLAB_REPO_DESCRIPTION
GITLAB_REPO_HOMEPAGE
GITLAB_REPO_GIT_SSH_URL
GITLAB_REPO_GIT_HTTP_URL
GITLAB_REPO_VISIBILITY_LEVEL
GITLAB_COMMIT_COUNT
GITLAB_COMMIT_ID_#
GITLAB_COMMIT_MESSAGE_#
GITLAB_COMMIT_TIMESTAMP_#
GITLAB_COMMIT_URL_#
GITLAB_COMMIT_AUTHOR_AVATAR_URL_#
GITLAB_COMMIT_AUTHOR_CREATED_AT_#
GITLAB_COMMIT_AUTHOR_EMAIL_#
GITLAB_COMMIT_AUTHOR_ID_#
GITLAB_COMMIT_AUTHOR_NAME_#
GITLAB_COMMIT_AUTHOR_STATE_#
GITLAB_COMMIT_AUTHOR_USERNAME_#
GITLAB_COMMIT_AUTHOR_WEB_URL_#
GITLAB_COMMIT_ADDED_#
GITLAB_COMMIT_MODIFIED_#
GITLAB_COMMIT_REMOVED_#
GITLAB_REQUEST_URL
GITLAB_REQUEST_STRING
GITLAB_REQUEST_TOKEN
GITLAB_REFS_HEAD
I try to configure Jenkins' seed job, where whole business is in provided DSL script. I want to seperate that script from its configuration, which I want to locate in additional yml file. When I try to read that file:
#Grab('org.yaml:snakeyaml:1.17')
import org.yaml.snakeyaml.Yaml
def workDir = SEED_JOB.getWorkspace()
def config = new Yaml().load(("${workDir}/config.yml" as File).text)
I receive error
java.io.FileNotFoundException: /var/lib/jenkins/workspace/test.dsl/config.yml (No such file or directory)
I suppose that Jenkins is looking for the file on a master host, not an agent node where workspace is located.
Is it possible to read yml file in DSL build step on the agent node? Or maybe I have to execute that seed job always on my master host?
This seems not possible as the jobDsl script is executed on master. You can try force to run the job on master with label master.
From the documentation in section Script location:
Job DSL scripts are executed on the Jenkins master node, but the seed job's workspace which contains the script files may reside on a build node. This mean that direct access to the file specified by FILE may not be possible from a DSL script. See Distributed builds for details.
I want to setup a Jenkins from code to
Create one initial pipeline
Create the Job DSL seed job and executing it to configure jobs used in the pipeline
Configure Jenkins settings
Locales - set locale to EN
Access control - Lock down system
I read many tutorials and questions and found the following ideas
Using the Jenkins CLI
Some Job DSL interface for setting up a job as described here at the bottom
Using JenkinsSCI interface within a Groovy file located in init.groovy.d - see below
For testing I use Docker and have the following sample already running.
Dockerfile
# https://github.com/jenkinsci/docker/blob/master/README.md
FROM jenkins/jenkins:lts
USER root
COPY groovy/* /usr/share/jenkins/ref/init.groovy.d/
USER jenkins
EXPOSE 8080
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
groovy/jobs/test1-basic.groovy
#!/usr/bin/env groovy
import hudson.model.*
import jenkins.model.Jenkins;
import hudson.tasks.Shell;
job = Jenkins.instance.createProject(FreeStyleProject, 'test1-basic')
job.buildersList.add(new Shell('echo hello world'))
job.save()
The sample sadly lacks the
configuration part, as I do not know how to access the locale plugin from within the groovy code
Job DSL integration, how to read the seed job and execute it ones
I really did an intensive research and could not find much about this initial setup part. It seems many people do this manually, or the legacy way copying XML files. Could you help me out solving this and making it a "best practice documentation" for other?
If you are familiar with configuration management tool like chef you can use it for configuring your jenkins instance. There is a jenkins community cookbook which can be utilized to write a wrapper to suite your needs.
jenkins_job resource in this cookbook lets you create any type of job be it pipeline, free style etc, you just need to supply the required job configuration. You can template this with variables so based on what you supplied, job will be created accordingly. Not just jobs, you can configure almost everything you do manually with chef using a resource corresponding to that.
One of the best part about using chef is you can source control it and update configuration based on requirements at any point of time.
If you are not planning to use a configuration management tool, you can check out the discussion here on how to achieve job creation with plugins
I have some groovy code which I am planning to re-use in Jenkins Groovy Post Build plugin of multiple jobs. How can I achieve this? Is there a place I can store the script in a global variable and call that in the jobs where ever I need?
You can load any groovy file living on the Jenkins master within the groovy postbuild and execute it. For example, you could have a special directory on the c drive where all the common scripts live. I'll update my answer later with some code that shows you how to load the script in.
Update
Assuming you have a test.groovy file on your C: drive, it should be as simple as the following in Groovy Postbuild:
evaluate(new File("C:\\test.groovy"))
Please view the comment section of the Groovy Postbuild for more examples and possibly other ways.
Here is the solution that worked for me:
Installed Scriptler plugin for Jenkins and saved the Groovy script in that. Now the script is available in JENKINS_HOME/scriptler/scripts directory. This way we can avoid manual step of copying files to Jenkins master.
Used the groovy file in Post build:
def env = manager.build.getEnvironment(manager.listener) evaluate(new File(env['JENKINS_HOME'] + "\\scriptler\\scripts\\GroovyForPostBuild.groovy"))
This is a copy of my answer to this similar question on StackOverflow:
If you wish to have the Groovy script in your Code Repository, and loaded onto the Build / Test Slave in the workspace, then you need to be aware that Groovy Postbuild runs on the Master.
For us, the master is a Unix Server, while the Build/Test Slaves are Windows PCs on the local network. As a result, prior to using the script, we must open a channel from the master to the Slave, and use a FilePath to the file.
The following worked for us:
// Get an Instance of the Build object, and from there
// the channel from the Master to the Workspace
build = Thread.currentThread().executable
channel = build.workspace.channel;
// Open a FilePath to the script
fp = new FilePath(channel, build.workspace.toString() + "<relative path to the script in Unix notation>")
// Some have suggested that the "Not NULL" check is redundant
// I've kept it for completeness
if(fp != null)
{
// 'Evaluate' requires a string, so read the file contents to a String
script = fp.readToString();
// Execute the script
evaluate(script);
}