I'd like to create a pipeline, where one of the parameters is a git repository from a list of available repositories. The list would be similar to the one you see when creating a multibranch pipeline with a bitbucket repository.
Is there a plugin that can do this? Is it hard to make one for myself?
I shoild think the relatively new REST List Parameter would work in these circumstances, as long as the repository host supports the plugin authentication mechanisms. It's kind of an HTTP Request extension for Build with Parameters plugin.
Plugin author (not me) is very responsive to positive feeback and useful enhancement requests.
Related
I am using a parameterized declarative downstream job in Jenkins.
During webhook where I need to pass the git_repo and git_branch as a parameter.
And I have defined this parameter in the Gitlab repository
Example
https://myjenkins.com/job/myjob-builder-downstream/buildWithParameters?token=1qqq1f54ff88e373b3c0&git_repo=git#mygitlab:development/myproduct.git&git_branch=master
During webhook, I don't know how to pass the branch name dynamically to my downstream job?
Thank you in advance for the help.
The major providers(bitbucket, github and gitlab) does not allow us this level of parametrization in the static webhook url registration step:
Bitbucket
Github
Gitlab
So, what is the solution?
These providers offer us an alternative: Webhook post payload interpretation.
How it works?
When github(example) invoke our webhook url, send a body http with a json with a lot of information about the event:
branch name
repository name
username who performed the push event
git commit message
etc
So in the backend of your webhook url, you must parse this json and get your desired values and start your custom logic. Here some samples of these json bodies:
gitlab json payload
github json payload
bitbucket json payload
Unfortunately, jsons are not the same for github, gitlab and bitbucket
Jenkins plugins
You can void this json parse if you use some of the jenkins plugins. One by provider. In you case gitlab-plugin. If you review the source code, you will view the json parse.
easy-webhook-plugin
If you work with several providers or custom plugins does not help you, you could try my generic plugin.
https://github.com/utec/easy-webhook-plugin
How it works?:
Plugin expose a public url similar to your approach or urls of others plugins:
https://myjenkins.com/project/myjob-builder-downstream/buildWithParameters?token=1qqq1f54ff88e373b3c0&git_repo=git#mygitlab:development/myproduct.git&git_branch=master
but with some differences and I think, more clean and easy:
http://my_jenkins.com/easy-webhook-plugin-RDcd4y3LkDcMKKhG/?scmId=gitlab&jobId=hello_word_job
In which you must indicate the scmId (gitlab or bitbucket) and an id of any jenkins job.
When git push is performed, gitlab will send the json to this url, my plugin will parse it and forward some standard parameters to your job:
repositoryName
branchName
authorId
eventMessage
You could access to these parameters with the classic "params" variable in jenkins and do whatever you want!
node {
echo 'New build detected with these incoming parameters: '+params
}
Follow the official readme and or feel free to contact me with an issue
Jenkins gitlab plugin has predefined variables:
https://plugins.jenkins.io/gitlab-plugin/
see Defined variables section. But in order to utilize them, you need to enable This project is parametrized section without adding any variable, this triggers webhook to automatically populate variables like gitlabSourceBranch or gitlabTargetBranch and many more in future builds..
Not sure if it's a bug, but without enabling this once, variables were not populated.
Just to quote as an example one can submit a remote-run with some tool like TeamCity (similar to Jenkins) where it will apply delta/patch on what user is trying to commit & produces result whether changes is good from set-of configured checks for that project.
With Github & Jenkins, can such validation be achieved with any plugins out there?, which will avoid breaking a build?
I know with pull-request & status check one can achieve similar end-result. But without commit/push to remote repo of Git - is there a way Jenkins can handle this validation & produce initial result ??
It isn't possible to have GitHub perform checks on data it doesn't have, so if you don't push the data to the remote server, GitHub won't know anything about it and therefore will do nothing.
Jenkins does have a REST API that you could use to do this, provided you equipped each developer with appropriate credentials. However, this is not a common situation and wouldn't be a recommended configuration.
You'd be better off with a script in the repository that users could install as a hook or invoke from a hook that would perform the testing you want. If your CI jobs run a script in your repository, then sharing code between them should be easy.
Note that you shouldn't mandate pre-commit hooks, since they can interfere with advanced users (who may make intentionally incomplete temporary commits) and they can be disabled by users. Any sort of required checks should be done as part of CI, where policy can be enforced appropriately.
In Jenkins, we want to get the Pipeline stages information through API, e.g. a stage is success or fail. From another answer it seems we can achieve it through Pipeline REST API Plugin.
My question is:
Can Jenkinsapi and Python-Jenkins achieve the same thing? It seems they're designed for bare metal Jenkins, instead of the Pipeline plugin, right ? If that's the case, do we have similar Python library for Pipeline plugin?
Thanks!
You have all the info to answer your own question in the documentation you linked. Just to put it together:
Jenkins has a REST API
The pipeline REST API plugin injects new endpoints in the Jenkins REST API to give you access to several pipeline info
Python-Jenkins and jenkinsapi you mentioned above are both wrappers around the Jenkins REST API to help you develop scripts/apps targeting the Jenkins API in Python. Since those modules/libs are based most probably based on the core API specification, they most probably don't provide specific methods to target the pipeline endpoints (but you can probably extend that quite easily).
If you want to stay in Jenkinsapi, get_data() defined in JenkinsBase class could be used for querying Pipeline REST API directly. Not very sure if it's recommended or not.
Following codes works for me.
from jenkinsapi.jenkins import Jenkins
import requests
requests.packages.urllib3.disable_warnings()
# GET /job/:job-name/:run-id/wfapi/describe
url = 'https://localhost/job/anyjob/10/wfapi/describe'
jenkins = Jenkins(
'https://localhost',
username='username',
password='api_token',
ssl_verify=False,
lazy=True)
print(jenkins.get_data(url))
I'm generating a presigned S3 URL as part of a workflow job, which is passed into a build step that essentially runs outside of the workspace (via ssh). I've been unable to identify an existing publish or archive plugin (workflow compatible or not) that will allow setting an external URL. Is there a plugin or workaround that enables setting a URL as an artifact or the addition of simple metadata to the build results?
I'm a bit surprised that there isn't a way to publish metadata directly from a workflow -- it seems like this would be insanely useful functionally. Have I missed something obvious?
JENKINS-26918 proposes support for some of the features in the Groovy Postbuild plugin. Using core Workflow, you can call
currentBuild.description = "Published"
(or whatever your installation’s markup formatter allows).
We have several repositories on BitBucket. For every repository we
have the issue tracker enabled. For our progress reports I would like
to be able to see all issues from all related repositories in one
overview.
Is this possible and how can it be done?
Thanks,
Paul
Bitbucket has an API. You could quickly put together a script (or webpage using JSONP) calling:
https://api.bitbucket.org/1.0/repositories/USERNAME/REPOSITORY/issues/
for each repository? (Replacing USERNAME and REPOSITORY with appropriate values)
For something more advanced you could query the USER, get all their repositories, then iterate.
I needed the same thing today so I made the following python script available here on github.
The two API call URLs you need are
https://bitbucket.org/api/1.0/user/repositories/
https://bitbucket.org/api/1.0/repositories/{owner}/{slug}/issues/
You'll have to authenticate with your BitBucket credentials which I did using the python requests library:
response = requests.get(url, auth=HTTPBasicAuth(_USERNAME, _PASSWORD))