I have instances of Jenkins and GitLab which are successfully connected. Part of setting up the GitLab plug-in involves providing an API token. I believe this is used by the plug-in to get details of the commit which is triggering a job.
I need to run a daily Jenkins job which collects some statistics using the API and then (using R) generates a report. So this is not triggered by an event in GitLab.
To complicate issues a little further, the R code will run inside a container (to avoid hving to install R and all the packages on every Jenkins slave),
It does not appear that the plug-in makes the token available to be used in jobs ... and I can see a sane reason for that.
Is it possible to access the API token from the plug-in?
There is a plugin which supports credential injecting (https://wiki.jenkins.io/display/JENKINS/Credentials+Binding+Plugin) to the builds which could be used later as an environment variable. Gitlab api token could be also selected.
Related
I am trying to trigger a build in Jenkins directly from Xray. I have been successfully able to create a trigger in jira and has provided the webhook url and other information needed to run the build. But on triggering the build from any Test Plan, I am getting the following error:
Error publishing web request. Response HTTP status:503
ERROR: The requested URL could not be retrieved.
When i'm hitting the same webhook url in any browser, then the build is getting triggered in jenkins, hence it seems there's no issue with the provided webhook url.
One thing to note is that our Xray is in Jira cloud whereas the Jenkins is running behind a VPN. Can anyone help me in resolving the above issue?
If your Xray on Jira Cloud is trying to acess Jenkins, the Jenkins server needs to have a public URL. If you have it behind the firewall then it's not possible for Xray or any other tool to trigger it directly.
You can call the Jenkins URL from your browser because you are probably with your vpn connected.
The possible workaround would be to use a tool such as ngrok, as mentioned in the docs, to make a tunnel, but beware with the security implications of exposing your Jenkins url to the world.
For reference, Xray cloud provides documentation/examples showcasing how to take advantage of Jira Cloud automation capabilities to trigger jobs, for example in Jenkins.
I am trying to start a VM that already exist in Google cloud with my jenkins to use it as a slave. The reason is because if I start the template of this VM I need to do a few things before I can use my Jenkins code.
Does anyone know how to start VM's that already exist in my VM Pool in Google Could via Jenkins?
There might be 2 approaches to this depending on the operations that you need to run before in your machine that is preventing you from just recreating it.
First and possibly the most straightforward given the restriction that the machine already exists would be talking directly to the GCE API in order to list and start the machine from Jenkins (using a build step).
Basically you can make requests to the GCE API to do operations with your instances. I suggest doing this using gcloud from within the Jenkins master node as it'll save you having to write your own client. It's straightforward as you only have to "install" it in your master and you can make it work safely using a service account.
Below is the outline of this approach:
Download the cloud-sdk to your master node following these release instructions.
You can do this once outside of Jenkins or directly in the build step, doesn't matter as long as Jenkins and its user is able to get the binary.
Create the service account, generate authentication keys and give it permissions to interact with GCE.
Using a service account is the way to go as you can restrict its permissions to the operations that are relevant for you.
Once you get the service account that will be bound to your gcloud client, you'll need to set it up in Jenkins. You might want to do this in a build step (I'm using Groovy here but it should be easy to translate it to the UI):
stage('Start_machine'){
steps{
//Considering that you already installed gcloud in this node, but you can also curl it from here if that's convenient
// You can set this as an scope env var in Jenkins or just hard code it
gcloud config set project ${GOOGLE_PROJECT_ID};
// This needs a json file location accessible by jenkins like: --key-file /var/lib/jenkins/..key.json
gcloud auth activate-service-account --key-file ${GOOGLE_SERVICE_ACCOUNT_KEY};
// Check the reference on this command: https://cloud.google.com/sdk/gcloud/reference/compute/instances/start
gcloud compute instances start my_existing_instance;
echo "Instance started"
}
post{
always{
println "Result : ${currentBuild.result}";
}
}
Wrapping up: You basically create a service account that has the permissions to start your instances. Download an client that can interact with the GCE API (gcloud), authenticate it and start the instance, all from within your pipeline.
The second approach might be easier if there were no constraints regarding the preexisting machine.
Jenkins has a plugin for Compute Engine that will automatically spin up new workers whenever needed.
I know that you need to do some previous operations before Jenkins sends some work to these slave machines. However, I want to bring to your attention that this plugin also considers the start up scripts.
So there's always the option to preload your operations there before the machine takes off and by the time it's ready, you might have everything done.
Hope this helps.
Our jenkins server (version 2.164.2) keeps making lot of requests to Github so it exceeds the Github API rate limit. Github support team found out that a user-agent string git-credential-manager contributed to 60% of the Api calls. Most of the calls are https://api.github.com/user/subscriptions and the agent is
git-credential-manager (Microsoft Windows NT 6.2.9200.0; Win32NT; x64) CLR/4.0.30319 git-tools/1.20.0 597,376 57.032%
From process explorer, it shows that Jenkins trigger lot of git-credential-manager:
We added some Log Recorders in Jenkins for org.jenkinsci.plugins.github and org.jenkinsci.plugins.github_branch_source but they don't make lot of github calls. Is there anyway to figure out which Jenkins plugins are making those calls?
Is there anyway to figure out which Jenkins plugins are making those calls?
I'm not familiar enough with Jenkins to comment on this, but I can tell you a bit more about the behaviour you're seeing as that might help you identify the root cause.
git.exe is invoking git-remote-https.exe to interact with GitHub over HTTPS, which then invokes git-credential-manager as that's the configured helper script to obtain credentials. git-credential-manager is then invoking api.github.com/v3/user/subscriptions to test if the credential it has (likely stored in the Windows Credential Store) is valid.
https://github.com/microsoft/Git-Credential-Manager-for-Windows/blob/1145b59498378a7ddbccdc45b216e17ef0d0a664/GitHub.Authentication/Src/Authority.cs#L190-L208
I don't believe this is tied to a specific plugin, but more to Jenkins working with private repositories and invoking Git on the machine. If you are working with private GitHub repositories, this could be consuming API calls for the account that is authenticating which results in it being rate-limited.
I'd check for these sorts of operations in your build process:
build setups that don't cache the repository on disk between builds (if that's supported)
builds that that include fetch/push/pull steps (more builds, more API calls)
cloning repositories or submodules on-the-fly in scripts
I have an automated Jenkins workflow that runs and tests a java project. I need to get all the data and results that are outputted by Jenkins into Report Portal (RP).
Initially, I was under the impression that you have to install the ReportPortal.io Jenkins plugin to be able to configure Jenkins to communicate with RP.
However, it appears that the plugin will eventually be deprecated.
According to one of the RP devs, there are APIs that can be used, but investigating them on our RP server does not give very clear instructions on what every API does or if it is what is required to get test data from Jenkins to RP.
How then do I get Jenkins to send all generated data to RP?
I am very familiar with Jenkins, but I am extremely new to Report Portal.
ReportPortal is intended for test execution results collection, not for jenkins logs gathering.
In two words, you need to find reporting agent at their github organization which depends on your testing framework (e.g. junit, testng, jbehave) and integrate it into your project.
Here is example for TestNG framework:
https://github.com/reportportal/example-java-TestNG/
How to start Jenkins build remotely with cloudbees authentication enabled?
Please let me know the command to start it from a script or throw commang like wget/curl
You can POST to https://accountname.ci.cloudbees.com/job/jobname/build passing your CloudBees username (typically an email address) and Jenkins API token (see /me/configure) for BASIC authentication.
But if you would rather not keep your full credentials in a script, you can pick an arbitrary token and configure that in the job definition, in which case POSTing to build?token=… without authentication will schedule a build. In the usual case that your Jenkins instance is not visible to anonymous users, this will not work (since the whole URL space is blocked), in which case the Build Token Root plugin may be used to accomplish the same thing.