I am not sure whats the best way to implement multiple parallel calls in Jenkins remotely.. Any inputs will be greatly appreciated.
How to get the build number for the multiple parallel calls (2-10 multiple calls) to the Jenkins Server for a parameteried job if it gets triggered remotely. One requirement is there will be no change in the build parameter. The development team is using tool/python program to invoke 50 POST calls in that case how do we track the build number.
Scenario -- I have a freestyle parameterized job with Enable concurrent build if necessary box checked for this job Sequential call for with same build parameters request when initiated remotely, we see build number using https://jenkinsurl/queue/item with filtering out the build number and then https://jenkins url/build/Consoletext -- This works
Scenario 2 -- Same request with no change with parameters when triggered multiple times more than 2 calls we can see the build number /Consoletext for the first call and later ones were unable to track with the build number.
Sorry, I am a beginner and trying to implement multiple parallel calls. My Jenkins job is configured to run a python script on the Jenkins server that will return success along with work id and other responses that the dev Team needs for further processing.
When the team triggers API remotely 50 times, only for the first call we see the build number/full response from the ConsoleText for the rest of the calls we don't see any build number. I don't see any failures also in Jenkins. FYI This is a free-style parameterized job concurrent build option enabled.
Related
I am using Jenkins integration server for my CI/CD.
I am working with freestyle projects.
I want to get build duration once finished (in seconds) using RESTFUL API (JSON).
This is what i tried:
"duration=$(curl -g -u login:token--silent "$BUILD_URL/api/json?pretty=true&tree=duration" | jq -r '.duration')"
Duration is always equel to 0 even though i did this shell script in a post buil task.
There are a few reasons why this may not work for you, but the most probable reason is that the build hasn't finished when you make the API call.
I tried it on our instance and for finished jobs it works fine and for running jobs it always returns 0. If your post build task is executed as part of the job, then the job has probably not finished executing yet, which is why you are always getting 0.
The Api call for the build will not contain the duration attribute as long as the build is running, and therefore you cannot use that mechanism during the build duration.
However you have a nice alternative for achieving what you want using Freestyle jobs.
The solution, which still uses the Api method, is to create a separate generic job for updating your data base with the results, this jobs will receive as parameters the project name and the build number, run the curl command for receiving the duration, update your database and run any other logic you need.
This job can now be called from any freestyle job using the Parameterized Trigger plugin post task with the relevant build environment parameters.
This has the additional benefit that the duration update mechanism is controlled in a single job and if updates are needed they can be made in a single location avoiding the need to update all separate jobs.
Assuming your job is called Update-Duration and it receives two parameters Project and Build the post trigger can look like the following:
And thats it, just add this tigger to any job that needs it, and in the future you can update the logic without changing the calling jobs.
Small thing, in order to avoid a race condition that can be caused if the caller job has not yet finished the execution you can increase the quite period of your data base updater job to allow enough time for the caller jobs to finish so the duration will be populated.
I have a set of chained freestyle jenkins jobs which we use as a build pipeline for several projects, we recently integrated it with our source repository management (phabricator) so it reports the continuous integration build result back to the merge request (whether if it failed or passed).
Due to some limitations with Phabricator, the way we are triggering the pipeline is through an AWS lambda function which knows what's the first job of the chain and starts it, since it's all chained together, triggering the first job executes the whole pipeline.
The issue is, technically we are triggering a single job (which triggers a downstream job as well and so on), so, if the first job passes, it will return a green build to Phabricator no matter if the second job fails, it won't wait for any of the downstream projects to finish, if the first one passed, it will say it build is green.
As I see it, there are two questions that come to my mind to solve this:
1.- Is there a way to mark the job as failed if the downstream project fails?
2.- Is there a way to trigger the chain instead of a single job? that way I think it will return the result of the chain instead of the single first job.
Any thoughts and advice are welcome.
Have you considered rewriting your Jenkins chain as a single pipeline job? I think this would make your life a lot easier.
Otherwise, you need to pass the Phabricator build ID down through your chain, and only post back a success response in the final job, with a fail response from any job that fails.
I currently have a Freestyle parametrized job, Job A (takes in 5 input parameters) and it is triggered remotely by an API. Whenever a user sends a request to the API with some input paramters, an equivalent build is triggered with a specific set of build parameter values.
Therefore, 1 API request = 1 Job A with 1 set of parameters. One more point to note is that I will never know the job build parameter values beforehand since they coming in from users' API requests.
I have 6 build executors on Jenkins meaning I can run 6 jobs in parallel. However, if the load to the API increases say 200 requests per minute... that means that 194 jobs will be queued while only 6 are executed.
I want to achieve more parallelism i.e. a scenario where multiple parallel and independent instances of the parametrized Job A are running with their respective (different) sets of parameters.
Any help with this is greatly appreciated. Thanks!
I manage a Jenkins server with a few hundred projects in the whole ecosystem. Many of the projects rely on upstream servers, that, unfortunately, are not always responsive. When I have a lag on these servers, my build queue can get to 10 or more. Is there a plugin or setting to send a warning email when the build queue exceeds a particular length?
I have been unable to find a plugin that does this, but you can query Jenkins for the information as detailed here: Jenkins command to get number of builds in queue.
If you have a Jenkins slave available you could set up a job that runs every 15 minutes and just hit each of the other Jenkins servers with the API call to get build queue counts (this is easy if you have just one master and many slaves.)
If you wanted to stay completely outside of Jenkins (not add another job to the mix) you could write a script to poll the Jenkins API for the information. You could then run that script under, say, a 15 minute (or some other relevant time step) timer using cron (or windows scheduled task). Admittedly then you have to dedicate some resources to running this job.
It looks like you could use python to get the build queue and check the length of the returned list. get_queue_info()
I haven't mucked about with the Jenkins API much myself so I'm not sure offhand exactly what the script would need, but it should be simple enough once you dig into it.
I've got two Jenkins jobs, a continuous integration job that runs whenever a new change is submitted, and a build/deploy job that runs everyday at noon. I do not want the builds to run simultaneously, so I am using the Throttle Concurrent Builds plugin to force only these two jobs to queue up if either of them is already running. Any of my other jobs are allowed to run at the same time as these two jobs.
I would like the build/deploy job to always run at noon, but sometimes a developer submits a change just before noon causing the build/deploy job to wait until the CI job finishes.
Is there a way to block only the CI job during a defined time frame, like 11:30am until 1pm, so that the build/deploy job will not be blocked at noon?
IMHO the are few possibilities:
a first pre-step which check the date and exit with error if the date do NOT match the requirements
two jobs which disable and enable the job programmatically calling the REST
eg job 1
curl -X POST http://usr:pwd#host/job/joobname/disable
and job 2
curl -X POST http://usr:pwd#host/job/joobname/enable
at given time.
You can try to add slave node which is offline from 11:30am until 1pm and restrict change-caused build to this.
Second possibility is to enable SCM poll schedule to check for changes during allowed time-frame.