I am wondering if there is a way in TFS to prohibit new builds from queuing if there are no agents online to run it.
I am asking this because we have a build job that kicks off every 15 minutes. Occasionally our test agents will become disconnected and unavailable. This causes a bunch of build jobs to stack up in the Queue.
Is there a way to tell an automated TFS build definition to not start if all agents are offline? I know we can limit the amount of time one can run before it stops but that only works once its started. If it cant kick off because no agents are available, then this guard will not work and the queue gets backed up.
Sorry, there is no way to do this. When a job add to the queue, the build agent downloads a listener OAuth token and uses it to listen to the job queue.
Periodically, the agent checks to see if a new job request has been posted for it in the job queue in Azure Pipelines/TFS. When a job is available, the agent downloads the job as well as a job-specific OAuth token. This token is generated by Azure Pipelines/TFS for the scoped identity specified in the pipeline. That token is short lived and is used by the agent to access resources (e.g., source code) or modify resources (e.g., upload test results) on Azure Pipelines or TFS within that job.
More details please take a look at agent communication with Azure Pipelines
Related
I am not sure whats the best way to implement multiple parallel calls in Jenkins remotely.. Any inputs will be greatly appreciated.
How to get the build number for the multiple parallel calls (2-10 multiple calls) to the Jenkins Server for a parameteried job if it gets triggered remotely. One requirement is there will be no change in the build parameter. The development team is using tool/python program to invoke 50 POST calls in that case how do we track the build number.
Scenario -- I have a freestyle parameterized job with Enable concurrent build if necessary box checked for this job Sequential call for with same build parameters request when initiated remotely, we see build number using https://jenkinsurl/queue/item with filtering out the build number and then https://jenkins url/build/Consoletext -- This works
Scenario 2 -- Same request with no change with parameters when triggered multiple times more than 2 calls we can see the build number /Consoletext for the first call and later ones were unable to track with the build number.
Sorry, I am a beginner and trying to implement multiple parallel calls. My Jenkins job is configured to run a python script on the Jenkins server that will return success along with work id and other responses that the dev Team needs for further processing.
When the team triggers API remotely 50 times, only for the first call we see the build number/full response from the ConsoleText for the rest of the calls we don't see any build number. I don't see any failures also in Jenkins. FYI This is a free-style parameterized job concurrent build option enabled.
IBM's version of JSR352 provides a Rest API which can be used to trigger jobs, restart them, get the job logs. Can it also be used to get the status of each step and each partition of the step?
I want to build a job monitoring console from where i can trigger the jobs and monitor the status of the steps and partitions in real time without actually having to look into the job log. (after i trigger the job it should periodically give me the status of the step and partitions)
How should i go about doing this?
You can subscribe to our batch events, a JMS topic tree where we publish messages at various stages in the batch job lifecycle, (job started/ended, step checkpointed, etc.)
See the Knowledge Center documentation and this whitepaper as well for more information.
I manage a Jenkins server with a few hundred projects in the whole ecosystem. Many of the projects rely on upstream servers, that, unfortunately, are not always responsive. When I have a lag on these servers, my build queue can get to 10 or more. Is there a plugin or setting to send a warning email when the build queue exceeds a particular length?
I have been unable to find a plugin that does this, but you can query Jenkins for the information as detailed here: Jenkins command to get number of builds in queue.
If you have a Jenkins slave available you could set up a job that runs every 15 minutes and just hit each of the other Jenkins servers with the API call to get build queue counts (this is easy if you have just one master and many slaves.)
If you wanted to stay completely outside of Jenkins (not add another job to the mix) you could write a script to poll the Jenkins API for the information. You could then run that script under, say, a 15 minute (or some other relevant time step) timer using cron (or windows scheduled task). Admittedly then you have to dedicate some resources to running this job.
It looks like you could use python to get the build queue and check the length of the returned list. get_queue_info()
I haven't mucked about with the Jenkins API much myself so I'm not sure offhand exactly what the script would need, but it should be simple enough once you dig into it.
In a scenario where continuous integration is important, for triggering builds which is a better option Webhooks or PollSCM.
These are my current understanding on both methods:
PollSCM is a heavy operation and depending on it to trigger build means we need to fire it frequently. But the configuration is easier and it is safer than web hooks as Jenkins will be communicating to code repo.
Web hooks can give you exact build trigger time without checking for it constantly. But on the other hand, there are security concerns when you are opening up a connection from outside and configuration is not easy compared to PollSCM.
Looking forward to know the exact pros and cons of both ways.
If your build cycle is very short (a few minutes) and if you want to trigger a build for each commit, the Webhooks solution is better.
But if your build cycle is longer (15/20 minutes) and if you don't need to build for each commit, the PollSCM is a good candidate :)
In my company, we are using Git/Stash and Jenkins + a Webhook to trigger a build every time something is committed. For the pull requests, we are using the Stash pullrequest builder plugin for Jenkins.
I have a job A running in Jenkins, which kicks off a process A on a VM, waits for it to finish, picks up the report generated by it and sends it as an attachment to the build notification. The problem is this process A takes too long to finish and job A keeps waiting on it. Is there any way I can start this process A, stop job A and when process A is done, trigger a new job B which would pick up the report generated by process A and sends it out with build sucess/failure status.
Any help is appreciated.
Thanks
Jenkins provides an API for kicking off jobs via simple HTTP requests. You kick off job B using curl or something like that, as the final step in process A on the VM.
The docs are on the Jenkins site. You can use your own Jenkins find the specific URLs for kicking off particular jobs; there's a link in the bottom right hand corner of the Jenkins page.
Perhaps an even better match for your use case would be a job of type "Monitor an external job". I have not used it myself, but from the documentation it sounds like a useful tool. The docs are at: https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs