I create two labels in one of our projects that requires builds on Windows and Linux, so the project.config for that project now looks as follows
[label "Verified"]
function = NoBlock
[label "Verified-Windows"]
function = MaxWithBlock
value = -1 Fails
value = 0 No score
value = +1 Verified
[label "Verified-Unix"]
function = MaxWithBlock
value = -1 Fails
value = 0 No score
value = +1 Verified
This works as intended. Submits require that one succesful build reports verified-windows and the other one verified-linux [1].
However, the two builds are now triggered by the same gerrit event (from 'different' servers, see note), but when they report back only one of the two labels 'survives'.
It seems as though, the plugin collates the two messages that arrive into one comment and only accepts whichever label was the first one to be set.
Is this by design or a bug? Can I work around this?
This is using the older version of the trigger: 2.11.1
[1] I got this to work by adding more than one server and then reconfiguring the messages that are sent back via SSH to gerrit. This is cumbersome and quite non-trivial. I think jobs should be able to override the label that a succesful build will set on gerrit.
This can be adressed by using more than one user name, so the verdicts on labels don't get mixed up. However this is only partially satisfactory, since multiple server connections for the same server also duplicate events from the event stream.
I am currently working on a patch for the gerrit trigger plugin for jenkins to address this issue and and make using different labels more efficient.
Maybe you can solve this challenge by using a post build groovy script.
I provided an example at another topic: https://stackoverflow.com/a/32825278
To be more specific as mentioned by arman1991
Install the Groovy Postbuild Plugin:
https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Postbuild+Plugin
Use the following example script as PostBuild action in each of your jobs. Modify it to your needs for the Linux verification.
It will do for you:
collect necessary environment variables and status of the job
build feedback message
build ssh command
execute ssh command -> send feedback to gerrit
//Collect all environment variables of the current build job
def env = manager.build.getEnvironment(manager.listener)
//Get Gerrit Change Number
def change = env['GERRIT_CHANGE_NUMBER']
//Get Gerrit Patch Number
def patch = env['GERRIT_PATCHSET_NUMBER']
//Get Url to current job
def buildUrl = env['BUILD_URL']
//Build Url to console output
def buildConsoleUrl = buildUrl + "/console"
//Verification will set to succeded (+1) and feedback message will be generated...
def result = +1
def message = "\\\"Verification for Windows succeeded - ${buildUrl}\\\""
//...except job failed (-1)...
if (manager.build.result.isWorseThan(hudson.model.Result.SUCCESS)){
result = -1
message = "\\\"Verification for Windows failed - ${buildUrl}\\\""
}
//...or job is aborted
if (manager.build.result == hudson.model.Result.ABORTED){
result = 0
message = "\\\"Verification for Windows aborted - ${buildConsoleUrl}\\\""
}
//Send Feedback to Gerrit via ssh
//-i - Path to private ssh key
def ssh_message = "ssh -i /path/to/jenkins/.ssh/key -p 29418 user#gerrit-host gerrit review ${change},${patch} --label=verified-Windows=${result} --message=${message}"
manager.listener.logger.println(new ProcessBuilder('bash','-c',"${ssh_message}").redirectErrorStream(true).start().text)
I hope this will help you to solve your challenge without using the Gerrit Trigger Plugin to report the results.
Related
First off the setup in question:
A Jenkins Instance with several build nodes and on prem Azure-Devops server containing the Git Repositories.
The Repo in question is too large to always build on push for all branches and all devs, so a small workaround was done:
The production branches have a polling enabled twice a day (because of testing duration which is handled downstream more builds would not help with quality)
All other branches have their automated building suppressed. They still can start it manually for Builds/Deployments/Unittests if they so choose.
The jenkinsfile has parameterization for which platforms to build, on prod* all the platforms are true, on all other branches false.
This helps because else the initial build of a feature branch would always build/deploy locally all platforms which would take too much of a load on the server infrastructure.
I added a service endpoint for Jenkins in the Azure Devops, added a Buildvalidation .yml - this basically works because when I call the sourcebranch of the pull request with the merge commitID i added a parameter
isPullRequestBuild which contains the ID of the PR.
snippet of the yml:
- task: JenkinsQueueJob#2
inputs:
serverEndpoint: 'MyServerEndpoint'
jobName: 'MyJob'
isMultibranchJob: true
captureConsole: true
capturePipeline: true
isParameterizedJob: true
multibranchPipelineBranch: $(System.PullRequest.SourceBranch)
jobParameters: |
stepsToPerform=Build
runUnittest=true
pullRequestID=$(System.PullRequest.PullRequestId)
Snippet of the Jenkinsfile:
def isPullRequest = false
if ( params.pullRequestID?.trim() )
{
isPullRequest = true
//do stuff to change how the pipeline should react.
}
In the jenkinsfile I look whether the parameter is not empty and reset the platforms to build to basically all and to run the unittests.
The problem is: if the branch has never run, Jenkins does not already know the parameter in the first run, so it is ignored, building nothing, and returning with 0 because "nothing had to be done".
Is there any way to only run the jenkins build if it hasnt run already?
Or is it possible to get information from the remote call if this was the build with ID 1?
The only other thing would be to Call the Jenkins via web api and check for the last successful build, but in that case I would have have the token somewhere stored in source control.
Am I missing something obvious here? I dont want to trigger the feature branch builds to do nothing more than once, because Devs could lose useful information about their started builds/deployments.
Any ideas appreciated
To whom it may concern with similar problems:
In the end I used the following workaround:
The Jenkins Endpoint is called via a user that only is used for automated builds. So, in case that this user triggered the build, I set everything to run a Pull Request Validation, even if it is the first build. Along the lines of
def causes = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
if (causes != null)
{
def buildCauses= readJSON text: currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').toString()
buildCauses.each
{
buildCause ->
if (buildCause['userId'] == "theNameOfMyBuildUser")
{
triggeredByAzureDevops = true
}
}
}
getBuildcauses must be allowed to run by a Jenkins Admin for that to work.
I want to call my rest API from Jenkins after build of job was ended with build status. As example when build was fail I want to send what are the recent changes has done, who start the build etc. I think there might be a specific plugin for doing this but I could not found. My next plane is to customize Jenkins Hipchat plugin to sent massage to my API. But it may difficult. Can anyone suggest me a plugin or method for do this.
The standard environment variables are listed here:
https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project
If you need additional parameters you have to query them yourself. I usually do it with groovy, and in you case I guess it would be lastBuild()
Here is an example to check if job has run on master or not:
import hudson.model.*
jenkins = Hudson.instance
items = jenkins.items
Collections.shuffle(items)
for (item in items){
def job = item.getLastBuild()
if(job != null){
if(job.getBuiltOnStr() == null || job.getBuiltOnStr() == "")
println "Built in master: " + job.toString()
}
}
Please check api documentation to determine what information you need: http://javadoc.jenkins-ci.org/hudson/model/AbstractBuild.html
I guess the user is is a build-cause: http://javadoc.jenkins-ci.org/hudson/model/Run.html#getCauses()
But you have to dig around yourself to get the information you need. If this is too much job, and you have everything you need in the hipchat plugin, that might be simpler for you.
I would like to check for or get notifications about SCM poll failures in Jenkins (for example, when the repository URL had changed, or branch got deleted). I thought about these:
a) A Jenkins console script, which would list such faulty jobs
b) Configuring/installing plugin for Jenkins to notify me somehow about that fact (e-mail, anything)
c) External script/executable (bash, python, ...), which would list builds which failed in last X hours due to SCM poll failure
As you mentioned in your question, one way to tackle this problem is by using a script. For example, Groovy Postbuild.
Since Groovy Postbuild scripts run on the master, you can access each job's scm-polling.log found on the file system using standard IO functions.
For example, assuming a Windows master, here is some (untested) pseudocode to give you some ideas:
def error = false;
def jobsDirectory = new File("C:\\Jenkins\\jobs");
jobsDirectory.eachFile {
def pollingLog = new File(it.path + "\\scm-polling.log");
if(pollingLog.text =~ "ERROR")
{
manager.listener.logger.println(it.path + " has polling errors.");
error = true;
}
}
if(error) {
manager.build.buildFailure();
}
Once you have marked the build as failure, you can use the standard email functionality of Jenkins to send an email or format it to look nice using the Email-ext plugin.
When you set up a Jenkins job various test result plugins will show regressions if the latest build is worse than the previous one.
We have many jobs for many projects on our Jenkins and we wanted to avoid having a 'job per branch' set up. So currently we are using a parameterized build to build eg different development branches using a single job.
But that means when I build a new branch any regressions are measured against the previous build, which may be for a different branch. What I really want is to measure regressions in a feature branch against the latest build of the master branch.
I thought we should probably set up a separate 'master' build alongside the parameterized 'branches' build. But I still can't see how I would compare results between jobs. Is there any plugin that can help?
UPDATE
I have started experimenting in the Script Console to see if I could write a post-build script... I have managed to get the latest build of master branch in my parameterized job... I can't work out how to get to the test results from the build object though.
The data I need is available in JSON at
http://<jenkins server>/job/<job name>/<build number>/testReport/api/json?pretty=true
...if I could just get at this data structure it would be great!
I tried using JsonSlurper to load the json via HTTP but I get 403, I guess because my script has no auth session.
I guess I could load the xml test results from disk and parse them in my script, it just seems a bit stupid when Jenkins has already done this.
I eventually managed to achieve everything I wanted, using a Groovy script in the Groovy Postbuild Plugin
I did a lot of exploring using the script console http://<jenkins>/script and also the Jenkins API class docs are handy.
Everyone's use is going to be a bit different as you have to dig down into the build plugins to get the info you need, but here's some bits of my code which may help.
First get the build you want:
def getProject(projectName) {
// in a postbuild action use `manager.hudson`
// in the script web console use `Jenkins.instance`
def project = manager.hudson.getItemByFullName(projectName)
if (!project) {
throw new RuntimeException("Project not found: $projectName")
}
project
}
// CloudBees folder plugin is supported, you can use natural paths:
project = getProject('MyFolder/TestJob')
build = project.getLastCompletedBuild()
The main test results (jUnit etc) seem to be available directly on the build as:
result = build.getTestResultAction()
// eg
failedTestNames = result.getFailedTests().collect{ test ->
test.getFullName()
}
To get the more specialised results from eg Violations plugin or Cobertura code coverage you have to look for a specific build action.
// have a look what's available:
build.getActions()
You'll see a list of stuff like:
[hudson.plugins.git.GitTagAction#2b4b8a1c,
hudson.scm.SCMRevisionState$None#40d6dce2,
hudson.tasks.junit.TestResultAction#39c99826,
jenkins.plugins.show_build_parameters.ShowParametersBuildAction#4291d1a5]
These are instances, the part in front of the # sign is the class name so I used that to make this method for getting a specific action:
def final VIOLATIONS_ACTION = hudson.plugins.violations.ViolationsBuildAction
def final COVERAGE_ACTION = hudson.plugins.cobertura.CoberturaBuildAction
def getAction(build, actionCls) {
def action = build.getActions().findResult { act ->
actionCls.isInstance(act) ? act : null
}
if (!action) {
throw new RuntimeException("Action not found in ${build.getFullDisplayName()}: ${actionCls.getSimpleName()}")
}
action
}
violations = getAction(build, VIOLATIONS_ACTION)
// you have to explore a bit more to find what you're interested in:
pylint_count = violations?.getReport()?.getViolations()?."pylint"
coverage = getAction(build, COVERAGE_ACTION)?.getResults()
// if you println it looks like a map but it's really an Enum of Ratio objects
// convert to something nicer to work with:
coverage_map = coverage.collectEntries { key, val -> [key.name(), val.getPercentageFloat()] }
With these building blocks I was able to put together a post-build script which compared the results for two 'unrelated' build jobs, then using the Groovy Postbuild plugin's helper methods to set the build status.
Hope this helps someone else.
The product I work on comprises 3/4 seperate (non-dependant) TFS builds.
I would like to create a single TFS build which queues the other 3/4 builds from within the ProcessTemplate AND, critically, pass process parameters to them. This build would wait for them all to complete and return an overall success/failure of the build.
So my questions are:
Can this be achieved by any existing 'standard' Workflow activities (my manager has had bad experiences with custom workflow activities)?
If not, I am able to 'shell out' to powershell. Can I achieve what I want from within Powershell (accessing the API)?
Maybe using TFSBuild.exe? But I can't find a way of passing the custom process parameters I need.
Any assistance or guidance would be appreciated.
UPDATE
The following powershell script will execute the build, but I'm still at a loss to be able to pass my custom process parameters :-(
function Get-BuildServer
{
param($serverName = $(throw 'please specify a TFS server name'))
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName ("Microsoft.TeamFoundation.Build.Client")
$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($serverName)
return $tfs.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
}
$buildserver = Get-BuildServer "http://tfsserver:8080/tfs/My%20Project%20Collection"
$teamProject = "ESI"
$buildDefinition = "iPrl_BuildMaster"
$definition = $buildserver.GetBuildDefinition($teamProject, $buildDefinition)
$request = $definition.CreateBuildRequest()
$buildserver.QueueBuild($request, "None")
Now after googling, I have found the following C# code to update the verbosity and, assuming it's the same for my custom process parameters, I need to convert this to work with the above powershell script. Any ideas?
IDictionary<String, Object> paramValues = WorkflowHelpers.DeserializeProcessParameters(processParameters);
paramValues[ProcessParameterMetadata.StandardParameterNames.Verbosity] = buildVerbosity;
return WorkflowHelpers.SerializeProcessParameters(paramValues);