We are using Jenkins 1.642.1 and we generate Cobertura reports for both build/deploy jobs and functional test jobs. (It appears we are using Cobertura plugin version 1.9.6) The generated report includes a link to the js source file which included red/green color coding to indicate line hits. This file includes non-ASCII characters so garbage chars are rendered in the html view of this file.
When I check the configuration of the build job (which is generated by groovy scripts) the Source Encoding setting for the 'Publish Cobertura Coverage Report' indicates "ASCII" as the setting. I need it to be UTF-8. I can choose this manually, but we don't mess with our builds manually. Everything is generated in CI style - using programming.
In this case, I see the line in the groovy script and it calls 'cobertura [path to file]'. There doesn't seem to be any room for additional arguments including that for the source encoding. Can someone point me the right direction to set this value programmatically?
Thanks,
Rob
Ah - It appears that you can micro-manage the resulting configuration.xml file via commands in your groovy file like this:
// Configuring Cobertura
project / publishers / "hudson.plugins.cobertura.CoberturaPublisher" {
coberturaReportFile('Directory path')
sourceEncoding(UTF_8)
}
Related
How do you automatically parse ansible warnings and errors in your jenkins pipeline jobs?
I greatly enjoy the power of leveraging in ansible in jenkins when it works. Upon a failure, the hunt to locate the actual error can be challenging.
I use WarningsNG which supports custom parsers (and allows their programmatic generation)
Do you know of any plugins or addons that already transform these logs into the kind charts similar to WarningsNG?
I figured I'd ask as I go off into deep regex land and make my own.
One good way to achieve this seems to be the following:
select an existing structured output ansible callback plugin (json, junit and yaml are all viable) . I selected junit as I can play with the format to get a really nice view into the playbook with errors reported in a very obvious way.
fork that GPL file (yes, so be careful with that license) to augment with the following:
store output as file
implement the missing callback methods (the three mentioned above do not implement the v2...item callbacks.
forward events to the default or debug callback to ensure operators see something when they execute the plan
add a secrets cleaner - if you use jenkins credentials-binding-plugin it will hide secrets from the console, it will not not hide secrets within stored files. You'll need to handle that in your playbook or via some groovy code (if groovy, try{...} finally { clean } seems a good pattern)
Snippet - forewarding to default callback
from ansible.plugins.callback.default import CallbackModule as CallbackModule_default
...
class CallbackModule(CallbackBase):
CALLBACK_VERSION = 2.0
CALLBACK_TYPE = 'stdout'
CALLBACK_NAME = 'json'
def __init__(self, display=None):
super(CallbackModule, self).__init__(display)
self.default_callback = CallbackModule_default()
...
def v2_on_file_diff(self, result):
self.default_callback.v2_on_file_diff(result)
... do whatever you'd want to ensure the content appears in the json file
I am working in jenkins.
I'd like to put some specified text in the Console Output in a specified colour.
For example:
My output is:
Commit message....
Creating dir: /var/lib/1
Creating dir: /var/lib/2
Job done
Build Successful
I'd like to out "Job Done" in red.
How can I do this?
I've looked at some plugins (The parsed log, AnsiColor Plugin) but I think I cannot do this using it.
The Log Parser plugin will do what you want.
It allows you to specify your own rules with a parsing rules file and regular expression like syntax.
Cobertura plugin in Jenkins has a support of ratcheting by ticking these boxes:
Health auto update
Stability auto update
When ticking this box, the coverage metric targets (in Jenkins configuration page) will be updated on every successful build:
These values will be overridden by job-dsl-plugin when seed job is triggered. How can I retain these values when my seed job is triggered?
Seems like I can't find a pretty way to do this right now, but here is my solution.
Solution
1. Execute a Groovy script and store all of the current job cobertura configuration in a JSON file.
Cobertura configuration can be retrieved like:
def coberturaPublisher = project.getPublishersList().get(CoberturaPublisher)
coberturaPublisher.**healthyTarget**.getTarget(**CoverageMetric.METHOD**)
2. job-dsl-plugin to configure cobertura by using the JSON file if it's available
job-dsl's CoberturaContext normal method can't be called here because the data represented in the first step is different with the method parameter:
80% is stored as 8000000 in the JSON file
80% must be passed in as 80 instead of 8000000 to CoberturaContext methods.
As of today, I can't simply divide the value by 100000 because the method is accepting Integer instead of double. To retain the precision of the ratcheted configuration, I have to bypass the validation by manipulating the target directly:
coberturaContext.targets = [
'METHOD': new CoberturaContext.CoberturaTarget(
targetType: CoberturaContext.TargetType.METHOD,
healthyTarget: 8000000,
unhealthyTarget: previousConfig ? previousConfig.cobertura.method.unhealthy : 0,
failingTarget: previousConfig ? previousConfig.cobertura.method.failing : 0
),
Why bother creating the JSON file while you can call Jenkins API directly?
My seed job is configured with this example here, hence I have an additional classpath used in the job configuration. When I tried to hit Jenkins API directly, I'm getting class loading issue for Cobertura plugin classes.
I was wondering if any of you guys had any experience generating code coverage reports in TFS Build Server 2010 while running NUnit tests.
I know it can be easily done with the packaged alternative (MSTest + enabling coverage on the testrunconfig file), but things are a little more involved when using NUnit. I've found some info here and there pointing to NCover, but it seems outdated. I wonder if there are other alternatives and whether someone has actually implemented this or not.
Here's more info about our environment/needs:
- TFS Build Server 2010
- Tests are in plain class libraries (not Test libraries - i.e., no testrunconfig files associated), and are implemented in NUnit. We have no MSTests.
- We are interested in running coverage reports as part of each build and if possible setting coverage threshold requirements for pass/fail criteria.
We 've done it with NUnit-NCover and are pretty happy with our results. NUnit execution is followed by NUnitTfs execution in order to get our testing results published in the Build Log. Then NCover kicks in, generating our code coverage results.
One major thing that poses as a disadvantage is fact that setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it.
Two things could pose as disadvantages:
NUnitTfs doesn't work well with NCover (at least I couldn't find a way to execute both in the same step, so (since NCover invokes NUnit) I have to run Unit tests twice: (1) to get the test results and (2) to get coverage results over NCover. Naturally, that makes my builds last longer.
Setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it .
In any case, the resulting reporting (especially the Trend aspect) is very useful in monitoring how our code evolves within time. Especially if you 're working on a Platform (as opposed to short-timed Projects), Trend reports are of great value.
EDIT
I 'll try to present in a quick & dirty manner how I 've implemented this, I hope it can be useful. We currently have NCover 3.4.12 on our build server.
Our simple naming convention regarding our NUnit assemblies is that if we have a production assembly "123.dll", then another assembly named "123_nunit.dll" exists that implements its tests. So, each build has several *_nunit.dll assemblies that are of interest.
The part in the build process template under "If not disable tests" is the one that has been reworked in order to achieve our goals, in particular the section that was named "Run MSTest for Test Assemblies". The whole implementation is here, after some cleanups to make the flow easier to be understood (pic was too large to be directly inserted here).
At first, some additional Arguments are implemented in the Build Process Template & are then available to be set in each build definition:
We then form the NUnit args in "Formulate nunitCommandLine":
String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL))
This is then used in the "Invoke NUnit"
In case this succeeds & we have set coverage for this build we move to "Generate NCover NCCOV" (the coverage file for this particular assembly). For this we invoke NCover.Console.exe with the following as Args:
String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""",
NUnitPath,
Path.GetFileName(nunitDLL),
Path.GetDirectoryName(nunitDLL),
Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)),
Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"),
Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""),
BuildDetail.BuildNumber)
All these run in the foreach loop "For all nunit dlls". When we exit the loop, we enter "Final NCover Activities" & at first the part "Merge NCCovs", where NCover.Console.exe is executed again - this time with different args:
String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject
)
When this has run, we have reached the point where all NCCOV files of this build are merged into one NCCOV-file named after the build + the Trend file (that monitors the build throughout its life) has been updated with the elements of this current build.
We now have to only generate the final HTML report, this is done in "Generate final NCover rep" where we invoke NCover.reporting with the following args:
String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
PathForNCoverResults,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject,
BuildType
)
I'm using Emma in my ant build to perform coverage reporting. For those that have used Emma, is there a way to get the build to fail if the line coverage (or any type of coverage stat) does not meet a particular threshold? e.g. if the line coverage is not 100%
Not out of the box.
However, the report.metrics property or attribute of <report></report> can be set for name, class, method, block, and line. See Coverage Metrics in the Emma reference.
Use a plain-text report then a regexp filter to set up a fail condition.
I wrote an ant task to do this.
You should be able to find all the information you need on my EmmaCheck site.