Question is how to run a jmeter script using jenkins job.
Is there any pre-requisites like standard variable names and all.
I have heard that there are different plugins, some can just execute the script as it is while some other can take inputs from gui as well as provides report as chart or graph.
Also some posts says first we need to create ant programs and then need to call them from jenkins.
If possible, please detail the exact procedure and suitable plugin.
I use the following two methods:
You can run Jmeter in non-GUI mode using command line ("Execute Windows batch command" or "Execute Shell" build step in Jenkins job settings)
jmeter.bat -n -t path_to_your_jmx_script.jmx
Of course, you can do the same by launching Jmeter as a java application or within shell script.
By "Invoke Ant" build step.
This method provides you all the benefits of ant. So, just include jmeter in appropriate ant target in your build.xml file. Here is an example (jvmarg and jmeterarg are not required):
<target name="test" depends="clean">
<jmeter jmeterhome="${jmeter-home}"resultlogdir="results/jtl">
<testplans dir="scripts" includes="*.jmx"></testplans>
<jmeterarg value="-Jbackend=${env.Backend_Address}"/>
<jvmarg value="-Xmx512m"/>
<jvmarg value="-Xdebug"/>
</jmeter>
</target>
And that's how you can generate nice report:
<target name="report" depends="test">
<xslt classpathref="xslt.classpath"
basedir="results/jtl"
destdir="results/html"
includes="*.jtl"
style="${jmeter-home}/extras/jmeter-results-detail-report_21.xsl">
<param name="showData" expression="${show-data}"/>
</xslt>
<copy file="${jmeter-home}/extras/expand.png" tofile="results/html/expand.png"></copy>
<copy file="${jmeter-home}/extras/collapse.png" tofile="results/html/collapse.png"></copy>
</target>
Regarding Jenkins plugins - the only one I know (and use) is Performance Plugin that can mark your build as failed/passed based on Jmeter results and generate nice graphs.
This site has the detailed information on Jmeter + Ant + Jenkins integration.
Other than running the test, you might also be interested in generating reports, charts and mailing the results.
Check here:
http://www.testautomationguru.com/jmeter-continuous-performance-testing-part2/
Jenkins doesn't provide any plugins to execute a JMeter test, it is up to you how you will launch it, the choices are in:
Command-line non-GUI mode
Ant task
Maven plugin
So you need to add a build step which will trigger JMeter test execution using one of above approaches. Also you can add Performance Plugin as a post-build action in order to be able to visualise performance trends and set pass and fail criteria.
See Continuous Integration 101: How to Run JMeter With Jenkins article for detailed walkthrough.
Related
Currently in our company we are executing tests from CCNet (in Automation VM) by specifying different build arguments, however I feel it is now best to migrate to better automation server which is Jenkins
I am new to Jenkins world however I started installing the jenkins (though had issue in starting the Jenkins Service, later found it was due to JDK version issue)
But now I have installed all the needed plugins(GIT - for gitlab repository, MSBuild and MStest)
But I do not know how do I specify build arguments in Jenkins the way we do it in CCNet.config files
Here is the CCNet Config File that we use to Force the build
<project name="AutoTests - Payments" queue="Q13">
<triggers>
<!--<scheduleTrigger time="03:30" buildCondition="ForceBuild" name="Scheduled" />-->
</triggers>
<workingDirectory>D:\GIT-TA</workingDirectory>
<artifactDirectory>D:\DEV\deploy\AutomatedTest\BuildLogs\BasicTests.Payments</artifactDirectory>
<category>AutomatedTest</category>
<modificationDelaySeconds>60</modificationDelaySeconds>
<labeller type="defaultlabeller">
<prefix>0.1.</prefix>
<incrementOnFailure>true</incrementOnFailure>
<labelFormat>000</labelFormat>
</labeller>
<tasks>
<exec>
<executable>D:\MSTestExtended\Runner.exe</executable>
<buildArgs>run-tests /project:payments /retriesCount:1</buildArgs>
<buildTimeoutSeconds>6800</buildTimeoutSeconds>
<successExitCodes>0</successExitCodes>
</exec>
</tasks>
<publishers>
<merge>
<files>
<file>D:\DEV\deploy\AutomatedTest\Results\TestPublish\BasicTests.Payments_Rerun.trx</file>
</files>
</merge>
<xmllogger />
</publishers>
Likewise we have many projects and then we do the build for that, but now in Jenkins I feel little lost - how do I configure it to with my according
Note: IN CCNet - it used to pick the branch whichever is checkout in VM, but I see in Jenkins we also need to manually setup the branch name; so in that case also Is there a way to specify the branch during build time?
And if I can add the selection for run-tests /project:payments /retriesCount:1 - when we initiate the jenkins build, it will aske to select the arguments
Can someone help me in that to understand the Jenkins way of doing this?
1.It can't perform when my script name is lotteryOnline:
2.But changed to other names (such as game), BUILD SUCCESSFUL：
Most likely your JMeter test execution for this lotteryOnline test plan has failed. The reason of failure you can see in the jmeter.log file. If you don't see the log file you can add jmeterlogfile parameter to jmeter Ant task like:
<jmeter
jmeterhome="${jmeter.home}"
testplan ="${testpath}/${test}.jmx"
resultlog="${testpath}/${test}.jtl"
jmeterlogfile="/path/to/jmeter.log"> <!-- add this line to enable logging-->
References:
JMeter Ant Task
Five Ways To Launch a JMeter Test without Using the JMeter GUI
Because there is a file, which is called game.itl. The parameter where you supply game is used to pick that file, as it seems.
We have an automated build process that uses Nant scripts, and we just inherited a Java project that has an existing, working Ant script. I have searched all over and it seems most people use a batch file to call Ant scripts from Nant. This seems a little hacky to me. Is there a way to call Ant scripts directly from Nant in task format?
This similar question addresses it with a batch file but I want to do it without.
<!-- calling the Ant build Process-->
<target name="AntBuildProcess" description="Created to call the Ant build during this NAnt build process" >
<echo message="Starting the Ant Build Process..." />
<exec program="ant" commandline='-buildfile YourAntBuild.xml' failonerror="true"/>
</target>
Then during your Build process, you just call this target at the point you need it to be built. <call target="AntBuildProcess" />
Ant is a batch file. Take a look, and you'll see a file called ant.bat in the %ANT_HOME%\bin directory.
Ant is actually a Java program, so you could launch it from the java command by running the class org.apache.tools.ant.launch.Launcher which is basically what the ant.bat file is doing.
However, why reinvent the wheel? The ant.bat runs the Java command the right way, gives you options to change the way it's executed, and makes sure everything is setup correctly.
Addendum
I see, Nant, unlike Ant, will always call cmd.exe and use suffixes and %PATHEXEC% to figure out if something is a batch script or other type of script. Thus, if you want to run Ant using Ant as a batch script via <exec/> you would do this:
<exec executable="cmd.exe"
dir="${working.dir}">
<arg value="/c"/>
<arg value="ant.bat"/>
</exec>
However, in Nant you can simply do it this way:
<exec program="ant"
workingdir=${working.dir}"/>
I have few target that I need to execute. I create a target naming all of them but I think it's not the way to do it ? Here is the target run that call all other target :
<target name="test.all" depends="build
echolaunching agent /echo
antcall target="RunJtfTests" /
antcall target="launchOpenAgent" /
antcall target="run.test" //target
target name="run.test" depends="build, launchOpenAgent, runJtfTests"
echo Launching test/echo
echo message="${toString:iControlSilk4J.classpath}" /
<java classname="com.miranda.icontrol.silk4j.installation.AdministrationCtrl"
classpath><fileset dir="${lib.dir}"
include name="**/*.jar" />
/fileset
pathelement path="${iControlSilk4J.classpath}" /
pathelement location="${jarPath}/Admin.jar" /
/classpath
</java>
</target>
It doesn't run and I do it to get the report and I get nothing ? What is wrong ? From what I read, antcall is like a goto loop which is not good. I want to call tests instead.
-> Here are all the tests I want to execute :
init
clean
AdministrationCtrl.Rollback
AdministrationCtrl.LatestInstallation
AdministrationCtrl.BackupiControl,
AdministrationCtrl.ChangeService
AdministrationCtrl.DefaultSetting
AdministrationCtrl.InitFailOver
AdministrationCtrl.RunDensite2Service
AdministrationCtrl.RunDensiteService
AdministrationCtrl.RunGSMService
AdministrationCtrl.RunLoudnessAnalyzerService
AdministrationCtrl.RunLoudnessLoggerService
AdministrationCtrl.RunRouterManagerService
AdministrationCtrl.RunttyR0Service
AdministrationCtrl.RunVirtualService
AdministrationCtrl.RestoreBkp
but this can be more general (regarding tests I will add in Silk4J). Is there a way to be more generic ?
Repeat after me:
Ant is not a programming language. It's a dependency matrix language.
This is an important distinction. You don't tell Ant what to execute. You tell Ant what you need, and Ant will figure out what to do.
I can tell you're having problems understanding Ant with all of those <antcall/>. That is a no-no because it could make you execute tasks more than once. Your build file also makes no sense.
Use the target's dependency parameter. For example, here's a skeleton build.xml file:
<project>
<target name="clean"/>
<target name="prepare"/>
<target name="compile"
depends="prepare"/>
<target name="package"
depends="compile"/>
<target name="test-compile
depends="compile"/>
<target name="test"
depends="test-compile"/>
<target name="deploy"
depends="package"/>
<target name="post-test-results"
depends="test"/>
<target name="all"
depends="clean,post-test-results,deploy"/>
</project>
When I want to run my target all, I mainly mean I want to do a clean build, post my test results, and deploy the build. This is true with Makefiles too. I don't list all of my tasks. Why do I care if I do my prep work for compilation? It's not my concern.
So I call, all, and that will call clean, post-test-results, and deploy. I have no idea what Ant will do beyond calling these three targets.
Wait... What do I need to do in order to post my test results? Well, I may need to run my tests. Therefore, I have a dependency to test for my post-test-results target. But, in order to run my tests, I may have to compile them. So, there's a dependency to test-compile on my test target.
In order to compile my tests, I have dependencies on the regular Java code. So, test-compile will depend upon compile. In order to compile, I have to prepare. Maybe that's building the necessary structure, or downloading the required jars. That's called before compile. Now, I can deploy. However, before I can deploy, I need to package my deployment. So, deploy depends upon package.
Package itself depends upon compile, but so did my compile-test. Since I've already called compile, my package target doesn't have to do that. All it has to do is package up the already compiled class files.
Thus, I'll probably execute the following targets in this order:
clean
prepare
compile
test-compile
post-test-results
package
deploy
My all target does hit all of my other targets, but I didn't have to list them all as dependencies or force them to build via <antcall/>s.
It looks like you need to learn about Ant and how it works. Your sample Ant file is simply not valid. Ant uses an XML style structure. All tasks are XML style entities.
Fortunately, there are a lot of good books on Ant. I would recommend Manning's Ant in Action as a good starting point.
One of the things you will find out is that you can specify batches of junit tests in a single <junit> task. This can be done via one or more <batchtest> sub-entities on the <junit> task. The <batchtest> will run all classifies that match a particular criteria. You can also use the <test> sub-entity on the <junit> task. The <test> sub-entity allows you to specify individual classfiles to run. Many times, these classfiles may simply call a specified set of other Junit classifies. This way, the developer can specify what tests to run and what tests to skip.
You can control what tests to run or not run by using properties and not by creating dozens of testing tasks. This allows you to specify sets of tests without having to spawn multiple JUnit processes.
I know that, we can very well use ANT and Maven together to build the project.We can run ANT scripts through Maven's POM.xml. But my question is can we run pom.xml through ANT's build.xml ?
i.e. can we create maven build from build.xml
Yes, using maven ant tasks.
The page lists out multiple maven tasks which can be integrated into an ant build script, thus combining the features of both. To take an example, there is the mvn task, which as documented can do a full maven build from ant.
<artifact:mvn mavenHome="/path/to/maven-3.0.x">
<arg value="install"/>
</artifact:mvn>
Besides this, there are
Dependencies task
Install and Deploy tasks
Pom task
each described with examples.
Maven and ANT are very different build tools. In ANT you write all the logic yourself, whereas a standard build process is "baked in" with Maven.
The POM file contains no logic, instead it contains a series of declarations about your project.
If you understand well how Maven works, it is theoretically possible to take a POM and generate an ANT build that emulates the behaviour of the Maven build. I'm not aware of any solution which can easily convert in the other direction, mainly because ANT is missing Maven functionality, such as dependency management.
Instead of trying to convert an ANT build into Maven, I'd recommend that you keep your existing build logic and delegate the management of your classpath to the ivy or Maven ANT tasks. These tools also provide tasks to publish your build output to a Maven repository, enabling your project to share with other projects using Maven.
Finally, I'm an ivy advocate and wrote an ant2ivy script which can assist in upgrade process. It creates an initial set of configuration files for downloading your projects dependencies from the Maven central repository.