I am running some performance tests from Jenkins. I do have two Windows machines with JMeter and I can configure from Jenkins which one to use. Everything works as expected here.
My issue: I did create another job for Stop/Shutdown the tests in case something goes wrong and you have a big run time. Whenever I try to summon Stoptest.sh/Shutdown.sh on the Windows machine that run tests, nothing happens.
How can I stop tests remotely? It has anything to do with the listening port? Thank you.
PS: Tests are ran using PSExec from Jenkin's Windows slave so there is no active CMD window on the screen.
Be aware that .sh is extension for Linux shell scripts, they cannot be executed by Windows command-line interpreter (CMD or Powershell) if you're running JMeter on Windows you need to go for shutdown.cmd or stoptest.cmd instead
There is also AutoStop Listener plugin which can be used for conditional stopping of JMeter test basing on various criteria, it can be installed using JMeter Plugins Manager
Related
i'm looking for some assistance in execution of Jmeter distributed test runs using Jenkins hosted on Windows platform.
I need to know how to
Start Jmeter master/slave agents (jmeter-server.bat) from Jenkins machine.
how to stop the above process once the test run completes.
A hint: if you don't "stop the above process once the test run completes" you won't have to "start" them.
If you want to control everything from Jenkins you need to install Jenkins agent process on each machine you intend to use as JMeter slave, see Step by step guide to set up master and agent machines on Windows Jenkins wiki page for more details.
Once done you can create a job which will start JMeter processes using simple "Execute windows batch command" step on the agent machines
With regards to your point 2 - it's sufficient to set server.exitaftertest JMeter property to true on slave machines in any convenient way, i.e. by adding -J command-line argument like:
jmeter-server.bat -Jserver.exitaftertest=true
For any Distributed Load Run via Jmeter, the pre-requisite is to get the Jmeter-server.bat running to support and enable master-slave communications by Jmeter between Load agents. Initially i had a struggle how to get this executed as first step in Jenkins and then followed execution of Jmeter commands for load test using Remote distributed run. Hence instead of running the pre-requisite condition of running Jmeter-server.bat from jenkins i made it as a Windows service (as load agents are windows machines) which is running all time and whenever Load test starts from Jenkins, the Jmeter-server running in background which will allow distributed run to go.
Selenium test (testNg) with mvn test from jenkins ,
I cant view the action performed on the browser, it is possible to view the real browser opened and performing action?
I suggest using SeleniumGrid.
You can then let the test run on a node, where you can see all the browser actions. For demonstration purposes, distributed and independent testing I did similar with teamcity for a regression-test project.
Basically there are two steps to setup the environment:
Start the grid for example as background-service.
Start a node which is running within a logged on user context (also as Service or autorun)
optional: Start another node for running tests in background
In your tests the creation of the the WebDriver instance changes like this:
WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
Here a link for more details to setup the Grid2: https://github.com/SeleniumHQ/selenium/wiki/Grid2
No. In typical setup, tests from Jenkins run on randomly available node. You don't know which node is available at a given point. Also typically Jenkins nodes tend to run headless browsers (like phantomjs) since they don't have displays.
Jenkins tests are meant to be run without any manual intervention, like on nightly basis for example. You develop tests locally, see how they run in a real browser and then push it to your main suite so that Jenkins can run it for you. If you want to run tests from Jenkins and view test running live then look for paid solutions like saucelabs or browserstack.
You can using BrowserStack. We have recently released a Plugin that allows you to view your test results right inside Jenkins. It's a paid tool of course :(
In Windows you should not install jenkins as windows application (because in this recent version, Microsoft decided to give services their own hidden desktop even you enable the functionality "interact with desktop" in jenkins service), you may have to deploy it from a war file as follows:
1) Download jenkins.war from Jenkins official site
2) Deploy it by the command prompt : java -jar {directoryOfJenkinsFile}/jenkins.war
3) Now you can access jenkins administration on http:// localhost:8080
Hope that helps you !
I would say that my problem rather lack of information and I need some confirmation than a real problem. It seems somebody else had similar question question.
I put together a machine (Windows Server 2012R2) for POC reasons where a Jenkins installed and it executes Selenium UI tests using nunit. The nunit tests are generated by Specflow.
I could do:
install jenkins
jenkins run by a valid user not by Service account
set up jenkins properly
it can pull the source code from TFS-GIT
it can compile the C# solution
it can execute the test project
the test results are correct
Selenium plugin installed on Jenkins but I don't think it is used in this case because the text execution is about executing nunit and it deals with everything else.
At the moment I don't need the capability to delegate test execution to other Jenkins slaves or machines because the Jenkins does have only one compile task. Compiling, executing and test running can go parallel, the machine able to deal with it.
But, when I log in the server where the Jenkins runs and I watch what happens during CI build (compile and test execution) I can't see that the browser (Firefox) starts, however, the test results and the logs show that a browser was executed.
What I did so far:
jenkins runs as service, the account is an existing account
If I remote to the machine with the account which is set up for the service, then I can't see the browser will be executed, however, the log shows that something had happened.
My question is that, what the hack is happening when my tests are executed by Jenkins? If I execute the command which is used by Jenkins from console on the same machine then I can see that Firefox starts, does what is programmed in the tests and the results are in the result.xml. Can I accept the result as valid result? Can I somehow set up Jenkins the way the browser really executed (I can believe it when I see it :) )?
I think this is because you run Jenkins as a service. Services do not show up in desktop. Workaround is to run Jenkins or slave from CMD.
Jenkins windows slave service does not interact with desktop
I am using Jenkins for my project CI. And several automation test cases developed by Coded UI(C# language) will be run when the new build is deployed. I created a job on the master(windows) that MSTest those test cases(note: with MSTest under cmd, test cases can be run successfully) but saw the below issue:
Error calling Initialization method for test class QuickUI.SmokeTests: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=159018)
Therefore,
1) Need I use Jenkins slave to run test cases?
2) If so, how to config since the dlls needs to be copied to slave firstly?
3) Is there any account or jenkins service configuration needs to be taken care?
Thanks in advance.
Sounds like you need to get mstest setup on your build machine. The easiest way is to use test agents (http://www.microsoft.com/en-us/download/details.aspx?id=38186) when installing chose test agents.
(below is copied from my answer in Coded UI build server integration process)
You going to want to put your CodedUI tests inside an orderedTest. Right Click on the solution -> add -> orderedTest.
You going to want to install https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin it's not a requirement, but it makes working with mstest in jenkins a little easier.
At this point you just need to configure the plugin in jenkins to run your orderedTest. It will need you to point to mstest and the location of your orderedtest.
1) Recommnd to run test on slave
2) To run GUI test, don't run the slave as windows service
3) Theoratically you only need the slave agent running. However the windows account logged in should have access to all the resources that your tests need.
Some other threads you could reference, pay special attention to this if you want to run the tests in unattended way.
Jenkins on Windows and GUI Tests without RDC
I am using Jenkins and Ant to execute my Selenium tests. I would like to make a job that would do the following:
starts Selenium server
executes tests and
kills Selenium server after all tests are run.
I am able to start the Selenium server manually with the following command:
java -jar selenium-2.16.1\selenium-server-standalone-2.25.jar
But I cannot find a Jenkins plugin that would do the start/stop for me. All I can find are some Selenium reporting plugins.
Not sure if there is a plugin, but you can just run the launch and kill commands via the Jenkins job.
Under Build, if you add an Execute shell build step, you can run your commands from there.
Note - The path to the selenium server is relative to the current work directory.
Looking at your error trace, looks like you are running this on a windows PC. you need to be running this as a "execute windows batch command" then run java -jar selenium-2.16.1\selenium-server-standalone-2.25.jar Assuming that you have your jar file on the workspace. or you beed to point it at the right path