I am running UFT Test Suite manually on the host machine, however i now have the requirement to run the smoke tests as soon as the new build is delivered in jenkins. For that i need to run the test suite automatically using Jenkings. Can someone help me in the direction, how this can be achieved? I have never used Jenkins before.
Looks like theres a plugin for Jenkins which enables integration with HP UFT. Take a look at the Jenkins wiki page for instructions about how to install and use it.
As you didn't mention that you are using ALM so I am assuming that you have some kind of driver script in vbs which runs your test suite. You do not necessarily require any plugin in jenkins to run your UFT suite(however you can also check the suggestion given in the previous answer). Create a job in jenkins and call your vbs file (driver file) or a bat file which kicks off your execution your test machine.
Your test execution machine can be configured as jenkins slave. Only thing which you need to take care off is to remove the settings which can let go your jenkins slave machine which is your uft execution machine in standby mode or locked user mode(uft scripts will fail otherwise). You can handle this by configuring windows settings(if admin allows) or by using some small scripts to move mouse after some intervals.
Your test execution job in jenkins(master) machine can be made dependent on the job which goes to success when new build is available. As soon as new build is available - that job goes to success then your Test job gets notified and connects to UFT execution machine(slave) and starts running your test.
I am trying to integrate ALM with jenkins but not working
Started by user admin
Running as SYSTEM
Building in workspace C:\JENKINSHOME\workspace\CLRP_AUTO_RN
[CLRP_AUTO_RN] $ C:\JENKINSHOME\workspace\CLRP_AUTO_RN\HpToolsLauncher.exe -paramfile props05092020010832244.txt
"Started..."
Timeout is set to: -1
Run mode is set to: RUN_PLANNED_HOST
Unable to retrieve test set folder: Node not found.
Node not found.
Starting test set execution
Test set name: CLRP Smoke Test Automation Run, Test set id: 104
Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))
Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form: http://almserver:8080/qcbin/TDConnectivity_index.html
Build step 'Execute Micro Focus functional tests from Micro Focus ALM' changed build result to FAILURE
Finished: FAILURE
Related
i'm looking for some assistance in execution of Jmeter distributed test runs using Jenkins hosted on Windows platform.
I need to know how to
Start Jmeter master/slave agents (jmeter-server.bat) from Jenkins machine.
how to stop the above process once the test run completes.
A hint: if you don't "stop the above process once the test run completes" you won't have to "start" them.
If you want to control everything from Jenkins you need to install Jenkins agent process on each machine you intend to use as JMeter slave, see Step by step guide to set up master and agent machines on Windows Jenkins wiki page for more details.
Once done you can create a job which will start JMeter processes using simple "Execute windows batch command" step on the agent machines
With regards to your point 2 - it's sufficient to set server.exitaftertest JMeter property to true on slave machines in any convenient way, i.e. by adding -J command-line argument like:
jmeter-server.bat -Jserver.exitaftertest=true
For any Distributed Load Run via Jmeter, the pre-requisite is to get the Jmeter-server.bat running to support and enable master-slave communications by Jmeter between Load agents. Initially i had a struggle how to get this executed as first step in Jenkins and then followed execution of Jmeter commands for load test using Remote distributed run. Hence instead of running the pre-requisite condition of running Jmeter-server.bat from jenkins i made it as a Windows service (as load agents are windows machines) which is running all time and whenever Load test starts from Jenkins, the Jmeter-server running in background which will allow distributed run to go.
As per the documentation, in order to run Coded-UI as part of the build you need to have your build agent running as interactive Mode (Not as a service) also your Test Agent need to be deployed as Interactive Mode.
Now the problem is when you deploy your test agent your build machine will restart automatically so your Build agent will not restart cause its not a a service.
How can you work around this ?
Two options:
Don't use the build server as a test agent. Depending on the complexity and size of the test suite, you can either run the tests directly on the deployed environment or have a dedicated group of test machines to parallelize the run.
Install the test agent outside of a build process and just use the "run functional tests" task to connect to the already-running agent. The "Install test agent" task is really for scenarios closer to option #1, especially if you're doing things like provisioning test machines in the cloud on-demand.
Since you would like to run Coded UI test, you have to configure the test agent to interact with the Desktop and build agent running as interactive Mode.
If you insist on installing the TFS Test agent on the same machine as the Build agent. You need to manually install and configure the agents first. Then specify the test agent installation media path in test agent deploy task and use Run Functional Test task to run Automated Test on agent machine.
I would say that my problem rather lack of information and I need some confirmation than a real problem. It seems somebody else had similar question question.
I put together a machine (Windows Server 2012R2) for POC reasons where a Jenkins installed and it executes Selenium UI tests using nunit. The nunit tests are generated by Specflow.
I could do:
install jenkins
jenkins run by a valid user not by Service account
set up jenkins properly
it can pull the source code from TFS-GIT
it can compile the C# solution
it can execute the test project
the test results are correct
Selenium plugin installed on Jenkins but I don't think it is used in this case because the text execution is about executing nunit and it deals with everything else.
At the moment I don't need the capability to delegate test execution to other Jenkins slaves or machines because the Jenkins does have only one compile task. Compiling, executing and test running can go parallel, the machine able to deal with it.
But, when I log in the server where the Jenkins runs and I watch what happens during CI build (compile and test execution) I can't see that the browser (Firefox) starts, however, the test results and the logs show that a browser was executed.
What I did so far:
jenkins runs as service, the account is an existing account
If I remote to the machine with the account which is set up for the service, then I can't see the browser will be executed, however, the log shows that something had happened.
My question is that, what the hack is happening when my tests are executed by Jenkins? If I execute the command which is used by Jenkins from console on the same machine then I can see that Firefox starts, does what is programmed in the tests and the results are in the result.xml. Can I accept the result as valid result? Can I somehow set up Jenkins the way the browser really executed (I can believe it when I see it :) )?
I think this is because you run Jenkins as a service. Services do not show up in desktop. Workaround is to run Jenkins or slave from CMD.
Jenkins windows slave service does not interact with desktop
I'm working on a team that is building a RESTful HTTP service. We're having trouble with setting up a Jenkins CI job which will build the service, run it in the background, execute some tests, and then terminate the servers.
Specifics
The server is built in Node.js using the hapi framework and has some unit tests written in mocha.
The tests are written in Java using Maven. (Why not node.js-based tests? Because our testing dept. has invested time in creating a Java-based REST-testing framework.)
The build should fail if the node-based unit tests fail or if the java tests fail.
Our Jenkins box is run by a support team elsewhere in the company; our builds execute on a Linux slave.
Current Attempt
We've got something that kind-of works right now, but it's unreliable. We use 3 build steps:
The first build step is an Execute Shell step with the following commands:
npm install
npm test
node server.js ./test-config.json &
Second we do a Invoke Maven 3 step that points to the test pom.xml.
And third we run Invoke Standalone Sonar Analysis to do static code analysis.
This mostly works, but we depend on Jenkins' ProcessTreeKiller to stop the services once the job completes. We always get the warnings stating: Process leaked file descriptors. See
http://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+buildfor
more information
Unfortunately, we've had cases where the service is terminated too soon (before the tests complete) or where the service doesn't get terminated at all (causing subsequent builds to fail because the port is already in use).
So we need something more reliable.
Failed Attempt
We tried setting up a single shell script which handled starting the service, running maven, killing the service, then outputting an exit code. But this didn't work out because the mvn command wasn't available on the command-line. Our Jenkins has multiple maven versions available (and jdks too) and I don't know where they live on the slaves or how to get at them without using the Invoke Maven 3 build step.
Ideas
We've toyed around with some ideas to solve this problem, but are hoping to get some guidance from others that may have solved similar problems with Jenkins.
Have the service self-terminate after some period of time. Problem is figuring out how long to let them run.
Add a build step to kill the services after we're done. Problem is that if the maven execution fails, subsequent steps won't run. (And if we tell maven to ignore test failures, then the build doesn't show as broken if they fail.)
Try killing any existing service process as the first and last steps of the build. Problem is that other teams also use these Jenkins slaves so we need to make sure that the service is terminated when we're done with our build.
Start and stop the node.js services via Maven doing something like this blog suggests. Problem is that we don't know if Jenkins will identify the spawned background task as a "leaked file descriptor" and kill it before we're done testing.
It would be nice if Jenkins had a "Post-build action" that let you run a clean-up script. Or if it had a "Execute background process" build step which would kill the background items at the end of the build. But I can't find anything like that.
Has anyone managed to get Jenkins to do anything remotely like this?
Some brainstorming:
You can turn off Jenkins ProcessTreeKiller, either globally or per invocation. I am not sure why that is not an option for you.
In response to #2, several options:
Post-build actions get executed regardless if build steps had failed or not. This would be a great way to trigger a "service cleanup" task that will run regardless of the build state.
You can setup any build step as post-build action, using Any Build Step plugin, or you can use Post Build Tasks plugin, the latter even gives options to define triggering criteria.
You can change the build state, based on RegEx criteria using Text-finder plugin
You can setup Conditional Build Steps. The "condition" could even be a result of some script execution
I am using Jenkins for my project CI. And several automation test cases developed by Coded UI(C# language) will be run when the new build is deployed. I created a job on the master(windows) that MSTest those test cases(note: with MSTest under cmd, test cases can be run successfully) but saw the below issue:
Error calling Initialization method for test class QuickUI.SmokeTests: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=159018)
Therefore,
1) Need I use Jenkins slave to run test cases?
2) If so, how to config since the dlls needs to be copied to slave firstly?
3) Is there any account or jenkins service configuration needs to be taken care?
Thanks in advance.
Sounds like you need to get mstest setup on your build machine. The easiest way is to use test agents (http://www.microsoft.com/en-us/download/details.aspx?id=38186) when installing chose test agents.
(below is copied from my answer in Coded UI build server integration process)
You going to want to put your CodedUI tests inside an orderedTest. Right Click on the solution -> add -> orderedTest.
You going to want to install https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin it's not a requirement, but it makes working with mstest in jenkins a little easier.
At this point you just need to configure the plugin in jenkins to run your orderedTest. It will need you to point to mstest and the location of your orderedtest.
1) Recommnd to run test on slave
2) To run GUI test, don't run the slave as windows service
3) Theoratically you only need the slave agent running. However the windows account logged in should have access to all the resources that your tests need.
Some other threads you could reference, pay special attention to this if you want to run the tests in unattended way.
Jenkins on Windows and GUI Tests without RDC