As per the documentation, in order to run Coded-UI as part of the build you need to have your build agent running as interactive Mode (Not as a service) also your Test Agent need to be deployed as Interactive Mode.
Now the problem is when you deploy your test agent your build machine will restart automatically so your Build agent will not restart cause its not a a service.
How can you work around this ?
Two options:
Don't use the build server as a test agent. Depending on the complexity and size of the test suite, you can either run the tests directly on the deployed environment or have a dedicated group of test machines to parallelize the run.
Install the test agent outside of a build process and just use the "run functional tests" task to connect to the already-running agent. The "Install test agent" task is really for scenarios closer to option #1, especially if you're doing things like provisioning test machines in the cloud on-demand.
Since you would like to run Coded UI test, you have to configure the test agent to interact with the Desktop and build agent running as interactive Mode.
If you insist on installing the TFS Test agent on the same machine as the Build agent. You need to manually install and configure the agents first. Then specify the test agent installation media path in test agent deploy task and use Run Functional Test task to run Automated Test on agent machine.
Related
i'm looking for some assistance in execution of Jmeter distributed test runs using Jenkins hosted on Windows platform.
I need to know how to
Start Jmeter master/slave agents (jmeter-server.bat) from Jenkins machine.
how to stop the above process once the test run completes.
A hint: if you don't "stop the above process once the test run completes" you won't have to "start" them.
If you want to control everything from Jenkins you need to install Jenkins agent process on each machine you intend to use as JMeter slave, see Step by step guide to set up master and agent machines on Windows Jenkins wiki page for more details.
Once done you can create a job which will start JMeter processes using simple "Execute windows batch command" step on the agent machines
With regards to your point 2 - it's sufficient to set server.exitaftertest JMeter property to true on slave machines in any convenient way, i.e. by adding -J command-line argument like:
jmeter-server.bat -Jserver.exitaftertest=true
For any Distributed Load Run via Jmeter, the pre-requisite is to get the Jmeter-server.bat running to support and enable master-slave communications by Jmeter between Load agents. Initially i had a struggle how to get this executed as first step in Jenkins and then followed execution of Jmeter commands for load test using Remote distributed run. Hence instead of running the pre-requisite condition of running Jmeter-server.bat from jenkins i made it as a Windows service (as load agents are windows machines) which is running all time and whenever Load test starts from Jenkins, the Jmeter-server running in background which will allow distributed run to go.
Issue Description:
We have release definitions set up to execute Coded UI test cases, the test agent deployment task has run into an issue where in it the execution does not end until the deployment is manually cancelled, the message displayed on the terminal is as follows "DistributedTests: Task 'ConfigureTestAgent' on machine 'HOSTNAME:5985' is taking time. Please Wait"
vsts_testagent.exe path is supplied from local path.
Environment Details:
TFS on-premises , TFS 2017 15.112.26307.0 (Tfs2017.Update1)
Build Agent : Private, OS: Windows Server 2012 R2, build agent version : 2.112.0
Test Agent : Private, OS: Windows 10
Test deployment task version: vstf_testagent.exe version version 14.0.23107.10
Test agent deployment step will spend longer time when the first time to deploy the test agents on a machine, if a server has been deployed successfully before, it would be much faster.
Please try below items to fix that:
Try with same credentials for build agent and test agent.
Try to download the test agent and install it on that machine manually, then try to queue build with the test agent deployment taskagain. （Install and configure test agents）
Try to deploy the test agent on another clean test machine.
Similar issue in Github for your reference: https://github.com/Microsoft/vsts-tasks/issues/2023
I have several automated tests that are associated to test cases in TFS 2017. These tests are running through as part of a functional tests step as a part of our build and they're running fine and I can see them running on the test machines. The physical browsers are there and navigating.
I'm attempting to run automated tests through TFS test hub using the new 2.* version build agents. This will allow us to run individual tests or tests plans.
However, I can't figure out a way to run tests on these agents where they're not running headless. They run headless fine, but I need to be able to see them run.
I read somewhere in Microsoft documentation that the agent must run as interactive instead of a service for UI tests. However that means the agent is running as a version 1.* which is not supported by the TFS test hub test runs.
Does anyone know a way to use the test hub to run test on physical browsers instead of headless?
The new agent 2.0 can also be configured to run in interactive mode, please see Deploy an agent on Windows for details.
Just for your information:
If you are running UI tests such as CodeUI or Selenium on physical browsers such as IE, Firefox, or Chrome, the agent on the machines must be running in interactive mode and not as a service.
If you are running UI tests such as coded UI or Selenium tests, the agent on
the test machines must be running in interactive mode, not as a
service, to allow the agent to launch a web browser. If you are using
a headless browser such as PhantomJS, the agent can be run as a
service or in interactive mode. See Build and Release Agents, Deploy
an agent on Windows, and Agent pools and queues.
Please see Run automated tests from test plans in the Test hub for details.
I am running UFT Test Suite manually on the host machine, however i now have the requirement to run the smoke tests as soon as the new build is delivered in jenkins. For that i need to run the test suite automatically using Jenkings. Can someone help me in the direction, how this can be achieved? I have never used Jenkins before.
Looks like theres a plugin for Jenkins which enables integration with HP UFT. Take a look at the Jenkins wiki page for instructions about how to install and use it.
As you didn't mention that you are using ALM so I am assuming that you have some kind of driver script in vbs which runs your test suite. You do not necessarily require any plugin in jenkins to run your UFT suite(however you can also check the suggestion given in the previous answer). Create a job in jenkins and call your vbs file (driver file) or a bat file which kicks off your execution your test machine.
Your test execution machine can be configured as jenkins slave. Only thing which you need to take care off is to remove the settings which can let go your jenkins slave machine which is your uft execution machine in standby mode or locked user mode(uft scripts will fail otherwise). You can handle this by configuring windows settings(if admin allows) or by using some small scripts to move mouse after some intervals.
Your test execution job in jenkins(master) machine can be made dependent on the job which goes to success when new build is available. As soon as new build is available - that job goes to success then your Test job gets notified and connects to UFT execution machine(slave) and starts running your test.
I am trying to integrate ALM with jenkins but not working
Started by user admin
Running as SYSTEM
Building in workspace C:\JENKINSHOME\workspace\CLRP_AUTO_RN
[CLRP_AUTO_RN] $ C:\JENKINSHOME\workspace\CLRP_AUTO_RN\HpToolsLauncher.exe -paramfile props05092020010832244.txt
"Started..."
Timeout is set to: -1
Run mode is set to: RUN_PLANNED_HOST
Unable to retrieve test set folder: Node not found.
Node not found.
Starting test set execution
Test set name: CLRP Smoke Test Automation Run, Test set id: 104
Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))
Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form: http://almserver:8080/qcbin/TDConnectivity_index.html
Build step 'Execute Micro Focus functional tests from Micro Focus ALM' changed build result to FAILURE
Finished: FAILURE
I am using Jenkins for my project CI. And several automation test cases developed by Coded UI(C# language) will be run when the new build is deployed. I created a job on the master(windows) that MSTest those test cases(note: with MSTest under cmd, test cases can be run successfully) but saw the below issue:
Error calling Initialization method for test class QuickUI.SmokeTests: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=159018)
Therefore,
1) Need I use Jenkins slave to run test cases?
2) If so, how to config since the dlls needs to be copied to slave firstly?
3) Is there any account or jenkins service configuration needs to be taken care?
Thanks in advance.
Sounds like you need to get mstest setup on your build machine. The easiest way is to use test agents (http://www.microsoft.com/en-us/download/details.aspx?id=38186) when installing chose test agents.
(below is copied from my answer in Coded UI build server integration process)
You going to want to put your CodedUI tests inside an orderedTest. Right Click on the solution -> add -> orderedTest.
You going to want to install https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin it's not a requirement, but it makes working with mstest in jenkins a little easier.
At this point you just need to configure the plugin in jenkins to run your orderedTest. It will need you to point to mstest and the location of your orderedtest.
1) Recommnd to run test on slave
2) To run GUI test, don't run the slave as windows service
3) Theoratically you only need the slave agent running. However the windows account logged in should have access to all the resources that your tests need.
Some other threads you could reference, pay special attention to this if you want to run the tests in unattended way.
Jenkins on Windows and GUI Tests without RDC