We are using TFS2015 VNext Run functional tests tasks to run our coded UI tests in VMs.
Our Code UI tests click controls on UI, and somehow the command shell of DTAExecutionHost.exe was in "Select" state which prevent the test run from finishing.
We had to log into the VM and manually un-select the DTAExecutionHost.exe to let the test run finish.
Any idea how to minimize the windows to prevent it being clicked by UI tests? Or if any workarounds to avoid the issue?
Thank you!
Angel
I managed to work around the issue by minimize all windows right after Test Agent being configured.
Related
I do want to run shell script between two XCUITests. But since package is created and installed on the device or simulator, how this can be achieved? Is there a way to execute shell script on the host machine which has device connected(Either for Simulator or Real iPhone) between the tests?
You should probably set up Client-Server communication between device and host machine in order to do things like this.
This exact approach has been already implemented in
https://github.com/Subito-it/SBTUITestTunnelHost
Another option is to move shell code entirely to the test code. For example, if you use a shell script to communicate with a remote server, you should consider doing it on the device.
I think you can try inserting a script inside the test plan.
From Xcode:
Go to tab product
Press Test Plan
Manage Test Plans
Then, from the right side, select Test, and insert your script inside the pre-actions and/or pos-actions.
PS: I'm not sure if you can do that for selected tests. Maybe the scripts will run before/after each test.
I am new to continuous integration. I have developed windows application and its smoke testing using coded ui. I want to create build and release process in TFS and want to execute coded ui tests bases on every time new build is created. I would like to know if there are any limitations for coded ui when it comes to continuous integration? I am not able to find any article so far which explains about behavior of coded ui in TFS continuous integration
Your guidance in this regard is highly appreciated.
First off, it's worth noting that Coded UI is effectively deprecated; Selenium is recommended for web apps, and Appium is recommended for desktop apps.
That said, you need to use the Deploy Test Agent task to deploy interactive test agents to your target machines, followed by a combination of the Windows Machine File Copy (to deploy your tests to the test agents) and Run Functional Tests task to kick off your tests.
When running a UI Test, how do I keep the simulator open so I can manually test additional steps?
After a UI Test completes, the simulator will shut down. I'd like to use the UI Test as an automation to reach a certain point in the app. Then I'll do a few additional things not covered by the test.
I don't think there's an option for that. You should separate your automatic and manual testing. Automatic testing ideally should be done on a CI. You should do your manual testing separately from UI tests.
We recently added Coded UI Tests to our solution. The tests complete successfully when ran through Test Explorer but when the code is checked in and a build is triggered, all the CUIT's fail (error message is below).
I have gone to each of the links in the error message. The first one details how to set up a test agent to run the tests. We don't really need tests on all our environments as we are setting up a lab environment to run them and adding the test agent would require me submitting a ticket which would probably take months to get a response back, not to mention walking the person through what needs to be done.
The second link is dead and I'm pretty sure I don't want to change the build to be interactive.
I am hoping there is an easy way to change the build definition (which I have access to) so that it can ignore all Coded UI Tests but still run the Unit Tests. Is this possible? Is there an easier way of going about this? Each set of tests have their own project file, all Coded UI Tests in one project and all Unit Tests in another project.
Thanks in advance.
Here is the error that is on all of the coded UI tests:
Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=255012)
If you are running the tests as part of your team build, you must also set up the build agent to run as an interactive process. For more information, see "How to: Configure and Run Scheduled Tests After Building Your Application" (http://go.microsoft.com/fwlink/?LinkId=254735).
One thing you can do is to group all the coded UI tests into a particular test category and exclude running those in the build. We do this with our "integration" tests and only choose to run our true unit tests for speed reasons.
See below link for how to use test categories:
https://msdn.microsoft.com/en-us/library/dd286683.aspx
Also see the below for how to edit your build definition to only run the tests you want:
Excluding tests from tfs build
You can use use some naming patterns and combine that with the Test case Filter option in the execute test step:
https://jessehouwing.net/xaml-build-staged-execution-of-unit-tests/
Something like:
FullyQualifiedName ~ .Ui.
This would require the Ui tests to have .Ui in the namespace
We have a Visual Studio test controller with 3 registered test agents in a specific test environment setup for our nightly automation runs. I've seen ample documentation on having the build agents run the tests, but we need the test execution to go through the controller and run from the test agents instead.
My thought was to edit the build process template so it would trigger the execution of these remotely executed tests and then wait for the test run results, but I have no experience with build templates and I've been unable to find any examples showing how I might accomplish this. And this is of course assuming that editing the build process is the best/correct solution in the first place.
Can someone with experience with triggering remote execution of tests at the end of a build/deploy cycle point me in the right direction please?
Actually, you don'have to change anything to your template. Just make sure your build definition refers to the correct tests and testsettings file that are configured for remote execution.
Step 1:
Please open http://msdn.microsoft.com/en-us/library/ee256991.aspx and scroll down to the section "Add a test settings for remote execution or data collection to your solution". Follow this to create a test settings file for remote execution.
Step 2:
Edit your build definition: go to the Process page, under heading "2. Basic", open the Automated Tests dialog by clicking the "..."at the end. It the Automated Tests dialog, click "Add". Then browse for your test settings file (for remote execution, the one you just created) and confirm your choices.
Now save your build definition and queue your build. Automagically, your tests are now performed on the remote system, because your testsettings file tells your build system to do so.
Hope that is enough to start your remote tests to work.