I've created a few tests for my project, some test need to be run async while others can run sync.
Currently the config file is set to run async.
I've set a test file which runs them in the correct order. I've imported all the tests that should run sync into a single file. And tried adding
test.describe.configure({ mode: 'parallel' })
This changes the whole test process to run in parallel.
I can't seem to find any documentation on how to only execute certain test async and other sync. Does anyone have experience with this?
The reason I need it to run async for certain files is to log in and authenticate before continuing, also certain actions affect the layout of the whole UI (even in a different browser) and will mess up other tests screenshots.
PW runs all test in parallel by default. But there are options to serialize tests per file. You cannot have both paralel and serial tests in one file.
https://playwright.dev/docs/test-parallel#serial-mode
Related
In Playwright, after all tests have been executed, I am running a globalTeardown file that would send all of the test results to an external test management tool by making a series of api calls.
My intention was to access the jUnit.xml file and get the temporary test results from there, but the problem is that that file doesn't generate until the entire run is over, which is after the globalTeardown call.
Is there a way with Playwright to access the temporary test results, before the run is over?
I guess the title is pretty self-explanatory. The reason I want that is so that I can make a live custom HTML reporter for my tests.
My test suite takes hours to complete, and although the tests generate HTML reports as soon as each test step is executed, it's only at post-build time that those report files get published.
Being able to see them as they get generated would reduce the time it takes for me and my teammates to analyze and act upon issues revealed by our test runs.
All I need is that Jenkins let me access the build files as the build executes. Nothing fancy; I can take care of the rest. Is that possible? How?
In our setup there is always an intermediate file (typically XML) but the HTML files are created at the end of the job.
What you can do, is use the progressive output (http://jenkins/job/jobName/buildNumber/logText/progressiveText?start=0). Although you don't state which framework you use, most of them output something that would be easy to parse. e.g. "Test xxx failed".
I'm struggling with something for quite a while and can't find a solution,
I got a test project (cucumber, maven) I configure jenkins to pull the project from github, build and execute the code (selenium test script) on jenkins slave and that works perfect, I added few more slave, tagged them and I'm able to execute the same job parallel(the same test cases on different machines)
my next step is to use grid extra (https://github.com/groupon/Selenium-Grid-Extras) in order to use some cool features like video recording, browser updating, selenium updates etc...
now, I know that in order to use the grid I need to address it via my code and also define the desired capabilities (browser, os etc...).
currently when I'm running the same job twice, my second request will be queued till the first one ends, if I will run the same code from two developers machine it will run at 2 different nodes and the grid can handle both request.
not sure what is wrong with my jenkins configuration or my grid hub configuration, I checked it again and again and all looks good :-)
so guess I'm missing something
any advice/direction/idea will be highly appreciated.
Thanks
Ronen
I have a scenario that requires me to revert to a clean snapshot before or after each Coded UI test method is executed. I have researched using the TFS Lab Management API (see http://blogs.microsoft.co.il/shair/2011/12/22/tfs-api-part-42-getting-started-with-lab-management-api/) to revert to a specific snapshot as part of the TestInitialize and/or TestCleanup method, but I can only get this to work when executed locally. When executed on a remote machine I get errors authenticating to the TFS service.
My other option is to somehow do a 'foreach test in testrun' into the build process template (LabDefaultTemplate.11.xaml). I have identified the area that I think this would fit best, but cannot find any documentation on running a loop on each test.
Is this something that is possible, or is there somehow a built in method to accomplish this that I have overlooked?
To do what you propose you should switch to Release Management and create a separate test run for each of your groupings, in your case each test. You can use RM to orchestrate looping through each of your runs and executing then.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
However running a UI test should not break your application and I would suggest that either your tests are way too long, or there is some flaw in the design of your application.
Is there a way to mark a fitnesse test such that it will not be run as part of a suite, but it can still be run manually?
We have our FitNesse tests running as part of our continuous integration, so new tests that are not yet implemented cause the build to fail. We'd like a way to allow our testers and BAs to be able to add new tests that will fail while still continuing to validate the existing tests as part of continuous integration.
Any suggestions?
The best way to do this is with suite tags. You can mark tests with a tag from the properties page and then you can filter for the or filter to exclude them.
In this case I would exclude with "NotOnCI" tag. Then add the following argument to the URL:
ExcludeSuiteFilter=NotOnCI
This might look like this then as the full URL:
Http://localhost:8080/FrontPage?test&ExcludeSuiteFilter=NotOnCI
You can select multiple tags by splitting with commas, but they act as "or",Not "and".
Check the FitNesse user guide for more details. http://fitnesse.org/FitNesse.UserGuide.TestSuites.TagsAndFilters
Would it make sense to have multiple Suites, one for regression tests that should always pass, and another one for the tests that are not yet implemented?
Testers and BAs can add tests/suites to the latter suite and the CI server only runs tests in the former suite.
Once a developer believes he has implemented the behavior they can move the test/suite relating to that functionality to the 'regression' suite so that it will be checked in continuous integration.
This might make the status of a test/suite a bit more explicit/obvious than just having a tag. It would also provide a clear handover from development to test/BA to indicate the implementation is finished.
If you just want to have a test/suite not run during an overall run of a suite that contains the particular test/suite you could also just tick 'Skip (Recursive)' in the properties page of that test/suite (below 'Page Type').