What is the cleanest way to Run Test1.java and Test2.java via ANT in Parallel taking into consideration that Test1.java and Test2.java have one test method inside. And what if the scenario was one class with multiple test methods? how to dynamically select which ones you want executed? and have reports for everything executed. I have seen this everywhere and have not seen a solid answer pointing to these problems.
Related
When I am running my test solution, single browser is getting launched but it is running two feature file simultaneously due to which test cases are failing. One step it is taking from one feature file and other from other feature file.
Contrary to the comment left on your question, I think I may have enough context to answer you.
You describe feature files that are sharing steps and concerns about multiple browser instances. This tells me that your various step files might each be containing a browser instance.
What you're likely looking to do instead is to use a SpecFlow Context -- SpecFlow provides it own ScenarioContext object you can use, or you can create your own context and inject it.
Some links that might help:
SpecFlow docs on sharing data between bindings, which explains about ScenarioContext and FeatureContext:
https://docs.specflow.org/projects/specflow/en/latest/Bindings/Sharing-Data-between-Bindings.html
Here's an article on using SpecFlow with Selenium and the Page Object Model: https://docs.specflow.org/projects/specflow/en/latest/ui-automation/Selenium-with-Page-Object-Pattern.html
The SpecFlow YouTube channel will likely be helpful as it's full of experts walking through these sort of examples: https://www.youtube.com/c/SpecFlowBDD/videos
Here's the first video in a 5 part series on how to automate a web application with Selenium and SpecFlow: https://www.youtube.com/watch?v=y1dAogvWVh8
Based on your question, it's also possible that your issue could be that you want things to run in parallel, but have the problem of your tests being dependent on one another or running in a certain order. This will be a bit more complex to solve.
I strongly suggest you treat your tests so that they can be run in isolation. You may need to add separate data to a database, or operate your tests so that they're not touching the same thing. This takes more work, but is more than worth it, because it will enable better maintainability and reliability of your tests and also ensure they can run in parallel successfully.
I hope this helps!
On Rails, each test case creates an ActiveRecord transaction, which allows to test everything and then revert the database to the original state, without having to drop all tables, or anything like that that might affect seeders, etc.
Is it possible to do something like this on TypeORM?
From what I've seen, the main issue with the way transactions are documented to work is that a call to another method would not be using the created transaction, but I'm hoping I'm missing some other way of implementing it.
Thanks!
I had exactly the same expectations as you. Coming from Rails and Spring, I expected to have transactional tests and found no solution directly from Typeorm.
It is hard to reuse the same transactions during the tests because the connection class always create a new QueryRunner for every database command or transaction. Diving into TypeORM, the solution I found was to monkey patch the method which creates the query runner, to reuse it during the tests. I created this library to reuse this code in several projects: https://github.com/viniciusjssouza/typeorm-transactional-tests.
I know this is quite late, but I also actually worked on a solution which you can see here: https://www.npmjs.com/package/typeorm-test-transactions
A disclaimer from my side is that you have to use the #Transactional() decorator, but I like how that makes the code a lot cleaner and you don't have to pass the transaction manager down.
#viniciusjssouza I checked your solution and I really like it! It's funny, I think we both had the same problem at the same time :P
You might consider an alternative approach. Instead of isolating tests through transactions you could isolate them through serialization + multiple databases.
At a high level the approach is as follows:
Split tests into N groups where each group gets its own database and N is roughly the number of cpu cores you have.
Within each group tests are run serially. Tests also reset the database when they start.
Each group runs in parallel to each other.
This approach is remarkably easy to set up with Jest and Docker and allows you to achieve a high degree parallelism.
I wrote a blog post describing this approach in more detail here:
https://blog.mikevosseller.com/2021/11/25/how-to-run-jest-with-multiple-test-databases.html
I'm developing an application in rails with cucumber.
The application includes a workflow that have multiple steps.
Some steps are
A user import files (3 different files),
Other user make make some checks to date that was imported,
Other user input some parameter,
Other user apply the parameter to the data that was imported,
etc.
The steps must be executed in the correct order, and I is necessary to run all the previous steps in order to execute each one, for example to apply the parameter its necessary to have the data imported and the parameters defined.
My problem is how to build cucumber scenarios/features in this situation.
I know that a scenario is not suppose to call all the previous scenario.But the only other idea that I have is to create a very long scenario performing all this steps, and that make sense because it will be a scenario with more than 2 hundred steps.
Any thought on a pragmatical way of implementing cucumber in this kind of situation?
Many Tks
It sounds as if you have to perform every thing every time.
Will every usage of your system include importing three files? Are there any cases where the user may only need to import two files? If the case is that there will always be three files imported, then you might abstract that step as
given the files are imported
Things that always have to be done may be combined into some generic setup. As the setup never changes, the details may not be necessary to mention explicit.
My experience though, is that at the beginning it is hard to separate scenarios and try to do too much in a few scenarios with many steps. If you don't see any other way, start there. Look at your scenario and see if they possible to separate into perhaps two independent scenarios. It may be possible to separate it into two scenarios that are independent. Next step would be to see if these two new scenarios are possible to divide into two smaller, independent scenarios. It happens that it is possible.
It is obviously always possible that Cucumber is not the tool you need. It is possible that you would be better off with a unit test framework.
I am wondering if anyone's ever tried tagging scenarios on the fly.
Here's the use case:
We have hundreds of scenarios for the regression tests, and some of the tests might fail because the API is down (which usually means that the next time it runs, this should pass), or the data changed (which means that either the scripts are not robust enough and we need to fix it, or we need to change the data), or the requirement changed (which means that we need to change the scripts).
For the latter 2 cases, the same scenario should fail multiple times.
We need to tag the ones that require human intervention (either rewrite the scripts or change data) with #quarantine, and take out the #regression tag so that it wouldn't be run over and over while we know it would fail anyways.
I've not seen anyone does this. Is this doable? Or is the only way to do this in Cucumber without resorting to complicated shell script?
Right now, I ended up using the hooks, in the "after" scenario part, to modify the feature file directly. A bit crude, but it works for now.
I am still looking for a better resolution.
I am trying to figure out a way to execute certain integration tests against an in memory DB (H2) and others against our Oracle test DB. Maybe its my limited test writing experience but it seems that some tests (such as search querying) are more suited for in memory as I can control the data set queried, and others such as testing transactions/persistence would benefit from going against our REAL schema and DB (Oracle).
I can think of 2 approaches but do not know how to implement either:
add a new test phase so that I can have integration-test-in-mem and integration-test (using oracle) and have different tests run in different phases and configure each for the different DB
have each test control which datasource is used
I would prefer the first as its cleaner and I don't have to pollute my test with logic to control which datasource it uses.
Also, the second is not simply setting different datasources by domain - I want to reuse the same domain in different tests against different DBs.
Any ideas appreciated and if you've done this please share! We do use SPOCK.
Here is a blog article I've found on adding custom test phases/types by Luke Daley. Has anyone implemented this? Now that I've read that and understand terminology better I think what I would like to do is set up new types - not phases. Unfortunately though since we are using spock we are already basically using a custom type. Though we could leave spock as one of the 2 types and potentially create a 'SPOCK-IN-MEM' type although this may require redefining the spock type which might not work. Any advice welcome. I would say that this seems to come up often enough (I've sen this question asked by others in other forums) that there should be a simpler way to go about it.
One more finding. There is an environment plugin for spock which adds an annotation to have tests run ONLY for the environment annotated. Its reusing the ignored tests capability of spock and is quite small, simple, and clean. The only downside is its for spock which is not an issue for our group.
A simpler way of defining phases would be nice - like a naming convention. It would be nice to be able to define phases/types with just a directory naming convention such as test//. Just create the folders and away you go. Then you could control execution by just explicitly setting phase/type/env in args when running test-app.