I am setting up the integration test framework for a Java rest api in our project and we want to run integration test in the gitlab pipeline. Since these tests are running in the same project as the API, we are wondering couple of things:
We dont want to run Karate tests during the maven build process. We want to run them only at integration test stage after the application deployment stage is complete. How do we do that as the maven build process runs both the junit unit tests and karate tests.
Since the API requires authentication, we need to run the karate test in a docker container, since we can inject our credentials only in the container as we are using hashi-corp vault to store the credentials. How do we launch a container with Karate client.
There are ways to run only a subset using Maven. What I do is define a different JUnit test, and call that from the command-line. Read the docs for more: https://github.com/karatelabs/karate#command-line
As long as you can pass environment variables (which you certainly can in Docker) you are good. Refer: https://stackoverflow.com/a/52821230/143475
Related
I am writing a client SDK library and use bazel as the build system. I want to write a test which spin up two docker instances. One is a server, which can accept client requests. And the other is test binary of SDK library. So this is an integration test. My goal is that this test can be run with bazel test //sdk/gtest_binary. I know that bazel has a rule called sh_test, which can be used to trigger docker compose to start the two docker instances. I am wondering if this is achievable and is there a better way to run such integration tests? Should I use bazel docker rule?
I created a VueJs project with some unit tests (using Jest) and integration tests using Cypress.
I have also a Jenkins pipeline in order to build, test and deploy the application.
I Integrated, as test stage, the unit tests but I would like to integrate also Cypress in order to run the integration tests into a dedicated pipeline step.
Is is possible to have this without installing any additional Cypress Jenkins plugin?
I mean, Is it possible to use a docker image to run the tests using Cypress?
Can you point me to some examples?
You should be able to use this with Jenkins Docker capabilities. Also here is another example you can refer to.
Our requirement for API testing is:
To deploy test-automation module (Karate feature files, custom java classes) into AWS ECS-Fargate cluster.
Trigger the tests via Jenkins pipeline after every build of the actual microservice.
In addition to above, test-automation module should be triggered to run test suite on-demand and/or at scheduled intervals (say nightly) and send reports.
I have gone through Karate Distributed Testing and stand-alone executable jar options, but doesn't seem suitable for my case. Is Distributed Testing supported only for "Web-UI" automation testing?
Any thoughts would be helpful.
For this use-case, just use a Maven + JUnit project and then there is no difference between Karate and any other Jenkins Java pipeline.
It should be Jenkin's responsibility to do a scheduled build. It is up to you how to get all this into Fargate, maybe building a Docker container is part of the answer, but I would recommend trying to keep it simple.
Here is some Docker related discussion that may help: https://github.com/intuit/karate/issues/396
Open a new question with specifics next time.
We currently have coded UI regression test suite (UI function test) which runs perfectly using MTM on test anent/controller.
As part of CI our team is using Jenkins. Here we want to integrate coded UI test with Jenkins. Can you please tell me how can I configure this?
Is there a way Jenkins will access the TFS build of my coded UI test project which will have test run setting file (with info of test controller and agent to run test)?
I don't want to run coded UI tests on Jenkins server (but on test agent) as it will need to have a active desktop session.
A great example of running UI with Jenkins is executing selenium tests. This approach suggests you to run browser and simulate user actions. The same you can do with your UI and basically this does not require any additional configuration steps except slave configuration. You need to use Slave Configuration via Java Web Start as only this gives you ability to interact with Desktop:
If you need display interaction (e.g. for GUI tests) on Windows and you have a dedicated (virtual) test machine, this is a suitable option.
Other slave configuration ways may just not run your UI.
I am using Jenkins for my project CI. And several automation test cases developed by Coded UI(C# language) will be run when the new build is deployed. I created a job on the master(windows) that MSTest those test cases(note: with MSTest under cmd, test cases can be run successfully) but saw the below issue:
Error calling Initialization method for test class QuickUI.SmokeTests: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=159018)
Therefore,
1) Need I use Jenkins slave to run test cases?
2) If so, how to config since the dlls needs to be copied to slave firstly?
3) Is there any account or jenkins service configuration needs to be taken care?
Thanks in advance.
Sounds like you need to get mstest setup on your build machine. The easiest way is to use test agents (http://www.microsoft.com/en-us/download/details.aspx?id=38186) when installing chose test agents.
(below is copied from my answer in Coded UI build server integration process)
You going to want to put your CodedUI tests inside an orderedTest. Right Click on the solution -> add -> orderedTest.
You going to want to install https://wiki.jenkins-ci.org/display/JENKINS/MSTestRunner+Plugin it's not a requirement, but it makes working with mstest in jenkins a little easier.
At this point you just need to configure the plugin in jenkins to run your orderedTest. It will need you to point to mstest and the location of your orderedtest.
1) Recommnd to run test on slave
2) To run GUI test, don't run the slave as windows service
3) Theoratically you only need the slave agent running. However the windows account logged in should have access to all the resources that your tests need.
Some other threads you could reference, pay special attention to this if you want to run the tests in unattended way.
Jenkins on Windows and GUI Tests without RDC