We have whole ui test suite written in Espresso for our app, we are trying to figure out best service to run this test on whole matrix of device and api's
I came across few services:
TestDroid - Looks like runs at about $49/mo and supports espresso tests, I have been playing around with their free account, it works nice, but I am wondering how well they support CI
Test Object - I guess you cannot use the trial version for automated test and it starts at about $199/mo. There manual test featured looks nice, I am just wondering why is this so expensive?
AWS Device Farm/App Thwak - Look like its really new and I didn't see any support for espresso tests.
Any input would be great in making decision on which service to go with?
I'm currently using AWS Device Farm.
The Android test system has been updated to be based on JUnit 4 during summer 2015 and the new implementation is working well on AWS.
Be careful, to make it work on AWS Device Farm, all your test methods have to start with "testXYZ()" or the farm will not detect them. and your test classes have to end with "Test".
There is a AWS Farm Device available with Jenkins and Gradle (for Android Studio integration). Also Spoon screenshot are embedded within AWS Device Farm test results.
Related
We create iOS and Android apps that are white-labeled. They all use a single code base (one for iOS and one for Android). Whenever we need to make changes to all of our apps (> 100 live in App Store) we rely on Fastlane. We have a "bulk" command that submits each new build to Apple, changing out config variables first and a few files so each app is unique.
This has worked well for us... but... its getting really slow. We'd love to be able to take advantage of some of the continuous development services out there. It seems like they weren't necessarily made for this use case but it might still work?
Ideally instead of running bulk on a local machine we could spin up 100 instances on something like CircleCI and they all run side by side, using our fastlane script to build, submit, etc.
We started by looking into CircleCI. The problem we are running into is they don't allow injection of variables into a job (https://ideas.circleci.com/ideas/CCI-I-690).
Is there a better service for this goal? Is there a tool that was built to achieve this? Struggling to find an alternative to hacking together a bunch of smaller tools.
I think you already identified your first step: You will have to split your fastlane (and other tooling) configuration, so it is possible to build each app in isolation.
Then you can trigger a job for each app on a CI service like for example Travis CI or Azure Pipelines (both have a simple API you can use to start jobs and give them some parameters that will be available to your job) that builds and releases the app.
All the other things (e.g. one big build vs. many small build steps etc.) are just implementation details and will depend on the individual service or tools you choose.
I am new to continuous integration. I have developed windows application and its smoke testing using coded ui. I want to create build and release process in TFS and want to execute coded ui tests bases on every time new build is created. I would like to know if there are any limitations for coded ui when it comes to continuous integration? I am not able to find any article so far which explains about behavior of coded ui in TFS continuous integration
Your guidance in this regard is highly appreciated.
First off, it's worth noting that Coded UI is effectively deprecated; Selenium is recommended for web apps, and Appium is recommended for desktop apps.
That said, you need to use the Deploy Test Agent task to deploy interactive test agents to your target machines, followed by a combination of the Windows Machine File Copy (to deploy your tests to the test agents) and Run Functional Tests task to kick off your tests.
What I need Actually?
We create the iPhone application for Mobile & iPad and the code is always checked in to the repository.
1) When ever the code is checked in to the code repository, that has to under go the automation testing and confirm the build does not failed or the app itself will works as per teh functional test scripts.
2) If there is any Build failure, mail has to be triggered to the developers.
3) The build is sucess and automation scripts are executed and that is also passed, next step is to deploy to the apple store and submit for review, necessary information for apple store is made available in configuration files.
Existing reference in stack overflow:
Continuous Integration for Xcode projects?
**Reference**: http://stackoverflow.com/questions/212999/continuous-integration-for-xcode-projects/17097018#17097018
Continuous integration for iphone xcode
**Reference**: http://stackoverflow.com/questions/1544119/continous-integration-for-iphone-xcode
Some of other references also was checked, which just give me the idea of how to execute functional script during code checkin, which is actual works like any CI tools likes Jenkins etc.
Above said reference are also discussed during 2009/2013, which are evry old.
What is available when researched?
I came to know about using using Hudson on the mac, which is very old version and not much supportive and also found Xcode OS X Server which is a product of apple itself where the reviews are not good and implementation is not feasible for my requirement.
Please share me the the approach of how to achieve this, also is that is possible to do CI process a one touch go for IOS, I found something similar to android with few confirmation from user.
At-least execution of Tests and creating an .ipa file in ios will be great.
How does CircleCI and other CI tools help ?
I am not able to fully understand the internals of these tools and how they help with faster deployment of apps.
Are these tools useful only for github based open source projects ? Since the testing requirements for every app is different how is it possible to seamlessly automate it using the CI tools ?
The reason you are using Continous Integration (CI) is to have a well defined build system and always a releasable latest successfullbuild.
You can also integrate unit tests or integration tests.
I think it is not only useful for github-based projects, but also for projects with projects where more developers develop in parallel.
For more information: Wikipedia: Continous-Integration
CI is a developer practice which allows them to integrate code into their shared repository several times a day along with other developers in their team.
Each time a developer checks-in code its been verified auto-build and detect errors early.
It solves issues like
long and intense builds
spend less time in debugging
build more features quickly
How it works
CI server monitors for any changes in the code repo and starts building unit & integration tests. It assigns label for each version of build and also informs the team for a successful build or if it fails, the team fixes the issue and again starts to integrate.
I am currently working on a distributed system consisting of two Grails apps (3.x), let's call them A and B, where A depends on B. For my functional tests I am wondering: How can I automatically start B when I am running the test suite of A? I am thinking about something like JUnit rules, but I could not find any docs on how to programmatically start/manage Grails apps.
As a side note, for nice and clean IDE integration I do not want to launch B as part of my build test phase, but as part of my test suite setup.
A couple of months later and more deeply in the topic of Microservices I would suggest not to consider system tests as candidates for one single project - while I would still keep my unit- and service-level tests (i.e. API testing with mocked collaborators) in one project with the affected service, I would probably spin up a system landscape via gradle and docker and then run an end-to-end test suite in the form of UI tests.