I need to run UI tests (using XCTest framework) to test app behavior that gets triggered after some number of days passes. Obviously, I cannot let the test actually wait for days naturally, so I need to trigger the event by moving the date forward. Is there any workaround to programmatically change the iOS simulator's date?
UPD:
I am aware of host machine time change approach, but it's not an option.
Related
I'm a pretty new DevOps engineer, and i mostly deal with CI processes.
I'm wondering if it makes sense to define both nightly build And build on each push.
Seemingly, it doesn't make sense, since if the code is built after each push, why do you need to build at midnight, it was already built when you pushed it to the repository.
Am i right?
IMHO you are right - it does not make sense to have a fixed time schedule if at the same time you have a push trigger.
A reason why you still want to have a nightly build (or other fixed schedules) could be if you cannot run a full test with every build.
For example you could decide that you only do minimum tests (or smoketests) with every push triggered build, but once per day (e.g. night) you do a full test run.
As far as I know, the advantage of midnight build is that tasks with long running/deployment time can be run at midnight.
After these tasks run at midnight, you can directly view the results the next day.
In this case, you can set the condition for a specific task to control whether it runs at midnight. You could use $(Build.Reason) to judge it.
On the other hand , we recommend that you can set a specific time schedule trigger.
CI triggers cause a pipeline to run whenever you push an update to the specified branches or you push specified tags. The build is only triggered when the code changes.
Changes in the pipeline itself and the operating environment will not trigger the build.
But they can sometimes determine whether the project can run successfully.
In this case, the schedule trigger can run the build at a specific time to ensure that the project is executable.
I will share what we do and maybe help you:
We have three build tiers, one to cover a case like the Push example you pointed, other with a set of PowerShell tests, and a Scheduled one with full set of tests that takes around 5 hours.
As you can picture each case have their one scenario based on time a number of tests.
When running a UI Test, how do I keep the simulator open so I can manually test additional steps?
After a UI Test completes, the simulator will shut down. I'd like to use the UI Test as an automation to reach a certain point in the app. Then I'll do a few additional things not covered by the test.
I don't think there's an option for that. You should separate your automatic and manual testing. Automatic testing ideally should be done on a CI. You should do your manual testing separately from UI tests.
The Simperium Android Github tells how to run the Android tests, but I can't find how to run the iOS tests. I tried opening Simperium.xcodeproj but Product->Test is grayed out.
Eventually I'd like to write my own unit tests that use Simperium, but I thought I'd start by looking into how Simperium structures their tests.
Thanks.
The process you describe adds Simperium's Integration Tests target to your own app's schema.
Normally, you would want to switch to the 3rd party library scheme first, and run the tests right there. To do so, please, click the Scheme picker (right by the Play / Stop buttons), and select 'Simperium'.
Make sure to select a simulator as well, since Tests are not supported in the real device.
Regarding failures, the Integration Tests simulate real interaction with the backend, and have several Timeouts.
Would it be possible you're running them on a slow internet connection?.
Thanks!
I figured out how to run the tests. In Xcode, I selected the Integration Tests scheme and edited that scheme. I selected 'Test' on the left side, then clicked the little plus at the bottom of the main pane. I added the 'Integration Tests' target. The list of tests to run appeared in the pane, and Product->Test could then be used to run the tests.
Unfortunately, 9 of the integration tests failed when I ran them.
i have build a new target on XCODE for my IOS app that contains code for UNIT TESTING, it works great when i am running the test when i am selecting the scheme and run a TEST. now i want this test to happen every time a build is happen. and every time i am pulling new code from git.
up until now iv'e understand that this is actually possible when working with a GIT and want to have control every time a new code is checked in.
anyone have some knowledge ?
Hi Automation/iOS Experts,
We have launched a new iPhone app project recently and would like to automate some basic acceptance tests using Apple's UIAutomation Instruments. Now we have a very basic framework for this task. This framework simply encapsulates the underlying JS functions provided by Apple in Java(to provide some debugging abilities) and drives the tests by Junit. The tests run in iPhone Simulator.
So the background is Instruments + Eclipse + Java + Junit + iPhone Simulator.
In writing the tests, I found the tests are greatly affected by the app's "states". For example,
The app shows some kind of "Terms of use" page when first run, but never again until the iPhone simulator is reset. After the user accepts the "Terms of use", she will be taken to a "Home" page, where she can input some search criteria and hit "search" and taken to search result page. Then she can be taken to a "View detail" page.
TOU -> Home -> Search Result -> View Detail.
Please keep in mind this is only a very simplified version of my real app.
Now here is the question:
In order to automatically test the view detail function, shall my test go through all the previous steps(assuming that the app is always started afresh without any states saved)?
or should my tests assume some pre-conditions(like "View Detail" tests assuming that my app is at "Search Result")?
Please give your reasons. And sorry if my English is hard to understand as it's not my mother tongue.
Thanks!
Vince
"Pre-conditions" / "known baseline" / "known state" are golden for automation. Even more so for UI testing since they have more variations that can cause test failures for issues unrelated to what your test was doing.
So from an automation perspective - go directly to the 'View Detail' test. The majority of your automated test scripts will be in those types of functional areas. TOU, etc. are one-time per reset/install. So two options:
First run an automated script to clear the TOU and exit, followed by all other tests that deal with home page, searching, etc. Or...
Clear the TOU manually, followed by all other tests.
Bonus option: you could also test that the TOU doesn't appear more than once per reset, since it shouldn't. That could be first and second test you always run each time. Then run the remaining tests.
If your automated always rely on the TOU appearing, then after the first test, the others will fail since the TOU won't appear until the next reset/test cycle. You could put a 'handler' at the start of all your automated tests to conditionally manage the TOU page. In this situation, I would go with option #1 above.