I want to test the contract of my API so if, for example, an object is created with some parameter to nil an assertion is raised, but if the assertions are disabled (Release config) it simply returns nil.
My test scheme has the build configuration to Debug, so in my API contract tests I check if the creation of an object with some parameters to nil returns a nil object. But the assertion of the constructor is raised before the XCTAssertNil is executed, so my test always fails.
Can I disable the assertions while testing? I tried to add NS_BLOCK_ASSERTIONS to the scheme arguments passed on launch but that doesn't work.
Solved, I added a new configuration, duplicated from Debug, called Test.
Then added NS_BLOCK_ASSERTIONS=1 in the build settings preprocessor macros.
Finally, change the test action in the test's scheme.
Now it works :)
As explained in your self-answer, it is useful to create a configuration dedicated to unit testing
In Objective-C you'll need to play with NS_BLOCK_ASSERTIONS and the more recent ENABLE_NS_ASSERTIONS indeed.
For Swift assertions, I found out from Apple Developer it depends on the compiler optimization level you chosed.
With -Onone the assertions will be triggered, but not with -O
Related
I have several assert(condition, "message") statements in my project.
They are used to check invariant conditions during development. I thought they would be ignored in production/release build (as stated in this answer). They are not. Instead they cause crashes during TestFlight testing. When I comment asserts the app does not crash. Something usually gets wrong a bit but it does not crash.
Can it be something with my build settings?
All my archive schemes use release configuration:
The asserts are in Cocoa Touch Framework project, that is used from custom keyboard extension.
All the targets in all projects (Cocoa Touch Framework, and the main project with keyboard extension target) have these Build Settings:
Enable Foundation Assertions
Debug YES
Release NO
Disable Safety Checks NO
What's wrong?
EDIT:
Sulthan's answer shows how to disable asserts globally for both debug and relase builds. That is not what I need. I want it to work as expected - asserts should be enabled in debug but disabled in release builds.
By default it works that way - and it also works that way in my main project. But it does not work for asserts located in Framework project that is linked from that main project (details in this question). Why? How to fix it?
The options you have tried:
Enable Foundation Assertions is in the preprocessing section (Macros). Swift is not preprocessed and does not use macros. This option disables NSAssert, NSParameterAssert and similar macros commonly used in Objective-C.
Disable Safety Checks is a performance option:
By default, the standard library guarantees memory safety. Many functions and methods document the requirements that must be satisfied by the caller, such as an array index being valid; memory safety is guaranteed even if a requirement is violated. However, violating a requirement can trigger a runtime error. APIs that include the word “unsafe” in their name let you explicitly disable safety checks in places where you need the additional performance. It’s your responsibility to verify the memory safety of code that uses unsafe APIs. Memory safety is also not guaranteed if there is a race condition in multithreaded code.
(Swift Library Reference)
You should probably try my answer here
(use -assert-config Release in Other Swift Flags).
Or just keep the asserts in production builds. Every failing assert is a bug and in general it's better to know about a bug as soon as possible.
So, i have already existing project at my hands and i’m trying to create some UI tests by using this new fancy UI Testing Bundle provided by apple. The problem is that test target doesn't have access to any external framework (and i need to do some setup with one of them). Adding framework in build phases and coping framework search path from main target doesn't do anything.
After day of browsing i found out only one thing, that ”makes things kinda different”. By setting up Bundle Loader and Test Host to $(BUILT_PRODUCTS_DIR)/App.app/App , i still couldn't import external frameworks to test.m, but i could import classes that do that for them self. And it all would be fine and dandy unless it didn't break some stuff. By setting Bundle and Host now my UI test is unable to execute launch method:
[[[XCUIApplication alloc] init] launch];
It crashes with error: Assertion Failure: UI Testing Failure - App state is still not terminated.
In the end i could remove launch method from setup and trigger every single test manually, so it restarts application every time before executing, but this solution seems so wrong (especially for some bigger projects). Does anyone know proper way to handle this problem?
What I've done for this is add an environment variable to XCUIApplication to specify UI tests are being run. I then have a pre-processor check for #DEBUG in a main part of the application, and then check if the test environment variable has been set; if it has, do the necessary steps for the UI tests.
Essentially, this will allow you to configure your app to how you need it for the UI tests to run. It also means the pre-processor check will strip out that setup code entirely for the release build.
I'm reading the Assertions section in the Swift e-book and it looks like assertions work very similarly to their Objective-C counterparts. However, nowhere in the docs can I find anything about runtime behaviour while running as a production app. Objective-C's NSAssert promises never to terminate a production application as a result of an assertion failure. Is it the same case in Swift?
Based upon the language Apple use in their documentation, I'd say assertions are ignored in a production environment.
If your code triggers an assertion while running in a debug
environment, such as when you build and run an app in Xcode, you can
see exactly where the invalid state occurred and query the state of
your app at the time that the assertion was triggered. An assertion
also lets you provide a suitable debug message as to the nature of the
assert.
And in the "Note" block:
Assertions cause your app to terminate and are not a substitute for
designing your code in such a way that invalid conditions are unlikely
to arise. Nonetheless, in situations where invalid conditions are
possible, an assertion is an effective way to ensure that such
conditions are highlighted and noticed during development, before your
app is published.
The difference between debug and release is the difference between compiler arguments. The most likely answer is that there will be some special compiler settings (similar to -ea in Java).
EDIT
The Swift compiler has an argument called -assert-config
-assert-config Specify the assert_configuration replacement. Possible values are Debug, Release, Replacement.
In Release, the assertions are ignored. Not sure about the difference between Debug and Replacement.
Check your Optimization Level and make sure it is not Onone for release configuration. See my note https://github.com/onmyway133/blog/issues/39
Asserts are documented along with preconditions in the Swift standard library documentation.
Debug -> Stop on assertion failure.
Release -> Compile with assertions ignored
Release and "Disable safety checks" -> Assume all assertion statements are statements of truth and hints to the compiler so that following and preceding code may be removed if only reached in conditions where the assertion would have failed. This means that if you follow the assertion by code on handle the exceptional case it may be ignored. If the assertion would ever have failed the behaviour is completely undefined.
I've not checked but the "Disable safety checks" may correlate with the -assert-config Replacement that #Sulthan mentions.
How do I make the test host/bundle loader dynamic based on the current scheme? Right now the value is set to:
$(BUILT_PRODUCTS_DIR)/MyApp1.app/MyApp1
The problem is I have 4 apps in the workspace and I would like to use the same unit testing suite for all of them. How do I dynamically change the "MyApp1" part based on the current scheme? Is it an environment variable based during build? I tried setting it to things like $(PROJECT_NAME) but those seem to get the name of the test suite.
To do this you need to have a variable inside the build settings - which seems simple, but it's not. If you set an environment variable through a Pre- or Post- step in the application or test scheme, it does not seem as though it will be picked up here. The build settings, after all, happen before the build. Same is true of a preprocessor macro, though doing this using xcodebuild and passing in a custom option may work.
The only way I know of to do this is to use an xcconfig file. Create one and apply it to (at the least) your test target. The content should include something like this:
THINGUNDERTEST=FooBar
Now in your project settings, wether in an xcconfig or the project file, set BUNDLER_LOADER to:
$(BUILT_PRODUCTS_DIR)/$(THINGUNDERTEST).app/$(THINGUNDERTEST)
That will work. Now you can change THINGUNDERTEST through various means and get at least some dynamic behavior. This may work for you or may not, depending on your needs - but it probably only a starting point.
I've found plenty of people using Xcode 3 to do this, but it no longer seems to work in Xcode 4. Or, it works partially…
I've added a preprocessor macro "TEST_TARGET" for Debug and Release under my Test target, and if I use #ifdef TEST_TARGET in the actual unit tests, that works as expected.
However, I really want to log some extra information from a source file that's just part of the main app when it's being run under unit tests (i.e. just a standard source file, not a unit test file). That source file doesn't appear to "see" the define. I've stepped through with the debugger, and the code within #ifdef is never executed.
Is there a way to tell my app is being run under a unit test target?
Here's how you can do it: you can test for something that's loaded when the tests are loaded, but not when they aren't.
For instance:
if (NSClassFromString(#"SenTest")) {
NSLog(#"Extra info when running tests");
}
You could also add categories to your classes that are only present in the test target, which might also be helpful.