I am using FitNesse with Xebium for documenting/running automated test scripts.
I was wondering how these test scripts can be improved e.g. by removing duplication:
|scenario |Given a customer check with status positive|
|start app with customerId|1000001 |
|scenario |Given a customer check with status negative|
|start app with customerId|1000002 |
|scenario |Given a customer check with status error|
|start app with customerId|1000003 |
(... and so on)
I have the feeling that this can be put in one table, but how?
I know its an old post, but I believe you can plan your acceptance tests something like below
|start app |
|status |given customer|whatever validation?|
|positive|1000001 |works fine |
|negative|1000002 |can not be launched |
|error |1000003 |can not be launched |
Related
Am using fastlane for my app and right now have configured to generate HTML reports for tests & code coverage.
Is it possible to fetch the stats as seen on command line so that I could use it to notify on separate chat app which I use ?
+------------------+---------+
| xcov Coverage Report |
+------------------+---------+
| 123456789012.app | 100.00% |
+------------------+---------+
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 1 |
| Number of failures | 0 |
+--------------------+---+
You can use tools like xcov and gcovr to gather coverage data on top of fastlane. Links are as follows to the documentation:
https://docs.fastlane.tools/actions/gcovr/
https://docs.fastlane.tools/actions/xcov/
For test results, i believe there is no straight forward answer for that because the logs you see in the console are not directly exposed by Xcode.
When you are running your application, Xcode creates a file called TestSummaries.plist, which has all the steps and results.
Some of the developers were in your position, where they ended up writing a macOS application, just to parse the plist and get all the data. Following is the reference to the project.
https://github.com/nacuteodor/ProcessTestSummaries
https://github.com/KrauseFx/trainer
Lastly, go through the following article which explains how test logs are structured in the testSummaries.plist
https://michele.io/test-logs-in-xcode/
Hope all the links and explanations help you out. :)
I am trying to add a table of variables and its values in FitNesse suite page, so that it can be used for all my tests.
I am using xmlHtttp tests for SOAP web services and fhoeben/hsac-fitnesse-fixtures (slim) for this.
Is it required to write separate fixtures to add a table?
Yes you can.
Using a scenario allows us to generate multiple request, only changing certain values.
!*> Scenario definition
!define POST_BODY_2 { {{{
<s11:Envelope xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/">
<s11:Body>
<ns1:GetCityWeatherByZIP xmlns:ns1="http://ws.cdyne.com/WeatherWS/">
<ns1:ZIP>#{zip}</ns1:ZIP>
</ns1:GetCityWeatherByZIP>
</s11:Body>
</s11:Envelope>
}}} }
|script|xml http test|
|table template |send request |
|post |${POST_BODY_2} |to |${URL} |
|check |response status|200 |
|show |response |
|register prefix|weather |for namespace|http://ws.cdyne.com/WeatherWS/|
|$City= |xPath |//weather:City/text() |
*!
|send request |
|zip |City? |
|10007|New York |
|94102|San Francisco|
In this example the variable zip is used so the request is sent with either 10007 or 94102.
I'm getting a weird error when testing my app using Capybara and poltergeist.
/features/document.feature
#javascript
Scenario: admin users can publish documents
Given a "admin" user exists and is signed in
And the following categories exist:
| name |
| Category 1 |
And the following documents exist:
| name | id |
| Test Document 1 | 1 |
And I goto "/admin/documents"
When I toggle publish document
And I wait for index
Then Document "Test Document 1" should be published
And I should see "Document published successfully!"
Result:
Failing Scenarios:
cucumber features/admin_features/document.feature:135 # Scenario: admin users can publish documents.
I tried to debug and i'm getting the output as expected ie, Document published successfully! but i dunno why the test failed after the scenario is successful. I think its realted to some config issue.
I figured out why the scenario was failing.... Actually i degraded cucumber from 2.0.0 to 1.3.8 and it showed me the exact error. Anyways thanks Mark
The following Gherking test defines the desired behaviour for one of my servers:
Scenario Outline: Calling the server with a valid JSON
Given A GIS update server
When I call /update with the json in the file <filename>
Then the response status code is <status_code>
And the response is a valid JSON
And the response JSON contains the key status
And the response JSON contains the key timestamp
And the response JSON contains the key validity
Examples:
| filename | status_code |
| valid_minimal.json | 200 |
| valid_minimal_minified.json | 200 |
| valid_full.json | 200 |
| valid_full_minified.json | 200 |
| valid_full_some_nulls.json | 200 |
| valid_full_all_nulls.json | 200 |
I wrote this code for unit testing a Flask server. The steps file, which interpret the Gherkin directives, open a test client and make the necessary calls and assertions:
#given(u'A GIS update server')
def step_impl(context):
context.app = application.test_client()
The feature file is similar for unit and functional tests. The only difference is in a few steps file which would have to make HTTP calls rather than calling the test client's methods.
What's the right way to re-use this behave feature file by passing parameters to the steps file?
Expanding on Parva's comments, I suggest sending parameters via the command line which will be detected in your step definitions and adjust behaviour for unit vs functional testing (the decision of whether you separate your unit and functional tests for clarity is up to you ;).
There is an example given in the Behave documentation around Debug on error, which gives a good example of using environmental attributes in modifying the execution of your steps method:
# -- FILE: features/environment.py
# USE: BEHAVE_DEBUG_ON_ERROR=yes (to enable debug-on-error)
from distutils.util import strtobool as _bool
import os
BEHAVE_DEBUG_ON_ERROR = _bool(os.environ.get("BEHAVE_DEBUG_ON_ERROR", "no"))
def after_step(context, step):
if BEHAVE_DEBUG_ON_ERROR and step.status == "failed":
# -- ENTER DEBUGGER: Zoom in on failure location.
# NOTE: Use IPython debugger, same for pdb (basic python debugger).
import ipdb
ipdb.post_mortem(step.exc_traceback)
You may change that to detect a command line passed variable such as UNIT_TESTING and have it hit a different endpoint or perform alternate functionality for your tests.
REQUIRES: behave >= 1.2.5
I think, the test stage concept should help you with your needs. It allows you to use different step implementations for different test stages.
behave --stage=functional
If your changes are minor, use the userdata concept to pass a flag to your step implementation, like:
behave -D test_stage=unit …
behave -D test_stage=functional …
I have a strange problem: My Delphi application raises an EOutOfRessources exception just after starting up on a Application.CreateForm call. Is somebody out there, who solved such a problem?
The strange things are
that this happens on a single machine only. I do not have this
problem on other computers
that the application runs properly, if a teleservice is active (we use Danware NetOP). If teleservice is not running (Netop waits for a guest log in), the application fails.
The application was developed under D7; OS is WinXP SP3.
Thanks for your help
--- Update 1 ---
Application uses EurekaLog to catch exceptions and to store error information. It says, the EOutOfRessources happens on Application.CreateForm (some 50 forms already created, a few other forms pending to create), the message is "out of system ressources". Exception address is 7C81EB2E.
The EurekaLog also provides the call stack :
|*Exception Thread: ID=2088; Priority=0; Class=; [Main] |
|-----------------------------------------------------------------------|
|7C81EB2E|kernel32.dll| | | | |
|77D56C4F|user32.dll | | |CreateIcon | |
|7C9205D4|ntdll.dll | | |RtlAllocateHeap | |
|7C9110ED|ntdll.dll | | |RtlLeaveCriticalSection| |
|77D2058E|user32.dll | | |SystemParametersInfoA | |
|77D205A3|user32.dll | | |SystemParametersInfoA | |
|7C809AE4|kernel32.dll| | |VirtualAllocEx | |
|7C809AA2|kernel32.dll| | |VirtualAllocEx | |
|7C809A94|kernel32.dll| | |VirtualAlloc | |
|0060E359|_765013.exe |_765013.dpr| | |235[58]|
|7C91E64C|ntdll.dll | | |NtSetInformationThread | |
-------------------------------------------------------------------------
Total memory use is about 60 MB; the application has some 20 MB in use.
I do not know the used number of handles; EurekaLog does not provide this.
--- Update 2 ---
Now we exchanged the PC by another one of same type. The exception did not raise again. However, we had a similar effect on another machine, now not being able to open a file during Application.CreateForm. The file name string was empty ... After a number of poor people resets (power shut down) the problem disappeared.
We suspect, that the exceptions are caused by a network problem. At this customer, we have four applications running (per two identical projects). They share data over a company network; for that there is a NAS. Network login is done with Windows start up, about 2 minutes before starting the applications.
Teleservice runs over company network too.
The question is now, if Application.CreateForm is trying to get connected to network. Our OnCreate event handlers do not require an open network.
The applications' source code is on the NAS too (encrypted by TrueCrypt). After compilation we copy the EXE and all other needed files to a local hard drive and run the application from that place. Normally, the TrueCrypt container is closed.
Could it happen, that the EXE is searching for some files on NAS resp. the TrueCrypt container?
Maybe, somebody is familiar with such issues. Thanks for your help.