Since updating my gems, one of my cucumber tests appears to be failing i.e. in the one-line printout of test outcomes it prints a red F:
However the test summary shows that all the tests passed which seems to contradict the red F.
What's a good way to try and unwrap what's going on here and to get some output from that apparently failing test?
Run your tests with cucumber --format=pretty ... and check out the full output
Related
I'm using hsac-fitnesse-fixtures for FitNesse tests.
When I started working on negative API tests I noticed FitNesse test suite fails if JSON HTTP Test fixture receives 4xx/5xx response, while no assertions were made in the actual test:
Adding assertions where we're actually waiting for 4xx/5xx response code does not make the difference: the assertion passes, but the test still fails.
I'm trying to find out how to make such negative tests pass. Any thoughts?
It turns out that it's necessary to add |reject| keyword before |get from| or any other HTTP action in hsac-fitnesse-fixtures to perform a negative test.
More details can be found in project's GitHub Issues and FitNesse Script Table Keywords.
When running minutest tests, is it possible to peek at the information about the errors that has happened?
For example, this test suite takes ten minutes to complete. But I would like some more info about the letter E appearing in the tests result.
I don't want to wait ten minutes.
*** Running FRONTEND component engine specs
Run options: --seed 29704
# Running:
......................................................................................................................................................................................E...........
That's E for "error", so one of your tests is failing. Normally you get output that explains more. Once you identify which test is failing you can run that test in a more focused capacity, like:
ruby test/unit/broken_test.rb --name=test_that_is_broken
Where that is the path to your test script and the name of the testing method that's failing.
You may need to make your tests self-contained, able to be run this way, by using:
require_relative '../test_helper'
Or whatever the helper stub is that kicks off the testing framework. Some skeleton files contain things like require 'test_helper' which won't be found in your current $LOAD_PATH.
I am writing Rails tests using the standard Test::Unit/TestCase.
Is there any way to somehow filter what gets printed to the log, so that you only print the stack for specific test cases.
I have a functional test file with many test cases in it, and I'm really only interested in debugging one test case. Printing my own log statements still requires searching through a few thousand lines of generated log. Or something similar to the RSpec 'pending' functionality.
Run from a command line ruby test/unit/my_model.rb to run one test suite. You can also use a debugger, such as (wrapped by) RubyMine or pry, to stop on a specific test case and look at the log.
But if a sledge-hammer does not solve the problem, you can use tweezers: config.logger.level = Logger::WARN in your test.rb, from Set logging levels in Ruby on Rails
It is probably better if instead of strangling the output to log/test.log, you become familiar with a command such as grep. Grep allows you to run very advanced search queries through files or directories, as long as your running on some flavor of *nix. The simplest use would be
grep search_term file_name
The reason I say you shouldn't constrict the log output is because someday that could bit you in the **s. Hope this helps.
Is there a way to basically mark a TestUnit test as a "todo"? I thought I could do it by using pending at the top of the test, but that still runs the test (and thus it's red). I'd like to basically do the same thing, but not actually run the test (just skip over it).
Does anybody know of a way to do this?
Thanks!
if these are test case stubs, you could just put assert true as the only assertion and a puts in the test to describe what needs to be done. this of course wouldn't help if your tests have failing assertions unless you comment them out.
In a Rails application I have a Test::Unit functional test that's failing, but the output on the console isn't telling me much.
How can I view the request, the response, the flash, the session, the variables set, and so on?
Is there something like...
rake test specific_test_file --verbose
You can add puts statements to your test case as suggested, or add calls to Rails.logger.debug() to your application code and watch your log/development.log to trace through what's happening.
In your test you have access to a bunch of resources you can user to debug your test.
p #request
p #response
p #controller
p flash
p cookie
p session
Also, remember that your action should be as simple as possibile and all the specific action execution should be tested by single Unit test.
Functional test should be reserved to the the overall action execution.
What does it mean in practice? If something doesn't work in your action, and your action calls 3 Model methods, you should be able to easily isolate the problem just looking at the unit tests. If one (or more) unit test fails, then you know which method is the guilty.
If all the unit tests pass, then the problem is the action itself but it should be quite easy to debug since you already tested the methods separately.
in the failing test use p #request etc. its ugly, but it can work
An answer to a separate question suggested
rake test TESTOPTS=-v
The slick way is to use pry and pry-nav gems. Be sure to include them in your test gem group. I use them in the development group as well. The great thing about pry and pry nav is you can step through your code with a console, so you can not only see the code as it's executed, but you can also enter console commands during the test.
You just enter binding.pry in the places in the code you want to trigger the console. Then using the 'step' command, you can move line by line through the code as it's executed.