I'm using hsac-fitnesse-fixtures for FitNesse tests.
When I started working on negative API tests I noticed FitNesse test suite fails if JSON HTTP Test fixture receives 4xx/5xx response, while no assertions were made in the actual test:
Adding assertions where we're actually waiting for 4xx/5xx response code does not make the difference: the assertion passes, but the test still fails.
I'm trying to find out how to make such negative tests pass. Any thoughts?
It turns out that it's necessary to add |reject| keyword before |get from| or any other HTTP action in hsac-fitnesse-fixtures to perform a negative test.
More details can be found in project's GitHub Issues and FitNesse Script Table Keywords.
Related
I am following along with the Ruby on Rails Tutorial by Michael Hartl and am up to the point of running tests. The test is a simple "supposed to fail" test shown here:
test "should get about" do
get static_pages_about_url
assert_response :success
end
with the error being simply that the about page, controller method, and route do not yet exist. However when I run the test I get a massive block of code which fills up my terminal. This is just a small sampling as an example of what i'm seeing.
E
Error:
StaticPagesControllerTest#test_should_get_about:
NameError: undefined local variable or method `static_pages_about_url' for #<StaticPagesControllerTest:0x00007fe6682cc760 #_routes=nil, #NAME="test_should_get_about", #failures=[#<Minitest::UnexpectedError: Unexpected exception>], #assertions=0, #integration_session=#<#<Class:0x00007fe6685342c8>:0x00007fe66859fc58 #_routes=nil, #app=#<SampleApp::Application:0x000055d7735cea68 #_all_autoload_paths=["/home/clay/Programs/sample_app/app/channels", "/home/clay/Programs/sample_app/app/controllers", "/home/clay/Programs/sample_app/app/controllers/concerns", "/home/clay/Programs/sample_app/app/helpers", "/home/clay/Programs/sample_app/app/jobs", "/home/clay/Programs/sample_app/app/mailers", "/home/clay/Programs/sample_app/app/models", "/home/clay/Programs/sample_app/app/models/concerns"], #_all_load_paths=["/home/clay/Programs/sample_app/lib", "/home/clay/Programs/sample_app/vendor", "/home/clay/Programs/sample_app/app/channels", "/home/clay/Programs/sample_app/app/controllers", "/home/clay/Programs/sample_app/app/controllers/concerns", "/home/clay/Programs/sample_app/app/helpers", "/home/clay/Programs/sample_app/app/jobs", "/home/clay/Programs/sample_app/app/mailers", "/home/clay/Programs/sample_app/app/models", "/home/clay/Programs/sample_app/app/models/concerns"], #app=#<ActionDispatch::HostAuthorization:0x000055d774009820 #app=#<Rack::Sendfile:0x000055d774009988 #app=#<ActionDispatch::Static:0x000055d773fe4f98 #app=#<ActionDispatch::Executor:0x000055d773fe5088 #app=#<ActiveSupport::Cache::Strategy::LocalCache::Middleware:0x000055d77382f4d8 #name="ActiveSupport::Cache::Strategy::LocalCache", #local_cache_key=:active_support_cache_null_store_local_cache_4420, #app=#<Rack::Runtime:0x000055d773fe51f0 #app=#<Rack::MethodOverride:0x000055d773fe5290 #app=#<ActionDispatch::RequestId:0x000055d773fe5330 #app=#<ActionDispatch::RemoteIp:0x000055d773fe5420 #app=#<Rails::Rack::Logger:0x000055d773fe54e8 #app=#<ActionDispatch::ShowExceptions:0x000055d773fe5588 #app=#<ActionDispatch::DebugExceptions:0x000055d773fe56c8 #app=#<ActionDispatch::ActionableExceptions:0x000055d773fe5790 #app=#<ActionDispatch::Reloader:0x000055d773fe58f8 #app=#<ActionDispatch::Callbacks:0x000055d773fe5998 #app=#<ActionDispatch::Cookies:0x000055d773fe5a38 #app=#<ActionDispatch::Session::CookieStore:0x000055d773fe5c68 #app=#<ActionDispatch::ContentSecurityPolicy::Middleware:0x000055d773fe5d58 #app=#<ActionDispatch::PermissionsPolicy::Middleware:0x000055d773fe5e20 #app=#<Rack::Head:0x000055d773fe5ec0 #app=#<Rack::ConditionalGet:0x000055d773fe5f60 #app=#<Rack::ETag:0x000055d773fe6050 #app=#<Rack::TempfileReaper:0x000055d773fe60f0
All I really need is the NameError: undefined local variable or method 'static_pages_about_url' and the location of the failing test right? Is this standard for failing tests in Ruby on Rails or is there a way to change how error messages are displayed from showing all of this less useful information?
In test driven development (TDD), you write the tests first. When you run them, all of them will be failing because you have not written the implementation yet (this is your "Red"). After that, you write the implementation that makes the tests pass (this is your "Green"). Once all tests are passsing, it comes the stage where you refactor the implementation (if required, this is your "Refactor"). Red, Green, Refactor.
In your particular example, you have written the test that checks if you can "GET" the about page but because you have not yet implemented the solution, your tests are failing.
In order to make the test pass, you'll need to add this to your route.rb file.
get 'static_pages/about'
When running minutest tests, is it possible to peek at the information about the errors that has happened?
For example, this test suite takes ten minutes to complete. But I would like some more info about the letter E appearing in the tests result.
I don't want to wait ten minutes.
*** Running FRONTEND component engine specs
Run options: --seed 29704
# Running:
......................................................................................................................................................................................E...........
That's E for "error", so one of your tests is failing. Normally you get output that explains more. Once you identify which test is failing you can run that test in a more focused capacity, like:
ruby test/unit/broken_test.rb --name=test_that_is_broken
Where that is the path to your test script and the name of the testing method that's failing.
You may need to make your tests self-contained, able to be run this way, by using:
require_relative '../test_helper'
Or whatever the helper stub is that kicks off the testing framework. Some skeleton files contain things like require 'test_helper' which won't be found in your current $LOAD_PATH.
I have a rails project that serves a JSON API with tests written in RSpec. Often when running specs (request specs, specifically), I’m interested in seeing some details about the HTTP request/response...i.e. the request URL, request body, and response body, ideally JSON pretty-formatted for readability. This isn't for the purposes of documentation but rather as part of the development / debugging process.
I have a helper method I wrote which does this...you just drop a method call into your spec and it prints this stuff out.
But, seems like it would be better if there was a switch that’s part of the running specs. RSpec has custom formatters which I thought might be the right direction, but in trying to build one, I can't figure out how to get access to the request/response objects like you can from inside of your spec.
How can I access the request/response objects in my custom RSpec formatter? Or, perhaps another way to approach the problem?
Here's an approach:
Assuming a rails project, in spec_helper.rb, define a global "after" hook like so:
config.after(:each) do #runs after each example
if ENV['PRINTHTTP']
#use request/response objects here, e.g. puts response.status
end
end
Then, you can conditionally enable by adding the environmental variable on the command-line:
$ PRINTHTTP=1 rspec
Since updating my gems, one of my cucumber tests appears to be failing i.e. in the one-line printout of test outcomes it prints a red F:
However the test summary shows that all the tests passed which seems to contradict the red F.
What's a good way to try and unwrap what's going on here and to get some output from that apparently failing test?
Run your tests with cucumber --format=pretty ... and check out the full output
In a Rails application I have a Test::Unit functional test that's failing, but the output on the console isn't telling me much.
How can I view the request, the response, the flash, the session, the variables set, and so on?
Is there something like...
rake test specific_test_file --verbose
You can add puts statements to your test case as suggested, or add calls to Rails.logger.debug() to your application code and watch your log/development.log to trace through what's happening.
In your test you have access to a bunch of resources you can user to debug your test.
p #request
p #response
p #controller
p flash
p cookie
p session
Also, remember that your action should be as simple as possibile and all the specific action execution should be tested by single Unit test.
Functional test should be reserved to the the overall action execution.
What does it mean in practice? If something doesn't work in your action, and your action calls 3 Model methods, you should be able to easily isolate the problem just looking at the unit tests. If one (or more) unit test fails, then you know which method is the guilty.
If all the unit tests pass, then the problem is the action itself but it should be quite easy to debug since you already tested the methods separately.
in the failing test use p #request etc. its ugly, but it can work
An answer to a separate question suggested
rake test TESTOPTS=-v
The slick way is to use pry and pry-nav gems. Be sure to include them in your test gem group. I use them in the development group as well. The great thing about pry and pry nav is you can step through your code with a console, so you can not only see the code as it's executed, but you can also enter console commands during the test.
You just enter binding.pry in the places in the code you want to trigger the console. Then using the 'step' command, you can move line by line through the code as it's executed.