I am using MiniTest out of the box with Rails 4 and the diff output looks kind of odd, and confusing. See the following output from a failed assert_equal (assert_equal 5, someBigDecimalValue)
--- expected
+++ actual
## -1 +1 ##
-5
+#<BigDecimal:7fa7db1dd528,'0.1E2',9(18)>
The expected result specified in the test was 5, but the - character preceding it makes the expected result look like -5. The ## -1 +1 ## part is also confusing.
Output looks fine when the expected value and the actual value are of the same type:
Expected: "Foo"
Actual: "Bar"
How can I get MiniTest to output the first test something like this:
Expected: 5
Actual: #<BigDecimal:7fa7db1dd528,'0.1E2',9(18)>
Related
z3py's LastIndexOf is supposed to return the "last index of substring within a string". However, it seems to return the wrong results when the substring is located at the end of the string, for example:
>>> import z3
>>> def check_lastindexof(s, sub_list):
... for c in sub_list:
... z_index = z3.simplify(z3.LastIndexOf(s, c))
... v_index = s.rindex(c)
... try:
... assert z_index == v_index
... except Exception as e:
... print(f'{c: <6} FAILED expected: {v_index: >2}, got: {z_index}')
...
>>>
>>> s = 'abcdefabcdef'
>>> tests = ['a', 'abc', 'f', 'ef', 'abcdef']
>>> check_lastindexof(s, tests)
f FAILED expected: 11, got: 5
ef FAILED expected: 10, got: 4
abcdef FAILED expected: 6, got: 0
Notably, LastIndexOf returns -1 if the substring exists only once in the string, which is clearly wrong as far as I know:
>>> s = 'abcdef'
>>> check_lastindexof(s, tests)
f FAILED expected: 5, got: -1
ef FAILED expected: 4, got: -1
abcdef FAILED expected: 0, got: -1
Am I misunderstanding how z3.LastIndexOf is supposed to work or is there maybe a bug somewhere?
z3-solver version: 4.8.14.0 (installed via pip)
This looks like a bug indeed. Please report it at https://github.com/Z3Prover/z3/issues
Is there a way to print the test description of an erlang test generator that uses fixtures? Using a generator makes it tricky to tell what test is actually failing and printing the description would help out.
Example:
-module(math_test).
-include_lib("eunit/include/eunit.hrl").
-define(test(Desc, F), {Desc, {setup, fun setup/0, fun cleanup/1, F}}).
setup() ->
ok.
cleanup(_) ->
ok.
math_test_ () ->
[
?test("adds two numbers", fun add_two_numbers/0),
?test("subtract two numbers", fun subtract_two_numbers/0),
?test("undefined method called", fun undefined_error/0)
].
add_two_numbers () ->
?assertEqual(2, 1 + 3).
subtract_two_numbers () ->
?assertEqual(1, 2 - 2).
undefined_error () ->
undefined_module:uh_oh().
And then running it
[root#a7c901c022bb src]# rebar3 eunit --module=math_test
===> Verifying dependencies...
===> Compiling math
===> Performing EUnit tests...
FFF
Failures:
1) math_test:math_test_/0
Failure/Error: ?assertEqual(2, 1 + 3)
expected: 2
got: 4
%% /src/_build/test/lib/math/src/math_test.erl:20:in `math_test:-add_two_numbers/0-fun-0-/1`
Output:
Output:
2) math_test:math_test_/0
Failure/Error: ?assertEqual(1, 2 - 2)
expected: 1
got: 0
%% /src/_build/test/lib/math/src/math_test.erl:23:in `math_test:-subtract_two_numbers/0-fun-0-/1`
Output:
Output:
3) math_test:math_test_/0
Failure/Error: {error,undef,[{undefined_module,uh_oh,[],[]}]}
Output:
The first 2 errors are ok, but not great -- you can at least see in the assertion where things actually went wrong.
However the 3rd error (calling undefined module/method) is where things go horribly wrong -- there's no real way to tell where it came from!
Is there a way to improve things, like printing the test description with the failure log?
One thing you can do is putting the test description on the test itself, not the entire setup tuple. That is, change this line:
-define(test(Desc, F), {Desc, {setup, fun setup/0, fun cleanup/1, F}}).
to:
-define(test(Desc, F), {setup, fun setup/0, fun cleanup/1, {Desc, F}}).
With that change, the test descriptions are printed:
Failures:
1) math_test:math_test_/0: adds two numbers
Failure/Error: ?assertEqual(2, 1 + 3)
expected: 2
got: 4
%% /tmp/math_test/mylib/_build/test/lib/mylib/test/math_test.erl:20:in `math_test:-add_two_numbers/0-fun-0-/0`
Output:
Output:
2) math_test:math_test_/0: subtract two numbers
Failure/Error: ?assertEqual(1, 2 - 2)
expected: 1
got: 0
%% /tmp/math_test/mylib/_build/test/lib/mylib/test/math_test.erl:23:in `math_test:-subtract_two_numbers/0-fun-0-/0`
Output:
Output:
3) math_test:math_test_/0: undefined method called
Failure/Error: {error,undef,[{undefined_module,uh_oh,[],[]}]}
Output:
Another thing to try is specifying the test functions with the ?_test macro instead of as plain fun terms:
math_test_ () ->
[
?test("adds two numbers", ?_test(add_two_numbers())),
?test("subtract two numbers", ?_test(subtract_two_numbers())),
?test("undefined method called", ?_test(undefined_error()))
].
The ?_test macro remembers the line number it appeared on, and includes it in the output on test failure:
1) math_test:math_test_/0:14: adds two numbers
[...]
2) math_test:math_test_/0:15: subtract two numbers
[...]
3) math_test:math_test_/0:16: undefined method called
[...]
Now you can tell which line those tests were invoked from.
Yet another way to do it is to have the individual functions return eunit "test objects" instead of just running the tests. That would involve using ?_assertEqual instead of ?assertEqual, or wrapping the entire thing in ?_test:
math_test_ () ->
[
?test("adds two numbers", add_two_numbers()),
?test("subtract two numbers", subtract_two_numbers()),
?test("undefined method called", undefined_error())
].
add_two_numbers () ->
?_assertEqual(2, 1 + 3).
subtract_two_numbers () ->
?_assertEqual(1, 2 - 2).
undefined_error () ->
?_test(undefined_module:uh_oh())
Then the output contains both the line numbers and the names of the individual test functions:
Failures:
1) math_test:add_two_numbers/0:20: adds two numbers
Failure/Error: ?assertEqual(2, 1 + 3)
expected: 2
got: 4
%% /tmp/math_test/mylib/_build/test/lib/mylib/test/math_test.erl:20:in `math_test:-add_two_numbers/0-fun-0-/0`
Output:
Output:
2) math_test:subtract_two_numbers/0:23: subtract two numbers
Failure/Error: ?assertEqual(1, 2 - 2)
expected: 1
got: 0
%% /tmp/math_test/mylib/_build/test/lib/mylib/test/math_test.erl:23:in `math_test:-subtract_two_numbers/0-fun-0-/0`
Output:
Output:
3) math_test:undefined_error/0:26: undefined method called
Failure/Error: {error,undef,[{undefined_module,uh_oh,[],[]}]}
Output:
The answers by #legoscia are good, but I also suspect that it's the error reporting implemented by rebar3 that's suboptimal for that kind of error. Running the tests directly from eunit with its default output, you get this:
2> eunit:test(math_test).
math_test: math_test_...*failed*
in function math_test:'-add_two_numbers/0-fun-0-'/0 (math_test.erl, line 22)
**error:{assertEqual,[{module,math_test},
{line,22},
{expression,"1 + 3"},
{expected,2},
{value,4}]}
output:<<"">>
math_test: math_test_...*failed*
in function math_test:'-subtract_two_numbers/0-fun-0-'/0 (math_test.erl, line 25)
**error:{assertEqual,[{module,math_test},
{line,25},
{expression,"2 - 2"},
{expected,1},
{value,0}]}
output:<<"">>
math_test: math_test_...*failed*
in function undefined_module:uh_oh/0
called as uh_oh()
**error:undef
output:<<"">>
=======================================================
Failed: 3. Skipped: 0. Passed: 0.
With the 'verbose' option, it also prints the descriptions before each setup. Furthermore, moving the description to the test fun, and using the ?_test(...) macro to create test funs with more positional information than a plain fun, as #legoscia suggested, you get this output:
math_test:18: math_test_ (undefined method called)...*failed*
in function undefined_module:uh_oh/0
called as uh_oh()
**error:undef
output:<<"">>
You could report this to the rebar3 maintainers.
I am currently writing a test suite using busted/luassert and since I have put some assertions in a separate function I am getting inaccurate stack traces. For example, consider the following test suite (a_spec.lua):
local function my_custom_assertion(x) -- 1
assert.is_true(x > 0) -- 2 <-
end -- 3
-- 4
describe("My test suite", function() -- 5
it("they are positive", function() -- 6
my_custom_assertion(-10) -- 7 <-
my_custom_assertion(-20) -- 8 <-
end) -- 9
end) -- 10
When I run it, my test case fails but the stack trace points to line 2 so I can't tell which of the two assertions was the one that failed.
$busted spec/a_spec.lua
◼
0 successes / 1 failure / 0 errors / 0 pending : 0.002242 seconds
Failure → spec/a_spec.lua # 6
My test suite they are positive
spec/a_spec.lua:2: Expected objects to be the same.
Passed in:
(boolean) false
Expected:
(boolean) true
Is there a way I could have it point to line 7 or 8 instead? One way this would be possible is if luassert's assert.is_true function had a level parameter similar to the builtin error function.
Looking at luassert's source code it seems that it does care about the stack level but I haven't been able to figure out if this feature is internal or if it is exposed to the user somehow.
Instead of creating a custom assertion by creating a function that calls assert.xyzz(), create a function that returns true or false and register it with assert:register.
See the second example in the README.
Turns out that there is a way to solve my actual problem of finding out which of the assertions was the one that fired without needing to change how I write my tests. By invoking busted with the -v (--verbose) option it prints a full stack trace when an assertion fails instead of just providing a single line number.
$ busted -v spec/a_spec.lua
0 successes / 1 failure / 0 errors / 0 pending : 0.003241 seconds
Failure → spec/a_spec.lua # 6
My test suite they are positive
spec/a_spec.lua:2: Expected objects to be the same.
Passed in:
(boolean) false
Expected:
(boolean) true
stack traceback:
spec/a_spec.lua:2: in upvalue 'my_custom_assertion'
spec/a_spec.lua:7: in function <spec/a_spec.lua:6>
That mention of line number 7 lets me know what assertion was the one that failed.
somewhat new to rails. I am working through the exercises on 4.4.1 in Michael Hartl's learn rails tutorial and i'm a bit unclear as to what the named constructor for the class Range is
when i type the literal constructor in the console it returns the appropriate value
(1..10)
=>1..10
but when i try the named constructor
Range.new(1..10)
i receive an error
ArgumentError: wrong number of arguments (given 1, expected 2..3)
from (irb):104:in `initialize'
from (irb):104:in `new'
Ive tried adding many types of extra arguments i.e
Range.new(1)..Range.new(10)
Range.new(1)..(10)
etc..
But I'm always met with an argument error.
ArgumentError: wrong number of arguments (given 1, expected 2..3)
should be a hint that you need at least 2 parameters :
Range.new(1,10)
The 3rd parameter is to specify if the last element of the range is excluded or not :
Range.new(1,10,false).to_a
# equivalent to (1..10)
# => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Range.new(1,10,true).to_a
# equivalent to (1...10)
# => [1, 2, 3, 4, 5, 6, 7, 8, 9]
See the documentation.
Your code
Range.new(1..10)
This is still Range#new with just one parameter : an already initialized Range!
Range.new(1)..Range.new(10)
This is a a..b syntax, so it tries to initialize a Range between a and b. What is a? Range.new(1), which isn't defined, because it only has 1 parameter. You cannot create a Range between 2 Ranges anyway :
Range.new(1,2)..Range.new(4,5)
#=> ArgumentError: bad value for range
Range.new(1)..(10)
Same problem as before. Only 1 parameter for Range.new and this would be a Range between a Range and an Integer!
You can read documentation with ri command
ri Range.new
Which prints
= Range.new
(from ruby site)
------------------------------------------------------------------------------
Range.new(begin, end, exclude_end=false) -> rng
------------------------------------------------------------------------------
Constructs a range using the given begin and end. If the exclude_end parameter
is omitted or is false, the rng will include the end object; otherwise, it
will be excluded.
NB, ri works both on the terminal commandline and inside the pry repl.
I'm using rspec rails to test my application, but I'm concerned that a few specs may actually be misleading. Can I get more information regarding passing specs in my console output to make sure the specs are testing behaviour I am happy with?
Here's how the output currently looks:
# spec.rb
require 'spec_helper'
context "With James Bonds' car" do
before do
#car = {make: "Aston Martin", color: "Silver", owner: "James", age: 3}
end
it "should not have an age of over 1 month" do
#car[:age].should_not == 1
end
end
Now, that expectation would pass, despite the car being over one month old:
$ rspec spec.rb
.
Finished in 0.12 seconds
1 examples, 0 failure, 0 pending
In order to make sure I've written good specs, how can I get ouput like this:
$ rspec spec.rb
.
Finished in 0.12 seconds
1 examples, 0 failure, 0 pending
Passing examples:
1) With James Bonds' car, it should not have an age of over 1 month
Pass: #car.age.should_not == 1
expected not: 1
got : 3
Rspec has different formatters for test output (and you can even write your own).
Try running your specs with the documentation format.
rspec -fd spec.rb
Which will have a more verbose output containing the text you put after your describe, context and it.
That doesn't necessarily assure you are testing the right thing (that's up to you when writing tests), but it does allow you to take stock of what tests are in your suite each time you run it.
Your logic in your spec is wrong - if you want to test if the age is less than 1, test that!
#car.age.should be < 1
The test suite for any decent sized app will span hundreds if not thousands of tests - you really don't want to be wading through the output of that for every single test run.
Your question is a bit confusing.
If the test failed, then you would get the output you were looking for.
If you want to check the actual value that is being tested, deliberately make the test fail e.g. change it to
#car.age.should == 1
Having said that, you seem to be checking the age with a method call, but you are setting car to be an Array of Hashes. In that sense, it will never be true because the method isn't there to be checked.