I'm using rspec rails to test my application, but I'm concerned that a few specs may actually be misleading. Can I get more information regarding passing specs in my console output to make sure the specs are testing behaviour I am happy with?
Here's how the output currently looks:
# spec.rb
require 'spec_helper'
context "With James Bonds' car" do
before do
#car = {make: "Aston Martin", color: "Silver", owner: "James", age: 3}
end
it "should not have an age of over 1 month" do
#car[:age].should_not == 1
end
end
Now, that expectation would pass, despite the car being over one month old:
$ rspec spec.rb
.
Finished in 0.12 seconds
1 examples, 0 failure, 0 pending
In order to make sure I've written good specs, how can I get ouput like this:
$ rspec spec.rb
.
Finished in 0.12 seconds
1 examples, 0 failure, 0 pending
Passing examples:
1) With James Bonds' car, it should not have an age of over 1 month
Pass: #car.age.should_not == 1
expected not: 1
got : 3
Rspec has different formatters for test output (and you can even write your own).
Try running your specs with the documentation format.
rspec -fd spec.rb
Which will have a more verbose output containing the text you put after your describe, context and it.
That doesn't necessarily assure you are testing the right thing (that's up to you when writing tests), but it does allow you to take stock of what tests are in your suite each time you run it.
Your logic in your spec is wrong - if you want to test if the age is less than 1, test that!
#car.age.should be < 1
The test suite for any decent sized app will span hundreds if not thousands of tests - you really don't want to be wading through the output of that for every single test run.
Your question is a bit confusing.
If the test failed, then you would get the output you were looking for.
If you want to check the actual value that is being tested, deliberately make the test fail e.g. change it to
#car.age.should == 1
Having said that, you seem to be checking the age with a method call, but you are setting car to be an Array of Hashes. In that sense, it will never be true because the method isn't there to be checked.
Related
Whenever I use QuickChick in Coq, it has a default of 10000 tests. Is there a way to change this default value to generate a different amount of tests?
I figured out a way:
Extract Constant Test.defNumTests => "42".
QuickChick test_predicate.
Will test test_predicate up to 42 times:
+++ Passed 42 tests (0 discards)
You could also try using QuickChickWith, stdArgs and updMaxSuccess as follows:
QuickChickWith (updMaxSuccess stdArgs 42) test_predicate.
However, most of the time I use your extraction hack to avoid large natural numbers.
a=0
describe "desc-1" do
it "it-1" do
puts "IT -1 "
a=1
end
if (a==1)
it "it-2" do
puts "IT-2"
end
end
end
the second it block must be executed,but now it is not executing can u please help thanks in advance
In RSpec your it blocks should be independent of each other and each test a specific piece of behavior. (Check out this similar question)
If you need to share a value across examples, assign the variable in a before(:all) block, but the order your tests are run should be random to ensure limited side effects (read this answer) so even this wouldn't really give you your answer.
If you want tests that test a larger system or series of actions look at this Rails Guide for functional testing or this great article on functional tests.
You have a scope issue. Local variable values assigned inside the it block will not be available outside of that context. In this case, if this is Rspec as it looks, your second example is not getting run because a evaluates to 0 in that context, but it will evaluate to 1 if you just allow the second block to run and test the expectation. This will work
a = 0
describe "a" do
it "is 0" do
expect(a).to eq 0
a = 1
end
it "is 1" do
expect(a).to eq 1
end
end
However now your second example is dependent on the first example running before it. A better idea would be to define the variable within the block testing the expectation, in which case it's a good idea to change it back afterward with an after block so you it resets to the default.
a = 0
describe "a" do
after { a = 0 }
it "is 0" do
expect(a).to eq 0
end
it "can be changed to 1" do
a = 1
expect(a).to eq 1
end
end
I'm evaluating the use of lua scrips in redis, and they seem to be a bit slow. I a benchmark as follows:
For a non-lua version, I did a simple SET key_i val_i 1M times
For a lua version, I did the same thing, but in a script: EVAL "SET KEYS[1] ARGV[1]" 1 key_i val_i
Testing on my laptop, the lua version is about 3x slower than the non-lua version. I understand that lua is a scripting language, not compiled, etc. etc. but this seems like a lot of performance overhead--is this normal?
Assuming this is indeed normal, are there any workaround? Is there a way to implement a script in a faster language, such as C (which redis is written in) to achieve better performance?
Edit: I am testing this using the go code located here: https://gist.github.com/ortutay/6c4a02dee0325a608941
The problem is not with Lua or Redis; it's with your expectations. You are compiling a script 1 million times. There is no reason to expect this to be fast.
The purpose of EVAL within Redis is not to execute a single command; you could do that yourself. The purpose is to do complex logic within Redis itself, on the server rather than on your local client. That is, instead of doing one set operation per-EVAL, you actually perform the entire series of 1 million sets within a single EVAL script, which will be executed by the Redis server itself.
I don't know much about Go, so I can't write the syntax for calling it. But I know what the Lua script would look like:
for i = 1, ARGV[1] do
local key = "key:" .. tostring(i)
redis.call('SET', key, i)
end
Put that in a Go string, then pass that to the appropriate call, with no key arguments and a single non-key argument that is the number of times to loop.
I stumbled on this thread and was also curious of the benchmark results. I wrote a quick Ruby script to compare them. The script does a simple "SET/GET" operation on the same key using different options.
require "redis"
def elapsed_time(name, &block)
start = Time.now
block.call
puts "#{name} - elapsed time: #{(Time.now-start).round(3)}s"
end
iterations = 100000
redis_key = "test"
redis = Redis.new
elapsed_time "Scenario 1: From client" do
iterations.times { |i|
redis.set(redis_key, i.to_s)
redis.get(redis_key)
}
end
eval_script1 = <<-LUA
redis.call("SET", "#{redis_key}", ARGV[1])
return redis.call("GET", "#{redis_key}")
LUA
elapsed_time "Scenario 2: Using EVAL" do
iterations.times { |i|
redis.eval(eval_script1, [redis_key], [i.to_s])
}
end
elapsed_time "Scenario 3: Using EVALSHA" do
sha1 = redis.script "LOAD", eval_script1
iterations.times { |i|
redis.evalsha(sha1, [redis_key], [i.to_s])
}
end
eval_script2 = <<-LUA
for i = 1,#{iterations} do
redis.call("SET", "#{redis_key}", tostring(i))
redis.call("GET", "#{redis_key}")
end
LUA
elapsed_time "Scenario 4: Inside EVALSHA" do
sha1 = redis.script "LOAD", eval_script2
redis.evalsha(sha1, [redis_key], [])
end
eval_script3 = <<-LUA
for i = 1,2*#{iterations} do
redis.call("SET", "#{redis_key}", tostring(i))
redis.call("GET", "#{redis_key}")
end
LUA
elapsed_time "Scenario 5: Inside EVALSHA with 2x the operations" do
sha1 = redis.script "LOAD", eval_script3
redis.evalsha(sha1, [redis_key], [])
en
I got the following results running on my Macbook pro
Scenario 1: From client - elapsed time: 11.498s
Scenario 2: Using EVAL - elapsed time: 6.616s
Scenario 3: Using EVALSHA - elapsed time: 6.518s
Scenario 4: Inside EVALSHA - elapsed time: 0.241s
Scenario 5: Inside EVALSHA with 2x the operations - elapsed time: 0.5s
In summary:
scenario 1 vs. scenario 2 show that the main contributor is the round trip time as scenario 1 makes 2 requests to Redis while scenario 2 only makes 1 and scenario 1 is ~2x the execution time
scenario 2 vs. scenario 3 shows that EVALSHA does provide some benefit and I am sure this benefit increases the more complex the script gets
scenario 4 vs scenario 5 shows the overhead of invoking the script is near minimal as we doubled the number of operations and saw a ~2x increase in execution time.
so there is now a workaround using a module created by John Sully. It works for Redis and KeyDB and allows you to use the V8 JIT engine which runs complex scripts much faster than Lua scripts. https://github.com/JohnSully/ModJS
Does anyone know if this is possible with SpecFlow? In the spirit of not having more than one assertion per test, I was hoping SpecFlow would treat each "Then" as a separate test.
I have a Scenario with multiple "Then" steps that looks something like this (snippet):
When a summary for day <day number> is requested
Then the summary well id should be "134134"
And the summary well name should be "My Well --oops!"
And the summary violated rules should be <violations>
And the summary variance should be <variance>
Examples:
| day number | violations | variance |
| 0 | BadTotal,HpOnLp | -33 |
| 3 | BadTotal | -133.33 |
| 5 | | 0 |
The second assertion "My Well --oops!" should fail. What I want is for SpecFlow to test the assertions that follow
I get:
When a summary for day 0 is requested
-> done: DprSummarySteps.WhenASummaryForDayIsRequested(0) (0.0s)
Then the summary well id should be "134134"
-> done: DprSummarySteps.ThenTheSummaryWellIdShouldBe("134134") (0.0s)
And the summary well name should be "My Well --oops!"
-> error: Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.
And the summary violated rules should be BadTotal,HpOnLp
-> skipped because of previous errors
And the summary variance should be -33
-> skipped because of previous errors
Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.
What I want:
When a summary for day 0 is requested
-> done: DprSummarySteps.WhenASummaryForDayIsRequested(0) (0.0s)
Then the summary well id should be "134134"
-> done: DprSummarySteps.ThenTheSummaryWellIdShouldBe("134134") (0.0s)
And the summary well name should be "My Well --oops!"
-> error: Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.
And the summary violated rules should be BadTotal,HpOnLp
-> done: DprSummarySteps.ThenTheViolatedRulesShouldBe("BadTotal,HpOnLp") (0.0s)
And the summary variance should be -33
-> done: DprSummarySteps.ThenTheVarianceShouldBe(-33) (0.0s)
Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.
I don't believe that you can do this but my question to you would be why do you want that? With any unit test it will fail at the first assertion. you wouldn't expect a test to continue execution after an assertion failed and this is no different. Surely knowing that the test failed for some reason is enough. In this specific case you might be able to make separate independent assertions which provide useful info, but in the general case assertions that come after one that has failed might be completely meaningless.
If you want to have each assertion independent of the others then you need to break it into several scenarios with each of you current And steps as its own Then step. You might be able to use a Background step to do the common setup.
I'm not sure that this will help you though as it seems like your assertions are all related to the examples so you end up needing to repeat the examples.
You can do this now if you upgrade to Specflow 2.4 and are using Nunit, see something like https://github.com/nunit/docs/wiki/Multiple-Asserts
Assert.Multiple(() =>
{
Assert.AreEqual(5.2, result.RealPart, "Real part");
Assert.AreEqual(3.9, result.ImaginaryPart, "Imaginary part");
});
I have identified one way to do this. So what happens internally in specflow, before running any step, the TestExecutionEngine checks the LastExecutionStatus of Scenario, and it does not proceed if it is not OK.. So what we can do is, in [AfterStep] hook, add the following line :
typeof(ScenarioContext).GetProperty("ScenarioExecutionStatus").SetValue(this.ScenarioContext, ScenarioExecutionStatus.OK);
Replace this.ScenarioContext with whatever object with type ScenarioContext exist. This will make the current status as OK and will allow you to proceed to next step.
Keep in mind this won't allow you to catch all assert failures in a single step.
I am not able to understand what I am being asked in this exercise(I have to get the following tests to pass):
require "performance_monitor"
require "time" # loads up the Time.parse method -- do NOT create time.rb!
describe "Performance Monitor" do
before do
#eleven_am = Time.parse("2011-1-2 11:00:00")
end
it "takes about 0 seconds to run an empty block" do
elapsed_time = measure do
end
elapsed_time.should be_within(0.1).of(0)
end
I try creating a empty method but it won't pass.
def measure(number = 1)
number
end
or
def measure
end
I think I am not understanding what the test is asking me to build. What is it asking, and how can I interpret it better.
measure should accept a block, time its execution and return number of seconds that the block took to run. Here is the pseudocode (it translates pretty much directly to ruby code you need)
def measure(&block)
time t1 = current time
execute block
return (current time - t1) # difference between current time *now* and
# time that was current before execution of the block
# which (the difference) means time the block took to run
end