I am coming across lot of production issues related to undefined variables. How can I FIND all undefined variables in Ruby file? Is there any gem/script which can scan through code files and detect possible issues.. This way I can test code before production deployment or accepting Pull Requests.
This can't really be done. There are too many ways a variable could be defined. There's also an equal number of ways a variable that should be defined could be set to nil for whatever reason.
You can run Ruby with warnings enabled (-w on the command-line), which will complain vigorously about these things. It will only complain about variables along the execution path of the code, so if there are sections you don't exercise, you won't get warnings.
This is why having a test suite that exhaustively tests branches is essential to shaking out bugs like this. If you have an if in your code, you need two tests for it. If you have two if clauses, you may need three or four. Anything with a case will need all branches tested. This can be a lot of work for a project that's got a lot of business logic in it.
Since Ruby isn't compiled per-se, it's not really able to detect these sorts of issues before the code is run. What's in your file and what actually gets executed can be worlds apart depending on the impact of other parts of code. This is not true in more conservative languages like C or Rust.
Test Unit is built into rails so write some unit tests. Then run with rake test.
Try the rspec and simplecov gems.
rspec allows you to write unit tests that find undefined variables.
simplecov measures code coverage to make sure your tests cover all variables.
Related
Is there any way to retest previously broken tests?
So, say, I run rspec and several tests in different files and directories fail.
I fix something and now I have to manually specify all files and folders I want to retest or run tests for whole project again(It takes considerable amount of time for big projets).
What I was looking for is something like a flag
rspec --prev-failed-only
I realize that such flag would require considerable amount of additional actions from rspec, like storing results of previous tests and so on. But I think it would be super convenient to me.
Is there any such(or similar) tool/gem?
rspec-rerun gem does what you want : https://github.com/dblock/rspec-rerun
https://github.com/rspec/rspec-core/issues/456 has a good discussion on the topic of making rspec itself be able to rerun failed tests.
On the Giant Robots podcast, Sam Phippen of the core team mentioned this feature is due to be added to RSpec soon.
In case anyone else finds this, a month after the question was asked (open source <3 ) rspec 3.3 introduced the --only-failures option, also the delightfully handy --next-failure (-n) option. rspec --help for more info.
In this video from GoGaRuCo 2011, Corey Haines shows some techniques for making Rails test suites much faster. I would summarize it as follows:
Put as much of your code as possible outside the Rails app, into other modules and classes
Test those separately, without the overhead of loading up Rails
Use them from within your Rails app
There were a couple of things I didn't understand, though.
He alternates between running tests with rspec and spn or spna (for example, at about 3:50). Is spn a commonly-known tool?
In his tests for non-Rails classes and modules, he includes the module or class being tested, but I don't see him including anything like spec_helper. How does he have Rspec available?
Sorry about the confusion. spn and spna are aliases I have that add my non-rails code to rspec's load path. There isn't anything special about them, other than adding a -I path_to_code on the command-line.
These days, I add something like this to my .rspec file:
-I app/mercury_app
Then I can do simple require 'object_name' at the top of my specs.
As for not including spec_helper: that is true, I don't. When you execute your spec file with rspec <path_to_spec_file>, it gets interpreted, so you don't need to require rspec explicitly.
For my db specs these days, I also have built an active_record_spec_helper which requires active_record, establishes a connection to the test database, and sets up database_cleaner; this allows me to simply require my model at the top of my spec file. This way, I can test the AR code against the db without having to load up my whole app.
A client I am working at where we are using these techniques is interested in supporting some blog posts about this, so hopefully they will start coming out towards the middle of June.
I have the following mocha mock that works great.
In a test.rb file:
setup do
Date.stubs(:today).returns(Date.new(2011, 7, 19))
Time.stubs(:now).returns(Time.new(2011,1,1,9,0))
end
The problem is that the timing is broken for the tests. After the tests run the date and time objects are still mocked.(!)
Finished in -21949774.01594216 seconds.
I added the following:
teardown do
Date.unstubs(:today)
Time.unstubs(:now)
end
This throws the following error for each test: WARNING: there is already a transaction in progress
Is this the proper way to unstub? Is it better to unstub at the end of the test file or even at the end of unit test suite?
Working in Rails 3.07 and Mocha 0.9.12
Thanks.
I don't know if this is fully your problem, but it is just unstub, not pluralized.
Other than that, there should be no issue. You definitely want to unstub after each test (or set of tests, if a bunch of tests need the stubbing) because once stubbed, it will stay stubbed, and that can screw up other tests.
The accepted answer is spreading misinformation and should be considered harmful.
One of the main purposes of a mocking library like Mocha is to provide automatic mock/stub teardown as part of the integration to various testing libraries. In fact if you look at the GitHub repo for Mocha you will see that significant maintenance effort is put into making Mocha work smoothly with all the versions of several different testing frameworks.
If this isn't working properly then you need to figure out why Mocha's built-in teardown isn't working. Unstubbing manually in your own teardown is just papering over the problem, and could hide subtler issues with stub leakage or Mocha otherwise misbehaving.
If I had to take a wild guess money would be on your stub somehow being run outside of an actual test because that's the most common cause I've seen for this kind of thing in the wild, but there's not enough information from the question to really ascertain.
Today I am going to get as far as I can setting up my testing environment and workflow. I'm looking for practical advice on how to setup the test environment from you guys who are very passionate and versed in Ruby Testing.
By the end of the day (6am PST?) I would like to be able to:
Type one 1-command to run test suites for ANY project I find on Github.
Run autotest for ANY Github project so I can fork and make TESTABLE contributions.
Build gems from the ground up with Autotest and Shoulda.
For one reason or another, I hardly ever run tests for projects I clone from Github. The major reason is because unless they're using RSpec and have a Rake task to run the tests, I don't see the common pattern behind it all.
I have built 3 or 4 gems writing tests with RSpec, and while I find the DSL fun, it's less than ideal because it just adds another layer/language of methods I have to learn and remember. So I'm going with Shoulda. But this isn't a question about which testing framework to choose.
So the questions are:
What is your, the SO reader and Github project committer, test environment setup using autotest so that whenever you git clone a gem, you can run the tests and autotest-develop them if desired?
What are the guys who are writing the Paperclip Tests and Authlogic Tests doing? What is their setup?
Thanks for the insight. There are tons of resources describing how to use the different testing frameworks, but almost nothing on the actual setup and workflow. Looking for answers that will make me a more effective tester.
The most common convention probably is rake test, rake spec, or maybe even just rake.
Of course, there is no question that this will fail with many projects, in particular the ones without tests or specs.
It might be possible to parse the output of rake -T if a Rakefile is there, and act on that, but there really is no way you will cover ALL projects on GitHub.
When I want to run all my unit tests, I run rake test:units. To run all my functional tests, I run rake test:functionals. If I want to run all the test cases in one file, I run
ruby test/unit/username_test.rb
A few people have been telling me I should run rake instead such as
rake test:units TEST=test/unit/username_test.rb
For running tests, they say I should always run rake. I know I should run rake if I'm testing all my unit tests. But what if it's just one file or one particular test method in a file that I'm testing? Should I still use rake? Is there any difference between the two? Do I get any benefit from running rake over ruby? Is there any disadvantage to running ruby rather than rake?
Sadly, the two are not the same. Running the tests under rake can wind up pulling things from different places than when you run the test directly (more a problem when you have multiple versions of gems, etc. on your system).
The intent is that tests run under rake should be in an environment that matches what rails would produce; I can not attest to how closely they match, but I have seen tests that passed when run directly but failed when run via rake or rails (and visa versa).
Before checking in at the very least I'd recommend running rake to hit everything, in order to be assured that nothing unexpected has broken.
Plain ruby seems ideal for fast testing of single files during iterations.
Be aware that running everything through rake can produce different results to running everything individually, as I found to my confusion recently - I was doing something slightly wrong in one test that worked successfully in isolation but that left a problem lying around for a subsequent test that only showed up when I used rake.
No I dont think so. Rake seems to be a convenient way to run all tests, all unit tests or all functional/controller tests.
For a single file, I use the ruby object_test.rb approach.. shorter and works fine for my rails home project.
They should be identical. if they are not, you're definitely doing something wrong.
As I said in my other comments, if you get tests that pass in one, but fail in the other, you're doing something very wrong; this indicates a poor test setup, and is usually caused by a different test run order between the two test approaches; one of which causes tests to fail.
Usually the cause of this is that you are not using transactions and/or tests are not cleaning up after themselves. For example, not properly requiring the fixtures they later test for, and instead relying on the pre-existing database state.
you are free to use either method. if something breaks, you're doing something wrong in your tests, and you should fix your code.
The two are not the same. Rake will do some preliminary test loading.
The intent is that tests run under rake should be in an environment that matches what rails would produce;
One difference I've noticed is with rake some of the fixture loading happens which could be by-passed with ruby.
I'd recommend using rake, unless you are using the ruby command line to one just one test in the file with the -n option.