How do you specify the exact order of Minitest tests? - ruby-on-rails

Question

Minitest lets you run tests in order by overriding test_order to alpha. (You can also use the i_suck_and_my_tests_are_order_dependent! method.)
After doing this, how do you control the order the tests are run across multiple files?
Is there a way to run some tests from one file and then switch to another file?
Looking at the source code, it seems one should be able to control how methods are sorted. But how do you specify that order?

Answer 1

i_suck_and_my_tests_are_order_dependent! (or def self.test_order; :alpha;end define an alphabetic order.
Beside the alphabetic order there is only a random order defined (at least in 'minitest', '>5.5.1', '<=5.6.1'.
But you can patch MiniTest::Test.runnable_methods to get another order.
gem 'minitest'
require 'minitest/autorun'
class MiniTest::Test
#Add a test order :defined
def self.runnable_methods
methods = methods_matching(/^test_/)
case self.test_order
when :random, :parallel then
max = methods.size
methods.sort.sort_by { rand max }
when :defined then # <-new
methods
when :alpha, :sorted then
methods.sort
else
raise "Unknown test_order: #{self.test_order.inspect}"
end
end
end
class TestOrder < MiniTest::Test
def self.test_order; :defined; end
#Alphabetic order
#~ def self.test_order; :alpha;end
#~ i_suck_and_my_tests_are_order_dependent!
def test_4; p __method__; end
def test_3; p __method__; end
def test_2; p __method__; end
def test_1; p __method__; end
end
But the test order is only defined per Test-subclass, not global for all tests. So this does not give you access to the order of test methods in multiple test classes.
I estimate you have different test-classes in your files, so this would correspond to your problem. (Not the files is the criteria, but the Test-class.)
If you define only one Test class in your test files then you have the possibility to define your own order.

Answer 2

You can add i_suck_and_my_tests_are_order_dependent! to your test class.
Example:
class TestClass < Minitest::Unit::TestCase
i_suck_and_my_tests_are_order_dependent!
def test_true
assert true
end
end
According to ruby-doc,
Call this at the top of your tests when you absolutely positively need to have ordered tests. In doing so, you’re admitting that you suck and your tests are weak.
You can also simply add the following to your test class:
def self.test_order
:alpha
end
The i_suck_and_my_tests_are_order_dependent! method uses that.

Answer 3

Can also change globally for all test suits:
# in your test_helper.rb
require "minitest" # or "minitest/autorun"
class Minitest::Test
def self.test_order
:alpha
end
end

Answer 4

Little addition ( or may be override %) ) to #knut answer. You don't need to patch Minitest, instead just override self.runnable_methods in your TestClass, and place your tests there in any suitable order.
def self.runnable_methods
super | ['run_last']
end
It's better then #knuts answer since original answer is dependent internally on Module method instance_methods(include_super=true), and it will return them in less predictable and partially alphabetically sorted way:
module A
def method3() end
end
class B
include A
def method2() end
end
class C < B
def method4() end
def method1() end
end
C.instance_methods(true) # -> [:method1, :method4, :method2, :method3 ...
so I assume in previously given solution your tests will need alpha sorted names to keep their order. So looks like it's just another way around to :alpha or :sorted way