I have a Rails app that interacts with some Mac software and I need to write some tests for it. How on earth do I do that? Where do I even start?
The Rails app connects to the Mac app through AppleScript and Terminal. Any ideas?
Update
Found this gem to help with Shell expectations. Is that as good as I'm gonna get?
https://github.com/matthijsgroen/rspec-shell-expectations
Testing external dependencies can certainly be a challenge.
Remember that tests you write should test the behavior of the Rails app, not the external dependency. You should have integration tests to verify that the code actually works with the application, but they should be "smoke tests", not a full suite to test every feature.
Write unit tests to verify the behavior of the code that relies on the dependency, and mock out the interactions. Typically with command line apps that means:
the app wrote to STDOUT
the app wrote to STDERR
the app read from STDIN
the app exited with a particular status code
The gem you mentioned is a good start, but you may find it worthwhile to look at rolling your own helper code using Open3 from the Ruby standard library, which can be useful for all the items in the list above.
Use a tag on the specs that need to use the Mac application to make it easier to filter out those specs as necessary.
You may already be familiar with vcr for mocking out HTTP interactions; its "playback" feature is a good source of inspiration.
Related
I'm a PHP dev mostly and just starting out with Ruby, so please correct me if I say something dumb.
I'm working on fixing some bugs in a "legacy" app written in rails. The app itself has never been unit-tested. I can see the test scaffolding generated by rails but tests are nowhere to be found.
The app is pretty big and the code quality is very bad, so I wanted to write some units tests for the functionality I'll be fixing, before writing any code.
The problem is that when I run rake test command, the testing DB is created, but if I write any tests it keeps crashing on me. There are several problems with some relations and keys which I tried to fix, but more problems just keep appearing. I do understand that the DB is created with schema.rb file, but I'm sure it is just outdated by now. It is another issue I will maybe fix, but for now I just want to write some basic unit tests not even using the DB itself.
So the question is: is it possible to write just unit tests for some methods without invoking all the test DB scripts? I'm aware that this is maybe not the best practice, but I will feel better modifying the app with some test coverage and starting with fixing the DB I do not yet understand seems like a bad idea to me.
I'm using Ruby version 2.1.10 and the app is written in Rails 4.0.4 - these seem to be the latest versions I managed to run the app on.
On edx.org there is Software as a Service course that grades all submitted assignments.
You upload a zip-file with a Rails project and they run a bunch of integration and unit tests.
How do they do it?
My thoughts are they mount an uploaded application as Rails engine. Is it possible to test one Rails app within another? I'd like to create a similar service, but I don't know what to start with.
I would imagine you could do this in a similar way that Jenkins runs continuous testing on a project. Where each project uploaded to your site gets expanded into a workspace, and then you shell out and execute commands. But that allows for a lot of variable configurations and complexity that probably doesn't make sense in the scope you've proposed. It also doesn't protect your underlying OS from the projects your testing.
You could also probably use an application container like docker and manage each uploaded application that way as well, which would keep everything self-contained, and isolate the application from the OS. It also puts the onus on the developer to package and manage dependencies correctly. I'm guessing they are probably using docker or something similar, here's an example Using Docker for MOOCS
At the point you want to capture the test results and report them back, I'd think they probably use something similar to the Junit formatter for Rspec or they just parse the rspec output directly.
For testing purposes for Ruby/Rails I use RSpec and I have well over 1000 tests that are already written in my application. Once the app goes live I want to have the tests run once a day on the production server (since all the code is there) and see if anything fails. The test itself will be run against test data and now production data. This is an effort so that any new code that is added in the future will not cause anything to unknowingly break.
I have found a few solutions:
Travis-CI (only open-source ... not suitable for closed-source projects)
Jenkins-CI (not sure if it works with or well with Ruby/RSpec/Rails)
Watir (not sure if Ruby/Rspec works with it, but the tool itself is written in Ruby).
Preferably something that checks the codebase daily and then emails me when something isn't working.
I also plan on integrating JavaScript testing with the a testing library of my choice (I just need the automation platform for testing it).
Can someone provide me some insight as to which tool to use? Or does anyone have any other tools to recommend?
Jenkins-CI works great with rspec, and can run your jasmine javascript testing, and your cucumber javascript tests as well.
The only thing I'd recommend is not to test on your production server itself. When you push changes to your source-control repository, Jenkins will download the new code and run your tests there. When you're green (tests pass), push the code to production.
How can I run a watir test in the context of the app that's being tested? I'd like my test to browse the app and then access ActionMailer::Base.deliveries for emails or check models directly. This is how I understand what's being described here.
UPDATE: They probably use Capybara to be able to acces the email array and be in the context of the "server" which is instantiated just for the test.
I suggest checking out the Rails unit testing docs, then writing a simple Rails test case that starts your app - then try adding a line or two of Watir code to access your app:
http://guides.rubyonrails.org/testing.html
As far as I know you should be able to write a Rails unit test, then put Watir code inside one of your test methods - and if all goes well you should be able to instantiate your web app, use Watir to launch a browser to test it, and in the same method(s) perform non-Watir low-level testing (e.g. checking models/data/etc.)
I've never used Watir inside a Rails test, but I don't see why it wouldn't work.
Watir is about driving browsers to automate functional testing. You could I suppose use it for unit testing of the top level UI stuff, but more often in a 'unit test' context that would be done using a headless browser emulation, Capybara, celerity, or watir-webdriver using the headless option, purely for speed of operation since driving an actual browser can be slow even with a fast browser like chrome.
Most of the times people use Watir it's for more functional tests, often from a test runner framework like Cucumber, sometimes Rspec depending on your needs. You might combine that with other ruby code to access or create test data, to validate something made it into the DB from the UI, but everything in the Watir gem is all about the browser and interacting with it much like a human would, and driving the browser is it's function within the set of tools you might use.
I had the same need and found the following solution: https://stackoverflow.com/a/9687562/90741. I reproduce it here, as the linked question seems to be dead, with its owner...
I had the same need (Watir+RSpec), but the problem is: there is no Rack stack running by default during tests...
I found this solution:
In spec_helper.rb:
HTTP_PORT = 4000
test_instance_pid = fork do
exec 'unicorn_rails -E test -p %d' % HTTP_PORT
end
at_exit do
Process.kill "INT", test_instance_pid
Process.wait
end
Which start the test stack once for all spec tests, and kill it at the end. (In this sample, I am using unicorn, but we can imagine using any other server)
In the spec, I reuse the HTTP_PORT constant to build URL:
browser.goto "http://localhost:#{HTTP_PORT}/"
We've recently upgraded our Rails application.
To be extra sure everything works, I've tried to get the tests and specs of the various used plugins (26 at current count) to work, thinking then to add those to our continuous integration, which only runs the main application's specs.
I've run into a lot of problems even getting the specs/tests to run at all, not even getting to any individual test failures. For example, I've run across this problem: http://rails_security.lighthouseapp.com/projects/15332/tickets/7-rake-spec-plugin-fails-on-rails-2-1 (thanks by the way for that ticket, even though the issue wasn't fixed).
So the question is: Are we unusual in that we've ever cared about running plugin tests ? It doesn't seem to feature much here on SO. My nagging feeling is that they should be run as much as the main specs, but you could also argue that since the main specs work, the plugins must also work.
Alot of it depends on the plugin/gem being used.
If I know the author/community of the gem is competant I will skip the tests and simply use the latest stable release and freeze that gem. I will then track the progress of the development using github.
If the plugin/gem is written by an unknown party I will run the tests and freeze the gem/plugin and again monitor the development.
Sometimes however I will write my own contributions to the gem and fork the code. I will clone the repo in github and base my installations from that. At which point any and all changes result in a complete test run.
With all things in the open source world there is an element of trust between the creator and the users of those pieces of code. The tests themselves don't tell me much about the codebase, it shows there are tests and thats it. Do they test everything ? Are there edge cases ? . Its this element of trust I have with certain developers in the community that means I forgo worrying over running tests for those gems.
Its a slippery slope testing everything, where does it stop ? Would you test rails every release ? No, you assume the community has done this for you already.