I want to create an online admin dashboard which will indicate whether our most important tests are passing. I've already written the tests, but how I could run them on demand and see what their results are in a non-test (production) environment?
I'd like to avoid rewriting the test code if possible.
I have 3 ideas for how to do this (not sure if they're any good):
Using system calls to grep the verbose (-v) output of the tests
Use a different test_runner that will give easier to parse ouptut via system call
Somehow abstract the testing functions so that they can be used in a non-test context. (Not sure how I would do this..
Related
Is it possible to tell rspec to restart Rails before an example is run? I'm building an Engine that hooks into the Rails initialization process and the users can make some configuration changes, in an initializer, that impact how Rails and the Engine are configured. I want to be able to simulate those configuration changes, restart rails and test the result.
I haven't done this feat yet, but as best practice I think your engine tests should be part of the engine and should have minimal dependencies.
Some approaches I've seen and believe you should try and combine:
Mock a minimal parent rails app to test your engine.
Write multiple dummy apps to test with.
Instead of loading the entire rails application, you can split spec_helper and rails_helper in smaller parts, also gaining in setup time.
You can write custom rake tasks to switch environment before spawning a new test thread.
You can also overwrite at runtime the configuration values which reflect in your test (plus: use dependency injection!).
If your initializer is complex enough, you could extract it in a testable helper and wire it up in your test initializers.
Also, there seems to be a gem for that: Combustion.
I have a Rails app that interacts with some Mac software and I need to write some tests for it. How on earth do I do that? Where do I even start?
The Rails app connects to the Mac app through AppleScript and Terminal. Any ideas?
Update
Found this gem to help with Shell expectations. Is that as good as I'm gonna get?
https://github.com/matthijsgroen/rspec-shell-expectations
Testing external dependencies can certainly be a challenge.
Remember that tests you write should test the behavior of the Rails app, not the external dependency. You should have integration tests to verify that the code actually works with the application, but they should be "smoke tests", not a full suite to test every feature.
Write unit tests to verify the behavior of the code that relies on the dependency, and mock out the interactions. Typically with command line apps that means:
the app wrote to STDOUT
the app wrote to STDERR
the app read from STDIN
the app exited with a particular status code
The gem you mentioned is a good start, but you may find it worthwhile to look at rolling your own helper code using Open3 from the Ruby standard library, which can be useful for all the items in the list above.
Use a tag on the specs that need to use the Mac application to make it easier to filter out those specs as necessary.
You may already be familiar with vcr for mocking out HTTP interactions; its "playback" feature is a good source of inspiration.
This question is about starting a rails server of the external project from a rspec environment.
There is 2 projects.
First project act as the Admin Back Office, it's the central application where users interact with web pages. I call it BackOffice
Second project is a Json API Server which will receive commands from the Admin Back Office through json requests.I call it ApiServer
I am trying to test API interaction between those 2 rails projects, and I would like to set-up rspec so I can write and maintain my spec files in BackOffice project. Those specs would start a ApiServer rails server and then play around to perform the tests.
My issue is about starting the ApiServer rails server. After looking at the rails app initialization files, I assumed I had to add a require to "config/environment".
But when I insert into BackOffice/spec/spec_helper.rb
require File.expand_path('../../../ApiServer/config/environment', __FILE__)
I get the error
`initialize!': Application has been already initialized. (RuntimeError)
# Backtrace to the file:
# ApiServer/config/environment.rb
# Line:
# Rails.application.initialize!
I also tried to simply call the following in backticks
`cd /api/path; bundle exec rails s -p 3002`
but got the same kind of error
Then I got inspiration from Capybara source code, and required the "ApiServer/application", then I am able to create a ApiServer.new object, but as soon as I call initialize! on it it I get the same message.
Any help is greatly appreciated. Cheers
Actually the second app is nothing more then an external service, which is better to stub for the tests.
There is one nice article from thoughtbot about using vcr gem to mock external web services:
https://robots.thoughtbot.com/how-to-stub-external-services-in-tests
Obligatory "don't do that unless you really need to".
However, since it seems you know what you need:
Short answer:
You need to isolate both application in system environment and launch it from there using system-calls syntax.
Long answer:
What you're trying to do is to run two Rails applications in the same environment. Since they both are Rails applications they share a lot of common names. Running them ends in name clash, which you're experiencing. Your hunch to try simple back ticks was good one, unfortunately you went with a bundler in already existing environment, which also clashes.
What you have to do in order to make it work is to properly isolate (in terms of code, not in terms of network i.e. communication layer ) application and then run launcher from rspec. There are multiple ways, you could:
Use Ruby process control (Check this graph, you could try to combine it with system level exec)
Daemonize from Operating System level (init.d etc.)
Encapsulate in VM or one of the wrappers (Virtualbox, Vagrant, etc.)
Go crazy and put code on separate machine and control it remotely (Puppet, Ansible, etc.)
Once there, you can simply run launcher (e.g. daemon init script or spawn new process in isolated environment) from RSpec and that's it.
Choosing which way to go with is highly dependent on your environment.
Do you run OSX, Linux, Windows? Are you using Docker? Do you manage Ruby libraries through things like RVM? Things like this.
Generally it's a bad idea to require launching another service/application to get your unit tests to pass. This type of interaction is usually tested by mocking or vcring responses, or by creating environment tests that run against deployed servers. Launching another server is outside the scope of rspec and generally, as you've discovered, will cause a lot of headaches to setup and maintain.
However, if you're going to have these rails projects tightly coupled and you want them to share resources, I'd suggest investigating Rails Engines. To do this will require a substantial amount of work but the benefits can be quite high as the code will share a repository and have access to each other's capabilities, while maintaining application isolation.
Engines effectively create a rails application within another rails application. Each application has it's own namespace and a few isolating guards in place to prevent cross app contamination. If you have many engines it becomes ideal to have a shell rails application with minimal capabilities serving each engine on a different route/namespace.
First you need to create housing for the new api engine.
$ rails plugin new apiserver --mountable
This will provide you with lib/apiserver/engine.rb as well as all the other scaffolding you'll need to run your API as an engine. You'll also notice that config/routes.rb now has a route for your engine. You can copy your existing routes into this to provide a route path for your engine. All of your existing models will need to be moved into the namespace and you'll need to migrate any associated tables to the new naming convention. You'll also have some custom changes depending on your application and what you need to copy over to the engine, however the rails guide walks your through these changes (I won't enumerate all of them here).
It took a coworker about a week of work to get a complicated engine copied into another complicated rails server while development on both apps was occurring and with preserving version control history. A simpler app -- like an api only service -- I imagine would be quicker to establish.
What this gives you is another namespace scope at the application root. You can change this configuration around as you add more engines and shared code to match various other directory structures that make more sense.
app
models
...
apiserver
app
...
And once you've moved your code into the engine, you can test against your engine routers:
require "rails_helper"
describe APIServer::UsersController do
routes { APIServer::Engine.routes }
it "routes to the list of all users" do
expect(:get => users_path).
to route_to(:controller => "apiserver/users", :action => "index")
end
end
You should be able to mix and match routes from both services and get cross-application testing done without launching a separate Rails app and without requiring an integration environment for your specs to pass.
Task rabbit has a great blog on how to enginize a rails application as a reference. They dive into the what to-do's and what not-to-do's in enginizing and go into more depth than can be easily transcribed to a SO post. I'd suggest following their procedure for engine decision making, though it's certainly not required to successfully enginize your api server.
You can stub requests like:
stub_request(:get, %r{^#{ENV.fetch("BASE_URL")}/assets/email-.+\.css$})
On edx.org there is Software as a Service course that grades all submitted assignments.
You upload a zip-file with a Rails project and they run a bunch of integration and unit tests.
How do they do it?
My thoughts are they mount an uploaded application as Rails engine. Is it possible to test one Rails app within another? I'd like to create a similar service, but I don't know what to start with.
I would imagine you could do this in a similar way that Jenkins runs continuous testing on a project. Where each project uploaded to your site gets expanded into a workspace, and then you shell out and execute commands. But that allows for a lot of variable configurations and complexity that probably doesn't make sense in the scope you've proposed. It also doesn't protect your underlying OS from the projects your testing.
You could also probably use an application container like docker and manage each uploaded application that way as well, which would keep everything self-contained, and isolate the application from the OS. It also puts the onus on the developer to package and manage dependencies correctly. I'm guessing they are probably using docker or something similar, here's an example Using Docker for MOOCS
At the point you want to capture the test results and report them back, I'd think they probably use something similar to the Junit formatter for Rspec or they just parse the rspec output directly.
How can I run a watir test in the context of the app that's being tested? I'd like my test to browse the app and then access ActionMailer::Base.deliveries for emails or check models directly. This is how I understand what's being described here.
UPDATE: They probably use Capybara to be able to acces the email array and be in the context of the "server" which is instantiated just for the test.
I suggest checking out the Rails unit testing docs, then writing a simple Rails test case that starts your app - then try adding a line or two of Watir code to access your app:
http://guides.rubyonrails.org/testing.html
As far as I know you should be able to write a Rails unit test, then put Watir code inside one of your test methods - and if all goes well you should be able to instantiate your web app, use Watir to launch a browser to test it, and in the same method(s) perform non-Watir low-level testing (e.g. checking models/data/etc.)
I've never used Watir inside a Rails test, but I don't see why it wouldn't work.
Watir is about driving browsers to automate functional testing. You could I suppose use it for unit testing of the top level UI stuff, but more often in a 'unit test' context that would be done using a headless browser emulation, Capybara, celerity, or watir-webdriver using the headless option, purely for speed of operation since driving an actual browser can be slow even with a fast browser like chrome.
Most of the times people use Watir it's for more functional tests, often from a test runner framework like Cucumber, sometimes Rspec depending on your needs. You might combine that with other ruby code to access or create test data, to validate something made it into the DB from the UI, but everything in the Watir gem is all about the browser and interacting with it much like a human would, and driving the browser is it's function within the set of tools you might use.
I had the same need and found the following solution: https://stackoverflow.com/a/9687562/90741. I reproduce it here, as the linked question seems to be dead, with its owner...
I had the same need (Watir+RSpec), but the problem is: there is no Rack stack running by default during tests...
I found this solution:
In spec_helper.rb:
HTTP_PORT = 4000
test_instance_pid = fork do
exec 'unicorn_rails -E test -p %d' % HTTP_PORT
end
at_exit do
Process.kill "INT", test_instance_pid
Process.wait
end
Which start the test stack once for all spec tests, and kill it at the end. (In this sample, I am using unicorn, but we can imagine using any other server)
In the spec, I reuse the HTTP_PORT constant to build URL:
browser.goto "http://localhost:#{HTTP_PORT}/"