I am using Steak and Capybara to do my integration tests. I also would like to test the serialization output (webservices using XML and JSON format) of my models. The problem is that JSON is only available by using a XML HTTP Request. So Capybara's visit method does not work. It also seems that I don't have access to xhr method in my acceptance tests (not sure why, because I require the normal spec_helper.rb in my acceptance_helper.rb).
How do you test web services? Using the xhr method (after requiring it somehow)? A special method with Capybara? Something else?
I also have some custom serialization (beside the normal my_object.to_xml). I guess it is better to do that in the model tests. Would you then still test the web service output of those custom serializations?
You can always create a "proxy" controller that makes requests to the web service and prints the result. It should be available only in test environment, obviously. Then your Capybara test visits the proxy and tests page content.
Related
I want to use Cypress as a testing tool with the cypress-on-rails plugin.
However during a cypress scenario I want to enable/wrap all rails backend requests with vcr so all requests are captured and replayed.
Typically you would tag a rspec or cucumber file that essentially wraps an entire block of code to perform this. The nature of cypress is that it's completely client/javascript driven and a scenario plays out with multiple ajax requests from the client.
The insert/eject methods on which the use_cassette method is built are publicly available.
https://www.rubydoc.info/gems/vcr/VCR#insert_cassette-instance_method
VCR.insert_cassette('my_cassette')
# do stuff
VCR.eject_cassette
My frontend is an EmberJS-based app. It's totally async in nature, so testing it with Capybara is pain and misery. On the other hand, Ember provides a fantastic test suite out of the box which makes acceptance testing fun and effective.
Normally, either fixtures or backend mocks are used to acceptance-test Ember apps. But testing against mocks does not satisfy me at all:
It will not reveal possible API inconsistencies between the backend and the frontend, race conditions, etc.
It is impossible to test backend business logic this way. Such tests are not integration tests.
Finally, acceptance tests require persistency, so you have to replicate backend behavior in a mock. It's very tedious and you effectively end up implementing two backends.
So I want to test against the real backend! It's trivial to set up Ember to use a local backend instance for testing. But the problem is that the backend will persist its state between individual tests and even test sessions.
That's why I'm thinking of implementing a special public API in Rails:
The API is only available when Rails is run with a specific flag or an env var.
Rails runs in a non-testing mode, serving normal API calls as it would in production.
Before each test, the frontend calls the special API, telling Rails which database setup is needed for this specific test.
When a call to the special API is received, Rails cleans the database and populates it with the requested data. For example, to test item deletion from the cart, the database should have three items in the cart.
Rails finishes the API request, and the frontend starts the test.
Frontend runs test steps, using the normal backend API as it would in production: log in, create posts, comment on them. It will also try doing some forbidden things, e. g. edit posts while not logged in, exceed text length constraints, etc and verify whether the backend rejects forbidden actions.
When the frontend runs next test, it will call the special API again. Rails will discard the state produced by the previous test and set up a new one, for this specific test.
I'm a frontend dev with a sketchy knowledge of Rails. Factory Girl and Database Cleaner seem to be the right tools for the job, but there is absolutely no information how to use them outside Rails' normal test environment. I guess I need a controller or a Rails engine or something.
So the question is: how do I make an API in Rails that can be used by the frontend to tell Rails to set up a certain database state with a fixture factory, while Rails are running in a non-test mode i. e. serving REST API and not running through RSpec/Capybara/Cucumber?
Bonus feature: fixture factory properties should be defined on the frontend so that test code is stored in one place. Thus, the backend should be able to accept fixture factory properties via the special API. Defaults can still be defined in the backend's codebase.
I believe this could become an acceptance/integration testing best practice. If I manage to implement it, I promise to publish a gem.
May be something like this
config/routes.rb
namespace 'test_api' do
resource 'db_transaction', only: [:create, :destroy]
end if Rails.env.test?
controllers/test_api/db_transactions_controller.rb
require 'database_cleaner'
def create
DatabaseCleaner.start
end
def destroy
DatabaseCleaner.clean
end
I'm trying to use webmocks to test part of my app that interacts with an external service. However, all of the API calls are actually happening within another gem. So it continues to make the requests, even though I've enabled webmock.
This properly is blocked and requires a mock:
it 'test webmock' do
Net::HTTP.get("www.google.com", "/")
end
But the test which contains API requests against the gem, which should be calling the external service is not.
Any idea how to make that test also require a mock?
Consider a Rails app that hits a (Sinatra app) API being developed separately from the Rails app. I want to test an API call from within the Rails tests.
The API code:
post '/foo/create' do
...
I created a mock, but that doesn't make sense because it is just a copy of the API file. That stinks.
It is possible to require the API file in the test. But how to call it from RSpec? There is no route in the Rails app for it.
One option is to start the API and make the HTTP call from the Rails test, but this is smelly because:
You have to start the API server to run the Rails tests
Why should a Rails test make a HTTP request? Rack::Test simulates this.
I don't think this will work because the apps have different test databases, but share the same production database.
EDIT: The point of the test is that the API call creates records that the Rails app is expecting. So the Rails app needs to test the state of the database after the API call is made.
Well. The perfect answer for you is a gem to mock the answer like webmock. It will fake a response when acessing that url, so on the test your app will make the requisition as it was for real, only that before it hits the web, it will hit your mock and respond with the desired answer.
Edit #2: Does anyone have a good method of testing the "middle" of a client-server application where we can intercept requests and responses, fake the client or server as needed, and which provides self-documentation of the api?
Cucumber might be a good solution in many cases, but it's not quite what I'm looking for. And this middle layer should be client/server implementation agnostic. (e.g., black-box).
Our client-server model is a ruby-on-rails server with a Flex client, using a RESTish interface with JSON as the data format. So anything the client posts to the server is usually a single JSON parameter. The server does it's thing and responds with a pure JSON model.
We have standard rails testing on the server and we're working on getting proper FlexUnit tests completed on the client (it's a moving target). However, there's a debate in my team about the effectiveness of the current testing model, since every change on the server seems to break part of the API. This is telling me that there is both a problem with API communication (between team members, self-documentation in code, etc..), and a lack of proper API sanity testing.
So I've been questioning whether we need to have a mock client for testing the server at a pure JSON level (without all the other complexities of a rich client), and possibly a mock-server for doing the same thing with the rich client. This would serve two purposes, to document the API and to provide more thorough testing of the API itself.
The reason there's a debate is that the rails guy claims that the rails integration testing is sufficient for testing all the server requests, and the middle-ground testing environment would simply be redundant.
So the question here is, given our situation, how should be go about self-documenting the API, and how should we test the API itself?
EDIT:
We have routes like /foo/444/bar.js, but the parameters can be virtually any complex JSON string depending on the action, e.g.:
json={
"foo":{
"x":1,
"y":2
},
"bar":[1,2,3,4,5]
}
but besides manually-edited API docs, there's no self-documentation. The rails controller often just deserializes and applies changes directly to the model. Would be nice to have common tests to tell us when it's changed, and what's expected.
I just started looking at this web functional testing tool called Maxq and I think it has the potential to solve your problem, Maxq acts as a proxy server between your web client and server application.
It sits on top of Junit so that means you could do proper unit testing for your API by asserting the behavior and responses of calls to your server app.
It basically captures and records all the requests you make from a web client and the responses you get back from a server, it also has the ability to generate test scripts of your request which you could use to play back and test on any server.
You should try it out http://maxq.tigris.org/
You can think of it as two different projects.. if you had two project, you would've writing two separate test suites right?
You should start by establishing the API between the server and the client- as if you wont have any communication between the teams after you start implementhing.
Then you build the client that consume the API and a server that produce the API (or the tests first, if you TDD).
For testing, one team need a mock-server to supply fake API responses to test the client, and the other team need to test the produced data of the server (i.e, the second team is using ails integration testing like your rails guy claims)
I would recommend Cucumber. It allows you to write specific tests for your application by emulating a browser. This way you can send requests and validate the JSON response easily.
One way you can do this is controller tests using rspec (you can also use test::unit)
describe PersonApiController
# GET /person/1.json
it "should respond with a person" do
person = Person.create(:name => "scott")
get :show, :id => person.id, :format => 'json'
response.should be_success
response.body.should have_selector('name', :content => person.name)
end
end