I have a website that I want to test using Selenium WebDriver (version 2.28), and to integrate it with my CI (Maven and Hudson).
The thing is that in order to properly test the front-end, I need to access the server (for the actual HTML and for REST resources). The test will not be fully encapsulated if I access the actual server that I build and deploy during the build process, since the data might be different each time (based on the data that resides in the DB of the built environment at the time of the build).
Therefore it seems to me that I have to mock the server access.
I just don't find a support for this in Selenium, and I can't quite think of the best way to do that without a Selenium support.
I can create mocks to all of my resources (HTML and REST) in the actual server, on different URLs, and access those in my tests instead of the production-level ones (with some kind of a flag to indicate that I should access mock resources instead of real ones). But that's not really mocking...
For reference, I saw that in the Sahi test framework there's a feature of addMock(url, class_function) - in which when Sahi runs into the specified URL, it will call the specified class and function instead of access the URL, whereas the class and function should provide the resource instead of the URL providing the resource.
I'm looking for something similar in Selenium (though perhaps it's not possible, since Sahi functions as a proxy).
Many thanks in advance,
Daniel
Related
I have two software components: My application and a library (which is owned by my company, but another team), which is used by the application. The library is a client library for some service and performs HTTP requests. The library also maps the HTTP response to the internal representation of the application. This is done by injecting a mapping class into the library by the application.
I already have unit tests for the mapping class and for the application, whereas the client library call is always mocked.
Now I'm thinking of integration test the library and I'm not sure what's the best way to do that:
Mock the library call and only check that it's called with the correct parameters
Pro: If the internals of the library change (with a non-breaking change) I don't have to adapt my tests.
Con: The mapping class isn't integration tested. I can't be sure that the library is correctly configured or that the parameters the mapper gets from the library are what I expect them to be.
Mock only the HTTP call done by the library
Pro: The mapping class and the configuration of the library (if I configured it correctly) are tested.
Con: I need to figure out the internals of the library and check how the HTTP call of each test case will look like. Also if the library upgrades to a new version of the service, I would need to adapt all the HTTP mocks as well and actually shouldn't care about how the library internally works.
Replace the HTTP call in the library with an in-memory fake (=dummy) implementation during testing
Pro: Everything is tested + the tests are resilient against library changes.
Con: It's an effort to implement and maintain a fake implementation. Depending on the service, this could mean rebuilding the functionalities of the service in the library. Who should be responsible (implementation + maintenance) for the fake strategy? My team or the team that owns the library?
I'm in favor of the last point, but given that the internals of our library rarely changes, I'm not sure if a fake strategy is worth the effort.
What's your opinion on this? Can you think of another solution?
I would create helpers inside the library that would allow you to mock the HTTP response. Therefore, you'd see the code running inside the library and you could use the libraries that verify the JSON format to make sure that the http request/response is the one you're expecting.
In this sense, you're checking i) the library actually works with your system; ii) the processes the correct HTTP response; thus, your helper could be easy enough so that developers would only need to provide the content of the http response
The application is using Minitest on Rails 4 with Capybara.
I'd like to write an integration or feature test that stubs a websocket connection (the application uses Faye as a client) to return a specific message (like I'm used to doing with Webmock).
Is this possible? If so, can you provide an example? My research has not turned up any examples.
Your research hasn't shown up any examples because it's not really what you're supposed to be doing in feature tests. Feature tests are supposed to be end-to-end black box tests where you configure the data as required for the app to generate the desired results and then all interaction is done via the browser (no mocking/stubbing which technically alter your apps code). Additionally when connections between the browser and a 3rd party service are involved there is nowhere in your app where you can mock it.
It may be possible to stub a websocket connection from the browser with a programmable proxy like puffing-billy, however it's generally cleaner to produce a small fake version of the 3rd party service for testing purposes (sinatra app, etc) and point the browser at that rather than the original service when you need to craft custom responses. Additionally, there are a lot of fakes already out there, depending on what service you are using (fake-stripe, fake-s3, etc), which may provide the functionality you're looking for.
We are using both Angular and Rails applications at our company. Is there a way to mock a web page to test the UI? Essentially I want to jump midway into an application, so I don't have to take time logging in, creating our object, tweaking the object, then finally getting to what I can test.
I was looking at something like MSL, but am unsure if it's really what I need.
Turns out MSL is what I wanted. My application depends on other applications for communication. In order to isolate my isolation, I can use MSL to mock the responses from and the requests to my dependent applications fairly easily. In my configuration I switch my external dependency to localhost:8001 (or whatever port msl is running on).
Edit
I rewrote MSL, specifically for my purposes and made it a little easier. The server is a node program, and the client is only in ruby as of now.
mobe-server
mobe-client
Lets say; I am developing a Web Application which talks to a RESTful web service for certain things.
The RESTful web service isn't third party, but is being developed parallely with main application (A good example would be, E commerce application and payment processor; or Social network and SSO system).
In such a system, acceptance(cucumber) or functional tests can be done in two ways:
By mocking out all external calls using object level mocking library; such as Mocha or JMock.
By doing mocking at http level, using libraries such as webmock.
By actually letting the main application make the actual call.
Problem with #1 and #2 is, if API of underlying application changes; my tests would keep passing and code will actual break and hence defeating the purpose of tests in first place.
Problem with #3 is, there is no way I can do rollback of data, the way test suite does on teardown. And I am running my tests parallely and hence if I let actual web services go, I will get errors such as "username taken" or stuff.
So the question to community is, what is the best practice?
Put your main application in a development or staging environment. Spin up your web service in the same environment. Have the one call the other. Control the fixture data for both. That's not mocking; you're not getting a fake implementation.
Not only will this allow you to be more confident in real-world stability, it will allow you to test performance in a staging environment, and also allow you to test your main application against a variety of versions of the web service. It's important that your tests don't do the wrong thing when your web service changes, but it's even more important that your main application doesn't either. You really want to know this confidently before either component is upgraded in production.
I can't see that there is a middleground between isolating your client from the service and actually hitting the service. You could have erroneously passing tests because the service has changed behavior, but don't you have some "contract" with the development team working on that service that holds them responsible for breakage?
You might try fakeweb and get a fresh copy of expected results each day so your tests won't run against stale data responses.
I use a web service to convert files. The service returns the converted file as an HTTP POST, along with identifier data. My app receives the response, updates its database and saves the file to the appropriate location.
At least that's the idea, but how do I develop and test this on a local machine? Since it isn't publicly facing, I can't provide a directive URL. What's the best way to handle this? I want to keep the process as clean as possible, and the only ideas I can come up with have seemed excessively kludgey.
Given how common REST API development is, I assume there are well-established best practices for this. Any help appreciated.
The solution will change a bit depending on which server your using.
But the generally accepted method is using the loopback address: 127.0.0.1 in place of a fully qualified domain name. Your server may need to be reconfigured to listen on this IP address, but that's usually a trivial fix.
example: http://127.0.0.1/path/to/resource.html
You can use curl or even your browser if your application has a proper frontend. There are many other similar tools to test this from a command line, and each language has a set of libraries for establishing http connections and transferring data along them.
If your machine isn't accessible to the service you are using, then your only option would really be to build a local implementation of the service that will exercise your API. A rake task that sends the POST with the file and the info would be a nice thing so you could start your rails app locally, and then kick off the task with some params to run your application through its paces.
This is the case any time you are trying to develop a system that can't connect to a required resource during development. You need to build a development harness of sorts so that you can exercise all the different types of actions the external service will call on your application.
This certainly won't be easy or straight forward, especially if your interface to this external service is complicated. Be sure to have your test cases send bad POSTs to your application so that you are sure you handle both what you expect, and what you don't.
Also make sure that you do some integration testing with the actual service before you "go-live" with the application. Hopefully you can deploy to an external server that the web service will be able to access in order to test. Amazon's EC2 hosting environment would let you set up a server very quickly, run your tests, and then shut down without much cost at all.
You have 2 options:
Set up dynamic dns and expose your app to the outside world. This only works if you have full control over your network.
Use something like webrat to fake the posts to your app. Since it's only 1 request, this seems pretty trivial.
Considering that you should be writing automated tests for this, I'd go with #2. I used to do #1 when developing facebook apps since there was far to many requests to mock them all out with webrat.
If your question is about testing, why don't you use mocks to fake the server? It's more elegant than using Webrat, and easier to deploy (you only have one app instead of an app and a test environment).
More info about mocks http://blog.floehopper.org/presentations/lrug-mock-objects-2007-07-09/
You've got some info about mocks with Rspec here http://rspec.info/documentation/mocks/