I am on a development team where we have 2 separate mobile apps. One of the apps is for Users. The other app is for Admin of those users. My main objective is to execute a test case in the Admin app, and then run a test case in the Users app to verify its working properly. How can I approach this?
For example, I want to run a test case in the Admin app that revokes some privilege. I then want to run a test case from the Users app to check to confirm that privilege was revoked.
Maybe this is not a good strategy at all -- but it makes sense for my team because we have 2 apps that work together -- and if we do some function in the Admin app -- we want to see the expected result in the Users app
My plan was to mark each test with a Category, for example, "Privilege"
On Jenkins:
Run "Privilege" Category on the Admin app where I revoke some privilege
Run "Privilege" Category on the User app where I confirm revoked privilege
This seems like an ok test strategy right now. But if I have 20 UITests that means I'll have 20 different Jenkins projects in my dashboard, one for each UITest (per device, per platform). It seems that with 20 UITests I'll end up with over 100 Jenkins projects. Thats not really ideal to me.
Has anybody else come up with a testing strategy where they needed to test 2 separate projects back and forth. I understand that this does not really fall under unit testing - and I may get some vague answers around unit testing and general. But I do believe mobile is a different animal in the UITest world
There are couple of points in your question
do some function in the Admin app -- we want to see the expected result in the Users app
If you need to test such integrations between the two apps, you can go with proper labels for ones that are
mark each test with a Category
in any case, you will need some way to organise your suites. Good way to do so, are test annotations. I think Lazy setup is aplicable in your case. It will set the desired state for all marked tests, when needed.
needed to test 2 separate projects back and forth
End-to-end tests are mandatory, for the most business critical features. My suggestion is to employ Backdoor manipulation. Your other tests should already have covered the simpler cases (e.g. setting a privilege in Admin app), so if you already did exercise this feature, no point of redundancy.
It seems that with 20 UITests I'll end up with over 100 Jenkins projects. Thats not really ideal
You actually don't need a Jenkins project per suite, just configure the tests via CLI arguments and your harness will pick that up for you. What you need is a tag (or platform, or device) to be passed to the runner.
Generally, you do NOT want tests to depend on each other. Have a look at this example:
In the admin app, you set the privilege.
You open the user app.
The privilege should be set, but it's not.
You know that something went wrong, but you don't know whether it's the admin app that's not working or the user app.
Therefore, you should test them independently by mocking (=faking) the backend:
Open the admin app.
Set the privilege.
Ask the mocked backend: Did you receive a call from the admin app to set the privilege?
In an independent UI test for the user app, you do the following:
Set up a fake backend where the privilege is set
Open the user app
See whether the privilege is set.
By separating the tests for both apps, you will know which of the two does not work.
The software that I use at work to do such things is called WireMock but there are others out there, too.
Related
I have a mobile app developed for number of different companies on AppStroe. Each company is using different endpoint to server/icon/logo. I have managed to add this in custom plist file and according to companies endpoint, I switched to different build setting.
Now these companies are going to different ways of authentication. One is using another app to authenticate and one is using server calls. Also for one company I am receiving datas one from server call and other one from local files.
I have to handle different login behaviour for different projects. It is mostly display/disable some extra views. I don't want to have two repositories or branches. Because almost %85 of functionalities are same. I want to add functionalities same time to both and some times to only one of them and run my tests and all.
I am looking for some way way to manage this app to maintain most functionalities and keep it only one app. How can I do that? Any suggestion?
This is a very old problem. Basically you have two options: build time and run-time; from your description it may be that you need a both (I would not trust configuration to drive my authentication code).
Build time means using conditional compile (e.g. Which conditional compile to use to switch between Mac and iPhone specific code?) and a different build profile for each customer. I assume that Xcode Targets (see How to manage the code of multiple, very similar Xcode projects) allows you to define different build profiles.
Run-time checks maps to Feature Toggling.
I suggest not using version control to manage nuances of the same application because it quickly becomes a merge nightmare, even with Git.
I am looking for a solution for the following scenario.
I am writing the cucumber-capybara tests for the Ruby on Rails application.
I have multiple cucumber feature file with several scenarios (say Scenario B...Z) which are dependent on one particular scenario (say Scenario A). I want to run the scenario only once for all the scenarios which are dependent on it.
So if i run the scenarios (B....Z) I wanted to run the dependent scenario (A) only once. I came across Before hook in cucumber but it will run once for every scenario.
I have one feature file and in that there is one scenario which provisions a server. (Scenario A)
I have multiple other feature files and multiple scenarios (Scenario B..Z) which will run the tests assuming that the server is actually provisioned (Scenario A)
So whenever someone runs the dependent scenarios (Scenario B..Z ), it should check if server is provisioned by some other scenario then it should not try to provision the server. As it will increase the no.of servers.
You could write a feature with scenario A, and then write a Given block which will set the initial expectation for what scenario A is about, Then you can call that before scenario B..Z
So, assuming scenario A is to sign in, then write a single feature for signing in, and then for scenario B to Z, you write a given block that set a user as signed-in, and use that as:
Given user is signed in
...
where Given user is signed in is defined like:
Given /^user is signed in$/ do
# code to sign a user in
end
There is a certain flow within our application that makes separate calls with related data. These are in different controllers, but interact with the same user.
We're trying to build a test to confirm that the full flow works as expected. We've built individual tests for the constituent parts, but want a test for the full flow.
E.g., we have a user who checks in to work (checkin) and then builds a widget (widgetize). We have methods that will filter our users between who have checked in, and who have widgetized (and checked in). We can build little objects with FactoryGirl to ensure that the filter works, but we want a test that will have a user check in, another user check in, and the second one widgetize so that we can confirm that our filtering methods only capture the users we want it to capture.
My first thought was to build an rspec test that simply made a direct call to checkin from the widgetize spec, and then confirm the filter methods -- but I discovered that rspec does not allow cross controller calls (or at least I could not figure out how to make it work; posts and gets to that controller were not working). Also, people told me this was very bad practice.
How should I go about testing this?
This article goes over how you can use request-specs to go through integration tests fairly well.
http://everydayrails.com/2012/04/24/testing-series-rspec-requests.html
Basically you want to use a gem like capybara so you can simulate user input and get your tests to run through your app and check to see if everything is going as you expect it to.
I have developed a jBehave story to test some work flow implemented in our system.
Let’s say this story is called customer_registration.story
That story is a starting point of some other more complex work flows that our system supports.
Those more complex work flows are also covered by different stories.
Let’s say we have one of our more complex work flows covered by a customer_login.story
So the customer_login.story will look somehow like below:
Story: Customer Login
Narrative:
In order to access ABC application
As a registered customer
I want to login into the application
Scenario: Successfully login into the application
GivenStories: customer_registration.story
Given I am at the login page
When I type a valid password
Then I am able to see the application main menu
All works perfectly and I am happy with that.
3.The story at point 1 above (customer registration) is something I need to run on different sets of data.
Let’s say our system supports i18n and we need to check that customer registration story runs OK for all supported languages, say we want to test our customer registration works OK with both en-gb and zh-tw
So I need to implement a multi_language_customer_registration.story that will look something like that:
Story: Multi language customer registration
Narrative:
In order to access ABC application
As a potential customer
I want to register for using the application
Scenario: Successfully customer registration using different supported languages
GivenStories: customer_registration.story
Then some clean up step so the customer registration story can run again
Examples:
|language|
|en-gb |
|zh-tw |
Any idea about how I could achieve this?
Note that something like below is not an option as I do need to run the clean up step between runs.
GivenStories: customer_registration.story#{0},customer_registration.story#{1}
Moving the clean up step inside the customer registration story is not an option too as then the login story will stop working.
Thanks in advance.
P.S. As you could guess in reality the stories we created are more complex and it is not an easy task to refactor them, but I am happy to do this for a real gain.
First off, BDD is not the same as testing. I wouldn't use it for every single i18n scenario. Instead, isolate the bit which deals with i18n and unit test that, manually test for a couple and call it done. If you really need more thorough then use it with a couple of languages, but don't do it with all of them - just enough examples to give you some safety.
Now for the bit with the customers. First of all, is logging in and registration really that interesting? Are you likely to change them once you've got them working? Is there anything special about logging in or registration that's particular to your business? If not, try to keep that stuff out of the scenarios - it'll be more of a pain to maintain than it's worth, and if it's never going to change you can just test it once, manually.
Scenarios which show what the user is logging in for are usually more enticing and interesting to the business (you are having conversations with the business, right?).
Otherwise, here are the three ways in which you can set up a context (Given):
By hacking the data (so accessing the database directly)
Through the UI (or controller if you're automating from that level)
By using existing data.
You can also look to see if data exists, and if it doesn't, set it up. So for instance, if your customer is registered and you don't want him to be registered, you can delete his registration as part of setting up the context (running the Given step); or if you need him to be registered and he isn't, you can go through the UI to register him.
Lastly, JBehave has an #AfterScenario annotation which you can use to denote a clean-up step for that scenario. Steps are reusable - you can call the steps of the scenario from within another step in code, rather than using JBehave's mechanism (this is more maintainable anyway IMO) and this will allow you to avoid clearing registration when you log in.
Hope one of these options works for you!
From a tactical standpoint, I would do this:
In your .story file
Given I set my language to {language}
When I type a valid password {pass}
Then I am able to see the application main menu
Examples:
|language|pass|
|en-gb |password1|
|zh-tw |kpassword2|
Then in your Java file,
#Given ("I set my language to $lang")
#Alias ("I set my language to {language}")
// method goes here
#When ("I type a valid password $pwrd")
#Alias ("I type a valid password {pass}")
// method goes here
#Then ("I am able to see the application main menu")
most unit testing frameworks support this.
Look how mstest you can specify DataSource, nunit is similar
https://github.com/leblancmeneses/RobustHaven.IntegrationTests
unfortunately some of the bdd frameworks i've seen try to replace existng unit test frameworks when it should instead work together to reuse the infrastructure.
https://github.com/leblancmeneses/BddIsTddDoneRight
is fluent bdd syntax that can be used with mstest/nunit and works with existing test runners.
It's quite common in sites- you have a "demo" version with a guest account full of data/posts/comments that you can play with, and all the data is reset every few hours so users wont spam the demo site.
I thought to have another rails environment, "mysite_demo" and use a cron job to call rake to reset it's database every X hours, and populate the seed data.
Then it hit me that all over my app I'll have to check if I'm running in "demo-mode":
For example, if the demo site has a login/register page too, a user might register, insert some data and wonder why his account is deleted after he logged in again.. so demosite shouldn't have a register option at all.
So I thought I'll make a "demo" branch of the code.. with the difference and just merge changes as I go... sounds like an overkill.
ideas?
In my application I started with a fixed demo user with an account that resets every hour. Something about that model didn't quite sit right - if there were multiple users hitting the demo at the same time you could get into some weird concurrency issues. And what if a user is in the middle of a demo and your reset the demo account? What happens?
I don't know if this model works for you but I ended up creating a brand new user account with a demo flag set in the database - I also automatically log the user in. This way the user gets to play around for as long as they like and I don't have to worry about data getting deleted/changed while a user demos my app. I run a cron job every night that deletes users with the demo flag set that are older than 24 hours.
If the demo version is running from its own database, how is it any different from the real thing? The demo site is just an instance of your product.
Just clean up the DB and redeploy the demo as needed. Is it just this simple or am I missing something?
Then it hit me that all over my app I'll have to check if I'm running in "demo-mode" (e.g, you cant register a new user in the demo) and make the site behave accordingly.
If the site is in demo, why does it matter what the users do? Anything they do will be wiped in a few hours, so they won't be able to actually do work with it.
It sounds like you are trying to handicap the site so they will pay. I don't know what your site does, but if its a host based service(web page that stores & display information) then the limited life span of the data should deter squatters.
If you website does something that can be used elsewhere, then I can see limiting it. An example might be a service that transforms media formats, or writes resumes. If the user can do something useful in the 2 hour window and walk away with it, then you might consider branching.
Why not allow the user to make an account even if it is deleted in an hour?
That allows them to see how the registration process of the script works for at least an hour, maybe give a message on the signup page that the account is only valid for an hour.
Just my thoughts
Is there any other functionality that is different in the demo version than the production environment? If it is just an issue of making the user register, you could just create a registered demo account in production, and give out the user name/password for people. Although this may not be an option depending on other business requirements.
If you are willing to use Authlogic you can take a look at this, then every X hours you can look through the database for users that start with anonymous_ and delete records that are associated with them.
Just make a separate demo site that works exactly like the production one, but the DB gets reset once an hour to clean example data. The only change you need to make is a banner across the top of every page that says its a demo. There are several ways to do it, (modify your site theme, or maybe use frames) but basically you should only have to change the code in one place, instead of throughout the site.
You could setup a new environment demo on your database.yml, with read-only privileges for the User table, and an additional demo_database. Then place some checks on your code to see if your RAILS_ENV is on DEMO.
That way, you only need to work with the same codebase and just show whatever you feel like it.
You can deploy it as a separate app with its own database to a separate domain or subdomain and then check the domain to decide what options should be available. For instance if you put it on demo.example.com you would use:
if request.domain =~ /demo/
If you use Capistrano you can set it up to update both apps when you deploy so they are in sync.