I have a VM where MS CRM is installed and can be access using http://localhost:5555/Orgname/main.aspx.
I have created Unit test cases in my VM by refering the Organization Url as
http://localhost:5555/XRMServices/2011/OrganizationService.svc?wsdl
When I build the test project it connects to CRM and executes the test methods without any error.
Where as when I do a check-in ,the build is getting failed due to the reference to url "localhost".
for Build we have a separate Build server.
Can any one let me know how to solve this.
Your tests are being executed on the build server and it looks like some of your tests are of the Integration kind and not of the Unit kind, as such it's looking for a configured CRM instance on that server (localhost resolves to the host itself for every machine), and can't find any. Which means you have a few options:
Install CRM on the build server, extend the build process to deploy CRM to the build server during build in order to run your tests. A build process like the one developed by Wael Hamze can be extremely helpful for such a solution.
Do not include a localhost address, but actually check in a location that points to a shared dev environment, the build server can connect to. This is not ideal, as the build may be dependent on specific data being present and concurrent builds may break due to strange interactions. If you configure the build agent to only run 1 concurrent builds, it may work.
Disable the tests that depend on an installed version of CRM. You could put a [TestCategory("Integration")] on these tests and then set a filter condition on the build to exclude this test category.
Or you could try to improve your tests by making them independent of your configured instance, using Fakes or any other mocking framework. There are several testing frameworks specifically made for CRM workflow activities and other parts specific to CRM.
You need to remove the Integration Test from the list of Unit Tests that are executed as part of the build. I recommend creating a new project called [projectundertest].IntegrationTests and adding all your integration tests there. Then configure the build to only execute UnitTests...
You build server is trying to execute all of the tests in your solution. Unfortunately you have an Integration Test masquerading as a Unit Test. When it is executed the test tried to access CRM and fails. This is correct behavior.
Related
I'm just getting started with Jenkins and I have a few doubts that must be silly, but I'm stuck at it.
After I build my project Jenkins save the build file in some specific path?
Using Ranorex for automation test, is it better to put my files locally on the server or push them to a repository?
Note: I just start tried to use this, at this moment I can check for changes at BitBucket, build the file, build the Ranorex test and run the test.
Jenkins is quite a versatile application that allows system setup to specific needs and requirements of the test project. So i'd say go with the way that seems most logical/easiest. It's kind of a learning process as well so you will be able to understand the working flow of Jenkins itself.
But to answer your 2 questions:
1) By build files i believe you mean the test reports? - For this I actually use the Jenkins UserContent folder. This requires the "Copy to slave" plugin to be installed. With this you will get an additional Post-build Action where you can specify the files that will be copied over to the UserContent folder. But don't forget to specify a common layout for the naming of report files through the Ranorex run parameters ("/rf"). The UserContent folder actually acts as a web server and you can directly link the URLs for email reports. (eg. "http://Jenkins-server.com/UserContent/Regression-Client-Test-#1.html")
2) This totally depends on the system setup. But i can give you an example on how our system is currently set up. So we have Jenkins which runs on a Linux machine. It is only used to manage and run the tests and the actual machine does not include the automation test project. Then we have the test machine which runs on Windows and holds the actual automation tests. This machine is connected to Jenkins through the Slave functionality. So basically when someone starts a test job Jenkins from the Linux machine sends a command to the slave to start the automated tests. When the test run has finished post-build actions take over and copy the report files from the Slave machine to the Linux machines UserContent folder.
Now when talking about the test project management. It's a good idea to use a gir repository which will add another layer of somewhat "security". But if you have a small team (or you are the only test developer) then there is no actual need for it. You just copy the project to the test machine to a specified folder whenever needed/updated and you are ready to run it.
Regards,
Martin
In our TFS 2013 project we have a set of simple MSBuild based integration tests alongside our unit tests which test stored procedures and other logic which need a database server to be present, for example
[TestMethod]
[TestCategory("Integration")]
public void SomeTest()
{
InitialiseData();
var results = RunStoredProcedure();
AssertResultIsCorrect(result);
}
As you can see we have tagged these tests as "Integration" so that we can supply a test case filter that excludes these tests when we don't want to run them. These tests are all designed to run against an installed copy of our database (we have a process which deploys an up-to-date copy of our database to a SQL Server instance).
What we would like to do is create an integration test build definition in TFS which we can schedule to deploy our database and then run all of these tests against a specific SQL Server instance.
At the moment we use the standard TFS build process with two test sources - the first runs a "unit test" which installs the database, the second contains all of the actual integration tests. The problem we have is passing the connection string & database name into the unit tests from the build definition. Currently the connection string is in app.config files for each of the test projects, however this is less than ideal as it means that we are constantly getting failing test runs, either due to developers checking in the wrong connection string, or running tests locally against the build database at the same time that a build is running. This setup also limits us to running one build at a time.
Is there a way that we can specify the connection string and database name to use as part of the build workflow template instead?
With a combination of SlowCheetah for your config transformation and VS linked files, I think you can solve this (and based on the OP you probably already have :). Make a new solution configuration in your solution for the scenario you described. This solution will not be used on dev machines, only by TFS build definition (under Process, Items to build, Configurations to build).
The Configuration Manager for the solution would then use the solution configuration only for the test proj.
Add your SlowCheetah transform for your new solution configuration and put in your db conn string you need for TFS for that new transform.
Now in the tests project, copy over all the config files as linked files. This will allow the test executions to respect the test config file that SlowCheetah will transform. You may have to adjust your configuration reading in your test proj(s) to account for this.
This isolates the solution configuration to only the TFS server since only it will be building with your new solution configuration. Now TFS will have a config file that points to your specific TFS database connection that no other machines respect.
I just pushed my project up to TFS online (visualstudio.com) for the first time and figured I'd try and run some tests while I'm at it. I ran them and they failed because I didn't set up a sql database - these tests have only ever run locally.
How does one go about running tests on tfs / configuring sql server? Do I have to somehow point to a database, something hosted on azure perhaps?
Totally new to this so any references / general tips would be much appreciated.
If you need to do some advance build / test, whether it's linking against libraries that aren't installed on the default build agent or setting up a SQL server to test against, then you will almost certainly need to provision your own build agent.
You can still use Team Foundation Service's source control and queueing of continuous integration, and you need not host your own build agent (you can set up an Azure VM). But if you've outgrown the simple build agent included, you'll need to configure your own.
You can attribute you test methods with the TestCategoryAttribute and the value of Database.
Then configure your Build Definition to filter only run test that are not database. This will stop them running on the Build Server, and just run them locally.
Example Test
[TestClass]
public class Test
{
[TestMethod, TestCategory("Database")]
public void TestMethod
{
Assert.Fail();
}
}
And in your build definition (if you are using the Visual Studio Test Runner):
TestCategory!=Database
You can set it here:
The background is : I am trying to implement an automated integration test solution. I want to have a FitNesse server running which QA/Users can maintain the test cases. During our nightly build, we want to have the test run locally in the build machine. (In our build script, we are going to startup Jetty, and FitNesse test cases are invoking the RESTful APIs)
When I am looking into the fitnesse-maven-plugin (http://mojo.codehaus.org/fitnesse-maven-plugin/), in the description of goal fitnesse:run, it said that:
This goal uses the fitnesse.runner.TestRunner class for calling a remote FitNesse web page and executes the tests or suites locally into a forked JVM
However, when I am using this plugin with FitNesse version 2009xxxx or 2008xxxx (with a special patch of this maven plugin), I found that the test is not running locally. Instead, I saw new test results created in the remote FitNesse wiki server.
May I know if it is due to change of behavior of FitNesse? (Coz the fitnesse maven plugin is depending on a much older version of FitNesse) Also, with the original Test Runner being deprecated, is it possible to have the behavior I am looking for? (Pages defined in remote server, but run locally in build machine)
Or, is such way of work no-longer a recommended approach to use FitNesse? (If so, I will need to change the approach of the automated test)
One solution I've used is the wiki import option feature. This can import the latest changes from the remote wiki to your local build server.
http://fitnesse.org/FitNesse.UserGuide.WikiImport
You can also tell it to auto-update when running the tests rather than having to re-import manually whenever they change.
Another possibility is to use a source control plugin to automatically commit changes by QA/Users from the remote wiki and pull them down as part of your build.
Hi I am using TFS and want do deploy two web applications after build to a test server.
After the deployment I want to call a web page at one of the web applications to trigger some work like, delete all data and insert test data.
How can I accomplish this?
You could have a look at TFSDeployer. If you are comfortable with Powershell, you can write a deployment script that does whatever you want, and TFSDeployer will execute it when it detects changes to the build quality (i.e. you set build quality to "deploy to test" for example and it will run the script to deploy to the test environment). There's more information on the site, and this is what I use almost always for deployments.
You can MS Deploy to deploy your applications to the test environments: http://www.ewaldhofman.nl/post/2010/04/12/Auto-deployment-of-my-web-application-with-Team-Build-2010-to-add-Interactive-Testing.aspx.
To call a web application, you can either create a codedui that you run as part of your testing, or you add an activity to your build template that calls the web site: http://www.ewaldhofman.nl/?tag=/build+2010+customization
You can use DeployToAzure. It publishes specified cloud project to Azure as a part of your TFS build workflow. Publishing is executed after solution is built and unit tested are passed. You can include some steps (like call web page) after that.
And you don't need any Powershell programming for that.
Check here for additional information: Deploying Azure Project from TFS 2010 Build Server