In my project we are using QC to execute our test cases(QTP), moving forward we would be eliminating QC (for cost reasons).
As far as I explored MSBuild & Jenkins, they would be suitable.
But MSBuild will trigger the execution when a new build pushed to the repository. Also it will automatically test on the latest build.
Is there any other CI tool available to execute test cases through QTP?
I will be executing automation once in a release. Also we install our application by manual since it requires lots of configuration.
Take a look at HP Application Automation Tools.
This plugin basically replaces the need for QC, and is developed by HP.
Create a Jenkins job using this plugin on the same Jenkins installation used to build your code, then you can configure your job to run your tests as soon the code is available (e.g. on a nightly basis).
See here for a helpful guide on how to implement a simple Jenkins job using this plugin.
They also host the code on Github, which is very useful if you need to change the behavior of the plugin to suit your needs.
Related
I'm just getting started with Jenkins and I have a few doubts that must be silly, but I'm stuck at it.
After I build my project Jenkins save the build file in some specific path?
Using Ranorex for automation test, is it better to put my files locally on the server or push them to a repository?
Note: I just start tried to use this, at this moment I can check for changes at BitBucket, build the file, build the Ranorex test and run the test.
Jenkins is quite a versatile application that allows system setup to specific needs and requirements of the test project. So i'd say go with the way that seems most logical/easiest. It's kind of a learning process as well so you will be able to understand the working flow of Jenkins itself.
But to answer your 2 questions:
1) By build files i believe you mean the test reports? - For this I actually use the Jenkins UserContent folder. This requires the "Copy to slave" plugin to be installed. With this you will get an additional Post-build Action where you can specify the files that will be copied over to the UserContent folder. But don't forget to specify a common layout for the naming of report files through the Ranorex run parameters ("/rf"). The UserContent folder actually acts as a web server and you can directly link the URLs for email reports. (eg. "http://Jenkins-server.com/UserContent/Regression-Client-Test-#1.html")
2) This totally depends on the system setup. But i can give you an example on how our system is currently set up. So we have Jenkins which runs on a Linux machine. It is only used to manage and run the tests and the actual machine does not include the automation test project. Then we have the test machine which runs on Windows and holds the actual automation tests. This machine is connected to Jenkins through the Slave functionality. So basically when someone starts a test job Jenkins from the Linux machine sends a command to the slave to start the automated tests. When the test run has finished post-build actions take over and copy the report files from the Slave machine to the Linux machines UserContent folder.
Now when talking about the test project management. It's a good idea to use a gir repository which will add another layer of somewhat "security". But if you have a small team (or you are the only test developer) then there is no actual need for it. You just copy the project to the test machine to a specified folder whenever needed/updated and you are ready to run it.
Regards,
Martin
Has anyone used Jenkins in Xcode for code management and automated deployment ? If yes, how can it be integrated ? How does automated deployment work with Jenkins ? Went through the documentation but didn't get much idea.
I went through the Jenkins plugins for iOS but not clear if we can publish the build to AppStore using any plugin.
We are using a Jenkins Server for Continuos Integration, by fetching the source from an SVN Server when the Jenkins is triggered by Commits to this repositories.
But, to be serious, i wouldn't recommend the jenkins... i'd rather use xcode server/bots to get rid of all the hassle with the jenkins...
fastlane is a tool for iOS and Android developers to automate tedious tasks like generating screenshots, dealing with provisioning profiles and releasing your application.
https://github.com/fastlane/fastlane
xctool is a replacement for Apple's xcodebuild that makes it easier to test iOS and Mac products. It's especially helpful for continuous integration.
https://github.com/facebook/xctool
The term "Jenkins in Xcode for code management" seems not fully correct. Because Jenkins server is for CI, in which we config it (create a job) that will fetch the source code from your repository (SVN, GIT), then compile it and run, probably execute your unit tests, UI automation tests or code coverage tool.
Based on your schedule, Jenkins server will automatically start its job or is triggered whenever there is a change in your repo (anybody commits the code for example).
The tool you mentioned which in Xcode is probably XCode Bots, the built-in CI tool.
In order to set up, firstly, you have to install and config XCode server. After that, connecting your server with your repositories. Secondly, creates a bot with your customize configuration and run it.
In my opinion in term of Jenkins and Xcode comparison, I would say that it depends on what tools or add-in functions that you want to set up for your CI server. Jenkins has many plugins might helpful like check style, measure code coverage while Xcode bot still having some limitation.
This document https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/ for your further reference.
I hope this would be helpful to you.
There is a gem called xcpretty, you can use it to output JUnit format test result, then use Jenkins JUnit Plugin to show the result.
Install xcpretty
gem install xcpretty
Use xcpretty to create junit format xml file
xcodebuild test ... | xcpretty --report junit --output [path_to_save]/unit_test_result.xml
Then this unit_test_result.xml can be used to report by Jenkins JUnit Plugin
I work in a team which maintains a Java website and back end java jobs and shell script jobs.
After all developers complete their updates, only the relevant ones are committed to source control system.
Later ant build scripts are run and war files are generated.
Along with these war files there will genrally be shell scripts etc to be copied to QA/PROD.
Then one fine day there is a team call the release management team which will transfer the code from our Dev environment to QA/PROD.
Recently I came across the Continuous Integration systems like Jenkins/Hudson.
Can these tools build all the changes committed and automatically transfer my code to QA/PROD.
BTW I work in a AIX Server environment and use Tomcat as the Container.
I am more curious whether the tool will be able to copy my code to QA/PROD.
Please Clarify.
The answer is almost certainly yes, depending on your particular setup for copying the code. There is a large number of plugins for this purposes at the appropriate Jenkins wiki page. You should be able to find something there for your needs.
The background is : I am trying to implement an automated integration test solution. I want to have a FitNesse server running which QA/Users can maintain the test cases. During our nightly build, we want to have the test run locally in the build machine. (In our build script, we are going to startup Jetty, and FitNesse test cases are invoking the RESTful APIs)
When I am looking into the fitnesse-maven-plugin (http://mojo.codehaus.org/fitnesse-maven-plugin/), in the description of goal fitnesse:run, it said that:
This goal uses the fitnesse.runner.TestRunner class for calling a remote FitNesse web page and executes the tests or suites locally into a forked JVM
However, when I am using this plugin with FitNesse version 2009xxxx or 2008xxxx (with a special patch of this maven plugin), I found that the test is not running locally. Instead, I saw new test results created in the remote FitNesse wiki server.
May I know if it is due to change of behavior of FitNesse? (Coz the fitnesse maven plugin is depending on a much older version of FitNesse) Also, with the original Test Runner being deprecated, is it possible to have the behavior I am looking for? (Pages defined in remote server, but run locally in build machine)
Or, is such way of work no-longer a recommended approach to use FitNesse? (If so, I will need to change the approach of the automated test)
One solution I've used is the wiki import option feature. This can import the latest changes from the remote wiki to your local build server.
http://fitnesse.org/FitNesse.UserGuide.WikiImport
You can also tell it to auto-update when running the tests rather than having to re-import manually whenever they change.
Another possibility is to use a source control plugin to automatically commit changes by QA/Users from the remote wiki and pull them down as part of your build.
Does anyone have specific experiences with using TFS 2008/2010 AND Jenkins for Continuous Integration (CI)? We are trying to decide which CI server to use. Our team works exclusively in Microsoft .NET/Visual Studio 2010/C#. We have the following requirements:
Automatically build our web project on every checkin.
Run unit tests with each build.
Automatically deploy green builds to development and/or test environments.
Provide pretty reports.
Provide build/deployment notifications via email.
I realize that installing a tool won't necessarily give us this functionality out-of-the-box and that we will have to integrate with other tools like MSBuild to achieve this.
I'm looking for specific features that Jenkins has that TFS 2008/2010 does not or vice versa. Also which is easier to maintain, use, etc.
I would highly recommend using Jenkins - it will do all of your requirements out of the box apart from possibly #3, but if you can script your deployments then it can do that as well.
Here are some links to help you get your builds up and running:
Blog about doing .NET builds in Jenkins
Jenkins Windows installers
Installing the Jenkins master and slaves as Windows services
Disclaimer: I have no experience with TFS, but I think open solutions are nearly always more flexible and extensible (and cheaper !) than proprietary products.
Late to this game, but I have used both TFS 2010 and Jenkins for CI. TFS 2010 has minimum set of CI tools in it. However, when you want to create a CI pipeline, it's a completely different story while Jenkins can easily create the pipeline.
If you are looking at only CI for one build either one should work. However, when it comes to the entire pipeline, Jenkins is way to go. With TFS it can be done, but Jenkins is better choice.
Here's quick bullet points:
TFS:
With a build definition you can compile, execute tests, return changeset/workitems, send an email when a build is broken
natural integration with visual studio
extremely hard to create CI pipeline. Requires custom handler and extensive workflow work. Not as intuitive as creating a build definition.
Because of the 3rd bullet, it's not easy to maintain/customize/scale CI pipeline
Jenkins:
Need to create a msbuild config file for CI, which is not much pain comparing to creating CI pipeline using TFS. However, TFS gives better/easier tool to create a build definition. however, it is not bad creating config file for msbuild for a project.
Creating a CI pipeline is very easy. Just chain them using upstream/downstream jenkins job trigger and passing an artifact from previous job.
Since Jenkins is very flexible, it is easy to create a jenkins plugin to meet your own needs and provide it to opensource community :)
In summary, if you need complete automated build, test, and deployment system go with Jenkins. If you just need only build and test, TFS might give you an edge over Jenkins.
If you're using Team 2010-2012, there is no reason whatsoever to bring in Jenkins. Team has all the features you listed, and the build process is ridiculously flexible.
Note that if you are stuck on Team 2008 or earlier, you should seriously look at Jenkins -- 2008 and earlier are quite primitive and inflexible compared to 2010 or later.