What kind of agile tools are you using for Erlang development? What continuous integration (CI) server are you using to build Erlang code? The only reference I got was from Quora question How do I integrate Erlang unit tests in Jenkins (Hudson)?.
I am also interested in the nifty details of setting them up and making talk to each other.
As a company using Erlang actively, Klarna (www.klarna.com) use Jenkins (formerly Hudson) for daily regression test on nearly every dev commit. It's an org with about 80 people total in rnd and we use distribute mode of Jenkins which allows us to have more than 10 build slaves mastered by only one Jenkins server. Basically we have a code base with Eralng code which is version controlled by tools like svn or git. All these testcases are under common test framework and all works well under Jenkins.
Previously, we tried Cruise Control and gave it up since Jenkins does much better.
As Lukas mentioned, you probably will need a tool to gen xml files sine common test doesn't export them directly. Haven't really tried that module though, we do have an implementation of common test event handler to do the job, but it was abandoned due to performance, we do have a a critical requirement on test time. right now, we use a own made script to export xml from common test log directly.
There are a lot more you could do with Erlang and Jenkins, like code coverage analyze if you compile properly and export formatted xml to Cobertour plugin, gui test with selenium etc.
For setting up Jenkins, I think Jenkins home page has a good introduction.
Regarding agile tools, I guess it's really hard to define what a agile tool. Also what I believe is it's very much depend on the size of you org. You will probably need a good process view tool (team level or depart level), a good ticket tracking tool, code review tool, communication tool. There are bunch of them implemented under open source. According to our exp, none of them seems to be able to work seamlessly with Jenkins which means you will need to select and tweak by your own requirement. BUT that's the beauty of open source isn't it :)?
If you want to do it using Jenkins, I have written a common test hook which generates JUnit XML output for your tests which Jenkins can use to produce test statistics.
https://github.com/garazdawi/cth_tools/blob/master/src/cth_junit.erl
We use Jenkins for our Python code, so I think you may use Jenkins with Erlang code.
We use buildbot with our own recipes to hook unit tests.
Related
I am working for a company now for a couple of weeks. The build process is done mostly manually and takes several hours spread over several days. The languages in use are C#, COBOL, Delphi, Visual Basic 6, and of course the database with T-SQL. For the version control, we use Apache Subversion (SVN), except for COBOL code and the documentation, which is kept in Microsoft Visual SourceSafe (VSS). I have the idea to improve the process using a continuous delivery tool. Do you think that Jenkins would do the job?
Thank you for your reply.
Jenkins is undoubtedly a tool that can help with CI/CD.
Whether it is the right tool for your particular needs you should be able to determine by doing your own research into the capabilities of Jenkins and the tooling that it supports. You may find that you struggle with finding adequate support for the older technologies that you mention and you will likely find that you need to uplift some of that legacy to make it usefully available to any viable, modern CI/CD tool.
e.g. get your code out of SourceSafe. You should do that anyway because .. SourceSafe. :)
Don't get bogged down in how to migrate your history. Just shutter SourceSafe (make it read-only) to retain as a reference to your history and move tip/head into a new repo. (SVN if you have to, though I'd highly recommend Git).
More generally, I would be surprised if you could not find some immediate quick-win improvements that can be made, without needing to invest time/effort/money into a "Silver Bullet" tool, just by putting some scripting in place to automate current manual processes.
Jenkins is definitely the right tool. We use Jenkins as a CI tool for building our Delphi (+Dunit+Innosetup), C# and Cordova/PhoneGap applications (all code in SVN).
I have no idea of the dependencies between the code in SVN of VSS, but if it depends on each other, I would advise to put all the code in a SVN or GIT repository.
There are some simple examples to integrate Delphi in Jenkins, see the following links:
https://community.embarcadero.com/blogs/entry/continuous-integration-with-svn-jenkins-and-dunit-delphi-with-craig-chapman
http://www.ictexpertise.com/blog/2016/02/10/continuous-integration-of-delphi-project-with-jenkins/
http://chapmanworld.com/2015/01/18/use-radstudio-with-jenkins-no-plugin/
I'm trying to get reporting working for Karate DSL, and it's proven a challenge because my team uses Circle CI instead of Jenkins. Cucumber reporting seems to only work for Jenkins.
I've had a look at this documentation, here:
https://github.com/intuit/karate/tree/master/karate-demo#example-report
https://github.com/jenkinsci/cucumber-reports-plugin
I was wondering if there is a circle friendly equivalent you could recommend? It'd be even better if the reports could be generated in the terminal. It's going to be a hard sell to convince my team to change CI tools just so I can implement a test framework.
Thanks!
Here's what I suggest:
If you follow the demo / doc instructions - you will get the HTML reports in say target/cucumber-html-reports, and this is "pure Maven and Java", no dependency on CircleCI at all so far.
Now all you need to do is somehow make these HTML reports accessible via the web. In Jenkins, there is an HTML Publisher Plugin. I am not familiar with CircleCI but a quick search suggests that there is a way to expose links to build artifacts.
Also note that when you follow the demo, Java JUnit XML reports would also be output to target/cucumber-reports. It looks like CircleCI has support for these which means that it should be able to derive the build pass/fail status and stats if configured right.
Also note that Karate now enables you to write custom reports: https://stackoverflow.com/a/66773839/143475
We are using Jenkins to run the selenium automation tests and my manager wants to see the list of failed builds and what percentage of the tests passed for the builds. We also have manual tests that get executed in JIRA. I need to combine both and derive the test metrics from them.
The way I think of proceeding is as follows:
Get the Jenkins data in JIRA first using the Jenkins plugin for JIRA.
Use the jira api to collect the testing results from Jenkins and manual tests run on jira.
Prepare a dashboard in JIRA to display all the metrics
Could you suggest if the above approach is correct and suggest something additional.
Thanks in advance!
Are you using cucumber? In that case you could use the cucumber reporting plugin for jenkins. If it doesn't suit your needs but you still use cucumber you can also generate reports in a format like JSON, which you could later parse and get your data.
I have the feeling what you want to do seems a bit complicated, and with not a big benefit. If the tests are failing it's likely you'll have to see what is happening. Having the percentage is sure nice, but I think you can spend some hours/days tailoring this just for having something cute that your manager wants but that has no specific purpose. I would opt for something simpler.
If the automated tests fail, create a jira issue automatically with jenkins. You could put the build number as a tag, or in the title. You can also create it always to indicate that build nr. ## was tested and everything went ok.
As a part of the manual testing process, report in jira what failed.
Create a dashboard and play a bit with tags and search to show which builds failed.
I would suggest AssertThat BDD & Test Management in Jira
Provides end-to-end integration - from features creation to manual and automated tests execution and reporting. Out of the box integration with test automation frameworks through plugins.
The plugin allows to download feature files stored in Jira before the run, execute the test in the usual way and then upload cucumber tests results back to Jira, which gives you a clear view on the testing progress in one place.
More info and usage examples on website https://www.assertthat.com/
I want to setup a continous integration system that upon a commit or similar trigger should:
run tests on a fortran/C/C++ code, if needed.
compile that code using cmake.
run tests on a rails app.
compile the rails ap.
restart the server.
I'm looking at Jenkins. Is it the best choice for this kind of work? Also, what's the difference between using a bash script that makes all that (if possible) and using jenkins? I'm asking not because I'm thinking about using a script, but to better understand jenkins.
It sounds like Jenkins would certainly be a reasonable choice for this. Apart from the ability to run arbitrary scripts as build steps, there's also a large number of plugins, which provide better integration with cmake for example.
Even if you're using a single bash script to do all of this, using Jenkins on top of it would still have a number of advantages. You get a web interface, email notifications and build history for free, with all that this entails. By integrating your tests "properly" with Jenkins, you can also get things like graphs that show how many tests succeeded/failed over time.
I am using Jenkins for java projects and have to say it is easy to configure. I used to add lots of plugins for better configuration of build steps, but tend to go back to using scripting languages for build and deploy steps because of two main reasons. If I have a build script, it's easier to configure the same job on a different Jenkins server or run the script manually if need be and the build configuration is not so cluttered (I still have one maven job with more than 50 post build steps). The second reason is, that it is easier to version the scripts in SVN, compared to having the build config in SVN.
So to answer your questions. I don't know if it is the 'best' tool, but it is good enough for me. Regarding scripting: use each tool for what it is build for. Jenkins a glorified cron deamon with great options when it comes to displaying analysis. The learning curve for people to use it is minimal (i.e. starting a job, seeing whether it failed.) Configuring Jenkins needs a little bit more learning, but it's very easy to set up simple jobs and go then to the more complicated tasks.
For the first four activities Jenkins will do the job and is rather the best choice nowadays, but for things like restarting the server (which is actually "remote execution"), better have a look at:
http://saltstack.com/
or:
https://wiki.opscode.com/display/chef/Home
http://cfengine.com/
http://puppetlabs.com/
http://cfengine.com/
Libraries like Fabric(Python) or Capistrano(Ruby) might be useful too.
I'm picking up support on a project that is currently built with MyEclipse and has a decent sized development team that has been working without any CI processes.
From what I can tell, the MyEclipse folks don't see any value in being able to build outside of the Eclipse platform, which makes no sense at all to me. Continuous Integration is extremely helpful when you have to integrate changes into a codebase from more than one development environment, and it's pretty tough to automate builds when you're tied to a GUI.
Does anyone have continuous integration processes set up around MyEclipse style project-sets? If so, what strategy did you use to accomplish it?
AFAIKT there is no OOTB feature that can generate an Ant script (or equivalent headless-build script) from MyEclipse, nor is there an exposed way to invoke MyEclipse builders from a build-script platform.
This would lead me to believe that I'll need to reverse engineer the scripts based on what MyEclipse generates, which I'd rather not have to do.
I'm not particularly concerned with a Maven-style solution for my needs, but if you know of one I'd like to hear about it. From my initial research it looks like Maven/MyEclipse integration is even worse.
This is remarkably similar to the problems I had working with a websphere 5.1 application that could only be built from WSAD6 running on build machine built from a disk image from the company IT dept. WSAD did have a headless mode. It was a real pain to get that working from Hudson.
I would not be surprised if there was a Maven plugin and/or Ant task for each of the builders you are using. I would start there.
Here is a Maven based solution so maybee a bit off topic for you..
In our company, we use MyEclipse as IDE and Hudson and Team City for continuous integration. The projects are Maven based, so Hudson and TC can work with them.
When you want to open the project in Eclipse, you have to check out the sources, setup maven repository path for eclipse with mvn eclispse:add-maven-repo, build them with mvn install and then run target mvn eclipse:eclipse, which creates the Eclipse project setup from the maven's POM configuration. Then it is possible to import the project into Eclipse and work with it seamlessly..
More information can be found on maven-eclipse-plugin project page
..seamlessly until you change something in the POM configuration - then you have to run the mvn eclipse:eclipse again and have the eclipse project configuration recreated acording to the new POM.. it's important not to forget about this step, unless your project in the IDE won't work properly and you'll be wondering why ;)
Me personally don't find this solution the best, but that's the way how Eclipse folks work with Maven :/
Hope this should inspire you at least :)
This is another reason why I intensely dislike Eclipse. The fact that an IDE can force you away from something that's acknowledged to be a best practice is shameful.
"AFAIKT there is no OOTB feature that can generate an Ant script (or equivalent headless-build script) from MyEclipse" - I'm not sure I understand why this is a problem. It's possible to write a simple Ant build.xml in an hour or two that would do the job for most Java EE apps packaged as WAR files. I don't know if you're using EJBs, but even adding app server specific tasks such as EJB and JSP compilation wouldn't be much of a challenge. If you can agree on a common directory structure it would even be reusable across projects.
With that Ant build.xml in hand, you should be able to drive your CI process simply by checking into Subversion. The Eclipse plug-ins to do that work well, I hear.
If it's really a problem, I'd recommend IntelliJ. It works nicely with CI based on either Cruise Control or Hudson or Jet Brains' own Team City. The cost isn't excessive, and it'll pay for itself quickly.
If I'm misreading your question, I apologize. But if I've got it right, there's no way I'd let the IDE dictate to the team this way.