I have some hudson servers for our CI process.
The build tasks use Ant scripts and also old-school *.bat files.
What do you prefer? What are the pros and cons?
(I think of readability, familiarity to developers and extensibility...)
Are there any other options? We have .Net, Java and PHP apps to test.
Batch is a programming language (and not a very good one at that). Ant is a dependency matrix language. What's the difference?
In a programming language, you specify the order everything occurs in. You're responsible to say what is built and the order.
In a dependency matrix language, you merely state the dependencies, and the program figures out what to do and the order it should be done in. One of the biggest issues developers have with Ant or Make is to try to force a build order instead of letting the build system take care of it.
Builds should always be done with a dependency matrix language like Ant.
Let Ant determine what needs to be built and the order it should be built. Don't use Batch. If you're using Batch scripts to call a bunch of Ant script in the order you think they should be called, you're doing it wrong. Have a master Ant script do it and use <subant> calls. Let Ant do the tricky stuff.
You can use batch script to do preliminary work (such as setting ANT_OPTS if Ant needs more memory, or setting environment variables like ANT_HOME and JAVA_HOME and your %PATH% variable to make sure you're using the correct Java and Ant versions. In Hudson, you can set all of this in the Hudson job itself, so you don't have to call the Batch script.
*.bat files pretty much restrict you to dos/windows, like how shell scripts are for linux, whereas Ant/Maven is cross-platform and gives you the option to use a non-Windows CI server
Related
originally, I use gradle to build my android project, but recently, I migrate it to bazel, and I find that bazel is truly fast than gradle, so I want to know why, but the doc of bazel doesn't give much idea about this, can anyone help me？
Thanks very much!
Full disclosure: I work on Bazel.
That's not an easy question to answer for two reasons. First, performance is highly dependent on the scenario. For example, we'd generally expect a clean build to be slower than a build where only a single file has changed. Second, I don't know how Gradle works internally, and they've done a lot of work recently to improve Gradle performance.
But I can talk about Bazel and what we're doing to make it fast. We've been working on build performance for ~10 years, starting long before we made it public.
The key feature is that we require all dependencies to be declared, and we track them explicitly. If you use a header file in C++, or depend on a Java library, you must declare this dependency in your BUILD file (and we enforce that these are declared by sandboxing individual actions). There are three effects from this:
First, we can heavily parallelize the build, because we know which things depend on which other things.
Second, we can make incremental builds very fast, because we can tell what parts of the build have to be re-done when you change a specific file (BUILD file, header file, source file, ...).
Third, we almost never have to do clean builds. Other build tools often require 'make clean' to get into a predictable state - since Bazel knows all the dependencies, it can get to a predictable state on every single build.
Another effect is that we can cache remotely (i.e., across users), and even execute on another machine, although neither of these are fully supported at the time of this writing.
So, at work, I frequently have to create virtually identical ant scripts. Basically the application we provide to our clients is designed to be easily extensible, and we offer a service of designing and creating custom modules for it. Because of the complexity of our application, with lots of cross dependencies, I tend to develop the module within our core dev environment, compile it using IntelliJ, and then run a basic ant script that does the following tasks:
1) Clean build directory
2) Create build directory and directory hierarchy based on package paths.
3) Copy class files (and source files to a separate sources directory).
4) Jar it up.
The thing is, to do this I need to go through the script line by line and change a bunch of property names, so it works for the new use case. I also save all the scripts in case I need to go back to them.
This isn't the worst thing in the world, but I'm always looking for a better way to do things. Hence my idea:
For each specific implementation I would provide an ant script (or other file) of just properties. Key-value pairs, which would have specific prefixes for each key based on what it's used for. I would then want my ant script to run the various tasks, executing each one for the key-value pairs that are appropriate.
For example, copying the class files. I would have a property with a name like "classFile.filePath". I would want the script to call the task for every property it detects that starts with "classFile...".
Honestly, from my current research so far, I'm not confident that this is possible. But... I'm super stubborn, and always looking for new creative options. So, what options do I have? Or are there none?
It's possible to dynamically generate ANT scripts, for example the following does this using an XML input file:
Use pure Ant to search if list of files exists and take action based on condition
Personally I would always try and avoid this level of complexity. Ant is not a programming language.
Looking at what you're trying to achieve it does appear you could benefit from packaging your dependencies as jars and using a Maven repository manager like Nexus or Artifactory for storage. This would simplify each sub-project build. When building projects that depend on these published libraries you can use a dependency management tool like Apache ivy to download them.
Hope that helps your question is fairly broad.
I want to setup a continous integration system that upon a commit or similar trigger should:
run tests on a fortran/C/C++ code, if needed.
compile that code using cmake.
run tests on a rails app.
compile the rails ap.
restart the server.
I'm looking at Jenkins. Is it the best choice for this kind of work? Also, what's the difference between using a bash script that makes all that (if possible) and using jenkins? I'm asking not because I'm thinking about using a script, but to better understand jenkins.
It sounds like Jenkins would certainly be a reasonable choice for this. Apart from the ability to run arbitrary scripts as build steps, there's also a large number of plugins, which provide better integration with cmake for example.
Even if you're using a single bash script to do all of this, using Jenkins on top of it would still have a number of advantages. You get a web interface, email notifications and build history for free, with all that this entails. By integrating your tests "properly" with Jenkins, you can also get things like graphs that show how many tests succeeded/failed over time.
I am using Jenkins for java projects and have to say it is easy to configure. I used to add lots of plugins for better configuration of build steps, but tend to go back to using scripting languages for build and deploy steps because of two main reasons. If I have a build script, it's easier to configure the same job on a different Jenkins server or run the script manually if need be and the build configuration is not so cluttered (I still have one maven job with more than 50 post build steps). The second reason is, that it is easier to version the scripts in SVN, compared to having the build config in SVN.
So to answer your questions. I don't know if it is the 'best' tool, but it is good enough for me. Regarding scripting: use each tool for what it is build for. Jenkins a glorified cron deamon with great options when it comes to displaying analysis. The learning curve for people to use it is minimal (i.e. starting a job, seeing whether it failed.) Configuring Jenkins needs a little bit more learning, but it's very easy to set up simple jobs and go then to the more complicated tasks.
For the first four activities Jenkins will do the job and is rather the best choice nowadays, but for things like restarting the server (which is actually "remote execution"), better have a look at:
http://saltstack.com/
or:
https://wiki.opscode.com/display/chef/Home
http://cfengine.com/
http://puppetlabs.com/
http://cfengine.com/
Libraries like Fabric(Python) or Capistrano(Ruby) might be useful too.
I haven't been able find an Ant task that works out of the box with ANTLR4. I want to be able to invoke options like -visitor to generate the *Visitor interfaces and classes. I suppose I could 'fake it' using the Java task.
You shouldn't need one. Take a look at the build script I use for the grammars in ANTLRWorks 2.
Downloads the correct version of the tool on-demand (so it doesn't have to be included in source control).
Only rebuilds grammars if they are out-of-date.
Builds grammars in parallel for improved build times.
https://github.com/sharwell/antlrworks2/blob/master/org-antlr-works-editor/build.xml
Android has an ant script file with generic targets for compile, clean, etc.
Are there any open source ant scripts that define standard targets such as these for general java projects?
I don't want to bother installing and using maven, but I also don't want to bother creating and maintaining ant scripts for each project.
I'm guessing you are new to both Maven and Ant. If I'm wrong you can ignore the rest of this post :)
I don't know of any generic open source Ant files. I've never had the need to look for one. I think its well worth the effort to learn Ant and create your own generic script. Ant is easy to learn and it is a very flexible, powerful and reliable build tool. You'll find it quite easy to create a build file that you can reuse over and over again on different projects.
With regards to having to maintain Ant scripts, its a bit of a fallacy. You'll find that once you've got a script that works in a generic way you'll hardly ever have to touch it again except when you need to do things in a custom way (usually around packaging and deployment). When that happens, you'll be glad you're using a tool like Ant that gives you the flexibility to do what you want to be able to do without getting in your way.