It is easy to find simple examples for declarative or scripted pipeline. But when the point comes where you go deep into scripting you need so much more information. When you're not familiar to the world of web, java and groovy you are running out of questions which can be asked to go future. Googeling helps you find some magic "hudson.model.Hudson..." or .methods and e.g. #NonCPS-operators solutions. Those solutions work, but I'm searching for the bigger context to work my self from the bottom up. Not from the top down. I'm looking for the knowledge, which is obvious to the insiders.
I'm looking for links/books/api-references or introductions to learn to find the entrance to knowledge around the jenkins scripted pipeline. e.g. like this one =).
I am not looking for answers to those questions below from the stackoverflow communety. This would be to much! I am looking for links of documentation to get deep into the topic. I assume that for an insider it's insider knowledge is not obvious. So I'm stating here some questions to make it obvious what I would describe as insider knowledge.
Example questions:
like : "hudson.model.Hudson..." but where do I get those magical dot.separated strings?
Is there a documentation of the Jenkins Api?
How can I find documentation of the classes and methods usable in jenkins like e.g. X.Y.collect?
Is there a way to debug a pipeline?
Is there a faster way in testing code than every time run it in a pipeline?
How does the inner mechanism work?
Is the Knowledge more about groovy or is it about general Jenkins? Or is it java?
Why println MyArrayList.getClass() class java.util.ArrayList which is a java class? does grooy inherit the types from java, or does the pipeline inherit the types from jenkins, which is java?
...
Asking one question at a time:
where do I get those magical dot.separated strings?
Those are inner java classes at the Jenkins core (or plugins). For the former, Javadoc is available, the latter have their code at Github
classes and methods usable in jenkins
Mostly every Java and Groovy class/method is usable
debug a pipeline?
You can only replay it, issuing changes on each Run
testing
you have two approaches: LesFurets one and the real-unit-one
innards
wide question and wider answer. pipelines are loaded, transformed and run as a near to groovy code (#NonCPS annotation alters this behaviour).
Knowledge about Java, Groovy and Jenkins will apply.
Groovy indeed extends Java hence, both languages apply
Related
So we use Jenkins at work, and I'd like to really learn how it works, particularly so I could customize our builds with Jenkins' flavor of Groovy scripts, understand what all of the build scripts mean, etc. But I just can't seem to find any solid, up-to-date documentation. Here's where I've looked so far:
I've read "Jenkins: The Definitive Guide" -- this is good as far as it goes, but it's outdated (2011) and doesn't seem to cover many of the current Jenkins features
The documentation on jenkins.io -- this is pretty minimal and clearly a work in progress
The wiki at wiki.jenkins-ci.org -- this has more information than jenkins.io, but seems to be mostly focused articles on specific topics, without a comprehensive discussion of how it all works
Stack Overflow -- there are a few questions on Jenkins documentation for specific use cases (e.g. writing a plugin), but as far as I can tell, none of those answers point to general, comprehensive documentation
I feel like I must be just missing something, since so many people use Jenkins -- they must be figuring out how to use it somehow.
Jenkins has a lot of plugins and generally I would google the plugin you intend to use.
I found some documentation about the pipeline plugin on jenkins.io and there is a lot more documentation there. If you want to accomplish something that you can't find there again try to google or ask here :)
I have found the Jenkins Handbook very useful. I would say it's got a lot of good information packed into 150 pages.
Due to the numerous implementation possibilities with Jenkins, it makes it somewhat difficult to fully document every possible use case.
Background:
We have a number of Jenkins top-level jobs that uses (and shares) other jobs as sort-of subroutines. To control the overall flow we use the Jenkins Parameterized Trigger Plugin.
The top-level jobs then gather test reports, build reports etc. from the sub-builds and conveniently publishes them in one go. It's working really well.
Problem at hand: Each of the top-level jobs are started with a number of parameters, and only a selection of these are passed on to the sub-jobs. For some sub-jobs, the parameters are the same as they were some time ago, when the sub-job was last called from this top-level job, but our top-level script isn't aware of this. In essence, we waste build time, building the sub-job again with the same parameters.
In a perfect world the Parameterized Trigger Plugin would have an option like
Do not rebuild job if identical parameters (and configuration unchanged).
which would perform the following steps:
Compare the build-parameters of all kept builds of the given job, to the current parameters.
If the job-configuration is unchanged since the found job was build, setup environmental variable to point to the old job that was found above.
If job not found or the job-configuration has been changed since the found job was build, perform the build as usual.
Unfortunately it does not seem to exist, nor can I find an alternative plugin that provides the functionality I seek.
Groovy to the rescue?
This is where I guess the Scriptler Plugin and a Groovy Script would come in handy, as it would allow me to perform the check in the sub-job, and then set an environment variable that I can use in the Conditional BuildStep Plugin to either perform the build as usual, or skip the build and setup the build environmental variables using the EnvInject Plugin.
My programming question: I'm new to Groovy, and to JAVA for that matter. I have lots of other (assembly, C and scripts) programming experience, though. I've searched for example scripts all over, but haven't found anything remotely similar to what I want to do here. Any hints on how to perform this, including alternative takes on the functionality, would be highly appreciated!
You're going in the right direction already. As there's no ready-made plugin available, the best way to implement a custom solution is to use Groovy.
Conceptually, it will be better to implement the "build-or-dont-build" decision on the triggering side (i.e., in the top-level job). This is because once a sub-job has been triggered, it will be difficult (and awkward) to prevent its actual execution or to re-use previous results in case of identical parameters. (The latter basically means implementing memoization for your sub-jobs; it's an interesting feature as such -- AFAIK there's no plugin for that, but it could be implemented with some scripting in the sub-jobs).
Regarding your programming question: personally, I also started from a more embedded/C-ish background. From my experience, if you're planning to work closer and longer with Jenkins, then learning Groovy will definitely pay off. I was reluctant initially to learn "just another scripting language", but Groovy has some very interesting concepts, and you will be able to control Jenkins in a much more flexible, powerful and efficient way than by just using plugins or the external REST/CLI APIs. You will also be less dependent on finding and running too many plugins, which is a plus from administration pov.
At my workplace, I've been tasked to look into some metrics that the Jenkins tool provides and somehow pull them programatically and display them in some presentable format. The metrics that I need to pull are:
How many unit tests are passing? Failing? Skipping? The total % of passing?
How many integration tests are passing? Failing? Skipping? The total % of passing?
How many acceptance tests are passing? Failing? Skipping? The total % of passing?
How long does it take to execute the test? Make the build?
What is the number of tests executing in pipelines?
... the list goes on
Now I have a very small 1000 ft understanding of Jenkins, and an even smaller understanding of the steps that I need to take to make this program come to life. I am an intern with not much programming experience either, but after some research, I learned that I can navigate through the Jenkins API by adding '.../api' to the link that I want to find API elements for, and I know that I'm going to need to develop a plugin. Aside from that I don't have much direction at all. I don't know what environment I need to develop these plugins (Maven? Never heard of it)... I don't know what languages are supported (I only know C++, Java, and JS)... I don't know how to even install a plugin or get to the plugin on the Jenkins site. I feel like I'm drinking from a firehose with this task and need some guidance.
Does anyone have good guides, advice, tips, tricks, videos... anything that might help me get started on Jenkins plugin development? Any insight into how I might solve this problem too would be much appreciated.
Thanks so much.
First of all there are tools out there which generates HTML reports. You can start there.
For example: MSTest report (.trx) can be converted to HTML by TRXER
and can be published using the HTML Publisher Plugin
However if you're into building your own plugin use NetBeans (I have tried it; and it works)
But creating Jenkins graphs you have to google and see.
We are in the process of developing a review tool for Gremlin/Cypher as we predominantly work with Neo4j graph databases in our project to reduce the manual review effort and also deliver quality code.
Are there any list of coding standards(formatting/performance tips etc.,) for Gremlin and Cypher scripts which can be used as a checklist for performing review of these scripts?
I don't think you're going to find one specific answer, as discussing coding standards can lead to very subjective (and debate-laden) answers. That said: I'll go with something more objective:
First step would be to decide on Gremlin vs Cypher since they're not the same thing nor the same style. When making that decision (and maybe that decision is use both), you should really take a close look at Neo4j 2.0 development (currently at Milestone 4), as Cypher is maturing rapidly and there's a lot of work being put into it, both from expressiveness point of view and performance point of view.
Assuming you go with Cypher, I'd suggest you look at the samples being published by Neo Technology, especially the Cypher learning module. I don't know of any published guidelines, but I'd think most of the guidelines would be similar to any scripting guidelines you already have (such as naming conventions, spacing, etc.). Going further, you're likely going to use Cypher via code as well as through the console. So you'll want to continue using your traditional programming style guidelines, as well as specifying the language-specific library you'll be using.
I can only give you an answer related to Gremlin. First, it is important to note that almost all examples, in the Gremlin wiki, GremlinDocs, the gremlin-users mailing list, etc. are meant for the REPL. Example traversals from these sources tend to be lengthy one-liners, that are written that way for easy transfer via copy/paste to the command line for execution. It is somewhat satisfying to get your answer in one-line in the REPL, but for production code that requires maintenance over time, consider avoiding the temptation to do so, unless there is some specific reason that dictates it.
Second, from a style/formatting perspective, Gremlin is a DSL built on Groovy. Whatever style you like for Groovy should generally work for Gremlin. Start with the Groovy recommendations and tweak them to your needs/liking. I would expect that a tool like CodeNarc would help with general style checks and identifying common Groovy coding problems.
Also see the message from the mailing list https://groups.google.com/forum/#!searchin/neo4j/coding$20standards/neo4j/JYz2APHV-_k/V1BKRwyv5hAJ
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I wanted to create a web application. I thought of selecting Groovy or Grails. But I don't know from where to start! To start with Groovy or Grails?
If you know Java, go directly to grails. Groovy can be used with pure Java syntax, and you'll be learning its special syntax extras on the way.
I've just been in the same situation.
With my last Java experiences dating back some 7 years, I tried to start with Grails directly using "The Definitive Guide to Grails".
I went about 1/3rd thru and thought I understood enough to start out. But as soon as I started to hack I noticed I simply ran into too many errors that would take me hours to resolve, simply because of missing Groovy knowledge.
So I went and got myself "Groovy in Action" (a.k.a GinA). I studied the whole book and programmed each and every listing myself. As I didn't focus on this task solely, it took me several weeks to complete the 600+ pages.
Then I went back to my Grails project and found that I had even forgotten the things I already had known from the first book.
In hindsight, I would go about the task like this:
Directly start your Grails project (I recommend you setup NetBeans SpringSource Tool Suite IntelliJ IDEA Ultimate as your IDE)
Work on it every day for about one
hour
Learn ~50 pages in your
preferred Groovy resource.
Learn ~20 pages in your preferred
Grails resource.
Apply what you
learned
Keep in mind that the current edition of Groovy in Action is dated 2007 and The Definitive Guide to Grails is dated 2009.
In both cases this is quite old, though almost all of the GinA examples still work.
The most important parts of the official Grails Framework Reference are written well structured in chapters, thus it will suite perfectly as your Grails resource.
The following combination would be my personal choice:
Groovy: Groovy in Action, Second Edition as MEAP (Manning Early
Access Program) ($34.99 or $49.99)
Grails: The Grails Framework - Reference Documentation (free)
I'd love to hear about your success & subscribed to your questions as I might be able to help with some of your simpler ones.
Enjoy!
It may be that you're misunderstanding what exactly Groovy and Grails are.
Groovy is a dynamic language that uses Java's JVM (virtual machine). It can compile to Java bytecode and run anywhere that a typical Java program can run. It's only a language, not a framework for developing web applications. You could use Groovy by itself to build a web application, leveraging Servlets, JSPs, and other libraries to help you. However...
Grails is a framework that uses the Groovy language to help you build web applications. Grails gives you easy ways to create all of the web application components that you'd normally have to build yourself (using Servlets, JSPs, etc.).
This may help explain to you why Bozho suggested to go straight to Grails (and why I agree with him).
If you have the opportunity to deploy on google appengine you should take a look at Gaelyk (http://gaelyk.appspot.com/), which is a Groovy framework that gets you up and running realy fast.
Based on your situation, I would start with grails. Here is the Quick Start tutorial: http://grails.org/Quick+Start
At some point you may want something lighter. When that happens check out Ratpack at https://github.com/bleedingwolf/Ratpack#readme
As a learner like you, here is what I've got in few last weeks.
Even if you're familiar with Java, Grails auto-generates templates of code, and knowing Groovy syntax helps to understand it better.
I very suggest books by Scott Davis, they are just brilliant -- short, clear, and straightforward.
Start from "Groovy Recipes", and then take a look at "Getting Started with Grails".
Gaelyk is good for some very custom webapps, but if you're doing ordinary dynamic website, then too much code has to be written manually, and from scratch. Grails provides much more help here.
If you do not know Groovy or Java, sure, you could go straight to Grails, although much of it will be lost on you. My path has been Grails, Groovy, Grails, and now Groovy full-time.
Groovy provides so much pure goodness (builders, command chaining, closure driven Annotations, Groovy SQL, POGOs, etc.), it would be a shame not to explore it in depth ;--)
I'll get back to Grails no doubt, but I am having too much fun with Groovy right now...