I'm fairly new to Groovy and Jenkins, so hopefully this question is coherent.
I have a Jenkinsfile written in Groovy and would like to validate one of the params as a valid URI. Without writing my own regex check, is there a library I could easily invoke during Jenkins startup?
You can try this:
try {
def foo = new java.net.URL("yourURI").openStream()
if (foo.getClass() == sun.net.www.protocol.http.HttpURLConnection$HttpInputStream) {
println 'valid'
foo.close()
}
}
catch (java.io.FileNotFoundException e) {
println 'not valid'
return
}
Unfortunately URL.toUri is not allowed at least in our setup. (It could possibly be allowed with a separate config.) Apparently opening the url (trying to connect to the host) could be possible, but that feels like it could cause other problems.
I ended up with this:
// Validation URLs in Jenkins script is hard (URL.toUri is banned). This regex roughly matches the subset of
// URLs we might want to use (and some invalid but harmless URLs). You can get a rough sense what
// this matches with a generation tool like https://www.browserling.com/tools/text-from-regex .
def saneUrlPattern = ~/^https:\/\/[-\w]{1,32}(\.[-\w]{1,32}){0,4}(:[0-9]{1,5})?(\/|(\/[-\w]{1,32}){1,10})?(\?([-\w]{1,32}=[-\w]{0,40}(&[-\w]{1,32}=[-\w]{0,40}){1,8})?)?(#[-\w]{0,40})?$/
if (!(params.sourceUrl =~ saneUrlPattern)) {
return [error: "Invalid url ${params.sourceUrl}. A simple https URL is expected."]
}
I realise that trying to validate URLs with a regular expression is difficult. I tried to strike a balance between strict and correct enough validation and a regular expression that has some hope of being understood by looking at it and being reasonably convinced as to what it actually matches.
Related
this is very tough to me to understand how Jenkins works. In general when you read documentation and define pipeline, things go smooth. I understand pipelines, stages, steps, scripts. What I don't understand is declaration vs runtime. Especially when it comes to WHEN declaration and evaluating expression. For example:
What is the form of expression? Should it return something like: return true; or maybe it should be statement like: true
When it gets executed? If I access params.MY_PARAMETER_FROM_INPUT, do WHEN has access to its value picked by user?
Is it possible to switch execution between runtime vs pipeline declaration time?
Can I ask for stage (input with message box) only if given condition within WHEN is meet and if not, then don't ask for it but run stage anyway?
When you use IF from script and when WHEN from stage. Can WHEN be defined else where? Within steps, scripts, pipeline?
For example in a stage I've put when { expression { params.ENV == 'prod' } } input { message "Really?" ok "Yeah!" } but the expression was ignored and the question was always asked (current understanding is that it should skip stage/abort whole pipeline when ENV input param is different than "prod" value)
Any thoughts?
OK. Using existing semantic of Jenkins I have manage to achieve what I wanted with following snippet:
stage('Confirm if production related') {
when {
beforeInput true
expression { params.ENV == 'production'; }
}
input {
message "Should I deploy to PRODUCTION?"
ok "Yes, do it!"
}
steps {
script { _ }
}
}
Not bad but not good either.
I'm working on a jenkinsfile and I'm getting and exception in the third stage:
an exception which occurred:
in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv#7bbae4fb
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.CaseEnv#6896a2e3
in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
in object com.cloudbees.groovy.cps.impl.BlockScopeEnv#605ccbbc
in field com.cloudbees.groovy.cps.impl.CallEnv.caller
in object com.cloudbees.groovy.cps.impl.FunctionCallEnv#7b8ef914
in field com.cloudbees.groovy.cps.Continuable.e
in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable#11e73f3c
in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
in object org.jenkinsci.plugins.workflow.cps.CpsThread#b2df9bb
in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup#2b30596a
in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup#2b30596a
Caused: java.io.NotSerializableException: java.util.regex.Matcher
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
I've been reading about it and I know I can't create non-serializable variables. So, I think it has to be with this part of my code:
def artifact_name = sh (
script: "ls -b *.jar | head -1",
returnStdout: true
).trim()
def has_snapshot = artifact_name =~ /-TEST\.jar/
if (has_snapshot) {
//Do something
}
My question is, how do I define that two variables in order to avoid that exception?
Your problem is this line:
def has_snapshot = artifact_name =~ /-TEST\.jar/
The =~ is the Groovy find operator. It returns a java.util.regex.Matcher instance, which is not Serializable. If Jenkins decides to pause your script after you have stored the result in a local variable that is serialized by Jenkins that is when you get the exception. This can be easily tested by immediately adding a sleep(1) step after your invocation and watch as that same exception is thrown.
To resolve this, you should :
Not store the java.util.regex.Matcher result in CPS transformed code
Move the usage into a #NonCPS annotated method or use the match operator (==~) which returns a boolean (if it fits your use case)
Building on the accepted answer I came up with this solution:
def hello = {
def matcher = ("Hello" =~ /Hello/)
matcher.find()
return matcher.group()
}.call()
I guess the stability of this is not so good, but I assume the likeliness of this failing to be very low. So if the impact of this code failing is also low it might be reasonable risk management to use this code.
The following seems to fit the case of running in a NonCPS context, but I am not 100% sure. It definitely is working though
#NonCPS def hello = {
def matcher = ("Hello" =~ /Hello/)
matcher.find()
return matcher.group()
}
hello = hello.call()
println hello
The accepted answer is certainly correct. In my case, I was trying to parse some JSON from an API response like so:
#NonCPS
def parseJson(rawJson) {
return new groovy.json.JsonSlurper().parseText(rawJson)
}
All this does is return a JsonSlurper that can then be used to walk down your JSON structure, like so:
def jsonOutput = parseJson(createIssueResponse)
echo "Jira Ticket Created. Key: ${jsonOutput.key}"
This snippet actually worked fine in my script, but later on in the script, it was using the jsonOutput.key to make a new web request. As stated in the other answer, if the script pauses when you have something stored into a local variable that cannot be serialized, you will get this exception.
When the script attempted to make the web request, it would pause (presumably because it was waiting for the request to respond), and the exception would get thrown.
In my scenario, I was able to fix this by doing this instead:
def ticketKey = parseJson(createIssueResponse).key.toString()
echo "Jira Ticket Created. Key: ${ticketKey}"
And later on when the script attempts to send the web request, it no longer throws the exception. Now that no JsonSlurper object is present in my running script when it is paused, it works fine. I previously assumed that because the method was annotated with #NonCPS that its returned object was safe to use, but that is not true.
I am trying to get build duration for our report but it always returns 0.
From reading docs, going through Slack plugin source and reading other resources I should be able to do one of the following:
def duration = currentBuild.duration
def duration = currentBuild.durationString
def duration = currentBuild.durationString()
def duration = currentBuild.getDurationString()
none of which works. From my understanding this might be because I am calling this before the build actually finished and so th duration is not available yet.
Structure of the pipeline looks something like this:
node {
try {
stage("Stage 1"){}
stage("Stage 2"){}
} catch (e) {
currentBuild.result = "FAILED"
throw e
} finally {
notifyBuild(currentBuild.result)
}
}
def notifyBuild(String buildStatus = 'STARTED') {
def duration = currentBuild.duration;
}
My question is:
Why don't I get the duration
Is there a way in pipeline to specify "after build" step? From what I read try-catching should work that way
My temporary solution is to use:
int jobDuration = (System.currentTimeMillis() - currentBuild.startTimeInMillis)/1000;
Which works fine, but always gives time in seconds and I think the currentBuild.duration should be smart enough to give different units (?)
Update 2018-02-19, this was fixed with 2.14 release of Pipeline Support API Plugin, see this issue
In unable to find any documentation about when duration is expected to be valid. But judging from the implementation it seems like it's set directly after the run/build completes. I'm guessing that it is available on the currentBuild object since it's the same object that is used for representing currentBuild.previousBuild, which may have completed.
So to answer your questions:
The duration field is only valid after the build has completed.
No, there is no way of specifying an "after build" step.
With that said, I think your workaround is a good solution (may be wrap it in a function and put it in an GPL (Global Public Library).
As for your final bonus question I think the currentBuild.duration should be smart enough to give different units (?). If you are referring to the nicely formatted string, like Took 10min 5sec, currentBuild.duration won't give you any nice formatting since it simply returns a long value with the number of seconds that has elapsed. Instead, what you can do is call hudson.Util#getTimeSpanString(long duration). Like this:
import hudson.Util;
...
echo "Took ${Util.getTimeSpanString(System.currentTimeMillis() - currentBuild.startTimeInMillis)}"
This will return a nicely formatted string with the current build duration.
I am new to Grails and Groovy however my problem is a simple but strange one.
I am making calls to a remote web service as follows:
public Boolean addInvites(eventid,sessionkey ){
String url = this.API_URL+"AddInvites?apikey=${sessionkey}&eventid=${eventid}&userids[]=5&userids[]=23";
def callurl = new URL(url);
println callurl;
def jsonResponse = callurl.getText();
println jsonResponse;
def jsonParsedObject = JSON.parse(jsonResponse);
if(jsonParsedObject){
println jsonParsedObject;
if(jsonParsedObject.code == 200){
return true;
}
}
}
return false;
}
The API_URL here is a "https://api..com/"
Normally making these calls works fine. Json gets returned and parsed. However with the above method, if I add only one userids[]=5 then it works fine but if i add a second one everything hangs after the "println callurl;"
I've checked on the webservice side and the call happens and everything works as expected. If I call it in the browser it works fine. but from the grails web app it simply hangs. I know I'm probably doing something silly here, but I am really stuck. Hope you guys can help.
Are you sure that the [] characters are supposed to be there? If you use the following at the end of your query string, it will effectively pass userids=[5,23] to the server:
&userids=5&userids=23
If the brackets really are necessary, use the URL-escaped values %5B and %5D for them instead.
first of all, you should consider the following bug entry:
http://jira.codehaus.org/browse/GROOVY-3921
URL.getText() does not specify a connection timeout.
if you have access to the server logs check if requests from your grails app are actually received (when adding a second userids[] parameter). if this is not the case, you will probably have to use tcpdump or Wireshark in order to debug on TCP level.
Anyone know how I can enter a multiline value in an Ant script? I'm prompting the user for a Subversion commit comment using the input task, and I'd like to be able to support multiple lines of text.
I'm running the standalone version of Ant at the Windows command prompt.
I thought I might be able to do a search and replace for \n, but I can't see any easy way to do a replace from property value to property value in Ant. It looks like I'd have to write a file, replace in the file, and then load the file into another property. I don't want it that badly.
I'm not 100% positive about this, but I took a look at the Ant source code, and it just does a readLine():
From /org/apache/tools/ant/input/DefaultInputHandler.java:
/**
* Prompts and requests input. May loop until a valid input has
* been entered.
* #param request the request to handle
* #throws BuildException if not possible to read from console
*/
public void handleInput(InputRequest request) throws BuildException {
String prompt = getPrompt(request);
BufferedReader r = null;
try {
r = new BufferedReader(new InputStreamReader(getInputStream()));
do {
System.err.println(prompt);
System.err.flush();
try {
String input = r.readLine();
request.setInput(input);
} catch (IOException e) {
throw new BuildException("Failed to read input from"
+ " Console.", e);
}
} while (!request.isInputValid());
} finally {
if (r != null) {
try {
r.close();
} catch (IOException e) {
throw new BuildException("Failed to close input.", e);
}
}
}
}
Here is what I would do if I were you:
If you are using Ant 1.7, then try implementing your own InputHandler, as described in the documentation. The Apache License permits you to basically copy-and-paste the above code as a starting point.
If you are using Ant 1.6 or earlier, then just create your own MultiLineInput task. You can extend the existing Input class and just read multiple lines.
In either case, you would need to decide how the user indicates "I'm done." You could use a blank line or a period or something.
Good luck!
P.S. When I did a Google search for "ant multi-line input", this page was the first hit :-). Pretty impressive for a question that was asked less than an hour ago.