I'm using GroovyConsole to evaluate scripts I get from external sources. So the code to evaluate is dynamic and I don't have control over it. Actually is written into a database and I have to read it as a String. Not perfect, but that's how it is.
What I'm doing right now:
private GroovyShell shell
def processScript( def script){
if (script) {
try{
shell.evaluate (script, 'some_random_name')
}catch( e ){
log.warn "Could not process script: $e"
}
}
}
This usually works. But now we got a large script (~3000 LOC) and it throws java.lang.RuntimeException: Method code too large! because the script is larger than 64K.
I tried to dump the script into a file and use a BufferedReader, but it throws the same Exception.
So is there a better way to evaluate dynamic Groovy code from within a Groovy method?
The problem is your script reach the java limit for a method. I think the only solution is to split your script in many scripts in some way.
See this answer
Related
in Jenkins file one of the variable is having the comma separated values like below.
infra_services=[abc,def,xyz]
when I write the below code it was throwing an error.
if ("{$Infra_Services}".contains("xyz"))
then
echo "$Infra_Services"
fi
yes you can do if statements in a Jenkinsfile. However if you are using declarative pipeline you need to brace it with the step script.
Your issue comes from the fact you did not put any double quotes around "abc" and all the elements of your array
infra_services=[abc,def,xyz]
A second error will raise after you fix this. If infra_services is an array, to manipulate it you should not try to cast it as string. It should throw when you do "{$Infra_Services}"
here is a working example
def Infra_Services = ["abc","def","xyz"]
if (Infra_Services.contains("xyz")) {
println "found"
}
My advice is to test your groovy before running it on jenkins, you will gain precious time. Here is a good online groovy console I use to test my code. running the groovy console from terminal is an alternative
https://groovyconsole.appspot.com/
Currently I'm refactoring our Jenkins build pipeline. In the stage of gathering our unittests I'm trying to enumerate all '**/.test.dll' files, or '.test.dll' at least. Read somewhere that this could be achieved using eachFileRecurse from the File-object.
But... all calls failed reporting FileNotFoundException.
Using the Scriptconsole on the specific slave I tried the same code and it works as expected. Adding some addition debug lines in our jenkins-file shows that the pipeline always returns false.
def TestFile(path)
{
def file = new File(path)
echo "File '${file}' exists: ${file.exists()}"
}
TestFile(WORKSPACE)
TestFile(pwd())
TestFile(BUILDPATH)
All result a 'exists: false', even though all these paths are already used during the build.
(How) can I use the File-object in a pipeline or how can I get the files I need?
Use fileExists together otherwise it will not work.
For example in your case it will be like this
echo "fileExists '${file}' exists: '${file}'"
If I have a Pipeline script method in Pipeline script (Jenkinsfile), my Global Pipeline Library's vars/ or in a src/ class, how can obtain the OutputStream for the console log? I want to write directly to the console log.
I know I can echo or println, but for this purpose I need to write without the extra output that yields. I also need to be able to pass the OutputStream to something else.
I know I can call TaskListener.getLogger() if I can get the TaskListener (really hudson.util.StreamTaskListener) instance, but how?
I tried:
I've looked into manager.listener.logger (from the groovy postbuild plugin) and in the early-build context I'm calling from it doesn't yield an OutputStream that writes to the job's Console Log.
echo "listener is a ${manager.listener} - ${manager.listener.getClass().getName()} from ${manager} and has a ${manager.listener.logger} of class ${manager.listener.logger.getClass().getName()}"
prints
listener is a hudson.util.LogTaskListener#420c55c4 - hudson.util.LogTaskListener from org.jvnet.hudson.plugins.groovypostbuild.GroovyPostbuildRecorder$BadgeManager#58ac0c55 and has a java.io.PrintStream#715b9f99 of class java.io.PrintStream
I know you can get it from a StepContext via context.get(TaskListener.class) but I'm not in a Step, I'm in a CpsScript (i.e. WorkflowScript i.e. Jenkinsfile).
Finding it from a CpsFlowExecution obtained from the DSL instance registered as the steps script-property, but I couldn't work out how to discover the TaskListener that's passed to it when it's created
How is it this hard? What am I missing? There's so much indirect magic I find it incredibly hard to navigate the system.
BTW, I'm aware direct access is blocked by Script Security, but I can create #Whitelisted methods, and anything in a global library's vars/ is always whitelisted anyway.
You can access the build object from the Jenkins root object:
def listener = Jenkins.get()
.getItemByFullName(env.JOB_NAME)
.getBuildByNumber(Integer.parseInt(env.BUILD_NUMBER))
.getListener()
def logger = listener.getLogger() as PrintStream
logger.println("Listener: ${listener} Logger: ${logger}")
Result:
Listener: CloseableTaskListener[org.jenkinsci.plugins.workflow.log.BufferedBuildListener#6e9e6a16 / org.jenkinsci.plugins.workflow.log.BufferedBuildListener#6e9e6a16] Logger: java.io.PrintStream#423efc01
After banging my head against this problem for a couple days I think I have a solution:
CpsThreadGroup.current().execution.owner.listener
It's ugly, and I don't know if it's correct or if there's a better way, but seems to work.
I think I followed very step on the document, but I still ran into this exception. (the only different is that I run this from Eclipse J2EE, but I won't expect this really maters, doesn't it?)
Code: (I didn't write this, it's right from the beam project example). I think you'd have to specify a google cloud platform project and provide the right credential to access it. However, I didn't find anywhere in this example project that does the setting up.
public static void main(String[] args) {
// Create a PipelineOptions object. This object lets us set various execution
// options for our pipeline, such as the runner you wish to use. This example
// will run with the DirectRunner by default, based on the class path configured
// in its dependencies.
PipelineOptions options = PipelineOptionsFactory.create();
// Create the Pipeline object with the options we defined above.
Pipeline p = Pipeline.create(options);
// Apply the pipeline's transforms.
// Concept #1: Apply a root transform to the pipeline; in this case, TextIO.Read to read a set
// of input text files. TextIO.Read returns a PCollection where each element is one line from
// the input text (a set of Shakespeare's texts).
// This example reads a public data set consisting of the complete works of Shakespeare.
p.apply(TextIO.Read.from("gs://apache-beam-samples/shakespeare/*"))
.....
)
Exception:
Exception in thread "main" java.lang.IllegalStateException: Failed to validate gs://apache-beam-samples/shakespeare/*
at org.apache.beam.sdk.io.TextIO$Read$Bound.expand(TextIO.java:309)
at org.apache.beam.sdk.io.TextIO$Read$Bound.expand(TextIO.java:205)
at org.apache.beam.sdk.runners.PipelineRunner.apply(PipelineRunner.java:76)
at org.apache.beam.runners.direct.DirectRunner.apply(DirectRunner.java:296)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:388)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:302)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:47)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:152)
at google.dataflow.beam.example.MinimalWordCount.main(MinimalWordCount.java:77)
Caused by: java.io.IOException: Unable to match files in bucket apache-beam-samples, prefix shakespeare/ against pattern shakespeare/[^/]*
at org.apache.beam.sdk.util.GcsUtil.expand(GcsUtil.java:234)
at org.apache.beam.sdk.util.GcsIOChannelFactory.match(GcsIOChannelFactory.java:53)
at org.apache.beam.sdk.io.TextIO$Read$Bound.expand(TextIO.java:304)
... 8 more
Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request
{
"error" : "invalid_grant"
}
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1070)
at com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:207)
at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149)
at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135)
at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96)
at com.google.cloud.hadoop.util.ChainingHttpRequestInitializer.initialize(ChainingHttpRequestInitializer.java:52)
at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:93)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.buildHttpRequest(AbstractGoogleClientRequest.java:300)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.ResilientOperation$AbstractGoogleClientRequestExecutor.call(ResilientOperation.java:166)
at com.google.cloud.hadoop.util.ResilientOperation.retry(ResilientOperation.java:66)
at com.google.cloud.hadoop.util.ResilientOperation.retry(ResilientOperation.java:103)
at org.apache.beam.sdk.util.GcsUtil.expand(GcsUtil.java:227)
... 10 more
Try to run it From command Prompt if using Windows.
Go to the folder containing pom.xml file and open cmd there.
then give command with the respective arguments.
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args=" --output=counts" -Pdirect-runner
If you want to run with your input file. Then make a txt file with any name and put it in the folder containing pom. And then Fire following Command.
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--inputFile=YOURFILENAME.txt --output=counts" -Pdirect-runner**
Hope this will do. Rest i am looking into your issue
If I use cucumber.api.groovy.Hooks.World then tests are working fine. But I cannot open declaration in steps definition
World() {
def world = new LucieWorld()
world.metaClass.mixin Lucie
world
}
step definition:
Given(~/^User is (.+)$/) { def username ->
login(username) //I cannot open declaration here
}
When I am in a step definition file (groovy), Intellij can't seem to see variables and methods defined in the cucumber "World" object Lucie. So don't get IDE support (auto-complete, etc) for those which is a bit annoying. How can I fix that?
I try to use this.metaClass.mixin(Lucie) in step definition, but I think, that this is not a good solution.