Unapprovable RejectedAccessException when using Tuple in Jenkinsfile - jenkins

I tried to use Tuple in a Jenkinsfile.
The line I wrote is def tupleTest = new Tuple('test', 'test2').
However, Jenkins did not accept this line and keep writing the following error to the console output:
No such constructor found: new groovy.lang.Tuple java.lang.String java.lang.String. Administrators can decide whether to approve or reject this signature.
...
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: No such constructor found: new groovy.lang.Tuple java.lang.Integer java.lang.String
...
When I visited the "Script Approval" configuration I could not see any scripts that pend approval.
Following this link, I tried to install and enable the "Permissive Security" plugin, but it did not help either - The error was the same.
I even tried to manually add the problematic signature to the scriptApproval.xml file. After I added it, I was able to see it in the list of approved signatures, but the error still remained.
Is there something I am doing wrong?

I had the same issue trying to use tuple on jenkins so I found out that I can simply use a list literal instead:
def tuple = ["test1", "test2"]
which is equivalent to
def (a, b) = ["test1", "test2"]
So now, instead of returning a tuple, I am returning a list in my method
def myMethod(...) {
...
return ["test 1", "test 2"]
}
...
def (a, b) = myMethod(...)

This is more or less a problem caused by groovy.lang.Tuple constructor + Jenkins sandbox Groovy mode. If you take a look at the constructor of this class you will see something like this:
package groovy.lang;
import java.util.AbstractList;
import java.util.List;
public class Tuple extends AbstractList {
private final Object[] contents;
private int hashCode;
public Tuple(Object[] contents) {
if (contents == null) throw new NullPointerException();
this.contents = contents;
}
//....
}
Groovy sandbox mode (enabled by default for all Jenkins pipelines) ensures that every invocation passes script approval check. It's not foolproof, and when it sees new Tuple('a','b') it thinks that the user is looking for a constructor that matches exactly two parameters of type String. And because such constructor does not exists, it throws this exception. However, there are two simple workarounds to this problem.
Use groovy.lang.Tuple2 instead
If your tuple is a pair, then use groovy.lang.Tuple2 instead. The good news about this class is that it provides a constructor that supports two generic types, so it will work in your case.
Use exact Object[] constructor
Alternatively, you can use the exact constructor, e.g
def tuple = new Tuple(["test","test2"] as Object[])
Both options require script approval before you can use them (however, in this case both constructors appear in the in-process script approval page).

Related

Jenkins | Use method from an imported java class in a groovy script

I want to be able to use a method from a Jenkins plugin via its java class
Just to point out I'm not a developer or a groovy/java expert - happy to learn!
The java class that my method is part of is com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator
From this I would like to use the method getRepoOwner()
What I've done is set my import and defined a new call to the class:
import com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator
def bbSCMNav = new BitbucketSCMNavigator()
When I run this I get the error below:
org.codehaus.groovy.runtime.metaclass.MethodSelectionException: Could not find which method <init>() to invoke from this list:
public com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator#<init>(java.lang.String)
public com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMNavigator#<init>(java.lang.String, java.lang.String, java.lang.String)
I've searched for the error above Could not find which method <init>() to invoke from this list
And I came across this ticket Could not find which method <init>() to invoke from this list on newInstance in groovy closure
Can't say that I entirerly understand the reply if it's helpful to me or not as I say I'm not a developer and groovy and java are relatively new to me but happy to understand if anyone can point me in the right direction with this
The goal of this exercise is to use the method during the run-time of a build to get the output of getRepoOwner() and use that in a variable to construct a URI
This question also seems similar to mine - Calling internal methods of Jenkins plugin (thinBackup)
But I'm not using maven or a pom.xml here
Cheers
Quick Answer
This error Could not find which method < init >() is related to a missing constructor.
Almost all internal jenkins class are ready to use in groovy.
In your case, BitbucketSCMNavigator does not have a default constructor. It have a constructor with one String argument. Check this line
Explanation
I could replicate your error with another internal class org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory:
node {
stage('internal') {
org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory obj =
new org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory();
}
}
hudson.remoting.ProxyException: org.codehaus.groovy.runtime.metaclass.MethodSelectionException: Could not find which method <init>() to invoke from this list:
private org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory#<init>(org.jenkinsci.plugins.workflow.cps.CpsFlowExecution, boolean, java.lang.ClassLoader, java.util.List)
But, reviewing this class CpsFlowExecution I could see that CpsGroovyShellFactory does not have a default constructor. It have a constructor with one argument : CpsGroovyShellFactory(this)
So, If I instance the constructor with one argument, no errors appear.
node {
stage('internal') {
org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory obj =
new org.jenkinsci.plugins.workflow.cps.CpsGroovyShellFactory(null);
}
}

Using an enum loaded from another Groovy file (Jenkins Pipeline issue)

I have the following Groovy script as part of a Jenkins pipeline
permissions.groovy
enum PermissionType {
ANONYMOUS,
AUTHENTICATED
}
def get_job_permissions(PermissionType permission) {
...
}
return this
I load this file into another Groovy file as part of my Jenkins pipeline and call get_job_permissions passing through one of the enums as a parameter.
pipeline.groovy
def job_permissions = load 'permissions.groovy'
job_permissions.get_job_permissions(job_permissions.PermissionType.AUTHENTICATED)
Jenkins fails on this with the following error (I've verified that in this case 'Script3' is the call to get_job_permissions with the enum parameter).
groovy.lang.MissingPropertyException: No such property: PermissionType for class: Script3
I know the script load and call is correct, as I can change the signature of get_job_permissions to the following, pass through a random string in pipeline.groovy and the call goes through correctly.
def get_job_permissions(def permission) {
...
}
If I don't change the signature, and still pass through a random string, Jenkins fails the build as it can't find the method it thinks I'm calling (which is true, it's not there, it's expecting a PermissionType type).
I've tried a number of different things to expose PermissionType to the calling script
Adding #Field (not legal Groovy)
Changing the enum definition to public def PermissionType (not legal Groovy)
Removing and adding public to the enum definition
Changing the case (though I believe enums need to start with an upper case character?)
None of these solutions allow me to reference the enum type from the calling script, and I understand this is because I'm trying to access a type by referencing it through an instance of the script.
But if I can't do it this way, what is the best way to do it?
Thanks
I managed to get something to work - I certainly know it's probably not the right, or even good, way to do it, but it unblocked me and gave me what I needed.
Rather than define the enum in a script as you normally would
enum PermissionType {
ANONYMOUS,
AUTHENTICATED
}
I created a class containing the enum with member variables initialised to the values within the enum.
permissions.groovy
public class PermissionTypes {
public enum Values {
ANONYMOUS,
AUTHENTICATED
}
public final PermissionTypes.Values ANONYMOUS = PermissionTypes.Values.ANONYMOUS
public final PermissionTypes.Values AUTHENTICATED = PermissionTypes.Values.AUTHENTICATED
}
#Field final PermissionTypes Permissions = new PermissionTypes()
I can then expose an instance of that class in the script, load it as normal and I finally get access to the enum values.
pipeline.groovy
def job_permissions = load 'permissions.groovy'
job_permissions.get_job_permissions(job_permissions.Permissions.AUTHENTICATED)
I think we can all agree this is slightly bonkers, but it gave me what I needed.
Only issues I have with this (which I can live with for now)
You can only load the file ones in a script otherwise you get a duplicate class exception
You can't use the type in an external method, only the values - OK for me since any methods taking in the type are local to the class definition
Would still love to know the right way to do this :)
I ran into this problem recently and found a different solution that looks less hacky.
enum PermissionType {
ANONYMOUS,
AUTHENTICATED
}
def get_job_permissions(PermissionType permission) {
...
}
// Do this before you return out to make the enum available as well
this.PermissionType = PermissionType
return this
I prefer to use:
MyEnumClass.groovy:
package cl.mypackage.utils
class MyEnumClass {
static enum MyEnum {
FOO, BAR, QWERTY
}
}
How to use:
import cl.mypackage.utils.MyEnumClass
def in_some_place() {
def fooEnum = MyEnumClass.MyEnum.FOO
}
Regards

Expected getter for property [tempLocation] to be marked with #default on all

I am trying to execute a Dataflow pipeline that writes to BigQuery. I understand that in order to do so, I need to specify a GCS temp location.
So I defined options:
private interface Options extends PipelineOptions {
#Description("GCS temp location to store temp files.")
#Default.String(GCS_TEMP_LOCATION)
#Validation.Required
String getTempLocation();
void setTempLocation(String value);
#Description("BigQuery table to write to, specified as "
+ "<project_id>:<dataset_id>.<table_id>. The dataset must already exist.")
#Default.String(BIGQUERY_OUTPUT_TABLE)
#Validation.Required
String getOutput();
void setOutput(String value);
}
And try to pass this to the Pipeline.Create() function:
public static void main(String[] args) {
Pipeline p = Pipeline.create(PipelineOptionsFactory.fromArgs(args).withValidation().as(Options.class));
...
}
But I am getting the following error. I don't understand why because I annotate with#Default:
Exception in thread "main" java.lang.IllegalArgumentException: Expected getter for property [tempLocation] to be marked with #Default on all [my.gcp.dataflow.StarterPipeline$Options, org.apache.beam.sdk.options.PipelineOptions], found only on [my.gcp.dataflow.StarterPipeline$Options]
Is the above snippet your code or a copy from the SDK?
You don't define a new options class for this. You actually want to call withCustomGcsTempLocation on BigQueryIO.Write [1].
Also, I think BQ should determine a temp location on it's own if you do not provide one. Have you tried without setting this? Did you get an error?
[1] https://github.com/apache/beam/blob/a17478c2ee11b1d7a8eba58da5ce385d73c6dbbc/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1402
Most users simply set the staging directory. To set the staging directory, you want to do something like:
DataflowPipelineOptions options = PipelineOptionsFactory.create()
.as(DataflowPipelineOptions.class);
options.setRunner(BlockingDataflowPipelineRunner.class);
options.setStagingLocation("gs://SET-YOUR-BUCKET-NAME-HERE");
However if you want to set gcpTemporaryDirectory, you can do that as well:
GcpOptions options = PipelineOptionsFactory.as(GcpOptions.class);
options.setGcpTempLocation()
Basically you have to do .as(X.class) to get to the X options. Then once you have that object you can just set any options that are part of X. You can find many additional examples online.

Dataflow output parameterized type to avro file

I have a pipeline that successfully outputs an Avro file as follows:
#DefaultCoder(AvroCoder.class)
class MyOutput_T_S {
T foo;
S bar;
Boolean baz;
public MyOutput_T_S() {}
}
#DefaultCoder(AvroCoder.class)
class T {
String id;
public T() {}
}
#DefaultCoder(AvroCoder.class)
class S {
String id;
public S() {}
}
...
PCollection<MyOutput_T_S> output = input.apply(myTransform);
output.apply(AvroIO.Write.to("/out").withSchema(MyOutput_T_S.class));
How can I reproduce this exact behavior except with a parameterized output MyOutput<T, S> (where T and S are both Avro code-able using reflection).
The main issue is that Avro reflection doesn't work for parameterized types. So based on these responses:
Setting Custom Coders & Handling Parameterized types
Using Avrocoder for Custom Types with Generics
1) I think I need to write a custom CoderFactory but, I am having difficulty figuring out exactly how this works (I'm having trouble finding examples). Oddly enough, a completely naive coder factory appears to let me run the pipeline and inspect proper output using DataflowAssert:
cr.RegisterCoder(MyOutput.class, new CoderFactory() {
#Override
public Coder<?> create(List<? excents Coder<?>> componentCoders) {
Schema schema = new Schema.Parser().parse("{\"type\":\"record\,"
+ "\"name\":\"MyOutput\","
+ "\"namespace\":\"mypackage"\","
+ "\"fields\":[]}"
return AvroCoder.of(MyOutput.class, schema);
}
#Override
public List<Object> getInstanceComponents(Object value) {
MyOutput<Object, Object> myOutput = (MyOutput<Object, Object>) value;
List components = new ArrayList();
return components;
}
While I can successfully assert against the output now, I expect this will not cut it for writing to a file. I haven't figured out how I'm supposed to use the provided componentCoders to generate the correct schema and if I try to just shove the schema of T or S into fields I get:
java.lang.IllegalArgumentException: Unable to get field id from class null
2) Assuming I figure out how to encode MyOutput. What do I pass to AvroIO.Write.withSchema? If I pass either MyOutput.class or the schema I get type mismatch errors.
I think there are two questions (correct me if I am wrong):
How do I enable the coder registry to provide coders for various parameterizations of MyOutput<T, S>?
How do I values of MyOutput<T, S> to a file using AvroIO.Write.
The first question is to be solved by registering a CoderFactory as in the linked question you found.
Your naive coder is probably allowing you to run the pipeline without issues because serialization is being optimized away. Certainly an Avro schema with no fields will result in those fields being dropped in a serialization+deserialization round trip.
But assuming you fill in the schema with the fields, your approach to CoderFactory#create looks right. I don't know the exact cause of the message java.lang.IllegalArgumentException: Unable to get field id from class null, but the call to AvroCoder.of(MyOutput.class, schema) should work, for an appropriately assembled schema. If there is an issue with this, more details (such as the rest of the stack track) would be helpful.
However, your override of CoderFactory#getInstanceComponents should return a list of values, one per type parameter of MyOutput. Like so:
#Override
public List<Object> getInstanceComponents(Object value) {
MyOutput<Object, Object> myOutput = (MyOutput<Object, Object>) value;
return ImmutableList.of(myOutput.foo, myOutput.bar);
}
The second question can be answered using some of the same support code as the first, but otherwise is independent. AvroIO.Write.withSchema always explicitly uses the provided schema. It does use AvroCoder under the hood, but this is actually an implementation detail. Providing a compatible schema is all that is necessary - such a schema will have to be composed for each value of T and S for which you want to output MyOutput<T, S>.

Strange behavior of gorm finder

In a controller I have this finder
User.findByEmail('test#test.com')
And works.
Works even if I write
User.findByEmail(null)
But if i write
User.findByEmail(session.email)
and session.email is not defined (ergo is null) it throw exception
groovy.lang.MissingMethodException: No signature of method: myapp.User.findByEmail() is applicable for argument types: () values: []
Is this behavior right?
If i evaluate "session.email" it give me null so I think it must work as it do when I write
User.findByEmail(null)
Even more strange....
If I run this code in groovy console:
import myapp.User
User.findByEmail(null)
It return a user that has null email but if I run the same code a second time it return
groovy.lang.MissingMethodException: No signature of method: myapp.User.findByEmail() is applicable for argument types: () values: []
You can't use standard findBySomething dynamic finders to search for null values, you need to use the findBySomethingIsNull version instead. Try
def user = (session.email ? User.findByEmail(session.email)
: User.findByEmailIsNull())
Note that even if User.findByEmail(null) worked correctly every time, it would not necessarily give you the correct results on all databases as a findBySomething(null) would translate to
WHERE something = null
in the underlying SQL query, and according to the SQL spec null is not equal to anything else (not even to null). You have to use something is null in SQL to match null values, which is what findBySomethingIsNull() translates to.
You could write a static utility method in the User class to gather this check into one place
public static User byOptEmail(val) {
if(val == null) {
return User.findByEmailIsNull()
}
User.findByEmail(val)
}
and then use User.byOptEmail(session.email) in your controllers.
Jeff Brown from grails nabble forum has identified my problem. It's a GORM bug. see jira
More info on this thread
This jira too
I tried with debugger and it looks it should be working, as you write. Maybe the groovy itself is a little bit confused here, try to help it this way:
User.findByEmail( session['email'] )

Resources