Jenkins - Passing array/list to Parameterized Remote Build - jenkins

I am using Jenkins to remotely run an Ansible playbook via the Publish Over SSH command.
This command:
curl -k -v -X POST https://jenkins.myhost.com/job/Ansible_Deploy/build?token=<appToken> --user <myUser>:<userToken> --data-urlencode json='{"parameter":[{"name":"thisIsAList","value":["one","two","three"]}]}'
should trigger a post-build action to remotely execute the following command over SSH:
ansible-playbook /home/<myUser>/test/practice.yml --extra-vars "thisIsAList=$thisIsAList"
thisIsAList is a string parameter under Job Notifications, and the job is parameterized. I have successfully executed similar commands, but this one fails, assumingly because the value is a list. I have tried both "String Parameter" as well as "Multi-line String Parameter" to no avail.
Here's the stack trace:
org.kohsuke.stapler.WrongTypeException: Got type array but no lister class found for type class java.lang.String
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:723)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:478)
at org.kohsuke.stapler.RequestImpl.instantiate(RequestImpl.java:777)
Caused: java.lang.IllegalArgumentException: Failed to convert the value parameter of the constructor public hudson.model.StringParameterValue(java.lang.String,java.lang.String)
at org.kohsuke.stapler.RequestImpl.instantiate(RequestImpl.java:779)
at org.kohsuke.stapler.RequestImpl.access$200(RequestImpl.java:83)
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:678)
Caused: java.lang.IllegalArgumentException: Failed to instantiate class hudson.model.StringParameterValue from {"name":"thisIsAList","value":["one","two","three"]}
at org.kohsuke.stapler.RequestImpl$TypePair.convertJSON(RequestImpl.java:680)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:478)
at org.kohsuke.stapler.RequestImpl.bindJSON(RequestImpl.java:474)
at hudson.model.StringParameterDefinition.createValue(StringParameterDefinition.java:88)
at hudson.model.ParametersDefinitionProperty._doBuild(ParametersDefinitionProperty.java:165)
Note: This may be a duplicate of How to pass an array to a jenkins parameterized job via remote access api? but it hasn't gotten a valid response.

Since this level of nesting isn't detailed anywhere in the Jenkins or Ansible documentation I'll shed some light on the topic now that I've solved my issue.
The command:
ansible-playbook /home/<myUsr>/test/practice.yml --extra-vars "thisIsAList=$thisIsAList"
Should have declared thisIsAList to be a dictionary object. I.e.:
ansible-playbook /home/<myUsr>/test/practice.yml --extra-vars "{thisIsAList=$thisIsAList}"
Furthermore, the data in the cURL command should've been formatted differently like so:
json='{"parameter":[{"name":"thisIsAList","value":"[one,two,three]"}]}'
Note: the double-quotes are around the whole list, rather than the individual elements.
Finally, with further nested items (such as dict inside a list) you have to escape the double-quotes like so:
{"parameter":[{"name":"thisIsADictNestedInAList","value":"[{\"name\":\"numbers\",\"value\":[1s, 2s, 3s]}]"}]}
It seems, that at this level of nesting, it is no longer required to double-quote the lists; probably because the quotes one level up already lead it to be interpreted correctly.

This is a bit of a guess, based on a similar problem I have seen with a choice parameter. Any documentation I have found seems to be wrong about how to handle these. It shouldn't be a list. Try passing as a string with newlines separating the items.
curl -k -v -X POST https://jenkins.myhost.com/job/Ansible_Deploy/build?token=<appToken> --user <myUser>:<userToken> --data-urlencode json='{"parameter":[{"name":"thisIsAList","value":"one\ntwo\nthree"}]}'
Let me know if this works. I'm interested to find out.
Edit: (based on comments)
Would this work:
curl -k -v -X POST https://jenkins.myhost.com/job/Ansible_Deploy/build?token=<appToken> --user <myUser>:<userToken> --data-urlencode json='{"parameter":[{"name":"thisIsAList","value":"'{\"thisIsAList\": [\"one\",\"two\",\"three\"]}'"}]}'
The nested quotes get a bit ugly. If you are using pipeline or can massage the data in a shell script first, it would probably be cleaner.

Related

How do I specify multiple node values in a buildWithParameters call to Jenkins?

I have a webhook service which kicks off a buildWithParameters Jenkins job, and I want to be able to specify which buildservers are being used.
This is easy enough in the job configuration - I've added a Node parameter which lets me specify which nodes are valid, and when starting the job manually in the Jenkins web UI, I can select which nodes I want:
I'm able to kick off the job via curl using the buildWithParameters Jenkins feature:
curl -vvv 'https://webhook:examplepassword#jenkins.example.com/job/build-sideboard-plugin/buildWithParameters?token=exampletoken&GIT_REPO=example/repo&YUM_REPO=example&BUILDSERVER=sideboard.build.dev.xr'
However, I can't figure out how to specify multiple parameters. I expected that I'd simply be able to add a second &BUILDSERVER=xxx value and have that work, but running this:
curl -vvv 'https://webhook:examplepassword#jenkins.example.com/job/build-sideboard-plugin/buildWithParameters?token=exampletoken&GIT_REPO=example/repo&YUM_REPO=example&BUILDSERVER=sideboard.build.dev.xr&BUILDSERVER=sideboard.rocky8.build.dev.xr'
Returns a 500 error. I also tried providing a single value with a comma separating the two values, i.e.
curl -vvv 'https://webhook:examplepassword#jenkins.example.com/job/build-sideboard-plugin/buildWithParameters?token=exampletoken&GIT_REPO=example/repo&YUM_REPO=example&BUILDSERVER=sideboard.build.dev.xr,sideboard.rocky8.build.dev.xr'
but Jenkins interpreted that as a single Node value which didn't match any node since there's no node named sideboard.build.dev.xr,sideboard.rocky8.build.dev.xr. I got the same result when submitting the two values separated by a space.
Is there any way to get Jenkins to do this while still using the buildWithParameter functionality? I'd hate to have to redo the structure of our build triggering or switch to Jenkins Pipeline. Even making 2 different curl commands would be somewhat of a pain given how our webhooks are structured, so I'd love to be able to provide both parameters just like I can in the Jenkins web UI.
I don't think it is possible using the query parameters like you have tried, due to the fact the the plugin actually triggers two different builds.
What you can do is pass the parameters with the submit command as JSON data, which will simulate the trigger of the build with multiple servers selected.
The general syntax will be something like:
curl -u USER:PASSWORD --show-error \
--data 'json={"parameter":[{"name":"PARAMNAME","value":["node1","node2"]}]}' \
http://localhost:8080/job/remote/build?token=TOKEN
or in your case:
curl -u webhook:examplepassword --show-error \
--data 'json={"parameter":[{"name":"BUILDSERVER","value":["sideboard.build.dev.xr","sideboard.rocky8.build.dev.xr"]}]}' \
https://jenkins.example.com/job/build-sideboard-plugin/build?token=exampletoken
You can of course pass all other needed parameters alongside the BUILDSERVERin the JSON data:
curl -u webhook:examplepassword --show-error \
--data 'json={"parameter":[{"name":"BUILDSERVER","value":["sideboard.build.dev.xr","sideboard.rocky8.build.dev.xr"]},{"name":"YUM_REPO","value":"example"},{"name":"GIT_REPO","value":"=example/repo"}]}' \
https://jenkins.example.com/job/build-sideboard-plugin/build?token=exampletoken
In addition it is probably better to use the --data-urlencode instead of the --data flag for the curl commands to avoid encoding issues in case the values of your parameters have special characters.
More info on submitting jobs via Remote Access API is available Here.

Validating Jenkinsfiles within a Jenkinsfile

The docs state to verify a Jenkinsfile you can do as follows
Linting via HTTP POST using curl
# curl (REST API)
# Assuming "anonymous read access" has been enabled on your Jenkins instance.
# JENKINS_URL=[root URL of Jenkins controller]
# JENKINS_CRUMB is needed if your Jenkins controller has CRSF protection enabled as it should
JENKINS_CRUMB=`curl "$JENKINS_URL/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,\":\",//crumb)"`
curl -X POST -H $JENKINS_CRUMB -F "jenkinsfile=<Jenkinsfile" $JENKINS_URL/pipeline-model-converter/validate
At the moment I am just trying to get the crumb. I am trying to do this programmatically in a pipeline, however issuing the curl command throws an error every time. script.sh: Syntax error: "(" unexpected
I have tried escaping the ( character but that is a different error, it works without the xpath extension but it would be cleaner to include this. I have tried to use jenkins Snippet Generator and when I plug in the command it gives response = sh returnStdout: true, script: '''curl -s $JENKINS_URL/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,\\":\\",//crumb) '''. However running this also gives an error.

Ansible Tower CLI pass Launch Parameters in one command without prompt

I am trying to launch an ansible-tower cli job through Jenkins. But I don't want a prompt that appears on Ansible Tower. I want to pass those parameters in the same command so that a prompt is not required.
I have tried:
tower-cli job launch --job-template=33 -e "param1" -e "param2"
This is the error I get:
Error: failed to pass some of the extra variables
According to the Ansible Tower-CLI documentation the parameter -e is wrong. You need to use --extra-vars. This differs from ansible-playbook command. So an easy example is
tower-cli job launch --job-template 1 --extra-vars '{"x":"y"}'
Be aware that you write all vars in one argument. The --extra-vars expects JSON or YAML format.
Be also aware, that the given job template MUST be configured to ask for extra-vars. Otherwise the argument is ignored on Ansible Tower side.
Also - not the question but a good advice - if your Jenkins needs to wait for the job result add --monitor to the tower-cli command. Then the cli waits for the response code and the stage could "fail" if there is a problem.

PBSPro qsub output error file directed to path with jobid in name

I'm using PBSPro and am trying to use qsub command line to submit a job but can't seem to get the output and error files to be named how I want them. Currently using:
qsub -N ${subjobname_short} \
-o ${path}.o{$PBS_JOBID} -e ${path}.e${PBS_JOBID}
... submission_script.sc
Where $path=fulljobname (i.e. more than 15 characters)
I'm aware that $PBS_JOBID won't be set until after the job is submitted...
Any ideas?
Thanks
The solution I came up with was following the qsub command with a qalter command like so:
jobid=$(qsub -N ${subjobname_short} submission_script.sc)
qalter -o ${path}.o{$jobid} -e ${path}.e${jobid} ${jobid}
This way, PBS Pro does not need to resolve the variables, as it failed to do so in our install (this may be a configuration issue)
If you want the ${PBS_JOBID} to be resolved by PBSPro, you need to escape it on the command line:
qsub -o \$PBS_JOBID
Otherwise, bash will attempt to resolve $PBS_JOBID before it gets to the qsub command. I don't know if $subjobname_short and $path are actual environment variables or ones you want pbs to resolve, but if you want pbs to resolve them you'll also need to escape these ones or place it inside the job script.
NOTE: I also notice that your -o argument says {$PBS_JOBID} and I'm pretty sure you want ${PBS_JOBID}. I don't know if that's a typo in the question or what you tried to pass to qsub.

Passing command line arguments through the Remote API

I am using the remote API to create and start containers, but I am not sure how to pass in the command line arguments I normally would when creating from the local machine. Specifically, I am using this image, which requires a bunch of arguments I would normally do when running 'docker run [arguments][image]'. Any ideas?
For argument passing, you can like this
curl -X POST localhost:2375/containers/create -H "Content-Type: application/json" -d '{"Cmd":["ping", "8.8.8.8"], "Image": "ubuntu"}'
Also see: http://blog.flux7.com/blogs/docker/docker-tutorial-series-part-8-docker-remote-api
This depends on which arguments you want to set. Up to port bindings,here you can find how to do that. In general, you have to use the JSON objects passed as body in your create and in your start request.

Resources