Jenkins Groovy pipeline, variable adds a newline and breaks curl statement - jenkins

I have a jenkins job, and part of the code looks like this:
def version = sh returnStdout: true, script: """ #!/bin/sh
find src/*/*/ -name *.ear | grep -Eo \'[0-9]+\\.[0-9]+\\.[0-9]+\'
"""
sh "echo $version"
def imageVersion = sh returnStdout: true, script: """
#!/bin/sh
curl -k -H 'X-JFrog-Art-Api:${api_key}' https://artifactory.xxx.com/api/search/latestVersion\\?g=com.xxx.bwce&a=billing-app-v1&v=$version-SNAPSHOT&repos=libs-snapshot-local
"""
And I noticed that during the execution, this curl statement is broken at this variable replacement.
here is how the output looks from Jenkins console log.
+ echo 1.0.6
1.0.6
[Pipeline] sh
+ curl -k -H X-JFrog-Art-Api:**** 'https://artifactory.xxx.com/api/search/latestVersion?g=com.xxx.bwce&a=billing-app-v1&v=1.0.6'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 23 0 23 0 0 34 0 --:--:-- --:--:-- --:--:-- 34
+ '-SNAPSHOT&repos=libs-snapshot-local'
Please suggest if there is a way to fix this. I am not sure why its adding a newline in the curl statement, but echo shows no newline in it.

You can trim the newline off the end of version by adding the line
version = version.trim()
After your def version = line

Related

Why is WLP installUtility not able obtain assets from feature repo?

I am running docker build with following Dockerfile, with main idea to use feature repo as described https://github.com/WASdev/ci.docker#installing-liberty-features-from-local-repository-19008:
FROM websphere-liberty-kernel-ubi-min:19.0.0.9
COPY usr/ /opt/ibm/wlp/usr/
USER root
ARG FEATURE_REPO_URL=http://xyz.openshift.local/19.0.0.9/repo.zip
ARG VERBOSE=true
RUN configure.sh
RUN chown -R 1001:0 /tmp \
&& chmod -R g+rw /tmp \
&& chown -R 1001:0 /opt/ibm/wlp/output \
&& chmod -R g+rw /opt/ibm/wlp/output \
&& chown -R 1001:0 /opt/ibm/wlp/usr/servers/defaultServer \
&& chmod -R g+rw /opt/ibm/wlp/usr/servers/defaultServer \
&& chown -R 1001:0 /opt/ibm/wlp/usr/shared/resources \
&& chmod -R g+rw /opt/ibm/wlp/usr/shared/resources
USER 1001
Docker build output shows that repo.zip is downloaded and missing features are detected:
+ '[' http://xyz.openshift.local/19.0.0.9/repo.zip ']'
+ curl -k --fail http://xyz.openshift.local/19.0.0.9/repo.zip % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 14 311M 14 44.8M 0 0 55.0M 0 0:00:05 --:--:-- 0:00:05 54.9M 31 311M 31 99.4M 0 0 54.8M 0 0:00:05 0:00:01 0:00:04 54.7M 49 311M 49 152M 0 0 54.2M 0 0:00:05 0:00:02 0:00:03 54.2M 66 311M 66 207M 0 0 54.4M 0 0:00:05 0:00:03 0:00:02 54.4M 83 311M 83 260M 0 0 54.0M 0 0:00:05 0:00:04 0:00:01 54.0M 99 311M 99 310M 0 0 53.4M 0 0:00:05 0:00:05 --:--:-- 53.1M100 311M 100 311M 0 0 53.4M 0 0:00:05 0:00:05 --:--:-- 52.7M
+ installUtility install --acceptLicense defaultServer --from=/tmp/repo.zip
Checking for missing features required by the server ...
The server requires the following additional features: mpconfig-1.3 transportsecurity-1.0 cdi-2.0 mpopenapi-1.0 jaxws-2.2 jsonp-1.1 jpa-2.2 mprestclient-1.3 mphealth-2.1 wssecurity-1.1 jaxrs-2.1. Installing features from the repository ...
Successfully connected to the configured repository.
but then installation of all features fails:
Preparing assets for installation. This process might take several minutes to complete.
CWWKF1259E: Unable to obtain the following assets: mpconfig-1.3 transportsecurity-1.0 cdi-2.0 mpopenapi-1.0 jaxws-2.2 jsonp-1.1 jpa-2.2 mprestclient-1.3 mphealth-2.1 wssecurity-1.1 jaxrs-2.1. Ensure that the specified assets are valid. To find the IDs of applicable assets, run the installUtility find command.
I have taken a look in downloaded repo.zip and I can find following file (that should match one of the missing features):
So, what is the reason for the error?
Using different image (ibmcom/websphere-liberty:some21.0.0.3version) and pointing FEATURE_REPO_URL to place where 21.0.0.3 repo is hosted works.
So whoever prepared 19.0.0.3 image for me and said which feature repo to use, pointed me to some incompatible artefacts.
What is interesting, when I combined different versions of image vs repo, configure.sh was nicely verbose (it explained there is incompatibility), but this error CWWKF1259E: Unable to obtain the following assets is really unhelpful.

Jenkins's Docker Swarm plugin not resolving environment variables

I am currently setting up a Cloudbees Jenkins cluster with the help of the docker swarm plugin. I got it to work once, but haven't gotten it to work ever since. I get several different errors related to the creation of the agent that executes the pipeline, but they all seem to be related to the environment variables that the plugin creates at runtime. Does someone know a solution to this?
[1:40:45 PM] Creating Service with Name : agt-vice_testing_59-72
[1:40:45 PM] ServiceSpec created with ID : kk0jxbuqvh4zyngruamxphl4b
[1:40:45 PM] ServiceSpec request JSON : {"TaskTemplate":{"ContainerSpec":{"Image":"docker:17.12","Command":["sh","-cx","docker run --rm --privileged java:8 sh -xc 'curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL' "],"Env":["DOCKER_SWARM_PLUGIN_JENKINS_AGENT_SECRET=5ee89222c397f86c7ca8ecc6af73268539fa3ea9f64e46f815c4cfc06322aa0f","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JAR_URL=http://199.18.14.456:8080/jnlpJars/agent.jar","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL=http://199.18.14.456:8080/computer/agt-vice_testing_59-72/slave-agent.jnlp","DOCKER_SWARM_PLUGIN_JENKINS_AGENT_NAME=agt-vice_testing_59-72"],"Dir":"/tmp","User":"root","DNSConfig":{"Nameservers":[],"Search":[],"Options":[]},"Mounts":[{"Target":"/var/run/docker.sock","Source":"/var/run/docker.sock","Type":"bind","VolumeOptions":null}],"Hosts":[],"Secrets":[],"Configs":[]},"RestartPolicy":{"Condition":"none","MaxAttempts":0},"Resources":{"Limits":{"NanoCPUs":0,"MemoryBytes":0},"Reservations":{"NanoCPUs":0,"MemoryBytes":0}},"Placement":{"Constraints":[]}},"EndpointSpec":{"Ports":[]},"Name":"agt-vice_testing_59-72","Labels":{"ROLE":"jenkins-agent"},"Networks":[]}
�+ docker run --rm --privileged java:8 sh -xc curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar && java -jar agent.jar -jnlpUrl $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL
d+ curl --connect-timeout 20 --max-time 60 -o agent.jar http://199.18.14.456:8080/jnlpJars/agent.jar
P % Total % Received % Xferd Average Speed Time Time Time Current
N Dload Upload Total Spent Left Speed
�
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 770k 100 770k 0 0 63.9M 0 --:--:-- --:--:-- --:--:-- 68.4M
+ java -jar agent.jar -jnlpUrl
#Option "-jnlpUrl" takes an operand
!java -jar agent.jar [options...]

Docker build: returned a non-zero code: 1

This line causes my docker build to fail. I've tried splitting it up but get different errors. What would be the proper way to define this line?
ARG lame=3.99.5
RUN curl -L http://sourceforge.net/projects/lame/files/lame/3.99/lame-${lame}.tar.gz | tar -xzf- && \
cd lame-${lame} && \
./configure && make install
I get this error:
Step 13/38 : RUN curl -L http://sourceforge.net/projects/lame/files/lame/3.99/lame-${lame}.tar.gz | tar -xzf- && cd lame-${lame} && ./configure && make install
---> Running in 1d9635c2e249
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 178 100 178 0 0 456 0 --:--:-- --:--:-- --:--:-- 456
0 345 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
0 15608 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
0 341 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
100 1411k 100 1411k 0 0 166k 0 0:00:08 0:00:08 --:--:-- 334k
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/usr/local/src/lame-3.99.5':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
The command '/bin/sh -c curl -L http://sourceforge.net/projects/lame/files/lame/3.99/lame-${lame}.tar.gz | tar -xzf- && cd lame-${lame} && ./configure && make install' returned a non-zero code: 1

Docker Build: xz(stdin): File format not recognized

Hi total Docker noob here, is there a dependency i can run so this error goes away?
Step 19/40 : RUN curl -OL https://www.haskell.org/ghc/dist/${ghc%[a-z]}/ghc-${ghc}-x86_64-deb7-linux.tar.xz | tar -xJf- && cd ghc-${ghc} && ./configure && make install
---> Running in 0141996d759f
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 161 100 161 0 0 335 0 --:--:-- --:--:-- --:--:-- 336
100 85.5M 100 85.5M 0 0 308k 0 0:04:44 0:04:44 --:--:-- 358k
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c curl -OL https://www.haskell.org/ghc/dist/${ghc%[a-z]}/ghc-${ghc}-x86_64-deb7-linux.tar.xz | tar -xJf- && cd ghc-${ghc} && ./configure && make install' returned a non-zero code: 2
The problem is the -O option to curl that writes to disk instead of stdout.
Try with: curl -L https://...

ERROR: Could not find 'which' command, make sure it's available first before continuing installation

I'm trying to deploy rails application in Cent-OS. When I run curl -L get.rvm.io | bash -s stable to setup ruby environment.
I got the following output
[root#186-aven-vps marketeurepo]# curl -L get.rvm.io | bash -s stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 184 100 184 0 0 581 0 --:--:-- --:--:-- --:--:-- 580
100 22721 100 22721 0 0 35092 0 --:--:-- --:--:-- --:--:-- 35092
ERROR: Could not find 'which' command, make sure it's available first before continuing installation.
I'm using Mac OS X on local machine and I'm new to this.Any suggestion will be appreciated.Thank you.
I solved it by running this command:
$ yum install which

Resources