I am running native Windows, and primarily using Git Bash as my terminal for programming.
I am trying to run the command
winpty docker run -it apache/spark-py /opt/spark/bin/pyspark
This works fine in the regular Windows Command Prompt terminal (without winpty), but it does not work in Git Bash. The error it throws back is
$ winpty docker run -it apache/spark-py /opt/spark/bin/pyspark
++ id -u
+ myuid=185
++ id -g
+ mygid=0
+ set +e
++ getent passwd 185
+ uidentry=
+ set -e
+ '[' -z '' ']'
+ '[' -w /etc/passwd ']'
+ echo '185:x:185:0:anonymous uid:/opt/spark:/bin/false'
+ '[' -z /usr/local/openjdk-11 ']'
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ env
+ grep SPARK_JAVA_OPT_
+ sort -t_ -k4 -n
+ sed 's/[^=]*=\(.*\)/\1/g'
+ readarray -t SPARK_EXECUTOR_JAVA_OPTS
+ '[' -n '' ']'
+ '[' -z ']'
+ '[' -z ']'
+ '[' -n '' ']'
+ '[' -z ']'
+ '[' -z ']'
+ '[' -z x ']'
+ SPARK_CLASSPATH='/opt/spark/conf::/opt/spark/jars/*'
+ case "$1" in
+ echo 'Non-spark-on-k8s command provided, proceeding in pass-through mode...'
Non-spark-on-k8s command provided, proceeding in pass-through mode...
+ CMD=("$#")
+ exec /usr/bin/tini -s -- C:/Users/xxxxxxx/AppData/Local/Programs/Git/opt/spark/bin/pyspark
[FATAL tini (14)] exec C:/Users/xxxxxxx/AppData/Local/Programs/Git/opt/spark/bin/pyspark failed: No such file or directory
It's like it somehow uses the Windows Git /opt/spark/bin/pyspark that I feed in to the docker run command. Any ideas?
Related
I have a script that works fine in sh on a linux host as well as inside an alpine container. But when I try executing that using docker exec <containerID> sh -c "<script>" it misbehaves. The script's function is to output stuff similar to ps.
systick=$(getconf CLK_TCK); for c in /proc/*/cmdline; do d=$(dirname $c); name=$(grep Name: $d/status); pid=$(basename $d); uid=$(grep Uid: $d/status); uid=$(echo ${uid#Uid:} | xargs); uid=${uid%% *}; user=$(grep :$uid:[0-9] /etc/passwd); user=${user%%:*}; cmdline=$(cat $c|xargs -0 echo); starttime=$(($(awk '{print $22}' $d/stat) / systick)); uptime=$(awk '{print int($1)}' /proc/uptime); elapsed=$(($uptime-$starttime)); echo $pid $user $elapsed $cmdline; done
EDIT: sh -c "<script>" has the same behavior.
You are not able to run this script from docker exec because the variables will be interpolated before they sent to the container (i.e., you are going to get values from your local machine, not from within the container).
In order to run it as you wish, you need to replace $ with \$ for every occurrence of $ in your script.
What might work better is to put your script into a file, then map the file to a location within the container, using -v (i.e., -v script.sh:/path/to/script.sh), and call the script via docker exec /path/to/script.sh
Part 1: A Working Answer
A Working One-Liner (Quoted For Use By Docker)
getProcessDataDef='shellQuoteWordsDef='"'"'shellQuoteWords() { sq="'"'"'"'"'"'"'"'"'"; dq='"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; for arg; do printf "'"'"'"'"'"'"'"'"'%s'"'"'"'"'"'"'"'"' " "$(printf '"'"'"'"'"'"'"'"'%s\n'"'"'"'"'"'"'"'"' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"; done; printf '"'"'"'"'"'"'"'"'\n'"'"'"'"'"'"'"'"'; }'"'"'; shellQuoteNullSeparatedStream() { xargs -0 sh -c "${shellQuoteWordsDef};"'"'"' shellQuoteWords "$#"'"'"' _; }; getProcessData() { systick=$(getconf CLK_TCK); for c in /proc/*/cmdline; do d=${c%/*}; pid=${d##*/}; name=$(awk '"'"'/^Name:/ { print $2 }'"'"' <"$d"/status); uid=$(awk '"'"'/^Uid:/ { print $2 }'"'"' <"$d"/status); pwent=$(getent passwd "$uid"); user=${pwent%%:*}; cmdline=$(shellQuoteNullSeparatedStream <"$c"); starttime=$(awk -v systick="$systick" '"'"'{print int($22 / systick)}'"'"' "$d"/stat); uptime=$(awk '"'"'{print int($1)}'"'"' /proc/uptime); elapsed=$((uptime-starttime)); echo "$pid $user $elapsed $cmdline"; done; }; getProcessData'
sh -c "$getProcessDataDef" # or docker exec <container> sh -c "$getProcessDataDef"
A Working One-Liner (Before Quoting/Escaping)
shellQuoteWordsDef='shellQuoteWords() { sq="'"'"'"; dq='"'"'"'"'"'; for arg; do printf "'"'"'%s'"'"' " "$(printf '"'"'%s\n'"'"' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"; done; printf '"'"'\n'"'"'; }'; shellQuoteNullSeparatedStream() { xargs -0 sh -c "${shellQuoteWordsDef};"' shellQuoteWords "$#"' _; }; getProcessData() { systick=$(getconf CLK_TCK); for c in /proc/*/cmdline; do d=${c%/*}; pid=${d##*/}; name=$(awk '/^Name:/ { print $2 }' <"$d"/status); uid=$(awk '/^Uid:/ { print $2 }' <"$d"/status); pwent=$(getent passwd "$uid"); user=${pwent%%:*}; cmdline=$(shellQuoteNullSeparatedStream <"$c"); starttime=$(awk -v systick="$systick" '{print int($22 / systick)}' "$d"/stat); uptime=$(awk '{print int($1)}' /proc/uptime); elapsed=$((uptime-starttime)); echo "$pid $user $elapsed $cmdline"; done; }; getProcessData "$#"
What Went Into That One-Liner
shellQuoteWordsDef='shellQuoteWords() { sq="'"'"'"; dq='"'"'"'"'"'; for arg; do printf "'"'"'%s'"'"' " "$(printf '"'"'%s\n'"'"' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"; done; printf '"'"'\n'"'"'; }'
shellQuoteNullSeparatedStream() {
xargs -0 sh -c "${shellQuoteWordsDef};"' shellQuoteWords "$#"' _
}
getProcessData() {
systick=$(getconf CLK_TCK)
for c in /proc/*/cmdline; do
d=${c%/*}; pid=${d##*/}
name=$(awk '/^Name:/ { print $2 }' <"$d"/status)
uid=$(awk '/^Uid:/ { print $2 }' <"$d"/status)
pwent=$(getent passwd "$uid")
user=${pwent%%:*}
cmdline=$(shellQuoteNullSeparatedStream <"$c")
starttime=$(awk -v systick="$systick" '{print int($22 / systick)}' "$d"/stat)
uptime=$(awk '{print int($1)}' /proc/uptime)
elapsed=$((uptime-starttime))
echo "$pid $user $elapsed $cmdline"
done
}
What Went Into The Shell-Quoting Helper Used By That One-Liner
To allow easier reading and editing, the function stringified above looks like:
# This is the function we're including in our code passed to xargs in-band above:
shellQuoteWords() {
sq="'"; dq='"'
for arg; do
printf "'%s' " "$(printf '%s\n' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"
done
printf '\n'
}
Part 2: How That Answer Was Created
Python has an excellent shlex.quote() function (or pipes.quote() in Python 2) that can be used to generate a shell-quoted version of a string. In this context, that can be used as follows:
Python 3.7.6 (default, Feb 27 2020, 15:15:00)
[Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> s = r'''
... shellQuoteWords() {
... sq="'"; dq='"'
... for arg; do
... printf "'%s' " "$(printf '%s\n' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"
... done
... printf '\n'
... }
... '''
>>> import shlex
>>> print(shlex.quote(s))
'
shellQuoteWords() {
sq="'"'"'"; dq='"'"'"'"'"'
for arg; do
printf "'"'"'%s'"'"' " "$(printf '"'"'%s\n'"'"' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"
done
printf '"'"'\n'"'"'
}
'
That result is itself a perfectly valid string in shell. That is to say, one can run:
s='
shellQuoteWords() {
sq="'"'"'"; dq='"'"'"'"'"'
for arg; do
printf "'"'"'%s'"'"' " "$(printf '"'"'%s\n'"'"' "$arg" | sed -e "s#${sq}#${sq}${dq}${sq}${dq}${sq}#g")"
done
printf '"'"'\n'"'"'
}
'
eval "$s"
shellQuoteWords "hello world" 'hello world' "hello 'world'" 'hello "world"'
...and get completely valid output.
The same process was followed to generate a string that evaluated to the definition of getProcessData.
I'm trying to run a Spark jar on Kubernetes. I have built my own docker image using the template spark-2.4.4-bin-hadoop2.7 and tried to run my yaml file with this docker image.
I have got below error from driver log /opt/entrypoint.sh:
line 133: /sbin/tini: No such file or directory.
I understand that /sbin/tini is not present in docker image. Anybody could help me solve this issue? How to get this tini in docker image path?
command entered:
kubectl create -f spark.yaml
driver log:
kubectl logs spark-wordcount-7-driver
++ id -u
+ myuid=0
++ id -g
+ mygid=0
+ set +e
++ getent passwd 0
+ uidentry=root:x:0:0:root:/root:/bin/bash
+ set -e
+ '[' -z root:x:0:0:root:/root:/bin/bash ']'
+ SPARK_K8S_CMD=driver
+ case "$SPARK_K8S_CMD" in
+ shift 1
+ SPARK_CLASSPATH=':/opt/spark/jars/*'
+ grep SPARK_JAVA_OPT_
+ env
+ sort -t_ -k4 -n
+ sed 's/[^=]*=\(.*\)/\1/g'
+ readarray -t SPARK_EXECUTOR_JAVA_OPTS
+ '[' -n '' ']'
+ '[' -n '' ']'
+ PYSPARK_ARGS=
+ '[' -n '' ']'
+ R_ARGS=
+ '[' -n '' ']'
+ '[' '' == 2 ']'
+ '[' '' == 3 ']'
+ case "$SPARK_K8S_CMD" in
+ CMD=("$SPARK_HOME/bin/spark-submit" --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" --deploy-mode client "$#")
+ exec /sbin/tini -s -- /opt/spark/bin/spark-submit --conf spark.driver.bindAddress=172.17.0.6 --deploy-mode client --properties-file /opt/spark/conf/spark.properties --class com.walmart.WordCount spark-internal
/opt/entrypoint.sh: line 133: /sbin/tini: No such file or directory
Since, you are using alpine image, you have to use /sbin/tini
NOTE: alpine has moved tini to /sbin/tini
In file: entrypoint.sh:
Do the following change
exec /usr/bin/tini -s -- /usr/bin/spark-operator "$#"
to
exec /sbin/tini -s -- /usr/bin/spark-operator "$#"
I have a basic docker-compose file file for wurstmeister/kafka
I'm trying to configure it to use SASL_PLAIN with SSL
However I keep getting this error no matter how many ways I try to specify my jaas file
This is the error I get
[2018-04-11 10:34:34,545] FATAL [KafkaServer id=1001] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'sasl_ssl.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
These are the vars I have. Last one is where I specify my jaas file
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_HOST_NAME: 10.10.10.1
KAFKA_PORT: 9092
KAFKA_ADVERTISED_PORT: 9093
KAFKA_ADVERTISED_HOST_NAME: 10.10.10.1
KAFKA_LISTENERS: PLAINTEXT://:9092,SASL_SSL://:9093
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://10.10.10.1:9092,SASL_SSL://10.10.10.1:9093
KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_SSL
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SSL_TRUSTSTORE_LOCATION: /kafka.server.truststore.jks
KAFKA_SSL_TRUSTSTORE_PASSWORD: password
KAFKA_SSL_KEYSTORE_LOCATION: /kafka.server.keystore.jks
KAFKA_SSL_KEYSTORE_PASSWORD: password
KAFKA_SSL_KEY_PASSWORD: password
KAFKA_OPTS: '-Djava.security.auth.login.config=/path/kafka_server_jaas.conf'
Also when I try to check the docker logs I see
/usr/bin/start-kafka.sh: line 96: KAFKA_OPTS=-Djava.security.auth.login.config: bad substitution
Any help is greatly appreciated!
equals '=' inside the last value is causing this issue.
KAFKA_OPTS: '-Djava.security.auth.login.config=/path/kafka_server_jaas.conf'
This is what I have got after debugging.
+ for VAR in $(env)
+ [[ KAFKA_OPTS=-
Djava.security.auth.login.config=/path/kafka_server_jaas.conf =~ ^KAFKA_ ]]
+ [[ ! KAFKA_OPTS=-
Djava.security.auth.login.config=/path/kafka_server_jaas.conf =~
^KAFKA_HOME ]]
++ echo KAFKA_OPTS=-
Djava.security.auth.login.config=/path/kafka_server_jaas.conf
++ sed -r 's/KAFKA_(.*)=.*/\1/g'
++ tr '[:upper:]' '[:lower:]'
++ tr _ .
+ kafka_name=opts=-djava.security.auth.login.config
++ echo KAFKA_OPTS=-
Djava.security.auth.login.config=/path/kafka_server_jaas.conf
++ sed -r 's/(.*)=.*/\1/g'
+ env_var=KAFKA_OPTS=-Djava.security.auth.login.config
+ grep -E -q '(^|^#)opts=-djava.security.auth.login.config='
/opt/kafka/config/server.properties
start-kafka.sh: line 96: KAFKA_OPTS=-Djava.security.auth.login.config: bad
substitution
and this is the piece of code that is performing this operation.
88 for VAR in $(env)
89 do
90 if [[ $VAR =~ ^KAFKA_ && ! $VAR =~ ^KAFKA_HOME ]]; then
91 kafka_name=$(echo "$VAR" | sed -r 's/KAFKA_(.*)=.*/\1/g' | tr '[:upper:]' '[:lower:]' | tr _ .)
92 env_var=$(echo "$VAR" | sed -r 's/(.*)=.*/\1/g')
93 if grep -E -q '(^|^#)'"$kafka_name=" "$KAFKA_HOME/config/server.properties"; then
94 sed -r -i 's#(^|^#)('"$kafka_name"')=(.*)#\2='"${!env_var}"'#g' "$KAFKA_HOME/config/server.properties" #note that no config values may contain an '#' char
95 else
96 echo "$kafka_name=${!env_var}" >> "$KAFKA_HOME/config/server.properties"
97 fi
98 fi
99
100 if [[ $VAR =~ ^LOG4J_ ]]; then
101 log4j_name=$(echo "$VAR" | sed -r 's/(LOG4J_.*)=.*/\1/g' | tr '[:upper:]' '[:lower:]' | tr _ .)
102 log4j_env=$(echo "$VAR" | sed -r 's/(.*)=.*/\1/g')
103 if grep -E -q '(^|^#)'"$log4j_name=" "$KAFKA_HOME/config/log4j.properties"; then
104 sed -r -i 's#(^|^#)('"$log4j_name"')=(.*)#\2='"${!log4j_env}"'#g' "$KAFKA_HOME/config/log4j.properties" #note that no config values may contain an'#' char
105 else
106 echo "$log4j_name=${!log4j_env}" >> "$KAFKA_HOME/config/log4j.properties"
107 fi
108 fi
109 done
Update: They have fixed it and it is merged now!
https://github.com/wurstmeister/kafka-docker/pull/321
There's a bug open now with wurstmeister/kafka but they have gotten back to me with a workaround as follows
I believe his is part of a larger namespace collision problem that
affects multiple elements such as Kubernetes deployments etc (as well
as other KAFKA_ service settings).
Given you are referencing an external file /kafka_server_jaas.conf,
i'm assuming you're OK adding/mounting extra files through; a
work-around is to specify a CUSTOM_INIT_SCRIPT environment var, which
should be a script similar to:
#!/bin/bash
export KAFKA_OPTS="-Djava.security.auth.login.config=/kafka_server_jaas.conf"
This is executed after the substitution part that is failing.
This could have been done inline, however there is currently a bug in
how we process the environment, where we need to specify the input
separator to make this work correctly.
Hopefully this works!
I'm trying to automate a JMX command using a tool called JMXterm.
The script looks like so:
#!/bin/bash
beanstemp="/tmp/jmx_beans"
read -r -p "Enter server name in FQDN " scrapername
bean1="BrokerName=localhost,Connection=Scraper_$scrapername,ConnectorName=openwire,Type=Connection"
echo beans -d $domain | $cmd > $beanstemp
idnum=$(grep "ID_$scrapername" $beanstemp | grep openwire | awk -F- '{print $2"-"$3}')
idname="$scrapername-$idnum"
bean2="BrokerName=localhost,Connection=ID_"$idname"2_0,ConnectorName=openwire,Type=Connection"
domain="org.apache.activemq"
server="scrapermq.sj.peer39.com:1099"
cmd="java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l $server"
echo run stop -b $bean1 -d $domain | $cmd -l $server
echo run stop -b $bean2 -d $domain | $cmd -l $server
When I run the script I get the proper response for the first command but the second one fails.
Here's the output of sh -x:
itaig#iganot-lt:~$ sh -x jmx.sh
+ beanstemp=/tmp/jmx_beans
+ read -r -p Enter server name in FQDN scrapername
Enter server name in FQDN selvmnj27.domain.company.com
+ bean1=BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
+ echo beans -d
+
+ + grep openwire
grep ID_selvmnj27.domain.company.com+ awk -F- {print $2"-"$3}
/tmp/jmx_beans
+ idnum=
+ idname=selvmnj27.domain.company.com-
+ bean2=BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
+ domain=org.apache.activemq
+ server=scrapermq.sj.peer39.com:1099
+ cmd=java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.sj.peer39.com:1099
+ echo+ run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connectionjava -jar -d Downloads/jmxterm-1.0-alpha-4-uber.jar -l org.apache.activemq scrapermq.sj.peer39.com:1099 -l
scrapermq.sj.peer39.com:1099
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#calling operation stop of mbean org.apache.activemq:BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
#operation returns:
null
$>+ + echo runjava stop -b -jar Downloads/jmxterm-1.0-alpha-4-uber.jar BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -l scrapermq.sj.peer39.com:1099 -l -d scrapermq.sj.peer39.com:1099
org.apache.activemq
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#InstanceNotFoundException: org.apache.activemq:BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
$>itaig#iganot-lt:~$
"Operation returns: null" is the response I'm looking for.
The problem begins with this line:
echo beans -d $domain | $cmd > $beanstemp
When it runs the output of the command should be written to the $beanstemp file but the file stays empty.
When I run the command manually from terminal it works:
itaig#iganot-lt:~$ echo beans -d org.apache.activemq | java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.domain.company.com:1099 > /tmp/jmx_beans
Welcome to JMX terminal. Type "help" for available commands.
$>beans -d org.apache.activemq
#domain = org.apache.activemq:
$>itaig#iganot-lt:~$ tail -5 /tmp/jmx_beans
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj28.domain.company.com,consumerId=ID_selvmnj28.domain.company.com-45117-1431866382308-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
itaig#iganot-lt:~$
So my question is: What am I doing wrong and why does the command work when running manually from terminal but not from the script?
Thanks in advance
Ok I found the problem.
Certain lines included variables which have not been declared in time.
After re-organizing the order of variable declaration the problem has been solved.
I am planning to run build and test cases and deploy using jenkins. I have installed Jenkins and creating job.
I have bitbucket repository with mercurial, So configured mercurial and build clone repository and do nothing else. Now I am writing commands in shell for given purpose:
source ~/.profile # load profile and working fine
mkvirtualenv test_build # create virtual environment using virtualenv wrapper. this fails with trace provided.
cd my_project # move to project directory
pip install -r requirements.txt # install packages using pip
Here is an trace for build console on jenkins.
[workspace] $ /usr/local/bin/bash -xe /tmp/hudson3781010986042746968.sh
+ source /usr/local/jenkins/.profile
++ PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/local/jenkins/bin
++ export PATH
++ BLOCKSIZE=K
++ export BLOCKSIZE
++ EDITOR=vi
++ export EDITOR
++ PAGER=more
++ export PAGER
++ ENV=/usr/local/jenkins/.shrc
++ export ENV
++ '[' -x /usr/games/fortune ']'
++ '[' -e /usr/local/bin/virtualenvwrapper.sh ']'
++ export WORKON_HOME=/usr/local/jenkins/virtualenvs
++ WORKON_HOME=/usr/local/jenkins/virtualenvs
++ source /usr/local/bin/virtualenvwrapper.sh
+++ '[' '' = '' ']'
++++ command which python
++++ which python
+++ VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python
+++ '[' '' = '' ']'
+++ VIRTUALENVWRAPPER_VIRTUALENV=virtualenv
+++ '[' '' = '' ']'
+++ VIRTUALENVWRAPPER_VIRTUALENV_CLONE=virtualenv-clone
+++ VIRTUALENVWRAPPER_ENV_BIN_DIR=bin
+++ '[' '' = Windows_NT ']'
+++ '[' .project = '' ']'
+++ virtualenvwrapper_initialize
++++ virtualenvwrapper_derive_workon_home
++++ typeset workon_home_dir=/usr/local/jenkins/virtualenvs
++++ '[' /usr/local/jenkins/virtualenvs = '' ']'
++++ echo /usr/local/jenkins/virtualenvs
++++ unset GREP_OPTIONS
++++ command grep '^[^/~]'
++++ grep '^[^/~]'
++++ echo /usr/local/jenkins/virtualenvs
++++ unset GREP_OPTIONS
++++ command egrep '([\$~]|//)'
++++ egrep '([\$~]|//)'
++++ echo /usr/local/jenkins/virtualenvs
++++ return 0
+++ export WORKON_HOME=/usr/local/jenkins/virtualenvs
+++ WORKON_HOME=/usr/local/jenkins/virtualenvs
+++ virtualenvwrapper_verify_workon_home -q
+++ RC=0
+++ '[' '!' -d /usr/local/jenkins/virtualenvs/ ']'
+++ return 0
+++ '[' /usr/local/jenkins/virtualenvs = '' ']'
+++ '[' /usr/local/jenkins/virtualenvs = '' ']'
+++ virtualenvwrapper_run_hook initialize
+++ typeset hook_script
+++ typeset result
++++ virtualenvwrapper_tempfile initialize-hook
++++ typeset suffix=initialize-hook
++++ typeset file
+++++ virtualenvwrapper_mktemp -t virtualenvwrapper-initialize-hook-XXXXXXXXXX
+++++ command mktemp -t virtualenvwrapper-initialize-hook-XXXXXXXXXX
+++++ mktemp -t virtualenvwrapper-initialize-hook-XXXXXXXXXX
++++ file=/tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
++++ '[' 0 -ne 0 ']'
++++ '[' -z /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1 ']'
++++ '[' '!' -f /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1 ']'
++++ echo /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
++++ return 0
+++ hook_script=/tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
+++ '[' -z /usr/local/jenkins/virtualenvs ']'
+++ /usr/local/bin/python -c 'from virtualenvwrapper.hook_loader import main; main()' --script /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1 initialize
+++ result=0
+++ '[' 0 -eq 0 ']'
+++ '[' '!' -f /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1 ']'
+++ source /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
++++ '[' -f /usr/local/jenkins/virtualenvs/initialize ']'
++++ source /usr/local/jenkins/virtualenvs/initialize
+++ command rm -f /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
+++ rm -f /tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX.jKEvY7Y1
+++ return 0
+++ virtualenvwrapper_setup_tab_completion
+++ '[' -n /usr/local/bin/bash ']'
+++ complete -o nospace -F _cdvirtualenv_complete -S/ cdvirtualenv
+++ complete -o nospace -F _cdsitepackages_complete -S/ cdsitepackages
+++ complete -o default -o nospace -F _virtualenvs workon
+++ complete -o default -o nospace -F _virtualenvs rmvirtualenv
+++ complete -o default -o nospace -F _virtualenvs cpvirtualenv
+++ complete -o default -o nospace -F _virtualenvs showvirtualenv
+++ return 0
+ mkvirtualenv test_build
+ typeset -a in_args
+ typeset -a out_args
+ typeset -i i
+ typeset tst
+ typeset a
+ typeset envname
+ typeset requirements
+ typeset packages
+ in_args=("$#")
+ '[' -n '' ']'
+ i=0
+ tst=-lt
+ '[' 0 -lt 1 ']'
+ a=test_build
+ case "$a" in
+ '[' 0 -gt 0 ']'
+ out_args=("$a")
+ i=1
+ '[' 1 -lt 1 ']'
+ set -- test_build
+ eval 'envname=$1'
++ envname=test_build
+ virtualenvwrapper_verify_workon_home
+ RC=0
+ '[' '!' -d /usr/local/jenkins/virtualenvs/ ']'
+ return 0
+ virtualenvwrapper_verify_virtualenv
+ virtualenvwrapper_verify_resource virtualenv
++ command which virtualenv
++ which virtualenv
++ unset GREP_OPTIONS
++ command grep -v 'not found'
++ grep -v 'not found'
+ typeset exe_path=/usr/local/bin/virtualenv
+ '[' /usr/local/bin/virtualenv = '' ']'
+ '[' '!' -e /usr/local/bin/virtualenv ']'
+ return 0
+ '[' -n '' ']'
+ virtualenvwrapper_cd /usr/local/jenkins/virtualenvs
+ '[' -n /usr/local/bin/bash ']'
+ builtin cd /usr/local/jenkins/virtualenvs
+ virtualenv test_build
New python executable in test_build/bin/python2.7
Also creating executable in test_build/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
+ '[' -d /usr/local/jenkins/virtualenvs/test_build ']'
+ virtualenvwrapper_run_hook pre_mkvirtualenv test_build
+ typeset hook_script
+ typeset result
++ virtualenvwrapper_tempfile pre_mkvirtualenv-hook
++ typeset suffix=pre_mkvirtualenv-hook
++ typeset file
+++ virtualenvwrapper_mktemp -t virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX
+++ command mktemp -t virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX
+++ mktemp -t virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX
++ file=/tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
++ '[' 0 -ne 0 ']'
++ '[' -z /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4 ']'
++ '[' '!' -f /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4 ']'
++ echo /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
++ return 0
+ hook_script=/tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
+ '[' -z /usr/local/jenkins/virtualenvs ']'
+ /usr/local/bin/python -c 'from virtualenvwrapper.hook_loader import main; main()' --script /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4 pre_mkvirtualenv test_build
virtualenvwrapper.user_scripts creating /usr/local/jenkins/virtualenvs/test_build/bin/predeactivate
virtualenvwrapper.user_scripts creating /usr/local/jenkins/virtualenvs/test_build/bin/postdeactivate
virtualenvwrapper.user_scripts creating /usr/local/jenkins/virtualenvs/test_build/bin/preactivate
virtualenvwrapper.user_scripts creating /usr/local/jenkins/virtualenvs/test_build/bin/postactivate
virtualenvwrapper.user_scripts creating /usr/local/jenkins/virtualenvs/test_build/bin/get_env_details
+ result=0
+ '[' 0 -eq 0 ']'
+ '[' '!' -f /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4 ']'
+ source /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
+ command rm -f /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
+ rm -f /tmp/virtualenvwrapper-pre_mkvirtualenv-hook-XXXXXXXXXX.rPgLzOe4
+ return 0
+ typeset RC=0
+ '[' 0 -ne 0 ']'
+ '[' '!' -d /usr/local/jenkins/virtualenvs/test_build ']'
+ '[' '!' -z '' ']'
+ workon test_build
+ typeset env_name=test_build
+ '[' test_build = '' ']'
+ virtualenvwrapper_verify_workon_home
+ RC=0
+ '[' '!' -d /usr/local/jenkins/virtualenvs/ ']'
+ return 0
+ virtualenvwrapper_verify_workon_environment test_build
+ typeset env_name=test_build
+ '[' '!' -d /usr/local/jenkins/virtualenvs/test_build ']'
+ return 0
+ activate=/usr/local/jenkins/virtualenvs/test_build/bin/activate
+ '[' '!' -f /usr/local/jenkins/virtualenvs/test_build/bin/activate ']'
+ type deactivate
Build step 'Execute shell' marked build as failure
Finished: FAILURE