I am trying to implement a script in gitlab CI to trigger a smoke test via Jenkins and then get the results.
So far I am able to trigger the job successfully and I am trying to follow this to implement a monitoring stage and then get the result once the job finishes.
my issue is that I implemented a while loop to monitor if the Jenkins job has finished so far the script is giving either syntax errors (when copied in gitlab) or if run in the terminal I get:
job is building? true
waiting...
parse error: Invalid numeric literal at line 2, column 0
job is still building?
job is building?
This is what I am using so far:
#!/bin/bash
running="true"
while [ "$running" != "false" ]
do
echo "job is building? ${running}"
echo "waiting...";
sleep 2;
running=$(curl -s --user ${EMAIL}:${TOKEN} ${URL}/${var}/lastBuild/api/json | jq .'building')
echo "job is still building? ${running}"
done
echo "Done!"
buildNumber=$(curl -s --user $EMAIL:$TOKEN ${URL}/$ENV-${var}/lastBuild/api/json | jq ".url" | awk -F "/" '{print $(NF-1)}')
echo "getting results for build ${buildNumber}"
curl -s --user ${EMAIL}:${TOKEN} ${URL}/${ENV}-${var}/lastBuild/api/json | jq ".url" | awk -F "/" '{print $(NF-1)}'
curl -v --silent --user ${EMAIL}:${TOKEN} ${URL}/${ENV}-${var}/lastBuild/consoleText 2>&1 | grep -i "finished:"
UPDATE
the script is running now ok in my local terminal
the change was
running="true"
while [ "$running" != "false" ]
do
echo "job is building? ${running}"
echo "waiting..."
sleep 2
curl -s --user $EMAIL:$TOKEN $URL/$ENV-${var}/lastBuild/api/json --output now.txt
running=$(jq .'building' now.txt)
echo "job is still building? ${running}"
done
The problem is still on Gitlab CI as after copy pasting this script I get in the pipeline
/bin/sh: eval: line 149: syntax error: unexpected "done"
I'm guessing that the error
parse error: Invalid numeric literal at line 2, column 0
comes from the line that does the curl
running=$(curl -s --user ${EMAIL}:${TOKEN} ${URL}/${var}/lastBuild/api/json | jq .'building')`
I'm assuming that the error is thrown by jq when it tries to parse the json and fails. Since this line has an error, the variable running never gets properly updated, meaning that the rest of the script doesn't work as intended.
If you fix this line, the rest of your pipeline should work. Consider looking into this question which has a similar problem and some solutions.
I have a Jenkins script that looks like this
stage ("Build and Deploy") {
steps {
script {
def statusCode = sh(script:"""ssh ${env.SERVER_NAME} << EOF
cd ${env.LOCATION}
git clone -b ${env.GIT_BRANCH} ${env.GIT_URL} ${env.FOLDER}
cd ${env.FOLDER}
... some other stuff goes here but isnt relevant ..
sudo docker-compose up -d --build
if [ ! \$(sudo docker container ls -f "name=config-provider-*" | wc -l ) -eq 4 ]
then
exit 1
fi
""", returnStatus:true).toString().trim()
if (statusCode == "1") {
error("At least one container failed to start")
}
}
}
}
What I want is to exit with error code 1 in the script if the number of running containers is not equal to 3 (wc -l == 4 including the header), but the if statement is evaluating true and exiting with error code 1 even though i know that the containers are successfully running.
I have tried
echo sh(script: """ssh ${env.SERVER_NAME} << EOF
echo \$(sudo docker container ls -f "name=config-provider-*" | wc -l)
""", returnStdout:true).toString()
and
echo sh(script: """ssh ${env.SERVER_NAME} << EOF
echo \$(sudo docker container ls -f "name=config-provider-*")
""", returnStdout:true).toString()
The latter outputted 4 lines within jenkins showing all of the running containers, as expected, but the former which includes "| wc -l" returned and printed out 0 in jenkins.
I have reproduced the steps of this script line by line manually from start to finish and it works as intended when not run from within jenkins.
Additionally, manually running the command:
[ ! $(sudo docker container ls -f "name=config-provider-*" | wc -l ) -eq 4 ] && echo failed
echoes nothing, and the following command returns an output of 4, which is expected.
echo $(sudo docker container ls -f "name=config-provider-*" | wc -l )
I have a script which has to move uploaded files from the first directory to the second directory.
The problem is that the script already moves the files during upload.
Anyone who can help?
#!/bin/sh
lockfile=/home/mediaze111/cronjobs/zenon_move.lock
if ( set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null; then
trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT
if [ "$(ls -A /home/mediaze111/domains/optimaal.fm/ZenonImport/Import1/)" ]; then
ps faux | grep -E 'UPLOAD' | grep -v 'grep' > /dev/null || mv -f /home/mediaze111/domains/optimaal.fm/ZenonImport/Import1/*.* /home/mediaze111/domains/optimaal.fm/ZenonImport/Import2/
fi
rm -f "$lockfile"
trap - INT TERM EXIT
fi
u can get the file size before upload . then check the destination file size . if same move uploaded. pul all the script in a loop so when was the same break it an move files.
I have a script to check the status of a url. I am trying to grep for the word connected from the below mentioned wget command output and print Running if it is found else print Not Running. How can I modify my grep to print only the word without the all the output from wget command
#!/bin/ksh
STAT=`wget 'http://server:port/ABC_Service/app' | grep connected`
if [ -z "$STAT" ] ; then
echo "Running"
else
echo "Not Running"
fi
Output of wget command:
--2013-05-31 11:09:32-- http://server:port/ABC_Service/app
Resolving server... 10.109.136.31
Connecting to server|10.109.136.31|:port... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.
Desired Output from my script:
Running
Check the return code
#!/bin/ksh
#use your wget command in place of echo below
echo "connected" 2>&1 | grep connected >/dev/null
retcode=$?
if [ $retcode = 0 ]
then
echo "Running"
else
echo "Not Running"
fi
If you can use Bash, then the variable $? reads the exit status of the last command executed.
I need to pass /DEF:c:\filepath\myLib.def" command line option from a bash script to MS compiler/linker. The path is generated as part of build process by a bash script. Basically, the argument that my script passes is:
-DEF:/c/filepath/myLib.def
MSYS path conversion can't handle it properly because it doesn't understand /DEF: part. It works if I do
-DEF=/c/filepath/myLib.def
but then ms tools don't understand this parameter. In short, what's the proper way to write that parameter in MSYS bash so that it converts it to proper argument?
On cygwin I could use cygpath, but there is no equivalent, because somebody from msys thinks that it's not needed (even if there are scripts for cygwin that uses cygpath).
Update (Aug-2016):
This question is no longer relevant, as msys2 now comes with cygpath in its installation.
...
I'll summarize my research here.
The cygpath equivalent in MSYS is to use this command:
{ cd /c/some/path && pwd -W; } | sed 's|/|\\|g'
The problem with this approach is that it requires existing path, e.g. the c:\some\path has to be an existing directory; however, real cygpath supports paths that do not exist.
So, if you need to get path to a directory that doesn't exist, then you can fallback to sed conversion of the path:
{ cd 2>/dev/null /c/some/path && pwd -W ||
echo /c/some/path | sed 's|^/\([a-z,A-Z]\)/|\1:/|'; } | sed 's|/|\\|g'
The mouthful of slashes is there to satisfy quoting rules of sed. So, if c:\some\path doesn't exist on your PC, it will try to convert forward to back slashes and replace /c/ with c:\ (or any other drive letter). The only drawback for this is that it won't work correctly non-existing paths that contain a mounted component, such as /bin/does-not-exist or /usr/bin/does-not-exist.
One more approach is to use cygpath from cygwin in MSYS. It seems that cygwin sets global environment variable CYGPATH, that is, you can use it from regular cmd.exe:
%CYGPATH% -w /c/some/path
C:\some\path
or from MSYS:
$CYGPATH -w /c/some/path
C:\some\path
as long as you set to point /c to /cygdrive/c in cygwin.
But this approach will print you /usr located in cygwin installation, not in MSYS.
In short, I think msys should really include real cygpath in the default set of tools just for some cases that aren't handled automatically by msys command line argument conversion logic
use pwd -W
or
download cygpath for msys from here http://mingw.5.n7.nabble.com/enhanced-version-of-cygpath-td28556.html
and use cygpath -wa
Similar to dmitri-rubinstein# above, I've cleaned up the code a bit and added the reverse conversion as well.
winpath() {
if [ ${#} -eq 0 ]; then
: skip
elif [ -f "$1" ]; then
local dirname=$(dirname "$1")
local basename=$(basename "$1")
echo "$(cd "$dirname" && pwd -W)/$basename" \
| sed \
-e 's|/|\\|g';
elif [ -d "$1" ]; then
echo "$(cd "$1" && pwd -W)" \
| sed \
-e 's|/|\\|g';
else
echo "$1" \
| sed \
-e 's|^/\(.\)/|\1:\\|g' \
-e 's|/|\\|g'
fi
}
unixpath() {
echo "$1" \
| sed -r \
-e 's/\\/\//g' \
-e 's/^([^:]+):/\/\1/'
}
I am using this with msysgit:
winpath() {
if [ -z "$1" ]; then
echo "$#"
else
if [ -f "$1" ]; then
local dir=$(dirname "$1")
local fn=$(basename "$1")
echo "$(cd "$dir"; echo "$(pwd -W)/$fn")" | sed 's|/|\\|g';
else
if [ -d "$1" ]; then
echo "$(cd "$1"; pwd -W)" | sed 's|/|\\|g';
else
echo "$1" | sed 's|^/\(.\)/|\1:\\|g; s|/|\\|g';
fi
fi
fi
}
My bash foo is weak and I couldn't get regexes working in bash 3.1 so I hacked out a perl script for it:
#!/bin/env perl
use strict;
my #r;
foreach my $e (#ARGV) {
$e=~s/\//\\/g;
$e=~s/^\\([A-Za-z])\\/\1:\\/;
push #r, $e;
}
print join(" ", #r);
MSYS cygpath
Program
This program convert a DOS path to a UNIX path and vice versa
#!/bin/env perl
# DOS to UNIX path conversion
# © John S. Peterson. License GNU GPL 3.
use strict;
use Getopt::Std;
# usage
if ($#ARGV == -1) {
print 'Usage: cygpath (-w) NAME...
Convert Unix and Windows format paths
Output type options:
-w, --windows print Windows form of NAMEs (C:\WINNT)
';
exit 0;
}
# option
my %opt;
getopts('w', \%opt);
# convert path
my #r;
foreach my $e (#ARGV) {
if ($opt{w}) {
# add drive letter suffix
$e =~ s,^\/([A-Za-z])\/,\1:\/,;
$e =~ s,\/,\\,g;
} else {
$e =~ s,\\,\/,g;
# add leading slash
$e = "/$e";
# remove drive letter suffix
$e =~ s,:,,;
}
push #r, $e;
}
print join("\n", #r);
Compared to Cygwin cygpath
The output from this program is better than the output from Cygwin cygpath in MSYS because
Cygwin cygpath remove the Cygwin home from a converted path, f.e.
cygpath "$CYGWIN/usr/local/bin"
/usr/local/bin
which is a problem because
it's sometimes useful to convert a DOS Cygwin path to a UNIX path for the purpose of copying files from Cygwin to MSYS
This program doesn't remove the Cygwin home
cygpath "$CYGWIN/usr/local/bin"
/c/file/program/cygwin/usr/local/bin
Compared to automatic MSYS path conversion
Manual path conversion has a use in MSYS because
the automatic path conversion is inadequate
for f.e.
devkitPro make
How about this one ?
cmd //c echo <your path>
It may not work always but it is the shortest I found
This works for me
df "$1" | tac >k
read b <k
rm k
set "$1" $b
echo ${1/$7/$2/}
ref
nearly pure GNU bash solution (which is what you commonly run in MSYS) (interestingly not working with MSYS2, leave a comment if you know why):
#!/bin/bash
# check if we have cygpath (cygwin, newer MSYS2), then just use that
which cygpath 1>/dev/null 2>&1
[[ $? = 0 ]] && (cygpath -wa "$1"; exit $?)
# check if it looks like a Windows path, in which case we directly convert and exit
[[ ${1} =~ ^[a-zA-Z]: ]] && \
echo "${1}" | sed -e 's|/|\\|g' -e 's/\(.\)/\u\1/' && exit 0
# split first path entry (if any) with trailing slash and filename
[[ ${1} =~ ^\([/a-zA-Z0-9_.-]\\w*/\)?\(.*\)$ ]]
chk_root="${BASH_REMATCH[1]}"
chk_rest="${BASH_REMATCH[2]}"
# check if the root path exists and more important: let pwd binary resolve the translation according to the mount
chk_winroot="$(cd "${chk_root}." 2>/dev/null && pwd -W)"
[[ "${chk_winroot}" == "" ]] && echo "${chk_root}: No such file or directory" && exit 1
# using substition to replace all / by \ and uppercasing the first character
# pure bash solution; sadly: the first part needs a newer bash than old MSYS have ...
# chk_drv="${chk_winroot:0:1}"
# chk_all="${chk_winroot:1}/${chk_rest}"
# echo "${chk_drv^^}${chk_all//\//\\}"
# ... so fallback to GNU sed
echo "${chk_winroot}/${chk_rest}" | sed -e 's|/|\\|g' -e 's/\(.\)/\u\1/'
There's still an issue with it: if MinGW's fstab contains an entry like /mnt/c the pwd -W of /mnt/. done in this script won't work.
To fix it: replace pwd -W by inspecting $ cat /etc/fstab | cut -d'#' -f1 | grep -v "^\s*$" entries manually and replace the first match - while this will never work for cygwin or msys2 which use a different format this is covered by using cygpath there.