We have 12 different projects inside the same repository and have a different job to run for each of these.
I want to know how I can trigger a job only when a change has happened in a specific folder, since running all 12 on every push takes too long to finish.
Well I have hacked a solution that works for us.
First, add an Execute Shell Build Step:
#!/bin/bash
export DIRS="api objects"
DIFF=`git diff --name-only develop`
echo "export RUN_TEST=0" > "$WORKSPACE/RUN_TEST"
for DIR in $DIRS; do
for LINE in $DIFF; do
# Is this file inside an interesting directory?
echo $LINE | grep -e "^$DIR/"
# Checking if it is inside
if [ $? -eq 0 ]; then
echo "export RUN_TEST=1" > "$WORKSPACE/RUN_TEST"
fi
done
done
Here:
api and objects are the 2 directories I want to trigger this Job
develop is the main branch we use, so I want to know how my directories compare to that branch in particular
I create a file $WORKSPACE/RUN_TEST to set a variable if I should or not run it
Then in the time-consuming build steps add:
#!/bin/sh
. "$WORKSPACE/RUN_TEST"
if [ $RUN_TEST -eq 1 ]; then
# Time consuming code here
fi
That way the job is triggered but runs as fast as if it wasn't triggered.
Now I modified it to:
#!/bin/bash
export DIRS="api objects"
DIFF=`git diff --name-only origin/develop`
RUN_TEST=111
for DIR in $DIRS; do
for LINE in $DIFF; do
# Is this file inside an interesting directory?
echo $LINE | grep -e "^$DIR/"
# Checking if it is inside
if [ $? -eq 0 ]; then
RUN_TEST=0
fi
done
done
echo "RUN_TEST=$RUN_TEST"
echo "return $RUN_TEST" > "$WORKSPACE/RUN_TEST"
exit $RUN_TEST
And set Exit code to set build unstable to 111 on all build steps. Then, in all following build steps I did:
#!/bin/bash
# Exit on any error
set -euo pipefail
. "$WORKSPACE/RUN_TEST"
# Rest of build step
Related
I have a pretty huge iOS(legacy) project in which I want to add SwiftLint. All the rules I wish to enforce, I would enforce only on new created files. In that way I do not have to go back and fix all the problems that would come up because the rules have not been followed in the past (as there was no SwiftLint in use). I am not sure how to achieve this?
you can run a script like this
# Run SwiftLint
START_DATE=$(date +"%s")
SWIFT_LINT=/usr/local/bin/swiftlint
# Run SwiftLint for given filename
run_swiftlint() {
local filename="${1}"
if [[ "${filename##*.}" == "swift" ]]; then
#${SWIFT_LINT} autocorrect --path "${filename}"
${SWIFT_LINT} lint --path "${filename}"
fi
}
if [[ -e "${SWIFT_LINT}" ]]; then
echo "SwiftLint version: $(${SWIFT_LINT} version)"
# Run for both staged and unstaged files
git diff --name-only | while read filename; do run_swiftlint "${filename}"; done
git diff --cached --name-only | while read filename; do run_swiftlint "${filename}"; done
else
echo "${SWIFT_LINT} is not installed."
exit 0
fi
END_DATE=$(date +"%s")
DIFF=$(($END_DATE - $START_DATE))
echo "SwiftLint took $(($DIFF / 60)) minutes and $(($DIFF % 60)) seconds to complete."
Simply add a build phase run script to Xcode with the following
"${SRCROOT}/Scripts/swiftlint.sh"
It comes from here https://github.com/realm/SwiftLint/issues/413#issuecomment-184077062
It doesn't work on the M1 chip because you need to say something like this
if test -d "/opt/homebrew/bin/"; then
PATH="/opt/homebrew/bin/:${PATH}"
fi
export PATH
if which swiftlint >/dev/null; then
swiftlint
else
echo "warning: SwiftLint not installed, download from https://github.com/realm/SwiftLint"
fine
based on this blog https://www.anotheriosdevblog.com/installing-swiftlint-on-a-m1/
If you are using feature branches you can run it as follows:
swiftlint --config PATH/TO/CONFIG/.swiftlint.yml $(git diff develop --name-only | grep .swift)
I want to ask how to find all the executable names of a package in ROS (Robot Operating System)? For example, find spawn_model in gazebo_ros package. When I inspect the package in my system, it just shows some .xml, .cmake files, without any executables. But I can run it, such as: rosrun gazebo_ros spawn_model.
Thank you!
An easy way to do this is to type: "rosrun name_of_package " and then press tab two times, it should show you all the executables built.
After looking in the bash autocompletion script for rosrun, it looks like the command catkin_find is used to find the location of the executables for a package, and the executables are filtered with a find command.
If you want to create a script to give you a list of the executables follow the instructions below:
Save the following script in a file called rospack-list-executables:
#!/bin/bash
if [[ $# -lt 1 ]]; then
echo "usage: $(basename $0) <pkg_name>"
echo ""
echo " To get a list of all package names use the command"
echo " 'rospack list-names'"
exit
fi
pkgname=${1}
pkgdir="$(catkin_find --first-only --without-underlays --libexec ${pkgname})"
if [[ -n "${pkgdir}" ]]; then
find -L "${pkgdir}" -executable -type f ! -regex ".*/[.].*" ! -regex ".*${pkgdir}\/build\/.*" -print0 | tr '\000' '\n' | sed -e "s/.*\/\(.*\)/\1/g" | sort
else
echo "Cannot find executables for package '${pkgname}'." >&2
exit 1
fi
Then make the rospack-list-executables script executable (chmod +x rospack-list-executables) and place it in a directory that can be found in your $PATH environment variable.
Run the script:
$ rospack-list-executables gazebo_ros
debug
gazebo
gdbrun
gzclient
gzserver
libcommon.sh
perf
spawn_model
You should get the same result that you get when you type the rosrun <pkgname> command and press Tab:
$ rosrun gazebo_ros
debug gazebo gdbrun gzclient gzserver libcommon.sh perf spawn_model
You can check the executables for all packages with the following bash code:
rospack list-names | while read pkgname; do
echo "Executables for package '${pkgname}':";
rospack-list-executables $pkgname; echo "";
done
To enable package autocompletion for your newly created command, type the following:
complete -F _roscomplete rospack-list-executables
If you do not want to have to type the complete command every time you login, you can append it to your .bashrc file:
echo "complete -F _roscomplete rospack-list-executables" >> ~/.bashrc
Now when you type the command rospack-list-executables and press the Tab key, you should get a list of all the available packages to choose from.
catkin_find --first-only --without-underlays --libexec <your package name>)
should give you the folder where the executables are
I'm entirely new to using bash and Xcode build scripts and so my code is probably a jungle full of errors.
The idea here is to trigger the script below which will scrape the directory that it is saved in for any .js automation scripts. It will then send these scripts to instruments to be run one at a time. I found some nifty code that created time stamped files and so I used that to create a more meaningful storage system.
#!/bin/bash
# This script should run all (currently only one) tests, independently from
# where it is called from (terminal, or Xcode Run Script).
# REQUIREMENTS: This script has to be located in the same folder as all the
# UIAutomation tests. Additionally, a *.tracetemplate file has to be present
# in the same folder. This can be created with Instruments (Save as template...)
# The following variables have to be configured:
#EXECUTABLE="Plans.app"
# Find the test folder (this script has to be located in the same folder).
ROOT="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Prepare all the required args for instruments.
TEMPLATE=`find $ROOT -name '*.tracetemplate'`
#EXECUTABLE=`find ~/Library/Application\ Support/iPhone\ Simulator | grep "${EXECUTABLE}$"`
echo "$BUILT_PRODUCTS_DIR"
echo "$PRODUCT_NAME"
EXECUTABLE="${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app/"
SCRIPTS=`find $ROOT -name '*.js'`
# Prepare traces folder
TRACES="${ROOT}/Traces/`date +%Y-%m-%d_%H-%M-%S`"
mkdir -p "$TRACES"
printf "\n" >> "$ROOT/results.log"
echo `date +%Y-%m-%d_%H-%M-%S` >> "$ROOT/results.log"
# Get the name of the user we should use to run Instruments.
# Currently this is done, by getting the owner of the folder containing this script.
USERNAME=`ls -l "${ROOT}/.." | grep \`basename "$ROOT"\` | awk '{print $3}'`
# Bring simulator window to front. Depending on the localization, the name is different.
osascript -e 'try
tell application "iPhone Simulator" to activate
on error
tell application "iOS Simulator" to activate
end try'
# Prepare an Apple Script that promts for the password.
PASS_SCRIPT="tell application \"System Events\"
activate
display dialog \"Password for user $USER:\" default answer \"\" with hidden answer
text returned of the result
end tell"
# Run all the tests.
for SCRIPT in $SCRIPTS; do
echo -e "\nRunning test script $SCRIPT"
TESTC="sudo -u ${USER} xcrun instruments -l -c -t ${TEMPLATE} ${EXECUTABLE} -e UIARESULTSPATH ${TRACES}/${TRACENAME} -e UIASCRIPT ${SCRIPT} >> ${ROOT}/results.log"
#echo "$COMMAND"
echo "Executing command $TESTC" >> "$ROOT/results.log"
echo "here $TESTC" >> "$ROOT/results.log"
OUTPUT=$(TESTC)
echo $OUTPUT >> "$ROOT/results.log"
echo "Finished logging" >> "$ROOT/results.log"
SCRIPTNAME=`basename "$SCRIPT"`
TRACENAME=`echo "$SCRIPTNAME" | sed 's_\.js$_.trace_g'`
for i in $(ls -A1t $PWD | grep -m 1 '.trace')
do
TRACEFILE="$PWD/$i"
done
if [ -e $TRACEFILE ]; then
mv "$TRACEFILE" "${TRACES}/${TRACENAME}"
fi
if [ `grep " Fail: " results.log | wc -l` -gt 0 ]; then
echo "Test ${SCRIPTNAME} failed. See trace for details."
open "${TRACES}/${TRACENAME}"
exit 1
break
fi
done
rm results.log
A good portion of this was taken from another Stack Overflow answer but because of the repository setup that I'm working with I needed to keep the paths abstract and separate from the root folder of the script. Everything seems to work (although probably not incredibly efficiently) except for the actual xcrun command to launch instruments.
TESTC="sudo -u ${USER} xcrun instruments -l -c -t ${TEMPLATE} ${EXECUTABLE} -e UIARESULTSPATH ${TRACES}/${TRACENAME} -e UIASCRIPT ${SCRIPT} >> ${ROOT}/results.log"
echo "Executing command $TESTC" >> "$ROOT/results.log"
OUTPUT=$(TESTC)
This is turned into the following by whatever black magic Bash runs on:
sudo -u Braains xcrun instruments -l -c -t
/Users/Braains/Documents/Automation/AppName/TestCases/UIAutomationTemplate.tracetemplate
/Users/Braains/Library/Developer/Xcode/DerivedData/AppName-
ekqevowxyipndychtscxwgqkaxdk/Build/Products/Debug-iphoneos/AppName.app/ -e UIARESULTSPATH
/Users/Braains/Documents/Automation/AppName/TestCases/Traces/2014-07-17_16-31-49/ -e
UIASCRIPT /Users/Braains/Documents/Automation/AppName/TestCases/Test-Case_1js
(^ Has inserted line breaks for clarity of the question ^)
The resulting error that I am seeing is:
posix spawn failure; aborting launch (binary ==
/Users/Braains/Library/Developer/Xcode/DerivedData/AppName-
ekqevowxyipndychtscxwgqkaxdk/Build/Products/Debug-iphoneos/AppName.app/AppName).
I have looked all over for a solution to this but I can't find anything because Appium has a similar issue. Unfortunately I don't understand the systems well enough to know how to translate the fixes to Appium to my own code but I imagine it's a similar issue.
I do know that the posix spawn failure is related to threading, but I don't know enough about xcrun to say what's causing the threading issue.
Related info:
- I'm building for the simulator but it'd be great to work on real devices too
- I'm using xCode 5.1.1 and iOS Simulator 7.1
- This script is meant to be run as a build post action script in xCode
- I did get it briefly working once before I broke it and couldn't get it back to the working state. So I think that means all of my permissions are set correctly.
UPDATE: So I've gotten to the root of this problem although I have not found a fix yet. First of all I have no idea what xcrun is for and so I dropped it. Then after playing around I found that my Xcode environment variables are returning the wrong path, probably because of some project setting somewhere. If you copy the Bash command from above but replace Debug-iphoneos with Debug-iphonesimulator the script can be run from the command line and will work as expected.
So for anyone who happens across this the only solution I could find was to hardcode the script for the simulator.
I changed EXECUTABLE="${BUILT_PRODUCTS_DIR}/${PRODUCT_NAME}.app/" to be EXECUTABLE="${SYMROOT}/Debug-iphonesimulator/${EXECUTABLE_PATH}". This is obviously not a great solution but it works for now.
I am looking for functionality where we have a directory with some files in it.
Whenever any one makes a change in any of the files in the directory, jenkins shoukd trigger a build.
Is there any plugin or mathod for this functionality. Please advise.
Thanks in advance.
I have not tried it myself, but The FSTrigger plugin seems to do what you want:
FSTrigger provides polling mechanisms to monitor a file system and
trigger a build if a file or a set of files have changed.
If you can monitor the directory with a script, you can trigger the build with a HTTP GET, for example with wget or curl:
wget -O- $JENKINS_URL/job/JOBNAME/build
Although slightly related.. it seems like this issue was about monitoring static files on system.. however there are many version control systems for just this purpose.
I answered this in another post if you're using git to track changes on the files themselves:
#!/bin/bash
set -e
job_name="whatever"
JOB_URL="http://myserver:8080/job/${job_name}/"
FILTER_PATH="path/to/folder/to/monitor"
python_func="import json, sys
obj = json.loads(sys.stdin.read())
ch_list = obj['changeSet']['items']
_list = [ j['affectedPaths'] for j in ch_list ]
for outer in _list:
for inner in outer:
print inner
"
_affected_files=`curl --silent ${JOB_URL}${BUILD_NUMBER}'/api/json' | python -c "$python_func"`
if [ -z "`echo \"$_affected_files\" | grep \"${FILTER_PATH}\"`" ]; then
echo "[INFO] no changes detected in ${FILTER_PATH}"
exit 0
else
echo "[INFO] changed files detected: "
for a_file in `echo "$_affected_files" | grep "${FILTER_PATH}"`; do
echo " $a_file"
done;
fi;
You can add the check directly to the top of the job's exec shell, and it will exit 0 if no changes detected.. Hence, you can always poll the top level of the repo for check-in's to trigger a build. And only complete a build if the files in question change.
I am trying to write a small shell script to find out the signed and unsigned jars in particular directory. While the script works fine until 4-5 jars, it starts showing unpredictable results if my directory has more jars ( currntly about 25 jars). I am not sure but a possible reason for this might be related to buffering of STDIN/STDOUT. I tried to find a resolution for this in related posts but could not get a clear answer.
Here goes my script: ( it takes JAVA_HOME as an argument ):
#! /bin/bash
JV_HOME=$1
for i in `ls *.jar`
do
echo "scanning $i ..."
FILE=$i
$JV_HOME/bin/jarsigner -verify -verbose -certs $FILE | grep "jar verified" ;
if [ $? -eq 0 ]; then
echo "\n$FILE is code-signed\n"
else
echo "\n$FILE is unsigned/unverified..\n"
fi
done
For some of the jars, it says the jar to be unsigned, when they are actually signed when checked individually with the following command:
$JAVA_HOME/bin/jarsigner -verify -verbose -certs
What could possibly be wrong with the above script?
Thanks in advance,
Pabi