I have a pre-build PowerShell script defined in my Build Definition. The PowerShell script execute a program:
& "c:\Program Files\nodejs\npm.cmd" "run" "build"
When npm runs, npm writes things to stdout/console. This output doesn't show up in my logs for the build though.
What do I need to do to make sure the build logs capture this console output?
In other words, is there something I can add to & "c:\Program Files\nodejs\npm.cmd" "run" "build" that will pipe the output to the build log?
EDIT:
I think it's one of these three but not sure which:
& "c:\Program Files\nodejs\npm.cmd" "run" "build-release" 2>&1 | Out-Host
& "c:\Program Files\nodejs\npm.cmd" "run" "build-release" 2>&1 | Write-Host
& "c:\Program Files\nodejs\npm.cmd" "run" "build-release" 2>&1 | Write-Output
If you set logging to verbose it will log everything...
I normally add a -Verbose at the end of every line
Related
Here is the folder structure I have.
Workspace folder: D:\Node\MyNode\
When Jenkins build runs on a node, the files from scm gets downloaded to the following folder: D:\Node\MyNode\xx_development
I need to do cd to the folder "xx_development" and this name xx can change for different strasms (RTC) but "_development" remains same.
how can I do cd to a folder with (*development) using a pipeline script?
Edit: I am using windows Nodes for Jenkins.
To change the current directory to the folder with the pattern *_development, you can use the following script:
For Windows:
def folder = bat(returnStdout: true, script: 'dir /b /ad | findstr "_development"').trim()
bat "cd ${folder}"
dir /b /ad | findstr "_development" --> lists all directories in the current folder and filters them by the pattern _development.
/b --> to list only the directory names.
/ad --> to list only directories.
findstr --> to filter the output by the pattern _development.
The second line changes the current directory to the directory stored in the Folder variable.
For Linux:
def Folder = sh(returnStdout: true, script: 'ls -d */ | grep "_development"').trim()
sh "cd ${Folder}"
ls -d */ | grep "_development" --> lists all directories in the folder and filters by the pattern _development.
trim() --> If there are any leading or trailing whitespaces, they are removed using this command.
The second line changes the current directory to the folder stored in the Folder variable.
I've set up a Jenkins build server that's running a nightly build for a Unity project, building two different instances of it. Once these builds are done it runs a job on a different node to copy over the build binaries and run them. What I'm running into is finding a good way for the job to (1) run both executables simultaneously, (2) wait for both of them to finish before moving to the next 'build step' in the job (where it verifies test logs etc).
Initially this seemed to work when I tested it on my own computer: https://stackoverflow.com/a/18762607/14764114
.. but it does not in Jenkins, because the Jenkins node runs as a Windows Service and thus cannot use the START command in Batch.
I'm reading that running separate services might be a solution to explore here, but before I start diving into that I figured I'd ask the community if there isn't a more elegant solution here. In summary, I want to:
Run two executables from a Jenkins build step at the same time (from a Jenkins node running on Windows)
Wait for both executables to exit before continuing to the next build step
In the end I went with this solution, as it seems the Task Scheduler seems to be the only thing capable of starting a Unity game window in my scenario. So I create a task, run it and then delete it, after which I just wait for the processes to disappear from the tasklist:
#echo off
echo "Run FirstApp"
schtasks /create /sc MONTHLY /tn FirstAppTask /tr "%TARGET_DIR%\%APP_First%\FirstApp.exe -automatedtest -duration=%TEST_DURATION_SECONDS%"
schtasks /run /tn FirstAppTask
schtasks /delete /f /tn FirstAppTask
echo "Run SecondApp"
schtasks /create /sc MONTHLY /tn SecondAppTask /tr "%TARGET_DIR%\%APP_Second%\SecondApp.exe -automatedtest -duration=%TEST_DURATION_SECONDS%"
schtasks /run /tn SecondAppTask
schtasks /delete /f /tn SecondAppTask
echo "Wait for FirstApp.exe to end"
:LOOP1
tasklist | find /i "FirstApp" >nul 2>&1
IF ERRORLEVEL 1 (
GOTO CONTINUE1
) ELSE (
ping -n 5 ::1 >NUL
GOTO LOOP1
)
:CONTINUE1
echo "Wait for SecondApp.exe to end"
:LOOP2
tasklist | find /i "SecondApp" >nul 2>&1
IF ERRORLEVEL 1 (
GOTO CONTINUE2
) ELSE (
ping -n 5 ::1 >NUL
GOTO LOOP2
)
:CONTINUE2
echo Done running tests
We have 12 different projects inside the same repository and have a different job to run for each of these.
I want to know how I can trigger a job only when a change has happened in a specific folder, since running all 12 on every push takes too long to finish.
Well I have hacked a solution that works for us.
First, add an Execute Shell Build Step:
#!/bin/bash
export DIRS="api objects"
DIFF=`git diff --name-only develop`
echo "export RUN_TEST=0" > "$WORKSPACE/RUN_TEST"
for DIR in $DIRS; do
for LINE in $DIFF; do
# Is this file inside an interesting directory?
echo $LINE | grep -e "^$DIR/"
# Checking if it is inside
if [ $? -eq 0 ]; then
echo "export RUN_TEST=1" > "$WORKSPACE/RUN_TEST"
fi
done
done
Here:
api and objects are the 2 directories I want to trigger this Job
develop is the main branch we use, so I want to know how my directories compare to that branch in particular
I create a file $WORKSPACE/RUN_TEST to set a variable if I should or not run it
Then in the time-consuming build steps add:
#!/bin/sh
. "$WORKSPACE/RUN_TEST"
if [ $RUN_TEST -eq 1 ]; then
# Time consuming code here
fi
That way the job is triggered but runs as fast as if it wasn't triggered.
Now I modified it to:
#!/bin/bash
export DIRS="api objects"
DIFF=`git diff --name-only origin/develop`
RUN_TEST=111
for DIR in $DIRS; do
for LINE in $DIFF; do
# Is this file inside an interesting directory?
echo $LINE | grep -e "^$DIR/"
# Checking if it is inside
if [ $? -eq 0 ]; then
RUN_TEST=0
fi
done
done
echo "RUN_TEST=$RUN_TEST"
echo "return $RUN_TEST" > "$WORKSPACE/RUN_TEST"
exit $RUN_TEST
And set Exit code to set build unstable to 111 on all build steps. Then, in all following build steps I did:
#!/bin/bash
# Exit on any error
set -euo pipefail
. "$WORKSPACE/RUN_TEST"
# Rest of build step
I want to ask how to find all the executable names of a package in ROS (Robot Operating System)? For example, find spawn_model in gazebo_ros package. When I inspect the package in my system, it just shows some .xml, .cmake files, without any executables. But I can run it, such as: rosrun gazebo_ros spawn_model.
Thank you!
An easy way to do this is to type: "rosrun name_of_package " and then press tab two times, it should show you all the executables built.
After looking in the bash autocompletion script for rosrun, it looks like the command catkin_find is used to find the location of the executables for a package, and the executables are filtered with a find command.
If you want to create a script to give you a list of the executables follow the instructions below:
Save the following script in a file called rospack-list-executables:
#!/bin/bash
if [[ $# -lt 1 ]]; then
echo "usage: $(basename $0) <pkg_name>"
echo ""
echo " To get a list of all package names use the command"
echo " 'rospack list-names'"
exit
fi
pkgname=${1}
pkgdir="$(catkin_find --first-only --without-underlays --libexec ${pkgname})"
if [[ -n "${pkgdir}" ]]; then
find -L "${pkgdir}" -executable -type f ! -regex ".*/[.].*" ! -regex ".*${pkgdir}\/build\/.*" -print0 | tr '\000' '\n' | sed -e "s/.*\/\(.*\)/\1/g" | sort
else
echo "Cannot find executables for package '${pkgname}'." >&2
exit 1
fi
Then make the rospack-list-executables script executable (chmod +x rospack-list-executables) and place it in a directory that can be found in your $PATH environment variable.
Run the script:
$ rospack-list-executables gazebo_ros
debug
gazebo
gdbrun
gzclient
gzserver
libcommon.sh
perf
spawn_model
You should get the same result that you get when you type the rosrun <pkgname> command and press Tab:
$ rosrun gazebo_ros
debug gazebo gdbrun gzclient gzserver libcommon.sh perf spawn_model
You can check the executables for all packages with the following bash code:
rospack list-names | while read pkgname; do
echo "Executables for package '${pkgname}':";
rospack-list-executables $pkgname; echo "";
done
To enable package autocompletion for your newly created command, type the following:
complete -F _roscomplete rospack-list-executables
If you do not want to have to type the complete command every time you login, you can append it to your .bashrc file:
echo "complete -F _roscomplete rospack-list-executables" >> ~/.bashrc
Now when you type the command rospack-list-executables and press the Tab key, you should get a list of all the available packages to choose from.
catkin_find --first-only --without-underlays --libexec <your package name>)
should give you the folder where the executables are
Now I am compiling my MetaTrader .mq4 files to .ex4 files with MetaEditor.
But my .mq4 files are generated by a Java-process, and I would like to automate the compilation process.
Is there a command-line compiler tool I could call programmatically?
To compile a source code file from a command line, you can use MetaEditor for that. For example:
metaeditor.exe /compile:"C:\Program Files\Platform\MQL5\Scripts\myscript.mq5"
For 64-bit use metaeditor64.exe instead.
In Linux/macOS, this can be achieved using Wine, e.g.:
wine metaeditor.exe /compile:"MQL4/Experts/MACD Sample.mq4"
For mass compilation, you can specify folder, like:
metaeditor.exe" /compile:"MQL5\Scripts"
To specify custom MQL5/MQL4 folder with include files, you can use /inc parameter, for example:
metaeditor.exe /compile:"./Scripts" /inc:"C:\Program Files\TradingPlatform 2\MQL5"
For additional information about the compilation process, you can use /log:
metaeditor.exe /compile:"C:\Program Files\Platform\MQL5\Scripts\myscript.mq5" /log
To check for the syntax only, add extra /s.
If the compilation fails, the MQL4.log file would be created in the platform folder with the relevant details. It's going to be in UTF-16 format, so you may need a special tool for it (such as Vim, Ruby, findstr or rg).
To specify the custom compilation log file, use /log:file.log parameter, e.g.
metaeditor.exe /log:errors.log /compile:.
Note: Display to the standard output is not supported (although on Linux you can use: /log:CON).
For more information, check: Compilation from the Command Line
Some time ago you could download the compiler of MQL4/MQL5 programs that runs separately from MetaEditor — MQL.exe. It was distributed separately from the terminal and you could download it at the following addresses:
https://download.mql5.com/cdn/web/metaquotes.software.corp/mt5/mql.exe
https://download.mql5.com/cdn/web/metaquotes.software.corp/mt5/mql64.exe
Usage (as per MQL4/MQL5 Compiler build 1162 from 02 Jul 2015):
mql.exe [<flags>] filename.mq5
/mql5 - compile mql5 source
/mql4 - compile mql4 source
/s - syntax check only
/i:<path> - set working directory
/o - use code optimizer
However the standalone compiler was intentionally removed, so now links point to the installer in favor of MetaEditor.
Much older version of MetaTrader prior to build 600 had metalang.exe included with the platform.
However in build 616, MetaQuotes intentionally has removed the compiler (mql.exe/mql64.exe) from the standard MetaTrader installation.
This means if you upgrade your MT platform (>616), the compiler executable will be removed.
This is a little late, but since I wrote a little script for UltraEdit/UEStudio and have received heaps of help from stackoverflow, here is my script. It compiles then copies the ex4 to a number of test MT4 installations:
The "Compile" button on UE does:
"MT4Compile.bat" "%FilePath" "%FileName"
Start in path eg: D:\Development\MQ4 (Location of MT4Compile.bat)
Normally my source code is in a library tree under D:\Development\MQ4[Group][ExpertName][FileName].mq4
Contents of D:\Development\MQ4\MT4Compile.bat:
#echo off
rem Version: 1.1
rem Date: 24 Sep 2013
rem Author: Shawky
rem Refer to HELP: for info
SET XC=xcopy /D /Y /V /F /I
SET PROGDIR=D:\Development\Go Pro Demo (MQ4 Testing)
SET DSTPATH=%PROGDIR%\experts
SET SIMPATH1=G:\Apps\MT4\BackTest IC (Recent)\experts
SET SIMPATH2=G:\Apps\MT4\BackTest IC (All)\experts
SET SIMPATH3=G:\Apps\MT4\BackTest Go (All)\experts
SET DEPLOYPATH=D:\Development\Deployment\experts
SET SRCPATH=%1
SET SRCPATH=%SRCPATH:"=%
IF "%SRCPATH%"=="" (
SET SRCPATH=[Arg1]
)
SET APPNAME=%2
SET APPNAME=%APPNAME:"=%
IF "%APPNAME%"=="" (
SET APPNAME=[Arg2]
)
SET SRCFILE=%APPNAME%.mq4
SET DSTFILE=%APPNAME%.ex4
SET CMD="%PROGDIR%\metalang.exe" "%SRCFILE%" "%DSTFILE%"
IF "%SRCPATH%"=="[Arg1]" GOTO HELP
IF "%APPNAME%"=="[Arg2]" GOTO HELP
cd %SRCPATH%
IF NOT EXIST "%SRCFILE%" (
SET ERROR=Error: File "%SRCFILE%" does not exist in %SRCPATH%
GOTO HELP
)
echo .
echo Compiling %SRCFILE% to %DSTPATH%\%DSTFILE%
echo .
DEL *.log
%CMD%
IF EXIST "%DSTFILE%" (
echo .
echo Distributing executable to SIM and Deployment paths
%XC% "%DSTFILE%" "%DSTPATH%\"
IF EXIST "%SIMPATH1%" %XC% "%DSTFILE%" "%SIMPATH1%\"
IF EXIST "%SIMPATH2%" %XC% "%DSTFILE%" "%SIMPATH2%\"
IF EXIST "%SIMPATH3%" %XC% "%DSTFILE%" "%SIMPATH3%\"
IF EXIST "%DEPLOYPATH%" copy /B /Y "%DSTFILE%" "%DEPLOYPATH%\%APPNAME% (Dev).ex4"
)
goto END
:HELP
echo . Metatrader 4 Command Line utility for compiling MT4 programmes.
echo .
echo . This batch files allows MT4 applications to be compiled from a directory other than .\experts.
echo . The output will be copied to experts after compilation.
echo .
echo . [Arg1] = Path to MT4 application directory
echo . [Arg2] = Name (without extension) of the main MQ4 source code to compile.
echo .
echo . Example:
echo . MT4Compile.bat "D:\Development\MQ4\MyExpert\" "PrimaryMQ4FileName"
echo .
echo . Programme Directory: %PROGDIR%
echo . Source Path: %SRCPATH%
echo . Source File: %SRCFILE%
echo . Destination File: %DSTFILE%
echo . Target Path: %DSTPATH%
echo .
echo . Argument 1: %SRCPATH%
echo . Argument 2: %APPNAME%
echo .
echo . Commands to execute would be:
echo .
echo . %CMD%
echo . %XC% "%DSTFILE%" "%DSTPATH%\"
echo .
echo . %ERROR%
echo .
pause
:END
All the best.
Shawky
Yes, there is an executable in the install directory of the terminal. It is called metalang.exe.