Need your kind advise on this. I am very new to Jenkins pipeline and groovy. I have also Googled quite a bit about this specific thing but can't find any definite answers (mostly due to how inexperience I am so I might be missing it).
I have a groovy that is used to run a Jenkins job, at one point I have to run a python script in $WORKSPACE (see below)
stage("Compile Canary") {
steps {
script {
Map bbMap = [
name: env.STAGE_NAME,
commitID: gitInfo.GIT_COMMIT,
url: env.BUILD_URL,
message: "Local Compile"
]
try {
updateBitBucket.inProgress(bbMap)
sh label: '', script: '''
#!/bin/bash
set -e
cd $WORKSPACE/automation/
pip install --upgrade pip
pip install botocore==1.19.50
pip install boto3==1.16.0
python bin/install-dependencies.py
All the pip install / upgrade and boto installs went fine, it's only when running this section:
python bin/install-dependencies.py
that is having issues due to the fact that it cannot find the file (see below for Jenkins output)
+ python bin/install-lambda-dependencies.py
python: can't open file 'bin/install-dependencies.py': [Errno 2] No such file or directory
The error is obvious, which begs the question, how can I import this file? Can I copy it to the git repo and then import it? I'm using this python script because there are quite a long list of dependencies, and also I want to use this script as a standard across my team.
FYI: the python file is currently residing in the same code repository as the groovy file, it is placed in "bin" directory.
Thank you very much!
Related
I'm setting up a CDK project that have some lambdas in Javascript and Python, I'm trying to figure out what's the best way to build these functions as I would normally pass the build command like this:
// Install dependencies, build and run cdk synth
commands: [
'npm ci',
'npm run build',
'npx cdk synth'
]
or
buildCommand: 'npm run build'
The only thing I can think of is to create a build.sh file inside each lambda, for the ones in JS I'd add npm run build and for the ones in Python pip install -r requirements.txt but I don't really know if this is a good practice and if there's a better way to accomplish this.
What you need is a Bundling docker container. You can
either configure the bundling yourself using the bundling option for Code.fromAsset()
or use the PythonFunction and NodejsFunction constructs which provide standard bundling for Python and Node.js, respectively.
This AWS Blog post gives some more examples for bundling.
I would love to be able to customize the UI to fit the guidelines of our brand.
I'm not very familiar with Sass and Angular.
Can you suggest which IDE, tools or plugins are best in this particular project to modify and test the UI without having to build the whole project every time?
Thank you!
thingsboard ui is using angularjs, react, webpack, babel and ...
but all of the ui is connected to all project and i couldn't run it separate and after npm install and npm run build had some problems . based on pom.xml file in ui folder the project first is build with maven and make a jar file in ui folder then run it as front-end module . if we want not to clean install maven every time we change the project, one way is to run commands in pom.xml file ourselves . because the project does not use our global installed npm and node first it installs the particular version of them (nodeVersion: v6.9.1 npmVersion: 3.10.8) locally in ui folder then move them to folder named node and make link to node_modules : ln -s node_modules/npm/bin/npm-cli.js npm after this run the commands:
node/npm install
node/npm run build
node/npm start
so you can write a script file to run all this commands together . don't forget to change your tb-gateway forwardHost and forwardPort in server.js file . the script file based on my os :
#!/usr/bin/env bash
if [ ! -d node-v6.9.1-linux-x64 ] ; then
wget https://nodejs.org/dist/v6.9.1/node-v6.9.1-linux-x64.tar.gz
tar -xzvf node-v6.9.1-linux-x64.tar.gz
mkdir node
mv node-v6.9.1-linux-x64/lib/node_modules node/
mv node-v6.9.1-linux-x64/bin/* node/
cd node
ln -s node_modules/npm/bin/npm-cli.js npm
cd ..
fi
node/npm install
node/npm run build
node/npm start
after all this every ui project build will take time . you can use webstorm IDE or some web ides to open ui folder .
How to run a python script file for ros ?
I have developed python script to make a drone to fly. I have kept the code inside
tumsimulator/src/scripts/DroneFly.py. ## catkin make is done in this directory
When I run the code in tumsimulator directory, it throws an error saying that scripts directory is not found.
I gave chmod 777 permission for directory as well as the file.
Can Somebody help me to run the python script in ros ?
Just to be on the same page
you need to create a catkin workspace a catkin package. For example you create a workspace called catkin_ws
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/src
catkin_init_workspace
Create your package called tumsimulator in ~/home/catkin_ws/src
catkin_create_pkg tumsimulator rospy
Put the scripts directory in tumsimulator/ not in tumsimulator/src. Once after running catkin_make, you should be able to run the script via
Run catkin_make
cd ~/catkin_ws
catkin_make
Finally, run your script
rosrun tumsimulator DroneFly.py
As a side note, scripts should note have the py extension. You can add the python shebang line at the top of your script file #! /usr/bin/env python. If your are writing a python module, you may put these files in tumsimulator/src/tumsimulator/ next to tumsimulator/src/tumsimulator/__init__.py.
I am running Jenkins on Tomcat7 - Windows 7. I have provided the node bin path in my jenkins configuration. Then running a shell script as follows:
echo $PATH
node --version
npm install -g grunt-cli
npm install
grunt cssmin
As suggested in other post Jenkins integration with Grunt, I have restarted my jenkins several times, and tried to work on all the answers written in that post, but still it shows error, grunt: command not found.
Error stack trace from jenkins console output:
/c/apache-maven-3.2.5/bin:/c/Program Files/Java/jdk1.7.0_79/bin:/c/Program Files/nodejs/bin:/c/Program Files/Java/jdk1.7.0_79/bin:/c/Program Files/nodejs/:
+ node --version
v0.10.30
+ npm install -g grunt-cli
C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\grunt -> C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\node_modules\grunt-cli\bin\grunt
grunt-cli#0.1.13 C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\node_modules\grunt-cli
├── resolve#0.3.1
├── nopt#1.0.10 (abbrev#1.0.7)
└── findup-sync#0.1.3 (lodash#2.4.2, glob#3.2.11)
+ npm install
npm WARN package.json Trademust#1.0.0 No repository field.
+ grunt cssmin
C:\Program Files\Apache Software Foundation\Tomcat 7.0\temp\hudson2968878175697925824.sh: line 6: grunt: command not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I have also followed the steps mentioned on this site grunt-on-jenkins
package.json and Gruntfile.js are in root directory the very first time when I executed the jenkins build, grunt installed all modules from my gruntfile.js, and after that in all other build's its showing the above output.
Can anyone please check what's going on wrong here.
After searching a lot, I found where my grunt is installed. As far as jenkins build is concerned it installs in drive:/.jenkins....../workspace/node_modules/.bin.
After providing this path in jenkins using shell script export path=$PATH:drive:/.jenkins....../workspace/node_modules/.bin, grunt started executing.
Also what I learnt in this process was checking where the executable's are available on system path or which path jenkins refers to is using which "executable_name" without qoutes. you can use this command both on windows as well as linux. Ex: which grunt will show the path where grunt executable file is present.
From the error message it seems, sheel is not able to find grunt. Could you please check if it is present in the $PATH variable. On which node this shell script is running? You cab check the $PATH of the particular node. You can also add grunt installation path to $PATH variable during the shell script.
grunt_path="grunt_installtion_path"
export PATH=${PATH}:${grunt_path}
I have a CI setup with Jenkins and Artifactory for Java. I would like also to build and deploy deb packages. For building deb packages, I might use a Maven plugin (called from Gradle), e.g., http://mojo.codehaus.org/deb-maven-plugin/.
I am now investigating Debian repository implementations. I would like to deploy a private Debian repository to host my packages (http://wiki.debian.org/HowToSetupADebianRepository).
Are there any plugin in Jenkins that would make it easier to deploy deb packages? Which debian repository implementation should I use?
Just adding my 2 cents to this post.
Internally we use Freight (https://github.com/rcrowley/freight#readme) as our Debian/Ubuntu repository.
A lot of us tend to use fpm (https://github.com/jordansissel/fpm#readme) by Jordan Sissel for creating debs for internal use.
This can be easily scripted inside your source repository like I do here:
https://github.com/stuart-warren/logit/blob/master/make-deb
#!/bin/bash
# SET SOME VARS
installdir='/usr/lib/logit'
NAME='logit-java'
VERSION='0.5.8'
ITERATION='1'
WEBSITE='https://github.com/stuart-warren/logit'
REPO='http://nexus.stuartwarren.com/nexus'
# REMOVE PREVIOUS BUILD IF PRESENT
echo "Delete ${installdir}"
rm -rf .${installdir}
# CREATE FOLDER STRUCTURE
echo "create base dir ${installdir}"
mkdir -p .${installdir}
# PUT FILES IN THE CORRECT LOCATIONS
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-tomcatvalve.jar -O .${installdir}/logit-${VERSION}-tomcatvalve.jar
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-jar-with-dependencies.jar -O .${installdir}/logit-${VERSION}-jar-with-dependencies.jar
wget https://raw.github.com/stuart-warren/logit/master/LICENSE -O .${installdir}/LICENCE
wget https://raw.github.com/stuart-warren/logit/master/README.md -O .${installdir}/README.md
pushd .${installdir}
ln -sf logit-${VERSION}-tomcatvalve.jar logit-tomcatvalve.jar
ln -sf logit-${VERSION}-jar-with-dependencies.jar logit-jar-with-dependencies.jar
popd
# REMOVE OLD PACKAGES
echo "Delete old packages"
rm ${NAME}_*_all.deb
# CREATE THE DEB
echo "Build new package"
fpm \
-n $NAME \
-v $VERSION \
--iteration ${ITERATION} \
-a all \
-m "Stuart Warren <stuart#stuartwarren.com>" \
--description "Library to extend Log4J 1.2 (plus now Logback 1.0,
Java.util.logging and Tomcat AccessLog Valve) by providing
json layouts (for logstash/greylog) and a zeromq appender" \
--url $WEBSITE \
--license 'Apache License, Version 2.0' \
--vendor 'stuartwarren.com' \
-t deb \
-s dir \
${installdir:1}
echo "Delete ${installdir}"
rm -rf .${installdir}
echo "Done!"
Obviously you could just copy in any compiled files directly rather than downloading from a server, maven repo in my case.
Then you can SCP the deb upto some 'incoming' directory on your repository server.
I am not aware of a Debian package plugin for Jenkins, and I didn't find the maven-deb-plugin suitable for my needs (see "What doesn't work" on page you linked to). Where I have a maven build job in Jenkins I add a post step shell script which increments the version in debian/changelog and runs dpkg-buildpackage -b -nc.
-nc suppresses a clean before build, which is necessary because my debian/rules file will otherwise try to run the maven targets to build jars, which Jenkins has already done. Snippet from my debian/rules:
pre-built-stamp
mvn package
touch pre-built-stamp
override_dh_auto_build: pre-built-stamp
So after the maven steps in Jenkins it runs the following
touch pre-built-stamp
dpkg-buildpackage -b -nc
This part is personal preference, but I do not have Jenkins push built debs straight to my repository. Instead it saves the .deb and .changes files as build artifacts so I can use the Promoted Builds Plugin to sign the .changes file and copy it to the repository (rsync). This lets my developers download and test the deb out before approving it to be pushed to our staging repository. A second promotion can then be used to push the package to a live repository.
I chose reprepro as our repository manager. Its one major drawback is that it cannot handle more than one version of a package in a distribution at once which makes rollback more painful. Aside from this found it reliable and usable, and now use it to completely mirror the main Debian repositories as well as using it to host my private repos.
Reprepro uses inoticoming to spot new incoming packages and verifies the signature on the changes file, ensuring that only Jenkins can add new packages.
I find some of the reprepro documentation online lacking, but I recommend installing it and reading the reprepro and inoticoming man pages.
Debian Package Builder Plugin for Jenkins
Yes there is a plugin that helps with deploying Debian packages into package repositories. The Debian Package Builder Plugin has two features: a build step (which you don't seem to need) and a post-build publishing step. Your target repositories are configured in the system configuration. Just select one of them in the job configuration. The plugin uses dupload(1) "under the hood".
As a repository manager for Debian packages I recommend Aptly. It is powerful, easy to use, well-documented and actively developed.