The bazel build is 90gb. I just want to use the commercial solvers. Is there any way to reduce build size?
Check https://drake.mit.edu/from_source.html near "Building the Python Bindings". The section title is misleading: it works for building all of Drake, not just the Python bindings.
The docs should do a better job of showing the GUROBI settings, though.
Here's a working example:
git clone https://github.com/RobotLocomotion/drake.git
mkdir drake-build
cd drake-build
env GUROBI_HOME=/path/to/Downloads/gurobi951/linux64 \
cmake -DWITH_GUROBI=ON \
../drake
make install
The install ends up in drake-build/install by default.
If you'd like to build Drake as an external in your existing CMakeLists project, see also drake-external-examples/drake_cmake_external.
Related
I have made a ROS workspace and inside a package.
I did catkin_make and everything is working well.
I would like to give this package (or should I give the entire workspace?) to another person.
I am thinking to give him a zip file of the files and folders (it contains launch files, python scripts, rviz files etc) so I am expecting he will unzip it in his machine
I would like he can run the launch files without problems
What is what he needs to do for this? (of course he will have ROS installed, that is no problem)
I am thinking perhaps he should do source devel/setup.bash but is this enough?
When sharing a workspace with somebody only the source space src has to be shared. It should contain all our packages with their launch files (*.launch), Python (*.py) and C++ nodes (*.cpp, *.hpp), YAML configuration files (*.yaml), RViz configurations (*.rviz), robot descriptions (*.urdf, *.xacro) and describe how each node should be compiled in a CMakeLists.txt. Additionally you are supposed to keep track of all the Debian packages you install inside the package.xml file of each package.
If for some obscure reason there are things that I have to do that can't be accommodated in the standard installation instructions given above, I will actually write a bash script that performs these steps for me and add it either to the package itself or the workspace. This way I can automate also more complex steps such as installing OpenCV or modifying the .bashrc. Here a small example of what such a minimal script (I generally name them install_dependencies.sh) might look like:
#!/bin/bash
# Get current workspace
WS_DIR="$(dirname "$(dirname "$(readlink -fm "$0")")")"
# Check if script is run as root '$ sudo ...'
if ["$EUID" -ne 0]
then
echo "Error: This script has to be run as root '$ sudo ./install_dependencies.sh'
exit 1
fi
echo "Installing dependencies..."
# Modify .bashrc
echo "- Modifying '~/.bashrc'..."
echo "source ${WS_DIR}/devel/setup.bash" >> ~/.bashrc
echo ""
echo "Dependencies installed."
If for some reason even that is not possible I make always sure to document it properly either in a Markdown *.md read-me either in a /doc folder inside your package, in the read-me.md inside the base folder of your repository or inside the root folder of your workspace.
The receiver then only has to
Create a new workspace
Copy or clone the package files to its src folder
Install all the Debian package dependencies listed in the package.xml files with $ rosdep install
(If any: Execute the bash scripts I created by hand $ sudo ./install_dependencies.sh or perform the steps given in the documentation)
Build the workspace with $ catkin_make or $ catkin build from catkin-tools
Source the current environment variables with $ source devel/setup.bash
Make sure that the Python nodes are executable either by $ chmod +x <filename> or right-clicking the corresponding Python nodes (located in src or scripts of your package), selecting Properties/Permissions and enabling Allow executing file as program.
Run the desired Python or C++ nodes ($ rosrun <package_name> <executable_name>) and launch files ($ roslaunch <package_name> <launch_file_name>)
It is up to you to share the code as a compressed file, in form of a Git repository or a more advanced way (see below) but I will introduce some best practices in the following paragraphs that will pay off in the long run with larger packages.
Sharing a package or sharing a workspace?
One can either share a single package or an entire workspace. I personally think that most of the time one should share the entire workspace instead of the package alone even if you only cloned the other packages from a public Github repo. This might save the receiver a lot of headache e.g. when checking out the wrong branch.
Version control with Git
Arguably the best way to arrange your packages is by using Git. I'd actually make a repository for every package you create (if a couple of packages are virtually inseparable you can also bundled them to a single Git repo or better use metapackages). Then create an additional repository for your workspace and include your own packages and packages from other sources as submodules. This allows your code to be modular and re-usable: You can share only a package or the entire workspace!
As a first step I always add a .gitignore file to each package repository which excludes *.pyc files and another one to the workspace repository that ignores the build, devel and install folders.
You can add a particular repository as submodule to your workspace Git repository by opening a console inside the src folder of your workspace repository and typing
$ git submodule add -b <branch_name> <git_url_to_package> <optional_directory_rename>
Note that you can actually track a particular branch of a repository that you include as a submodule. In case you need a submodule at some point follow this guide.
If you share the workspace repository with someone they will have to have access to each individual submodule repository and they will have to not only pull the repository but also update the submodules with
$ git clone --recurse-submodules <git_url_to_workspace_repository>
and potentially update them to the latest commit with
$ git submodule update --remote
After these two steps they should have a full version of the repository with submodules and they should be able to progress with the steps listed in the section above.
1.1 Unit-testing and continuous integration
Before sharing a repository you will have to verify that everything is working correctly. This can take a decent amount of time, in particular if the code base is large and you are modifying it frequently. In the ideal case you would have to install it on a brand new machine or inside a virtual box in order to make sure that the set-up works which would take quite some time. This is where unit testing comes into play: For every class and function you program you will write a test. This way you can simply run these tests and make sure everything is working correctly. Generally these unit tests will be performed automatically and the working branches merged continuously. Generally the test routines are written with the libraries Boost::Test (C++), GoogleTest (generally used in ROS with C++), unittest (for Python) and QtTest (for GUIs). For ROS launch files there is additionally rostest. How this can be done in ROS is described here and here.
ROSjects
If you do not even want the person you are sending the code to to go through the hassle to set it up you might consider sending them a ROSject. A ROSject is an online virtual ROS environment (by the guys behind The Construct, the main source of ROS courses and of ROS tutorials on Youtube) that can be created and shared very easily from your existing Git repository as can be seen here. The simulation runs entirely in the cloud on a virtual machine. This way the potential of failure is very low but it is not a choice if your code is supposed to run on hardware and not only in simulation.
Docker
If your installation procedure is complex you might as well use a container such as a Docker.
More information about using Docker in combination with ROS can be found here. The Docker container might introduce though a bit of overhead and it is probably no choice for code which should have real-time priority in combination with a real-time patched operating system.
Debian or snap package
Another way of sending somebody a ROS package is by packing it into a Debian or snap package. This process takes a while and is in particular favourable if you want to give your code to a large number of users that should use the code out of the box. Instructions on how this can be done for Debian packages can be found here and here, while a guide for snap can be found here.
[91mERROR: unsatisfiable constraints:
icu-libs-64.2-r0:
conflicts: icu-libs-62.1-r0
satisfies: xerces-c-3.2.2-r2[so:libicuuc.so.64]
icu-libs-62.1-r0:
conflicts: icu-libs-64.2-r0
satisfies: .postgresql-rundeps-0[so:libicui18n.so.62]
.postgresql-rundeps-0[so:libicuuc.so.62
My build has started failing after a year of no issue due to this conflict with icu-libs after it updated on alpine's community channel. The only APK repos I have added are
--repository http://dl-cdn.alpinelinux.org/alpine/edge/main \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/community \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
I would revert to 3.9 but libgeos, proj, and gdal are only available on edge, which I need for this build.
Sorry if this question is naive but there any way to have apk add both versions? I've been trying to pick apart the mdillon/postgis image that's available on dockerhub since I can't cache unofficial images but it won't build and results in the same error no matter what.
I've been upping postgres versions and no dice.
For some reason alpine decided to explicitly add the icu package version to the icu-depends-dev, instead of just saying depends on icu package.
From the git repo for icu:
depends_dev="$pkgname=$pkgver-r$pkgrel"
That means any package you are running in your build environment
that required icu-dev to build, now in turn requires a specific version of the icu package.
The only way to get around is to remove any packages in your build environment that required icu-dev to build.
At the end of the day I am not sure why they would not just have done
depends_dev="$pkgname"
If I want to install code from a release version in Github in Docker, how can I do that taking up the least space possible in the image? Currently, I've done something like:
RUN wget https://github.com/some/repo/archive/v1.5.1.tar.gz¬
RUN tar -xvzf v1.5.1.tar.gz¬
WORKDIR /unzipped-1.5.1/¬
RUN make; make install
Issue here is the final image will have the downloaded tar, the unzipped version, and everything that gets created during make. I don't need the vast majority of this. How do I install my library in my image without keeping all of this extra data?
This is the textbook definition of the problem that the docker multi-stage build aims to solve.
The idea is to use a separate build with the dependencies and use that docker image to build the final product.
Note that this is available only in the new versions of Docker (17.05 onwards).
I'm pretty new to Cloudbees and Jenkins, and need to get a gradle build to have access to a compiled version of protoc from Google's protobuf project to generate some necessary Java code for a project.
My best guess offhand would be that I would have to add either a build with an output artifact to build it for the Jenkins server and pull it in with the other project... or add it as a build target for the dependent project... though that seems pretty messy. Is there a better/right way to do what I want?
Turns out it was not too hard to add another Jenkins build, export the protoc binary as an artifact, and import the protoc artifact into the required build.
Build-step:
wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2
tar xjvf protobuf-2.5.0.tar.bz2
cd protobuf-2.5.0
./configure --disable-shared
make -j4
strip src/protoc
Leaving this as inspiration, though hopefully someone else comes up with a cleaner solution that I don't know have any clue exists.
I have a CI setup with Jenkins and Artifactory for Java. I would like also to build and deploy deb packages. For building deb packages, I might use a Maven plugin (called from Gradle), e.g., http://mojo.codehaus.org/deb-maven-plugin/.
I am now investigating Debian repository implementations. I would like to deploy a private Debian repository to host my packages (http://wiki.debian.org/HowToSetupADebianRepository).
Are there any plugin in Jenkins that would make it easier to deploy deb packages? Which debian repository implementation should I use?
Just adding my 2 cents to this post.
Internally we use Freight (https://github.com/rcrowley/freight#readme) as our Debian/Ubuntu repository.
A lot of us tend to use fpm (https://github.com/jordansissel/fpm#readme) by Jordan Sissel for creating debs for internal use.
This can be easily scripted inside your source repository like I do here:
https://github.com/stuart-warren/logit/blob/master/make-deb
#!/bin/bash
# SET SOME VARS
installdir='/usr/lib/logit'
NAME='logit-java'
VERSION='0.5.8'
ITERATION='1'
WEBSITE='https://github.com/stuart-warren/logit'
REPO='http://nexus.stuartwarren.com/nexus'
# REMOVE PREVIOUS BUILD IF PRESENT
echo "Delete ${installdir}"
rm -rf .${installdir}
# CREATE FOLDER STRUCTURE
echo "create base dir ${installdir}"
mkdir -p .${installdir}
# PUT FILES IN THE CORRECT LOCATIONS
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-tomcatvalve.jar -O .${installdir}/logit-${VERSION}-tomcatvalve.jar
wget ${REPO}/content/repositories/releases/com/stuartwarren/logit/${VERSION}/logit-${VERSION}-jar-with-dependencies.jar -O .${installdir}/logit-${VERSION}-jar-with-dependencies.jar
wget https://raw.github.com/stuart-warren/logit/master/LICENSE -O .${installdir}/LICENCE
wget https://raw.github.com/stuart-warren/logit/master/README.md -O .${installdir}/README.md
pushd .${installdir}
ln -sf logit-${VERSION}-tomcatvalve.jar logit-tomcatvalve.jar
ln -sf logit-${VERSION}-jar-with-dependencies.jar logit-jar-with-dependencies.jar
popd
# REMOVE OLD PACKAGES
echo "Delete old packages"
rm ${NAME}_*_all.deb
# CREATE THE DEB
echo "Build new package"
fpm \
-n $NAME \
-v $VERSION \
--iteration ${ITERATION} \
-a all \
-m "Stuart Warren <stuart#stuartwarren.com>" \
--description "Library to extend Log4J 1.2 (plus now Logback 1.0,
Java.util.logging and Tomcat AccessLog Valve) by providing
json layouts (for logstash/greylog) and a zeromq appender" \
--url $WEBSITE \
--license 'Apache License, Version 2.0' \
--vendor 'stuartwarren.com' \
-t deb \
-s dir \
${installdir:1}
echo "Delete ${installdir}"
rm -rf .${installdir}
echo "Done!"
Obviously you could just copy in any compiled files directly rather than downloading from a server, maven repo in my case.
Then you can SCP the deb upto some 'incoming' directory on your repository server.
I am not aware of a Debian package plugin for Jenkins, and I didn't find the maven-deb-plugin suitable for my needs (see "What doesn't work" on page you linked to). Where I have a maven build job in Jenkins I add a post step shell script which increments the version in debian/changelog and runs dpkg-buildpackage -b -nc.
-nc suppresses a clean before build, which is necessary because my debian/rules file will otherwise try to run the maven targets to build jars, which Jenkins has already done. Snippet from my debian/rules:
pre-built-stamp
mvn package
touch pre-built-stamp
override_dh_auto_build: pre-built-stamp
So after the maven steps in Jenkins it runs the following
touch pre-built-stamp
dpkg-buildpackage -b -nc
This part is personal preference, but I do not have Jenkins push built debs straight to my repository. Instead it saves the .deb and .changes files as build artifacts so I can use the Promoted Builds Plugin to sign the .changes file and copy it to the repository (rsync). This lets my developers download and test the deb out before approving it to be pushed to our staging repository. A second promotion can then be used to push the package to a live repository.
I chose reprepro as our repository manager. Its one major drawback is that it cannot handle more than one version of a package in a distribution at once which makes rollback more painful. Aside from this found it reliable and usable, and now use it to completely mirror the main Debian repositories as well as using it to host my private repos.
Reprepro uses inoticoming to spot new incoming packages and verifies the signature on the changes file, ensuring that only Jenkins can add new packages.
I find some of the reprepro documentation online lacking, but I recommend installing it and reading the reprepro and inoticoming man pages.
Debian Package Builder Plugin for Jenkins
Yes there is a plugin that helps with deploying Debian packages into package repositories. The Debian Package Builder Plugin has two features: a build step (which you don't seem to need) and a post-build publishing step. Your target repositories are configured in the system configuration. Just select one of them in the job configuration. The plugin uses dupload(1) "under the hood".
As a repository manager for Debian packages I recommend Aptly. It is powerful, easy to use, well-documented and actively developed.