As part of our efforts to create a bazel-maven transition interop tool (that creates maven sized jars from more granular sized bazel jars),
we want the aspect that runs on bazel build to access the target's java_common.provider in order to fetch the jars and ijars from it.
Is that possible?
The short answer is yes, that's possible.
You can use the java_common module in an aspect implementation the same way you would use it in a rule implementation.
From the documentation on java_common.provider:
java_common.provider.compile_jars and java_common.provider.transitive_compile_time_jars refer to the ijars used at compilation time
java_common.provider.transitive_runtime_jars refer to the full jars used at runtime.
The full jars at compilation time are not yet available, but someone is working on exposing this feature. (Issue #3528 on GitHub.)
Make sure you also read the blog post on this topic: https://blog.bazel.build/2017/03/07/java-sandwich.html
Related
I would like to use a very large non-bazel system in a bazel project. Specifically, ROS2. This dependency provides a large number of python, C, and C++ libraries which are built using its own hand-rolled buildsystem. Obviously, I would like to avoid having to translate the entire buildsystem over to bazel.
Broadly, what's the best way of me doing this? In instinct was to use a custom repository rule to download the source (since it's split across many repositories), then use a genrule to call the ROS2 build system. Then write my simple cc_import and py_library rules for each of the individual components that I need.
However, I'm having trouble with the bit where I need to call the foreign build system. It seems that genrules require a list of output files to be specified, while I would like it to make an entire build directory available.
Before I spent any more time on this, I thought I'd ask whether I'm on the right lines since I'm new to bazel. Is this a good strategy? How would you approach this problem? Are there any other projects that mainly use bazel, but call other build systems in this way that I can look at?
As of recent, you can use rules_foreign_cc to call native CMake or make/configure like projects.
I want to build a rule that is very similar to cc_proto_library. The key features are that it would apply an aspect to all the transitive proto_library dependencies and generate .cc and .h files for all of the dependencies. In addition it would generate actions that would compile these into object files.
While I understand how I can do the file generation, I don't see how to easily do the object generation. The native module is not available for rule (or aspect) implementations, and I cannot use a macro on top of the aspect as I need the object files to be generated in the same package as the proto_library so that it is generated only once.
cc_proto_library can do this I believe because it is not written in Skylark and thus has access to more primitives. Is there anyway to do this with just Skylark?
This is unfortunately currently not possible. There is no Skylark API to the C++ rules/actions (what we call C++ sandwich). We have plans to implement this in Q1 2018. There are many tracking issues, this one looks the most relevant: https://github.com/bazelbuild/bazel/issues/2163.
I have a large application to manage consisting of of three or four executables and as many as fifty .dlls. Many of the source code files are shared across many of the projects.
The problem is a familiar one to many of us - if I change some source code I want to be able to identify which of the binaries will change and, therefore, what it is appropriate to retest.
A simple approach would be simply to compare file sizes. That is an 80% acceptable solution, but there is at least a theoretical possibility of missing something. Secondly, it gives me very little indication as to WHAT has changed; It would be ideal to get some form of report on this so I can then filter out irrelevant (e.g. dates/versions copyrights etc..)
On the plus side :
all my .dcus are in a row - I mean they are all built into a single folder
the build is controlled by a script (.bat)(easy, for example, to emit .obj files if that helps)
svn makes it easy to collect together any (two) revisions for comparison
On the minus side
There is no policy to include all used units in all projects; some units get included because they are on a search path.
Just knowing that a changed unit is used/compiled by a project is not sufficient proof that the binary is affected.
Before I begin writing some code to solve the problem I would like to ask the panel what suggestions they might have as to how to approach this.
The rules of StackOverflow forbid me to ask for recommended software, but if anyone has any positive experiences of continuous integration tools that would help - great
I am open to any suggestion or observation that is relevant in this context.
It seems to me that your question boils down to knowing which units are contained in your various executables. Since you are using search paths, it will be hard for you to work this out ahead of time. The most robust way to find out is to consult the .map file that the compiler emits. This contains a list of all units contained in your executable.
Once you know which units are contained in each executable, you need to know whether or not anything has changed in those units. That information is contained in your revision control system. Put this all together and you have the information that you need.
Of course, just because the source code for a unit has changed, you might argue that re-testing is not needed. Perhaps the only change made was the version, or the date in a copyright label or some such. But it is asking too much to be able to ask a computer to make such a judgement. At some point you need a human to step up and take responsibility.
What is odd about this though is that you are asking the question at all. It seems to me to be enormously risky to attempt partial testing. I cannot understand why you don't simply retest the entire product.
After using it for > 10 years for commercial in-house and freelancer work in large projects, I can recommend to try Apache Ant. It is a build tool which supports dependencies, and has many very helpful features.
Apache Ant also integrates nicely with CI tools such as Hudson/Jenkins, Bamboo etc.
Another suggestion - based on experience with Maven - is to design the general software architecture as modular as possible. If modules (single or multiple source or DCU files in one directory) use a version number in the directory name as a version number, it is possible to control exactly how application are composed from these modules.
If you want to program such a tool yourself the approach would be something like this:
First you need to detect wheter there were any changes made to seperate source files. As you already figured out comparing the file size is bad idea as the file size can stay the same despite lots of changes made to it (as long as there is same amount of text in pas file its size won't change). So instead you could check the last modification time for specific file or create some hash value like MD5 hash for comparison (can be quite slow).
Then you need to generate yourself a dependancy tree which will tell you which files are used for which project/subproject.
Finally based on changes detected in seperate files you check the dependancy tree to see which projects needs to be recompiled.
The problem of such approach is that you would probably have to update the dependancy tree manually each time when new unit is added to the project or an existing one is removed from the project.
But the best way would be to go and use some version controll software istead of reinventing the wheel. I myself like the way how GIT works and I belive that with proper implementation of GIT into the project mannager itself could be quite powerfull do to GIT support of branching/subbranching (each project is its own branch, each version of your software can be its own subbranch).
Now latest version of Delphi does have GIT integration done though SVN but this unfortunately limits some of best GIT functionality. So if you maybe decide to go and integrate GIT support directly into Delphi I'm first in line to use it.
So I really cant understand how this work but late me explain. First, just in case you need it, I am running Ubuntu 12.04 64-bit on a laptop.
As a building tool am using CMake. I want to load in to my project OpenCV, MRPT (http://www.mrpt.org/) and libfreenect. All of them have a "source code". What I don't understand is when they say "build from source". How to I make a project with all of them?
Do I need to build each one individually and with some way but then in my project OR do I down load the source code and build them all together at ones? As you can see I'm really confused what I have to do... do I run the CMakeList.txt from each source code and the run one CMakeList.txt that has all the other CMakeList.txt?
In fewer world, if I want to build from source, two or more libraries, how do I do that?
I would like a general answer (how this "build from source" works) and an answer specifically on the the ones I mentioned (CMake, OpenCV, MRPT, libfreenect). I hope I made clear what I don't really understand.
It depends of the 'master' project. In general in the c/c++ universe your project must know how to invoke the build process of each subproject/library OR your project needs to know how to include&link the results after building each external project yourself.
You can also mix the two approaches if needed but I think it cleaner to try to use one if possible.
In the first case if all the subprojects offer cmake building files (CMakeLists.txt) you may try to add_subdirectory() each and see if there are any conflicts. For example google test can be easily included this way and it gives your project some global variables that easy linking later.
Alternatively or if the above approach gives problems or the sub project doesn't provide CMakeLists.txt you can use ExternalProject_add(). It takes more work and you have to handle includes/linking configurations with your project manually but it makes the subproject more independent. For example if there are conflicting targets with your project or the subproject doesn't provide CMakeLists.txt.
The last approach involves building and installing the sub projects separately, using configuration variables in your project to point the includes/libraries paths of the sub project. Check CMake:How To Find Libraries for details.
The team I work for manages a large collection of technical documentation which is written in LaTeX.
Currently all the documentation we have is manually built by the editors and then checked into a version control system. Sometimes people forget to compile their documents so we have a situation where the PDF and .tex files are often out of step. Unfortunately when this happens our users find themselves reading old versions of our document.
I've managed to hack a simple script to build PDFs using Make - it's rather clumsy.
I was wondering if there was a better way to do it? Most people in our department use Eclipse + Pydev for a Python project which means we are all very familiar with this IDE. I know that Ant plays nicely with Eclipse, so might we be able to use this tool for our doc building?
So what's the best way of doing this? I hope I will not have to learn everything there is to know about a new build-system in order to automate the building of some quite simple docs.
There is an external Ant task for LaTeX PDF generation, though the site is in German.
To use it, download the jar to a location on your machine, then define a taskdef as follows:
<taskdef name="latex" classname="de.dokutransdata.antlatex.LaTeX"
classpath="/path/to/ant/lib/ant_latex.jar"/>
Then to use it, define a target like this:
<target name="doLaTeX">
<latex
latexfile="${ltx2.file}"
verbose="on"
clean="on"
pdftex="off"
workingDir="${basedir}"
/>
</target>
Where ltx2.file is the file to process.
This is a link to the howto page listing the parameters. If you need any more options, my German is just about passable enough to explain, maybe.
There is also a maven plugin for LaTeX, but I can't find any documentation.
Haven't tried it, but I remember seeing a blog post about it.
If you know python, this blog post might be interesting
EDIT: Also, I would assume that you're using some kind of version control system, and I can't say for sure, but I use git to manage all my latex docs, and it might be possible to use some kind of post-commit hook to execute a script to rebuild the document. This would depend on how your repository is structured... just thinking out loud, so to speak.
I went into great detail on a large number of build systems for latex in this question, but its slightly different in your case. I think you want rubber or latexmk. The latex-makefile seems a good idea, but only supports building via postscript, which might not be your build process.
In general, its a good idea to keep generated files outside of version control for just this reason. A good exception is when specialist build tools are not widely available, and your situation sounds similar. You might do better with a commit-hook to build automatically upon commit.
I guess I should also point out that committing something without first building it and checking it is a deadly sin, so a better solution might be to stamp that out.
Maven is a better alternative as build system compared to Ant. So I would recommend a maven-plugin to generate PDF from LaTeX sources. Have a look at mathan-latex-maven-plugin