When debugging Bazel BUILD files, it can be useful to know what values the system has on by default or has determined from the host/target setup (eg. debugging select rules).
Is there a way, either through a query or directly in the BUILD file, to view platform and other configuration values (eg. cpu, os, etc)?
Is it also possible to produce a list of all of these variables? With many different configuration options available, it's easy to lose track of what's there and what they may be set to(eg. crosstool_top, host_crosstool_top, etc).
Not a perfect answer, but adding the --toolchain_resolution_debug flag to the command line gives some visibility into a couple of these values.
https://docs.bazel.build/versions/master/command-line-reference.html#flag--toolchain_resolution_debug
Related
I would like to version build artefacts with build number for CI passed to bazel via workspace_status_command. Sometimes I would like to include build number to the name of the artefact.
Is there a way how do I access ctx when writing a macro(as I was trying to use ctx.info_file)? So far it seems that I am able to access such info just in new rule when creating a new rule which in this case is a bit awkward.
I guess that having a build number or similar info is pretty common use case so I wonder if thre is a simpler way how to access such info.
No, you really need to define a custom rule to be able to consume information passed from workspace_status_command through info_file and version_file file and even then you cannot just access it's values from Starlark, you can pass the file to your tooling (wrapper) and process the inputs there. After all, (build) rules do not execute anything, they emit actions to be executed at a later phase.
Be careful though, because if you depend on info_file (STABLE_* entries), changes to the file invalidate targets depending on it. For something like CI build number, it's usually not what you want and version_file is more likely what you are after. You may want to record the id, but you usually do not want to rebuild stuff just because the build ID has changed (it's a new CI run). However, even simple inclusion of IDs could be considered problematic, if you want your results to be completely reproducible.
Having variable artifact names is a whole new problem and there would be good reasons why not to. But generally since as proposed the name would be decided during execution of actions (reading in version_file in your tool), you're past the analysis phase to decide what comes out of the action. The only way I am currently aware of (that is for out of tree source of variable input, you can of course always define a Starlark variable and load it from your BUILD file) to be able to do that is to use tree artifacts (using declare_directory in your rule.
Is there a mechanism to select a default config based on the platform or the environment variable? For example:
common --config=$DEFAULT_CONFIG
I'm working on a project which uses bazelrc configurations to specify default build and test flags for separate platforms. It looks something like this:
build:macos --compiler=clang
...
build:linux --compiler=gcc
...
build:windows --compiler=msvc
...
This would be really useful for a multi-platform CI setup and in general would avoid having to type bazel <command> --config=<some config> <target> every time.
Related question: Default, platform specific, Bazel flags in bazel.rc
There is now! If you add
common --enable_platform_specific_config to your .bazelrc, then --config=windows will automatically apply on windows hosts, --config=macos on mac, --config=linux on linux, etc.
You can then add lines to your .bazelrc like:
build:windows --windows-flags
build:linux --linux-flags
There is one downside, though. This works based on the host rather than the target. So if you're cross-compiling, e.g. to mobile, and want different flags there, you could wrap all the library rules you use in macros, or (probably better) add transitions into your graph targets. (See discussion here and here. "Flagless builds" are still under development, but there are usable hacks in the meantime.) You could also use the temporary platform_mappings API.
References:
Commit that added this functionality.
Where it appears in the Bazel docs.
Yes, that's what toolchains are for.
I am unable to figure out how to change default build/compile settings. The little default checkbox in the lower left of the project options dialog is gone. The documentation states:
The Default checkbox that appeared at the lower edge of many Project Options pages has been removed from the product. If you want to specify options as the default for multiple projects, the suggested alternative is to use option sets instead.
I'm going round and round about "Options Sets", "Configuration Manager" etc.. Is this even possible? What does "specifying options as the default for multiple projects" mean? If I have multiple projects then that means those projects and their options exists, how can I set a default value to something already set? What about new projects?
That feature really has gone and there is nothing like it any more in the product, to the very best of my knowledge. I think the best you can do is as follows:
Create a new project.
Change the project settings to whatever you want them to be.
Change anything else in the default project that you don't like, for example { Private declarations }.
Add this project to the repository.
use File | New | Customize to move this project template onto the File | New menu for easy access.
Project->Options->Target. You can set up a base configuration, and then provide different options that differ from the base for Debug and Release. You can also create custom option sets, which means that they're different from the standard Debug and Release. You can also have different configurations based on different targets (VCL app's Debug build has different options than a FMX app's Debug build, etc.)
To change the default options first starts with defining "default". You can start as low as the "base configuration" through Project->Options->Delphi Compiler, and then choosing the All Configuration target. You can refine it somewhat by altering the base configuration for the Debug and Release configurations. You can also define your own option sets, using the Save button next to the Target list.
Your specific question about "specifying options as the default for multiple projects" means is the base configuration. From there, you refine those base options to give you debug settings and release settings (which can also be saved as your initial defaults, and refined on a per-project basis).
So, for a specific answer, you can change the default by modifying the base configuration, or by getting more specific by modifying the debug or release configurations that inherit from that base, depending on what your end result needs to be and what you're trying to accomplish.
Our team is sharing a Jenkins server with other teams, and this currently means that we are sharing the same OS-level build-user account. The different teams' OS-level build-user settings (Maven settings, bash settings, user-level Ant libraries, etc...) have collided a few times--"fixing" the settings for one team's jobs inadvertently "breaks" another team's jobs. The easiest sol'n that occurs to me is giving each team its own OS-level build-user account with which to execute its Jenkins jobs--but I cannot find a way to do this.
I have checked with Google, and also here
https://wiki.jenkins-ci.org/display/JENKINS/Use+Jenkins
and here
https://wiki.jenkins-ci.org/display/JENKINS/Plugins
to no avail.
Is there a way to do this? If not, can you recommend any best practices for segregating sets of builds from one another?
Maven Specific
You have two options that come to mind,
Add additional installations of Maven into your Jenkins global configuration, each using their own Home directory, and thus settings files. This will allow you to use totally different version of Maven, and selected based on Job requirements (You are given the option to select which "version" of maven you wish to use on the job itself.
Similar to (1), but specify specific settings configurations using Maven command line arguments. Its a little less "obvious" but may be quicker to implement
Multi-slave
You could possibly make use of multiple slaves on each machine. It increases the overheads of the builds quite significantly, and the implementation is such that you'd have multiple user accounts on a machine, each setup as needed, and then one slave instance for each user.
I'm not sure these solutions will totally answer your problem, I'll have a think and see if anything else pops into mind, but it might give some starting points
Key builds to a specific team directory that contains that team's settings. For example, provide a parameter 'TEAM' to every build, set its default value to the appropriate team name, and use that parameter as a key to a directory that contains the team's settings (so instead of using ${HOME} as in what you want to do, you'll use something like ${TEAM_SETTINGS}/${TEAM}).
You can set per-job users (who has access to/can build a particular job).
Under "Manage Jenkins" > "Configure System" >
Click on Enable Security
Check Project-based Matrix Authorization Strategy
However, I do not think there is a "per-build" option for a single job.
If you have the same project that you are sharing between teams, you could (and probably should) create two jobs for this project, and have different libraries/scripts be used in each.
You could also parametrize the build (On the Job Page, "Configure" > This build is parametrized) and supply the library versions, etc via string parameters.
You could also use a parameter to be the team's name, and in your build script change libraries based on the parameter:
For example, have a parameter called "TEAM", with choices: TEAM_A and TEAM_B, and in your script, have
if [ $TEAM == "TEAM_A" ]
then
ANT_HOME=/opt/ant/libA
else
ANT_HOME=/opt/ant/libB
fi
======================================================================
Have you considered sourcing your settings? In Linux, you could do this by saving your OS settings in a script file (for example paths, etc), and using source /path/to/settings/file, in Windows it would be call /path/to/settings/batch/file.
Can you give examples of OS level settings that you would require and per-build user for?
You problem is a common one.
Whenever something nonstandard is installed on a build server, something will break for someone.
The only solutions I know are
Set up a separate build slave for each team or product. Then they can install whatever they want on the build slave and any mess they create is all their own fault.
Any dependencies required by a job need to come with the job. This is my preferred way of working. For example: If a job needs a library or a tool, the library or tool is not installed on the build server but in the source tree and the build uses it from the source tree.
Sometimes the latter way is more work. You need to set up the tools or library so it works when it is installed in the source tree. Some tools have hard-coded paths and they do not work. In that case you can install the source of the tool and compile the tool during the build.
An even better solution is to set up separate Jenkins jobs for all the tools and libraries and the jobs that need a library or tool will download them from the Jenkins jobs.
This way you can control all your dependencies and different jobs do not conflict when e.g. one needs an older version of a library and one a newer version. And if someone upgrades the library, it is immediately visible in the version control who did what.
I have a complex project group that has about 10 "final" build configurations configured in a tree, where each node has its own option set. Something like this:
Base
Release
Release Generic 1
Release Final 1
...
Release final 5
Release Generic 2
Release Final 6
...
Debug
same as release, but for debug
so, all base, Release, Release Generic X, Release Final Y have their own option set saved in files and added as reference. So you will have
base.optset
Release.optset
Release Generic 1.optset
Release Final 1.optset
..
The main differences between these option sets are various compiler defines enabling/disabling certain features that are organized in the tree mentioned above (hence the build targets and the option sets) and obviously search paths that include the needed stuff.
So each option set also contains different search paths that depend on the respective build configuration.
There are over 100 projects in the project group and most of them relate to one another on different level of configurations, so there are also a bunch of Build Groups, but this is irrelevant for the question as right now I'm trying to build the projects one at a time.
The problem is that every option set is configuring the search path. Up to and including "Release Generic 1.optset" the search path is correctly updated and used. However, the "Release Final 1" and siblings don't get the search paths introduced by "Release Final 1.optset". (I know this because I looked at the compiler messages and checked the parameters passed to dcc32) It appears like the option set is ignored.
Everything is configured with "inherit=true" in the option set.
There is nothing specific in the build configurations, nowhere. Every project uses the configuration in the IDE and option set files only.
Is this issue/limitation documented anywhere/known about? Is there a workaround? Other than applying the option set as value and not by reference.
Thanks.
later edit:
I decided to implement a pretty ugly workaround in order to get moving: basically, replicate teh configuration from "Release Generic 1.optset" into each "Release Final ?.optset"
this has the major disadvantage of having to edit 5 option set files propagating the same modification, when in need of adding something to "Release Generic 1.optset"
Finally I managed to figure out the problem. It's actually a bug in the Delphi IDE in that it messes up the imports in the dproj file.
Specifically, if an optionset is added once, it will not be maintained with the necessary import condition for the rest of the configs.
This usually happens when you add/delete a configuration, the dproj gets busted and optset files no longer get imported for all the configs.
But once your configs are stable and you no longer change them, the dproj remains stable.
I mocked up an application to validate dproj files against these issues, verifying that each config has its optset files in an import directive. I'm yet to make it auto-correct this, as I'm rather busy. But once I manage to find the time, or if there are more people affected by it and require an automatic fix, I'll try to make a release.
Until then, if you noticed the option set is not applied for a specific config, simply find the configs id in dproj (something like Cfg_10 ) and see if you can find an
<Import Condition="'$(Cfg_10)'!='' And Exists('optset file path name')" Project="optio file path name"/>
if not, then add it after the other <import calls