My WORKSPACE file is below:
new_local_repository(
name = "opencv",
path = "/usr/local",
build_file = "opencv.BUILD",
)
new_local_repository(
name = "pcl",
path = "/usr/local",
build_file = "pcl.BUILD"
)
my pcl.BUILD file is below:
cc_library(
name = "pcl",
srcs = glob(["lib/libopencv_*.so*"]),
hdrs = glob(["include/pcl-1.7/pcl/**/*.hpp", "include/pcl-1.7/pcl/**/*.h"]),
includes = ["include"],
visibility = ["//visibility:public"],
linkstatic = 1,
)
my code is very sample, just like below:
#include <pcl/common/common_headers.h>
#include <pcl/visualization/pcl_visualizer.h>
int main() {
std::cout << "hello pcl" << std::endl;
return 0;
}
and i run bazel build command, using bazel query //... and bazel build target.
now I get the error like "fatal error: pcl/common/common_headers.h: No such file or directory". Why?
Someone can give suggestion for this error?
Normally in the #include you must use a path relative to the repository's root:
#include "include/pcl-1.7/pcl/common/common_headers.h"
In order to be able to include like so:
#include "pcl/common/common_headers.h"
...I believe you need to change cc_library.includes to this:
cc_library(
name = "pcl",
...
includes = ["include/pcl-1.7"],
...
)
Also, I believe you have to use quotes in #include, not angle brackets:
#include "pcl/common/common_headers.h"
Related
I have some html files in "root/html_files/*.html" directory. I want to iterate on these html files and run some bazel rules on them, from "root/tests/" directory.
So, I made a filegroup of those html files and tried to get access to them in "root/tests/" directory, but that is't working.
I wonder if it is possible?
My BUILD file in "root/" directory:
HTMLS_LIST = glob(["html_files/*.html",])
filegroup(
name = "html_files",
srcs = HTMLS_LIST,
visibility = [ "//visibility:public" ],)
My BUILD file in "root/tests/" directory:
load("//tests:automation_test.bzl", "make_and_run_html_tests")
make_and_run_html_tests(
name = 'test_all_htmls',
srcs = ['test/automation_test.py'],
html_files = '//:html_files')
My bzl file in "root/tests/" directory:
def make_and_run_html_tests(name, html_files, srcs):
tests = []
for html_file in html_files: # I want to iterate on sources of html filegroup here
folders = html_file.split("/")
filename = folders[-1].split(".")[0]
test_rule_name = 'test_' + filename + '_file'
native.py_test(
name = test_rule_name,
srcs = srcs,
main = srcs[0],
data = [
html_file,
],
args = [html_file],
)
testname = ":" + test_rule_name
tests.append(testname)
native.test_suite(
name = name,
tests = tests,
)
And my python unittest file in "root/tests/" directory:
import sys
import codecs
import unittest
class TestHtmlDocumetns(unittest.TestCase):
def setUp(self):
self.html_file_path = sys.argv[1]
def test_html_file(self):
fid = codecs.open(self.html_file_path, 'r')
print(fid.read())
self.assertTrue(fid)
if __name__ == '__main__':
unittest.main(argv=[sys.argv[1]])
You can't access the references to the files inside another filegroup / rule from within a macro like that. You'd need to create a rule and access them via the ctx.files attr
However, you can iterate over them if you were to remove the filegroup, and pass the glob directly to the macro:
HTMLS_LIST = glob(["html_files/*.html"])
make_and_run_html_tests(
name = 'test_all_htmls',
srcs = ['test/automation_test.py'],
html_files = HTMLS_LIST
)
The glob is resolved to an array before expanding the macro
If I have a header-only library (Eigen in my case) like this:
cc_library(
name = "eigen",
hdrs = glob(
["Eigen/**"],
exclude = [
"Eigen/src/OrderingMethods/Amd.h",
"Eigen/src/SparseCholesky/**",
"Eigen/Eigen",
"Eigen/IterativeLinearSolvers",
"Eigen/MetisSupport",
"Eigen/Sparse",
"Eigen/SparseCholesky",
"Eigen/SparseLU",
],
),
defines = [
"EIGEN_MPL_ONLY",
"EIGEN_NO_DEBUG",
"EIGEN_DONT_PARALLELIZE",
"EIGEN_UNALIGNED_VECTORIZE=0",
"EIGEN_MAX_ALIGN_BYTES=32",
"EIGEN_MAX_STATIC_ALIGN_BYTES=32",
"EIGEN_NO_AUTOMATIC_RESIZING"
],
)
Are the defines observed at all, or do I have to manually add these defines to each dependend library?
additionally, say I have two libraries in my project that depend on Eigen, and I want to compile one with AVX512 and the other without. How can I hide the respective compiled eigen symbols from each other, so that it does not reuse the ones compiled with different defines?
A target that depedens on eigen will also inherit the defines (e.g. EIGEN_MPL_ONLY).
See for details here:
Each string, ..., is
prepended with -D and added to the compile command line to this
target, as well as to every rule that depends on it
A can also setup a small testbed to check how defines work:
WORKSPACE.bazel
# Empty
main/BUILD.bazel
cc_binary(
name = "main",
srcs = ["main.cpp"],
deps = ["//my_header_only_lib:my_header_only_lib"]
)
main/main.cpp
#include "header.h"
#include <iostream>
int main() {
Vector3<float> vec;
vec.print();
#ifdef FOO_BAR
std::cout << "FOO_BAR is also known in main" << std::endl;
#endif
}
my_header_only_lib/BUILD.bazel
package(
default_visibility = ["//visibility:public"],
)
cc_library(
name = "my_header_only_lib",
hdrs = ["header.h"],
includes=["."],
defines = ["FOO_BAR"],
)
my_header_only_lib/my_header_only_lib.h
#pragma once
#include <iostream>
template<typename ScalarType>
class Vector3 {
public:
ScalarType x, y, z;
void print() {
#ifdef FOO_BAR
std::cout << "FOO_BAR defined" << std::endl;
#endif
}
};
Output
FOO_BAR defined
FOO_BAR is also known in main
I'm experimenting with using libhttpserver and, by extension, libmicrohttpd from inside a Bazel build. The build process for these libraries seems to go:
./bootstrap
mkdir build
cd build
../configure
make
which is a slight variation, that I haven't seen before, on the more classic configure && make workflow.
Has anyone managed to make such a library work under Bazel?
Does anyone have a public example I can crib from?
The closest thing I've found to supporting this is #rules_foreign_cc//tools/build_defs:configure.bzl#configure_make but that seems to have no concept of the bootstrap step. Even hacking it doesn't seem to work as the bootstrap script ends up failing with:
mkdir: cannot create directory 'tmpwrk23': Read-only file system
autopoint: *** cannot create directory tmpwrk23
autopoint: *** Stop.
autoreconf: autopoint failed with exit status: 1
I'm about ready to just reach for a genrule() but that seems very error prone...
I went down the same path as you and was able to get libhttpserver compiling with bazel using the rules_foreign_cc project after coming across the 6/23/20 update on this blog post. I added code below, but in general rules_foreign_cc has a make rule, which you can set override make_commands and call ./bootstrap.
WORKSPACE:
...
http_archive(
name = "rules_foreign_cc",
strip_prefix = "rules_foreign_cc-master",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/master.zip",
)
load("#rules_foreign_cc//:workspace_definitions.bzl", "rules_foreign_cc_dependencies")
rules_foreign_cc_dependencies(register_default_tools = True)
all_content = """filegroup(name = "all", srcs = glob(["**"]), visibility = ["//visibility:public"])"""
http_archive(
name = "rules_cc",
urls = ["https://github.com/bazelbuild/rules_cc/archive/262ebec3c2296296526740db4aefce68c80de7fa.zip"],
strip_prefix = "rules_cc-262ebec3c2296296526740db4aefce68c80de7fa",
)
http_archive(
name = "libgnutls",
build_file_content = all_content,
strip_prefix = "gnutls-3.6.15",
urls = ["https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.15.tar.xz"],
)
http_archive(
name = "libhttpserver",
build_file_content = all_content,
strip_prefix = "libhttpserver-master",
urls = ["https://github.com/etr/libhttpserver/archive/master.zip"],
)
http_archive(
name = "libmicrohttpd",
build_file_content = all_content,
strip_prefix = "libmicrohttpd-0.9.71",
urls = ["https://ftp.gnu.org/gnu/libmicrohttpd/libmicrohttpd-0.9.71.tar.gz"],
)
BUILD:
load("#rules_foreign_cc//tools/build_defs:configure.bzl", "configure_make")
load("#rules_foreign_cc//tools/build_defs:make.bzl", "make")
load("#rules_cc//cc:defs.bzl", "cc_binary", "cc_library")
package(default_visibility = ["//visibility:public"])
configure_make(
name = "libgnutls",
lib_source = "#libgnutls//:all",
configure_options = ["--with-included-unistring"],
out_include_dir = "include/gnutls",
shared_libraries = ["libgnutls.so"],
)
configure_make(
name = "libmicrohttpd",
lib_source = "#libmicrohttpd//:all",
deps = [":libgnutls"],
)
make(
name = "libhttpserver",
lib_source = "#libhttpserver//:all",
make_commands = [
"./bootstrap",
"mkdir build_dir",
"cd build_dir",
"../configure --prefix=${INSTALLDIR}",
"make",
"make install",
],
deps = [":libmicrohttpd", ":libgnutls"],
)
cc_binary(
name = "hello-world",
srcs = ["hello_world.cc"],
deps = [
":libhttpserver"
],
)
hello_world.cc (example on libhttpserver github page, run "curl -XGET -v http://localhost:8080/hello" to test)
#include <iostream>
#include <httpserver.hpp>
using namespace std;
using namespace httpserver;
class hello_world_resource : public http_resource {
public:
const std::shared_ptr<http_response> render(const http_request&) {
return std::shared_ptr<http_response>(new string_response("Hello, World!"));
}
};
int main(int argc, char** argv) {
cout << "hello!" << std::endl;
webserver web_server = create_webserver(8080);
hello_world_resource resource;
web_server.register_resource("/hello", &resource);
web_server.start(true);
return 0;
}
I'm trying to build a rule for bazel which emulates the CMake *.in template system.
This has two challenges, the first is generate the template output. The second is make the output available to both genrules, filegroups and cc_* rules. The third is to get that dependency to transitively be passed to further downstream rules.
I have it generating the output file version.hpp in genfiles (or bazel-bin), and I can get the initial library rule to include it, but I can't seem to figure out how to make my cc_binary rule, which depends on the cc_library and transitively the header_template rule to find the header file.
I have the following .bzl rule:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
headers=depset([ctx.outputs.out])
)),
# Also create a provider referencing this header ???
DefaultInfo(files=depset(
[ctx.outputs.out]
))
]
header_template = rule(
implementation = _header_template_impl,
attrs = {
"vars": attr.string_dict(
mandatory = True
),
"extension": attr.string(default=".hpp"),
"template": attr.label(
mandatory = True,
allow_single_file = True,
),
},
outputs = {
"out": "%{name}%{extension}",
},
output_to_genfiles = True,
)
elsewhere I have a cc_library rule:
load("//:tools/header_template.bzl", "header_template")
# version control
BONSAI_MAJOR_VERSION = '2'
BONSAI_MINOR_VERSION = '0'
BONSAI_PATCH_VERSION = '9'
BONSAI_VERSION = \
BONSAI_MAJOR_VERSION + '.' + \
BONSAI_MINOR_VERSION + '.' + \
BONSAI_PATCH_VERSION
header_template(
name = "bonsai_version",
extension = ".hpp",
template = "version.hpp.in",
vars = {
"#BONSAI_MAJOR_VERSION#": BONSAI_MAJOR_VERSION,
"#BONSAI_MINOR_VERSION#": BONSAI_MINOR_VERSION,
"#BONSAI_PATCH_VERSION#": BONSAI_PATCH_VERSION,
"#BONSAI_VERSION#": BONSAI_VERSION,
},
)
# ...
private = glob([
"src/**/*.hpp",
"src/**/*.cpp",
"proto/**/*.hpp",
])
public = glob([
"include/*.hpp",
":bonsai_version",
])
cc_library(
# target name matches directory name so you can call:
# bazel build .
name = "bonsai",
srcs = private,
hdrs = public,
# public headers
includes = [
"include",
],
# ...
deps = [
":bonsai_version",
# ...
],
# ...
)
When I build, my source files need to be able to:
#include "bonsai_version.hpp"
I think the answer involves CcInfo but I'm grasping in the dark as to how it should be constructed.
I've already tried add "-I$(GENDIR)/" + package_name() to the copts, to no avail. The generated header still isn't available.
My expectation is that I should be able to return some kind of Info object that would allow me to add the dependency in srcs. Maybe it should be a DefaultInfo.
I've dug through the bazel rules examples and the source, but I'm missing something fundamental, and I can't find documentation that discuss this particular.
I'd like to be able to do the following:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.src", ":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
and include the generated header like so:
#include "some_header.hpp"
void bar(){
}
The answer looks like it is:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
# pass out the include path for finding this header
includes=depset([ctx.outputs.out.dirname]),
# and the actual header here.
headers=depset([ctx.outputs.out])
))
]
elsewhere:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.cpp"],
deps = [":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
If your header has a generic name (eg config.h) and you want it to be private (ie srcs instead of hdrs), you might need a different approach. I've seen this problem for gflags, which "leaked" config.h and affected libraries that depended on it (issue).
Of course, in both cases, the easiest solution is to generate and commit header files for the platforms you target.
Alternatively, you can set copts for the cc_library rule that uses the generated private header:
cc_library(
name = "foo",
srcs = ["foo.cpp", "some_header.hpp"],
copts = ["-I$(GENDIR)/my/package/name"],
...
)
If you want this to work when your repository is included as an external repository, you have a bit more work cut out for you due to bazel issue #4463.
PS. You might want to see if cc_fix_config from https://github.com/antonovvk/bazel_rules works for you. It's just a wrapper around perl but I found it useful.
In the Bazel official documentation there is an example explaining how to create a Java library built from regular java files and files generated by a :gen_java_srcs rule. I rewrite this code here for ease of reading:
java_library(
name = "mylib",
srcs = glob(["*.java"]) + [":gen_java_srcs"],
deps = "...",
)
genrule(
name = "gen_java_srcs",
outs = [
"Foo.java",
"Bar.java",
],
...
)
Now in a C++ perspective, I am in a scenario where the genrule generates two kind of files: .hpp and .cpp:
genrule(
name = "gen_cpp_srcs",
outs = [
"myFile_1.hpp","myFile_2.hpp",...,"myFile_N.hpp",
"myFile.cpp","myFile_2.cpp",...,"myFile_N.cpp",
],
...
)
where N is some tens.
My problem/question is: how to write the cc_library rule, with an automatic dispatching of the hpp and cpp files into hdrs and srcs field?
I want something like:
cc_library(
name = "mylib",
srcs = glob(["*.cpp"]) + (howto: .cpp files of [":gen_cpp_srcs"]),
hdrs = glob(["*.hpp"]) + (howto: .hpp files of [":gen_cpp_srcs"]),
...
)
Some magic like:
output_filter(":gen_cpp_srcs","*.cpp")
would be perfect, but I do not know enough of Bazel to make it real.
Globs only get expanded when they're passed into rules, so you'll need to write a simple rule. I would package it like this (in a file named filter.bzl):
# The actual rule which does the filtering.
def _do_filter_impl(ctx):
return struct(
files = set([f for f in ctx.files.srcs if f.path.endswith(ctx.attr.suffix)]),
)
_do_filter = rule(
implementation = _do_filter_impl,
attrs = {
"srcs": attr.label_list(
mandatory = True,
allow_files = True,
),
"suffix": attr.string(
mandatory = True,
),
},
)
# A convenient macro to wrap the custom rule and cc_library.
def filtered_cc_library(name, srcs, hdrs, **kwargs):
_do_filter(
name = "%s_hdrs" % name,
visibility = ["//visibility:private"],
srcs = hdrs,
suffix = ".hpp",
)
_do_filter(
name = "%s_srcs" % name,
visibility = ["//visibility:private"],
srcs = srcs,
suffix = ".cpp",
)
native.cc_library(
name = name,
srcs = [ ":%s_srcs" % name ],
hdrs = [ ":%s_hdrs" % name ],
**kwargs
)
This is what my demo BUILD file looks like (I changed the globs so they both include *.cpp and *.hpp files; using the label of a genrule will work the same way):
load("//:filter.bzl", "filtered_cc_library")
filtered_cc_library(
name = "mylib",
srcs = glob(["*.*pp"]),
hdrs = glob(["*.*pp"]),
)
This is easy to extend to more sophisticated filtering by changing _do_filter_impl. In particular, changing suffix to an attr.string_list so you can accept multiple C/C++ source/header extensions seems like a good idea.
Depending on the genrule by name (:gen_cpp_srcs) will give you all of the outputs of the genrule, as you have noted. Instead, you can depend on the individual outputs of the genrule (e.g. hdrs = [:myFile.hpp] and srcs = [:myFile.cpp]).
See also the answer to Bazel & automatically generated cpp / hpp files.
Looks like you know the total number of files that should be generated. Can you put those in their own variables and then reuse them in both targets. Something like this in your BUILD file:
output_cpp_files = [
"myFile_1.cpp",
"myFile_2.cpp",
"myFile_3.cpp"
]
output_hpp_files = [
"myFile_1.hpp",
"myFile_2.hpp",
"myFile_3.hpp"
]
genrule(
name = "gen_cpp_srcs",
outs = output_cpp_files + output_hpp_files,
cmd = """
touch $(OUTS)
"""
)
cc_library(
name = "mylib",
srcs = output_cpp_files,
hdrs = output_hpp_files
)