I am newbie to bazel. I read through doc and know I can retrieve package via http_archive and use it if it is already bazel project. But when I am trying to build with cppkafka, I am lost and don't know what to do to build it with my project.
You have to bazelize cppkafka and its dependencies (e.g. librdkafka).
librdkafka.BUILD:
cc_library(
name = "librdkafka",
srcs = [
"src/configuration.cpp",
"src/configuration.cpp",
"src/topic_configuration.cpp",
"src/configuration_option.cpp",
"src/exceptions.cpp",
"src/topic.cpp",
"src/buffer.cpp",
"src/queue.cpp",
"src/message.cpp",
"src/message_timestamp.cpp",
"src/message_internal.cpp",
"src/topic_partition.cpp",
"src/topic_partition_list.cpp",
"src/metadata.cpp",
"src/group_information.cpp",
"src/error.cpp",
"src/event.cpp",
"src/kafka_handle_base.cpp",
"src/producer.cpp",
"src/consumer.cpp",
"src/utils/backoff_performer.cpp",
"src/utils/backoff_committer.cpp",
"src/utils/poll_strategy_base.cpp",
"src/utils/roundrobin_poll_strategy.cpp",
...
In the WORKSPACE file:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("#bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")
maybe(
http_archive,
name = "librdkafka",
build_file = ":librdkafka.BUILD",
strip_prefix = "...",
sha256 = "...",
urls = ["https://github.com/..."],
)
Similar for cppkafka with deps attribute to librkafka
Related
I have a file hello.txt that lives in src/main/resources folder.
I am using a java_binary rule and need to pass this hello.txt file as an argument.
java_binary(
name = "Hello",
srcs = glob(["src/main/java/**"]),
args = ["/Users/jdoe/repo1/libraries/myproj/src/main/resources/hello.txt"],
deps = [...],
)
The above works when I provide the full path, however it fails if I try with a relative path like src/main/resources/hello.txt.
How do I provide a relative path to the args attribute?
Your binary depends on the resource file hello.txt, but Bazel is not aware of this. Make hello.txt a data dependency of your java_binary, i. e. add the attribute data = ["src/main/resources/hello.txt"]. Bazel runs your executable in a sandbox, i.e. somewhere where your hello.txt is not present. The data dependency makes sure that the file is copied to the place where it is needed.
I am trying to set up a simple Bazel workspace to build a pybind library and I am unable to get Bazel to use a specific non-system python (pulled using the rules_python Bazel pakage).
This is my setup:
.
├── BUILD
├── WORKSPACE
└── example.cpp
BUILD:
load("#pybind11_bazel//:build_defs.bzl", "pybind_extension")
pybind_extension(
name = "example",
srcs = ["example.cpp"],
deps = [ ],
)
WORKSPACE:
oad("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_python",
sha256 = "8c8fe44ef0a9afc256d1e75ad5f448bb59b81aba149b8958f02f7b3a98f5d9b4",
strip_prefix = "rules_python-0.13.0",
url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.13.0.tar.gz",
)
load("#rules_python//python:repositories.bzl", "python_register_toolchains")
python_register_toolchains(
name = "python3_9",
python_version = "3.9",
)
load("#python3_9//:defs.bzl", "interpreter")
load("#rules_python//python:pip.bzl", "pip_parse")
pip_parse(
python_interpreter_target = interpreter,
)
http_archive(
name = "pybind11_bazel",
strip_prefix = "pybind11_bazel-9a24c33cbdc510fa60ab7f5ffb7d80ab89272799",
urls = ["https://github.com/pybind/pybind11_bazel/archive/9a24c33cbdc510fa60ab7f5ffb7d80ab89272799.zip"],
)
http_archive(
name = "pybind11",
build_file = "#pybind11_bazel//:pybind11.BUILD",
strip_prefix = "pybind11-2.10.0",
urls = ["https://github.com/pybind/pybind11/archive/refs/tags/v2.10.0.zip"],
)
load("#pybind11_bazel//:python_configure.bzl", "python_configure")
python_configure(
name = "local_config_python",
python_interpreter_target = "**what goes here**",
)
example.cpp:
#include <pybind11/pybind11.h>
namespace py = pybind11;
void do_something() { }
PYBIND11_MODULE(example, m) {
m.def("do_something", &do_something, "A function that does something");
}
If I remove the python_interpreter_target = "**what goes here**" line from the WORKSPACE everything works, but it is using the system python which I am trying to avoid. But I can't find where/what bazel target the python3 binary is from the rules_bazel pakage pull.
In case it helps, these are all the targets I can see for #python3_9:
% bazel query #python3_9//...
#python3_9//:files
#python3_9//:includes
#python3_9//:libpython
#python3_9//:pip
#python3_9//:py3_runtime
#python3_9//:python3
#python3_9//:python_headers
#python3_9//:python_runtimes
Loading: 0 packages loaded
(None of these seem to be what python_interpreter_target wants)
Please try passing the actual path to your target python executable. Kindly check the python executable into the third_party dir and instead of invoking your rules_python using simply python pass the relative path to that executable relative to WORKSPACE. Thank you!
You're almost there, just need to pass in the interpreter:
load("#python3_9//:defs.bzl", "interpreter")
...
python_configure(
name = "local_config_python",
python_interpreter_target = interpreter,
)
I'm trying to build my project running Bazelbuild //... but I'm getting the following error
CPTI/WORKSPACE:1:1: name 'git_repository' is not defined
There's my WORKSPACE file:
git_repository(
name = "org_pubref_rules_protobuf",
remote = "https://github.com/pubref/rules_protobuf",
tag = "v0.8.1",
# commit = "d9523f3d443b6a4f3fabc72051d84eb5474d7745"
)
load("#org_pubref_rules_protobuf//cpp:rules.bzl", "cpp_proto_repositories")
#cpp_proto_repositories()
#BTW, #org_pubref_rules_protobuf already contains #com_google_googletest
load("//tools/build_defs:externals.bzl",
"new_patched_http_archive",
)
# The sparsehash BUILD is copied from https://github.com/livegrep/livegrep
new_patched_http_archive(
name = "com_github_sparsehash",
url = "https://github.com/sparsehash/sparsehash/archive/sparsehash-2.0.3.tar.gz",
sha256 = "05e986a5c7327796dad742182b2d10805a8d4f511ad090da0490f146c1ff7a8c",
build_file = "//third_party:BUILD.sparsehash",
strip_prefix = "sparsehash-sparsehash-2.0.3/",
patch_file = "//third_party:sparsehash.patch",
)
I found another post with the answer Solve dependency issue when using gRPC cpp in bazel
I added and it worked. Sorry, I didn't found it before.
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
I need the path to external (or internal) dependency to pass it as an argument to a function inside. We need the location to the folder, not specific files. Also, sometimes, we need the path to the folder where a shared library, generated by cc_library.
Python file
import cppyy
cppyy.add_include_path('path/to/external/dependency/1')
cppyy.add_library_path('path/to/another/external/dependency/2')
cppyy.add_include_path('path/to/another/internal/dependency')
cppyy.include('file/in/external/dependency')
BUILD file
py_binary(
name = "sample",
srcs = ["sample.py"],
deps = [
"#cppyy_archive//:cppyy",
],
data = [
"#external-dependency//location:target",
"//internal-dependency/location:target2"
]
)
From https://docs.bazel.build/versions/master/external.html#layout:
You can see the external directory by running:
ls $(bazel info output_base)/external
How the paths in external actually look like really depends on the rule used for the archive.
For example, if it's declared using an http_file in the WORKSPACE file:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
http_file(
name = "fenix",
urls = ["https://github.com/mozilla-mobile/fenix/archive/v76.0.0-beta.2.tar.gz"],
sha256 = "94050c664e5ec5b66cd2ca9f6a8b898987ab63d9602090533217df1a3f2dc5a9"
)
You will find that v76.0.0-beta.2.tar.gz file as external/fenix/file/downloaded:
user#host:~$ file $(bazel info output_base)/external/fenix/file/downloaded
/home/user/.cache/bazel/_bazel_user/761044447e04744e746cd54d0b4b5056/external/fenix/file/downloaded: gzip compressed data, from Unix, original size modulo 2^32 15759360
So I am trying to convert a monorepo of micro services (C#, Go, NodeJS) to use bazel. Just playing with it for now.
I focus on one go service to get started and isolated it as a WORKSPACE.
The go service is gRPC service that uses protobuf obviously, grpc-gateway with the protoc-gen-swagger and also protoc-gen-gorm (this one does not support bazel).
The code builds using a command like go build cmd/server/server.go
I am hoping to get some guidance on how to get started to build this project with all the dependencies.
I see several rules available for protobuf/go and I am not yet comfortable browsing through them or deciding which is better (i cannot get any to work due to grpc gateway or protoc gen gorm)
- https://github.com/stackb/rules_proto
- https://github.com/bazelbuild/rules_go
- https://github.com/stackb/rules_proto/tree/master/github.com/grpc-ecosystem/grpc-gateway
Code structure looks like this:
/repo
svc1
svc2
svc3
cmd/server
BUILD.bazel
server.go
pkg
contains folders and some go files and a BUILD.bazel in each
proto
BUILD.bazel
test.proto
WORKSPACE
BUILD.bazel
Right now I only work on svc3. Later i will probably move the WORKSPACE to the parent folder.
My WORKSPACE looks like this:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "io_bazel_rules_go",
sha256 = "96b1f81de5acc7658e1f5a86d7dc9e1b89bc935d83799b711363a748652c471a",
urls = [
"https://storage.googleapis.com/bazel-mirror/github.com/bazelbuild/rules_go/releases/download/0.19.2/rules_go-0.19.2.tar.gz",
"https://github.com/bazelbuild/rules_go/releases/download/0.19.2/rules_go-0.19.2.tar.gz",
],
)
load("#io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
go_rules_dependencies()
go_register_toolchains()
http_archive(
name = "bazel_gazelle",
urls = [
"https://storage.googleapis.com/bazel-mirror/github.com/bazelbuild/bazel-gazelle/releases/download/0.18.1/bazel-gazelle-0.18.1.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/0.18.1/bazel-gazelle-0.18.1.tar.gz",
],
sha256 = "be9296bfd64882e3c08e3283c58fcb461fa6dd3c171764fcc4cf322f60615a9b",
)
load("#bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
gazelle_dependencies()
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
name = "com_google_protobuf",
commit = "09745575a923640154bcf307fba8aedff47f240a",
remote = "https://github.com/protocolbuffers/protobuf",
shallow_since = "1558721209 -0700",
)
load("#com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
+ a bunch of go_repository() created by Gazelle
Running gazelle created a bunch of build.bazel files for my go project in each folder.
Next to the .proto, I have a generated build.bazel file:
load("#io_bazel_rules_go//go:def.bzl", "go_library")
load("#io_bazel_rules_go//proto:def.bzl", "go_proto_library")
proto_library(
name = "svc_proto",
srcs = ["test.proto"],
visibility = ["//visibility:public"],
deps = [
# the two github below are referenced as go_repository
"#com_github_infobloxopen_protoc_gen_gorm//options:proto_library", # not sure what to put after the colon
"#com_github_grpc_ecosystem_grpc_gateway//protoc-gen-swagger/options:proto_library",
"#go_googleapis//google/api:annotations_proto",
],
)
go_proto_library(
name = "svc_go_proto",
compilers = ["#io_bazel_rules_go//proto:go_grpc"],
importpath = "src/test/proto/v1",
proto = ":svc_proto",
visibility = ["//visibility:public"],
deps = [
"//github.com/infobloxopen/protoc-gen-gorm/options:go_default_library",
"//github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger/options:go_default_library",
"#go_googleapis//google/api:annotations_go_proto",
],
)
go_library(
name = "go_default_library",
embed = [":svc_go_proto"],
importpath = "src/test/proto/v1",
visibility = ["//visibility:public"],
)
Now the questions:
not sure what to put to reference other proto files: "#com_github_infobloxopen_protoc_gen_gorm//options:proto_library" ? and not sure this is the best way to reference other external libraries from git.
if i build the above using bazel build //proto/v1:svc_proto, i get: no such target '#com_github_grpc_ecosystem_grpc_gateway//protoc-gen-swagger/options:proto_library': target 'proto_library' not declared in package 'protoc-gen-swagger/options'. Probably linked to 1.
i am not sure which rule to use. As I need grpc gateway, I guess i
need to exclusively use
https://github.com/stackb/rules_proto/tree/master/github.com/grpc-ecosystem/grpc-gateway
but i can't make them to work either.
I use statik (https://github.com/rakyll/statik) to package the
swagger file in go to server the swagger. Is there any alternative
or if not, how can i call a custom bash/command as part of the build
process in the chain?
In summary, I am pretty sure my BUILD.bazel file to build the proto and library is structured wrong and would appreciate some up-to-date guidance (github is full of repos that are outdated, using outdated rules or simply not working).