Bazel Maven migration Transitive Dependencies Scope - bazel

I am trying to use the generate_workspace on one of the project which has deps and transitive dependencies. Once thegenerate_workspace.bzl has been generated and I copied it to the WORKSPACE and followed the instruction in the bazel docs. Though I see the deps and their transitive deps listed in the generate_workspace.bzl my project during java_library phase is not able resolve transitive deps.. when I import the same project in IDEA I don't see the jars correctly loaded.
My doubt is for the deps I see the generate_workspace.bzl is adding its transitve deps as runtime_deps which means they are available only during runtenter code hereime
I have created gist of all the files here
https://gist.github.com/kameshsampath/8a4bdc8b22d85bbe3f243fa1b816e464
Ideally in my maven project I just need https://gist.github.com/kameshsampath/8a4bdc8b22d85bbe3f243fa1b816e464#file-src_main_build-L8-L9, though generate_workspace.bzl has resolved rightly i thought its enough if my src/main/BUILD looks like
java_binary(
name = "main",
srcs = glob(["java/**/*.java"]),
resources = glob(["resources/**"]),
main_class = "com.redhat.developers.DemoApplication",
# FIXME why I should import all the jars when they are transitive to spring boot starter
deps = [
"//third_party:org_springframework_boot_spring_boot_starter_actuator",
"//third_party:org_springframework_boot_spring_boot_starter_web",
],
)
But sadly that gives lots of compilation errors as transitive deps are not getting loaded as part the above declaration. eventually I have to define like how I did in the https://gist.github.com/kameshsampath/8a4bdc8b22d85bbe3f243fa1b816e464#file-src_main_build
src_main_build is BUILD file under package src/main/BUILD
third_party_BUILD is the BUILD under package third_party/BUILD

Bazel expects you to declare all your direct dependencies. I.e. if you directly use a class from jar A, you need to have it in your direct dependencies.
What you are looking for is a deploy jar. This is a bit hacky but you can actually do it that way (in third_party/BUILD):
java_binary(
name = "org_springframework_boot_spring_boot_starter_actuator_bin",
main_class = "not.important",
runtime_deps = [":org_springframework_boot_spring_boot_starter_actuator"], )
java_import(
name = "springframework_actuator",
jars = [":org_springframework_boot_spring_boot_starter_actuator_bin_deploy.jar"],
)
This will bundle all dependencies except the neverlink one in a jar (the _deploy.jar) and reexpose it.

An update: rules_jvm_external is the officially maintained ruleset by the Bazel team to fetch and resolve artifacts transitively.
You can find the example for Spring Boot here. The declaration in the WORKSPACE file looks something like this:
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"org.hamcrest:hamcrest-library:1.3",
"org.springframework.boot:spring-boot-autoconfigure:2.1.3.RELEASE",
"org.springframework.boot:spring-boot-test-autoconfigure:2.1.3.RELEASE",
"org.springframework.boot:spring-boot-test:2.1.3.RELEASE",
"org.springframework.boot:spring-boot:2.1.3.RELEASE",
"org.springframework.boot:spring-boot-starter-web:2.1.3.RELEASE",
"org.springframework:spring-beans:5.1.5.RELEASE",
"org.springframework:spring-context:5.1.5.RELEASE",
"org.springframework:spring-test:5.1.5.RELEASE",
"org.springframework:spring-web:5.1.5.RELEASE",
],
repositories = [
"https://jcenter.bintray.com",
]
)

Related

With Bazel how do I make part of one genrules' source files (e.g. header files) available to another genrule?

Maybe this is a no-brainer and I just didn't get the concept yet.
I have a genrule, basically wrapping an existing make/config workflow to integrate it into a Bazel-based build configuration. In my example I'd like to build openssl, and then (with the same approach) some library depending on openssl, say xmlsec1
My (shortened) rule for openssl looks like this:
genrule(
name = "build",
visibility = ["//visibility:public"],
srcs = glob(["**/*"], exclude=["bazel-*"]),
outs = [
"libssl.a",
"libcrypto.a",
"include/openssl/opensslconf.h",
],
cmd = """
OUT_DIR="$$(realpath $(RULEDIR))"
pushd "$$(dirname $(location config))"
./config
make
make -j6 DESTDIR="$$OUT_DIR" install_sw install_ssldirs
"""
)
This builds fine and $OUT_DIR contains all files I need to build against openssl
I'd now like to create another genrule building xmlsec1 which needs the path to openssls header files.
Now if I want to access a header, say include/opensslv.h it won't be part of #openssl//:builds artifacts since I didn't explicitly listing it in outs. But doing so results in
ERROR: Traceback (most recent call last):
File "/bla/blubb/.cache/bazel/_bazel_me/f68917ddf601b6533d6db04f8101d580/external/openssl/BUILD.bazel", line 37, column 8, in <toplevel>
genrule(
Error in genrule: rule 'build' has file 'include/openssl/opensslv.h' as both an input and an output
which is correct of course, but what can I do about it?
Removing those header files from srcs doen't work neither, since they wouldn't be available at build time.
One way would be to make install openssl to some destination directory, listing each of the dozens of header files explicitly and using that prefix in all dependent projects. But that doesn't feel right.
What's the recommended way to pass lists of files from one genrule to another?
xmlsec1 could have include/openssl/opensslv.h in its own srcs directly. The build genrule shouldn't really need include/openssl/opensslv.h in its outs both because that would be a circular dependency as bazel said, and because the genrule doesn't really build that file: it already exists on disk (I assume it's getting captured by the glob())
There may be nicer way to organize the library though, something like this:
genrule(
name = "build_openssl",
visibility = ["//visibility:private"],
outs = [
"libssl.a",
"libcrypto.a",
"include/openssl/opensslconf.h",
],
.....,
)
cc_library(
name = "openssl",
srcs = [":build_openssl"],
hdrs = [
"include/openssl/opensslv.h",
# other headers that openssl should provide
],
)
then your other rules can depend on the openssl cc_library and get both the .a files and the header files. (I have not tested this though)

Can't seem to get simple Bazel python script to run

I have just switched to bazel and I am trying to figure out how to get things working. I have added the following to my bazel BUILD file
package(default_visibility = ["//visibility:public"])
py_binary(
name = "Test",
srcs = [ "Test.py" ],
deps = [
"#numpy",
],
)
How on earth do i get Bazel to import numpy?
When I run it, it says no such package '#numpy//'
And of course if i dont have any deps it says no module named numpy.
What is the syntax to be used here?
#numpy is a not valid bazel label. To use dependencies from PyPI repositories you need to use external python rules: bazelbuild/rules_python

How to add an external dependency into a Swift project using Bazel (without using Cocoapod)

The dependency I'm trying to integrate is TensorFlowLite:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/swift
I'm already using Carthage and SwiftPackageManager to manage dependencies of an Xcode project.
Since TensorFlowLite supports Bazel, I tried to download, build, and link the dependency using the Bazel configuration.
I downloaded the Bazel iOS demo app from here:
https://github.com/bazelbuild/examples/tree/master/tutorial
And followed the instruction in this page:
https://docs.bazel.build/versions/master/tutorial/ios-app.html
When I try to build the demo iOS app using the following command:
bazel build //ios-app
I get the following error:
ERROR: /Users/nebil/Downloads/examples-master/tutorial/ios-app/BUILD:15:14:
no such package '#TensorFlowLite//tensorflow/lite/experimental/swift': BUILD file not found in directory 'tensorflow/lite/experimental/swift' of external repository #TensorFlowLite.
Add a BUILD file to a directory to mark it as a package. and referenced by '//ios-app:TensorFlowLite'
ERROR: Analysis of target '//ios-app:ios-app' failed; build aborted: Analysis failed
In the main WORKSPACE file I added the following dependency:
git_repository(
name = "TensorFlowLite",
remote = "https://github.com/tensorflow/tensorflow",
tag = "v2.2.0",
)
And the BUILD file of the ios_app package:
load("#build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load("#build_bazel_rules_swift//swift:swift.bzl", "swift_library")
swift_library(
name = 'TensorFlowLite',
deps = [
"#TensorFlowLite//tensorflow/lite/experimental/swift:TensorFlowLite",
],
)
ios_application(
name = "ios-app",
bundle_id = "Google.UrlGet",
families = [
"iphone",
"ipad",
],
infoplists = [":UrlGet/UrlGet-Info.plist"],
launch_storyboard = "UrlGet/UrlGetViewController.xib",
minimum_os_version = "8.0",
visibility = ["//visibility:public"],
deps = [:TensorFlowLite"],
)
I'm using the same project structure of the same project and simply added a new dependency to the main app.
The build is failing because the BUILD file is missing from the third party code, but this is not true:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/swift/BUILD.apple
I'm not an expert with Bazel but I don't see why this shouldn't be possibile. Any clarification would be much appreciated.
Thank you

"undeclared inclusion(s)" in bazel with headers in subdirectories

I'm trying to use ArduinoJson in a C++ bazel project. It's a header only library with headers in subdirectories under src. But I get an undeclared inclusion(s) error.
This is how the //Third-Party/ArduinoJson cc_library def looks like:
cc_library(
name = "ArduinoJson",
hdrs = glob(["5.12.0/src/**"]),
includes = [
"5.12.0/src",
"5.12.0/src/ArduinoJson",
],
visibility = ["//visibility:public"],
)
The target that uses it (Serialization) does have //Third-Party/ArduinoJson in it's deps
Here is the error:
ERROR: /[...]/Serialization/BUILD:1:1: undeclared inclusion(s) in rule '//[...]/Serialization:Serialization':
this rule is missing dependency declarations for the following files included by '[...]/Serialization/JsonDeserializer.cpp':
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Data/Encoding.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Serialization/FloatParts.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Polyfills/math.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/TypeTraits/FloatTraits.hpp'
Target //[...]/Serialization:Serialization failed to build
Since it is only complaining about files in subdirectories of src, e.g. src/Data/Encoding.hpp, I'm guessing that might have something to do with it?
This is not a duplicate of "How to resolve bazel “undeclared inclusion(s)” error?" since that one is solved by doing what I'm already doing with deps.
Make sure there are no BUILD files in the subdirectories of Third-Party/ArduinoJson, otherwise Bazel regards those as packages and the glob won't pick up files from them.
If you do have BUILD files in those subdirectories, you need to create cc_library rules in them, exporting the headers that you wanted in //Third-Party/ArduinoJson:ArduinoJson, and depending on these cc_libraries from ArduinoJson.

Bazel cc_library dependency on other cc_library when each compile with a different crosstool

I have a code generator tool that generates C/C++ code. This code generator tool is compiled with crosstool1. The generated C/C++ code needs to be compiled with crosstool2.
So the actions are:
Using Crosstool1 compile 'code_generator'.
Execute 'code_generator' and generate 'generated_code.cpp'
Using Crosstool2 compile 'generated_code.cpp'
Is it possible to make a cc_library() determine the crosstool to use? I saw that Skylark rules now allow a 'toolchains' parameter which I'm not sure how this is used, also I do not want to do the heavy lifting of C/C++ compiling bare bone with Skylark.
Is there an example of using a proper Host Crosstool and Target Crosstool except for the Tenserflow example? I get a headache each time I read it :D
Assume //crosstool1:toolchain is a label for cc_toolchain_suite rule describing first crosstool, //crosstool2:toolchain is a label for cc_toolchain_suite for second crosstool, and the build file for the project is:
cc_binary(
name = "generator",
srcs = [ "main.cc" ],
)
genrule(
name = "generate",
outs = ["generated.cc"],
cmd = "$(location :generator) > $#",
tools = [":generator"],
)
cc_binary(
name = "generated",
srcs = [ "generated.cc" ],
)
Then running:
bazel build --host_crosstool_top=//crosstool1:toolchain --crosstool_top=//crosstool2:toolchain :generated
will do exactly what you describe, it will use crosstool1 to build :generator, and crosstool2 to build generated. Genrules use host configuration by default, so all should just work.

Resources