Link to external Bazel dependency - bazel

Problem
I'm trying to modify this tutorial for compiling a TensorFlow network for use with C++. It currently requires you to copy your network file into the TensorFlow source code so the dependencies can be found, but I'd rather not do that. Note that TensorFlow is also built with Bazel.
Here is my BUILD file:
cc_binary(
name = "mnistpredict_keras",
srcs = ["mnist_keras.cc", "MNIST.h"],
deps = [
"//tensorflow/core:tensorflow",
],
)
When I try to run $ bazel build :mnistpredict_keras I get the error:
ERROR: /home/saubin/git/tf-keras-speed-test/loadgraph/BUILD:17:1: no such package 'tensorflow/core': BUILD file not found on package path and referenced by '//:mnistpredict_keras'.
ERROR: Analysis of target '//:mnistpredict_keras' failed; build aborted.
INFO: Elapsed time: 0.105s
Obviously, the problem is I'm trying to compile something in my folder ~/git/tf-keras-speed-test/loadgraph but it can't find the dependency //tensorflow/core:tensorflow. How do I properly give the path to the dependency? The documentation for deps appears to be non-existent.
Attempted Solutions
Adding a local_repository to my WORKSPACE:
local_repository(
name = "tensorflow",
path = "/home/saubin/src/tensorflow",
)
This changed nothing.
Trying to pass the full path:
cc_binary(
name = "mnistpredict_keras",
srcs = ["mnist_keras.cc", "MNIST.h"],
deps = [
"/home/saubin/src/tensorflow/tensorflow/core:tensorflow",
],
)
But I get the same error:
ERROR: /home/saubin/git/tf-keras-speed-test/loadgraph/BUILD:17:1: no such package 'tensorflow/core': BUILD file not found on package path and referenced by '//:mnistpredict_keras'.
ERROR: Analysis of target '//:mnistpredict_keras' failed; build aborted.
INFO: Elapsed time: 0.287s
Mark TensorFlow as an external repository dependency:
cc_binary(
name = "mnistpredict_keras",
srcs = ["mnist_keras.cc", "MNIST.h"],
deps = [
"#tensorflow//tensorflow/core:tensorflow",
],
)
But this gives me this error:
WARNING: /home/saubin/.cache/bazel/_bazel_saubin/74f664e7cf53364557da8b57a716c919/external/tensorflow/WORKSPACE:1: Workspace name in /home/saubin/.cache/bazel/_bazel_saubin/74f664e7cf53364557da8b57a716c919/external/tensorflow/WORKSPACE (#org_tensorflow) does not match the name given in the repository's definition (#tensorflow); this will cause a build error in future versions.
ERROR: /home/saubin/git/tensorgraph/loadgraph/BUILD:1:1: error loading package '#tensorflow//tensorflow/core': Encountered error while reading extension file 'sycl/build_defs.bzl': no such package '#local_config_sycl//sycl': error loading package 'external': The repository named 'local_config_sycl' could not be resolved and referenced by '//:mnistpredict'.
ERROR: Analysis of target '//:mnistpredict' failed; build aborted.
INFO: Elapsed time: 0.326s

The documentation for external dependencies is https://bazel.build/versions/master/docs/external.html.
The syntax that you are looking for is #tensorflow//tensorflow/core:tensorflow.
Labels that start with // refer to the current repository. Labels that start with #reponame// refer to the reponame repository.

Related

Swift Package Manager failed to resolve dependencies: artifacts already exists in file system. fatalError

I have uploaded my binary artifacts in maven-central repository. I am fetching these libraries through SPM. I am getting the below error most of the times.
xcodebuild: error: Could not resolve package dependencies:
failed downloading which is required by binary target:
DerivedData/SourcePackages/artifacts/MyLib/MyLib.xcframework.zip already exists in file system. fatalError
My Swift package manifest targets as below:
targets: [
// ---- Binaries -----
.binaryTarget(
name: "MyLib",
url: "\(myPrivateArtefactURL)/MyLib.xcframework.zip",
checksum: "checksum"
)
]
However dependencies are getting resolved if I delete artficats folder manually and run the resolving dependencies command. I am not understanding what is the problem?. I want to get rid off this manually deleting the folder.

'#io_bazel_rules_docker//toolchains/docker': Unable to load file '#bazel_skylib//:bzl_library.bzl': file doesn't exist

i've tried to load io_bazel_rules_docker to push a container i have in bazel. And the thing that have me stuck is the bazel_skylib dependency.
The error is that is not found, i've tried everything, adding it to my Workspace, copying it in my directory and it doesn't work. This is my Workspace part where i declare it.
http_archive(
name = "bazel_skylib",
urls = [
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz",
],
sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
)
load("#bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
git_repository(
name = "io_bazel_rules_docker",
remote = "https://github.com/bazelbuild/rules_docker.git",
tag = "v0.7.0"
)
load(
"#io_bazel_rules_docker//java:image.bzl",
_java_image_repos = "repositories",
)
This is the error that i have:
ERROR: While resolving toolchains for target ******: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#io_bazel_rules_docker//toolchains/docker': Unable to load file '#bazel_skylib//:bzl_library.bzl': file doesn't exist
ERROR: Analysis of target '******' failed; build aborted: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#io_bazel_rules_docker//toolchains/docker': Unable to load file '#bazel_skylib//:bzl_library.bzl': file doesn't exist
I've removed the bazel skylib declaration and is the same error. Still don't know what to do.

How to add an external dependency into a Swift project using Bazel (without using Cocoapod)

The dependency I'm trying to integrate is TensorFlowLite:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/swift
I'm already using Carthage and SwiftPackageManager to manage dependencies of an Xcode project.
Since TensorFlowLite supports Bazel, I tried to download, build, and link the dependency using the Bazel configuration.
I downloaded the Bazel iOS demo app from here:
https://github.com/bazelbuild/examples/tree/master/tutorial
And followed the instruction in this page:
https://docs.bazel.build/versions/master/tutorial/ios-app.html
When I try to build the demo iOS app using the following command:
bazel build //ios-app
I get the following error:
ERROR: /Users/nebil/Downloads/examples-master/tutorial/ios-app/BUILD:15:14:
no such package '#TensorFlowLite//tensorflow/lite/experimental/swift': BUILD file not found in directory 'tensorflow/lite/experimental/swift' of external repository #TensorFlowLite.
Add a BUILD file to a directory to mark it as a package. and referenced by '//ios-app:TensorFlowLite'
ERROR: Analysis of target '//ios-app:ios-app' failed; build aborted: Analysis failed
In the main WORKSPACE file I added the following dependency:
git_repository(
name = "TensorFlowLite",
remote = "https://github.com/tensorflow/tensorflow",
tag = "v2.2.0",
)
And the BUILD file of the ios_app package:
load("#build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load("#build_bazel_rules_swift//swift:swift.bzl", "swift_library")
swift_library(
name = 'TensorFlowLite',
deps = [
"#TensorFlowLite//tensorflow/lite/experimental/swift:TensorFlowLite",
],
)
ios_application(
name = "ios-app",
bundle_id = "Google.UrlGet",
families = [
"iphone",
"ipad",
],
infoplists = [":UrlGet/UrlGet-Info.plist"],
launch_storyboard = "UrlGet/UrlGetViewController.xib",
minimum_os_version = "8.0",
visibility = ["//visibility:public"],
deps = [:TensorFlowLite"],
)
I'm using the same project structure of the same project and simply added a new dependency to the main app.
The build is failing because the BUILD file is missing from the third party code, but this is not true:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/swift/BUILD.apple
I'm not an expert with Bazel but I don't see why this shouldn't be possibile. Any clarification would be much appreciated.
Thank you

error while reading extension file 'intellij_info_bundled.bzl'

we are trying to create a Scala project which uses Spark also but we are facing issue Encountered error while reading extension file 'intellij_info_bundled.bzl': no such package '#intellij_aspect//': No WORKSPACE file found in C:/users//_bazel_user/i45wuf6d/external/intellij_aspect. Is it has something missing in Intellij?
Scala file
package src.main.scala
object HelloWorld extends App {
def main(args: Array[String]) {
println("Hello, world!")
}
}
Build file
package(default_visibility = ["//visibility:public"])
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_test")
scala_library(
name = "hello-world",
srcs = glob(["src/main/scala/*.scala"]),
)
scala_test(
name = "Hello_test",
srcs = glob(["src/main/scala/*.scala"]),
size = "small", # Expect this test to run quickly
)
Work Space
workspace(name = "scala_example")
rules_scala_version="7522c866450cf7810eda443e91ff44d2a2286ba1" # update this as needed
http_archive(
name = "io_bazel_rules_scala",
url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip"%rules_scala_version,
type = "zip",
strip_prefix= "rules_scala-%s" % rules_scala_version
)
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
scala_repositories()`enter code here`
# register default scala toolchain
load("#io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()
Command and Error from console
Command: C:\ProgramData\chocolatey\bin\bazel.exe build --tool_tag=ijwb:IDEA:community --keep_going --curses=no --color=yes --experimental_ui=no --progress_in_terminal_title=no --aspects=#intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect --override_repository=intellij_aspect=C:\Users\ADMIN.IdeaIC2017.3\config\plugins\ijwb\aspect --output_groups=intellij-compile-java,intellij-compile-py -- //...:all
INFO: Loading complete. Analyzing...
ERROR: Encountered error while reading extension file 'intellij_info_bundled.bzl': no such package '#intellij_aspect//': No WORKSPACE file found in C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/external/intellij_aspect.
INFO: Found 3 targets...
WARNING: failed to create one or more convenience symlinks for prefix 'bazel-':
cannot create symbolic link bazel-out -> C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/execroot/scala_example/bazel-out: Cannot create junction (name=C:\users\admin\scalaprojects\example1\bazel-out, target=C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file-jni.cc(86): nativeCreateJunction(C:\users\admin\scalaprojects\example1\bazel-out, C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file.cc(128): CreateJunction(\?\C:\users\admin\scalaprojects\example1\bazel-out): Cannot create a file when that file already exists.
cannot create symbolic link bazel-out -> C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/execroot/scala_example/bazel-out: Cannot create junction (name=C:\users\admin\scalaprojects\example1\bazel-out, target=C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file-jni.cc(86): nativeCreateJunction(C:\users\admin\scalaprojects\example1\bazel-out, C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file.cc(128): CreateJunction(\?\C:\users\admin\scalaprojects\example1\bazel-out): Cannot create a file when that file already exists.
.
INFO: Building...
ERROR: command succeeded, but not all targets were analyzed.
INFO: Elapsed time: 18.108s, Critical Path: 0.05s
Make failed
This is a sample Helloworld program only
In general, like #Ittai, I would suggest you open an issue in the intellij plugin github repo.
Unfortunately, your version of the plugin is no longer supported. I, too, previously ran into an issue with an older version of the plugin and was recommended to upgrade to the latest version. Which resolved the specific issue I was facing.
When reporting the issue make sure to include the following bits of information:
intellij build number
plugin version number
rules_scala version
operating system (it seems your using Windows, while most users use unix based systems)
bazel release number
how you have opened the intellij project (BUILD file, WORKSPACE, .blazeproject)
Additionally, to verify this is in fact an issue with the plugin, I would also suggest you try to reproduce this issue on a Unix based system. It seems you are using Intellij
compile on Windows. This may be Windows specific issue with aspects not being recognized.
When attempting to reproduce, make sure to clone your repository in a separate directory, close the intellij project, and reopen the project

ClassNotFoundException using Spark library through Bazel

I am trying to run a "hello world" server in Spark building it with Bazel, but I am getting this error:
$ bazel run //:app
INFO: Analysed target //:app (0 packages loaded).
INFO: Found 1 target...
Target //:app up-to-date:
bazel-bin/app.jar
bazel-bin/app
INFO: Elapsed time: 0.201s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at spark.Service.<clinit>(Service.java:56)
at spark.Spark$SingletonHolder.<clinit>(Spark.java:51)
at spark.Spark.getInstance(Spark.java:55)
at spark.Spark.<clinit>(Spark.java:61)
at io.app.server.Main.main(Main.java:7)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
BUILD:
java_binary(
name = "app",
main_class = "io.app.server.Main",
srcs = ["src/main/java/io/app/server/Main.java"],
deps = [
"#org_slf4j_slf4j_simple//jar",
"#com_sparkjava_spark_core//jar",
]
)
The same error happens if I don't include slf4j, and it should not be a required dependency of spark.
WORKSPACE:
maven_jar(
name = "com_sparkjava_spark_core",
artifact = "com.sparkjava:spark-core:2.7.2"
)
maven_jar(
name = "org_slf4j_slf4j_simple",
artifact = "org.slf4j:slf4j-simple:1.7.21"
)
And finally, src/main/java/io/app/server/Main.java:
package io.app.server;
import static spark.Spark.*;
public class Main {
public static void main(String[] args) {
port(3000);
get("/", (req, res) -> "Hello World");
}
}
Any idea of what I could be doing wrong here?
Found what I was missing. It seems that maven_jar does not automatically fetch the "transitive dependencies" that the library itself has, see this.
Bazel only reads dependencies listed in your WORKSPACE file. If your
project (A) depends on another project (B) which list a dependency on
a third project (C) in its WORKSPACE file, you'll have to add both B
and C to your project's WORKSPACE file. This requirement can balloon
the WORKSPACE file size, but hopefully limits the chances of having
one library include C at version 1.0 and another include C at 2.0.
Large WORKSPACE files can be generated using the tool
generate_workspace. For details, see Generate external dependencies
from Maven projects.
So the solution seems to be to write a pom.xml and use generate_workspace.
EDIT: generate_workspace seems to be deprecated, use bazel_deps instead.
Another solution might be to use maven_install
git_repository(
name = "rules_jvm_external",
commit = "22b463c485f31b240888c89d17e67c460d7e68c0",
remote = "https://github.com/bazelbuild/rules_jvm_external.git",
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"org.apache.spark:spark-core_2.12:3.1.2",
"org.apache.spark:spark-sql_2.12:3.1.2",
],
repositories = [
"https://repo.maven.apache.org/maven2/",
]
)

Resources