How to generate protobuf problem with importing - dart

I have those 2 protobuf files:
user.proto:
syntax = "proto3";
import "google/protobuf/timestamp.proto";
message User{
string name = 1;
int32 age = 2;
string email = 3;
google.protobuf.Timestamp created_at = 4;
}
API.proto:
syntax = "proto3";
import "google/protobuf/empty.proto";
import "user.proto";
service ServerAPI {
rpc Init(google.protobuf.Empty) returns (google.protobuf.Empty) {}
rpc Get(User) returns (Result) {}
}
message Result {
bool result = 1;
}
Both of them are in the same dir called proto_files:
When I am trying to run protoc -I=. --dart_out=grpc:build/generated proto_files/*.proto I am getting the following error message.
google/protobuf/empty.proto: File not found.
user.proto: File not found.
proto_files/API.proto:2:1: Import "google/protobuf/empty.proto" was not found or had errors.
proto_files/API.proto:3:1: Import "user.proto" was not found or had errors.
proto_files/API.proto:6:12: "google.protobuf.Empty" is not defined.
proto_files/API.proto:6:44: "google.protobuf.Empty" is not defined.
proto_files/API.proto:7:11: "User" is not defined.
But if I am going to remove the dependencies between those 2 files, everything is going to work properly, if I will not use the import anylonger.
Can anyone help me to understand what I am doing wrong and how I can fix it?
Thx :D

protoc is very particular that you must fully (and consistently) reference the path(s) to your protos.
Because proto_files is used to reference your protos (i.e. proto_files/*.proto), proto_files must be in the --proto_path (-I) too.
protoc should (!?) include Google's Well-Known Types. The binary (protoc) is in the /bin folder and the Well-Known Types should be in the /include folder.
It appears that, on your machine, these aren't being automatically discovered. You can add these directly to the --proto_path too but you will need to specify the location.
Try:
# Replace with path to `protoc-${VERS}-${ARCH}` folder
# The `protoc` binary is in `${PROTOC}/bin`
# Google's Well-Known Types should be in `${PROTOC}/include`
PROTOC="..."
protoc \
--proto_path=proto_files \
--proto_path=${PROTOC}/include \
--dart_out=grpc:build/generated \
proto_files/*.proto

Related

If I provide CLANG_DATABASE_PATH in Doxyfile do I still have to provide INPUT and INCLUDE_PATH?

I am using Doxygen for documentation of my project. I have a compile_commands.json file describing the source code of my project inside the directory C:\dev\project_dir. I set the following variables:
CLANG_ASSISTED_PARSING = YES
CLANG_DATABASE_PATH = C:\dev\project_dir
So how does this work? Do I also have to set the variables INPUT and INCLUDE_PATH? It seems that all the files and the instructions to compile them, including where to get header files from, are written in the compilation database.
And if I do have to set the variables INPUT and INCLUDE_PATH also, what should I set them to? The compilation database lists the source and header files of the project, which are scattered among multiple different directories. How should I proceed in this situation.
I found the answer.
So I set the following variables in the Doxyfile.
CLANG_ASSISTED_PARSING = YES
CLANG_DATABASE_PATH = C:\dev\project_dir
And I set the following variables as blank:
INPUT =
INCLUDE_PATH =
INPUT specifies the paths to source code *.c *.cpp files and/or directories to be processed. INCLUDE_PATH specifies the paths to header code *.h *.hpp files and/or directories to be processed.
The CLANG_ASSISTED_PARSING = YES enables using clang compiler as the parser instead of the default doxygen parser. So if INPUT and INCLUDE_PATH are not set, then it gets the source code files and header code files from the compilation database itself. The CLANG_DATABASE_PATH specifies the directory in which the compilation database is stored. It grabs the file named compile_commands.json by default from that directory, implying that the name of the compilation database is fixed. If you name your compilation database JSON file anything else other than compile_commands.json doxygen won't be able to find it.
So if a clang compilation database JSON file is used all the *.c *.cpp files that are being compiled are placed in the INPUT. And all the header code files are placed in the INCLUDE_PATH. The clang used by doxygen parses the JSON, and every time it encounters a -I compiler flag it recognizes that file as a header file, adding it to the INCLUDE_PATH. This means that setting INPUT and INCLUDE_PATH are not mandatory. So if the compilation database is properly formatted, and all the header files are explicitly marked with the -I, only setting the CLANG_DATABASE_PATH is sufficient.
But there is a certain situation when the INCLUDE_PATH also needs to be set explicitly. For example if you have a source code file, which includes a header file, which includes another header file inside of it.
first.h
int one(int);
second.h
#include "first.h"
int two(int);
code.cpp
#include "second.h"
int main(void) {}
And the command in the compilation database is such:
clang -I path/to/second.h -c code.cpp
So in this case doxygen would read that file, and it would internally set the following variables as such:
INPUT = code.cpp
INCLUDE_PATH = path/to/second.h
This means that although doxygen will index second.h, it will miss first.h since that header file isn't explicitly provided in the -I compilation database. That would be an error. So we need to list it explicitly in the doxyfile, an an additional include path.
INPUT =
INCLUDE_PATH = path\to\first.h
CLANG_ASSISTED_PARSING = YES
CLANG_DATABASE_PATH = C:\dev\project_dir

How to write Bazel rules that work with external repositories?

The Bazel Starlark API does strange things with files in external repositories. I have the following Starlark snippet:
print(ctx.genfiles_dir)
print(ctx.genfiles_dir.path)
print(output_filename)
ret = ctx.new_file(ctx.genfiles_dir, output_filename)
print(ret.path)
It is creating the following output:
DEBUG: build_defs.bzl:292:5: <derived root>
DEBUG: build_defs.bzl:293:5: bazel-out/k8-fastbuild/genfiles
DEBUG: build_defs.bzl:294:5: google/protobuf/descriptor.upb.c
DEBUG: build_defs.bzl:296:5: bazel-out/k8-fastbuild/genfiles/external/com_google_protobuf/google/protobuf/descriptor.upb.c
That extra external/com_google_protobuf comes seemingly out of nowhere, and it makes my rule fail:
I tell protoc to generate into ctx.genfiles_dir.path (which is bazel-out/k8-fastbuild/genfiles).
So protoc generates bazel-out/k8-fastbuild/genfiles/google/protobuf/descriptor.upb.c
Bazel fails because I didn't generate bazel-out/k8-fastbuild/genfiles/external/com_google_protobuf/google/protobuf/descriptor.upb.c
Likewise, when I try to call file.short_path on a source file from an external repository, I get a result like ../com_google_protobuf/google/protobuf/descriptor.proto. This seems quite unhelpful, so I just wrote some manual code to strip off the leading ../com_google_protobuf/.
Am I missing something? How can I write this rule in a way that doesn't feel like I'm fighting Bazel the whole time?
Am I missing something?
The basic problem, as you already realized, is that you have two path "namespaces" the one that protoc sees (i.e. import paths) and the one that bazel sees (i.e. the path you pass to declare_file().
2 things to note:
1) All paths declared with declare_file() get the path <bin dir>/<package path incl. workspace>/<path you passed to declare_file()>
2) All actions are executed from <bin dir> (unless output_to_genfils=True in which case this switches to <gen dir> as in your example.
Trying to solve the exact same problem you encountered, I resorted to stripping the known path from the output_file's path to determine which directory to pass as p:
# This code is run from the context of the external protobuf dependency
proto_path = "google/a/b.proto"
output_file = ctx.actions.declare_file(proto_path)
# output_file.path would be `<gen_dir>/external/protobuf/google/a/b.proto`
# Strip the known proto_path from output_file.path
protoc_prefix = output_file.path[:-len(proto_path)]
print(protoc_prefix) # Prints: <gen_dir>/external/protobuf
command = "{protoc} {proto_paths} {cpp_out} {plugin} {plugin_options} {proto_file}".format(
...
cpp_out = "--cpp_out=" + protoc_prefix,
...
)
Alternatives
You may also be able to construct the same path with ctx.bin_dir, ctx.label.workspace_name, ctx.label.package, and ctx.label.name.
Misc.
proto_library recently gained an attribute strip_import_prefix. When used, the above is not correct, as all dependent files are symlinked into a new directory from which they have the relative paths declared with strip_import_prefix.
The path format is:
<bin dir>/<repo>/<package>/_virtual_base/<label name>/<path `import`ed in .proto files>
i.e.
<bin dir>/external/protobuf/_virtual_base/b_proto/google/a/b.proto
Assuming you are building an external repo called protobuf, which contains a BUILD file at its root with a target named b_proto, which in turn, relies on a proto_library wrapping google/a/b.proto AND uses the strip_import_prefix attribute.

How do we refer to etc package from NixOS configuration?

I want to get a path, which leads to nixos /etc location (any one of /run/current-system/etc or /nix/store/hashhere-etc-1.0). I use this path to configure pppd connect script, some kind of the following,
environment.etc."huawei" =
{ text = ''
/dev/ttyUSB0
38400
lock
crtscts
nodetach
noipdefault
# Below here what I've struggled
connect ${pkgs.etc}/${environment.etc."huawei-script".target}
'';
mode = "0777";
target = "ppp/peers/huawei"; };
I have tried to write ${pkgs.etc} or ${system.build.etc} or even ${environment.etc} resulting errors.
The directory structure is actually relative, but I think it's safer to use absolute path.
/nix/store/...etc.../ppp/peers
|- huawei
|- huawei.d
|- huawei.sh
|- huawei.chat
You can refer to path to file in /nix/store/...etc... like this:
{ config, pkgs, lib, ... }:
{
environment.etc."test".text = "helo";
environment.etc."test2".text = "${config.environment.etc."test".source.outPath}";
}
Now I have in /etc/test2:
$ cat /etc/test2
/nix/store/1igc2rf011jmrr3cprsgbdp3hhm5d4l0-etc-test
If I understand correctly your problem is you simply need to pass the string value of the target attribute to the huawei.text connect directive. As per the description for the target attribute the value is a path relative to /etc so you should be able to either:
Make the value of the connect directive the string literal connect /etc/ppp/peers/huawei or
make the etc.huaweiattribute set a recursive one so that the attributes can refer to each other then do
environment.etc.huawei = rec {
target = "ppp/peers/huawei";
text = ''...
# Below here what I've struggled
connect ${target}
'';
};
Sorry, I was overlook a fact where NixOS actually map any files in /nix/store/...etc../ into the /etc itself.
So, to refer to a file, it is better to use /etc directly.
connect /etc/${environment.etc."huawei-script".target}

excel xlwings_udfs module is empty

Using xlwings 0.7.1 UDF on Windows in 64-bit virtual env python 2.7.6.
I see now that instead of requiring full path to module, it takes module names. However it fails silently to import any UDFs when the module name has package name prefixed. Eg:
PYTHONPATH = ThisWorkbook.Path & ";C:\pathTo\Pydev\myproj\src"
UDF_MODULES = "pkg.myudfs"
If I move the package name 'pkg' from UDF_MODULES to PYTHONPATH, then it fails at imports inside myudfs.py (like 'import pkg.module2').
After hit & trial, I fixed it by adding multiple source folders:
PYTHONPATH = ThisWorkbook.Path & ";C:\pathTo\Pydev\myproj\src\pkg;C:\pathTo\Pydev\myproj\src"
Am I expected to do this? Can't I just point UDF_MODULES to base src folder and provide qualified module name like 'pgk.myudfs'?
You're actually doing it right for right now (v0.7.1). I have, however, opened an issue on GitHub so we might make this easier in a future release.

mogenerator: use of unresolved identifier

Trying to get coredata working with mogenerator. I have generated the human and machine files by adding an entry in the build phases:
/usr/local/bin/mogenerator --v2 --model virgin1/Model.xcdatamodeld -M virgin1/Model/GerneratedClasses/machine -H virgin1/Model/GerneratedClasses/human --swift
It generates (in my case) the Spot.swift and _Spot.swift files. So far so good. But if I use the Spot class in the project like:
let result = Spot.importSpots(dict)
it gives me a "use of unresolved identifier 'Spot'"
Do I have to include the Spot class anywhere?
Thanks in advance!

Resources