Bazel overwriting test logs and BuildEventProtocol via BuildEventService - bazel

Question

I am playing around with implementing a server of the BuildEventService so that I can have bazel export its BuildEventProtocol messages to it. I'm trying to figure out how to read the logs for a test run without race conditions, and in particular this seems very difficult due to bazel reusing the same path on the local machine for multiple runs and the default asynchronous nature of BES.
Example:
As part of the event stream I get the following:
EventStream event:
stream_id {
build_id: "a4a34ca2-fc4b-483d-b4ab-b4546bdb2c4e"
component: TOOL
invocation_id: "b09c0b08-b096-4673-9521-4980506207f7"
}
sequence_number: 11
event {
event_time {
seconds: 1504560960
nanos: 778000000
}
bazel_event {
[type.googleapis.com/build_event_stream.BuildEvent] {
id {
test_summary {
label: "//libraries:types-test"
configuration {
id: "fe35dfece8e09ba054305e51187b3316"
}
}
}
test_summary {
total_run_count: 1
failed {
uri: "file:///private/var/tmp/_bazel_endobson/f851d7f6c7010ae7d7a3db153bed36de/execroot/yaspl/bazel-out/darwin_x86_64-fastbuild/testlogs/libraries/types-test/test.log"
}
overall_status: FAILED
}
}
}
}
I would like to read the file in the uri:
file:///private/var/tmp/_bazel_endobson/f851d7f6c7010ae7d7a3db153bed36de/execroot/yaspl/bazel-out/darwin_x86_64-fastbuild/testlogs/libraries/types-test/test.log
but it seems that every time I run the test I get the same uri. Thus I want to read it before the next test run recreates it. But bazel by default does the uploading asynchronously, so it seems there is nothing preventing another run of bazel of starting up and recreating the file even before the BES server receives this stream message.
How can I avoid this race and still read these files?

Answer 1

It depends on whether you are in control of the Bazel client. If so then yes you can avoid the race. Else you can't.
You can specify a different --output_base on each invocation of
Bazel (The output base is the path prefix
/private/var/tmp/_bazel_endobson/f851d7f6c7010ae7d7a3db153bed36de in
your example). However, that --output_base is a startup option and
thus requires a Bazel server restart when it's changed. That would
work but it's slow and you need to specify the different
--output_base before the invocation, which might be fine if you invoke Bazel programmatically.
You can specify --bes_best_effort=false in which case the BES upload
is synchronous i.e. Bazel waits for the upload to finish. If the
upload fails, the build also fails.
You could wrap the bazel client in a shell script and additionally to uploading to your BES service, also write the BEP to a file and then at the end of the invocation parse the file for test.log files and upload these before giving control back to the user.