Bazel TestNG XML output - bazel

Question

I have a small Java project: one package with dependencies on Google Truth, Google Guava, the JSR305 annotations, and TestNG for unit tests. I've been having some trouble running the tests with Bazel. I can create a java_test rule and run it with bazel test, but Bazel's XML output gives me a single pass/fail for the entire test suite, with no information on individual failures. The XML from TestNG gets cleaned up along with the sandbox.
To get around this, I've created a genrule for TestNG's XML, but the documentation explicitly says "don't use genrules for testing" so I'm wondering if there's a better approach.
My BUILD file looks like this:
java_library(
name='myproject',
srcs=glob(['src/main/java/**/*.java']),
deps=[
'#com_google_code_findbugs_jsr305//jar',
'#com_google_guava_guava//jar',
],
)
java_library(
name='myproject-test-lib',
srcs=glob(['src/test/java/**/*.java']),
deps=[
':myproject',
'#com_google_code_findbugs_jsr305//jar',
'#com_google_guava_guava//jar',
'#com_google_truth_truth//jar',
'#org_testng_testng//jar',
],
)
java_test(
name='myproject-test',
size='small',
runtime_deps=[
':myproject',
':myproject-test-lib',
'#org_testng_testng//jar',
'#com_beust_jcommander//jar', # Used by TestNG CLI
'#org_yaml_snakeyaml//jar', # Used by TestNG to parse YAML
'#junit_junit//jar', # Dependency of Truth
],
data=['testng.yaml'],
use_testrunner=False,
main_class='org.testng.TestNG',
args=['testng.yaml'],
)
genrule(
name='myproject-test-report',
srcs=['testng.yaml'],
tools=[
':myproject',
':myproject-test-lib',
'#com_google_code_findbugs_jsr305//jar',
'#com_google_guava_guava//jar',
'#com_google_truth_truth//jar',
'#org_testng_testng//jar',
'#com_beust_jcommander//jar', # Used by TestNG CLI
'#org_yaml_snakeyaml//jar', # Used by TestNG to parse YAML
'#junit_junit//jar', # Dependency of Truth
],
outs=['testng_report'],
cmd='$(JAVA) -cp $(location :myproject):$(location :myproject-test-lib):$(location #com_google_code_findbugs_jsr305//jar):$(location #com_google_guava_guava//jar):$(location #com_google_truth_truth//jar):$(location #org_testng_testng//jar):$(location #com_beust_jcommander//jar):$(location #org_yaml_snakeyaml//jar):$(location #junit_junit//jar) org.testng.TestNG -d $(OUTS) -usedefaultlisteners false testng.yaml'
)
...I suspect there's also a better way to deal with the classpath. My WORKSPACE file, for completeness:
workspace(name='com_example_myproject')
maven_jar(
name='com_google_code_findbugs_jsr305',
artifact='com.google.code.findbugs:jsr305:3.0.1',
sha1='f7be08ec23c21485b9b5a1cf1654c2ec8c58168d',
)
maven_jar(
name='com_google_guava_guava',
artifact='com.google.guava:guava:21.0',
sha1='3a3d111be1be1b745edfa7d91678a12d7ed38709',
)
maven_jar(
name='com_google_truth_truth',
artifact='com.google.truth:truth:0.32',
sha1='e996fb4b41dad04365112786796c945f909cfdf7',
)
maven_jar(
name='org_testng_testng',
artifact='org.testng:testng:6.11',
sha1='1fdd5e22f50b14f6d846163456e8c9a7657626fb',
)
maven_jar(
name='com_beust_jcommander',
artifact='com.beust:jcommander:1.64',
sha1='456a985ac9b12d34820e4d5de063b2c2fc43ed5a',
)
maven_jar(
name='org_yaml_snakeyaml',
artifact='org.yaml:snakeyaml:1.17',
sha1='7a27ea250c5130b2922b86dea63cbb1cc10a660c',
)
maven_jar(
name='junit_junit',
artifact='junit:junit:4.10',
sha1='e4f1766ce7404a08f45d859fb9c226fc9e41a861',
)

Answer 1

By default bazel test will output just a summary of the test results. To see a more detailed report you can use --test_output all. You could also set --test_summary detailed.
If this won't give you the desired output and you would prefer the testng log, I can think of 2 alternatives:
Disable sandboxing.
Declare testng_report as an input file (using
data attribute of java_test). Bazel needs to know the set of input/output files
and will remove everything not declared beforehand. Since for
java_test there is no way to declare additional output files, try
to declare it as an input if having it at all times in the package
is not an inconvenience. This is a bit hackish and I wouldn't prefer it.
Hope this helps.

Answer 2

I think this is a new option, but I was able to use --sandbox_writable_path to make a directory in CI writable, and then specify my test output to go to that directory.
--sandbox_writable_path=<a string> multiple uses are accumulated
For sandboxed actions, make an existing directory writable in the sandbox (if supported by the sandboxing implementation, ignored otherwise).