I'm naively passing along some variable test metadata to some py_test targets to inject that metadata into some test result artifacts that later get uploaded to the cloud. I'm doing so using either the --test_env or --test_arg values at the bazel test invocation.
Would this variable data negatively affect the way test results are cached such that running the same test back to back would effectively disturb the bazel cache?
Command Line Inputs
Command line inputs can indeed disturb cache hits. Consider the following set of executions
BUILD file
py_test(
name = "test_inputs",
srcs = ["test_inputs.py"],
deps = [
":conftest",
"#pytest",
],
)
py_library(
name = "conftest",
srcs = ["conftest.py"],
deps = [
"#pytest",
],
)
Test module
import sys
import pytest
def test_pass():
assert True
def test_arg_in(request):
assert request.config.getoption("--metadata")
if __name__ == "__main__":
args = sys.argv[1:]
ret_code = pytest.main([__file__, "--log-level=ERROR"] + args)
sys.exit(ret_code)
First execution
$ bazel test //bazel_check:test_inputs --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.40s
INFO: Critical path 0.57s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.72s (preparation 0.12s, execution 0.60s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.4s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
Second execution: same argument value, cache hit!
$ bazel test //bazel_check:test_inputs --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 1 process: 1 internal (100.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.00s
INFO: Critical path 0.47s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.61s (preparation 0.12s, execution 0.49s)
INFO: Build completed successfully, 1 total action
//bazel_check:test_inputs (cached) PASSED in 0.4s
Executed 0 out of 1 test: 1 test passes.
INFO: Build completed successfully, 1 total action
Third execution: new argument value, no cache hit
$ bazel test //bazel_check:test_inputs --test_arg --metadata=kk
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 93 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.30s
INFO: Critical path 0.54s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.71s (preparation 0.14s, execution 0.57s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.3s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
Fourth execution: reused same argument as first two runs
Interestingly enough there is no cache hit despite the result being cached earlier. Somehow it did not persist.
$ bazel test //bazel_check:test_inputs --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.34s
INFO: Critical path 0.50s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.71s (preparation 0.17s, execution 0.55s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.3s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
Environment Inputs
The same exact behavior applies for --test_env inputs
import os
import sys
import pytest
def test_pass():
assert True
def test_env_in():
assert os.environ.get("META_ENV")
if __name__ == "__main__":
args = sys.argv[1:]
ret_code = pytest.main([__file__, "--log-level=ERROR"] + args)
sys.exit(ret_code)
First execution
$ bazel test //bazel_check:test_inputs --test_env META_ENV=33
INFO: Build option --test_env has changed, discarding analysis cache.
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 7285 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.29s
INFO: Critical path 0.66s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 1.26s (preparation 0.42s, execution 0.84s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.3s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
Second execution: same env value, cache hit!
$ bazel test //bazel_check:test_inputs --test_env META_ENV=33
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 1 process: 1 internal (100.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.00s
INFO: Critical path 0.49s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.67s (preparation 0.15s, execution 0.52s)
INFO: Build completed successfully, 1 total action
//bazel_check:test_inputs (cached) PASSED in 0.3s
Executed 0 out of 1 test: 1 test passes.
INFO: Build completed successfully, 1 total action
Third execution: new env value, no cache hit
$ bazel test //bazel_check:test_inputs --test_env META_ENV=44
INFO: Build option --test_env has changed, discarding analysis cache.
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 7285 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.29s
INFO: Critical path 0.62s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 1.22s (preparation 0.39s, execution 0.83s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.3s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
Fourth execution: reused same env value as first two runs
$ bazel test //bazel_check:test_inputs --test_env META_ENV=33
INFO: Build option --test_env has changed, discarding analysis cache.
INFO: Analyzed target //bazel_check:test_inputs (0 packages loaded, 7285 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.28s
INFO: Critical path 0.66s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 1.25s (preparation 0.40s, execution 0.85s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_inputs PASSED in 0.3s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions
I'm trying to do performance test using the https://github.com/thingsboard/gatling-mqtt project.
Although current versions of the tools are a bit different, I managed to have it running.
I've tested a couple of basic scripts with Gatling using HTTP and Gatling worked as expected.
But when using the scripts provided with the tool, that use the plugin for MQTT it does not work.
In fact it runs but it doesn't do anything. No connections, no logs, no reports, no errors.
On the other tests it increments the global OK=0 as the test progresses, and generates logs and reports.
But as you can see below, when running with the MQTT plugin it doesn't increment the count
.
Simulation MqttSimulation_localhost started...
================================================================================
2021-10-14 17:55:02 5s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- MQTT Test -----------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 10 / done:0
================================================================================
================================================================================
2021-10-14 17:55:07 10s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- MQTT Test -----------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 10 / done:0
================================================================================
================================================================================
2021-10-14 17:55:12 15s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
---- MQTT Test -----------------------------------------------------------------
[--------------------------------------------------------------------------] 0%
waiting: 0 / active: 10 / done:0
================================================================================
================================================================================
2021-10-14 17:55:17 20s elapsed
---- Requests ------------------------------------------------------------------
> Global (OK=0 KO=0 )
... and it goes on forever
Any ideas on what could be happening?
I would certainly appreciate any help on this!
Thank you!
A bazel binary that I am building completes unsuccessfully during the analysis phase. What flags and tools can I use to debug why it fails during analysis.
Currently, clean builds return the following output
ERROR: build interrupted
INFO: Elapsed time: 57.819 s
FAILED: Build did NOT complete successfully (133 packages loaded)
If I retry building after failed completion, I receive the following output
ERROR: build interrupted
INFO: Elapsed time: 55.514 s
FAILED: Build did NOT complete successfully (68 packages loaded)
What flags can I use to identify
what packages are being loaded
what package the build is being interrupted on
whether the interruption is coming from a timeout or an external process.
Essentially, something similar to --verbose_failures but for the analysis phase rather than the execution phrase.
So far I have ran my build through the build profiler, and have not been able to glean any insight. Here is the output of my build:
WARNING: This information is intended for consumption by Blaze developers only, and may change at any time. Script against it at your own risk
INFO: Loading /<>/result
INFO: bazel profile for <> at Mon Jun 04 00:10:11 GMT 2018, build ID: <>, 49405 record(s)
INFO: Aggregating task statistics
=== PHASE SUMMARY INFORMATION ===
Total launch phase time 9.00 ms 0.02%
Total init phase time 91.0 ms 0.16%
Total loading phase time 1.345 s 2.30%
Total analysis phase time 57.063 s 97.53%
Total run time 58.508 s 100.00%
=== INIT PHASE INFORMATION ===
Total init phase time 91.0 ms
Total time (across all threads) spent on:
Type Total Count Average
=== LOADING PHASE INFORMATION ===
Total loading phase time 1.345 s
Total time (across all threads) spent on:
Type Total Count Average
CREATE_PACKAGE 0.67% 9 3.55 ms
VFS_STAT 0.69% 605 0.05 ms
VFS_DIR 0.96% 255 0.18 ms
VFS_OPEN 2.02% 8 12.1 ms
VFS_READ 0.00% 5 0.01 ms
VFS_GLOB 23.74% 1220 0.93 ms
SKYFRAME_EVAL 24.44% 3 389 ms
SKYFUNCTION 36.95% 8443 0.21 ms
SKYLARK_LEXER 0.19% 31 0.29 ms
SKYLARK_PARSER 0.68% 31 1.04 ms
SKYLARK_USER_FN 0.03% 5 0.27 ms
SKYLARK_BUILTIN_FN 5.91% 349 0.81 ms
=== ANALYSIS PHASE INFORMATION ===
Total analysis phase time 57.063 s
Total time (across all threads) spent on:
Type Total Count Average
CREATE_PACKAGE 0.30% 138 3.96 ms
VFS_STAT 0.05% 2381 0.03 ms
VFS_DIR 0.19% 1020 0.35 ms
VFS_OPEN 0.04% 128 0.61 ms
VFS_READ 0.00% 128 0.01 ms
VFS_GLOB 0.92% 3763 0.45 ms
SKYFRAME_EVAL 31.13% 1 57.037 s
SKYFUNCTION 65.21% 32328 3.70 ms
SKYLARK_LEXER 0.01% 147 0.10 ms
SKYLARK_PARSER 0.03% 147 0.39 ms
SKYLARK_USER_FN 0.20% 343 1.08 ms
As far as my command, I am running
bazel build src:MY_TARGET --embed_label MY_LABEL --stamp --show_loading_progress
Use the --host_jvm_debug startup flag to debug Bazel itself during a build.
From https://bazel.build/contributing.html:
Debugging Bazel
Start creating a debug configuration for both C++ and
Java in your .bazelrc with the following:
build:debug -c dbg
build:debug --javacopt="-g"
build:debug --copt="-g"
build:debug --strip="never"
Then you can rebuild Bazel with bazel build --config debug //src:bazel and use your favorite debugger to start debugging.
For debugging the C++ client you can just run it from gdb or lldb as
you normally would. But if you want to debug the Java code, you must
attach to the server using the following:
Run Bazel with debugging option --host_jvm_debug before the command (e.g., bazel --batch --host_jvm_debug build //src:bazel).
Attach a debugger to the port 5005. With jdb for instance, run jdb -attach localhost:5005. From within Eclipse, use the remote
Java application launch configuration.
Our IntelliJ plugin has built-in debugging support
I have cluster RHEL6,
cman, corosync, pacemaker.
After adding new memebers I got error in GFS mounting. GFS never mounts on servers.
group_tool
fence domain
member count 4
victim count 0
victim now 0
master nodeid 1
wait state none
members 1 2 3 4
dlm lockspaces
name clvmd
id 0x4104eefa
flags 0x00000000
change member 4 joined 1 remove 0 failed 0 seq 1,1
members 1 2 3 4
gfs mountgroups
name lv_gfs_01
id 0xd5eacc83
flags 0x00000005 blocked,join
change member 3 joined 1 remove 0 failed 0 seq 1,1
members 1 2 3
In processes:
root 2695 2690 0 08:03 pts/1 00:00:00 /bin/bash /etc/init.d/gfs2 start
root 2702 2695 0 08:03 pts/1 00:00:00 /bin/bash /etc/init.d/gfs2 start
root 2704 2703 0 08:03 pts/1 00:00:00 /sbin/mount.gfs2 /dev/mapper/vg_shared-lv_gfs_01 /mnt/share -o rw,_netdev,noatime,nodiratime
fsck.gfs2 -yf /dev/vg_shared/lv_gfs_01
Initializing fsck
jid=1: Replayed 0 of 0 journaled data blocks
jid=1: Replayed 20 of 21 metadata blocks
Recovering journals (this may take a while)
Journal recovery complete.
Validating Resource Group index.
Level 1 rgrp check: Checking if all rgrp and rindex values are good.
(level 1 passed)
RGs: Consistent: 183 Cleaned: 1 Inconsistent: 0 Fixed: 0 Total: 184
2 blocks may need to be freed in pass 5 due to the cleaned resource groups.
Starting pass1
Pass1 complete
Starting pass1b
Pass1b complete
Starting pass1c
Pass1c complete
Starting pass2
Pass2 complete
Starting pass3
Pass3 complete
Starting pass4
Pass4 complete
Starting pass5
Block 11337799 (0xad0047) bitmap says 1 (Data) but FSCK saw 0 (Free)
Fixed.
Block 11337801 (0xad0049) bitmap says 1 (Data) but FSCK saw 0 (Free)
Fixed.
RG #11337739 (0xad000b) free count inconsistent: is 65500 should be 65502
RG #11337739 (0xad000b) Inode count inconsistent: is 15 should be 13
Resource group counts updated
Pass5 complete
The statfs file is wrong:
Current statfs values:
blocks: 12057320 (0xb7fae8)
free: 9999428 (0x989444)
dinodes: 15670 (0x3d36)
Calculated statfs values:
blocks: 12057320 (0xb7fae8)
free: 9999432 (0x989448)
dinodes: 15668 (0x3d34)
The statfs file was fixed.
Writing changes to disk
gfs2_fsck complete
gfs2_edit -p 0xad0047 field di_size /dev/vg_shared/lv_gfs_01
10 (Block 11337799 is type 10: Ext. attrib which is not implemented)
Howto drop flag blocked,join from GFS?
I solved it by reboot all servers which have GFS,
it is one of the unpleasant behavior of GFS.
GFS lock based on kernel and in the few cases it solved only with reboot.
there is very usefull manual - https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Global_File_System_2/index.html
I have a problem with jenkins. It reports that the build of my symfony2 project ist successful, but directly after that it crashes with
ERROR: Publisher hudson.plugins.violations.ViolationsPublisher aborted due to exception
java.io.EOFException: input contained no data
I have no clue why and I don't find any helpfull on the net for this error.
It apperead out of the blue. yesterday the build was done correctly.
here is my log:
jslint:
[exec] Result: 1
build:
BUILD SUCCESSFUL
Total time: 13 minutes 21 seconds
[CHECKSTYLE] Collecting checkstyle analysis files...
[CHECKSTYLE] Finding all files that match the pattern build/logs/checkstyle.xml
[CHECKSTYLE] Parsing 1 files in /var/lib/jenkins/jobs/DEVELOPMENT/workspace
[CHECKSTYLE] Successfully parsed file /var/lib/jenkins/jobs/DEVELOPMENT/workspace/build/logs/checkstyle.xml of module with 3 warnings.
[CHECKSTYLE] Computing warning deltas based on reference build #54
[DRY] Collecting duplicate code analysis files...
[DRY] Finding all files that match the pattern build/logs/pmd-cpd.xml
[DRY] Parsing 1 files in /var/lib/jenkins/jobs/DEVELOPMENT/workspace
[DRY] Successfully parsed file /var/lib/jenkins/jobs/DEVELOPMENT/workspace/build/logs/pmd-cpd.xml of module with 207 warnings.
[DRY] Computing warning deltas based on reference build #54
[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing PHPUnit-3.x (default)
[xUnit] [INFO] - [PHPUnit-3.x (default)] - 1 test report file(s) were found with the pattern 'build/logs/junit.xml' relative to '/var/lib/jenkins/jobs/DEVELOPMENT/workspace' for the testing framework 'PHPUnit-3.x (default)'.
[xUnit] [INFO] - Check 'Failed Tests' threshold.
[xUnit] [INFO] - Check 'Skipped Tests' threshold.
[xUnit] [INFO] - Setting the build status to SUCCESS
[xUnit] [INFO] - Stopping recording.
[JDepend] JDepend plugin is ready
[JDepend] Found 894 classes in 253 packages
ERROR: Publisher hudson.plugins.violations.ViolationsPublisher aborted due to exception
java.io.EOFException: input contained no data
at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:3003)
at org.xmlpull.mxp1.MXParser.more(MXParser.java:3046)
at org.xmlpull.mxp1.MXParser.parseProlog(MXParser.java:1410)
at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1395)
at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
at hudson.plugins.violations.parse.AbstractParser.expectNextTag(AbstractParser.java:262)
at hudson.plugins.violations.types.pmd.PMDParser.execute(PMDParser.java:39)
at hudson.plugins.violations.parse.AbstractTypeParser.parse(AbstractTypeParser.java:57)
at hudson.plugins.violations.ViolationsCollector.doType(ViolationsCollector.java:187)
at hudson.plugins.violations.ViolationsCollector.invoke(ViolationsCollector.java:114)
at hudson.plugins.violations.ViolationsCollector.invoke(ViolationsCollector.java:25)
at hudson.FilePath.act(FilePath.java:906)
at hudson.FilePath.act(FilePath.java:879)
at hudson.plugins.violations.ViolationsPublisher.perform(ViolationsPublisher.java:74)
at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
at hudson.model.Build$BuildExecution.post2(Build.java:183)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:726)
at hudson.model.Run.execute(Run.java:1618)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:247)
As I see, the problem comes at a post-build action! Maybe, you don't have any data in the input file for JDepend, as I see from the error log. You must see if you give the correct path to the file and also, you must be sure that the file have some content... maybe the path is correct but the file is empty...