I'm running E2E test using some bazel test target (the scala flavour of java_test).
In Maven I used to dump logs to target/logs folder that was created during test time, and then if something failed - I could have looked in this folder and find the logs.
In bazel - what path can I put in my test logs configuration so it would be writable and conveniently available upon test finish / test failure?
I know that the java.io.tmp dir is writable but gets deleted immediately after test finishes.
So digging through bazel docs I found this:
https://docs.bazel.build/versions/master/test-encyclopedia.html#initial-conditions
Seems like I can read env variable "TEST_UNDECLARED_OUTPUTS_DIR" and it will give me a writable path. Anything I write there would be zipped and saved under ./bazel-out/darwin-fastbuild/testlogs/<package-name>/<target-name>/test.outputs/outputs.zip
Pretty cool!
You can set a writable path for bazel test by using the environment variable TEST_TMPDIR="<.file directory.>".
Related
For years I've been running a Docker container on my local machine and using it as a remote Python interpreter via SSH in PyCharm. This works great (though 2022.2.1 brought a lot of new bugs that have been slowly being ironed out) for running my code! I'm now on 2022.2.3.
However, I'm having issues running unit tests. In the past (i.e. before version 2022.2.1), I could simply right click my tests directory (a direct child of my main project directory) and click Run Python tests in test... and it would all work as expected.
Now, though, when I click this, I receive an error message about "No such file or directory."
I've tried everything I can think of- I've setup my path mappings in the Python test run config to exactly match those shown in my Python run config, and have tried every version of directory and subdirectory in the mappings and working directory, but I always receive an error about either having an empty test suite (no tests found), or that the directory "must be in the project."
It seems like no matter what I do, PyCharm is trying to create a temp directory somewhere, or is trying to read from some temp directory that I never specified, because I see errors this like:
AssertionError: /tmp/pycharm_project_405/docker/tests: No such file or directory
Yet I never created, specified, or requested a temp directory of any sort, let alone one named /tmp/pycharm_project_405/; this is a mystery to me.
PyCharm with an SSH interpreter is rapidly becoming unusable for me and my team because we cannot figure out how to set this up. Can anybody please offer some guidance on what we need to do?
Thank you all so very much!
I tried:
Changing run config for Python tests to match the working directory and path mapping of Python run configs (which work)
Directly specifying the path to the tests from the container's perspective
Setting up run config templates
Specifying one directory up/down from the actual tests
Expected:
Unit tests to be found and run as they were in previous versions of PyCharm
Answer
Create a run config for testing
In the testing run config, set Target: to Custom
Set the correct remote interpreter
Set Working directory to the test folder
Set TWO path mappings: 1) Map the code directory (in my case, the parent directory of the tests folder) and 2) Map the test directory itself
Voila!!!
I am trying to run rego test cases as part of jenkins pipeline run. I previously had my test case and the input data for the test case in a single file and they all ran successfully.
I modified my test cases to read the data from an external json file and the current folder structure is as follows: test.rego file and input.json file in same folder.
This is what I use in Jenkins to run the tests *./opa test -b -v .*
The tests are failing with the error data.utilities.test_no_OwnerContact: FAIL
I am assuming this is because the rego test is unable to read the data from input.json
When I am trying to pass the input file explicitly using -i/--input, I see an error in jenkins as Error: unknown flag: --input
I am not sure if I am using the required tags or if it is something else.
Any help would be appreciated.
Thanks!
Edit: I printed the files in the bundle, and I see the data.json file present. So the OPA engine is not able to read the data file from the Bundle. How can I fix this issue?
The way you'd specify input data in tests is not with an input.json file, but by mocking the input using the with .. as construct. An example test mocking input might look something like this:
test_with_mocked_input {
allow with input as {
"deployment_name": "testing",
"replicas": 2
}
}
See the docs on data mocking for further info.
I have just created a .env file to separate my environment variables from my main docker-compose file. I can run this document on my local machine fine with no errors or issues but when I try run it through my CD pipeline I get the following error.
[error]Top level object in 'C:\BuildAgent_work\r38\a\"Myproject Name"\drop\ .env' needs to be an object not 'class 'str'.
I first thought this was because I had set up my build/CI process wrong but I have played around with it and have had no luck.
I have also done some research online to find others with the same problem but none relate to DevOps in anyway so it has been unhelpful
I am not sure how to reproduce this problem but if anyone knows I can try provide some of my code if needed
Edit:
Here is a snippet of my .env file. Check comment below for my thoughts
ContainerInfrastructure_Version=6.7.93-beta.1
ContainerInfrastructureCore_Version=6.7.41-beta.1
AuthenticationWebService_Version=6.7.52-beta.1
CRM_Version=6.7.52-beta.1
Expected result:
Deploys successfully
What I'm getting during the docker-compose task:
[error]Top level object in 'C:\BuildAgent_work\r38\a\Goldpine.ReleaseManagement\drop.env' needs to be an object not 'class 'str'.
Ok so I figured it out. I'm not sure how to explain this briefly but I'll do my best.
So the problem was within DevOps itself not my code. It turns out a .env file only works/gets picked up if you run the docker-compose command from within the working directory that the docker-compose.yml file exists.
When it went into DevOps it was not running the command from within the downloaded artefact directory but by creating a path to it using the -f command.
So long story short, If you use a .env file you need to set the working directory within the CD pipeline to your artefact folder for it to be able to see the .env file correctlly.
I hope this is clear enough if not just let me know and I'll try change it accordingly :)
According to How does AOSP 9.0 build system link the executable? and What does # mean in this clang command in AOSP build log? , when linking a module, AOSP seems to produce a .rsp file that contains all the obj files that the module need,and pass the file name as a parameter to the link command, for example:
prebuilts/clang/host/linux-x86/clang-4691093/bin/clang++ /OpenSource/Build/Android/9.0.0_r30/soong/.intermediates/bionic/libc/crtbegin_so/android_x86_64_core/crtbegin_so.o #/OpenSource/Build/Android/9.0.0_r30/soong/.intermediates/frameworks/base/libs/hwui/libhwui/android_x86_64_core_shared/libhwui.so.rsp ......
But the .rsp files seems to be removed after build.
The question is, how are these file generated and how to get these files? This may require to learn and modify the build scripts which is out of reach for me.
There maybe the answer for you, read the ninja build manual , in that manual .rsp file mentioned.
https://ninja-build.org/manual.html
the following is info that I copy out.
rspfile, rspfile_content
if present (both), Ninja will use a response file for the given command, i.e. write the selected string (rspfile_content) to the given file (rspfile) before calling the command and delete the file after successful execution of the command.
This is particularly useful on Windows OS, where the maximal length of a command line is limited and response files must be used instead.
I have a test which tries to start an embedded mysql.
It tries to write a socket file to Files.createTempFile(null, ".sock") which returns with the following error:
mysql start failed with error: [ERROR] The socket file path is too long (> 103): /private/var/tmp/_bazel_user/1c8ed8d84f6cb79483aa3cc4da758c86/bazel-sandbox/2478112867584790357/execroot/some_workspace/_tmp/dfebe48cda4dfdc8739653efedfa4933/394798020705754292.sock.
I worked around it by re-pointing java temp dir to /tmp using jvm_flags but this doesn't work when I try to use sandboxing since I guess the test isn't allowed to write there.
I've tried setting a symbolic link from the java code (like so Files.createSymbolicLink(Paths.get("/tmp/foo"),Paths.get(System.getProperty("java.io.tmpdir"))) but this doesn't seem to help.
I've also tried setting the output_base but that didn't help either.
Would really appreciate pointers and tips since I currently can't run my tests under sandbox and so can't parallelize them.
I think your approach of re-pointing Java to /tmp should work. The macOS sandbox always allows writing to a number of directories and /tmp and /private/tmp are included in this set. I tried to reproduce the failure with a genrule: genrule(name = "write_to_tmp", outs = ["out.txt"], cmd = "touch /tmp/something.sock && touch $#"), which works fine and creates the file /tmp/something.sock.
I think in general using /tmp should work fine, although it does seem to be a bit unusual on macOS. $TMPDIR is set to a user-specific folder with a randomized name underneath /var/folders by the OS and it seems to be generally encouraged to use that instead of /tmp. But if you know what you're doing, I don't see a real problem.
Note that we don't have tmpfs or similar mechanisms available on macOS, so we can't automatically guarantee that your usage of writable folders like /tmp by actions is hermetic, won't leak state between runs or that file names won't conflict. Make sure to generate file names in a secure way via mkstemp or similar.
In a bazel test, you can use the TEST_TMPDIR environment variable for a test-private writeable area.
See https://docs.bazel.build/versions/master/test-encyclopedia.html