I have a complex config search path consisting of multiple locations where each location looks similar to this:
├── conf
│ └── foo
│ ├── foo.yaml
│ └── bar.yaml
└── files
├── foo.txt
└── bar.txt
with foo.yaml:
# #package _group_
path: "../../files/foo.txt"
and bar.yaml:
# #package _group_
path: "../../files/bar.txt"
Now the problem is: how do I find the correct location of the files specified in the configurations? I am aware of the to_absolute_path() method provided by hydra, but it interprets the path relative to the directory in which the application was started. However, I would like to interpret that path relative to the position of the configuration file. I cannot do this manually in my code, because I don't know how hydra resolved the configuration file and where exactly it is used to.
Is there some mechanism to determine the location of a config file from hydra? I really want to refrain from putting hard coded absolute paths in my configurations.
You can't get the path of a config file. In fact, it may not be a file at all (such as the case for Structured Configs), or it can be inside a python wheel (even in a zipped wheel).
You can do something like
path = os.path.join(os.path.dirname(__file__), "relative_path_from_config")
You can use also APIs designed for loading resources files from Python modules.
Here is a good answer in the topic.
Related
With Bazel I'm building an external library using http_archive together with
some patches which bring additional features:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name="some-lib",
build_file="#my-project//some-lib:BUILD.some-lib",
url="https://example/download/some-lib.tar.gz",
sha256="8d9405baf113a9f25e4fb961d56f9f231da02e3ada0f41dbb0fa4654534f717b",
patches=[
"//some-lib/patches:01-add-additional-features.dif",
],
patch_args=["-p1"],
patch_tool="patch",
)
The file structure looks like this:
some-lib
├── BUILD
├── BUILD.some-lib
├── include/additional_features.h
├── some-lib.bzl
└── patches
├── 01-add-additional-features.dif
└── BUILD
This basically works but I still struggle with adding include/additional_features.h
into the extracted source folder.
I first tried to just list the file in the filegroup I use to later run
configure_make like this:
filegroup(
name="all_srcs",
srcs=glob(["**"]) + ["#my-project//some-lib:include/additional_features.h"],
)
then I'm getting
no such target '//some-lib:includes/additional_features.h': target 'includes/additional_features.h' not declared in package 'some-lib'; however, a source file of this name exists.
My next idea was to use the tools http_archive provides to make the file part of the source folder.
While you can use patches to modify the extracted folder you'd need a dedicated
dif file just to create the extra header file, which then you would have to
create in advance (i.E. create a Bazel rule) and declare it a dependency etc..
which I'd like to avoid to keep things simple.
There is also patch_cmds which next to patches can be used to modify the
extracted source folder by running arbitrary bash commands so I tried something like this:
patch_cmds=[
"cp #my-project//some-lib:include/additional_features.h include/",
],
but this does not work for me, I'm getting
Error in fail: Error applying patch command cp #my-project//some-lib:include/additional_features.h include/:
cp: cannot stat '#my-project//some-lib:include/additional_features.h': No such file or directory
So it looks like the syntax for specifying a path like I do with build_file does
not work with patch_cmds or that file can't be accessed at that specific stage.
Does one of the approaches I tried actual work and I just didn't use the right
syntax?
What's the Bazel-way to add (a bunch of) readily available files (i.e. in the same
repository as the Bazel-rules I provide) to a http_archive based source directory?
Try putting exports_files(["include/additional_features.h"], visibility=["//visibility:public"]) in some-lib/BUILD, so that Bazel will let you reference source files from the //some-lib package in the external repository (and elsewhere).
I even thought the "no such target" error message suggested exports_files?
Here is my issue: I want to include/exclude specific file when building a service called django. But when playing with Dockerfile.dockerignore file, I do not succeed what I want and I can't figure out which syntax to adopt in order to do so.
Repo structure is the following:
.
└── root_dir
├── django
│ ├── Dockerfile
│ ├── Dockerfile.ignore
│ └── stuff
├── documentation
│ └── stuff
├── useless dir
└── another useless dir
My docker-compose.yml used to make the build is the following:
version: '3.5'
services:
[...]
django:
build:
context: $ROOT_DIR
dockerfile: $ROOT_DIR/django/Dockerfile
[...]
As you can see, context is root_dir and not django (because I need to copy a few things from documentation dir at build time).
What I need when building my image:
django rep
documentation rep
What I want to ignore when building my image:
everything but django and documentation dirs
Dockerfile and Dockerfile.dockerignore
and a few other things (virtual env, *.pyc etc, but we'll stick with Dockerfile and Dockerfile.dockerignore for that question!)
What my Dockerfile.dockerignore looks like in order to do so:
# Ignore everything
*
# Allows django and documentation rep (context is root of project)
!/django
!/documentation
# Ignoring some files
**/Dockerfile
**/Dockerfile.dockerignore
but my issue is that the files I want to ignore are still present in my image! And this is not good for me. I think I tried any syntax possible (**/, */*/......) but I can't find one that suits my need.
Many thanks if you can help me with that!
EDIT: for those wondering about that Dockerfile.dockerignore file, you can have a look here: https://docs.docker.com/engine/reference/commandline/build/#use-a-dockerignore-file
Docker ignore file name should be .dockerignore
You can check the documentation here https://docs.docker.com/engine/reference/builder/#dockerignore-file
Running this command before the 'docker build' fixed it for me:
export DOCKER_BUILDKIT=1
FYI: I did not need to install 'buildkit' first.
I'm doing a small F# program where I am reading a markdown file, and store it as a string:
open System.IO
[<EntryPoint>]
let main argv =
let text = File.ReadAllText("/home/ask/RiderProjects/ConsoleApp1/ConsoleApp1/File.md")
printfn "%s" text
0
It works like it is here, where I provide the full path. But I need to use a relative path, since I need to put this on a server at some point.
My Program.fs and File.md is in the same directory, So i've tried all these paths, that have'nt worked.
- File.md
- /File.md
- ~/File.md
NOthing works. How do I provide the correct path?
I'm running ubuntu
EDIT
I accidentally wrote "txt" as the extension for some of my examples above. This is now corrected.
The directory that I am working in looks like this:
├── ConsoleApp1
│ ├── bin
│ ├── ConsoleApp1.fsproj
│ ├── File.md
│ ├── obj
│ └── Program.fs
└── ConsoleApp1.sln
Program.fs is the F# file, with my code in. File.md is the file I want to read
One of the more useful utilities is __SOURCE_DIRECTORY__ so you can reference things like files relative to the source file's directory. So just a simple Path.Join will get you the file.
let mdPath = Path.Join(__SOURCE_DIRECTORY__, "File.md")
I've an external dependency declared in WORKSPACE as a new_git_repository and provided a BUILD file for it.
proj/
├── BUILD
├── external
│ ├── BUILD.myDep
│ └── code.bzl
└── WORKSPACE
in the BUILD.myDep file, I want to load code.bzl nearby, but when I load it (load("//:external/code.bzl", "some_func")) bazel tries to load #myDep//:external/code.bzl instead!
Of course it's not a target in #myDep repository, but in my local worksapce.
Seems I Rubber Duck ed the Stackoverflow. since the solution appeared when writing the question!
However, the solution is to explicitly mention the local workspace when loading the .bzl file:
Suppose we have declared the name in the WORKSPACE as below:
workspace(name = "local_proj")
Now instead of load("//:external/code.bzl", "some_func"), just load it explicitly as a local workspace file:
load("#local_proj//:external/code.bzl", "some_func")
NOTE: When using this trick just be careful about potential dependency loops (i.e. loading a generated file that itself is produced by a rule depending on the same external repo!)
In the blog How to generate language fragment bundles to localize Carbon products by Tanya Madurapperuma, I am having the following problem. Once generated the language bundles with ant localize command, these bundles are generated in the CARBON_HOME/repository/components/dropins/ folder. The problem is that when I run the tool I'm not looking to change the language to Spanish. I would appreciate help to correct what I may be missing to do?
Note: All resources.properties files are translated into Spanish.
If you have the jars with translated resources.properties files in you dropins folder, you need to restart the server and set the Locale setting of your browser to Spanish.
Locale should be changed in the browser, and then the server will pick the matching resources files to use.
UPDATE:
There are some problems here.
First, there's a bug if you have multiple directories in /resources directory. For now, you can make sure that you have only one directory inside resources directory when you run localize task.
You should have the properties files inside a directory with the bundle name, without the tree structure. So your resources directory should look like this.
../resources/
└── org.wso2.carbon.i18n_4.2.0
├── JSResources_es.properties
└── Resources_es.properties
You need to include the locale code as _es in your files as shown above.
Also the localize tool seems to append i18n at the end of the folder structure of the built jar. This works with ui bundles but in the case of org.wso2.carbon.i18n it looks as org/wso2/carbon/i18n/i18n. So open the built jar in dropins folder and remove the extra i18n folder so that the jar tree structure looks like following.
../repository/components/dropins/org.wso2.carbon.i18n.languageBundle_4.2.0.jar
├── META-INF
│ └── MANIFEST.MF
└── org
└── wso2
└── carbon
└── i18n
├── JSResources_es.properties
└── Resources_es.properties
Did you get this to work?
The place I doubt that you might have gone wrong is the folder structure in the resources folder. (You can place your resource files anywhere and execute command as ant localize -Dresources.directory=path_to_your_resources_directory)
Also note that a resource folder should have the proper naming conventions of the osgi bundle.
Ex: org.wso2.carbon.claim.mgt.ui_4.2.0 (This entire thing is name of the folder)
If you still couldn't get this to work mail me your resources folder to tanyamadurapperuma#gmail.com