I'm trying to create a very simple copy task but it seems grunt-copy is not made for my usecase :-(.
All I needed is the following:
Source directory structure:
vendor
L subdir1
L assets
L subdir2
L subdir3
L file1.js
L file2.js
L subdir4
L subdir5
L subdir6
L file3.js
config:
src: [
'vendor/subdir1/assets/**/*',
'vendor/subdir4/**/*'
],
dest: 'build/assets/'
Destination file structure:
build
L assets
L subdir2
L subdir3
L file1.js
L file2.js
L subdir5
L subdir6
L file3.js
As you can see I want to preserve directory structure, but omit the basepath in src. Normally this can be done with cwd. But actually for this case I would need multiple cwds.
Sounds like a fair requirement to me. Is there really no easy solution?
Thanks for help,
You can use more comma separated 'file' definitions in your Gruntfile.js:
copy: {
dist: {
files: [
{
src: ['assets/**/*'],
dest: 'build/assets/',
cwd: 'vendor/subdir1'
},
{
src: ['**/*'],
dest: 'build/assets/',
cwd: 'vendor/subdir4/'
}]
}
}
In Gruntfile you then run for example this task:
grunt.registerTask('build', [
copy:dist
]);
And then run the task grunt build
Related
Suppose I am writing a custom Bazel rule for foo-compiler.
The user provides a list of source-files to the rule:
foo_library(
name = "hello",
srcs = [ "A.foo", "B.foo" ],
)
To build this without Bazel, the steps would be:
Create a config file config.json that lists the sources:
{
"srcs": [ "./A.foo", "./B.foo" ]
}
Place the config alongside the sources:
$ ls .
A.foo
B.foo
config.json
Call foo-compiler in that directory:
$ foo-compiler .
Now, in my Bazel rule implementation I can declare a file like this:
config_file = ctx.actions.declare_file("config.json")
ctx.actions.write(
output = config_file,
content = json_for_srcs(ctx.files.srcs),
)
The file is created and it has the right content.
However, Bazel does not place config.json alongside the srcs.
Is there a way to tell Bazel where to place the file?
Or perhaps I need to copy each source-file alongside the config?
You can do this with ctx.actions.symlink e.g.
srcs = []
# Declare a symlink for each src files in the same directory as the declared
# config file.Then write that symlink.
for f in ctx.files.srcs:
src = ctx.actions.declare_file(f.basename)
srcs.append(src)
ctx.actions.symlink(
output = src,
target_file = f,
)
config_file = ctx.actions.declare_file("config.json")
ctx.actions.write(
output = config_file,
content = json_for_srcs(ctx.files.srcs),
)
# Run compiler
ctx.actions.run(
inputs = srcs + [config_file],
outputs = # TODO: Up to you,
tools = [ctx.file.__compiler], #TODO: Update this to match your rule.
command = ctx.file.__compiler.path,
args = ["."],
#...
)
Note that when you return your provider that you should only return the result of your compilation not the srcs. Otherwise, you'll likely run into problems with duplicate outputs.
I'm making a haskell program and I'm setting buildInput like this to include pkgs.ffmpeg-full:
(myHaskellPackages.callCabal2nix "App" (./.) {}).overrideAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.ffmpeg-full ];
})
However this seems to make the ffmpeg package accessible during build time only rather than runtime of the application.
What attribute do I need to set for ffmpeg-full to be available during runtime - being able to invoke the ffmpeg executable?
There is a section about runtime dependencies in nix pills but I don't understand that section, it doesn't make sense how it can always determine runtime dependencies by hashes alone? I mean if I reference an executable in a shell script - surely nix does not parse the shell script to determine the executable I reference. https://nixos.org/guides/nix-pills/automatic-runtime-dependencies.html#idm140737320205792
Something is different for runtime dependencies however. Build
dependencies are automatically recognized by Nix once they are used in
any derivation call, but we never specify what are the runtime
dependencies for a derivation.
There's really black magic involved. It's something that at first
glance makes you think "no, this can't work in the long term", but at
the same time it works so well that a whole operating system is built
on top of this magic.
In other words, Nix automatically computes all the runtime
dependencies of a derivation, and it's possible thanks to the hash of
the store paths.
default.nix:
{
ghc ? "ghc8106",
pkgs ? import <nixpkgs> {}
}:
with pkgs.haskell.lib;
let
haskellPkgs = pkgs.haskell.packages.${ghc};
inherit (pkgs) lib;
mySourceRegexes = [
"^app.*$"
"^.*\\.cabal$"
"package.yaml"
];
myApp = (haskellPkgs.callCabal2nix "my-hello"
(lib.sourceByRegex ./. mySourceRegexes) { });
in myApp
.overrideAttrs(
oa: {
nativeBuildInputs = oa.nativeBuildInputs ++ [pkgs.hello pkgs.makeWrapper];
installPhase = oa.installPhase + ''
ln -s ${pkgs.hello.out}/bin/hello $out/bin/hello
'';
postFixup = ''
wrapProgram $out/bin/x-exe --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.hello ]}
'';
})
src/Main.hs:
module Main where
import System.Process (callCommand)
main :: IO ()
main = do
putStrLn "HELLO"
callCommand "hello"
putStrLn "BYE"
Seems this is not directly supported with an explicitly stated list of dependencies. However we can indirectly achieve this with "wrapping".
I found more information about wrapping here: https://nixos.wiki/wiki/Nix_Cookbook#Wrapping_packages
So I can do a ls that references the package.
...
appPkg = (myHaskellPackages.callCabal2nix "HaskellNixCabalStarter" (./.) {}).overrideAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.ffmpeg-full ];
});
in
(pkgs.writeScriptBin "finderapp" ''
#!${pkgs.stdenv.shell}
ls ${pkgs.ffmpeg-full}/bin/ffmpeg
exec ${appPkg}/bin/app
''
)
We can verify the built package(?) correctly depends on the appropriate with:
nix-store -q --references result
/nix/store/0cq84xic2absp75ciajv4lfx5ah1fb59-ffmpeg-full-4.2.2
/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23
/nix/store/wlvnjx53xfangaa4m5rmabknjbgpvq3d-HaskellNixCabalStarter-0.1.0.0
I would like to restrict the number of files to include in my build src to a select few.
Directly passing the list of files to src or srcs isn't allowed as an archive or a directory is expected.
I couldn't find a function to do this and builtins.filterSource does not seem to work for me (no idea why -- the intermediate derivation containing the filtered files ends up empty):
files = [
./Cargo.toml
./Cargo.lock
./cpu.rs
];
src = builtins.filterSource (p: t: builtins.elem p files) ./.;
Note: I'm using the rustPlatform builder but I don't think it matters.
filterSource passes the full path as a string.
in order to compare the paths of your list with it you need to convert the string to a path:
$ cd /Users/fghibellini/code/nix
$ nix-instantiate --eval -E './a == "/Users/fghibellini/code/nix/a"'
false
$ nix-instantiate --eval -E './a == (/. + "/Users/fghibellini/code/nix/a")'
true
i.e. the following code should work fine:
files = [
./Cargo.toml
./Cargo.lock
./cpu.rs
];
src = builtins.filterSource (p: t: builtins.elem (/. + p) files) ./.;
You can use builtins.typeOf and builtins.trace to debug such issues.
I have a directory like this:
assigment
|
|__ src
| |
| |__ Main.scala
|
|
|__ testcase
|
|__ Simple.in
In Main.scala, Simple.in is read by Source.fromFile():
val inputFile = "Simple.in"
val lines = Source.fromFile("./testcase/" + inputFile).getLines
But when I run Main.scala in sbt the FileNoutFoundException appear. When I change the path to "../testcase/" + inputFile then it works fine. The original path is from my teacher, so I wonder which path is actually correct? Oh, I'm using Linux btw...
./ means: the current path
../ means: the directory "above" the current directory
Thus: when you run your Scala class from "src", "./testcase" makes it look for a directory testcase within "src"; or using full path names:
"assignment/src/" + "./testcase" turns into "assignment/src/testcase"
Whereas, when you use
"assignment/src/" + "../testcase" turns into "assignment/testcase"
therefore, the version with ".." finds the valid path. That is all the magic here!
. => current dir
.. => one above curren dir
But standard way to access resources is using the resources folder of sbt project structure.
This way helps you to access files independent of where (which class) you are accessing the resource in the code.
Folder to put your files
src/main/resources
val stream : InputStream = getClass.getResourceAsStream("/readme.txt")
val lines = scala.io.Source.fromInputStream( stream ).getLines
I need to copy some files to binary directory while preserving their names. What I've got so far:
filegroup(
name = "resources",
srcs = glob(["resources/*.*"]),
)
genrule(
name = "copy_resources",
srcs = ["//some/package:resources"],
outs = [ ],
cmd = "cp $(SRCS) $(#D)",
local = 1,
output_to_bindir = 1,
)
Now I have to specify file names in outs but I can't seem to figure out how to resolve the labels to obtain the actual file names.
To make a filegroup available to a binary (executed using bazel run) or to a test (when executed using bazel test) then one usually lists the filegroup as part of the data of the binary, like so:
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
data = [
"//your_project/other/deeply/nested/resources:other_test_files",
],
)
# known to work at least as of bazel version 0.22.0
Usually the above is sufficient.
However, the executable must then recurse through the directory structure "other/deeply/nested/resources/" in order to find the files from the indicated filegroup.
In other words, when populating the runfiles of an executable, bazel preserves the directory nesting that spans from the WORKSPACE root to all the packages enclosing the given filegroup.
Sometimes, this preserved directory nesting is undesirable.
THE CHALLENGE:
In my case, I had several filegroups located at various points in my project directory tree, and I wanted all the individual files of those groups to end up side-by-side in the runfiles collection of the test binary that would consume them.
My attempts to do this with a genrule were unsuccessful.
In order to copy individual files from multiple filegroups, preserving the basename of each file but flattening the output directory, it was necessary to create a custom rule in a bzl bazel extension.
Thankfully, the custom rule is fairly straightforward.
It uses cp in a shell command much like the unfinished genrule listed in the original question.
The extension file:
# contents of a file you create named: copy_filegroups.bzl
# known to work in bazel version 0.22.0
def _copy_filegroup_impl(ctx):
all_input_files = [
f for t in ctx.attr.targeted_filegroups for f in t.files
]
all_outputs = []
for f in all_input_files:
out = ctx.actions.declare_file(f.basename)
all_outputs += [out]
ctx.actions.run_shell(
outputs=[out],
inputs=depset([f]),
arguments=[f.path, out.path],
# This is what we're all about here. Just a simple 'cp' command.
# Copy the input to CWD/f.basename, where CWD is the package where
# the copy_filegroups_to_this_package rule is invoked.
# (To be clear, the files aren't copied right to where your BUILD
# file sits in source control. They are copied to the 'shadow tree'
# parallel location under `bazel info bazel-bin`)
command="cp $1 $2")
# Small sanity check
if len(all_input_files) != len(all_outputs):
fail("Output count should be 1-to-1 with input count.")
return [
DefaultInfo(
files=depset(all_outputs),
runfiles=ctx.runfiles(files=all_outputs))
]
copy_filegroups_to_this_package = rule(
implementation=_copy_filegroup_impl,
attrs={
"targeted_filegroups": attr.label_list(),
},
)
Using it:
# inside the BUILD file of your exe
load(
"//your_project:copy_filegroups.bzl",
"copy_filegroups_to_this_package",
)
copy_filegroups_to_this_package(
name = "other_files_unnested",
# you can list more than one filegroup:
targeted_filegroups = ["//your_project/other/deeply/nested/library:other_test_files"],
)
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
data = [
":other_files_unnested",
],
)
You can clone a complete working example here.