I've just started playing with NixOS, and have so far managed to edit /etc/nixos/configuration.nix in my NixOS 18.09 VM to have PHP-FPM and the Caddy webserver enabled.
{ config, pkgs, ... }:
{
imports = [ <nixpkgs/nixos/modules/installer/virtualbox-demo.nix> ];
users = {
mutableUsers = false;
groups = {
caddy = { };
php-project = { };
};
users = {
hello = {
group = "php-project";
};
};
};
environment.systemPackages = [
pkgs.htop
pkgs.httpie
pkgs.php # for PHP CLI
];
services.caddy = {
enable = true;
email = "david#example.com";
agree = true;
config = ''
(common) {
gzip
header / -Server
header / -X-Powered-By
}
:8080 {
root /var/www/hello
fastcgi / /run/phpfpm/hello.sock php
log syslog
import common
}
'';
};
services.phpfpm = {
phpOptions = ''
date.timezone = "Europe/Berlin"
'';
poolConfigs = {
hello = ''
user = hello
listen = /run/phpfpm/hello.sock
; ...
pm.max_requests = 500
'';
};
};
}
A PHP-processed response is available at at localhost:8080. (Yay!)
To enable Caddy plugins when compiling from source, Go imports are added to caddy's run.go, e.g.:
_ "github.com/mholt/caddy/caddyhttp" // plug in the HTTP server type
// This is where other plugins get plugged in (imported)
_ "github.com/nicolasazrak/caddy-cache" // added to use another plugin
)
How can I set such line insertion to be performed after the source is downloaded and before the build takes place? (If this is a reasonable approach when using Nix?)
The NixOS 18.09 caddy package.
The NixOS 18.09 caddy service.
I believe that when writing a package a builder script (Bash or otherwise) can be assigned, and I'm thinking the line insertion could be done in it. But I'm lost as to how to assign a script to an existing package in this situation (override an attribute/use an overlay?) and where to put the script on the disk.
Status update
I've been doing some reading on customising packages in general and it sounds like overlays might be what I need. However, I don't seem to be able to get my overlay evaluated.
I'm using overriding of the package name as a test as it's simpler than patching code.
Overlay attempt 1
/etc/nixos/configuration.nix:
{ config, pkgs, options, ... }:
{
imports = [ <nixpkgs/nixos/modules/installer/virtualbox-demo.nix> ];
nix.nixPath = options.nix.nixPath.default ++ [
"nixpkgs-overlays=/etc/nixos/overlays-compat/"
];
# ...
}
/etc/nixos/overlays-compat/overlays.nix:
self: super:
with super.lib;
let
# Using the nixos plumbing that's used to evaluate the config...
eval = import <nixpkgs/nixos/lib/eval-config.nix>;
# Evaluate the config,
paths = (eval {modules = [(import <nixos-config>)];})
# then get the `nixpkgs.overlays` option.
.config.nixpkgs.overlays
;
in
foldl' (flip extends) (_: super) paths self
/etc/nixos/overlays-compat/caddy.nix:
self: super:
{
caddy = super.caddy.override {
name = "caddy-override";
};
}
Overlay attempt 2
/etc/nixos/configuration.nix:
nixpkgs.overlays = [ (self: super: {
caddy = super.caddy.override {
name = "caddy-override";
};
} ) ];
error: anonymous function at /nix/store/mr5sfmz6lm5952ch5q6v49563wzylrkx-nixos-18.09.2327.37694c8cc0e/nixos/pkgs/servers/caddy/default.nix:1:1 called with unexpected argument 'name', at /nix/store/mr5sfmz6lm5952ch5q6v49563wzylrkx-nixos-18.09.2327.37694c8cc0e/nixos/lib/customisation.nix:69:12
overrideAttrs
I previously managed to override the package name with this:
{ config, pkgs, options, ... }:
let
caddyOverride = pkgs.caddy.overrideAttrs (oldAttrs: rec {
name = "caddy-override-v${oldAttrs.version}";
});
in {
{
# ...
services.caddy = {
package = caddyOverride;
# ...
}
}
I could see in htop that the caddy binary was in a folder called /nix/store/...-caddy-override-v0.11.0-bin/. But I understand that overriding in this way has been superseded by overlays.
In order to add plugins to Caddy, it seems that the method is to modify the source.
You will need to adapt the Nixpkgs expression for Caddy to make that possible. That can be done outside the Nixpkgs tree, using services.caddy.package = callPackage ./my-caddy.nix {} for example, or by forking the Nixpkgs repository and pointing your NIX_PATH to your clone.
There is an issue for Caddy plugins: https://github.com/NixOS/nixpkgs/issues/14671
PR welcome!
This is my first time creating a luarock and writing .rockpec file. I have a small lua script and some static text files that this script requires to use. How should I pack my luarock so that these static files are available for my script?
For example, here is my myscript.rockspeck file:
package = "myscript"
version = "1.0-1"
source = {
url = "https://github.com/me/myscript/raw/master/myscript-1.0.tar.gz",
tag = "v1.0"
}
description = {
summary = "My script.",
detailed = [[
Some lua script.
]],
homepage = "https://github.com/me/myscript",
license = "MIT"
}
dependencies = {
"lua >= 5.1, < 5.4"
}
build = {
type = "builtin",
modules = {
myfun = "src/myscript.lua"
},
copy_directories = {"my_data"}
}
I am able to do :
luarocks pack myscript-1.0-1.rockspec
sudo luarocks install myscript-1.0-1.src.rock
However, upon importing myfun module I can see that the my_data directory with the required files inside is not accessible by the module as it complains accordingly.
When I do laurocks pack my working directory contains myscript-1.0.tar.gz with the following sctructure:
myscript-1.0/
src/myscript.lua
tiny_data/
data1.txt
data2.txt
How should I correctly include my_data static files?
Does anyone know how to create multiple output paths in a webpack.config.js file? I'm using bootstrap-sass which comes with a few different font files, etc. For webpack to process these i've included file-loader which is working correctly, however the files it outputs are being saved to the output path i specified for the rest of my files:
output: {
path: __dirname + "/js",
filename: "scripts.min.js"
}
I'd like to achieve something where I can maybe look at the extension types for whatever webpack is outputting and for things ending in .woff .eot, etc, have them diverted to a different output path. Is this possible?
I did a little googling and came across this *issue on github where a couple of solutions are offered, edit:
but it looks as if you need to know the entry point in able to specify an output using the hash method
eg:
var entryPointsPathPrefix = './src/javascripts/pages';
var WebpackConfig = {
entry : {
a: entryPointsPathPrefix + '/a.jsx',
b: entryPointsPathPrefix + '/b.jsx',
c: entryPointsPathPrefix + '/c.jsx',
d: entryPointsPathPrefix + '/d.jsx'
},
// send to distribution
output: {
path: './dist/js',
filename: '[name].js'
}
}
*https://github.com/webpack/webpack/issues/1189
however in my case, as far as the font files are concerned, the input process is kind of abstracted away and all i know is the output. in the case of my other files undergoing transformations, there's a known point where i'm requiring them in to be then handled by my loaders. if there was a way of finding out where this step was happening, i could then use the hash method to customize output paths, but i don't know where these files are being required in.
Webpack does support multiple output paths.
Set the output paths as the entry key. And use the name as output template.
webpack config:
entry: {
'module/a/index': 'module/a/index.js',
'module/b/index': 'module/b/index.js',
},
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name].js'
}
generated:
└── module
├── a
│ └── index.js
└── b
└── index.js
I'm not sure if we have the same problem since webpack only support one output per configuration as of Jun 2016. I guess you already seen the issue on Github.
But I separate the output path by using the multi-compiler. (i.e. separating the configuration object of webpack.config.js).
var config = {
// TODO: Add common Configuration
module: {},
};
var fooConfig = Object.assign({}, config, {
name: "a",
entry: "./a/app",
output: {
path: "./a",
filename: "bundle.js"
},
});
var barConfig = Object.assign({}, config,{
name: "b",
entry: "./b/app",
output: {
path: "./b",
filename: "bundle.js"
},
});
// Return Array of Configurations
module.exports = [
fooConfig, barConfig,
];
If you have common configuration among them, you could use the extend library or Object.assign in ES6 or {...} spread operator in ES7.
You can now (as of Webpack v5.0.0) specify a unique output path for each entry using the new "descriptor" syntax (https://webpack.js.org/configuration/entry-context/#entry-descriptor) –
module.exports = {
entry: {
home: { import: './home.js', filename: 'unique/path/1/[name][ext]' },
about: { import: './about.js', filename: 'unique/path/2/[name][ext]' }
}
};
If you can live with multiple output paths having the same level of depth and folder structure there is a way to do this in webpack 2 (have yet to test with webpack 1.x)
Basically you don't follow the doc rules and you provide a path for the filename.
module.exports = {
entry: {
foo: 'foo.js',
bar: 'bar.js'
},
output: {
path: path.join(__dirname, 'components'),
filename: '[name]/dist/[name].bundle.js', // Hacky way to force webpack to have multiple output folders vs multiple files per one path
}
};
That will take this folder structure
/-
foo.js
bar.js
And turn it into
/-
foo.js
bar.js
components/foo/dist/foo.js
components/bar/dist/bar.js
Please don't use any workaround because it will impact build performance.
Webpack File Manager Plugin
Easy to install copy this tag on top of the webpack.config.js
const FileManagerPlugin = require('filemanager-webpack-plugin');
Install
npm install filemanager-webpack-plugin --save-dev
Add the plugin
module.exports = {
plugins: [
new FileManagerPlugin({
onEnd: {
copy: [
{source: 'www', destination: './vinod test 1/'},
{source: 'www', destination: './vinod testing 2/'},
{source: 'www', destination: './vinod testing 3/'},
],
},
}),
],
};
Screenshot
If it's not obvious after all the answers you can also output to a completely different directories (for example a directory outside your standard dist folder). You can do that by using your root as a path (because you only have one path) and by moving the full "directory part" of your path to the entry option (because you can have multiple entries):
entry: {
'dist/main': './src/index.js',
'docs/main': './src/index.js'
},
output: {
filename: '[name].js',
path: path.resolve(__dirname, './'),
}
This config results in the ./dist/main.js and ./docs/main.js being created.
In my case I had this scenario
const config = {
entry: {
moduleA: './modules/moduleA/index.js',
moduleB: './modules/moduleB/index.js',
moduleC: './modules/moduleB/v1/index.js',
moduleC: './modules/moduleB/v2/index.js',
},
}
And I solve it like this (webpack4)
const config = {
entry: {
moduleA: './modules/moduleA/index.js',
moduleB: './modules/moduleB/index.js',
'moduleC/v1/moduleC': './modules/moduleB/v1/index.js',
'moduleC/v2/MoculeC': './modules/moduleB/v2/index.js',
},
}
You definitely can return array of configurations from your webpack.config file. But it's not an optimal solution if you just want a copy of artifacts to be in the folder of your project's documentation, since it makes webpack build your code twice doubling the overall time to build.
In this case I'd recommend to use the FileManagerWebpackPlugin plugin instead:
const FileManagerPlugin = require('filemanager-webpack-plugin');
// ...
plugins: [
// ...
new FileManagerPlugin({
onEnd: {
copy: [{
source: './dist/*.*',
destination: './public/',
}],
},
}),
],
You can only have one output path.
from the docs https://github.com/webpack/docs/wiki/configuration#output
Options affecting the output of the compilation. output options tell Webpack how to write the compiled files to disk. Note, that while there can be multiple entry points, only one output configuration is specified.
If you use any hashing ([hash] or [chunkhash]) make sure to have a consistent ordering of modules. Use the OccurenceOrderPlugin or recordsPath.
I wrote a plugin that can hopefully do what you want, you can specify known or unknown entry points (using glob) and specify exact outputs or dynamically generate them using the entry file path and name. https://www.npmjs.com/package/webpack-entry-plus
I actually wound up just going into index.js in the file-loader module and changing where the contents were emitted to. This is probably not the optimal solution, but until there's some other way, this is fine since I know exactly what's being handled by this loader, which is just fonts.
//index.js
var loaderUtils = require("loader-utils");
module.exports = function(content) {
this.cacheable && this.cacheable();
if(!this.emitFile) throw new Error("emitFile is required from module system");
var query = loaderUtils.parseQuery(this.query);
var url = loaderUtils.interpolateName(this, query.name || "[hash].[ext]", {
context: query.context || this.options.context,
content: content,
regExp: query.regExp
});
this.emitFile("fonts/"+ url, content);//changed path to emit contents to "fonts" folder rather than project root
return "module.exports = __webpack_public_path__ + " + JSON.stringify( url) + ";";
}
module.exports.raw = true;
u can do lik
var config = {
// TODO: Add common Configuration
module: {},
};
var x= Object.assign({}, config, {
name: "x",
entry: "./public/x/js/x.js",
output: {
path: __dirname+"/public/x/jsbuild",
filename: "xbundle.js"
},
});
var y= Object.assign({}, config, {
name: "y",
entry: "./public/y/js/FBRscript.js",
output: {
path: __dirname+"/public/fbr/jsbuild",
filename: "ybundle.js"
},
});
let list=[x,y];
for(item of list){
module.exports =item;
}
The problem is already in the language:
entry (which is a object (key/value) and is used to define the inputs*)
output (which is a object (key/value) and is used to define outputs*)
The idea to differentiate the output based on limited placeholder like '[name]' defines limitations.
I like the core functionality of webpack, but the usage requires a rewrite with abstract definitions which are based on logic and simplicity... the hardest thing in software-development... logic and simplicity.
All this could be solved by just providing a list of input/output definitions... A LIST INPUT/OUTPUT DEFINITIONS.
Vinod Kumar's good workaround is:
module.exports = {
plugins: [
new FileManagerPlugin({
events: {
onEnd: {
copy: [
{source: 'www', destination: './vinod test 1/'},
{source: 'www', destination: './vinod testing 2/'},
{source: 'www', destination: './vinod testing 3/'},
],
},
}
}),
],
};
Following this example, I've just created a rockspec for a rock with just .lua files. I don't need to build anything, so I set the build option to
build = {
type = "none",
install = {
lua = {
"a.lua",
"b.lua",
...
}
}
}
When I run luarocks make it works. However, I noticed that all the files are dumped into my /home/<username>/torch/install/share/lua/5.1/ directory. I'd like for them to be in ../share/lua/5.1/<package_name> directory. I tried doing something like
lua = {
["<package_name>"] = "a.lua",
...
or
lua = {
["<package_name>.<package_name>"] = "a.lua",
...
but neither method works.
Is there a way to put these files in a directory in the rockspec?
It's easy using the builtin build mode of rockspecs:
-- ...
build = {
type = "builtin",
modules = {
["mypackage.a"] = "a.lua",
["mypackage.b"] = "b.lua"
}
}
This should install a.lua as .../share/lua/5.1/mypackage/a.lua and b.lua as .../share/lua/5.1/mypackage/b.lua, so that require("mypackage.a") (or require("mypackage.b")) just works.
Lets say I have a project that uses bower, grunt, bowerify(with shim) and since I love Jest so much I want to test with that. How in the world do I get jest to see my browserify shim modules when it runs tests. I use grunt, to kick off the npm test command.
Here is my package.json file.
"browser": {
"jquery": "./bower_components/jquery/dist/jquery.js",
"foundation": "./bower_components/foundation/js/foundation/foundation.js",
"fastclick": "./bower_components/fastclick/lib/fastclick.js",
"greensock-tm": "./bower_components/gsap/src/uncompressed/TweenMax.js",
"greensock-css": "./bower_components/gsap/src/uncompressed/plugins/CSSPlugin.js",
"greensock-time": "./bower_components/gsap/src/uncompressed/TimelineMax.js",
"scrollmagic": "./bower_components/ScrollMagic/js/jquery.scrollmagic.js",
"handlebars": "./bower_components/handlebars/handlebars.runtime.js"
},
"browserify-shim": {
"jquery": "$",
"greensock-css": "CSSPlugin",
"fastclick": "FastClick",
"greensock-tm": "TweenMax",
"greensock-time": "TimelineMax",
"scrollmagic": "ScrollMagic",
"foundation": "foundation",
"handlebars": "Handlebars"
},
"browserify": {
"transform": [
"browserify-shim"
]
},
Right now I almost have this worked out by doing this in my grunt file before I run the test.
grunt.registerTask("shimBowerForTests",function(){
var readJson = require('read-package-json');
var fs = require('fs');
var remapify = require('remapify');
readJson('./package.json', console.error, false, function (er, data) {
if (er) {
throw "There was an error reading the file";
}
var packages = data.browser;
var browserify = require('browserify');
for (var key in packages){
var b = browserify();
var wstream = fs.createWriteStream("devjs/test/modules/"+key+'.js');
b.add(packages[key]);
b.bundle().pipe(wstream);
}
});
});
and.
exec: {
jestTest: {
command: 'cp -r devjs/modules devjs/test/modules && npm test'
}
}
The problem is that using browserify so combine everything for the browser works great with my setup and I can require my shimmed modules like this.
require('jquery') //example but in the jest cli the test fail because they can find the module unless I somehow prefix it with ./, like so require('./jquery')
I'm guessing that the problem is that you've only installed your shimmed modules with bower. If you want them to work in node/jest, you'll have to install them with npm as well. Then just make sure Jest isn't mocking anything in the node_modules directory, and it should find all the required modules in there as long as the names match up.
Your Jest config in package.json should look like:
"jest": {
"unmockedModulePathPatterns": [
"./node_modules"
]
}
And then just download all the dependencies.
npm install jquery --save-dev
UPDATE
Instead of using my below solution you should opt for using Karma,karma browserify. I have converted the below solution into using karma and it is working much much better.
----------------------OLD ANSWER
What I actually did to solve this was, used the Jest source preprocessor to rewrite the require statement to look for a module in a certain directory in my /tests/ folder that I have created using grunt. The Folder contains the files listed in my browserify-shim, browser section of the package.json file.
EDIT: Here is how I shim bower, I made this script in the Gruntfile.js that puts all the bower modules and any commonjs modules that I need into an accessible directory.
grunt.registerTask("shimBowerForTests", function() {
var readJson = require('read-package-json');
var fs = require('fs');
readJson('./package.json', console.error, false, function(er, data) {
if (er) {
throw "There was an error reading the file";
}
var packages = data.browser;
var shim = data['browserify-shim'];
var browserify = require('browserify');
var exclude = ["jquery.maskedinput", "jquery"];
for (var key in packages) {
var b = browserify();
var wstream = fs.createWriteStream("devjs/test/modules/" + key + '.js');
if (shim[key] !== undefined && exclude.indexOf(key) === -1) {
b.add(packages[key]);
b.bundle().pipe(wstream);
} else {
var rstream = fs.createReadStream(packages[key]);
rstream.pipe(wstream);
}
}
});
});
Then in the Jest pre processor file I do this.
module.exports = {
process: function(src, path) {
var src2= src.replace(/require\([\"\']([^\.\'\"]+)[\"\']\)/g, "require(\'../modules/$1\')");
src2= src2.replace(/jest\.dontMock\([\"\']([^\.\'\"]+)[\"\']\)/g, "jest.dontMock(\'../modules/$1\')");
return src2;
}
};