How to efficiently manage and process Bower packages with Gulp? (VS2015, Visual Studio 2015) - asp.net-mvc

Background:
Visual Studio 2015 has introduced Gulp and Bower for client side package management. .Net previously had a very efficient method of bundling / minification and package management, but for an unspecified reason this has been removed in ASP.Net 5 / MVC 6, and the advice is to use Gulp and Bower instead.
I have a number of vendor files that I wish to use in my project, including jquery.appear, isotope, owl-carousel etc, etc; some are simple JavaScript files, others have CSS, still others have assets such as fonts, images.
Scenario:
At the moment I am evaluating how to best utilise Bower to manage versions of packages, while using Gulp to extract only the necessary files from bower_components, and uglify / minify / concat them into bundles.
I am currently using CDN available versions of scripts, but best practice would suggest I implement fail-over to local copies - IF I can find a way to manage them using Bower / Gulp OR just download them locally, and forgo package management.
Package management would be my preferred approach, but not if this is high maintenance in terms of scripts, configurations, overrides etc.
What I have tried:
I have looked at Gulp packages such as bower-main-files, gulp-bower-src (which apparently is blacklisted by Gulp), and I am currently using gulp-concat-vendor; with this I can process basic packages which contain only single JavaScript files (i.e. not CSS, not related assets such as images).
Problems:
Some of the bower packages do not include correct information for exporting their main files (some have no main declarations at all).
Some of the packages download dependencies into bower_components at the top level, which becomes cluttered with files I do not need (I want only the main (core) exported files, and the dependencies are usually already met elsewhere). These additional packages need yet more configuration to exclude them from being processed as part of 'Bower Main Files'.
In general, Bower 'standards' are loose, and are not adhered to, even for popular packages.
During concatenation, sometimes a specific order needs to be achieved. I have been unable to find an elegant way to do this automatically - I have created an array of source files, but this is not ideal, as it requires manually checking and editing for each package, which mostly negates the whole concept of package management.
Questions:
Do experienced front-end developers attempt to follow the same approach as I am attempting (using bower_components as a source), or simply manually copy required files from GitHub?
If you do use bower-components, can you please outline the workflow with Gulp, and what plug-ins you use to filter out only the files you need.
Is it possible to prevent unneeded dependencies, tests, etc from being downloaded by Bower in the first place?
When processing files that include relative references (e.g. CSS containing a reference to an image), is it possible to correct the relative path, to be relative to the specified output directory for such assets?

Yes.
See below.
Well, bower package is package, you get whats included. For your build you either rely on components bower.json which specifies main files or do filtering yourself. It is simple enough.
You can use filter = require('gulp-filter') to filter files like that:
var gulp = require('gulp'),
bower = require('gulp-main-bower-files'),
filter = require('gulp-filter'),
concat = require('gulp-concat'),
rename = require('gulp-rename'),
srcmaps = require('gulp-sourcemaps'),
jsminify = require('gulp-uglify')
cssminify = require('gulp-csso'),
del = require('del');
var src = {
js: 'app/**/*.js',
css: 'app/**/*.css',
content: ['app/**/*.jpg', 'app/**/*.svg', 'app/**/*.png', 'app/**/*.ico', 'app/**/*.html']
}
var dst = {
pub: 'pub/',
lib: 'pub/lib/'
}
gulp.task('bower', ['start-build'], function () {
var jsfilter = filter('**/*.js')
var cssfilter = filter('**/*.css')
return gulp.src('bower.json')
.pipe(bower())
.pipe(jsfilter)
.pipe(concat('lib.min.js'))
.pipe(jsminify())
.pipe(gulp.dest(dst.lib))
.pipe(jsfilter.restore())
.pipe(cssfilter)
.pipe(concat('lib.min.css'))
.pipe(cssminify())
.pipe(gulp.dest(dst.lib))
.pipe(cssfilter.restore())
.pipe(rename(function (path) {
if (~path.dirname.indexOf('fonts')) {
path.dirname = '/fonts'
}
}))
.pipe(gulp.dest(dst.lib));
})
gulp.task('js', ['start-build'], function () {
return gulp.src([src.js])
.pipe(srcmaps.init())
.pipe(concat('app.min.js'))
.pipe(jsminify())
.pipe(srcmaps.write())
.pipe(gulp.dest(dst.pub));
})

Related

How to get the file path to an asset included in a Dart package?

I am writing a Dart package (not Flutter). I have included a few bitmap images as public assets, e.g., lib/assets/empty.png. When this package is running as a command-line app for an end-user, how can I get the file path to these assets on the user's system?
Use-case: My Dart package calls out to FFMPEG, and I need to tell FFMPEG where to find these asset files on the system that's using my package. For example, the call to FFMPEG might look like:
ffmpeg -i "path/to/lib/assets/empty.png" ...
Accessing a Dart package's assets can happen in two modalities:
Running a Dart CLI app with the dart tool and accessing a dependency's assets, or
Running an executable CLI app
The difference between these two situations is that when you're running a CLI app using the dart tool, all of your dependencies are available as structured packages in a local cache on your system. However, when you're running an executable, all relevant code is compiled into a single binary, which means you no longer have access at runtime to your dependencies' packages, you only have access to your dependencies' tree-shaken, compiled code.
Accessing assets when running with dart
The following code will resolve a package asset URI to a file system path.
final packageUri = Uri.parse('package:your_package/your/asset/path/some_file.whatever');
final future = Isolate.resolvePackageUri(packageUri);
// waitFor is strongly discouraged in general, but it is accepted as the
// only reasonable way to load package assets outside of Flutter.
// ignore: deprecated_member_use
final absoluteUri = waitFor(future, timeout: const Duration(seconds: 5));
final file = File.fromUri(absoluteUri);
if (file.existsSync()) {
return file.path;
}
This resolution code was adapted from Tim Sneath's winmd package: https://github.com/timsneath/winmd/blob/main/lib/src/metadatastore.dart#L84-L106
Accessing assets when running an executable
When compiling a client app to an executable, that client app simply cannot access any asset files that were stored with the dependent package. However, there is a work around that may work for some people (it did for me). You can store Base64 encoded versions of your assets in your Dart code, within your package.
First, encode each of your assets into a Base64 string and store those strings somewhere in your Dart code.
const myAsset = "iVBORw0KGgoAAA....kJggg==";
Then, at runtime, decode the string back to bytes, and then write those bytes to a new file on the local file system. Here's the method I used in my case:
/// Writes this asset to a new file on the host's file system.
///
/// The file is written to [destinationDirectory], or the current
/// working directory, if no destination is provided.
String inflateToLocalFile([Directory? destinationDirectory]) {
final directory = destinationDirectory ?? Directory.current;
final file = File(directory.path + Platform.pathSeparator + fileName);
file.createSync(recursive: true);
final decodedBytes = base64Decode(base64encoded);
file.writeAsBytesSync(decodedBytes);
return file.path;
}
This approach was suggested by #passsy
Have a look at the dcli package.
It has a 'pack' command designed to solve exactly this problem.
It encodes assets into dart files that can be unpacked at runtime.

Have all Bazel packages expose their documentation files (or any file with a given extension)

Bazel has been working great for me recently, but I've stumbled upon a question for which I have yet to find a satisfactory answer:
How can one collect all files bearing a certain extension from the workspace?
Another way of phrasing the question: how could one obtain the functional equivalent of doing a glob() across a complete Bazel workspace?
Background
The goal in this particular case is to collect all markdown files to run some checks and generate a static site from them.
At first glance, glob() sounds like a good idea, but will stop as soon as it runs into a BUILD file.
Current Approaches
The current approach is to run the collection/generation logic outside of the sandbox, but this is a bit dirty, and I'm wondering if there is a way that is both "proper" and easy (ie, not requiring that each BUILD file explicitly exposes its markdown files.
Is there any way to specify, in the workspace, some default rules that will be added to all BUILD files?
You could write an aspect for this to aggregate markdown files in a bottom-up manner and create actions on those files. There is an example of a file_collector aspect here. I modified the aspect's extensions for your use case. This aspect aggregates all .md and .markdown files across targets on the deps attribute edges.
FileCollector = provider(
fields = {"files": "collected files"},
)
def _file_collector_aspect_impl(target, ctx):
# This function is executed for each dependency the aspect visits.
# Collect files from the srcs
direct = [
f
for f in ctx.rule.files.srcs
if ctx.attr.extension == f.extension
]
# Combine direct files with the files from the dependencies.
files = depset(
direct = direct,
transitive = [dep[FileCollector].files for dep in ctx.rule.attr.deps],
)
return [FileCollector(files = files)]
markdown_file_collector_aspect = aspect(
implementation = _file_collector_aspect_impl,
attr_aspects = ["deps"],
attrs = {
"extension": attr.string(values = ["md", "markdown"]),
},
)
Another way is to do a query on file targets (input and output files known to the Bazel action graph), and process these files separately. Here's an example querying for .bzl files in the rules_jvm_external repo:
$ bazel query //...:* | grep -e ".bzl$"
//migration:maven_jar_migrator_deps.bzl
//third_party/bazel_json/lib:json_parser.bzl
//settings:stamp_manifest.bzl
//private/rules:jvm_import.bzl
//private/rules:jetifier_maven_map.bzl
//private/rules:jetifier.bzl
//:specs.bzl
//:private/versions.bzl
//:private/proxy.bzl
//:private/dependency_tree_parser.bzl
//:private/coursier_utilities.bzl
//:coursier.bzl
//:defs.bzl

What is the purpose of buildResources folder in electron-builder building process?

I'm reading through electron and electron-builder docs, but I still do not quite understand what is the purpose of the buildResources folder?
Here's what a configuration doc for electron-builder says:
buildResources = build String - The path to build resources.
Kind of self-explanatory... But how or when they are involved in the build process, especially having that:
...build resources is not packed into the app. If you need to use some
files, e.g. as tray icon, please include required files explicitly
Can we simply put those icon files in an arbitrary folder and then copy over into the app/ manually (since we need to include buildResources manually anyway)?
TL;DR:
As far as I can tell from a quick glance at the source code, the buildResources folder is used to hold additional scripts, plugins, etc. that can be used by the package building software. Electron-builder doesn't generate the packages itself, it uses tools like NSIS.
Explanation:
I've had the same question and unfortunately find an answer for this isn't very straight-forward. The docs entry is pretty useless. I found out that someone asked about it in the GitHub issues but never got an answer.
I decided to dig in the code a bit myself to find out what it does. In NsisTargets.ts, you can see that the buildResources folder can contain custom includes and plugins for NSIS.
// NsisTargets.ts
taskManager.add(async () => {
const userPluginDir = path.join(packager.info.buildResourcesDir, pluginArch)
const stat = await statOrNull(userPluginDir)
if (stat != null && stat.isDirectory()) {
scriptGenerator.addPluginDir(pluginArch, userPluginDir)
}
})
// [...]
taskManager.add(async () => {
const customInclude = await packager.getResource(this.options.include, "installer.nsh")
if (customInclude != null) {
scriptGenerator.addIncludeDir(packager.info.buildResourcesDir)
scriptGenerator.include(customInclude)
}
})
and in pkg.ts it's used to load additional scripts to the pkg builder:
// pkg.ts
if (options.scripts != null) {
args.push("--scripts", path.resolve(this.packager.info.buildResourcesDir, options.scripts))
}
It appears as though buildResources can contain assets/scripts specifically used for the build process. That also explains why the contents of buildResources aren't included in the resulting app.asar file.
So, I'm going to say straight away that the documentation for this option is just awful.
Files included in buildResources will appear in the asar file which you can find documentation about on electron's website.
The option files will include files such as pictures which are not accessible in the asar file.
I.E.
given I have a folder called assets in my build folder I want to include with my app.
"files": [
"./build/**/*"
],
"directories": {
"buildResources": "assets"
}
This will put all folders inside build into the asar file, which you can then unpack by including,
"asarUnpack": "**/assets/*"
This will put the folder assets into the build folder in the app directory.

Using NPM/Bower over Nuget MVC Site with bundling

I am trying to train myself on the new preferred way of referencing Front-end libraries in .Net Applications, so I have started to read into doing this with Gulp. I have read this article on replacing the Nuget packages with ones from bower
https://blogs.msdn.microsoft.com/cdndevs/2015/02/17/using-bower-with-visual-studio/
And this one to use Gulp
http://www.davepaquette.com/archive/2014/10/08/how-to-use-gulp-in-visual-studio.aspx
However I am having some trouble putting it all together. I want together some process that will not only replace all the Pre-Installed Nuget packages with bower packages, but also minify and bundle them with gulp instead of using the Web.Optimization library. The first link walks through doing that, but the gulp script is not providing the output I would expect (no dist folders for instance). This SO question has some good answers, but I am not seeing how I bundle all the libraries from bower (I read through all the comments and answers).
Using Grunt, Bower, Gulp, NPM with Visual Studio 2015 for a ASP.NET 4.5 Project
Obviously this stuff is new to me so I will get confused, but I want to make sure I do it the right way.
I am using Visual Studio 2015 and am creating a new MVC 4.5.2 Application, and I want to have all front-end libraries referenced and bundled/minified without using any .Net libraries. It seems at this point far easier to do it the old way
The question is a little broad but Ill hit on the key points since I've been doing this exact thing for a few weeks now.
Break what you are doing into two phases - replacing packages from nuget as phase one and replacing the .net bundling as the other.
Phase one - pretty simple actually, remove (uninstall) all the packages you have from nuget that have bower equivalents, then add those packages via the bower commands (don't forget to --save and --save-dev where needed). Then replace your script locations in the .net bundling with the new locations in bower_components. Once you can confirm your site works under this new layout while still using .net bundling you are ready for phase two.
Now for phase two, first and formost you need to learn the concept of 'globs' which are basically wild card based include and excludes for files in gulp. Probably the single most helpful thing I've found to help with this is the gulp plugin main-bower-files. So to create a good glob to start with I had this...
var paths = {};
paths.baseReleaseFolder = "app";
paths.baseJSReleaseFile = "site.min.js";
paths.jsSource = mainBowerFiles();//this gets all your bower scripts in an array
.concat([
"node_modules/angular/angular.js",//my angular is loaded via npm not bower because the bower version was giving me dependency issues (bug)
"node_modules/angular-route/angular-route.js",
"Source/output.js",//generated by MY typescript compiler
"!/Scripts", //this is an exclude
"!**/*.min.js" //also an exclude
]);
This is basically saying I want to select all the DISTRO bower plugins files needed to run whatever, include my angular js, and exclude anything in my scripts folder (typescript files and outputs) and exclude any ALREADY minified files (I want to do it all myself as one concatenated file).
Now I separate the js and css operations and wrap another event to call both so I end up with
gulp.task("min", ["min:js", "min:css"]);
gulp.task("min:js", function () {});
gulp.task("min:css", function () {});
Now as an example of how that works I have the following in my min:js
gulp.task("min:js", function () {
var jsFilter = filter('**/*.js', { restore: true });//extra file safty incase my glob is getting non js files somehow
return gulp
.src(paths.jsSource) //this is the 'glob' IE the file selector
.pipe(jsFilter) //enforce my filter from above (gulp-filter)
//.pipe(debug()) //useful to output what files are in use (gulp-debug)
.pipe(sourcemaps.init({loadMaps:true})) //create sourcemaps for final output(gulp-sourcemaps)
.pipe(uglify()) //min and ugilfy files on the way to next step (gulp-uglify)
.pipe(concat(paths.baseReleaseFolder + "/" + paths.baseJSReleaseFile)) //this takes all the files in the glob and creates a single file out of them in app/site.min.js
.pipe(rev()) //ignore this, cache-busting but requires work on the .net side to load the files, basically adds the file hash to the file name in the output
.pipe(sourcemaps.write(".")) //actually write my .map.js file to the final destination
.pipe(gulp.dest(".")) //write the final site.min.js to its location
.pipe(jsFilter.restore); //not sure but all filter examples do this.
});
So when this is all said and done I end up with a single site.min.js file in the 'app' folder that is the concatenated, minified, uglified version off all my bower components (and whatever else the glob hit). Now just to give you an idea on how plugin intensive using gulp is this is the declaration of all plugins I load into my main gulp script to do bascailly what .net bundling does for you....
var gulp = require('gulp'),
rimraf = require("gulp-rimraf"),
concat = require("gulp-concat"),
cssmin = require("gulp-cssmin"),
debug = require("gulp-debug"),
uglify = require("gulp-uglify"),
filter = require("gulp-filter"),
rename = require("gulp-rename"),
rev = require('gulp-rev'),
sourcemaps = require('gulp-sourcemaps'),
csslint = require('gulp-csslint'),
jslint = require('gulp-jslint'),
typescript = require("gulp-typescript"),
tslint = require("gulp-tslint"),
runSequence = require('run-sequence'),
mainNodeFiles = require("npm-main-files"),
mainBowerFiles = require('main-bower-files');
You probably don't need all of these (some are typescript, some are linters)
Edit: Heres my css function
gulp.task("min:css", function () {
var cssFilter = filter('**/*.css', { restore: true });
return gulp
.src(paths.cssSource)
.pipe(cssFilter)
//.pipe(debug())
.pipe(sourcemaps.init({ loadMaps: true }))
.pipe(concat(paths.baseReleaseFolder + "/" + paths.baseCSReleaseFile))
.pipe(cssmin())
.pipe(rev())
.pipe(sourcemaps.write("."))
.pipe(gulp.dest("."))
.pipe(cssFilter.restore);
});

Using StealJS to load Bower components lacking a bower.json file

I am using the StealJS + Bower integration in my application, but a couple of my Bower components (including es6-collections) do not contain a bower.json file. Because es6-collections is one of the dependencies in my project's bower.json file, StealJS tries to load the es6-collections component's bower.json file, cannot find it because it does not exist, and complains: Unable to load the bower.json for es6-collections. I tried using System.config({ path: { 'es6-collections': '...' } }) to notify StealJS of the path to the script to use when loading es6-collections, but that does not help. What can I do to get StealJS to load this component?
Assumptions
So I am going to make a few assumptions:
you are using steal from bower
you are "bower install"'ing es6-collections from github directly
you are implicitly using the system-bower plugin by using the HTML <script src="bower_components/steal/steal.js" main="main"></script> to load your "main" file
If these things seem mostly true-ish then you may just have to add some configuration in your bower.json file to silence the error/warning and have everything work as expected.
Explanation:
So because the system-bower plugin (which you are using implicitly because steal detects it is being loaded from a bower_components directory) uses the components bower.json files to determine entry points, so in this case the error/warning comes from not being able to find es6-collections bower.json file.
Solution:
So we just need to tell System (used by steal) where to find that module and that it can stop looking for it's bower.json file.
We can do that by adding a "system" property to the bower.json and adding some configuration data like this...
"system": {
"paths": {
"es6-collections": "bower_components/es6-collections/index.js"
},
"bowerIgnore": ["es6-collections"],
"meta": {
"es6-collections": {
"format": "global"
}
}
}
The paths configuration there, tells System where to find the module
The bowerIgnore array tells system-bower to not look for the bower.json for that module
and the meta config is there to tell System to treat this module like it's a script that is going to add to the global object (window in the browser), which you should probably do for this particular module because of the way es6-collections was written: it exports an empty object if it has nothing to pollyfill so you can't use the exported object, best to just use it as if it was a global module.
For more information on all these things...
http://stealjs.com/docs/bower.html
https://github.com/systemjs/systemjs/wiki/Meta-Configuration
http://stealjs.com/docs/steal.html
Just to have a working example here https://gist.github.com/BigAB/c108bb0860c9cfee3d6a are three files you can copy-paste/clone and then do a bower install and see it working.

Resources