I am trying to exclude a list of files or directories when building a web application with dart's pub build.
Using this, as suggested by the documentation:
transformers:
- simple_transformer:
$exclude: "**/CVS"
does not work:
Error on line 10, column 3 of pubspec.yaml: "simple_transformer" is not a dependency.
- simple_transformer:
Is there a way to do it (using SDK 1.10.0) ?
Sadly there is currently no support to mark files as ignored by pub build as Günter already mentioned. The .gitignore feature was removed as it was undocumented and caused more trouble than it solved.
But you can execlude files from the build output. This means that the files are still processed (and still take time to process =/ ) but aren't present in the output directiory. This is useful for generating a deployable copy of your application in one go.
In our application we use a simple ConsumeTransformer to mark assets as consumed so that they are not written to the output folder:
library consume_transformer;
import 'package:barback/barback.dart';
class ConsumeTransformer extends Transformer implements LazyTransformer {
final List<RegExp> patterns = <RegExp>[];
ConsumeTransformer.asPlugin(BarbackSettings settings) {
if (settings.configuration['patterns'] != null) {
for (var pattern in settings.configuration['patterns']) {
patterns.add(new RegExp(pattern));
}
}
}
bool isPrimary(AssetId inputId) =>
patterns.any((p) => p.hasMatch(inputId.path));
void declareOutputs(DeclaringTransform transform) {}
void apply(Transform transform) => transform.consumePrimary();
}
The consumer requires a list of regex patterns as an argument an consumes the matched files. You need to add the transformer to you pubspec.yaml file as the last transformer:
transformers:
- ... # Your other transformers
- packagename/consume_transformer:
patterns: ["\\.psd$"]
The example configuration ignores all files that have the psd extension, but you can add pattern as you need them.
I created a pub package that contains the transformer, take a look here.
simple_transformer is the name of the transformer you want to inform to exclude the files. If you want to apply this to dart2js you need to use the name $dart2js instead of simple_transformer.
For more details about configuring $dart2js see https://www.dartlang.org/tools/pub/dart2js-transformer.html
Related
I am writing a Dart package (not Flutter). I have included a few bitmap images as public assets, e.g., lib/assets/empty.png. When this package is running as a command-line app for an end-user, how can I get the file path to these assets on the user's system?
Use-case: My Dart package calls out to FFMPEG, and I need to tell FFMPEG where to find these asset files on the system that's using my package. For example, the call to FFMPEG might look like:
ffmpeg -i "path/to/lib/assets/empty.png" ...
Accessing a Dart package's assets can happen in two modalities:
Running a Dart CLI app with the dart tool and accessing a dependency's assets, or
Running an executable CLI app
The difference between these two situations is that when you're running a CLI app using the dart tool, all of your dependencies are available as structured packages in a local cache on your system. However, when you're running an executable, all relevant code is compiled into a single binary, which means you no longer have access at runtime to your dependencies' packages, you only have access to your dependencies' tree-shaken, compiled code.
Accessing assets when running with dart
The following code will resolve a package asset URI to a file system path.
final packageUri = Uri.parse('package:your_package/your/asset/path/some_file.whatever');
final future = Isolate.resolvePackageUri(packageUri);
// waitFor is strongly discouraged in general, but it is accepted as the
// only reasonable way to load package assets outside of Flutter.
// ignore: deprecated_member_use
final absoluteUri = waitFor(future, timeout: const Duration(seconds: 5));
final file = File.fromUri(absoluteUri);
if (file.existsSync()) {
return file.path;
}
This resolution code was adapted from Tim Sneath's winmd package: https://github.com/timsneath/winmd/blob/main/lib/src/metadatastore.dart#L84-L106
Accessing assets when running an executable
When compiling a client app to an executable, that client app simply cannot access any asset files that were stored with the dependent package. However, there is a work around that may work for some people (it did for me). You can store Base64 encoded versions of your assets in your Dart code, within your package.
First, encode each of your assets into a Base64 string and store those strings somewhere in your Dart code.
const myAsset = "iVBORw0KGgoAAA....kJggg==";
Then, at runtime, decode the string back to bytes, and then write those bytes to a new file on the local file system. Here's the method I used in my case:
/// Writes this asset to a new file on the host's file system.
///
/// The file is written to [destinationDirectory], or the current
/// working directory, if no destination is provided.
String inflateToLocalFile([Directory? destinationDirectory]) {
final directory = destinationDirectory ?? Directory.current;
final file = File(directory.path + Platform.pathSeparator + fileName);
file.createSync(recursive: true);
final decodedBytes = base64Decode(base64encoded);
file.writeAsBytesSync(decodedBytes);
return file.path;
}
This approach was suggested by #passsy
Have a look at the dcli package.
It has a 'pack' command designed to solve exactly this problem.
It encodes assets into dart files that can be unpacked at runtime.
I have a truststore file(a binary file) that I need to provide during helm upgrade. This file is different for each target env(dev,qa,staging or prod). So I can only provide this file at time of deployment. helm upgrade --set-file does not take a binary file. This seem to be the issue I found here: https://github.com/helm/helm/issues/3276. This truststore files are stored in Jenkins Credential store.
As the command itself is described below:
--set-file stringArray set values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2)
it is also important to know The Format and Limitations of
--set.
The error you see: Error: failed parsing --set-file data... means that the file you are trying to use does not meet the requirements. See the example below:
--set-file key=filepath is another variant of --set. It reads the
file and use its content as a value. An example use case of it is to
inject a multi-line text into values without dealing with indentation
in YAML. Say you want to create a brigade project with certain value
containing 5 lines JavaScript code, you might write a values.yaml
like:
defaultScript: |
const { events, Job } = require("brigadier")
function run(e, project) {
console.log("hello default script")
}
events.on("run", run)
Being embedded in a YAML, this makes it harder for you to use IDE
features and testing framework and so on that supports writing code.
Instead, you can use --set-file defaultScript=brigade.js with
brigade.js containing:
const { events, Job } = require("brigadier")
function run(e, project) {
console.log("hello default script")
}
events.on("run", run)
I hope it helps.
I'd like to modify some of my Grails domain classes at compilation time. I initially thought this was a job for Groovy's global ASTTransformation since I don't want to annotate my domain classes (which local transformers require). What's the best way to do this?
I also tried mimicking DefaultGrailsDomainClassInjector.java by creating my own class in the same package, implementing the same interfaces, but I probably just didn't know how to package it up in the right place because I never saw my methods get invoked.
On the other hand I was able to manually create a JAR which contained a compiled AST transformation class, along with the META-INF/services artifacts that plain Groovy global transformations require. I threw that JAR into my project's "lib" dir and visit() was successfully invoked. Obviously this was a sloppy job because I am hoping to have the source code of my AST transformation in a Grails plugin and not require a separate JAR artifact if I don't have to, plus I couldn't get this approach to work by having the JAR in my Grails plugin's "lib" but had to put it into the Grails app's "lib" instead.
This post helped a bit too: Grails 2.1.1 - How to develop a plugin with an AstTransformer?
The thing about global transforms the transform code should be available when the compilation starts. Having the transformer in a jar was what i did first! But as you said it is a sloppy job.
What you want to do is have your ast transforming class compile before others gets to the compilation phase. Here is what you do!
Preparing the transformer
Create a directory called precompiled in src folder! and add the Transformation class and the classes (such as annotations) the transformer uses in this directory with the correct packaging structure.
Then create a file called org.codehaus.groovy.transform.ASTTransformation in called precompiled/META-INF/services and you will have the following structure.
precompiled
--amanu
----LoggingASTTransformation.groovy
--META-INF
----services
------org.codehaus.groovy.transform.ASTTransformation
Then write the fully qualified name of the transformer in the org.codehaus.groovy.transform.ASTTransformation file, for the example above the fully qualified name would be amanu.LoggingASTTransformation
Implementation
package amanu
import org.codehaus.groovy.transform.GroovyASTTransformation
import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.control.SourceUnit
#GroovyASTTransformation(phase=CompilePhase.CANONICALIZATION)
class TeamDomainASTTransformation implements ASTTransformation{
public void visit(ASTNode[] nodes, SourceUnit sourceUnit) {
println ("*********************** VISIT ************")
source.getAST()?.getClasses()?.each { classNode ->
//Class node is a class that is contained in the file being compiled
classNode.addProperty("filed", ClassNode.ACC_PUBLIC, new ClassNode(Class.forName("java.lang.String")), null, null, null)
}
}
}
Compilation
After implementing this you can go off in two ways! The first approach is to put it in a jar, like you did! and the other is to use a groovy script to compile it before others. To do this in grails, we use _Events.groovy script.
You can do this from a plugin or the main project, it doesn't matter. If it doesn't exist create a file called _Events.groovy and add the following content.
The code is copied from reinhard-seiler.blogspot.com with modifications
eventCompileStart = {target ->
...
compileAST(pluginBasedir, classesDirPath)
...
}
def compileAST(def srcBaseDir, def destDir) {
ant.sequential {
echo "Precompiling AST Transformations ..."
echo "src ${srcBaseDir} ${destDir}"
path id: "grails.compile.classpath", compileClasspath
def classpathId = "grails.compile.classpath"
mkdir dir: destDir
groovyc(destdir: destDir,
srcDir: "$srcBaseDir/src/precompiled",
classpathref: classpathId,
stacktrace: "yes",
encoding: "UTF-8")
copy(toDir:"$destDir/META-INF"){
fileset(dir:"$srcBaseDir/src/precompiled/META-INF")
}
echo "done precompiling AST Transformations"
}
}
the previous script will compile the transformer before others are compiled! This enable the transformer to be available for transforming your domain classes.
Don't forget
If you use any class other than those added in your classpath, you will have to precompile those too. The above script will compile everything in the precompiled directory and you can also add classes that don't need ast, but are needed for it in that directory!
If you want to use domain classes in transformation, You might want to do the precompilation in evenCompileEnd block! But this will make things slower!
Update
#Douglas Mendes mentioned there is a simple way to cause pre compilation. Which is more concise.
eventCompileStart = {
target -> projectCompiler.srcDirectories.add(0, "./src/precompiled")
}
I have a use-case for a pub transformer that isn't very typical, so I'm wondering if it's possible. I'd like to individually gzip every css, html, and javascript file produced during pub build. I have two questions about this:
Is it possible to transform compiled javascript?
Is it possible to configure the order that the transformer is run? Obviously, the gzip transformer would need to be the last transformer run.
In case you're wondering why I want to do this, I'm serving my app via S3 which doesn't support on-the-fly gzipping. I figure pub transformers would be the most appropriate place to do this.
You can specify order by the line position of the transformer:
transformers:
- $dart2js
- YourGzipTransformer
The above specifies that you want the dart2js transformer to run before YourGzipTransformer, thereby giving YourGzipTransformer access to the Javascript files generated by dart2js. If you swapped the order, or did not specify the order of the dart2js transformer, YourGzipTransformer would run before it and not have access to the compiled Javascript.
If you didn't care what order some transformers ran in, you could specify that too:
transformers:
- [$dart2js, SomeOtherTransformer]
- YourGzipTransformer
You can read more about this in the Assets and Transformers article.
Here's a basic transformer that will generate a gzipped version of every CSS, HTML, and Javascript file:
import 'dart:io';
import 'package:barback/barback.dart';
class GzipTransformer extends Transformer {
final BarbackSettings _settings;
GzipTransformer.asPlugin(this._settings);
#override
Future apply(Transform transform) {
return transform.primaryInput.readAsString().then((content) {
var id = transform.primaryInput.id;
var gzipId = id.changeExtension(id.extension +".gzip");
var gzippedContent = GZIP.encode(content.codeUnits);
transform.addOutput(new Asset.fromBytes(gzipId, gzippedContent));
});
}
String get allowedExtensions => ".js .css .html";
}
Barback package description:
An asset build system. Given a set of input files and a set of
transformations (think compilers, preprocessors and the like), will
automatically apply the appropriate transforms and generate output
files. When inputs are modified, automatically runs the transforms
that are affected. Runs transforms asynchronously and in parallel when
possible to maximize responsiveness.
Docs on assets and transformers say:
For pub serve, the transformers run when the dev server starts up and
whenever a source asset changes. The pub build command runs the
transformers once and then exits.
So I took this example:
// Copyright (c) 2014, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
import 'package:barback/barback.dart';
import 'package:markdown/markdown.dart';
import 'dart:async';
class ConvertMarkdown extends Transformer {
// A constructor named "asPlugin" is required. It can be empty, but
// it must be present. It is how pub determines that you want this
// class to be publicly available as a loadable transformer plugin.
ConvertMarkdown.asPlugin();
// Any markdown file with one of the following extensions is
// converted to HTML.
String get allowedExtensions => ".md .markdown .mdown";
Future apply(Transform transform) {
return transform.primaryInput.readAsString().then((content) {
// The extension of the output is changed to ".html".
var id = transform.primaryInput.id.changeExtension(".html");
String newContent = "<html><body>"
+ markdownToHtml(content)
+ "</body></html>";
transform.addOutput(new Asset.fromString(id, newContent));
});
}
}
It runs as expected with pub build, but does nothing with pub serve except printing:
Build completed successfully
every time I change any file (not only appropriate asset) in the directory.
After reading this I think that Dart has some problems with watching files(not only directories) on Windows platform.
It is true that pub serve runs transformers after each file modification. But compared to pub build it is not outputting the results to the build/ folder. pub serve is a development server, so you need to request you assets via HTTP, for example in a browser.
See the pub serve documentation for more details