Saving scoped object outside scope in Dart - dart

The following leaves s null after the file read exits:
String s;
new File('etc.stk').readAsString().then((String contents) {
s = contents;
});
// s is null here.
Is there a way to save (or clone) s, or am I compelled to use it only in the .then scope?
I have a few thousand lines of compiler/interpreter code that parses and runs the file contents, and would prefer not to have them all inside the new File scope.
EDIT
To provide more context, what I am trying to do is something like
new File('etc1.stk').readAsString()
.then((String script) {
syntaxTree1 = buildTree(script);
});
new File('etc2.stk').readAsString()
.then((String script) {
syntaxTree2 = buildTree(script);
});
and have access to both syntaxTree1 and syntaxTree2 in subsequent code. I will wrap my mind around the Dart Way if I can.

EDIT
(this code is tested)
import 'dart:async' as async;
import 'dart:io' as io;
void main(args) {
// approach1: inline
async.Future.wait([
new io.File('file1.txt').readAsString(),
new io.File('file2.txt').readAsString()
]).then((values) {
values.forEach(print);
});
// approach2: load files in another function
getFiles().then((values) {
values.forEach(print);
});
}
async.Future<List> getFiles() {
return async.Future.wait([
new io.File('file1.txt').readAsString(),
new io.File('file2.txt').readAsString()
]);
}
output:
file1
file2
file1
file2
EDIT END
hint: the code is not tested
// s is null here
is because this line is executed before
s = contents
This code
new File('etc.stk').readAsString()
returns a future that is enlisted in the event queue and is executed when the actual 'thread' of execution is finished.
If you had provided more code I would have a better context for a proposed solution.
What you could do is
String s;
new File('etc.stk').readAsString().then((String contents) {
s = contents;
}).then((_) {
// s is **NOT** null here.
});
or
//String s;
new File('etc.stk').readAsString().then((String contents) {
//s = contents;
someCallback(s)
});
// s is null here.
void someCallback(String s) {
// s is **NOT** null here
}
or
Future<String> myReadAsString() {
return new File('etc.stk').readAsString();
}
myReadAsString().then((s) {
// s is **NOT** null here
}
see also:
https://www.dartlang.org/slides/2013/06/dart-streams-are-the-future.pdf
async programming in dart
https://www.dartlang.org/docs/tutorials/#futures
https://api.dartlang.org/apidocs/channels/stable/dartdoc-viewer/dart:async
https://www.dartlang.org/articles/event-loop/
https://www.dartlang.org/articles/futures-and-error-handling/
and maybe
https://www.dartlang.org/articles/creating-streams/

Related

How to store csv data in a list

I want to read a csv file and store the content in a list of abjects. So I created the respective object and made an inputstream to read the file. But when I try to acces the list from outside of the function it's empty, so I thing, that the data is stored in a copy of listOfStocks and not directly in the list, like I want. So how can I store the content in the list?
import 'dart:io';
import 'dart:async';
import 'dart:convert';
main(List arguments) {
final File file = new File("../Data/In.List.toDart.csv");
Stream<List> inputStream = file.openRead();
List<Stock> listOfStocks = new List();
inputStream
.transform(utf8.decoder)
.transform(new LineSplitter())
.listen((String line) {
List row = line.split(',');
Stock stock = new Stock();
stock.setSymbol(row[0]);
stock.setDesc(row[1]);
stock.setExchange(row[2]);
listOfStocks.add(stock);
print(listOfStocks.length);
},
onDone: () { print('All read.'); },
onError: (e) { print(e.toString()); });
print(listOfStocks.length);
}
class Stock {
String symbol;
String desc;
String exchange;
void setSymbol(String symbol) {
this.symbol = symbol;
}
void setDesc(String desc) {
this.desc = desc;
}
void setExchange(String exchange) {
this.exchange = exchange;
}
String getSymbol() {
return symbol;
}
String getDesc() {
return desc;
}
String getExchange() {
return exchange;
}
}
The inputStream is asynchronous, so it will take some time before the file has been read.
The inputStream.transform(...).transform(...).listen(...) call completes immediately, then the stream will emit events when something has been read from the disk, and the listOfStocks will be gradually populated.
If you need the content of the list "immediately", that is: before you do anything else, you will have to block until the list is fully populated.
You can do this in two ways:
Read the file synchronously. This makes the operating system block until the file has been read completely into memory.
var content = file.readAsStringSync(); // Defaults to UTF-8 decoding.
for (var line in LineSplitter.split(content))
...
lineOfStocks.add(stock);
}
print(listOfStocks.length);
Read the file asynchronously, but wait for the transformation to complete:
main(List arguments) async {
...
await inputStream
.transform(utf8.decoder)
.transform(const LineSplitter())
.forEach((String line) { // Use forEach instead of listen
...
listOfStocks.add(stock);
});
print(listOfStocks.length);
}
I you use forEach instead of listen, it returns a future which completes
when the stream is done. With listen, you get a done event that you have
to handle manually.

Generate one file for a list of parsed files using source_gen in dart

I have a list of models that I need to create a mini reflective system.
I analyzed the Serializable package and understood how to create one generated file per file, however, I couldn't find how can I create one file for a bulk of files.
So, how to dynamically generate one file, using source_gen, for a list of files?
Example:
Files
user.dart
category.dart
Generated:
info.dart (containg information from user.dart and category.dart)
Found out how to do it with the help of people in Gitter.
You must have one file, even if empty, to call the generator. In my example, it is lib/batch.dart.
source_gen: ^0.5.8
Here is the working code:
The tool/build.dart
import 'package:build_runner/build_runner.dart';
import 'package:raoni_global/phase.dart';
main() async {
PhaseGroup pg = new PhaseGroup()
..addPhase(batchModelablePhase(const ['lib/batch.dart']));
await build(pg,
deleteFilesByDefault: true);
}
The phase:
batchModelablePhase([Iterable<String> globs =
const ['bin/**.dart', 'web/**.dart', 'lib/**.dart']]) {
return new Phase()
..addAction(
new GeneratorBuilder(const
[const BatchGenerator()], isStandalone: true
),
new InputSet(new PackageGraph.forThisPackage().root.name, globs));
}
The generator:
import 'dart:async';
import 'package:analyzer/dart/element/element.dart';
import 'package:build/build.dart';
import 'package:source_gen/source_gen.dart';
import 'package:glob/glob.dart';
import 'package:build_runner/build_runner.dart';
class BatchGenerator extends Generator {
final String path;
const BatchGenerator({this.path: 'lib/models/*.dart'});
#override
Future<String> generate(Element element, BuildStep buildStep) async {
// this makes sure we parse one time only
if (element is! LibraryElement)
return null;
String libraryName = 'raoni_global', filePath = 'lib/src/model.dart';
String className = 'Modelable';
// find the files at the path designed
var l = buildStep.findAssets(new Glob(path));
// get the type of annotation that we will use to search classes
var resolver = await buildStep.resolver;
var assetWithAnnotationClass = new AssetId(libraryName, filePath);
var annotationLibrary = resolver.getLibrary(assetWithAnnotationClass);
var exposed = annotationLibrary.getType(className).type;
// the caller library' name
String libName = new PackageGraph.forThisPackage().root.name;
await Future.forEach(l.toList(), (AssetId aid) async {
LibraryElement lib;
try {
lib = resolver.getLibrary(aid);
} catch (e) {}
if (lib != null && Utils.isNotEmpty(lib.name)) {
// all objects within the file
lib.units.forEach((CompilationUnitElement unit) {
// only the types, not methods
unit.types.forEach((ClassElement el) {
// only the ones annotated
if (el.metadata.any((ElementAnnotation ea) =>
ea.computeConstantValue().type == exposed)) {
// use it
}
});
});
}
});
return '''
$libName
''';
}
}
It seems what you want is what this issue is about How to generate one output from many inputs (aggregate builder)?
[Günter]'s answer helped me somewhat.
Buried in that thread is another thread which links to a good example of an aggregating builder:
1https://github.com/matanlurey/build/blob/147083da9b6a6c70c46eb910a3e046239a2a0a6e/docs/writing_an_aggregate_builder.md
The gist is this:
import 'package:build/build.dart';
import 'package:glob/glob.dart';
class AggregatingBuilder implements Builder {
/// Glob of all input files
static final inputFiles = new Glob('lib/**');
#override
Map<String, List<String>> get buildExtensions {
/// '$lib$' is a synthetic input that is used to
/// force the builder to build only once.
return const {'\$lib$': const ['all_files.txt']};
}
#override
Future<void> build(BuildStep buildStep) async {
/// Do some operation on the files
final files = <String>[];
await for (final input in buildStep.findAssets(inputFiles)) {
files.add(input.path);
}
String fileContent = files.join('\n');
/// Write to the file
final outputFile = AssetId(buildStep.inputId.package,'lib/all_files.txt');
return buildStep.writeAsString(outputFile, fileContent);
}
}

Groovy/Grails promises/futures. There is no .resolve(1,2,3) method. Strange?

I am developing in a Grails application. What I want to do is to lock the request/response, create a promise, and let someone else resolve it, that is somewhere else in the code, and then flush the response.
What I find really strange is that the Promise promise = task {} interface has no method that resembles resolve or similar.
I need to lock the response until someone resolves the promise, which is a global/static property set in development mode.
Promise interface:
http://grails.org/doc/latest/api/grails/async/Promise.html
I have looked at the GPars doc and can't find anything there that resembles a resolve method.
How can I create a promise, that locks the response or request, and then flushes the response when someone resolves it?
You can call get() on the promise which will block until whatever the task is doing completes, but I imagine what that is not what you want. What you want seems to be equivalent to a GPars DataflowVariable:
http://gpars.org/1.0.0/javadoc/groovyx/gpars/dataflow/DataflowVariable.html
Which allows using the left shift operator to resolve the value from another thread. Currently there is no way to use the left shift operator via Grails directly, but since Grails' promise API is just a layer over GPars this can probably be accomplished by using the GPars API directly with something like:
import org.grails.async.factory.gpars.*
import groovyx.gpars.dataflow.*
import static grails.async.Promise.*
def myAction() {
def dataflowVar = new DataflowVariable()
task {
// do some calculation and resolve data flow variable
def expensiveData = ...
dataflowVar << expensiveData
}
return new GParsPromise(dataflowVar)
}
It took me quite some time to get around this and have a working answer.
I must say that it appears as if Grails is quite a long way of making this work properly.
task { }
will always execute immediatly, so the call is not put on hold until dispatch() or whatever is invoked which is a problem.
Try this to see:
public def test() {
def dataflowVar = new groovyx.gpars.dataflow.DataflowVariable()
task {
// do some calculation and resolve data flow variable
println '1111111111111111111111111111111111111111111111111111'
//dataflowVar << expensiveData
}
return new org.grails.async.factory.gpars.GparsPromise(dataflowVar);
}
If you are wondering what this is for, it is to make the lesscss refresh automatically in grails, which is a problem when you are using import statements in less. When the file is touched, the lesscss compiler will trigger a recompilation, and only when it is done should it respond to the client.
On the client side I have some javascript that keeps replacing the last using the refresh action here:
In my controller:
/**
* Refreshes link resources. refresh?uri=/resource/in/web-app/such/as/empty.less
*/
public def refresh() {
return LessRefresh.stackRequest(request, params.uri);
}
A class written for this:
import grails.util.Environment
import grails.util.Holders
import javax.servlet.AsyncContext
import javax.servlet.AsyncEvent
import javax.servlet.AsyncListener
import javax.servlet.http.HttpServletRequest
/**
* #Author SecretService
*/
class LessRefresh {
static final Map<String, LessRefresh> FILES = new LinkedHashMap<String, LessRefresh>();
String file;
Boolean touched
List<AsyncContext> asyncContexts = new ArrayList<AsyncContext>();
String text;
public LessRefresh(String file) {
this.file = file;
}
/** Each request will be put on hold in a stack until dispatchAll below is called when the recompilation of the less file finished **/
public static AsyncContext stackRequest(HttpServletRequest request, String file) {
if ( !LessRefresh.FILES[file] ) {
LessRefresh.FILES[file] = new LessRefresh(file);
}
return LessRefresh.FILES[file].handleRequest(request);
}
public AsyncContext handleRequest(HttpServletRequest request) {
if ( Environment.current == Environment.DEVELOPMENT ) {
// We only touch it once since we are still waiting for the less compiler to finish from previous edits and recompilation
if ( !touched ) {
touched = true
touchFile(file);
}
AsyncContext asyncContext = request.startAsync();
asyncContext.setTimeout(10000)
asyncContexts.add (asyncContext);
asyncContext.addListener(new AsyncListener() {
#Override
void onComplete(AsyncEvent event) throws IOException {
event.getSuppliedResponse().writer << text;
}
#Override
void onTimeout(AsyncEvent event) throws IOException {
}
#Override
void onError(AsyncEvent event) throws IOException {
}
#Override
void onStartAsync(AsyncEvent event) throws IOException {
}
});
return asyncContext;
}
return null;
}
/** When recompilation is done, dispatchAll is called from LesscssResourceMapper.groovy **/
public void dispatchAll(String text) {
this.text = text;
if ( asyncContexts ) {
// Process all
while ( asyncContexts.size() ) {
AsyncContext asyncContext = asyncContexts.remove(0);
asyncContext.dispatch();
}
}
touched = false;
}
/** A touch of the lessfile will trigger a recompilation **/
int count = 0;
void touchFile(String uri) {
if ( Environment.current == Environment.DEVELOPMENT ) {
File file = getWebappFile(uri);
if (file && file.exists() ) {
++count;
if ( count < 5000 ) {
file << ' ';
}
else {
count = 0
file.write( file.getText().trim() )
}
}
}
}
static File getWebappFile(String uri) {
new File( Holders.getServletContext().getRealPath( uri ) )
}
}
In LesscssResourceMapper.groovy of the lesscsss-recources plugin:
...
try {
lessCompiler.compile input, target
// Update mapping entry
// We need to reference the new css file from now on
resource.processedFile = target
// Not sure if i really need these
resource.sourceUrlExtension = 'css'
resource.contentType = 'text/css'
resource.tagAttributes?.rel = 'stylesheet'
resource.updateActualUrlFromProcessedFile()
// ==========================================
// Call made here!
// ==========================================
LessRefresh.FILES[resource.sourceUrl.toString()]?.dispatchAll( target.getText() );
} catch (LessException e) {
log.error("error compiling less file: ${originalFile}", e)
}
...
In the index.gsp file:
<g:set var="uri" value="${"${App.files.root}App/styles/empty.less"}"/>
<link media="screen, projection" rel="stylesheet" type="text/css" href="${r.resource(uri:uri)}" refresh="${g.createLink(controller:'home', action:'refresh', params:[uri:uri])}" resource="true">
JavaScript method refreshResources to replace the previous link href=...
/**
* Should only be used in development mode
*/
function refreshResources(o) {
o || (o = {});
var timeoutBegin = o.timeoutBegin || 1000;
var intervalRefresh = o.intervalRefresh || 1000;
var timeoutBlinkAvoid = o.timeoutBlinkAvoid || 400 ;
var maxErrors = o.maxErrors || 200 ;
var xpath = 'link[resource][type="text/css"]';
// Find all link[resource]
$(xpath).each(function(i, element) {
refresh( $(element) );
});
function refresh(element) {
var parent = element.parent();
var next = element.next();
var outer = element.clone().attr('href', '').wrap('<p>').parent().html();
var uri = element.attr('refresh');
var errorCount = 0;
function replaceLink() {
var link = $(outer);
link.load(function () {
// The link has been successfully added! Now remove the other ones, then do again
errorCount = 0;
// setTimeout needed to avoid blinking, we allow duplicates for a few milliseconds
setTimeout(function() {
var links = parent.find(xpath + '[refresh="'+uri+'"]');
var i = 0;
// Remove all but this one
while ( i < links.length - 1 ) {
links[i++].remove();
}
replaceLinkTimeout();
}, timeoutBlinkAvoid );
});
link.error(function(event, handler) {
console.log('Error refreshing: ' + outer );
++errorCount;
if ( errorCount < maxErrors ) {
// Load error, it happens. Remove this & redo!
link.remove();
replaceLink();
}
else {
console.log('Refresh: Aborting!')
}
});
link.attr('href', urlRandom(uri)).get(0);
link.insertBefore(next); // Insert just after
}
function urlRandom(uri) {
return uri + "&rand=" + Math.random();
}
function replaceLinkTimeout() {
setTimeout(function() {
replaceLink();
}, intervalRefresh ) ;
}
// Waith 1s before triggering the interval
setTimeout(function() {
replaceLinkTimeout();
}, timeoutBegin);
}
};
Comments
I am unsure why Javascript style promises have not been added to the Grails stack.
You can not render or stuff like that in the onComplete. render, redirect and what not are not available.
Something tells me that Grails and Promises/Futures are not there yet. The design of the GPars libraries seems not take into account of the core features which is to resolve later. At least it is not simple to do so.
It would be great if the dispatch() method actually could be invoked with some paramaters to pass from the resolving context. I am able to go around this using static properties.
I might continue to write my own solution and possibly contribute with a more fitting solutions around the AsyncContext class, but for now, this is enough for me.
I just wanted to refresh my less resources automatically.
Phew...
EDIT:
I made it to support several number of files. It is complete now!

Prevent to read file many times

I am trying to write an i18n app. The program read a json file, that contains translation from languages and it based on json structure.
{
"EN": {
"TEXT1": "Hello",
"TEXT2": "March"
},
"DE": {
"TEXT1": "Hallo",
"TEXT2": "März"
}
}
My program read the json file in async way with the file class, the whole code
import 'dart:io';
import 'dart:async';
import 'package:json_object/json_object.dart';
abstract class I18n {
static _I18n _i18n;
factory I18n(String file, String lang) {
if(_i18n == null) {
_i18n = new _I18n(file, lang);
return _i18n;
}
return _i18n;
}
Future<String> getTextByMap(String textId);
}
class _I18n implements I18n {
File _file;
String _lang;
JsonObject _jsonContainer;
JsonObject _jsonFiltered;
Future<JsonObject> _imme;
// Parameters:
// file: The whole path and filename
// lang: Expected language
_I18n(String file, this._lang) {
this._file = new File(file);
}
// Read file and return the content of file.
Future<String> _readFileFromStream() {
var com = new Completer();
this._file.exists()
.then((fileExists) {
if(!fileExists) {
throw new StateError('File not found');
}
return this._file.readAsString()
.then((stream) => com.complete(stream));
});
return com.future;
}
void _convertContentToJson(String stream) {
this._jsonContainer = new JsonObject.fromJsonString(stream);
}
Future<JsonObject> _prepareData() {
return this._readFileFromStream().then((stream) {
_convertContentToJson(stream);
this._jsonFiltered = this._jsonContainer[this._lang];
return this._jsonFiltered;
});
}
Future<String> getTextByMap(String textId) {
return this._prepareData().then((filterd) {
return filterd[textId];
});
}
}
and the main code
import 'package:i18n/i18n.dart';
void main() {
var i18n = new I18n('../hello.json', 'EN');
i18n.getTextByMap('TEXT1').then((val) => print(val));
i18n.getTextByMap('TEXT2').then((val) => print(val));
}
Everything here, happen in dart async way, read json file etc. And everytime, when i call the method
i18n.getTextByMap('TEXT1').then((val) => print(val));
it gonna read the json file again and again. I tried to rewrite the method to prevent reading json file many times
Future<String> getTextByMap(String textId) {
if(this._jsonFiltered == null)
{
return this._prepareData().then((filterd) {
return filterd[textId];
});
}
return new Future(() => this._jsonFiltered[textId]);
}
but it doesn't work too, because dart works in async way.
My question is, how can i keep this json file content in an object? Read json file only one time and keep the contents in an object, it is better then read json file everytime, that is my opinion.
It could do everything in sync way, then i wouldn't have such as problem but this is not dart terminology.
In which order do dart execute I/O operations, like this?
Future
I/O Events
My solution would be to create a class with a factory constructor. The factory constructor always returns a object of that file.
Your problem is that futures are parallel. So both calls are executed in parallel. The solution is to let the first future complete and then do other stuff to be able to get cached results.
Then you can have a read() method that reads the value of the file if it is not present in the classes "contents" attribute for example - or if that attribute is not null, it loads the file in background.
In both cases a completer or future is returned you can listen on.
EDIT Example Code:
example_async_file_factory.dart
import 'dart:io';
import 'dart:async';
class FileHolder {
String _contents = null;
String path;
static Map<String, FileHolder> _files;
factory FileHolder(String path) {
if (_files == null) {
_files = {};
}
if (_files.containsKey(path)) {
return _files[path];
} else {
final fh = new FileHolder._internal(path);
_files[path] = fh;
return fh;
}
}
FileHolder._internal(this.path);
Future<String> getContents() {
if(_contents != null) {
print("cached");
return new Future.value(_contents);
} else {
print("read");
File f = new File(this.path);
Future<String> future = f.readAsString();
Completer completer = new Completer();
future.then((String c) {
_contents = c;
completer.complete(_contents);
});
return completer.future;
}
}
}
void main() {
FileHolder f = new FileHolder("example_async_file_factory.dart");
f.getContents().then((String contents) {
print(contents.length);
FileHolder f2 = new FileHolder("example_async_file_factory.dart");
f2.getContents().then((String contents) {
print(contents.length);
});
f2.getContents().then((String contents) {
print(contents.length);
});
f.getContents().then((String contents) {
print(contents.length);
});
});
}
Output:
read
1411
cached
cached
cached
1411
1411
1411
Regards
Robert

How return Future from Future? Or this is prohibited in async library?

How I can return Future value from Future object?
This code does not work.
import 'dart:async';
void main() {
var temp = foo();
temp.then((Future<int> future) {
future.then((int result) {
print(result);
});
});
}
Future<Future<int>> foo() {
return new Future<Future<int>>(() {
return new Future<int>(() => 5);
});
}
How to prevent unnecessary unwrapping?
In this case in async library 'Future' declared as generic class.
abstract class Future<T> {
}
If I create expression as the following
new Future<Future<int>>();
Then with type T specified as Future<int> which result expected from generic class Future?
I thing that result must be as specified in type argument T.
I.e. Future<int>.
But result is not as expected.
There is no information found about this abnormal behavior on Dart API site.
If this is a "feature" (but I think that abnormal behavior wrongly to call "feature') then why it not documented in Dart API?
How can be explained this discrepancy?
Why this code not generated errors and warnings?
Another IDENTICAL example but w/o using Future.
void main() {
var temp = foo();
temp.baz((Foo<int> foo) {
foo.baz((int result) {
print(result);
});
});
}
Foo<Foo<int>> foo() {
return new Foo<Foo<int>>(() {
return new Foo<int>(() => 5);
});
}
If in this case result will be as when using Future (i.e. unexpected) then how we can call this code?
Normal or abnormal?
Or maybe the Future in Dart some special (magic)?
Look at the api documentation
http://api.dartlang.org/docs/releases/latest/dart_async/Future.html
It says there:
If the returned value is itself a Future, completion of the created future will wait until
the returned future completes, and will then complete with the same result.
I guess that means you can't return a Future from a Future.
But you could return a list of futures.
void main() {
var temp = foo();
temp.then((List<Future<int>> list) {
list[0].then((int result) {
print(result);
});
});
}
Future<List<Future<int>>> foo() {
return new Future<List<Future<int>>>(() {
return [new Future<int>(() => 5)];
});
}
There is no need for any of that extra wrapping. According to the Future documentation:
If the returned value is itself a [Future], completion of the created
future will wait until the returned future completes, and will then
complete with the same result.
This means you can rewrite your code as:
import 'dart:async';
void main() {
var temp = foo();
temp.then((int result) {
print(result);
});
}
Future<int> foo() {
return new Future<int>(() {
return new Future<int>(() => 5);
});
}
This is a lot cleaner to work with and provides the expected result.

Resources