Prevent to read file many times - dart

I am trying to write an i18n app. The program read a json file, that contains translation from languages and it based on json structure.
{
"EN": {
"TEXT1": "Hello",
"TEXT2": "March"
},
"DE": {
"TEXT1": "Hallo",
"TEXT2": "März"
}
}
My program read the json file in async way with the file class, the whole code
import 'dart:io';
import 'dart:async';
import 'package:json_object/json_object.dart';
abstract class I18n {
static _I18n _i18n;
factory I18n(String file, String lang) {
if(_i18n == null) {
_i18n = new _I18n(file, lang);
return _i18n;
}
return _i18n;
}
Future<String> getTextByMap(String textId);
}
class _I18n implements I18n {
File _file;
String _lang;
JsonObject _jsonContainer;
JsonObject _jsonFiltered;
Future<JsonObject> _imme;
// Parameters:
// file: The whole path and filename
// lang: Expected language
_I18n(String file, this._lang) {
this._file = new File(file);
}
// Read file and return the content of file.
Future<String> _readFileFromStream() {
var com = new Completer();
this._file.exists()
.then((fileExists) {
if(!fileExists) {
throw new StateError('File not found');
}
return this._file.readAsString()
.then((stream) => com.complete(stream));
});
return com.future;
}
void _convertContentToJson(String stream) {
this._jsonContainer = new JsonObject.fromJsonString(stream);
}
Future<JsonObject> _prepareData() {
return this._readFileFromStream().then((stream) {
_convertContentToJson(stream);
this._jsonFiltered = this._jsonContainer[this._lang];
return this._jsonFiltered;
});
}
Future<String> getTextByMap(String textId) {
return this._prepareData().then((filterd) {
return filterd[textId];
});
}
}
and the main code
import 'package:i18n/i18n.dart';
void main() {
var i18n = new I18n('../hello.json', 'EN');
i18n.getTextByMap('TEXT1').then((val) => print(val));
i18n.getTextByMap('TEXT2').then((val) => print(val));
}
Everything here, happen in dart async way, read json file etc. And everytime, when i call the method
i18n.getTextByMap('TEXT1').then((val) => print(val));
it gonna read the json file again and again. I tried to rewrite the method to prevent reading json file many times
Future<String> getTextByMap(String textId) {
if(this._jsonFiltered == null)
{
return this._prepareData().then((filterd) {
return filterd[textId];
});
}
return new Future(() => this._jsonFiltered[textId]);
}
but it doesn't work too, because dart works in async way.
My question is, how can i keep this json file content in an object? Read json file only one time and keep the contents in an object, it is better then read json file everytime, that is my opinion.
It could do everything in sync way, then i wouldn't have such as problem but this is not dart terminology.
In which order do dart execute I/O operations, like this?
Future
I/O Events

My solution would be to create a class with a factory constructor. The factory constructor always returns a object of that file.
Your problem is that futures are parallel. So both calls are executed in parallel. The solution is to let the first future complete and then do other stuff to be able to get cached results.
Then you can have a read() method that reads the value of the file if it is not present in the classes "contents" attribute for example - or if that attribute is not null, it loads the file in background.
In both cases a completer or future is returned you can listen on.
EDIT Example Code:
example_async_file_factory.dart
import 'dart:io';
import 'dart:async';
class FileHolder {
String _contents = null;
String path;
static Map<String, FileHolder> _files;
factory FileHolder(String path) {
if (_files == null) {
_files = {};
}
if (_files.containsKey(path)) {
return _files[path];
} else {
final fh = new FileHolder._internal(path);
_files[path] = fh;
return fh;
}
}
FileHolder._internal(this.path);
Future<String> getContents() {
if(_contents != null) {
print("cached");
return new Future.value(_contents);
} else {
print("read");
File f = new File(this.path);
Future<String> future = f.readAsString();
Completer completer = new Completer();
future.then((String c) {
_contents = c;
completer.complete(_contents);
});
return completer.future;
}
}
}
void main() {
FileHolder f = new FileHolder("example_async_file_factory.dart");
f.getContents().then((String contents) {
print(contents.length);
FileHolder f2 = new FileHolder("example_async_file_factory.dart");
f2.getContents().then((String contents) {
print(contents.length);
});
f2.getContents().then((String contents) {
print(contents.length);
});
f.getContents().then((String contents) {
print(contents.length);
});
});
}
Output:
read
1411
cached
cached
cached
1411
1411
1411
Regards
Robert

Related

Flutter: Reading a file, parsing as JSON, then converting to an object in order to use dot notation

I am trying to make a way to read a file with data saved in a specific format, parse it to JSON then convert it to an object so that I can use dot notation.
The problem here is using dot notation as it just returns null
CoreData.dart
import 'dart:convert';
import 'dart:io';
import 'dart:async';
import 'package:path_provider/path_provider.dart';
#proxy
class CoreObject {
Map _data;
CoreObject([String source]) {
Map json = (source == null) ? new Map() : JSON.decode(source);
_data = new Map.from(json);
json.forEach((k, v) {
print(k);
_data[k] = v;
});
}
static encode(List<CoreObject> list) {
String result = "";
for (CoreObject item in list) {
result += "${item.toString()};";
}
return result;
}
#override toString() {
print(this._data);
return JSON.encode(this._data);
}
#override
noSuchMethod(Invocation invocation) {
var name = invocation.memberName.toString().replaceFirst('Symbol(\"', "");
print("_data.keys ${_data.keys}");
print("_data.values ${_data.values}");
if (invocation.isGetter) {
print("name ${name.replaceAll("\")", "")}");
var ret = _data[name.replaceAll("\")", "")];
print("ret $ret");
print(ret.toString());
return ret;
}
if (invocation.isSetter) {
_data[name.replaceAll("=\")", "")] = invocation.positionalArguments.first;
} else {
super.noSuchMethod(invocation);
}
}
}
class Person extends CoreObject {
Person([source]): super(source);
#override noSuchMethod(Invocation invocation) {
super.noSuchMethod(invocation);
}
}
class CoreContainer {
String _object;
var returnNew;
var path;
_map(String source) {
var result = [];
for (var line in source.split(";")) {
// print("line $line");
if (line != "") result.add(returnNew(line));
}
print("result $result");
return result;
}
encode(List<CoreObject> list) {
// print("list $list");
String result = "";
list.forEach((CoreObject item) {
// print("item ${item.toString()}");
result += "${item};";
});
// print("result $result");
return result;
}
CoreContainer(this._object, this.returnNew);
Future<File> _getFile() async {
String dir = path ?? (await getApplicationDocumentsDirectory()).path;
this.path = dir;
return new File('$dir/$_object.txt');
}
Future<List<CoreObject>> getAll() async {
return _getFile().then((File file) {
String contents = file.readAsStringSync();
print("contents $contents");
return this._map(contents);
})
.catchError((Error error) {
print('error: $error');
_getFile().then((File file) {
file.writeAsStringSync("");
});
return [];
});
}
save(List<CoreObject> data) async {
_getFile().then((file) {
try {
file.writeAsStringSync(this.encode(data));
}
catch (error) {
print("error: $error");
}
}).catchError((Error error) {
print("error: $error");
});
}
clear() async {
return _getFile().then((file) {
file.writeAsStringSync("");
}).catchError((Error error) {
print("error: $error");
});
}
Future<List<CoreObject>> get(query) async {
return this.getAll().then((List data) {
data.retainWhere(query);
return data;
}).catchError((error) {
print("error: $error");
});
}
Future<List<CoreObject>> remove(query) async {
return this.getAll().then((List data) {
// print(data);
data.removeWhere(query);
save(data);
return data;
}).catchError((error) {
print("error: $error");
});
}
Future<List<CoreObject>> add(obj) async {
return this.getAll().then((data) {
data.add(obj);
return save(data).then(() {
return data;
})
.catchError((Error error) {
throw error;
});
}).catchError((Error error) {
print("error: $error");
});
}
}
Using it:
CoreContainer corePerson = new CoreContainer("Person", (source) => new Person(source));
corePerson.getAll().then((List<CoreObject> array) {
var tempItems = [];
var i = 0;
print("array $array");
while (i < array.length) {
Person person = array[i];
print(person); //{"name":"<whatever 'i' is>"}
print(person.name); //null
tempItems.add(new ListTile(
title: new Text("$i"),
subtitle: new Text("${person.name}"),
));
i++;
}
print(tempItems.length);
count = tempItems.length;
setState(() {
items = tempItems;
});
}).catchError((Error error) {
print("error: $error, ${error.stackTrace}");
});
Code is hard to read because of a lot of print debugging.
But I suppose you need a way to convert JSON data into a Dart class.
You should use library like jaguar_serializer that do the job for you.
https://pub.dartlang.org/packages/jaguar_serializer
Dart doesn't use dot notation like dynamic languages (Python, JavaScript). In Python and JavaScript, for example, every single object is actually internally a HashMap, and . is actually a hash lookup of the property name:
a.bar // Loosely the same as a.lookup('bar')
The Python/JS VM though can "see" that a.bar is used like a property on a class-like object a, and optimize it to use a true property/field access - this is part of the "optimization" phase of a JIT (just-in-time compiler).
It is features like this that make it almost impossible to ahead-of-time compile either Python or JS - they require runtime profile information to generate fast code. Dart (and specifically Dart 2.0) is implementing a sound type system where a.bar, when a is known, is always a property accessor, not a hash lookup.
That means at runtime you can't take an arbitrary hash map and force it to act like an object, which is why your code seems awkward. I'd recommend using code generation if you need a typed object with . notation, or settling for a HashMap [] if you do not.
Check also mapping json into class objects answers for example of clean basic way of json -> dart class mapping.

Questions about using Futures and Completers

Hey have I read all I can find about futures, but I would like some more advice on proper usage.
I am writing an API library, that bridges the gap between HTTP Requests and the app. So I use the future returned by HTTP in most cases, however sometimes the data is already retrieved. Is that the appropriate time to use a Completer?
ex.
String _someData = "";
Future<String> getSomeData(){
if (_someData == ""){
return Api.getSomeData().then((String someData){
_someData = someData;
return _someData;
});
} else {
var completer = new Completer();
completer.complete(_someData);
return completer.future;
}
}
-edit- Also if I create a Completer, but end up not using its future or calling complete. Will that cause a mem leak? Should I call its complete method or dispose of it somehow?
Thanks :)
You can use the named constructor Future.value if the value is immediately accessible. You won't need a Completer.
String _someData = "";
Future<String> getSomeData(){
if (_someData == ""){
return Api.getSomeData().then((String someData){
_someData = someData;
return _someData;
});
} else {
return new Future.value(_someData);
}
}
And for your second question, if you create a Completer without using it, I guess the garbage collector will simply free its memory when there won't be anymore references to it in your code.
Use an async function instead.
import "dart:async";
String _someData = "";
Future<String> getSomeData() async {
if (_someData == "") {
_someData = await Api.getSomeData();
}
return _someData;
}
Compiler generates approximately the following code:
import "dart:async";
String _someData = "";
Future<String> getSomeData() {
var $_awaiter = new Completer<String>();
try {
if (_someData == "") {
Api.getSomeData().then(($) {
$_awaiter.complete($);
}).catchError((e, s) {
$_awaiter.completeError(e, s);
});
return $_awaiter.future;
} else {
$_awaiter.complete(_someData);
return $_awaiter.future;
}
} catch (e, s) {
$_awaiter.completeError(e, s);
return $_awaiter.future;
}
}

What is the best/shortest way to convert an Iterable to a Stream, in Dart?

I have an Iterable, and I'd like to convert it to a Stream. What is the most efficient/shortest-amount-of-code to do this?
For example:
Future convert(thing) {
return someAsyncOperation(thing);
}
Stream doStuff(Iterable things) {
return things
.map((t) async => await convert(t)) // this isn't exactly what I want
// because this returns the Future
// and not the value
.where((value) => value != null)
.toStream(); // this doesn't exist...
}
Note: iterable.toStream() doesn't exist, but I want something like that. :)
Here's a simple example:
var data = [1,2,3,4,5]; // some sample data
var stream = new Stream.fromIterable(data);
Using your code:
Future convert(thing) {
return someAsyncOperation(thing);
}
Stream doStuff(Iterable things) {
return new Stream.fromIterable(things
.map((t) async => await convert(t))
.where((value) => value != null));
}
In case you are using the Dart SDK version 1.9 or a newer one, you could easily create a stream using async*
import 'dart:async';
Future convert(thing) {
return new Future.value(thing);
}
Stream doStuff(Iterable things) async* {
for (var t in things) {
var r = await convert(t);
if (r != null) {
yield r;
}
}
}
void main() {
doStuff([1, 2, 3, null, 4, 5]).listen(print);
}
Maybe it is easier to read as it has less braces and "special" methods, but that is a matter of taste.
If you want to sequentially process each item in the iterable you can use Stream.asyncMap:
Future convert(thing) {
return waitForIt(thing); // async operation
}
f() {
var data = [1,2,3,4,5];
new Stream.fromIterable(data)
.asyncMap(convert)
.where((value) => value != null))
}

How to combine traditional and Future-based API in one interface in Dart?

I would like to parse binary file (from some old game) as on desktop as in browser.
So, I should use abstract class, which can read binary data from array of bytes:
abstract class BinData {
int readByte();
String readNullString(){
var buffer = new StringBuffer();
int char;
do {
char = readByte();
if (char == 0){
break;
}
buffer.writeCharCode(char);
} while(true);
return buffer.toString();
}
}
Now I can implement my parser. For example:
class Parser {
BinData _data;
void load(BinData data){
...
}
}
For desktop console application I use dart:io RandomAccessFile:
class FileBinData extends BinData {
RandomAccessFile _file;
FileBinData.from(RandomAccessFile file){
this._file = file;
}
int readByte(){
return this._file.readByteSync();
}
}
For web application I have to use dart:html FileReader. However, this class has only Future-based API, which isn't compatible with my interface:
class WebFileBinData extends BinData {
File _file;
int _position = 0;
WebFileBinData.from(File file){
this._file = file;
}
int readByte(){
Blob blob = _file.slice(_position, _position + 1);
FileReader reader = new FileReader();
var future = reader.onLoad.map((e)=>reader.result).first
.then((e) { ... });
reader.readAsArrayBuffer(blob);
...
}
}
How can I solve it?
Your readByte() should return Future<int> instead of int. You can return a Future from a function/method even when it doesn't do any async operation (return new Future.value(5);) but you can not return int (or any non-Future value) from a function which executes async operations, at least not when the value should be returned as result of the async operation.
You also need to ensure to connect all async calls.
Future<int> readByte(){
return reader.onLoad.map((e)=>reader.result).first
.then((e) {
...
return reader.readAsArrayBuffer(blob);
});
** readNullString
Future<String> readNullString() {
var buffer = new StringBuffer();
int char;
return Future.doWhile(() {
return readByte().then((char) {
if (char == 0) {
return false; // end doWhile
}
buffer.writeCharCode(char);
return true; // continue doWhile
});
}).then((_) => buffer.toString()); // return function result
}

How do I create a blank Future in Dart + how do I return a future currently in progress?

I'm trying to create a server-side Dart class that performs various data-related tasks. All of these tasks rely on the database having been first initialized. The problem is that the init of the database happens asynchronously (returns a Future). I first tried to put the init code into the constructor, but have given up on this approach as it seems to not be viable.
I am now attempting to figure out how to force the DB initialization as a first step in any method call that accesses data. So in other words, when attemptLogin() is called below, I'd like to first check if the DB has been initialized and initialize it if necessary.
However, there are two obstacles. If the database hasn't been initialized, the code is straightforward - initialize the db, then use the then() method of the returned future to do the rest of the function. If the db is not yet initialized, what do I attach my then() method to?
Second related question is what happens when a database is currently being initialized but this process is not yet complete? How can I pull in and return this "in-progress" Future?
This is the basic gist of the code I'm trying to wrangle:
class DataManager {
bool DbIsReady = false;
bool InitializingDb = false;
Db _db;
Future InitMongoDB() {
print("Initializing MongoDB");
InitializingDb = true;
_db = new Db("mongodb://127.0.0.1/test");
return _db.open().then((_) {
DbIsReady = true;
InitializingDb = false;
});
}
Future<List> attemptLogin(String username, String password) {
Future firstStep;
if ((!DbIsReady) && (!InitializingDb) {
Future firstStep = InitMongoDB()
}
else if (InitializingDb) {
// Need to return the InitMongoDB() Future that's currently running, but how?
}
else {
// How do I create a blank firstStep here?
}
return firstStep.then((_) {
users = _db.collection("users");
return // ... rest of code cut out for clarity
});
}
}
Thanks in advance for your help,
Greg
Just return
return new Future<bool>.value(true);
// or any other value instead of `true` you want to return.
// or none
// return new Future.value();
Just keep the future alive:
class DataManager {
Future _initializedDb;
Future initMongoDb() { ... }
Future<List> attemptLogin(String username, String password) {
if (_initializedDb == null) {
_initializedDb = initMongoDB();
}
return _initializedDb.then((db) {
users = db.collection("users");
return // ... rest of code cut out for clarity
});
}
}
You might need to pay attention for the error-case. It's up to you if you want to deal with errors in the initMongoDB or after it.
One of the possible solutions:
import "dart:async";
void main() {
var dm = new DataManager();
var selectOne = dm.execute("SELECT 1");
var selectUsers = dm.execute("SELECT * FROM users");
var users = selectOne.then((result) {
print(result);
return selectUsers.then((result) {
print(result);
});
});
users.then((result) {
print("Goodbye");
});
}
class Event {
List<Function> _actions = new List<Function>();
bool _raised = false;
void add(Function action) {
if (_raised) {
action();
} else {
_actions.add(action);
}
}
void raise() {
_raised = true;
_notify();
}
void _notify() {
if (_actions.isEmpty) {
return;
}
var actions = _actions.toList();
_actions.clear();
for (var action in actions) {
action();
}
}
}
class DataManager {
static const int _STATE_NOT_INITIALIZED = 1;
static const int _STATE_INITIALIZING = 2;
static const int _STATE_READY = 3;
Event _initEvent = new Event();
int _state = _STATE_NOT_INITIALIZED;
Future _init() {
if (_state == _STATE_NOT_INITIALIZED) {
_state = _STATE_INITIALIZING;
print("Initializing...");
return new Future(() {
print("Initialized");
_state = _STATE_READY;
_initEvent.raise();
});
} else if (_state == _STATE_INITIALIZING) {
print("Waiting until initialized");
var completer = new Completer();
_initEvent.add(() => completer.complete());
return completer.future;
}
return new Future.value();
}
Future execute(String query, [Map arguments]) {
return _init().then((result) {
return _execute(query, arguments);
});
}
Future _execute(String query, Map arguments) {
return new Future.value("query: $query");
}
}
Output:
Initializing...
Waiting until initialized
Initialized
query: SELECT 1
query: SELECT * FROM users
Goodbye
I think that exist better solution but this just an attempt to answer on your question (if I correctly understand you).
P.S. EDITED at 11 July 2014
Slightly modified (with error handling) example.
import "dart:async";
void main() {
var dm = new DataManager();
var selectOne = dm.execute("SELECT 1");
var selectUsers = dm.execute("SELECT * FROM users");
var users = selectOne.then((result) {
print(result);
return selectUsers.then((result) {
print(result);
});
});
users.then((result) {
print("Goodbye");
});
}
class DataManager {
static const int _STATE_NOT_INITIALIZED = 1;
static const int _STATE_INITIALIZING = 2;
static const int _STATE_READY = 3;
static const int _STATE_FAILURE = 4;
Completer _initEvent = new Completer();
int _state = _STATE_NOT_INITIALIZED;
Future _ensureInitialized() {
switch (_state) {
case _STATE_NOT_INITIALIZED:
_state = _STATE_INITIALIZING;
print("Initializing...");
new Future(() {
print("Initialized");
_state = _STATE_READY;
// throw null;
_initEvent.complete();
}).catchError((e, s) {
print("Failure");
_initEvent.completeError(e, s);
});
break;
case _STATE_INITIALIZING:
print("Waiting until initialized");
break;
case _STATE_FAILURE:
print("Failure detected");
break;
default:
print("Aleady intialized");
break;
}
return _initEvent.future;
}
Future execute(String query, [Map arguments]) {
return _ensureInitialized().then((result) {
return _execute(query, arguments);
});
}
Future _execute(String query, Map arguments) {
return new Future.value("query: $query");
}
}
For those that are still wondering how to create a blank Future in Dart and later complete them, you should use the Completer class like in the next example.
class AsyncOperation {
final Completer _completer = new Completer();
Future<T> doOperation() {
_startOperation();
return _completer.future; // Send future object back to client.
}
// Something calls this when the value is ready.
void finishOperation(T result) {
_completer.complete(result);
}
// If something goes wrong, call this.
void _errorHappened(error) {
_completer.completeError(error);
}
}
Future<Type> is non nullable in Dart, meaning that you have to initialize it to a value. If you don't, Dart throws the following error:
Error: Field should be initialized because its type 'Future<Type>' doesn't allow null.
To initialize a Future<Type>, see the following example:
Future<String> myFutureString = Future(() => "Future String");
Here "Future String" is a String and so the code above returns an instance of Future<String>.
So coming to the question of how to create a blank/empty Future, I used the following code for initializing an empty Future List.
Future<List> myFutureList = Future(() => []);
I found this link to be quite useful in understanding Futures in Flutter and Dart: https://meysam-mahfouzi.medium.com/understanding-future-in-dart-3c3eea5a22fb

Resources