With M3 the classes like StringInputStream are replaced with Stream. How can I read stdin input on a server application?
Try this:
import 'dart:io';
import 'dart:async';
void main() {
print("Please, enter a line \n");
Stream cmdLine = stdin
.transform(new StringDecoder())
.transform(new LineTransformer());
StreamSubscription cmdSubscription = cmdLine.listen(
(line) => print('Entered line: $line '),
onDone: () => print(' finished'),
onError: (e) => /* Error on input. */);
}
Related
I am creating a dart cli that allows to display one or more colors at random but what I would like to do is that when we press the "space" key it displays a new color.
Code in nodeJS that I want to reproduce in dart. Here the module used is Terminal-kit
term.grabInput(true)
function checkKeyPress(name) {
//Si la touche espace est appuyée, on relance le programme
if (name === ' ') {
term.grabInput(false)
term.removeListener('key', checkKeyPress)
//Faire attendre 0.1 secondes
setTimeout(() => {
colornew()
}, 80)
}
if (name === 'ESCAPE') {
term.grabInput(false)
term.removeListener('key', checkKeyPress)
setTimeout(() => {
console.clear()
process.stdout.write('\x1Bc');
return main()
}, 80)
}
}
term.addListener('key', checkKeyPress)
}
colornew()
You can do it by disabling lineMode on stdin which makes it so your terminal are sending data to your data program right away instead of waiting for the next line.
I would also recommend disabling echoMode so you can't see the space in the terminal.
So something like this:
import 'dart:io';
final int spaceChar = ' '.codeUnitAt(0);
void main() async {
stdin.echoMode = false;
stdin.lineMode = false;
await for (final event in stdin) {
if (event.first == spaceChar) {
print('SPACE!!!!!');
}
}
}
Just starting with Deno, I am trying to figure out how to calculate a binary file checksum. It seems to me that the problem is not with the methods provided by the hash module of the standard library, but with the file streaming method and/or the type of the chunks feeding the hash.update method.
I have been trying a few alternatives, related to file opening and chunk types,with no success. A simple example is in the following:
import {createHash} from "https://deno.land/std#0.80.0/hash/mod.ts";
const file= new File(["my_big_folder.tar.gz"], "./my_big_folder.tar.gz");
const iterator = file.stream() .getIterator();
const hash = createHash("md5");
for await( let chunk of iterator){
hash.update(chunk);
}
console.log(hash.toString()); //b35edd0be7acc21cae8490a17c545928
This code compiles and runs with no errors, pity that the result is different from what I get running the functions of the crypto module provided by node and the md5sum provided by linux coreutils. Any suggestion ?
nodejs code:
const crypto = require('crypto');
const fs = require('fs');
const hash = crypto.createHash('md5');
const file = './my_big_folder.tar.gz';
const stream = fs.ReadStream(file);
stream.on('data', data=> { hash.update(data); });
stream.on('end', ()=> {
console.log(hash.digest('hex')); //c18f5eac67656328f7c4ec5d0ef5b96f
});
The same result in bash:
$ md5sum ./my_big_folder.tar.gz
$ c18f5eac67656328f7c4ec5d0ef5b96f ./my_big_folder.tar.gz
on Windows 10 this can be used:
CertUtil -hashfile ./my_big_folder.tar.gz md5
The File API isn't used to read a File in Deno, to do that you need to use the Deno.open API and then turn it into an iterable like this
import {createHash} from "https://deno.land/std#0.80.0/hash/mod.ts";
const hash = createHash("md5");
const file = await Deno.open(new URL(
"./BigFile.tar.gz",
import.meta.url, //This is needed cause JavaScript paths are relative to main script not current file
));
for await (const chunk of Deno.iter(file)) {
hash.update(chunk);
}
console.log(hash.toString());
Deno.close(file.rid);
import { crypto, toHashString } from 'https://deno.land/std#0.176.0/crypto/mod.ts';
const getFileBuffer = (filePath: string) => {
const file = Deno.openSync(filePath);
const buf = new Uint8Array(file.statSync().size);
file.readSync(buf);
file.close();
return buf;
};
const getMd5OfBuffer = (data: BufferSource) => toHashString(crypto.subtle.digestSync('MD5', data));
export const getFileMd5 = (filePath: string) => getMd5OfBuffer(getFileBuffer(filePath));
I was wondering if anyone knew how to correctly perform asynchronous file I/O in Dart using Streams, because clearly my method is not working. I have seen other people's solutions by converting the stream to a list, but I don't really want to do that because of speed concerns and was wondering if there was any way to do this operation using just streams?
Anyways, here is my code:
import 'dart:io';
import 'dart:collection';
import 'dart:convert';
final validCharacters = RegExp(r'^[a-zA-Z]+$'); // a-z, A-Z
main() async {
final dict1File = 'american-english';
final dict2File = 'british-english';
final cleanedDictFile = 'Cleansed_English_Dictionary';
SplayTreeSet<String> wordSet = new SplayTreeSet<String>();
await readFile(dict1File, wordSet);
await readFile(dict2File, wordSet);
await writeFile(cleanedDictFile, wordSet);
}
readFile(String fileName, SplayTreeSet<String> wordSet) async {
File file = new File(fileName);
if (await file.exists()) {
file
.openRead()
.transform(Utf8Decoder())
.transform(LineSplitter())
.where((String data) => validCharacters.hasMatch(data))
.listen((String data) => wordSet.add(data),
onDone: () => print('${wordSet.length}'), onError: (e) => print(e));
} else {
print('The desired file $fileName does not exist');
}
}
writeFile(String fileName, SplayTreeSet<String> wordSet) async {
File file = new File(fileName);
if (await file.exists()) {
file.openWrite().writeAll(wordSet, '\n');
} else {
print('The desired file $fileName does not exist');
}
}
Here is me performing a similar operation in Java (my native language) :p
import java.io.*;
import java.util.Set;
import java.util.TreeSet;
public class DictCleaner {
public static void main(String[] args) throws IOException {
System.out.println("Beginning process");
Set<String> words = new TreeSet<>();
long time1 = System.currentTimeMillis();
readFile("american-english", words);
readFile("british-english", words);
long time2 = System.currentTimeMillis();
System.out.println(
"Time to read files: " + (time2 - time1) + " milliseconds"
);
System.out.println("Number of words: " + words.size());
String filename = "Cleansed_English_Dictionary";
System.out.println("Outputting new file, " + filename);
outputFile(filename, words);
long time3 = System.currentTimeMillis();
System.out.println(
"Total execution time: " + (time3 - time1) + " milliseconds"
);
File file = new File(filename);
double fileSize = file.length() >> 10;
System.out.println(
"Output file size of " + filename + "= " + fileSize + " kilobytes"
);
}
public static void readFile(String fileName, Set<String> words)
throws IOException {
BufferedReader br;
try {
br = new BufferedReader(new FileReader(fileName));
String currentLine;
while ((currentLine = br.readLine()) != null) {
if (currentLine.matches("^[a-zA-Z]") && currentLine.length() <= 15) {
words.add(currentLine);
}
}
br.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
public static void outputFile(String filename, Set<String> words)
throws FileNotFoundException {
BufferedOutputStream bos = new BufferedOutputStream(
new FileOutputStream(filename)
);
try {
for (String s : words) {
bos.write(s.getBytes());
bos.write(System.lineSeparator().getBytes());
}
bos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Anyways, I am basically just cleaning up a dictionary for use with my app. Does anyone have any guidance on how I can make the Dart code work, or perhaps a more generic explanation?
My new code after comments!
import 'dart:io';
import 'dart:collection';
import 'dart:convert';
final validCharacters = RegExp(r'^[a-zA-Z]+$'); // a-z, A-Z
main() async {
Stopwatch stopwatch = new Stopwatch()..start();
final dict1File = 'american-english';
final dict2File = 'british-english';
final cleanedDictFile = 'Cleansed_English_Dictionary';
SplayTreeSet<String> wordSet = new SplayTreeSet<String>();
await readFile(dict1File, wordSet);
await readFile(dict2File, wordSet);
await writeFile(cleanedDictFile, wordSet);
print('Total execution time: ${stopwatch.elapsed}');
}
Future<void> readFile(String fileName, SplayTreeSet<String> wordSet) async {
File file = new File(fileName);
if (await file.exists()) {
await for (final data in file
.openRead()
.transform(Utf8Decoder())
.transform(LineSplitter())
.where((String data) => validCharacters.hasMatch(data))) {
wordSet.add(data);
}
print('${wordSet.length}');
} else {
print('The desired file $fileName does not exist');
}
}
Future<void> writeFile(String fileName, SplayTreeSet<String> wordSet) async {
File file = new File(fileName);
if (await file.exists()) {
IOSink sink = file.openWrite();
sink.writeAll(wordSet, '\n');
sink.close();
} else {
print('The desired file $fileName does not exist');
}
}
The problem is that .listen does not halt the program until all events has been consumed.
I think the most clean way to fix this is to use "await for-each" like this:
Future<void> readFile(String fileName, SplayTreeSet<String> wordSet) async {
File file = new File(fileName);
if (await file.exists()) {
await for (final data in file
.openRead()
.transform(Utf8Decoder())
.transform(LineSplitter())
.where((String data) => validCharacters.hasMatch(data))) {
wordSet.add(data);
}
print('${wordSet.length}');
} else {
print('The desired file $fileName does not exist');
}
}
More optimized solution
As requested, I have tried to make the program run faster. If performance are the main goal, you can often save some time by using the "sync" edition of the IO file operations. In this case, the program are really not doing anything else, so it is fine to let the main thread wait on e.g. reading the file.
One optimization is more about code readability but Dart does already provide readAsLines and readAsLinesSync on File so we don't need to use a Utf8Decoder and LineSplitter if we just want to split the lines. This will potentially take some more memory since we now get a list containing all lines instead of iterating each line. The list are short-lived so it is properly not a problem for this program. But also, the list contains String objects where most of them are going to be used in our Set so most of the used memory is not really something we can avoid.
Another optimization is the writing. Here we generate the final string first in memory (wordSet.join('\n')) and then write the whole string to disk as one operation. This is much faster but does take some additional memory.
import 'dart:collection';
import 'dart:io';
final validCharacters = RegExp(r'^[a-zA-Z]+$'); // a-z, A-Z
void main() {
const dict1File = 'american-english-insane';
const dict2File = 'british-english-insane';
const cleanedDictFile = 'Cleansed_English_Dictionary';
final wordSet = SplayTreeSet<String>();
readFile(dict1File, wordSet);
readFile(dict2File, wordSet);
writeFile(cleanedDictFile, wordSet);
}
void readFile(String fileName, Set<String> wordSet) {
final file = File(fileName);
if (file.existsSync()) {
file
.readAsLinesSync()
.where((data) => validCharacters.hasMatch(data))
.forEach(wordSet.add);
print(wordSet.length);
} else {
print('The desired file $fileName does not exist');
}
}
void writeFile(String fileName, Iterable<String> wordSet) =>
File(fileName).writeAsStringSync(wordSet.join('\n'));
Is there an easy way to open a file and continously read from it without the stream getting closed on EOF? Like the Unix command tail -f.
Just reading until EOF is described in the API docs. But I can't see an obvious/simple way to block or pause the stream for more input.
One solution would be to repeatedly reopen the file and continue reading from the last known length when I detect that the file size has changed.
Somethink like this
import 'dart:io';
void main(List<String> args ) {
var file = new File("test.txt");
print(file.absolute);
var openFuture = file.open(mode: FileMode.READ);
openFuture.then((raf) => raf.length().then((len) => raf.setPosition(len)
.then((raf) => raf.readXxx...
}
You can also use Directory.watch to get notified about changes and then reopen and read from the last known position.
(even though this questions is a bunch of years old now, I stumbled across the same issue today and couldn't find a viable solution and therefore had to roll my own and wanted to share my findings with future generations of Dart programmers. ;-))
The dart:io package in conjunction with a bit of stream and async-await magic should offer everything that is needed to achieve a "tail -f"-like functionality:
import 'dart:typed_data';
import 'dart:io';
Stream<List<int>> tail(final File file) async* {
final randomAccess = await file.open(mode: FileMode.read);
var pos = await randomAccess.position();
var len = await randomAccess.length();
// Increase/decrease buffer size as needed.
var buf = Uint8List(8192);
Stream<Uint8List> _read() async* {
while (pos < len) {
final bytesRead = await randomAccess.readInto(buf);
pos += bytesRead;
yield buf.sublist(0, bytesRead);
}
}
// Step 1: read whole file
yield* _read();
// Step 2: wait for modify events and read more bytes from file
await for (final event in file.watch(events: FileSystemEvent.modify)) {
if ((event as FileSystemModifyEvent).contentChanged) {
len = await (randomAccess.length());
yield* _read();
}
}
}
This function (tail) might then be used like that:
import 'dart:convert';
import 'dart:io';
void main() {
final file = File('/var/log/messages');
tail(file).transform(utf8.decoder).transform(LineSplitter()).forEach((line) {
// Do something with the line that has been read, e.g. print it out...
print(line);
});
}
Could someone please explain what's wrong with the following code. I'm making two calls to the function fInputData. The first works ok, the second results in an error :
"unhandled exception"
"Bad state: Stream already has subscriber"
I need to write a test console program that inputs multiple parameters.
import "dart:async" as async;
import "dart:io";
void main() {
fInputData ("Enter Nr of Iterations : ")
.then((String sResult){
int iIters;
try {
iIters = int.parse(sResult);
if (iIters < 0) throw new Exception("Invalid");
} catch (oError) {
print ("Invalid entry");
exit(1);
}
print ("In Main : Iterations selected = ${iIters}");
fInputData("Continue Processing? (Y/N) : ") // this call bombs
.then((String sInput){
if (sInput != "y" && sInput != "Y")
exit(1);
fProcessData(iIters);
print ("Main Completed");
});
});
}
async.Future<String> fInputData(String sPrompt) {
async.Completer<String> oCompleter = new async.Completer();
stdout.write(sPrompt);
async.Stream<String> oStream = stdin.transform(new StringDecoder());
async.StreamSubscription oSub;
oSub = oStream.listen((String sInput) {
oCompleter.complete(sInput);
oSub.cancel();
});
return oCompleter.future;
}
void fProcessData(int iIters) {
print ("In fProcessData");
print ("iIters = ${iIters}");
for (int iPos = 1; iPos <= iIters; iPos++ ) {
if (iPos%100 == 0) print ("Processed = ${iPos}");
}
print ("In fProcessData - completed ${iIters}");
}
Some background reading:
Streams comes in two flavours: single or multiple (also known as
broadcast) subscriber. By default, our stream is a single-subscriber
stream. This means that if you try to listen to the stream more than
once, you will get an exception, and using any of the callback
functions or future properties counts as listening.
You can convert the single-subscriber stream into a broadcast stream
by using the asBroadcastStream() method.
So you've got two options - either re-use a single subscription object. i.e. call listen once, and keep the subscription object alive.
Or use a broadcast stream - note there are a number of differences between broadcast streams and single-subscriber streams, you'll need to read about those and make sure they suit your use-case.
Here's an example of reusing a subscriber to ask multiple questions:
import 'dart:async';
import 'dart:io';
main() {
var console = new Console();
var loop;
loop = () => ask(console).then((_) => loop());
loop();
}
Future ask(Console console) {
print('1 + 1 = ...');
return console.readLine().then((line) {
print(line.trim() == '2' ? 'Yup!' : 'Nope :(');
});
}
class Console {
StreamSubscription<String> _subs;
Console() {
var input = stdin
.transform(new StringDecoder())
.transform(new LineTransformer());
_subs = input.listen(null);
}
Future<String> readLine() {
var completer = new Completer<String>();
_subs.onData(completer.complete);
return completer.future;
}
}