I'm using the ZLibDeflater to compress a file, by reading it as a stream and transforming it:
new File(filePath)
.openRead()
.transform(new ZLibDeflater())
.pipe(new File(gzipPath).openWrite())
.then(...);
As the ZLibDeflater is now deprecated, how can I convert the code to use the new GZipCodec class?
You can also use ZLIB :
new File(filePath)
.openRead()
.transform(ZLIB.decoder)
.pipe(new File(zlibPath).openWrite())
.then(...);
Gave up to early, you can simply do it using the Converter.bind() method to transform a stream:
const GZipCodec()
.encoder.bind(new File(filePath).openRead())
.pipe(new File(gzipPath).openWrite())
.then(...)
Related
I have the in-memory bytes of a "blob", but the API that I want to process this "blob" with only accepts dart:io File objects.
Is there a way to create a "fake" dart:io File , simply wrapping my in-memory bytes, so that I can pass this "fake" File to my API?
Assume that a file system doesn't exist, and assume that I can't write the in-memory bytes to a "real" file.
Thanks!
You can create a memory-file using MemoryFileSystem from the package file:
Example:
File file = MemoryFileSystem().file('test.dart')
..writeAsBytesSync(blobBytes);
Add path provider dependency on pubspec.yaml
dependencies:
path_provider: 0.2.2
Write byte data to file, use it, then delete it.
import 'dart:io';
import 'package:path_provider/path_provider.dart';
main() async {
String dir = (await getTemporaryDirectory()).path;
File temp = new File('$dir/temp.file');
var bytes = [0, 1, 2, 3, 4, 5];
await temp.writeAsBytes(bytes);
/*do something with temp file*/
temp.delete();
}
You can override the io using the IOOverides class. See the below example:
await IOOverrides.runZoned(() async {
// Write your code here.
},
createDirectory: (p0) {
return MockDirectory()
},
createFile: (p0) {
return MockeFile();
},
);
You can check the doc here for more info.
I always use cross_file when designing APIs that work with files.
In addition to working on multiple platforms, it also has an XFile.fromData constructor that allows you to create a file in memory.
I have a FileStream (GLib.stdout really) and I want to pass it to a function that needs an OutputStream (Vte.Terminal.write_contents). How do you convert between them?
Create a new UnixOutputStream using the file descriptor from the stream you want. (e.g., new UnixOutputStream(stdout.fileno(), false)
I used to do something like this:
HttpResponse res = req.response;
String dataReceived;
await req.listen((List<int> buffer) {
dataReceived = new String.fromCharCodes(buffer);
}).asFuture();
Map data = JSON.decode(dataReceived);
When I needed UTF8 support, I modified it to:
Map data = JSON.decode(await new Utf8Codec().decodeStream(request));
Kevin Moore suggested to encode/decode like this:
https://dartpad.dartlang.org/1d229cfdc1c1fd2ab877
So I've got:
Map data;
await request.listen((List<int> buffer) {
data = JSON.fuse(UTF8).decode(buffer);
}).asFuture();
Not sure that I need the asFuture():
Map data;
await request.listen((List<int> buffer) => data = JSON.fuse(UTF8).decode(buffer));
Or do I? And this method requires that I encode it into bytes on the client side:
sendData: new JsonUtf8Encoder().convert({'model': message, 'authToken': app.authToken}))
What are the benefits of this? Isn't it more to send over the wire?
I believe Shelf and/or the new RPC lib would handle this stuff for me? Shall I move to one of those? Right now, it's all homegrown.
HttpRequest is a Stream<List<int>>. You don't want to use listen because you'll only get the first "chunk" of data.
Instead you'll want to do something like this:
import 'dart:async';
import 'dart:convert';
main() async {
var input = {'a':1, 'b':2};
var decoder = JSON.fuse(UTF8).decoder;
var json = await decoder.bind(toByteStream(input)).single;
print(json);
}
Stream<List<int>> toByteStream(json) =>
_encoder.bind(new Stream.fromIterable([json]));
final _encoder = new JsonUtf8Encoder();
https://dartpad.dartlang.org/9807d0c5ed89360c9f53
Yes as you can see on https://github.com/dart-lang/shelf/blob/master/lib/src/message.dart#L136 shelf defaults to UTF-8
I am likely biased but I would definitely recommend moving over to shelf. You have several options depending on what you prefer, like:
shelf_rpc as you mentioned. I haven't used it but likely full featured API support
shelf_bind if you simply want to bind a handler function parameter to a JSON body. This is lower level, more flexible and less prescriptive but does less. e.g.
router.post('/foo', (#RequestBody() Foo foo) => ...)
shelf_rest. Adds higher level more prescriptive API support (similar to shelf_rpc).
full frameworks like redstone, mojito etc. These do more for you but you need to buy into more
Had a chat w/ Kevin to better understand his answer, and thought it best to share my learnings as a new answer.
HttpRequest is always a Stream<List<int>> – a streamed list of integers. Those integers are bytecodes, and this is commonly referred to as a bytestream. You can be sure that no matter what API you use to send data over the wire, that it is sent as a bytestream.
The HttpRequest.request() method accepts sendData in several forms...
* If specified, `sendData` will send data in the form of a [ByteBuffer],
* [Blob], [Document], [String], or [FormData] along with the HttpRequest.
Source:
https://api.dartlang.org/apidocs/channels/stable/dartdoc-viewer/dart:html.HttpRequest#id_request
...but these are just abstractions, and ultimately your data is sent as a Stream<List<int>> bytestream.
So on the server we first set up a decoder that will decode both JSON and UTF8 (for correct char handling), and then we bind that to the HttpRequest request, which is a bytestream. I think single just serves to ensure we throw an exception if we received more than one data event. Here's all the code we need to interpret an HttpRequest:
import 'dart:async';
import 'dart:convert';
static handleRequest(HttpRequest request) async {
var decoder = JSON.fuse(UTF8).decoder;
var data = await decoder.bind(request).single;
print('The decoded data received is:\n\n$data');
}
With the release of the new dart:io libraries, in order to read the data from an HttpRequest we now need to:
Listen to the body to handle data and notified of it's completion.
In order to accomplish this, I've been using something similar to the following to get all of the data from the request (taken from the dart:io tests):
List<int> body = new List<int>();
request.listen(body.addAll, onDone: () {
var str = new String.fromCharCodes(body);
// Now do something with the string of data
});
Is there some way of using transform and/or StringDecoders to already provide a constructed result. The only way I can think of, I would still need to create a StringBuffer and use writeAll to ensure all data is passed in the event it doesn't all arrive at once, and still call the onDone before using the string in the buffer.
So I guess ultimately the question is: is there some way I can use a HttpRequest (or in all actuality any Stream) without needing to buffer/build the results and can just pass a callback or handler which receives the entire contents?
I haven't tried this, but I think you should be able to use StringDecoder, toList() and join() to get the whole body:
request.transform(new StringDecoder()).toList().then((data) {
var body = data.join('');
print(body);
}
I am looking for a Java DataOutputStream equivalent for Dart where I can write arbitrary types (int, string, float, byte array etc). There is RandomAccessFile but it does not provide byte array or float-double values. ByteArray seems to have some necessary functions but I am not sure how to write it to a file or an OutputStream.
Here is some simple code showing how to write a ByteArray into an OutputStream:
#import('dart:io');
#import('dart:scalarlist');
main() {
File file = new File("c:\\temp\\foo.txt");
OutputStream os = file.openOutputStream();
os.onNoPendingWrites = () {
print('Finished writing. Closing.');
os.flush();
os.close();
};
Uint8List byteList = new Uint8List(64);
ByteArray byteArray = byteList.asByteArray();
int offset = 0;
offset = byteArray.setUint8(offset, 72);
offset = byteArray.setUint8(offset, 101);
offset = byteArray.setUint8(offset, 108);
offset = byteArray.setUint8(offset, 108);
offset = byteArray.setUint8(offset, 111);
offset = byteArray.setUint8(offset, 0);
byteArray.setFloat32(offset, 1.0);
os.write(byteList);
}
This has been around for a while, but I searched and didn't find good DataInput/OutputStream interoperability classes. I wanted a version that works with streams, so I could process files that don't comfortably fit in RAM. So I wrote one.
It's published over at https://pub.dev/packages/jovial_misc in io_streams, or if you prefer, https://github.com/zathras/misc/tree/master/dart/jovial_misc. I made it so it interoperates with java.io.DataInputStream and java.io.DataOutputStream. Code using it looks a little like this:
import 'package:convert/convert.dart';
import 'package:jovial_misc/io_utils.dart';
void main() async {
final acc = ByteAccumulatorSink();
final out = DataOutputSink(acc);
out.writeUTF8('Hello, world.');
out.close();
final stream = Stream<List<int>>.fromIterable([acc.bytes]);
final dis = DataInputStream(stream);
print(await dis.readUTF8());
await dis.close();
}
The Stream<List<int>> would of course typically come from a socket, or File.openRead(), etc. There's also a DataInputStream variant that is synchronous and takes an Iterable, if you do have all the byte data available up front.
DataInputStream and DataOutputSink are pretty much the obvious mapping of the java.io classes. The tricky part is the buffer management, since a stream shoves data at you in List<int> instances that probably aren't lined up with the data you want. And, of course, it's necessary to do everything asynchronously.
HTH.
You are essentially asking for arbitrary object serialization. And while the Dart VM has one, it isn't exposed to programmers (it is only used for snapshotting and message passing). I'd say that it would be a mistake to expose it -- in different situations, we have different requirements for serialization and "one true solution" isn't gonna work (Java showed us that already).
For example, I'm working on a MsgPack implementation for Dart, I know that Protobuf port is also in the works, maybe someone will start a Thrift port... the possibilities are endless.
The closest thing I could find is this package: https://github.com/TomCaserta/dart_io/ . Unfortunately there is a bug when reading to the end of the byte array - see my pull request in GitHub.
You could use this class:
https://github.com/TomCaserta/dart_io/blob/master/lib/data_output.dart
Unfortunately (a) it doesn't handle streams; (b) writeLong doesn't take a single integer. I have raised an issue for the Dart SDK: https://github.com/dart-lang/sdk/issues/31166
Edit: I have forked the dart_io package and fixed the two problems described above. My new package is published as dart_data_io:
https://github.com/markmclaren2/dart_data_io