I am pretty new to Dart, and still wrapping my head around streams. Specifically I am having some difficulty with finding the proper way of making a function that takes a window of N elements from a stream, applies a function to it and restreams the results.
To clarify what I mean, I include an example that I implemented myself which led me to this question. The code takes a byte stream from a file and converts 4 byte chunks to an integer stream. By using an await for I was able to accomplish what I wanted but I am looking for a more idiomatic stream based function that accomplishes the same thing, more succinctly.
Stream<int> loadData(String path) async* {
final f = File(path);
final byteStream = f.openRead();
var buffer = Uint8List(8);
var i = 0;
// This is where I would like to use a windowing function
await for(var bs in byteStream) {
for(var b in bs) {
buffer[i++] = b;
if(i == 8) {
var bytes = new ByteData.view(buffer.buffer);
yield bytes.getUint16(0);
i = 0;
}
}
}
}
Look at bufferCount method from RxDart package.
Buffers a number of values from the source Stream by count then emits the buffer and clears it, and starts a new buffer ...
Here is an example:
import 'dart:typed_data';
import 'package:rxdart/rxdart.dart';
main() {
var bytes = Uint8List.fromList([255, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 2, 1, 0, 0]);
Stream<int>.fromIterable(bytes)
.bufferCount(4)
.map((bytes) => Uint8List.fromList(bytes).buffer)
.map((buffer) => ByteData.view(buffer).getInt32(0, Endian.little))
.listen(print); // prints 255 256 257 258
}
It is worth noting that this particular task can be performed much easier:
bytes.buffer.asInt32List();
Related
List<int> l = [ 1, 2, 3, 4 ];
var b = ByteData(10);
May I know what is the easiest way to fill b (position 4 ~ 7) with data from l.
I can certainly iterate through l and then fill b one by one. But this is just part of a larger solution. So I hope there is a simpler way (for easy maintenance in future).
ByteData represent an area of memory counted in bytes but does not tell us anything how we want to represent the data inside this block of memory.
Normally, we would use one of the specific data types from dart:typed_data like e.g. Uint8List, Int8List, Uint16List and so on which have a lot more functionality.
But you can easily get the same by making a view over your ByteData. In this example I guess you want to insert your numbers as Uint8:
import 'dart:typed_data';
void main() {
List<int> l = [ 1, 2, 3, 4 ];
var b = ByteData(10);
var uInt8ListViewOverB = b.buffer.asUint8List();
uInt8ListViewOverB.setAll(4, l);
print(uInt8ListViewOverB); // [0, 0, 0, 0, 1, 2, 3, 4, 0, 0]
}
I recommend reading the documentation for the different methods on ByteBuffer (returned by buffer). You can e.g. make subview of a limited part of your ByteData if your ByteData needs to contain different types of data:
https://api.dart.dev/stable/2.15.0/dart-typed_data/ByteBuffer/asUint8List.html
How do I convert the below java code to equivalent dart.
private static final byte[] mIdBytes = new byte[]{(byte) 0x01, (byte) 0x02, (byte) 0x03, (byte) 0x7E};
byte[] data;
System.arraycopy(mIdBytes, 2, data, 0, 4);
Is there any Dart method that does a similar kind of operation?
I was looking into this:
https://pub.dev/documentation/ckb_dart_sdk/latest/ckb-utils_number/arrayCopy.html
To match Java's System.arrayCopy(source, sourceOffset, target, targetOffset, length)
you should use
target.setRange(targetOffset, targetOffset + length, source, sourceOffset);
This is more efficient than using List.copyRange for some lists, for example copying between typed-data lists with the same element size (like two Uint8Lists).
Well, I found the way to do it.
you can just use
List.copyRange(data, 0, mIdBytes, 2);
This is a workaround I kinda found to be done in your case. This is called sublist(), this method will take the start index, and an end index.
IDEA:
Use sublist(), and copy the elements to be started from, that sourcePos = you_pos
Source array will be used like sourceArray.sublist(startIndext, endIndex)
The destination array will be initialized with the value using sublist()
Till what length the item should be added would be mentioned in the end index+2, since it will ignore the last item, and copy till the index-1
FINAL CODE
void main() {
List<int> source = [1, 2, 3, 4, 5, 6];
List<int> target = [];
int startPos = 1;
int length = 4;
// to ensure the length doesn't exceeds limit
// length+2 because, it targets on the end index, that is 4 in source list
// but the end result should be length+2 to contain a length of 5 items
if(length+1 <= source.length-1){
target = source.sublist(startPos, length+2);
print(target);
}else{
print('Cannot copy items till $length: index out of bound');
}
}
//OUTPUT
[2, 3, 4, 5, 6]
Is there any way to generate random numbers without duplication?
For instance I want to generate 50 random numbers from 1 to 100 no duplication, any way to do this or do I have to check every time incoming number is already created or not?
you can use shuffle as following code.
import 'dart:math';
var list = new List<int>.generate(10, (int index) => index); // [0, 1, 4]
list.shuffle();
print(list);
You can use Set. Each object can occur only once when using it. Just try this:
Set<int> setOfInts = Set();
while (setOfInts.length < 50) {
setOfInts.add(Random().nextInt(range) + 1);
}
You can read the documentation here: Set Doc
Here is an alternative that avoids creating an array of all the possible values, and avoids repeatedly looping until no collision occurs. It may be useful when there is a large range to select from.
import 'dart:math';
class RandomList {
static final _random = new Random();
static List<int> uniqueSample({int limit, int n}) {
final List<int> sortedResult = [];
final List<int> result = [];
for (int i = 0; i < n; i++) {
int rn = _random.nextInt(limit - i); // We select from a smaller list of available numbers each time
// Increment the number so that it picks from the remaining list of available numbers
int j = 0;
for (; j < sortedResult.length && sortedResult[j] <= rn; j++) rn++;
sortedResult.insert(j, rn);
result.add(rn);
}
return result;
}
}
I haven't tested it exhaustively but it seems to work.
I have a command line Dart script client interacting over a [dart:io] websocket with a Jetty server. I've implemented a custom message subprotocol that uses reflection (both sides) to exchange Dart objects with Java objects in Jetty using binary encoding. PODO -> Uint8List -> wire -> ByteBufer -> POJO (and in reverse for the return trip). The local round trip unit tests execute correctly on either side (i.e. PODO -> Uint8List -> PODO; POJO -> ByteBuffer -> POJO). I've tested the connection with a different service endpoint using a series of simple 'string' exchanges. The transmission from Dart to Jetty works but the response data, although correctly received, produces an odd type that I don't understand and which the decoder doesn't understand as either a Uint8List, ByteBuffer, etc.
Although I can't easily distill this into a small example, here is the relevant code and some output:
Dart Client:
WebSocket.connect(url).then((WebSocket socket) {
_log.finer('connected');
_websocket = socket
..listen(_onResponse, onError: (e, StackTrace st) => print('Session error: $e; $st'));
...
}
_onResponse(data) {
print('response raw data: $data');
InstanceMirror im = reflect(data);
print('instance: $im');
...
decode(data)
}
decode(Uint8List data) {
var b = data.buffer;
ByteData bd = new ByteData.view(b);
int offset = 0;
const ENDIANNESS = Endianness.LITTLE_ENDIAN;
int msgLength = bd.getInt32(offset, ENDIANNESS); // is 6645122; should be 101
...
}
Output:
response raw data: [101, 0, 0, 0, ...]
instance: InstanceMirror on Instance of '_Uint8ArrayView'
The IntelliJ debugger shows:
data = {List[id=1]} size = 101
> im = {_LocalInstanceMirror[id=2]} InstanceMirror on Instance of '_Uint8ArrayView'
_reflectee = {List[id=1]}
_type = null
hasReflectee = true
Dart supports the general ByteBuffer type which just represents a list of bytes as you can see here:
https://api.dartlang.org/apidocs/channels/stable/dartdoc-viewer/dart:typed_data.ByteBuffer
As the ByteBuffer class is abstract, you create lists from either receiving a ByteBuffer object or by instantiating a new list. The list usually is fixed length and working with it is hard. That's why you can create views from a ByteBuffer. A view represents a subset of the ByteBuffers bytes, given by an offset and a length which could also be the whole buffer.
Let's look at a small example:
import 'dart:typed_data';
void main() {
Uint8List data8 = new Uint8List.fromList([1,2,3,4,5,6,256]);
Uint16List data16 = new Uint16List.fromList([1,2,3,4,5,6,256]);
print(data8);
print(data16);
print(data8.buffer.lengthInBytes);
print(data16.buffer.lengthInBytes);
print(data16.buffer.asUint8List());
print(data16.buffer.asUint16List());
}
which gives you:
[1, 2, 3, 4, 5, 6, 0] // List
[1, 2, 3, 4, 5, 6, 256] // List
7
14
[1, 0, 2, 0, 3, 0, 4, 0, 5, 0, 6, 0, 0, 1] // View
[1, 2, 3, 4, 5, 6, 256] // View
This means: ByteBuffer is some abstract class which gives you the interface and basic functionality and you can get views from a buffer to access the data as you need it.
The buffer accessed by 'data.buffer' is an internal buffer backing the Uint8List view referenced by 'data'. The transmitted payload -- what I want to wrap with the ByteData view -- is offset by some number of bytes into this buffer. The 'offset' of the payload can be determined through the 'offsetInBytes' property of the 'data' object. The following changes to decode() make it possible to use the ByteData API to decode various fragments of the payload byte array:
int offset = data.offsetInBytes; // sets offset to the starting position of the payload
int msgLength = bd.getInt32(offset);
double somethingElse = bd.getFloat64(offset+4);
// etc.
In javascript it always bothered me people use objects as vectors like {x: 1, y: 2} instead of using an array [1,2]. Access time for the array is much faster than the object but accessing by index is more confusing especially if you need a large array. I know dart has fixed arrays but is there a way to name the offsets of an array like you would a struct or a tuple/record in another language? Define enum/constants maybe?
I'd want something like
List<int> myVector = new List([x,y]);
myVector.x = 5;
is there an equivalent or idiomatic way to do this?
That sounds like a class.
class MyVector {
int x;
int y;
MyVector(this.x, this.y);
}
There is no simpler and more efficient way to create a name-indexed structure at runtime. For simplicity you could usually use a Map, but it's not as efficient as a real class.
A class should be at least as efficient (time and memory) as a fixed length list, after all it doesn't have to do an index bounds check.
In Dart 3.0, the language will introduce records. At that point, you can use a record with named fields instead of creating a primitive class:
var myVector = (x: 42, y: 37);
print(myVector.x);
A record is unmodifiable, so you won't be able to update the values after it has been created.
For me, i see 2 way to do this. I will sort by best in my point of view
Class based method
Here, the approach is to encapsulate your need, in a dedicated object
Pros:
It's encapsultate
You can propose several way to access variable, depend of the need
You can extend functionality without break everything
I love it :p
Cons
More time spend to create class, etc.
Do you really need what i say in pros ?
Maybe weird for js people
example :
class Vector {
int x;
int y;
static final String X = "x";
static final String Y = "y";
Vector({this.x, this.y});
Vector.fromList(List<int> listOfCoor) {
this.x = listOfCoor[0];
this.y = listOfCoor[1];
}
// Here i use String, but you can use [int] an redefine static final member
int operator[](String coor) {
if (coor == "x") {
return this.x;
} else if (coor == "y") {
return this.y;
} else {
// Need to be change by a more adapt exception :)
throw new Exception("Wrong coor");
}
}
}
void main() {
Vector v = new Vector(x: 5, y: 42);
Vector v2 = new Vector.fromList([12, 24]);
print(v.x); // print 5
print(v["y"]); // print 42
print(v2.x); // print 12
print(v2[Vector.Y]); // print 24
}
Enum based method:
You can also defined a "enum" (actually not really implement but will be in the future version) that will contains "shortcut" to your value
Pros
More simple to implement
Is more like your example ;p
Cons
Less extendable
i think is not very pretty
Not OOP think
example:
class Vector {
static final int x = 0;
static final int y = 1;
}
void main() {
List<int> myVector = new List(2);
myVector[Vector.x] = 5;
myVector[Vector.y] = 42;
}
Make your choice ;p
This is only possible with a class in Dart.
There are some open feature requests at http://dartbug.com
introduce struct (lightweight class)
Give us a way to structure Bytedata
If you have reasonably big data structure, you can use "dart:typed_data" as a model and provide lightweight view for the stored data. This way the overhead should be minimal.
For example, if you need 4X4 matrix of Uint8 values:
import "dart:typed_data";
import "dart:collection";
import "package:range/range.dart";
class Model4X4Uint8 {
final Uint8List _data;
static const int objectLength = 4 * 4;
final Queue<int> _freeSlotIndexes;
Model4X4Uint8(int length): _data = new Uint8List((length) * objectLength),
_freeSlotIndexes = new Queue<int>.from(range(0, length));
int get slotsLeft => _freeSlotIndexes.length;
num operator [](int index) => _data[index];
operator []=(int index, int val) => _data[index] = val;
int reserveSlot() =>
slotsLeft > 0 ? _freeSlotIndexes.removeFirst() : throw ("full");
void delete(int index) => _freeSlotIndexes.addFirst(index);
}
class Matrix4X4Uint8 {
final int offset;
final Model4X4Uint8 model;
const Matrix4X4Uint8(this.model, this.offset);
num operator [](int index) => model[offset + index];
operator []=(int index, int val) => model[offset + index] = val;
void delete() => model.delete(offset);
}
void main() {
final Model4X4Uint8 data = new Model4X4Uint8(100);
final Matrix4X4Uint8 mat = new Matrix4X4Uint8(data, data.reserveSlot())
..[14] = 10
..[12] = 256; //overlow;
print("${mat[0]} ${mat[4]} ${mat[8]} ${mat[12]} \n"
"${mat[1]} ${mat[5]} ${mat[9]} ${mat[13]} \n"
"${mat[2]} ${mat[6]} ${mat[10]} ${mat[14]} \n"
"${mat[3]} ${mat[7]} ${mat[11]} ${mat[15]} \n");
mat.delete();
}
But this is very low level solution and can easily create sneaky bugs with memory management and overflows.
You could also use an extension on List to create aliases to specific indexes.
Although it will be difficult to set up mutually exclusive aliases, in some cases, it may be a simple solution.
import 'package:test/test.dart';
extension Coordinates<V> on List<V> {
V get x => this[0];
V get y => this[1];
V get z => this[2];
}
void main() {
test('access by property', () {
var position = [5, 4, -2];
expect(position.x, 5);
expect(position.y, 4);
expect(position.z, -2);
});
}
The Tuple package https://pub.dev/packages/tuple might be what you are looking for when a class is too heavy.
import 'package:tuple/tuple.dart';
const point = Tuple2<int, int>(1, 2);
print(point.item1); // print 1
print(point.item2); // print 2