I have noticed that some Dart data structures implement EfficientLengthIterable like List and Set but Iterable is not implementing it, so I have measured the taken time to calculate the length of List, Set and Iterable that containing 10000000 elements and the results was little strange for me.
Code:
void main() {
listChecker();
iterableChecker();
setChecker();
}
var list = List.generate(10000000, (index) => '${index} + a');
var iterable = Iterable.generate(10000000, (index) => '${index} + a');
var setCollection = Set.from(iterable);
void listChecker() {
final stopwatch = Stopwatch()..start();
print(list.length);
print('${stopwatch.elapsed} List execution Time');
}
void iterableChecker() {
final stopwatch = Stopwatch()..start();
print(iterable.length);
print('${stopwatch.elapsed} Iterable execution Time');
}
void setChecker() {
final stopwatch = Stopwatch()..start();
print(setCollection.length);
print('${stopwatch.elapsed} Set execution Time');
}
Results (each one in a separated execution):
List
10000000
0:00:01.402543 List execution Time
Iterable
10000000
0:00:00.001933 Iterable execution Time
Set
10000000
0:00:06.180889 Set execution Time
It seems like the fastest one was the Iterable but the dart documentation says that about the length property of the Iterable class:
/// Returns the number of elements in [this].
///
/// Counting all elements may involve iterating through all elements and can
/// therefore be slow.
/// Some iterables have a more efficient way to find the number of elements.
So what is meant by efficiency when implementing EfficientLengthIterable? I know that not iterating over each element should be more efficient but how is this reflected to the usage of these data structures? is it means space complexity? what are the cases I should use a List over Iterable in terms of efficiency as it seems that Iterable is too much faster regarding the time.
From the Dart Language Tour:
Top-level and class variables are lazily initialized; the initialization code runs the first time the variable is used.
Since you've declared each Iterable as a top-level variable, they aren't initialized until they are actually used; which means that you are implicitly generating them within your benchmarks. The Set is the one that takes the longest to construct, and that's why the benchmark for Set is taking so long.
Try this instead, and you should find the results are more what you would expect:
void main() {
var list = List.generate(10000000, (index) => '${index} + a');
var iterable = Iterable.generate(10000000, (index) => '${index} + a');
var set = Set.from(iterable);
listChecker(list);
iterableChecker(iterable);
setChecker(set);
}
void listChecker(List list) {
final stopwatch = Stopwatch()..start();
print(list.length);
print('${stopwatch.elapsed} List execution Time');
}
void iterableChecker(Iterable iterable) {
final stopwatch = Stopwatch()..start();
print(iterable.length);
print('${stopwatch.elapsed} Iterable execution Time');
}
void setChecker(Set set) {
final stopwatch = Stopwatch()..start();
print(set.length);
print('${stopwatch.elapsed} Set execution Time');
}
However, also note that Iterable.generate probably still creates a list under the hood, and therefore it will probably be equally as efficient in its length calculation, it's just that an object which conforms to Iterable won't necessarily conform to EfficientLengthIterable.
Related
I am learning Dart and I'm following the Codelabs tutorial on iterable collections.
I have just read about the firstWhere method of iterables for finding the first element that satisfies some criterion.
The tutorial gives an example similar to the following:
bool predicate(String item, {int minLength = 6}) => item.length > minLength;
void main() {
const items = ['Salad', 'Popcorn', 'Toast', 'Lasagne'];
var foundItem = items.firstWhere(predicate);
print(foundItem);
}
Which would print Popcorn as it is the first string with 6 or more characters.
I'm wondering whether it is possible to pass the minLength argument to predicate when calling items.firstWhere(predicate).
sure, but like this:
final minLength = 6;
final foundItem = items.firstWhere((String item) => item.length > minLength));
what you example is doing is just extracting the method (String item) => item.length > minLength; to a separate global variable. which isn't necessary and I wouldn't recommend.
I am trying to create a custom collector in order to count valid elements of a list. I have done it using one of the already provide collectors:
arr.stream()
.filter(e -> e.matches("[:;][~-]?[)D]"))
.map(e -> 1)
.reduce(0, Integer::sum);
but as a challenge for myself, I wanted to create my own custom collector in order to understand it better. And this is where I got stuck.
It is probably something trivial but I am learning this and can't figure a supplier, an accumulator, and a combiner. I guess I still don't understand something about them. For instance, I have a similar stream:
arr1.stream()
.filter(e -> e.matches("[:;][~-]?[)D]"))
.map(e -> 1)
.collect(temporary array, adding to array, reduce);
AFAIK supplier is a function without arguments, which returns something. I studied standard examples and it is usually a method reference for a new collection, e.g. ArrayList::new. I tried to use constant 0, e -> 0 because I want to return a scalar. I think it is wrong because it makes the stream returning 0. If using method reference for a temporary collection, Java complains about a mismatch of types of a supplier and returning object. I am also confused about using an accumulator if the final result is a number as a combiner would reduce all elements to a number, e.g. (a,b) -> a + b`.
I'm completely stumped.
Probably part of your problem is that you cannot obviously create an accumulator for an Integer type since it is immutable.
You start with this:
System.out.println(IntStream.of(1,2,3).reduce(0, Integer::sum));
You can extend to this:
System.out.println(IntStream.of(1,2,3).boxed()
.collect(Collectors.reducing(0, (i1,i2)->i1+i2)));
Or even this, which has an intermediate mapping function
System.out.println(IntStream.of(1,2,3).boxed()
.collect(Collectors.reducing(0, i->i*2, (i1,i2)->i1+i2)));
You can get this far with your own Collector
Collector<Integer, Integer, Integer> myctry = Collector.of(
()->0,
(i1,i2)->{
// what to do here?
},
(i1,i2)->{
return i1+i2;
}
);
The accumulator is A function that folds a value into a mutable result container with mutable being the keyword here.
So, make a mutable integer
public class MutableInteger {
private int value;
public MutableInteger(int value) {
this.value = value;
}
public void set(int value) {
this.value = value;
}
public int intValue() {
return value;
}
}
And now:
Collector<MutableInteger, MutableInteger, MutableInteger> myc = Collector.of(
()->new MutableInteger(0),
(i1,i2)->{
i1.set(i1.intValue()+i2.intValue());
},
(i1,i2)->{
i1.set(i1.intValue()+i2.intValue());
return i1;
}
);
And then:
System.out.println(IntStream.of(1,2,3)
.mapToObj(MutableInteger::new)
.collect(myc).intValue());
Reference:
Example of stream reduction with distinct combiner and accumulator
EDIT: The finisher just does whatever with the final result. If you don't set it on purpose then it is set by default to IDENTITY_FINISH which is Function.identity() which says just to return the final result as is.
EDIT: If you're really desperate:
Collector<int[], int[], int[]> mycai = Collector.of(
()->new int[1],
(i1,i2)->i1[0] += i2[0],
(i1,i2)->{i1[0] += i2[0]; return i1;}
);
System.out.println(IntStream.of(1,2,3)
.mapToObj(v->{
int[] i = new int[1];
i[0] = v;
return i;
})
.collect(mycai)[0]);
Assume I want to concatenate N Uint8Lists into a single one.
The naive approach is to simple copy all elements into a new list. However, that seems rather memory in efficient. Instead, I want to create a single Uint8List "view" which simply indexes into the appropriate underlying list instead of copying all its content.
In C++ I'd usually just overwrite operator[] but I am not quite certain how to do this with Uint8Lists in Dart.
In C++, you can make a View class that overrides operator[]. In Dart, you could do the same thing:
class View<T> {
View(this._lists);
List<List<T>> _lists;
T operator [](int index) {
for (var list in _lists) {
if (index < list.length) {
return list[index];
}
index -= list.length;
}
throw RangeError('...');
}
}
You could stop there, but doing just that usually wouldn't be enough in either language. In C++, you'd also want to provide begin() and end() methods for range-based for loops to work. Similarly, in Dart, you'd want to provide the Iterable interface so that for-in would work.
Luckily package:collection (note that this is separate from dart:collection) provides a CombinedListView class that does that work for you. For example:
import 'dart:typed_data';
import 'package:collection/collection.dart';
void main() {
var list1 = Uint8List.fromList([1, 2, 3]);
var list2 = Uint8List.fromList([4, 5, 6]);
var list3 = Uint8List.fromList([7, 8, 9]);
var view = CombinedListView<int>([list1, list2, list3]);
for (var i in view) {
print(i);
}
}
In pubspec.yaml, I'm using english_words library to generate wordpairs:
dependencies:
flutter:
sdk: flutter
# Contains a few thousand of the most used English words
# plus some utility functions.
english_words: ^3.1.0
Now the WordPair Class is not a subtype of String and so I can't use the Iterable's lambdas or functions like cast or retype to 'cast' the 'WordPairs' to Strings.
So, I had to write the function called getWords().
See below the Dart file, Model.dart, that contains this implementation.
You'll see the old line commented out where it was returning in the getter the type Iterable.
Would there be a more efficient way to do this?
For example, I didn't want to involve a List Class in the conversion, but I can't find any other way to successfully do this.
Thanks.
---------------- Model.dart
import 'package:english_words/english_words.dart' show WordPair, generateWordPairs;
import 'dart:collection';
/// Model Class
///
class Model {
String get randomWordPair => new WordPair.random().asPascalCase;
// Iterable<WordPair> get wordPairs => generateWordPairs().take(10);
Iterable<String> get wordPairs => getWords();
Iterable<String> getWords(){
Iterable<WordPair> pairs = generateWordPairs().take(10);
ListWords<String> words = new ListWords();
for (var pair in pairs) {
words.add(pair.asString);
}
return words;
}
}
class ListWords<E> extends ListBase<E> {
final List<E> l = [];
set length(int newLength) { l.length = newLength; }
int get length => l.length;
E operator [](int index) => l[index];
void operator []=(int index, E value) { l[index] = value; }
}
In Dart 2 you can use
iterable.cast<NewType>()
but it is prone to lead to inefficiency if the resulting list is accessed often, because it wraps the original iterable into a new one and has to forward every access.
Usually more efficient are
new List<NewType>.of(oldList)
or
new List.from<NewType.from(oldList)
I was not able to derive the difference between .of() and from() from the docs though (https://api.dartlang.org/dev/2.0.0-dev.50.0/dart-core/List/List.from.html, https://api.dartlang.org/dev/2.0.0-dev.50.0/dart-core/List/List.of.html)
At first glance, a loop that is collecting the result of an expression can generally be replaced with an appropriate .map method invocation on an Iterable. See if that will help.
In my application I have to show only the top 5 items of a long list that have the highest rating. I have implemented this as follows:
The long list is an array that I have turned into an observable array with all elements as observables with ko.mapping, and the top 5 items are a computed array that depends on the long list. Whenever anything in the long list changes the computed array resorts the long list and takes the top 5 items.
My problem is that the long array with ko.mapping takes up 60MB of memory, whereas without ko.mapping it only takes 4MB. Is there a way to achieve this effect without ko.mapping the long array?
Here is a fiddle in which I have recreated the scenario with a smaller and simpler long array, but it's just for you to understand what I'm talking about.
This is the long array, for the demo I've made it 12 elements long:
this.longArray = ko.mapping.fromJS([
{name:"Annabelle"},
{name:"Vertie"},
{name:"Charles"},
{name:"John"},
{name:"AB"},
{name:"AC"},
{name:"AD"},
{name:"AE"},
{name:"AF"},
{name:"AG"},
{name:"AH"},
{name:"AJ"}
]);
And this is the computed array(showing only the top 5):
this.sortedTopItems = ko.computed(function() {
return self.longArray().sort(function(a, b) {
if(a.name() < b.name()) return -1;
if(a.name() > b.name()) return 1;
return 0;
}).slice(0, 5);
}, this);
The change one button is to simulate the long array changing and the reset button is to reset the array to its initial state.
You sure can, but the simplest way would be to filter the data before putting into knockout. If you only ever care about the first 5. Let's assume your long array of items is called data. Note that I'm not able to test this right now, but it should give you a good idea.
const sortedTopItems = ko.observableArray([]);
// Call with new data
const update = (data) => {
data.sort((a,b) => a.name - b.name);
sortedTopItems(data.slice(0, 5));
}
This handles the case for simple data where it's not observable. If you want the actual data items (rows) to be observable then I'd do the following:
const length = 5;
// Create an empty array and initialize as the observable
const startData = new Array(length).map(a => ({}));
const sortedTopItems = ko.observableArray(startData);
// Call with new data
const update = (data) => {
data.sort((a,b) => a.name - b.name);
for(let i = 0; i < length; i++) {
const item = sortedTopItems()[i];
ko.mapping.fromJS(data[i], item); // Updates the viewModel from data
}
}