Poor performance of Flux::mergeComparing - project-reactor

Question

I'm migrating my application to Reactor and noticed weird performance issue with Flux::mergeComparing. Here's a benchmark for my existing implementation and Flux alternative.
#Fork(value = 1)
#State(Scope.Thread)
#BenchmarkMode(Mode.Throughput)
#Measurement(iterations = 3)
#OutputTimeUnit(TimeUnit.SECONDS)
public class MergeComparingBenchmark
{
private List<List<Integer>> numbers;
#Setup
public void setUp()
{
numbers = IntStream.range(0, 1_000)
.mapToObj(i -> IntStream.range(0, 10_000))
.map(IntStream::boxed)
.map(stream -> stream.collect(toList()))
.collect(toList());
}
#Benchmark
public List<Integer> myMerge()
{
final List<Stream<Integer>> data = numbers.stream().map(Collection::stream).collect(toList());
final MergingIterator mergingIterator =
new MergingIterator(data);
return Streams.stream(mergingIterator).collect(toList());
}
#Benchmark
public List<Integer> mergeComparing()
{
final Flux<Integer>[] data = numbers.stream().map(Flux::fromIterable)
.toArray(Flux[]::new);
return Flux.mergeComparing(data)
.collectList()
.block();
}
static class MergingIterator implements Iterator<Integer>
{
private final PriorityQueue<Tuple2<Integer, Iterator<Integer>>> queue;
MergingIterator(final List<? extends Stream<Integer>> iterators)
{
this.queue = new PriorityQueue<>(Comparator.comparingInt(entry -> entry.v1));
checkNotNull(iterators).stream()
.map(Stream::iterator)
.filter(Iterator::hasNext)
.forEach(iterator -> queue.add(Tuple.tuple(iterator.next(), iterator)));
}
#Override
public boolean hasNext()
{
return !queue.isEmpty();
}
#Override
public Integer next()
{
final Tuple2<Integer, Iterator<Integer>> element = queue.poll();
final Iterator<Integer> iterator = element.v2;
if (iterator.hasNext()) {
queue.add(Tuple.tuple(iterator.next(), iterator));
}
return element.v1;
}
}
}
And the results are:
MergeComparingBenchmark.mergeComparing thrpt 3 0.181 ± 0.664 ops/s
MergeComparingBenchmark.myMerge thrpt 3 5.146 ± 1.390 ops/s
Why is the reactor case so slow? What can I do to improve it?
I tried to change the subscriber thread pools, changing prefetch size and reactor buffers, but there was not difference.

Answer 1

If your intention is to get a list of sorted integers, then I think you can simply call the collectSortedList method :
return Flux.merge(data)
.collectSortedList()
.block();
I also don't understand why you implemented your own iterator, you basically just need to call the sorted method :
return numbers.stream()
.flatMap(Collection::stream)
.sorted()
.collect(Collectors.toList())
MergeComparing
From the documentation
Merge data from provided Publisher sequences into an ordered merged sequence, by picking the smallest values from each source (as defined by the provided Comparator). This is not a sort(Comparator), as it doesn't consider the whole of each sequences.
Instead, this operator considers only one value from each source and picks the smallest of all these values, then replenishes the slot for that picked source.
It only consider a single element from each sources to do the comparison :
Flux<Integer> flux1 = Flux.just(9, 6, 11);
Flux<Integer> flux2 = Flux.just(10, 2, 13);
Flux.mergeComparing(flux1, flux2)
.doOnNext(System.out::println)
.subscribe();
/**
Output => 9 6 10 2 11 13
Compare 9 and 10 => 9
Compare 6 and 10 => 6
Compare 11 and 10 => 10
Compare 11 and 2 => 2
Compare 11 and 13 => 11
=> 13
**/
This is probably not what you need