How to change the final type after reduction of a downstream collector in a Java 8 stream? - java-stream

I got a legacy application using data structures like those in the following toy snippet and I can't easily change these data structures.
I use a Java 8 (only) stream to do some stats and I failed to get the wished type using Collectors.
package myIssueWithCollector;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.BinaryOperator;
import java.util.stream.Collectors;
public class MyIssueWithCollector {
public static Double latitude(Map<String, String> map) {
String latitude = map.get("LATITUDE");
return Double.valueOf(latitude);
}
private static int latitudeComparator(double d1, double d2) {
// get around the fact that NaN > +Infinity in Double.compare()
if (Double.isNaN(d1) && !Double.isNaN(d2)) {
return -1;
}
if (!Double.isNaN(d1) && Double.isNaN(d2)) {
return 1;
}
return Double.compare(Math.abs(d1), Math.abs(d2));
}
public static Map<String, String> createMap(String city, String country, String continent, String latitude) {
Map<String, String> map = new HashMap<>();
map.put("CITY", city);
map.put("COUNTRY", country);
map.put("CONTINENT", continent);
map.put("LATITUDE", latitude);
return map;
}
public static void main(String[] args) {
// Cities with dummies latitudes
// I can not change easily these legacy data structures
Map<String, String> map1 = createMap("London", "UK", "Europa", "48.1");
Map<String, String> map2 = createMap("New York", "USA", "America", "42.4");
Map<String, String> map3 = createMap("Miami", "USA", "America", "39.1");
Map<String, String> map4 = createMap("Glasgow", "UK", "Europa", "49.2");
Map<String, String> map5 = createMap("Camelot", "UK", "Europa", "NaN");
List<Map<String, String>> maps = new ArrayList<>(4);
maps.add(map1);
maps.add(map2);
maps.add(map3);
maps.add(map4);
maps.add(map5);
//////////////////////////////////////////////////////////////////
// My issue starts here:
//////////////////////////////////////////////////////////////////
Map<String, Map<String, Double>> result = maps.stream()
.collect(Collectors.groupingBy(m -> m.get("CONTINENT"),
Collectors.groupingBy(m -> m.get("COUNTRY"), Collectors.reducing(Double.NaN, m -> latitude(m),
BinaryOperator.maxBy((d1, d2) -> latitudeComparator(d1, d2))))));
System.out.println(result);
}
}
I need the result type to be
Map<String, Map<String, String>> instead of Map<String, Map<String, Double>>
by converting back "LATITUDE" from Double to String (using a custom format, not Double.toString() ).
I failed to achieve this with Collectors methods like andThen or collectingAndThen,...
I am currently stuck with Java 8.
Is there a way to get a Map<String, Map<String, String>> result using the same stream ?

Instead of using Collectors.reducing(…) with BinaryOperator.maxBy(…) you can also use Collectors.maxBy. Since this collector doesn’t support an identity value, it requires a finisher function to extract the value from an Optional, but your task requires a finisher anyway, to format the value.
Map<String, Map<String,String>> result = maps.stream()
.collect(Collectors.groupingBy(m -> m.get("CONTINENT"),
Collectors.groupingBy(m -> m.get("COUNTRY"),
Collectors.mapping(MyIssueWithCollector::latitude,
Collectors.collectingAndThen(
Collectors.maxBy(MyIssueWithCollector::latitudeComparator),
o -> format(o.get()))))));
This assumes format to be your custom format function like
private static String format(double d) {
return String.format("%.2f", d);
}
But sometimes, it might be worthwhile to implement your own collector instead of combining multiple built-in collectors.
Map<String, Map<String,String>> result = maps.stream()
.collect(Collectors.groupingBy(m -> m.get("CONTINENT"),
Collectors.groupingBy(m -> m.get("COUNTRY"),
Collector.of(
() -> new double[]{Double.NEGATIVE_INFINITY},
(a, m) -> {
double d = latitude(m);
if(!Double.isNaN(d)) a[0] = Double.max(a[0], d);
},
(a, b) -> a[0] >= b[0]? a: b,
a -> format(a[0])))));
A collector maintains its state using a mutable container, this custom collector uses an array of length one to be able to hold a double value (which eliminates the need to box it to Double objects). Instead of implementing a special comparator to treat NaN specially, it uses a conditional, to never let NaN get into the array in the first place. That’s why the combiner doesn’t need to care about NaN; it can simply return the larger of the two values.
The finisher function just invokes the custom format function with the double value.

You can use Collectors.collectingAndThen to convert the reduced double value to a corresponding String:
Map<String, Map<String, String>> result = maps.stream().collect(
Collectors.groupingBy(
m -> m.get("CONTINENT"),
Collectors.groupingBy(
m -> m.get("COUNTRY"),
Collectors.collectingAndThen(
Collectors.reducing(
Double.NaN,
m -> latitude(m),
BinaryOperator.maxBy(
(d1, d2) -> latitudeComparator(d1, d2)
)
),
MyIssueWithCollector::myToString
)
)
)
);
Here, myToString is some method in the MyIssueWithCollector class to return String from double with your custom format, for example,
public static String myToString(double d) {
return "[latitude=" + d + "]";
}

Using Collectors reducing, you can maintain the latitude's String type in the identity so that the downstream collector is returning a String.
Map < String, Map < String, String >> result = maps.stream()
.collect(
Collectors.groupingBy(m - > m.get("CONTINENT"),
Collectors.groupingBy(m - > m.get("COUNTRY"),
Collectors.reducing("NaN", m - > m.get("LATITUDE"),
BinaryOperator.maxBy((s1, s2) - > latitudeComparator(Double.valueOf(s1), Double.valueOf(s2)))))));

Related

Reduce hanging when trying to sum up grouped values

I'm trying use a Project Reactor chain set up to collect and group values to finally sum them up by group. The collection is split into two parts and blocking.
In a simplified example I'm able to reproduce the problem. First I gather some generic data in createWrappers() which reads data from a the network (blocking calls). As data is retrieved objects are emitted. in the second step details are gathered from a different blocking network location and that information is added to the wrapper part. Then data gets transformed into a list of details, grouped by the details key and finally summed up by details key. In the end a map should be produced which looks like this (values are specific for the testcase):
key value
------------------
detail-0 1000
detail-1 2000
detail-2 3000
...
As soon as I add the block() to the reduce() part everything hangs in the sample code below:
import org.junit.jupiter.api.Test;
import reactor.core.publisher.Flux;
import reactor.core.publisher.FluxSink;
import reactor.core.scheduler.Schedulers;
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
public class TestBlockingIssue
{
#Test
public void testBlockingMap()
{
final Flux<Wrapper> source = Flux.create( sink -> createWrappers( 1000, sink ) );
final Map<String, BigDecimal> block = source.parallel( 10 ).runOn( Schedulers.boundedElastic() )
.map( wrapper -> enhanceWrapper( wrapper, 100 ) )
.flatMap( wrapper -> Flux.fromIterable( wrapper.detailsList ) )
.sequential()
.groupBy( details -> details.detailKey )
.cache()
.collectMap( group -> group.key(), group -> group.reduce( new BigDecimal( 0 ), ( x, y ) -> x.add( y.value ) ).block() ).block();
System.out.println( block );
}
private Wrapper enhanceWrapper( final Wrapper wrapper, final int count )
{
for ( int i = 0; i < count; i++ )
{
wrapper.detailsList.add( new Details( "detail-" + i, new BigDecimal( i +1 ) ) );
}
return wrapper;
}
private void createWrappers( final int count, final FluxSink<Wrapper> sink )
{
for ( int i = 0; i < count; i++ )
{
sink.next( new Wrapper( "Wrapper-" + i ) );
}
sink.complete();
}
private class Details
{
final String detailKey;
final BigDecimal value;
private Details( final String detailKey, final BigDecimal value )
{
this.detailKey = detailKey;
this.value = value;
}
}
private class Wrapper
{
final String lookupKey;
final List<Details> detailsList = new ArrayList<>();
private Wrapper( final String lookupKey )
{
this.lookupKey = lookupKey;
}
}
}
How could I resolve the issue with the hanging chain or which alternatives do I have to generate the map?
This occurs when using groupBy with too much groups and downstream isn't fast enough to consume the group. In your sample you should not block in the collect map but you should consume the group before collecting like:
final Map<String, BigDecimal> block = source.parallel( 10 ).runOn( Schedulers.boundedElastic() )
.map( wrapper -> enhanceWrapper( wrapper, 100 ) )
.flatMap( wrapper -> Flux.fromIterable( wrapper.detailsList ) )
.sequential()
.groupBy( details -> details.detailKey )
.cache()
.flatMap(g -> g.reduce( new BigDecimal( 0 ), ( x, y ) -> x.add( y.value ) ).map(v -> Tuples.of(g.key(), v)))
.collectMap(Tuple2::getT1, Tuple2::getT2)
.block();
So now downstream is fast enough but you might need to adjust concurrency depending on the number of group. And be sure that you have a low number of groups.

How to combine different mono and use the combined result with error handling?

I have a scenario where i need to use different mono which could return me errors and set map values to null if error is returned.
Ex:
Mono<A> a=Some api call;
Mono<A> b=Some api giving error;
Mono<A> c=Some api call;
Now i want to set the resulting response to map
Map<String,A> m=new HashMap<>();
m.put("a",a);
m.put("b",null);
m.put("c",c);
Can anyone help on how to do all this in reactive non blocking way.
I tried zip but it will not execute if any of the api return error or if i use onErrorReturn(null).
Thanks in advance
To solve your problems, you will have to use some tricks. The problem is that :
Giving an empty mono or mono that ends in error cancel zip operation (source: Mono#zip javadoc)
Reactive streams do not allow null values (source: Reactive stream spec, table 2: Subscribers, bullet 13)
Also, note that putting a null value in a hash map is the same as cancelling any previous value associated with the key (it's important in case you're updating an existing map).
Now, to bypass your problem, you can add an abstraction layer, and wrap your values in domain objects.
You can have an object that represents a query, another a valid result, and the last one will mirror an error.
With that, you can design publishers that will always succeed with non null values.
That's a technic used a lot in functional programming : common errors are part of the (one possible) result value.
Now, let's see the example that create a new Map from multiple Monos:
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import java.time.Duration;
import java.util.Map;
public class BypassMonoError {
/**
* An object identified by a key. It serves to track which key to associate to computed values
* #param <K> Type of the key
*/
static class Identified<K> {
protected final K id;
Identified(K id) {
this.id = id;
}
public K getId() {
return id;
}
}
/**
* Describe the result value of an operation, along with the key associated to it.
*
* #param <K> Type of the identifier of the result
* #param <V> Value type
*/
static abstract class Result<K, V> extends Identified<K> {
Result(K id) {
super(id);
}
/**
*
* #return Computed value on success, or null if the operation has failed. Note that here, we cannot tell from
* a success returning a null value or an error
*/
abstract V getOrNull();
}
static final class Success<K, V> extends Result<K, V> {
private final V value;
Success(K id, V value) {
super(id);
this.value = value;
}
#Override
V getOrNull() {
return value;
}
}
static final class Error<K, V> extends Result<K, V> {
private final Exception error;
Error(K id, Exception error) {
super(id);
this.error = error;
}
#Override
V getOrNull() {
return null;
}
public Exception getError() {
return error;
}
}
/**
* A request that can asynchronously generate a result for the associated identifier.
*/
static class Query<K, V> extends Identified<K> {
private final Mono<V> worker;
Query(K id, Mono<V> worker) {
super(id);
this.worker = worker;
}
/**
* #return The operator that computes the result value. Note that any error is silently wrapped in an
* {#link Error empty result with error metadata}.
*/
public Mono<Result<K, V>> runCatching() {
return worker.<Result<K, V>>map(success -> new Success<>(id, success))
.onErrorResume(Exception.class, error -> Mono.just(new Error<K, V>(id, error)));
}
}
public static void main(String[] args) {
final Flux<Query<String, String>> queries = Flux.just(
new Query("a", Mono.just("A")),
new Query("b", Mono.error(new Exception("B"))),
new Query("c", Mono.delay(Duration.ofSeconds(1)).map(v -> "C"))
);
final Flux<Result<String, String>> results = queries.flatMap(query -> query.runCatching());
final Map<String, String> myMap = results.collectMap(Result::getId, Result::getOrNull)
.block();
for (Map.Entry<String, String> entry : myMap.entrySet()) {
System.out.printf("%s -> %s%n", entry.getKey(), entry.getValue());
}
}
}
Note : In the above example, we silently ignore any occurred error. However, when using the flux, you can test if a result is an error, and if it is, you are free to design your own error management (log, fail-first, send in another flux, etc.).
This outputs:
a -> A
b -> null
c -> C

groupingBy on stream<String[]>

I have a situation where i have to generate a Map from List
I have a List as below
List<String> li = new ArrayList<String>();
li.add("A:abc");
li.add("B:xyz");
li.add("C:mno");
li.add("B:bbb");
li.add("A:aaa");
li.add("C:xxx");
I want to generate a Map from this as below
Map<String, String[]> or Map<String, List<String>>
I wrote the below lambda expression
li.stream().map(i->i.split(":")).collect(Collectors.groupingBy(i->i[0]));
but the resultant map is Map<String, List<String[]>> , which is not expected
Try it like this.
stream the list
split the string.
groupingBy on first element of array from split
the mapping to put the second element from array in the list.
List<String> li = new ArrayList<String>();
li.add("A:abc");
li.add("B:xyz");
li.add("C:mno");
li.add("B:bbb");
li.add("A:aaa");
li.add("C:xxx");
Map<String, List<String>> map =
li.stream().map(str -> str.split(":"))
.collect(Collectors.groupingBy(a -> a[0],
Collectors.mapping(a -> a[1],
Collectors.toList())));
map.entrySet().forEach(System.out::println);
prints
A=[abc, aaa]
B=[xyz, bbb]
C=[mno, xxx]
The answer from #WJS is perfect, however, if you want to get a Map<String, String[]> like you asked in the question, you might consider extending it using Collectors.collectingAndThen and List#toArray
Map<String, String[]> result = li.stream()
.map(x -> x.split(":"))
.collect(Collectors.groupingBy(
x -> x[0],
Collectors.mapping(
x -> x[1],
Collectors.collectingAndThen(
Collectors.toList(),
x -> x.toArray(String[]::new)
)
)
));
result.forEach((key, value) -> System.out.println(key + " : " + Arrays.toString(value)));
Prints the expected result
A : [abc, aaa]
B : [xyz, bbb]
C : [mno, xxx]
I want to generate a Map from this as below Map<String, String[]> or
Map<String, List<String>>
You do not need to call .map; just collect the stream using Collectors.groupingBy(s -> s.split(":")[0]).
Demo:
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
public class Main {
public static void main(String[] args) {
List<String> li = new ArrayList<String>();
li.add("A:abc");
li.add("B:xyz");
li.add("C:mno");
li.add("B:bbb");
li.add("A:aaa");
li.add("C:xxx");
Map<String, List<String>> map =
li.stream().collect(Collectors.groupingBy(s -> s.split(":")[0]));
map.entrySet().forEach(System.out::println);
}
}
Output:
A=[A:abc, A:aaa]
B=[B:xyz, B:bbb]
C=[C:mno, C:xxx]
However, if you are expecting a result like
A=[abc, aaa]
B=[xyz, bbb]
C=[mno, xxx]
you can call .map as you are doing and then group by the first element of the array, obtained as a result of splitting, mapping against the list of the second element of the array.
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
public class Main {
public static void main(String[] args) {
List<String> li = new ArrayList<String>();
li.add("A:abc");
li.add("B:xyz");
li.add("C:mno");
li.add("B:bbb");
li.add("A:aaa");
li.add("C:xxx");
Map<String, List<String>> map =
li.stream()
.map(s -> s.split(":"))
.collect(
Collectors.groupingBy(
arr -> arr[0], Collectors.mapping(
arr -> arr[1], Collectors.toList()
)
)
);
map.entrySet().forEach(System.out::println);
}
}

Dart HashMap initial value

I have a final HashMap in a class, How can I have a default value for it?
class RoomsState {
final HashMap<int, int> myMap;
RoomsState({
this.myMap= const {}
});
}
as const {} is a Map and not HashMap I cannot do it, Also HashMap is not a const constructor
Not sure if this is the only way to do it, but it is an option:
import 'dart:collection';
class RoomsState {
final HashMap<int, int> myMap;
RoomsState({
HashMap<int, int>? myMap
}) : this.myMap = myMap ?? HashMap();
}
The myMap parameter are here nullable since we are using the null value to identify if we got any argument.
You could expect a Map in your constructor and then convert it to a HashMap in an initializer.
class RoomsState {
final HashMap<int, int> myMap;
RoomsState({
Map<int, int> map = const {},
}) : myMap = HashMap.from(map);
}
HashMap has the construtor of that creates a HashMap that contains all key/value pairs of other. Check this for reference.
Example:
HashMap<String, String> englishToFrench = HashMap.of({
"go": "aller",
"buy": "acheter",
"sleep": "dormir",
});
Usage example:
void main() {
print(englishToFrench["go"]);
}
Output:
aller

Send multiple arguments to the compute function in Flutter

I was trying to use the compute function in Flutter.
void _blockPressHandler(int row, int col) async {
// Called when user clicks any block on the sudoku board . row and col are the corresponding row and col values ;
setState(() {
widget.selCol = col;
}
});
bool boardSolvable;
boardSolvable = await compute(SudokuAlgorithm.isBoardInSudoku , widget.board , widget.size) ;
}
isBoardInSudoku is a static method of class SudokuAlgorithm. Its present in another file. Writing the above code , tells me that
error: The argument type '(List<List<int>>, int) → bool' can't be assigned to the parameter type '(List<List<int>>) → bool'. (argument_type_not_assignable at [just_sudoku] lib/sudoku/SudokuPage.dart:161)
How do i fix this? Can it be done without bringing the SudokuAlgorithm class's methods out of its file ? How to send multiple arguments to the compute function ?
static bool isBoardInSudoku(List<List<int>>board , int size ){ } is my isBoardInSudoku function.
Just put the arguments in a Map and pass that instead.
There is no way to pass more than one argument to compute because it is a convenience function to start isolates which also don't allow anything but a single argument.
Use a map. Here is an example:
Map map = Map();
map['val1'] = val1;
map['val2'] = val2;
Future future1 = compute(longOp, map);
Future<double> longOp(map) async {
var val1 = map['val1'];
var val2 = map['val2'];
...
}
In OOP and in general, it is more elegant to create a class for that with fields you need, that gives you more flexibility and less hassle with hardcoded strings or constants for key names.
For example:
boardSolvable = await compute(SudokuAlgorithm.isBoardInSudoku , widget.board , widget.size) ;
replace with
class BoardSize{
final int board;
final int size;
BoardSize(this.board, this.size);
}
...
boardSolvable = await compute(SudokuAlgorithm.isBoardInSudoku, BoardSize(widget.board, widget.size)) ;
Use a Tuple
Here is some example code from my app:
#override
Future logChange(
String recordId, AttributeValue newValue, DateTime dateTime) async {
await compute(
logChangeNoCompute, Tuple2<String, AttributeValue>(recordId, newValue));
}
Future<void> logChangeNoCompute(Tuple2<String, AttributeValue> tuple) async {
_recordsById[tuple.item1]!.setAttributeValue(tuple.item2);
await storage.setItem(AssetsFileName, toJson());
}
You can have a function whose only argument is a Map so that you can pass multiple parameters by passing a Map with properties and values. However, the problem that I'm encountering now is that I cannot pass functions. If the value of a Map's property is a function I get an error when I run the compute function.
This example works(keep in mind that I've imported libraries and that's the reason why some functions and classes definitions aren't in this example)
Future<List<int>> getPotentialKeys({
#required int p,
#required int q,
})async{
return await compute(allKeys,{
"p" : p,
"q" : q,
});
}
List<int> allKeys(Map<String,dynamic> parameters){
AdvancedCipherGen key = AdvancedCipherGen();
List<int> possibleE = key.step1(p: parameters["p"], q: parameters["q"]);
return possibleE;
}
This does not work(same thing with a function as the value of a property thows an error)
Future<List<int>> getPotentialKeys({
#required int p,
#required int q,
#required Function(AdvancedCipherGen key) updateKey,
})async{
return await compute(allKeys,{
"p" : p,
"q" : q,
"updateKey" : updateKey,
});
}
List<int> allKeys(Map<String,dynamic> parameters){
AdvancedCipherGen key = AdvancedCipherGen();
List<int> possibleE = key.step1(p: parameters["p"], q: parameters["q"]);
//TODO: Update the key value through callback
parameters["updateKey"](key);
return possibleE;
}
easily use a Class, you can Also Use Map or List But using class is Better and Cleaner
class MyFunctionInput{
final int first;
final int second;
MyFunctionInput({required this.first,required this.second});
}
change your function like this
doSomething(MyFunctionInput input){
}
and use it like below
compute(doSomething,MyFunctionInput(first: 1, second: 4));

Resources