RxJava2 has a doAfterNext operator that emits items downstream, and then invokes the consumer. It doesn't seem like Project Reactor has such an operator so I'd like to get some pointers on the best way to create my own to achieve the same thing.
The use case is freeing memory after the subscriber has received the item
Not sure if leavering doOnEach is a valid solution:
public class ByteBufferSafeReleaseConsumer implements Consumer<Signal<ByteBuffer<?>>> {
private final List<ByteBuffer<?>> elements = new ArrayList<>();
#Override
public void accept(Signal<ByteBuffer<?>> signal) {
if (signal.isOnNext()) {
ByteBuffer<?> next = signal.get();
if (next != null) {
elements.add(next);
}
}
if (signal.isOnComplete() || signal.isOnError()) {
for (ByteBuffer<?> buffer : elements) {
ByteBufferUtils.safeRelease(buffer);
}
}
}
}
ByteBufferSafeReleaseConsumer consumer = new ByteBufferSafeReleaseConsumer()
Flux.from(byteBufferPublisher).doOnEach(consumer)
I have a combo box over my GUI in JavaFX.
This Combo Box is composed of a complex type elements :
public class DureeChoiceBoxElement extends ObservableValueBase<DureeChoiceBoxElement> {
private IntegerProperty duree;
#Override
public String toString() {
return duree.get() + " an";
}
}
I want to map (or bind) the selected complex element with my model which contains the simple type :
public class Pel {
private IntegerProperty duree = new SimpleIntegerProperty(1);
public Property<Number> dureeProperty() {
return duree;
}
public void setDuree(Integer duree) {
this.duree.setValue(duree);
}
public Integer getDuree() {
return duree.getValue();
}
}
How to do it ?
I tried in the controller with :
public class PelController {
#FXML
private ChoiceBox<DureeChoiceBoxElement> duree;
//etc..
pel.dureeProperty().bind(createElapsedBindingByBindingsAPI2(duree.getValue()));
/*
* #return an ObjectBinding of immutable TimeElapsed objects for the player
*/
private ObjectBinding<Property<Number>> createElapsedBindingByBindingsAPI2(
final DureeChoiceBoxElement dureeChoiceBoxElement) {
return Bindings.createObjectBinding(new Callable<Property<Number>>() {
#Override
public IntegerProperty call() throws Exception {
return dureeChoiceBoxElement.dureeProperty();
}
}, dureeChoiceBoxElement.dureeProperty());
}
}
But it doesn't work (even not compile). I want to say that "Bind this simple property to this complex Object calling the method I give you through the method named "createElapsedBindingByBindingsAPI2(..)".
It is logical read but I didn't managed to make it works anyway.
That's poor ....
Any help please :).
Example that (obviously) works with legacy code style (Swing coding) :
duree.getSelectionModel().selectedItemProperty().addListener(new ChangeListener<DureeChoiceBoxElement>() {
#Override
public void changed(ObservableValue<? extends DureeChoiceBoxElement> observable,
DureeChoiceBoxElement oldValue, DureeChoiceBoxElement newValue) {
// changement durée
log.debug("Durée sélectionnée : {}", duree.getSelectionModel().getSelectedItem().getDuree());
log.debug("Durée bindée ? : {}", pel.getDuree());
pel.setDuree(duree.getSelectionModel().getSelectedItem().getDuree());
}
});
Like this my model is set to selected item. But it implies some boilerplate code. Any better idea based on high level bindings of JavaFX ?
I've just recently started using Rhino-Etl for very simple ETL processes and have had great success with it. I have a slightly more complicated scenario to address now and I didn't find the ConventionInputCommandOperation behaving the way I expected.
I've done up a very simplified example of what I'm trying to do. Basically I have two systems involved and I don't know what I want to get from system 2 until I first query system 1. I thought registering an InputOperation immediately after another InputOperation would behave like a loop. So that each row in operation 1 would be fed to operation 2. The below code fails with "Failed to execute operation DetailReader: Must declare the scalar variable #PlanetAbbrv." So my question is how are you meant to handle situations where the input operation is dependent a previous input operation?
Thanks,
Brian
using System;
using Rhino.Etl.Core;
using Rhino.Etl.Core.ConventionOperations;
namespace ETLTest
{
class Program
{
static void Main()
{
new MainProcess().Execute();
Console.ReadLine();
}
}
public class MainProcess : EtlProcess
{
protected override void Initialize()
{
Register(new MainReader());
Register(new DetailReader());
}
protected override void PostProcessing()
{
foreach (var exception in GetAllErrors())
{
throw exception;
}
}
}
public class MainReader : ConventionInputCommandOperation
{
public MainReader() : base("Galactic1")
{
Command = #"select * from Planet";
}
}
public class DetailReader : ConventionInputCommandOperation
{
public DetailReader() : base("Galactic2")
{
Command = #"select * from Delivery where DeliveryPlanetAbbrv = #PlanetAbbrv";
}
}
}
You'll need to have your DetailReader select all rows (take out the where operation).
Then use a JoinOperation to match the details to the main information.
Register(new JoinPlanets()
.Right(new MainReader())
.Left(new DetailReader()));
public class JoinPlanets: JoinOperation
{
protected override Row MergeRows(Row leftRow, Row rightRow)
{
Row row = leftRow.Clone();
foreach (var column in leftRow.Columns)
row[column] = leftRow[column];
return row;
}
protected override void SetupJoinConditions()
{
FullOuterJoin.Left("PlanetAbbrv")
.Right("DeliveryPlanetAbbrv");
}
}
I'm rolling my own ActivatableCollection<T> for db4o but cribbing heavily from the builtin ActivatableList<T> implementation. I'm running into the problem where transparent persistence doesn't seem to be working correctly. In the test code below:
[Fact]
void CanStoreActivatableCollection()
{
var planets = new ActivatableCollection<Planet>();
var pagingMemoryStorage = new PagingMemoryStorage();
var config = Db4oEmbedded.NewConfiguration();
config.Common.Add(new TransparentActivationSupport());
config.Common.Add(new TransparentPersistenceSupport());
config.File.Storage = pagingMemoryStorage;
var objectContainer = Db4oEmbedded.OpenFile(config, "Memory.yap");
planets.Add(new Planet("Mercury"));
objectContainer.Store(planets);
planets.Add(new Planet("Venus"));
planets.Add(new Planet("Earth"));
objectContainer.Commit();
objectContainer.Close();
config = Db4oEmbedded.NewConfiguration();
config.Common.Add(new TransparentActivationSupport());
config.Common.Add(new TransparentPersistenceSupport());
config.File.Storage = pagingMemoryStorage;
objectContainer = Db4oEmbedded.OpenFile(config, "Memory.yap");
planets = objectContainer.Query<ActivatableCollection<Planet>>().FirstOrDefault();
Assert.NotNull(planets);
Assert.Equal(3, planets.Count);
objectContainer.Close();
}
The planet "Mercury" is stored, but not "Venus" and "Earth". If I change from ActivatableCollection to ActivatableList, then all 3 planets are stored.
What am I missing? My ActivatableCollection is just minimal implementation of ActivatableList as best as I can tell.
Below is my implementation of ActivatableCollection:
public class ActivatableCollection<T>
: ICollection<T>
, IActivatable
, INotifyCollectionChanged
{
List<T> _list;
List<T> List
{
get
{
if (_list == null)
_list = new List<T>();
return _list;
}
}
public ActivatableCollection()
{
}
public int Count
{
get
{
ActivateForRead();
return List.Count;
}
}
public bool IsReadOnly
{
get
{
ActivateForRead();
return ((IList) List).IsReadOnly;
}
}
public void Add(T t)
{
ActivateForWrite();
List.Add(t);
OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Add, t));
}
public void Clear()
{
ActivateForWrite();
List.Clear();
OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset));
}
public bool Contains(T t)
{
ActivateForRead();
return List.Contains(t);
}
public void CopyTo(T[] array, int index)
{
ActivateForRead();
List.CopyTo(array, index);
}
public IEnumerator<T> GetEnumerator()
{
ActivateForRead();
return List.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
public bool Remove(T t)
{
ActivateForWrite();
bool removed = List.Remove(t);
if (removed)
OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Remove, t));
return removed;
}
[Transient]
private IActivator _activator;
public virtual void Bind(IActivator activator)
{
if (_activator == activator)
return;
if (activator != null && _activator != null)
throw new InvalidOperationException();
_activator = activator;
}
public virtual void Activate(ActivationPurpose purpose)
{
if (_activator == null)
return;
_activator.Activate(purpose);
}
protected virtual void ActivateForRead()
{
Activate(ActivationPurpose.Read);
}
protected virtual void ActivateForWrite()
{
Activate(ActivationPurpose.Write);
}
[Transient]
public event NotifyCollectionChangedEventHandler CollectionChanged;
protected virtual void OnCollectionChanged(NotifyCollectionChangedEventArgs e)
{
if (CollectionChanged != null)
CollectionChanged(this, e);
}
}
I've also tried copying the code from GenericTypeHandlerPredicate and registering my ActivatableCollection to use the GenericCollectionTypeHandler. That results in a crash in GenericTypeFor() throwing an InvalidOperationException() when "Mercury" is being stored.
Just want to mention my answers from the db4o forums also here, for people with a similar problem:
First part of the issue:
From db4o's point of view nothing has changed in the 'ActivatableCollection' object and therefore no changes are stored. This is what is happening:
When you add the items, the ActivatableCollection is marked as changed.
When you commit the changes are stored. However the ' ActivatableCollection' holds the reference to the same object. db4o only stores the changes in the ActivatableCollection-object, which is the reference to the List. Since it is the same, no actual change is stored.
The List of the ActivatableCollection is never updated, because it wasn't marked as 'changed'
So the transparent activation doesn't see the changes in the list. You can fix your issue simply by using an ActivatableList in you're ActivatableCollection implementation. Just change the List with a IList interface and instantiate a ActivatableList instead of an List.
The second part of the issue: Why doesn't it work even when registering the GenericCollectionTypeHandler for this type? Here we hit a implementation detail. The GenericCollectionTypeHandler has an internal list of supported types, which doesn't include the self made 'ActivatableCollection'. GenericCollectionTypeHandler is not really part of the public API and intendet for internal use only.
Workaround / Fix
Just use an ActivatableList<T> instead of a List<T>. then everything works fine.
The problem I have is that I need to do about 40+ conversions to convert loosely typed info into strongly typed info stored in db, xml file, etc.
I'm plan to tag each type with a tuple i.e. a transformational form like this:
host.name.string:host.dotquad.string
which will offer a conversion from the input to an output form. For example, the name stored in the host field of type string, the input is converted into a dotquad notation of type string and stored back into host field. More complex conversions may need several steps, with each step being accomplished by a method call, hence method chaining.
Examining further the example above, the tuple 'host.name.string' with the field host of name www.domain.com. A DNS lookup is done to covert domain name to IP address. Another method is applied to change the type returned by the DNS lookup into the internal type of dotquad of type string. For this transformation, there is 4 seperate methods called to convert from one tuple into another. Some other conversions may require more steps.
Ideally I would like an small example of how method chains are constructed at runtime. Development time method chaining is relatively trivial, but would require pages and pages of code to cover all possibilites, with 40+ conversions.
One way I thought of doing is, is parsing the tuples at startup, and writing the chains out to an assembly, compiling it, then using reflection to load/access. Its would be really ugly and negate the performance increases i'm hoping to gain.
I'm using Mono, so no C# 4.0
Any help would be appreciated.
Bob.
Here is a quick and dirty solution using LINQ Expressions. You have indicated that you want C# 2.0, this is 3.5, but it does run on Mono 2.6. The method chaining is a bit hacky as i didn't exactly know how your version works, so you might need to tweak the expression code to suit.
The real magic really happens in the Chainer class, which takes a collection of strings, which represent the MethodChain subclass. Take a collection like this:
{
"string",
"string",
"int"
}
This will generate a chain like this:
new StringChain(new StringChain(new IntChain()));
Chainer.CreateChain will return a lambda that calls MethodChain.Execute(). Because Chainer.CreateChain uses a bit of reflection, it's slow, but it only needs to run once for each expression chain. The execution of the lambda is nearly as fast as calling actual code.
Hope you can fit this into your architecture.
public abstract class MethodChain {
private MethodChain[] m_methods;
private object m_Result;
public MethodChain(params MethodChain[] methods) {
m_methods = methods;
}
public MethodChain Execute(object expression) {
if(m_methods != null) {
foreach(var method in m_methods) {
expression = method.Execute(expression).GetResult<object>();
}
}
m_Result = ExecuteInternal(expression);
return this;
}
protected abstract object ExecuteInternal(object expression);
public T GetResult<T>() {
return (T)m_Result;
}
}
public class IntChain : MethodChain {
public IntChain(params MethodChain[] methods)
: base(methods) {
}
protected override object ExecuteInternal(object expression) {
return int.Parse(expression as string);
}
}
public class StringChain : MethodChain {
public StringChain(params MethodChain[] methods):base(methods) {
}
protected override object ExecuteInternal(object expression) {
return (expression as string).Trim();
}
}
public class Chainer {
/// <summary>
/// methods are executed from back to front, so methods[1] will call method[0].Execute before executing itself
/// </summary>
/// <param name="methods"></param>
/// <returns></returns>
public Func<object, MethodChain> CreateChain(IEnumerable<string> methods) {
Expression expr = null;
foreach(var methodName in methods.Reverse()) {
ConstructorInfo cInfo= null;
switch(methodName.ToLower()) {
case "string":
cInfo = typeof(StringChain).GetConstructor(new []{typeof(MethodChain[])});
break;
case "int":
cInfo = typeof(IntChain).GetConstructor(new[] { typeof(MethodChain[]) });
break;
}
if(cInfo == null)
continue;
if(expr != null)
expr = Expression.New(cInfo, Expression.NewArrayInit( typeof(MethodChain), Expression.Convert(expr, typeof(MethodChain))));
else
expr = Expression.New(cInfo, Expression.Constant(null, typeof(MethodChain[])));
}
var objParam = Expression.Parameter(typeof(object));
var methodExpr = Expression.Call(expr, typeof(MethodChain).GetMethod("Execute"), objParam);
Func<object, MethodChain> lambda = Expression.Lambda<Func<object, MethodChain>>(methodExpr, objParam).Compile();
return lambda;
}
[TestMethod]
public void ExprTest() {
Chainer chainer = new Chainer();
var lambda = chainer.CreateChain(new[] { "int", "string" });
var result = lambda(" 34 ").GetResult<int>();
Assert.AreEqual(34, result);
}
}
The command pattern would fit here. What you could do is queue up commands as you need different operations performed on the different data types. Those messages could then all be processed and call the appropriate methods when you're ready later on.
This pattern can be implemented in .NET 2.0.
Do you really need to do this at execution time? Can't you create the combination of operations using code generation?
Let me elaborate:
Assuming you have a class called Conversions which contains all the 40+ convertions you mentioned like this:
//just pseudo code..
class conversions{
string host_name(string input){}
string host_dotquad(string input){}
int type_convert(string input){}
float type_convert(string input){}
float increment_float(float input){}
}
Write a simple console app or something similar which uses reflection to generate code for methods like this:
execute_host_name(string input, Queue<string> conversionQueue)
{
string ouput = conversions.host_name(input);
if(conversionQueue.Count == 0)
return output;
switch(conversionQueue.dequeue())
{
// generate case statements only for methods that take in
// a string as parameter because the host_name method returns a string.
case "host.dotquad": return execute_host_dotquad(output,conversionQueue);
case "type.convert": return execute_type_convert(output, conversionQueue);
default: // exception...
}
}
Wrap all this in a Nice little execute method like this:
object execute(string input, string [] conversions)
{
Queue<string> conversionQueue = //create the queue..
case(conversionQueue.dequeue())
{
case "host.name": return execute_host_name(output,conversionQueue);
case "host.dotquad": return execute_host_dotquad(output,conversionQueue);
case "type.convert": return execute_type_convert(output, conversionQueue);
default: // exception...
}
}
This code generation application need to be executed only when your method signatures changes or when you decide to add new transformations.
Main advantages:
No runtime overhead
Easy to add/delete/change the conversions (code generator will take care of the code changes :) )
What do you think?
I apologize for the long code dump and the fact that it is in Java, rather than C#, but I found your problem quite interesting and I do not have much C# experience. Hopefully you will be able to adapt this solution without difficulty.
One approach to solving your problem is to create a cost for each conversion -- usually this is related to the accuracy of the conversion -- and then perform a search to find the best possible conversion sequence to get from one type to another.
The reason for needing a cost function is to choose among multiple conversion paths. For example, converting from an integer to a string is lossless, but there is no guarantee that every string can be represented by an integer. So, if you had two conversion chains
string -> integer -> float -> decimal
string -> float -> decimal
You would want to select the second one because it will reduce the chance of a conversion failure.
The Java code below implements such a scheme and performs a best-first search to find an optimal conversion sequence. I hope you find it useful. Running the code produces the following output:
> No conversion possible from string to integer
> The optimal conversion sequence from string to host.dotquad.string is:
> string to host.name.string, cost = -1.609438
> host.name.string to host.dns, cost = -1.609438 *PERFECT*
> host.dns to host.dotquad, cost = -1.832581
> host.dotquad to host.dotquad.string, cost = -1.832581 *PERFECT*
Here is the Java code.
/**
* Use best-first search to find an optimal sequence of operations for
* performing a type conversion with maximum fidelity.
*/
import java.util.*;
public class TypeConversion {
/**
* Define a type-conversion interface. It converts between to
* user-defined types and provides a measure of fidelity (accuracy)
* of the conversion.
*/
interface ITypeConverter<T, F> {
public T convert(F from);
public double fidelity();
// Could use reflection instead of handling this explicitly
public String getSourceType();
public String getTargetType();
}
/**
* Create a set of user-defined types.
*/
class HostName {
public String hostName;
public HostName(String hostName) {
this.hostName = hostName;
}
}
class DnsLookup {
public String ipAddress;
public DnsLookup(HostName hostName) {
this.ipAddress = doDNSLookup(hostName);
}
private String doDNSLookup(HostName hostName) {
return "127.0.0.1";
}
}
class DottedQuad {
public int[] quad = new int[4];
public DottedQuad(DnsLookup lookup) {
String[] split = lookup.ipAddress.split(".");
for ( int i = 0; i < 4; i++ )
quad[i] = Integer.parseInt( split[i] );
}
}
/**
* Define a set of conversion operations between the types. We only
* implement a minimal number for brevity, but this could be expanded.
*
* We start by creating some broad classes to differentiate among
* perfect, good and bad conversions.
*/
abstract class PerfectTypeConversion<T, F> implements ITypeConverter<T, F> {
public abstract T convert(F from);
public double fidelity() { return 1.0; }
}
abstract class GoodTypeConversion<T, F> implements ITypeConverter<T, F> {
public abstract T convert(F from);
public double fidelity() { return 0.8; }
}
abstract class BadTypeConversion<T, F> implements ITypeConverter<T, F> {
public abstract T convert(F from);
public double fidelity() { return 0.2; }
}
/**
* Concrete classes that do the actual conversions.
*/
class StringToHostName extends BadTypeConversion<HostName, String> {
public HostName convert(String from) { return new HostName(from); }
public String getSourceType() { return "string"; }
public String getTargetType() { return "host.name.string"; }
}
class HostNameToDnsLookup extends PerfectTypeConversion<DnsLookup, HostName> {
public DnsLookup convert(HostName from) { return new DnsLookup(from); }
public String getSourceType() { return "host.name.string"; }
public String getTargetType() { return "host.dns"; }
}
class DnsLookupToDottedQuad extends GoodTypeConversion<DottedQuad, DnsLookup> {
public DottedQuad convert(DnsLookup from) { return new DottedQuad(from); }
public String getSourceType() { return "host.dns"; }
public String getTargetType() { return "host.dotquad"; }
}
class DottedQuadToString extends PerfectTypeConversion<String, DottedQuad> {
public String convert(DottedQuad f) {
return f.quad[0] + "." + f.quad[1] + "." + f.quad[2] + "." + f.quad[3];
}
public String getSourceType() { return "host.dotquad"; }
public String getTargetType() { return "host.dotquad.string"; }
}
/**
* To find the best conversion sequence, we need to instantiate
* a list of converters.
*/
ITypeConverter<?,?> converters[] =
{
new StringToHostName(),
new HostNameToDnsLookup(),
new DnsLookupToDottedQuad(),
new DottedQuadToString()
};
Map<String, List<ITypeConverter<?,?>>> fromMap =
new HashMap<String, List<ITypeConverter<?,?>>>();
public void buildConversionMap()
{
for ( ITypeConverter<?,?> converter : converters )
{
String type = converter.getSourceType();
if ( !fromMap.containsKey( type )) {
fromMap.put( type, new ArrayList<ITypeConverter<?,?>>());
}
fromMap.get(type).add(converter);
}
}
public class Tuple implements Comparable<Tuple>
{
public String type;
public double cost;
public Tuple parent;
public Tuple(String type, double cost, Tuple parent) {
this.type = type;
this.cost = cost;
this.parent = parent;
}
public int compareTo(Tuple o) {
return Double.compare( cost, o.cost );
}
}
public Tuple findOptimalConversionSequence(String from, String target)
{
PriorityQueue<Tuple> queue = new PriorityQueue<Tuple>();
// Add a dummy start node to the queue
queue.add( new Tuple( from, 0.0, null ));
// Perform the search
while ( !queue.isEmpty() )
{
// Pop the most promising candidate from the list
Tuple tuple = queue.remove();
// If the type matches the target type, return
if ( tuple.type == target )
return tuple;
// If we have reached a dead-end, backtrack
if ( !fromMap.containsKey( tuple.type ))
continue;
// Otherwise get all of the possible conversions to
// perform next and add their costs
for ( ITypeConverter<?,?> converter : fromMap.get( tuple.type ))
{
String type = converter.getTargetType();
double cost = tuple.cost + Math.log( converter.fidelity() );
queue.add( new Tuple( type, cost, tuple ));
}
}
// No solution
return null;
}
public static void convert(String from, String target)
{
TypeConversion tc = new TypeConversion();
// Build a conversion lookup table
tc.buildConversionMap();
// Find the tail of the optimal conversion chain.
Tuple tail = tc.findOptimalConversionSequence( from, target );
if ( tail == null ) {
System.out.println( "No conversion possible from " + from + " to " + target );
return;
}
// Reconstruct the conversion path (skip dummy node)
List<Tuple> solution = new ArrayList<Tuple>();
for ( ; tail.parent != null ; tail = tail.parent )
solution.add( tail );
Collections.reverse( solution );
StringBuilder sb = new StringBuilder();
Formatter formatter = new Formatter(sb);
sb.append( "The optimal conversion sequence from " + from + " to " + target + " is:\n" );
for ( Tuple tuple : solution ) {
formatter.format( "%20s to %20s, cost = %f", tuple.parent.type, tuple.type, tuple.cost );
if ( tuple.cost == tuple.parent.cost )
sb.append( " *PERFECT*");
sb.append( "\n" );
}
System.out.println( sb.toString() );
}
public static void main(String[] args)
{
// Run two tests
convert( "string", "integer" );
convert( "string", "host.dotquad.string" );
}
}