Chaining InputOperations in Rhino-Etl - rhino-etl

I've just recently started using Rhino-Etl for very simple ETL processes and have had great success with it. I have a slightly more complicated scenario to address now and I didn't find the ConventionInputCommandOperation behaving the way I expected.
I've done up a very simplified example of what I'm trying to do. Basically I have two systems involved and I don't know what I want to get from system 2 until I first query system 1. I thought registering an InputOperation immediately after another InputOperation would behave like a loop. So that each row in operation 1 would be fed to operation 2. The below code fails with "Failed to execute operation DetailReader: Must declare the scalar variable #PlanetAbbrv." So my question is how are you meant to handle situations where the input operation is dependent a previous input operation?
Thanks,
Brian
using System;
using Rhino.Etl.Core;
using Rhino.Etl.Core.ConventionOperations;
namespace ETLTest
{
class Program
{
static void Main()
{
new MainProcess().Execute();
Console.ReadLine();
}
}
public class MainProcess : EtlProcess
{
protected override void Initialize()
{
Register(new MainReader());
Register(new DetailReader());
}
protected override void PostProcessing()
{
foreach (var exception in GetAllErrors())
{
throw exception;
}
}
}
public class MainReader : ConventionInputCommandOperation
{
public MainReader() : base("Galactic1")
{
Command = #"select * from Planet";
}
}
public class DetailReader : ConventionInputCommandOperation
{
public DetailReader() : base("Galactic2")
{
Command = #"select * from Delivery where DeliveryPlanetAbbrv = #PlanetAbbrv";
}
}
}

You'll need to have your DetailReader select all rows (take out the where operation).
Then use a JoinOperation to match the details to the main information.
Register(new JoinPlanets()
.Right(new MainReader())
.Left(new DetailReader()));
public class JoinPlanets: JoinOperation
{
protected override Row MergeRows(Row leftRow, Row rightRow)
{
Row row = leftRow.Clone();
foreach (var column in leftRow.Columns)
row[column] = leftRow[column];
return row;
}
protected override void SetupJoinConditions()
{
FullOuterJoin.Left("PlanetAbbrv")
.Right("DeliveryPlanetAbbrv");
}
}

Related

How would one implement a doAfterNext operator in Project Reactor?

RxJava2 has a doAfterNext operator that emits items downstream, and then invokes the consumer. It doesn't seem like Project Reactor has such an operator so I'd like to get some pointers on the best way to create my own to achieve the same thing.
The use case is freeing memory after the subscriber has received the item
Not sure if leavering doOnEach is a valid solution:
public class ByteBufferSafeReleaseConsumer implements Consumer<Signal<ByteBuffer<?>>> {
private final List<ByteBuffer<?>> elements = new ArrayList<>();
#Override
public void accept(Signal<ByteBuffer<?>> signal) {
if (signal.isOnNext()) {
ByteBuffer<?> next = signal.get();
if (next != null) {
elements.add(next);
}
}
if (signal.isOnComplete() || signal.isOnError()) {
for (ByteBuffer<?> buffer : elements) {
ByteBufferUtils.safeRelease(buffer);
}
}
}
}
ByteBufferSafeReleaseConsumer consumer = new ByteBufferSafeReleaseConsumer()
Flux.from(byteBufferPublisher).doOnEach(consumer)

Binding between an Object and a SimpleIntegerProperty

I have a combo box over my GUI in JavaFX.
This Combo Box is composed of a complex type elements :
public class DureeChoiceBoxElement extends ObservableValueBase<DureeChoiceBoxElement> {
private IntegerProperty duree;
#Override
public String toString() {
return duree.get() + " an";
}
}
I want to map (or bind) the selected complex element with my model which contains the simple type :
public class Pel {
private IntegerProperty duree = new SimpleIntegerProperty(1);
public Property<Number> dureeProperty() {
return duree;
}
public void setDuree(Integer duree) {
this.duree.setValue(duree);
}
public Integer getDuree() {
return duree.getValue();
}
}
How to do it ?
I tried in the controller with :
public class PelController {
#FXML
private ChoiceBox<DureeChoiceBoxElement> duree;
//etc..
pel.dureeProperty().bind(createElapsedBindingByBindingsAPI2(duree.getValue()));
/*
* #return an ObjectBinding of immutable TimeElapsed objects for the player
*/
private ObjectBinding<Property<Number>> createElapsedBindingByBindingsAPI2(
final DureeChoiceBoxElement dureeChoiceBoxElement) {
return Bindings.createObjectBinding(new Callable<Property<Number>>() {
#Override
public IntegerProperty call() throws Exception {
return dureeChoiceBoxElement.dureeProperty();
}
}, dureeChoiceBoxElement.dureeProperty());
}
}
But it doesn't work (even not compile). I want to say that "Bind this simple property to this complex Object calling the method I give you through the method named "createElapsedBindingByBindingsAPI2(..)".
It is logical read but I didn't managed to make it works anyway.
That's poor ....
Any help please :).
Example that (obviously) works with legacy code style (Swing coding) :
duree.getSelectionModel().selectedItemProperty().addListener(new ChangeListener<DureeChoiceBoxElement>() {
#Override
public void changed(ObservableValue<? extends DureeChoiceBoxElement> observable,
DureeChoiceBoxElement oldValue, DureeChoiceBoxElement newValue) {
// changement durée
log.debug("Durée sélectionnée : {}", duree.getSelectionModel().getSelectedItem().getDuree());
log.debug("Durée bindée ? : {}", pel.getDuree());
pel.setDuree(duree.getSelectionModel().getSelectedItem().getDuree());
}
});
Like this my model is set to selected item. But it implies some boilerplate code. Any better idea based on high level bindings of JavaFX ?

How to get Parse.com object count in class

I want to get the object count from 5 classes in Parse.com (to check if all objects were fetched successfully).
Because I'm using findObjectsInBackgroundWithBlock: sometimes not all the objects are fetched before I'm using it, that's why I want to check.
How can I do that?
Update 2: just noticed you were asking about iOS. its basically the same principle, use fetchAllIfNeeded like so:
https://parse.com/docs/ios/api/Categories/PFObject(Synchronous).html#/c:objc(cs)PFObject(cm)fetchAllIfNeeded:
Update: a better way than the naive one (below) would probably be using fetchAllIfNeededInBackground:
ArrayList<ParseObject> objectsToFetch = new ArrayList<>();
objectsToFetch.add(object1);
objectsToFetch.add(object2);
objectsToFetch.add(object3);
ParseObject.fetchAllIfNeededInBackground(objectsToFetch, new FindCallback<ParseObject>() {
#Override
public void done(List<ParseObject> objects, ParseException e) {
//all are fetched, do stuff
}
});
My native way of doing this is adding an outer boolean array where each boolean is responsible for one of the classes.
When a findObjectsInBackgroundWithBlock's 'done' function runs, I set that boolean to "true" and run a function that checks whether all array is true, if so -> all classes have been fetched and I can continue.
example from my code:
boolean[] itemFetched;
protected void getAllClassesAndDoStuffWithThem() {
itemFetched= new boolean[NUM_OF_CLASSES];
for (int i=0;i<NUM_OF_CLASSES;i++){
final int finalI = i;
itemFetched[i] = false;
parseObjectArray[i].fetchIfNeededInBackground(new GetCallback<ParseObject>() {
#Override
public void done(ParseObject object, ParseException e) {
itemFetched[finalI] = true;
finishInitIfPossible();
}
});
}
}
private void finishInitIfPossible() {
for (int i=0;i<NUM_OF_CLASSES;i++){
if (!itemFetched[i])
return;
}
//all classes fetched
finishInit();
}
private void finishInit() {
//do stuff with all 5 classes
}

connecting vertices in jung by edges results in another extra vertex being created

I am implementing an interface for taking commands for creating , connecting and coloring vertices in JUNG when I want to connect two already existing vertices JUNG connects to vertices and creates an extra vertex , why?
Here is my code for connect method:
public class Connect extends Command {
private CommandMaster cm;
private BehGraphUndirected behGraph;
private static int edgenumber=0;
#Override
public Object run(BehGraphUndirected behGraph, VisualizationImageServer panel, InterpretMaster interpretMaster, String... args) {
System.out.print("connect Runs\n");
this.cm = new CommandMaster();
this.behGraph = behGraph;
if(cm.exists(args[0]))
{
//got to another command
}else
{
switch (args[0]) {
case "edge":
this.createEdge(args[1] , args[2]);
break;
}
}
interpretMaster.refreshAndRepaint();
return null;
}
public void createEdge(String nodeName1 , String nodeName2)
{
this.behGraph.addEdge(edgenumber++,nodeName1, nodeName2);
System.out.println(this.behGraph.getVertexCount());
System.out.println("edge between: "+nodeName1+" and "+ nodeName2+" added");
}
And it's the create method just in case you want to know the way I implemented the code:
package interpreter.command;
import GraphHandling.BehGraphUndirected;
import edu.uci.ics.jung.visualization.VisualizationImageServer;
import interpreter.Command;
import interpreter.CommandMaster;
import interpreter.InterpretMaster;
/**
*
* #author Administrator
*/
public class Create extends Command{
private CommandMaster cm;
private BehGraphUndirected behGraph;
#Override
public Object run(BehGraphUndirected behGraph, VisualizationImageServer panel, InterpretMaster interpretMaster, String... args) {
System.out.print("create Runs \n");
this.cm = new CommandMaster();
this.behGraph = behGraph;
if(cm.exists(args[0]))
{
//got to another command
}else
{
switch (args[0]) {
case "node":
this.createNode(args[1]);
break;
case "label":
this.createLabel(args[1]);
break;
}
}
interpretMaster.refreshAndRepaint();
return null;
}
public void createNode(String nodeName)
{
this.behGraph.addVertex(nodeName);
System.out.print("vertex: "+nodeName+" added");
}
private void createLabel(String string) {
}
class str
{
int i;
long j;
}
}
Graph images before and after connecting two nodes:
and Here is my BehGraphUndirected class:
package GraphHandling;
import edu.uci.ics.jung.graph.UndirectedSparseGraph;
import java.util.LinkedList;
/**
*
* #author Administrator
*/
public class BehGraphUndirected extends UndirectedSparseGraph{
private final LinkedList<Node> nodeList;
public BehGraphUndirected()
{
this.nodeList = new LinkedList<>();
}
public void addNode(Node newNode)
{
this.nodeList.add(newNode);
}
}
You should look at what BehGraphUndirected is doing; it's not a JUNG class or interface.
What is the name of the vertex that's being created, and how does that relate to what's being passed to the create method?
I have compiled and tested your code , The Jung library seems working right and It extinguishes the different nodes by the different object that was given to it It seems you have some other problem , Like a problem in processing the input strings that are used as objects that create nodes.

How can i call dynamic object like a function?

I am studying orchard architecture.i have faced with a strange concept in display management section.
in Partial view page there is a 'function call like' syntax like so Display(Model.Head). that is not a function thought, it is a dynamic object defined in orchard WebViewPage.
I am wondering how to define a dynamic object so that you can call it (and pass it an argument as well) like a function as i mentioned.
thanks in advance.
A lighter weight way to do it without clay would be to subclass the built-in DynamicObject class.
public static dynamic Display;
void Main()
{
Display = new MyCallableObject();
//this is what i was after
Console.Write(Display("bla bla bla"));
}
public class MyCallableObject:DynamicObject
{
public override bool TryInvoke(InvokeBinder binder, object[] args, out Object result)
{
result = string.Format("This is response for {0}",args.FirstOrDefault());
return true;
}
}
I finally found it my self!
all the operations have done with Clay Library behind the scene.i have wrote a sample console app for demonstrating the process.
class Program
{
static void Main(string[] args)
{
Display = ClayActivator.CreateInstance<MyResponser>(new List<IClayBehavior> {new MyFunctionCallBehavior()});
//this is what i was after
Console.Write(Display("bla bla bla"));
}
public static dynamic Display;
}
public class MyFunctionCallBehavior : IClayBehavior
{
public object InvokeMember(Func<object> proceed, object self, string name, INamedEnumerable<object> args)
{
return ((MyResponser)self).ResponseForRequest(args.FirstOrDefault().ToString());
}
}
public class MyResponser
{
public string ResponseForRequest(string param)
{
return string.Format("This is response for {0}",param);
}
}

Resources