I need to implement a listener that receives multiple messages then groups them base on a certain criteria then send the grouped messages to the next queue. In other words batching them in a logical way.
I'm using Grails and JMS integration plugin.
In my mind, it can be implemented in a listener that runs a single thread that runs an infinite loop that forever consume messages from a queue then do the grouping logic there.
Runnable runnable = new Runnable() {
#Override
public void run() {
while(true) {
def obj = consumer.receive()
//group the messages and store on a list then send them in groups to the next queue
producer.send(groupedObjList)
}
}
}
def thread = new Thread(runnable);
thread.start();
I'm not sure if this is a clean way to do it. So I'm looking for alternatives on how to implement this.
Related
I'm trying to create a Flux from incoming messages received from a queue.
For instance, if I'm using Amazon SQS how do I achieve to write the following code:
Flux<String> messages = connectionToSQS.receiveFromQueue(queueName);
messages.map(s -> log.info("message: {}", s).subscribe();
After experimentation, I found the following issues:
How do I keep requesting messages from the queue (loop forever)? Do I create one thread that has a loop that keeps on requesting from the queue?
How do I make the Flux cold? I don't want to request messages from SQS unless the consumer asks for it. This allows me to use backpressure.
First pass over this problem yielded something like the following code as per Reactor documentation:
Flux<String> bridge = Flux.create(sink -> {
myEventProcessor.register(new MyEventListener<String>() {
public void onDataChunk(List<String> chunk) {
for(String s : chunk) {
sink.next(s);
}
}
public void processComplete() {
sink.complete();
}
});
});
The idea being to create a single thread that keeps on requesting for messages in a loop and then using an observer pattern like above to do a next() on each message received.
I have requirement where workflow should start execution but it should wait for next subsequent events to do the execution. Events are generated based of API calls that the service receives from client. The events semantics is based on a state machine. Please let me know how this can be implemented in java. Please provide any sample code for reference.
If you are using AWS Flow Framework an external event should use Signal API to notify a workflow instance. Inside a workflow it becomes a handler method like:
#Workflow
#WorkflowRegistrationOptions(
defaultExecutionStartToCloseTimeoutSeconds = 60,
defaultTaskStartToCloseTimeoutSeconds = 10)
public interface MyWorkflow
{
#Execute(version = "1.0")
void startMyWF();
#Signal
void signal1();
}
public class MyWFImpl implements MyWorkflow
{
MyActivitiesClient client = new MyActivitiesClientImpl();
// Used to block the workflow until a signal is received.
Settable<Void> signal1Called = new Settable<Void>();
#Override
public void startMyWF(){
Promise<Integer> result = client.activity1();
// Continues when both result and signal1 are ready.
client.activity2(result, signal1);
}
#Override
public void signal1() {
//Process signal
signal1.set(null);
}
}
http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/features.workflow.html describes how to write a workflow interface.
http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/workflowimpl.html describes how to implement workflows.
http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/clients.html describes how to send signals using external client.
I encapsulated JCSMP API in a class and provide methods for other classes to send and consume messages.
I have the following method:
public void send(byte[] data, String queueName) throws JCSMPException {
// Acquire a message producer
if (producer == null) { //producer is an instance of XMLMessageProducer
producer = session.getMessageProducer(new PublishCallback());
}
Queue queue = JCSMPFactory.onlyInstance().createQueue(queueName);
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
msg.setData(data);
msg.setDeliveryMode(DeliveryMode.PERSISTENT);
logger.info("Sending to \"{}\"", queueName);
producer.send(msg, queue);
}
When I have multiple threads calling this method, once in a while, I notice the following exception:
com.solacesystems.jcsmp.ClosedFacilityException: Tried to perform operation on a closed XML message producer
I wonder whether the producer is closed after each call to send(). How should I make this method thread-safe?
Thank you.
XMLMessageProducer does not close itself after each call to send.
The first step here is to investigate why your XMLMessageProducer is closed. The easiest option to do this is to enable the Solace API logging to INFO (or even DEBUG), and edit your question to include the Solace API logs prior to the first ClosedFacilityException.
One possible reason is that your application has disconnected, but was not able to automatically reconnect to the Solace appliance/VMR.
I have a Struts2 Action class that places a JMS Fetch request for a list of Trade in a JMS Queue. This JMS Fetch message is processed by an external process and can take either a few seconds or even few minutes depending on the number of Trade files to be processed by the external task processing app.
I want to know how to handle this HTTP Request with an appropriate response. Does the client wait till the list of Trades is returned? (client (UI) has to action on it and has nothing else to do meanwhile).
The way I approached it is
HTTP Request -->
Struts2 Action -->
Invokes a Runnable to run in a separate Thread (separate from Action class)
UI waits
Action class thread sleeps till runnable does it's job
When Task completed, return list of Trades to UI
Flow is as follows:
Place JMS Fetch Request on Queue1
ExecutorService for Runnable
CClass cclass = new CClass();
final ExecutorService execSvc = Executors.newFixedThreadPool(1);
execSvc.execute(cclass);
Where CClass implements runnable returning a list of Trades:
List<Trade> tradesList = new ArrayList<Trade>();
#Override
public void run() {
while (true) {
try {
Message message = msgConsumer.receive(); // SYNCHRONOUS / NO MDB
if (message == null){
break;
}
if (message instanceof TextMessage) {
TextMessage txtMessage = (TextMessage) message;
Trade trade = TradeBuilder.buildTradeFromInputXML(txtMessage);
if (trade != null) {
tradesList.add(trade); // tradeList is a CClass class variable
}
}
} catch (JMSException e) {
logger.error("JMSException occurred ", e);
}
}
closeConnection();
}
And while this runnableis executing, I do a Thread.sleep in Action class (to let the Runnable execute in the separate Thread)
// In Action class
try {
Thread.sleep(5000); // some time till when the runnable will get executed
} catch (InterruptedException e) {
e.printStackTrace();
}
execSvc.shutdown();
Problem is If I use Callable with a FutureTask and do a get() , that will be blocking till any result is returned. If I do a Runnable, I have to put Action class Thread to sleep till runnable has executed and tradeList is available.
Using Runnable approach, I am able to get couple of hundred records back to UI giving a 5 second Thread.sleep() in main Action class, but only partially constructed tradeList when thousands of records are to be fetched and shown in UI.
This is clearly Not a fail-proof approach.
Any better approach to suggest ? Please elucidate steps for processing in one complete request - response flow.
Yes there is a much better approach when making a standard HTTP request (with ajax you can do other things).
You want to look at the Struts2 Execute and Wait Interceptor Which has most of the functionality you've already implemented. Also look at the token interceptor... which could be useful (it prevents duplicate requests, but doesn't provide a happy wait screen like exec and wait does).
After complete of asynchronous call to WCF service I want set success message into session and show user the notification .
I tried use two ways for complete this operation.
1) Event Based Model.
client.GetDataCompleted += new EventHandler<GetDataCompletedEventArgs>(GetDataCompleted);
client.GetDataAsync(id, client);
private void GetDataCompleted(object obj, GetDataCompletedEventArgs e)
{
this.SetNotification(new Notification() { Message = e.Result, Type = NotificationType.Success });
}
In MyOperationCompleted event i can set notification to HttpContext.Current.Session, but I must waiting before this operation will completed and can't navigate to others pages.
2) IAsyncResult Model.
In this way I can navigate to other pages and make asynchronous calls to wcf service, but in GetDataCallback method can't set notification, becouse session = null.
client.BeginGetData(id, GetDataCallback, client);
private void GetDataCallback(IAsyncResult ar)
{
string name = ((ServiceReference1.Service1Client)ar.AsyncState).EndGetData(ar);
this.SetNotification(new Notification() { Message = name, Type = NotificationType.Success });
}
"Generate asynchronous operations" in service reference enabled.
Please help me with this trouble. Thanks.
I'm no wcf expert, but what I've found to work is wrapping your call to the Async version of your method in ThreadPool.QueueUserWorkItem. Without this, I had same blocking issue. So this seems to free up the main thread in your asp mvc to move on while another worker thread waits for the callback.
Also, I used AsyncController, although that alone was not enough without the worker thread.
See this: http://msdn.microsoft.com/en-us/library/ee728598.aspx
I used this as a guide, but still needed the ThreadPool.
Cheers