Spring AMQP how to access response message when connection is lost while sending using #SendTo - spring-amqp

we have a microservice which consumes a message using #RabbitListener and persist data into database, generate a response on successful processing of message and send it using #sendTO to different queue for auditing.
When running Rabbit in HA failover, while sending response if connection is lost the message currently being processed is correctly returned to the queue but database transaction (jpa transaction in our case) is not rolled back , response is never sent.
I read from this issue(https://github.com/spring-projects/spring-amqp/issues/696) that this is "best effort 1PC" transaction synchronization; RabbitMQ does not support XA transactions. The Rabbit tx is committed after the DB tx and there is a possibility the DB tx might commit and the rabbit rolled back; you have to deal with the small possibility of duplicate messages.
But in our case when we retry request, we are treating it as duplicate message and response is never created for this request. is there a way where we can only retry sending response message in case of connection lost exceptions rather than reprocessing request again? I looked at ConditionalRejectingErrorHandler.DefaultExceptionStrategy, it has access only to original request,no way to access response lost during connection failure. Please suggest what's the best way to handle this?
our code looks like:
SpringBootApplication
#EnableJpaRepositories("com.***")
#EnableJpaAuditing
#EnableTransactionManagement
#EnableEncryptableProperties
public class PcaClinicalValidationApplication {
#RabbitListener(queues = "myqueue"
#SendTo("exchange/routingKey")
#Timed) description = "Time taken to process a request")
public Message receivemessage(HashMap<String, Object> myMap, Message requestMessage)
throws Exception {
//business logic goes here
Message message = MessageBuilder.fromMessage(requestMessage)
//add some headers
return message;
}
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setRetryTemplate(new RetryTemplate());
factory.setReplyRecoveryCallback(ctx -> {
Message failed = SendRetryContextAccessor.getMessage(ctx);
Address replyTo = SendRetryContextAccessor.getAddress(ctx);
Throwable t = ctx.getLastThrowable();
//wrote to a file
serializer.serialize(failed);
return null;
});
return factory;
}

The listener container factory uses a RabbitTemplate in its replyTemplate property - this is used to send the reply.
You can configure a RetryTemplate into that RabbitTemplate to retry sending the reply.
When retries are exhausted, you can add a RecoveryCallback which will get the failed reply and you can save it off someplace and use it when the redelivery occurs.

Related

How to wait in integration test for some operations

I have integration test with docker using test containers. On container I run jms. In test I am putting message on queue.
How I can wait in test to make it populated on jms?
On local machine it works, but on jenkins it fails, so I have to add
Thread.sleep(3000);
but this is nasty.
org.awaitility seems to be missed usage:
await().atMost(2, TimeUnit.SECONDS).until(() -> return true));
I just need to do a pause to make jms propagate (put on jms queue) and wait for listener to act, which is putting message to database.
Then I have to call get rest endpoint to see it worked.
With topic it would be easier, because I would create test listener on topic.
But it is queue, there can be on listener that will get message.
Use org.awaitility with a JMS QueueBrowser, e.g.:
#Test
public void myTest() throws Exception {
...
await().atMost(2, TimeUnit.SECONDS).until(() -> return queueIsEmpty(queueName)));
...
}
private boolean queueIsEmpty(String queueName) {
ConnectionFactory cf = new MyBrokersConnectionFactory();
Connection connection = cf.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
QueueBrowser browser = session.createBrowser(session.createQueue(queueName));
Enumeration enumeration = senderBrowser.getEnumeration();
while (enumeration.hasMoreElements()) {
return false;
}
return true;
}
A QueueBrowser is read only so there is no danger that it will actually consume the message.
Another potential option would be to create a consumer with a transacted session and then try to receive the message. If you actually did receive a message you could rollback the transaction and close the consumer.
Use retries (e.g. Spring RetryTemplate or Failsafe Retry Policy) to improve integration test execution time:
Retry the SQL query until record is present
Retry the REST endpoint until it is successful
Here an example to wait for a DB record; tweak the policies to your needs:
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setBackOffPolicy(new FixedBackOffPolicy());
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(
10, Collections.singletonMap(AssertionError.class, true)));
retryTemplate.execute(retryContext -> {
List<MyRecord> records = jdbcTemplate.query("select ...");
Assert.assertEquals(1, records.size());
return null;
});
My solution is to use org.awaitility lib and replace asserts with return statement:
await().atMost(30, TimeUnit.SECONDS).until(
() -> {
//
// assertTrue(condition);
return condition == true;
}

Fetch message details in Spring RecoveryCallback

I'm publishing messages into RabbitMQ and I would like to track the errors when RabbitMQ is down, for this I added one RetryTemplate with the recovery callback, but the recovery callback only provides this method getLastThrowable() and I'm not sure how to provide the details of the messages that failed when RabbitMQ is down. (as per documentation "The RecoveryCallback is somewhat limited in that the retry context only contains the
lastThrowable field. For more sophisticated use cases, you should use an external
RetryTemplate so that you can convey additional information to the RecoveryCallback via
the context’s attributes") but I don't know how to do that, if anyone could help me with one example that will be awesome.
Rabbit Template
public RabbitTemplate rabbitMqTemplate(RecoveryCallback publisherRecoveryCallback) {
RabbitTemplate r = new RabbitTemplate(rabbitConnectionFactory);
r.setExchange(exchangeName);
r.setRoutingKey(routingKey);
r.setConnectionFactory(rabbitConnectionFactory);
r.setMessageConverter(jsonMessageConverter());
RetryTemplate retryTemplate = new RetryTemplate();
ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
backOffPolicy.setInitialInterval(500);
backOffPolicy.setMultiplier(10.0);
backOffPolicy.setMaxInterval(10000);
retryTemplate.setBackOffPolicy(backOffPolicy);
r.setRetryTemplate(retryTemplate);
r.setRecoveryCallback(publisherRecoveryCallback);
return r;
}
Recovery Callback
#Component
public class PublisherRecoveryCallback implements RecoveryCallback<AssortmentEvent> {
#Override
public AssortmentEvent recover(RetryContext context) throws Exception {
log.error("Error publising event",context.getLastThrowable());
//how to get message details here??
return null;
}
}
AMQP Outbound Adapter
return IntegrationFlows.from("eventsChannel")
.split()
.handle(Amqp.outboundAdapter(rabbitMqTemplate)
.exchangeName(exchangeName)
.confirmCorrelationExpression("payload")
.confirmAckChannel(ackChannel)
.confirmNackChannel(nackChannel)
)
.get();
The isn't possible because the function RabbitTemplate.execute() is already not aware about message you send, because it may be performed from any other method, where we might not have messages to deal:
return this.retryTemplate.execute(
(RetryCallback<T, Exception>) context -> RabbitTemplate.this.doExecute(action, connectionFactory),
(RecoveryCallback<T>) this.recoveryCallback);
What I suggest you to do is like storing message to the ThreadLocal before send and get it from there from your custom RecoveryCallback.

Why XMLMessageProducer is closed when called by multiple threads?

I encapsulated JCSMP API in a class and provide methods for other classes to send and consume messages.
I have the following method:
public void send(byte[] data, String queueName) throws JCSMPException {
// Acquire a message producer
if (producer == null) { //producer is an instance of XMLMessageProducer
producer = session.getMessageProducer(new PublishCallback());
}
Queue queue = JCSMPFactory.onlyInstance().createQueue(queueName);
BytesMessage msg = JCSMPFactory.onlyInstance().createMessage(BytesMessage.class);
msg.setData(data);
msg.setDeliveryMode(DeliveryMode.PERSISTENT);
logger.info("Sending to \"{}\"", queueName);
producer.send(msg, queue);
}
When I have multiple threads calling this method, once in a while, I notice the following exception:
com.solacesystems.jcsmp.ClosedFacilityException: Tried to perform operation on a closed XML message producer
I wonder whether the producer is closed after each call to send(). How should I make this method thread-safe?
Thank you.
XMLMessageProducer does not close itself after each call to send.
The first step here is to investigate why your XMLMessageProducer is closed. The easiest option to do this is to enable the Solace API logging to INFO (or even DEBUG), and edit your question to include the Solace API logs prior to the first ClosedFacilityException.
One possible reason is that your application has disconnected, but was not able to automatically reconnect to the Solace appliance/VMR.

How to auto update/refresh in grails view/site?

I am making an xmpp webchat application in Grails. I have a message listener that can log the recieved messages in console. But how can I do this dinamycally in a website ? I am new to webapp development so please dont down vote. I am guessing Ajax but not sure.
My listener:
private MessageListener messageListener = new MessageListener() {
#Override
public void processMessage(Chat chat, Message message) {
// 'from' and 'to' fields contains senders ids, e.g.
// 17792-1028#chat.quickblox.com/mac-167
// 17744-1028#chat.quickblox.com/Smack
String from = message.getFrom().split("#")[0];
String to = message.getTo().split("#")[0];
org.jivesoftware.smack.packet.Message.Type type = message.getType();
String tajp = type.toString();
println String.format(">>> Message received (from=%s, to=%s, type=%s): %s", from, to, tajp, message.getBody())
if (onMessageReceivedListener != null) {
onMessageReceivedListener.onMessageReceived(message);
}
}
}
I suggest you to take a look on the Events Push plugin. With that you can propagate your messages and notify your clients.
You will need:
Service that notify a new message
Controller method to call this service
Ajax request to the controller, sending the user message
JavaScript to handle incoming messages
I am new to webapp development
Web development is pretty different from desktop, so I suggest you to slow down to something more easy. There's other topics on StackOverflow that will introduce you to Grails.
Also, for webdev, it's essential to know about JavaScript, CSS and HTML for your front end.

TCPClient in asp.net mvc application

I have asp.net mvc app that shows varios events. All events stored in a database. But now, I have to load data from the database and remote program. This program have external service (this is simple program that listening specific TCP port and recieve a query and send xml back).
And, I wrote simple page for test that connects to external program. The code got from MSDN:
static string Connect(String server, String message)
{
try
{
// Create a TcpClient.
Int32 port = 9197;
TcpClient client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
stream.Close();
client.Close();
return responseData;
}
catch (ArgumentNullException e)
{
//
}
catch (SocketException e)
{
//
}
}
This is my action:
public ActionResult GetData()
{
string query = "some query";
var response = Connect("192.168.0.1", query);
var model = ParseResponse(response);
return View(model);
}
I think this solution will reduce the perfomance.
What is best practicies to use TCPClient in ASP.NET MVC 3 app?
What you think about my code?
Any suggestions are welcome.
I think this solution will reduce the perfomance.
Well. All/most database operations are done over sockets. And you do not notice that, do you?
The most likely performance issues are:
Your server
Server location
Connection setup
The only thing I would do now is to build in checks in the client to monitor the response time and write to a log (or send an email) when the response times are too high.
Don't try to optimize performance until that happen.
Solutions for the above mentioned issues:
Refactor and optimize
Either put the server on the same lan or create a cache proxy server.
Use connection pooling instead of disconnecting the connections every time.
I think this solution will reduce the perfomance.
It's as any other remote request that your server does - an I/O intensive operation. So you could use an asynchronous controller and the asynchronous versions of the TcpClient methods. This way you won't be jeopardizing any worker threads on your server during the execution of the remote request.

Resources