Spring Cloud Spring Service Connector with RabbitMQ - spring-amqp

I use Spring cloud Spring service connector to connect Rabbitmq service on CloudFoundry.
public class CloudConfig extends AbstractCloudConfig {
#Bean
public ConnectionFactory rabbitFactory()
{
return connectionFactory().rabbitConnectionFactory();
}
}
But I need to declare a CachingConnectionFactory and set its PublisherConfirms true. Because we need use publisherConfirm to check ack when we send message to queue. I have no idea about how to inject the connectionFactory which is got from cloud spring service connector. Or how we could handle this situation.

The documentation includes examples of customizing details of the connection provided by Connectors.
In your case, you should be able to do something like this:
#Bean
public RabbitConnectionFactory rabbitFactory() {
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("publisherConfirms", true);
RabbitConnectionFactoryConfig rabbitConfig = new RabbitConnectionFactoryConfig(properties);
return connectionFactory().rabbitConnectionFactory(rabbitConfig);
}

You can reconfigure the CCF created by the connector as follows:
#Bean
public SmartInitializingSingleton factoryConfigurer() {
return new SmartInitializingSingleton() {
#Autowired
private CachingConnectionFactory connectionFactory;
#Override
public void afterSingletonsInstantiated() {
this.connectionFactory.setPublisherConfirms(true);
}
};
}
You must be sure not to perform any RabbitMQ operations before the application context is fully initialized (which is best practice anyway).

This is RabbitTemplate
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMandatory(true);
template.setMessageConverter(new Jackson2JsonMessageConverter());
template.setConfirmCallback((correlationData, ack, cause) -> {
if (!ack) {
System.out.println("send message failed: " + cause + correlationData.toString());
} else {
System.out.println("Publisher Confirm" + correlationData.toString());
}
});
return template;
}
This is spring-cloud config:
#Bean
public ConnectionFactory rabbitConnectionFactory() {
Map<String, Object> properties = new HashMap<String, Object>();
properties.put("publisherConfirms", true);
RabbitConnectionFactoryConfig rabbitConfig = new RabbitConnectionFactoryConfig(properties);
return connectionFactory().rabbitConnectionFactory(rabbitConfig);
}
When I use this sender to send message.The result is not expected.
#Component
public class TestSender {
#Autowired
private RabbitTemplate rabbitTemplate;
#Scheduled(cron = "0/5 * * * * ? ")
public void send() {
System.out.println("===============================================================");
this.rabbitTemplate.convertAndSend(EXCHANGE, "routingkey", "hello world",
(Message m) -> {
m.getMessageProperties().setHeader("tenant", "aaaaa");
return m;
}, new CorrelationData(UUID.randomUUID().toString()));
Date date = new Date();
System.out.println("Sender Msg Successfully - " + date);
}
}

Related

Listen to RabbitMQ Queues at same time in different Virtual Host using SpringBoot

I have multiple Clients each client has virtual host defined.
I need to provide ability to listen queues which are located in different virtual host.
SpringBoot , RabbitMQ I am using in the application.
Spring Boot can only auto configure one connection factory.
You will need to configure two or more sets of infrastructure beans (connection factory, container factory, template, etc).
You have to configure both because when Boot detects one, it disables its auto configuration.
EDIT
#SpringBootApplication(exclude = RabbitAutoConfiguration.class)
#EnableRabbit
public class So72953705Application {
public static void main(String[] args) {
SpringApplication.run(So72953705Application.class, args);
}
#Bean
ConnectionFactory fooConn() {
CachingConnectionFactory ccf = new CachingConnectionFactory("localhost", 5672);
ccf.setVirtualHost("foo");
return ccf;
}
#Bean
RabbitTemplate fooTemplate(ConnectionFactory fooConn) {
return new RabbitTemplate(fooConn);
}
#Bean
RabbitAdmin fooAdmin(ConnectionFactory fooConn) {
return new RabbitAdmin(fooConn);
}
#Bean
SimpleRabbitListenerContainerFactory fooContFactory(ConnectionFactory fooConn) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(fooConn);
return factory;
}
#Bean
ConnectionFactory barConn() {
CachingConnectionFactory ccf = new CachingConnectionFactory("localhost", 5672);
ccf.setVirtualHost("bar");
return ccf;
}
#Bean
RabbitTemplate barTemplate(ConnectionFactory barConn) {
return new RabbitTemplate(barConn);
}
#Bean
RabbitAdmin barAdmin(ConnectionFactory barConn) {
return new RabbitAdmin(barConn);
}
#Bean
SimpleRabbitListenerContainerFactory barContFactory(ConnectionFactory barConn) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(barConn);
return factory;
}
}
#Component
class FooListener {
#RabbitListener(queues = "fooQ", containerFactory = "fooContFactory")
public void listen(String in) {
System.out.println(in);
}
}
#Component
class BarListener {
#RabbitListener(queues = "barQ", containerFactory = "barContFactory")
public void listen(String in) {
System.out.println(in);
}
}
EDIT2
Dynamic registration of infrastructure beans.
#SpringBootApplication(exclude = RabbitAutoConfiguration.class)
#EnableRabbit
public class So72953705Application {
public static void main(String[] args) {
SpringApplication.run(So72953705Application.class, args);
}
#Bean
ApplicationRunner runner(DynamicRegistrar registrar) {
return args -> {
registrar.registerInfrastructure("foo");
registrar.registerInfrastructure("bar");
};
}
}
#Component
class DynamicRegistrar {
private final ConfigurableListableBeanFactory beanFactory;
DynamicRegistrar(ConfigurableListableBeanFactory beanFactory) {
this.beanFactory = beanFactory;
}
void registerInfrastructure(String virtualHost) {
CachingConnectionFactory ccf = new CachingConnectionFactory("localhost", 5672);
ccf.setVirtualHost(virtualHost);
this.beanFactory.registerSingleton(virtualHost + ".cf", ccf);
this.beanFactory.initializeBean(ccf, virtualHost + ".cf");
RabbitTemplate template = new RabbitTemplate(ccf);
this.beanFactory.registerSingleton(virtualHost + ".template", template);
this.beanFactory.initializeBean(template, virtualHost + ".template");
RabbitAdmin admin = new RabbitAdmin(ccf);
this.beanFactory.registerSingleton(virtualHost + ".admin", admin);
this.beanFactory.initializeBean(admin, virtualHost + ".admin");
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(ccf);
container.setQueueNames("same.queue.name");
container.setMessageListener(msg -> {
System.out.println(msg + " from VH: " + msg.getMessageProperties().getHeader("fromVirtualHost"));
});
container.setAfterReceivePostProcessors(msg -> {
msg.getMessageProperties().setHeader("fromVirtualHost", virtualHost);
return msg;
});
this.beanFactory.registerSingleton(virtualHost + ".container", container);
this.beanFactory.initializeBean(container, virtualHost + ".container");
container.start();
}
}

How to create rabbit listeners using spring AMQP to listen to queues across multiple vhosts

I have multiple virtual hosts each with a request queue and a response queue. These virtual hosts serve different clients. The names for the request queue and the response queue remain the same across the virtual hosts.
I have created a SimpleRoutingConnectionFactory with the clientName()+"ConnectionFactory" as the lookup key and a corresponding CachingConnectionFactory as the value in the map. I'm able to publish message to the request queues by binding and the RabbitTemplate to a virtual host before convertAndSend and then unbinding it.
I'm not able to consume messages from the response queues from different virtual hosts. I have created a SimpleRabbitListenerContainerFactory for each client. I implemented RabbitListenerConfigurer and registered a SimpleRabbitListenerEndpoint for each SimpleRabbitListenerContainerFactory. I also set the connectionFactory on each SimpleRabbitListenerContainerFactory as the client's CachingConnectionFactory.
#Configuration
public class RabbitConfiguration implements RabbitListenerConfigurer {
#Autowired
private ApplicationContext applicationContext;
#Autowired
private ClientList clients;
#Bean
#Primary
public SimpleRoutingConnectionFactory routingConnectionFactory() {
final var routingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> routeMap = new HashMap<>();
applicationContext.getBeansOfType(ConnectionFactory.class)
.forEach((beanName, bean) -> {
routeMap.put(beanName, bean);
});
routingConnectionFactory.setTargetConnectionFactories(routeMap);
return routingConnectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(routingConnectionFactory());
}
#Bean
public DirectExchange orbitExchange() {
return new DirectExchange("orbit-exchange");
}
#Bean
public Queue requestQueue() {
return QueueBuilder
.durable("request-queue")
.lazy()
.build();
}
#Bean
public Queue responseQueue() {
return QueueBuilder
.durable("response-queue")
.lazy()
.build();
}
#Bean
public Binding requestBinding() {
return BindingBuilder.bind(requestQueue())
.to(orbitExchange())
.with("orbit-request");
}
#Bean
public Binding responseBinding() {
return BindingBuilder.bind(responseQueue())
.to(orbitExchange())
.with("orbit-response");
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
clients.get()
.stream()
.forEach(client -> {
var endpoint = createEndpoint(client);
var listenerContainerFactory = applicationContext.getBean(client.getName() + "ListenerContainerFactory");
listenerContainerFactory.setConnectionFactory((ConnectionFactory)applicationContext.getBean(client.getName() + "ConnectionFactory"));
registrar.registerEndpoint(endpoint, listenerContainerFactory);
});
}
}
private SimpleRabbitListenerEndpoint createEndpoint(Client client) {
var endpoint = new SimpleRabbitListenerEndpoint();
endpoint.setId(client.getName());
endpoint.setQueueNames("response-queue");
endpoint.setMessageListener(new MessageListenerAdapter(new MessageReceiver(), "receive"));
return endpoint;
}
}
However, I get org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer: Failed to check/redeclare auto-delete queue(s). java.lang.IllegalStateException: Cannot determine target ConnectionFactory for lookup key [null]
I'm not able to figure out whats causing this as I'm not using the SimpleRoutingConnectionFactory for message consumption at all.
EDIT:
Full stack trace below -
ERROR [2020-07-09T04:12:38,028] [amdoListenerEndpoint-1] [TraceId:] org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer: Failed to check/redeclare auto-delete queue(s).
java.lang.IllegalStateException: Cannot determine target ConnectionFactory for lookup key [null]
at org.springframework.amqp.rabbit.connection.AbstractRoutingConnectionFactory.determineTargetConnectionFactory(AbstractRoutingConnectionFactory.java:120)
at org.springframework.amqp.rabbit.connection.AbstractRoutingConnectionFactory.createConnection(AbstractRoutingConnectionFactory.java:98)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:214)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2089)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2062)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2042)
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueInfo(RabbitAdmin.java:407)
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:391)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.attemptDeclarations(AbstractMessageListenerContainer.java:1836)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.redeclareElementsIfNecessary(AbstractMessageListenerContainer.java:1817)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.initialize(SimpleMessageListenerContainer.java:1349)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1195)
at java.base/java.lang.Thread.run(Thread.java:834)
EDIT2:
I used the routingConnectionFactory with every listener and used the setLookUpKeyQualifier. No more exceptions but, the listeners don't seem to be doing anything i.e., the queues are not being listened to.
#Import(MqConfig.class)
//This is to import CachingConnectinFactory beans and SimpleRabbitListenerContainerFactory beans for all clients
#Configuration
public class RabbitConfiguration implements RabbitListenerConfigurer {
#Autowired
private ApplicationContext applicationContext;
#Autowired
private ClientList clients;
#Bean
#Primary
public SimpleRoutingConnectionFactory routingConnectionFactory() {
final var routingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> routeMap = new HashMap<>();
applicationContext.getBeansOfType(ConnectionFactory.class)
.forEach((beanName, bean) -> {
routeMap.put(beanName+"[response-queue]", bean);
});
routingConnectionFactory.setTargetConnectionFactories(routeMap);
return routingConnectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
return new RabbitTemplate(routingConnectionFactory());
}
#Bean
public DirectExchange orbitExchange() {
return new DirectExchange("orbit-exchange");
}
#Bean
public Queue requestQueue() {
return QueueBuilder
.durable("request-queue")
.lazy()
.build();
}
#Bean
public Queue responseQueue() {
return QueueBuilder
.durable("response-queue")
.lazy()
.build();
}
#Bean
public Binding requestBinding() {
return BindingBuilder.bind(requestQueue())
.to(orbitExchange())
.with("orbit-request");
}
#Bean
public Binding responseBinding() {
return BindingBuilder.bind(responseQueue())
.to(orbitExchange())
.with("orbit-response");
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
clients.get()
.stream()
.forEach(client -> {
var endpoint = createEndpoint(client);
var listenerContainerFactory = getListenerContainerFactory(Client client);
listenerContainerFactory.setConnectionFactory((ConnectionFactory)applicationContext.getBean(client.getName() + "ConnectionFactory"));
registrar.registerEndpoint(endpoint, listenerContainerFactory);
});
}
}
private SimpleRabbitListenerEndpoint createEndpoint(Client client) {
var endpoint = new SimpleRabbitListenerEndpoint();
endpoint.setId(client.getName());
endpoint.setQueueNames("response-queue");
endpoint.setMessageListener(new MessageListenerAdapter(new MessageReceiver(), "receive"));
return endpoint;
}
private SimpleRabbitListenerContainerFactory getListenerContainerFactory(Client client) {
var listenerContainerFactory = (SimpleRabbitListenerContainerFactory) applicationContext.getBean(client.getName() + "ListenerContainerFactory");
listenerContainerFactory.setConnectionFactory(routingConnectionFactory());
listenerContainerFactory.setContainerCustomizer(container -> {
container.setQueueNames("response-queue");
container.setLookupKeyQualifier(client.getName());
container.setMessageListener(message -> log.info("Received message"));
});
return listenerContainerFactory;
}
}
There is something very strange going on; [null] implies that when we call getRoutingLookupKey() the cf is not a routing cf but when we call getConnectionFactory() it is.
It's not obvious how that can happen. Perhaps you can figure out why in a debugger?
One solution would be to inject the routing cf and use setLookupKeyQualifier(...).
The lookup key will then be clientId[queueName].

RabbitListenerConfigureIntegrationTests Example

I am looking for some integration test examples for RabbitListenerConfigurer and RabbitListenerEndpointRegistrar and calling #rabbitListner annotation and test the message conversion and pass additional paramenters such as Channel and message properties etc.
Some thing like this
#RunWith(SpringJUnit4ClassRunner.class)
public class RabbitListenerConfigureIntegrationTests {
public final String sampleMessage="{\"ORCH_KEY\":{\"inputMap\":{},\"outputMap\":{\"activityId\":\"10001002\",\"activityStatus\":\"SUCCESS\"}}}";
#Test
public void testRabiitListenerConfigurer() throws Exception {
AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(
EnableRabbitConfigWithCustomConversion.class);
RabbitListenerConfigurer registrar = ctx.getBean(RabbitListenerConfigurer.class);
/* I want to get the Listener instance here */
Message message = MessageBuilder.withBody(sampleMessage.getBytes())
.andProperties(MessagePropertiesBuilder.newInstance()
.setContentType("application/json")
.build())
.build();
/* call listener.onmessage(message) and that intern pass the call back to #rabbit listener and by that time MessageHandler which is registered should kick off and convert the message */
}
#Configuration
#EnableRabbit
public static class EnableRabbitConfigWithCustomConversion implements RabbitListenerConfigurer {
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
#Bean
public ConnectionFactory mockConnectionFactory() {
return mock(ConnectionFactory.class);
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(mockConnectionFactory());
factory.setAutoStartup(false);
return factory;
}
#Bean
MessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory messageHandlerMethodFactory = new DefaultMessageHandlerMethodFactory();
messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter());
return messageHandlerMethodFactory;
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public Listener messageListener1() {
return new Listener();
}
}
public class Listener {
#RabbitListener(queues = "QUEUE")
public void listen(ExchangeDTO dto, Channel chanel) {
System.out.println("Result:" + dto.getClass() + ":" + dto.toString());
/*ExchangeDTO dto = (ExchangeDTO)messageConverter.fromMessage(message);
System.out.println("dto:"+dto);*/
}
}
EDIT 2
I am not getting Exchange DTO populated with values. instead I get Null values
Here is Log :
15:00:50.994 [main] DEBUG org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter - Processing [GenericMessage [payload=byte[93], headers={contentType=application/json, id=8bf86bf1-7e45-d136-9126-69959f92f100, timestamp=1552680050993}]]
Result:class com.dsicover.dftp.scrubber.subscriber.ExchangeDTO:DTO [inputMap={}, outputMap={}]
public class ExchangeDTO implements Serializable {
/**
*
*/
private static final long serialVersionUID = 1L;
private HashMap<String, Object> inputMap = new HashMap<String, Object>();
private HashMap<String, Object> outputMap = new HashMap<String, Object>();
public HashMap<String, Object> getInputMap() {
return inputMap;
}
public void setInputMap(HashMap<String, Object> inputMap) {
this.inputMap = inputMap;
}
public HashMap<String, Object> getOutputMap() {
return outputMap;
}
public void setOutputMap(HashMap<String, Object> outputMap) {
this.outputMap = outputMap;
}
#Override
public String toString() {
return "DTO [inputMap=" + this.inputMap + ", outputMap=" + this.outputMap + "]";
}
}
Is there any thing i am missing in Jackson2MessageConverter.
Give the #RabbitListener an id
RabbitListenerEndpointRegistry.getListenerContainer(id);
cast container to AbstractMessageListenerContainer
container.getMessageListener()
cast listener to ChannelAwareMessageListener
call onMessage().
use a mock channel and verify expected call
EDIT
#Autowired
private RabbitListenerEndpointRegistry registry;
#Test
public void test() throws Exception {
AbstractMessageListenerContainer listenerContainer =
(AbstractMessageListenerContainer) this.registry.getListenerContainer("foo");
ChannelAwareMessageListener listener =
(ChannelAwareMessageListener) listenerContainer.getMessageListener();
Channel channel = mock(Channel.class);
listener.onMessage(new Message("foo".getBytes(),
MessagePropertiesBuilder
.newInstance()
.setDeliveryTag(42L)
.build()), channel);
verify(channel).basicAck(42L, false);
}
EDIT2
Your json does not look like a DTO, it looks like a Map<String, DTO>.
This works fine for me...
#SpringBootApplication
public class So55188061Application {
public static void main(String[] args) {
SpringApplication.run(So55188061Application.class, args);
}
#RabbitListener(id = "foo", queues = "foo")
public void listen(Map<String, Foo> in, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag) throws IOException {
System.out.println(in);
channel.basicAck(tag, false);
}
#Bean
public MessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
public static class Foo {
private HashMap<String, Object> inputMap = new HashMap<String, Object>();
private HashMap<String, Object> outputMap = new HashMap<String, Object>();
public HashMap<String, Object> getInputMap() {
return this.inputMap;
}
public void setInputMap(HashMap<String, Object> inputMap) {
this.inputMap = inputMap;
}
public HashMap<String, Object> getOutputMap() {
return this.outputMap;
}
public void setOutputMap(HashMap<String, Object> outputMap) {
this.outputMap = outputMap;
}
#Override
public String toString() {
return "Foo [inputMap=" + this.inputMap + ", outputMap=" + this.outputMap + "]";
}
}
}
and
#RunWith(SpringRunner.class)
#SpringBootTest
public class So55188061ApplicationTests {
public final String sampleMessage =
"{\"ORCH_KEY\":{\"inputMap\":{},"
+ "\"outputMap\":{\"activityId\":\"10001002\",\"activityStatus\":\"SUCCESS\"}}}";
#Autowired
private RabbitListenerEndpointRegistry registry;
#Test
public void test() throws Exception {
AbstractMessageListenerContainer listenerContainer = (AbstractMessageListenerContainer) this.registry
.getListenerContainer("foo");
ChannelAwareMessageListener listener = (ChannelAwareMessageListener) listenerContainer.getMessageListener();
Channel channel = mock(Channel.class);
listener.onMessage(MessageBuilder.withBody(sampleMessage.getBytes())
.andProperties(MessagePropertiesBuilder.newInstance()
.setContentType("application/json")
.setDeliveryTag(42L)
.build())
.build(),
channel);
verify(channel).basicAck(42L, false);
}
}
and
{ORCH_KEY=Foo [inputMap={}, outputMap={activityId=10001002, activityStatus=SUCCESS}]}
According to your complex requirement to have everything on board, I don't see how we can make a deal with the mock(ConnectionFactory.class). We would need to mock much more to have everything working.
Instead, I would suggest to take a look into the real integration test against existing RabbitMQ or at least embedded QPid.
In addition you may consider to use a #RabbitListenerTest to spy your #RabbitListener invocation without interfering your production code.
More info is in the Reference Manual: https://docs.spring.io/spring-amqp/docs/2.1.4.RELEASE/reference/#test-harness

Handling Connections in Spring-Boot-RabbitMQ

Hi I am developing Spring-boot-RabbitMQ version 1.6.I am having few queries while developing the application. Read the docs and browsed other stack overflow question but i cannot get few things clear(Might be because of my bad memory).
It would be great if some one answers my questions.
1) Currently i am having 4-Producers and 4-Consumers.Producer may produce millions of messages or events so using a single connection for both producer & consumer will block consumer to consume the messages.So what i would thought is creating separate connections for producer and consumer so that both will not block and will give some performance improvement.Am i correct with this approach?
2) I am using CachingConnectionFactory in order to create connection using SimpleRabbitListenerContainerFactory.While making call to this factory whether it will return new connection for us?So if we use CachingConnectionFactory do we really need to write a separate connection factories for both Producer & consumer.Please find my below
1)Configuration class
#Configuration
#EnableRabbit
public class RabbitMqConfiguration{
#Autowired
private CachingConnectionFactory cachingConnectionFactory;
#Value("${concurrent.consumers}")
public int concurrent_consumers;
#Value("${max.concurrent.consumers}")
public int max_concurrent_consumers;
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(cachingConnectionFactory);
factory.setConcurrentConsumers(concurrent_consumers);
factory.setMaxConcurrentConsumers(max_concurrent_consumers);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public MessageConverter jsonMessageConverter()
{
final Jackson2JsonMessageConverter converter = new Jackson2JsonMessageConverter();
return converter;
}
}
2)Producer Class
#Configuration
public class TaskProducerConfiguration extends RabbitMqConfiguration {
#Value("${queue1}")
public String queue1;
#Value("${queue2}")
public String queue2;
#Value("${queue3}")
public String queue1;
#Value("${queue4}")
public String queue2;
#Value("${spring.rabbit.exchange}")
public String exchange;
#Autowired
private CachingConnectionFactory cachingConnectionFactory;
#Primary
#Bean
public RabbitTemplate getQueue1Template()
{
RabbitTemplate template = new RabbitTemplate(cachingConnectionFactory);
template.setRoutingKey(this.queue1);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public RabbitTemplate getQueue2Template()
{
RabbitTemplate template = new RabbitTemplate(cachingConnectionFactory);
template.setRoutingKey(this.queue2);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public RabbitTemplate getQueue3Template()
{
RabbitTemplate template = new RabbitTemplate(cachingConnectionFactory);
template.setRoutingKey(this.queue3);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean
public RabbitTemplate getQueue4Template()
{
RabbitTemplate template = new RabbitTemplate(cachingConnectionFactory);
template.setRoutingKey(this.queue4);
template.setMessageConverter(jsonMessageConverter());
return template;
}
#Bean(name="queue1Bean")
public Queue queue1()
{
return new Queue(this.queue1);
}
#Bean(name="queue2Bean")
public Queue queue2()
{
return new Queue(this.queue2);
}
#Bean(name="queue3Bean")
public Queue queue3()
{
return new Queue(this.queue3);
}
#Bean(name="queue4Bean")
public Queue queue4()
{
return new Queue(this.queue4);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(exchange);
}
#Bean
List<Binding> bindings(Queue queue1Bean,Queue queue2Bean,Queue queue3Bean,Queue queue4Bean, TopicExchange exchange) {
List<Binding> bindingList = new ArrayList<Binding>();
bindingList.add(BindingBuilder.bind(queue1Bean).to(exchange).with(this.queue1));
bindingList.add(BindingBuilder.bind(queue2Bean).to(exchange).with(this.queue2));
bindingList.add(BindingBuilder.bind(queue3Bean).to(exchange).with(this.queue3));
bindingList.add(BindingBuilder.bind(queue4Bean).to(exchange).with(this.queue4));
return bindingList;
}
}
3) Receiver Class(Just Shared one receiver class rest of the 3-receiver classes are one and the same except queue name & routing key).
#Component
public class Queue1Receiver {
#Autowired
private TaskProducer taskProducer;
#Value("${queue1}")
public String queue1;
#RabbitListener(id="queue1",containerFactory="rabbitListenerContainerFactory",queues = "#{queue1Bean}")
public void handleQueue1Message(TaskMessage taskMessage,#Header(AmqpHeaders.CONSUMER_QUEUE) String queue)
{
System.out.println("Queue::"+queue);
System.out.println("CustomerId: " + taskMessage.getCustomerID());
if(taskMessage.isHasQueue2()){
taskProducer.sendQueue2Message(taskMessage);
}
if(taskMessage.isHasQueue3()){
taskProducer.sendQueue3Message(taskMessage);
}
if(taskMessage.isHasQueue4()){
taskProducer.sendQueue4Message(taskMessage);
}
}
#Bean
public Queue queue1Bean() {
// This queue has the following properties:
// name: my_durable,durable: true,exclusive: false,auto_delete: false
return new Queue(queue1, true, false, false);
}
}
Your help should be appreciable.
Note : Down Voters please register your comment before down voting so that in future i can avoid the mistake.
Edited based on comments by Gary Russell:
1)RabbitMqConfiguration
#Configuration
#EnableRabbit
public class RabbitMqConfiguration{
#Value("${concurrent.consumers}")
public int concurrent_consumers;
#Value("${max.concurrent.consumers}")
public int max_concurrent_consumers;
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(concurrent_consumers);
factory.setMaxConcurrentConsumers(max_concurrent_consumers);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public CachingConnectionFactory connectionFactory()
{
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
connectionFactory.setCacheMode(CacheMode.CONNECTION);
return connectionFactory;
}
#Bean
public MessageConverter jsonMessageConverter()
{
final Jackson2JsonMessageConverter converter = new Jackson2JsonMessageConverter();
return converter;
}
}
using a single connection for both producer & consumer will block consumer to consume the messages`
What leads you to believe that? A single connection will generally be fine. If you really want separate connections, change the connection factory cacheMode to CONNECTION.
You can use connection pooling in the same case keeping the pool size appropriate may solve the problem.As suggested in the above answer both producer and consumer are using the same connection so pooling might help you out instead.

Two way dead lettering between work queue and retry queue

I am trying to implement dead lettering in recursive way. I have two queues, a main queue and a retry queue for it.
The messages will first be put in the main queue. When there is a exception during consumer process, it throws AmqpRejectAndDontRequeueException so that the message gets removed from queue and added to the retry queue. It sits there for 5 secs, and then get added to tail of the work queue again.
when the consumer throws 'AmqpRejectAndDontRequeueException', it does not get delivered to the retry queue
Could you help me how i can resolve this?
Below is the configuration, producer and consumer files.
producer config:
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory()
connectionFactory.setUsername(userName)
connectionFactory.setPassword(password)
connectionFactory.setAddresses(hosts)
return connectionFactory
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory())
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory())
template.setMessageConverter(jsonMessageConverter());
return template
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter()
}
#Bean
public DirectExchange searchNotificationExchange() {
return new DirectExchange("search.notification.exchange", false, true)
}
#Bean
public Queue searchNotificationQueue() {
Map<String, Object> args = new HashMap<String, Object>()
args.put("x-dead-letter-exchange", "search.notification.retry.exchange")
return new Queue(SEARCH_NOTIFICATION_QUEUE_NAME, true, false, false, args)
}
#Bean
public Binding binding() {
return BindingBuilder.bind(searchNotificationQueue()).to(searchNotificationExchange()).with(SEARCH_NOTIFICATION_QUEUE_NAME);
}
#Bean
public DirectExchange searchNotificationRetryExchange() {
return new DirectExchange("search.notification.retry.exchange", false, true)
}
#Bean
public Queue searchNotificationRetryQueue() {
Map<String, Object> args = new HashMap<String, Object>()
args.put("x-dead-letter-exchange", "search.notification.exchange")
args.put("x-message-ttl",5000)
return new Queue(SEARCH_NOTIFICATION_RETRY_QUEUE_NAME, true, false, false, args)
}
#Bean
public Binding retryBinding() {
return BindingBuilder.bind(searchNotificationRetryQueue()).to(searchNotificationRetryExchange()).with(SEARCH_NOTIFICATION_RETRY_QUEUE_NAME);
}
Consumer config that extend from producer config
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory()
factory.setConnectionFactory(connectionFactory())
factory.setConcurrentConsumers(3)
factory.setMaxConcurrentConsumers(10)
factory.setMessageConverter(jsonMessageConverter())
return factory
}
producer:
try {
rabbitTemplate.convertAndSend(RabbitConfiguration.SEARCH_NOTIFICATION_QUEUE_NAME, sd);
println sd.deliveryId
} catch(AmqpException amqbe) {
log.error("Error putting message into search notification queue.", amqbe)
}
Consumer:
#RabbitListener(queues =RabbitConfiguration.SEARCH_NOTIFICATION_QUEUE_NAME)
public void handleMessage(SearchNotificationMQMessage messageBody, Message message) {
try {
//process message
} catch(Throwable t) {
if(message.messageProperties.deliveryTag < 3) {
throw new AmqpRejectAndDontRequeueException()
}
}

Resources