Following is the MQTT configuration to listening event.
For high message load around 100 message per second I noticed messages not received realtime on handler.
public class VehicleEventMqttConfig {
#Value("${mqtt.auto-startup.vehicleEvent:false}")
private boolean autoStartup;
#Value("${mqtt.completion-timeout.vehicleEvent:30000}")
private int completionTimeout;
#Bean
public MessageChannel vehicleMqttInputChannel() {
return new DirectChannel();
}
#Bean
public MessageProducer inboundVehicleEvent(
final MqttPahoClientFactory mqttPahoClientFactory,
final MqttAdapters adapters,
#Value("${mqtt.topic.vehicleEvent}") final String topic) {
log.info("Register vehicleEvent mqtt");
if (StringUtils.isEmpty(topic)) {
log.warn("vehicleEvent disabled!");
return null;
}
final MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter(
getClientIdWithHost("inboundVehicleEvent"), mqttPahoClientFactory, topic);
adapter.setCompletionTimeout(completionTimeout);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setOutputChannel(vehicleMqttInputChannel());
adapter.setAutoStartup(autoStartup);
adapter.setQos(1);
adapters.add(adapter);
return adapter;
}
#Bean
#ServiceActivator(inputChannel = "vehicleMqttInputChannel")
public MessageHandler vehicleEventHandler() {
return new VehicleEventMessageHandler();
}
}
Related
I want to implement a MQTT client which can auto-reply when it get a message.
for example:
client1 send a "request" message to client2 (by topic "/toClient2").
client2 get the "request" message and reply a "response" message to client1 (by topic "/toClient1").
I use org.eclipse.paho.client.mqttv3 v1.2.5 and mosquitto as broker to implement it. And it works when client1 send 1 request to client2 in one time. (client2 can reply "response" correctly)
But I found a problem that if client1 sent a lot (e.g. 100, 10000) "request" Continuously to client2, client2 can't publish message or receive message.
Then client2 would disconnect.
my code is here:
package test;
import org.eclipse.paho.client.mqttv3.*;
import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;
public class Client implements MqttCallback {
private final int id;
private MqttClient mqttClient;
private int qos = 1;
private int numGet = 0;
private int numSend = 0;
public Client(int id) {
this.id = id;
String HOST = "tcp://127.0.0.1:1883";
MqttConnectOptions options = new MqttConnectOptions();
try {
mqttClient = new MqttClient(HOST, String.valueOf(id), new MemoryPersistence());
options.setCleanSession(true);
options.setConnectionTimeout(10);
options.setKeepAliveInterval(20);
options.setMaxInflight(1000);
mqttClient.setCallback(this);
mqttClient.connect(options);
System.out.println("Connected to MQTT Broker");
} catch (MqttException e) {
e.printStackTrace();
}
}
private void subscribe(String topic) {
try {
mqttClient.subscribe(topic, qos);
} catch (Exception e) {e.printStackTrace();}
}
public void publish(String topic, String payload) {
try {
boolean retained = false;
mqttClient.publish(topic, payload.getBytes(), qos, retained);
} catch (Exception e) {e.printStackTrace();}
}
#Override
public void connectionLost(Throwable cause) {}
#Override
public void messageArrived(String topic, MqttMessage message) throws Exception {
System.out.println(topic + " get: " + message);
if (id == 2)
publish("/toClient1", "Response");
numGet++;
}
#Override
public void deliveryComplete(IMqttDeliveryToken token) { numSend++; }
public static void main(String[] args) throws InterruptedException {
Client client1 = new Client(1);
Client client2 = new Client(2);
String topic1 = "/toClient1";
String topic2 = "/toClient2";
client1.subscribe(topic1);
client2.subscribe(topic2);
int n = 100;
for (int i = 0; i < n; i++) {
client1.publish(topic2, "request");
}
Thread.sleep(1000);
System.out.println("Client1 sent messages: " + client1.numSend);
System.out.println("Client2 sent messages: " + client2.numSend);
}
}
I got the output like this:
Connected to MQTT Broker
Connected to MQTT Broker
/toClient2 get: request
/toClient1 get: Response
Client1 sent messages: 100
Client2 sent messages: 0
1月 29, 2023 1:02:10 下午 org.eclipse.paho.client.mqttv3.internal.ClientState checkForActivity
严重: 2: Timed out as no activity, keepAlive=20,000,000,000 lastOutboundActivity=6,831,960,648,000 lastInboundActivity=6,811,951,900,600 time=6,851,960,579,800 lastPing=6,831,960,650,700
等待来自服务器的响应时超时 (32000)
at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:31)
at org.eclipse.paho.client.mqttv3.internal.ClientState.checkForActivity(ClientState.java:747)
at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:818)
at org.eclipse.paho.client.mqttv3.internal.ClientComms.checkForActivity(ClientComms.java:804)
at org.eclipse.paho.client.mqttv3.TimerPingSender$PingTask.run(TimerPingSender.java:79)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
I suspect that I should not do publish() in public void messageArrived(String topic, MqttMessage message) callback, so I try to use a queue to enqueue the "request" message and use a new Thread to consume the queue and do publish(). But it get the same result.
After addded concurrrency to the #RabbitListened faced with problem that sometimes events are stuck in app.
When restarting app it's continue works normally. But then could aslo stuck suddenly after sometime.
#RabbitListener(
queues = "${app.queue}",
ackMode = "MANUAL",
concurrency = 2-5,
messageConverter = "jsonMessageConverter")
public void consumeEvent(AppEvent event, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long deliveryTag) {
try {
.....
doAck(channel, deliveryTag);
} catch (Throwable e) {
.....
doNack(channel, deliveryTag);
}
}
#Bean
public PooledChannelConnectionFactory connectionFactory(){
ConnectionFactory rabbitConnectionFactory = new ConnectionFactory();
rabbitConnectionFactory.setHost(host);
rabbitConnectionFactory.setPort(port);
rabbitConnectionFactory.setUsername(userName);
rabbitConnectionFactory.setPassword(password);
return new PooledChannelConnectionFactory(rabbitConnectionFactory);
}
#Bean
public RabbitTemplate rabbitTemplate(){
final var rabbitTemplate = new RabbitTemplate(connectionFactory());
rabbitTemplate.setMessageConverter(getJsonMessageConverter());
return rabbitTemplate;
}
#Bean("jsonMessageConverter")
public Jackson2JsonMessageConverter getJsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
In thread dump there are 3 threads like on picture
I have the following code snipped that creates a TCPServer, and attaches a ChannelHandler to the channel in the doOnChannelInit() function. The server is to process byte data from an embedded device.
#Component
#RequiredArgsConstructor
public class NettyServer {
Logger log = LoggerFactory.getLogger(NettyServer.class);
private final NettyProperties nettyProperties;
private final NettyServerHandler nettyServerHandler;
private TcpServer server;
public void run() {
server = TcpServer
.create()
.host("localhost")
.port(nettyProperties.getTcpPort())
.doOnChannelInit((connectionObserver, channel, remoteAddress) -> {
log.info("Connection from " + remoteAddress);
channel.pipeline()
.addLast("idleStateHandler", new IdleStateHandler(0, 0, 4, TimeUnit.MINUTES))
.addLast(new ByteArrayDecoder())
.addLast(new ByteArrayEncoder())
.addLast(nettyServerHandler);
});
server.bindNow();
log.info("Server running");
}
}
Channel handler
#Component
#RequiredArgsConstructor
#ChannelHandler.Sharable
public class NettyServerHandler extends SimpleChannelInboundHandler<byte[]> {
Logger log = LoggerFactory.getLogger(NettyServerHandler.class);
private final AttributeKey<byte[]> dataKey = AttributeKey.valueOf("dataBuf");
private final AttributeKey<Integer> dataLen = AttributeKey.valueOf("dataBufLen");
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
log.info("New Meter connection from : " + ctx.channel());
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
if (ctx.channel() != null) {
log.info(String.format("Meter/Client Disconnected. No: %s ; Channel : %s", meterNo, ctx.channel()));
}
ctx.close();
}
#Override
protected void channelRead0(ChannelHandlerContext ctx, byte[] msg) throws Exception {
log.info("Message received: " + new String(msg);
ctx.channel().read();
}
}
I'm able to connect to the server, but when i send a message, nothing happens, the log statements are not triggered.
I'm not sure what I'm missing here, would appreciate some help.
Thanks
There is no need to add custom handlers to the Netty pipeline. The example above can be written like this:
#Component
#RequiredArgsConstructor
public class NettyServer {
Logger log = LoggerFactory.getLogger(NettyServer.class);
private final NettyProperties nettyProperties;
private TcpServer server;
public void run() {
server = TcpServer
.create()
.host("localhost")
.port(nettyProperties.getTcpPort())
.doOnChannelInit((connectionObserver, channel, remoteAddress) -> {
log.info("Connection from " + remoteAddress);
channel.pipeline()
.addLast("idleStateHandler", new IdleStateHandler(0, 0, 4, TimeUnit.MINUTES));
})
.handle((in, out) ->
in.receive()
.asString()
.doOnNext(s -> log.info("Message received: " + s))
.then());
server.bindNow();
log.info("Server running");
}
}
Consider checking the Reference Documentation
The incoming data can be transformed to String with (asString), to byte[] with (asByteArray) etc. If there is no suitable transformation you can use map(byteBuf -> ...) and transform the ByteBuf to the needed abstraction.
I have two applications communicating with each other using rabbit.
I need to send (from app1) an object to a listener (in app2) and after some process (on listener) it answer me with another object, now I am receiving this error:
ClassNotFound
I am using this config for rabbit in both applications:
#Configuration
public class RabbitConfiguration {
public final static String EXCHANGE_NAME = "paymentExchange";
public final static String EVENT_ROUTING_KEY = "eventRoute";
public final static String PAYEMNT_ROUTING_KEY = "paymentRoute";
public final static String QUEUE_EVENT = EXCHANGE_NAME + "." + "event";
public final static String QUEUE_PAYMENT = EXCHANGE_NAME + "." + "payment";
public final static String QUEUE_CAPTURE = EXCHANGE_NAME + "." + "capture";
#Bean
public List<Declarable> ds() {
return queues(QUEUE_EVENT, QUEUE_PAYMENT);
}
#Autowired
private ConnectionFactory rabbitConnectionFactory;
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(rabbitConnectionFactory);
}
#Bean
public DirectExchange exchange() {
return new DirectExchange(EXCHANGE_NAME);
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate r = new RabbitTemplate(rabbitConnectionFactory);
r.setExchange(EXCHANGE_NAME);
r.setChannelTransacted(false);
r.setConnectionFactory(rabbitConnectionFactory);
r.setMessageConverter(jsonMessageConverter());
return r;
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
private List<Declarable> queues(String... nomes) {
List<Declarable> result = new ArrayList<>();
for (int i = 0; i < nomes.length; i++) {
result.add(newQueue(nomes[i]));
if (nomes[i].equals(QUEUE_EVENT))
result.add(makeBindingToQueue(nomes[i], EVENT_ROUTING_KEY));
else
result.add(makeBindingToQueue(nomes[i], PAYEMNT_ROUTING_KEY));
}
return result;
}
private static Binding makeBindingToQueue(String queueName, String route) {
return new Binding(queueName, DestinationType.QUEUE, EXCHANGE_NAME, route, null);
}
private static Queue newQueue(String nome) {
return new Queue(nome);
}
}
I send the message using this:
String response = (String) rabbitTemplate.convertSendAndReceive(RabbitConfiguration.EXCHANGE_NAME,
RabbitConfiguration.PAYEMNT_ROUTING_KEY, domainEvent);
And await for a response using a cast to the object.
This communication is between two different applications using the same rabbit server.
How can I solve this?
I expected rabbit convert the message to a json in the send operation and the same in the reply, so I've created the object to correspond to a json of reply.
Show, please, the configuration for the listener. You should be sure that ListenerContainer there is supplied with the Jackson2JsonMessageConverter as well to carry __TypeId__ header back with the reply.
Also see Spring AMQP JSON sample for some help.
I'm using Spring-AMQP-rabbit in one of applications which acts as a message-consumer. The queues are created and subscribed to the exchange at startup.
My problem:
When the RabbitMq server is restarted or removed and added completely, the Queue's are not recreated. The connection to the RabbitMq server is re-stored, but not the Queues.
I've tried to do the queue admin within a ConnectionListener but that hangs on startup. I guess the admin is connection aware and should do queue management upon connection restore isn't?
My Queues are created by a service:
#Lazy
#Service
public class AMQPEventSubscriber implements EventSubscriber {
private final ConnectionFactory mConnectionFactory;
private final AmqpAdmin mAmqpAdmin;
#Autowired
public AMQPEventSubscriber(final AmqpAdmin amqpAdmin,
final ConnectionFactory connectionFactory,
final ObjectMapper objectMapper) {
mConnectionFactory = connectionFactory;
mAmqpAdmin = amqpAdmin;
mObjectMapper = objectMapper;
}
#Override
public <T extends DomainEvent<?>> void subscribe(final Class<T> topic, final EventHandler<T> handler) {
final EventName topicName = topic.getAnnotation(EventName.class);
if (topicName != null) {
final MessageListenerAdapter adapter = new MessageListenerAdapter(handler, "handleEvent");
final Jackson2JsonMessageConverter converter = new Jackson2JsonMessageConverter();
converter.setJsonObjectMapper(mObjectMapper);
adapter.setMessageConverter(converter);
final Queue queue = new Queue(handler.getId(), true, false, false, QUEUE_ARGS);
mAmqpAdmin.declareQueue(queue);
final Binding binding = BindingBuilder.bind(queue).to(Constants.DOMAIN_EVENT_TOPIC).with(topicName.value());
mAmqpAdmin.declareBinding(binding);
final SimpleMessageListenerContainer listener = new SimpleMessageListenerContainer(mConnectionFactory);
listener.setQueues(queue);
listener.setMessageListener(adapter);
listener.start();
} else {
throw new IllegalArgumentException("subscribed Event type has no exchange key!");
}
}
}
Part of my handler app:
#Component
public class FooEventHandler implements EventHandler<FooEvent> {
private final UserCallbackMessenger mUserCallbackMessenger;
private final HorseTeamPager mHorseTeamPager;
#Autowired
public FooEventHandler(final EventSubscriber subscriber) {
subscriber.subscribe(FooEvent.class, this);
}
#Override
public void handleEvent(final FooEvent event) {
// do stuff
}
}
I wonder why out-of-the-box feature with the RabbitAdmin and beans for Broker entities doesn't fit your requirements:
A further benefit of doing the auto declarations in a listener is that if the connection is dropped for any reason (e.g. broker death, network glitch, etc.) they will be applied again the next time they are needed.
See more info in the Reference Manual.