RabbitMQ: Why don't channels reopen after a reconnection? - spring-amqp

After my app loses connection to the RabbitMQ Server, I see in the log
ERROR ... o.s.a.r.c.CachingConnectionFactory : Channel shutdown: connection error
And then when my app reconnects to RabbitMQ Server, I see in the log
INFO ... Created new connection: SimpleConnection#7d9717bc [delegate=amqp://******]
However when the app reconnects to RabbitMQ Server, The channels are not reopened, in the RabbitMQ Admin Console, I can see in the 'Channels' tab that there aren't any.
In the RabbitMQ Server log after the reconnection:
=INFO REPORT==== 13-Jul-2017::10:33:39 ===
accepting AMQP connection (*.*.*.*:* -> *.*.*.*:5672)
=INFO REPORT==== 13-Jul-2017::10:33:39 ===
Connection (*.*.*.*:* -> *.*.*.*:5672) has
a client-provided name: rabbitConnectionFactory#1
=INFO REPORT==== 13-Jul-2017::10:33:39 ===
connection (*.*.*.*:* -> *.*.*.*:5672 -
rabbitConnectionFactory#1): user '***' authenticated and granted access to vhost '***'
I'm using Spring Boot 1.5.3 and spring-boot-starter-amqp
It looks like Spring AMQP 1.7 disables the rabbitmq client's 'enableAutomaticRecovery', and uses it's own recovery mechanism
The 4.0.x client enables automatic recovery by default; while compatible with this feature, Spring AMQP has its own recovery mechanisms and the client recovery feature generally isn’t needed. It is recommended to disable amqp-client automatic recovery, to avoid getting AutoRecoverConnectionNotCurrentlyOpenException s when the broker is available, but the connection has not yet recovered. Starting with version 1.7.1, Spring AMQP disables it unless you explicitly create your own RabbitMQ connection factory and provide it to the CachingConnectionFactory. RabbitMQ ConnectionFactory instances created by the RabbitConnectionFactoryBean will also have the option disabled by default.
I'm not sure if this has anything to do with the issue.
Note that if I shutdown my app and restart it, it behaves as expected.

Spring AMQP turns off the automaticRecovery by default:
private static com.rabbitmq.client.ConnectionFactory newRabbitConnectionFactory() {
com.rabbitmq.client.ConnectionFactory connectionFactory = new com.rabbitmq.client.ConnectionFactory();
connectionFactory.setAutomaticRecoveryEnabled(false);
return connectionFactory;
}
You can turn it on via RabbitConnectionFactoryBean:
/**
* Set to true to enable amqp-client automatic recovery. Note: Spring AMQP
* implements its own connection recovery and this is generally not needed.
* #param automaticRecoveryEnabled true to enable.
* #since 1.7.1
*/
public void setAutomaticRecoveryEnabled(boolean automaticRecoveryEnabled) {
this.connectionFactory.setAutomaticRecoveryEnabled(automaticRecoveryEnabled);
}
The Spring AMQP out-of-the-box automatic recovery is good for the listener container, but I guess you deal with RabbitTemplate. So, consider to switch it on: http://docs.spring.io/spring-amqp/reference/html/_reference.html#auto-recovery

Related

KeyCloak: Connection has been refused by the server. Connection timed out

Occasionally I receive a connection timeout when calling the /userinfo endpoint of my KeyCloak-Server.
So far, I have no indication what's wrong and what causes the timeouts. There are no errors in the server.log I configured. Also, I cannot reproduce the issue, I just see the errors in the logs of the application trying to authenticate with keycloak.
Is there some sort of connection limit that my keycloak might use?
List item
What additional logs can I activate to narrow down the problem?
I am currently on version 17.0.1
Try running keycloak in debug mode kc.sh start --log-level=debug If the /userinfo call reached the keycloak then there will be a debug log for that, you can match the time when error occurred to the keycloak log.
Do you have any other components in between your application and keycloak such as proxy, a DNS server etc ? You would need to check their logs as well.
Also check out this document regarding rest api in keycloak -> https://github.com/keycloak/keycloak-community/blob/main/design/rest-api-guideline.md#rate-lmiting

Service bus connection is not working from on-premise console application

I have created a .net core 3.1 console application for reading azure service bus queue messages and deployed the EXE in a client on-premise VM. It was working initially but now it is not working from VM (now also working from local machine). I am getting a time-out (socket exception) while executing the exe in on-premise VM. I am using shared access policy connection strung to connect the service bus.
Exception :
Azure.Messaging.ServiceBus.ServiceBusException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ErrorCode: TimedOut (ServiceCommunicationProblem)
---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
at Microsoft.Azure.Amqp.Transport.TransportStream.EndRead(IAsyncResult asyncResult)
at Microsoft.Azure.Amqp.Transport.TransportStream.<>c__DisplayClass22_0.b__1(IAsyncResult a)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
The most common connection issue in an enterprise environment is that the ports needed for AMQP over TCP (5671/5672) are not open. Changing the transport to AMQP over WebSockets often helps, as it will use port 443 and may be routed through a proxy, if needed.
Both the transport and the proxy (if needed) can be specified using the ServiceBusClientOptions when creating your client:
var options = new ServiceBusClientOptions
(
TransportType = ServiceBusTransportType.AmqpWebSockets,
WebProxy = new WebProxy("https://proxyserver:80", true)
};
var client = new ServiceBusClient("<< CONNECTION STRING >>", options);
For more information, you may want to look at the Service Bus troubleshooting guide.

Paho MQTT connecting to MQXR broker

I have a mqtt java code (paho) written which works fine when it attempts to connect to Mosquitto broker. It is able to establish, publish all messages to a topic. But with MQXR (broker included with MQ), I get
SEVERE: Could not connect to MQTT broker, requests may be lost.
Connection lost (32109) - java.io.EOFException
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:138)
at java.lang.Thread.run(Thread.java:736)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:250)
at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.readMqttWireMessage(MqttInputStream.java:56)
at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(CommsReceiver.java:100)
... 1 more
The QoS is set as 2. Not able to find why it fails in MQXR? Any thoughts is greatly appreciated.
Thanks
There are two possibilities for losing connection:
1. Some MQTT Brokers will not support QOS-2 Implementations like Azure IoT-Hub. Verify that your broker supports QOS-2.
2. What Port are you connecting to?. If its SSL/TLS encrypted, then you cannot connect with unencrypted connect process. You have to enable SSL connection using bouncycastle during your connect process.

Connection lost during mqtt connection with rabbit mq server

Actually i am using rabbit mq server and mosquitto mqtt client for the connection
but after sometime server is disconnecting from the client
and the error in not traceable.
You should check the rabbitmq log (and log configuration) and also test with different keep alive values.

Rabbitmq server connection closing abruptly

I have tried to use Rabbitmq server for some reason the connection closes abruptly even though I passed the correct username and password.
Rabbitmq server is running on port 5672 and telneting to my server at port 5672 says its running fine.
I have installed rabbitmq server in CentOS and my rabbitmq server log are as follows:
=INFO REPORT==== 19-Dec-2012::06:25:44 ===
accepted TCP connection on [::]:5672 from <host>:42048
=INFO REPORT==== 19-Dec-2012::06:25:44 ===
starting TCP connection <0.357.0> from <host>:42048
=WARNING REPORT==== 19-Dec-2012::06:25:44 ===
exception on TCP connection <0.357.0> from <host>:42048
connection_closed_abruptly
=INFO REPORT==== 19-Dec-2012::06:25:44 ===
closing TCP connection <0.357.0> from <host>:42048
What might be the possible reasons for this to happen.
Thanks
connection_closed_abruptly means the client closed the TCP connection without going through the proper AMQP connection termination process.
Is your rabbit server behind a load balancer? A common cause for connections being abruptly closed as soon as they're started is a TCP load balancer's heartbeat. If this is the case you should see these messages at very regular intervals, and the generally accepted practice seems to be to ignore them. To avoid log file buildup you could also consider raising the log level to "error".
On the other hand, if your client connects to the rabbitmq server directly, this probably means your client does not close the connection in an AMQP-approved way. You could try a different client to confirm whether this is the case.
Btw, telnetting to your server is likely to cause abrupt closings too. :)
Check your connection limit
Your connection is time short, there is usually caused by your client improper use

Resources