I have a project in which I use Spring AMQP. I have two SimpleMessageListenerContainer, one with a self-declared queue by the server (amq-gen), and one with a queue with a given name.
I use a SimpleRoutingConnectionFactory with two CachingConnectionFactory. For error detection I have a ConnectionListener, ListenerContainerConsumerFailedEvent, and a ConditionalExceptionLogger.
The idea is to switch between two Rabbit servers once an error is detected in the AMQP connection, but when there is an error in the AMQP connection several errors are thrown in the ConditionalExceptionLogger, several events of type ListenerContainerConsumerFailedEvent, and complicates the fact of switching automatically Between servers.
What could be the best way to do that switching automatically given a number of retries?
Thank you
one with a self-declared queue by the server (amq-gen)
You can't do that; if you use broker-declared queue names, the second broker doesn't know about it, and the container will try to declare it, which is not allowed.
Instead use a Spring AMQP AnonymousQueue, which has the same characteristics as a broker declared queue (auto delete, not durable) but has a name generated by the framework so it can be declared when you fail over.
Related
I'm trying to create a reasonable setup for client-client-communication for our existing infrastructure. I've been reading the docs for Spring, Websocket, STOMP, SockJS and ActiveMQ for over a week now and I'm not not sure whether what I am trying to do is feasible or sensible. Spring server and JavaScript client were up and running relatively quickly and sending messages between clients just works (direct connection from JS client to Spring server). This setup won't suffice for our needs so we decided to put dedicated brokers in between. Configuring ActiveMQ is a nightmare probably because I don't really know where to start. I have not worked with a dedicated broker so far.
Environment
170 independent servers (Tomcat, Spring, SockJS, STOMP)
2 ActiveMQ (Artemis) brokers (load balance, failure safety)
a few thousand clients (JavaScript/.NET, SockJS, STOMP)
Requirement
I need every client to be able to talk to every other client. Every message has to be curated by one of the servers. I'd like the clients to connect to one of the ActiveMQ brokers. The ActiveMQ brokers would hold a single connection to every single server. The point is to avoid that all my clients would have to open 170 WebSocket connections to all the servers. The servers do not need to talk to each other (yet/necessarily) since they are independent with different responsibilities.
Question
Is ActiveMQ or any other dedicated broker viable as transparent proxy/relay i.e. can it handle this situation and are there ways to dynamically decide the correct recipients or should I go another route like rolling my own Spring-based relay?
In a traditional messaging use-case (e.g. using ActiveMQ Artemis with STOMP) the broker manages "destinations" and any messages sent to those destinations. Messages are only dispatched to clients if the client specifically creates a consumer on a destination.
In your use-case all of your 170 "servers" would actually be messaging clients. They would need to create a consumer on the broker in order to receive messages. To be clear, once the consumer is created then the broker would dispatch messages to it as soon as they arrived.
I'm not sure what exactly you mean by "transparent," but if it means that the process(es) receiving the message(s) don't have to do anything then no message broker will handle your use-case.
We are implementing an MQ/IIB architecture where we will have one QM and one Broker each on 2 RHEL servers load-balanced with each other to divide incoming traffic.
We have consumer applications which connect our servers through JMS bindings file. We also have IIB applications running on both of them.
Now, since one bindings file could have only one QMGR name while creating a connection factory, it's not recommended to keep different QM/Broker names on each servers. Since this bindings file would be shared with consumers, it has to be with unique QM name.
But if we have same QM/Broker names on each server, all logs on IIB record and replay tool will have one Broker name (from both servers) which is again difficult to identify which server actually served the incoming request.
Could you please suggest best possible approach in such scenario?
Or else suggest if above approach can be modified to achieve our goal.
In general it is not a good practice to have two queue managers with the same name. The same would be true for IIB brokers for the reasons you stated.
In the Binding file you can leave QMANAGER blank (null). This will allow the application to connect to any queue manager listening on the HOSTNAME and PORT that you specify.
If the queue managers on the 2 RHEL servers use the same port you could even set hostname to localhost and use the same binding file on both servers.
Example is below if both queue managers listened on the same port:
DEFINE CF(CF_NAME) QMANAGER() TRANSPORT(CLIENT) CHANNEL(MY.SVRCONN) HOSTNAME(localhost) PORT(1414)
We have a requirement for our project to listen to queues hosted in different machines. For example, we have 4 virtual hosts. I have created new instance of SimpleMessageListenerContainer for each hosts but i use one instance of MessageListener(it implements ChannelAwareListener so that i can manually ack). MessageListener is a bean managed by spring. I maintain a map of host and container instance when i create the containers. On receiving the message, check if i received the desired message from the host, get the container instance from the map (using #Resource) and stop listening to the host. Also manually ACK and store the message in the cassandra database.
Right now, there are times when some of the messages dont get persisted in the database and gets lost. I think it might be a race condition or due to the fact that i am using only one instance of messagelistener but i had to do that so that i can get the map(#Resource). Sorry if I am not making any sense. I am using the AMQP for the first time and trying to understand it. Any suggestions will be great. Thank you!
Why do you need manual ack? It's generally better to let the container take care of acks (AUTO). It will ack the message on success and nack it if the listener throws an exception.
we are currently trying to determine a application architecture for an application that will need to accept a number of SOAP calls and also make SOAP calls. One of the design goals is simplicity and robustness which we need to take into account.
In the Grails space we could all tie this into one big Grails application but this gives headaches in the robustness aspect as and update of the Grails application will disable all incoming SOAP request.
I was wondering if splitting up the Grails app and combining this with something like ActiveMQ/ServiceMix/Mule etc is recommend? Any advice or comments are appreciated! And what kind of solution woud be a good candidate?
You can achieve some robustness with your monolithic Grails app by running it behind a network load balancer. This would allow you to perform no-downtime rolling upgrades.
Now this doesn't address other concerns like the need to deal with possibly unreachable remote SOAP services, etc... This is when a tool/framework, like Mule, can become helpful as it will provide you exception handling, retries and whatnot.
This is conditioned by the intended behavior of your SOAP bridge: is it asynchronous (ie. fire and forget, send the message to the bridge, get an immediate ACK and let the bridge do the remote dispatch whenever possible) or is it synchronous (ie. the caller of the bridge is held until a remote response is received and forwarded back to it).
If your bridge is fundamentally synchronous, I'd say you can stick with your single Grails app and use a load balancer. It will be up to the caller to deal with retries.
Otherwise, if it's async, consider a messaging middleware to help with the temporary message persistence and redelivery in case of failure.
I read in forum that while implementing any application using AMQP it is necessary to use fewer queues. So would I be completely wrong to assume that if I were cloning twitter I would have a unique and durable queue for each user signing up? It just seems the most natural approach and if not assign a unique queue for each user how would one design something like that.
What is the most used approach for web messaging. I see RabbitHUb and Rabbit WebHooks but Webhooks doesn't seem to be a scalable solution. i am working with Rails and my AMQP server as running as a Daemon.
In RabbitMQ, queues are quite cheap. They're effectively lightweight Erlang processes, and you can run tens to hundreds of thousands of queues on a single commodity machine (i.e. my laptop). Of course, each will consume a bit of RAM, but unused-recently queues will hibernate, so they'll consume as little memory as possible. In addition, if Rabbit runs low on memory for messages, it will page old messages to disk.
The above only applies to a single machine. RabbitMQ supports a form of lightweight clustering. When you join several Rabbit nodes into a cluster, each can see the queues and exchanges on the other nodes but each runs only its own queues. So, you'll be able to have even more queues! (to the limit of Erlang clusters, which is usually a few hundred nodes) So, a cluster forms a logical broker distributed over several machines; clients connect to it and use it transparently through any of the nodes.
That said, having a single durable queue for each user seems a bit strange: in AMQP, you cannot browse messages while they're on the queue; you may only get/consume messages which takes them off the queue and publish which adds the to the end of the queue. So, you can use AMQP as a message router, but you can't use it as a sort of message database.
Here is a thread that just talks about that: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2009-February/003041.html