I am trying to learn to Build Microservices based app where Microservices would be communicating via some service bus. In my case, I am using Azure Service bus.
With referring to below link an initial system is set up. Messages are reaching to the queue.
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-multi-tier-app-using-service-bus-queues
But as next step or mimicking real app, I have OrderAPI to handle the orders.
This is how my WorkerRole class looks like
namespace OrderProcessingRole
{
public class WorkerRole : RoleEntryPoint
{
// The name of your queue
const string QueueName = "ProcessingQueue";
// QueueClient is thread-safe. Recommended that you cache
// rather than recreating it on every request
QueueClient Client;
ManualResetEvent CompletedEvent = new ManualResetEvent(false);
public override void Run()
{
Trace.WriteLine("Starting processing of messages");
// Initiates the message pump and callback is invoked for each message that is received, calling close on the client will stop the pump.
Client.OnMessage((receivedMessage) =>
{
try
{
// Process the message
Trace.WriteLine("Processing Service Bus message: " + receivedMessage.SequenceNumber.ToString());
}
catch
{
// Handle any message processing specific exceptions here
}
});
CompletedEvent.WaitOne();
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// Create the queue if it does not exist already
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.QueueExists(QueueName))
{
namespaceManager.CreateQueue(QueueName);
}
// Initialize the connection to Service Bus Queue
Client = QueueClient.CreateFromConnectionString(connectionString, QueueName);
return base.OnStart();
}
public override void OnStop()
{
// Close the connection to Service Bus Queue
Client.Close();
CompletedEvent.Set();
base.OnStop();
}}}
I am not sure, how & where should I call the OrdersAPI when service bus is in place.
I am guessing it would be in
OnStart() ->
Client.OnMessage((receivedMessage) =>
{
try
{
//Order API call here
}
catch
{
}
});
If my guess is right, then how I call my OrderAPI which is hosted on
http://localhost:8090/api/order
Thanks.
Assuming OrdersAPI is your Web API endpoint receiving requests from a browser, it would construct Azure Service Bus messages and send to a queue. Your Worker Role then would receive those messages and process. Processing would not be performed in Web API.
The Orders API should be called inside
Run() ->
Client.OnMessage((receivedMessage) =>
{
try
{
//Order API call here
}
catch
{
}
});
Use HttpClient for triggering the API. By doing this, the API will be called whenever a message is received from the Queue.
As the worker role is hosted in Azure environment, the APIs hosted somewhere public can only be called from here. It does not identify the API hosted in local machine. Try hosting the API in Azure App Service or Azure Function App.
Related
There is a Vaadin 23 tutorial which shows how to send messages to all users (broadcast) https://vaadin.com/docs/latest/advanced/server-push
But what if I need to send a Push message from me to only one other specific user? Is it possible with server push and Vaadin and if so - how ? For example, an Admin user updated something for another User and would like to immediately notify such user about that.
UPDATED
Based on the answer and comments, I updated the Broadcaster from the example to the following:
public class Broadcaster {
static Executor executor = Executors.newSingleThreadExecutor();
static Map<String, List<Consumer<String>>> listeners = new ConcurrentHashMap<>();
public static synchronized Registration register(String userUuid, Consumer<String> listener) {
addListener(userUuid, listener);
return () -> {
synchronized (Broadcaster.class) {
listeners.remove(listener);
}
};
}
private static synchronized void addListener(String userUuid, Consumer<String> listener) {
List<Consumer<String>> consumers = listeners.get(userUuid);
if (consumers == null) {
consumers = new LinkedList<>();
}
consumers.add(listener);
listeners.put(userUuid, consumers);
}
public static synchronized void broadcast(String userUuid, String message) {
List<Consumer<String>> consumers = listeners.get(userUuid);
if (CollectionUtils.isNotEmpty(consumers)) {
for (Consumer<String> consumer : consumers) {
executor.execute(() -> consumer.accept(message));
}
}
}
}
Will such implementation properly work in case I'd like to push a message to the listeners of the specific user?
You need some PubSub in place.
Push is pretty much agnostic to distribution events or what you do with
it in general. It just allows the server to notify the client-side
out-of-band. What means you use this features for, is up to you.
E.g. each client could register to the pub-sub on session-init with
their user-name or -group (or a subject in general) and later some admin
publishes notifications with the target. Only clients registered to
that target react by e.g. pushing.
The poor-mans version would be all clients listening to the same stream
of messages but only react if they are mentioned as the target. This is
most likely less efficient.
I have an app that's using Boot 2.0 with webflux, and has an endpoint returning a Flux of ServerSentEvent. The events are created by leveraging spring-amqp to consume messages off a RabbitMQ queue. My question is: How do I best bridge the MessageListener's configured listener method to a Flux that can be passed up to my controller?
Project Reactor's create section mentions that it "can be very useful to bridge an existing API with the reactive world - such as an asynchronous API based on listeners", but I'm unsure how to hook into the message listener directly since it's wrapped in the DirectMessageListenerContainer and MessageListenerAdapter. Their example from the create section:
Flux<String> bridge = Flux.create(sink -> {
myEventProcessor.register(
new MyEventListener<String>() {
public void onDataChunk(List<String> chunk) {
for(String s : chunk) {
sink.next(s);
}
}
public void processComplete() {
sink.complete();
}
});
});
So far, the best option I have is to create a Processor and simply call onNext() each time in the RabbitMQ listener method to manually produce an event.
I have something like this:
#SpringBootApplication
#RestController
public class AmqpToWebfluxApplication {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = SpringApplication.run(AmqpToWebfluxApplication.class, args);
RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
for (int i = 0; i < 100; i++) {
rabbitTemplate.convertAndSend("foo", "event-" + i);
}
}
private TopicProcessor<String> sseFluxProcessor = TopicProcessor.share("sseFromAmqp", Queues.SMALL_BUFFER_SIZE);
#GetMapping(value = "/sseFromAmqp", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> getSeeFromAmqp() {
return this.sseFluxProcessor;
}
#RabbitListener(id = "fooListener", queues = "foo")
public void handleAmqpMessages(String message) {
this.sseFluxProcessor.onNext(message);
}
}
The TopicProcessor.share() allows to have many concurrent subscribers which we get when we return this TopicProcessor as a Flux to our /sseFromAmqp REST request via WebFlux.
The #RabbitListener just delegates its received messages to that TopicProcessor.
In the main() I have a code to confirm that I can publish to the TopicProcessor even if there is no subscribers.
Tested with two separate curl sessions and published messages to the queue via RabbitMQ Management Plugin.
By the way I use share() because of: https://projectreactor.io/docs/core/release/reference/#_topicprocessor
from multiple upstream Publishers when created in the shared configuration
That' because that #RabbitListener really can be called from different ListenerContainer threads, concurrently.
UPDATE
Also I moved this sample to my Sandbox: https://github.com/artembilan/sendbox/tree/master/amqp-to-webflux
Let's suppose you want to have a single RabbitMQ listener that somehow puts messages to one or more Flux(es). Flux.create is indeed a good way how to create such a Flux.
Let's start with Messaging with RabbitMQ Spring guide and try to adapt it.
The original Receiver would have to be modified in order to be able to put received messages to a FluxSink.
#Component
public class Receiver {
/**
* Collection of sinks enables more than one subscriber.
* Have to keep in mind that the FluxSink instance that the emitter works with, is provided per-subscriber.
*/
private final List<FluxSink<String>> sinks = new ArrayList<>();
/**
* Adds a sink to the collection. From now on, new messages will be put to the sink.
* Method will be called when a new Flux is created by calling Flux.create method.
*/
public void addSink(FluxSink<String> sink) {
sinks.add(sink);
}
public void receiveMessage(String message) {
sinks.forEach(sink -> {
if (!sink.isCancelled()) {
sink.next(message);
} else {
// If canceled, don't put any new messages to the sink.
// Sink is canceled when a subscriber cancels the subscription.
sinks.remove(sink);
}
});
}
}
Now we have a receiver that puts RabbitMQ messages to sink. Then, creating a Flux is rather simple.
#Component
public class FluxFactory {
private final Receiver receiver;
public FluxFactory(Receiver receiver) { this.receiver = receiver; }
public Flux<String> createFlux() {
return Flux.create(receiver::addSink);
}
}
Receiver bean is autowired to the factory. Of course, you don't have to create a special factory. This only demonstrates the idea how to use the Receiver to create the Flux.
The rest of the application from Messaging with RabbitMQ guide may stay the same, including the bean instantiation.
#SpringBootApplication
public class Application {
...
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queueName);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(Receiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
...
}
I used similar design to adapt Twitter streaming API sucessfuly. Though, there may be a nicer way how to do it.
I have many windows services that I have installed using Top-Shelf.
I wondered if there is a way to set a timeout when stopping the service to enable on-going process that the service is currently handling to finish before the service actually closes?
I want to wait X seconds before I terminate the Top-Shelf.
This is my console which I have installed as a service:
static int Main(string[] args)
{
return (int)HostFactory.Run(x =>
{
string serviceName = new ServiceNameResolverImpl().GetServiceName();
x.SetServiceName(serviceName);
x.SetDisplayName(serviceName);
x.SetDescription(serviceName);
x.UseAssemblyInfoForServiceInfo();
x.RunAsLocalSystem();
x.StartAutomaticallyDelayed();
x.SetStartTimeout(TimeSpan.FromMinutes(1));
x.Service<ServiceImpl>();
});
}
Thanks!
In your service implementation, use the HostControl to request additional shutdown time (i'm presuming your service already implements ServiceControl as that's what .Service<T> expects) - note that the Service Control Manager may or may not grant the request:
public class ServiceImpl : ServiceControl
{
public bool Start(HostControl hostControl)
{
return true;
}
public bool Stop(HostControl hostControl)
{
hostControl.RequestAdditionalTime(TimeSpan.FromMinutes(1));
return true;
}
}
similar to the start timeout
x.SetStopTimeout(TimeSpan.FromMinutes(1));
Overview:
Our application extends a UIApplication and has a SMS Listener class that is registered on boot. When a message is received that fits our criteria we process the message and then we want to save it to a local SQLite database as well as upload it to a Web Server. It is important that this happens as soon as possible after the SMS is received, even if the UI Application is not open at that stage.
Problem:
When the SMSListener Instance is running in the background, with no UIApplication instance active, and wants to access the SQLite database or tries to create a HTTP Connection a “No Application Instance” exception is thrown.
Desired outcome:
We want to process, save and upload all the messages from the SMSListener background thread even if the UIApplication is not active. Currently the SMSListener background thread would store the messages in the RuntimeStore; when the UI Application is started it reads the messages from the RuntimeStore and saves it to the database. This is not an optimal solution though, because the synchronisation with the Web Server would also then only happen when the UI Application is next opened. It is important that it rather syncs when the message is received.
Application Pseudo Code:
Main Class, checks for startup and creates a SMSListener instance or gets the instance from the RuntimeStore.
public class OurAppUi extends UiApplication {
public static void main(String[] args) {
if (args != null && args.length > 0 && args[0].endsWith("gui")) {
// Create a new instance of the application and make the currently
// running thread the application's event dispatch thread.
OurAppUi theApp = new OurAppUi();
theApp.enterEventDispatcher();
} else {
// Entered through the alternate application entry point
SmsListener.waitForSingleton();
}
}
}
The SMSListener Class listens for any incoming messages, makes use of the RuntimeStore Singleton Model. This is working as expected.
public class SmsListener implements javax.wireless.messaging.MessageListener {
public static SmsListener waitForSingleton() {
//Ensure this is a singleton instance.
//Open RuntimeStore and obtain the reference of BackgroundListener
RuntimeStore store = RuntimeStore.getRuntimeStore();
Object obj = store.get(ID_BACKGROUND_LISTENER);
//If obj is null, there is no current reference to BackgroundListener
//Start a new instance of BackgroundLIstener if one is not running
if(obj == null) {
store.put(ID_BACKGROUND_LISTENER, new SmsListener());
return (SmsListener)store.get(ID_BACKGROUND_LISTENER);
} else {
return(SmsListener)obj;
}
}
public void notifyIncomingMessage(MessageConnection conn) {
new Thread() {
MessageConnection connection;
Thread set (MessageConnection con) {
this.connection = con;
return (this);
}
public void run() {
try {
Message m = connection.receive();
String msg = null;
if (m instanceof TextMessage) {
TextMessage tm = (TextMessage)m;
msg = tm.getPayloadText();
}
// Process the SMS
SMSObject sms = processSMS(msg);
// Save to DataBase { Exception is Thrown Here }
SQLManager.getInstance().save(sms);
// Upload to Web Server { Exception is Thrown Here }
WebServer.upload(sms);
} catch(Exception error) {
}
}
}.set(conn).start();
}
}
When the SmsListener Instance wants to access the SQLite database or tries to create a HTTP Connection a “No Application Instance” exception is thrown.
public final class SQLManager {
private SQLManager() {
try {
db = OpenOrCreateDatabase();
} catch (MalformedURIException e) {
Debug.log(TAG, "Get connection: URI: " + e.getMessage());
} catch (ControlledAccessException e) {
Debug.log(TAG, "Get connection: Controlled Access: " + e.getMessage());
} catch (DatabasePathException e) {
Debug.log(TAG, "Get connection: Database Path: " + e.getMessage());
} catch (DatabaseIOException e) {
Debug.log(TAG, "Get connection: Database IO: " + e.getMessage());
} catch (Exception e) {
Debug.log(TAG, e);
}
}
public static synchronized SQLManager getInstance() {
if (instance == null) {
instance = new SQLManager();
}
return instance;
}
}
We’ve tried store the SQLite instances in the RuntimeStore, using the same Singleton Model as the SMSListener but received errors when the UI Application tried to access the stored DB Instance.
In general the way to handle this type of activity is to divide the application into two parts:
The user interactive parts that need the UI and need only be run when the user wants to interact with the application;
The background processing part that will store the data and communicate with the remote server.
The background processing should take place under the context of an extension of a net.rim.device.api.system.Application which probably should be a RuntimeStore based singleton. This portion should be started from your auto run code, register the listeners and remain active. There are some complexities involved in making sure the code executes in the right context. I have a blog post which may be helpful.
I'm struggling with the concept of creating a Jedis-client which listens infinitely as a subscriber to a Redis pubsub channel and handles messages when they come in.
My problem is that after a while of inactivity the server stops responding silently. I think this is due to a timeout occurring on the Jedis-client I subscribe with.
Would this likely indeed be the case? If so, is there a way to configure this particular Jedis-client to not timeout? (While other Jedispools aren't affected with some globally set timeout)
Alternatively, is there another (best practice) way of what I'm trying to achieve?
This is my code, (modified/ stripped for display) :
executed during web-server startup:
new Thread(AkkaStarter2.getSingleton()).start();
AkkaStarter2.java
private Jedis sub;
private AkkaListener akkaListener;
public static AkkaStarter2 getSingleton(){
if(singleton==null){
singleton = new AkkaStarter2();
}
return singleton;
}
private AkkaStarter2(){
sub = new Jedis(REDISHOST, REDISPORT);
akkaListener = new AkkaListener();
}
public void run() {
//blocking
sub.psubscribe(akkaListener, AKKAPREFIX + "*");
}
class AkkaListener extends JedisPubSub {
....
public void onPMessage(String pattern, String akkaChannel,String jsonSer) {
...
}
}
Thanks.
ermmm, the below solves it all. Indeed it was a Jedis thing
private AkkaStarter2(){
//0 specifying no timeout.. Overlooked this 100 times
sub = new Jedis(REDISHOST, REDISPORT,0);
akkaListener = new AkkaListener();
}