Disclaimer: .Net guy trying to learn grails.
I've gotten used to building services with a distributed and durable messaging layer for inter-service communication with NServiceBus and MSMQ.
For anyone unfamiliar, NServiceBus provides messaging simply by referencing the assembly, doing some quick dependency injection.
Then, to work with it, I can send a message simply by doing something like bus.Send("location", messageObject) for a command, and bus.publish(messageObject) for a publish/subscribe situation. Then, all I have to do is create a service that "listens" for my messageObject type and I get the message.
It also provides something they call timeouts - which basically will trigger some event handler after x amount of time (useful for sending reminders or doing something on a schedule).
I'm looking for something similar. I found an article that suggest using grails itself as an ESB, but I don't see how grails can provide reliable and durable messaging. What I mean by that - if service A sends a message to service B, and service B is down, service A will retry later. A more involved example would be that of a saga - where the client starts a saga, service A does something, and service B does something, both report to the saga when they're done processing, and then the saga sends a message to service C so it can do its thing, knowing that both service A and B have done their job.
PS: if this question is too broad, please let me know how I can refine it. I'm at the very beginning of learning grails, so I'm not even sure where I need to start researching stuff.
EDIT: realized I forgot to add the article- http://jlorenzen.blogspot.com/2009/03/grails-create-app-esb
I'd probably use some AMQP queue that has a Grails plugin (like RabbitMQ).
While this wouldn't give you all the features of NServiceBus on MSMQ, you would get the durable messaging behavior you wanted. Things you'd give up / have to implement yourself include some of the retry logic, sagas, and message idempotence.
Related
I am new to web development but I have a project done in VB6, it is a real time send and receive sms application. It uses a GSM modem connected to the server. My company asked me to make a web based version of my application, using ASP.NET MVC.
So I studied MVC about a month ago, created basic CRUD apps, read about SignalR but I don't want to start up a project that I am not confident if its feasible or not. So my question is, is this project possible with the said technology? (MVC, SignalR, and a GSM Modem)
If it is, can you point me somewhere I could start reading for maybe you know a similar project that can guide me through.
And if it is not, can you suggest me the appropriate tech to use for this project to be feasible? (I prefer ASP.NET MVC)
A little detail about the functionality of the project:
Has User Authentication - this user will have a simple "SMS box" type of thing which will update like a chatbox if an SMS is received in the GSM modem and this user can reply and use the GSM to send the SMS back to the sender.
IF anyone could help me, guide me, point me to some reading materials to make this project it would be greatly appreciated. Please post if you need clarifications and further explanation. Thanks!
User,
There are many ways to achieve this, so I will explain a method that worked for me a while back doing exactly this.
Based on the information provided, I will assume that you have a windows desktop app, that can already talk to the modem and perform various functions.
Trying to perform that same tasks in a web application can be a hard path to go down, what I opted for was rather simple in my opinion. You really have two separate problems, one is getting the received messages to the web application and the other getting the web application to send messages to the modem.
For handling received messages, I created a simple web api in the web application and defined a post method that is able to receive a sms object. You will need to provide the url for this call to your code that resides in your modem project, which I would convert to a windows service, if it is not already. Then when your code fires when the modem has received a sms, you simply create a object, serialize it to xml or json and then call the post method in your web api. This will give you the chance to use SignalR to notify clients or whatever else you might need to do. This is honestly is the simple part and you have the "real-time" handling ability now for receive.
For sending, there again where many options, I opted for database polling from the code inside the windows service, so when a user wants to reply or send a message, the message gets stored in a table in a database, from where the service code will pick it up and send it, once sent it will mark a field in the table as sent and possibly add a date and time as to when this happened. Again your service code could call another method on your web api, that will notify your app, that a message was sent and again you could notify the end user.
As for database polling, there are various methods, you could simply run some code every minute or some configurable value to see if there are any unsent messages.
You could use SqlDependency and handle the events in real time, you would register an event against the table where messages to be sent will be inserted into and when this happens, your service code will receive a real time event which can be handled.
You could also in the windows service, create a service, even a self hosted web api again, which can be called from your web application when it wants to send an email.
I did various implementations based on client requirements, but for me hosting either a wcf service or web api in the windows service project, proved to be the best in my opinion. You then simply have a configuration section in your web application, and provide the service address, or uri for the post method.
I hope that I have given you enough ideas to get a starting point for your project.
I'm developing an application that has various types of Notifications. Examples of notifications:
Message Created
Listing Submitted
Listing Approved
I'd like to tie all of these up to SignalR so that any connected clients get updates in real-time.
As far as architecture goes - right now the application is entirely within a single solution hosted on an Azure Website. The triggers for each of these notification types live within this application.
When a trigger is hit, I'd like to tell signalR, "Hey, send this message to the following clients" along with a list of userIds. I'm assuming that it's possible to identify connected clients based on userId... and I'm assuming that the process of send message to clients should be executed outside of the web application, so as to not slow down the MVC app or risk losing data in a broken async call. First question - are these assumptions correct?
Assuming so, this means that I'll need something like a dedicated web/worker role to be sending messages to clients. I could pass messages from my web application directly to this process, but what happens if the process dies? The resiliency concerns lead me to believe that the proper way to pass messages would be via a queue of some sort. Second question - is this a valid train of thought?
Assuming so, this means that I can either use a good ol' Azure SQL database as a queue, but it seems like there are some specialized (and maybe cheaper) services to handle message queueing, such as this:
http://www.windowsazure.com/en-us/develop/net/how-to-guides/queue-service/
Third question: Should this be used as a queueing mechanism for signalR? I'm interested in using Redis for caching in the future... would Redis be better or worse than the queue service?
Final Question:
I've attempted to illustrate my proposed architecture here:
What I'm most unclear on here is how the MVC app will know when to queue, or how the SignalR processes will know when to broadcast. Should the MVC app queue blindly, without caring about connected clients? This seems to introduce a lot of wasted space on the queue, and wasted cycles in the worker roles, since a very small percentage of clients will ever be connected.
The only other approach I can think of is to somehow give the MVC app visibility into the SignalR processes to see if the client is connected... and if they are, then Enqueue. This makes me uncomfortable though because it means I have to hit that red line on the diagram for every trigger that gets hit, which - even if done async - gets me worrying about performance and reliability.
What is the recommended architecture for scalable, performant SignalR message broadcasting? Performance is top priority, followed closely by cost.
Bonus question:
What if some messages are of higher priority than others? Should two queues be used, one of which always gets checked before the other?
If you want to target some users, you'll have to come up with a mechanism, off the top of my head I can give an example, if any user hits a page, you can create a group for that page and push to all users in that group/in that page.
It's not clear to me why you need the queues. Usually users subscribe to some events when hitting a page or by some action like join a chat room, and the server pushes data using those events/functions when appropriate.
For scalability, you can run signalr in different servers, in which case you should use sql server, or service bus or redis as a backplane.
Firstly you need to create a SignalR server to which all the users can connect to. This SignalR server can be created either in the web role or worker role. If you have a huge user base then its better to create the SignalR server on a separate role.
Then wherever the trigger is hit and you want to send messages to users, you have to create a SignalR client (.NET or javascript) and then connect to SignalR server. Then you can send the message to SignalR server which in turn will broadcast to all the other users connected. After that you can disconnect the connection with SignalR server. This way you dont have to use queues to communicate with the SignalR role.
And also to send messages to specific users you can store the socket id's along with their user id's in a table (azure table storage should do) when they connect to SignalR server. Then using socket id you can send messages to specific user.
I have an existing complex website built using ASP.NET MVC, including a database backend, data layer, as well as the Web UI layer. Rebuilding this website in another language is not a feasible option.
There are some UI elements on some views (client side) which would benefit from live interactivity, involving both push and pull, so rather than implement some kind of custom long polling or websocket server in asp.net, I am looking to leverage node.js for Windows, and Socket.io.
My problem is that I need two way communication between both applications. Each user should only be able to receive data once they are authorised on the ASP.NET website, so I first need communication for this. Secondly, once certain events occur on the ASP.NET website I want to immediately push this data to the Node server, to be broadcast to specific users or groups of users. Thirdly, I would like any data sent to the node.js server to be pushed to the ASP.NET website for processing, as this is where all our business logic lies. The sole reason for adding Node.js is to have the possibility to push data directly to the client, I do not want to build any business logic into it (or as little as possible).
I would like to know what the fastest method of two-way push communication is between Node.Js and ASP.NET. The only good option I'm aware of so far is to create a special listener on a specific port on the node.js server and connect to that, but I was wondering if there's a more elegant or more efficient method? I also know that you could use a database inbetween but surely this would need to be polled and would be less efficient? Both servers will be running on the same server under a Visual Studio project.
Many thanks for any help you can provide.
I'm not an ASP.NET expert, but I think there are multiple ways you can achieve this:
1) As you said, you could make Node listen on a specific port for data and then react based on the data received (TCP)
2) You can make POST requests to Node.js (HTTP) and also send an auth-key in the process to be extra-secure. Like on 1) Node would react to the data you send.
3) Use something like Redis for pub-sub, send messages from ASP.NET (pub) and get them on the Node.js part (sub). This is even better if you want to scale your app across multiple machines etc.
The only good option I'm aware of so far is to create a special
listener on a specific port on the node.js server and connect to that,
but I was wondering if there's a more elegant or more efficient
method?
You can try to look at redis pub/sub model where ASP.NET MVC application and node.js would communicate through separate channels in order to achieve full-duplex communication. Or you can also try to use CouchDB change nofitications.
I also know that you could use a database inbetween but surely this
would need to be polled and would be less efficient?
Former techniques do not require you to poll for changes, but instead they will notify you when the changes happens or channel message arrives.
apologies in advance for this question being dumb, or previously covered. I have researched far and wide but have not found any resources on WCF/ Windows Services that cover this question.
I have a managed Windows Service which is working nicely. Every n (>5) seconds it checks on the status (e.g. memory consumption) of some processes and other Windows services and also does some database logging and raises events where necessary.
I intend to make an ASP.NET website that would allow users to query the status of the processes that the Windows Service is monitoring. Having researched the options it looks like the up-to-date method would be to use a WCF Service, hosted in the Windows Service, to act as intermediary between the ASP.NET website and the Windows Service. Such that, a user could request through the browser a snapshot of the current status of whatever set of processes the Windows Service was monitoring, and have this request and subsequent response relayed through the WCF service (using named pipes, I think).
So, my difficulty is that there a set of methods and events in the Windows Service for which a single root object exists (let's say MonitorObject). I don't see how the ServiceHost can be instantiated with the reference to MonitorObject so that the WCF Service can call the methods in the Windows Service. I am thinking that perhaps I need to make the Monitor object a shared (I am VB'ing) member of the Windows Service class (that contains OnStart and OnStop) and make all the events shared so that the WCF Service can just access the WindowsService.SharedMonitorObject without needing to be passed the object....
However, I am lost in the subject and am seeking any advice on how best to proceed.
Thanks in advance.
I think you're going down the right track. I wouldn't necessarily make the entire MonitorObject shared, but you might put a shared method in that object that will return the single root object to the caller.
There is a design pattern called the Singleton Pattern that will help you with this. Jon Skeet has written an excellent article on some of the things to be aware of when using this pattern in .NET. His article uses C# for the examples, but here's a SO question referencing this pattern using VB.
While it's unclear from your description, my guess is that your Windows Service is essentially single-threaded right now. Just keep in mind that once you add the WCF service, you'll need to make the methods that it references thread-safe.
I have some delphi code which, given a list of items, calculates the total price taking into account any special deals that might apply.
This code is non-trivial to rewrite in another language.
How should I set it up to communicate with a website running on the same server? The website will need to ask it for a price every time the user updates their shopping cart. It's possible that there will be multiple concurrent requests.
The delphi code needs to maintain an in-memory list of special deals, periodically refreshed from a database. So it cannot simply be executed every time or anything as simple as that.
I don't know what the website is written in, or even which http server it runs under, so I'm just looking for ideas or standard methods.
It sounds like the win32 app is already running as a Windows Service on the box. So, if you can't modify that service, you are going to have to deal with whatever way it wants to accept and respond to requests. This could be through sockets or some higher level communication protocol like web services.
You could do a couple of things. Write an assembly that knows how to communicate with the service and have your web site use that assembly. Or you could build a shim service that knows how to communicate with the legacy service, but exposes communication over higher level protocols such as web services. Either way will have the benefit of hiding the concurrency, threading and communications issue behind an easy to call interface, but the latter will make communicating with the service easier for everyone going forward.
If you can modify the delphi app to take an XML request and respond with an XML answer over a TCP socket (ideally using the HTTP protocol), you will be able to make it interoperate with most web server frameworks relatively easily. But the exact details of how to make that integration happen will depend on the language/framework it was written in.
If the web server is on windows you can compile your delphi app as a DLL that can return XML or HTML, taking parameters as part of the URL or a POST operation. Some details on making a Delphi DLL for web servers are here.
It doesn't matter what web server or OS the existing system is running under. What matters is what you want YOUR code to run under. If it is windows then the easiest solution would be to use WebBroker and write a custom ISAPI application, or use SOAP to expose web services. The first method could be used if you wanted to write a rest like API for instance, the second if your web application has the ability to consume web services.
Another option, if you are running both on the same box under IIS, is to create a COM/Automation object which you then invoke via server side scripting (ASP). If the application is an ASP.NET application, then I would use PRISM to port your code into an assembly.
I have done this with a quite complicated workers compensation calculator. I created a windows service using RemObjects Sdk. The calculations are exposed as a soap method so it can be accessed by nearly anything.
It's not necessary to use RemObjects in the service but it makes it much easier to do as it handles a lot of the underlying plumbing. The clients don't need RemObjects, they just need to be able to call soap methods. Nearly any programming langugae can do that.
You could also create an isapi dll for IIS that exposes a soap interface. This would be useful if other websites on different servers needed access to the methods. However I have handled this in my case by opening a port in the firewall to access my windows service.
There is a lot of examples on the web. A couple of places to start reading are About.Com and Dr Bob.
Torn this app into Windows Service. Write Web Service that will communicate with your windows service. You should spend some time designing your Web Service, because this Web Service is going to be your consistent interface, shielding old Delphi app. So in the future whenever you will want to write web app, mobile app, or whatever you will imagine, you will have one consistent interface – XML Web Service.
A popular way to integrate a web application with background services is a message broker.
The message flow would be:
the web application sends a "calculation request" message to a message destination on the message broker, which contains all needed parameters and also a correlation id to match the calculation request with the response from the Delphi service
one (or, in a high availability / load balanced environment more) Delphi services handle the messages: pull the next incoming message, process it by feeding the parameters to the calculation engine, and send a "calculation result message" back to the web server
the web server can either synchronously wait for the response (and discard responses which have no matching correlation ide) and build the result HTML document, or continue with other tasks and asynchronously receive the calculation result in a separate thread, for example in a Ajax based web application
See for an introduction this slideshow about the Dopplr image service:
http://de.slideshare.net/carsonified/dopplr-its-made-of-messages-matt-biddulph-presentation
If you can make it a service (but not a library), you have to do inter-process communication somehow - there are a few ways to do this on Windows:
Sockets directly which is hardest since you have to do marshalling/auth yourself
Shared Memory (yuck!)
RPC which works great but isn't trivial
DCOM which is easier but a pain to configure
WCF - but can you call it from your Windows Service written in Delphi?