I need to create a Message for unit-testing. How to create Message in Vertx?
You can mock the Message with your favorite mocking library. But in general, Vert.x users simply start a Vert.x instance for testing, since it's small and embeddable. Then you no longer mock but register test consumers and publishers.
Related
I building multitenancy into the unit of work for a set of services. I want to keep the tenancy question out of the way of day-to-day business domain work, and I do not want to touch every existing consumer in the system (I am retrofitting the multitenancy onto a system without any prior concept of a tenant).
Most messages in the system will be contexted by a tenant. However, there will be some infrastructure messages which will not be, particularly for the purpose of automating tenant creation. I need a way of determining whether to use a tenant-contexted unit of work, or a infrastructure unit of work uncontexted by a tenant because the way I interact with the database is different depending on whether I have tenant context. The unit of work is built in the process of spinning up the dependencies of the consumer.
As such I need a way of peeking at the message or its metadata before consuming it, and specifically, I need to be able to peek at it during the dependency building. I was intended to have a tag interface to mark tenant management messages out from normal business domain messages, but any form of identifying the difference could work. If I am in a unit of work resulting from an HTTP request, I can look at WebApi's HttpContext.Current and see the headers of the current request, etc. How do I do something analogous to this if I am in a unit of work resulting from messaging?
I see there is a way to intercept messages with BeforeConsumingMessage() but I need a way of correlating it to the current unit of work I am spinning up and I'm not seeing how that would work for me. Pseudocode for what I am trying to do:
if MessageContext.Message.GetType() = typeof<ITenantInfrastructureMessage>:
database = new Database(...)
else:
tenantId = MessageContext.Headers.TenantId;
database = new TenantDatabase(..., tenantId)
I am working in C#/.NET using MassTransit with RabbitMQ and Autofac with MassTransit's built-in support for both.
Your best option is to override at the IConsumerFactory<T> extension point, and extract the tenant from the message (either via a message header, or some message property) and register that in the container child lifetime scope so that subsequent resolutions from the actual consumer class (and it's dependencies) are properly matched to the tenant in the message.
In our systems, we have a TenantContext that is registered in a newly created LifetimeScope (we're using Autofac), after which we resolve the consume from the child scope, and the dependencies that use the tenant context get the proper value since it's registered as part of building the child container for the message scope.
It works extremely well, we even built up extension methods to make it easy for developers registering consumers to specify "tenant context providers" that go from a message type to the proper tenant id, which is used to build the TenantContext.
You can do similar things with activity factories in Courier routing slips (which are a specialization of a consumer).
Preamble: I'm trying to put together a proposal for what I assume to be a very common use-case, and I'd like to use Amazon's SWF and SQS to accomplish my goals. There may be other services that will better match what I'm trying to do, so if you have suggestions please feel free to throw them out.
Problem: The need at its most basic is for a client (mobile device, web server, etc.) to post a message that will be processed asynchronously without a response to the client - very basic.
The intended implementation is to for the client to post a message to a pre-determined SQS queue. At that point, the client is done. We would also have a defined SWF workflow responsible for picking up the message off the queue and (after some manipulation) placing it in a Dynamo DB - again, all fairly straightforward.
What I can't seem to figure out though, is how to trigger the workflow to start. From what I've been reading a workflow isn't meant to be an indefinite process. It has a start, a middle, and an end. According to the SWF documentation, a workflow can run for no longer than a year (Setting Timeout Values in SWF).
So, my question is: If I assume that a workflow represents one message-processing flow, how can I start the workflow whenever a message is posted to the SQS?
Caveat: I've looked into using SNS instead of SQS as well. This would allow me to run a server that could subscribe to SNS, and then start the workflow whenever a notification is posted. That is certainly one solution, but I'd like to avoid setting up a server for a single web service which I would then have to manage / scale according to the number of messages being processed. The reason I'm looking into using SQS/SWF in the first place is to have an auto-scaling system that I don't have to worry about.
Thank you in advance.
I would create a worker process that listens to the SQS queue. Upon receiving a message it calls into SWF API to start a workflow execution. The workflow execution id should be generated based on the message content to ensure that duplicated messages do not result in duplicated workflows.
You can use AWS Lambda for this purpose . A lambda function will be invoked by SQS event and therefore you don't have to write a queue poller explicitly . The lambda function could then make a post request to SWF to initiate the workflow
I am building an application which will send status requests to users (via email & sms) on a regular basis. I want to execute the service each hour which will:
Query the database for all requests that need to be sent (based on some logic)
Send the requests through Amazon's Simple Email Service (this is already working)
Write a record of the status request notification back to the data store
I am considering wrapping up this series of operations into a single controller with an end point that can be called remotely to kick off the process within the rails app.
Longer term, I will break this process out into an app that can be run independently of my rails app, but for now I'm just trying to keep it simple.
My first inclination is to build the following:
Controller with the following elements:
A method which will orchestrate the steps outlined above (and can be called externally)
A call to the status_request model which will bring back a collection of request needing to be sent
A loop to iterate through the pending requests, which will:
Make a call to my AWS Simple Email Service module to actually send the email, and
Make a call to the status_request model to log the request back to the database
Model:
A method on my status_request model which will bring back a collection of requests that need to be sent
A method in my status_request model which will log that a notification was sent
Since this will behave as a service that gets called periodically from an outside scheduler I don't think I'll need a view for this operation. (Will, of course, need views to show users and admins what requests have been sent, but that's later...).
As someone new to Rails, I'm asking for review of this approach and any suggestions you may have.
Thanks!
Instead of a controller which Jeff pointed out exposes a security risk, you may just want to expose a rake task and use cron to invoke it on an hourly basis.
If you are still interested in building a controller, look at devise gem and its single access token, token_authenticatable, for securing the methods you are exposing.
You may also want to look at delayed_job or resque to offload the call to status_request and the loop to AWS simple service to a background worker process.
You may want a seperate controller and view for the log file so you can review progress on demand.
And if you want to get real fancy use Amazon SNS to send you alerts when the service reaches some unacceptable level of failures, backlog, etc.
Since you are trying to invoke this from an outside process, your approach should work. You could also have a worker process that processes task when they are there.
You will need routes to expose your service, and you may want to also make security decisions. How will the service that invokes your application authenticate so all others can't hit it at will?
Another consideration should be how many emails are you sending. If there are enough, we may want to look into the fact that writing this sort of loop is going to be extremely top heavy; and may affect users on the current system if it's a web application.
In the end, there are many ways to do this. I would focus on the performance/usage you expect as well as security. There's never one perfect way to solve a problem like this, and your way should just be aware of the variables it will need to be operating within.
Resque and Redis might be helpful to you in scheduling and performing operatio n .They are simple and superfast, [here](http://railscasts.com/episodes/271-resque] is a simple tut on same.
We have an ASP.NET MVC web app which allows users to publish messages onto a web site. Alongside this, the user is also able to syndicate that message content to other 3rd party systems when they post the message.
At present, this is done synchronously, so when they click the 'Post' button, we persist their message to the database and then notify each 3rd party systems in turn. We need to improve the scalability and durability of this operation so I would like to make the notification aspect of the action asynchronous in some way.
I can think of the following possiblities
Save the 3rd party messages into a database table and have some worker process read items from the table and post to the 3rd party systems.
Use a "proper" message queue of some sort like nServiceBus or RabbitMQ (I have no experience with either of these)
Is there a better way to do this? I'm particularly interested in how to notify the user that the message has been syndicated correctly (since it's ansynchronous) and also how to handle multiple retry failures, at which point the sender should just give up.
Thanks
James
NServiceBus is a great framework for implementing asynchronous communication. If you use it for this use case, you will see many other opportunities for applying messaging for improving the scalability and reliability of your system.
Create a MessagePosted event message that is published after a message is persisted to the database. For each third party system that might be notified of the message, create an event handler class that implements IHandleMessages.
Multiple retry failures is facilitated by NServiceBus, just throw an exception within the event handler if something goes wrong. The event will be resubmitted to the event handler for a configurable number of retries before the event is moved to the error queue.
To notify the user you can for instance create a status view or widget which shows the notification results of the latest messages. If a third party system cannot be notified you can consider sending the user an e-mail so that he can take action.
Use this publish subscribe sample to get up to speed quickly: http://docs.particular.net/samples/pubsub/
You should read this: It explains how to use pub sub with RabbitMQ http://www.rabbitmq.com/tutorials/tutorial-three-java.html
What is the required IOC instance lifecycle I need to use in conjuction with a NServiceBus message handler if I want an instance per message handled.
PerRequest won't work, since given the numerous constructor dependenices and dependency graph I have, it gives me many instances when I require only one per Handle(MessageX msg) call. I can't or don't want to inject it directly into the message handler since it is required further down the object graph. E.g. inject IPersonService, depends on IPersonRepository, they can be per request (Default), but IPersonDBContext need to be per message call.
PerThread won't work since NServiceBus uses the same worker threads over and over.
Singleton, HttpContext, etc.. are obviously not applicable.
Has anyone came across this with either StructureMap or Castle?
I might be missing something here, but PerRequest will give you a new instance for each MessageHandler ( the messagehandlers are them selves registered as PerRequest). I have just committed a fix for a bug that caused messagehandlers to fire multiple times for each message. I wonder if that bug has been misleading you (try to get the lastest 2.0 build from CI and see if that does it for you)
http://teamcity.codebetter.com/viewLog.html?buildId=7087&tab=artifacts&buildTypeId=bt96
Hope this helps!