I'm having some issues configuring MassTransit with SQS. My goal is to have N consumers which create N queues and each of them accept a different message type. Since I always have a 1 to 1 consumer to message mapping, I'm not interested in having any sort of fan-out behaviour. So publishing a message of type T should publish it directly to that queue. How exactly would I configure that? This is what I have so far:
services.AddMassTransit(x =>
{
x.AddConsumers(Assembly.GetEntryAssembly());
x.UsingAmazonSqs((context, cfg) =>
{
cfg.Host("aws", h =>
{
h.AccessKey(mtSettings.AccessKey);
h.SecretKey(mtSettings.SecretKey);
h.Scope($"{mtSettings.Environment}", true);
var sqsConfig = new AmazonSQSConfig() { RegionEndpoint = RegionEndpoint.GetBySystemName(mtSettings.Region) };
h.Config(sqsConfig);
var snsConfig = new AmazonSimpleNotificationServiceConfig()
{ RegionEndpoint = RegionEndpoint.GetBySystemName(mtSettings.Region) };
h.Config(snsConfig);
});
cfg.ConfigureEndpoints(context, new BusEnvironmentNameFormatter(mtSettings.Environment));
});
});
The BusEnvironmentNameFormatter class overrides KebabCaseEndpointNameFormatter and adds the environment as a prefix, and the effect is that all the queues start with 'dev', while the h.Scope($"{mtSettings.Environment}", true) line does the same for topics.
I've tried to get this working without configuring topics at all, but I couldn't get it working without any errors. What am I missing?
The SQS docs are a bit thin, but is at actually possible to do a bus.Publish() without using sns topics or are they necessary? If it's not possible, how would I use bus.Send() without hardcoding queue names in the call?
Cheers!
Publish requires the use of topics, which in the case of SQS uses SNS.
If you want to configure the endpoints yourself, and prevent the use of topics, you'd need to:
Set ConfigureConsumeTopology = false – this prevents topics from being created and connected to the receive endpoint queue.
Set PublishFaults = false – this prevents fault topics from being created when a consumer throws an exception.
Don't call Publish, because, obviously that will create a topic.
If you want to somehow establish a convention for your receive endpoint names that aligns with your ability to send messages, you could create your own endpoint name formatter that would use message types and then use those same names to call GetSendEndpoint using the queue:name short name syntax to Send messages directly to those queues.
Related
I use gatling to send data to an ActiveMQ. The payload is generated in a separate method. The response should also be validated. However, how can I access the session data within the checks
check(bodyString.is()) or simpleCheck(...)? I have also thought about storing the current payload in a separate global variable, but I don't know if this is the right approach. My code's setup looks like this at the moment:
val scn = scenario("Example ActiveMQ Scenario")
.exec(jms("Test").requestReply
.queue(...)
.textMessage{ session => val message = createPayload(); session.set("payload", payload); message}
.check(simpleCheck{message => customCheck(message, ?????? )})) //access stored payload value, alternative: check(bodystring.is(?????)
def customCheck(m: Message, string: String) = {
// check logic goes here
}
Disclaimer: providing example in Java as you don't seem to be a Scala developper, so Java would be a better fit for you (supported since Gatling 3.7).
The way you want to do things can't possibly work.
.textMessage(session -> {
String message = createPayload();
session.set("payload", payload);
return message;
}
)
As explained in the documentation, Session is immutable, so in a function that's supposed to return the payload, you can't also return a new Session.
What you would have to do it first store the payload in the session, then fetch it:
.exec(session -> session.set("payload", createPayload()))
...
.textMessage("#{payload}")
Regarding writing your check, simpleCheck doesn't have access to the Session. You have to use check(bodyString.is()) and pass a function to is, again as explained in the documentation.
I'm building an application with microservices communicating through RabbitMQ (request-response pattern). Everything works fine but still I have a problem with error "There is no matching message handler defined in the remote service." - When I send POST to my Client app, it should simply send the message with data through client (ClientProxy) and the Consumer app should response. This functionality actually works, but always only for the second time. I know it sounds strange but on my first POST request there is always the error from Client and my every second POST request works. However this problem is everywhere in my whole application, so the particular POST request is just for the example.
Here is the code:
Client:
#Post('devices')
async pushDevices(
#Body(new ParseArrayPipe({ items: DeviceDto }))
devices: DeviceDto[]
) {
this.logger.log('Devices received');
return this.client.send(NEW_DEVICES_RECEIVED, devices)
}
Consumer:
#MessagePattern(NEW_DEVICES_RECEIVED)
async pushDevices(#Payload() devices: any, #Ctx() context: RmqContext) {
console.log('RECEIVED DEVICES');
console.log(devices);
const channel = context.getChannelRef();
const originalMsg = context.getMessage();
channel.ack(originalMsg);
return 'ANSWER';
}
Client has the RMQ settings with queueOptions: {durable: true} and the consumer as well queueOptions: {durable: true} with noAck: false
Please do you have any ideas what may causes the problem? I have tried sending the data with JSON.stringify and changing the message structure to {data: devices} but the error is still there.
I had same error and finally solve it today.
In my project, there is an api-gateway as a hybrid application to receive requests and pass data to other systems, every second request gives an error like below.
error: There is no matching message handler defined in the remote service.
Then I tried to remove the api-gateway hybrid application scope in the code below, the error is gone, hope this helps you out with this.
// api-gateway main.ts
const app = await NestFactory.create(AppModule);
// run as a hybrid app —→ remove it
app.connectMicroservice({
transport: Transport.RMQ,
noACK: false,
options: {
urls: [`amqp://${rmqUser}:${rmqPassword}#127.0.0.1:5672`],
queue: 'main_queue',
queueOptions: {
durable: false,
},
},
});
// run hybrid app
await app.startAllMicroservices(); —→ remove it
await app.listen(3000);
I solved this issue by placing the #EventPattern decorator on to a #Controller decorator method
I had this error while NOT using RabbitMQ. I found very little help online around this error message outside of it being related to RabbitMQ.
For me it was an issue where I was importing a DTO from another microservice in my microservice's Controller. I had a new DTO in my microservice that has a similar name to one in another microservice. I accidentally selected the wrong one from the automated list.
Since there wasn't any real indicator that my build was bad, just this error, I wanted to share in case others made the same mistake I did.
I encountered this same issue today and could not find any solution online and stumbled upon your question. I solved it in a hacky way and am not sure how it will behave when the application scales.
I basically added one #EventPattern (#MessagePattern in your case) in the controller of the producer microservice itself. And I called the client.emit() function twice.
So essentially the first time it gets consumed by the function that is in the producer itself and the second emit actually goes to the actual consumer.
This way only one POST call is sufficient.
Producer Controller:
#EventPattern('video-uploaded')
async test() {
return 1;
}
Producer client :
async publishEvent(data: VideosDto) {
this.client.emit('video-uploaded', data);
this.client.emit('video-uploaded', data);
}
I've experienced the same error in my another project and after some research I've found out that problem is in the way of distributing messages in RabbitMQ - named round-robin. In my first project I've solved the issue by creating a second queue, in my second project I'm using the package #golevelup/nestjs-rabbitmq instead of default NestJS library, as it is much more configurable. I recommend reading this question
I dont see in the documentation anywhere the ability to hide the chat.
https://www.twilio.com/docs/flex/flex-webchat-basic-configuration
Essentially, I want the chat to NOT show on a website if no agents are available. Is this possible?
It looks like I have to call twilio flex to get avaialble workers, and set this property accordingly.
const defaultConfiguration: Config = {
...
available: {BoolValueDependingOnAgentAvail},
Here is an example of javascript that leverages the necessary API to obtain the worker count. Based on this variable, you can set the BoolValueDependingOnAgentAvail accordingly
client.taskrouter
.workspaces('WSxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
.workers.list()
.then(workers => {
data = {
availWorkersCount: Object.keys(workers.filter(x=> x.available === true && x.attributes.includes("sales"))).length
};
I am making a worklist application using SAPUI5. The problem is that when I create an entry and then create another one right after that, I get the following error:
Default changeset implementation allows only one operation.
I checked the $batch header and I see that there is a MERGE and a POST, with the MERGE updating the previous entry for some reason. Can anyone shed some light? Could it be a backend error and not a UI5 error?
Creating the new entry:
_onMetadataLoaded: function() {
var oModel = this.getView().getModel();
var that = this;
// ...
oModel.read("/USERS_SET", {
success: function(oData) {
var oProperties = {
Qmnum: "0",
Otherstuff: "cool"
};
that._oContext = that._oView.getModel().createEntry("/ENTITYSET", {
properties: oProperties
});
that.getView().setBindingContext(that._oContext);
// ...
}
});
},
handleSavePress: function(oEvent) {
// ...
this.getView().getModel().submitChanges({
success: function(oData) {
// ...
},
error: function(oError) {
// ...
}
});
},
tl-dr: Apparently you must be using the SAP Gateway. If you do not need to process those requests in one transaction then send them in different changesets. If you do not need batch calls at all consider turning it off by supplying your model with "useBatch": false upon instantiation. However if you need to process the requests together in one transaction then you have to read the details below.
In order to understand the problem you have to understand how the gateway and the batch and changeset requests work.
Batch requests consist of multiple requests bundled together. The purpose is to open only one connection and group together relevant requests so that the overhead is minimalized. Changesets form smaller blocks inside batch requests, where modification requests can be bundled and processed together in order to ensure an all-or-nothing characteristic.
So on the gateway side: there are two relevant classes for your OData service, assuming that you have used the SAP Gateway Service Builder (SEGW transaction). There is one with the ending ...DPC and one with ...DPC_EXT. Don't touch the former, it will be always regenerated when you update your service in the service builder. The latter is the one that we will need in this example. You will have to redefine at least two methods:
/IWBEP/IF_MGW_APPL_SRV_RUNTIME~CHANGESET_BEGIN
/IWBEP/IF_MGW_APPL_SRV_RUNTIME~CHANGESET_PROCESS
By default the changeset_begin method will only allow changeset processing for changesets where the number of requests equals to one. This can be handled automatically that's why a limitation exists. If there were more requests one could not ensure their processing automatically as they could have a business dependency on each other.
So make sure to allow a bundled (deferred mode) processing of changesets under the desired conditions:
/IWBEP/IF_MGW_APPL_SRV_RUNTIME~CHANGESET_BEGIN: first call the super->/iwbep/if_mgw_appl_srv_runtime~changeset_begin method in a try catch block, then loop at it_operation_info to decide and narrow down processing only in selected cases and then allow cv_defer_mode only for the selected cases, otherwise throw a /iwbep/cx_mgw_tech_exception=>changeset_not_supported exception.
/IWBEP/IF_MGW_APPL_SRV_RUNTIME~CHANGESET_PROCESS: all requests will be available in the it_changeset_request. Make sure to fill the ct_changeset_response table with the responses.
METHOD /iwbep/if_mgw_appl_srv_runtime~changeset_process.
DATA:
lv_operation_counter TYPE i VALUE 0,
lr_context TYPE REF TO /iwbep/cl_mgw_request,
lr_entry_provider TYPE REF TO /iwbep/if_mgw_entry_provider,
lr_message_container TYPE REF TO /iwbep/if_message_container,
lr_entity_data TYPE REF TO data,
ls_context_details TYPE /iwbep/if_mgw_core_srv_runtime=>ty_s_mgw_request_context,
ls_changeset_response LIKE LINE OF ct_changeset_response.
FIELD-SYMBOLS:
<fs_ls_changeset_request> LIKE LINE OF it_changeset_request.
LOOP AT it_changeset_request ASSIGNING <fs_ls_changeset_request>.
lr_context ?= <fs_ls_changeset_request>-request_context.
lr_entry_provider = <fs_ls_changeset_request>-entry_provider.
lr_message_container = <fs_ls_changeset_request>-msg_container.
ls_context_details = lr_context->get_request_details( ).
CASE ls_context_details-target_entity.
WHEN 'SomeEntity'.
"Do the processing here
WHEN OTHERS.
ENDCASE.
ENDLOOP.
ENDMETHOD.
From the error I can tell you must be using SAP GW :-) This happens only for batch requests containing more than one create/delete/update calls and it's related to transaction security ("all or nothing"). What you have to do is redefining the corresponding GW method, I think it was CHANGESET_BEGIN. See https://archive.sap.com/discussions/thread/3562720 for some details (can't offer more for now...).
I am currently working on a client-server application using netty, some of the clients are not going to be doing anything until they recieve a message. I have read the api and can´t find a way to do so. I mean I could try to have "in.readline()" on the main so it won´t end but it Doesn´t feel right. Also could have endless loops but I don´t think its the right way either.
The question here is: is there a way to bind the socket for incoming messages just like the server having the main method ending?
public void run(){
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap()
.group(group)
.channel(NioSocketChannel.class)
.handler(new ChatClientInitializer());
Channel channel = bootstrap.connect(host,port).sync().channel();
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Inserte su nombre");
String nombre = in.readLine();
MyClientChannel canal = new MyClientChannel(channel,nombre);
canal.write("SM",nombre);
in.readLine();
See that at the end I had to write "in.readline()" so the program wouldn´t end and the handler would be still up for incomming messages
The easiest thing to do would be to replace:
in.readLine();
With:
channel.closeFuture().await();
When the connection to the server is disconnected, the client will terminate.
You will also want to spend some time defining your client's life-cycle, so that the channel's state doesn't affect when your application is running and when it's not.