We are using quickfixj in our system via apache camel:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-quickfix</artifactId>
<version>${camel.version}</version>
</dependency>
We are now trying to troubleshoot the problem with our FIX channel and want to understand where the delay with message communication is – in our network while sending the message or on the bank side.
quickfixj has an outgoing message logging but the problem is logging is done before sending the message and the message itself then sent asynchronously
private boolean send(String messageString) {
getLog().onOutgoing(messageString); // message is logged here
...
return responder.send(messageString); // async sent inside
}
So is there any way to make some log only after the message is sent? This way we will understand if there is really some delay between the first log and actually sending the message and what time that sending takes.
Thanks.
UPD:
Based on Christoph's answer, trying the following:
val engine = (camelContext.getEndpoint(fixConfiguration.fixEndpoint) as QuickfixjEndpoint).engine
engine::class.java.getDeclaredField("acceptor").let {
it.isAccessible = true
val acceptor = it.get(engine) as SessionConnector
val loggingFilter = LoggingFilter()
loggingFilter.sessionOpenedLogLevel = LogLevel.NONE // don't log this event
loggingFilter.sessionCreatedLogLevel = LogLevel.NONE // don't log this event
acceptor.setIoFilterChainBuilder { chain -> chain.addLast("LoggingFilter", loggingFilter) }
}
I tested around a little bit and think the way to go would be to employ a org.apache.mina.filter.logging.LoggingFilter.
I assume you are using an Initiator but it will of course also work with an Acceptor.
LoggingFilter loggingFilter = new LoggingFilter();
loggingFilter.setSessionOpenedLogLevel(LogLevel.NONE); // don't log this event
loggingFilter.setSessionCreatedLogLevel(LogLevel.NONE); // don't log this event
initiator.setIoFilterChainBuilder(chain -> chain.addLast("LoggingFilter", loggingFilter));
This will create events like these in your log:
Feb 13, 2021 1:05:11 AM org.apache.mina.filter.logging.LoggingFilter log
INFO: SENT: 8=FIX.4.29=6735=A34=149=TW52=20210213-00:05:11.18456=ISLD98=0108=30141=Y10=244
...
Feb 13, 2021 1:05:11 AM org.apache.mina.filter.logging.LoggingFilter log
INFO: RECEIVED: 8=FIX.4.29=6735=A34=149=ISLD52=20210213-00:05:11.22456=TW98=0108=30141=Y10=239
Related
I have a question / problem about Persistent Session and Queuing Messages.
Here is the scenario:
I have a publisher (java server) which is publish message and I have a receiver (android client). When android client it online it gets the messages which amazing, working very well.
However, when I kill the android app and keep sending message from server and when I open android app, android does not receive previous messages.
Server side:
final Mqtt5BlockingClient client = MqttClient.builder()
.useMqttVersion5()
.serverHost(host)
.serverPort(8883)
.sslWithDefaultConfig()
.buildBlocking();
// connect to HiveMQ Cloud with TLS and username/pw
client.connectWith()
.simpleAuth()
.username(username)
.password(UTF_8.encode(password))
.applySimpleAuth()
.noSessionExpiry()
.cleanStart(false)
.send();
// This code is running every 15 sec
String now = LocalDateTime.now().toString();
String message = String.format("Hello: %s", now);
// publish a message to the topic "my/test/topic"
client.publishWith()
.topic("hasan-device/sayHello")
.payload(UTF_8.encode(message))
.retain(true)
.qos(MqttQos.AT_LEAST_ONCE)
.noMessageExpiry()
.send();
Client side:
// create an MQTT client
final Mqtt5BlockingClient client = MqttClient.builder()
.identifier("my-device-1")
.useMqttVersion5()
.serverHost(host)
.serverPort(8883)
.sslWithDefaultConfig()
.automaticReconnectWithDefaultConfig()
.buildBlocking();
// connect to HiveMQ Cloud with TLS and username/pw
client.connectWith()
.simpleAuth()
.username(username)
.password(UTF_8.encode(password))
.applySimpleAuth()
.noSessionExpiry()
.cleanStart(false)
.send();
// subscribe to the topic "my/test/topic"
client.subscribeWith()
.topicFilter("hasan-device/sayHello")
.retainHandling(Mqtt5RetainHandling.SEND)
.send();
// set a callback that is called when a message is received (using the async API style)
client.toAsync().publishes(ALL, publish -> {
byte[] message = publish.getPayloadAsBytes();
LOGGER.info("Received message: {} -> {}, ", publish.getTopic(), new String(message, UTF_8));
});
Expecting to message arrive when device back to online
When the Android app restarts with the persistent session, brokers will send down pending messages immediately. This can happen before the application callbacks get initialised.
Here is an example from when I did some testing with this:
To fix, move this bit of code to execute just before the connectWith call:
// set a callback that is called when a message is received (using the async API style)
client.toAsync().publishes(ALL, publish -> {
byte[] message = publish.getPayloadAsBytes();
LOGGER.info("Received message: {} -> {}, ", publish.getTopic(), new String(message, UTF_8));
});
I have developed a Quarkus app with which I want to receive and process MQTT messages.
This also works so far.
My problem is that when the internet goes down at the MQTT broker and the app reconnects afterwards, the app reconnects to the broker but no messages are received. I think that the "subscribe" method is not called anymore.
How can I solve this problem?
Here is my Config:
mp.messaging.incoming.smarthome/electricity.connector=smallrye-mqtt
mp.messaging.incoming.smarthome/electricity.host=192.168.1.88
mp.messaging.incoming.smarthome/electricity.port=1883
mp.messaging.incoming.smarthome/electricity.reconnect-attempts=3000
mp.messaging.incoming.smarthome/electricity.reconnect-interval-seconds=10
mp.messaging.incoming.smarthome/electricity.qos=1
mp.messaging.incoming.smarthome/electricity.failure-strategy=ignore
Here is my Controller:
#Incoming("smarthome/electricity")
public void consume(byte[] raw) {
String price = new String(raw,StandardCharsets.UTF_8);
String[] parts = price.split(",");
String watt = parts[0].trim();
String timeStamp = parts[1].trim();
byte wattH = Byte.parseByte(watt.replace("WH", ""));
ZonedDateTime now = ZonedDateTime.now(ZoneId.of("Europe/Vienna"))
.withHour(Integer.parseInt(timeStamp.split(":")[0]))
.withMinute(Integer.parseInt(timeStamp.split(":")[1]));
Message message = new Message(wattH,now);
System.out.println(message);
service.addToPackage(message);
scheudler.check();
}
Stack Output if i cut the Connection:
2022-09-20 07:50:09,683 ERROR [io.sma.rea.mes.mqtt] (vert.x-eventloop-thread-0) SRMSG17105: Unable to establish a connection with the MQTT broker: java.net.SocketException: Connection reset
If the Connection is back:
2022-09-20 07:50:26,751 INFO [io.ver.mqt.imp.MqttClientImpl] (vert.x-eventloop-thread Connection with 192.168.1.88:1883 established successfully
So the connection seems to be back, but there are no more incoming messages.
I solved the Problem by myself.
I set :
quarkus.arc.remove-unused-beans=none
And now it works fine.
I tried many ways to fix the problem, but this seems to be the issue.
I think there is some bean removed in the runtime when the connection is lost for a too long time.
If anyone can explain why this happens please tell me
I am building a slack application that will schedule a message when someone posts a specific type of workflow in a channel.
It will schedule a message, and if someone from a specific group of users replies before it has sent, it will delete the scheduled message.
Unfortuantely these messages are still sending, even though the list of scheduled messages is empty and the response when deleting the message is a successful one. I am also deleting the message within the 60 second limit that is noted on the API.
Scheduling the message gives me a success response, and if I use the list scheduled messages I get:
[
{
id: 'MESSAGE_ID',
channel_id: 'CHANNEL_ID',
post_at: 1620428096, // 2 minutes in the future for testing
date_created: 1620428026,
text: 'thread_ts: 1620428024.001300'
}
]
Canceling the message:
async function cancelScheduledMessage(scheduled_message_id) {
const response = await slackApi.post("/chat.deleteScheduledMessage", {
channel: SLACK_CHANNEL,
scheduled_message_id
})
return response.data
}
response.data returns { "ok": true }
If I use the list scheduled message API to retrieve what is scheduled I get an empty array []
However, the message will still send to the thread.
Is there something I am missing? I have the proper scopes set up and the API calls appear to be working.
If it helps, I am using AWS Lambda, and DynamoDB to store/retrieve the thread_ts and message IDs.
Thanks all.
For messages due in 5 minutes or less, chat.deleteScheduleMessage has a bug (as of November 2021) [1]. Although this API call may return OK, the actual message will still be delivered due to the bug.
Note that for messages within 60 seconds, this API does return an proper error code, as described in the documentation [2]. For the range (60 seconds, ~5 minutes), the API call returns OK but fails behind the scenes.
Before this bug is fixed, the only thing one can do is to only delete messages scheduled 5 minutes (the exact threshold may vary, according to Slack) or more (yes not very ideal and may not be feasible for some applications).
[1] Private communication with Slack support.
[2] https://api.slack.com/methods/chat.deleteScheduledMessage
Our application is using org.springframework.cloud, spring-cloud-starter-stream-rabbit framework and we are trying to avoid sending specific messages to DLQ and also retrying them, this behaviour should be somehow dynamically, because, for the default messages, retries and DLQ should work.
According to this documentation:
Putting it All Together
And this useful post:
DLX in rabbitmq and spring-rabbitmq - some considerations of rejecting messages
It seems that ImmediateAcknowledgeAmqpException could be used in spring AMQP to mark a message as acknowledged and no further process it. However, when we use this code:
#StreamListener(LogSink.INPUT)
public void handle(Message<Map<String, Object>> message) {
if (message.getPayload().get("condition1").equals("abort")) {
throw new ImmediateAcknowledgeAmqpException("error, we don't want to send this message to DLQ");
}
...
}
The message is always send to DLQ
Our current configuration:
spring.cloud.stream:
bindings:
log:
consumer.concurrency: 10
destination: log
group: myGroup
content-type: application/json
rabbit.bindings:
log:
consumer:
autoBindDlq: true
republishToDlq: true
transacted: true
Are we missing something? Is there any other alternative to avoid publishing to DLQ and requeuing?
republishToDlq does not look at that exception; it only applies when the exception is thrown to the container (method causeChainHasImmediateAcknowledgeAmqpException()).
Republishing subverts that logic since no exception is thrown to the container.
Please open an issue against the Rabbit binder, republishToDlq should honor that exception and discard the failed message.
I've been searching Google for awhile and there seems to be no offers on fixing this problem I have here.
I am using LuaSocket as a simple way to connect to a external server I created, and I am able to connect to it successfully and send a signal.
However, when I try to send a second message later on, the external server does not seem to be receiving the message, even though I am still connected to the socket.
socket = require("socket")
host, port = ip, port
tcp = assert(socket.tcp())
tcp:settimeout( 0 )
tcp:connect(host, port);
msg = {
["status"]="connect",
["usrName"]=username
}
msg = Json.Encode(msg)
tcp:send(msg); -- This message, the server received this message.
-- Later in my code, I attempt to send another message.
msg = {
["status"]="anotherMessage",
["usrName"]=username
};
msg = Json.Encode(msg)
tcp:send(msg); -- This message is not sending, even though i'm still connected.
You need to show what happens on the other side as it may be simply not reading even though the connection may be open. You also don't say what exactly happens when "message is not sending"; do you get an error? the script finishes but the message is not sent?
There are several things you can try:
Switch to the (default) synchronous send until you get it working; remove tcp:settimeout(0), as your send may simply fail with "timeout" message if the other side is not ready to read the message.
Check the error message from :send call to see if it's timing out or not.
local ok, err = tcp:send(msg)
Use socket.select to check if the other side it ready to accept the message you are sending.
Try adding "\r\n" at the end of your serialized JSON.