I have an SMS app that sends the SMS message to be sent to a sidekiq worker, which then pings twilio to actually send the message. The problem I'm running into is that by sending the messages to a worker, sometimes messages above 160 characters get sent in the wrong order. I assume this is because sidekiq is running the jobs concurrently. How do I solve this problem?
Previously, I would cycle through each 160 characters of a message and send each 160 character string to a worker to be sent. This caused issues because the workers would get setup and run concurrently to the messages were out of order. To solve this, I moved the 160 character logic into the worker, which I believe solved the issue of a single message.
However, if multiple messages come through within 1-2 seconds, they get sent concurrently so it's possible it will be out of order again. How do I make sure sidekiq processes the messages in the order I call the perform_async method? Here's my code:
//messages_controller.rb
SendSMSWorker.new.perform(customer.id, message_text, 'sent', false, true)
//send_sms_worker.rb
def perform(customer_id, message_text, direction, viewed, via_api)
customer = Customer.find(customer_id)
company = customer.company
texts = message_text.scan(/.{1,160}/) # split the messages up into 160 char pieces
texts.each do |text|
message = customer.messages.new(
user_id: company.admin.id, # set the user_id to the admin's ID when using the api
company_id: company.id,
text: text,
direction: 'sent',
viewed: false,
via_api: true
)
# send_sms returns nil if successful
if error = message.send_sms
customer.mark_as_invalid! if error.code == 21211
else
# only save the message if the SMS was successfully sent
puts "API_SEND_MESSAGE company_id: #{company.id}, customer_id: #{customer.id}, message_id: #{message.id}, texts_count: #{texts.count}"
message.save
Helper.publish(company.admin, message.create_json_with_extra_attributes(true))
end
end
end
to be clear, the message.send_sms is the method on the message model that actually sends the sms via twilio. thanks!
If you're sending multiple messages, each message takes it's own route to the destination carrier. Even if they're sent in the correct sequence there isn't a guarantee they'll be received at the handset in the correct order. A way to overcome this is using concatenated messages up to 1600 characters (in the US). If you send a long message via the Messages resource it will be received as a single long message. Just make sure you're using the Messages resource:
#client.account.messages.create()
instead of
client.account.sms.messages.create()
You can read more here:
https://www.twilio.com/help/faq/sms/does-twilio-support-concatenated-sms-messages-or-messages-over-160-characters
http://twilio-ruby.readthedocs.org/en/latest/usage/messages.html
Related
I am building a slack application that will schedule a message when someone posts a specific type of workflow in a channel.
It will schedule a message, and if someone from a specific group of users replies before it has sent, it will delete the scheduled message.
Unfortuantely these messages are still sending, even though the list of scheduled messages is empty and the response when deleting the message is a successful one. I am also deleting the message within the 60 second limit that is noted on the API.
Scheduling the message gives me a success response, and if I use the list scheduled messages I get:
[
{
id: 'MESSAGE_ID',
channel_id: 'CHANNEL_ID',
post_at: 1620428096, // 2 minutes in the future for testing
date_created: 1620428026,
text: 'thread_ts: 1620428024.001300'
}
]
Canceling the message:
async function cancelScheduledMessage(scheduled_message_id) {
const response = await slackApi.post("/chat.deleteScheduledMessage", {
channel: SLACK_CHANNEL,
scheduled_message_id
})
return response.data
}
response.data returns { "ok": true }
If I use the list scheduled message API to retrieve what is scheduled I get an empty array []
However, the message will still send to the thread.
Is there something I am missing? I have the proper scopes set up and the API calls appear to be working.
If it helps, I am using AWS Lambda, and DynamoDB to store/retrieve the thread_ts and message IDs.
Thanks all.
For messages due in 5 minutes or less, chat.deleteScheduleMessage has a bug (as of November 2021) [1]. Although this API call may return OK, the actual message will still be delivered due to the bug.
Note that for messages within 60 seconds, this API does return an proper error code, as described in the documentation [2]. For the range (60 seconds, ~5 minutes), the API call returns OK but fails behind the scenes.
Before this bug is fixed, the only thing one can do is to only delete messages scheduled 5 minutes (the exact threshold may vary, according to Slack) or more (yes not very ideal and may not be feasible for some applications).
[1] Private communication with Slack support.
[2] https://api.slack.com/methods/chat.deleteScheduledMessage
I'm making this addons that have to send to the raid my interrupt cooldown.
The problem is that whenever i send a message to the raid i am the only one that receive it.
This is the code that send the message:
C_ChatInfo.SendAddonMessage("KickRotation",string.format( "%0.2f",remainingCd ), "RAID")
This is the event handler:
frame:RegisterEvent("PLAYER_ENTERING_WORLD")
frame:RegisterEvent("CHAT_MSG_ADDON")
frame:SetScript("OnEvent", function(self, event, ...)
local prefix, msg, msgType, sender = ...;
if event == "CHAT_MSG_ADDON" then
if prefix == "KickRotation" then
print("[KickRotation]" ..tostring(sender) .." potrĂ interrompere tra: " ..msg);
end
end
if event == "PLAYER_ENTERING_WORLD" then
print("[KickRotation] v0.1 by Galfrad")
end
end)
Basically when the message is sended it is printed only to me.
Network messages are handled and transferred to the recipient channel (in this case, Raid Group) by the server. The reason that you are seeing the message locally, but the other people do not see it is that the message will be handled on the local system (sender) to reduce the repetition of data transmit.
Server however, only accepts and sends messages that are registered to it.
Therefore, you must first register your add-on messages to the server so the other players in the requested channel be able to receive it.
First, register your add-on messages with the name you have given already (But be sure to call the registration method only once per client):
local success = C_ChatInfo.RegisterAddonMessagePrefix("KickRotation") -- Addon name.
Next, check if your message was accepted and registered to the server. In case success is set to false (failure), you may want to handle proper warning messages and notifications to the user. The case of failure means that either server has disabled add-on messages or you have reached the limit of add-on message registrations.
Finally, send your message again check if it did not fail.
if not C_ChatInfo.SendAddonMessage("KickRotation",string.format( "%0.2f",remainingCd ), "RAID") then
print("[KickRotation] Failed to send add-on message, message rejected by the server.")
end
I have an AWS SQS FIFO queue configured to deduplicate messages based on content. My rails app uses Shoryuken worker to get messages from SQS. Here is the worker code:
class MyJob
include Shoryuken::Worker
shoryuken_options queue: "myjobs-#{ENV['RAILS_ENV']}.fifo",
auto_delete: true,
body_parser: JSON
def perform(message_meta, message_body)
# do stuff
end
end
As you can see, it's configured to automatically delete messages from queue, once received. But today something strange happened. I noticed that the worker performs a large number of identical tasks. When I opened the SQS Queue in AWS Console, I saw there was a message in it, which looked it was received multiple times by the worker. Here are its attributes, notice the Receive Count:
Message ID: 9207017f-ad15-4de8-97c4-cf391c8f3840
Size: 1.3 KB
MD5 of Body: 55918bf431e31e4badae0720453aea35
Sent: 2018-12-11 10:40:53.978 GMT-08:00
First Received: 2018-12-11 10:40:54.045 GMT-08:00
Receive Count: 2654
Message Attribute Count: 0
Message Group ID: default Message
Deduplication ID: c5fb9acda5e3c9c82dc0ae3f0b1cff5bd7067d0cf942075c4c38dddd1fbc1ed1
Sequence Number: 37288893882837472512
Any idea how that could happen?
Platform details: Ubuntu, ruby 2.5.3, Rails: 5.2.2, Shoryuken: 4.0.2
Turns out, the problem was with the queue's VisibilityTimeout setting. By default it is set to 30 seconds, but often messages would arrive to the receiver side outside of the allowed 30 seconds, and this would mean that Shoryuken would fail to delete the received message from the queue with the following error:
ERROR: Could not delete 0, code: 'ReceiptHandleIsInvalid', message:
'The receipt handle has expired', sender_fault: true
The solution is to increase the VisibilityTimeout. I set it to the maximum allowed 12 hours, and that resolved the issue.
More about VisibilityTimeout:
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
The thread that put me on the right track:
https://github.com/aws/aws-sdk-java/issues/705
I am developing messenger IOS app based on Firebase Realtime Database.
I want that all messages to be ordered based on timestamp.
There is a scenario as like below.
There are 3 clients. A, B and C.
1)
All clients register 'figure-1' listener to receive messages from others.
<figure-1>
ref.queryOrdered(byChild: "timestamp").queryStarting(atValue: startTime).observe(.childAdded, with:
{
....
// do work for the messages, print, save to storage, etc.
....
// save startTime to storage for next open.
startTime = max(timeOfSnapshot, startTime)
saveToStorage(startTime)
}
2)
Client A write message 1 to server with ServerValue.timestamp().
Client B write message 2 to server with ServerValue.timestamp().
Client C write message 3 to server with ServerValue.timestamp().
They sent messages extremely the same moment.
All clients have good speed wifi.
So, finally. Server data saved like 'figure-2'
<figure-2>
text : "Message 1", timestamp : 100000001
text : "Message 2", timestamp : 100000002
text : "Message 3", timestamp : 100000003
As my listener's code, i keep messages on storage and next listening timestamp for preventing downloading duplicated messages.
In this case.
Does Firebase always guarantee to trigger callback in order as like below?
Message 1
Message 2
Message 3
If it is not guaranteed, my strategy is absolutely wrong.
For example, some client received messages as like below.
Message 3 // the highest timestamp.
// app crash or out of storage
Message 1
Message 2
The client do not have chance to get message 1, 2 anymore.
I think if there are some nodes already, Firebase might trigger in order for those. Because, that is role of 'queryOrdered' functionality.
However, there are no node before register the listener and added new nodes additionally after then. What is will happen?
I suppose Firebase might send 3 packets to clients. (No matter how quickly the message arrives, Firebase has to send it out as soon as it arrives.)
Packet1 for message1
Packet2 for message2
Packet3 for message3
ClientA fail to receive for packet 1,2
ClientA success to receive for packet 3
Firebase re-send packet 1,2 again.
ClientA success to receive for packet 1,2
Eventually, all datas are consistent. But ordering is corrupted.
Does Firebase guarantee to occur events in order?
I have searched stack overflow and google and read official documents many times. However, i could not find the clear answer.
I have almost spent one week for this. Please give me piece of advice.
The order in which the data for a query is returns is consistent, and determined by the server. So all clients are guaranteed to get the results in the same order.
For new data that is sent to the database after the listeners are attached, all remote clients will receive it in the same order. The local client will see events for it's write operation right away though, before the data even reaches the database server.
In figure 2, it is actually quite simple: since each node has a unique timestamp, and they will be returned in the order of that timestamp. But even if they'd have the same timestamp, they'd be returned in the same order (timestamp first, then key) for each client.
Remote iOS client successfully connects to me, send subscribe command (it works fine), but on "unsubscribe" command I get next error:
Unsubscribing from channel: {"channel":"Assessor::StationChannel", "station_id": 1}
Could not execute command from {"command"=>"unsubscribe", "identifier"=>"{\"channel\":\"Assessor::StationChannel\", \"station_id\": 1}"}) [NoMethodError - undefined method `unsubscribe_from_channel' for nil:NilClass]: /app/vendor/bundle/ruby/2.2.0/gems/actioncable-5.0.0/lib/action_cable/connection/subscriptions.rb:44:in `remove_subscription' | /app/vendor/bundle/ruby/2.2.0/gems/actioncable-5.0.0/lib/action_cable/connection/subscriptions.rb:40:in `remove' | /app/vendor/bundle/ruby/2.2.0/gems/actioncable-5.0.0/lib/action_cable/connection/subscriptions.rb:16:in `execute_command' | /app/vendor/bundle/ruby/2.2.0/gems/actioncable-5.0.0/lib/action_cable/connection/base.rb:88:in `dispatch_websocket_message' | /app/vendor/bundle/ruby/2.2.0/gems/actioncable-5.0.0/lib/action_cable/server/worker.rb:58:in `block in invoke'
Subscribe message format:
{"command": "subscribe", "identifier": "{\"channel\":\"Assessor::StationChannel\", \"station_id\": 1}"}
Unsubscribe message format:
{"command": "unsubscribe", "identifier": "{\"channel\":\"Assessor::StationChannel\", \"station_id\": 1}"}
I cannot reproduce this problem on localhost, so maybe somebody can help me?
I saw a similar error. I was trying to unsubscribe via the client (JS). I eventually figured out it was because the javascript to .remove(subscription) takes the subscription and not the subscription identifier.
This is how I got it to work without error. Perhaps it will help you find out why you are getting the error from the server side.
subscription = App.cable.subscriptions.subscriptions[0]
App.cable.subscriptions.remove(subscription);
(Note, I'm just pulling the first subscription from the array, TODO: Search for the subscription I want to remove)
Here is the bug I was seeing and how I eventually found the source code/answer. I ran these from the webclient console:
App.cable.subscriptions.create({channel: "RoomChannel", room_id: 2})
That line works and I get a "... is transmitting the subscription confirmation" on stdout for rails s
App.cable.subscriptions.remove({channel: "RoomChannel", room_id: 2})
That line blows up, yells at my kids, and insults my wife which looks like:
[NoMethodError - undefined method `unsubscribe_from_channel' for nil:NilClass]: /usr/local/lib64/ruby/gems/2.3.0/gems/actioncable-5.0.0.1/lib/action_cable/connection/subscriptions.rb:44:in `remove_subscription'
I also noted the following line before the crash.
Unsubscribing from channel:
The code to produce that is: logger.info "Unsubscribing from channel: #{data['identifier']}". Which means it wasn't finding the data['identifier']. So I started debugging and I see that line 88 of base.rb in actioncable only gets {"command":"unsubscribe"} and not something like {"command":"unsubscribe", "identifier":" (channel name here)}
Which brought me to action_cable.js. (I would have started here, but I hate JS.). Here was my problem: function(subscription). I was sending the identifier and not the subscription object.
Subscriptions.prototype.remove = function(subscription) {
this.forget(subscription);
if (!this.findAll(subscription.identifier).length) {
this.sendCommand(subscription, "unsubscribe");
}
return subscription;
};
App.cable.subscriptions.create({channel: "RoomChannel", room_id: 2}) returns a subscription object you have to pass that into the remove function
var subscription = App.cable.subscriptions.create({channel: "RoomChannel", room_id: 2});
Then later
App.cable.subscriptions.remove(subscription);
After several tries I eventually figured it out. Late reply but this worked for me hope it does for you.
App["identifier"].disconnected() #e.g App["chat_4"].disconnect()
=> YourChannel stopped streaming from chat_4 #rails terminal
Above is the line to stop the streaming and below is how you subscribe to the channel
App['chat' + id] = App.cable.subscriptions.create({channel:
'YourChannel', chat_id: id}, {
disconnected: function () {
App.cable.subscriptions.remove(this)
},
)}
I know this is an old question, but I suffered from the same issue. None of the answers above worked for me and in fact, at least in Rails 5.0.1, they are incorrect. It is a frontend issue all right, but here's why it doesn't matter whether you call App.yourChannelName.unsubscribe() or App.yourChannelName.remove(App.yourChannelName)
Eg. if you have something like this (example code is in coffeescript, so ignore the lack of vars and other stuff from vanilla JS):
App.yourChannel = App.cable.subscriptions.create({channel: 'YourChannel', id: id})
...
// do stuff, execute callbacks, whatnot
...
// try to execute unsubscribe
App.yourChannel.unsubscribe()
The .unsubscribe() is a method on Subscription prototype which only does return this.consumer.subscriptions.remove(this)
The this.consumer.subscriptions returns the instance of your subscription, in example above, it would be App.yourChannel and calls .remove method with the instance of Subscription - ie. with App.yourChannel
So App.yourChannel.unsubscribe() is the same as calling App.cable.subscriptions.remove(App.yourChannel) (or whichever variable you choose to store the instance of Subscription in.
I have also been seeing the same error as OP, except that in my case, it was caused by App.yourChannel.unsubscribe() being called two times - the first time it was called immediately after I received specific data via the channel and the second time was due to a custom cleanup being run in a specific scenario before the App.yourChannel was re-subscribed.
So if you see a similar error, I suggest you look at the server logs.
You'll probably see something like
Registered connection (some-id) <-- initial subscription
YourChannel is transmitting the subscription confirmation
...
// other stuff while user is subscribed to the channel
...
Unsubscribing from channel: {"channel":"YourChannel","id":"some-id"} <-- initial unsubscribe call
YourChannel stopped streaming from your_channel_some-id
// some other requests potentially, perhaps some DB queries
...
// there are no requests to subscribe to the channel with the some-id, eg. you won't see this
// Registered connection (some-id)
// YourChannel is transmitting the subscription confirmation
Unsubscribing from channel: {"channel":"YourChannel","id":"some-id"} <-- duplicated unsubscribe call
Could not execute command from {"command"=>"unsubscribe", "identifier"=>"{\"channel\":\"YourChannel\",\"id\":\"some-id\"}"}) [NoMethodError - undefined method `unsubscribe_from_channel' for nil:NilClass]:
Basically, the user subscribes, unsubscribes, then tries to unsubscribe again (even though they are not subscribed to that channel anymore)