Sniffing the packets transmitted for a simple echo service Python gRPC example shows that there's 17 packets transmitted for a single message exchange (including SYNs) if the channel is new and 6-8 packets when a channel is reused.
Here's the example protofile:
// The greeting service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
And here's the Python code to run it:
import grpc
import helloworld_pb2
import helloworld_pb2_grpc
channel = grpc.insecure_channel('localhost:50051')
stub = helloworld_pb2_grpc.GreeterStub(channel)
# Try with a brand new channel first
print(stub.SayHello(helloworld_pb2.HelloRequest(name='you'))) # 17 packets
# Let's reuse the channel this time
print(stub.SayHello(helloworld_pb2.HelloRequest(name='you'))) # 6-8 packets; still too much :-(
A Wireshark capture shows that all the packets be between 66 and 176 bytes long, so there's obviously no need to transmit a huge amount of data in total. They could well fit into 1 for each request and response (totalling to 2 packets), but for some reason I'm consistently seeing them fragmented like this (note: testing on loopback, Linux 5.4).
Is there any way to reduce the number of packets?
Related
I borrowed code from Toms Hardware on how to use MQTT and subscribe. JRMI is the publisher of the messages and it keeps repeating them over and over again. Is there anyway to have the message sent only once? I dont have this problem when I subscribe to MQTT via http://www.hivemq.com/demos/websocket-client/ The MQTT service I'm using is broker.hivemq.com
For those not familiar with JRMI, it is the JAVA program that model railroads use to control tracks,lighting, DCC etc. Ref: https://www.jmri.org/
The link to Tom's is here https://www.tomshardware.com/how-to/send-and-receive-data-raspberry-pi-pico-w-mqtt
The code adapted from Tom's is
import network
import time
from machine import Pin
from umqtt.simple import MQTTClient
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect("whatever","pwd")
time.sleep(5)
print(wlan.isconnected())
mqtt_server = 'broker.hivemq.com'
client_id = 'bigles'
topic_sub = b'/trains/track/turnout/#'
def sub_cb(topic, msg):
print("New message on topic {}".format(topic.decode('utf-8')))
msg = msg.decode('utf-8')
print(msg)
def mqtt_connect():
client = MQTTClient(client_id, mqtt_server, keepalive=60)
client.set_callback(sub_cb)
client.connect()
print('Connected to %s MQTT Broker'%(mqtt_server))
return client
def reconnect():
print('Failed to connect to MQTT Broker. Reconnecting...')
time.sleep(5)
machine.reset()
try:
client = mqtt_connect()
except OSError as e:
reconnect()
while True:
client.subscribe(topic_sub)
time.sleep(1)
The setup inside JRMI for MQTT (edit->preferences) is as follows:
JMRI, by default, publishes with "the retain option on". When you subscribe to a topic the broker will send you the most recent (if any) retained message. This occurs even if you already had an identical subscription as per the MQTT Spec:
If a Server receives a SUBSCRIBE Packet containing a Topic Filter that is identical to an existing Subscription’s Topic Filter then it MUST completely replace that existing Subscription with a new Subscription. The Topic Filter in the new Subscription will be identical to that in the previous Subscription, although its maximum QoS value could be different. Any existing retained messages matching the Topic Filter MUST be re-sent, but the flow of publications MUST NOT be interrupted [MQTT-3.8.4-3].
In your code you are calling Subscribe in a loop:
while True:
client.subscribe(topic_sub)
time.sleep(1)
To avoid the repeated messages move the subscribe out of the loop (you only need to subscribe once!). Something like the following (simplified!) code:
client = mqtt_connect()
client.subscribe(topic_sub)
while True:
client.wait_msg() // Use client.check_msg() if you have other stuff to do
I'm using Twilio Programmable Voice to process phone calls.
I want to use bi-directional stream feature to send some raw audio data to play by twilio, the initialization code looks like,
from twilio.twiml.voice_response import Connect, VoiceResponse, Stream
response = VoiceResponse()
connect = Connect()
connect.stream(url='wss://mystream.ngrok.io/audiostream')
response.append(connect)
Then when got wss connection from twilio, I start to send raw audio data to twilio, like this
async def send_raw_audio(self, ws, stream_sid):
print('send raw audio')
import base64
import json
with open('test.wav', 'rb') as wav:
while True:
frame_data = wav.read(1024)
if len(frame_data) == 0:
print('no more data')
break
base64_data = base64.b64encode(frame_data).decode('utf-8')
print('send base64 data')
media_data = {
"event": "media",
"streamSid": stream_sid,
"media": {
"playload": base64_data
}
}
media = json.dumps(media_data)
print(f"media: {media}")
await ws.send(media)
print('finished sending')
test.wav is a wav file encoded audio/x-mulaw with a sample rate of 8000.
But when run, I can't hear anything, and on twilio console, it said
31951 - Stream - Protocol - Invalid Message
Possible Causes
- Message does not have JSON format
- Unknown message type
- Missing or extra field in message
- Wrong Stream SID used in message
I have no idea which part is wrong. Does anyone know what's my problem? I can't find an example about this scenario, just follow instructions here, really appreciate it if someone knows there is an example about this, thanks.
Not sure if this will fix it but I use .decode("ascii"), not "utf-8"
Question is probably not relevant anymore, but I came across this while debugging my bi-directional stream, so it might be useful for someone:
Main reason why were you receiving this error because of the typo in json content. You are sending "playload" instead of "payload".
Another issue when sending data to twilio stream is that you should send mark message at the end of data stream to notify twilio that complete payload was sent. https://www.twilio.com/docs/voice/twiml/stream#message-mark-to-twilio
When sending data back to twilio stream, be aware that payload should not contain audio file type header bytes, so make sure you remove them from your recording or alternatively skip them while sending data to twilio.
Using the twilio-ruby package to connect to the REST API for Twilio's IP Messaging service and attempting to compute an unread message count.
The REST API is paginating the messages so that something like
channel.messages.list.last.index
Will return 49 once there are more than 50 messages in the channel.
Is there a way to get just the last message on the channel (as seems to be possible in the android/ios SDK) to avoid paginating through all message history?
In regards to computing an unread message count, take a look at the Message Consumption Horizon and subtract the lastConsumedMessageIndex from the total number of messages in the list - 1.
For the messages list (in Python):
https://www.twilio.com/docs/api/ip-messaging/rest/messages#list-all-messages
# Download the Python helper library from twilio.com/docs/python/install
from twilio.rest.ip_messaging import TwilioIpMessagingClient
# Your Account Sid and Auth Token from twilio.com/user/account
account = "ACCOUNT_SID"
token = "AUTH_TOKEN"
client = TwilioIpMessagingClient(account, token)
service = client.services.get(sid="SERVICE_SID")
channel = service.channels.get(sid="CHANNEL_ID")
messages = channel.messages.list()
See also, Sending a Consumption Report (the example in JavaScript):
//determine the newest message index
var newestMessageIndex = activeChannel.messages.length ?
activeChannel.messages[activeChannel.messages.length-1].index : 0;
//check if we we need to set the consumption horizon
if (activeChannel.lastConsumedMessageIndex !== newestMessageIndex) {
activeChannel.updateLastConsumedMessageIndex(newestMessageIndex);
}
I have build a socket to transfer message between client and server on IOS.
if(CFReadStreamSetClient(readStream, registeredEvents, readCallBack, &myContext))
{
CFReadStreamScheduleWithRunLoop(readStream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
}
if (!CFReadStreamOpen(readStream)) {
CCLog("Error Open Read Stream");
/* error handling */
}
and readCallBack function
void readCallBack(CFReadStreamRef stream, CFStreamEventType eventType, void *clientCallBackInfo)
{
switch(eventType) {
case kCFStreamEventHasBytesAvailable:{
UInt8 bufr[10240];
int bytesRead = CFReadStreamRead(stream, bufr, sizeof(bufr));
if(bytesRead >0 ){
NSLog(#"Read: %d", bytesRead);
}
break;
}
case kCFStreamEventErrorOccurred:
NSLog(#"A Read Stream Error Has Occurred!");
case kCFStreamEventEndEncountered:
NSLog(#"A Read Stream Event End!");
default:
break;
}
}
But when client send multi message to server by multi time.
Server always read it as one message.
Example:
Client send message 1st: Message1
Client send message 2nd: Message2
But when server read message from client:
Result is: Message1Message2
How can i split it as 2 messages. (I don't know the size of each message)
Thanks.
You have to make up a protocol of your own. For example, clients can append \n to each message so that the server can split messages by \n. However if your messages can have \n character in them, you can modify your protocol to first send the length of the message, again split by \n:
Client sends: 8\nMessage1
Client sends: 14\nAnotherMessage
Server receives: 8\nMessage114\nAnotherMessage
So you read up to first \n and get the content length. Then you read that many characters.
Be careful with the difference between byte streams and text streams though. You can google about TCP text streams to learn more about them. Your best bet is to send number of bytes being sent, instead of number of characters.
And be aware that sometimes, you will not receive a message as a whole. For example the following is possible:
Client sends: 8\nMessage1
Client sends: 14\nAnotherMessage
Server receives: 8\nMessage11
Server receives: 4\nAnotherMessage
I have code
DatagramConnection _dc =(DatagramConnection)Connector.open("sms://");
Datagram d = _dc.newDatagram(_dc.getMaximumLength());
_dc.receive(d); //Receive the sms
byte[] bytes = d.getData();
String address = d.getAddress(); //The address of the sms is put on a string.
String msg = new String(bytes);
Does the above code listen for incoming SMS's on a continuous basis, or does it just listen for 1 sms?
If it just listens for 1 SMS can you please provide me with code to listen for SMS's on a continuous basis.
Your code reads only a single SMS. If you need to read every SMS delivered, you need a loop like this one posted in the official knowledge base article:
DatagramConnection _dc = (DatagramConnection)Connector.open("sms://");
for(;;) {
Datagram d = _dc.newDatagram(_dc.getMaximumLength());
_dc.receive(d);
byte[] bytes = d.getData();
String address = d.getAddress();
String msg = new String(bytes);
System.out.println( "Received SMS text from " + address + " : " + msg);
}
Now a question arises: Is the BB OS delivering SMSs to listening apps in a serial way? If it were (I think so but never tested to that extent), then all you need to do is to forward the message as quickly as possible to a consumer (otherwise you would be hogging the connection listener thread during the SMS processing).