websocket client packet unframe/unmask - erlang
I am trying to implement latest websocket spec. However, i am unable to get through the unmasking step post successful handshake.
I receive following web socket frame:
<<129,254,1,120,37,93,40,60,25,63,71,88,92,125,80,81,73,
51,91,1,2,53,92,72,85,103,7,19,79,60,74,94,64,47,6,83,
87,58,7,76,87,50,92,83,70,50,68,19,77,41,92,76,71,52,
70,88,2,125,90,85,65,96,15,14,20,107,31,14,28,100,27,9,
17,122,8,72,74,96,15,86,68,37,68,18,76,48,15,28,93,48,
68,6,73,60,70,91,24,122,77,82,2,125,80,81,85,45,18,74,
64,47,91,85,74,51,21,27,20,115,24,27,5,37,69,80,75,46,
18,68,72,45,88,1,2,40,90,82,31,37,69,76,85,103,80,94,
74,46,64,27,5,60,75,87,24,122,25,27,5,47,71,73,81,56,
21,27,93,48,88,76,31,57,77,74,11,55,73,68,73,115,65,81,
31,104,26,14,23,122,8,75,68,52,92,1,2,110,24,27,5,53,
71,80,65,96,15,13,2,125,75,83,75,41,77,82,81,96,15,72,
64,37,92,19,93,48,68,7,5,62,64,93,87,46,77,72,24,40,92,
90,8,101,15,28,83,56,90,1,2,108,6,13,21,122,8,82,64,42,
67,89,92,96,15,93,19,56,28,8,65,101,31,94,16,105,28,10,
20,56,30,14,65,56,27,93,71,106,16,11,17,63,25,4,17,57,
73,89,17,59,29,88,29,106,24,27,5,46,65,72,64,54,77,69,
24,122,66,93,93,49,5,12,8,109,15,28,76,59,90,93,72,56,
76,1,2,41,90,73,64,122,8,89,85,50,75,84,24,122,25,15,
23,105,25,5,19,106,26,14,20,111,25,27,5,53,77,85,66,53,
92,1,2,110,26,13,2,125,95,85,65,41,64,1,2,108,27,10,19,
122,7,2>>
As per base framing protocol defined here (https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-17#section-5.2) i have:
fin:1, rsv:0, opcode:1, mask:1, length:126
Masked application+payload data comes out to be:
<<87,58,7,76,87,50,92,83,70,50,68,19,77,41,92,76,71,52,70,88,2,125,90,85,65,96,
15,14,20,107,31,14,28,100,27,9,17,122,8,72,74,96,15,86,68,37,68,18,76,48,15,
28,93,48,68,6,73,60,70,91,24,122,77,82,2,125,80,81,85,45,18,74,64,47,91,85,
74,51,21,27,20,115,24,27,5,37,69,80,75,46,18,68,72,45,88,1,2,40,90,82,31,37,
69,76,85,103,80,94,74,46,64,27,5,60,75,87,24,122,25,27,5,47,71,73,81,56,21,
27,93,48,88,76,31,57,77,74,11,55,73,68,73,115,65,81,31,104,26,14,23,122,8,75,
68,52,92,1,2,110,24,27,5,53,71,80,65,96,15,13,2,125,75,83,75,41,77,82,81,96,
15,72,64,37,92,19,93,48,68,7,5,62,64,93,87,46,77,72,24,40,92,90,8,101,15,28,
83,56,90,1,2,108,6,13,21,122,8,82,64,42,67,89,92,96,15,93,19,56,28,8,65,101,
31,94,16,105,28,10,20,56,30,14,65,56,27,93,71,106,16,11,17,63,25,4,17,57,73,
89,17,59,29,88,29,106,24,27,5,46,65,72,64,54,77,69,24,122,66,93,93,49,5,12,8,
109,15,28,76,59,90,93,72,56,76,1,2,41,90,73,64,122,8,89,85,50,75,84,24,122,
25,15,23,105,25,5,19,106,26,14,20,111,25,27,5,53,77,85,66,53,92,1,2,110,26,
13,2,125,95,85,65,41,64,1,2,108,27,10,19,122,7,2>>
While the 32-bit masking key is:
<<37,93,40,60,25,63,71,88,92,125,80,81,73,51,91,1,2,53,92,72,85,103,7,19,79,60,
74,94,64,47,6,83>>
As per https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-17#section-5.2 :
j = i MOD 4
transformed-octet-i = original-octet-i XOR masking-key-octet-j
however, i doesn't seem to get my original octet sent from client side, which is basically a xml packet. Any direction, correction, suggestions are greatly appreciated.
I think you've mis-read the data framing section of the protocol spec.
Your interpretation of the first byte (129) is correct - fin + opcode 1 - final (and first) fragment of a text message.
The next byte (254) implies that the body of the message is masked and that the following 2 bytes provide its length (lengths of 126 or 127 imply longer messages whose length's can't be represented in 7 bits. 126 means that the following 2 bytes hold the length; 127 mean that its the following 4 bytes).
The following 2 bytes - 1, 120 - imply a message length of 376 bytes.
The following 4 bytes - 37,93,40,60 - are your mask.
The remaining data is your message which should be transformed as you write, giving the message
<body xmlns='http://jabber.org/protocol/httpbind' rid='2167299354' to='jaxl.im' xml:lang='en' xmpp:version='1.0' xmlns:xmpp='urn:xmpp:xbosh' ack='1' route='xmpp:dev.jaxl.im:5222' wait='30' hold='1' content='text/xml; charset=utf-8' ver='1.1
0' newkey='a6e44d87b54461e62de3ab7874b184dae4f5d870' sitekey='jaxl-0-0' iframed='true' epoch='1324196722121' height='321' width='1366'/>
Related
How to receive CAN data from 29 bit CAN id using TP layer in CAPL
My requirement is to receive/send more than 8 bytes from 29 bit CAN id. I'm using TP layer for this. Is there any library functions for receiving the can data which is from 29 bit id? Ex: 0x1CDA00FE is tester and 0x1CDAFE00 is server.
non CAN-FD protocol on Data Link Layer does not support payload greater than 8 bytes. Payload in TP greater than 8 bytes is handled as consecutive frames, and usually are incorporated in a packaging protocol, like UDS.
If this is a diagnostic message ID, and you have diag layer (UDS/KWP) configured on this message, there is a complete diag library in CAPL to send/receive diag requests on the configured Diag Layer.
But to do this "by hand" (I do not know what protocol uses your message IDs to send consecutive frames, like in UDS) :
Receiving
message 0x1CDA00FE msgContainer; /*Create/ define an arbitrary instance of CAN message element*/
Here is how you capture the first iteration of the consecutive frames sent (all of them will be 8 bytes, but you will wait for them as individual 8 byte messages until the intended TP layer payload is completely sent to you). So you basically repeat this snippet 6 times (if your TP payload is 6*8=48bytes long), and do with the msgContainer (message with your payload) whatever you want, until the next one comes.
testwaitformessage(0x1CDA00FE,1000); //wait for the message ID to arrive in 1s
testGetWaitEventMsgData(msgContainer); /*Capture the message content from tail to toe in the msgContainer*/
write("The Byte(0) selector of the captured message for example selects the first byte of the 8 byte payload: 0x%X",msgContainer.byte(0)); // see message selectors for more options
Sending:
byte qword QPayload=0x11223344x55667788; /*You can do with byte array also, I don't fancy to write fors now*/
message 0x1CDAFE00 messagetoSend;
messagetoSend.qword(0)=QPayload;
output(messagetoSend);
/*Prepare another payload to send as the next consecutive frame*/
...
Twilio queue overflow error: how large is the queue?
Twilio's Message resource has a "status" property that indicates whether a SMS message is "queued", "sending", "failed", etc. If a Message instance failed to deliver, one possible error message is "Queue overflow". In the Twilio documentation, the description for this error case is: "You tried to send too many messages too quickly and your message queue overflowed. Try sending your message again after waiting some time." Is the queue referenced in error code 30001 an instance of this resource? https://www.twilio.com/docs/api/rest/queue Or is the queue (in the case of a 30001 error code) something that Twilio maintains on their end? If Twilio does throttling behind the scenes (queueing SMS messages per sending phone number), what is the size of that queue? How much would we have to exceed the rate limit (per phone number) before the queue overflow referenced in error code 30001 occurs?
Emily, message queue is not related to the queue resource you linked to above and it is something maintained on our end. Twilio can queue up to 4 hours of SMS. This means, we can send out 1 sms per second, if there are more than 14,400 messages in the queue, all the messages queued up after will fail with 30001 error queue overflow and will not be sent. This info is for Long Code numbers. The link above explains processing for other scenarios. A few suggestions to avoid the error: Keep messages to at most 160 characters if possible. But if not possible, calculate how many SMS messages each message will be (if you are not sure you can always send 1 test message and see how much you are charged for that message). Based on the assumption that your messages is 160 characters, throttle the sending rate to 3600 messages per hour (1 messaage/sec * 60 sec/min * 60 min/hr). Please let me know if you've got any other questions.
Each of the Twilio phone numbers(senders) has a separate queue in which 14400(4 hr x 60 min x 60 sec) message segments can be queued. 1 segment is sent in one second. What is a message segment? A message segment is not a complete message but a part of a message. Normally SMS is sent in terms of message segments and all message segments are combined on the user’s mobile to create the actual SMS. Twillio message segment config: 1 character = 8 bits(1 byte) GSM encoding = 7 bit per character UCS-2 encoding = 16 bit per character Data Header = 6 bytes per segment Summary: Each character takes 8 bits, If GSM encoding is used, each character will take 7 bits or if UCS-2 encoding is used, each char will take 16 bits. In the case of multiple segments, 6 bytes per segment will be used for data headers(responsible for combining all segments of the same SMS on user mobile) Character per Message Segment: GSM encoding when single segment = (140 char bytes x 8 bits)/ 7 bits = 160 characters UCS-2 encoding when single segment = (140 char bytes x 8 bits)/ 16 bits = 70 characters GSM encoding when multiple segment = ((140 char bytes - 6 header bytes) x 8 bits)/ 7 bits = 154 characters UCS-2 encoding when multiple segment = ((140 char bytes - 6 header bytes) x 8 bits)/ 16 bits = 67 characters Based on what encoding is used(check via Twilio Admin) for your message, you can calculate how many SMS can be in the queue at a time. References: https://support.twilio.com/hc/en-us/articles/115002943027-Understanding-Twilio-Rate-Limits-and-Message-Queues https://www.twilio.com/blog/2017/03/what-the-heck-is-a-segment.html
CocoaAsyncSocket processing hex values
I'm using CocoaAsyncSocket to process a fixed length header message. My node implementation is as follows var output = "\xA5\xA5" + jsonStr; socket.write(output); I intend to read the header first (\xA5\xA5) and then process the body of the message. The problem that I have is that I've noticed that A5 takes 4 bytes to be represented in iOS. So when I read 2 bytes I receive c2a5. CocoaAsync reading code socket?.readDataToLength(2, withTimeout: -1, tag: 1) What am I doing wrong here ?. I would like to keep a fixed length header on my server side. Any help is much appreciated.
How to manual set minimal value for dynamic buffer in Netty 3.2.6? For example 2048 bytes
I need to receive full packet from other IP (navigation device) by TCP/IP.
The device has to send 966 bytes periodically (over one minute), for example.
In my case first received buffer has length 256 bytes (first piece of packet), the second is 710 bytes (last piece of packet), the third is full packet (966 bytes).
How to manual set minimal value for first received buffer length?
This is piece of my code:
Executor bossExecutors = Executors.newCachedThreadPool();
Executor workerExecutors = Executors.newCachedThreadPool();
NioServerSocketChannelFactory channelsFactory =
new NioServerSocketChannelFactory(bossExecutors, workerExecutors);
ServerBootstrap bootstrap = new ServerBootstrap(channelsFactory);
ChannelPipelineFactory pipelineFactory = new NettyServerPipelineFactory(this.HWController);
bootstrap.setPipelineFactory(pipelineFactory);
bootstrap.setOption("child.tcpNoDelay", true);
bootstrap.setOption("child.keepAlive", true);
bootstrap.setOption("child.receiveBufferSizePredictorFactory",
new FixedReceiveBufferSizePredictorFactory(2048)
);
bootstrap.bind(new InetSocketAddress(this.port));
No matter what receiveBufferSizePredictorFactory you specify, you will see a message is split into multiple MessageEvents. It's because TCP/IP is not a message-oriented protocol but a stream-oriented one. Please read the user guide that explains how to write a proper decoder that deals with this common issue.
TCP/IP Client / Server commands data
I have a Client/Server architecture (C# .Net 4.0) that send's command packets of data as byte arrays. There is a variable number of parameters in any command, and each paramater is of variable length. Because of this I use delimiters for the end of a parameter and the command as a whole. The operand is always 2 bytes and both types of delimiter are 1 byte. The last parameter_delmiter is redundant as command_delmiter provides the same functionality. The command structure is as follow: FIELD SIZE(BYTES) operand 2 parameter1 x parameter_delmiter 1 parameter2 x parameter_delmiter 1 parameterN x ............. ............. command_delmiter 1 Parameters are sourced from many different types, ie, ints, strings etc all encoded into byte arrays. The problem I have is that sometimes parameters when encoded into byte arrays contain bytes that are the same value as a delimiter. For example command_delmiter=255.. and a paramater may have that byte inside of it. There is 3 ways I can think of fixing this: 1) Encode the parameters differently so that they can never be the same value as a delimiter (255 and 254) Modulus?. This will mean that paramaters will become larger, ie Int16 will be more than 2 bytes etc. 2) Do not use delimiters at all, use count and length values at the start of the command structure. 3) Use something else. To my knowledge, the way TCP/IP buffers work is that SOME SORT of delimiter has to be used to seperate 'commands' or 'bundles of data' as a buffer may contain multiple commands, or a command may span multiple buffers.. So this BinaryReader / Writer seems like an obvious candidate, the only issue is that the byte array may contain multiple commands ( with parameters inside). So the byte array would still have to be chopped up in order to feel into the BinaryReader. Suggestions? Thanks.
The standard way to do this is to have the length of the message in the (fixed) first few bytes of a message. So you could have the first 4 bytes to denote the length of a message, read those many bytes for the content of the message. The next 4 bytes would be the length of the next message. A length of 0 could indicate end of messages. Or you could use a header with a message count. Also, remember TCP is a byte stream, so don't expect a complete message to be available every time you read data from a socket. You could receive an arbitrary number of bytes at ever read.