Problem using mosquitto broker with netcat - mosquitto
I trying to use mosquitto broker for an IoT application. I have a embedded hardware, actually not much documented/exampled on the internet. I've succesfully implemented an TCP client on this hardware, and now i can send/listen messages throught any port i want via TCP and i listen via netcat. But when i tried to connect mosquitto, it doesn't accept the literal language. I digged on the internet. The broker take messages like shown below, but even this one not working.
I can not found any documentation. I even tried to watch Wireshark packages, and i can not find any pattern. Any help will appreciated.
$ echo -en "\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a" |nc localhost 1883|hd
00000000 20 02 01 00 | ...|
00000004
Had a similar usecase like you and this is how I managed to decode this message and create my own connect request.
echo -en "\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a" |nc localhost 1883
The above message is broken down as follows:
nc localhost 1883, opens a tcp socket to the mqtt port 1883 (to the broker (on localhost) listening to port 1883)
\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a is the connect packet sent to the socket that was opened. This connect packet can be broken down as follows:
\x10: MQTT connect packet (Constitutes the control field with the 1st 4 bits representing the command type "0001" and the 2nd 4 bits the control flag.
\x0d: Remaining length: is the total length of both the variable header and the payload. This needs to be set after the whole payload is complete.
*Variable header = Protocol Name + Protocol level + Connect Flag byte + Keep Alive
Payload = ClientId, username, password, e.t.c, each entry is provided in the format lengthOfEntry + Entry e.g for a clientId and username and password, this gives: lengthOfClientId + ClientId + lengthOfUsername + Username + lengthOfPassword + Password. *Note: the length is always provided as two bytes.
\x00\x04MQTT: Represents the protocol name "MQTT". The first two bytes 0x00 and 0x04 are the protocol length and MQTT is then the protocol. These give a total of 6 bytes.
\x04: is the protocol level. From MQTT specification, the value of the Protocol Level field for the version 3.1.1 of the protocol is 4 (0x04)
\0x00: is the connect flag byte. each byte is represented as follows:
UsernameFlag|PasswordFlag|Will Retain|Will|QoS|Will Flag|CleanSession|Reserved. 0x00 means none of the flags are set and persistent sessions will be used. Hence the payload doesn't require a username or password as will be seen in the payload bytes.
\0x00\0x00: two bytes that represent the keepAlive time. In this case 0 is provided which means that the server is not required to disconnect the client on grounds of inactivity. keepalive mechanism is turned off
The next bytes are already the payload. Since the connect flag was set such that no username or password is required, then we only have to provide the client id. In the above example the clientid is "a". This is of length 1 byte.
\0x00\0x01: represents the length of the clientId. since we have the character "a" as the clientId we have just the length 1. The length is always given in 16bit (2 bytes).
a: the clientId.
If we count all the bytes, we come up to a total of 13 bytes which gives us our remaining le0gnth as 0x0d.
If you need to add a username and password, set the connect flag accordingly 0b11000000 = 0xC0. The username and password are added to the payload right after the clientId in the order lengthofClientId, clientId, lengthOfUSername, Username, LengthofPassword, Password. The reminaining length needs to be adjusted to reflect this.
Tip: The variable header has a fixed number of bytes of 10. The payload length always depends on the data provided. the length of each element is always 2 bytes. so for the above case, we have a clientid of 1 byte and the length which take 2 bytes to give a total of 10 + 2 + 1 = 13.
I hope this helps you out and answers your question.
Useful links:
https://openlabpro.com/guide/mqtt-packet-format/
http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718024
Related
How to receive CAN data from 29 bit CAN id using TP layer in CAPL
My requirement is to receive/send more than 8 bytes from 29 bit CAN id. I'm using TP layer for this. Is there any library functions for receiving the can data which is from 29 bit id? Ex: 0x1CDA00FE is tester and 0x1CDAFE00 is server.
non CAN-FD protocol on Data Link Layer does not support payload greater than 8 bytes. Payload in TP greater than 8 bytes is handled as consecutive frames, and usually are incorporated in a packaging protocol, like UDS.
If this is a diagnostic message ID, and you have diag layer (UDS/KWP) configured on this message, there is a complete diag library in CAPL to send/receive diag requests on the configured Diag Layer.
But to do this "by hand" (I do not know what protocol uses your message IDs to send consecutive frames, like in UDS) :
Receiving
message 0x1CDA00FE msgContainer; /*Create/ define an arbitrary instance of CAN message element*/
Here is how you capture the first iteration of the consecutive frames sent (all of them will be 8 bytes, but you will wait for them as individual 8 byte messages until the intended TP layer payload is completely sent to you). So you basically repeat this snippet 6 times (if your TP payload is 6*8=48bytes long), and do with the msgContainer (message with your payload) whatever you want, until the next one comes.
testwaitformessage(0x1CDA00FE,1000); //wait for the message ID to arrive in 1s
testGetWaitEventMsgData(msgContainer); /*Capture the message content from tail to toe in the msgContainer*/
write("The Byte(0) selector of the captured message for example selects the first byte of the 8 byte payload: 0x%X",msgContainer.byte(0)); // see message selectors for more options
Sending:
byte qword QPayload=0x11223344x55667788; /*You can do with byte array also, I don't fancy to write fors now*/
message 0x1CDAFE00 messagetoSend;
messagetoSend.qword(0)=QPayload;
output(messagetoSend);
/*Prepare another payload to send as the next consecutive frame*/
...
Wireshark: Flag abbreviations and Exchange type
I was told to ask this here: 10:53:04.042608 IP 172.17.2.12.42654 > 172.17.2.6.6000: Flags [FPU], seq 3891587770, win 1024, urg 0, length 0 10:53:04.045939 IP 172.17.2.6.6000 > 172.17.2.12.42654: Flags [R.], seq 0, ack 3891587770, win 0, length 0 This states that the flags set are FPU and R. What flags do these stand for and what kind of exchange is this?
The flags are: F - FIN, used to terminate an active TCP connection from one end. P - PUSH, asks that any data the receiving end is buffering be sent to the receiving process. U - URGENT, indicating that there is data referenced by the urgent "pointer." R - RESET, indicating that a packet was received that was NOT part of an existing connection. It looks like the first packet was manufactured, or possibly delayed. The argument for it being manufactured is the urgent flag being set, with no urgent data. If it was delayed, it indicates the normal end of a connection between .12 and .6 on port 6000, along with a request that the last of any pending data sent across the wire be flushed to the service on .6. .6 has clearly forgotten about this connection, if it even existed. .6 is indicating that while it got the FIN packet, it believes that the connection that FIN packet refers to did not exist. If .6 had a current matching connection, it would have replied with a FIN-ACK instead of RST, acknowledging the termination of the connection.
tracert command returns timed out
tracert returns requested time out. What I understand from this is the packets lost some where on the network. Does it mean the issue is with the ISP or with the hosting provider or my windows system? 10 * * * Request timed out. 11 * * * Request timed out. 12 * * * Request timed out. 13 * * * Request timed out. 14 * * * Request timed out. 15 * * * Request timed out. 16 * * * Request timed out. 17 * * * Request timed out. 18 * * * Request timed out. 19 * * * Request timed out. 20 * * * Request timed out. 21 * * * Request timed out. 22 * * * Request timed out. 23 * * * Request timed out. 24 * * * Request timed out. 25 * * * Request timed out. 26 * * * Request timed out. 27 * * * Request timed out. 28 * * * Request timed out. 29 * * * Request timed out. 30 * * * Request timed out. The first 9 were successful.
I can't see the first 9 hops but if they are all the same then you may have a firewall configuration issue that prevents the packets from either getting out or getting back. Try again turning off your firewall (temporarily!). The other option is that your ISP may drop ICMP traffic as a matter of course, or only when they are busy with other traffic. ICMP (the protocol used by traceroute) is of the lowest priority, and when higher priority traffic is ongoing the router may be configured to simply drop ICMP packets. There is also the possibility that the ISP drops all ICMP packets as a matter of security since many DOS (Denial of Service) attacks are based on probing done with ICMP packets.
Some routers view all pings as a Port-Scan, and block for that reason. (as the first step in any attack is determining which ports are open.) However, blocking ping packets / tracert packets, etc. is only partially effective at mitigating a Denial-of-service attack, as such an attack could use ANY PROTOCHOL it wanted (such as by using TCP or UDP packets, etc.) So long as there is an open port to receive the packet on the machine targeted for Denial-Of-Service. For example, if we wanted to target an http server, we only need use an intercepting proxy to repeatedly send a null TCP packet to the server on port 80 or port 8080, since we know that these are the two most common ports for http. Likewise, if the target machine is running an IRCd, we know the port is most likely 6667 (unless the server is using SSL), which would be the most common port for that kind of service. Therefore, dropping ping packets does not prevent a DdOS attack- it just makes that type of attack a bit more difficult.
This is what I found from the Wireshark documentation(I had the same problem): "The tracert program provided with Windows does not allow one to change the size of the ICMP message sent by tracert. So it won’t be possible to use a Windows machine to generate ICMP messages that are large enough to force IP fragmentation. However, you can use tracert to generate small,fixed-length packets" https://danielgraham.files.wordpress.com/2021/09/wireshark_ip_v8.1-2.pdf
use tracert -h 1 this will limit the number of times it tries a particular ip address to 1 try. h = hops. I had written a batch script a while back to scan my entire network to get a list of ips and computer networks and it would waste time on the fire wall that wouldnt answer and ip addresses that weren't assigned to any computers. Wicked annoying!!!!!! so I added the -h 1 to the script!! I runs through and makes a list in a text file. I hope to improve it in the future by running arp -a first to get a quck list of ips, then feeding that list into a script similar to this one. that way it doesn't waste time on unassigned IP's. enter code here#echo off enter code hereset trace=tracert enter code hereset /a byte1=222 enter code hereset /a byte2=222 enter code hereset /a byte3=222 enter code hereset /a byte4=100 enter code hereset loop=0 enter code here:loop enter code here#echo enter code here%trace% %byte1%.%byte2%.%byte3%.%byte4%>>ips.txt enter code hereset /a loop=%loop% + 1 enter code hereset /a byte4=%byte4% + 1 enter code here#echo %byte4% enter code hereif %loop%==255 goto next enter code heregoto loop enter code here:next enter code hereend
Your antivirus blocks the incoming packets , and in no case this option can be turned off because its the basic property of an antivirus i.e to block packets to prevent computer from normal as well as DOS (Denial of Service) attacks .
websocket client packet unframe/unmask
I am trying to implement latest websocket spec. However, i am unable to get through the unmasking step post successful handshake. I receive following web socket frame: <<129,254,1,120,37,93,40,60,25,63,71,88,92,125,80,81,73, 51,91,1,2,53,92,72,85,103,7,19,79,60,74,94,64,47,6,83, 87,58,7,76,87,50,92,83,70,50,68,19,77,41,92,76,71,52, 70,88,2,125,90,85,65,96,15,14,20,107,31,14,28,100,27,9, 17,122,8,72,74,96,15,86,68,37,68,18,76,48,15,28,93,48, 68,6,73,60,70,91,24,122,77,82,2,125,80,81,85,45,18,74, 64,47,91,85,74,51,21,27,20,115,24,27,5,37,69,80,75,46, 18,68,72,45,88,1,2,40,90,82,31,37,69,76,85,103,80,94, 74,46,64,27,5,60,75,87,24,122,25,27,5,47,71,73,81,56, 21,27,93,48,88,76,31,57,77,74,11,55,73,68,73,115,65,81, 31,104,26,14,23,122,8,75,68,52,92,1,2,110,24,27,5,53, 71,80,65,96,15,13,2,125,75,83,75,41,77,82,81,96,15,72, 64,37,92,19,93,48,68,7,5,62,64,93,87,46,77,72,24,40,92, 90,8,101,15,28,83,56,90,1,2,108,6,13,21,122,8,82,64,42, 67,89,92,96,15,93,19,56,28,8,65,101,31,94,16,105,28,10, 20,56,30,14,65,56,27,93,71,106,16,11,17,63,25,4,17,57, 73,89,17,59,29,88,29,106,24,27,5,46,65,72,64,54,77,69, 24,122,66,93,93,49,5,12,8,109,15,28,76,59,90,93,72,56, 76,1,2,41,90,73,64,122,8,89,85,50,75,84,24,122,25,15, 23,105,25,5,19,106,26,14,20,111,25,27,5,53,77,85,66,53, 92,1,2,110,26,13,2,125,95,85,65,41,64,1,2,108,27,10,19, 122,7,2>> As per base framing protocol defined here (https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-17#section-5.2) i have: fin:1, rsv:0, opcode:1, mask:1, length:126 Masked application+payload data comes out to be: <<87,58,7,76,87,50,92,83,70,50,68,19,77,41,92,76,71,52,70,88,2,125,90,85,65,96, 15,14,20,107,31,14,28,100,27,9,17,122,8,72,74,96,15,86,68,37,68,18,76,48,15, 28,93,48,68,6,73,60,70,91,24,122,77,82,2,125,80,81,85,45,18,74,64,47,91,85, 74,51,21,27,20,115,24,27,5,37,69,80,75,46,18,68,72,45,88,1,2,40,90,82,31,37, 69,76,85,103,80,94,74,46,64,27,5,60,75,87,24,122,25,27,5,47,71,73,81,56,21, 27,93,48,88,76,31,57,77,74,11,55,73,68,73,115,65,81,31,104,26,14,23,122,8,75, 68,52,92,1,2,110,24,27,5,53,71,80,65,96,15,13,2,125,75,83,75,41,77,82,81,96, 15,72,64,37,92,19,93,48,68,7,5,62,64,93,87,46,77,72,24,40,92,90,8,101,15,28, 83,56,90,1,2,108,6,13,21,122,8,82,64,42,67,89,92,96,15,93,19,56,28,8,65,101, 31,94,16,105,28,10,20,56,30,14,65,56,27,93,71,106,16,11,17,63,25,4,17,57,73, 89,17,59,29,88,29,106,24,27,5,46,65,72,64,54,77,69,24,122,66,93,93,49,5,12,8, 109,15,28,76,59,90,93,72,56,76,1,2,41,90,73,64,122,8,89,85,50,75,84,24,122, 25,15,23,105,25,5,19,106,26,14,20,111,25,27,5,53,77,85,66,53,92,1,2,110,26, 13,2,125,95,85,65,41,64,1,2,108,27,10,19,122,7,2>> While the 32-bit masking key is: <<37,93,40,60,25,63,71,88,92,125,80,81,73,51,91,1,2,53,92,72,85,103,7,19,79,60, 74,94,64,47,6,83>> As per https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-17#section-5.2 : j = i MOD 4 transformed-octet-i = original-octet-i XOR masking-key-octet-j however, i doesn't seem to get my original octet sent from client side, which is basically a xml packet. Any direction, correction, suggestions are greatly appreciated.
I think you've mis-read the data framing section of the protocol spec. Your interpretation of the first byte (129) is correct - fin + opcode 1 - final (and first) fragment of a text message. The next byte (254) implies that the body of the message is masked and that the following 2 bytes provide its length (lengths of 126 or 127 imply longer messages whose length's can't be represented in 7 bits. 126 means that the following 2 bytes hold the length; 127 mean that its the following 4 bytes). The following 2 bytes - 1, 120 - imply a message length of 376 bytes. The following 4 bytes - 37,93,40,60 - are your mask. The remaining data is your message which should be transformed as you write, giving the message <body xmlns='http://jabber.org/protocol/httpbind' rid='2167299354' to='jaxl.im' xml:lang='en' xmpp:version='1.0' xmlns:xmpp='urn:xmpp:xbosh' ack='1' route='xmpp:dev.jaxl.im:5222' wait='30' hold='1' content='text/xml; charset=utf-8' ver='1.1 0' newkey='a6e44d87b54461e62de3ab7874b184dae4f5d870' sitekey='jaxl-0-0' iframed='true' epoch='1324196722121' height='321' width='1366'/>
How to manual set minimal value for dynamic buffer in Netty 3.2.6? For example 2048 bytes
I need to receive full packet from other IP (navigation device) by TCP/IP.
The device has to send 966 bytes periodically (over one minute), for example.
In my case first received buffer has length 256 bytes (first piece of packet), the second is 710 bytes (last piece of packet), the third is full packet (966 bytes).
How to manual set minimal value for first received buffer length?
This is piece of my code:
Executor bossExecutors = Executors.newCachedThreadPool();
Executor workerExecutors = Executors.newCachedThreadPool();
NioServerSocketChannelFactory channelsFactory =
new NioServerSocketChannelFactory(bossExecutors, workerExecutors);
ServerBootstrap bootstrap = new ServerBootstrap(channelsFactory);
ChannelPipelineFactory pipelineFactory = new NettyServerPipelineFactory(this.HWController);
bootstrap.setPipelineFactory(pipelineFactory);
bootstrap.setOption("child.tcpNoDelay", true);
bootstrap.setOption("child.keepAlive", true);
bootstrap.setOption("child.receiveBufferSizePredictorFactory",
new FixedReceiveBufferSizePredictorFactory(2048)
);
bootstrap.bind(new InetSocketAddress(this.port));
No matter what receiveBufferSizePredictorFactory you specify, you will see a message is split into multiple MessageEvents. It's because TCP/IP is not a message-oriented protocol but a stream-oriented one. Please read the user guide that explains how to write a proper decoder that deals with this common issue.