How to filter MQTT traffic on base of topic name in tcpdump - mqtt

Question

I am capturing MQTT traffic for troubleshooting using below command
tcpdump -i team0 -w mqtt-trace.pcap src 10.x.x.x
But it results in very big file within minutes, Can i filter tcpdump on base of topic name
Following is tcp payload, I want it only capture payload which has PKGCTRL/1/status/frequency or if tcpdump can directly support filter on application layer protocol like wireshark mqtt.topic == PKGCTRL/1/status/frequency
0000 00 13 95 36 2e ef 00 10 7e 07 87 3d 08 00 45 00 ...6....~..=..E.
0010 00 77 2e 0d 40 00 40 06 f6 78 0a 0b 80 f3 0a 0b .w..#.#..x......
0020 80 f2 c3 6a 75 83 e4 f8 f7 7a 0a 89 67 76 50 18 ...ju....z..gvP.
0030 ea 60 59 f8 00 00 30 4d 00 1a 50 4b 47 43 54 52 .`Y...0M..PKGCTR
0040 4c 2f 31 2f 73 74 61 74 75 73 2f 66 72 65 71 75 L/1/status/frequ
0050 65 6e 63 79 0a 11 09 c2 7a 85 14 d0 71 37 16 12 ency....z...q7..
0060 06 08 01 10 01 18 00 12 1c 0a 0d 09 0b 46 25 75 .............F%u
0070 02 f2 48 40 10 21 18 00 11 00 60 76 14 d0 71 37 ..H#.!....`v..q7
0080 16 20 00 28 00 . .(.

Answer 1

I don't think the previously accepted answer necessarily does what you think it does and possibly not even what you want it to do. The original question stated, "But it results in very big file within minutes, Can i filter tcpdump on base of topic name"
If you're trying to limit the size of the capture file, then the previously accepted answer isn't doing that because it uses the exact same capture filter as was originally provided, namely src 10.x.x.x. This means that you're capturing the same amount of data as you were before. Just because a capture file name wasn't specified doesn't mean that packets aren't being written to a file; they are. In the case of tshark, packets are written to a temporary file, which will continue to grow until the capture session is terminated and then ideally it will be deleted, but not always. The location of the temporary file varies depending on the platform that tshark is run on, but you should be able to easily locate the directory by running tshark -G folders | grep "^Temp".
Now, if you want to reduce the size of the capture file, or the number of packets that you see, then you should be able to modify the tcpdump or tshark command-line arguments to accomplish that.
First off, if you don't need the entire payload, you can apply a snaplen to cut the packets short after some appropriate value. This is done using the -s option, and it's the same option for either capture tool.
OK, but whether you decide to apply a snaplen or not, if you want to filter based on the specific topic name, most likely you can achieve this; however there are a couple of caveats that I listed below. The main idea is to use the slice operator, [] (see the pcap-filter man page) to compare various bytes of the TCP payload to specific values. (NOTE: Neither tcpdump itself nor pcap-filter refers to this operator as the slice operator, but wireshark-filter does, so I do as well.) So the filter should:
Match packets only to/from a particular host, in this case 10.x.x.x
Match only MQTT packets (typically by port number, which I'll assume to be the standard tcp/1883 port)
Match only PUBLISH messages with QoS 0
Match only PUBLISH messages where the topic length is 26 bytes
Match only PUBLISH messages where the topic is "PKGCTRL/1/status/frequency"
Here's a command that should work (at least in most cases -> see caveats below):
tcpdump -i team0 -w mqtt-trace.pcap \
"(src host 10.x.x.x) and \
(tcp port 1883) and \
((tcp[20]&0xf6)=0x30) and \
(tcp[22:2]=26) and \
(tcp[24:4]=0x504b4743 and tcp[28:4]=0x54524c2f and \
tcp[32:4]=0x312f7374 and tcp[36:4]=0x61747573 and \
tcp[40:4]=0x2f667265 and tcp[44:4]=0x7175656e and tcp[48:2]=0x6379)"
It should be obvious from the description of the desired filter above what each separate component of the filter is doing for you.
Here, you'll end up with a capture file of the desired packets that you can reference later on if you need to. You can even apply this same filter to the tshark solution too if you prefer as well as writing to the named capture file, because as I explained earlier, tshark is writing packets to a file whether you explicitly specify one or not.
Caveats:
The filter assumes TCP headers are 20 bytes for simplicity in illustrating the solution, but that may not be the case. If you want a more robust solution to accommodate any TCP header size, then you'll need to determine the TCP header size from the data offset field of the TCP header, which is done in the filter using ((tcp[12]&0xf0)>>4)*4, and then replace every occurrence of 20 in the slice operator's offset field with that value. So for example, tcp[22:2]=26 becomes tcp[(((tcp[12]&0xf0)>>4)*4)+2:2]=26, etc.
Because the MQTT remaining message length field is variable-length encoded per MQTT3.1.1 section 2.2.3 Remaining Length, the filter, as provided above, will only work for values of the remaining length field from 0 to 127, i.e., the remaining length field must only be a single byte. Given that the topic in this case is 26 bytes and the topic length itself is 2 bytes, this means the filter will only work for MQTT message payloads of 99 bytes or less (127 - (2 + 26)).

Answer 2

As #hardillb suggested, use tshark instead. Due to tshark's architecture, you can't write a file at the same time as a display filter (it would be too slow).
To get the information you need, it would look something like this
$ tshark -i team0 -f "src 10.x.x.x" \
-Y "mqtt.topic == PKGCTRL/1/status/frequency" -T fields -e mqtt.topic
-i team0: Filter on interface team0
-f "src 10.x.x.x": Use a capture filter, which is the same as tcpdump's filtering. This will speed up processing as it's faster than a display filter (next bullet).
-Y "mqtt.topic == PKGCTRL/1/status/frequency": Filter for packets that match this display filter
-T fields -e mqtt.topic: Output only the mqtt.topic field, as that is the target information.
-T fields will output columnar data vertically delimited by newlines. It won't be horizontally delimited because there is only column, but the default is \t.