(My question differs from this one.)I am connected to a AP in a wireless network and I've send a simple ping request to www.google.com. When I analyze the packet in wireshark, I can see, that there are 48 bytes written in the data section of ICMP. After 8 bytes of trash values, the values are sequentially increasing from 0x10 to 0x37.Is there any particular reason, why ICMPv4 fits values instead of just using a bunch of zeroes?
The hexdump of the ICMPv4 data section:
0030 09 d9 0d 00 00 00 00 00 10 11 12 13 14 15 .Ù............
0040 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 .......... !"#$%
0050 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 &'()*+,-./012345
0060 36 37 67
After 8 bytes of trash values
First of all, these are not trash values. In some implementations of ping, the 1st 8 bytes may represent a timestamp.
As #ross-jacobs mentioned, RFC 792 describes the ICMP Echo Request/Reply Packets. For clarity, these two packets are described, in relevant part, as follows:
Echo or Echo Reply Message
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Code | Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identifier | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data ...
+-+-+-+-+-
...
Description
The data received in the echo message must be returned in the echo
reply message.
Here you can see that the contents of the Data is not described at all; therefore an implementation is free to use whatever data it wishes, including none at all.
Now, since ping is a network test tool, one of the things it can help test is fragmentation/reassembly. Every ping implementation I'm aware of allows the user to specify the size of the payload, and if you exceed the MTU, you should see the ICMP packet fragmented/reassembled. If you examine the payload of the first fragment, you can tell where the second fragment should start just by looking at the sequence of bytes in the payload of the first fragment. If the data was all 0's, it wouldn't be possible to do this. Similarly, if an ICMP packet wasn't reassembled properly, not only would the checksum likely be wrong, but you would most likely be able to tell exactly where the reassembly failed due to a gap or other inconsistency in the payload. This is just one example of why a payload with a sequence of bytes instead of all 0's is more useful to the user.
Related
I am capturing MQTT traffic for troubleshooting using below command
tcpdump -i team0 -w mqtt-trace.pcap src 10.x.x.x
But it results in very big file within minutes, Can i filter tcpdump on base of topic name
Following is tcp payload, I want it only capture payload which has PKGCTRL/1/status/frequency or if tcpdump can directly support filter on application layer protocol like wireshark mqtt.topic == PKGCTRL/1/status/frequency
0000 00 13 95 36 2e ef 00 10 7e 07 87 3d 08 00 45 00 ...6....~..=..E.
0010 00 77 2e 0d 40 00 40 06 f6 78 0a 0b 80 f3 0a 0b .w..#.#..x......
0020 80 f2 c3 6a 75 83 e4 f8 f7 7a 0a 89 67 76 50 18 ...ju....z..gvP.
0030 ea 60 59 f8 00 00 30 4d 00 1a 50 4b 47 43 54 52 .`Y...0M..PKGCTR
0040 4c 2f 31 2f 73 74 61 74 75 73 2f 66 72 65 71 75 L/1/status/frequ
0050 65 6e 63 79 0a 11 09 c2 7a 85 14 d0 71 37 16 12 ency....z...q7..
0060 06 08 01 10 01 18 00 12 1c 0a 0d 09 0b 46 25 75 .............F%u
0070 02 f2 48 40 10 21 18 00 11 00 60 76 14 d0 71 37 ..H#.!....`v..q7
0080 16 20 00 28 00 . .(.
I don't think the previously accepted answer necessarily does what you think it does and possibly not even what you want it to do. The original question stated, "But it results in very big file within minutes, Can i filter tcpdump on base of topic name"
If you're trying to limit the size of the capture file, then the previously accepted answer isn't doing that because it uses the exact same capture filter as was originally provided, namely src 10.x.x.x. This means that you're capturing the same amount of data as you were before. Just because a capture file name wasn't specified doesn't mean that packets aren't being written to a file; they are. In the case of tshark, packets are written to a temporary file, which will continue to grow until the capture session is terminated and then ideally it will be deleted, but not always. The location of the temporary file varies depending on the platform that tshark is run on, but you should be able to easily locate the directory by running tshark -G folders | grep "^Temp".
Now, if you want to reduce the size of the capture file, or the number of packets that you see, then you should be able to modify the tcpdump or tshark command-line arguments to accomplish that.
First off, if you don't need the entire payload, you can apply a snaplen to cut the packets short after some appropriate value. This is done using the -s option, and it's the same option for either capture tool.
OK, but whether you decide to apply a snaplen or not, if you want to filter based on the specific topic name, most likely you can achieve this; however there are a couple of caveats that I listed below. The main idea is to use the slice operator, [] (see the pcap-filter man page) to compare various bytes of the TCP payload to specific values. (NOTE: Neither tcpdump itself nor pcap-filter refers to this operator as the slice operator, but wireshark-filter does, so I do as well.) So the filter should:
Match packets only to/from a particular host, in this case 10.x.x.x
Match only MQTT packets (typically by port number, which I'll assume to be the standard tcp/1883 port)
Match only PUBLISH messages with QoS 0
Match only PUBLISH messages where the topic length is 26 bytes
Match only PUBLISH messages where the topic is "PKGCTRL/1/status/frequency"
Here's a command that should work (at least in most cases -> see caveats below):
tcpdump -i team0 -w mqtt-trace.pcap \
"(src host 10.x.x.x) and \
(tcp port 1883) and \
((tcp[20]&0xf6)=0x30) and \
(tcp[22:2]=26) and \
(tcp[24:4]=0x504b4743 and tcp[28:4]=0x54524c2f and \
tcp[32:4]=0x312f7374 and tcp[36:4]=0x61747573 and \
tcp[40:4]=0x2f667265 and tcp[44:4]=0x7175656e and tcp[48:2]=0x6379)"
It should be obvious from the description of the desired filter above what each separate component of the filter is doing for you.
Here, you'll end up with a capture file of the desired packets that you can reference later on if you need to. You can even apply this same filter to the tshark solution too if you prefer as well as writing to the named capture file, because as I explained earlier, tshark is writing packets to a file whether you explicitly specify one or not.
Caveats:
The filter assumes TCP headers are 20 bytes for simplicity in illustrating the solution, but that may not be the case. If you want a more robust solution to accommodate any TCP header size, then you'll need to determine the TCP header size from the data offset field of the TCP header, which is done in the filter using ((tcp[12]&0xf0)>>4)*4, and then replace every occurrence of 20 in the slice operator's offset field with that value. So for example, tcp[22:2]=26 becomes tcp[(((tcp[12]&0xf0)>>4)*4)+2:2]=26, etc.
Because the MQTT remaining message length field is variable-length encoded per MQTT3.1.1 section 2.2.3 Remaining Length, the filter, as provided above, will only work for values of the remaining length field from 0 to 127, i.e., the remaining length field must only be a single byte. Given that the topic in this case is 26 bytes and the topic length itself is 2 bytes, this means the filter will only work for MQTT message payloads of 99 bytes or less (127 - (2 + 26)).
As #hardillb suggested, use tshark instead. Due to tshark's architecture, you can't write a file at the same time as a display filter (it would be too slow).
To get the information you need, it would look something like this
$ tshark -i team0 -f "src 10.x.x.x" \
-Y "mqtt.topic == PKGCTRL/1/status/frequency" -T fields -e mqtt.topic
-i team0: Filter on interface team0
-f "src 10.x.x.x": Use a capture filter, which is the same as tcpdump's filtering. This will speed up processing as it's faster than a display filter (next bullet).
-Y "mqtt.topic == PKGCTRL/1/status/frequency": Filter for packets that match this display filter
-T fields -e mqtt.topic: Output only the mqtt.topic field, as that is the target information.
-T fields will output columnar data vertically delimited by newlines. It won't be horizontally delimited because there is only column, but the default is \t.
I have a check printer that I want to connect in Delphi 7 by COM port and operate.
I have a command that I extracted with Serial Port Monitor:
STX "PIRI(781" FS NULL ETX "0B" wich is 02 50 49 52 49 28 37 38 31 1c 00 03 30 42 hex
The manual says the following:
CRC (which is the last two digits after the ETX) - packet checksum. It
is calculated by the following algorithm: executing XOR for every
byte of the block including ETX by excluding STX. The data of the
checksum take up two bytes and are a symbolic representation of the
numeric in a hexadecimal calculation system.
I tried to an ONLINE CRC calculator and return a 1B result and a 27 numeric.
How to do it? For "PIRI(781" FS NULL ETX it should be 0B
The documentation incorrectly identifies the check value as a CRC. It is not. It is simply the exclusive-or of the noted bytes. The exclusive-or of 50 49 52 49 28 37 38 31 1c 00 03 is 0b. You then convert the 0b to hex (with an upper case B, i.e. 0B), and get 30 42.
I have implemented a websocket server in my existing server. Handshake is good and then I can even send first message and client browser receives it. But any subsequent messages disconnect websocket. On chrome I get following error.
failed: A server must not mask any frames that it sends to the client.
My frame is created like this
Len = erlang:size(Msg),
if
Len < 126 ->
Message = [<<1:1, 0:3,2:4,0:1,Len:7>>,Msg];
Len < 65536 ->
Message = [<<1:1, 0:3,2:4,0:1,126:7,Len:16>>,Msg];
true ->
Message = [<<1:1, 0:3,2:4,0:1,127:7,Len:64>>,Msg]
end,
Now one sample data to be transmitted looks like this
<<130,46,60,115,110,112,95,105,110,102,111,32,97,118,95,112,111,116,61,34,49,49,34,32,104,97,110,100,115,95,112,101,114,95,104,111,117,114,61,34,48,46,50,48,34,32,47,62>>
As you can see in the code above, my mask bit is always set to 0, but I don't why the same message works for the first time and then I send the same message again, it complains about mask bit set.
Anybody got any idea why?
Thanks
The frame sample data you pasted is sane.
48 bytes
ErlangSampleBytes[48] = 82 2E 3C 73 6E 70 5F 69 6E 66 6F 20 61 76 5F 70
6F 74 3D 22 31 31 22 20 68 61 6E 64 73 5F 70 65
72 5F 68 6F 75 72 3D 22 30 2E 32 30 22 20 2F 3E
It Parses/Translates to:
82 = fin:true
rsv1:false
rsv2:false
rsv3:false
opcode:2
2E = masked:false
payloadLength:46 (decimal)
Payload = UTF8 valid:
<snp_info av_pot="11" hands_per_hour="0.20" />
You don't even have a mismatch on the payload length declaration and what is left in the sample buffer.
At this point, it would be wise to look at the chrome inspection tooling to find out what the frames it thinks it got. And a next step after that would be to try to capture the network traffic with a tool like WireShark in an effort to see what is going on.
Suspect that the # of bytes you are sending on the wire, vs what you are declaring in the payloadLength are differing. (common one i've seen is sending more bytes than you declare)
Such as allocating a buffer of 1024 bytes, filling in the first 48 properly, then sending the entire 1024 bytes. That will mess parsing of the next frame.
I am trying to use Indy10 ReadBytes() in Delphi 2007 to read a large download of a series of data segments formatted as [#bytes]\r\n where #bytes indicates the number of bytes. My algorithm is:
Use ReadBytes() to get the [#]\r\n text, which is normally 10 bytes.
Use ReadBytes() to get the specified # data bytes.
Go to step 1 if more data segments need to be processed, i.e., # is negative.
This works well but frequently I don't get the expected text at step 1. Here's a short example after 330 successful data segments:
Data received from last step 2 ReadBytes(). NOTE embedded Step 1 [-08019]\r\n text.
Line|A033164|B033164|C033164|D033164|E033164|F033164|G033164|H033164|EndL\r|Begin
Line|A033165|B033165|C033165|D033165|E033165|F033165|G033165|H033165|EndL\r|Begin
Line|A033166|B033166|C033166|D033166|E033166|F033166|G033166|H033166|EndL\r[-08019]
\r\n|Begin
Line|A033167|B033167|C033167|D033167|E033167|F033167|G033167|H033167|EndL\r|Begin
Line|A033168|B033168|C033168|D033168|E033168|F033168|G033168|H033168|EndL\r|Begin
Socket data captured by WireShark.
0090 30 33 33 31 36 36 7c 42 30 33 33 31 36 36 7c 43 033166|B033166|C
00a0 30 33 33 31 36 36 7c 44 30 33 33 31 36 36 7c 45 033166|D033166|E
00b0 30 33 33 31 36 36 7c 46 30 33 33 31 36 36 7c 47 033166|F033166|G
00c0 30 33 33 31 36 36 7c 48 30 33 33 31 36 36 7c 45 033166|H033166|E
00d0 6e 64 4c 0d ndL.
No. Time Source Destination Protocol Length Info
2837 4.386336000 000.00.247.121 000.00.172.17 TCP 1514 40887 > 57006 [ACK] Seq=2689776 Ack=93 Win=1460 Len=1460
Frame 2837: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits) on interface 0
Ethernet II, Src: Cisco_60:4d:bf (e4:d3:f1:60:4d:bf), Dst: Dell_2a:78:29 (f0:4d:a2:2a:78:29)
Internet Protocol Version 4, Src: 000.00.247.121 (000.00.247.121), Dst: 000.00.172.17 (000.00.172.17)
Transmission Control Protocol, Src Port: 40887 (40887), Dst Port: 57006 (57006), Seq: 2689776, Ack: 93, Len: 1460
Data (1460 bytes)
0000 5b 2d 30 38 30 31 39 5d 0d 0a 7c 42 65 67 69 6e [-08019]..|Begin
0010 20 4c 69 6e 65 7c 41 30 33 33 31 36 37 7c 42 30 Line|A033167|B0
0020 33 33 31 36 37 7c 43 30 33 33 31 36 37 7c 44 30 33167|C033167|D0
0030 33 33 31 36 37 7c 45 30 33 33 31 36 37 7c 46 30 33167|E033167|F0
Does anyone know why this happens? Thanks
More information. We do socket reading from a single thread and don't call Connected() while reading. Here's relevant code snippet:
AClientDebugSocketContext.Connection.Socket.ReadBytes(inBuffer,byteCount,True);
numBytes := Length(inBuffer);
Logger.WriteToLogFile(BytesToString: '+BytesToString(inBuffer,0,numBytes),0);
Move(inBuffer[0], Pointer(Integer(Buffer))^, numBytes);
Embedded data like you describe, especially at random times, usually happens when you read from the same socket in multiple threads at the same time without adequate synchronization between them. One thread may receive a portion of the incoming data, and another thread may receive another portion of the data, and they end up storing their data in the InputBuffer in the wrong order. Hard to say for sure if that your problem since you did not provide any code. The best option is to make sure you never read from the same socket in multiple threads at all. That includes any calls to Connected(), since it performs a read operation internally. You should do all of your reading within a single thread. If that is not an option, then at least wrap your socket I/O with some kind of inter-thread lock, such as a critical section or mutex.
Update: You are accessing a TIdContext object via your own AClientDebugSocketContext variable. Where is that code being used exactly? If it is not in the context of the server's OnConnect, OnDisconnect, OnExecute, or OnException events, then you are reading from the same socket across multiple threads, because TIdTCPServer internally calls Connected() (which does a read) in between calls to the OnExecute event for that TIdContext object.
I have a problem trying to parse a midi file. I am trying to parse the notes files used by the frets on fire game (it just uses midi files so i don't think this is relevent) if any of you are familiar with it, the problem i am having is a general midi problem. I have a file with a track called guitar part, the hex, as viewed in a hex editor is as follows:
4D 54 72 6B 00 00 1E 74 00 FF 03 0B 50 41 52 54 20 47 55 49 54 41 52 A9 20 90 61 40 9A 20 61 00 83 60 63 40 BC
My program parses this fine as follows:
4D M
54 T
72 R
6B K
00 < --
00 size of
1E track part
74 -- >
00 time of this event
FF event type (this is meta)
03 meta event type
0B length of data
50 "P"
41 "A"
52 "R"
54 "T"
20 " "
47 "G"
55 "U"
49 "I"
54 "T"
41 "A"
52 "R"
A9 time of event (variable length) 10101001
20 time of event (variable length) 00100000
90 event,channel (non-meta) 1001=note on,channel=0000
61 note on has 2 params this is the first
40 this is the second
9A variable time 10011010
20 variable time 00100000
This is where my problem lies, there is no event that has event type 0x6, since 0x61 is 01100001 we have to assume it's non meta, therefore the event type should be 6 (0110) and the channel is (0001) but the midi specification contains no identification for this event.. I've added a few of the bytes after this incase they are somehow relevent but obviously at the moment my program hits the next byte, doesn't recognise the event and bombs out.
61
00
83
60
63
40
BC
If anyone thinks they could shed any light on where my parsing logic has gone wrong i'd be most appreciative, sorry for the formatting, i couldn't think of a better way to illustrate my problem.
I have been using this site: http://www.sonicspot.com/guide/midifiles.html as a reference and it hasn't led me wrong so far. I figured this might be something directly relating to frets on fire but it doesn't seem to be as i downloaded another notes file for the game and that file did not contain this event.
Thanks in advance.
It's called running status. If an event is of the same type as the previous event, the MIDI status byte can be eliminated. So if the first byte after the timing info is < $80, use the previous status. In the case of your $61 byte, the previous status was $90, so it's Note On, channel 0. Which makes sense since the previous event was note number $61 velocity $40. This event is note number $61 velocity 0 (releasing the previously played note). The next event is note number $63 velocity $40.