Plot RTT histogram using wireshark or other tool - histogram

I have a little office network and I'm experiencing a huge internet link latency. We have a simple network topology: a computer configured as router running ubuntu server 10.10, 2 network cards (one to internet link, other to office network) and a switch connecting 20 computers. I have a huge tcpdump log collected at the router and I would like to plot a histogram with the RTT time of all TCP streams to try to find out the best solution to this latency problem. So, could somebody tell me how to do it using wireshark or other tool?

Wireshark or tshark can give you the TCP RTT for each received ACK packet using tcp.analysis.ack_rtt which measures the time delta between capturing a TCP packet and the ACK for that packet.
You need to be careful with this as most of your ACK packets will be from your office machines ACKing packets received from the internet, so you will be measuring the RTT between your router seeing the packet from the internet and seeing the ACK from your office machine.
To measure your internet RTT you need to look for ACKS from the internet (ACKing data sent from your network). Assuming your office machines have IP addresses like 192.168.1.x and you have logged all the data on the LAN port of your router you could use a display filter like so:
tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24
To dump the RTTs into a .csv for analysis you could use a tshark command like so;
tshark -r router.pcap -Y "tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d > rtt.csv
The -r option tells tshark to read from your .pcap file
The -Y option specifies the display filter to use (-R without -2 is deprecated)
The -e option specifies the field to output
The -T options specify the output formatting
You can use the mergecap utility to merge all your pcap files into one one file before running this command. Turning this output into a histogram should be easy!

Here's the 5-min perlscript inspired by rupello's answer:
#!/usr/bin/perl
# For a live histogram of rtt latencies, save this file as /tmp/hist.pl and chmod +x /tmp/hist.pl, then run:
# tshark -i wlp2s0 -Y "tcp.analysis.ack_rtt and ip.dst==192.168.0.0/16" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d | /tmp/hist.pl
# Don't forget to update the interface "wlp2s0" and "and ip.dst==..." bits as appropriate, type "ip addr" to get those.
#t[$m=0]=20;
#t[++$m]=10;
#t[++$m]=5;
#t[++$m]=2;
#t[++$m]=1;
#t[++$m]=0.9;
#t[++$m]=0.8;
#t[++$m]=0.7;
#t[++$m]=0.6;
#t[++$m]=0.5;
#t[++$m]=0.4;
#t[++$m]=0.3;
#t[++$m]=0.2;
#t[++$m]=0.1;
#t[++$m]=0.05;
#t[++$m]=0.04;
#t[++$m]=0.03;
#t[++$m]=0.02;
#t[++$m]=0.01;
#t[++$m]=0.005;
#t[++$m]=0.001;
#t[++$m]=0;
#h[0]=0;
while (<>) {
s/\"//g; $n=$_; chomp($n); $o++;
for ($i=$m;$i>=0;$i--) { if ($n<=$t[$i]) { $h[$i]++; $i=-1; }; };
if ($i==-1) { $h[0]++; };
print "\033c";
for (0..$m) { printf "%6s %6s %8s\n",$t[$_],sprintf("%3.2f",$h[$_]/$o*100),$h[$_]; };
}
The newer versions of tshark seem to work better with a "stdbuf -i0 -o0 -e0 " in front of the "tshark".
PS Does anyone know if wireshark has DNS and ICMP rtt stats built in or how to easily get those?
2018 Update: See https://github.com/dagelf/pping

Related

tshark returns 0 results for filter icmp.no_resp but wireshark returns 12 resutls with the same filter

I am trying to do packet capture analysis with tshark on about 30000 files looking for a needle in the haystack.The files containing interesting needles contain icmp failures. I wrote a script which iterates though these files with tshark but they all return 0 results.
tshark -r <filename> -Y "icmp.no_resp"
tshark -r <filename> -Y "icmp.resp_not_found"
Both ofthese commands yield 0 results. However when I open a specific file and use the display filter "icmp.no_resp" or "icmp.resp_not_found" I see results.
Is this a bug in T-shark where it can't identify response not found?
I'm running tshark/wireshark v3.6.7 on Ubuntu
I figured it out.
tshark requires multiple passes to identify certain display filters. Doing a command like so creates this.
tshark -r <filename> -Y "icmp.resp_not_found" -2
I hope this helps someone in the future.

how to remove zero packets (empty streams) records in wireshark

I am very new to wireshark. in my day to day job i need to remove the packet bytes zero records from captured PCAP file. please help me in this process. attached image is for reference
wireshark packets zero.png
Since you have 47 TCP Streams and 28 that you want to remove, it might be a bit faster to filter for all the TCP streams that you do want to keep since there are only 19 of those.
For the 19 streams you want:
Right-click on the first TCP conversation and choose "Prepare a Filter -> Selected -> A<-->B".
For the next 17 TCP conversations, right-click on each one and choose "Prepare a Filter -> ... And Selected -> A<-->B".
Finally, for the last TCP stream, right-click on the TCP conversation and choose "Apply as Filter -> ... And Selected -> A<-->B".
You may wish to export the resulting filtered packets to a new file via "File -> Export Specified Packets... -> All packets:Displayed" so you won't have to keep filtering for those streams anymore.
If you have a large number of streams to filter, then you are better off scripting something. Here's a script you can use that seems to work well in my testing on my Linux machine. If you're using Windows, you will need to write an equivalent batch file, or you may be able to use it as is if you have Cygwin installed.
#!/bin/sh
# Check usage
if [ ${#} -lt 2 ] ; then
echo "Usage: $0 <infile> <outfile>"
exit 0
fi
infile=${1}
outfile=${2}
# TODO: Could also pass the filter on the command-line too.
filter="ip.dst eq 192.168.10.44 and tcp.len > 0"
stream_filter=
for stream in $(tshark -r ${infile} -Y "${filter}" -T fields -e tcp.stream | sort -u | tr -d '\r')
do
if [[ -z ${stream_filter} ]] ; then
stream_filter="tcp.stream eq ${stream}"
else
stream_filter+=" or tcp.stream eq ${stream}"
fi
done
tshark -r ${infile} -Y "${stream_filter}" -w ${outfile}
echo "Wrote ${outfile}"

wire shark log file conversion to text file through cli (in windows7)

For some automation purpose I have below requirements for the Wireshark log file(.pcap).
1-Conversion of Wireshark logs(.pcap file ) to text file with detail of packets.
2-Conversion of Wireshark logs (.pcap file) to text file with some filter (eg: bssgp.pdu_type == 0x00) with detail of packets.
I know how to convert the wireshark files to text file through GUI,
But I need the cli commands for the same to automate the procedure.
Thanks in advance
To convert a .pcap file to text output, you can run:
tshark -V -r file.pcap > file.txt
If you only want to convert certain packets that match a Wireshark display filter, then using your filter, you can run:
tshark -Y "bssgp.pdu_type == 0x00" -V -r file.pcap > file.txt
If the -V option provides too much detail, you can limit the detail to specific protocol(s) by using the -O option instead. For example, to provide details for bssgp only and a summary for all other protocols, try:
tshark -Y "bssgp.pdu_type == 0x00" -O bssgp -r file.pcap > file.txt
Refer to the tshark man page for more details about these options.

How to extract full set of features from an existing pcap file using tshark or any other tool?

I am new to network traffic analysis.
I have used the following Tshark command, but no luck.
C:\Program Files\Wireshark>tshark -r C:\Users\Ravi\Desktop\IDS-augustdocuments\iscxdataset\testbed13jun.pcapCopy\split\small_00057_20100613213752.pcap separator=, -R "tcp.dat
a" -T fields frame.number -e appName -e totalSourceBytes > C:\Users\Ravi\Desktop\IDS-augustdocuments\iscxdataset\testbed13jun.pcapCopy\split\18oct.csv
tshark: "=" was unexpected in this context.
Any suggestions to extract features like Direction ( for the flows), totalSourceBytes, totalDestinationBytes, totalDestinationPackets, totalSourcePackets, sourceTCPFlagsDescription etc.
Yes. Bro IDS or Argus (Auditing Network Activit).
Argus example:
racluster -L0 -m proto -r filepcap.arg -s proto saddr daddr spkts dpkts sbytes dbytes
Proto SrcAddr DstAddr SrcPkts DstPkts SrcBytes DstBytes
udp 84.125.xxx.xxx 0.0.0.0 2634 2580 205131 317889
tcp 84.125.xxx.xxx 0.0.0.0 34143 42585 6078099 48276978
arp 84.125.xxx.xxx 84.xxx.xxx.x 3 3 126 180
Best Regards,
You have to use quotes:
separator=","
I used Bro IDS to get the required fields from the conn.log file.
1) Configure the Bro IDS
(Follow this link to install Bro IDS)
https://www.digitalocean.com/community/tutorials/how-to-install-bro-ids-2-2-on-ubuntu-12-04
2) Start the Bro IDs
3) use the command "bro -r your pcap file.pcap" and this will generate a .log files in the current directory.
4) Inspect the logs like conn.log, dns.log, http.log, etc. for different information from the pcap log file.

ffmpeg udp/tcp stream receive frame not same as sent

I am streaming a video on raspberrypi using command:
ffmpeg -re -threads 2 -i sample_video.m2v -f mpegts - | \
ffmpeg -f mpegts -i - -c copy -f mpegts udp://192.168.1.100:12345
The remote PC with 192.168.1.100 uses ffmpeg library to listen to the input stream. For example:
informat = ffmpeg::av_find_input_format("mpegts");
avformat_open_input(&pFormatCtx, "udp://192.168.1.100:12345", informat, options);
However, when I compute the hash value of each decoded frame on two sides (i.e. raspberrypi and PC), they DON'T MATCH at all. A weird thing is, among ~2000 frames, there are in total ~10 frames whose hash value are the same on the sender and receiver side. The match result look like this:
00000....00011000...00011110000...000
where 0 indicates non-match and 1 indicates match. The matched frame appeared 2~6 in sequence and appeared rarely while most of the other frames has different hash value.
The hash is computed on the frame data buffer extracted using avpicture_layout(). On the Pi side, I just stream the video to a local port and there's a local process using the same code to decode and hash the frames:
ffmpeg -re -threads 2 -i sample_video.m2v -f mpegts - | \
ffmpeg -f mpegts -i - -c copy -f mpegts udp://localhost:12345
...
The streaming source raspberry pi, is connected directly to the PC using cable. I don't think it is a packet loss problem. Because, first, I rerun the same process several times and the hash value of the received frames are the same (otherwise the result should be different because packet loss is probabilistic). Secondly, I even try to stream on tcp://192.168.1.100:12345 (and "tcp://192.168.1.100:12345?listen" on PC), and the received frame hash are still the same - different than the hash result on the Pi.
So, does anyone know why the streaming to a remote address will yield different decoded frames? Maybe I am missing some details.
Thanks in advance!!

Resources