My approach is to:
1 - Use tshark and export the list in a txt file.
tshark -r file.cap -T fields -e ip.src > output.txt
2 - Use sort to delete the double ips
sort output.txt | uniq > uniqueip.txt
3 - use uniqueip.txt to count the lines with
wc -l output.txt
I noticed right after i get the output.txt has some strange formatting where some ips are in line? why are they not in a new line?
This it the output.txt
"58.176.204.64"
"180.168.211.204"
"103.248.63.253"
"216.245.214.196,146.231.254.240"
"112.104.105.79"
"216.245.214.196,146.231.254.131"
"112.104.105.79"
"10.0.61.65,146.231.254.12"
The reason why some lines contain more than 1 IP address separated by a comma is because the packet itself contains more than 1 IP header. Such is the case for tunneling protocols or for ICMP error packets whose payload contains the original IP header that caused the ICMP error packet to be sent in the first place, or for other types of packets as well.
I have a PCAP file I exported from wireshark that's rather large. The PCAP looks as follows:
No. Time Source Destination Protocol Length Info
1 0 192.168.100.180 8.8.8.8 DNS 95 Standard query 0xf948 A detectportal.firefox.com OPT
2 0.000159827 192.168.100.180 8.8.8.8 DNS 95 Standard query 0xaf8a AAAA detectportal.firefox.com OPT
3 0.002743676 8.8.8.8 192.168.100.180 DNS 206 Standard query response 0xf948 A detectportal.firefox.com CNAME detectportal.prod.mozaws.net CNAME prod.detectportal.prod.cloudops.mozgcp.net A 34.107.221.82 OPT
4 0.002774349 8.8.8.8 192.168.100.180 DNS 218 Standard query response 0xaf8a AAAA detectportal.firefox.com CNAME detectportal.prod.mozaws.net CNAME prod.detectportal.prod.cloudops.mozgcp.net AAAA 2600:1901:0:38d7:: OPT
12 0.168337688 192.168.100.180 8.8.8.8 DNS 82 Standard query 0x6b67 A mozilla.org OPT
13 0.170840019 8.8.8.8 192.168.100.180 DNS 98 Standard query response 0x6b67 A mozilla.org A 63.245.208.195 OPT
14 0.201381247 192.168.100.180 8.8.8.8 DNS 82 Standard query 0xce22 AAAA
...
I want to split the PCAP file into 10 smaller PCAPs based on the time column. Basically, 10 PCAP files, each file spanning about 50 seconds.
I think editcap may help me, but I'm not sure what the proper command would be. (Never even heard of editcap until ~2 days ago lol)
You can use editcap as below:
editcap -i <seconds per file> <input_file> <output_file_format>
More details here - https://www.wireshark.org/docs/man-pages/editcap.html
The option -Y, -2 and -R in tshark confuse me a long time.
After I read the manual, I know that -Y is used in single-pass filter and -2 in 2-pass filter (in case where we can not get some info until 1st pass filter is over)
But I still can not understand what is the difference between -2 -Y 'blabla' and -2 -R 'balabala' and -2 -Y 'blalal' -R 'blala'
And I also did an experiment that drive me crazy:
tshark -n -r test.pcap -2 -R 'frame.number > 0'
1 0.000000 10.140.28.17 -> 10.74.68.58 TCP 80 62276 > 8989 [SYN, ECN, CWR] Seq=0 Win=65535 Len=0 MSS=1460 WS=32 TSval=330325315 TSecr=0 SACK_PERM=1
2 0.000056 10.74.68.58 -> 10.140.28.17 TCP 76 8989 > 62276 [SYN, ACK, ECN] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=2078759468 TSecr=330325315 WS=128
3 0.000678 10.140.28.17 -> 10.74.68.58 TCP 68 62276 > 8989 [ACK] Seq=1 Ack=1 Win=131744 Len=0 TSval=330325316 TSecr=2078759468
4 0.000756 10.140.28.17 -> 10.74.68.58 HTTP 158 GET /index.html HTTP/1.1
5 0.000770 10.74.68.58 -> 10.140.28.17 TCP 68 8989 > 62276 [ACK] Seq=1 Ack=91 Win=29056 Len=0 TSval=2078759468 TSecr=330325316
But when I execute tshark -n -r test.pcap -2 -R 'frame.number > 1', there is nothing printed. How to explain this?
My tshark version is: TShark 1.10.6 (v1.10.6 from master-1.10)
Can you help me with this problem? Thank you in advance!
I answered this question on http://ask.wireshark.org, but I'll paste my answer here as well in case anyone looks here for an answer instead of there.
-R specifies a read filter, so only matching packets from a file are read and processed; unmatched packets are essentially treated as if the file didn't contain them at all. Contrast this with -Y, which specifies a display filter, so only matching packets are displayed, but all packets are still read and processed.
The problem you're seeing with frame.number is a known bug that was determined not worth fixing long ago. See Bug 380, "wireshark -R doesn't support 'frame.number' as a read filter ".
You can also follow along some of the history behind -R vs. -Y here:
Bug 8223
Bug 8316
Bug 8529
Bug 9048
Wireshark-dev mailing list discussion:
Wireshark's Local archive
MARC archives
The Mail Archive
I am interested in getting a count of all LDAP/Kerberos/DNS packets.
I tried the following, but this captures the full packet.
tcpdump -i any -Z root "tcp port 389 or tcp port 88 or udp port 53" -w ~/ldap_kerberos_dns.cap
Is there a way I can just capture how many ldap/Kerberos/DNS packets were exchanged
without actually capturing the full packet.
Expected output should be something like:
LDAP: 100
Kerberos: 200
UDP: 300
Take a look at tshark statistics:
$ tshark -r 04.pcap -q -z io,phs
===================================================================
Protocol Hierarchy Statistics
Filter:
eth frames:649 bytes:124780
ipv6 frames:605 bytes:116558
udp frames:212 bytes:33686
dhcpv6 frames:171 bytes:28044
dns frames:25 bytes:2914
ntp frames:10 bytes:1300
cldap frames:6 bytes:1428
icmpv6 frames:80 bytes:7008
tcp frames:313 bytes:75864
nbss frames:108 bytes:24063
smb frames:7 bytes:1554
smb2 frames:101 bytes:22509
tcp.segments frames:1 bytes:103
dcerpc frames:16 bytes:4264
epm frames:2 bytes:544
tcp.segments frames:1 bytes:214
drsuapi frames:8 bytes:2352
kerberos frames:16 bytes:9358
tcp.segments frames:8 bytes:2130
dcerpc.cn_deseg_req frames:1 bytes:1514
ldap frames:16 bytes:5945
tcp.segments frames:3 bytes:1101
ldap frames:1 bytes:803
ip frames:40 bytes:8018
udp frames:40 bytes:8018
nbdgm frames:30 bytes:7300
smb frames:30 bytes:7300
mailslot frames:30 bytes:7300
browser frames:30 bytes:7300
dns frames:10 bytes:718
arp frames:4 bytes:204
===================================================================
See the man-page for more information.
tshark approach
If you have Wireshark (based on question tags, not the actual question) then tshark along the lines of #joke's comment is one way to go, if you don't mind its verbose stats output:
tshark -i any -n -q -z 'io,stat,0,FRAMES()tcp.port==389,FRAMES()tcp.port==88,FRAMES()udp.port==53'
Capturing on Pseudo-device that captures on all interfaces
^C142 packets captured
=============================================
| IO Statistics |
| |
| Interval size: 4.319 secs (dur) |
| Col 1: FRAMES()tcp.port==389 |
| 2: FRAMES()tcp.port==88 |
| 3: FRAMES()udp.port==53 |
|-------------------------------------------|
| |1 |2 |3 |
| Interval
| Interval | FRAMES | FRAMES | FRAMES |
|-------------------------------------------|
| 0.000 <> 4.319 | 100 | 200 | 300 |
=============================================
Though the output from that is verbose, I don't think you can get closer with tshark alone. Another approach, still more verbose, would be:
tshark -q -z io,phs "tcp port 389 or tcp port 88 or udp port 53"
These two commands do not write a capture file. Use Ctrl+C when you're ready to quit, or see the comment below about -a and autostop conditions. One caveat is that those statistics rely on the protocol dissectors, not the source/destination ports, so there can be discrepancies (e.g. connections with no data, or content non-compliant with protocol), "malformed" packets will be reported instead. It also means you won't get reliable results if you optimize and only capture 40 bytes per packet ("short" TCP or UDP packets will be reported instead).
tcpdump approach
A simple, but inelegant way is to run multiple tcpdump instances (assume bash as a shell) --
for pp in "tcp port 88" "tcp port 389" "udp port 53"; do
tcpdump -i any -Z root $pp -w /dev/null 2> ${pp// /-}.stats &
done
Packets are not written to a capture file (discarded via /dev/null).
Then wait as required, kill the tcpdump processes (by PID as listed, or kill %1 %2 %3 if no other background jobs), and inspect the .stats files:
grep captured *.stats
tcp-port-389.stats:0 packets captured
tcp-port-88.stats:0 packets captured
udp-port-53.stats:4 packets captured
perl approach
Since this is stackoverflow, here's a quick and dirty perl/libpcap solution (just add error handling):
#!/usr/bin/perl
use Net::Pcap;
use NetPacket::Ethernet;
use NetPacket::IP;
use NetPacket::TCP;
use strict;
use warnings;
my ($dev,$err,$address, $netmask,$pcap,$filter,$cfilter,$fd,%stats);
my $bail=0;
my ($rin,$rout)=('','');
$SIG{INT} = sub { print "quit...\n"; $bail=1; };
$filter='(tcp port 88 or tcp port 389 or udp port 53)';
$dev=Net::Pcap::lookupdev(\$err);
Net::Pcap::lookupnet($dev, \$address, \$netmask, \$err);
$pcap = Net::Pcap::open_live($dev, 60, 1, 0, \$err);
Net::Pcap::pcap_setnonblock($pcap, 1, \$err);
$fd=Net::Pcap::pcap_get_selectable_fd($pcap);
vec($rin,$fd,1)=1;
Net::Pcap::compile( $pcap, \$cfilter, $filter, 0, $netmask);
Net::Pcap::setfilter($pcap, $cfilter);
while (Net::Pcap::dispatch($pcap,0, \&handlepackets, '')>=0) {
select($rout=$rin,undef,undef,0.250);
printf("."); $|=1;
$bail && last;
};
Net::Pcap::freecode($cfilter);
Net::Pcap::close($pcap);
sub handlepackets() {
my ($user_data, $header, $packet) = #_;
my $ether = NetPacket::Ethernet::strip($packet);
my $ip = NetPacket::IP->decode($ether);
my $tcp = NetPacket::TCP->decode($ip->{'data'});
if ($tcp->{'src_port'} < 1024) {
$stats{$tcp->{'src_port'}}++;
} elsif ($tcp->{'dest_port'} < 1024) {
$stats{$tcp->{'dest_port'}}++;
}
}
END {
if (keys %stats) {
for my $kk (sort keys %stats) {
my ($name, $aliases, $port, $prot)=getservbyport($kk,"tcp");
printf("%12s %4i\n",$name,$stats{$kk});
}
} else { printf("No stats...\n"); }
}
I have a little office network and I'm experiencing a huge internet link latency. We have a simple network topology: a computer configured as router running ubuntu server 10.10, 2 network cards (one to internet link, other to office network) and a switch connecting 20 computers. I have a huge tcpdump log collected at the router and I would like to plot a histogram with the RTT time of all TCP streams to try to find out the best solution to this latency problem. So, could somebody tell me how to do it using wireshark or other tool?
Wireshark or tshark can give you the TCP RTT for each received ACK packet using tcp.analysis.ack_rtt which measures the time delta between capturing a TCP packet and the ACK for that packet.
You need to be careful with this as most of your ACK packets will be from your office machines ACKing packets received from the internet, so you will be measuring the RTT between your router seeing the packet from the internet and seeing the ACK from your office machine.
To measure your internet RTT you need to look for ACKS from the internet (ACKing data sent from your network). Assuming your office machines have IP addresses like 192.168.1.x and you have logged all the data on the LAN port of your router you could use a display filter like so:
tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24
To dump the RTTs into a .csv for analysis you could use a tshark command like so;
tshark -r router.pcap -Y "tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d > rtt.csv
The -r option tells tshark to read from your .pcap file
The -Y option specifies the display filter to use (-R without -2 is deprecated)
The -e option specifies the field to output
The -T options specify the output formatting
You can use the mergecap utility to merge all your pcap files into one one file before running this command. Turning this output into a histogram should be easy!
Here's the 5-min perlscript inspired by rupello's answer:
#!/usr/bin/perl
# For a live histogram of rtt latencies, save this file as /tmp/hist.pl and chmod +x /tmp/hist.pl, then run:
# tshark -i wlp2s0 -Y "tcp.analysis.ack_rtt and ip.dst==192.168.0.0/16" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d | /tmp/hist.pl
# Don't forget to update the interface "wlp2s0" and "and ip.dst==..." bits as appropriate, type "ip addr" to get those.
#t[$m=0]=20;
#t[++$m]=10;
#t[++$m]=5;
#t[++$m]=2;
#t[++$m]=1;
#t[++$m]=0.9;
#t[++$m]=0.8;
#t[++$m]=0.7;
#t[++$m]=0.6;
#t[++$m]=0.5;
#t[++$m]=0.4;
#t[++$m]=0.3;
#t[++$m]=0.2;
#t[++$m]=0.1;
#t[++$m]=0.05;
#t[++$m]=0.04;
#t[++$m]=0.03;
#t[++$m]=0.02;
#t[++$m]=0.01;
#t[++$m]=0.005;
#t[++$m]=0.001;
#t[++$m]=0;
#h[0]=0;
while (<>) {
s/\"//g; $n=$_; chomp($n); $o++;
for ($i=$m;$i>=0;$i--) { if ($n<=$t[$i]) { $h[$i]++; $i=-1; }; };
if ($i==-1) { $h[0]++; };
print "\033c";
for (0..$m) { printf "%6s %6s %8s\n",$t[$_],sprintf("%3.2f",$h[$_]/$o*100),$h[$_]; };
}
The newer versions of tshark seem to work better with a "stdbuf -i0 -o0 -e0 " in front of the "tshark".
PS Does anyone know if wireshark has DNS and ICMP rtt stats built in or how to easily get those?
2018 Update: See https://github.com/dagelf/pping