How to capture count of particular packets with tcpdump? - wireshark

I am interested in getting a count of all LDAP/Kerberos/DNS packets.
I tried the following, but this captures the full packet.
tcpdump -i any -Z root "tcp port 389 or tcp port 88 or udp port 53" -w ~/ldap_kerberos_dns.cap
Is there a way I can just capture how many ldap/Kerberos/DNS packets were exchanged
without actually capturing the full packet.
Expected output should be something like:
LDAP: 100
Kerberos: 200
UDP: 300

Take a look at tshark statistics:
$ tshark -r 04.pcap -q -z io,phs
===================================================================
Protocol Hierarchy Statistics
Filter:
eth frames:649 bytes:124780
ipv6 frames:605 bytes:116558
udp frames:212 bytes:33686
dhcpv6 frames:171 bytes:28044
dns frames:25 bytes:2914
ntp frames:10 bytes:1300
cldap frames:6 bytes:1428
icmpv6 frames:80 bytes:7008
tcp frames:313 bytes:75864
nbss frames:108 bytes:24063
smb frames:7 bytes:1554
smb2 frames:101 bytes:22509
tcp.segments frames:1 bytes:103
dcerpc frames:16 bytes:4264
epm frames:2 bytes:544
tcp.segments frames:1 bytes:214
drsuapi frames:8 bytes:2352
kerberos frames:16 bytes:9358
tcp.segments frames:8 bytes:2130
dcerpc.cn_deseg_req frames:1 bytes:1514
ldap frames:16 bytes:5945
tcp.segments frames:3 bytes:1101
ldap frames:1 bytes:803
ip frames:40 bytes:8018
udp frames:40 bytes:8018
nbdgm frames:30 bytes:7300
smb frames:30 bytes:7300
mailslot frames:30 bytes:7300
browser frames:30 bytes:7300
dns frames:10 bytes:718
arp frames:4 bytes:204
===================================================================
See the man-page for more information.

tshark approach
If you have Wireshark (based on question tags, not the actual question) then tshark along the lines of #joke's comment is one way to go, if you don't mind its verbose stats output:
tshark -i any -n -q -z 'io,stat,0,FRAMES()tcp.port==389,FRAMES()tcp.port==88,FRAMES()udp.port==53'
Capturing on Pseudo-device that captures on all interfaces
^C142 packets captured
=============================================
| IO Statistics |
| |
| Interval size: 4.319 secs (dur) |
| Col 1: FRAMES()tcp.port==389 |
| 2: FRAMES()tcp.port==88 |
| 3: FRAMES()udp.port==53 |
|-------------------------------------------|
| |1 |2 |3 |
| Interval
| Interval | FRAMES | FRAMES | FRAMES |
|-------------------------------------------|
| 0.000 <> 4.319 | 100 | 200 | 300 |
=============================================
Though the output from that is verbose, I don't think you can get closer with tshark alone. Another approach, still more verbose, would be:
tshark -q -z io,phs "tcp port 389 or tcp port 88 or udp port 53"
These two commands do not write a capture file. Use Ctrl+C when you're ready to quit, or see the comment below about -a and autostop conditions. One caveat is that those statistics rely on the protocol dissectors, not the source/destination ports, so there can be discrepancies (e.g. connections with no data, or content non-compliant with protocol), "malformed" packets will be reported instead. It also means you won't get reliable results if you optimize and only capture 40 bytes per packet ("short" TCP or UDP packets will be reported instead).
tcpdump approach
A simple, but inelegant way is to run multiple tcpdump instances (assume bash as a shell) --
for pp in "tcp port 88" "tcp port 389" "udp port 53"; do
tcpdump -i any -Z root $pp -w /dev/null 2> ${pp// /-}.stats &
done
Packets are not written to a capture file (discarded via /dev/null).
Then wait as required, kill the tcpdump processes (by PID as listed, or kill %1 %2 %3 if no other background jobs), and inspect the .stats files:
grep captured *.stats
tcp-port-389.stats:0 packets captured
tcp-port-88.stats:0 packets captured
udp-port-53.stats:4 packets captured
perl approach
Since this is stackoverflow, here's a quick and dirty perl/libpcap solution (just add error handling):
#!/usr/bin/perl
use Net::Pcap;
use NetPacket::Ethernet;
use NetPacket::IP;
use NetPacket::TCP;
use strict;
use warnings;
my ($dev,$err,$address, $netmask,$pcap,$filter,$cfilter,$fd,%stats);
my $bail=0;
my ($rin,$rout)=('','');
$SIG{INT} = sub { print "quit...\n"; $bail=1; };
$filter='(tcp port 88 or tcp port 389 or udp port 53)';
$dev=Net::Pcap::lookupdev(\$err);
Net::Pcap::lookupnet($dev, \$address, \$netmask, \$err);
$pcap = Net::Pcap::open_live($dev, 60, 1, 0, \$err);
Net::Pcap::pcap_setnonblock($pcap, 1, \$err);
$fd=Net::Pcap::pcap_get_selectable_fd($pcap);
vec($rin,$fd,1)=1;
Net::Pcap::compile( $pcap, \$cfilter, $filter, 0, $netmask);
Net::Pcap::setfilter($pcap, $cfilter);
while (Net::Pcap::dispatch($pcap,0, \&handlepackets, '')>=0) {
select($rout=$rin,undef,undef,0.250);
printf("."); $|=1;
$bail && last;
};
Net::Pcap::freecode($cfilter);
Net::Pcap::close($pcap);
sub handlepackets() {
my ($user_data, $header, $packet) = #_;
my $ether = NetPacket::Ethernet::strip($packet);
my $ip = NetPacket::IP->decode($ether);
my $tcp = NetPacket::TCP->decode($ip->{'data'});
if ($tcp->{'src_port'} < 1024) {
$stats{$tcp->{'src_port'}}++;
} elsif ($tcp->{'dest_port'} < 1024) {
$stats{$tcp->{'dest_port'}}++;
}
}
END {
if (keys %stats) {
for my $kk (sort keys %stats) {
my ($name, $aliases, $port, $prot)=getservbyport($kk,"tcp");
printf("%12s %4i\n",$name,$stats{$kk});
}
} else { printf("No stats...\n"); }
}

Related

Extract Unique IP sources from pcap file

My approach is to:
1 - Use tshark and export the list in a txt file.
tshark -r file.cap -T fields -e ip.src > output.txt
2 - Use sort to delete the double ips
sort output.txt | uniq > uniqueip.txt
3 - use uniqueip.txt to count the lines with
wc -l output.txt
I noticed right after i get the output.txt has some strange formatting where some ips are in line? why are they not in a new line?
This it the output.txt
"58.176.204.64"
"180.168.211.204"
"103.248.63.253"
"216.245.214.196,146.231.254.240"
"112.104.105.79"
"216.245.214.196,146.231.254.131"
"112.104.105.79"
"10.0.61.65,146.231.254.12"
The reason why some lines contain more than 1 IP address separated by a comma is because the packet itself contains more than 1 IP header. Such is the case for tunneling protocols or for ICMP error packets whose payload contains the original IP header that caused the ICMP error packet to be sent in the first place, or for other types of packets as well.

how to remove zero packets (empty streams) records in wireshark

I am very new to wireshark. in my day to day job i need to remove the packet bytes zero records from captured PCAP file. please help me in this process. attached image is for reference
wireshark packets zero.png
Since you have 47 TCP Streams and 28 that you want to remove, it might be a bit faster to filter for all the TCP streams that you do want to keep since there are only 19 of those.
For the 19 streams you want:
Right-click on the first TCP conversation and choose "Prepare a Filter -> Selected -> A<-->B".
For the next 17 TCP conversations, right-click on each one and choose "Prepare a Filter -> ... And Selected -> A<-->B".
Finally, for the last TCP stream, right-click on the TCP conversation and choose "Apply as Filter -> ... And Selected -> A<-->B".
You may wish to export the resulting filtered packets to a new file via "File -> Export Specified Packets... -> All packets:Displayed" so you won't have to keep filtering for those streams anymore.
If you have a large number of streams to filter, then you are better off scripting something. Here's a script you can use that seems to work well in my testing on my Linux machine. If you're using Windows, you will need to write an equivalent batch file, or you may be able to use it as is if you have Cygwin installed.
#!/bin/sh
# Check usage
if [ ${#} -lt 2 ] ; then
echo "Usage: $0 <infile> <outfile>"
exit 0
fi
infile=${1}
outfile=${2}
# TODO: Could also pass the filter on the command-line too.
filter="ip.dst eq 192.168.10.44 and tcp.len > 0"
stream_filter=
for stream in $(tshark -r ${infile} -Y "${filter}" -T fields -e tcp.stream | sort -u | tr -d '\r')
do
if [[ -z ${stream_filter} ]] ; then
stream_filter="tcp.stream eq ${stream}"
else
stream_filter+=" or tcp.stream eq ${stream}"
fi
done
tshark -r ${infile} -Y "${stream_filter}" -w ${outfile}
echo "Wrote ${outfile}"

GREP - Search only in first n characters of each line, searching for number after special character

my first post here.
In summary: I have a netstat output, using "netstat -an" command in Windows, and I would like to get the top number of one of the columns.
The output of netstat is something like this:
TCP 10.45.43.232:50387 10.42.48.61:902 ESTABLISHED
TCP 10.45.43.232:50559 46.228.47.115:443 CLOSE_WAIT
TCP 10.45.43.232:52501 10.48.48.128:3389 ESTABLISHED
TCP 10.45.43.232:58000 10.46.48.243:63713 ESTABLISHED
The result I want is:
58000
That number is the biggest value on the second column, after the ":" character
So, in essence, I want a grep (and/or sed, awk, etc) which can search through a file, only look in the first 25 characters of each line, and get the highest number after a ":" character.
Tell me if you need more information, thanks in advance!
This can be an approach:
netstat ... | sort -t':' -nrk2 | awk -F"[ :]" '{print $8; exit}'
By pieces:
Sort it based on : as delimiter and taking second column into account:
$ netstat ... | sort -t':' -nrk2
TCP 10.45.43.232:58000 10.46.48.243:63713 ESTABLISHED
TCP 10.45.43.232:52501 10.48.48.128:3389 ESTABLISHED
TCP 10.45.43.232:50559 46.228.47.115:443 CLOSE_WAIT
TCP 10.45.43.232:50387 10.42.48.61:902 ESTABLISHED
Print the biggest:
$ netstat ... | sort -t':' -nrk2 | awk -F"[ :]" '{print $8; exit}'
58000
Or better, using Mark Setchell's approach to fetch the last item:
$ netstat ... | sort -t':' -nrk2 | awk '{sub(/.*:/,"",$2); print $2; exit}'
58000
if the output has leading space/tabs:
netstat...|awk -F':|\\s*' '{p=$4>p?$4:p}END{print p}'
if there is no leading spaces:
netstat ..| awk -F':|\\s*' '{p=$3>p?$3:p}END{print p}'
I would go with this:
netstat -an | awk '{sub(/.*:/,"",$2); if($2>max)max=$2} END{print max}'
The sub() part strips all characters up to and including a colon, off the second field thereby extracting the port. If that is greater than max, max is updated. At the end, max is printed.
Here is yet another way using GNU awk:
netstat ... | awk '{split($2,tmp,/:/); a[tmp[2]]++}END{n=asorti(a);print a[n]}'
We split the second field of second column (delimited by :) in a tmp array
We populate the values as keys in array a.
In the END we use GNU awk asorti function which sorts the keys and print the highest.
You can also do it with coreutils alone:
netstat ... | cut -d: -f2 | cut -d' ' -f1 | sort -nr | head -n1
Output:
58000

selecting major flows at once in a huge pcap in wireshark

i have a large pcap with more than 1000 tcp flows. i want to filter major flows say with packets greater than 100. if i go to conversations and right click on those flows, i can filter those flows, but then i have to do it several times and since i have huge pcap, it may exceed 100. is there any other quick display filter i can use which will give me flows having number of packets > n (n being any +ve integer).
say some filter like :
flow.num_pkt > 100
which can give me all such flows.
thanks a lot,
any help will be greatly appreciated.
Bro is an apt tool for connection-oriented analysis. To find the number of packets per flow, you run simply run Bro on the trace and extract the value from the logs:
bro -r trace.pcap
bro-cut id.orig_h id.orig_p id.resp_h id.resp_p orig_pkts resp_pkts < conn.log \
| awk '$5+$6 > 100 {print $1,$2,$3,$4,$5,$6}' \
| sort -rn -k 5 \
| head
This gives the following output:
192.168.1.105 49325 137.226.34.227 80 73568 146244
192.168.1.105 49547 198.189.255.74 80 16764 57098
192.168.1.105 49531 198.189.255.74 80 5186 14843
192.168.1.105 49255 198.189.255.73 80 4749 32164
192.168.1.104 1422 69.147.86.184 80 2657 2656
192.168.1.105 49251 198.189.255.74 80 2254 13854
192.168.1.1 626 224.0.0.1 626 2175 0
192.168.1.105 49513 198.189.255.82 80 2010 3852
192.168.1.103 2026 151.207.243.129 80 1953 2570
192.168.1.105 49330 143.166.11.10 64334 1514 3101
The tool bro-cut ships with Bro and provides a convenient way to extract certain named columns from the logs. For this task, you want:
id.orig_h: IP of the connection originator (source)
id.orig_p: Transport-layer port of the connection originator (source)
id.resp_h: IP of the connection responder (destination)
id.resp_p: Transport-layer port of the connection responder (source)
orig_pkts: Number of packets sent by the originator
resp_pkts: Number of packets sent by the responder
Note the awk filter expression:
awk '$5+$6 > 100 {print ...}'
It restricts the output to those connections that have a total number of packets greater than 100.
Unless you have fixed-size packets, I encourage you to also investigate other metrics, such as packet size (IP or TCP payload). These are readily in the connection logs via the orig_bytes and resp_bytes columns.

Plot RTT histogram using wireshark or other tool

I have a little office network and I'm experiencing a huge internet link latency. We have a simple network topology: a computer configured as router running ubuntu server 10.10, 2 network cards (one to internet link, other to office network) and a switch connecting 20 computers. I have a huge tcpdump log collected at the router and I would like to plot a histogram with the RTT time of all TCP streams to try to find out the best solution to this latency problem. So, could somebody tell me how to do it using wireshark or other tool?
Wireshark or tshark can give you the TCP RTT for each received ACK packet using tcp.analysis.ack_rtt which measures the time delta between capturing a TCP packet and the ACK for that packet.
You need to be careful with this as most of your ACK packets will be from your office machines ACKing packets received from the internet, so you will be measuring the RTT between your router seeing the packet from the internet and seeing the ACK from your office machine.
To measure your internet RTT you need to look for ACKS from the internet (ACKing data sent from your network). Assuming your office machines have IP addresses like 192.168.1.x and you have logged all the data on the LAN port of your router you could use a display filter like so:
tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24
To dump the RTTs into a .csv for analysis you could use a tshark command like so;
tshark -r router.pcap -Y "tcp.analysis.ack_rtt and ip.dst==192.168.1.255/24" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d > rtt.csv
The -r option tells tshark to read from your .pcap file
The -Y option specifies the display filter to use (-R without -2 is deprecated)
The -e option specifies the field to output
The -T options specify the output formatting
You can use the mergecap utility to merge all your pcap files into one one file before running this command. Turning this output into a histogram should be easy!
Here's the 5-min perlscript inspired by rupello's answer:
#!/usr/bin/perl
# For a live histogram of rtt latencies, save this file as /tmp/hist.pl and chmod +x /tmp/hist.pl, then run:
# tshark -i wlp2s0 -Y "tcp.analysis.ack_rtt and ip.dst==192.168.0.0/16" -e tcp.analysis.ack_rtt -T fields -E separator=, -E quote=d | /tmp/hist.pl
# Don't forget to update the interface "wlp2s0" and "and ip.dst==..." bits as appropriate, type "ip addr" to get those.
#t[$m=0]=20;
#t[++$m]=10;
#t[++$m]=5;
#t[++$m]=2;
#t[++$m]=1;
#t[++$m]=0.9;
#t[++$m]=0.8;
#t[++$m]=0.7;
#t[++$m]=0.6;
#t[++$m]=0.5;
#t[++$m]=0.4;
#t[++$m]=0.3;
#t[++$m]=0.2;
#t[++$m]=0.1;
#t[++$m]=0.05;
#t[++$m]=0.04;
#t[++$m]=0.03;
#t[++$m]=0.02;
#t[++$m]=0.01;
#t[++$m]=0.005;
#t[++$m]=0.001;
#t[++$m]=0;
#h[0]=0;
while (<>) {
s/\"//g; $n=$_; chomp($n); $o++;
for ($i=$m;$i>=0;$i--) { if ($n<=$t[$i]) { $h[$i]++; $i=-1; }; };
if ($i==-1) { $h[0]++; };
print "\033c";
for (0..$m) { printf "%6s %6s %8s\n",$t[$_],sprintf("%3.2f",$h[$_]/$o*100),$h[$_]; };
}
The newer versions of tshark seem to work better with a "stdbuf -i0 -o0 -e0 " in front of the "tshark".
PS Does anyone know if wireshark has DNS and ICMP rtt stats built in or how to easily get those?
2018 Update: See https://github.com/dagelf/pping

Resources