I have a problem in "Date first seen" column in the result generated by nfdump. I have enabled netflow on an ESXi 5.5 to send netflow data to a netflow server. up to now everything is OK and I can capture netflow data with nfcapd with the following command:
nfcapd -D -z -u netflow -p 9996 -n Esxi,192.168.20.54,/data/nfdump -S2 -e
but the problem is that when I filter the traffic with nfdump (e.g. with nfdump -R nfdump5/2016/ -c 10) I see "1970-01-01 03:30:00.000" for "Date first seen" column in all entries!!! What should I do to get the right time stamps?
Any help is appreciated.
The NetFlow header has a timestamp for the whole datagram; most likely, their export is using the "first seen" field as an offset from that. It's possible nfdump isn't correctly interpreting that field; I'd recommend having a look at the capture in Wireshark, which I've found to be pretty reliable in decoding NetFlow. That will also let you examine the flow records directly to see if the timestamps are really coming across that small, or are just being misinterpreted. Just remember that if you're capturing NetFlow v9 or IPFIX, you'll need to make sure that your capture includes a template datagram.
If the ESXi's NetFlow isn't exporting timestamps correctly, you can also look into monitoring using a small virtual machine running a software flow exporter (there are a number of free ones available - just Google "free flow exporter") with an interface in promiscuous mode.
Related
I am using wireshark to decode gsm packets. Wireshark picks up all communication at that frequency and decodes it.
Overview of the packets obtained.
Now in the system information 5 packet, we have the required parameter.Details of system information 5 packet
I wish to write a code such that if there is only a single value in the last entry of the packet i.e. the neighbour list, an alert inform of a pop up message or something is generated.
These packets have to captured and processed in real time.
I have installed pyshark but cant figure out how to move forward.
Please help
I can only give a partial answer, which is how to detect if there's only a single value or not. My suggestion is to use tshark and then post-process the data using another tool, such as wc. For example:
tshark -i lo -Y "gsm_a.rr.arfcn_list" -T fields -e gsm_a.rr.arfcn_list | wc -w
If the result is 1, then there's only 1 entry in the list.
How you generate an alert from this, I'm not sure.
I have fragmented packets coming from multiple sources stored in a *.pcap file. I need to merge all these payloads coming from the same source and extract the payloads in a file.
I need to do the above task using tcpdump or tshark commands.
Please let me know if I have missed something or if you need more clarification.
With pdml2flow (pypi) you can merge/aggregate frames based on any fields extracted by tshark / wireshark and print them as JSON, XML or any other format. You can also easily create a new plugin which extracts, rearranges and saves the payload for your use-case.
For example if you'd like to aggregate all frames with the same source and destination mac and print them as JSON you could do:
tshark -i interface -Tpdml | pdml2flow -f eth.src -f eth.dst +json
This might help if you can find a set of fields based on which you would like to merge/aggregate.
Disclosure: I am the author of pdml2flow.
I am capturing the router interface from my Fritzbox modem then using Wireshark to view it.
I'd like a script to filter a number of Source IP's and then sum all the Length's (data quantity) associated with them. Effectively giving me the total data usage of each IP address I monitor.
Conceptually is sounds simple, but after a look at Lua, I think I'm in over my head.
Thanks.
Maybe tshark can help you achieve your goal directly, without the need for a script at all? Have you tried something like:
tshark -r file.pcap -z endpoints,ip,"ip.src eq 1.1.1.1" -q
... where 1.1.1.1 represents the IP address of the endpoint you're interested in gathering statistics for? You can specify as many endpoints as you need to by or ing them together, or even use a subnet such as "ip.src eq 1.1.1.0/24", for example.
I want to have a list of source IP addresses of an interface outbound traffic. How could I find the direction of a packet whether it's inbound or outbound reading traffic using libpcap? I don't know the subnet information of either side. And there are clients/servers on both sides, so I can't rely on port number ranges to filter traffic.
Why there is no information in libpcap packet header about direction, or filter option like inbound in pcap-filter?
Netsniff-NG, while not relying on libpcap, supports Linux kernel packet type extensions.
They're documented
here
One of the packet types is outgoing and commented as "outgoing of any type".
The following example will capture all packets leaving your interface.
$ netsniff-ng --in eth0 --out outgoing.pcap --type outgoing
Using this you can utilize other command-line tools to read the PCAP file and pull out all the source
addresses. Maybe something *nix-ey like this:
$ tcpdump -nnr outgoing.pcap | cut -d " " -f3 | cut -d . -f1-4
Note: I haven't tried this on a router.
you could use "ether src" or "ether dst" to filter packet direction. This require you to know the mac address of the interface.
You can select a direction that packets will be capture calling pcap_setdirection() before pcap_loop().
For example, to capture incoming packets only you need to write:
handle = pcap_open_live("eth0", 65535, 1, 0, errbuf);
pcap_setdirection(handle, PCAP_D_IN);
pcap_loop(handle, -1, process_packet, NULL);
Possible directions are: PCAP_D_IN, PCAP_D_OUT, PCAP_D_INOUT.
See tcpdump.org/manpages/pcap_setdirection.3pcap.txt
The PCAP file format does not contain a field that holds the interface used during the capture. With that said, the newer PCAP-NG file format, currently used by Wireshark & Tshark, supports it along with packet direction.
Existing pcap-ng features:
packet dropped count
annotations (comments)
local IP address
interface & direction
hostname <-> IP address database
PcapNg
It sounds like you're capturing from a router or firewall so something like the following
would not work.
ip src 192.168.1.1
Capturing the traffic into flows may be an option but it still will not provide you with direction information. Though, you will be able to determine the source and destinations address easily. If you have an existing pcap you can convert it to the ARGUS format:
argus -r capture.pcap -w capture.argus
ra -nnr capture.argus
Other tools, some w/ examples, that can easily obtain end-points/hosts are:
ntop -f capture.pcap
tcpprof -nr capture.pcap
Wireshark Endpoints
flow-tools
You'll have to parse out the information you want, but I don't think that's too much trouble. I recommend taking a look at PCAP-NG if you can't work with this.
Is there a way to capture only the data layer and disregard the upper layers in wireshark? If not, is there a different packet dump utility that can do this? PREFERABLY 1 file per packet!
What I am looking for: A utility that dumps only the data (the payload) layer to a file.
This is programming related...! What I really want to do is to compare all of the datagrams in order to start to understand a third party encoding/protocol. Ideally, and what would be great, would be a hex compare utility that compares multiple files!
You should try right-clicking on a packet and select "Follow TCP Stream". Then you can save the TCP communication into a raw file for further processing. This way you won't get all the TCP/IP protocoll junk.
There is a function to limit capture size in Wireshark, but it seems that 68bytes is the smallest value. There are options to starting new files after a certain number of kilo, mega, gigabytes, but again the smallest is 1-kilobyte, so probably not useful.
I would suggest looking at the pcap library and rolling your own. I've done this in the past using the PERL Net::Pcap library, but it could easily be done it other languages too.
If you have Unix/Linux available you might also look into tcpdump. You can limit amount of data captured with -s. For example "-s 14" would typically get you the Ethernet header, which I assume is what you mean by the datalink layer. There are also options for controlling how often files are created by specifying file size with -C. So theoretically if you set the file size to the capture size, you'll get one file per packet.
Using tshark I was able to print data only, by decoding as telnet and printing field telnet.data
tshark -r file.pcap -d tcp.port==80,telnet -T fields -e telnet.data
GET /test.js HTTP/1.1\x0d\x0a,User-Agent: curl/7.35.0\x0d\x0a,Host: 127.0.0.1\x0d\x0a,Accept: */*\x0d\x0a,\x0d\x0a
HTTP/1.1 404 Not Found\x0d\x0a,Server: nginx/1.4.6 (Ubuntu)\x0d\x0a,Date: Fri, 15 Jan 2016 11:32:58 GMT\x0d\x0a,Content-Type: text/html\x0d\x0a,Content-Length: 177\x0d\x0a,Connection: keep-alive\x0d\x0a,\x0d\x0a,<html>\x0d\x0a,<head><title>404 Not Found</title></head>\x0d\x0a,<body bgcolor=\"white\">\x0d\x0a,<center><h1>404 Not Found</h1></center>\x0d\x0a,<hr><center>nginx/1.4.6 (Ubuntu)</center>\x0d\x0a,</body>\x0d\x0a,</html>\x0d\x0a
Not perfect but it was good enough for what I needed, I hope it helps some one.