I have a long running tcpdump that runs over ssh which outputs to a file.
ssh remotehost.example.com -p "tcpdump -i eth0 -w - " > capture-`date '+%Y%m%d-%H%M'`.pcap
How can I rotate that file by day or week? It is important to not duplicate or lose any content.
From the tcpdump man page:
-G If specified, rotates the dump file specified with the -w option every rotate_sec‐
onds seconds. Savefiles will have the name specified by -w which should include a
time format as defined by strftime(3). If no time format is specified, each new
file will overwrite the previous.
If used in conjunction with the -C option, filenames will take the form of
`file<count>'.
For example, if you want to rotate every 10 seconds:
tcpdump -i eth0 -G 10 -w capture-%Y%m%d-%H%M%S.pcap
In your case, you can rotate daily like so:
tcpdump -i eth0 -G 86400 -w capture-%Y%m%d-%H%M.pcap
Related
I want to convert a pcap file to csv/tsv with "Tshark" where each line corresponds to a packet and have the following format:
timestamp src_ip dst_ip protocol
I use this command:
tshark -r <file_name.pcap> -T fields -e frame.time_epoch -e ip.src -e ip.dst -e ip.proto
However, in the displayed output I see some lines to have two src,dst IP addresses and protocol number like this:
1525794300.3842 92.153.107.1,203.46.108.229 203.46.108.229,85.50.172.78 1,1
While for the rest, each line has one src IP, one dst IP and one protocol like the following:
1525794300.3843 185.61.46.124 163.218.99.213 6
Is there any reason that tshark is displaying multiple src and dst ip addresses in a single line? Can we do something so tshark does not do this?
Thanks!
The reason tshark displays multiple src and dst IP addresses as well as multiple protocol numbers is because there are multiple IP headers in the packet. In this case, it's ICMP packet carrying information about another ICMP packet, perhaps a "Time to live exceeded in transit" or some other such error. If you open the file in Wireshark or run tshark -r <file_name.pcap> -Y "icmp", you will be able to see this for yourself.
If you're only interested in the first (outer) IP src and dst addresses and protocol number, then you can limit the output to the first occurrence of each field as follows:
tshark -r <file_name.pcap> -T fields -E occurrence=f -e frame.time_epoch -e ip.src -e ip.dst -e ip.proto
Alternatively, you can specify columns while limiting the field occurrences this way:
On *nix:
tshark -r <file_name.pcap> -o 'gui.column.format:"Epoch Time","%Cus:frame.time_epoch","Src","%Cus:ip.src:1","Dst","%Cus:ip.dst:1","Proto","%Cus:ip.proto:1"'
On Windows:
tshark.exe -r <file_name.pcap> -o "gui.column.format:\"Epoch Time\",\"%Cus:frame.time_epoch\",\"Src\",\"%Cus:ip.src:1\",\"Dst\",\"%Cus:ip.dst:1\",\"Proto\",\"%Cus:ip.proto:1\""
Run tshark -G column-formats for additional help with the column formats.
I have a pcap of ICMP packets. I am trying to use tshark to extract the payload data so that I can extract a specific byte offset.
The tshark documentation is highly convoluted, especially for me, a beginner.
I've been searching around a lot and I'm trying to piece together a command for the purpose of my goal.
I can run the following command:
shark -r test.pcapng -Y icmp -z flow,icmp,network > output.bin
But it only outputs the packet list as it were shown in Wireshark.
For example, I am trying to extract the following byte offset from each packet (offset 22):
How would I go about extracting a specific byte offset with tshark?
EDIT:
Issuing the following command only returns a portion of the payload data, how can I get all of it?
tshark -r test.pcapng -Y "frame.number == 13" -T fields -e data -w output.bin
I've provided an answer over at https://ask.wireshark.org/question/14795/extract-specific-byte-offset-using-tshark/, but for convenience, I'll summarize the 2 possible solutions I provided here. In a nutshell:
The highlighted byte in the image appears to be the TTL field of the IP header. If that's the field you're interested in, you can obtain it via:
tshark -r test.pcapng -Y "frame.number == 13" -T fields -e ip.ttl -w output.bin
If you're looking for a more general solution to print the 22nd byte of the packet, regardless of whether it's the ip.ttl field or not, then you can use a solution such as:
tshark -r test.pcapng -Y "frame.number == 13" -x -w output.bin | grep "^0010" | cut -d ' ' -f 9
The 2nd solution above also illustrates how you can dump all the bytes; it's done using tshark's -x option.
Using the below, I'm able to get both the raw predictions and the final predictions as a file:
cat train.vw.txt | vw -c -k --passes 30 --ngram 5 -b 28 --l1 0.00000001 --l2 0.0000001 --loss_function=logistic -f model.vw --compressed --oaa 3
cat test.vw.txt | vw -t -i model.vw --link=logistic -r raw.txt -p predictions.txt
However, I'm unable to get the raw predictions when I run VW as a daemon:
vw -t -i model.vw --daemon --port 26542 --link=logistic
Do I have a pass in a specific argument or parameter to get the raw predictions? I prefer the raw predictions, not the final predictions. Thanks
On systems supporting /dev/stdout (and /dev/stderr), you may try this:
vw -t -i model.vw --daemon --port 26542 --link=logistic -r /dev/stdout
The daemon will write raw predictions into standard output which in this case end up in the same place as localhost port 26542.
The relative order of lines is guaranteed because the code dealing with different prints within each example (e.g non-raw vs raw) is always serial.
Since November 2015, the easiest way how to obtain probabilities is to use --oaa=N --loss_function=logistic --probabilities -p probs.txt. (Or if you need label-dependent features: --csoaa_ldf=mc --loss_function=logistic --probabilities -p probs.txt.)
--probabilities work with --daemon as well. There should be no more need for using --raw_predictions.
--raw_predictions is a kind of hack (the semantic depends on the reductions used) and it is not supported in --daemon mode. (Something like --output_probabilities would be useful and not difficult to implement and it would work in daemon mode, but so far no one had time to implement it.)
As a workaround, you can run VW in a pipe, so it reads stdin and writes the probabilities to stdout:
cat test.data | vw -t -i model.vw --link=logistic -r /dev/stdout | script.sh
According to https://github.com/VowpalWabbit/vowpal_wabbit/issues/1118 you can try adding --scores option in command line:
vw --scores -t -i model.vw --daemon --port 26542
It helped me with my oaa model.
I run iperf command like this :
iperf -c 10.0.0.1 -t 2 -f m -w 1K | grep -Po '[0-9.]*(?= Mbits/sec)'
I want to display throughput only such as 0.32 but because I use 1K here, there is a warning and the display becomes
WARNING: TCP window size set to 1024 bytes. A small window size will give poor performance. See the Iperf documentation.
0.32
How to delete this warning so I can get "0.32" only?
Just send the warning message to /dev/null, after that you get output only.
So your command would be,
iperf -c 10.0.0.1 -t 2 -f m -w 1K 2> /dev/null | grep -Po '[0-9.]*(?= Mbits/sec)'
Given a pcap file, I'm able to extract a lot of information from the reconstructed HTTP request and responses using the neat filters provided by Wireshark. I've also been able to split the pcap file into each TCP stream.
Trouble I'm running into now is that of all the cool filters I'm able to use with tshark, I can't find one that will let me print out full request/response bodies. I'm calling something like this:
tshark -r dump.pcap -R "tcp.stream==123 and http.request" -T fields -e http.request.uri
Is there some filter name I can pass to -e to get the request/response body? The closest I've come is to use the -V flag, but it also prints out a bunch of information I don't necessary want and want to avoid having to kludge out with a "dumb" filter.
If you are willing to switch to another tool, tcptrace can do this with the -e option. It also has an HTTP analysis extension (xHTTP option) that generates the HTTP request/repsonse pairs for each TCP stream.
Here is a usage example:
tcptrace --csv -xHTTP -f'port=80' -lten capturefile.pcap
--csv to format output as comma sperated variable
-xHTTP for HTTP request/response written to 'http.times' this also switches on -e to dump the TCP stream payloads, so you really don't need -e as well
-f'port=80' to filter out non-web traffic
-l for long output form
-t to give me progress indication
-n to turn off hostname resolution (much faster without this)
If you captured a pcap file, you can do the following to show all requests+responses.
filename="capture_file.pcap"
for stream in `tshark -r "$filename" -2 -R "tcp and (http.request or http.response)" -T fields -e tcp.stream | sort -n | uniq`; do
echo "==========BEGIN REQUEST=========="
tshark -q -r "$filename" -z follow,tcp,ascii,$stream;
echo "==========END REQUEST=========="
done;
I just made diyism answer a bit easier to understand (you don't need sudo, and multiline script is imo simple to look at)
This probably wasn't an option when the question was asked but newer versions of tshark can "follow" conversations.
tshark -nr dump.pcap -qz follow,tcp,ascii,123
I know this is a super old question. I'm just adding this for anyone that ends up here looking for a current solution.
I use this line to show last 10 seconds request body and response body(https://gist.github.com/diyism/eaa7297cbf2caff7b851):
sudo tshark -a duration:10 -w /tmp/input.pcap;for stream in `sudo tshark -r /tmp/input.pcap -R "tcp and (http.request or http.response) and !(ip.addr==192.168.0.241)" -T fields -e tcp.stream | sort -n | uniq`; do sudo tshark -q -r /tmp/input.pcap -z follow,tcp,ascii,$stream; done;sudo rm /tmp/input.pcap