I have a fwcapture.cap file, which is used by Wireshark.
in it, there have many IP addresses source IPs and destination IPs.
How can I extract the unique IP addresses(no mater source or destination) as a list?
You can use tshark, which already in Wireshark installation.
tshark -T json -e 'ip.src' -e 'ip.dst' -r filename.pcap | grep '\.[0-9]' | sort -u
Related
When I want to count the number of bytes getting out a Linux system through a particular port I can use iptables, adding a specific rule that can be checked whenever I need.
For instance lets imagine that I need to know how many bytes go out through port 22. I can add the following rule:
iptables -A OUTPUT -p tcp --sport 22
And when I need to know the answer to my question, I run:
iptables -L -nvx
Or if I need just the bytes:
iptables -L -nvx | grep :22 | awk '{ print $2 }'
My problem is that if that system is running Docker, the iptables are changed as explained here and I can't get the desired effect (if I repeat the process above I always get 0 bytes even knowing that was traffic on that port).
Can someone please explain how can I obtain the same bytes count in this case?
I used a regex to grep and output only IPv4 addresses from the file content.
But when I try to use the same regex to exclude all IPv4 addresses, it just does not work.
File content:
# cat IPs
172.16.1.125
172.16.1.4
172.16.1.143
172.16.1.140
172.16.1.77
/dev/nvme101
/dev/sda1
/dev/sdb2
172.16.1.60
172.16.1.146
172.16.1.5
172.16.1.51
172.16.1.99
172.16.1.10
172.16.1.189
To grep only IPv4 addresses:
# grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" IPs
172.16.1.125
172.16.1.4
172.16.1.143
172.16.1.140
172.16.1.77
172.16.1.60
172.16.1.146
172.16.1.5
172.16.1.51
172.16.1.99
172.16.1.10
172.16.1.189
When I try to exclude the IPv4 addresses using the same regex:
# grep -voE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" IPs
#
No output at all.
I was expecting the following output:
/dev/nvme101
/dev/sda1
/dev/sdb2
Get rid of the -o. The -o flag says to only show what was matched rather than the entire line. That doesn't make sense when using -v for lines that do NOT match.
In ack, if you try to use -o and -v together, it throws an error.
After googling for hours and trying not to get lost in the different tshark versions I still can't figure out what command line options to tshark I should use to get the full (reassembled) JSON requests and responses (the JSON data structrues).
tshark 2.2.2 used on a live eth0 interface, not to parse pcap.files.
The requests and responses are gziped and need to be decoded.
All the related wireshark issues that seemed related are marked as "fixed" so I think in the 2.2.2 it should be possible.
I found a working solution. It doesn't work on a live interface and requires to first save a pcap file but it is the best I managed to do with tshark.
Step1 (capture network trafic):
tshark -i eth0 -f "port 9088" -w capture.pcap
Step2 (list captured tcp streams):
tshark -r capture.pcap -T fields -e tcp.stream | sort -u
Step3 (dump the content of one particular tcp stream):
tshark -nr capture.pcap -q -d tcp.port==9088,http -z follow,http,ascii,_your_stream_number
Noice the "-d tcp.port==9088,http" option to force http decoding on this port as in my case it is a socks5 proxy running on that port.
Most importantly "-z follow,http,ascii,_your_stream_number" where the "follow,http" feature decodes gziped http body content and is undocumented and only available from version 2.2.0 of wireshark/tshark.
I tried recursively parsing email addresses from a directory of text/html files with xargs and grep but this command keep including the path (I just want the email addresses in my resulting emails.csv file).
find . -type f | xargs grep -E -o "\b[A-Za-z0-9._%+-]+#[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" >> ~/emails.csv
Can you explain what's wrong with my grep command? I don't need this to be sorted or unique. I want to match all occurrences of email addresses in files. I need to use xargs cause I'm parsing emails in 20 GB worth of text files.
Thanks.
When you tell grep to search in more than one file, it prepends the corresponding filename to the search result. Try the following to see the effect...
First, search in a single file:
grep local /etc/hosts
# localhost is used to configure the loopback interface
127.0.0.1 localhost
Now search in two files:
grep local /etc/hosts /dev/null
/etc/hosts:# localhost is used to configure the loopback interface
/etc/hosts:127.0.0.1 localhost
To suppress the filename in which the match was found, add the -h switch to grep like this
grep -h <something> <somewhere>
Currently I am transporting files back and forth over telnet and I would like to send multiple files at once.
However, my target platform (a Blackfin processor) does not have "tar" enabled in its kernel/busybox configuration (a uClinux distribution).
As you all know the normal command is:
nc -p 12345 -l | tar -x
tar -c * | nc 192.168.0.100 12345 # with x.100 the robot IP address
How can I send multiple files using netcat without using tar?
Please, consider that I cannot easily add binaries on the platform. It would be best to do it with basic utilities and/or shell scripts.
Finally managed myself to do this, it can be done!
Here $l> stands for your machine with IP 192.168.0.10. And $e> is done on the embedded device without tar, in my case a robot. It uses old-fashioned dd which is able to copy an entire disk.
$l> nc -p 12345 -l | dd obs=4K of=/tmp/file.jffs2
$e> dd ibs=4K if=/dev/mtdblock2 | nc 192.168.0.10 12345
This is it, but because not everybody knows how to read a filesystem that is in this form, this is how you mount it:
file /tmp/file.jffs2
/tmp/file.jffs2: Linux jffs2 filesystem data little endian
sudo su #careful
mknod /tmp/mtdblock0 b 31 0
modprobe loop
losetup /dev/loop0 /tmp/file.jffs2
modprobe mtdblock
modprobe block2mtd
echo "/dev/loop0,128KiB" > /sys/module/block2mtd/parameters/block2mtd
modprobe jffs2
mkdir /media/robot
mount -t jffs2 /tmp/mtdblock0 /media/robot
Ctrl-D #back as normal user
And yes, you need the loopback device, or else:
sudo mount -t jffs2 /tmp/file.jffs2 /media/robot
mount: /tmp/file.jffs2 is not a block device (maybe try `-o loop'?)
Logically, it is a file (chars), and not a block device. The only thing I do not know is if there is a syntax for dd in which the command on the embedded device, can only select a subset of the filesystem to be included. I don't think this is likely because that would require dd to understand jffs2 while its strength is its raw byte copying behaviour.