I am trying to automate the exporting of full dissections of a pcap to a .txt file using tshark. I am aware of the file->export packet dissections as option, but I am working to automate that. Right now I have tshark -X lua_script: -r > . The files enclosed in <> are paths. The lone > is the command for printing text. It will export packet summaries but not the full dissection. IS there any way to export the full dissection to the command line. These sample line are what is exported right now,
1 0.000000000 02:00:00:00:00:67 -> IPv4mcast_01:05:ee 0x8903 1467 Data Center Ethernet (DCE) protocol(Cisco)
2 0.000001180 10.81.130.23 -> 239.1.5.238 ST 1451 Messages: 14
3 0.006327070 02:00:00:00:00:67 -> IPv4mcast_01:05:ee 0x8903 1467 Data Center Ethernet (DCE) protocol(Cisco)
4 0.006328250 10.81.130.23 -> 239.1.5.238 ST 1451 Messages: 14
5 0.019039770 02:00:00:00:00:67 -> IPv4mcast_01:05:ee 0x8903 1467 Data Center Ethernet (DCE) protocol(Cisco)
This is what I want the exports to look like
No. Time Source Destination Protocol Length Info
2 0.000001180 10.81.130.23 239.1.5.238 ST 1451 Messages: 14
Frame 2: 1451 bytes on wire (11608 bits), 1451 bytes captured (11608 bits)
Ethernet II, Src: Solarfla_0e:e4:a1 (00:0f:53:0e:e4:a1), Dst: IPv4mcast_01:05:ee (01:00:5e:01:05:ee)
Internet Protocol Version 4, Src: 10.81.130.23 (10.81.130.23), Dst: 239.1.5.238 (239.1.5.238)
User Datagram Protocol, Src Port: 43464 (43464), Dst Port: 25238 (25238)
ST Block
Block Header
Sanity: 23559 (Should be 23559)
Header Version (Major: 0 Minor: 1)
Header Size in Bytes: 19
Payload Size in Bytes: 1386
Messages: 14
Environment Id: 0
Feed Id: 1 (Uqdf)
Compression Type: 0
Sender Id: 1
Sequence: 37495844
Message Header
Header Version (Major: 0 Minor: 1)
Header Length in Bytes: 31
Msg Type: 1 (Equity Quote)
Message Version (Major: 0 Minor: 1)
Msg Length in Bytes: 68
Flags: 0
Data Type: 1 (Equity)
Feed Id: 1 (Uqdf)
Feed Line: 1
Feed Seq Num: 7123431
Feed Sub Seq Num: 0
Exchange Time (10:59:59.978517000)
High: 9220
Low: 380047880
Note: this is what the packet dissections look like when using file->export packet dissections
Thank you in advance!
After looking at the problem I have figured it out. It is tshark -X lua_script:filename -r p.pcap -V -T text > file.txt.
The key was the -V, as that prints packet details.
Related
How to view the content of a file in raw format in hex editor? and how to find the header offset and tailer offset of a document in raw format in hex editor?
1. How to view the content of a file in raw format in hex editor?
On Linux / Mac you can use xxd, which also has a lot of formatting options of the output, but a simple example:
xxd file.pdf | less
00000000: 2550 4446 2d31 2e37 0d25 e2e3 cfd3 0d0a %PDF-1.7.%......
00000010: 3131 3837 3420 3020 6f62 6a0d 3c3c 2f4c 11874 0 obj.<</L
00000020: 696e 6561 7269 7a65 6420 312f 4c20 3330 inearized 1/L 30
00000030: 3934 3237 392f 4f20 3131 3837 372f 4520 94279/O 11877/E
00000040: 3133 3334 3538 2f4e 2037 362f 5420 3238 133458/N 76/T 28
00000050: 3536 3638 312f 4820 5b20 3136 3733 2034 56681/H [ 1673 4
00000060: 3331 315d 3e3e 0d65 6e64 6f62 6a0d 2020 311]>>.endobj.
...
...
002f36c0: 4134 3534 3437 3444 4434 3337 3e3c 3036 A454474DD437><06
002f36d0: 3839 3542 4133 4234 4341 3434 3044 4232 895BA3B4CA440DB2
002f36e0: 3435 3937 3645 3545 3331 3231 3738 3e5d 45976E5E312178>]
002f36f0: 3e3e 0d73 7461 7274 7872 6566 0d31 3136 >>.startxref.116
002f3700: 0d25 2545 4f46 0d .%%EOF.
You can also open any file using popular hex editor HxD on Windows ( screenshot from https://mh-nexus.de/en/graphics/HxDShotLarge.png )
2. how to find the header offset and tailer offset
Let's take a look at file signatures and magic bytes. As you can see, the lenght of them can differ:
1F 9D .. 0 z tar.z compressed file (often tar zip) using Lempel-Ziv-Welch algorithm
25 50 44 46 2d %PDF- 0 pdf PDF document[16]
ed ab ee db í«îÛ 0 rpm RedHat Package Manager (RPM) package [3]
If you don't want to manually inspect based on the previous list, but rather programatically identify file signatures, there are some libraries for different languages, such as pyfsig, and they maintain a list of current file signatures under current list that they can deal with.
I am trying to extract the text of page 5 in pdf.
The pdf have a font YLJAAA+CMSY10 which has no mappings (CMap) or even encodings (default encoding or /Differences). While extracting text, after string "tetex package" CGPDFScanner returns "\x15" character which is encountered many times. When this character is encountered current font is the above mentioned font which has nothing to extract the text from pdf string.
What is this \x15 character?
Thanks.
I found 2 (not "many") occurrences of this:
[ (\025) ] TJ
which is a number in octal – this is the number that is \x15 in hexadecimal.
The font definition for "YLJAA+CMSY10" in the PDF carries no special encoding, so it has the default encoding for "CMSY" ("Computer Modern Symbol"):
114 0 obj
<<
/Type /Font
/Subtype /Type1
/BaseFont 210 0 R % -> "/YLJAAA+CMSY10"
/FirstChar 0
/FontDescriptor 211 0 R
/LastChar 127
/Widths 204 0 R
>>
211 0 obj
<<
/Ascent 750
/CapHeight 683
/CharSet (/bullet/greaterequal/arrowright/arrowdblright/element/negationslash/backslash/radical)
/Descent 0
/Flags 4
/FontBBox [ -29 -960 1116 775 ]
/FontFile 205 0 R
/FontName 210 0 R % -> '/YLJAAA+CMSY10'
/ItalicAngle -14
/StemV 85
/XHeight 430
>>
endobj
In itself, this still says nothing definitive: a PDF producer may reorder glyphs and encodings at will, as long as it does the same with the embedded font). Assuming the font set is not reordered, checking a random list of CMxx encodings shows that the character code 0x1F could well be GREATER-THAN OR EQUAL TO (Unicode U+2265).
Acrobat agrees; inspecting the font in the PDF shows that character code 21 (decimal) is named 'GREATER-THAN OR EQUAL' and looks like it as well.
Im trying to train a haar cascade. For that as a test run, I'm taking 5 positive images (which have the image). I use a program called objectmarker.exe to mark the object in the image and store the coordinates as well as the height and width of the rectangle in a text file (positives.txt)
Now when I try to create a .vec file using the the text file from command line, the program executes, but i get the following:
positive(1).txt : parse errorDone. Created 0 samples
The .vec file does get generated but if i try to view it, it opens a window and crashes.
I use the following code
C:\Sahil\Major Project\Haartraining Stuff\Haartraining Stuff\STEPS\step 02>openc
v_createsamples.exe -info positives.txt -num5 -vec vec5.vec -w 20 -h 20
Info file name: positives.txt
Img file name: (NULL)
Vec file name: vec5.vec
BG file name: (NULL)
Num: 1000
BG color: 0
BG threshold: 80
Invert: FALSE
Max intensity deviation: 40
Max x angle: 1.1
Max y angle: 1.1
Max z angle: 0.5
Show samples: FALSE
Width: 20
Height: 20
Create training samples from images collection...
positives.txt(1) : parse errorDone. Created 0 samples
my postives.txt is in the following format
C:/Sahil/Major Project/Haartraining Stuff/Haartraining Stuff/STEPS/step 02/rawdata/00007 001 (3).bmp_0000_0065_0107_0107_0199.bmp 1 1 2 106 193
C:/Sahil/Major Project/Haartraining Stuff/Haartraining Stuff/STEPS/step 02/rawdata/00007 001 (4).bmp_0000_0065_0107_0107_0199.bmp 1 1 2 108 195
C:/Sahil/Major Project/Haartraining Stuff/Haartraining Stuff/STEPS/step 02/rawdata/00007 001.bmp_0000_0065_0107_0107_0199.bmp 1 2 5 110 195
C:/Sahil/Major Project/Haartraining Stuff/Haartraining Stuff/STEPS/step 02/rawdata/img1.bmp 1 4 4 103 190
C:/Sahil/Major Project/Haartraining Stuff/Haartraining Stuff/STEPS/step 02/rawdata/img2.bmp 1 3 5 118 217
kindly suggest what i can do to correct this error. as i cannot proceed further
How is opencv_createsamples.exe distinguishing image file names? It might be written not to check white characters in paths/file names. Try without spaces either in the paths and file names.
My memory is DDR2 800MHz
dmidecode -t memory
Handle 0x1100, DMI type 17, 27 bytes
Memory Device
Array Handle: 0x1000
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 1024 MB
Form Factor: DIMM
Set: None
Locator: DIMM_1
Bank Locator: Not Specified
Type: DDR2
Type Detail: Synchronous
Speed: 800 MHz (1.2 ns)
Manufacturer: 7F98000000000000
Serial Number: 2DCCDD00
Asset Tag: 050916
Part Number:
I want to estimate the write or read speed by this info.
Is 800MHz means read/write 800bits per second?
Should we multiple 64 bit datawidth? eg: 800*64 bits/sec. So we can read/write 800*64/8 bytes/sec?
Thanks advance!
Excluding options field in IPv4 header, after 20 bytes of header, data follows. That data may be TCP packet, or UDP etc.
Now given a IPv4 packet (with header and data), How to find out which type of transport layer packet (TCP/UDP/etc.) is present in data? Actually I am parsing a IPv4 packet so I need to understand this.
The protocol field of the IPv4 header (see RFC791) will tell you:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Options | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The protocol numbers are assigned by IANA and are listed here:
http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
Some example protocol numbers are:
1 ICMP
6 TCP
17 UDP
Deep packet inspection? Ipoque release some open source code for this task: opendpi.