In my logs i am able to see database password.I want to remove the fields when ever it matches to one particular method.Below is the sample log format.
Date=2016-02-23:00:36:29:242, Level=INFO , class=abc_class, method=abc_method, line=266, name=dataSourceUser, message=returning Property, value=password
below is the sample logstashclient filter am using but still am able to see value field(password)
filter {
grok {
match => {"message"=>"Date=(?<LogTime>%{YEAR:Year}-%{MONTHNUM:Month}-%{MONTHDAY:Day}:%{TIME:Time}), (Level=)(?<SeverityLevel>[A-Z]*)(\s)?, %{GREEDYDATA:message}"}
break_on_match => false
}
date {
match => ["LogTime", "YYYY-MM-dd:HH:mm:ss:SSS"]
target => "#timestamp"
}
if[method=="abc_method"]{
mutate {
remove_field => ["%{Value}"]
}
}
ETA : when ever it matches to method=abc_method it should not pass value field,for other metods it should pass
Try:
remove_field => [ "value" ]
found the answer we can use gsub filter
eg :
gsub => [ "message", "value=%{Value}", "" ]
Related
I am trying to go over a list of items, cache the URL and then update the URL in the list before drawing it.
However I don't seems to be able to do so.
Any idea?
external cache: option(string) => Js.Promise.t({. "uri": string, "filePath": string }) = "get";
let items = result##data##data->Option.getWithDefault([||]);
let cachedUri = items
->Belt.Array.map(
item =>
cache(item##hosted_video_url)
->FutureJs.fromPromise(Js.String.make)
->Future.flatMapOk(cachedObj => (
{. "hosted_video_url": cachedObj##uri }
)
))
->Belt.Array.toList
->Future.all;
The idea is that you cannot exit a future, so you need to update the state inside the future thing rather than trying to get an equivalent of an asyncio.gather or something similar.
I changed:
setVerticalData(_ => {items-> Js.Array2.sortInPlaceWith((a, b) => {...})});
with
items
->List.fromArray;
->List.map(item =>
cache(item##hosted_video_url)
->FutureJs.fromPromise(Js.String.make)
)
->Future.all
->Future.map(cachedList => {
setVerticalData(_ => {items: cachedList})
})
I'm fetching some data from an camera's API.
There's a big string returned in the response with this format VCA.Ch0.licenseid=0
Here's a part of the response
VCA.Ch0.licenseid=0\r\nVCA.Ch0.enable=yes\r\nVCA.Ch0.enablemovobj=yes\r\nVCA.Ch0.enablecntline=yes\r\nVCA.Ch0.trackmode=surv\r\n
Is it possible to parse this string?
Let's say that I want to query about the param VCA.Ch0.enable.
How can I do that?
▶ input.split.map { |kv| kv.split('=') }.to_h
#⇒ {
# "VCA.Ch0.enable" => "yes",
# "VCA.Ch0.enablecntline" => "yes",
# "VCA.Ch0.enablemovobj" => "yes",
# "VCA.Ch0.licenseid" => "0",
# "VCA.Ch0.trackmode" => "surv"
# }
You can do this...
string = 'VCA.Ch0.licenseid=0\r\nVCA.Ch0.enable=yes\r\nVCA.Ch0.enablemovobj=yes\r\nVCA.Ch0.enablecntline=yes\r\nVCA.Ch0.trackmode=surv\r\n'
parsed_result = string.split("\r\n").reduce({}){|result, unit| key, data = unit.split('='); result[key] = data; result }
this lets you do...
parsed_result['VCA.CH0.enable']
=> 'yes'
Following the code in http://www.w3resource.com/mongodb/mongodb-elemmatch-projection-operators.php I have set up a test database using the ruby mongodb driver.
For those following along at home, you first need to install the mongo driver as described at https://docs.mongodb.com/ecosystem/tutorial/ruby-driver-tutorial/#creating-a-client, then run the following commands.
client = Mongo::Client.new([ '127.0.0.1:27017'], :database => 'mydb')
test = client['test']
doc = {
"_id" => 1,
"batch" =>10452,
"tran_details" =>[
{
"qty" =>200,
"prate" =>50,
"mrp" =>70
},
{
"qty" =>250,
"prate" =>50,
"mrp" =>60
},
{
"qty" =>190,
"prate" =>55,
"mrp" =>75
}
]
}
test.insert_one(doc)
Insert all of the different docs created in the w3 tutorial.
If you look at example 2 in the w3 tutorial, the translated ruby find is:
test.find({"batch" => 10452}).projection({"tran_details" => {"$elemMatch" => {"prate" => 50, "mrp" => {"$gte" => 70}}}}).to_a
which returns the same result as in the example.
=> [{"_id"=>1, "tran_details"=>[{"qty"=>200, "prate"=>50, "mrp"=>70}]}, {"_id"=>3}, {"_id"=>4}]
My problem is that I would like to constrain the results with the constraints above (mrp gte 70 etc) while also specifying which fields are returned.
For instance, constraining only the tran_details that have a mrp gte 70, but in the results returned only include the prate field (or any subset of the fields).
I can return only the prate field with the query:
test.find({"batch" => 10452}).projection({"tran_details.prate" => 1}).to_a
I would like to combine the effects of the two different projections, but I haven't seen any documentation about how to do that online. If you string the two projections to each other, only the final projection has an effect.
To anyone out there --
The problem can be solved up to one element by using $elemMatch on projection. However, $elemMatch only returns the first result found. To return only parts of embedded documents multiple layers down that fit certain criteria, you need to use the aggregation framework.
test.find({
'tran_details.prate' => { '$gt' => 56 }
}).projection({
tran_details: {
'$elemMatch' => {
prate: { '$gt' => 56 }
}
},
'tran_details.qty' => 1,
'tran_details.mrp' => 1,
batch: 1,
_id: 0
}).to_a
To return only parts of embedded documents multiple layers down that fit certain criteria, you need to use the aggregation framework.
Here is example code
test.aggregate([{
'$match': {
'$or': [
{'batch': 10452}, {'batch': 73292}]}},
{'$project':
{trans_details:
{'$filter': {
input: '$tran_details',
as: 'item',
cond: {'$and':[
{'$gte' => ['$$item.prate', 51]},
{'gt' => ['$$item.prate', 53]}
]}
}
}
}
}]).to_a
If anyone sees this and knows how to dynamically construct queries in ruby from strings please let me know! Something to do with bson but still trying to find relevant documentation. Thanks -
Here's an example hash and an example array to search in the hash:
nicknames = { "Black Mamba" => "Kobe Bryant",
"Half Man Half Amazing" => "Vince Carter",
"The Big Fundamental" => "Tim Duncan",
"Big Ticket" => "Kevin Garnett",
"Obi-Wan Ginobili" => "Manu Ginobili",
"The Answer" => "Allen Iverson" }
names = [ "Vince Carter", "Manu Ginobili", "Allen Iverson" ]
I want to return:
selected = { "Half Man Half Amazing" => "Vince Carter", "Obi-Wan Ginobili" => "Manu Ginobili", "The Answer" = "Allen Iverson" }
What's a good way to do this? Thanks!
You can simply do the following:
nicknames.select { |key, value| names.include?(value) }
(copy-paste the code you provided and mine in your IRB console and you'll see it working).
If the values in the hash are unique, then you can reverse the keys and the values. MrYoshiji's code works, but here is a more efficient way.
hash = nicknames.invert
names.to_enum.with_object({}){|k, h| h[hash[k]] = k}
When I access my nested hash something weird happens.
Below is my nested hash.
{
"http://example.com"=>{
"a-big_word"=>{
"Another word"=>[]
}
},
"www.example.com"=>{
"a-big_word"=>{
"Another word"=>[]
}
}
}
If I try and add something to it with the following
hash['www.example.com']['a-big_word']['Another word'] << {"key"=>"value"}
This happens
{
"http://example.com"=>{
"a-big_word"=>{
"Another word"=>[{"key"=>"value"}]
}
},
"www.example.com"=>{
"a-big_word"=>{
"Another word"=>[{"key"=>"value"}]
}
}
}
Use strings instead of symbols as keys. I took your hash and changed the keys to be strings. Now it looks like this:
{"http://example.com"=>
{"sublink"=>
{"A word"=>[], :"Another word"=>[]},
"sublinktwo"=>
{"Hello"=>[], "World"=>[]}},
"www.example.com"=>
{"sublink"=>
{"hi"=>[], "goodbye"=>[]},
"sublinkthree"=>
{"word"=>[], "bye"=>[]}
}
}
If you haven't see the difference, for keys I'm using => instead of :. In that way Ruby will not convert the keys into symbols, it will leave it as they are.
How to access the values? Check out the following irb session.
> hash["www.example.com"]
=> {"sublink"=>{"hi"=>[], "goodbye"=>[]}, "sublinkthree"=>{"word"=>[], "bye"=>[]}}
> hash["www.example.com"]["sublink"]
=> {"hi"=>[], "goodbye"=>[]}
> hash["www.example.com"]["sublink"]["hi"]
=> []
Change value:
> hash["www.example.com"]["sublink"]["hi"] << {"key"=>"value"}
=> [{"key"=>"value"}]