Only collect logs with specific field by fluent-bit - fluentd

For example, my logs look like:
{
"fieldA":"logA",
"foo":"bar"
}
or
{
"fieldA":"logA",
"fieldB":"logB"
}
In my case, I only want logs with field "foo" to be collected, while dropping other logs.

Use grep filter plugin to only include events with foo field:
[FILTER]
Name grep
Match *
Regex foo .*

Related

How can I use a regex with flags in a Jenkins pipeline when/changelog?

I have a Jenkins pipeline stage that I want to skip if there isn't a magic string in the commit message of the form Release: (major|minor|patch) (case and some whitespace insensitive) not on the first line. I'm trying to use the when directive and changelog condition, but it doesn't seem to like my regex, either because it's malformed or literally interpreting the flags .../im or escaped whitespace \s. I don't quite know how to debug it efficiently...
stage('Publish') {
when {
changelog '/.*^release:\\s*(major|minor|patch)\\s*$/im'
}
# ...
}
When it was a "bare" regex like changelog /.*^... it had a syntax error.
Is there a way to get the regex to work, or would another solution like
when {
expression {
???? ==~ /.*^release:\\s*(major|minor|patch)\\s*$/im
}
be better? Not sure what can be dropped into ????.
In Groovy/Java regex flags should come at the beginning and be wrapped in parentheses like this /(?im).*^release:\s*(major|minor|patch)\s*$/.
Also you can get commit change sets from currentBuild.changeSets. For git it returns a set of GitChangeSet.
The following should work
when {
expression {
currentBuild.changeSets.any { it.comment =~ /(?im).*^release:\s*(major|minor|patch)\s*$/ }
}
}

Trying to run a groovy script to populate a string's description, but the script returns no output, what could be the reason?

I'd like to configure a choice parameter in Jenkins.
The parameter I'd like to configure is called CIDR.
I tried using "Extended choice parameter" plugin but to no avail.
What I'm trying to do, is to let the user manually insert the chosen CIDR, considering the CIDRs which are already in use -> I want to run a groovy script to populate the string description with CIDRs which are already in use.
In order to list the already in use CIDRs, I wrote the following Groovy code:
#!/usr/local/bin/groovy
def p = ['/usr/local/bin/aws', 'ec2', 'describe-vpcs'].execute() | 'grep CidrBlock'.execute() | ['awk', '{print $2}'].execute() | ['tr', '-d', '"\\"\\|,"'].execute()
p.waitFor()
println p.text
The script runs properly in terminal:
itai#Itais-MacBook-Pro ~ - $ groovy cidrs.groovy
172.31.0.0/16
172.51.0.0/16
172.51.0.0/16
I even accepted a suspicious signature in Jenkins in-script approvals to allow the script to run.
But when I insert it to the Groovy script section of the string description and run the "build the job with parameters", the string dropdown stays empty.
What am I doing wrong?
Looks trivial issue. Try below.
Change From :
println p.text
To:
return p.text
The reason why the parameter kept being empty is that as it seems, the "Extended Choice Parameter" plugin expects the output to be an array.
Changing the script to the following code solved the issue:
#!/opt/groovy-2.4.12/bin/groovy
def p = ['/usr/bin/aws', 'ec2', 'describe-vpcs'].execute() | 'grep CidrBlock'.execute() | ['awk', '{print $2}'].execute() | ['tr', '-d', '"\\"\\|,"'].execute()
p.waitFor()
def output = []
p.text.eachLine { line ->
output << line
}
output.each {
println it
}
Now the parameter is populated with the available CIDRs.

How can I include the current workspace name in the default argument value of a rule?

Let's say I have a rule:
blah = rule(
attrs = {
"foo": attr.string(default = "#HELP#"),
},
)
I want the default value of foo to contain the name of the workspace that invokes the rule. How can I accomplish this?
(Note: An acceptable approach is to leave a placeholder in the value and replace it when the rule uses the attribute, but I can't figure out how to get the current workspace there either. The closest I can find is ctx.label.workspace_root, but that is empty for the "main" workspace, and e.g. external/foo for other things.)
ctx.workspace_name does not give the correct answers. For example, if I print("'%s' -> '%s'", (ctx.label.workspace_root, ctx.workspace_name)), I get results like:
'externals/foo' -> 'main'
'externals/bar' -> 'main'
...which is wrong; those should be 'foo' and 'bar', not 'main' ('main' being my main/root workspace). Note that labels from those contexts are e.g. '#foo//:foo', so Bazel does apparently know the correct workspace name.
You can use a placeholder attribute and then use ctx.workspace_name in the implementation.
def _impl(ctx):
print("ws: %s" % ctx.workspace_name)
blah = rule(
implementation = _impl,
)
As far as getting the workspace name, this seems sub-optimal, but also seems to work:
def _workspace(ctx):
"""Compute name of current workspace."""
# Check for meaningful workspace_root
workspace = ctx.label.workspace_root.split("/")[-1]
if len(workspace):
return workspace
# If workspace_root is empty, assume we are the root workspace
return ctx.workspace_name
Per Kristina's answer and comment in the original question, this can then be used to replace a placeholder in the parameter value.

Setting up ruby on rails logs

I was looking into logging actions for example when you create a new user it sends it to the logger etc.. so every action is logged. I can see how the logger.info sends information into the development.log file.
I was wondering how I Would set-up a different file e.g. users.log and then when I log a line or variable, it saves inside that log file instead of the development.log?
Ruby has a Logger class in its standard lib: http://ruby-doc.org/stdlib-2.1.0/libdoc/logger/rdoc/Logger.html
You would instantiate that and pass it the file path of your new log file, like this:
user_log = File.open('logs/users.log', File::WRONLY | File::APPEND)
You can place that in a controller method that your controllers can use. The first string argument is the path to the log file, and the following are opening the file for writing and appending only (so that each log line is added to the log rather than overwriting it each time).
You can customize the format of each log line by setting a formatter:
user_log.formatter = proc { |severity, datetime, progname, msg|
"#{severity}, #{datetime}, #{progname}, #{msg.dump}"
}
You can specify the file path used in the config file, which can vary according to the environment, as so:
config.paths.log = "/some/path/#{Rails.env}.log"
If you want to create different log files for each model, you can simply create a logger object when needed, as explained in this answer.
However if you just want to somehow mark different logs according to where they were generated, it may be easier to use tagged logging:
logger = ActiveSupport::TaggedLogging.new(Logger.new(STDOUT))
logger.tagged("BCX") { logger.info "Stuff" } # Logs "[BCX] Stuff"
logger.tagged("BCX", "Jason") { logger.info "Stuff" } # Logs "[BCX] [Jason] Stuff"
logger.tagged("BCX") { logger.tagged("Jason") { logger.info "Stuff" } } # Logs "[BCX] [Jason] Stuff"

Better parameter error handling; and better parameter parsing/handling?

I have a script like this:
param(
[Alias('a')]
[string]$aval,
[Alias('b')]
[switch]$bval,
[Alias('c')]
[string]$cval
)
if($aval.length -gt 1)
{
Do-Something
}
elseif($bval)
{
Do-Something-Else
}
elseif($cval.length -gt 1)
{
Do-Another-Thing
}
else
{
Do-This
}
If someone calls my script like so, an ugly error is displayed saying it is missing an argument for parameter 'aval/bval/cval':
PS C:\> .\MyScript.ps1 -a
C:\MyScript.ps1 : Missing an argument for parameter 'aval'. Specify a
parameter of type 'System.String' and try again.
At line:1 char:18
+ .\MyScript.ps1 -n <<<<
+ CategoryInfo : InvalidArgument: (:) [MyScript.ps1], ParameterBindingException
+ FullyQualifiedErrorId : MissingArgument,MyScript.ps1
Is there any way to make a cleaner, possibly one line, error appear instead? Also, is there a better way to handle parameters then a list of elseif statements (my actual script has ~10 parameters)?
The script sometimes passes an argument with a parameter as well:
EX:
PS C:\> .\MyScript.ps1 -b ServerName
Thanks for any help!
There are a few things that you can look at here. First, if the parameter will never have an associated value and you just want to know if the script was called with the parameter or not, then use a [switch] parameter instead of a string.
Here is a very simple example of using a switch parameter:
param(
[switch]$a
)
if($a){
'Switch was present'
}else{
'No switch present'
}
Save that as a script and run it with and without the -a parameter.
If you will sometimes have the parameter present with some value being passed in but other times without the value, then give the parameter a default value when you define it:
[Alias('a')]
[string]$aval = '',
Then in your logic if something was passed in, the length of the string will be gt 1.
As for the if-then structure that you have, there are a plethora of options for handling this sort of logic. with the little bit of information that you have shared, I suspect that using switch structure will be the best plan:
Get-Help about_Switch

Resources