How can i set alert on gcp dataflow pipeline based on log pattern matching - google-cloud-dataflow

Let see, I want to notify with alert on Dataflow pipeline using log based.But in log based alert , i cant see any option if i have to set alert on particular pattern matching like, if in log any message "Job failed due to ....", here i want to trigger an alert based on condition like "job failed *". . It should alert me if any message in log occur. Highly appreciated your help
I tried Dataflow-log entries option, but didnt get anything on pattern matching log.

jsonPayload.message=~"some pattern"
https://cloud.google.com/logging/docs/view/logging-query-language

Related

Grafana - Modify Alert Message

I am using Grafana version v8.3.3 and have set up Slack alerts but the alerts are formatted in a certain way and I don't see an option to change it.
There is a section Add details for your alert which is great but the alert contains all these fields and I can't find where to control which one to display in Slack message and which ones to hide. I just want to send a message with value and a brief description without annotations, labels, source, and silence.
**Firing**
Value: [ metric='disk_used_percent' labels={InstanceId=i-..., path=/} value=28.528058875324813 ]
Labels:
- alertname = CI Jenkins instance FREE space alert
- notification = slack
Annotations:
- summary = CI Jenkins instance is running out of space. Please investigate.
Source: http.....
Silence: http....
By providing custom message in contact point, I could able to avoid source, silence, query etc., being sent as part of alert notification message.

How to write email configuration in jenkins pipeline to send different set of information for build success and build failure?

I have pipeline which will trigger from SCM. I want to capture most of the information about what went wrong if build fails and needed information if build succeed. All the captured info i will be using it in mail body(As detail as possible). I want to know how to capture those info and Do i need to use try catch or is there any other way?
Could anyone help me with the solution please?
I did something similar in the scripted pipeline.
Certainly, you need to use try/catch/finally as the sending email step needs to run no matter build pass or fail; you need to catch the exception so that you can take useful information out of it and then finally run the send email step.
At a high level, it will look like this
try {
// put your stages/logic here
} catch (ex) {
// get exception details ex.message etc
throw ex
} finally {
// put the email step
}

Getting batch script task error highlighted in TFS Release Management

When deploying, the log lists blanks for Error and Warning highlight patterns:
Error message highlight pattern:
Warning message highlight pattern:
Where can I set values for these so I can highlight my messages?
There is no such values can highlight your messages for now in TFS.
However, you should be able to return warnings and errors from your script using logging commands. For example:
Write-Host "##vso[task.logissue type=warning;]Test warning"
Write-Host "##vso[task.logissue type=error;]Test error"
will return something like this:
Also take a look at this similar question: Is it possible to configure the 'message highlight pattern' for the TFS2015.2 build definition command line step?

OSSEC Slack Integration

I want all OSSEC notifications to be routed to a Slack room instead of email. 2.9.Beta5 has a ossec-slack.sh active response script. The relevant parts of my ossec.conf are:
<command>
<name>ossec-slack</name>
<executable>ossec-slack.sh</executable>
<expect>srcip</expect>
<timeout_allowed>no</timeout_allowed>
</command>
<active-response>
<command>ossec-slack</command>
<location>local</location>
<level>1</level>
</active-response>
This works for SSH logins (failed and successful), but as far as I can tell doesn't trigger anything else. What am I doing wrong/how are others doing this? Is this just beta software being beta software?
First make sure your ossec-slack.sh file has the correct information in the top:
# FILE: /var/ossec/active-response/bin/ossec-slack.sh
SLACKUSER="ossec"
CHANNEL="#slack_chanel" # include the hash "#"
SITE="https://hooks.slack.com/services/TOKEN"
SOURCE="ossec2slack"
Your "SLACKUSER" is the same as the "Customize Name" field that you set in your Slack WebHook Integrations page.
Now that your ossec-slack.sh file is set up you can test your Slack integration manually:
/var/ossec/active-response/bin/ossec-slack.sh
Running the script manually will post recent entries from your alerts log file:
/var/ossec/logs/alerts/alerts.log
When this script is triggered as an active-response, it will only post the information for the current alert, rather than posting from your log file.
When you have verified that you can post Slack messages manually, add the following XML blocks to your ossec.conf file:
<!-- FILE: /var/ossec/etc/ossec.conf -->
<ossec_config>
<command>
<name>ossec-slack</name>
<executable>ossec-slack.sh</executable>
<expect></expect> <!-- no expect args required -->
<timeout_allowed>no</timeout_allowed>
</command>
<active-response>
<command>ossec-slack</command>
<location>local</location>
<level>3</level>
</active-response>
</ossec_config>
The settings above will post to your Slack channel whenever a level 3 or above alert is triggered.
Note: no arguments are required within the <expect> tag. But the <expect> tag itself, is required. See OSSEC's active-response documentation for more information.
To test this integration, restart your ossec server:
/var/ossec/bin/ossec-control restart
You should see the "OSSEC Started" alert very quickly:
If you do not see the alert, check your logs for any misconfigurations:
tail /var/ossec/etc/logs/ossec.log
tail /var/ossec/logs/active-responses.log
Not a full answer, but adding on here. To ensure this works, make sure you don't have this set in /var/ossec/etc/ossec.conf. If it's there, just remove.
<active-response>
<disabled>yes</disabled>
</active-response>

Write to the system's standard error in Progress

I am writing a small program in Progress that needs to write an error message to the system's standard error. What ways, simple if at all possible, can I use to print to standard error?
I am using OpenEdge 11.3.
When on Windows (10.2B+) you can use .NET:
System.Console:Error:WriteLine ("This is an error message") .
together with
prowin32 2> stderr.out
Progress doesn't provide a way to write to stderr - the easiest way I can think of is to output-through an external program that takes stdin and echoes it to stderr.
You could look into LOG-MANAGER:WRITE-MESSAGE. It won't log to standard output or standard error, but to a client-specific log. This log should be monitored in any case (specifically if the client is an application server).
From the documentation:
For an interactive or batch client, the WRITE-MESSAGE( ) method writes the log entries to the log file specified by the LOGFILE-NAME attribute or the Client Logging (-clientlog) startup parameter. For WebSpeed agents and AppServer servers, the WRITE-MESSAGE() method writes the log entries to the server log file. For DataServers, the WRITE-MESSAGE() method writes the log entries to the log file specified by the DataServer Logging (-dslog) startup parameter.
LOG-MANAGER:WRITE-MESSAGE("Got here, x=" + STRING(x), "DEBUG1").
Will write this in the log:
[04/12/05#13:19:19.742-0500] P-003616 T-001984 1 4GL DEBUG1 Got here, x=5
There are quite a lot of options regarding the LOG-MANAGER system, what messages to display, where the file is placed, etc.
There is no easy way, but in Unixen you can always do something like this using OUTPUT THROUGH (untested):
output through "cat >&2" no-echo unbuffered.
Alternatively -- and this is tested -- if you just want error messages from a batch-mode program to go to standard out then
output through "tee" ...
...definitely works.

Resources