I can view the task manager's logs from web ui:
But suppose I submit a job, and use log4j2 to write some logs to a specific file, can I make some configurations so I can view the logs from web ui?
In the script :
$FLINK_HOME/bin/flink
You will find these 2 lines:
log=$FLINK_LOG_DIR/flink-$FLINK_IDENT_STRING-client-$HOSTNAME.log
log_setting=(-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j-cli.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml)
Either you configure log4j to store the logs in flink path, i.e: $FLINK_HOME/log, or modify $FLINK_HOME/bin/flink script by adding log4j logs path location.
Hope it helps :) !
Related
When running Apache Beam Google Dataflow jobs and using the SLF4J logger we don't get anything beyond the log message in Stack Driver.
Example of additional information would be function, line number etc.
Is there anyway to configure the logger like a log4j.xml or java logging properties file?
There is no way to customize logs messages in Dataflow other than what is shown in this logging pipelines messages
Have you looked at Cloud Logging? It has several features such as Custom logs / Ingestion API. In case you haven't, take a look at this guide to setup the SLF4J logging facade through Logback appender and Cloud Logging. Once you have configured Logback to use the Cloud Logging, you can use the SLF4J logging API. Another option is to use the Cloud Logging API with a default Java Logging API handler, which can be added programmatically or by using a configuration file, here is an example using logger.
Isaac Miliani, I tried the same option google cloud logging as provided in the google cloud docs,
Added logback.xml to src/main/resources (classpath).
Created loggingeventenhancer and enhancer class to add new labels.
Added markers to logger error, to find the type of error in Stackdriver.
But the logs in stackdriver doesnt have new labels added via logging appender. I think the logback.xml is not found by the maven compile command to deploy the job in dataflow.
Can you provide whats going on wrong here?
I'm sending build logs to Logstash via the logstashSend method at the end of a Jenkins declarative pipeline. The logs are being written to Logstash and I can query them in Kibana. The "data" section of the message contains what looks like a pre-configured set of Jenkins job properties. I'd like to add some properties to this set but I can't find any documentation that talks about how those properties are set.
Is there any way to add to/modify the properties in the data section of the message?
No, it is not supported yet, but is a highly requested feature.
You can see that there is an open issue (JENKINS-50455) for implementing this new feature.
I've gone through the ReportPortal docs and it states which appender should be used to send logs to report portal.
I guess what I'm missing here is how to configure where is ReportPortal configured in the first place (i.e., what's the host name & credentials that should be used, etc)
Thanks
first of all you need to male test framework integration
http://reportportal.io/docs/Test-Framework-Integration
choose one relevant to you.
And main config for those integrations stays in reportportal.properties file
http://reportportal.io/docs/JVM-based-clients-configuration
I occasionally want to get notified when a particular jenkins job that is building finishes. Is there any way to do this?
Scripting it through the API would be fine. I already have the jenkins IRC bot that notifies me of many things, so if I could just dynamically modify the running job build, that would be enough to do what I want -- I'm just having a hard time finding how to accomplish that.
AFAIK, you cannot change a job's config while it's running.
Here is an idea: Use a post-build step to check for an external resource status (like a file containing an action by text) and running an action based on the content of the file.
The external file can be modified while the build is running, so when the post-build is executed, it will follow the logic defined based on the content of the file.
I hope this helps.
You can use email notifier, It will send you an email
https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
This is a broad question, so any answers are deeply appreciated. I need to continually log the size of several build files (in this case some CSS and JS files), preserve this log and ideally show it as a dashboard in Jenkins.
I know that I can setup a cron job and execute a bash script to grab the files and log their size, but I'm not sure where this file would live and how to display it. Ideally the result would be a dashboard plot or bar graph over time.
Thanks.
P.S. I'm open to other logging suggestions, but Jenkins seems like the appropriate system to do this in.
Update: this isn't perfect but it works. Google Spreadsheets has a simple API for posting data, so this can work as an endpoint for any script you want to write that logs your data.
It's not a Jenkins solution, but gets the job done.
In my search leading up to this, I did come across JMeter, and the Performance Plugin for Jenkins, which were contenders for a possible solution.