I have been trying to monitor a custom JMX metrics using stackdriver,
. I saw the below GIT page for fetching MBEAN metrics,
https://github.com/Stackdriver/stackdriver-agent-service-configs/blob/master/etc/collectd.d/jvm-sun-hotspot.conf
I require some additional custom metrics thats needs to be monitored from Stackdriver. Basically, i need
openejb.management:ObjectType=datasources,DataSource=jdbc_db
I tried modifying the conf file from Github by removing the OOb Mbeans and added the custom MBEAN, and i changed the target set to
<Target "set">
MetaData "stackdriver_metric_type" "custom.googleapis.com/jvm/jmx/datasource/jdbc_db"
I see a new metric is getting propagated to Stack driver, but other OOb metrics are affected, Can anyone provide a conf file which i can deplot in /stackdriver/collectd/etc/collectd.d/ location which will sends the custom MBean to Stachdriver
Related
Not able to enable the debug|trace level logging of the dataflow workers.
The documentation :https://cloud.google.com/dataflow/docs/guides/logging#SettingLevels
indicates the usage of DataflowWorkerLoggingOptions to programmatically overrides the default log level on the worker and enable the debug|trace level logging; however the interface is deprecated and no more present in bean-sdk 2.27.0 .
Has anyone been able to enable the worker level debugging in cloud dataflow; in any way.
The documentation is still up to date and the interface is still present and will work.
The interface is deprecated because the Java-based Dataflow worker is not used when running a pipeline using Beam's portability framework. Quoting the deprecation message:
#deprecated This interface will no longer be the source of truth for worker logging configuration once jobs are executed using a dedicated SDK harness instead of user code being co-located alongside Dataflow worker code. Please set the option below and also the corresponding option within org.apache.beam.sdk.options.SdkHarnessOptions to ensure forward compatibility.
So what you should do is follow the instructions that you linked and also set up logging in SdkHarnessOptions.
When running Apache Beam Google Dataflow jobs and using the SLF4J logger we don't get anything beyond the log message in Stack Driver.
Example of additional information would be function, line number etc.
Is there anyway to configure the logger like a log4j.xml or java logging properties file?
There is no way to customize logs messages in Dataflow other than what is shown in this logging pipelines messages
Have you looked at Cloud Logging? It has several features such as Custom logs / Ingestion API. In case you haven't, take a look at this guide to setup the SLF4J logging facade through Logback appender and Cloud Logging. Once you have configured Logback to use the Cloud Logging, you can use the SLF4J logging API. Another option is to use the Cloud Logging API with a default Java Logging API handler, which can be added programmatically or by using a configuration file, here is an example using logger.
Isaac Miliani, I tried the same option google cloud logging as provided in the google cloud docs,
Added logback.xml to src/main/resources (classpath).
Created loggingeventenhancer and enhancer class to add new labels.
Added markers to logger error, to find the type of error in Stackdriver.
But the logs in stackdriver doesnt have new labels added via logging appender. I think the logback.xml is not found by the maven compile command to deploy the job in dataflow.
Can you provide whats going on wrong here?
I can view the task manager's logs from web ui:
But suppose I submit a job, and use log4j2 to write some logs to a specific file, can I make some configurations so I can view the logs from web ui?
In the script :
$FLINK_HOME/bin/flink
You will find these 2 lines:
log=$FLINK_LOG_DIR/flink-$FLINK_IDENT_STRING-client-$HOSTNAME.log
log_setting=(-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_CONF_DIR"/log4j-cli.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml)
Either you configure log4j to store the logs in flink path, i.e: $FLINK_HOME/log, or modify $FLINK_HOME/bin/flink script by adding log4j logs path location.
Hope it helps :) !
I was playing a bit with Cygnus, and I was wondering how to properly configure it. I’ve seen both agent_<id>.conf and cygnus_instance_<id>.conf files are needed. I understand the purpose of the first one, but not the second one. In addition, what about the grouping_rules.conf file? Is there any other configuration file?
File by file:
agent_<id>.conf file is the main configuration file for Cygnus. It inherits the syntax from Apache Flume, the base technology used by Cygnus, thus it is used to declare and specify the sources, channels and sinks of your Cygnus agent.
cygnus_instance_<id>.conf file is used to configure other Cygnus parameters that cannot be configured as part of the agent configuration, such as the logging file, the management interface port, etc. Cygnus service will run as many instances as cygnus_instance_<id>.conf files are configured. That's why an <id> must be provided, since this <id> will be used to find the proper agent_<id>.conf file.
grouping_rules.conf file is used when the Grouping Rules advanced feature is wanted to be used. Usually, this file may be empty (but it must exist) and Cygnus will run after all.
flume-env.sh file has been inherited from Apache Flume, and it is used in order to configure certain Flume paramters such as the classpath overwritting the default one, some Java options (-Xms, -Xmx, etc)...
Configuring log4j2's flume appender https://logging.apache.org/log4j/2.0/manual/appenders.html#FlumeAppender
for HA seems like a difficult task if you want a generic file that can be used for different clusters. Each flume 'agent' requires its own line in configuration like
<Agent host="192.168.10.101" port="8800"/>
How can I specify agents using system property. It is possible to do so. Also, how can redistribute load across all agents.
LoadBalancingLog4jAppender - a Log4j compliant appender that flume provides seems to solve the problems I am facing but I am really looking for log4j2 solution