I'm trying to monitor my network usage with Graphite but I can't figure out how to do this, could you help me please?
In addition to this, I would like to monitor some other services, as nginx, mysql, etc,..
Thanks for your help!
Best.
Ofcourse there are multiple solutions possible. The solution I'm using is collectd. With collectd you can collect statistic data from plugins in rrd files. It has a lot of plugins like network, nginx and mysql.
It does not generate graphs by itself, but there are multiple ways to generate graphs. One of them is to send the collected data to graphite with the graphite plugin.
Related
We need to monitor the Neo4j hosted in GCP VM's instance for which we are using Prometheus. Neo4j Natively supports sending metrics to Prometheus.
Now we need to create a dashboard using the stack driver monitoring with the exposed prometheus metrics.
Any suggestions/help will be useful.
Thanks in advance !!
Well, I tried to find something elaborated about this particular scenario but all I found was related with GKE and prometheus as a sidecar. However, the configuration must be similar and probably you will find this documentation useful. This guide will also lead you to this official documentation.
Also found another user of neo4j trying to do something similar and there is an answer that maybe you should check. For logging neo4j to Stackdriver Logging, see this.
I have a Dropwizard application I am working with where I need to be able to monitor for active HTTP connections. I know Metrics provides classes for Instrumenting Jetty, and of interest to me is measuring total number of active connections....however the javadoc doesn't help me much, and I can't find any examples of how this specific functionality is implemented. Does anyone have any examples they can share?
I'm not sure what your exact use-case is but if you just need to be able to see active connections I think the simplest solutions are just using monitoring solutions like JMX, Datadog, New Relic, or Appdynamics. If you need it in the code I think you'd need to manually implement something. I'd recommend StatsD or Redis is that's the path you go down.
I am looking for monitoring tool for the following use cases:
Collect basic metrics about virtual machine (cpu usage, memory usage, i/o, available space)
Extract metrics from SQL Server (probably running some queries)
Extract information from external service about processing i.e how many processing are currently running and for how long. I am thinking about writing python scripts, but don't know how to combine with monitoring tool
Have the ability to plot charts and manage alerts and it will nice to have ability to send not only mails, but send message to slack/ms teams.
I was thing about Prometheus, because it has wmi_exporter, node_exporter, sql exporter, alert manager with possibility to send notifications to multiple destinations, but I don't know what to do with this external service and python scripts.
Any suggestions?
Prometheus can definitely do what you say you need done. Some of it may not be trivial, but you can definitely fill in the blanks yourself.
E.g. you can get machine metrics basically out of the box by firing up a node_exporter and having it scraped by Prometheus, but I don't think it has e.g. information on all running processes. The latter might require you to write an agent/exporter: a simple web server that exposes metrics on /metrics; there exists a Python client library to help with that. Or have said processes (assuming they're your code) push metrics to a Pushgateway instead, if they're short lived batch jobs.
Oh, and for charts/dashboards you probably want Grafana, as Prometheus' abilities in that area are rather limited and Grafana integrates rather well with Prometheus.
Sorry I just started to learn docker. My question may seem stupid for some of you.
In fact, I would like to know if there is a way to collect performance metrics from "CAdvisor" container (not from cgroup) at runtime ? I mean, extract performance values from the curves designed by cadvisor like memory usage or network traffic.
I need to record this values and save them in a database so that, I can perform a statistic analyzes upon these generated values (like comparing memory consumption for two docker containers at t=50s).
Thanks in advance.
As other answers mention, cAdvisor doesn't provide its own performance data API, instead it exposes metrics which are typically handled in a separate database if one wants to derive performance data beyond "real time". For example, cAdvisor exports Prometheus metrics natively:
http://prometheus.io/docs/instrumenting/exporters/
The Prometheus metric types:
http://prometheus.io/docs/concepts/metric_types/
Prometheus supports a fairly rich functional expression language that can be used for querying and visualization:
http://prometheus.io/docs/querying/basics/
cAdvisor does provide a rest endpoint to get any stats in real time. By default, it keeps latest two minute of data. You can configure it to keep more or less. It also supports a storage backend to keep dumping stats to an influxdb database.
REST Api:
eg. /api/v1.3/containers
doc: https://github.com/google/cadvisor/blob/master/docs/api.md
Doc on setting up InfluxDB:
https://github.com/google/cadvisor/blob/master/docs/influxdb.md
I think you could use https://github.com/tutumcloud/container-metrics for this. Basically what that would be doing is using influxdb http://influxdb.com/ as a time series data store.
There is some more information available here: http://blog.tutum.co/2014/08/25/panamax-docker-application-template-with-cadvisor-elasticsearch-grafana-and-influxdb/
A couple of people seemed to be looking into the ELK stack (Elastic Search, Logstash, Kibana) for visualising some of this data here: https://github.com/google/cadvisor/issues/634
I am interested in monitoring my multi-agent apache flume setup. I have enabled the inbuilt ganglia server which provides me the flume metrics through JSON data. Now I am interested in viewing these info in graphs/charts. TO achieve this I am using ganglia web ui, I have these questions - Do I have to install gmond and gmetad to achieve it, if not then how I will use the existing ganglia info with the ganglia web ui ?
Thanks in advance.
You'll need both, IMHO. Moreover, I think Flume can communicate directly to a gmond by appending some stuff in JAVA_OPTS, see hortonworks docs.
You'll need gmetad because it stores your data in RRD files, and the web UI query on it to display graphs.
Graphite can do the job too.