Iguazio is supposed to be monitoring models and also resource metrics across the cluster like resource usage per pod / service, etc. When I open up the grafana service I only see a few pre-built dashboards for model monitoring and nuclio functions but nothing at the cluster level. Do I have to create my own dashboards for this?
There are actually different grafana dashboards being used in Iguazio. The grafana dashboard you are referring to is for the platform users and is found on the services page. This includes all your model monitoring dashboards as you mentioned.
There is another grafana service that is used by the Iguazio system which comes pre-installed. You can access it by navigating to the Clusters page, under the Application tab, Status Dashboard column there is an icon that is a link which takes you to that grafana service. See this screenshot:
There you will find about 15 dashboards related to kubernetes, NGINX, GPUs, Nuclio, and Iguazio services resources monitoring. This constitutes the "cluster-level" monitoring that you were referring to.
Related
I'm trying to monitor Keycloak on dynatrace, but I see only process metrics.
is there a way to see metrics about sessions, Connected Users...
an existing plugin for keycloak on dynatrace?
If you want to monitor Keycloak you will see it as a process only and not on a transaction level unfortunately.
Supported technologies can be viewed here: https://www.dynatrace.com/support/help/shortlink/supported-technologies
If there is option of collecting of extra Parameters you can always create a custom OneAgent extension. There is currently not a plugin available. You will be able to define the metrics that you would like to see in the UI: https://www.dynatrace.com/support/help/shortlink/oneagent-extensions-tutorial
Iam working on a project of big data, where Iam trying to get tweets from Twitter and analyse these tweets and make predictions out of it.
I have followed this tutorial : http://blog.cloudera.com/blog/2012/10/analyzing-twitter-data-with-hadoop-part-2-gathering-data-with-flume/
for getting the tweets. Now Iam planning to build a microservice which can replicate itself as I increase the number of topics on which I want tweets. Now whatever code I have written to gather the tweets with that I want to make a microservice that can take a keyword and create a instance of that code for that keyword and gather tweets, for each keyword an instance should be created.
It will also be helpful if you inform me what tools to use for such application.
Thank you.
I want to make a microservice that can take a keyword and create a instance of that code for that keyword and gather tweets, for each keyword an instance should be created.
You could use kubernetes as an underlying cluster/deployment infrastructure. It has an API that allows you to deploy new services programmatically. So what you would have to do is:
Set up a basic service container for your twitter-service that is available in a container repository.
Then you deploy a first service based on your container. The service configuration will contain the keyword that the service uses as well as information about the kubernetes cluster (how to access the cluster API and where to find the container in the repository).
Now your first service has all the information it needs to automatically create additional service descriptions for kubernetes (with other key words) and deploy those additional services by calling the kubernetes cluster API.
Since the additional services will be passed all the necessary information as well, they themselves can then start even more services and so on.
You probably need to put some effort into figuring out the cluster provisioning, but that can also be done automatically with auto-scaling (available for Google or AWS clouds for example).
A different approach would be to run a horizontally scaled cluster of your basic twitter services that use a self organization algorithm to involve all the keywords put into a database or event queue.
I am trying to connect to cassandra (which have firewall rule tags) from google dataflow batch, , moreover the dataflow service will create workers with 'dataflow' tagged.
is it possible to change the default tag ? or add more tags? .
I believe it is currently not possible to change the labels with which Dataflow service creates the VMs. However, it is possible to control which GCE network the Dataflow workers will belong to, by specifying the parameter --network: The Google Compute Engine network for launching Compute Engine instances to run your pipeline (see Specifying Pipeline Execution Parameters).
You may consider creating a network for your Dataflow workers and using that in the firewall rules.
There does not appear to be a way to change the tag. However, firewall rules can select instances by service account in lieu of network tag, and the service account can be specified through the --serviceAccount account option. See "Specifying a user-managed controller service account" within the Dataflow documentation.
I am trying to give some web user interface to the dropwizard metrics. Checked Graphite and Ganglia but I need something which can be easily installed on windows and linux,as well.
Could not configure metrics-watcher because the metrics in my application are dynamic. Also configured jclawson/metrics-ui but did not find the ui appealing. Please suggest me if there are any other sources which can be easily integrated.
Pipelined the metrics to Kibana. I was expecting a web interface where can I search the metrics contents, configuring meaningful widgets and dashboards for the same, Kibana fulfills my requirement.
In the Kubernetes/Docker ecosystem there is a convention of using /healthz as a health-check endpoint for applications.
Where does the name 'healthz' come from, and are there any particular semantics associated with that name?
It historically comes from Google’s internal practices. They're called "z-pages".
The reason it ends with z is to reduce collisions with actual application endpoints with the same name (like /status). See this talk for more: https://vimeo.com/173610242
Similar endpoints (at least inside Google) are /varz, /statusz, /rpcz. Services developed at Google automatically get these endpoints to export their health and metrics and there are tools that collect the exposed metrics/statuses from all the deployed services.
Open source tools like Prometheus implement this pattern (since original authors of Prometheus are also ex-Googlers) by coming to a well-known endpoint to collect metrics from your application. Similarly OpenCensus allows you to expose z-pages from your app (ideally on a different port) to diagnose problems.