I'm using FUSE ESB and i wondering, is there any possibilities to connect some JMX monitor ?
I have connectec JMX monitor to normal tomcat, but i think that it is good idea, to have controll over serwer load, where i have FUSE ESB instance.
Do you have any experience with it?
I will be greatefull for any help
You may want to read this QA as well where its discussing monitoring of SMX / FuseESB
Administration and Monitoring of Apache-Camel routes in ServiceMix
But rule of thumb is that SMX / Fuse ESB is running on a JVM and offers JMX management capabilities, and any standard JMX compliant tooling can access these information.
For example with Camel we have an extensive number of JMX mbeans, you can gain details about your Camel applications, such as performance statistics, control lifecycle of Camel routes, consumers, etc. And see thread pool utilization, and so forth.
FuseSource offers documentation about Fuse ESB. For example there is some details about configuring JMX here: http://fusesource.com/docs/esb/4.4.1/esb_runtime/ESBRuntimeJMXConfig.html
yep, you can use JMX (jconsole, visualVM, etc)...its enabled by default (see the /bin/servicemix shell script and /etc/system.properties for config)
see these links for more details (though they are a bit dated)...
https://cwiki.apache.org/confluence/display/SMX4/Remote+JMX+connection
http://servicemix.apache.org/docs/4.4.x/users-guide/jmx.html
Related
Jenkins's openstack-plugin uses openstack4j for talking to an openstack cloud. I'm looking for a way that we can we can monitor the amount of http(s) API calls openstack4j does, from client side perspective.
Some possible things to know:
Jenkins can tell me that? (although I believe openstack4j does the http(s) call independently)
it's running inside a container, some https call monitoring tools that I could use on that level?
Regarding your questions:
I don't think Jenkins can do this monitoring for you, in the end, it's just a big, distributed, job scheduler and runner. If there's no plugin purposefully written for this, it can't. You'd have to write it yourself.
Regarding the monitoring, there's a bunch of questions to answer, actually:
Do you want just a Java based solution?
Surprisingly, I couldn't find anything Java based, the standard Java Management Extensions (JMX) apparently do not have direct support for investigating a process' open network connections.
If it doesn't have to be Java-specific, you could use tcpdump or tshark to analyze the traffic, as long as you know where the calls go, for example.
Another generic Linux based alternative is to launch the process through strace. You might need to make some adjustments for Java.
Is the connection HTTP or HTTPS (it matters a lot)?
For HTTPS one option would be to man-in-the-middle the HTTPS connection with some sort of proxy. Then you can just check the logs of the proxy for the connections
I'm using Spring Cloud Dataflow local server and deploying 60+ streams with a Kafka topic and custom sink. The memory/cpu usage cost is not currently scalable. I've set the Xmx to 64m for most streams.
Currently exploring my options.
Disable embedded Tomcat server. I tried this once and SCDF couldn't tell the deployment status of the stream.
Group multiple Kafka "source" topics to a single sink app. This is allowed by Kafka but unclear if SCDF will permit subscribing to multiple topics.
Switch to using Kubernetes deployer. Won't exactly reduce the memory/cpu usage but distribute it across multiple machines. Haven't pursued this option because Kubernetes isn't used in my org yet. Maybe this will force the issue.
Open to other ideas. Might also be able to tweak Kafka configs such as max.poll.records and reduce memory usage.
Thanks!
First, I'd like to clarify the differences between SCDF and Stream/Task apps in the data pipeline.
SCDF is a lightweight Spring Boot app that includes the DSL, REST-APIs, and the Dashboard. Simply put, it serves as the orchestrator to define and deploy stream and task/batch data pipelines made of stream and task applications respectively.
The actual business logic, its performance, and the underlying resource consumption are at the individual Stream/Task application level. SCDF doesn't interfere with the app's operation, nor it contributes to the resource load. Everything, in the end, is standalone Boot apps - standalone Java processes.
Now, to your exploratory steps.
Disable embedded Tomcat server. I tried this once and SCDF couldn't tell the deployment status of the stream.
SCDF is a REST server and it requires the application container (in this case Tomcat), you cannot disable it.
Group multiple Kafka "source" topics to a single sink app. This is allowed by Kafka but unclear if SCDF will permit subscribing to multiple topics.
Again, there is no relation between SCDF and the apps. SCDF orchestrates full-blown Stream/Task (aka: Boot apps) into coherent data pipeline. If you have to produce or consumer to/from multiple Kafka topics, it is done at application level. Checkout the multi-io sample for more details.
There's the facility to consume from multiple topics directly via named-destination, too. SCDF provides a DSL/UI capability to build fan-in and fan-out pipelines. Refer to docs for more details. This video could be useful, too.
Switch to using Kubernetes deployer.
SCDF's Local-server is generally recommended for development. Primarily because there's no resiliency baked into the Local-server implementation. For example, if the streaming apps crash for any reason, there's no mechanism to restart them automatically. This is exactly why we recommend either SCDF's Kubernetes or Cloud Foundry server implementations in production. The platform provides the resiliency and fault-tolerance by automatically restarting the apps under fault scenarios.
From resourcing standpoint, once again, it depends on each application. They are standalone microservice application doing a specific operation at runtime, and it is up to how much resources the business logic requires.
I need to monitor memory consumption, class loading, thread details specific to each web application deployed in a tomcat server. In my use case, there will be a number of WAR files deployed in a single tomcat instance.
I know i can enable JMX to monitor tomcat and can get few details about the applications deployed in it. But this will not give any information about memory consumption and other details specific to a each web application.
We can enable JMX for a jar file with the below configurations.
java -Dcom.sun.management.jmxremote.port=3333 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false \
YourJavaApp
Similar to this, is there any way to enable JMX for a web application. Again, what I'm looking is to get details like memory consumption, class-loading, thread details separately for each web application.
Thanks,
nks
JMX only provides high level metrics, you may want to use an APM tool to get deeper into your code and provide far better visibility. The three leading products for this are AppDynamics, Dynatrace, and New Relic. I work for AppDynamics if you have questions, I believe we have the best solution on the market today. We also monitor JMX and do much deeper code analysis, and transaction stitching.
I am trying to write a small script that will help me automate some of my IT tasks regarding to VLAN management.
I do not want to log-in to my switch via command-line - I want to send commands to it and get response (over the NET).
Are there any alternatives? I have started to search the web but so far I did not found anything.
I know SNMP is an option to gain info but I want to check other alternatives
thanks.
You can try Netconf Configuration Protocol, it is RPC-like management protocol which is supported by Cisco and many other vendors.
SNMP is the only widely and commonly used option here.
You can use WMI to manage Windows-based infrastructure.
There is also legacy SYSLOG protocol (RFC3164) which is UDP based.
For traffic monitoring and billing purposes there are NetFlow,
sFlow, jFlow, IPFIX and RADIUS protocols.
There are some other protocols but mostly proprietary.
So I'd suggest using SNMP which is nowadays a de-facto standard in network monitoring domain.
You might look at Expect as a scripting language solution. It is commonly used to do exactly what you are needing:
log into device (with result cases)
execute commands
save config
logout
As you build out a script library, tasks become simplified as you could do things like run scripts with parameters and have Expect do all the detail work.
See the wikipedia article for an overview.
I have also used SNMP for this kind of thing but the functionality is different because you are using an SNMP read-write privilege to upload new parts or complete configs, saving the running config to flash and/or saving the config off-device.
Try NETCONF+YANG protocol because it is currently the best option for network device configuration. More about SNMP alternatives:
https://bestmonitoringtools.com/top-snmp-alternatives-because-snmp-is-dying/
I have a Java app running on Tomcat, and I would like to monitor counters using Windows Performance Monitor. Is this possible using a JMX adapter for the Java MBeans or by some other means?
The following tutorial might be of use:
http://www.developer.com/java/ent/article.php/3087741/Hooking-to-PerfMon-from-Java.htm
It shows how a Java application defines a custom counter that can be monitored in Perfmon. It basically boils down to using an extension DLL to the performance monitor and communicating with that via a memory mapped file. You could then hook your JMX counters into a similar mechanism so that they can be monitored from Perfmon.
Since you tagged this with JMX and MBeans, I assume your counters are accessible from a Java MBean. If so you could use jconsole, provided with the Java SDK, to monitor the counters. Once you find your MBean in the MBeans tab, double click on the value and it will draw a nice line graph for easy monitoring.
See JConsole Guide for more info.
This seems to be exactly what you're looking for, but it's not free:
http://www.adventnet.com/products/snmpadaptor/faqs/general.html
If you want to do this programatically, you can create a JMXConnection to the machine, then accessing the MBeans from there. We did this ourselves for a function test that involved keeping track of the number of threads in the application, which there's a counter for that you can access through JMX. If you want nice graphs and things like that I suggest using JConsole, as mentioned above. There's also a program called JProfiler that works like JConsole on steroids, basically. There's a free trial so you can try it out.
This article outlines how to access Windows PerfMon stats from Java and expose them as MBean attributes through JMX.