I am trying to create a stream of samples of resource utilization of selected tasks. For example a simple command line like this:
perf stat -I 400 -e task-clock,instructions,cycles
can give me a good measure of CPU utilization. I want to be able to get the same result for memory usage too (not considering the LLC stuff yet). Does "Linux Perf" able to do this? can I see how many pages are resident in memory for this task (number of pages if different kinds e.g. 4K, 1M, ...). Or how much of the pages are swapped?
Related
My PS 1.7 is using a lot of memory(6-7GB of a total of 7 GB memory) over an unspecified amount of time.
Is there a way to see what is causing this?
I can see that when indexing all products, either by cron-job or via BO, the memory hits the roof and after the cron-job is done, the memory usage don't decrease back to normal. It stays consistent at a very high numer of usage.
I use cPanel on a shared host, but quite fast and strong server...
If you want to see the memory usage of individual processes launched from the backoffice, simply activate debug mode from the Performance tab and have a check at the Symfony bar at the bottom, which, among other things, also shows the resources usage.
If, on the other hand, you want to check in more detail at a "lower" level, you will need a PHP profiler like Xdebug, blackfire or similar.
When running a simple Linux container on ACI there is a huge discrepancy between the 'graphed' CPU usage in the portal compared to running 'top' in the container itself.
I can see my process running in 'top' and the cpu usage stays at around 5% and the load on the machine is below 0.10 but the portal reports around 60% usage. It's a single processor container.
Under heavier loads I have seen CPU usage of 300-400 % which feels like an issue related to the number of processors but even this does not add up and as previously stated it's a single processor container
Any thoughts ??
the ACI CPU Usage metrics seems to be in millicores, not in %. So when you see 300-400 it would be in fact .3 to .4 CPU which for a single CPU would represent 30-40%.
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-monitor#available-metrics
Hoping this helps.
I create my docker (python flask).
How can I calculate what is the limit to put for memory and CPU?
Do we have some tools that run performance tests on docker with different limitation and then advise what is the best limitation numbers to put?
With an application already running inside of a container, you can use docker stats to see the current utilization of CPU and memory. While there it little harm in setting CPU limits too low (it will just slow down the app, but it will still run), be careful to keep memory limits above the worst case scenario. When apps attempt to exceed their memory limit, they will be killed and usually restarted by a restart policy/orchestration tool. If the limit is set too low, you may find your app in a restart loop.
This is more about the consumption of your specific Flask application, you can probably take use the resource module in Python to calculate them.
More information here and here.
On a Linux machine, I need to count the number of read and write accesses to memory (DRAM) performed by a process. The machine has a NUMA configuration and I am binding the process to access memory from a single remote NUMA node using numactl. The process is running on CPUs in node 0 and accessing memory in node 1.
Currently, I am using perf to count the number of LLC load miss and LLC store miss events to serve as an estimate for read and write accesses to memory. Because, I guessed LLC misses will need to be served by memory accesses. Is this approach correct i.e. is this event relevant ? And, are there any alternatives to obtain the read and write access information ?
Processor : Intel Xeon E5-4620
Kernel : Linux 3.9.0+
Depending on your hardware you should be able to acess performance counter located on the memory side, to exactly count memory accesses. On Intel processor, these events are called uncore events. I know that you can also count the same thing on AMD processors.
Counting LLC misses is not totally correct because some events such as the hardware prefetcher may lead a significant number of memory accesses.
Regarding your hardware, unfortunately you will have to use raw events (in the perf terminology). These events can't be generalized by perf because they are processor's specifics and as a consequence you will have to look into your processor's manual to find the raw encoding of the event to give to perf. For your Intel processor you should look at chapter 18.9.8 Intel® Xeon® Processor E5 Family Uncore Performance Monitoring Facility and CHAPTER 19 PERFORMANCE-MONITORING EVENTS of the Intel software developer manual document available here In these documents you'll need the exact ID of your processor that you can get using /proc/cpuinfo
I have windows 2003 terminal servers, multi-core. I'm looking for a way to monitor individual CPU core usage on these servers. It is possible for an end-user to have a run-away process (e.g. Internet Explorer or Outlook). The core for that process may spike to near 100% leaving the other cores 'normal'. Thus, the overall CPU usage on the server is just the total of all the cores or if 7 of the cores on a 8 core server are idle and the 8th is running at 100% then 1/8 = 12.5% usage.
What utility can I use to monitor multiple servers ? If the CPU usage for a core is "high" what would I use to determine the offending process and then how could I automatically kill that process if it was on the 'approved kill process' list?
A product from http://www.packettrap.com/ called PT360 would be perfect except they use SMNP to get data and SMNP appears to only give total CPU usage, it's not broken out by an individual core. Take a look at their Dashboard option with the CPU gauge 'gadget'. That's exactly what I need if only it worked at the core level.
Any ideas?
Individual CPU usage is available through the standard windows performance counters. You can monitor this in perfmon.
However, it won't give you the result you are looking for. Unless a thread/process has been explicitly bound to a single CPU then a run-away process will not spike one core to 100% while all the others idle. The run-away process will bounce around between all the processors. I don't know why windows schedules threads this way, presumably because there is no gain from forcing affinity and some loss due to having to handle interrupts on particular cores.
You can see this easily enough just in task manager. Watch the individual CPU graphs when you have a single compute bound process running.
You can give Spotlight on Windows a try. You can graphically drill into all sorts of performance and load indicators. Its freeware.
perfmon from Microsoft can monitor each individual CPU. perfmon also works remote and you can monitor farious aspects of Windows.
I'm not sure if it helps to find run-away processes because the Windows scheduler dos not execute a process always on the same CPU -> on your 8 CPU machine you will see 12.5 % usage on all CPU's if one process runs away.