I am trying to get total CPU usage of the servers running the VMware ESXi 6.7.0 Update 3 to monitor performance.
I was able to view the memory usage using the command- vsish -e get /memory/comprehensive
However, I can't find a command for CPU usage. There isn't a top command or /proc/stat file.
Is there a way I can get this info?
Thanks
I'm using ESXi 6.0.0U2 and after looking through the vsish nodes for a bit, I found some relevant information in the power/pcpu tree.
[root#server:~] vsish -e ls power
policy/
pcpu/
hardwareSupport
hostStats
resetStats
currentPolicy
[root#server:~] vsish -e ls power/pcpu
0/
1/
2/
3/
4/
5/
6/
7/
8/
9/
10/
11/
12/
13/
14/
15/
My server has 8 hyperthreaded CPUs, so 16 virtual cores show up in ESXi. Here's what it shows about the first core:
[root#server:~] vsish -e ls power/pcpu/0
cstate/
tstate/
pstate/
cres
perf
state
[root#server:~] vsish -e get power/pcpu/0/perf
PCPU performance statistics {
PCPU core busy (current frequency): 40 %
PCPU core busy (maximum frequency): 40 %
APERF/MPERF ratio: -1 %
Halted time: 176111536370 usec
}
Maybe there's a better way to do this, but this shows the info for all the cores:
[root#server:~] vsish -e get $(printf 'power/pcpu/%sperf ' $(vsish -e ls power/pcpu))
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 175721833728 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 188174562097 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 24 %
PCPU core busy (maximum frequency): 24 %
APERF/MPERF ratio: -1 %
Halted time: 174102490208 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 20 %
PCPU core busy (maximum frequency): 20 %
APERF/MPERF ratio: -1 %
Halted time: 187428554783 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 65 %
PCPU core busy (maximum frequency): 65 %
APERF/MPERF ratio: -1 %
Halted time: 172642150099 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 55 %
PCPU core busy (maximum frequency): 55 %
APERF/MPERF ratio: -1 %
Halted time: 185952136508 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 172496627680 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 185168370294 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 174843857000 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 100 %
PCPU core busy (maximum frequency): 100 %
APERF/MPERF ratio: -1 %
Halted time: 188132157292 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 32 %
PCPU core busy (maximum frequency): 32 %
APERF/MPERF ratio: -1 %
Halted time: 174204368068 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 63 %
PCPU core busy (maximum frequency): 63 %
APERF/MPERF ratio: -1 %
Halted time: 185497348425 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 43 %
PCPU core busy (maximum frequency): 43 %
APERF/MPERF ratio: -1 %
Halted time: 172301626490 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 44 %
PCPU core busy (maximum frequency): 44 %
APERF/MPERF ratio: -1 %
Halted time: 185424645378 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 40 %
PCPU core busy (maximum frequency): 40 %
APERF/MPERF ratio: -1 %
Halted time: 172333434508 usec
}
PCPU performance statistics {
PCPU core busy (current frequency): 41 %
PCPU core busy (maximum frequency): 41 %
APERF/MPERF ratio: -1 %
Halted time: 184481550403 usec
}
If you were looking for an average CPU usage percentage across all the cores, I haven't found that directly yet but you could compute it:
[root#server:~] vsish -e get $(printf 'power/pcpu/%sperf ' $(vsish -e ls power/pcpu)) | awk '/current/ {cpus+=1;total+=$6} END {print total/cpus "%"}'
78.125%
I've been wanting to know this too and I didn't know about the vsish command so thanks for pointing it out.
However, I can't find a command for CPU usage. There isn't a top command or ...
There is a top command but it is "esxtop". See https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-D89E8267-C74A-496F-B58E-19672CAB5A53.html for the latest documentation. Note that although this is documentation for ESXi 7.0 esxtop has been around for over a decade so you should be able to use in ESXi 6.7.
Related
I'm running a load test with wrk2 as a job on Jenkins. I'd like to send the results of the load test to Graylog but I only want to store the Requests/Sec and average latency.
Here's what the output looks like:
Running 30s test # https://example.com
1 threads and 100 connections
Thread calibration: mean lat.: 8338.285ms, rate sampling interval: 19202ms
Thread Stats Avg Stdev Max +/- Stdev
Latency 16.20s 6.17s 29.64s 65.74%
Req/Sec 5.00 0.00 5.00 100.00%
Latency Distribution (HdrHistogram - Recorded Latency)
50.000% 15.72s
75.000% 20.81s
90.000% 24.58s
99.000% 29.13s
99.900% 29.66s
99.990% 29.66s
99.999% 29.66s
100.000% 29.66s
Detailed Percentile spectrum:
Value Percentile TotalCount 1/(1-Percentile)
4497.407 0.000000 1 1.00
7561.215 0.100000 11 1.11
11100.159 0.200000 22 1.25
12582.911 0.300000 33 1.43
14565.375 0.400000 44 1.67
15720.447 0.500000 54 2.00
16416.767 0.550000 60 2.22
17301.503 0.600000 65 2.50
18464.767 0.650000 71 2.86
19185.663 0.700000 76 3.33
20807.679 0.750000 81 4.00
21479.423 0.775000 84 4.44
22347.775 0.800000 87 5.00
22527.999 0.825000 90 5.71
23216.127 0.850000 93 6.67
23478.271 0.875000 95 8.00
23805.951 0.887500 96 8.89
24723.455 0.900000 98 10.00
25067.519 0.912500 99 11.43
25395.199 0.925000 101 13.33
26525.695 0.937500 102 16.00
26525.695 0.943750 102 17.78
26705.919 0.950000 103 20.00
28065.791 0.956250 104 22.86
28065.791 0.962500 104 26.67
28377.087 0.968750 105 32.00
28377.087 0.971875 105 35.56
28475.391 0.975000 106 40.00
28475.391 0.978125 106 45.71
28475.391 0.981250 106 53.33
29130.751 0.984375 107 64.00
29130.751 0.985938 107 71.11
29130.751 0.987500 107 80.00
29130.751 0.989062 107 91.43
29130.751 0.990625 107 106.67
29655.039 0.992188 108 128.00
29655.039 1.000000 108 inf
#[Mean = 16199.756, StdDeviation = 6170.105]
#[Max = 29638.656, Total count = 108]
#[Buckets = 27, SubBuckets = 2048]
----------------------------------------------------------
130 requests in 30.02s, 13.44MB read
Socket errors: connect 0, read 0, write 0, timeout 1192
Requests/sec: 4.33
Transfer/sec: 458.47KB
Does anyone know how I could go about extracting Requests/sec (at the bottom) and the latency average to send as JSON parameters?
The expected output would be: "latency": 16.2, "requests_per_second": 4.33
You didn't provide the expected output so your question isn't clear but is this what you want?
$ awk 'BEGIN{a["Latency"]; a["Requests/sec:"]} ($1 in a) && ($2 ~ /[0-9]/){print $1, $2}' file
Latency 16.20s
Requests/sec: 4.33
Updated based on you adding expected output to your question:
$ awk '
BEGIN { map["Latency"]="latency"; map["Requests/sec:"]="requests_per_second" }
($1 in map) && ($2 ~ /[0-9]/) { printf "%s\"%s\": %s", ofs, map[$1], $2+0; ofs=", " }
END { print "" }
' file
"latency": 16.2, "requests_per_second": 4.33
net.rubyeye.xmemcached.XMemcachedClient:get:1016 (method time = 0 ms, total time = 74 ms)
net.rubyeye.xmemcached.XMemcachedClient:get0:1058 (method time = 0 ms, total time = 74 ms)
net.rubyeye.xmemcached.XMemcachedClient:fetch0:644 (method time = 0 ms, total time = 74 ms)
net.rubyeye.xmemcached.XMemcachedClient:latchWait:2516 (method time = 0 ms, total time = 74 ms)
java.util.concurrent.locks.LockSupport:parkNanos:215 (method time = 74 ms, total time = 74 ms)
I need to realise a PWM output with 5 kHz +/- 5%. (Presumably due to a filter circuit into which this feeds over which I have no control.)
Is this realisable with a ESP8266 (ideally with NodeMCU)?
I realise that the software PWM of the ESP8266 has a maximum frequency of 1 kHz while the sigma-delta can be used to implement a PWM with a fixed frequency of about 300 kHz.
So is there a reliable way to achieve 5 kHz? I know some people experimented with the I2S peripheral for waveform output but I am unsure whether it can be used for 5kHz output.
Has anybody looked at into a similar problem before?
The essential code is:
#define qap 2 // Quick As Possible ... Duty cycle only 0, 50, 100%
#define HFreq 5150
#define pPulse D2 // a NodeMCU/ESP8266 GPIO PWM pin
analogWriteRange(qap); analogWriteFreq( HFreq ); analogWrite(pPulse, 1); // start PWM
TL;DR
Just did some crude benchamrks
// had HFreq=126400 with 75 KHz pulse at 80 MHz ESP clock, every 1 sec but proof of evidence lost
nominally with 80 MHz clock got
// 72.24 KHz 361178 2015061929
// 72.23 KHz 361163 2415062390
// 72.23 KHz 361133 2815062824
and
// 141.52 KHz 353809 2009395076
// 141.54 KHz 353846 2409395627
// 141.52 KHz 353806 2809395946
with 160 MHz clock
CAVEATS!!!
1. The duty cycle range is 2!
2. As well the system had no other conditioning, so other timing artifacts were still present from Serial IO etc. In particular instability due to the WDT and WiFi penultimately appeared. (Anyhow, presumably this is an issue only when the ESP8266 is under stress and duress.)
// 141.50 KHz 353754 619466806
// 141.52 KHz 353810 1019467038
// ...
// ad infinitum cum tempore finitum, infinitus est ad nauseum?
// ...
// 141.54 KHz 353857 735996888
//
// ets Jan 8 2013,rst cause:4, boot mode:(1,7)
//
//wdt reset
When using the following code to generate a 5 KHz square wave signal, the considerations above are not an issue and do not occur.
// ------------ test results for 80 MHz clock --------------
//
//
// PWM pulse test
//
// F_CPU: 80000000L
// ESP8266_CLOCK: 80000000UL
// PWM "freq.": 5150
//
//
// connect D1 to D2
//
//
// raw MPU
// frequency count cycle
// 0.00 KHz 1 407976267
// 4.74 KHz 9482 567976702
// 5.00 KHz 10007 727977137
// 5.00 KHz 10006 887977572
// 5.00 KHz 10006 1047978007
// 5.00 KHz 10007 1207978442
// 5.00 KHz 10006 1367978877
// 5.00 KHz 10006 1527979312
// 5.00 KHz 10007 1687979747
// 5.00 KHz 10006 1847980182
// 5.00 KHz 10006 2007980617
// 5.00 KHz 10007 2167981052
// 5.00 KHz 10006 2327981487
// 5.00 KHz 10006 2487981922
// 5.00 KHz 10007 2647982357 ...
//
// crude testing for 5KHz signal
// extracted from:
// highest frequency / shortest period pin pulse generate / detect test
//
// uses raw ESP8266 / NodeMCU V1.0 hardware primitive interface of Arduino IDE (no included libraries)
//
// timing dependencies: WDT, WiFi, I2S, I2C, one wire, UART, SPI, ...
//
// Arduino GPIO 16 5 4 0 2 14 12 13 15 3 1 0 1 2 3 4 5 ... 12 13 14 15 16
// NodeMCU D pin 0 1 2 3 4 5 6 7 8 9 10 3 10 4 9 2 1 ... 6 7 5 8 0
// | | | | | | | | | | |
// a WAKE | | F Tx1 | | Rx2 Tx2 Rx0 Tx0
// k (NO PWM or | | L blue | | | |
// a' interrupt) | S A * H | H | H | | * led's
// s red S D S S M S H
// * C A H C I I C
// L T L S M S
// K A K O O
// └ - - - - └----UART's----┘
// └--I2C--┘ └-----SPI------┘
//
// rules of engagement are obscure and vague for effects of argument values for the paramters of these functions:
// analogWriteRange(qap); analogWriteFreq( HFreq ); analogWrite(pPulse, 1);
//
// http://stackoverflow.com/questions/42112357/how-to-implement-esp8266-5-khz-pwm
//
// system #defines: F_CPU ESP8266_CLOCK
#define pInt D1 // HWI pin: NOT D0 ie. GPIO16 is not hardwared interrupt or PWM pin
#define pPulse D2 // PWM pulsed frequency source ... ditto D0 (note: D4 = blue LED)
#define countFor 160000000UL
#define gmv(p) #p // get macro value
#define em(p) gmv(p) // evaluate macro
#define qap 2 // minimal number of duty cycle levels (0, 50, 100% ) Quick As Possible ...
#define HFreq 5150 //((long int) F_CPU==80000000L ? 125000 : 250000) // ... to minimize time of a cycle period
// max values ^ and ^ found empirically
// had HFreq=126400 with 75 KHz pulse at 80 MHz ESP clock, every 1 sec but proof of evidence lost
#define infoTxt (String) \
"\n\n\t PWM pulse test " \
"\n F_CPU: " em(F_CPU) \
"\n ESP8266_CLOCK: " em(ESP8266_CLOCK) \
"\n PWM \"freq.\": " + HFreq + "\n" \
"\n\n connect " em(pInt) " to " em(pPulse) "\n" \
"\n\n raw MPU " \
" \n frequency count cycle "
long int oc=1, cntr=1;
unsigned long int tc=0;
void hwISR(){ cntr++; } // can count pulses if pInt <---> to pPulse
void anISR(){ tc=ESP.getCycleCount(); timer0_write( countFor + tc ); oc=cntr; cntr=1; }
void setup() { // need to still confirm duty cycle=50% (scope it or ...)
noInterrupts();
Serial.begin(115200); Serial.println(infoTxt); delay(10); // Serial.flush(); Serial.end(); // Serial timing?
analogWriteRange(qap); analogWriteFreq( HFreq ); analogWrite(pPulse, 1); // start PWM
pinMode( pInt, INPUT ); attachInterrupt(pInt, hwISR, RISING); // count pulses
timer0_isr_init(); timer0_attachInterrupt( anISR ); anISR(); //
interrupts();
}
void loop() { delay(10); if (oc==0) return;
Serial.println((String)" "+(oc/1000.0*F_CPU/countFor)+" KHz "+oc+" "+tc); oc=0; }
//
You can also use the hardware SPI interface which has an adjustable clock speed.
By writing continuous data, the output waveform appears on SCLK.
I've figured out the abovementioned while creating a knight rider effect with IC 74hc595 on ESP8266, using MicroPython (which is indeed slow, as being a script language).
It worked for me up to 4MHz SPI clock speed.
The disadvantage is, for permanent waveform, you need to write data to MOSI forever (as when SPI data buffer goes empty, there is no signal on SCLK anymore).
I am performing logistic regression using Vowpal Wabbit on a dataset with 25 features and 48 million instances. I have a question on current predict values. Should it be within 0 or 1.
average since example example current current current
loss last counter weight label predict features
0.693147 0.693147 1 1.0 -1.0000 0.0000 24
0.419189 0.145231 2 2.0 -1.0000 -1.8559 24
0.235457 0.051725 4 4.0 -1.0000 -2.7588 23
6.371911 12.508365 8 8.0 -1.0000 -3.7784 24
3.485084 0.598258 16 16.0 -1.0000 -2.2767 24
1.765249 0.045413 32 32.0 -1.0000 -2.8924 24
1.017911 0.270573 64 64.0 -1.0000 -3.0438 25
0.611419 0.204927 128 128.0 -1.0000 -3.1539 25
0.469127 0.326834 256 256.0 -1.0000 -1.6101 23
0.403473 0.337820 512 512.0 -1.0000 -2.8843 25
0.337348 0.271222 1024 1024.0 -1.0000 -2.5209 25
0.328909 0.320471 2048 2048.0 -1.0000 -2.0732 25
0.309401 0.289892 4096 4096.0 -1.0000 -2.7639 25
0.291447 0.273492 8192 8192.0 -1.0000 -2.5978 24
0.287428 0.283409 16384 16384.0 -1.0000 -3.1774 25
0.287249 0.287071 32768 32768.0 -1.0000 -2.7770 24
0.282737 0.278224 65536 65536.0 -1.0000 -1.9070 25
0.278517 0.274297 131072 131072.0 -1.0000 -3.3813 24
0.291475 0.304433 262144 262144.0 1.0000 -2.7975 23
0.324553 0.357630 524288 524288.0 -1.0000 -0.8995 24
0.373086 0.421619 1048576 1048576.0 -1.0000 -1.2076 24
0.422605 0.472125 2097152 2097152.0 1.0000 -1.4907 25
0.476046 0.529488 4194304 4194304.0 -1.0000 -1.8591 25
0.476627 0.477208 8388608 8388608.0 -1.0000 -2.0037 23
0.446556 0.416485 16777216 16777216.0 -1.0000 -0.9915 24
0.422831 0.399107 33554432 33554432.0 -1.0000 -1.9549 25
0.428316 0.433801 67108864 67108864.0 -1.0000 -0.6376 24
0.425511 0.422705 134217728 134217728.0 -1.0000 -0.4094 24
0.425185 0.424860 268435456 268435456.0 -1.0000 -1.1529 24
0.426747 0.428309 536870912 536870912.0 -1.0000 -2.7468 25
Predictions are in the range [-50, +50] (theoretically any real number, but Vowpal Wabbit truncates it to [-50, +50]).
To convert them to {-1, +1}, use --binary. Positive predictions are simply mapped to +1, negative to -1.
To convert them to [0, +1], use --link=logistic.
This uses the logistic function 1/(1 + exp(-x)).
You should also use --loss_function=logistic if you want to interpret the numbers as probabilities.
To convert them to [-1, +1], use --link=glf1.
This uses formula 2/(1 + exp(-x)) - 1 (generalized logistic function with limits of 1).
I would like to know of any commands or utilities there are to check the runtime footprint of application in vxworks (target board).
And I'd also like to know how to find the CPU usage of target board.
Not quite sure what you mean by "runtime footprint", but the ti command will show you the stack usage for a particular task:
-> ti tNet0
NAME ENTRY TID PRI STATUS PC SP ERRNO DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
tNet0 ipcomNetTask 1040fad0 50 PEND 1014c42b 1078ff10 0 0
full task name : tNet0
task entry : ipcomNetTask
process : kernel
options : 0x9007
VX_SUPERVISOR_MODE VX_UNBREAKABLE VX_DEALLOC_STACK VX_DEALLOC_TCB
VX_DEALLOC_EXC_STACK
STACK BASE END SP SIZE HIGH MARGIN
--------- -------- -------- -------- ------- ------- -------
execution 10790000 10780000 1078ff10 65536 2404 63132
exception 10428fb8 10426030 12168 360 11808
For cpu usage, enable INCLUDE_SPY in your BSP and execute spy to start the display of the cpu usage:
->spy
NAME ENTRY TID PRI total % (ticks) delta % (ticks)
------------ ------------ ---------- --- --------------- ---------------
tJobTask 0x10098410 0x103eeb00 0 0% ( 0) 0% ( 0)
tExcTask 0x10097880 0x101d6560 0 0% ( 0) 0% ( 0)
tLogTask logTask 0x103efa58 0 0% ( 0) 0% ( 0)
tShell0 shellTask 0x1058c5c8 1 0% ( 2) 0% ( 2)
tWdbTask 0x10141e80 0x104ae950 3 0% ( 0) 0% ( 0)
tSpyTask spyComTask 0x1042ecb8 5 0% ( 8) 0% ( 2)
ipcom_tickd 0x10060090 0x1058fb50 20 0% ( 0) 0% ( 0)
tVxdbgTask 0x10051810 0x104ae658 25 0% ( 0) 0% ( 0)
tAioIoTask1 aioIoTask 0x1040df78 50 0% ( 0) 0% ( 0)
tAioIoTask0 aioIoTask 0x1040e3a0 50 0% ( 0) 0% ( 0)
tNet0 ipcomNetTask 0x1040fad0 50 0% ( 0) 0% ( 0)
ipcom_syslog 0x10055190 0x1042e5a8 50 0% ( 0) 0% ( 0)
tNetConf 0x100887e0 0x1044f8b8 50 0% ( 0) 0% ( 0)
tAioWait aioWaitTask 0x1040aa40 51 0% ( 0) 0% ( 0)
KERNEL 0% ( 0) 0% ( 0)
INTERRUPT 0% ( 0) 0% ( 0)
IDLE 99% ( 2495) 99% ( 498)
TOTAL 99% ( 2505) 99% ( 502)
->spyStop