How to measure CPU memory bandwidth in C RTOS? - memory

Question

I am working on an ARMv7 Embedded system, which uses an RTOS in the cortex-A7 SOC(2 cores, 256KB L2 Cache).
Now I want to measure the memory bandwidth of the CPU in this system, so I wrote following functions to do the measurement.
The function allocates 32MB memory and does memory reading in 10 loops. And measure the time of the reading, to get the memory reading bandwidth.. (I know there is dcache involved in it so the measurement is not precise).
#define T_MEM_SIZE 0x2000000
static void print_summary(char *tst, uint64_t ticks)
{
uint64_t msz = T_MEM_SIZE/1000000;
float msec = (float)ticks/24000;
printf("%s: %.2f MB/Sec\n", tst, 1000 * msz/msec);
}
static void memrd_cache(void)
{
int *mptr = malloc(T_MEM_SIZE);
register uint32_t i = 0;
register int va = 0;
uint32_t s, e, diff = 0, maxdiff = 0;
uint16_t loop = 0;
if (mptr == NULL)
return;
while (loop++ < 10) {
s = read_cntpct();
for (i = 0; i < T_MEM_SIZE/sizeof(int); i++) {
va = mptr[i];
}
e = read_cntpct();
diff = e - s;
if (diff > maxdiff) {
maxdiff = diff;
}
}
free(mptr);
print_summary("memrd", maxdiff);
}
Below is the measurement of reading, which tries to remove the caching effect of Dcache.
It fills 4Bytes in each cache line (CPU may fill the cacheline), until the 256KB L2 cache is full and be flushed/reloaded, so I think the Dcache effect should be minimized. (I may be wrong, correct me please).
#define CLINE_SIZE 16 // 16 * 4B
static void memrd_nocache(void)
{
int *mptr = malloc(T_MEM_SIZE);
register uint32_t col = 0, ln = 0;
register int va = 0;
uint32_t s, e, diff = 0, maxdiff = 0;
uint16_t loop = 0;
if (mptr == NULL)
return;
while (loop++ < 10) {
s = read_cntpct();
for (col = 0; col < CLINE_SIZE; col++) {
for (ln = 0; ln < T_MEM_SIZE/(CLINE_SIZE*sizeof(int)); ln++) {
va = *(mptr + ln * CLINE_SIZE + col);
}
}
e = read_cntpct();
diff = e - s;
if (diff > maxdiff) {
maxdiff = diff;
}
}
free(mptr);
print_summary("memrd_nocache", maxdiff);
}
After running these 2 functions, I found the bandwith is about,
memrd: 1973.04 MB/Sec
memrd_nocache: 1960.67 MB/Sec
The CPU is running at 1GHz, with DDR3 on die， the two testing has the similar data!? It is a big surprise to me.
I had worked with lmbench in Linux ARM server, but I don't think it can be ran in this embedded system.
So I want to get a software tool to measure the memory bandwidth in this embedded system, get one from community or do it by myself.