A question about cache

[mahdi259]

Hi @stnolting
Can I ask you guide me about analyzing cache activity of a sample program?

[stnolting]

Hey @mahdi259.

How can I help you? Do want to benchmark the efficiency of a certain cache configuration?

The CPU features some "hardware performance monitors" (HPMs) which can be used to count certain events like instruction fetch wait cycles (see 📚 Hardware Performance Monitors (HPM) CSRs).

Unfortunately, we do not (yet?) have HPM events for cache misses. However, there are some HPM events that can give some insights about cache efficiency:

• HPMCNT_EVENT_WAIT_DIS: instruction dispatch wait cycle (wait for instruction prefetch-buffer refill (CPU Control Unit IPB); caused by a fence instruction, a control flow transfer or a instruction fetch bus wait cycle); this event will also triggered if there is a (I-)cache miss.
• HPMCNT_EVENT_WAIT_LSU: any memory/bus/cache/etc. delay/wait cycle while executing any load or store operation (caused by a data bus wait cycle))

[mahdi259]

Thanks a lot. I will try HPMCNT_EVENT_WAIT_LSU. I'm also looking at some low level signals that show cache activity like ctrl in dcache_inst and external memory signals.

[mahdi259]

Hi @stnolting
One interesting thing that I found is that the direct mapped strategy may result in odd execution times. For example consider a matrix multiplication of two arrays (X and Y) with data defined in a header file.

int X[len] = {1, 2, 3, ...};
int Y[len] = {99, 100, 101, ...};
int O[len];

for (int i=0; i<len; i++)
{
    O[i] = X[i] * Y[i];
}

In this scenario, CPU loads X[i] and Y[i] (and the associated blocks) in cache before multiplication. Here if X and Y go into one block (due to similar LSB bits), then cache will go to miss state every time CPU wants to read the block of X[i] or Y[i], and will try to reload them. This leads to inefficiency in loading data from main memory because other block elements are not used and are always flushed. This way the same block is loaded into data cache multiple times and multiplication will experience long memory delays (ping pong effect).

One possible solution is to increase cache block numbers and decrease cache block size. This way it's more likely that X and Y go into different blocks (due to larger MSB bits that determine cache block number) and avoid cache collision. The other solution is to use set associative caches in which multiple blocks with the same MSB bits can be stored in the cache without the need to flush each other.

[stnolting]

What cache configuration did you use? I-cache and D-cache ("L1") or are you also using the X-cache ("L2")?

Using the I- and D-caches for a setup with internal memory only does not make much sense, as these memories are quite fast.

This way the same block is loaded into data cache multiple times and multiplication will experience long memory delays (ping pong effect).

Right, the current caches are not very efficient as they are direct-mapped. This was much easier to implement than a full-features set-associative cache. However, optimizing the caches is still on my todo-list.

One possible solution is to increase cache block numbers and decrease cache block size

Right. The "best" cache configuration highly depends on the workload of the program being executed. Its trial and error I guess... 🙈

[mahdi259]

I use neorv32_tb in which Icache and Dcache are enabled and there is an external memory. Block numbers * block sizes are as below:
Icache: 128 * 32
Dcache: 512 * 16
L2 cache: 4 * 32
I didn't care L2 cache since I didn't know that xbus cache is L2 cache.

[stnolting]

I didn't care L2 cache since I didn't know that xbus cache is L2 cache.

You can see the "cache hierarchy" in the documentation's 📚 Bus System section:

I am not sure if "level 2" cache really fits here. The X-cache is used to cache accesses to processor-external memory that is attached via the external bus interface. However, it can be combined with the CPU-caches. In that case, I recommend to make the cache blocks of the L2 cache much larger than the ones in the L1 caches (rule of thumb 😅).

[mahdi259]

Thanks a lot