Some questions about ARM/ARM64 write combine(NORMAL_NC) memory attribute read/write and memory barriers
I want to test the correctness of DDR, that is, write a value to a certain address in DDR, ensure that it is written to DDR, then read the value from DDR and check if it is equal to the written value. For example, for the c code: unsigned long test(volatile unsigned long *vaddr, unsigned long val) { *vaddr = val; __asm__ volatile ("dsb sy":::"memory"); return *vaddr; } Assuming the input vaddr uses write combine(NORMAL_NC) memory attribute and the physical address is DDR RAM, I want to ensure that val is indeed written to DDR before reading the contents of vaddr from DDR. There are two key points here: 1. Ensure that `val` is written to DDR, at least ensuring that the CPU issues a write request to the external bus, and not that it's in the CPU's internal cache. 2. Ensure that `vaddr` is read from DDR, at least ensuring that the CPU issues a read request to the external bus, and not that it uses data predicted from the CPU cache or instructions. I'm not an expert in architecture or hardware. I've consulted many resources, but I still don't quite understand. Below is some information I've compiled; please correct me if anything is incorrect: (1) `write combine` is a concept in x86; the `NORMAL_NC` attribute in ARM/ARM64 provides similar functionality. In Linux, the `pgprot_writecombile` attribute corresponds to the `NORMAL_NC` mapping attribute in ARM64. https://elixir.bootlin.com/linux/v5.10.258/source/arch/arm64/include/asm/pgtable.h#L490 https://elixir.bootlin.com/linux/v5.10.258/source/arch/arm64/include/asm/memory.h#L136 https://elixir.bootlin.com/linux/v5.10.258/source/arch/arm64/mm/proc.S#L56 https://elixir.bootlin.com/linux/v5.10.258/source/arch/arm64/include/asm/sysreg.h#L636 https://developer.arm.com/documentation/111107/2026-03/AArch64-Registers/MAIR-EL1--Memory-Attribute-Indirection-Register--EL1- oooo is encoded as follows: 0b0100 Normal memory, Outer Non-cacheable. iiii is encoded as follows: 0b0100 Normal memory, Inner Non-cacheable. According to the arm64 documentation, the NORMAL_NC attribute does not use L1/L2/L3 caches, which is consistent with x86. (2) In x86, the `write combine` memory attribute does use Write Buffer, which appears to be a dedicated buffer for the it. Does the ARM/ARM64 NORMAL_NC mode have a similar Write Buffer? (3) No read buffer exists, whatever x86/arm/arm64. (4) The `write combine (NORMAL_NC)` attribute in x86/arm/arm64 may do the memory reordering. When performing mixed read and write operations on related memory, I found two possible behaviors: 4.1 Read operations will pause to ensure that the previous Write Buffer has been written out of the CPU and cleared before proceeding with the read operation: https://fgiesen.wordpress.com/2013/01/29/write-combining-is-not-your-friend/ 4.2 A read operation might read an incorrect value because a previous write operation might still be stored in the write buffer and not yet written out, while a read operation sends a read request directly to the outside of the CPU without going through any buffer. However, both of the above possible behaviors are for x86; I didn't find much relevant information for ARM/ARM64. (5) DMA memory might be mapped using write combine, and since DMA runs outside the CPU, and looking for the Linux code found thath `dma_mb()` uses `dmb(osh)`: https://elixir.bootlin.com/linux/v5.10.258/source/arch/arm64/include/asm/barrier.h#L48 I believe that in the above C example, at least an osh barrier ensures that the write request will be sent outside the CPU. My guesses: 1. In ARM/ARM64 NORMAL_NC memory attribute, CPU may do the speculative execution. Therefore, without a barrier, scenario (4).2 might occur, or the read operation might not happen at all (the CPU instruction executor assumes that the value read immediately after writing must have just been written, so it simply doesn't read it). 2. In the C code example above, `dmb osh` ensures that the write operation sends a write request to the CPU's external bus (it might not be written to DDR, but it definitely reaches the external CPU), and only then does the read operation occur. The read operation will definitely send a read request to the CPU's external bus and use the value returned by the bus. Is my guess correct?
Take Your Experience to the Next Level
NewDownload our mobile app for a faster and better experience.
Comments
0U
Join the discussion
Sign in to leave a comment