About those `compiler_fence` calls in the barrier functions.

[cbiffle]

We're going through doing an audit of barrier/fence usage in DMA situations in a separate codebase, and I was comparing my implementations to yours when I noticed something odd that I wanted to ask about.

#264 introduced calls to compiler_fence in the inline asm versions of the barrier functions, such as dmb. The current implementation of dmb reads:

#[inline(always)]
pub unsafe fn __dmb() {
    asm!("dmb");
    compiler_fence(Ordering::SeqCst);
}

I'm concerned about what the presence of compiler_fence implies. One of two things must be true here:

1. asm! without the nomem attribute participates in the compiler's ordering of memory accesses, such that no loads/stores above the asm! will be moved to below it in the generated instruction stream. In this case, the compiler_fence is unnecessary and arguably misleading (as it has misled at least one person: me).

2. The compiler_fence really is necessary to prevent motion of memory accesses, in which case the compiler is free to move instructions after the dmb but before the compiler_fence. This would be, well, bad.

I think (1) is likely the case, though it is not explicitly documented as such in the inline asm RFC. Before I go hassle the UCG folks again, I was curious if I'm missing something and y'all had a more detailed rationale for this pattern.

Thanks!

[adamgreig]

I think this point was more or less unresolved in https://github.com/rust-embedded/cortex-m/pull/264/files#r483971671. Maybe @jonas-schievink has had further insight since.

I'm still not sure if the semantics are fully defined, but I'd welcome reassurance. In the unstable book it says that without nomem, "the compiler will assume that inline assembly can read or write any memory address that is accessible to it (e.g. through a pointer passed as an operand, or a global)".

That doesn't seem equivalent to saying "will never re-order memory accesses around the inline assembly", because in the example of DSB/ISB/etc we don't pass any pointers to it, and we care about accesses to memory that is not globally accessible. Whether this would ever happen in practice, though...? It's certainly not made simpler by how these functions may be FFI, LTO-inlined FFI, or straight up inline Rust code when using our inline-asm feature.

The book goes on to reference the unsafe code guidelines (re what memory locations FFI calls may access), but as far as I could tell they don't yet say anything about this.

At first glance you're right on (2), though; perhaps we should add a compiler fence before each inline asm block as well.

[jonas-schievink]

1. asm! without the nomem attribute participates in the compiler's ordering of memory accesses, such that no loads/stores above the asm! will be moved to below it in the generated instruction stream.

I think this is not the case – the assembly block is likely only considered to access memory whose address has previously been observed or is generally known (by virtue of being a static).

2. The compiler_fence really is necessary to prevent motion of memory accesses, in which case the compiler is free to move instructions after the dmb but before the compiler_fence. This would be, well, bad.

This sounds correct to me, which means that they need to be swapped.

[adamgreig]

This sounds correct to me, which means that they need to be swapped.

In principle, if we just swapped them around without adding a second fence, could the compile re-order an access to before the inline asm block but after the preceding compiler fence?

In other words, we specify:

1. Fence
2. DSB
3. Memory access

But it re-orders to:

1. Fence
2. Memory access
3. DSB

[jonas-schievink]

Hmm, yeah, I guess we need 2 fences then 🤔