munmap need not point to the start of an object

[Zoxc]

Using munmap with a partial deallocation results in the following error:
error: Undefined Behavior: deallocating 0x2d0000[alloc75617]<212514> which does not point to the beginning of an object

However munmap supports such partial deallocation so this is not actually UB.

[RalfJung]

munmap supports this, but the LLVM memory model, and thus by extension every language compiled via LLVM, does not. LLVM optimizations assume that allocations can never be partially freed. Consequently, in Rust, allocations can currently not change their shape or size once they are created.

[RalfJung]

So, closing in favor of rust-lang/unsafe-code-guidelines#430 -- the question really is if and how the Rust memory model can support partial munmap. (The same question arises for C, people just don't seem to talk about that.)

To be clear, I think we should support at least munmap at the "end" of an allocation (allocations with gaps have their own challenges). But we first need to convince LLVM to support that and potentially fix optimizations that rely on allocations never shrinking.

[Zoxc]

You're assuming that munmap and mmap create Rust allocations here, which is something I would not expect. Do LLVM count these as allocation functions?

[RalfJung]

I am assuming that they generate memory that Rust can access. If Rust can't access that memory then they become kind of pointless. :)

LLVM assumes that if at moment A, one can access both ptr and ptr.add(N), and then later one can still access ptr, then it follows that one can also still access ptr.add(N). Partial munmap clearly violates this. Therefore partial munmap is incompatible with LLVM's assumptions.

rust-lang/unsafe-code-guidelines#430 is tracking whether we want Rust to make such assumptions. But currently even if we wanted the answer to be "no", we'd first have to fix LLVM.

[Zoxc]

The use case I have is aligned allocation where I overallocate then munmap the unneeded parts (on both ends). My impression is that LLVM's behavior would be fine for that use case at least.

[RalfJung]

If there is never a memory access nor an in-bounds pointer arithmetic for the part of the allocation that you remove again, then the sketched optimization would still be fine, yeah. (But you better use wrapping_add and not add to move around the parts of the allocation that you will later remove again.)

But the intermediate state before you "pruned" your allocation is a state that doesn't really exist in the LLVM nor Rust memory model, so this is rather shaky ground to stand on.