C++ | Avoiding Move Semantics

	Motivated and inspired by Decl. In development and soon to be open-sourced.

C++ | Avoiding Move Semantics

For the video overview of this week, let's go with C++ | Avoiding Move Semantics if we really have to and determine how expensive the move operation is on the assembly level - whether that is the move constructor or assignment.

The move in question is a very simple swap operation that renders the moved entity "unused" or "dumped" but still needs to be part of the C++ RAII.

S(S&& other) {
  std::cout << "S(S&&)" << std::endl;
  i_ptr = other.i_ptr;
  other.i_ptr = nullptr;
}

The assembly of this move constructor effectively involves about 4 assembly instructions:

mov     rax, QWORD PTR [rbx]
mov     QWORD PTR [rsp+24], rdx
mov     QWORD PTR [rdx], rax
mov     QWORD PTR [rbx], 0

The unavoidable move always takes place even for a prvalue for vector::push_back or any function similar to it.

For example,

{
  std::vector<S> v;

  v.push_back(S(1));
}

This code produces the following RAII output:

S(int) # OK! This is a just constructor
S(S&&) # OVERHEAD! Move involved in order to "move" the entity into the destination memory region of the vector item
~S()   # OVERHEAD! Destructor invoked for the prvalue at the end of the expression - the semi colon
~S()   # OK! Destructor invoked by the "~std::vector<S>()"

Avoiding these two overheads is more tricky than it appears at first, we need to get around, or essentially escape:

• The automatic [variable, primitive stack] restoration (the assembly code generated by the compiler that reclaims the memory on stack at the end of the scope - required for primitive structs and static or fixed-size arrays e.g. unsigned char[])
• C++ RAII (Constructor and Destructor Life Cycle)

Both of these C and C++ requirements implicitly happen at the end of the [current] scope - generated by the compiler. So the catch is that we need to somehow allocate memory on stack that is not gonna be affected by the current scope but this memory still needs to reside somewhere - this goes for every entity conceivable in a C/C++ program except for constant literals.

To pull this off, we need to rely on the custom memory [management] allocator that we introduced in the C++ | Custom Memory Management video.

The customor_new<T> returns the pointer address so there will be no C++ RAII or automatic stack restoration in the current scope as it is a primitive type and, most importantly, it does not reside in the current scope!

static CustomMemoryStorage<1000> allocator;

int main() {
  std::vector<S*> v1;

  v1.push_back(
    allocator.custom_new<S>(1) // Same as "S(2)" - just via a custom memory allocator
  );

  defer: {
    for(auto start = v1.begin(); start != v1.end(); start++) {
      (*start)->~S();
    }
  }

  return 0;
}

This code produces the following output:

S(int)
~S()
# No moves or additional destructors in between whatsoever!

Another upside is that the vector does not have to hold the entire sizeof(S) but, rather, the 8 byte as the pointer reference is sizeof == 8 on 64-bit machine and it is essentially uintptr_t so it is just an integer all things considered. The bottom line is - storing it into the destination container buffer like std::vector takes one mov assembly instruction!

The primary and major downside of this is that we don't have automatic stack restoration [and implicit destructor invocation, less importantly] that would otherwise be generated by the compiler, which means each time we need to restore or destruct a piece of allocated memory - it must be handled explicitly like the ~S() destructor in the example above. To feature this type of restoration on stack [that happens implicitly or automatically], the usage of the custom allocator can get really complicated, hence something along the lines of memory pools would be the preferable implementation.

Even though that kind of memory would reside on heap [for memory pools], the important bottleneck to eliminate is the individual malloc [system/IO] calls per block allocation - as long as we allocate ample memory for the next cycle of memory blocks to be used. In other words, even though it is memory on heap - that is the trade-off we can afford.

For example,

malloc(sizeof(MyObj) + sizeof(MyOtherObj));
// Better than
malloc(sizeof(MyObj));
malloc(sizeof(MyOtherObj));

We may also refer to this technique as "Lazy [Memory] Allocation".

Also, another thing is that we need to be careful how we use these references as the point to the same pointer so mutating:

v1[i]->i = 1

will affect every other reference to that object. Make sure any mutation is intentionally applied through the vector or respective container buffer and does not produce unintended side effects across aliases.

Just by trying to get around the move semantics of C++, as a result - we also bypassed the copy and destructor semantics. In other words, we went entirely nuclear!

Last but not least, using a custom memory allocator like this, which fetches and "reserves" a piece of memory to be used, suppresses all compiler optimizations that could have otherwise been done to the variables - for example, storing variables on register for super fast READ/WRITE is NOT an optimization compiler can do in this case.

📘 Why not use std::unique_ptr with the custom allocator that operates on stack?

It is important to note that using a std::unique_ptr in this case (even with a custom memory allocator) is the std::unique_ptr() + ~std::unique_ptr() RAII overhead.

In conclusion, using a primitive pointer fully escapes the automatic (primitive or stack) restoration and the C++ RAII.

Conclusion and YouTube Video

I AM NOT advising that you prefer doing this for general code IN ANY SHAPE OR FORM. This is rather an experimental overview on how to escape the automatic stack restoration and C++ RAII. However, if your use case is specific enough with many repeated std::vector reallocations in your application or other heavy workloads performed by the container buffers such as [shallow] copying of objects and similar, approaching structs/objects in C++ like this just might pay off for some performance purposes (e.g. JIT, or bytecode interpreters written in C++).

Keep in mind that this is just a brief overview of the YouTube video! For the full experience with all the details we missed here, check out my YouTube channel and C++ | Avoiding Move Semantics if we really have to.

For the source code of this overview and the video, check out my GitHub repository at https://github.com/BeAllAround/S2S-Compilers-Blog-Series-Public.