Debunking "bad_alloc" and out of memory errors in C++

[lefticus]

Some have said that there's no meaningful way to deal with out of memory in C++, but I have students who regularly deal with this stuff, so I'm recording an episode about it.

[LB--]

I agree, it's certainly quite manageable on some operating systems, though there are issues in the language that make it sketchy. A typical example is software that allows opening multiple projects in the same process, if opening a project results in running out of memory it can safely be rolled back and the other open projects (which may have unsaved changes) can remain open and unharmed. The problem is that the way C++ exceptions are dynamically allocated means the exception throwing code can itself fail. You could hope and pray that compiler implementers carve out some special memory or special handling for std::bad_alloc, but that doesn't help much if you need to use your own exception types, especially since std::bad_alloc doesn't give you any information about the size of the allocation that failed, meaning you have to use your own workarounds if you want that information (such as to tell the user and let them decide how much trouble they are in). So, there are certainly things you can do but it's not very comfortable to do so, and it can still result in process termination for no reason.

Another issue is that some operating systems use overcommit, where they give you a pointer to memory that they tell you they've allocated, when actually all they've done is allocate the address space and not the backing memory. Then you don't find out there's a problem until you access every page of that memory and force the OS to fault in the pages, at which point it's usually too late (or too troublesome) to do anything but terminate the process. You don't ever get std::bad_alloc on these systems unless all address space has been used up. I have seen advice in the linux world that you should instead hook into APIs for monitoring overall memory usage and avoid doing heavy things if free memory starts looking low. Absolutely terrifying situation, and not something C++ can really do anything about.

[lefticus]

Coming in Ep451