I would say that it's likely the opposite - larger flat containers will have more expensive insert and erase.
That's true, but often the data is relatively stable, in which case the flat containers are a good choice.
Flat containers are especially useful with small number of (small) elements because they are cache-friendly, and small size optimization makes perfect sense to improve that use case even further.
It is my experience that the flat containers *always* outperform the node based containers on searching. So when the data is stable, I always use a flat container regardless of how many elements are in the container. One of the problems with that approach is that, because insert is slow, I often have to build up a std::vector of items, sort it and then copy them into the flat container using the ordered_unique_t overload. If this change was done in a way that allowed me to pass ownership of that sorted vector to the flat_* container, that would be an additional optimisation I would find useful. -- chris