Question about boost::multi_index emplace operations
Hi there, Is there a way to emplace (key, value) in boost multi_index hashed index in such a way that the value is not moved/stolen if the key is already present? As far as I checked the implementation for emplace (and emplace_) the node is eagerly created and the key and the value are moved into the node and then the insertion operation takes place. If the insertion fails the node is destroyed but the outside value is lost/stolen in all cases. What I actually need is to create the value only if the key is not present, because the creation is expensive, and I'd like to avoid the find+emplace or insert empty value + modify call. (As far as I checked, the value can't be modified directly from the returned iterator because it's const and will need const_cast to work). I thought to use emplace with some machinery for lazy creation struct creator { operator value_type () { return ...; } // create the value lazily, on demand }; The creator is passed to emplace instead of the actual value. However, this doesn't help because the node is eagerly created and the above operator is always called. Thanks, Pavel.
El 01/11/2022 a las 16:44, Pavel Vazharov via Boost-users escribió:
Hi there,
Is there a way to emplace (key, value) in boost multi_index hashed index in such a way that the value is not moved/stolen if the key is already present? As far as I checked the implementation for emplace (and emplace_) the node is eagerly created and the key and the value are moved into the node and then the insertion operation takes place. If the insertion fails the node is destroyed but the outside value is lost/stolen in all cases. What I actually need is to create the value only if the key is not present, because the creation is expensive, and I'd like to avoid the find+emplace or insert empty value + modify call. (As far as I checked, the value can't be modified directly from the returned iterator because it's const and will need const_cast to work). [...]
Insert+modify works fine as far as I can see:
(http://coliru.stacked-crooked.com/a/eb5dc2b569dff95e )
// C++20, easily portable to older C++ versions
#include
On Wed, Nov 2, 2022 at 10:05 PM Joaquin M López Muñoz < joaquinlopezmunoz@gmail.com> wrote:
Insert+modify works fine as far as I can see: (http://coliru.stacked-crooked.com/a/eb5dc2b569dff95e )
// C++20, easily portable to older C++ versions
#include
#include #include #include <cassert> #include <string> #include <utility> using namespace boost::multi_index;
struct element { int id; std::string str; };
using container=multi_index_container< element, indexed_by< hashed_unique
> > >; template
auto lazy_emplace(Index& i,int id,Str&& str) { auto p=i.emplace(id,std::string{}); const auto& [it,b]=p; if(p.second)i.modify(it,[&](auto& e){e.str=std::forward<Str>(str);}); return p; } int main() { container c={{0,"hello"}}; std::string str="boost";
auto [it,b]=lazy_emplace(c,0,std::move(str)); assert(b==false&&!str.empty());
std::tie(it,b)=lazy_emplace(c,1,std::move(str)); assert(b==true&&str.empty()); }
Your particular problem hints at the more general issue that Boost.MultiIndex indices do not have something like try_emplace, which is the semantics you're after. The reason why this function is not available is that we have set-like indices, not map-like indices. For instance, this expression
c.try_emplace(0,"hello");
assumes that value_type is constructible from 0 and "hello" *and* that 0 is the key. With Boost.MultiIndex, this assumption does not hold true in general, as the key is gotten via a key extractor and there's no mapping between construction args and keys or anything. That said, the following could be a potentially interesting addition to the library:
template
c.try_emplace(const Key& k,Args&&... args); where the value is constructed from args... and not from (k,args...), so, going back to your example, the syntax would be:
s.try_emplace(0,0,"hello"); // first 0 is the key, (0,"hello") constructs element
I have to think about this.
Best,
Joaquín M López Muñoz
Hi, Thanks for the response. Yes, you are right, I'm looking for an API with semantics like try_emplace and now I understand the complications for it for the case of boost multi_index. I'm currently using emplace + modify API. My initial question was not very clear about this. However, my understanding is that the modify call will try to reposition the node because it can't be sure if the user callback hasn't changed something in the entry which would require repositioning of the node in any of the indices. Is my understanding of the modify API correct? As I'm using hashed + sequenced indices, I know that setting only the mapped_type in the modify call won't cause any repositioning in the both of the above indices. Thus I was trying to avoid the additional work which happens internally in the modify call. BTW, thanks for working on boost::unordered_flat_map because it'll allow us to remove 3rd party dependency from our projects as we are currently using https://greg7mdp.github.io/parallel-hashmap/. Regards, Pavel.
El 03/11/2022 a las 8:06, Pavel Vazharov escribió:
On Wed, Nov 2, 2022 at 10:05 PM Joaquin M López Muñoz
wrote: Insert+modify works fine as far as I can see: [...]
Your particular problem hints at the more general issue that Boost.MultiIndex indices do not have something like try_emplace, which is the semantics you're after. The reason why this function is not available is that we have set-like indices, not map-like indices. [...]
Hi,
Thanks for the response.
Yes, you are right, I'm looking for an API with semantics like try_emplace and now I understand the complications for it for the case of boost multi_index.
I'm currently using emplace + modify API. My initial question was not very clear about this. However, my understanding is that the modify call will try to reposition the node because it can't be sure if the user callback hasn't changed something in the entry which would require repositioning of the node in any of the indices. Is my understanding of the modify API correct? As I'm using hashed + sequenced indices, I know that setting only the mapped_type in the modify call won't cause any repositioning in the both of the above indices. Thus I was trying to avoid the additional work which happens internally in the modify call.
modify repositions the element after the modifier is called, but *only* if necessary: in the interest of performance, a pre-check is done for each index, and if the element remains in place no further work is done. This in-place check is faster than actually re-positioning the element: https://github.com/boostorg/multi_index/blob/develop/include/boost/multi_ind... To sum up, there's some overhead even if your modifier does not touch keys, but not has high as actually reinserting the element.
BTW, thanks for working on boost::unordered_flat_map because it'll allow us to remove 3rd party dependency from our projects as we are currently using https://greg7mdp.github.io/parallel-hashmap/.
Glad to hear about that! It'd be great if you could download the upcoming beta and provide early feedback before Boost 1.81 ships. Joaquín M López Muñoz
On Thu, Nov 3, 2022 at 10:26 AM Joaquin M López Muñoz < joaquinlopezmunoz@gmail.com> wrote:
Glad to hear about that! It'd be great if you could download the upcoming beta and provide early feedback before Boost 1.81 ships.
As far as I can see the 1.81 beta is planned for November 16th. I'll try to replace then the currently used open addressing hash map with boost::unordered_flat_map in one of our projects which is still in testing anyway. Pavel.
participants (2)
-
Joaquin M López Muñoz
-
Pavel Vazharov