[MultiIndex] Prevent automatic re-indexing after "modify"?
Hi guys, Hi Joaquín, I have a question concerning the automatic re-indexing of Boost.MultiIndex indexes. Is it somehow possible to modify a member of the element-type that is used as key for (another) index without triggering automatic re-indexing? (Of course, I want to be able to trigger manual re-indexing later.) Or is it at least possible to adjust the code so that this can be achieved? To understand my question, here is my use-case and why I need it. I have a boost::multi_index_container with multiple indexes (one hashed_unique and four hashed_non_unique). I am accessing a range of elements through a hashed_non_unique index 'A'. Then I am iterating through these elements and want to modify the value of a member 'M' of each element. However, that member is used when calculating the key for another hashed_non_unique index 'B'. Therefore, I must use the "modify" function of index 'A'. The problem is now that it takes forever to modify the entire range. (We are talking about 100 thousands if not millions of entries that need to be modified.) Probably because re-indexing is triggered after each call to "modify". However, in my case it should be perfectly fine to postpone the re-indexing after I modified the entire range, because I am not going to access any element through index 'B' until I finished modifying the entire range. As a workaround I removed index 'B' from the container and declared the element-member 'M' as mutable. This allows me to directly change the value of 'M'. The speedup is astonishing. It now finishes almost in no time. But this workaround is not really feasible as it requires reimplementing index 'B' by somehow tracking the information outside of the multi_index_container. Therefore, my question really is: Is it possible to provide an "unsafe_modify" function, which does the same as "modify" but does not trigger automatic re-indexing, and an "refresh_index" function, which (manually) triggers the re-indexing? Thanks, Deniz
Therefore, my question really is:
Is it possible to provide an "unsafe_modify" function, which does the same as "modify" but does not trigger automatic re-indexing, and an "refresh_index" function, which (manually) triggers the re-indexing?
I second this idea. Hit the same issue not so long ago. Or event better, something like modify_range(<first>, <last>, <modifier>) that would automatically re-index after modifying entire range thus avoiding the danger of user forgetting to re-index. Best regards, Leon
On Wed, Dec 14, 2016 at 3:00 PM, Leon Mlakar <leon@digiverse.si> wrote:
Therefore, my question really is:
Is it possible to provide an "unsafe_modify" function, which does the same as "modify" but does not trigger automatic re-indexing, and an "refresh_index" function, which (manually) triggers the re-indexing?
I second this idea. Hit the same issue not so long ago. Or event better, something like modify_range(<first>, <last>, <modifier>) that would automatically re-index after modifying entire range thus avoiding the danger of user forgetting to re-index.
If it takes <first>, <last>, it's not a range :) Joking apart, better to accept anything that range-for can accept, and use Boost.Range to adapt a pair of iterators into an actual range. Yes, a modify_range() or bulk_modify() or insert_better_name() would be great. But then you don't know which index needs to be rebuilt, so they must all be rebuilt. So a new API to do this could additionally explicitly spell out which ones perhaps. Although then nothing prevents other keys to be modified, breaking BMI invariants. But I for one would be willing to accept the risk for the performance benefits. --DD PS: The same applies to the current "scalar" modify I guess, which must update all indices too.
On 14.12.2016 15:22, Dominique Devienne wrote:
On Wed, Dec 14, 2016 at 3:00 PM, Leon Mlakar <leon@digiverse.si <mailto:leon@digiverse.si>> wrote:
Therefore, my question really is:
Is it possible to provide an "unsafe_modify" function, which does the same as "modify" but does not trigger automatic re-indexing, and an "refresh_index" function, which (manually) triggers the re-indexing?
I second this idea. Hit the same issue not so long ago. Or event better, something like modify_range(<first>, <last>, <modifier>) that would automatically re-index after modifying entire range thus avoiding the danger of user forgetting to re-index.
If it takes <first>, <last>, it's not a range :) Joking apart, better to accept anything that range-for can accept, and use Boost.Range to adapt a pair of iterators into an actual range. Well, possibly ... I was sort of trying to model on equal_range search, which returns pair of first, last iterators... and since it's likely it would be used on the results of equal_range search, passing them to modify_range (either as pair or as separate parameters) would be fit.
Yes, a modify_range() or bulk_modify() or insert_better_name() would be great. But then you don't know which index needs to be rebuilt, so they must all be rebuilt. So a new API to do this could additionally explicitly spell out which ones perhaps. Although then nothing prevents other keys to be modified, breaking BMI invariants. I'm not sure I understand this - afaik you can already modify other key(s) via index::modify so it has to adjust/rebuild all indices.
Cheers, Leon
Hi Joaquín, I just wanted to bring this topic up again and ask you if you can give me some hints on where I might have to look to possibly modify/extend the behavior of Boost.MultiIndex as described in my original mail below: Am 14.12.2016 um 14:06 schrieb Deniz Bahadir:
Hi guys, Hi Joaquín,
I have a question concerning the automatic re-indexing of Boost.MultiIndex indexes.
Is it somehow possible to modify a member of the element-type that is used as key for (another) index without triggering automatic re-indexing? (Of course, I want to be able to trigger manual re-indexing later.) Or is it at least possible to adjust the code so that this can be achieved?
To understand my question, here is my use-case and why I need it.
I have a boost::multi_index_container with multiple indexes (one hashed_unique and four hashed_non_unique). I am accessing a range of elements through a hashed_non_unique index 'A'. Then I am iterating through these elements and want to modify the value of a member 'M' of each element. However, that member is used when calculating the key for another hashed_non_unique index 'B'. Therefore, I must use the "modify" function of index 'A'.
The problem is now that it takes forever to modify the entire range. (We are talking about 100 thousands if not millions of entries that need to be modified.) Probably because re-indexing is triggered after each call to "modify". However, in my case it should be perfectly fine to postpone the re-indexing after I modified the entire range, because I am not going to access any element through index 'B' until I finished modifying the entire range.
As a workaround I removed index 'B' from the container and declared the element-member 'M' as mutable. This allows me to directly change the value of 'M'. The speedup is astonishing. It now finishes almost in no time.
But this workaround is not really feasible as it requires reimplementing index 'B' by somehow tracking the information outside of the multi_index_container.
Therefore, my question really is:
Is it possible to provide an "unsafe_modify" function, which does the same as "modify" but does not trigger automatic re-indexing, and an "refresh_index" function, which (manually) triggers the re-indexing?
Thanks, Deniz
Thanks again, Deniz -- BENOCS GmbH Dipl.-Inform. Deniz Bahadir Winterfeldtstr. 21 10781 Berlin Germany Phone: +49 - 30 / 577 0004-22 Email: deniz.bahadir@benocs.com www.benocs.com Advisory Board (Chairman): Stephan Schröder Board of Management: Dr.-Ing. Oliver Holschke, Dr.-Ing. Ingmar Poese Commercial Register: Amtsgericht Bonn HRB 19378
participants (2)
-
Deniz Bahadir -
Dominique Devienne -
Leon Mlakar