pt., 1 mar 2019 o 01:17 Peter Dimov via Boost napisał(a):

Andrzej Krzemienski wrote:

...

Hi Peter, Thank you for writing and sharing this implementation of variant.

I am sorry if I am bringing back an issue that has been already discussed and solved before, but I do not understand the rationale behind doing compromises in order to provide the never-empty guarantee.

The short rationale provided in Boost.Variant library:

https://www.boost.org/doc/libs/1_69_0/doc/html/variant/design.html#variant.d...

...

was provided before we got C++11 the moved-from state in the language. Today, it should come as no surprise to programmers that a type might be in a "moved from" state where only a limited interface of an object can be used.

C++11 introduces no such "moved-from" state. After a move, objects are not in a special state. The desire to have variant contain one of X, Y, or Z, is as valid today as it was when Boost.Variant was developed.

...

The guarantee provided by variant's assignment is not a strong exception safety guarantee: it is possible that my variant has value A, I want to assign value B, and (due to an exception) I end up with value C.

That's correct. Strong guarantee requires double buffering when two or more of the alternatives have a throwing move constructor. This would affect significantly more of the real-world cases than the current scheme, which doubles only occasionally.

...

If this happens, the only thing I can reasonably do is to either abandon whatever I was doing or reset the variant to the state that I need. So the guarantee that it is not left empty does not seem to be of much use. But the cost to be paid is noticeable.

Well, if you want a variant without this guarantee, there are plenty to be found. Those who don't, however, have no options at present.

So, does the following recommendation correctly capture the design goals for boost::variant2? If you require the never-empty guarantee (and accept the costs) use boost::variant2. If you do not require the never empty guarantee use std::variant. (For the sake of the discussion, I do not consider the additional conversions offered by variant2). Also, I am not entirely satisfied with the reply, "those who want this guarantee". Could you, or anyone else, give me a real-world use case where a never-empty guarantee is needed, but a strong exception guarantee is not? It might be just my lack of imagination, so I would appreciate any help understanding this. Regards, Andrzej

pt., 1 mar 2019 o 10:59 Peter Dimov via Boost napisał(a):

Andrzej Krzemienski wrote:

...

So, does the following recommendation correctly capture the design goals for boost::variant2?

If you require the never-empty guarantee (and accept the costs) use boost::variant2.

If you do not require the never empty guarantee use std::variant.

Kind of, but as written this implies that std::variant has no costs, which is not true. The checks for valueless do carry a cost. Each visit(), for example, starts with `if(valueless) throw`, which is not necessary in variant2.

...

Also, I am not entirely satisfied with the reply, "those who want this guarantee". Could you, or anyone else, give me a real-world use case where a never-empty guarantee is needed, but a strong exception guarantee is not?

My reply was unsatisfactory because I was really not looking forward to rehashing the arguments against singular states. Singular state is bad when a result of two-phase construction (which is why we no longer use two-phase construction), it's bad when a result of exception (which is why we don't use destroy-only exception safety but basic exception safety), it's bad when a result of default initialization of built-in types (but we can do nothing about it), and it's bad when a result of a move (which is why move semantics, as originally specified, do not put the moved-from object in a singular state.)

Singular states introduce implicit "is_valid" preconditions on all your normal functions, and partition the program into two worlds, a normal world where no object is singular, and an "exceptional" world where objects may be singular. It's _possible_ to program in this way, but it's not fun, because world #2 may never call into world #1 under penalty of undefined behavior, and singular objects are never to enter world #1, because this sets up a delayed explosion.

If you avoid singular states, this removes all these implicit "is_valid" preconditions, which removes the partitioning and collapses the two worlds back into one; "type 2" code can call "type 1" code and nothing undefined will happen.

Now in principle, for the specific case of move, it's possible and sound to specify it to leave the object in a singular state, provided that you only ever move from objects that are about to be immediately destroyed. But that's not the approach that was taken. In this timeline, move does not leave objects in a singular state, so there is no requirement to only ever use it on objects that are about to be destroyed.

For variant specifically, the guarantee that variant can only ever either hold an X, or hold a Y, simplifies the specification of all code taking variant, because it's not required to document what happens in the event of the variant not holding X or Y.

"Never empty" is somewhat a misnomer, because variant2 can be empty, you just have to request it explicitly: variant. Of course then you have to explicitly handle the possibility of the variant being empty. If the variant is an implementation detail of some component of yours that has behavior X' when in state X and behavior Y' when in state Y, you would need to decide what happens when the variant is empty. Do you emulate X' or Y', or does the component behave in a third way, E'? Up to you, but undefined behavior is probably not acceptable, unless you introduce a singular state for your component.

What are the alternatives? One is to do what std::variant does and try to have the cake and eat it too. Have the empty state, but don't acknowledge it in the interface as equal to other states, throw an exception instead. This is a bit like sweeping the problem under the carpet, it allows people to pretend that the variant delivers the "never empty" guarantee and program as if it did, whereas it doesn't.

Thanks for a very long reply. I am sorry if my questions look like trying to repeat the same discussion again. In fact my goal is only to *understand* your rationale behind your design choices; not to argue with them. In your reply you mostly address the issue of a "singular state" in general, but I was hoping for an answer specific to variant. I think the two are different. A general "singular state" is dangerous (I whole-heartedly agree with you here) because users can create this state by using a default constructor or moving from the object. But this is not the case for variant: both default-constructed state and the moved-from state is not "valueless", even in std::variant. The only way to get to a valueless state is to trigger an exception from a move constructor. And it seems to me that when this happens, the only reasonable choice for the user is to either reset or destroy the variant. Therefore the practical implication of having such "variant-speciffic singular state" seems to me not noticeable. If you were serious about your convictions you would make accessing a

valueless variant undefined behavior, which avoids the valueless checks but see above about singular states and time bombs.

Yes: it makes a lot of sense to me to make an attempt to access the value of a valueless variant an undefined behavior. I do not associate this decision with the problems of types with singular states in general, because there is no easy way to obtain the valueless state in variant; and it seems to me that if we get an exception that puts the variant into this state, and the user makes an effort to intercept the exception and do things with the variant, the user must surely be doing something wrong. So, let me restate my question in more precise terms. Is there a real-world use case for a variant (as opposed to any other UDT), given that default-constructed and moved-from variant is not valueless, where it makes the difference if the variant is guaranteed not to be valueless but does not provide a strong exception safety guarantee? Regards, Andrzej

pt., 1 mar 2019 o 11:57 Peter Dimov via Boost napisał(a):

Andrzej Krzemienski wrote:

...

But this is not the case for variant: both default-constructed state and the moved-from state is not "valueless", even in std::variant. The only way to get to a valueless state is to trigger an exception from a move constructor.

This is not true, by the way (although it should have been). std::variant also goes valueless on emplace when the construction throws. I have argued against this behavior in https://wg21.link/p0308r0, which also contains other interesting things. :-)

Thanks for the correction. Although it does not change the reasoning in this thread: valueless state can only be observed during stack unwinding (or when the unwinding was stopped too soon).

(It was even less true before P0308 was partially adopted.)

...

And it seems to me that when this happens, the only reasonable choice for the user is to either reset or destroy the variant.

I can only repeat what I said about singular states. Stack unwinding in this case is an example of "world #2" code, which can observe a valueless variant. This means that it must be careful to never ever call an ordinary "world #1" function on a variant object.

This is technically true. I argue that an attempt to observe the states of local objects, during stack unwinding is something wrong on itself: you do not want to do this. Therefore being careful not to call ordinary "world #1" function should come automatically from following other good practices: do not fiddle with local objects during stack unwinding: just let them be destroyed.

There's really nothing variant-specific here; the same argument applies to any function having the basic exception safety guarantee. When you assign to a vector, and the assignment throws, you can only reasonably reset or destroy the vector. But we don't make it destroy-only.

Yes, when you use vec.assign(b, e); you may end up in an unintended state. In that case you can only reasonably reset or destroy the vector.If you try to count its elements, there is no UB, but at the same time you are doing something wrong. What do you want to do with such count? In case of a vector, in this state you can do more things with it. But this is a guarantee that adds no value to anyone. This is at least what I am trying to investigate with this thread. And also note that if I wanted to get from state A to state B and due to an exception I ended up in a valid but unspecified state, I cannot call just any operations that were valid on A and B: std::vector<X> v(10), w(10); v[9]; // valid w[9]; // valid try{ v.assign(w.begin(), w.end()); } catch(...) { v[9]; // maybe UB }

And if you do have a type that is destroy-only, it breaks basic exception safety everywhere it's used. If you assign vector<variant> and it throws, the vector is now destroy-only.

No: it is normal that in "valid but unspecified state" you can do less than other states: E.g., I can call `*p = 0` when I know that pointer `p` points to an object within its lifetime, but I cannot call this operation safely when a `p` is in a "valid but unspecified state".

...

Yes: it makes a lot of sense to me to make an attempt to access the value of a valueless variant an undefined behavior. I do not associate this decision with the problems of types with singular states in general, because there is no easy way to obtain the valueless state in variant;

It doesn't have to be easy to be a problem. In fact, easy is better, because it happens more frequently and is therefore tested.

I would still want to hear from you or other people if they ever use a variant (or in fact any other object with assignment that offers basic exception safety guarantee) in this way: An exception is thrown, but I stop the stack unwinding so that the object is still in scope, and I do (intentionally) anything else than resetting it. Does anyone do it? Regards, Andrzej

On Fri, 1 Mar 2019 at 09:59, Peter Dimov via Boost wrote:

Kind of, but as written this implies that std::variant has no costs, which is not true. The checks for valueless do carry a cost. Each visit(), for example, starts with `if(valueless) throw`, which is not necessary in variant2.

That is not true. A typical implementation would just add an extra value to the switch, there is no extra branch.

On Sat, Mar 2, 2019 at 3:49 PM Steven Watanabe via Boost wrote:

AMDG

On 3/2/19 12:47 PM, Mathias Gaunard via Boost wrote:

...

On Fri, 1 Mar 2019 at 09:59, Peter Dimov via Boost wrote:

...

Kind of, but as written this implies that std::variant has no costs, which is not true. The checks for valueless do carry a cost. Each visit(), for example, starts with `if(valueless) throw`, which is not necessary in variant2.

That is not true. A typical implementation would just add an extra value to the switch, there is no extra branch.

That may be true in theory, but both libc++ and libstdc++ have if(__v.valueless_by_exception()) scattered everywhere. Not to mention that neither uses switch.

In Christ, Steven Watanabe

The code might say that, but the compiler probably still turns it into a switch. The variant only has one internal variable, such as "_which". It holds both the "is it A or B or C" and "or is it valueless". So the resulting code should only need to look at it once (ie something switch-like). If compilers and std::variant haven't yet eliminated the cost of valueless_by_exception on visitation, I think it is only a matter of time before they do. Tony

...

I would disagree with this assessment of the strong never empty guarantee provided by variant2. [snip] Or do you disagree with my judgement that the never empty guarantee is not of much use?

This one.

I agree that this is how you could observe the valueless state. But I also claim that such code (I wait to be convinced otherwise) has already more serious problems than observing the valueless state on variant. You typically do not want your objects to outlive the stack unwinding. And if you do, for globals, you want to provide a transactional-like guarantee. Leaving such objects in "valid but unspecified states" is a design bug.

My issue with std::variant is that it needlessly does not conform to the usual assignment and emplacement guarantees of other standard library types. For example, resizing a std::vector implements the strong guarantee that state will be restored to before the resize if an exception is thrown in the middle of the resize. Boost.Optional implements the strong guarantee for assignment, but not for emplacement (which I think it should, but fair enough that emplacement generally has the basic guarantee. You might note that Outcome deliberately omits an emplacement modifier). See https://www.boost.org/doc/libs/1_69_0/libs/optional/doc/html/boost_optional/... Thus I would hold that, unless there is a *very* good reason why not, so should std::variant propagate the strong guarantee where it is able to do so. To my knowledge, there is no good reason that is does not, as proven by Peter's variant2.

Or am I wrong?

I suppose it depends on whether you consider variant a sort of container with responsibilities or not. I would say it is.

My position (until I see examples that will convince me otherwise) is that if a programmer observes the valueless state in a variant, then the programmer is doing something wrong with handling exceptions.

My issue is that it is a point of unintended failure that need not exist. The committee decided that exception throws during assignment or emplacement ought to create a trap state which renders the variant always throwing exceptions on use thereafter. I can see the logic, but it is wrong in my opinion. The variant should be put back into the state it was in beforehand, in my opinion. (Personally speaking, I find the double buffering a step too far. I remember debating this with Anthony Williams a few years ago at ACCU. I think that if double buffering is necessary, then you weaken your guarantees to basic, and you provide a constexpr bool for static asserting when the guarantees are basic or strong. In any case, I find the valueless by exception state to be an abomination, it should never have been allowed, it litters the code with potential throw paths none of which aid codegen)

You are contrasting boost::variant2 with std::variant, but the design space I see is more than just either of them. If std::variant is wrong (which I tend to agree with) it does not immediately imply that boost::variant2is right. Another alternative that Peter indirectly suggested is that it is UB if you try to observe the valuelsess state in std::variant. In this case some usages after a throw are banned, but the model still guarantees the never emptiness.

You're right that it doesn't mean variant2 is right. And I do have some issues with it as it is currently, which I have already covered in enough detail here. I feel far more strongly about propagation of triviality than I do about double buffering. I only have a weakly held disagreement with double buffering. It stems mainly from my belief that the compile time extra cost is not worth it for supporting pathological types (i.e. ones without noexcept move constructors). But that's a belief, not a fact, I have no empirical evidence to prove my belief. Niall

pt., 1 mar 2019 o 17:41 Niall Douglas via Boost napisał(a):

...

...

I would disagree with this assessment of the strong never empty guarantee provided by variant2. [snip] Or do you disagree with my judgement that the never empty guarantee is not of much use?

This one.

...

I agree that this is how you could observe the valueless state. But I also claim that such code (I wait to be convinced otherwise) has already more serious problems than observing the valueless state on variant. You typically do not want your objects to outlive the stack unwinding. And if you do, for globals, you want to provide a transactional-like guarantee. Leaving such objects in "valid but unspecified states" is a design bug.

My issue with std::variant is that it needlessly does not conform to the usual assignment and emplacement guarantees of other standard library types.

For example, resizing a std::vector implements the strong guarantee that state will be restored to before the resize if an exception is thrown in the middle of the resize.

Boost.Optional implements the strong guarantee for assignment, but not for emplacement (which I think it should, but fair enough that emplacement generally has the basic guarantee. You might note that Outcome deliberately omits an emplacement modifier). See

https://www.boost.org/doc/libs/1_69_0/libs/optional/doc/html/boost_optional/...

Thus I would hold that, unless there is a *very* good reason why not, so should std::variant propagate the strong guarantee where it is able to do so. To my knowledge, there is no good reason that is does not, as proven by Peter's variant2.

Hold on. Most standard library types *do not* provide strong exception safety guarantee on assignment. It is only the subset of STL containers that are implemented as pointers, such as std::vector or std::list. But consider std::array, which has to store its elements directly. It has no way of providing strong guarantee on assignment. Or consider std::pair: pair p1 {"Niall", "Douglas"}; pair p2 {"Peter", "Dimov"}; try { p1 = p2; } catch(...) {} If the assignment of p2.second() throws, you end up with unintended person {"Peter", "Douglas"}. The same with nearly every aggregate you might use in the program: struct Person { string firstName, lastName; }; The generated assignment only provides *basic* exception safety guarantee. Most of the types we use only provide basic exception safety guarantee in assignment. But it is usually not a problem, because in a program that correctly handles exceptions all objects that cause exceptions are immediately destroyed in stack unwinding. variant2 also only provides *basic* exception safety guarantee: you can be assigning a variant containing type B to a variant containing type A and end up with variant containing type C. Here's an example: https://wandbox.org/permlink/AObFiUKgeXIEiXQa My point is, you can only observe the valueless state in variant if you work with objects that threw from a basic-guarantee operation. But if you do this, you already have mess in your program regardless if you are using variants or not. Nobody so far has shown an example when they are using (other than resetting or destroying) objects in a "valid but unspecified state" in a correct program. Regards, Andrzej

...

Or am I wrong?

I suppose it depends on whether you consider variant a sort of container with responsibilities or not. I would say it is.

...

My position (until I see examples that will convince me otherwise) is that if a programmer observes the valueless state in a variant, then the programmer is doing something wrong with handling exceptions.

My issue is that it is a point of unintended failure that need not exist.

The committee decided that exception throws during assignment or emplacement ought to create a trap state which renders the variant always throwing exceptions on use thereafter.

I can see the logic, but it is wrong in my opinion. The variant should be put back into the state it was in beforehand, in my opinion.

(Personally speaking, I find the double buffering a step too far. I remember debating this with Anthony Williams a few years ago at ACCU. I think that if double buffering is necessary, then you weaken your guarantees to basic, and you provide a constexpr bool for static asserting when the guarantees are basic or strong. In any case, I find the valueless by exception state to be an abomination, it should never have been allowed, it litters the code with potential throw paths none of which aid codegen)

...

You are contrasting boost::variant2 with std::variant, but the design space I see is more than just either of them. If std::variant is wrong (which I tend to agree with) it does not immediately imply that boost::variant2is right. Another alternative that Peter indirectly suggested is that it is UB if you try to observe the valuelsess state in std::variant. In this case some usages after a throw are banned, but the model still guarantees the never emptiness.

You're right that it doesn't mean variant2 is right. And I do have some issues with it as it is currently, which I have already covered in enough detail here.

I feel far more strongly about propagation of triviality than I do about double buffering. I only have a weakly held disagreement with double buffering. It stems mainly from my belief that the compile time extra cost is not worth it for supporting pathological types (i.e. ones without noexcept move constructors). But that's a belief, not a fact, I have no empirical evidence to prove my belief.

Niall

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

sob., 2 mar 2019 o 01:03 Emil Dotchevski via Boost napisał(a):

On Fri, Mar 1, 2019 at 2:18 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

variant2 also only provides *basic* exception safety guarantee: you can be assigning a variant containing type B to a variant containing type A and end up with variant containing type C. Here's an example: https://wandbox.org/permlink/AObFiUKgeXIEiXQa

My point is, you can only observe the valueless state in variant if you work with objects that threw from a basic-guarantee operation. But if you do this, you already have mess in your program regardless if you are using variants or not.

This is true only if we define "mess" to mean "an unspecified but valid state".

I meant "mess" in the sense of stopping stack unwinding in arbitrary, not well planned, places, not paying attention which objects go out of scope and which are preserved. I would appreciate someone giving me example of where: 1. stack unwinding is stopped in places that leave objects in a "valid but unspecified state" and 2. this state is actually read instead of being destroyed (or overwritten) and 3. this makes sense (it is not immediately classified as programmer doing something wrong) Regards, Andrzej

sob., 2 mar 2019 o 06:04 Emil Dotchevski via Boost napisał(a):

On Fri, Mar 1, 2019 at 8:55 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2. this state is actually read instead of being destroyed (or overwritten)

Why does it have to be read instead of destroyed?

Because it is only when you try to read the state of such object, being observably valueless or not makes any difference. The reason why

valid-but-unspecified is useful is not because reading it is somehow desirable, but because it could happen. It's better than seeing pink elephants.

My hypothesis is that reading valid-but-unspecified can only happen in a buggy program in an unintended path. And that making design compromises to address this path is not necessarily the best approach to take. Regards, Andrzej

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

sob., 2 mar 2019 o 07:35 Emil Dotchevski via Boost napisał(a):

On Fri, Mar 1, 2019 at 9:37 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

My hypothesis is that reading valid-but-unspecified can only happen in a buggy program in an unintended path.

Running out of memory, or out of some other resource, does not indicate a bug. In response, under the basic exception guarantee, you may get a state which I'm saying shouldn't be merely "destructable" but also valid. For example, if this was a vector<T>, it shouldn't explode if you call .size(), or if you iterate over whatever elements it ended up with.

This is where my imagination fails me. I cannot imagine why upon bad_alloc I would be stopping the stack unwinding and determining size of my vectors. This is why I ask about others' experience with real-world correct code.

...

And that making design compromises to address this path is not necessarily the best approach to take.

Consider that if you choose to allow, after an error, to have objects left in such a state that they may explode if you attempt to do anything but destroy them, there may not be any way to detect that state.

Yes. and I do not see how this is a problem in practice. In my experience objects that failed on operation with basic guarantee can only be safely removed from the scope. (I do not even reset them.) Assuming you

don't want your program to crash, your only choice may be to support this as a "valid" state anyway, if not in the object itself, then through some external indicator.

I achieve no-UB guarantee by removing objects in "valid but unspecified state" from scope. I agree: to be able to safely destroy an object you need some sort of an indicator. A variant may hold it internally and never expose it. If this is relevant for the sake of the purity of the design, it can be exposed with member `.valueless_by_exception()` (but it should be UB if I try to visit such variant). For the sake of the purity of the design we can call it a normal variant state, as any other state. The fact that you cannot create this state other than by throwing from operator= and .emplace() makes a real practical difference.

It's just better if all elephants you encounter are grey, even if they show up where you don't expect them. :)

I sort of understand this point of view. But to me it sounds like an aesthetic goal rather than a practical or technical one. No real world example of using "valid but unspecified state" has been shown in this thread. The arguments put forth use phrases "it is better", "I prefer", "it is simpler". Normally we do not need to distinguish between aesthetic an technical goals because in most of the cases they both can be achieved without compromises. But in case of variant there is a tension/conflict between aesthetic and technical goals. Maybe this is the root of controversy. Regards, Andrzej

sob., 2 mar 2019 o 10:36 Andrey Semashev via Boost napisał(a):

On 3/2/19 11:56 AM, Andrzej Krzemienski via Boost wrote:

...

sob., 2 mar 2019 o 07:35 Emil Dotchevski via Boost < boost@lists.boost.org> napisał(a):

...

On Fri, Mar 1, 2019 at 9:37 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

My hypothesis is that reading valid-but-unspecified can only happen in a buggy program in an unintended path.

Running out of memory, or out of some other resource, does not indicate a bug. In response, under the basic exception guarantee, you may get a state which I'm saying shouldn't be merely "destructable" but also valid. For example, if this was a vector<T>, it shouldn't explode if you call .size(), or if you iterate over whatever elements it ended up with.

This is where my imagination fails me. I cannot imagine why upon bad_alloc I would be stopping the stack unwinding and determining size of my vectors. This is why I ask about others' experience with real-world correct code.

That is not an unimaginable scenario. If you have two branches of code, one requiring more memory but better performance, and the other that is slower (or maybe lacking some other qualities but still acceptable) and less resource consuming, operating on the same vector, you will want the vector to stay valid if memory allocation fails. Although not specifically with vectors, I had cases like this in real world.

Thanks for sharing your experience. I am not sure we are on the same page here. Are you describing an operation where an operation on vector fails to allocate more storage and therefore throws and leaves the *value* of the vector unchanged (i.e., an operation with strong exception safety guarantee)? Or are you describing an operation that reuses memory owned by a vector and discards the vector's value? Boith these cases can be described as "never observing the value after a failed operation with a basic exception safety guarantee". Or are you describing a different situation? Regards, Andrzej

However, in my experience, if I want to handle OOM condition gracefully, I tend to not trust any third party components except the lowest level ones, like C runtime, and write the relevant code myself. Especially, this concerns components that allocate memory, like containers. Unfortunately, it is often the case that either I don't trust implementations to take OOM into account and handle it well or I want some specific guarantees about how much memory is allocated and what the state of the program is when OOM happens.

...

...

...

And that making design compromises to address this path is not necessarily the best approach to take.

Consider that if you choose to allow, after an error, to have objects left in such a state that they may explode if you attempt to do anything but destroy them, there may not be any way to detect that state.

Yes. and I do not see how this is a problem in practice. In my experience objects that failed on operation with basic guarantee can only be safely removed from the scope. (I do not even reset them.)

Removing the objects may be wasteful or require expensive operations. In the vector example, that vector may be initially large or expensive or even impossible to reconstruct. If you strive for the "destroy upon failure" logic, you would have to duplicate the vector before attempting the operation that may fail with an exception. Which is a point of failure on its own, BTW. Generally, you want to minimize the number of points of failure while also minimizing amount of work needed to be done to complete the program. There is also a third subjective limit of code quality or simplicity, design quality, etc., but that is not relevant to my point.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 3/2/19 2:24 PM, Andrey Semashev wrote:

On 3/2/19 1:44 PM, Andrzej Krzemienski via Boost wrote:

...

sob., 2 mar 2019 o 10:36 Andrey Semashev via Boost napisał(a):

...

On 3/2/19 11:56 AM, Andrzej Krzemienski via Boost wrote:

...

This is where my imagination fails me. I cannot imagine why upon

bad_alloc

...

I would be stopping the stack unwinding and determining size of my vectors. This is why I ask about others' experience with real-world correct code.

That is not an unimaginable scenario. If you have two branches of code, one requiring more memory but better performance, and the other that is slower (or maybe lacking some other qualities but still acceptable) and less resource consuming, operating on the same vector, you will want the vector to stay valid if memory allocation fails. Although not specifically with vectors, I had cases like this in real world.

Thanks for sharing your experience. I am not sure we are on the same page here. Are you describing an operation where an operation on vector fails to allocate more storage and therefore throws and leaves the *value* of the vector unchanged (i.e., an operation with strong exception safety guarantee)? Or are you describing an operation that reuses memory owned by a vector and discards the vector's value? Boith these cases can be described as "never observing the value after a failed operation with a basic exception safety guarantee". Or are you describing a different situation?

As I mentioned, I usually don't rely on standard containers in situations when I want complete control over memory allocations, so vector is not the component I was talking about. However, the program logic is often closer to the strong guarantee because I would normally try allocating memory early, before the real work is done. Though, in some instances some data is modified in non-harmful way in case of OOM.

There are a few instances when I want to "undo" the failed operation, in which case the code relies on a particular "half-way through" state of the program. The rollback process requires that there is a known number of steps that were completed before failure, and the data associated with those steps is in a valid state. From the failed operation standpoint, this is a "strict" basic guarantee, from the underlying components, like containers, this is almost certainly a strong guarantee.

I call "strict" basic guarantee, as opposed to just basic guarantee, a case, where the program is allowed to have a *subset* of valid states upon failure. Maybe there is a better term for this. I mean, for a hypothetical initially non-empty vector, a basic guarantee push_back() is allowed to leave the vector empty upon failure, but that might not be an acceptable outcome for a higher level user's program. In my case, I require a particular state of the program upon my operation failure. This state may not be the same as before the operation started, but it must at least include results of the steps the failed operation has managed to complete. This allows these steps to be rolled back.

I'll add that I could probably agree with you that observing the result of a failed "purely" basic guarantee operation is not useful. You basically have no guarantees about the state of the program, except that the objects can be destroyed or re-initialized from scratch. However, "strict" basic guarantee is more prevalent in practice, even though it is often not documented so, and observing result of failed such operation can be useful, provided that the restrictions on the operation failure outcome match your requirements.

Andrzej Krzemienski wrote:

This is where my imagination fails me. I cannot imagine why upon bad_alloc I would be stopping the stack unwinding and determining size of my vectors.

Why would you need to be stopping stack unwinding in order to access a variable? Variables can also be accessed during stack unwinding.

On Sat, Mar 2, 2019 at 12:12 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

sob., 2 mar 2019 o 07:35 Emil Dotchevski via Boost

...

It's just better if all elephants you encounter are grey, even if they

show

...

up where you don't expect them. :)

I sort of understand this point of view. But to me it sounds like an aesthetic goal rather than a practical or technical one.

How is it an aesthetic goal when I say that it's best if the objects don't get in a funny state in which they explode except if you destroy them? Keep in mind, we're talking about error-handling code which is always difficult to test.

AMDG On 3/2/19 11:25 AM, Niall Douglas via Boost wrote:

...

...

Thus I would hold that, unless there is a *very* good reason why not, so should std::variant propagate the strong guarantee where it is able to do so. To my knowledge, there is no good reason that is does not, as proven by Peter's variant2.

Hold on. Most standard library types *do not* provide strong exception safety guarantee on assignment. It is only the subset of STL containers that are implemented as pointers, such as std::vector or std::list.

You see, I would consider any union-based storage as in the same category. By definition union-based storage must contain a "pointer" to the correct way to interpret that union-based storage.

I don't understand this. Andrzej says pointers and means it literally: T*. You say "pointers" and I have no idea what you mean by it and what relation it has to exception safety.

...

<snip> variant2 also only provides *basic* exception safety guarantee: you can be assigning a variant containing type B to a variant containing type A and end up with variant containing type C. Here's an example: https://wandbox.org/permlink/AObFiUKgeXIEiXQa

I just don't get why variant2 would set a state of C when it had state A, and setting state B failed. It should have spotted the lack of common noexcept move, employed double buffers, and alternated between them such that state A is untouched should setting state B fail.

If it's going to do weirdness like setting state C out of the blue, then better dispose entirely any double buffered implementation as not adding value.

variant2 won't use double buffering in this example. In Christ, Steven Watanabe

AMDG On 3/2/19 3:31 PM, Niall Douglas via Boost wrote:

...

...

You see, I would consider any union-based storage as in the same category. By definition union-based storage must contain a "pointer" to the correct way to interpret that union-based storage.

I don't understand this. Andrzej says pointers and means it literally: T*. You say "pointers" and I have no idea what you mean by it and what relation it has to exception safety.

If an object *selects* another object, then in my opinion it needs to propagate the strongest possible exception guarantees it can for assignment, swap and emplace.

If an object *aggregates* other objects, then in my opinion it is permitted to have partial operations. So swap can get half way into swapping the individual members in the aggregate, and bail out.

This might help:

T*: Selects a T[1...N] unique_ptr: Selects a T[x] vector<T>: Selects a T[0...N] optional<T>: Selects a T[0], or a T[1] variant: Selects one of a A[1], B[1], or C[1], with all others [0]

---

struct: Aggregate of heterogeneous types T[x]: Aggregate of x homogeneous T's array: Aggregate of x homogeneous T's

Can you see the difference yet? The first group selects an aggregate *elsewhere*. The second grouo IS an aggregate.

Nope. The second group makes sense to me. The first is just a grab bag of everything that isn't the second. In Christ, Steven Watanabe

niedz., 3 mar 2019 o 22:10 Andrzej Krzemienski napisał(a):

sob., 2 mar 2019 o 23:32 Niall Douglas via Boost napisał(a):

...

...

...

You see, I would consider any union-based storage as in the same category. By definition union-based storage must contain a "pointer" to the correct way to interpret that union-based storage.

I don't understand this. Andrzej says pointers and means it literally: T*. You say "pointers" and I have no idea what you mean by it and what relation it has to exception safety.

If an object *selects* another object, then in my opinion it needs to propagate the strongest possible exception guarantees it can for assignment, swap and emplace.

If an object *aggregates* other objects, then in my opinion it is permitted to have partial operations. So swap can get half way into swapping the individual members in the aggregate, and bail out.

This might help:

T*: Selects a T[1...N] unique_ptr: Selects a T[x] vector<T>: Selects a T[0...N] optional<T>: Selects a T[0], or a T[1] variant: Selects one of a A[1], B[1], or C[1], with all others [0]

---

struct: Aggregate of heterogeneous types T[x]: Aggregate of x homogeneous T's array: Aggregate of x homogeneous T's

Can you see the difference yet? The first group selects an aggregate *elsewhere*. The second grouo IS an aggregate.

Sorry, I cannot see how you are making this distinction. I can see a different distinction though:

T* unique_ptr vector<T>

The above types use indirection through a pointer, so providing strong assignment is cheap and quite trivial, and this is why it is provided: because it is cheap to implement: no other reason.

aggregates -- no indirection, members stored directly inside the aggregate; implementing strong assignment is expensive (double buffering), therefore we only get basic.

optional and variant also do not use indirection: they store their values in their storage, therefore providing strong assignment can be difficult.

So the driver for deciding on strong versus basic guarantee is not how we want to think about these types but a practical factor: how easy it is to implement. In case of optional<T> strong assignment is implementable "by luck": because it has the "monostate" as part of the contract. But even optional<T> has its problems: its swap() is potentially throwing if T potentially throws from move constructor or assignment, even if T's swap() is noexcept.

Correction: optional<T>::swap() 's noexcept depends on T's swap()'s and move constructor's noexcept. It does not depend on T's move assignment's noexcept.

Regards, Andrzej

On 3/03/2019 07:25, Niall Douglas wrote:

I just don't get why variant2 would set a state of C when it had state A, and setting state B failed. It should have spotted the lack of common noexcept move, employed double buffers, and alternated between them such that state A is untouched should setting state B fail.

If it's going to do weirdness like setting state C out of the blue, then better dispose entirely any double buffered implementation as not adding value.

While I agree that switching to an apparently unrelated state is surprising, it could be anticipated and "solved" by always using monostate as the first type where the types might throw on move. (And perhaps variant could issue a warning or error if you try to do otherwise?) It seems reasonable to require a monostate state to exist unless using types that can guarantee that it is never needed. And with it "in their face" in the type declaration, consumers of the variant should be less likely to forget about handling it, which is one of the problems with valueless_by_exception. (On a peripherally related note, why did variant introduce monostate instead of reusing nullopt_t?) I find it a lot more common that people writing move constructors/assignment forget to declare it noexcept than it actually being exception-prone, so having more diagnostics ("if you really meant to do that, add monostate") rather than silently degrading performance seems like a good thing.

On Mon, Mar 4, 2019 at 3:07 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

nullopt_t was a compromise: the Committee didn't feel comfortable with introducing a generic boost::none_t with this name with well defined semantics (comparability, ordering, interaciotns with nullptr_t) that soon in the process, so we provided something that was supposed to be a temporary solution, a tag (like std::piecewise_construct) whose only purpose is to indicate an intention to initialize std::optional to a state of not containing a value. Now std::monostate has taken the role of boost::none_t. It would make sense to use it in std::optional:

std::optional<int> oi = std::monostate{};

Which breaks this case: std::optional> ov = std::monostate{}; Having to spend any brain power at all to figure out the corner cases makes it not worth having a universal "none" type. You aren't just burdening library designers; you are burdening the users too. -- Nevin ":-)" Liber > +1-847-691-1404

...

If it's going to do weirdness like setting state C out of the blue, then better dispose entirely any double buffered implementation as not adding value.

While I agree that switching to an apparently unrelated state is surprising, it could be anticipated and "solved" by always using monostate as the first type where the types might throw on move.  (And perhaps variant could issue a warning or error if you try to do otherwise?)

I'd much prefer that the code not compile instead of unrelated states magically appearing in corner case situations. Let the user add in monostate to the list of state options if they'd like to explicitly opt into monostate, and only monostate, magically appearing if there is no alternative in the single buffer variant implementation only. Niall

At the risk of inserting something totally irrelevant, I'd offer my own experience with variant types. In my case this comes up as the result of an integer operation which might produce an integer or some error condition such as positive_overflow. I've characterized this as "extended integer type" as opposed to "integer" which has infinite range. These types can take on a value from 0 to some N or one value of and enum. Arithmetic operations are defined so that if i and j are both extended integer types, i + j is defined for all types. For example if i & j both hold values "positive_overflow" then i + j would result a value of "positive_overflow". (For those who are interested - the set of values representable by this type and the addition operator constitute a non-associative, commutative group. So the concept is well defined.) Also, of late, I'm been influenced by the discussions about "monad" and it's value in provably correct programs generally. Of course "variant" comes to mind immediately when a C++ programmer starts thinking about this. I looked into it and was dissatisfied with the complexity and ambiguity of the current offerings. I started to think about writing my own. This is always a red flag that something's very wrong somewhere. I eventually concluded that the source of my difficulties was the assignment operation. This of course is derived from the idea that a C++ type should be in general mutable. But the idea that an instance of s set "extended integer" can change to a different member of the same set is mathematically weird. This is why assignability conflicts on fundamental level with the ideas behind functional programming - an idea which has influenced me. So I implemented "extended integer" in the simplest way possible - and deleting the assignment operator. Seems to me that this resolved all my issues. In the few occasions I've been inclined to use assignment, I've found that code was either flawed or could easily be refactored to avoid the need for assignment. I didn't need any "library" type solution as my the implentation of my ad hoc solution is/was trivially obvious. For those interested you can see in the safe numerics library as the type "checked_integer". C++ starts from the idea that every instance should be mutable and hence gives you an assignment operator by default. The design discussions around variant (expected, outcome, ...) presume that this operation must be implemented. I question this. Not only for variant, but for many other C++ data types. Robert Ramey

On 3/1/19 9:13 AM, Ivan Matek via Boost wrote:

On Fri, Mar 1, 2019 at 5:50 PM Robert Ramey via Boost wrote:

...

C++ starts from the idea that every instance should be mutable and hence gives you an assignment operator by default. The design discussions around variant (expected, outcome, ...) presume that this operation must be implemented. I question this. Not only for variant, but for many other C++ data types.

Robert Ramey

It is part history,

I'd say it's all history.

part performance, for example your immutable integers work fine and fast,

but if you had immutable strings and pushback was O(n) or string was no longer an array that would be problematic.

This is interesting. For my extended integer case it wasn't apparent to me the assignability was a problem. I eventually came around the idea that it was easier and better to tweak my code not to require assignment than to permit assignment in the "variable" type. So as an exercise, you might experiment with your own use case regarding strings. I don't claim that variant with assignment is not useful/necessary. I'm really suggesting that perhaps might be interesting to consider a variant of variant (const variant?) which didn't assignment. The only problem with this is that implementation is so trivial that perhaps a library is not appropriate other than as a tool to motivate people to think about how they are using the variant in the first place. Robert Ramey

AMDG On 3/1/19 9:50 AM, Robert Ramey via Boost wrote:

C++ starts from the idea that every instance should be mutable and hence gives you an assignment operator by default.  The design discussions around variant (expected, outcome, ...) presume that this operation must be implemented.  I question this.  Not only for variant, but for many other C++ data types.

If you want a variant without assignment, it's called const variant<> In Christ, Steven Watanabe