2017-05-27 18:53 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

If they do, you only have a basic guarantee: you can destroy, assign to,

...

or maybe call valueless_by_exception(). Nothing more.

This is not the basic guarantee. It's one step lower than basic and has no name because it deliberately hasn't been named (by Dave Abrahams).

How come? I thought bsic guarantee menas I just should be able to safely destroy it without UB or resource leaks, and perhaps to reset it. How does the above not meet these guarantees? Regards, &rzej;

2017-05-27 19:25 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

How come? I thought bsic guarantee menas I just should be able to safely

...

destroy it without UB or resource leaks, and perhaps to reset it. How does the above not meet these guarantees?

No. Basic means you can safely use the object. It's in an unspecified state, but it's usable. Same as move - unspecified, but valid.

Destroy-only is a completely different animal.

What? How is that different? Surely, when a basic-guarantee operation failed on object `o`, you cannot safely use all operations on `o` that you were allowed to use before the throw. You need to first inspect `o` to see what you can safely do with it. If you get a std::variant<> just after the throw, you can inspect it with valueless_by_exception() and then you know the safe subset of operations is assignment or destructor. valueless_by_exception() is a valid state, even though you cannot set an object to this state easily. Am I wrong here? Regards, &rzej;

2017-05-27 20:56 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

...

...

Destroy-only is a completely different animal.

What? How is that different? Surely, when a basic-guarantee operation failed on object `o`, you cannot safely use all operations on `o` that you were allowed to use before the throw.

Yes, you can. That's the whole point. It's as if someone handed you a valid object `o`. You don't know what's inside, but you know that it's not destroy-only. It has some value, you just don't know what it is.

Ok, but even without any exception you cannot call some functions on your objects, because individual member functions still have preconditions. Consider std::optional: optional<int> o = 1; use_int(*o); // fine o.emplace(this_returns_int_but_throws()); // basic exception safety use_int(*o); // UB, even though o is in "valid state"

Sure, you can abide by the letter of the basic guarantee while trampling all over its spirit by littering all member functions with a "!valueless()" precondition, but this doesn't change much.

Ok, I understand.

In other words, under basic guarantee, r.has_value() returns true or false, under destroy-only it crashes.

In yet other words, under basic guarantee, you in general have wide contract functions, under destroy-only, you never do (except for the destructor and possibly assignment.)

I have checked "Exception-Safety in Generic Components" at http://www.boost.org/community/exception_safety.html It says, "The basic guarantee: that the *invariants of the component are preserved*, and no resources are leaked." I suppose either interpretation is correct, depending on how strong you wan the invariant to be. But tell me this. Consider the example with class Man above: ``` struct Man { std::string fist_name, last_name; }; Man m1 = {"April", "Jones"}; Man m2 = {"Theresa", "May"}; try { m2 = m1; // suppose it throws } catch(...) { } ``` Object m2 after recovering from stack unwinding may be in the state {"April", "May"}, which is a "valid state". Would you call it a valid state? It is "valid" in the sense that reading values from its members does not cause UB, but its "high-level invariant" (that m1 should refer to an existing person) is broken. It conveys no useful information. Once you observe it sneaked out of "stack unwinding bubble", the only reasonable thing you can do with it is to either reset it to a new meaningful value, of just continue with stack unwinding until m1 gets out of scope. Maybe you can see other usages, but I don't. And because I am only interested in reset or destroy, the fact that I can read values from its members is of no use to me. Same with outcome<T>: if I assign to it and it fails: o1 = o2; // assume basic guarantee provided I get transactional guarantee, I know what its value is. But if I get "basic guarantee" as you describe it (valid but unspecfied state), what good does it make that I can safely call has_value() if the object contains a different value than o1 or o2 had initially? How can I trust such value even if I can read it? I can only discard it. Regards, &rzej;

2017-05-28 0:45 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

...

But tell me this. Consider the example with class Man above:

``` struct Man { std::string fist_name, last_name; }; Man m1 = {"April", "Jones"}; Man m2 = {"Theresa", "May"};

try { m2 = m1; // suppose it throws } catch(...) { } ```

Object m2 after recovering from stack unwinding may be in the state {"April", "May"}, which is a "valid state". Would you call it a valid state?

It depends on the invariant of Man. Assuming no invariant, yes, it's valid. It doesn't crash if I access it.

Same with outcome<T>: if I assign to it and it fails:

...

o1 = o2; // assume basic guarantee

provided I get transactional guarantee, I know what its value is. But if I get "basic guarantee" as you describe it (valid but unspecfied state), what good does it make that I can safely call has_value() if the object contains a different value than o1 or o2 had initially?

What good it makes is that you can call has_value() on it without crashing.

Without this guarantee, you could never safely call has_value() on a foreign object without checking valueless() first.

Hold on. I am talking about "valueless_by_exception" which cannot happen for just any reason: only when you tried to mutate the object and it threw an exception in the middle. I am not talking about a default constructed state. I get the result from the function: ``` expected o = fun(); // (*) if (o.has_value()) ... ``` Upon the call to has_value() how is it possible that I am experiencing the valueless_by_exception state? Either I was trying to assign a different value in between (what I marked with (*)) and caught exception (in which case I am doing something fishy), or function fun() is already returning a valueless_by_exception, but why? because it inside again is assigning, and try-catching the assignment, which again is fishy. IOW, my claim is that if in your code you have concerns whetherr calling has_value is safe in the face of valueless_by_exception, you are doing something wrong (catching exceptions prematurely). Rgards, &rzej;

2017-05-28 1:27 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

...

``` expected o = fun(); // (*) if (o.has_value()) ... ```

Upon the call to has_value() how is it possible that I am experiencing the valueless_by_exception state?

It's not possible to experience that state here. It's possible to experience it in other scenarios. If o is non-local, for example, the assignment throws, and a catch block either catches the exception and further code examines o, or if the catch block or a destructor calls a function that examines o.

My first reaction to this is: such a code is again fishy, and it is better to avoid writing such code that offer stronger guarantees. "Examines`o`" -- for what purpose? (after the throw, nowing that it ia in a valid but unspecified state). But maybe it is my inability to imagine a practical use case for it. It would help me, if you could show an example that illustrates a practical value of such examination of an object in a valid-but-unspecified state. (Other than to determine if it is in a valid-but-unspecified state.) Regards, &rzej;

2017-05-28 2:10 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

My first reaction to this is: such a code is again fishy, ...

...

That may be so, but it changes not the fact that there is the basic guarantee, the standard library offers it because Dave, and then there's the destroy-only guarantee, which the standard library does not use.

Ok, I will agree to this statement provided that you agree that "only destroy or reset" is equivalent to basic guarantee when the invariant is super-wide. I think we are agreeing here. I also acknowledge that the STL requires of T's that after move they are in "vaid but unspecified state".

You are not the only person who feels that destroy-only is better, but this doesn't really matter here.

If the subject of the argument is the formal definition of basic guarantee, then agreed, my preference is irrelevant. But what interests me most what is the optimum guarantee for `expected` and `outcome` in the context ot this library review. And in this case my opinion is relevant to Niall or Vicente, even if it is incorrect.

(It's the same for move, by the way. There are people who prefer destroy-only for moved-from objects. The standard library does not agree with them, this time because Howard.)

Acknowledged. But what do the STL containers/algorithms do with the moved-from objects other than to destroy them or reset them?

I realize that I'm a bit curt, so here's some explanation.

Under the destroy-only guarantee, all your member functions have a precondition on valid(), assuming that you even have such an accessor (of you don't, it's even worse). Most of the time, this doesn't matter, because, as you assume, you'll never encounter invalid objects. Except when you do.

I understand this. But exactly the same observation is to be made for all move-only RAII classes that wrap resource handling. Once you have moves, you get the moved-from state, and hence every member function has a weak invariant: you have to check if it represents not-a-resource. Take std::fstream for instance. And we are using these types and recommend using them. And people do not put defensive if-s anywhere. Some do, but this also isn't a good solution.

So you have to split your code into two categories: functions that will never be called on invalid objects, and functions that can. You then need to make sure that first category functions are never, directly or indirectly, called during stack unwinding, either from a catch block, or from a destructor.

Yes, as with movable objects.

This is possible in principle, even though it's not very reliable, because "never happen" objects, well, never happen, and therefore you often aren't testing the scenarios in which they happen. So when they do happen, it's in production.

But in the context of concern you are describing, valueless_by_exception is no different than the moved-from state. moved-from state is "safe" on `std::vector` but not on `std::fstream`. We are facing this problem in amny many places.

Under the basic guarantee, when your member functions don't have a "valid" precondition because objects are always valid, and when functions taking objects don't need to be split into two categories, some that can only take valid objects, and others that can take invalid ones, the world is simply a better place.

With move semantic world is not this good place any more. You have moved-from states potentially everywhere. And this is smething we are used to. Regards, &rzej;

2017-05-29 6:15 GMT+02:00 Howard Hinnant via Boost :

On May 28, 2017, at 6:36 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2017-05-28 2:10 GMT+02:00 Peter Dimov via Boost :

...

(It's the same for move, by the way. There are people who prefer destroy-only for moved-from objects. The standard library does not agree with them, this time because Howard.)

Acknowledged. But what do the STL containers/algorithms do with the moved-from objects other than to destroy them or reset them?

They are allowed to do everything with moved-from objects that they are allowed to do with objects that haven’t been moved from. For example sort is allowed to move construct from, move assign to, move assign from, swap, and compare objects, whether or not they are in a moved from state. std::sort requires objects to satisfy strict weak ordering. This means among other things that x < x is always false, even if x is in a moved-from state.

In summary, being in a moved-from state does not excuse an object from meeting the requirements of the algorithm it is being used with. It only excuses the object from having a specified state (in most cases).

Thanks. that was useful.

Real-world examples:

An old std::reverse implementation, when given a range with an odd length, would swap the middle element with itself. This results in a moved-from object being self-move-assigned if the generic std::swap is being used. It just has to work. By work, I mean that the self-move-assignment of a moved-from-object has to leave the object in a valid but unspecified state. If it does this, then self-swap is a non-modifying operation, and the reverse algorithm gives the correct result.

Ok, self-assignment is something different than only "being able to assign to". But that could be added to the minimum safety requirements.

The std::remove algorithm returns moved-from objects back to the client (those at the end of the sequence that have been “removed”). The client is free to do anything he wants with those objects as long as that operation does not have a precondition. Although the typical use case is for the client to delete those “removed” elements, that is by no means required, or the only use case.

Ok, so who of the people who are following this thread ever did anything else with these past-removal objects, other than to destroy them? I am not being sarcastic. I just can't imagine anyone doing anything else with them.

I know of no sort implementation that compares moved-from objects (though I have not done a rigorous survey either). But it is quite possible that a sort algorithm could compare a moved-from object against itself, and still be a correct algorithm, as long as x < x (or comp(x, x)) _always_ returns false.

Variant's semantics meet these requirements, even in valueless_by_exception state. Right? Thanks, &rzej;

Mere moments ago, quoth I:

For types that do have a well-defined empty state (like most smart pointers) it can be safer to make assumptions that the pointer is now empty after being moved-from. (As long as you remember that it's up to both the method you're passing the object to and the object itself whether the move actually happens -- calling std::move by itself is a declaration of intent, not an action. That trips many people up at first.)

Another example (which I believe is correct, but perhaps I've missed something): If you move-from a std::vector, or any other STL container, you are guaranteed that the vector is in a sane state which you can still call any methods on. You are *not* guaranteed that the vector is empty (although this is reasonably likely since that's a typical side-effect of the performant implementation). Importantly, this means that it's legal to move a vector, then clear() it and add more items to it. If you forget to clear it, then it's still legal (there's no memory leaks or other UB) but it's implementation-defined whether you end up adding to an empty vector or appending to the original one (or some weird hybrid, though that's less likely), so it's probably not something you should actually do. You could probably call empty() to figure out which it was, but there's usually not much point -- you generally just want to clear() it and move on (pun somewhat intended).

On May 30, 2017, at 7:13 AM, Andrzej Krzemienski via Boost wrote:

2017-05-29 6:15 GMT+02:00 Howard Hinnant via Boost :

...

On May 28, 2017, at 6:36 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2017-05-28 2:10 GMT+02:00 Peter Dimov via Boost :

...

(It's the same for move, by the way. There are people who prefer destroy-only for moved-from objects. The standard library does not agree with them, this time because Howard.)

Acknowledged. But what do the STL containers/algorithms do with the moved-from objects other than to destroy them or reset them?

They are allowed to do everything with moved-from objects that they are allowed to do with objects that haven’t been moved from. For example sort is allowed to move construct from, move assign to, move assign from, swap, and compare objects, whether or not they are in a moved from state. std::sort requires objects to satisfy strict weak ordering. This means among other things that x < x is always false, even if x is in a moved-from state.

In summary, being in a moved-from state does not excuse an object from meeting the requirements of the algorithm it is being used with. It only excuses the object from having a specified state (in most cases).

Real-world examples:

An old std::reverse implementation, when given a range with an odd length, would swap the middle element with itself. This results in a moved-from object being self-move-assigned if the generic std::swap is being used. It just has to work. By work, I mean that the self-move-assignment of a moved-from-object has to leave the object in a valid but unspecified state. If it does this, then self-swap is a non-modifying operation, and the reverse algorithm gives the correct result.

The std::remove algorithm returns moved-from objects back to the client (those at the end of the sequence that have been “removed”). The client is free to do anything he wants with those objects as long as that operation does not have a precondition. Although the typical use case is for the client to delete those “removed” elements, that is by no means required, or the only use case.

I know of no sort implementation that compares moved-from objects (though I have not done a rigorous survey either). But it is quite possible that a sort algorithm could compare a moved-from object against itself, and still be a correct algorithm, as long as x < x (or comp(x, x)) _always_ returns false.

Ok, I think I asked the wrong question. I asked:

what do the STL containers/algorithms do with the moved-from objects

...

But what I actually meant was:

"when an STL algorithm *causes* an object to obtain a valid-but-unspecified *during its operation* , what does it later do with this object other than to destroy it or re-set it (or leave it like this for the caller -- for the caller to destroy it or reset it)?"

Because, if I myself am passing objects in valid-but-unspecified state to STL algorithms, I am already doing something wrong (I should have probably destroyed or reset these objects).

I had interpreted your question the way you meant it. My answer doesn’t change. Howard

On May 30, 2017, at 11:30 AM, Andrzej Krzemienski via Boost wrote:

std::sort, apart from what it is allowed to do, is also require to produce the output range that is the permutation of the input range. Assuming no moved-from states on input, if for some reason it starts to put objects in a moved-from state and then using this value for sorting further, even if moved-from state is well-ordered, you will get this moved from state in the output range, but it was not in the input range.

Theoretically, a std::sort algorithm could move from x[i], then subsequently perform x[i] < x[i], act on the return of false, and then move assign a new value into x[i], and still get the right answer. I know of no shipping std::sort that actually does this. Howard

On May 30, 2017, at 5:45 PM, Andrzej Krzemienski via Boost wrote:

2017-05-30 17:43 GMT+02:00 Howard Hinnant via Boost :

...

On May 30, 2017, at 11:30 AM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

std::sort, apart from what it is allowed to do, is also require to produce the output range that is the permutation of the input range. Assuming no moved-from states on input,

if

...

for some reason it starts to put objects in a moved-from state and then using this value for sorting further, even if moved-from state is well-ordered, you will get this moved from state in the output range, but it was not in the input range.

Theoretically, a std::sort algorithm could move from x[i], then subsequently perform x[i] < x[i], act on the return of false, and then move assign a new value into x[i], and still get the right answer. I know of no shipping std::sort that actually does this.

Yes, it could do it :) I guess the only purpose would be to verify if the moved-from state is well ordered with other values.

I was thinking more along the lines of a performance bug, much like the std::reverse implementation that did one-too-many swaps on odd-numbered ranges. Howard

2017-05-28 0:55 GMT+02:00 Emil Dotchevski via Boost :

On Sat, May 27, 2017 at 3:15 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

I suppose either interpretation is correct, depending on how strong you wan the invariant to be.

Nope, this is a matter of definition -- basic exception safety is _defined_ to mean something. Other interpretations are incorrect, or rather, disagree with the definition.

...

But tell me this. Consider the example with class Man above:

``` struct Man { std::string fist_name, last_name; }; Man m1 = {"April", "Jones"}; Man m2 = {"Theresa", "May"};

try { m2 = m1; // suppose it throws } catch(...) { } ```

Object m2 after recovering from stack unwinding may be in the state {"April", "May"}, which is a "valid state". Would you call it a valid state? It is "valid" in the sense that reading values from its members does not cause UB, but its "high-level invariant" (that m1 should refer to an existing person) is broken. It conveys no useful information.

This is not about usefulness but validity. When we say that vector::push_back has basic exception safety guarantee (in general, it is strong in case no reallocation occurs), it doesn't mean that you're going to find anything "useful" in the vector in case of failure. It only means that any objects that remain in the vector are in valid state, and the vector itself is in a valid state.

Yes, it is "valid, and the elements are "valid"; and what do you do next? Because the only ways to proceed I see is to leave the scope and have the vector destroyed. Or clear it, ao assign a different, meaningful vector.

What does it mean for an object to be in valid state? This means that its invariants are in place. In C++ this is formally supported by the semantics of constructors: a constructor succeeds in establishing the invariants of the type or it does not return.

Yes, I understand that; and in the context of this thread for the purpose of describing valueless_by_exception state, I claim that the invariant for expected is: `this->valueless_by_exception() || /* other parts of invariant*/` That is, I artificially widen the invariant. Peter's claim (if I understand correctly) is that by this artificial invariant widening I make the semantics fuzzy beyond reason. But my claim is quite the opposite. Any more guarantee is never needed. I cannot see where/why we disagree.

Note that there is no provision to report a failure (to establish the invariants) except by throwing. This is not an omission but a deliberate design choice.

Interesting. I do not know what you mean here. Maybe, "there is no other way to report such failure except to throw an exception"? If so, I agree, but how does this relate to the discussed topic? Regards, &rzej;

2017-05-28 3:02 GMT+02:00 Emil Dotchevski via Boost :

On Sat, May 27, 2017 at 4:09 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2017-05-28 0:55 GMT+02:00 Emil Dotchevski via Boost < boost@lists.boost.org

...

:

...

On Sat, May 27, 2017 at 3:15 PM, Andrzej Krzemienski via Boost <> Note that there is no provision to report a failure (to establish the invariants) except by throwing. This is not an omission but a deliberate design choice.

Interesting. I do not know what you mean here. Maybe, "there is no other way to report such failure except to throw an exception"? If so, I agree, but how does this relate to the discussed topic?

What I am demonstrating is that the C++ semantics for initialization and destruction of objects have a built-in assumption about what constitutes a valid object. Can you define this as "the only safe thing to do with x is call is_valid()" on it? Sure, but then you're operating as a C programmer. Consider what a C programmer has to do:

struct foo { /*state*/ };

void init_foo( foo * x ) { /*initialization*/ assert(is_valid(x)); }

void destroy_foo( foo * x ) { assert(is_valid(x)); //destroy *x }

void use_foo( foo * x ) { assert(is_valid(x)); /*use foo*/ }

And now compare this to a C++ program:

struct foo { /*state*/ foo() { /*initialization*/ }

~foo() { /*destruction*/ }

void use() { /*use foo*/ } };

Note that in well designed C++ programs not only the asserts are not necessary, they're downright silly. Is the object valid? Duh, of course it is, or else the constructor would not have returned. But if you introduce a "not quite valid" state, not only you need the asserts, you're making it much more difficult for the user to reason about the state of the objects in his program, just like a C programmer must.

Emil, thanks for being patient with me. I understand what you are saying here. It is convincing. But ultimately I have found it to be incorrect after C++11 introduced move semantics. Let me show you a typical implementation of a RAII-like type for representing file-handles. First, in C++03, without moves ``` class File { int _handle; public: explicit File(string_view name) : _handle(system::open_file(name)) { if (_handle == 0) throw FileProblem{}; } char read() { // no precondition: _handle always valid return system::read_char(_handle); } ~File() { // no precondition: _handle always valid system::close(_handle); } }; ``` It is as you say: if we have an object, we know we have a file open, ready to be used. But now, lets's add C++11's move semantics: ``` class File { int _handle; public: explicit File(string_view name) : _handle(system::open_file(name)) { if (_handle == 0) throw FileProblem{}; } File(File && rhs) : _handle(rhs._handle) { rhs._handle = 0; // now rhs obtains an invalid state (or, not-a-file state) } char read() { // precondition: _handle != 0 return system::read_char(_handle); } ~File() { if (_handle) // defensive if system::close(_handle); } }; ``` Now, because I have a moved-from state, it weakens all my invariants. As you say, every function now has a precondition: either I put defensive if's everywhere, or expect the users to be putting them. And this is a normal moveable RAII class (or maybe a movable type is no longer "RAII" because of this). And we have lived with it for years now. I often have functions returning std::unique_ptr's, and I am not defensive-checking everywher if the function did not return a null. I just trust that if someone is returning a unique_ptr it is because they wanted to return a heap allocated object: not null. My point: moved-from state is quite similar to valueless_by_exception, it exposes the same problems (weak invariants on RAII-like types), and no-one complains about it.

If the user is handed an object, is it safe to use it? He could assume that it is, but your design choice has rendered that an unsafe assumption, because you're effectively reserving the right for the object to be unusable. Alternatively, he could assume that he can't use it unless he checks first. But now he's forced to write if( obj.is_valid() ) obj.use() rather than obj.use().

Same with all movable types. And we use them, and I don't think we recommend putting defensive if-s everywhere in the code.

What he is supposed to do in theory is analyze the program carefully and determine for sure if in this particular case the object can be unusable. In practice, there are so many corner cases that it is easy to get this wrong, if not when the code is initially written, then under maintenance. Such bugs are very difficult to detect, and infinitely more difficult to detect in error handling code (where this "not quite valid" state would occur, per your design.)

Again, I understand your reasoning, but I think it equally well applies to moved-from state. Are you also describing shortcommings of moves? Regards, &rzej;

2017-05-29 1:20 GMT+02:00 Emil Dotchevski via Boost :

On Sun, May 28, 2017 at 3:15 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2017-05-28 3:02 GMT+02:00 Emil Dotchevski via Boost < boost@lists.boost.org

...

:

...

On Sat, May 27, 2017 at 4:09 PM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

2017-05-28 0:55 GMT+02:00 Emil Dotchevski via Boost < boost@lists.boost.org

...

:

...

On Sat, May 27, 2017 at 3:15 PM, Andrzej Krzemienski via Boost <> Note that there is no provision to report a failure (to establish the invariants) except by throwing. This is not an omission but a deliberate design choice.

Interesting. I do not know what you mean here. Maybe, "there is no other way to report such failure except to throw an exception"? If so, I agree, but how does this relate to the discussed topic?

What I am demonstrating is that the C++ semantics for initialization and destruction of objects have a built-in assumption about what constitutes a valid object. Can you define this as "the only safe thing to do with x is call is_valid()" on it? Sure, but then you're operating as a C programmer. Consider what a C programmer has to do:

struct foo { /*state*/ };

void init_foo( foo * x ) { /*initialization*/ assert(is_valid(x)); }

void destroy_foo( foo * x ) { assert(is_valid(x)); //destroy *x }

void use_foo( foo * x ) { assert(is_valid(x)); /*use foo*/ }

And now compare this to a C++ program:

struct foo { /*state*/ foo() { /*initialization*/ }

~foo() { /*destruction*/ }

void use() { /*use foo*/ } };

Note that in well designed C++ programs not only the asserts are not necessary, they're downright silly. Is the object valid? Duh, of course it is, or else the constructor would not have returned. But if you introduce a "not quite valid" state, not only you need the asserts, you're making it much more difficult for the user to reason about the state of the objects in his program, just like a C programmer must.

Emil, thanks for being patient with me. I understand what you are saying here. It is convincing. But ultimately I have found it to be incorrect after C++11 introduced move semantics. Let me show you a typical implementation of a RAII-like type for representing file-handles. First, in C++03, without moves

``` class File { int _handle;

public: explicit File(string_view name) : _handle(system::open_file(name)) { if (_handle == 0) throw FileProblem{}; }

char read() { // no precondition: _handle always valid return system::read_char(_handle); }

~File() { // no precondition: _handle always valid system::close(_handle); } }; ```

It is as you say: if we have an object, we know we have a file open, ready to be used. But now, lets's add C++11's move semantics:

``` class File { int _handle;

public: explicit File(string_view name) : _handle(system::open_file(name)) { if (_handle == 0) throw FileProblem{}; }

File(File && rhs) : _handle(rhs._handle) { rhs._handle = 0; // now rhs obtains an invalid state (or, not-a-file state) }

char read() { // precondition: _handle != 0 return system::read_char(_handle); }

~File() { if (_handle) // defensive if system::close(_handle); } }; ```

Now, because I have a moved-from state, it weakens all my invariants. As you say, every function now has a precondition: either I put defensive if's everywhere, or expect the users to be putting them.

Indeed, and this is not a good thing. Consider that the int handle doesn't have move semantics, even though (in theory) it is possible to define some type with invalid "moved from" state, even in C.

When dealing with things like file handles or any other resourse it's best to use shared_ptr. This entire File class you wrote can be reduced to a function that returns shared_ptr<int const> that closes the file in a custom deleter. Things are even more straight-forward if you use FILE *, then it's just shared_ptr<FILE>.

Are you suggesting that one should use shared_ptrs instead of movable types? Are you saying that the design of a movable std::fstream is wrong?

...

And this is a normal moveable RAII class (or maybe a movable type is no longer "RAII" because of this). And we have lived with it for years now. I often have functions returning std::unique_ptr's, and I am not defensive-checking everywher if the function did not return a null. I just trust that if someone is returning a unique_ptr it is because they wanted to return a heap allocated object: not null.

IMO unique_ptr is way overused and shared_ptr way underused. I know, "Overhead!!",

Not only overhed. also the fact that you are implying shared ownership semantics (possibly across threads), even though you have none.

but like in the case of exception handling the overhead is not a problem in general and it comes with the benefit of weak_ptr.

Maybe if you had a weak_ptr for a unique_ptr it would convince me.

In practice many of the concerns you expressed can be dealt with if it's possible to hold on to the object just a bit longer until you're done with it. Using shared/weak_ptr replaces all of the defensive ifs you otherwise need to sprinkle around with a single if at lock time:

if( shared_ptr<foo> sp=wp.lock() ) { sp->do_a(); sp->do_b(); }

You also have one `if` here. I can also have a one-if solution with a unique_ptr: ``` if( unique_ptr<foo> sp = get() ) { sp->do_a(); sp->do_b(); } ``` It looks like the same (in)convenience to me.

Again, I understand your reasoning, but I think it equally well applies to

...

moved-from state. Are you also describing shortcommings of moves?

Not necessarily. Note that a "moved from" std::vector is still a good vector. I understand that sometimes it does make sense to define a less than completely valid state and the language does support that, but these should be treated as unfortunate necessities rather than good C++ programming practices, especially because they effectively butcher RAII.

What about std::fstream. Does it butcher RAII?

Again, granted, this is a moot point if you've already disabled exception handling,

No, no. I was raising doubts about the cases with exceptions as well. which also effectively butchers RAII. Interestingly, this also

leads to having to write ifs all over the place, this time to manually enforce postconditions.

I think this can also be dealt with one-if solution as you described. ``` if( outcome<foo> sp = foo::make() ) // factory instead constructor { sp->do_a(); sp->do_b(); } ``` Regards, &rzej;

Le 30/05/2017 à 04:13, Emil Dotchevski via Boost a écrit :

On Mon, May 29, 2017 at 12:50 AM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

...

...

Now, because I have a moved-from state, it weakens all my invariants. As you say, every function now has a precondition: either I put defensive if's everywhere, or expect the users to be putting them.

Indeed, and this is not a good thing. Consider that the int handle doesn't have move semantics, even though (in theory) it is possible to define some type with invalid "moved from" state, even in C.

When dealing with things like file handles or any other resourse it's best to use shared_ptr. This entire File class you wrote can be reduced to a function that returns shared_ptr<int const> that closes the file in a custom deleter. Things are even more straight-forward if you use FILE *, then it's just shared_ptr<FILE>.

Are you suggesting that one should use shared_ptrs instead of movable types? Are you saying that the design of a movable std::fstream is wrong?

Why is it better to use shared_ptr instead of move-only wrappers when dealing with (file) handles? Because handles are copyable types and (by design) they can be shared. I would say that it depends on the application, we need movable and shared handles. BTW, I don't know what (std| boost) Filesystem provides?

...

IMO unique_ptr is way overused and shared_ptr way underused. I know,

...

"Overhead!!", Not only overhed. also the fact that you are implying shared ownership semantics (possibly across threads), even though you have none.

Your argument is that if it is incorrect to have more than one thread have access to the object then shared_ptr is the wrong design choice. But it doesn't necessarily follow, because move-only semantics don't _prevent_ multiple threads from accessing the object.

Multi-threading and ownership are two different things. Vicente

2017-05-30 4:13 GMT+02:00 Emil Dotchevski via Boost :

On Mon, May 29, 2017 at 12:50 AM, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

...

...

Now, because I have a moved-from state, it weakens all my invariants. As you say, every function now has a precondition: either I put defensive if's everywhere, or expect the users to be putting them.

Indeed, and this is not a good thing. Consider that the int handle doesn't have move semantics, even though (in theory) it is possible to define some type with invalid "moved from" state, even in C.

When dealing with things like file handles or any other resourse it's best to use shared_ptr. This entire File class you wrote can be reduced to a function that returns shared_ptr<int const> that closes the file in a custom deleter. Things are even more straight-forward if you use FILE *, then it's just shared_ptr<FILE>.

Are you suggesting that one should use shared_ptrs instead of movable types? Are you saying that the design of a movable std::fstream is wrong?

Why is it better to use shared_ptr instead of move-only wrappers when dealing with (file) handles? Because handles are copyable types and (by design) they can be shared.

A raw file handle, (a pointer or an int) is indeed copyable and shareable. Agreed. But now, this is inherited from C, and is it because it was the optimal design choice, or is it only because in C you could not do it in a better way (e.g. by applying unique ownership)?

...

IMO unique_ptr is way overused and shared_ptr way underused. I know,

...

"Overhead!!",

Not only overhed. also the fact that you are implying shared ownership semantics (possibly across threads), even though you have none.

Your argument is that if it is incorrect to have more than one thread have access to the object then shared_ptr is the wrong design choice. But it doesn't necessarily follow, because move-only semantics don't _prevent_ multiple threads from accessing the object.

Agreed. In general, there is no way to prevent two threads from observing the same object. My point is that you can prevent it when you use value semantics. When I return shared_ptr-s even employing value semantics does no longer guarantee shared ownership, even though it could if I used a movable-only type. Besides, I don't think shared_ptr's move constructor does what you describe it does. From the Standard: shared_ptr(shared_ptr&& r) noexcept;

template<class Y> shared_ptr(shared_ptr<Y>&& r) noexcept;

Remarks: The second constructor shall not participate in overload resolution unless Y* is compatible with T*. Effects: Move constructs a shared_ptr instance from r. *Postconditions:* *this shall contain the old value of r. r shall be empty. r.get() == nullptr.

...

but like in the case of exception handling the overhead is

...

not a problem in general and it comes with the benefit of weak_ptr.

Maybe if you had a weak_ptr for a unique_ptr it would convince me.

You don't need weak_ptr for unique_ptr, because shared_ptr can do everything unique_ptr can do.

Yes, and far far more. And this is my objection. This type is doing far more than what I need, and therefore using it sends the wrong message to other developers (e.g., that I may be using shared ownership semantics. You already said that you can pass around naked handles anyway. But my view is that I want to wrap them in more constrained type to send a message to other developers and to compiler, that I will not be doing certain things, even though I know how to do them. This is my idea of safety: to constrain myself (with the choice of types) to do only the operations I require, and no more.

...

...

In practice many of the concerns you expressed can be dealt with if it's possible to hold on to the object just a bit longer until you're done with it. Using shared/weak_ptr replaces all of the defensive ifs you otherwise need to sprinkle around with a single if at lock time:

if( shared_ptr<foo> sp=wp.lock() ) { sp->do_a(); sp->do_b(); }

You also have one `if` here. I can also have a one-if solution with a unique_ptr:

``` if( unique_ptr<foo> sp = get() ) { sp->do_a(); sp->do_b(); } ```

It looks like the same (in)convenience to me.

I don't understand, what is get()?

I am just illustrating some function that returns a unique_ptr. Like a factory function.

The point I was making is that with shared_ptr, as long as you keep the shared_ptr afloat, the object isn't going anywhere and it is safe to use. Contrast this with an object which could have been moved-from and left in a not-quite-valid state.

You can also move from a shared_ptr, and it no longer owns the object. This is my understanding of the specification of std::shared_ptr.

...

...

Again, I understand your reasoning, but I think it equally well applies to

...

moved-from state. Are you also describing shortcommings of moves?

Not necessarily. Note that a "moved from" std::vector is still a good vector. I understand that sometimes it does make sense to define a less than completely valid state and the language does support that, but these should be treated as unfortunate necessities rather than good C++ programming practices, especially because they effectively butcher RAII.

What about std::fstream. Does it butcher RAII?

"move constructor: Acquires the contents of x. First, the function move-constructs both its base iostream class from x and a filebuf object from x's internal filebuf object, and then associates them by calling member set_rdbuf. x is left in an unspecified but valid state."

"Unspecified but valid state" tells me that there is nothing wrong with using a moved-from std::fstream.

Really, and will you put more characters to it with operator<

But yes, I do think that the fstream invariants are too weak.

And would you make it so strong that operator<< still works with well defined semantics? And does what?

I think this can also be dealt with one-if solution as you described.

...

``` if( outcome<foo> sp = foo::make() ) // factory instead constructor { sp->do_a(); sp->do_b(); } ```

Compare to:

shared_ptr<foo> x=foo::make(); x->do_a(); x->do_b();

The if is gone because foo::make won't return upon failure.

How come the if is gone? shared_ptr has a weak invariant: it can be a nullptr. I personally do not check shared_ptr-s returned from factories for null, but it is my understanding that your position is that you should always be sure you are not having the weak state.

Also this is repeated every time you call a function which may fail: when using exception handling, exception-neutral contexts don't have to worry about that possibility. If you don't use exceptions, you have to make sure you communicate the failure, which is prone to errors.

I absolutely agree: when you throw exceptions to communicate failures, and you do not catch them prematurely, you do not have to worry about defensive ifs. We seem to agree here. But this is exactly why I am in favor of having the valueless_by_exception state (if preventing it comes at efficiency cost): if you use exceptions correctly, you will never observe this state: only destructors will (or sometimes assignments).

More formally, exception handling allows you to enforce postconditions: the code that follows foo::make() requires that the object was created successfully. Therefore, the code that calls make() has to enforce the postcondition that make was successful.

I agree with you here.

Either you write ifs, or you use exceptions and (effectively) the compiler does it for you.

I agree. I never wanted to promote defensive if-s. My claim is, if you use exceptions correctly, and design functions with exception safety in mind. The minimum guarantee (destroy-and-reset only) is not worse than "valid but unspecified state". I have not yet been convinced to the contrary. Regards, &rzej;

On 5/27/17 10:25 AM, Peter Dimov via Boost wrote:

Andrzej Krzemienski wrote:

...

How come? I thought bsic guarantee menas I just should be able to safely destroy it without UB or resource leaks, and perhaps to reset it. How does the above not meet these guarantees?

Hmmm - perhaps this refers to the "strong" guarentee. Calling David Abrahams.

No. Basic means you can safely use the object. It's in an unspecified state, but it's usable. Same as move - unspecified, but valid.

Destroy-only is a completely different animal.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

2017-05-28 0:13 GMT+02:00 Niall Douglas via Boost :

...

This is in connection with Vicente's question: what exception safety should we expect of copy assignment of `expected`?

Vicente's paper promises a never empty guarantee. So if during a transition of state during assignment the new state's assignment operator throws, the previous state in the Expected is preserved.

Outcome master branch doesn't implement that. Outcome develop branch does.

...

First, let's consider the use case for `expected`, where we want to disable exception handling altogether. This means `T` or `E` cannot throw on any operation.

Vicente's paper allows T to throw during move or copy. E must be nothrow move. This makes possible the never empty guarantee.

...

So my view, as of today, is not to strive for a strong or even never-empty guarantee. Provide a conditional guarantee, if types T and E don't throw, you get no-fail guarantee. If they do, you only have a basic guarantee: you can destroy, assign to, or maybe call valueless_by_exception(). Nothing more. I think `std::variant` made the optimal choice.

std::variant could provide a never empty guarantee without increasing storage required if it imposed restrictions on its types e.g. all but one of the possible types must have nothrow move construction.

If it were allowed to double the storage required, it could use Anthony Williams' double buffer technique to ensure never empty guarantee. This eliminates the need for valueless_by_exception() entirely.

None of this affects Expected nor Outcome. Outcome hard codes EC and E, and both those types are guaranteed nothrow move constructible, so we can guarantee we never lose a previous state if assignment of a new state throws.

I'll ask the same I asked Peter, because I no longer see the value in never-empty guarantee. If I have two objecte of type variant, where A, B can throw on copy/move, and C is trivial: ``` variant a = A{}, b = B{}; try { a = b; // throws } catch(...) {} // at this point a holds a C ``` What good does it make to me that I had an A, wanted to assign a B and got a C? (unless C represents empty_state.) Same with outcomes: if I assign `o1 = o2` and because of an exception I get value different than `o1` or `o2` had initially, what good does it make? Of course, the bug in my examples is that I try to catch exceptions prematurely. If I let the stack unwinding continue, all these objects are gone and I can start anew. Regards, &rzej;

2017-05-28 12:59 GMT+02:00 Niall Douglas via Boost :

...

I'll ask the same I asked Peter, because I no longer see the value in never-empty guarantee.

If I have two objecte of type variant, where A, B can throw on copy/move, and C is trivial:

``` variant a = A{}, b = B{};

try { a = b; // throws } catch(...) {}

// at this point a holds a C ```

What good does it make to me that I had an A, wanted to assign a B and got a C?

Why would you get a C in 'a'?

'a' previously was set to A.

The assignment of 'b' with B state to 'a' failed due to exception.

I am describing `boost::variant` which tries to provide both "never-empty" guarantee and "no double storage" guarantee. But it does not provide the strong guarantee: In short, if you assign to a, first the A is destroyed, second, we attempt to copy-construct a B. It fails: we have no A (already destroyed) and no B (construction failed), so it will default-construct a C, because this is guaranteed not to throw. So the never-empty guarantee is preserved, but you get some other value, you have no use for. This is documented here: http://www.boost.org/doc/libs/1_64_0/doc/html/variant/design.html#variant.de...

So the strong never empty guarantee implemented by both Outcome and Expected means that 'a' still contains its original A untouched and undamaged. 'b' also contains its B untouched and undamaged, though that is up to B's copy constructor.

I wonder if this is possible. In `result` you have a T an an error_code_extended. If you are assigning to an errored result: ``` outcome<T> o = make_errored_result(xxx); outcome<T> v = T{yyy}; o = v; ``` What do you do? First you need to destroy the error_code_extended stored by `o`. Then you try to copy-construct T in the same storage. If it fails, you have no error to return to: you have just destroyed it, and possibly overwrote the storage. So I fail to see how you can implement a strong guarantee for the assignment, unless you apply no-throw restrictions on type T. Regards, &rzej;

2017-05-29 1:38 GMT+02:00 Niall Douglas via Boost :

...

...

So the strong never empty guarantee implemented by both Outcome and Expected means that 'a' still contains its original A untouched and undamaged. 'b' also contains its B untouched and undamaged, though that is up to B's copy constructor.

I wonder if this is possible. In `result` you have a T an an error_code_extended. If you are assigning to an errored result:

``` outcome<T> o = make_errored_result(xxx); outcome<T> v = T{yyy};

o = v; ```

What do you do? First you need to destroy the error_code_extended stored by `o`. Then you try to copy-construct T in the same storage. If it fails, you have no error to return to: you have just destroyed it, and possibly overwrote the storage.

So I fail to see how you can implement a strong guarantee for the assignment, unless you apply no-throw restrictions on type T.

It's very straightforward.

Yes, I just realized it is, because....

If the existing state's type is nothrow move but the new state's type is throwing move/copy, you move it onto the stack. Then you clear storage, and assign in the T. If that throws, you restore the stacked previous state.

If the existing state's type is throwing move and the new state's type is nothrow move, you destruct, clear storage, and do the assignment.

Only if the existing state's type is throwing and the new state's type is throwing is there a problem. That's why I said earlier that to implement the strong never empty guarantee without additional storage, all but one of a variant's types must be nothrow move.

Yes, and in both expected<> and outcome<> you are guaranteed that the E is nothrow movable (and copyable). This is why you do not have the problem that variant in the general case has.

Does this make sense?

Yes, it does now. In that case I understand Vicente's recommendation, and I support it: the constraint (the static_assert-s) that E is nothrow-movable should be imposed on the copy/move-assignment, but not on the entire type. If I am not doing assignments on `expected` there is nothing wrong in having an E that throws upon copying. (As in my example with validating input.) Regards, &rzej;