Niall Douglas wrote:

Now, some may say that calling .exception() on an errored object is wasteful because you must silently construct an exception_ptr for a system_error. But equally, I'd counter that if you actually care about that, check .has_error() and yank out the error_code before calling .exception().

How I see things here is, you specify what error() and exception() return after calls to set_value(v), set_error(e), and set_exception(x), respectively, as if conceptually initialization stores the appropriate return values into independent error_ and exception_ member variables. Then, the synthesis of an exception_ptr from the error code in exception() is just an optimization that doesn't change behavior but is just a different tradeoff - it favors copy performance (no exception_ptr to copy after set_error) at the expense of exception() performance (have to make_exception_ptr each time.)

After all:

sizeof(error_code_extended) = 24 sizeof(exception_ptr) = 8

So for a type predominantly used solely for returning results from functions where you will only ever be returning one of them at a time, there is a very strong argument in favour of ditching the variant storage in exchange for considerable improvements in:

- implementation complexity - compile times for end users - load on the compiler's optimiser

... for the hardcoded EC = error_code_extended and E = exception_ptr Outcome.

I see this as a promising direction because it would allow you to store both an error and an exception, as per the other thread. Then you can fully represent a Filesystem function with an outcome return value, because you'll be able to store the error_code return in error() and the filesystem_error exception in exception(). The actual behavior of Outcome won't change that much, because if you only look at its observable state as told by the accessors, logically it's not a variant, because both has_error and has_exception report true at the same time. In fact has_error == has_exception if I'm not mistaken?

On June 5, 2017 8:06:27 AM EDT, Peter Dimov via Boost wrote:

...

I see this as a promising direction because it would allow you to store both an error and an exception, as per the other thread.

To be more precise, you'll be able to represent the exact equivalent of the two Filesystem overloads with a single function.

R filesystem_api(); // throws filesystem_error R filesystem_api( error_code& ec ) noexcept;

->

outcome<R> filesystem_api();

That changes the behavior for what was the throwing overload, doesn't it? -- Rob (Sent from my portable computation device.)

2017-06-05 19:59 GMT+02:00 Niall Douglas via Boost :

...

I see this as a promising direction because it would allow you to store both an error and an exception, as per the other thread. Then you can fully represent a Filesystem function with an outcome return value, because you'll be able to store the error_code return in error() and the filesystem_error exception in exception().

The actual behavior of Outcome won't change that much, because if you only look at its observable state as told by the accessors, logically it's not a variant, because both has_error and has_exception report true at the same time. In fact has_error == has_exception if I'm not mistaken?

Currently has_error() = true if and only there is an error stored. has_exception() = true if error or exception stored.

If one is removing the variant storage and gaining .has_failure(), I think .has_exception() would simply report true if there is an exception in there.

I do feel considerable concern with your idea of using the exception_ptr as a payload for the error code. I appreciate the use case for the Filesystem TS, but I feel the fact that the Filesystem TS returns more information via the exception throw mechanism than via the error_code mechanism to be an awful mistake. Some other mechanism should have been chosen to make both of equal information returning value instead of splitting the two and making the error code returning overloads inferior like that. After all, the latter are the natural fit for a Filesystem library. The former are inappropriately heavyweight.

What I could live with though is this synopsis:

template class outcome { T _value; EC _error; union { P _payload; E _exception; }; };

I personally really wish that shared_ptr could be constructible from an exception_ptr as they surely are implemented the same way on any standard library implementation I could think of, but at least the above correctly lets payload be any type-erased shared_ptr, which is the correct and appropriate type for returning type-erased payload unlike exception_ptr.

The question now becomes this: surely the Filesystem TS is cleaner if when returning an errored outcome it supplied a shared_ptr> instead of the exception_ptr to the filesystem_error that would have been thrown? Advantages:

1. I would also be fairly sure, without having benchmarked it, that make_shared>(path1, path2) will be many times faster than make_exception_ptr(filesystem_error(path1, path, ec)) on most standard library implementations.

2. make_exception_ptr(filesystem_error(path1, path, ec)) will return a null pointer if C++ exceptions are disabled on at least libstdc++, and thus your proposal would be a problem for those users running with exceptions off.

What do you think?

I will not address your question directly; but let me offer a remark. It was my understanding that `error_code_extended`, along with its ring buffer usage, was provided to address exactly this problem: provide additional payload to `std::error_code`. The current exercise with adapting the Filesystem TS can be considered a test of the usefulness of `error_code_extended`. Some questions arise that i will surely ask in the second review of Boost.Outcome (I hope we will have one): 1. Can `result` be used to successfully replace error handling in Filesystem TS? 2. If I carry arbitrary payload with a `result` do I need the ring buffer for anything? Regards, &rzej;

2017-06-06 11:04 GMT+02:00 Niall Douglas via Boost :

...

...

I do feel considerable concern with your idea of using the exception_ptr as a payload for the error code. I appreciate the use case for the Filesystem TS, but I feel the fact that the Filesystem TS returns more information via the exception throw mechanism than via the error_code mechanism to be an awful mistake. Some other mechanism should have been chosen to make both of equal information returning value instead of splitting the two and making the error code returning overloads inferior like that. After all, the latter are the natural fit for a Filesystem library. The former are inappropriately heavyweight.

What I could live with though is this synopsis:

template class outcome { T _value; EC _error; union { P _payload; E _exception; }; };

I personally really wish that shared_ptr could be constructible from an exception_ptr as they surely are implemented the same way on any standard library implementation I could think of, but at least the above correctly lets payload be any type-erased shared_ptr, which is the correct and appropriate type for returning type-erased payload unlike exception_ptr.

The question now becomes this: surely the Filesystem TS is cleaner if when returning an errored outcome it supplied a shared_ptr> instead of the exception_ptr to the filesystem_error that would have been thrown? Advantages:

1. I would also be fairly sure, without having benchmarked it, that make_shared>(path1, path2) will be many times faster than make_exception_ptr(filesystem_error(path1, path, ec)) on most standard library implementations.

2. make_exception_ptr(filesystem_error(path1, path, ec)) will return a null pointer if C++ exceptions are disabled on at least libstdc++, and thus your proposal would be a problem for those users running with exceptions off.

What do you think?

I will not address your question directly; but let me offer a remark.

It was my understanding that `error_code_extended`, along with its ring buffer usage, was provided to address exactly this problem: provide additional payload to `std::error_code`. The current exercise with adapting the Filesystem TS can be considered a test of the usefulness of `error_code_extended`.

With result<T> which is T|error_code_extended, then yes error_code_extended can be used to optionally transport a *fixed* set of payload. By its nature, you get storage for one short string currently a maximum of 191 characters.

With outcome<T>, we are storing an exception_ptr anyway in this proposed simplified non-variant design, so I am suggesting go ahead and union that with a shared_ptr, that way you can optionally supply arbitrary type-erased payload.

...

Some questions arise that i will surely ask in the second review of Boost.Outcome (I hope we will have one):

1. Can `result` be used to successfully replace error handling in Filesystem TS?

Not entirely. outcome<T> as proposed above can. The problem is the *two* paths which Filesystem may return as part of filesystem_error. And each of those paths could be 16Kb or more.

...

2. If I carry arbitrary payload with a `result` do I need the ring buffer for anything?

Sorry, this misunderstanding is the fault of me cutting out some exposition. Let's do it out in its entirety:

``` template class result { T _value; EC _error; };

template class outcome : public result { union { P _payload; E _exception; }; }; ```

So result<T> retains its trivial copy, move and destruction which is important for optimisation. outcome<T> always forces observable behaviour because of the potential atomics in exception_ptr, so we lose nothing by potentially making the same storage alternatively a shared_ptr.

As mentioned above, the shared_ptr possibility means the Filesystem TS now works equally well with C++ exceptions disabled if that TS used outcome<T>.

Thoughts?

Let's see if I got it correctly. I, the user of your library, have to make the choice: 1. Either I get minimum-overhead (trivially-copyable) `result<T>`, with limited ability to carry payload. 2. Or I get ability to carry any payload, but at the expense of paying the cost of copying `outcome<T>`s. Since filesystem_error already has non-trivial copy, I loose nothing when going with option 2 (using `outcome<T>`). Right?

Gottlob Frege wrote:

So converting a Circle to a Triangle might result in a Rectangle, because Rectangle is the fallback?

If an exception is thrown, yes. That's how the basic guarantee works. "Converting" from a vector with 5 elements to a vector with three elements may result in a vector with zero elements on failure, and "converting" from a Circle to a Circle may result in a Circle that is neither of those two. I argued that variant's assignment should provide the strong guarantee, as the usual idiom of copy+swap that is used for other types doesn't work for variant, but was persuaded that it in fact does, in the final wording. I have to verify whether this holds in my implementation, and under what conditions.

I'd rather have double buffering or valueless_by_exception. ie correctness or acknowledgement of error. Not silent incorrectness.

It's your prerogative to be wrong. :-)

Why not add "throw E if derived from std::exception" as part of that fallback? It is probably what users want.(?)

This would imply that expected<> would be used to return types derived from std::exception, and I don't see this as common. Although with that said, yes, it would be easy to add that as part of the default behavior if there's demand for it. It would open the door to the subsequent question of whether it ought to be possible to override the behavior for a whole hierarchy rooted in a user class in a similar manner, though, which would imply that the default would need to be made a worse match in some way. There are pros and cons for that.

Gottlob Frege wrote:

It just means I need to double buffer higher up. I really want my drawing program to just fail (rollback) the transaction if it can't complete - that is what users really want.

That's true, I understand that, and I tried to communicate the need for variant to support the strong guarantee on assignment for this reason. Normally, if your type was, say, vector<>, you don't need all vectors to be double-buffered. You do vector<> tmp( v ); process( tmp ); v.swap( tmp ); // commit transaction at the places where you want the strong guarantee. The problem with variant<> is that the above doesn't quite produce the strong guarantee when the types aren't nothrow move constructible. I also tried to provide a solution to this problem in the form of pilfering, because types can typically be pilfered even when they can't be nothrow-moved-from, but this didn't go anywhere.

On 05/06/2017 15:23, Gottlob Frege via Boost wrote:

On Mon, Jun 5, 2017 at 7:57 AM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

Some days I'm like "man just accept empty, it would be a simple API", but then I think "it is stupid for variant to be empty". Other days I think "just double buffer when necessary" (I assume that's your direction Peter?), but then I think "I don't want double-buffering variant to go on/off based on whether I use MS std vs libc++ etc" and also "I don't want double buffering for cases that only happen in theory, not in practice".

That is my direction, yes. My variant uses double storage when (1) not all types have noexcept move constructors and (2) there isn't a noexcept default constructible type in the list.

So converting a Circle to a Triangle might result in a Rectangle, because Rectangle is the fallback? I'd rather have double buffering or valueless_by_exception. ie correctness or acknowledgement of error. Not silent incorrectness.

I hate with a passion any possibility of a variant changing its state without me explicitly telling it to do so. The only tolerable state a variant should ever choose on its own is empty/valueless. I repeat my suggestion that for variant2: 1. If at least all types minus one have nothrow move construction, you get the strongest possible never empty never change guarantee. 2. If the user is dumb enough to supply more than one type with a throwing move constructor, they get the current C++ 17 variant valueless by exception semantics i.e. go valueless if changing state threw an exception and the previous state threw an exception on attempt to restore. 3. We thus never double storage. 4. And we never, EVER change state silently to something the user did not explicitly ask for. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

...

I hate with a passion any possibility of a variant changing its state without me explicitly telling it to do so. The only tolerable state a variant should ever choose on its own is empty/valueless.

I repeat my suggestion that for variant2:

1. If at least all types minus one have nothrow move construction, you get the strongest possible never empty never change guarantee.

2. If the user is dumb enough to supply more than one type with a throwing move constructor, they get the current C++ 17 variant valueless by exception semantics i.e. go valueless if changing state threw an exception and the previous state threw an exception on attempt to restore.

3. We thus never double storage.

4. And we never, EVER change state silently to something the user did not explicitly ask for.

I would recommend not mixing the design discussion of `variant2` with the design considerations of `outcome`. `outcome` has its own design constraints: it is natural to have only one type with a potentially throwing move constructor. In contrast, `variant2` cannot make any assumptions or guesses about `Ts...`, so it may need different implementation of the assignment.

I am not mixing the design discussion. I am talking about variant2 only. I am proposing the above mix of strong never-empty never-change guarantees when the user supplies a reasonable set of types. So: 1. If all but one of the types has a nothrow move constructor, you get the strong never-empty never-change guarantee. 2. If more than one of the types has a throwing move constructor, assignment still moves old state onto the stack before attempting to set new state. If, *and only if*, **both** the move of the new state throws and the attempt to restore the old state ALSO throws, then and only then do we enter valueless-by-exception state. My view here is that C++ 17 std::variant already has valueless_by_exception(). My proposal is that it becomes a very rare situation indeed to ever occur, and guaranteed not to occur if the types supplied meet the conditions requested. End user code can test if the never empty guarantee is in place using 'if constexpr(!variant.valueless_by_exception()) ...' because bool valueless_by_exception() would become constexpr false in the implementation with the strong never empty guarantee. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

You are basically saying: provide the implementation that gives me strong guarantee when I meet condition X. ("X" being up to one type with potentially throwing move constructor).

Your expectation is reasonable, but (I think) it is incompatible with other peoples' expectation: provide implementation that gives me never-empty guarantee when I meet condition Y. ("Y" in that case means I have a type with nothrow default constructor.)

I do not think both expectations can be satisfied in one implementation.

I think it daft that a variant require any default constructors at all. My understanding of C++ 17 std::variant<> is that it does not require any of its types to have default constructors. I certainly feel no warmth to the idea of a variant which will default construct to any of its possible states instead of to explicit valueless. If I just lost previous state due to a throw during assignment, I **want** that reflected in the variant state. Much better still of course is that you don't lose me my previous state. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On Tue, Jun 6, 2017 at 5:29 AM, Andrzej Krzemienski via Boost wrote:

You are basically saying: provide the implementation that gives me strong guarantee when I meet condition X. ("X" being up to one type with potentially throwing move constructor).

Your expectation is reasonable, but (I think) it is incompatible with other peoples' expectation: provide implementation that gives me never-empty guarantee when I meet condition Y. ("Y" in that case means I have a type with nothrow default constructor.)

I do not think both expectations can be satisfied in one implementation.

Regards, &rzej;

Agreed. But I don't see much value in the never-empty guarantee if it doesn't give you the strong guarantee. Tony

Andrzej Krzemienski wrote:

...

...

Your expectation is reasonable, but (I think) it is incompatible with other peoples' expectation: provide implementation that gives me never-empty guarantee when I meet condition Y. ("Y" in that case means I have a type with nothrow default constructor.)

It's actually "give me single buffer when Y". The aim was to make the typical variant (that often contains a fallback type by accident) single buffer. Surprising pushback against this one. I'll take this feedback into account.

Gottlob Frege wrote:

...

Agreed. But I don't see much value in the never-empty guarantee if it doesn't give you the strong guarantee.

I'm not sure I understand this fully; could you please explain from what expressions, and under what conditions, you expect the strong guarantee?

variant v1, v2; X x;

v1= v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

Anyone? This is a genuine inquiry. How can I give you strong guarantee if you don't tell me when and where you want it?

On 9/06/2017 01:22, Andrzej Krzemienski wrote:

2017-06-08 15:01 GMT+02:00 Peter Dimov:

...

...

I'm not sure I understand this fully; could you please explain from what expressions, and under what conditions, you expect the strong guarantee?

variant v1, v2; X x;

v1= v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

Anyone? This is a genuine inquiry. How can I give you strong guarantee if you don't tell me when and where you want it?

Maybe nobody needs the strong guarantee? This is definitely the case for my programs. I put variants in different containers. But if exception is thrown while modifying them I am destroying the entire data structure. I do not need the previous state. If I cannot put the new one, I cannot proceed anyway.

(I am not saying such guarantee is useless. I just observe that the need occurred in my programs.)

The only case that I can think of at the moment is for something like a state machine, where each type in the variant is a possible state and the current value is the current state. If a transition fails via throwing from the new state constructor then presumably the state machine should remain in the previous state. Having said that, the proposed implementations of strong guarantee wouldn't work for that case anyway, since that sort of state machine also generally assumes that the current state destructor is run strictly before the new state constructor, and that wouldn't be the case if you stacked a backup in case of exception. (Mind you, it also wouldn't be the case even for basic assignment; they'd have to explicitly assign a none_t or something first to make that happen.) So the right place to handle that guarantee is in the state machine transition machinery, not in variant itself. People generally require that for x = f() then if f() throws x should be unmodified (which is automatic based on how the language works); if it's the assignment itself that throws then things are usually beyond recovery anyway, since assignment is not really expected to throw for any reason other than out of memory.

On 9/06/2017 18:09, degski wrote:

On 9 June 2017 at 02:05, Gavin Lambert wrote:

...

... since assignment is not really expected to throw for any reason other than out of memory.

Are there really any modernish OSes that *do* run out of memory? Stack space (on those type of machines) maybe, but now we are talking programming error, I would say, because that situation is foreseeable (and should be expected) by the programmer. It's not particularly hard for certain application types to run out of memory (or at least address space) on 32-bit platforms.

I suppose you could argue (with some justification) that those are not "modern", but at least in the Windows world it's still fairly common for applications to be 32-bit (even when the OS is almost exclusively 64-bit now) until they actually prove they need more memory. It's also not uncommon for applications to have memory leaks that inevitably end in out of memory errors if left long enough. And some people religiously disable swap, which can have a similar result.

From what I understand, if one is programming for embedded systems or iot-devices, dynamic allocation is not the common way to do things and all memory required is allocated upon boot. Depends on the type of application. For tiny embedded IoT devices, sure. For larger ones that are basically just a PC running a kiosk app, dynamic allocation is not uncommon. There's a broad spectrum in between.

On 9/06/2017 22:39, degski wrote:

...

It's also not uncommon for applications to have memory leaks that inevitably end in out of memory errors if left long enough.

As a former C99-guy, I would consider fixing ones' leaky applications with exceptions to be SB (Suspect Behaviour) by the developer.

There's no "fixing" involved. The simple fact is that copy-assignment will most likely throw std::bad_alloc if memory allocation is required but the available space is exhausted. And this can happen much sooner than 2GB application memory due to heap fragmentation. (In fact on average it seems to occur closer to 1.2GB, although that depends a bit on your typical object size and allocation pattern.) Most developers don't do anything to defend against this because there typically isn't any way to do so -- out of memory is usually application-fatal. As it should be. The point being that since std::bad_alloc is usually the only expected way for operator= to fail (assuming you weren't already in UB territory with a bad pointer or something), I can't think why anyone would have any expectation of exception guarantees for it in general. The exception (pun somewhat intended) to that is if some type explicitly provides additional reasons for assignment to fail -- eg. perhaps a one-time-assignable type or one that can be write-locked at runtime or something like that; but such a type should also clearly advertise what guarantees it provides in that case. The question is then whether something like variant should try to provide those same guarantees or whether (like struct) it doesn't.

2017-06-09 8:09 GMT+02:00 degski via Boost :

On 9 June 2017 at 02:05, Gavin Lambert via Boost wrote:

...

... since assignment is not really expected to throw for any reason other than out of memory.

Are there really any modernish OSes that *do* run out of memory? Stack space (on those type of machines) maybe, but now we are talking programming error, I would say, because that situation is foreseeable (and should be expected) by the programmer. From what I understand, if one is programming for embedded systems or iot-devices, dynamic allocation is not the common way to do things and all memory required is allocated upon boot.

When you are implementing a "cache": you want to store the most "hot" part of your database data in RAM on a dedicated server, which obviously cannot cache the entire database. You have to make some predictions, how much data you can accept. If the actual data is bigger, you probably want to terminate with apologize message. Regards, &rzej;

2017-06-09 12:46 GMT+02:00 degski via Boost :

On 9 June 2017 at 10:29, Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

...

When you are implementing a "cache": you want to store the most "hot" part of your database data in RAM on a dedicated server, which obviously cannot cache the entire database. You have to make some predictions, how much data you can accept. If the actual data is bigger, you probably want to terminate with apologize message.

LOFL: If a software engineer in my company would do what you are suggesting, the only thing that would be terminated is his contract, without apology. Certainly, there must be better ways of doing just that

This was supposed to illustrate that you might observe that you have run out of memory. Not what you do with it later. Regards, &rzej;

On 9 June 2017 at 14:28, Peter Dimov via Boost wrote:

Yes, it's not hard for a 32 bit app to run out of memory (which usually means address space). Not hard at all nowadays.

As a programmer, I *can* detect whether any source code is compiled 32 bit. Now I know I have a possible address space issue. The code should address this issue (no pun intended). Andrzej concluded that only out of memory errors should trigger an exception on assignment. I'm argueing that that should not happen either, because I would consider that a short-coming in the code, just like f.e. non-tail-recursive deep recursive functions are not safe. Life would be so much easier if we had actual Turing-machines. degski -- "*Ihre sogenannte Religion wirkt bloß wie ein Opiat reizend, betäubend, Schmerzen aus Schwäche stillend.*" - Novalis 1798

On 9/06/2017 19:43, Andrzej Krzemienski wrote:

It is funny that you should say that. I remember I wanted at some point to test one of Boost's state machines in my toy projects, and I had the exact opposite expectation: when the the new state B was being initialized I wanted to be able to still access the previous state A, to steal (move) some data from it.

Perhaps we had a similar experience then. I remember running into a problem when perfect forwarding was added to a state machine transition method, because it ended up trying to move some data of the prior state after it had been destroyed (it was explicitly destroying the prior state and then forwarding the parameters to the constructor of the new state, in that order).

Gottlob Frege wrote:

On Wed, Jun 7, 2017 at 10:28 AM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

Agreed. But I don't see much value in the never-empty guarantee if it doesn't give you the strong guarantee.

I'm not sure I understand this fully; could you please explain from what expressions, and under what conditions, you expect the strong guarantee?

variant v1, v2; X x;

v1= v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

How about "all of the above"? At least when X and Y each offer the strong guarantee?

I'm interested in a practical answer, not a theoretically sound one which is of no use. Suppose that X is something that occurs in practice, such as std::vector, not some hypothetical X with a strong assignment, which doesn't. Unless of course you only put types with strong assignment operators into your variants, which in practice confines you to built-ins, in which case all of the above will indeed be not just strong, but nonthrowing as well.

2017-06-12 21:00 GMT+02:00 Gottlob Frege via Boost :

On Mon, Jun 12, 2017 at 1:29 PM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

On Wed, Jun 7, 2017 at 10:28 AM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

Agreed. But I don't see much value in the never-empty guarantee if

it

...

...

...

...

> doesn't give you the strong guarantee.

I'm not sure I understand this fully; could you please explain from what

...

expressions, and under what conditions, you expect the strong guarantee?

variant v1, v2; X x;

v1= v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

How about "all of the above"? At least when X and Y each offer the strong guarantee?

I'm interested in a practical answer, not a theoretically sound one which is of no use. Suppose that X is something that occurs in practice, such as std::vector, not some hypothetical X with a strong assignment, which doesn't.

Unless of course you only put types with strong assignment operators into your variants, which in practice confines you to built-ins, in which case all of the above will indeed be not just strong, but nonthrowing as well.

I think I'm going with Niall's comment some time earlier - to have variant give as strong a guarantee as the types it holds. If X = X (assignment) is only basic, I don't really need variant to somehow magically make variant<X> = variant<X> strong.

Maybe I just don't have enough time to think about this all the way through - your comments about how strong doesn't compose, and using the swap idiom at the correct level are interesting. Are you saying your variant2 fixes swap, but doesn't go full strong, and that that is all we really need?

First of all. I haven't seen anyone give an example where they need a strong guarantee for variant's assignment. There was one attempt with a state machine, but even there it seamed only theoretical. I think the questions if we can implement it are secondary to if anyone really needs it. If whoever has a real (as opposed to theoretical) need for a strong guarantee, please respond. Regards, &rzej;

First theoretical, then more real-ish: I have a drawing program that draws rectangles, circles, triangles. I want to change a circle to a triangle. The user expects it to work, or a reason why it failed. They don't want the circle to become a rectangle. More realistic, but exactly the same: In my codebase of projection mapping (multiple projectors projecting a seamless image onto a surface) we have different screen types - flat, cylinder, spherical, and custom (mesh file). Each type comes with a bunch of settings, so it isn't just a enum, each is a separate type. The user can select which they want from a drop-down. This is currently _not_ using a variant, but I think it should. However, when the user switches from cylinder to "custom" and the mesh is invalid or out of memory, etc, I don't want the cylinder to turn into a flat screen. I haven't coded it yet, but I think it will come down to a strong assignment guarantee (or I handle it myself elsewhere). Tony On Tue, Jun 13, 2017 at 3:31 AM, Andrzej Krzemienski via Boost wrote:

2017-06-12 21:00 GMT+02:00 Gottlob Frege via Boost :

...

On Mon, Jun 12, 2017 at 1:29 PM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

On Wed, Jun 7, 2017 at 10:28 AM, Peter Dimov via Boost wrote:

...

Gottlob Frege wrote:

...

Agreed. But I don't see much value in the never-empty guarantee if

it

...

...

...

>> doesn't give you the strong guarantee.

I'm not sure I understand this fully; could you please explain from what

...

expressions, and under what conditions, you expect the strong guarantee?

variant v1, v2; X x;

v1= v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

How about "all of the above"? At least when X and Y each offer the strong guarantee?

I'm interested in a practical answer, not a theoretically sound one which is of no use. Suppose that X is something that occurs in practice, such as std::vector, not some hypothetical X with a strong assignment, which doesn't.

Unless of course you only put types with strong assignment operators into your variants, which in practice confines you to built-ins, in which case all of the above will indeed be not just strong, but nonthrowing as well.

I think I'm going with Niall's comment some time earlier - to have variant give as strong a guarantee as the types it holds. If X = X (assignment) is only basic, I don't really need variant to somehow magically make variant<X> = variant<X> strong.

Maybe I just don't have enough time to think about this all the way through - your comments about how strong doesn't compose, and using the swap idiom at the correct level are interesting. Are you saying your variant2 fixes swap, but doesn't go full strong, and that that is all we really need?

First of all. I haven't seen anyone give an example where they need a strong guarantee for variant's assignment. There was one attempt with a state machine, but even there it seamed only theoretical. I think the questions if we can implement it are secondary to if anyone really needs it.

If whoever has a real (as opposed to theoretical) need for a strong guarantee, please respond.

Regards, &rzej;

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Jun 13, 2017 at 11:48 AM, Andrzej Krzemienski via Boost wrote:

2017-06-13 17:22 GMT+02:00 Gottlob Frege via Boost :

...

First theoretical, then more real-ish:

Thank you. It is easier to think about real problems.

...

I have a drawing program that draws rectangles, circles, triangles. I want to change a circle to a triangle. The user expects it to work, or a reason why it failed. They don't want the circle to become a rectangle.

More realistic, but exactly the same:

In my codebase of projection mapping (multiple projectors projecting a seamless image onto a surface) we have different screen types - flat, cylinder, spherical, and custom (mesh file). Each type comes with a bunch of settings, so it isn't just a enum, each is a separate type. The user can select which they want from a drop-down.

Does any of this types throw upon move construction?

...

This is currently _not_ using a variant, but I think it should. However, when the user switches from cylinder to "custom" and the mesh is invalid

At which point is it determined that the mesh is invalid? My point with this is that in the assignment of the form:

variant = Mesh{params...};

If constructing a mesh fails (because it would be invalid), no assignment is even attempted.

Probably true.

...

or out of memory,

But what does it mean that you get out-of-memory situation when a user tries to change the projection? Will you be able to recover from it other than resetting everything?

Well, I find there tends to be two kinds of "out of memory": 1. large allocations - Images, etc. - can't open giant image file because you need gigs of memory 2. small allocations - C++ objects - can't allocate ~ 100s of bytes. The first is "normal" and a program needs to handle that. The second means you are screwed. A Mesh actually falls into the 1st category. Of course that also means that you tend not to copy Mesh objects, you move them. But unless Mesh was made move-only, someone will likely inadvertently copy instead of moving. (I do tend to make things like this move-only, then add an explicit copy() function, but that doesn't always happen.) This is basically why I'm satisfied with std::variant - move should never throw. If it does, it was a tiny allocation, and you were screwed anyhow. No one should have a move that does a large allocation. So in my world, std::variant already has the never-empty (and never valueless_by_exception) guarantee. So std::variant made some pragmatic choices, and my code makes some pragmatic choices, but if we want "perfection", that would be nice, and the examples above are as realistic as I can give. Tony

...

etc, I don't want the cylinder to turn into a flat screen.

Agreed.

Regards, &rzej;

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Jun 13, 2017 at 1:07 PM, Peter Dimov via Boost wrote:

Gottlob Frege wrote:

...

This is basically why I'm satisfied with std::variant - move should never throw. If it does, it was a tiny allocation, and you were screwed anyhow. No one should have a move that does a large allocation.

So in my world, std::variant already has the never-empty (and never valueless_by_exception) guarantee.

Why are you then so insisting on the strong guarantee? It's only relevant when an operation throws.

I'm happy with std::variant. It makes some trade-offs, but I can live with them. But once someone tries to make a variant with less trade-offs, it seems to me you should just go all the way - no trade-offs. I can and will live with trade-offs, but who wouldn't like no trade-offs? Basically, I was surprised that there was something between std::variant and no-compromises-variant. It seems you want to explore that area - _different_ trade-offs, or just _less_ trade-offs. But you are approaching zero, I think you should just do zero. I could easily be wrong. Does that make sense? Tony

2017-06-14 0:21 GMT+02:00 Peter Dimov via Boost :

Gottlob Frege wrote:

I'm happy with std::variant. It makes some trade-offs, but I can live

...

with them. But once someone tries to make a variant with less trade-offs, it seems to me you should just go all the way - no trade-offs. I can and will live with trade-offs, but who wouldn't like no trade-offs?

Basically, I was surprised that there was something between std::variant and no-compromises-variant. It seems you want to explore that area - _different_ trade-offs, or just _less_ trade-offs. But you are approaching zero, I think you should just do zero. I could easily be wrong.

All right.

My current iteration is "zero" unless you specifically opt into nonzero by placing the special type "valueless" as a first alternative.

For the example use cases in my previous message,

variant v1, v2;

...

X x;

v1 = v2; // do you expect strong guarantee here? v1 = std::move(v2); // here? v1 = x; // here? v1 = std::move(x); // here? v1.emplace<X>(); // here?

where X is not the type 'valueless', this would translate to:

v1 = v2; // as strong as X::op=(X const&) v1 = std::move(v2); // as strong as X::op=(X&&)

v1 = x; // as strong as X::op=(X const&) v1 = std::move(x); // as strong as X::op=(X&&)

v1.emplace<X>(); // always strong

So, you mean -- unless I am using the special "valueless" type -- all assignments/emplacements/swaps are strong (possibly at the expense of double buffering in some cases)? That is, there is no "get C upon unsuccessful assignment of A to B"? Regards, &rzej;

2017-06-14 15:27 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

So, you mean -- unless I am using the special "valueless" type -- all

...

assignments/emplacements/swaps are strong (possibly at the expense of double buffering in some cases)? That is, there is no "get C upon unsuccessful assignment of A to B"?

The only situation in which you get a C is when C is the special `valueless` type, yes.

This makes sense. If I ask for it, I get it.

This does not mean that all assignments are strong. In

variant v1, v2;

v1 = v2;

the assignment is basic, because that's the guarantee vector<string>::operator= gives.

Interesting. This satisfies the expectation that a number of people have expressed. But on the other hand, now that you are doing the double buffering anyway, making the assignment strong would come almost fo free. If we continue the example with projections: if I change from the drop down menu from one mesh projection to another mesh projection, they are different projections, but can be encoded in the same type; maybe we should get the strong guarantee. If not in the assignment, then perhaps in a dedicated function. Regards, &rzej;

2017-06-14 18:06 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

...

...

This does not mean that all assignments are strong. In

variant v1, v2;

v1 = v2;

the assignment is basic, because that's the guarantee > vector<string>::operator= gives.

Interesting. This satisfies the expectation that a number of people have expressed.

Exactly. For me basic is basic, regardless of whether the type stays the same, but apparently this does not match people's intuitive expectations.

But on the other hand, now that you are doing the double buffering

...

anyway, making the assignment strong would come almost fo free.

Not free here because the above variant doesn't double-buffer (because all alternatives are nothrow move constructible).

Ok, I see.

If we continue the example with projections: if I change from the drop

...

down menu from one mesh projection to another mesh projection, they are different projections, but can be encoded in the same type; maybe we should get the strong guarantee. If not in the assignment, then perhaps in a dedicated function.

You get the strong guarantee here in move assignment (because vector's move assignment doesn't throw (... except for unequal allocators ...)):

v1 = std::move(v2); // strong v1 = variant(v2); // strong

v1 = vector<string>{ "s1", "s2" }; // strong

and in emplace, because it doesn't use assignment:

v1.emplace( ... ); // strong too

If you are not doing double buffering, how can you guarantee that `emplace` is strong? You move the original to the side? Regards, &rzej;

2017-06-14 18:21 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

If you are not doing double buffering, how can you guarantee that

...

`emplace` is strong? You move the original to the side?

I could do that, but I construct on the side instead and move it into place. This is another instance where people may disagree because emplace should as a principle always construct in-place. But realistically, construct on the side + move is never inferior to move original to the side, construct in-place, optionally move original back if constructor throws.

Given that you have to move something anyway, I guess you are right. Now I realize this all means the types inside variant need to be movable. This will not be a problem in most of the cases, but does it mean I cannot use variant2 with non-movable types? Regards, &rzej;

2017-06-14 19:10 GMT+02:00 Peter Dimov via Boost :

Andrzej Krzemienski wrote:

Now I realize this all means the types inside variant need to be movable.

...

This will not be a problem in most of the cases, but does it mean I cannot use variant2 with non-movable types?

You can, it will double-buffer to keep emplace strong. Or you can use valueless.

All types nothrow move constructible -> single buffer;

Otherwise, first alternative is valueless -> single buffer;

Otherwise -> double buffer.

So, If I got all this right: * All types nothrow move constructible -> single buffer, requires MoveConstructible; * Otherwise, first alternative is valueless -> single buffer (no MoveConstructible); * Otherwise -> double buffer (no MoveConstructible). Is that correct? Some other thoughts: When valueless is required as the first type, it also implies that the default constructor of variant puts it into valueless state, which some might find useful, but others inferior to std::variant. Would it be possible to also assign a special meaning to putting `valueless` at the end, which would assign semantics of std::variant? Maybe the choice how you want to implement the assignment should be explicitly controlled by the users (in form of a policy)? You could still provide the defaults based on properties of the types, but it looks to me sometimes people might want to override the defaults. For instance, we have seen examples where I want strong guarantee on assignment even when T does not offer one. Or to put it differently, it may be that this double buffering may be useful on its own, even outside the variant. Regards, &rzej;

Andrzej Krzemienski wrote:

Some other thoughts:

When valueless is required as the first type, it also implies that the default constructor of variant puts it into valueless state, which some might find useful, but others inferior to std::variant. Would it be possible to also assign a special meaning to putting `valueless` at the end, which would assign semantics of std::variant?

`valueless` is proving to be a nuisance. I have for instance expected using variant under the hood. But this means that expected falls into the variant special case and it really shouldn't. So either I ignore this or would have to replace valueless internally with something else. So, yes, I did consider putting it at the end instead. But then it won't default construct to valueless, and you may actually want that. So we're going into variant then, which isn't what I'd call elegant. Maybe the right thing to do is just drop valueless and point people to std::variant if they need it. My ambition was to produce a superset of std::variant, but this may not be the right choice design-wise.

Maybe the choice how you want to implement the assignment should be explicitly controlled by the users (in form of a policy)? You could still provide the defaults based on properties of the types, but it looks to me sometimes people might want to override the defaults. For instance, we have seen examples where I want strong guarantee on assignment even when T does not offer one.

You use emplace then, or move assignment. Or swap, although I need to double-check whether it does what it needs to do in all the various cases.

Niall Douglas wrote:

I hate with a passion any possibility of a variant changing its state without me explicitly telling it to do so. The only tolerable state a variant should ever choose on its own is empty/valueless.

I wonder whether the people who insist on the strong guarantee on assignment realize that when the variant holds a vector, and you assign it a vector, assignment delegates to vector<>::operator= and as a result you get the basic guarantee because that's what all standard types provide. That is, the "variant state" - by which you probably mean the index - doesn't change but the variant state can change into holding a vector that is neither the old nor the new one. It's possible to define a variant that never delegates to the assignment operator of the contained type. I've also argued for this possibility, because it makes it possible for one to put non-assignable types into variant (and optional, if it's changed to match), and still get an assignable variant. Nevertheless, this is not the case today, and I suspect that you don't realize that the strong guarantee requires this change.

1. If at least all types minus one have nothrow move construction, you get the strongest possible never empty never change guarantee.

What do you do on emplace(), when the variant already contains 'throwing_move'? And what do you do on operator=( throwing_move ), when the variant already contains 'throwing_move' and throwing_move::operator= doesn't provide the strong guarantee?

Niall Douglas wrote:

...

I wonder whether the people who insist on the strong guarantee on assignment realize that when the variant holds a vector, and you assign it a vector, assignment delegates to vector<>::operator= and as a result you get the basic guarantee because that's what all standard types provide.

I'd expect nothing else. No state change means you always pass through to operator=().

I don't understand of what value is a guarantee that is strong when you change type, but basic if you don't, but perhaps I'm missing something here.

Niall Douglas wrote: On 06/06/2017 12:14, Peter Dimov via Boost wrote:

...

I don't understand of what value is a guarantee that is strong when you change type, but basic if you don't, but perhaps I'm missing something here.

My opinion would be that most users would expect a variant object to not be involved with the type being worked with, certainly not overriding its implementation. If the variant's state is pointing to type Foo, and I assign to the variant, it's 100% onto Foo's assignment operators what happens next, including any throwing or whatever guarantees.

That would mean the basic guarantee, for nearly all types in existence for which the assignment can fail. Types don't generally provide the strong guarantee on assignment as it doesn't compose. If type T has strong assign, type U has strong assign, struct { T t; U u; } no longer does. So it's more efficient to just do basic everywhere, which does compose, and then at the specific point you want the strong guarantee, use f.ex. the copy and swap trick to get it.

4. I do NOT want any additional runtime overhead whatsoever if all the variant's types have nothrow move constructors and nothrow destructors.

That's the easy case, but if you demand the same for N-1, things are more convoluted. :-)

Andrzej Krzemienski wrote:

2017-06-06 14:20 GMT+02:00 Peter Dimov via Boost :

...

Types don't generally provide the strong guarantee on assignment as it doesn't compose. If type T has strong assign, type U has strong assign, struct { T t; U u; } no longer does. So it's more efficient to just do basic everywhere, which does compose, and then at the specific point you want the strong guarantee, use f.ex. the copy and swap trick to get it.

I think the reasoning behind Niall's position is that a type `X` can have a custom guarantee: upon its assignment it guarantees that after a throw the type is either unchanged or goes to fallback state `0`. This guarantee is stronger than basic, and weaker than strong.

Types _could_ provide such a guarantee, but don't, for the same reason they don't provide strong - as outlined in the quoted paragraph above, if types X and Y provide it, struct { X x; Y y; } no longer does. So it's additional work, and buys nothing much.

Le 05/06/2017 à 13:57, Peter Dimov via Boost a écrit :

Gottlob Frege wrote:

...

Some days I'm like "man just accept empty, it would be a simple API", but then I think "it is stupid for variant to be empty". Other days I think "just double buffer when necessary" (I assume that's your direction Peter?), but then I think "I don't want double-buffering variant to go on/off based on whether I use MS std vs libc++ etc" and also "I don't want double buffering for cases that only happen in theory, not in practice".

That is my direction, yes. My variant uses double storage when (1) not all types have noexcept move constructors and (2) there isn't a noexcept default constructible type in the list.

I think that in practice few variants will hit the double case, as it's rare to have variant without a scalar alternative, although who knows.

For variant specifically, when going from libstdc++ to MS STL the difference is that sizeof(variant) changes, but it changes for std::variant, too. Obviously, double storage can never be as good as single storage, but I view this as an acceptable compromise. You can guarantee single storage by putting a nothrow default constructible type in the list of alternatives. Alternatively, we could as well add a phantom type that says, please, use double buffering if absolutely needed.

...

- I think std::expected should always throw something deriving from std::exception. So if E derives from std::exception, we can throw it. If E is exception_ptr we can (re)throw it. If E is error_code/system_error/etc we can figure out what to throw. If E is user-type, then we wrap it in bad_expected or whatever.

My suggested expected<> just calls `throw_on_unexpected(e)` unqualified (it's a customization point.) There are overloads for std::error_code and std::exception_ptr and the fallback default is to throw bad_expected_access<E>.

The current proposal (not necessary accepted) throw always bad_expected_access. If we can throw different exceptions, I believe that we should have as well a type_trait that gives the exception thrown by an error. This could be useful for generic code. I have not a concrete use case in mind yet. Vicente

...

I always wished that ASIO made the const error_code& passed in non-const, and you could set it on return from the handler. You can still throw an exception of course to abort everything, but sometimes you don't want to abort everything. Sometimes you just want to fail.

What kind of errors do you want to return to Asio, and how do you want it to respond to them?

Right now if you throw an exception from within a handler the default action is to throw out the thread calling ioservice.run() which ran the handler. This isn't ideal - usually one would prefer failure to pop out of the initiating function instead. You can implement this manually using a custom async_result or easier, using future-promise, but I've always wished there was a lightweight non-allocating default propagation of handler failure => error_code => initiating function. So a super lightweight, non-allocating future-promise like object which can solely transport an error_code would be the default, that way end users can do .get() on the initiating function's return if they care, if not throw it away. Anyway, I figure that ship has sailed unfortunately. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/