[outcome] To variant, or not to variant?

newer
[outcome] expected, etc. why are...

Gavin Lambert

31 May 2017 31 May '17

8:26 a.m.

I probably should have thought of asking this earlier, but it occurs to me now that my own mental model of how an "outcome"-ish type should act is probably not suited to variant storage at all. So just out of curiosity I thought I'd ask whether people prefer this sort of interface: template<typename T> class outcome { public: bool has_value() const { return m_storage.has<T>(); } T& value() { return m_storage.get<T>(); } bool has_error() const { return m_storage.has(); } error_code& error() { return m_storage.get(); } bool has_exception() const { return m_storage.has(); } exception_ptr& exception() { return m_storage.get(); } void set(none_t) { m_storage = none; } void set(const T& val) { m_storage = val; } void set(error_code err) { m_storage = err; } void set(exception_ptr ep) { m_storage = ep; } private: variant m_storage; }; Or this sort of interface: template<typename T> class outcome { public: bool has_value() const { return !!m_value; } T& value() { ensure(); return m_value.value(); } //bool has_error() const { return !!m_error; } const error_code& error() const { return m_error; } //bool has_exception() const { return !!m_exception; } const exception_ptr& exception() const { return m_exception; } void ensure() const { if (m_exception) { rethrow_exception(m_exception); } if (m_error) { throw system_error(m_error); } } void set(none_t) { m_value = none; m_error = error_code(); m_exception = nullptr; } void set(const T& val) { m_value = val; m_error = error_code(); m_exception = nullptr; } void set(error_code err) { m_value = none; m_error = err; m_exception = nullptr; } void set(exception_ptr ep) { m_value = none; m_error = ep ? error_code(errc::has_exception) : error_code(); m_exception = ep; } private: optional<T> m_value; error_code m_error; exception_ptr m_exception; }; (Don't get too hung up on the specifics. This is a sketch, not a real implementation. I've obviously omitted things that a real implementation would need such as more const methods and move support, and construction and assignment rather than using set methods. The focus is on variant vs. non-variant.) Anyway, the point is that this could actually transport multiple things; in particular as above both an error_code and an exception_ptr; perhaps set(error_code) could construct an exception as well, although I've explained elsewhere why I don't like that option. Perhaps this could even be exposed even more so that user code could explicitly provide both a value and an error -- think "here's a value, but it was truncated" or "here's the 5 values you asked for, but there's more", both fairly common in system APIs. Or for things like ambiguous matches where you still return the most likely candidate. Or dictionary insertion (duplicate key, but here's the value that's already there). Or many more such examples. (Though obviously in this case value() couldn't call ensure(). But that should make some people happy.) Also note has_error() and has_exception() don't even need to be provided in this version since you can always call error() and exception() in a boolean context anyway, with the same result. (Or actually a slightly better result than the implementation shown here.) The obvious downside is that it now uses more storage, which might hinder some inlining cases.

Show replies by date

Andrzej Krzemienski

31 May 31 May

8:44 a.m.

2017-05-31 10:26 GMT+02:00 Gavin Lambert via Boost :

...

In your mental model what is the interpretation of the situation when you have both an error_code and an exception_ptr? Regards, &rzej;

Gavin Lambert

11:06 p.m.

On 31/05/2017 20:44, Andrzej Krzemienski wrote:

...

Mostly where error() returns the error_code and exception() returns the system_error(error()), so that the caller could choose to treat it either way, as you'd expect. Conceivably there could be cases where someone might want to have a different type of exception (eg. errc::invalid_argument plus std::out_of_range or a derived type, which might convey some additional information), although that could also be a can of worms best left unopened.

Andrzej Krzemienski

1 Jun 1 Jun

7:29 a.m.

2017-06-01 1:06 GMT+02:00 Gavin Lambert via Boost :

...

So, are you saying that exception() and error() observe the same "failure report", just in two different ways? Regards, &rzej;

Gavin Lambert

8:25 a.m.

On 1/06/2017 19:29, Andrzej Krzemienski wrote:

...

Of course. It's a single object containing the single outcome of a method call. There might be multiple related sub-states but it's still a single event with a single result.

Vicente J. Botet Escriba

31 May 31 May

10:31 a.m.

Le 31/05/2017 à 10:26, Gavin Lambert via Boost a écrit :

...

I probably should have thought of asking this earlier, but it occurs to me now that my own mental model of how an "outcome"-ish type should act is probably not suited to variant storage at all.

So just out of curiosity I thought I'd ask whether people prefer this sort of interface:

template<typename T> class outcome { public: bool has_value() const { return m_storage.has<T>(); } T& value() { return m_storage.get<T>(); }

bool has_error() const { return m_storage.has(); } error_code& error() { return m_storage.get(); }

bool has_exception() const { return m_storage.has(); } exception_ptr& exception() { return m_storage.get(); }

void set(none_t) { m_storage = none; } void set(const T& val) { m_storage = val; } void set(error_code err) { m_storage = err; } void set(exception_ptr ep) { m_storage = ep; }

private: variant m_storage; };

Does none_t mean success or failure? For what I see it means failure as it is not the result of value(). In addition it default construct to none_t. The mental model for me is variant> m_storage; I know it is the same and your representation is more efficient.

...

Or this sort of interface:

template<typename T> class outcome { public: bool has_value() const { return !!m_value; } T& value() { ensure(); return m_value.value(); }

//bool has_error() const { return !!m_error; } const error_code& error() const { return m_error; }

//bool has_exception() const { return !!m_exception; } const exception_ptr& exception() const { return m_exception; }

void ensure() const { if (m_exception) { rethrow_exception(m_exception); } if (m_error) { throw system_error(m_error); } }

void set(none_t) { m_value = none; m_error = error_code(); m_exception = nullptr; } void set(const T& val) { m_value = val; m_error = error_code(); m_exception = nullptr; } void set(error_code err) { m_value = none; m_error = err; m_exception = nullptr; } void set(exception_ptr ep) { m_value = none; m_error = ep ? error_code(errc::has_exception) : error_code(); m_exception = ep; }

private: optional<T> m_value; error_code m_error; exception_ptr m_exception; };

I guess you want a variant here. You don't want to store all of them, isn't it? I could understand varaint>

...

(Don't get too hung up on the specifics. This is a sketch, not a real implementation. I've obviously omitted things that a real implementation would need such as more const methods and move support, and construction and assignment rather than using set methods. The focus is on variant vs. non-variant.)

Anyway, the point is that this could actually transport multiple things; in particular as above both an error_code and an exception_ptr; perhaps set(error_code) could construct an exception as well, although I've explained elsewhere why I don't like that option.

Perhaps this could even be exposed even more so that user code could explicitly provide both a value and an error -- think "here's a value, but it was truncated" or "here's the 5 values you asked for, but there's more", both fairly common in system APIs. Or for things like ambiguous matches where you still return the most likely candidate. Or dictionary insertion (duplicate key, but here's the value that's already there). Or many more such examples. (Though obviously in this case value() couldn't call ensure(). But that should make some people happy.)

This corresponds to the status_value [1] model where you store the reason why you don't have a value or why you have it and an optional<value>. As described by L. Crowl in [2], there are use cases for status_value, expected and exceptions. pair> Where we have a value of error_code to mean success.

...

Also note has_error() and has_exception() don't even need to be provided in this version since you can always call error() and exception() in a boolean context anyway, with the same result. (Or actually a slightly better result than the implementation shown here.)

I don't see why we need to choose.between the narrow and the wide functions yet. First you need to state what is your mental model. The provide the operations.

...

The obvious downside is that it now uses more storage, which might hinder some inlining cases.

I'm not sure of this possible missing optimization. Best, Vicente [1] P0262R1 A Class for Status and Optional Value http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0262r1.html [2] P0157R0 Handling Disappointment in C++ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0157r0.html

Niall Douglas

1:15 p.m.

On 31/05/2017 11:31, Vicente J. Botet Escriba via Boost wrote:

...

Surely it is actually: variant std::monostate at the front of the typelist gives std::variant default-construct-to-empty semantics.

...

Personally speaking, I have not found Lawrence's status_value proposal sufficiently value-adding over returning a std::pair to be worth implementing. Also, the STL already uses std::pair throughout for status returns, it's the convention, though I will agree to use it is mildly clunky e.g. unordered_map::insert().first. Now, that said, if expected and status_value could be combined into a single object, that I can see significant value in. It's something I've often pondered for Outcome as well because it could, if done right, let me eliminate error_code_extended. The main thing which has stopped me is the potential confusion. For example, right now we have result<T> which can be empty|T|error_code_extended. If I instead made: template class result; Now result could be (empty|T|std::error_code U Payload). But if this review to date with 600+ emails has been complicated, imagine a review of such a payload-carrying result object? std::error_code + payload is obvious. But, what does an empty state + payload mean? I have no idea. Indeed, what does a T state + payload mean? I guess that is Lawrence's status_value<> use case, but the problem is that the Payload type is fixed between variant states of empty|T|std::error_code and that surely is not particularly useful, you'd want different payload types with each of T or std::error_code. That's why I didn't implement it. Still, I'd be interested in what people think. The other option is empty|T|status_value, that would make more sense, but at the potential cost of bloating the stack significantly which could surprise end users in a way error_code_extended can not. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Vicente J. Botet Escriba

5:06 p.m.

...

On 31/05/2017 11:31, Vicente J. Botet Escriba via Boost wrote:

...
Does none_t mean success or failure? For what I see it means failure as it is not the result of value(). In addition it default construct to none_t.

The mental model for me is variant> m_storage; Surely it is actually:

variant

std::monostate at the front of the typelist gives std::variant default-construct-to-empty semantics.

Le 31/05/2017 à 15:15, Niall Douglas via Boost a écrit : optional<T> also give you that :)

...

...
...
Perhaps this could even be exposed even more so that user code could explicitly provide both a value and an error -- think "here's a value, but it was truncated" or "here's the 5 values you asked for, but there's more", both fairly common in system APIs. Or for things like ambiguous matches where you still return the most likely candidate. Or dictionary insertion (duplicate key, but here's the value that's already there). Or many more such examples. (Though obviously in this case value() couldn't call ensure(). But that should make some people happy.)

This corresponds to the status_value [1] model where you store the reason why you don't have a value or why you have it and an optional<value>. As described by L. Crowl in [2], there are use cases for status_value, expected and exceptions. Personally speaking, I have not found Lawrence's status_value proposal sufficiently value-adding over returning a std::pair to be worth implementing. Also, the STL already uses std::pair throughout for status returns, it's the convention, though I will agree to use it is mildly clunky e.g. unordered_map::insert().first.

Things are changing in the standard. We want more explicit types. status_value is a very good class that responds to real cases.

...

Now, that said, if expected and status_value could be combined into a single object, that I can see significant value in.

Why and how do you want to combine them. One is a sum type the other a product type. These classes are useful on its own use cases.

...

It's something I've often pondered for Outcome as well because it could, if done right, let me eliminate error_code_extended.

The main thing which has stopped me is the potential confusion. For example, right now we have result<T> which can be empty|T|error_code_extended. If I instead made:

template class result;

Now result could be (empty|T|std::error_code U Payload). But if this review to date with 600+ emails has been complicated, imagine a review of such a payload-carrying result object?

std::error_code + payload is obvious. But, what does an empty state + payload mean? I have no idea.

Indeed, what does a T state + payload mean? I guess that is Lawrence's status_value<> use case, but the problem is that the Payload type is fixed between variant states of empty|T|std::error_code and that surely is not particularly useful, you'd want different payload types with each of T or std::error_code. That's why I didn't implement it. You lost me.

Still, I'd be interested in what people think. The other option is empty|T|status_value, that would make more sense, but at the potential cost of bloating the stack significantly which could surprise end users in a way error_code_extended can not.

You lost me definitely. status_value is a product type because we need some additional information even when the operation succeeds. The status is there to convey this information, but conveys also the information explaining why the result is not there. status_value could be seen as pair, optional<T>> There is an invariant Success <=> optional present and Failure <=> optional not present. You could as well see it as variant, Failure> or expected, Failure> Vicente

Andrzej Krzemienski

1 Jun 1 Jun

7:26 a.m.

2017-05-31 19:06 GMT+02:00 Vicente J. Botet Escriba via Boost < boost@lists.boost.org>:

...

Le 31/05/2017 à 15:15, Niall Douglas via Boost a écrit :

...
On 31/05/2017 11:31, Vicente J. Botet Escriba via Boost wrote:

...
Does none_t mean success or failure? For what I see it means failure as it is not the result of value(). In addition it default construct to none_t.

The mental model for me is variant> m_storage;

Surely it is actually:

variant

std::monostate at the front of the typelist gives std::variant default-construct-to-empty semantics.

optional<T> also give you that :)

...
Perhaps this could even be exposed even more so that user code could

...
...
explicitly provide both a value and an error -- think "here's a value, but it was truncated" or "here's the 5 values you asked for, but there's more", both fairly common in system APIs. Or for things like ambiguous matches where you still return the most likely candidate. Or dictionary insertion (duplicate key, but here's the value that's already there). Or many more such examples. (Though obviously in this case value() couldn't call ensure(). But that should make some people happy.)

This corresponds to the status_value [1] model where you store the reason why you don't have a value or why you have it and an optional<value>. As described by L. Crowl in [2], there are use cases for status_value, expected and exceptions.

Personally speaking, I have not found Lawrence's status_value proposal sufficiently value-adding over returning a std::pair to be worth implementing. Also, the STL already uses std::pair throughout for status returns, it's the convention, though I will agree to use it is mildly clunky e.g. unordered_map::insert().first.

Things are changing in the standard. We want more explicit types. status_value is a very good class that responds to real cases.

...
Now, that said, if expected and status_value could be combined into a single object, that I can see significant value in.

Why and how do you want to combine them. One is a sum type the other a product type. These classes are useful on its own use cases.

...
It's something I've often pondered for Outcome as well because it could, if done right, let me eliminate error_code_extended.

The main thing which has stopped me is the potential confusion. For example, right now we have result<T> which can be empty|T|error_code_extended. If I instead made:

template class result;

Now result could be (empty|T|std::error_code U Payload). But if this review to date with 600+ emails has been complicated, imagine a review of such a payload-carrying result object?

std::error_code + payload is obvious. But, what does an empty state + payload mean? I have no idea.

Indeed, what does a T state + payload mean? I guess that is Lawrence's status_value<> use case, but the problem is that the Payload type is fixed between variant states of empty|T|std::error_code and that surely is not particularly useful, you'd want different payload types with each of T or std::error_code. That's why I didn't implement it.

You lost me.

...
Still, I'd be interested in what people think. The other option is empty|T|status_value, that would make more sense, but at the potential cost of bloating the stack significantly which could surprise end users in a way error_code_extended can not.

You lost me definitely.

status_value is a product type because we need some additional information even when the operation succeeds. The status is there to convey this information, but conveys also the information explaining why the result is not there.

status_value could be seen as pair, optional<T>>

There is an invariant Success <=> optional present and Failure <=> optional not present.

You could as well see it as variant, Failure> or expected, Failure>

What the OP means is that status_value can bee seen as a "superset" of expected. It can cover all cases `expected` covers plus more. Of course, this "more" is not necessarily better or harmless. Regards, &rzej;

Vicente J. Botet Escriba

5:20 p.m.

Le 01/06/2017 à 09:26, Andrzej Krzemienski via Boost a écrit :

...

2017-05-31 19:06 GMT+02:00 Vicente J. Botet Escriba via Boost < boost@lists.boost.org>:

...
Le 31/05/2017 à 15:15, Niall Douglas via Boost a écrit :

...
On 31/05/2017 11:31, Vicente J. Botet Escriba via Boost wrote:

It's something I've often pondered for Outcome as well because it could, if done right, let me eliminate error_code_extended.

The main thing which has stopped me is the potential confusion. For example, right now we have result<T> which can be empty|T|error_code_extended. If I instead made:

template class result;

Now result could be (empty|T|std::error_code U Payload). But if this review to date with 600+ emails has been complicated, imagine a review of such a payload-carrying result object?

std::error_code + payload is obvious. But, what does an empty state + payload mean? I have no idea.

Indeed, what does a T state + payload mean? I guess that is Lawrence's status_value<> use case, but the problem is that the Payload type is fixed between variant states of empty|T|std::error_code and that surely is not particularly useful, you'd want different payload types with each of T or std::error_code. That's why I didn't implement it.

You lost me.

...
Still, I'd be interested in what people think. The other option is empty|T|status_value, that would make more sense, but at the potential cost of bloating the stack significantly which could surprise end users in a way error_code_extended can not.

You lost me definitely. status_value is a product type because we need some additional information even when the operation succeeds. The status is there to convey this information, but conveys also the information explaining why the result is not there.

status_value could be seen as pair, optional<T>>

There is an invariant Success <=> optional present and Failure <=> optional not present.

You could as well see it as variant, Failure> or expected, Failure>

What the OP means is that status_value can bee seen as a "superset" of expected. It can cover all cases `expected` covers plus more. Of course, this "more" is not necessarily better or harmless.

I will not say is a super set as expected, Failure> could be used instead of status_value. I will say just it provides a different interface and can be more adapted to people that use to use status with positive and negative meaning. Vicente

Gavin Lambert

31 May 31 May

11:15 p.m.

On 31/05/2017 22:31, Vicente J. Botet Escriba wrote:

...

Does none_t mean success or failure?

Either; that's up to the method in question. It also wasn't really the focus of the question so you can pretend it doesn't exist if that makes you happier.

...

For what I see it means failure as it is not the result of value().

That was a consequence of trying to keep it returning by reference as some people seem to prefer. If it returned by value then it could return an optional<T>, which might be better.

...

In addition it default construct to none_t.

That was intentional.

...

...
private: optional<T> m_value; error_code m_error; exception_ptr m_exception; };

I guess you want a variant here. You don't want to store all of them, isn't it?

No, that was the entire point: to store all of them, and not as a variant.

...

I could understand

varaint>

Nested variants are utterly pointless. In an ideal world variant would automatically collapse nested sub-variants to reduce storage, although that does unfortunately complicate return-by-reference semantics.

...

This corresponds to the status_value [1] model where you store the reason why you don't have a value or why you have it and an optional<value>. As described by L. Crowl in [2], there are use cases for status_value, expected and exceptions.

pair>

Where we have a value of error_code to mean success.

That's closer to my second suggestion, yes, although I think using pair<> is a disservice. I also don't think error_code and exception_ptr are necessarily exclusive.

Daniela Engert

10:59 a.m.

Am 31.05.2017 um 10:26 schrieb Gavin Lambert via Boost:

...

This is pretty much what I expect from an 'outcome' vocabulary type. In my mental model, a 'none_t' would cover the 'partially formed' nascent and/or forgot-to-produce-an-outcome state which is never looked at besides contract checking. The failure to produce a T (i.e. violation of postcondition) is reflected in error_code or exception_ptr. Ciao Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5

Asbjørn

11:26 a.m.

On 31.05.2017 10:26, Gavin Lambert via Boost wrote:

...

I'm just a plain C++ user, and my gut is leaning strongly towards the variant-like storage. However...

...

This sort of thing is very common in system APIs as you mention. So if outcome is meant to be able to wrap those "directly" then clearly it needs to be able to do both. But for these "non-error error results", I think I would prefer the code wrapping it in an outcome to put the non-error bits as part of the value. For example, a "here's some data, but I got more" error code could be handled by outcome's value storing a tuple or similar rather than just value_t. A truncated output has always been a proper error in my world, and handled as such (fex retrying after increasing buffer). Then again, just a plain user so :) Cheers - Asbjørn

Peter Dimov

2:26 p.m.

Gavin Lambert wrote:

...

Neither. I prefer a combination of the two. Like a variant, exactly one of has_value(), has_error(), has_exception() should report true, depending on whether you called set_value, set_error, or set_exception. The accessors however, should work as you previously outlined. // throws when !has_value() T value() const; // error_code() when has_value() // error when has_error() // errc::has_exception when has_exception() error_code error() const noexcept; // nullptr when has_value() // either nullptr or make_exception_ptr(system_error()) when has_error() // exception when has_exception() exception_ptr exception() const noexcept; Proposed additional narrow: // nullptr when !has_value(), otherwise &value_ T* operator->() noexcept; T const* operator->() const noexcept; // *operator->() T& operator() & noexcept; T const& operator() const & noexcept; T&& operator() && noexcept; T const&& operator() const && noexcept; value() can also have the four-overload form, not shown for brevity. Is there anyone that objects to that model? I am sympathetic to people's desire to have a statically checkable value accessor, but I see no reason to provide such for error() or exception(). The "exactly one is true" requirement allows all possible checks to be expressed in a concise manner. If you want to test for error or exception, say, that's !has_value.

...

Thomas Heller

2:43 p.m.

Am 31.05.2017 4:26 nachm. schrieb "Peter Dimov via Boost" < boost@lists.boost.org>: Gavin Lambert wrote:

...

... Or this sort of interface:

...

... Neither. I prefer a combination of the two. Like a variant, exactly one of has_value(), has_error(), has_exception() should report true, depending on whether you called set_value, set_error, or set_exception. The accessors however, should work as you previously outlined. // throws when !has_value() T value() const; // error_code() when has_value() // error when has_error() // errc::has_exception when has_exception() error_code error() const noexcept; // nullptr when has_value() // either nullptr or make_exception_ptr(system_error()) when has_error() Why not this: make_exception_ptr(system_error(error()) when has_error()? // exception when has_exception() exception_ptr exception() const noexcept;

Peter Dimov

3:17 p.m.

Thomas Heller wrote:

...

Yes, this was what I meant, sorry. Shortened it too much.

Andrzej Krzemienski

3:04 p.m.

2017-05-31 16:26 GMT+02:00 Peter Dimov via Boost :

...

Gavin Lambert wrote:

...
I probably should have thought of asking this earlier, but it occurs to me now that my own mental model of how an "outcome"-ish type should act is probably not suited to variant storage at all.

So just out of curiosity I thought I'd ask whether people prefer this sort of interface:

...

...
Or this sort of interface:

...

Neither. I prefer a combination of the two. Like a variant, exactly one of has_value(), has_error(), has_exception() should report true, depending on whether you called set_value, set_error, or set_exception. The accessors however, should work as you previously outlined.

// throws when !has_value() T value() const;

// error_code() when has_value() // error when has_error() // errc::has_exception when has_exception() error_code error() const noexcept;

What when `if_empty() == true`?

...

// nullptr when has_value() // either nullptr or make_exception_ptr(system_error()) when has_error() // exception when has_exception() exception_ptr exception() const noexcept;

What when `if_empty() == true`?

...

Proposed additional narrow:

// nullptr when !has_value(), otherwise &value_ T* operator->() noexcept; T const* operator->() const noexcept;

This is not narrow: this is wide. I disagree.

...

// *operator->() T& operator() & noexcept; T const& operator() const & noexcept; T&& operator() && noexcept; T const&& operator() const && noexcept;

This is operator*, right? This is technically narrow, but without essential benefits of direct narrow contract, as I tried to explain in another thread. I disagree.

...

value() can also have the four-overload form, not shown for brevity.

Is there anyone that objects to that model?

Yes, it des not offer a *proper* narrow contract.

...

I am sympathetic to people's desire to have a statically checkable value accessor, but I see no reason to provide such for error() or exception().

I am not sure about this. I have no strong opinion at this point.

...

The "exactly one is true" requirement allows all possible checks to be expressed in a concise manner. If you want to test for error or exception, say, that's !has_value.

What about is_empty()? Or are we considering the type without empty state? Regards, &rzej;

Peter Dimov

3:19 p.m.

Andrzej Krzemienski wrote:

...

What when `if_empty() == true`?

Nothing, I don't have an empty state.

...

...
Proposed additional narrow:

// nullptr when !has_value(), otherwise &value_ T* operator->() noexcept; T const* operator->() const noexcept;

This is not narrow: this is wide. I disagree.

Why do you disagree?

...

...
// *operator->() T& operator() & noexcept; T const& operator() const & noexcept; T&& operator() && noexcept; T const&& operator() const && noexcept;

This is operator*, right? This is technically narrow, but without essential benefits of direct narrow contract, as I tried to explain in another thread. I disagree.

What essential benefits are lost?

Andrzej Krzemienski

1 Jun 1 Jun

7:03 a.m.

2017-05-31 17:19 GMT+02:00 Peter Dimov via Boost :

...

Ok. Not having an empty state would work for me. I am not sure about others.

...

I tried to outline my reasoning in this post: http://boost.2283326.n4.nabble.com/outcome-narrow-contract-wide-contract-and... In short, when the narrow contract is directly in the library's interface, I have a place where I can put BOOST_ASSERT(), or similar code for assisting instrumented builds. In what you propose, the library has a wide contract, it returns another type (pointer) with a narrow contract; but it is somebody else's narrow contract and I cannot put this librarie's sanity checks. Let me just mention here that the LWG recommendation, while it probably makes sense for the Standard Library description, is not necessary in Boost. We can just make narrow-contract functions noexcept. Regards, &rzej;

Peter Dimov

12:15 p.m.

Andrzej Krzemienski wrote:

...

That's what I was thinking - that you want to place an assert there - I just wanted to confirm that this is the only objection. Do we agree that returning nullptr from operator-> is good for informing the static analyzer and the optimizer that a value is present? result<X> r; // ... r->x; // same as __builtin_assume( has_value() ) X x = *r; // ditto I can understand the desire to assert() inside, but at the same time, view the idea of returning a X* to something that is not an X as monumentally misguided in an error handling library. The goal here is to help people write robust code, not introduce subtle bugs and security vulnerabilities by allowing them scribble all over the stack. (Note that with the above spec, you can still assert in op* and it would be conforming, and you can still have a mode in which you assert in op->, but it will not conform.) The root of our disagreement is the idea that undefined behavior is good because it supposedly allows you to do this or that. It isn't, it never is. Defining the good things that we want to happen is good. Not defining them in the hope they'll occur is not.

Gavin Lambert

11:38 p.m.

On 2/06/2017 00:15, Peter Dimov wrote:

...

I assume you meant "not present" there. Having an explicit assert is still probably better than just returning nullptr and hoping for a static analyzer to pick up on that (or to crash at runtime). There are some platforms where unfortunately addresses around zero are valid (and while the standards allow the compiler to make nullptr not zero in this case, I don't know of any compilers that actually do so, probably because it's hard to actually do that alongside code that assumes it can memset(0) to get null pointers). I think the assert should be in there regardless; so then the question becomes whether you just reinterpret_cast<> after that or if you check type and return nullptr if you somehow survive the assert. It's not even really a narrow vs. wide contract thing, because it's never well-defined behaviour for -> to return nullptr. If assertions are disabled then both of these behaviours are UB (since returning nullptr from -> will result in dereferencing nullptr, which is inherently UB), but the second one is at least more likely to crash on those platforms that don't have valid memory around 0, and that's a good thing. On the other hand, some people would argue that the extra branch is pointless, especially in contexts where you check up front once and then use -> multiple times, which is not an uncommon pattern. (Sure, you could use * once and capture a reference, then avoid the issue entirely, but that can make the code look more awkward.)

Peter Dimov

2 Jun 2 Jun

12:43 a.m.

Gavin Lambert wrote:

...

No, I do mean "present". If you do `r->x` in your code, the compiler from this point on will optimize based on the assumption that a value was present - otherwise what you did would have been undefined. Actually, it will even propagate the "has_value()" assumption backwards from the `r->x` point, not just forwards. Returning a reinterpret_cast'ed pointer, on the other hand, will not do so, although maybe an `assert( has_value() )` will, depending. (Last time I recall Microsoft tried feeding asserts to the optimizer, they reverted it, as it broke too much code, but things may have changed in the meantime.)

Vicente J. Botet Escriba

3 Jun 3 Jun

6:31 a.m.

Le 02/06/2017 à 02:43, Peter Dimov via Boost a écrit :
> Gavin Lambert wrote:
>> On 2/06/2017 00:15, Peter Dimov wrote:
>> > That's what I was thinking - that you want to place an assert there 
>> - I > just wanted to confirm that this is the only objection. Do we 
>> agree that > returning nullptr from operator-> is good for informing 
>> the static > analyzer and the optimizer that a value is present?
>>
>> I assume you meant "not present" there.
>
> No, I do mean "present". If you do `r->x` in your code, the compiler 
> from this point on will optimize based on the assumption that a value 
> was present - otherwise what you did would have been undefined.
>
> Actually, it will even propagate the "has_value()" assumption 
> backwards from the `r->x` point, not just forwards.
>
> Returning a reinterpret_cast'ed pointer, on the other hand, will not 
> do so, although maybe an `assert( has_value() )` will, depending. 
> (Last time I recall Microsoft tried feeding asserts to the optimizer, 
> they reverted it, as it broke too much code, but things may have 
> changed in the meantime.)

Peter, I now that we are designing a class in C++11/.../C++17. I want to 
know what do you think if we had contracts and that compilers (or static 
analysis tools) where smart enough to detect UB.
Would you expect that operator-> return a nullptr when there is no value?
Or you want it to return nullptr because you believe that this could 
help the current UB sanitizers or static analysis tools to detect some UB?

Best,
Vicente

P.S. Sorry fro the late responses; all you mails were considered as spams.

Andrzej Krzemienski

2 Jun 2 Jun

7:44 a.m.

2017-06-01 14:15 GMT+02:00 Peter Dimov via Boost :

...

Andrzej Krzemienski wrote:

...
...
...
// *operator->()

...
T& operator() & noexcept; T const& operator() const & noexcept; T&& operator() && noexcept; T const&& operator() const && noexcept;

This is operator*, right? This is technically narrow, but without >> essential benefits of direct narrow contract, as I tried to explain in >> another thread. I disagree.

What essential benefits are lost?

I tried to outline my reasoning in this post: http://boost.2283326.n4.nabble.com/outcome-narrow-contract- wide-contract-and-value-if-td4695003.html

In short, when the narrow contract is directly in the library's interface, I have a place where I can put BOOST_ASSERT(), or similar code for assisting instrumented builds.

That's what I was thinking - that you want to place an assert there - I just wanted to confirm that this is the only objection.

It is not _the only_ objection. I thought that this one would be the easiest to communicate across.

...

Do we agree that returning nullptr from operator-> is good for informing the static analyzer and the optimizer that a value is present?

result<X> r;

// ...

r->x; // same as __builtin_assume( has_value() )

X x = *r; // ditto

I cannot answer this question with certainty. It looks like it has the potential to address the static-analyzer/UB-sanitizer use cases. On the other hand, static analyzers are not ideal, and they get lost when the reason for UB spans across to wide space. It is conceivable that putting an ASSERT(), or some such, earlier (directly in `outcome` could help detect UB, but your trick with operator-> would not. But I have no data to support this.

...

I can understand the desire to assert() inside, but at the same time, view the idea of returning a X* to something that is not an X ....

Here, the first disagreement. The way I see it is that the blame is on the user side. I would put it, "requesting the call X* to something you know not to be an X is misguided".

...

as monumentally misguided in an error handling library.

A question to you. Is your verdict affected by the fact that this is "an error handling library" or that this is a "vocabulary type"? IOW, would your position be different if we were discussing a design of some bigger library, like ASIO. In yet other words, do you object the same to vector::operator[] allowing UB when over-indexed? I fail to be convinced that "error handling library" should make different decisions than any other library. It is my understanding that `outcome and `expected` are intended to carry the information about unavailability of resources, and similar conditions in the environment. In contrast, what we are arguing about is what happens upon programmer using the library incorrectly.

...

The goal here is to help people write robust code,

Here is the second disagreement. That is, I do agree with you (and I guess everybody agrees) that this is always the important goal: help people write robust code. I just fail to see how the decision to assign *just any* semantics to something that is a misguided decision (I mean "requesting the call X* to something you know not to be an X") helps people write robust code. It is not robust to requesting the call X* to something you know not to be an X and you are not preventing that.

...

not introduce subtle bugs

In general, you cannot protect form users planting bugs in their logic.

...

and security vulnerabilities by allowing them scribble all over the stack.

So, is this your goal? In case the programmer does not know what he is doing (that he is using the library incorrectly), the library itself should take actions to minimize the damage? I am somewhat convinced by that. Although I am not sure that this is the library that should take this responsibility. In my view, the library should provide information what the invalid usages are, and there should be another tool (like static analyzer) that does the job of minimizing damage.

...

(Note that with the above spec, you can still assert in op* and it would be conforming,

Do you mean that you are ok with operator* having a narrow contract?

...

and you can still have a mode in which you assert in op->, but it will not conform.)

But then I do not understand what you get in this compromise that operator* has narrow- and operator-> has a wide contract. And we should also bare in mind Niall's opinion, that if he agrees to a narrow contract, such function should spell longer than `value()`.

...

The root of our disagreement is the idea that undefined behavior is good because it supposedly allows you to do this or that.

It is not in this that I see the root of our disagreement. In fact, I do not yet see where this root is. What I fight for is not an UB but narrow contracts. The different between the two is the following. In order for UB to occur you require two things: a function, call it `f` with a narrow contract, and an another function that calls `f` out of the contract. In the ideal world you have functions with narrow contracts and nobody calling them out of contract. And defining contracts, as formally as possible, help users identify when they are violating the contract. Not every design requires narrow contract. For instance, if we allowed only variant-like inspection of `expected`, every usage makes sense and is wide in contract. But the design choice that you first ask in which of the states `expected` is in, and then "get_the_T` and `get_the_E` allows for incorrect usage. Narrow contract simply reflects the fact that there is a way to use this interface incorrectly. Narrow contract is not incorrect, it simply reflects the fact that there might exist incorrect programs.

...

It isn't, it never is.

No, I do not believe that UB guarantees something in run-time, or rely on it.

...

Defining the good things that we want to happen is good. Not defining them in the hope they'll occur is not.

I agree with this statement in isolation, but I think you misunderstood my position about narrow contracts. Regards, &rzej;

Niall Douglas

10:29 a.m.

On 02/06/2017 08:44, Andrzej Krzemienski via Boost wrote:

...

I don't mind all-narrow at all. Hard to accidentally get wrong. I do object to mixed narrow and wide where narrow is quicker to type than wide, and the object is being used to transport uncertainty. It demands too much understanding of detail by the average programmer. Regarding the debate about narrow contracts, if one bases outcome/result/expected on std::variant as I believe one probably ought to, then as std::variant doesn't offer narrow contracts, neither could outcome/result/expected. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

12:02 p.m.

Andrzej Krzemienski wrote: ...

...

...
That's what I was thinking - that you want to place an assert there - I just wanted to confirm that this is the only objection.

It is not _the only_ objection. I thought that this one would be the easiest to communicate across.

What are the others?

...

...
I can understand the desire to assert() inside, but at the same time, view the idea of returning a X* to something that is not an X ....

Here, the first disagreement. The way I see it is that the blame is on the user side. I would put it, "requesting the call X* to something you know not to be an X is misguided".

Where is the disagreement? The blame is on the user side. This does not change what actually transpires.

...

...
as monumentally misguided in an error handling library.

A question to you. Is your verdict affected by the fact that this is "an error handling library" or that this is a "vocabulary type"?

No, this being an error handling library just makes the argument easier.

...

In yet other words, do you object the same to vector::operator[] allowing UB when over-indexed?

Possibly, today. In the past, not. It depends on whether the optimizer is good enough to avoid the repeated bounds check, and because of aliasing concerns, it may not be likely that it will ever be able to optimize it out in this case. vector::operator[] accesses occur in tight inner loops, the check is hard to optimize out, so we swallow the UB. Doesn't mean we have to like it, it's a necessary evil here.

...

I fail to be convinced that "error handling library" should make different decisions than any other library.

It shouldn't, I already said the operator-> for smart pointers should behave the same.

...

...
The goal here is to help people write robust code,

Here is the second disagreement. That is, I do agree with you (and I guess everybody agrees) that this is always the important goal: help people write robust code. I just fail to see how the decision to assign *just any* semantics to something that is a misguided decision (I mean "requesting the call X* to something you know not to be an X") helps people write robust code.

"Robust" is not the same as "correct". Robust code means code that doesn't misbehave when faced with unforeseen circumstances; having a logic error is such one unforeseen circumstance. Nobody is perfect.

...

...
not introduce subtle bugs

In general, you cannot protect form users planting bugs in their logic.

And I do not. Accessing a null pointer crashes. I however can mitigate the effects of their bugs, which is what I'm doing.

...

...
and security vulnerabilities by allowing them scribble all over the stack.

So, is this your goal? In case the programmer does not know what he is doing (that he is using the library incorrectly), the library itself should take actions to minimize the damage? I am somewhat convinced by that.

Yes, exactly. Here you have a union of an arbitrary type and std::error_code. std::error_code contains a pointer to a type with a vtable. Allowing the ability to write arbitrary data over that pointer is a security concern because this could lead to arbitrary code execution.

...

...
(Note that with the above spec, you can still assert in op* and it would be conforming,

Do you mean that you are ok with operator* having a narrow contract?

It already has. I have deliberately left it implicit though.

...

And we should also bare in mind Niall's opinion, that if he agrees to a narrow contract, such function should spell longer than `value()`.

Yes, there is that.

...

...
The root of our disagreement is the idea that undefined behavior is good because it supposedly allows you to do this or that.

It is not in this that I see the root of our disagreement. In fact, I do not yet see where this root is. What I fight for is not an UB but narrow contracts.

Traditionally the two are synonyms in C++. If you have a narrow contract, the behavior on contract violation is undefined. Can't have one without the other.

Andrzej Krzemienski

4:40 p.m.

2017-06-02 14:02 GMT+02:00 Peter Dimov via Boost :

...

In code reviews, when I see someone using the function out of contract, I know that this someone has a bug. When wide contracts are employed, even if people inadvertently do silly things, I can never easily detect bugs, because they operate within contract by definition. Now maybe they have bug, and maybe they just learned a way to do a hacky usage of the "rescue semantics" (he effect of artificially widening the contract).

...

Should I read the above as saying that in case of vector it would cause too much potential run-time overhead; whereas in case of `expected` it implies the existence of function `open_file` in the vicinity, and compared to the cost of that function, the cost of the defensive check is negligible? Is that what you are saying?

...

I fail to be convinced that "error handling library" should make different

...
decisions than any other library.

It shouldn't, I already said the operator-> for smart pointers should behave the same.

...
The goal here is to help people write robust code,

Here is the second disagreement. That is, I do agree with you (and I guess everybody agrees) that this is always the important goal: help people write robust code. I just fail to see how the decision to assign *just any* semantics to something that is a misguided decision (I mean "requesting the call X* to something you know not to be an X") helps people write robust code.

"Robust" is not the same as "correct". Robust code means code that doesn't misbehave when faced with unforeseen circumstances; having a logic error is such one unforeseen circumstance. Nobody is perfect.

...
not introduce subtle bugs

In general, you cannot protect form users planting bugs in their logic.

And I do not. Accessing a null pointer crashes. I however can mitigate the effects of their bugs, which is what I'm doing.

...
and security vulnerabilities by allowing them scribble all over the > stack.

So, is this your goal? In case the programmer does not know what he is doing (that he is using the library incorrectly), the library itself should take actions to minimize the damage? I am somewhat convinced by that.

Yes, exactly.

Here you have a union of an arbitrary type and std::error_code. std::error_code contains a pointer to a type with a vtable. Allowing the ability to write arbitrary data over that pointer is a security concern because this could lead to arbitrary code execution.

...
(Note that with the above spec, you can still assert in op* and it would

...
be conforming,

Do you mean that you are ok with operator* having a narrow contract?

It already has. I have deliberately left it implicit though.

And we should also bare in mind Niall's opinion, that if he agrees to a

...
narrow contract, such function should spell longer than `value()`.

Yes, there is that.

...
The root of our disagreement is the idea that undefined behavior is good

...
because it supposedly allows you to do this or that.

It is not in this that I see the root of our disagreement. In fact, I do not yet see where this root is. What I fight for is not an UB but narrow contracts.

Traditionally the two are synonyms in C++. If you have a narrow contract, the behavior on contract violation is undefined. Can't have one without the other.

Maybe this calls for a new kind of precondition specification: that calling a function in given circumstances is formally incorrect (tools are allowed to make use of this information), but the component still guarantees a rescue action. Regards, &rzej;

Peter Dimov

4:55 p.m.

Andrzej Krzemienski wrote:

...

But this doesn't apply here, does it? When you see r->x for !r, you do know that someone has a bug, right?

...

Not merely potential. Actual slowdown on the order of 100. You should read it as "as much as we'd like to define the behavior of operator[], doing so would be prohibitively expensive, so we won't."

...

That's kind of what we all want, but there's at present no way of getting there. Ideally -- let's assume we don't want to allow uses of the form `r.operator->()` -- we would want `operator->` when `!r` to either invoke a precondition violation handler that is guaranteed to terminate the program and never return, or to return `nullptr`. We have no tradition in expressing the above, so within the current vocabulary I prefer guaranteeing the `nullptr` instead of leaving the behavior undefined in the hope that it will end up being defined to the above. (It won't be.)

Andrzej Krzemienski

5:08 p.m.

2017-06-02 18:55 GMT+02:00 Peter Dimov via Boost :

> Andrzej Krzemienski wrote:
>
>> >> > That's what I was thinking - that you want to place an assert there
>> - >> > I just wanted to confirm that this is the only objection.
>> >>
>> >> It is not _the only_ objection. I thought that this one would be the
>> >> easiest to communicate across.
>> >>
>> >
>> > What are the others?
>>
>> In code reviews, when I see someone using the function out of contract, I
>> know that this someone has a bug.
>>
>
> But this doesn't apply here, does it?
>
> When you see r->x for !r, you do know that someone has a bug, right?
>

Yes, do. I am giving you the second reason why I want the narrow contract.
Within this second reason, I can live with your semantics. But previously I
have given you other reason, and there the operator-> trick will not work.


>
> > vector::operator[] accesses occur in tight inner loops, the check is >
>> hard to optimize out, so we swallow the UB. Doesn't mean we have to like >
>> it, it's a necessary evil here.
>>
>> Should I read the above as saying that in case of vector it would cause
>> too much potential run-time overhead;
>>
>
> Not merely potential. Actual slowdown on the order of 100. You should read
> it as "as much as we'd like to define the behavior of operator[], doing so
> would be prohibitively expensive, so we won't."
>

Why the same argument with "compiler will see that I am checking the same
condition twice, and remove redundant checks" does not apply here?


>
> > > The root of our disagreement is the idea that undefined behavior is >
>> > good because it supposedly allows you to do this or that.
>> >>
>> >> It is not in this that I see the root of our disagreement. In fact, I
>> >> do not yet see where this root is. What I fight for is not an UB but >>
>> narrow contracts.
>> >>
>> >
>> > Traditionally the two are synonyms in C++. If you have a narrow >
>> contract, the behavior on contract violation is undefined. Can't have > one
>> without the other.
>>
>> Maybe this calls for a new kind of precondition specification: that
>> calling a function in given circumstances is formally incorrect (tools are
>> allowed to make use of this information), but the component still
>> guarantees a rescue action.
>>
>
> That's kind of what we all want, but there's at present no way of getting
> there. Ideally -- let's assume we don't want to allow uses of the form
> `r.operator->()` -- we would want `operator->` when `!r` to either invoke a
> precondition violation handler that is guaranteed to terminate the program
> and never return, or to return `nullptr`.
>

Actually, now I am thinking not about Boost.Outcome but specifying these
preconditions in the Standard. If you want to std::terminate upon some
condition rather than let the program do random things, you could say.

  *Requires:* `!r` with rescue action `std::terminate()`.



>
> We have no tradition in expressing the above, so within the current
> vocabulary I prefer guaranteeing the `nullptr` instead of leaving the
> behavior undefined in the hope that it will end up being defined to the
> above. (It won't be.)


That is true, "it wont' be". And that was never the goal.

But what is the gain with the nullptr trick? someone can still cause UB
with it, so it does not seem much "safer". Are you increasing the chances
that it will be trapped by the operating system?

Regards,
&rzej;

Peter Dimov

5:24 p.m.

Andrzej Krzemienski wrote:

...

Because we can test it and see that it doesn't. This is an empirical argument, not a theoretical argument. "But it's the same here!" Well no, it isn't, we can measure it.

...

It's again a practical argument - we know what it does. In principle, it should be the same, but it isn't. We know that if we write the specification this way, so and so happens, and if we write it another way, different things occur. :-)

Vicente J. Botet Escriba

3 Jun 3 Jun

6:52 a.m.

Le 02/06/2017 à 19:24, Peter Dimov via Boost a écrit :

...

Peter, I not be against an implementation that returns nullptr of operator-> as far as this is not documented and something the user could use. If returning a nulptr is the best thing that we can do today in order to help the current tools to catch UB, why not. I don't agree on documenting it, because this could make some user code more complex. When you have a narrow contract you know that the check must be done before. If you document that it the user can be tempted to check it. Well operator-> is particular in some way as the user doesn't use to use it as x.operator->(). Consider for a moment that some compiler manage better UB when we do a check and assert for unreachable code (as other are suggesting). Requiring a nullptr as result will forbid this implementation, isn't it? My question is, why do you consider that it is good to document it? Vicente

Peter Dimov

9:42 a.m.

Vicente J. Botet Escriba wrote:

...

Peter, I not be against an implementation that returns nullptr of operator-> as far as this is not documented and something the user could use. If returning a nulptr is the best thing that we can do today in order to help the current tools to catch UB, why not. I don't agree on documenting it, because this could make some user code more complex. When you have a narrow contract you know that the check must be done before. If you document that it the user can be tempted to check it.

The expression `r->x` still has a narrow contract under my formulation. You (and the static analyzer) still know that a check needs to be done. It's just specified differently because the real world consequences of this specification are more in line with the semantics I want to express.

...

Well operator-> is particular in some way as the user doesn't use to use it as x.operator->().

Yes, `r.operator->()` is now defined, which some consider desirable. Presumably, if one writes `auto* p = r.operator->();`, one is aware of what one's doing. I wouldn't presume that this code is a logic error without a check, and neither should a static analyzer.

...

Consider for a moment that some compiler manage better UB when we do a check and assert for unreachable code (as other are suggesting). Requiring a nullptr as result will forbid this implementation, isn't it?

No, it doesn't. This formulation just lifts the undefined behavior in `r->x` from the library into user code, and what happens there depends on the compiler. g++ takes care to trap on null pointer access, clang++ optimizes out the checks under the assumption that null pointer accesses don't occur. The balance here is hard to strike, because there are security implications of just allowing `p->m = ...` to write over the error; at the same time, there's an argument to be made to optimize correct programs more at the expense of incorrect programs. The main feature of my formulation here is that `r->x` and `p->x` (where p is a raw pointer) elicit a consistent response from the compiler when they invoke UB, and are therefore covered by the same optimization decisions or command line switches.

Andrzej Krzemienski

10 a.m.

2017-06-02 19:24 GMT+02:00 Peter Dimov via Boost :

...

Peter, I do not understand what you are trying to say here. You say you have observed some useful behavior when your trick with operator-> is applied. I am asking what it is. It is still an UB when used, so there must be something to it. My guess would be that UB on null pointer is more tool-friendly than UB on just any bad pointer. But if it should be the case, why did you say you would like the same trick in shared_ptr? It deals with dereferencing a nullptr both with and wothout your hack. Regards, &rzej;

Peter Dimov

10:11 a.m.

Andrzej Krzemienski wrote:

...

Trick?

...

As I say in my reply to Vicente, specifying operator-> in this way leads to consistent behavior for the expression `p->x` when p is a raw pointer, a smart pointer, or, in this case, result<>/outcome<>. The same optimization decisions apply, the same command-line compiler switches apply, in a consistent manner.

Andrzej Krzemienski

10:29 a.m.

2017-06-03 12:11 GMT+02:00 Peter Dimov via Boost :

...

Sorry, I did not mean to imply any opinion or evaluation of quality. I just needed a short phrase to refer to your proposed alternative. Maybe I should call it "operator-> gives nullptr on bad access". Regards, &zej;

Niall Douglas

31 May 31 May

8:24 p.m.

On 31/05/2017 15:26, Peter Dimov via Boost wrote:

...

Proposed additional narrow:

// nullptr when !has_value(), otherwise &value_ T* operator->() noexcept; T const* operator->() const noexcept;

// *operator->() T& operator() & noexcept; T const& operator() const & noexcept; T&& operator() && noexcept; T const&& operator() const && noexcept;

value() can also have the four-overload form, not shown for brevity.

Is there anyone that objects to that model?

I don't mind those operators being wide. I do object to them being narrow as it's too little typing.

...

I am sympathetic to people's desire to have a statically checkable value accessor, but I see no reason to provide such for error() or exception().

I would assume that if the programmer writes .error_raw() on a valued object, they'd like to see a compiler warning.

...

The "exactly one is true" requirement allows all possible checks to be expressed in a concise manner. If you want to test for error or exception, say, that's !has_value.

Such a design would not allow identical code to work correctly with an empty-capable or non-empty-capable objects. It should be very simple, even simpler than yours: .empty() true => object is empty .has_value() true => value() observes a T .has_error() true => error() returns the error code .has_exception() true => exception() returns an exception_ptr to what would be thrown if you called .value() Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Robert Ramey

4:09 p.m.

On 5/31/17 1:26 AM, Gavin Lambert via Boost wrote: well since people are chiming in with their own ideas I might has well demonstate mine. The save numerics library (which I'm still working on trying to make all the required changes), includes a type called checked_result. It turns out that I needed this so I just made it. I was familar with expected and of course variant, but I didn't find a "finished" implementation of expected - boost didn't have one, and variant, ... I just didn't think of deriving from it - which would be my usual inclination. Basically I thought my needs were just simple to need to use a library or more general facility. After all, I'm a C++ programmer and to a man with a hammer in his hand, the whole world looks like a nail (not Naill). So I just wrote a simple class which I called "checked_result". It's part of the safe numerics library implementation so I didn't feel required to document it, but I did anyway because I thought it would useful to do so, I have a simple easy to use documentation system, and it's helpful for me to do this to clarify my own thinking. Here a link to the documenation for checked_result. https://htmlpreview.github.io/?https://raw.githubusercontent.com/robertramey... and here is a link to the code https://github.com/robertramey/safe_numerics/blob/master/include/checked_res... which comes to about 230 lines (way more than I thought it would). The template includes a parameter for the value type but doesn't include one for the error/exception type - because I didn't need it. I might "improve" this in the future by making the error/exception type another template parameter. Readers of this humble missive are free to make of this what they desire. When the review of outcome was announced, I had the expectation that the proposal would be for something of like this of similar scale, simplicity, and functionality - but done more formally and carefully. So I'm sort of blown away at how this thing has grown. I've already posted my review, so I'm not going to try to compare these things, I'm just thinking that when something seems to grow way, way out of proportion to what one original expectation, perhaps it's time to step back and re-consider. Maybe the idea of "outcome" is too ambitious, or maybe we're trying to solve too many problems at once, or maybe we're trying to reconcile fundamentally irreconcilable view points. Robert Ramey

Andrzej Krzemienski

1 Jun 1 Jun

7:22 a.m.

2017-05-31 18:09 GMT+02:00 Robert Ramey via Boost :

...

On 5/31/17 1:26 AM, Gavin Lambert via Boost wrote:

well since people are chiming in with their own ideas I might has well demonstate mine.

The save numerics library (which I'm still working on trying to make all the required changes), includes a type called checked_result. It turns out that I needed this so I just made it. I was familar with expected and of course variant, but I didn't find a "finished" implementation of expected - boost didn't have one, and variant, ... I just didn't think of deriving from it - which would be my usual inclination. Basically I thought my needs were just simple to need to use a library or more general facility. After all, I'm a C++ programmer and to a man with a hammer in his hand, the whole world looks like a nail (not Naill). So I just wrote a simple class which I called "checked_result". It's part of the safe numerics library implementation so I didn't feel required to document it, but I did anyway because I thought it would useful to do so, I have a simple easy to use documentation system, and it's helpful for me to do this to clarify my own thinking. Here a link to the documenation for checked_result.

https://htmlpreview.github.io/?https://raw.githubusercontent .com/robertramey/safe_numerics/master/doc/html/checked_result.html

and here is a link to the code

https://github.com/robertramey/safe_numerics/blob/master/ include/checked_result.hpp

which comes to about 230 lines (way more than I thought it would). The template includes a parameter for the value type but doesn't include one for the error/exception type - because I didn't need it. I might "improve" this in the future by making the error/exception type another template parameter.

Readers of this humble missive are free to make of this what they desire.

When the review of outcome was announced, I had the expectation that the proposal would be for something of like this of similar scale, simplicity, and functionality - but done more formally and carefully. So I'm sort of blown away at how this thing has grown. I've already posted my review, so I'm not going to try to compare these things, I'm just thinking that when something seems to grow way, way out of proportion to what one original expectation, perhaps it's time to step back and re-consider. Maybe the idea of "outcome" is too ambitious, or maybe we're trying to solve too many problems at once, or maybe we're trying to reconcile fundamentally irreconcilable view points.

A valid observation. In a way `expected` and `outcome` (both from Boost.Outcome) have different goals in mind. `expected` might in fact cover your case. They could be two libraries. The reason they come together is that they share 95% of the same implementation. They differ only by interfaces. Two interfaces for two different problems. Regards, &rzej;

Robert Ramey

2:23 p.m.

On 6/1/17 12:22 AM, Andrzej Krzemienski via Boost wrote:

...

I haven't investigated too deeply into the code so of course I didn't know that. I would have "expected" that outcome, expected, et.al would be derived from variant which would be the shared code. Also all the questions about narrow/wide interface, no empty guarantees, etc. would be resolved (for better or worse) in the variant library so the design, review, maintenance, etc. of outcome, expected et al. would be confined to the particular aspects of these components - thus being a more economic application of limited brain surface area. Code size would be smaller as well.

...

They differ only by interfaces. Two interfaces for two different problems

Right. It's common for different "types/libraries/concepts/functions/etc." to factor out commonality for just the reasons mentioned above. Robert Ramey

Vicente J. Botet Escriba

5:24 p.m.

Le 01/06/2017 à 16:23, Robert Ramey via Boost a écrit :

...

Robert Ramey

5:43 p.m.

On 6/1/17 10:24 AM, Vicente J. Botet Escriba via Boost wrote:

...

Right - so isn't variant the place to "fix" it? if outcome, expected, optional ... need a never-empty guarantee, should variant (std, boost, whatever) have that guarantee as well? Shouldn't these discussions take place in the context of variant? Now if the "best" implementation of variant isn't the "best" one for one of it's derivatives - then I would ask why not? How are they different. Of course this can windup as variant having a policy. I haven't seen this but I would be very surprised if it hasn't come up. In any case it seems that it would be much more efficient for these types to derive from variant then repeat implementations - each with a little different sauce. Not only more efficient as in reducing code. But more efficient in consumption of my limited brain power: OK I understand variant, never empty or not, etc... template<class T> using outcome = std::variant // + some global functions void f(const global &g){ do something } or template<class T> struct outcome : public std::variant { // some special member functions outcome(const std::error_code & e){...} // construct as error outcome(const T & t){...} // construct legitimate result }; (I'm obviously winging it here as I don't remember exactly what outcome does.) Basically, I'm arguing that all the discussion/design of these "derivative" types belong in variant. Of course if there is not agreement (not that this would ever happen) one could template using outcome = V In any case, I would be factoring out common code, design, rationale, maintenance problems, documentation, etc up into the base class variant. FWIW - this was my concern when I saw your presentation a few years ago. Not that it was without merit, but I saw it as more of a use case for variant rather than a new library component. Robert Ramey

Peter Dimov

5:47 p.m.

Robert Ramey wrote:

...

There were prolonged and furious debates on this topic while std::variant was getting standardized, and the side who wanted a never-valueless variant (that would be me) lost.

Niall Douglas

6:47 p.m.

On 01/06/2017 18:47, Peter Dimov via Boost wrote:

...

I haven't met a soul from WG21 who thinks variant should go empty as frequently as the C++ 17 standard says it should. I think it's acceptable if the user supplies types more than one of which has a throwing move constructor. Otherwise you'd need to double storage used. If one or fewer types supplied to variant have throwing move constructors, then a hard never-empty guarantee needs to be made, and with some static flag or observer to let you know. Or just don't implement .valueless_by_exception() :) Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

6:30 p.m.

Robert Ramey wrote:

...

Something like this? https://github.com/pdimov/variant2/blob/develop/include/boost/variant2/resul... https://godbolt.org/g/EFGkCj

Niall Douglas

6:49 p.m.

On 01/06/2017 19:30, Peter Dimov via Boost wrote:

...

Hmm, maybe you could also implement expected, and that would kinda eliminate the need for a Boost.Outcome? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

6:55 p.m.

Niall Douglas wrote:

...

My original idea was to implement my idea of expected, but I took a detour through never-valueless variant<> first.

Niall Douglas

10:02 p.m.

...

Would you be intending to submit your variant2 into Boost, along with an expected implementation sometime soon? You see, from my perspective, getting Outcome into Boost is nothing like as important as getting **any** sort of variant-stored result<T> type thing into Boost. I need one of those into Boost to make getting AFIO v2 into Boost feasible as every single AFIO API returns a result<T>. In other words Peter, if you'd like to do the heavy lifting getting your own Outcome implementation extending Variant2 into Boost instead of me ... I'm all for it. I'll still be implementing my agreed changes to my own Outcome as lots of my code depends on my specific formulation, and I don't want a Boost dependency anyway. But most of the work of getting a Boost library past review is not implementing and testing a high quality implementation, it's all the other stuff like endless documentation and indeed, writing 300+ emails as part of a two week long review. If I could skip all that other stuff because you were taking it on, I'd be **overjoyed**. I could get back to working on AFIO v2 instead, a big gain for me. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

10:54 p.m.

Niall Douglas wrote:

...

We'll see.

...

That's not and hasn't been my intention. I wanted to write expected so that we have something concrete to discuss regarding standardization, and for that, I needed a never-valueless variant, which is also something that I would rather see in the standard at some point, preferably instead of the current one. The result and outcome classes I threw together are only toys at this point, their purpose is both to let us see how the compilers optimize, and for me to demonstrate how I would like the interface of boost::result/boost::outcome/std::result/std::outcome to look like. But I don't really want to enter these implementations in Boost instead of yours. Your implementation has been tested in practice and is more feature-rich. The error_code_extended infrastructure adds significant value, and if you agree to a larger (or customizable at runtime) ring buffer and chaining support, they'll become even better. (Attaching information to an error code is a very good idea of yours.) In short, I'd rather see your implementations in Boost, but with something close to the interface I present - if the community can agree on it, of course. It's just hard to keep the design discussion focused without a reference implementation. Ideally, we all say at some point OK, we're fine with that interface, this is it, there we go.

Robert Ramey

11:08 p.m.

On 6/1/17 3:54 PM, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...

I think we spend too much time thinking about the standard. Let us focus on getting it right, getting it out there, getting people to use it, getting feedback on it, making sure there is a version available that is maintained, making sure it's well documented and plays well with everything else. Let the standards committee spend their time catching up to us. This has worked well in the past and I believe will continue to work best. I'm 69 years old as I write this. I could be dead before the it's all standardized. Robert Ramey

Niall Douglas

2 Jun 2 Jun

10:14 a.m.

...

A shame. I already invested six months of effort taking a mature library and getting it ready for review. I had been hoping to avoid more work on this, AFIO languishes until Outcome is done. Still that said, if Outcome changes significantly, better it happens now before more AFIO code is hard wired around Outcome. Some of the AFIO filesystem algorithms are quite subtle and rely very heavily on Outcome behaving exactly just so. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

1 Jun 1 Jun

6:52 p.m.

...

And here's outcome for comparison: https://github.com/pdimov/variant2/blob/develop/include/boost/variant2/outco... https://godbolt.org/g/I4gM8D The generated code does look a bit worse, the destructor is not optimized out.

Robert Ramey

10:07 p.m.

On 6/1/17 11:30 AM, Peter Dimov via Boost wrote:

...

Probably I suppose ....

...

https://godbolt.org/g/EFGkCj

But this isn't really my point. The point I'm trying to make is that all the design disputes are also found in variant. It would seem to me that we're repeating all the same arguments ... with pretty much similar results. If the problem is that variant (std, boost, ...) is "broken" or not suitable for that reason it would seem to me that THAT problem would need to be addressed. Were that the case, then all the issues with outcome, optional, result, expected would disappear. Maybe I'm over simplifying here, so feel free to demonstrate I'm wrong. In any case, I would think that if a consensus can't be agreed upon after all this effort, maybe we should step back and except as a fact that "concensus cannot be reached". So we'll permit variant implementations of variant either with policy or simpler yet, just a separate implementation. Of course these separate implementations would also share a common base class so we get nested russian dolls. But at least we don't get so much repetition. Robert Ramey

Niall Douglas

2 Jun 2 Jun

10:09 a.m.

...

There is one big difference with std::optional and std::variant - their design is now **the standard**, for better or for worse. All new code written henceforth ought to be designed around the C++ standard in my book, with hacks/workarounds as appropriate where the standard object falls short. But otherwise regarding discussion of Outcome, I think there are three main use patterns for failure returning objects, and a single object design can't fulfill more than two of them at best. I personally believe that "single implementation, multiple personalities" solves the problem, but I recognise that I have failed to persuade anyone of that during this review, which was a surprise to me actually. But finding out what others really care about, and you had no idea that they did, is the whole point of a successful Boost review irrespective of outcome. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Andrzej Krzemienski

10:36 a.m.

2017-06-02 12:09 GMT+02:00 Niall Douglas via Boost :

...

You have never put it this way. Can you list the three use patterns here? Regards, &rzej;

Niall Douglas

1:44 p.m.

...

Sure. 1. User wants failure reason type to be local to only the code it applies to. So think custom E type, nonconvertible to any other E type, and E is never error_code nor exception_ptr. They specifically want to keep failure handling local a small patch of code e.g. constexpr evaluation contexts. All-narrow observers make sense for this use case as object usage is tightly integrated to the code using it, and failure is handled entirely locally. 2. User wants failure reason type to be globally understood and will be choosing either E = std::error_code, or E = std::exception_ptr. They specifically want failure handling to be able to traverse any or all code, just like C++ exception throws. This is where Outcome was primarily aimed at. All-wide observers make sense for this use case, as due to handling a wide degree of uncertainty, most failure handling code is *generic* i.e. we test for some specific failure causes, and for everything else we either ignore or hand it off to other code to handle. 3. User wants to write functional programming logic using the basic vocabulary of Maybe, Either and i/o monads and basic operators of bind, fmap, do etc and probably some subset or refinement of Hana for the collections monads, though my GSoC student may be making the Ranges TS a choice here as well later this summer. All-narrow observers make sense for this use case as the monadic operators ensure your function will never be called with the wrong state. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Peter Dimov

2:57 p.m.

Niall Douglas wrote:

...

There's no need for narrow (value) observers when using the monadic interface; it's all wide and since you get the value directly, you don't need to observe it.

Niall Douglas

9:51 p.m.

On 02/06/2017 15:57, Peter Dimov via Boost wrote:

...

Not always. Sometimes your function will be fed a wrapped value and you'll need to observe it. I still think narrow makes more sense here. I also forget another thing which use case 3 needs, and that is immutability, which in turn probably in C++ implies that the types used must be trivially destructible, else it would be unusable. In other words, the code must be written as if able to run constexpr, probably. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Robert Ramey

11:45 p.m.

On 6/2/17 2:51 PM, Niall Douglas via Boost wrote:

...

On 02/06/2017 15:57, Peter Dimov via Boost wrote:

...
Niall Douglas wrote:

...

I also forget another thing which use case 3 needs, and that is immutability, which in turn probably in C++ implies that the types used must be trivially destructible, else it would be unusable. In other words, the code must be written as if able to run constexpr, probably.

LOL - I've already run into problems with error_code due the fact that it's constructor is not constexpr friendly. So I would guess you're on to something here.

...

Niall

Vicente J. Botet Escriba

3 Jun 3 Jun

6:17 a.m.

Le 02/06/2017 à 16:57, Peter Dimov via Boost a écrit :

...

You are right Peter. You can just provide the Functor, Applicative, Monad operations directly. You can as well provide the SumType interface (to be defined) but that will have at least the visit function. My idea is to see any TypeConstructible PossiblyValued as a Functor, an Applicative and a Monad. expected<_,E>/optional<_>/unique_ptr<_,D>/shared_ptr<_> are all PossiblyValued. A PossiblyValued is the sum/variant of a type T and something else. A TypeConstructible type is one that has a factory that makes the type from its underlying type. I know that see unique_ptr<_,D>/shared_ptr<_> as PossiblyValued types can be surprising, but any NullablePointer either has a pointee (pointer not null) or not (pointer null). So we can see it as the sum of T and nullptr_t. We don't need this and we can have direct mappings for all these concrete types. Nevertheless, if a type has already access to its possibly value types, why not have also a narrow contract. The question is do we want optional/expected to provide only a monadic interface. Do we want it to provide the visit function as well? Why not provide a direct access? I know (thanks to Niall) variant has a wide narrow access. We have lost the train for std::variant. If we had to define a monadic interface for std::variant, I'm sure that the std::variant implementer would have a hidden narrow access to its elements and will not use neither get_if nor visit. Well, this is what I will do to avoid to have to ensure/check that there is no cost while using get_if/visit. The problem is that a user of variant can not change variant or see hidden functions. Maybe I'm wrong and I'm doing premature micro optimizations. While I think compilers could optimize a lot of things, I believe that a developer should try to define the minimal interface that helps the compiler to generate the best code. A narrow contract (like the one of optional, smart pointers) is like accessing the alternative directly and this is could be considered bad. The point is that it allows to build the high abstraction efficiently. Again, we don't need a narrow contract access, but then we need all the high level abstractions that we can built on to of it. I'm a fan of safe interface, but also the ones that are efficient enough. Vicente P.S. Sorry if I have repeated myself more than once. P.S. You can take a look at the current state of my work concerning this a more at https://github.com/viboes/std-make/tree/master/include/experimental/fundamen... https://github.com/viboes/std-make/tree/master/doc/proposal

Peter Dimov

9:49 a.m.

Vicente J. Botet Escriba wrote:

...

On gcc/clang there is no cost for using *get_if<> instead of a private accessor, and depending on how get<> is written, there can be no cost to using it as well. I did switch from get<> to private accessors within the variant implementation, but that's mostly because I want to generate aesthetically pleasing assembly on Compiler Explorer and not because there is a measurable cost to using get<>.

Robert Ramey

2 Jun 2 Jun

2:59 p.m.

On 6/2/17 3:09 AM, Niall Douglas via Boost wrote:

...

There is one big difference with std::optional and std::variant - their design is now **the standard**, for better or for worse.

...

Ahhhh - I argued in a different thread that we should pay less attention to the C++ standard. Traditionally, they've followed Boost. Most of the stuff in the standard have been test driven in Boost first. Even so, they make mistakes - as is likely has been done in this case. We should strive to make good well conceived, well designed, well implemented library components. Let them pick and choose what they want to standardize. We don't work for them, the work for us - promulgating the stuff we've already implemented and proven useful. Robert Ramey

Niall Douglas

9:54 p.m.

On 02/06/2017 15:59, Robert Ramey via Boost wrote:

...

I'm not talking about future standards. I'm talking about the existing ones. Ok, you can argue C++ 17 and its std::variant isn't standard *quite* just yet, but I can guarantee you that std::variant will not be changing in the C++ 17 standard (apart from defects). Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Vicente J. Botet Escriba

3 Jun 3 Jun

6:56 a.m.

Le 01/06/2017 à 20:30, Peter Dimov via Boost a écrit :

...

Thanks Peter for working on this. We need such a variant2 at least in Boost. As you show, building on top of a sound class makes the implementation much simpler. Best, Vicente

Andrzej Krzemienski

2 Jun 2 Jun

7:50 a.m.

2017-06-01 19:43 GMT+02:00 Robert Ramey via Boost :

...

outcome, expected, optional -- they all have a nice property, they always have at least one type that is no-throw upon copy/move constructor/assignment. Under these constraints it is easy to implement a variant with strong assignment. But generic variant needs to work on generic Ts, and cannot make this assumption. If we provided a second variant, with stricter assumptions, or added a specialization to the current one, it would work, I guess. Regards, &rzej;

2866

Age (days ago)

2869

Last active (days ago)

List overview

Download

63 comments

9 participants

participants (9)

Andrzej Krzemienski
Asbjørn
Daniela Engert
Gavin Lambert
Niall Douglas
Peter Dimov
Robert Ramey
Thomas Heller
Vicente J. Botet Escriba