[Interprocess] Named pipe interface proposal

Geoff Shannon

10 Aug 2013 10 Aug '13

1:45 a.m.

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it. Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing. As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing. Thanks! -- Geoff Nothing is ever easy.

Attachments:

named_pipe.hpp (text/x-c++hdr — 3.1 KB)

Show replies by date

Edward Diener

10 Aug 10 Aug

12:25 p.m.

On 8/9/2013 9:45 PM, Geoff Shannon wrote:

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

std::vector<TYPE> is the C++ way of specifying arrays.

...

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

Sounds good.

Rob Stewart

11 Aug 11 Aug

12:59 a.m.

On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...

On 8/9/2013 9:45 PM, Geoff Shannon wrote:

...
Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid. Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right. There's also std::array and boost::array. You could even combine the two into one class to get SBO, though I think that's not likely to be useful if the caller allocates the memory. (It would be useful if the API allocates space.) ___ Rob (Sent from my portable computation engine)

Edward Diener

1:50 p.m.

On 8/10/2013 8:59 PM, Rob Stewart wrote:

...

On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
On 8/9/2013 9:45 PM, Geoff Shannon wrote:

...
Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

...

Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right.

Do you actually do this in your programming ? To "prevent mistakes" ?

...

There's also std::array and boost::array.

Sure, but why not use the normal C++ construct.

Rob Stewart

7:39 p.m.

On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...

On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written. What I often do is the following: std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity()); That can be simplified by a wrapper class.

...

...
Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right.

Do you actually do this in your programming ? To "prevent mistakes" ?

I've been doing that enough lately that I should. Indeed, I intend to do that this week because I've thought over the number of times I've managed that by hand over the years. It's past time for me to wrap that logic.

...

...
There's also std::array and boost::array.

Sure, but why not use the normal C++ construct.

How are those array classes not "normal C++ construct[s]"? They are appropriate for local allocations of modest sizes, though they can be used in free store allocated objects with much larger sizes. (My 1024-byte example, for example, would do well with array rather than vector.) ___ Rob (Sent from my portable computation engine)

Edward Diener

12 Aug 12 Aug

12:45 a.m.

On 8/11/2013 3:39 PM, Rob Stewart wrote:

...

On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

I don't know what "read(&v[0], v.capacity())" is doing here. is this supposed to be a named pipe 'read'-like function ? Whatever it is either the 'read' will: 1) read a byte at a time and add it to the end of the vector via a push_back or 2) will copy some number of bytes already read into the vector. In either case your above code does not make sense to me, and seems unnecessary. If I am doing 1) there is no need for it. If I am doing 2) I better resize the vector to the number of bytes I need or use an inserter. if my 'read' function isn't smart enough to just take a vector and fill it with what's read, I have programmed the wrong function.

...

That can be simplified by a wrapper class.

...
...
Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right.

Do you actually do this in your programming ? To "prevent mistakes" ?

I've been doing that enough lately that I should. Indeed, I intend to do that this week because I've thought over the number of times I've managed that by hand over the years. It's past time for me to wrap that logic.

I don't think you like std::vector. For some reason you want something else, but I don't know what that is or why.

...

...
...
There's also std::array and boost::array.

Sure, but why not use the normal C++ construct.

How are those array classes not "normal C++ construct[s]"? They are appropriate for local allocations of modest sizes, though they can be used in free store allocated objects with much larger sizes. (My 1024-byte example, for example, would do well with array rather than vector.)

Because when you already have a dynamically allocated array in std::vector you need some reason for giving it up and doing something else. Why re-invent the wheel for no reason ?

Marshall Clow

12:53 a.m.

On Aug 11, 2013, at 5:45 PM, Edward Diener wrote:

...

On 8/11/2013 3:39 PM, Rob Stewart wrote:

...
On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

I don't know what "read(&v[0], v.capacity())" is doing here. is this supposed to be a named pipe 'read'-like function ? Whatever it is either the 'read' will:

1) read a byte at a time and add it to the end of the vector via a push_back

I can't see it doing that - because it would need a reference to the vector to do the push_back.

...

or

2) will copy some number of bytes already read into the vector.

If it did this, v.size() would still == 1.

...

In either case your above code does not make sense to me, and seems unnecessary.

I don't know about "unnecessary", but I agree this call does not make sense to me. Maybe there's some more information here that I'm missing.

...

If I am doing 1) there is no need for it. If I am doing 2) I better resize the vector to the number of bytes I need or use an inserter. if my 'read' function isn't smart enough to just take a vector and fill it with what's read, I have programmed the wrong function.

...
That can be simplified by a wrapper class.

...
...
Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right.

Do you actually do this in your programming ? To "prevent mistakes" ?

I've been doing that enough lately that I should. Indeed, I intend to do that this week because I've thought over the number of times I've managed that by hand over the years. It's past time for me to wrap that logic.

I don't think you like std::vector. For some reason you want something else, but I don't know what that is or why.

...
...
...
There's also std::array and boost::array.

Sure, but why not use the normal C++ construct.

How are those array classes not "normal C++ construct[s]"? They are appropriate for local allocations of modest sizes, though they can be used in free store allocated objects with much larger sizes. (My 1024-byte example, for example, would do well with array rather than vector.)

Because when you already have a dynamically allocated array in std::vector you need some reason for giving it up and doing something else. Why re-invent the wheel for no reason ?

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-- Marshall Marshall Clow Idio Software mailto:mclow.lists@gmail.com A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki

Edward Diener

4:49 a.m.

On 8/11/2013 8:53 PM, Marshall Clow wrote:

...

On Aug 11, 2013, at 5:45 PM, Edward Diener wrote:

...
On 8/11/2013 3:39 PM, Rob Stewart wrote:

...
On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

I don't know what "read(&v[0], v.capacity())" is doing here. is this supposed to be a named pipe 'read'-like function ? Whatever it is either the 'read' will:

1) read a byte at a time and add it to the end of the vector via a push_back

I can't see it doing that - because it would need a reference to the vector to do the push_back.

In my prefereed interface it is passing a reference to a vector which is being filled in with the data read. Gee, that's too simple. Must be something seriously wrong with it. I acknowledge it might also be advantage to pass some limit as a second parameter, so only that number of elements gets filled in at the max. But in either case it does not make sense to me to prime the data with some mythical first element, or worry about what has been reserved. I don't design with C-style pointers, to some possible array of some possible size, of elements. I thought everyone had given up on that sort of unnecessary complication a long time ago when std::vector became the defacto standard in C++ for dynamic arrays. I am not against fixed size arrays if that is what some interface calls for, but if we are talking about dynamic arrays of elements then std::vector seems to me to be the way to do it.

Rob Stewart

9:28 a.m.

On Aug 12, 2013, at 12:49 AM, Edward Diener wrote:

...

In my prefereed interface it is passing a reference to a vector which is being filled in with the data read. Gee, that's too simple. Must be something seriously wrong with it.

That's restrictive. An interface that takes a pointer plus length, packaged how you like, leaves the caller free to allocate the buffer however it makes most sense in the calling context. Using vector like that also forces the container choice on the caller. Either way, the caller must copy the data elsewhere, but what if there are only a few hundred bytes to be read? Your interface requires a free store allocation. For larger lengths, unless your API first calls reserve(), the vector may reallocate a time or two. Finally, note that your scheme requires allocating a buffer to use when calling a low level OS or runtime API, and then copying those bytes to the vector in order to transfer them to the caller. Why not have the caller provide the buffer in the first place and avoid that overhead?

...

I acknowledge it might also be advantage to pass some limit as a second parameter, so only that number of elements gets filled in at the max. But in either case it does not make sense to me to prime the data with some mythical first element, or worry about what has been reserved.

A wrapper would hide these irksome implementation details.

...

I don't design with C-style pointers, to some possible array of some possible size, of elements. I thought everyone had given up on that sort of unnecessary complication a long time ago when std::vector became the defacto standard in C++ for dynamic arrays.

Many of us care about performance.

...

I am not against fixed size arrays if that is what some interface calls for, but if we are talking about dynamic arrays of elements then std::vector seems to me to be the way to do it.

Perhaps you're beginning to rethink your position a little. ___ Rob (Sent from my portable computation engine)

Rob Stewart

9:01 a.m.

On Aug 11, 2013, at 8:45 PM, Edward Diener wrote:

...

On 8/11/2013 3:39 PM, Rob Stewart wrote:

...
On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

I don't know what "read(&v[0], v.capacity())" is doing here. is this supposed to be a named pipe 'read'-like function ? Whatever it is either the 'read' will:

1) read a byte at a time and add it to the end of the vector via a push_back

You need the container for push_back().

...

or

2) will copy some number of bytes already read into the vector.

That's not a read function.

...

In either case your above code does not make sense to me, and seems unnecessary.

reserve() allocates uninitialized, contiguous memory, which is ideal for a read buffer. In order to use that memory, you need a pointer to it. v[0] is the first element in that allocation, so &v[0] is the start of that allocation, but referencing that element is UB without first adding it

...

If I am doing 1) there is no need for it. If I am doing 2) I better resize the vector to the number of bytes I need

As I noted previously, resize() initializes each element, which is often unneeded.

...

or use an inserter. if my 'read' function isn't smart enough to just take a vector and fill it with what's read, I have programmed the wrong function.

...or missed the point.

...

...
That can be simplified by a wrapper class.

Does a wrapper class make more sense now?

...

...
...
...
Having said all that, it really should be wrapped in a buffer class to prevent mistakes. Perhaps that has the makings of a useful Boost addition in its own right.

Do you actually do this in your programming ? To "prevent mistakes" ?

I've been doing that enough lately that I should. Indeed, I intend to do that this week because I've thought over the number of times I've managed that by hand over the years. It's past time for me to wrap that logic.

I don't think you like std::vector. For some reason you want something else, but I don't know what that is or why.

std::vector manages memory for me quite nicely, thank you. It can even be used in a loop that dynamically determines the needed allocation.

...

...
...
...
There's also std::array and boost::array.

Sure, but why not use the normal C++ construct.

How are those array classes not "normal C++ construct[s]"? They are appropriate for local allocations of modest sizes, though they can be used in free store allocated objects with much larger sizes. (My 1024-byte example, for example, would do well with array rather than vector.)

Because when you already have a dynamically allocated array in std::vector you need some reason for giving it up and doing something else. Why re-invent the wheel for no reason ?

Allocating from the free store is costlier than off the stack. For local allocations, within reason, stack is much better. A vector data member in an object allocated on the free store means a second free store allocation plus an indirection. The array classes are better in these cases. HTH ___ Rob (Sent from my portable computation engine)

Marshall Clow

12:56 a.m.

On Aug 11, 2013, at 12:39 PM, Rob Stewart wrote:

...

On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class.

Rob -- Could you have meant "resize()" instead of "reserve()" here? -- Marshall Marshall Clow Idio Software mailto:mclow.lists@gmail.com A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki

Jason Roehm

1:25 a.m.

On 08/11/2013 08:56 PM, Marshall Clow wrote:

...

...
Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class. Rob --

Could you have meant "resize()" instead of "reserve()" here?

I think he meant reserve() here, for the reason indicated above the code example. Using std::vector::resize() causes any newly-added elements to be initialized, which may be a waste of effort if you're just passing that vector to a function whose job is to fill it with new data. Using reserve() instead will allocate the required memory but won't initialize it. I agree that the intent of the above code perhaps isn't readily apparent (and if you're going to use it that way, it would make sense to wrap the code to hide that implementation detail), but I can see the argument for using it. I've had some applications in the past where large (multi-GB) buffers were being allocated using std::vector instances, and in some cases, the time it took to default-initialize all of the elements was non-trivial (or even a bottleneck!) Jason

Marshall Clow

2:32 a.m.

On Aug 11, 2013, at 6:25 PM, Jason Roehm wrote:

...

On 08/11/2013 08:56 PM, Marshall Clow wrote:

...
...
Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class. Rob --

Could you have meant "resize()" instead of "reserve()" here?

I think he meant reserve() here, for the reason indicated above the code example. Using std::vector::resize() causes any newly-added elements to be initialized, which may be a waste of effort if you're just passing that vector to a function whose job is to fill it with new data. Using reserve() instead will allocate the required memory but won't initialize it.

Yes, but - it will initialize the contents of the vector by calling (the moral equivalent of) char::char() - which is a no-op - and will generate no code.

...

I agree that the intent of the above code perhaps isn't readily apparent (and if you're going to use it that way, it would make sense to wrap the code to hide that implementation detail), but I can see the argument for using it. I've had some applications in the past where large (multi-GB) buffers were being allocated using std::vector instances, and in some cases, the time it took to default-initialize all of the elements was non-trivial (or even a bottleneck!)

The problem with calling reserve (instead of resize) is that v.size() is still 1, and so the vector believes that it only contains a single character. Accessing the elements of the vector (past the first element) is technically undefined behavior. -- Marshall Marshall Clow Idio Software mailto:mclow.lists@gmail.com A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki

Rob Stewart

9:15 a.m.

On Aug 11, 2013, at 10:32 PM, Marshall Clow wrote:

...

On Aug 11, 2013, at 6:25 PM, Jason Roehm wrote:

...
On 08/11/2013 08:56 PM, Marshall Clow wrote:

...
...
Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class. Rob --

Could you have meant "resize()" instead of "reserve()" here?

I think he meant reserve() here, for the reason indicated above the code example. Using std::vector::resize() causes any newly-added elements to be initialized, which may be a waste of effort if you're just passing that vector to a function whose job is to fill it with new data. Using reserve() instead will allocate the required memory but won't initialize it.

Yes, but - it will initialize the contents of the vector by calling (the moral equivalent of) char::char() - which is a no-op - and will generate no code.

reserve() does not initialize the memory. That's the point.

...

...
I agree that the intent of the above code perhaps isn't readily apparent (and if you're going to use it that way, it would make sense to wrap the code to hide that implementation detail), but I can see the argument for using it. I've had some applications in the past where large (multi-GB) buffers were being allocated using std::vector instances, and in some cases, the time it took to default-initialize all of the elements was non-trivial (or even a bottleneck!)

The problem with calling reserve (instead of resize) is that v.size() is still 1, and so the vector believes that it only contains a single character. Accessing the elements of the vector (past the first element) is technically undefined behavior.

I'm using std::vector to manage a chunk of memory. The start is given by &v[0] and the extent by v.capacity(). In C, you'd malloc() the memory and track its length in a variable. In C++, you could also use shared_array and other types, but the allocation code is explicit, you must still track the length separately, and shared_array has overhead not helpful in this use case. vector is a handy tool for the job, though this is certainly an abuse of the class since you must avoid the temptation to access more than the first element; that one element is the means to an end (or, rather, a beginning ;). ___ Rob (Sent from my portable computation engine)

Rob Stewart

9:03 a.m.

On Aug 11, 2013, at 9:25 PM, Jason Roehm wrote:

...

On 08/11/2013 08:56 PM, Marshall Clow wrote:

...
...
Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class. Rob --

Could you have meant "resize()" instead of "reserve()" here?

I think he meant reserve() here, for the reason indicated above the code example. Using std::vector::resize() causes any newly-added elements to be initialized, which may be a waste of effort if you're just passing that vector to a function whose job is to fill it with new data. Using reserve() instead will allocate the required memory but won't initialize it. I agree that the intent of the above code perhaps isn't readily apparent (and if you're going to use it that way, it would make sense to wrap the code to hide that implementation detail), but I can see the argument for using it. I've had some applications in the past where large (multi-GB) buffers were being allocated using std::vector instances, and in some cases, the time it took to default-initialize all of the elements was non-trivial (or even a bottleneck!)

Bingo! ___ Rob (Sent from my portable computation engine)

Rob Stewart

9:02 a.m.

On Aug 11, 2013, at 8:56 PM, Marshall Clow wrote:

...

On Aug 11, 2013, at 12:39 PM, Rob Stewart wrote:

...
On Aug 11, 2013, at 9:50 AM, Edward Diener wrote:

...
On 8/10/2013 8:59 PM, Rob Stewart wrote:

...
On Aug 10, 2013, at 8:25 AM, Edward Diener wrote:

...
std::vector<TYPE> is the C++ way of specifying arrays.

If you do that, don't use resize() and size(). Use reserve() and capacity(), and don't forget to push_back() one element to make &v[0] valid.

I do not follow this. Especially the bit about pushing back an element. Vectors can be empty as I am sure you know and there are still valid.

Accessing an element of an empty vector is undefined behavior. Calling resize() to reserve space also causes elements to be initialized, which is often not needed when calling an API that indicates the number of bytes written.

What I often do is the following:

std::vector<char> v; v.reserve(1024); v.push_back(0); read(&v[0], v.capacity());

That can be simplified by a wrapper class.

Rob --

Could you have meant "resize()" instead of "reserve()" here?

No. See my reply to Edward. ___ Rob (Sent from my portable computation engine)

niXman

10 Aug 10 Aug

12:37 p.m.

2013/8/10 Geoff Shannon:

...

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

named_pipe_object named_pipe_server::accept(); There is synchronous accept, but there is no asynchronous accept. Why? template<typename F> void named_pipe_server::async_accept(F); -- Regards, niXman ___________________________________________________ Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows: http://sourceforge.net/projects/mingwbuilds/ ___________________________________________________ Another online IDE: http://liveworkspace.org/

Geoff Shannon

10:41 p.m.

On Saturday, August 10, 2013, niXman wrote:

...

named_pipe_object named_pipe_server::accept();

There is synchronous accept, but there is no asynchronous accept. Why?

Mainly for simplicity of the first implementation and because that seems to start overlapping a lot with Boost.Asio territory. Once the initial implementation is working that would be high on the list of new features ;) My intention for implementation targets are linux and windows since I have some experience with those platforms. Under Windows I intend to use the Named Pipe feature so your input would be invaluable Edward. For the unix side I've determined that the most compatible (feature and semantics-wise) mechanism would be Unix domain sockets so it should be fairly trivial to port to OS X and other *nix like platforms (if anything is needed to port it at all). I'll be able to test it personally under Linux, Windows 7 and Mac OS X 10.7 The code (such as it is) is hosted at Githubhttps://github.com/ezephyr/named-pipe . -- Geoff Nothing is ever easy.

Edward Diener

11 Aug 11 Aug

1:46 p.m.

On 8/10/2013 6:41 PM, Geoff Shannon wrote:

...

On Saturday, August 10, 2013, niXman wrote:

...
named_pipe_object named_pipe_server::accept();

There is synchronous accept, but there is no asynchronous accept. Why?

Mainly for simplicity of the first implementation and because that seems to start overlapping a lot with Boost.Asio territory.

OK, but you still need an explanation of how the end-user should use your named pipe functionality to do asio with each member function in your class where asio is possible. Using asio with named pipes in Windows involves making a connection, accepting a connection, sending data, and receiving data. In other words using asio with named pipes means never waiting on anything, but checking periodically if some operation has completed and doing other things if it has not.

...

Once the initial implementation is working that would be high on the list of new features ;)

My intention for implementation targets are linux and windows since I have some experience with those platforms. Under Windows I intend to use the Named Pipe feature so your input would be invaluable Edward.

Glad to give it, but you don't have much code yet on which to comment. I would suggest you keep things really, really simple in your main interface. There should be the creation of a named pipe, listening for and accepting a connection, making a connection, reading data, and writing data. I think you should also consider your named pipe being outside of Boost.ASIO rather than as part of it, mainly because named pipes can be synchronous if the user wants to wait, let's say in some background thread, for operations to complete.

Geoff Shannon

12 Aug 12 Aug

12:09 a.m.

On Sun, Aug 11, 2013 at 6:46 AM, Edward Diener wrote:

...

OK, but you still need an explanation of how the end-user should use your named pipe functionality to do asio with each member function in your class where asio is possible. Using asio with named pipes in Windows involves making a connection, accepting a connection, sending data, and receiving data. In other words using asio with named pipes means never waiting on anything, but checking periodically if some operation has completed and doing other things if it has not.

I'll keep that in mind. For now I'm going to completely punt on asio behavior. Glad to give it, but you don't have much code yet on which to comment.

...

I was speaking more in the abstract. I'll be sure to let you know when I've actually got an attempt at the Windows implementation.

...

I would suggest you keep things really, really simple in your main interface. There should be the creation of a named pipe, listening for and accepting a connection, making a connection, reading data, and writing data.

I think you should also consider your named pipe being outside of Boost.ASIO rather than as part of it, mainly because named pipes can be synchronous if the user wants to wait, let's say in some background thread, for operations to complete.

This is an excellent point. I think I was getting a little focused on Asio because it seems like a good idea, but I really know next to nothing about it so there's no point in trying to implement it in the first pass. On Sun, Aug 11, 2013 at 6:55 AM, Edward Diener wrote:

...

I heartily agree with this last comment. KISS. There's no need for two different classes. A single named pipe class should be able to handle creation of a named pipe, listening for and accepting a connection, making a connection, reading data, and writing data. It really is that simple at the basic level.

Fair enough. Since both you and Rob seem to be clear on that I'll pursue that approach. -- Geoff Nothing is ever easy.

Edward Diener

10 Aug 10 Aug

5:20 p.m.

On 8/9/2013 9:45 PM, Geoff Shannon wrote:

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

Please point out where your named pipe implementation exists. Also documenting your name pipe implementation, for whatever you have implemented or intend to implement, is always the best way to get other people's interest. I have worked extensively with named pipes on Windows so I have some decent knowledge of how things might work.

Rob Stewart

11 Aug 11 Aug

1:55 a.m.

On Aug 9, 2013, at 9:45 PM, Geoff Shannon wrote:

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

named_pipe_object is an odd name. Why "object"? Construction with char const * attaches to an existing pipe, while with a std::string it creates a pipe? That's horrible. Those ctors should have the same behavior. Use an enumerated type argument to distinguish on behavior from the other or use the Named Constructor Idiom. "some" in read_some() and write_some() is not helpful. How does read_some() know the size of the supplied buffer? You need a size argument (or buffer type discussed separately). write_some() also needs a size argument; you must account for binary data. I don't think named_pipe_server is needed. One creates or connects to a named pipe. A single class can handle both aspects, through different constructors. ___ Rob (Sent from my portable computation engine)

Geoff Shannon

3:47 a.m.

Lots of good feedback, thank you! On Sat, Aug 10, 2013 at 6:55 PM, Rob Stewart wrote:

...

named_pipe_object is an odd name. Why "object"?

For consistency with the interprocess library. For instance creating a shared memory object you write:

shared_memory_object shm; I actually don't like it either,

...

Construction with char const * attaches to an existing pipe, while with a std::string it creates a pipe? That's horrible. Those ctors should have the same behavior. Use an enumerated type argument to distinguish on behavior from the other or use the Named Constructor Idiom.

Thanks for the catch, but that's really just a "typo" of me editing one piece of documentation and not the other. They will definitely have the same behaviour.

...

"some" in read_some() and write_some() is not helpful.

This again was for consistency with the Boost.Asio convertion of differentiating reads/writes that will block until the size is full, vs the more low level construct of write-what-you-can/read-what-is-available. How does read_some() know the size of the supplied buffer? You need a size

...

argument (or buffer type discussed separately).

I definitely favour the buffer type instead of size argument. It looks like Boost.Asio already has a buffer typehttp://think-async.com/Asio/asio-1.4.8/doc/asio/reference/buffer.htmlthat can be used to wrap builtin arrays, std::vector or boost::array. It seems useful enough that it would be worth the coupling to use it. Also, it seems like this functionality wants to live halfway between Asio and Interprocess :) I don't think named_pipe_server is needed. One creates or connects to a

...

named pipe. A single class can handle both aspects, through different constructors.

I disagree, and here's why. With a named pipe you can either act as a server, and be prepared to accept multiple incoming connections or you can be a client and connect to a "server." With a separate "server" object this is fairly natural I think: named_pipe_server pipeserv("pipename"); for (;;) { named_pipe cli_conn = pipeserv.accept(); // Do stuff with the new client } Whereas if you had to use named_pipe for everything there's the potential for changing the name, and thus creating a completely new pipe (instead of another connection on it). There is definitely another use-case for a one-to-one connection. That would be a good candidate for an alternate factory function on named_pipe, but I think the separate server object is still useful. Also, here's the new version with your all's feedback incorporated. -- Geoff Nothing is ever easy.

Rob Stewart

11:24 a.m.

On Aug 10, 2013, at 11:47 PM, Geoff Shannon wrote:

...

On Sat, Aug 10, 2013 at 6:55 PM, Rob Stewart wrote:

...
named_pipe_object is an odd name. Why "object"?

For consistency with the interprocess library.

Consistency is good, but not when carried too far. You intend this code to be part of Interprocess, so consistency with that library is good, when sensible. I don't think shared_memory_object is an appropriate class name, so copying it perpetuates the fault, IMO.

...

For instance creating a shared memory object you write:

shared_memory_object shm;

I actually don't like it either,

Even if you keep your server class, the connection manager type doesn't need "object" in its name to distinguish it.

...

...
"some" in read_some() and write_some() is not helpful.

This again was for consistency with the Boost.Asio convertion of differentiating reads/writes that will block until the size is full, vs the more low level construct of write-what-you-can/read-what-is-available.

How many other libraries will you try to be consistent with? Seriously, precedent can be good, but you can get carried away. Will you offer blocking and non-blocking read/write calls? If not, I see no advantage to "some".

...

...
How does read_some() know the size of the supplied buffer? You need a size argument (or buffer type discussed separately). I definitely favour the buffer type instead of size argument. It looks like Boost.Asio already has a buffer typehttp://think-async.com/Asio/asio-1.4.8/doc/asio/reference/buffer.htmlthat can be used to wrap builtin arrays, std::vector or boost::array. It seems useful enough that it would be worth the coupling to use it. Also, it seems like this functionality wants to live halfway between Asio and Interprocess :)

IIRC, that class doesn't manage any memory. It's just an adapter for various forms of buffers. There's still room for something like I mentioned in my other mail, especially if the API allocates on the caller's behalf.

...

...
I don't think named_pipe_server is needed. One creates or connects to a named pipe. A single class can handle both aspects, through different constructors. I disagree, and here's why. With a named pipe you can either act as a server, and be prepared to accept multiple incoming connections or you can be a client and connect to a "server."

Perhaps you're thinking differently than I. On Linux, for example, creating a named pipe (FIFO) means calling mkfifo() followed by two calls to open(), all using the same pathname....

...

With a separate "server" object this is fairly natural I think:

named_pipe_server pipeserv("pipename");

for (;;) { named_pipe cli_conn = pipeserv.accept();

// Do stuff with the new client }

I presume your server ctor would call mkfifo() and accept() would call open(). How does accept() wait for a connection? Are you implying some underlying IPC for the client process to send notification of its desire to connect?

...

Whereas if you had to use named_pipe for everything there's the potential for changing the name, and thus creating a completely new pipe (instead of another connection on it).

I don't see it. If the create + open and open only constructors are distinct, there's no more protection in separate class names. Use the Named Constructor Idiom for more clarity if you like.

...

There is definitely another use-case for a one-to-one connection. That would be a good candidate for an alternate factory function on named_pipe, but I think the separate server object is still useful.

I don't understand your server idea.

...

Also, here's the new version with your all's feedback incorporated.

Much improved. FYI, "setup" is a noun, not a verb, so "open and setup" should be "open and set up". ___ Rob (Sent from my portable computation engine)

Geoff Shannon

12 Aug 12 Aug

12:20 a.m.

On Sun, Aug 11, 2013 at 4:24 AM, Rob Stewart wrote:

...

How many other libraries will you try to be consistent with? Seriously, precedent can be good, but you can get carried away. Will you offer blocking and non-blocking read/write calls? If not, I see no advantage to "some".

For the most part the consistency idea came from deriving a place to start from. It was a helpful thing to do but your point is well taken.

...

IIRC, that class doesn't manage any memory. It's just an adapter for various forms of buffers. There's still room for something like I mentioned in my other mail, especially if the API allocates on the caller's behalf.

I don't think the API should allocate on the caller's behalf. Hence the adaptor buffer seems like a good fit because it allows the user to use whatever is natural for their application. If you have a good reason for thinking that it should I'd like to hear it. Perhaps you're thinking differently than I. On Linux, for example, creating

...

a named pipe (FIFO) means calling mkfifo() followed by two calls to open(), all using the same pathname....

I presume your server ctor would call mkfifo() and accept() would call open(). How does accept() wait for a connection? Are you implying some underlying IPC for the client process to send notification of its desire to connect?

On Sat, Aug 10, 2013 at 3:41 PM, Geoff Shannon wrote:

...

For the unix side I've determined that the most compatible (feature and semantics-wise) mechanism would be Unix domain sockets.

I'm not going to be using mkfifo. It's not full duplex, where unix domain sockets are. Plus the semantics of creation and destruction are slightly more similar for unix sockets and windows named pipes.

...

...
Whereas if you had to use named_pipe for everything there's the potential for changing the name, and thus creating a completely new pipe (instead of another connection on it).

I don't see it. If the create + open and open only constructors are distinct, there's no more protection in separate class names. Use the Named Constructor Idiom for more clarity if you like.

This was a specious argument on my part, though it seemed real to me at the time. As mentioned in my other email I'll be pursuing this change in interface. -- Geoff Nothing is ever easy.

Rob Stewart

9:41 a.m.

On Aug 11, 2013, at 8:20 PM, Geoff Shannon wrote:

...

On Sun, Aug 11, 2013 at 4:24 AM, Rob Stewart wrote:

...
IIRC, that class doesn't manage any memory. It's just an adapter for various forms of buffers. There's still room for something like I mentioned in my other mail, especially if the API allocates on the caller's behalf.

I don't think the API should allocate on the caller's behalf. Hence the adaptor buffer seems like a good fit because it allows the user to use whatever is natural for their application. If you have a good reason for thinking that it should I'd like to hear it.

I wasn't advocating for API allocation, just allowing for it. Sometimes, the API knows how many bytes are needed to read the available data and the caller doesn't. To get the information to the caller, some APIs use a zero length buffer to signal the return of the needed length so the caller can allocate and call the API again. In those cases, it is simpler for the API to allocate. However, that also means the caller is responsible for releasing the memory. In those cases, I resort to returning std::auto_ptr (gasp!) (unique_ptr in C ++ 11). That, of course, forces free store allocation.

...

...
Perhaps you're thinking differently than I. On Linux, for example, creating a named pipe (FIFO) means calling mkfifo() followed by two calls to open(), all using the same pathname....

I presume your server ctor would call mkfifo() and accept() would call open(). How does accept() wait for a connection? Are you implying some underlying IPC for the client process to send notification of its desire to connect?

On Sat, Aug 10, 2013 at 3:41 PM, Geoff Shannon wrote:

...
For the unix side I've determined that the most compatible (feature and semantics-wise) mechanism would be Unix domain sockets.

I'm not going to be using mkfifo. It's not full duplex, where unix domain sockets are.

Creating two pipes gains full duplex.

...

Plus the semantics of creation and destruction are slightly more similar for unix sockets and windows named pipes.

I can see how that would be desirable. ___ Rob (Sent from my portable computation engine)

Edward Diener

11 Aug 11 Aug

1:55 p.m.

On 8/10/2013 9:55 PM, Rob Stewart wrote:

...

On Aug 9, 2013, at 9:45 PM, Geoff Shannon wrote:

...
I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

named_pipe_object is an odd name. Why "object"?

Construction with char const * attaches to an existing pipe, while with a std::string it creates a pipe? That's horrible. Those ctors should have the same behavior. Use an enumerated type argument to distinguish on behavior from the other or use the Named Constructor Idiom.

"some" in read_some() and write_some() is not helpful.

How does read_some() know the size of the supplied buffer? You need a size argument (or buffer type discussed separately).

write_some() also needs a size argument; you must account for binary data.

I don't think named_pipe_server is needed. One creates or connects to a named pipe. A single class can handle both aspects, through different constructors.

I heartily agree with this last comment. KISS. There's no need for two different classes. A single named pipe class should be able to handle creation of a named pipe, listening for and accepting a connection, making a connection, reading data, and writing data. It really is that simple at the basic level.

Boris Schaeling

12:11 p.m.

On Sat, 10 Aug 2013 03:45:38 +0200, Geoff Shannon wrote: Hi Geoff,

...

[...]As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

I'm interested in named pipe support as Boost.Process could benefit a lot (not an official Boost library but everyone calls it like that). We have a function called create_pipe() which creates an anonymous pipe (see http://www.highscore.de/boost/process0.5/boost/process/create_pipe_id341354....). The function returns an object which is similar to a pair of file descriptors/Windows handles (see http://www.highscore.de/boost/process0.5/boost/process/pipe.html). The library refers to these things as pipe ends. The pipe ends can be used to create a boost::iostreams::file_descriptor_source or boost::iostreams::file_descriptor_sink object to do synchronous I/O (see http://www.highscore.de/boost/process0.5/boost_process/tutorial.html#boost_p...). For asynchronous I/O Windows requires a named pipe. Boost.Process doesn't provide anything out of the box to help users though. There is a create_async_pipe() function used in the documentation. But it's left as an exercise to the reader to implement it. :/ I actually implemented it in the unit tests. But there is too much hardcoded that it's of any use to others. This function create_async_pipe() returns the same pair of file descriptors/Windows handles as create_pipe() does. Instead of putting these pipe ends into Boost.Iostreams objects they are now used to instantiate boost::asio::windows::stream_handle or boost::asio::posix::stream_descriptor (see http://www.highscore.de/boost/process0.5/boost_process/tutorial.html#boost_p...). These classes from Boost.Asio can be used as is for asynchronous I/O with named pipes. All of that is really a low-level view as you actually deal with file descriptors/Windows handles. I don't know whether you want to go that way (your interface makes me think you are higher level). If you can provide a Boost.Asio I/O object for named pipes, or if you can provide access to the pipe ends of your named pipe it would however help to make asynchronous I/O with Boost.Process easier to use. Boris

yuri kilochek

12 Aug 12 Aug

5:09 a.m.

Why not derive from std::streambuf overriding appropriate functions? Isn't it the standard way to deal with custom data streams? 10 авг. 2013 г. 11:42 пользователь "Geoff Shannon" написал:

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

Thanks! -- Geoff

Nothing is ever easy.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Geoff Shannon

5:21 a.m.

On Sun, Aug 11, 2013 at 10:09 PM, yuri kilochek wrote:

...

Why not derive from std::streambuf overriding appropriate functions? Isn't it the standard way to deal with custom data streams?

It doesn't seem like it's quite what I'm looking for. For instance this SO questionhttp://stackoverflow.com/questions/8116541/what-exactly-is-streambuf-how-do-...about streambuf seems to indicate that it's something of an implementation detail, and thus not appropriate for a high-level interface. I think the Boost.Asio buffer class is much more appropriate. -- Geoff Nothing is ever easy.

yuri kilochek

7:26 a.m.

Exactly. It is a low level thing supposed to be wrapped in std::basic_istream/std::basic_ostream for high level stuff. Do not reinvent the wheel, study the design of standard library if you want consistency. 12 авг. 2013 г. 11:12 пользователь "Geoff Shannon" написал:

...

On Sun, Aug 11, 2013 at 10:09 PM, yuri kilochek
...
wrote:

...
Why not derive from std::streambuf overriding appropriate functions? Isn't it the standard way to deal with custom data streams?

It doesn't seem like it's quite what I'm looking for. For instance this SO question< http://stackoverflow.com/questions/8116541/what-exactly-is-streambuf-how-do-...

...
about streambuf seems to indicate that it's something of an implementation detail, and thus not appropriate for a high-level interface. I think the Boost.Asio buffer class is much more appropriate.

-- Geoff

Nothing is ever easy.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Rob Stewart

9:30 a.m.

On Aug 12, 2013, at 1:09 AM, yuri kilochek wrote:

...

Why not derive from std::streambuf overriding appropriate functions? Isn't it the standard way to deal with custom data streams?

Please don't top post or overquote on this list. streambufs have virtual functions which add unwanted generalization for this purpose. ___ Rob (Sent from my portable computation engine)

yuri kilochek

4:21 p.m.

...

Please don't top post or overquote on this list. Okay, though it feels strange to reply to the latest post when I actually am commenting on the first one. streambufs have virtual functions which add unwanted generalization for

On 12 Aug 2013 at 13:30 Rob Stewart wrote: this purpose. Why exactly is generalization unwanted? On 12 Aug 2013 at 19:59 Niall Douglas wrote:

...

I would URGE you to make this exclusively a Boost.ASIO implementation. This is an even better idea. +1.

Niall Douglas

3:58 p.m.

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

I would URGE you to make this exclusively a Boost.ASIO implementation. That is the correct fit for pipes, moreover Boost.ASIO does almost all the heavy lifting for you, especially on Windows where I suspect a Win32 named pipe implementation will take about eight lines of new code as the ASIO IOCP backend "just works". There is an IOCP example in the ASIO documentation which I believe even implements those eight lines for you :) That solves your data buffers problem, because ASIO already implements scatter/gather buffer i/o for you. ASIO also already integrates with STL iostreams and Boost.iostreams for you. Regarding some of the other comments on this list regarding your proposal: 1. Any async op can be made synchronous by waiting on it. Therefore an ASIO based named pipe is just as easy to use synchronously as asynchronously. 2. ASIO is the proposed foundation for the future C++ standard networking library, so adding named pipes there is exactly the right place to enter any future C++ standard. 3. There seemed to be some confusion regarding named pipes on POSIX. The only difference between Windows and POSIX named pipes is that the former use the NT kernel namespace, within which the filing systems are mount points, whereas the latter use the filing system namespace directly. In my own code, I use a magic directory in /tmp as the namespace for all my named pipes in the system in an attempt to replicate a similar behavior to Windows, but there are many other ways of doing the same thing. Anonymous pipes are also slightly different - on POSIX they really are anonymous, whereas on Windows they get given a random hex id for their name. There is absolutely no reason why on POSIX you can't also use a random hex id for their name, and indeed that is also exactly what my own code does. BTW are you aware of the +24 buffer size bug in Windows' pipe implementation? It's a known bug since very early NT days :) Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.

Geoff Shannon

11:27 p.m.

On Mon, Aug 12, 2013 at 8:58 AM, Niall Douglas wrote:

...

I would URGE you to make this exclusively a Boost.ASIO implementation. That is the correct fit for pipes, moreover Boost.ASIO does almost all the heavy lifting for you, especially on Windows where I suspect a Win32 named pipe implementation will take about eight lines of new code as the ASIO IOCP backend "just works". There is an IOCP example in the ASIO documentation which I believe even implements those eight lines for you :)

That solves your data buffers problem, because ASIO already implements scatter/gather buffer i/o for you. ASIO also already integrates with STL iostreams and Boost.iostreams for you.

Regarding some of the other comments on this list regarding your proposal:

1. Any async op can be made synchronous by waiting on it. Therefore an ASIO based named pipe is just as easy to use synchronously as asynchronously.

2. ASIO is the proposed foundation for the future C++ standard networking library, so adding named pipes there is exactly the right place to enter any future C++ standard.

I like this idea. I wasn't sure it was appropriate though and frankly, from having poked through the source a bit, ASIO is a bit intimidating in the way it's structured. But I'm sure that's mostly a lack of familiarity with it and only vaguely understanding the core principles which can surely be remedied by some study of the documentation.

...

3. There seemed to be some confusion regarding named pipes on POSIX. The only difference between Windows and POSIX named pipes is that the former use the NT kernel namespace, within which the filing systems are mount points, whereas the latter use the filing system namespace directly. In my own code, I use a magic directory in /tmp as the namespace for all my named pipes in the system in an attempt to replicate a similar behavior to Windows, but there are many other ways of doing the same thing.

Hmmm, I see what you're saying and I agree with it, but I'm also not sure that it's strictly true that this is the "only" difference. Also, to clarify when you say POSIX named pipes, you are talking about the mkfifo call, correct? And is there a reason why the mkfifo call is more desirable to use than UNIX domain sockets?

...

Anonymous pipes are also slightly different - on POSIX they really are anonymous, whereas on Windows they get given a random hex id for their name. There is absolutely no reason why on POSIX you can't also use a random hex id for their name, and indeed that is also exactly what my own code does.

BTW are you aware of the +24 buffer size bug in Windows' pipe implementation? It's a known bug since very early NT days :)

I was not! Thanks for the tip :) Also, FYI, I'm doing this project as an independent study through my CS program. I'm getting close to the end of the quarter and I need to have something concrete to show for my efforts. Since I've already started down the path of implementing this not inside of Boost.ASIO, for the purposes of my school project I'm going to continue with that. However, I plan to continue working on it after the scholastic bit is finished, and then I would be interested in implementing it as part of ASIO. -- Geoff Nothing is ever easy.

Niall Douglas

13 Aug 13 Aug

3:02 p.m.

...

...
I would URGE you to make this exclusively a Boost.ASIO implementation.

I like this idea. I wasn't sure it was appropriate though and frankly, from having poked through the source a bit, ASIO is a bit intimidating in the way it's structured. But I'm sure that's mostly a lack of familiarity with it and only vaguely understanding the core principles which can surely be remedied by some study of the documentation.

...

...
3. There seemed to be some confusion regarding named pipes on POSIX. The only difference between Windows and POSIX named pipes is that the former use the NT kernel namespace, within which the filing systems are mount points, whereas the latter use the filing system namespace directly. In my own code, I use a magic directory in /tmp as the namespace for all my named pipes in the system in an attempt to replicate a similar behavior to Windows, but there are many other ways of doing the same thing.

Hmmm, I see what you're saying and I agree with it, but I'm also not sure

I hear you on ASIO's intimidation! (https://ci.nedprod.com/view/All/job/Boost.AFIO%20Build%20Documentation/Boos t.AFIO_Documentation/doc/html/afio/introduction.html) Due to a failure of its maintainer to timely address the bug https://svn.boost.org/trac/boost/ticket/8669 (with possible dupes https://svn.boost.org/trac/boost/ticket/8933 and https://svn.boost.org/trac/boost/ticket/8935), I ended up having to reimplement a bug free alternative version which actually was quite painless to do once you figure out how ASIO works. The hard part was definitely trawling the source code to figure out why and how, but once you reach understanding it's all fairly obvious. BTW you should read the comments at https://svn.boost.org/trac/boost/ticket/8933 as they mention an existing named pipe implementation for Windows ASIO. Contacting the author might save you some time. that

...

it's strictly true that this is the "only" difference.

Correct, I meant "only semantic difference".

...

Also, to clarify when you say POSIX named pipes, you are talking about the mkfifo call, correct?

I'm talking about any pipe which can be uniquely identified within the system by some string identifier.

...

And is there a reason why the mkfifo call is more desirable to use than UNIX domain sockets?

...

Also, FYI, I'm doing this project as an independent study through my CS

Only consistency with Windows. Windows' pipes do offer message datagram semantics like unix domain sockets, so one could duplicate semantics there too. In my own personal code though I have never found the need. program.

...

I'm getting close to the end of the quarter and I need to have something concrete to show for my efforts. Since I've already started down the path of implementing this not inside of Boost.ASIO, for the purposes of my school project I'm going to continue with that. However, I plan to continue working on it after the scholastic bit is finished, and then I would be interested in implementing it as part of ASIO.

Sure, no problem. I would still believe though that you'll write far less code and it will be less effort to debug if it's exclusively ASIO. On that basis, I would even say it's worth chalking up your existing implementation as a learning experience and starting again. That said, if your compsci department was like mine back in the day, they give better grades to longer code which doesn't test the domain knowledge of the grader. An ASIO implementation would be too short and too domain knowledge specific for good grades I think. It's one of the reasons I got an average 32% during my Software Engineering degree. Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.

Geoff Shannon

18 Aug 18 Aug

1:49 a.m.

On Fri, Aug 9, 2013 at 6:45 PM, Geoff Shannon wrote:

...

I emailed the list a while ago that I was starting to work on a named pipe implementation for potential inclusion in the Boost.Interprocess library, and now I finally have some concrete evidence of progress. I've come up with a header file for my proposed interface and I would really appreciate any and all feedback on it.

Particularly, I'm interested in ideas for an appropriate replacement for using char *'s as the data buffers for reading and writing.

As a point of interest this interface is meant to be extremely simple. I don't intend to initially support ANYTHING except for basic creation and reading and writing.

Thanks! -- Geoff

Nothing is ever easy.

Hey all, So I'm officially finished with my independent study on named pipes, and I thought I'd post and let you all know that a working implementation exists. It's ugly, primitive and lacking most of the features one would actually want to use, but it exists :) The source is on github https://github.com/ezephyr/named-pipe. After I get back from a week-long camping trip, I'll be starting to work on reimplementing this as part of Boost.ASIO, which will be a much more useful end product I think. Thanks for the feedback and suggestions! Cheers, -- Geoff Nothing is ever easy.

4208

Age (days ago)

4216

Last active (days ago)

List overview

Download

36 comments

9 participants

participants (9)

Boris Schaeling
Edward Diener
Geoff Shannon
Jason Roehm
Marshall Clow
Niall Douglas
niXman
Rob Stewart
yuri kilochek