On Wed, Nov 19, 2014 at 1:47 PM, Sylvester-Bradley, Gareth wrote:

...

shared_ptr/unique_ptr work fine for me with FILE*, one only has to write a deleter. A dedicated class would be needed for POSIX file descriptors though.

Peter Sommerlad and Andrew L. Sandoval's unique_resource RAII proposal uses both FILE* and POSIX file descriptors as examples. I think this is the latest revision: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4189.pdf

It does solve the RAII part but it doesn't provide a standard interface for native IO. -- Olaf

On Nov 26, 2014, at 11:10 AM, Olaf van der Spek wrote: [Please don't top-post.]

On Thu, Nov 20, 2014 at 9:10 AM, Olaf van der Spek wrote:

...

On Wed, Nov 19, 2014 at 1:47 PM, Sylvester-Bradley, Gareth wrote:

...

...

shared_ptr/unique_ptr work fine for me with FILE*, one only has to write a deleter. A dedicated class would be needed for POSIX file descriptors though.

Peter Sommerlad and Andrew L. Sandoval's unique_resource RAII proposal uses both FILE* and POSIX file descriptors as examples. I think this is the latest revision: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4189.pdf

It does solve the RAII part but it doesn't provide a standard interface for native IO.

Not many responses... Does noone use native IO?

I use POSIX I/O calls frequently. I adapted Lisa Lippincott and Marshall Clow's work on Nitrogen to create a library called poseven, based on the same 'nucleus' core library. p7::fd_t is an enum type for a file descriptor, and n::ownedp7::fd_t and n::sharedp7::fd_t are smart resources that work like unique_ptr and shared_ptr (loosely speaking) respectively. Much of it is about a decade old, early in my exposure to generic programming, though just recently I added p7::coprocess. Let me know if you're interested in more info. https://github.com/jjuran/metamage_1/tree/master/posix/poseven/poseven/types https://github.com/jjuran/metamage_1/tree/master/posix/poseven/poseven/funct... Josh

On Nov 29, 2014, at 6:16 AM, Olaf van der Spek wrote:

On Wed, Nov 26, 2014 at 9:28 PM, Josh Juran wrote:

...

I use POSIX I/O calls frequently. I adapted Lisa Lippincott and Marshall Clow's work on Nitrogen to create a library called poseven, based on the same 'nucleus' core library. p7::fd_t is an enum type for a file descriptor, and n::ownedp7::fd_t and n::sharedp7::fd_t are smart resources that work like unique_ptr and shared_ptr (loosely speaking) respectively. Much of it is about a decade old, early in my exposure to generic programming, though just recently I added p7::coprocess.

Interesting. Why are you using an enum to store the fd?

For type-safety and efficiency. I don't use int, because that would allow any constant to be misinterpreted as a file descriptor. Nitrogen used to use complicated wrapper classes, but these had several defects: (a) they were slow to compile in the compiler used at the time (and which I still use for classic Mac OS development[1]), (b) the same compiler wouldn't place their values in registers, (c) their values couldn't be used in switch statements, and (d) creating constants for them was problematic, typically resulting in many `static const Foo x = Foo( ::x );` in headers. Using an enum addressed all of these issues (with (d) being at least a partial improvement). One of the downsides of C-style enums is non-portable memory layout. You can ensure an enum is large enough my declaring max_Foo = uint32_t( -1 ), but there's no guarantee that enum Boolean { False, True } occupies a single byte instead of, say, four. I'm new to C++11, but if I understand correctly, its enum classes address this very issue.

Do you think it'd be useful to move parts of this functionality into Boost?

Quite possibly. Poseven is useful if you want to add type-safety and exception-safety to a C-style POSIX codebase (or subset thereof) without rewriting it to use higher-level semantics like ASIO. In debug mode, it throws exceptions containing a backtrace. A key question is how to approach the overlap of poseven with existing Boost libraries like Filesystem and Thread. I think the first step toward adoption would be for me to familiarize myself with Boost.System. Josh [1] Classic Mac OS is relevant in that POSIX programs are supported through MacRelix. The latest compiler to fully support classic Mac OS is Metrowerks C++ 2.4.1 from CodeWarrior Pro 6.3, released in summer 2000.

On November 26, 2014 3:57:06 PM EST, Andrey Semashev wrote:

...

On Thu, Nov 20, 2014 at 9:10 AM, Olaf van der Spek wrote:

...

On Wed, Nov 19, 2014 at 1:47 PM, Sylvester-Bradley, Gareth

wrote:

...

...

shared_ptr/unique_ptr work fine for me with FILE*, one only has to write a deleter. A dedicated class would be needed for POSIX file descriptors though.

Peter Sommerlad and Andrew L. Sandoval's unique_resource RAII

...

...

...

uses both FILE* and POSIX file descriptors as examples. I think

On Wednesday 26 November 2014 20:10:31 Olaf van der Spek wrote: proposal this is

...

...

...

the latest revision: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4189.pdf

It does solve the RAII part but it doesn't provide a standard interface for native IO.

Does noone use native IO?

I do, from time to time. But I always found RAII wrappers sufficient and used the native API to work with file descriptors. I usually need something low-level or non-portable anyway when I do this.

I use filesystem or IOStreams when I want portability, and otherwise use RAII and native APIs when the code isn't portable. ___ Rob (Sent from my portable computation engine)

On November 29, 2014 9:14:50 AM EST, Olaf van der Spek wrote:

On Wed, Nov 26, 2014 at 10:21 PM, Rob Stewart wrote:

...

...

I do, from time to time. But I always found RAII wrappers sufficient and used the native API to work with file descriptors. I usually need something low-level or non-portable anyway when I do this.

I use filesystem or IOStreams when I want portability, and otherwise use RAII and native APIs when the code isn't portable.

What RAII wrappers do you use in those cases?

In some cases, just my scope_guard class to invoke close(), for example, at end-of-scope. In other cases, I have wrappers around native handles to permit testing them and, of course, closing them. ___ Rob (Sent from my portable computation engine)

On December 15, 2014 6:25:44 PM EST, Nat Goodspeed wrote:

Interestingly, I just stumbled across a bit of live legacy code that relies on __gnu_cxx::stdio_filebuf to allow constructing an istream or ostream from an open FILE*. This does not port to libc++.

What surprised me was that I could find no support for FILE* in Boost.Iostreams. Writing a streambuf isn't rocket science, but I would think that this particular use case would be common enough to warrant library support.

Am I overlooking something?

FILE* is buffered and streambuf provides buffering. You can create a streambif that doesn't actually do any buffering, but it is contrary to the purpose of one. ___ Rob (Sent from my portable computation engine)

On 11/26/2014 08:10 PM, Olaf van der Spek wrote:

Not many responses... Does noone use native IO?

The reason I have not responded is because I would much rather have an asynchronous file I/O API (along the lines of Asio) with synchronous convenience functions. But this is a major task.

On 11/29/2014 03:13 PM, Olaf van der Spek wrote:

I don't understand this kind of reasoning. It's not an exclusive or deal is it?

While it is not mutually exclusive, it is easier to build a sync API on top of an async API than the other way around -- see the Half-Sync/ Half-Async design pattern in POSA2. You could design the sync API first, but it should not enforce an awkward design for the async API, and how are you going to ensure that without a reasonable idea about the latter? For example, the async API would need some kind of execution context (same as or similar to asio::io_service), and that should be reflected in the sync API. Regarding your wrapper class, I suggest that you make better use of C++ types. For instance, use filesystem::path (whether std::experimental or boost) for file names, std::io_base::openmode for the mode parameter, define a native_handle_type type instead of exposing FILE directly, and use a native_handle() member function to return it instead of using the conversion operator (this pattern is used in Asio and Networking TS.)

On 12/05/2014 12:29 PM, Olaf van der Spek wrote:

...

std::io_base::openmode for the mode parameter, define a native_handle_type type instead of exposing FILE directly, and

Do you mean a real class (unowned handle) or just a typedef?

Just a typedef. I am thinking about something similar to: http://www.boost.org/doc/html/boost_asio/reference/basic_stream_socket/nativ...

On 29 Nov 2014 at 14:59, Bjorn Reese wrote:

On 11/26/2014 08:10 PM, Olaf van der Spek wrote:

...

Not many responses... Does noone use native IO?

The other problem with anything Boost.Iostreams based is that Iostreams has a severely flawed design in not supporting async i/o. This makes Iostreams useless for almost any task I've needed it for in recent years. It also doesn't help that it lacks a maintainer, and has many annoying bugs (I recently had to use Iostreams due to Boost.Process 0.5 forcing me, and I lost over a week of my time fighting Iostreams to behave when combined with ASIO. It was quite frankly a disaster).

The reason I have not responded is because I would much rather have an asynchronous file I/O API (along the lines of Asio) with synchronous convenience functions. But this is a major task.

As Bjorn only knows too well but Olaf may not, proposed Boost.AFIO (https://boostgsoc13.github.io/boost.afio/doc/html/afio.html) provides portable asynchronous file i/o implementation library extending ASIO. It however does not use the ASIO handler callback idiom as that is (in my opinion, though TBH I am right on this) unsuited for file i/o where _ordering_ of operations is paramount. The ability to easily program ordering makes an enormous difference to ease of use and tweaking algorithms for performance on cold and warm cache file systems. Instead AFIO uses a functional result passing design currently based on plain old futures. Benchmarks for the v1.2 engine are more than plenty for even a top end SSD (https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/closure_pe rformance.html) but latencies are in my opinion not good enough (https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/closure_la tency.html), so the work plan is as follows: v1.3 engine: Port Boost.AFIO to proposed Boost.BindLib, thus making AFIO standalone capable. This part is nearly done, it's mainly all dotting i's and crossing t's to go, mainly on older compilers like GCC 4.6. The red items on the CI dashboard (https://boostgsoc13.github.io/boost.afio/) are those todo fixme support items apart from anything before VS2013 support for which I am dropping. v1.4 engine: Replace spinlocked unordered_map with proposed Boost concurrent_unordered_map. This should yield a substantial performance increase due to hugely improved concurrency and a large decrease in latencies due to eliminating much use of malloc. v1.5 engine: Replace async_io_op with a custom future type based on the non-allocating constexpr continuations based basic_future I proposed on this list some time ago. This should completely eliminate malloc from the AFIO core engine, and I would assume we shall see perfectly optimal performance and latencies in AFIO. A "free" bonus is that AFIO becomes monadically programmable using either proposed Boost.Hana or proposed Boost.Expected, or indeed any monadic C++ programming library of your choice (if it provides basic_future support). This ought to make "simple" use of AFIO much more amenable and its current batch based API is a big gnarly for occasional users. If approved to present at C++ Now, I shall present on all this there for those interested, and I currently intend to submit a talk or two to CppCon. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/