Hi Gavin,
In theory the filtering_stream 'filter' and 'device' layers should pass characters, EOF, and WOULD_BLOCK. Unfortunately the code snippet I posted suppresses the propagation of the WOULD_BLOCK return value from the 'device' layer to the 'filter' layer.
Generally I would recommend using only blocking or async code. Non-blocking is a holdover from when async didn't really exist yet, and with ASIO you shouldn't have that excuse.
Right, I'd be happy using async code, but this particular feature is new with filtering_streams - per the documentation http://www.boost.org/doc/libs/1_57_0/libs/iostreams/doc/guide/asynchronous.h... Since async support is new, it might not be fully supported (or at least fully debugged). Now that I am getting more familiar with the boost code, I need to dig around a little more in the Boost.Iostream example and test folders.
The Boost chat client/server has similar features to my problem, i.e., it involves encoding/decoding messages. The encoding/decoding is different than my case, since the chat protocol adds a header with the message length. This simplifies the socket read code, since you can read the fixed-length header, then read the now known) message length, i.e., each call to read has a fixed length parameter. In my case, the message length is unknown, and depends on the content of the message (since data can be escaped). The data stream is generated by a field-programmable gate array (FPGA), and adding a buffer to determine the message length before sending the response would use too many resources, so the encoding protocol cannot easily be changed.
My point is that theoretically this shouldn't be a problem, as long as the lowest level (actual device reading) behaves as described earlier.
When you read(buffer, 512) it would block until the device provides some data; perhaps it sends a 28 byte packet, of which the driver eagerly grabs the first 8 bytes and so read() returns 8; you look at that data, decide it's not a complete message yet, so you stash it into a separate buffer and call read(buffer, 512) again. (There are other approaches if you want zero-copy.) This time the driver already had the remaining 20 bytes queued up and so read() immediately returns 20. You tack those bytes onto the end of your prior stash, parse the whole thing, and now you've got a message you can return to a higher layer, and then go back to reading. Async works similarly, you just break the code up a little more so that it can return while it's waiting.
Yes, this is what I'm in the process of getting working with my modified version of the chat client/server. The async_read_until callbacks do nicely sequence through my simplified packet protocol.
(Just remember that it might also return a few bytes from the start of the *next* message, so you need to strip exactly one message out of the stash and keep accumulating bytes for the next message.)
Yep, I ran a few tests where I ensured that two complete messages were received into a boost::asio::streambuf and confirmed that read_until '[' returned the SOP first index, read_until ']' returned the first EOP index, I then consume()'d those characters and repeated the call to read_until and confirmed it returned immediately based on the streambuf contents. So this all works as described by the documentation. --- What is the policy of this list with regards to posting code inline in messages? Once I finish my variation on the chat client/server, I'd be happy to post the code. At a minimum it would provide code for people on the list to review/comment on, and any final version of the code would benefit anyone interested in reading streams containing a different style of packet than that used in the boost example chat client/server. I'll then go back to looking at what I did wrong with filtering_streams 'filter' and 'devices', or see if I can get the async stuff working. Thanks again for the helpful discussion. Cheers, Dave