On 11/08/2016 07:16, knarks@online.de wrote:
Thank you for the response so far. My idea is to prevent unnecessary copy operations when receiving large files with tcp. For this reason I thought about implementing my own stream, which writes direct to file or memory, depending on the situation.
I know that I can connect std::iostream to the asio streambuf but what about moving the received memory to a std::ofstream or std::stringstream without having the data twice in memory. Is that possible?
ASIO does provide posix::stream_descriptor and windows::stream_handle, which can wrap native file descriptors and handles (resp.) and provide async operations on them. Thus you can read from the network into one asio streambuf and then pass the same streambuf to the file write operation. This is probably as close to zero-copy as you can get. Though other than scalability and blocking concerns there's not much difference between this and just doing a blocking write to a regular fstream -- at the point you're hitting disk the cost of copying memory is negligible. For writing direct to memory, you can simply supply the target memory as a buffer to the initial read (making sure to keep track of the origin and size correctly as data blocks incrementally arrive, remembering that short reads are likely). This will quite literally be zero-copy (bar any copies that occur inside the network stack); no special streams required.