Em sex., 8 de abr. de 2022 às 16:52, Marcelo Zimbres Silva < mzimbres@gmail.com> escreveu:
Now, from your example:
std::string request, read_buffer, response; // ... co_await resp3::async_read(socket, dynamic_buffer(read_buffer)); // Hello (ignored). co_await resp3::async_read(socket, dynamic_buffer(read_buffer), adapt(response)); // Set co_await resp3::async_read(socket, dynamic_buffer(read_buffer)); // Quit (ignored)
By recreating the dynamic buffer every time, you discard the "native" capacity property from the dynamic buffer.
Also I don't see a return value to indicate the current message size
The async_read completion handler has the following signature void(boost::system::error_code, std::size_t), where the second argument is the number of bytes that have been read i.e. the size of the message. Users can keep it if they judge necessary, namely
auto n = co_await resp3::async_read(...);
so I know what to discard from the current buffer. You always discard the current message yourself (and a second argument must be supplied if the user wants to save the response).
Yes, Aedis will consume the buffer as it parses the message, in a manner similar to the async_read_until - erase pattern [1]. Each new chunk of data is handed to the user and erased afterwards, this is efficient as it doesn't require much memory to read messages.
Can you clarify what you mean by "erased afterwards"? Afterwards when? Before or after the delivery to the user? When? I need to know when before I can comment much further. It's not clear to me at all how aedis manages the buffer. If the buffer were an internal implementation detail (as in wrapping the underlying socket) I wouldn't care, but as it is... it's part of the public interface and I must understand how to use it. The downside of the *async_read_until - erase* pattern however is that
it rotates the buffer constantly with x.consume(n)), adding some latency.
However, this could be mitigated with a dynamic_buffer implementation that consumes the content less eagerly (in exchange to increased memory use).
Golang's bufio.Scanner implementation will avoid excessive memcopies to the head of the buffer by using a "moving window" over the buffer. It only uses the tail of the buffer to new read operations. Only when the buffer fully fills it'll memcpy the current message to the head of the buffer as to have more space.
If the current message was kept in the buffer then the
response could just hold string_views to it. What are your thoughts?
As I said above, response adapters will have to act on each new chunk of resp3 data, afterwards that memory will be overwritten with new data when the buffer is rotated and won't be available anymore.
I consider this a good thing, for example, if you are receiving json files from Redis, it is better to deserialize it as content becomes available than keeping it in intermediate storage to be processed later. I can't come up with a use case where this could be desirable.
The pattern to parse the textual protocol is simple: get message, process
message, discard message.
Upon accumulating a whole message, you decode its fields.
Does redis's usage pattern feel similar to this? If it doesn't, then how
does it differ? If it differs, I should reevaluate my thoughts for this
discussion.
As for "[rather] than keeping it in intermediate storage", that's more
complex. The deserialized object *is* intermediate storage. The question
is: can I use pointers to the original stream to put less pressure on the
allocator (even if we customize the allocator, the gains only accumulate)?
For instance, suppose the deserialized object is map