[serialization] suggestion for the extension of the functionality
Hello
I would like to ask for consideration of some new functionality for
boost::serialization.
If I have a stream and I do not want to close the stream at the end of
serialization I have to make easy, but nasty trick using stringstream as a
buffer between my stream and serialized stream.
More concrete example. I'am using easy library for sockets. Connection is
represented as stream and after connection I do not want to close it. So when
I try to use serialization though the socket, serialisation is hanging - it is
waiting for the end of the stream, but there is no end of stream because it is
socket.
It will be useful just to add to library one fuction for manual sending data -
"simulation" of the end of stream. To avoid or closing socket either using
stringstreams buffers.
Below is example code for this problem - this code is using buffers, but IMHO
better will be that inside library will be functionality for such a
situation.
Best regards, Seweryn Habdank-Wojewódzki.
Ps. Full code I can send if will be needed.
void do_server()
{
string message("Hello Socket!\nHelloWorld!");
// create class instance
const gps_position g(35, 59, 24.567f);
try
{
TCPSocketWrapper sockserver;
// listen on some port
sockserver.listen(IPportnumber);
cout << "server is ready" << endl;
// accept connection from client
TCPSocketWrapper sock(sockserver.accept());
cout << "accepted connection from: "
<< sock.address() << endl;
// make the stream around the socket wrapper
TCPStream stream(sock);
bool oncemore = true;
int command;
while (oncemore)
{
// read the command
stream >> command;
switch (command)
{
case 1:
{
cout << "command 1" <
Seweryn Habdank-Wojewódzki wrote:
Hello
I would like to ask for consideration of some new functionality for boost::serialization.
If I have a stream and I do not want to close the stream at the end of serialization
I can't see why this is a problem. Does the following not work? { o?stream os("stream_name"); os << ... ; // your own stuff here { text_oarchive oa(os); oa << .... os.flush(); // actually I think this is invoked by default when archive is destroyed } // archive destructor called here - stream stays open // stream is still open here. os << ... ; // more of your own stuff os.close(); } Robert Ramey I have to make easy, but nasty trick using stringstream
as a buffer between my stream and serialized stream.
More concrete example. I'am using easy library for sockets. Connection is represented as stream and after connection I do not want to close it. So when I try to use serialization though the socket, serialisation is hanging - it is waiting for the end of the stream, but there is no end of stream because it is socket.
It will be useful just to add to library one fuction for manual sending data - "simulation" of the end of stream. To avoid or closing socket either using stringstreams buffers.
Below is example code for this problem - this code is using buffers, but IMHO better will be that inside library will be functionality for such a situation.
Best regards, Seweryn Habdank-Wojewódzki.
Ps. Full code I can send if will be needed.
void do_server() { string message("Hello Socket!\nHelloWorld!");
// create class instance const gps_position g(35, 59, 24.567f);
try { TCPSocketWrapper sockserver;
// listen on some port sockserver.listen(IPportnumber);
cout << "server is ready" << endl;
// accept connection from client TCPSocketWrapper sock(sockserver.accept());
cout << "accepted connection from: " << sock.address() << endl;
// make the stream around the socket wrapper TCPStream stream(sock);
bool oncemore = true; int command; while (oncemore) { // read the command stream >> command;
switch (command) { case 1: { cout << "command 1" <
// ------------------------------------------------
// buffer
// ------------------------------------------------
std::ostringstream s;
{ boost::archive::text_oarchive oa(s);
// write class instance to archive oa << g; } //------------------------------------------------------------------
// buffer
//------------------------------------------------------------------ stream << s.str() << endl; }
break; default: cout << "END command" << endl; oncemore = false; break; } } } catch (const SocketRunTimeException &e) { cout << "socket exception: " << e.what() << endl; } }
void do_client() { string message;
// ... some time later restore the class instance to its orginal state gps_position newg;
try { TCPClientStream stream(IPserveraddress, IPportnumber);
bool oncemore = true; int command; while (oncemore) { cout << "1. request for a serialized object" << endl; cout << "other - end" << endl;
cin >> command; stream << command << endl;
switch (command) { case 1: { // ------------------------------------------------
// buffer
// ------------------------------------------------
std::string line; std::getline(stream, line); istringstream s(line);
// ------------------------------------------------
// buffer will be filled
// ------------------------------------------------
{ boost::archive::text_iarchive ia(s);
ia >> newg; }
cout << "received: " << newg << endl; } break; default: oncemore = false; break; } } } catch (const SocketRunTimeException &e) { cout << "socket exception: " << e.what() << endl; } }
\/\/| Seweryn Habdank-Wojewódzki `\/\/ =
Hello # pią stycznia 27 2006 18:31, @ Robert Ramey:
Seweryn Habdank-Wojewódzki wrote:
I would like to ask for consideration of some new functionality for boost::serialization.
If I have a stream and I do not want to close the stream at the end of serialization
I can't see why this is a problem. Does the following not work?
The following works fine. This is just a small suggestion. IMHO it is very nice if I can serialize object (as binary, text or xml) through the socket, isn't it? Using additional buffers, just for creating "end of stream" sequence could make code buggy. Just one function as Pauld said eg. close() or send() will be nice. Best regards. -- |\/\/| Seweryn Habdank-Wojewódzki `\/\/
Seweryn Habdank-Wojewódzki wrote:
The following works fine. This is just a small suggestion. IMHO it is very nice if I can serialize object (as binary, text or xml) through the socket, isn't it?
This is indeed an important application. But the serialization library is not the place to implement it !!! The serialization library depends upon the underlying streambuf (member of ?stream) to implement any actual i/o - it is here that the connection to sockets or special buffering should be made. In short ?socket_ostream os("name of socket, pipe or whatever")" { xml_archive oa(os); for(;;){ oa << ....; os.flush(); // send object to socket } }
Using additional buffers, just for creating "end of stream" sequence could make code buggy. Just one function as Pauld said eg. close() or send() will be nice.
Look in to makeing a custom stream buffer for your purposes. Investigate the iostreams library for this purpose. Robert Ramey
Hi Robert Ramey wrote:
The following works fine. This is just a small suggestion. IMHO it is very nice if I can serialize object (as binary, text or xml) through the socket, isn't it?
This is indeed an important application. But the serialization library is not the place to implement it !!! The serialization library depends upon the underlying streambuf (member of ?stream) to implement any actual i/o - it is here that the connection to sockets or special buffering should be made. In short
?socket_ostream os("name of socket, pipe or whatever")"
{ xml_archive oa(os); for(;;){ oa << ....; os.flush(); // send object to socket } }
Ok, I understand it. It is good solution, thanks.
Look in to makeing a custom stream buffer for your purposes. Investigate the iostreams library for this purpose.
Ok. Best regards. -- |\/\/| Seweryn Habdank-Wojewódzki `\/\/
Hello To sum up our work with boost::serialization. Maybe useful would be to add comment to the serialization documentation that std::endl and std::eof could finalize serialization procedure. IHMO it is not clear there (in documentation). Best regards. -- |\/\/| Seweryn Habdank-Wojewódzki `\/\/
Even better - I've tweaked the destructor on basic_oarchive_primitive to append an std::endl to the stream. So this problem will no longer appear. Note that problem only came up with text archives. Other oarchives never read beyonnd the last data serialized. So on 1.34 the problem is solved once and for all. And all archives classes will be consistent in their behavior in this regard. Robert Ramey Seweryn Habdank-Wojewódzki wrote:
Hello
To sum up our work with boost::serialization. Maybe useful would be to add comment to the serialization documentation that std::endl and std::eof could finalize serialization procedure.
IHMO it is not clear there (in documentation).
Best regards.
\/\/| Seweryn Habdank-Wojewódzki `\/\/ =
Hello At present after clarifing all that thinks with sockets and serialisation it is possible to easily do something like RMI. In serialized object I could send to server parameters and name of function. Then on server side I could take it and do the function (it could be a functor) and than I can send results to the client. Moreover if it is functor I could remember last operation and If I heve another new call but with the same parametres I could just send results without executing function. Best regards, -- |\/\/| Seweryn Habdank-Wojewódzki `\/\/
Seweryn Habdank-Wojewódzki wrote:
Hello
I would like to ask for consideration of some new functionality for boost::serialization.
If I have a stream and I do not want to close the stream at the end of serialization I have to make easy, but nasty trick using stringstream as a buffer between my stream and serialized stream.
I raised this issue a few months ago. It turns out that the only way of closing a stream at the end of serialisation is to leave the scope in which the stream was opened. The reason for this is, I believe, so that a "footer" can be written to the stream indicating the end of the file. It would be handy, as Seweryn suggests, if there could be a close() function available to users that did this, so that users could close the stream when they needed to. One application of this is allowing a stream to be used for writing and then reading within the same scope. At the moment, a new scope is required to do this. Paul
Paul Giaccone wrote:
Seweryn Habdank-Wojewódzki wrote:
I raised this issue a few months ago. It turns out that the only way of closing a stream at the end of serialisation is to leave the scope in which the stream was opened. The reason for this is, I believe, so that a "footer" can be written to the stream indicating the end of the file.
This is a requirement of the xml_archive - I suppose a "close" function could be added to the xml_archive so that one woudn't have to depend on the destructor being called - but then one would want to add it to all the archives where it would be a no-op. It seems to me that it would just not be worth it to avoid the { } alternative. In fact, I rather prefer the {} alternative - it makes more sense to me.
It would be handy, as Seweryn suggests, if there could be a close() function available to users that did this, so that users could close the stream *** you mean "close the archive" - leaving the stream open?*** when they needed to.
One application of this is allowing a
stream to be used for writing and then reading within the same scope. At the moment, a new scope is required to do this.
Robert Ramey
Robert Ramey wrote:
Paul Giaccone wrote:
Seweryn Habdank-Wojewódzki wrote:
I raised this issue a few months ago. It turns out that the only way of closing a stream at the end of serialisation is to leave the scope in which the stream was opened. The reason for this is, I believe, so that a "footer" can be written to the stream indicating the end of the file.
This is a requirement of the xml_archive - I suppose a "close" function could be added to the xml_archive so that one woudn't have to depend on the destructor being called - but then one would want to add it to all the archives where it would be a no-op.
It seems to me that it would just not be worth it to avoid the { } alternative. In fact, I rather prefer the {} alternative - it makes more sense to me.
It would be handy, as Seweryn suggests, if there could be a close() function available to users that did this, so that users could close the stream *** you mean "close the archive" - leaving the stream open?*** when they needed to.
One application of this is allowing a
stream to be used for writing and then reading within the same scope. At the moment, a new scope is required to do this.
Robert Ramey
OK, if it is more hassle than it is worth, that is fine, as I saw you posted a way of writing and reading in the same scope a few days ago. Is the necessity of closing the scope to complete the write in the documentation?
Paul Giaccone wrote:
OK, if it is more hassle than it is worth, that is fine,
well it IS more hassle than its worth - but that's only one reason I would be reluctant to add it. I'm wary about accidently tying serialization to streams. That's why I'm inclined to drag my feet regarding adding "stream like" api names to serialiazation. Of course I'm known to drag my feet regarding adding anything to the serialization library that doesn't absolutely have to be there. This is to keep things actually manageable. Up until now - all the included achives have depended on a stream whose reference is passed during archive construction. In the next version - this native binary archives will depend only on a streambuf. This change is being made to enance performance and will break little if any usercode.
Is the necessity of closing the scope to complete the write in the documentation?
If you don't see it there - its not there. I look for a good place to add a note. Robert Ramey
participants (3)
-
Paul Giaccone
-
Robert Ramey
-
Seweryn Habdank-Wojewódzki