On Friday, July 27, 2007 at 20:47:54 (+0100) Richard writes:

Have you tried investing in a replacement memory manager?

Such as?

It might give you a bigger overall payback than just tinkering with one aspect.

Certainly possible, however, it is also possible that the sheer size and number of objects we have in our application makes it inherently difficult for dynamically-allocated memory for small objects to work very well at very high memory usages. Thus the hope for a hybrid static store and ostringstream. Bill

Most of std classes support custom allocators. basic_stringstream and vector do this as well. You can preallocate the memory needed and create objects in this memory locations. For more help see the placement new operator. But your question here is very general. It is for example possible use some COW implementation of your objects together with reference counted content. You objects are not duplicated on every copy, but only when one tries to write to the content. To give you better advices you should provide more specific examples. With Kind Regards, Ovanes On Fri, July 27, 2007 22:17, Richard wrote:

Well - I have zero experience here.

I remember that they were frequently advertised in the back of computer mags a few years back. Try Google? I saw...

http://www.freedownloadscenter.com/Programming/C_and_C++_Tools_and_Component s/SwiftAlloc_Memory_manager_replacement.html

and

http://www.newcodeinc.com/

Another possibility might be to look at how your app uses memory.

If you have vast numbers of objects all the same size you might benefit from a home written replacement for operator new. If you special case them all into a special pool it might get them out of the way of everything else. The general purpose manager that ships with the compiler is just that - general! Sounds like you have a specific problem.

HTH,

- Richard

-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Bill Lear Sent: 27 July 2007 21:01 To: boost-users@lists.boost.org Subject: Re: [Boost-users] Stupid ostringstream question

On Friday, July 27, 2007 at 20:47:54 (+0100) Richard writes:

...

Have you tried investing in a replacement memory manager?

Such as?

...

It might give you a bigger overall payback than just tinkering with one aspect.

Certainly possible, however, it is also possible that the sheer size and number of objects we have in our application makes it inherently difficult for dynamically-allocated memory for small objects to work very well at very high memory usages. Thus the hope for a hybrid static store and ostringstream.

Bill _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

With Kind Regards, Ovanes Markarian

On Fri, July 27, 2007 23:11, Bill Lear wrote:

On Friday, July 27, 2007 at 22:49:22 (+0200) Ovanes Markarian writes:

...

Most of std classes support custom allocators. basic_stringstream and vector do this as well. You can preallocate the memory needed and create objects in this memory locations. For more help see the placement new operator.

Hmm, yes, they do, but they also have a pitfall. If I create a new templated ostringstream with a custom allocator, it then means that the string that is returned also must use this custom allocator, and is incompatible with standard strings. I guess that's not surprising, however, now that I think about it...

I don't think it is so. std::string and a stream class are containers of elements. How they contain them is their own beer. They exchange them for example via operators << and >> where it is defined how the strings are serialized or deserialized to the stream. In these operators you provide as template params Elems stored in stream and traits about these Elems. And the rest is how you handle it. With Kind Regards, Ovanes Markarian

On Sat, July 28, 2007 00:07, Bill Lear wrote: [...]

Well, stringstream defines a type, from g++ header:

typedef basic_string __string_type;

and str() is defined to return this:

__string_type str() { ...}

So, if I have a std::string, and an ostringstream with a different allocator, say my_ostringstream, I cannot do this:

my_ostringstream os; os << ...; std::string s = os.str();

Ok, but you understand that this approach, will probably consume twice as much memory, since str() leaves stream as it is and does not remove characters read. A was meaning this: my_ostringstream os; os << ...; std::string s; os >> s; And this should work with custom allocators in stringstream and string. With Kind Regards, Ovanes Markarian

On 27/07/07, Bill Lear wrote:

Certainly possible, however, it is also possible that the sheer size and number of objects we have in our application makes it inherently difficult for dynamically-allocated memory for small objects to work very well at very high memory usages. Thus the hope for a hybrid static store and ostringstream.

Not sure if it is exactly what you're looking for, but Boost.Pool might be of some use here. Regards, Darren

On Friday, July 27, 2007 at 16:40:05 (-0400) Jeff Flinn writes:

Bill Lear wrote:

...

I have been searching the world in vain for a stringstream that works with a static buffer, and does not use the heap for memory allocation. We have an application that uses lots (12-16Gb) of memory. This puts a strain on the memory allocation routines, and when we want to create

Which platform?

Linux, with Intel and gcc compilers.

Have you looked at the boost::iostream library which abstracts out sources/sinks that you can use with buffers that you control?

Also have you looked at the boost::format library as a typesafe replacement for sprintf?

Hmm, not yet, but thanks for the pointers. Bill

Well, I'd be grateful if you sent it to me. If it's good enough, perhaps one day it could be accepted into boost ... who knows?

ok I send it to 'laer@zopyra.com' wkr, me