Re: [Boost-users] Boost.Thread memory leak reported
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Peter Dimov Sent: Friday, August 26, 2005 1:27 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Boost.Thread memory leak reported
Sohail Somani wrote:
Christian Henning wrote:
But Peter, we do agree that there is problem and it should be fixed, dont't we?
I see no problems in the code.
Any dynamically allocated memory that isn't freed is a Bad Thing imho.
Because?
Was it forgotten? Was it intentional? If the assumption is it will always be freed, then there is no ambiguity.
Sohail Somani wrote:
Any dynamically allocated memory that isn't freed is a Bad Thing imho.
Because?
Was it forgotten? Was it intentional? If the assumption is it will always be freed, then there is no ambiguity.
Seems clearly intentional to me - the allocation is invoked by call_once and is a textbook implementation of a thread safe singleton. A "this is not a leak" comment may help.
Ah, but does it work on AIX? :-) Sorry, I couldn't resist. (My posts are getting ignored, so I assume nobody uses boost on AIX. Either that or I'm an impatient crybaby. Bummer!) On Aug 26, 2005, at 1:45 PM, Peter Dimov wrote:
Sohail Somani wrote:
Any dynamically allocated memory that isn't freed is a Bad
Thing imho.
Because?
Was it forgotten? Was it intentional? If the assumption is it will always be freed, then there is no ambiguity.
Seems clearly intentional to me - the allocation is invoked by call_once and is a textbook implementation of a thread safe singleton. A "this is not a leak" comment may help.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- The primary task of most software projects is to discover and resolve these unknowns rather than to build a system. -- Philip G. Armour -- Richard Cook ✇ Lawrence Livermore National Laboratory Bldg-453 Rm-4037, Mail Stop L-557 7000 East Avenue, Livermore, CA, 94550, USA ☎ (925) 423-9605 (925) 423-6961 --- Information Management & Graphics Grp., Services & Development Div., Integrated Computing & Communications Dept. (opinions expressed herein are mine and not those of LLNL)
Rich Cook wrote:
Ah, but does it work on AIX? :-) Sorry, I couldn't resist. (My posts are getting ignored, so I assume nobody uses boost on AIX.
This might well be the case. Have you tried replacing the problematic get_ptr_impl()->T::~T(); construct at line 388 of optional.hpp with get_ptr_impl()->~T(); or perhaps get_ptr_impl()->~value_type();
That fixed it! Thank you very much for your valuable help. To be clear, when I replaced line 388 of optional.hpp: get_ptr_impl()->T::~T(); with get_ptr_impl()->~T(); My program compiled fine. On Aug 26, 2005, at 2:20 PM, Peter Dimov wrote:
Rich Cook wrote:
Ah, but does it work on AIX? :-) Sorry, I couldn't resist. (My posts are getting ignored, so I assume nobody uses boost on AIX.
This might well be the case. Have you tried replacing the problematic
get_ptr_impl()->T::~T();
construct at line 388 of optional.hpp with
get_ptr_impl()->~T();
or perhaps
get_ptr_impl()->~value_type();
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Richard Cook ✇ Lawrence Livermore National Laboratory Bldg-453 Rm-4037, Mail Stop L-557 7000 East Avenue, Livermore, CA, 94550, USA ☎ (925) 423-9605 (925) 423-6961 --- Information Management & Graphics Grp., Services & Development Div., Integrated Computing & Communications Dept. (opinions expressed herein are mine and not those of LLNL)
Well, since I started this whole thing, I have to ask... If it's a singleton, doesn't it need to clean itself up? If the assumption is that the OS will clean it up when the process is killed, isn't that a bit shaky as far as sound assumptions go? Elisha
-----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users- bounces@lists.boost.org] On Behalf Of Peter Dimov Sent: Friday, August 26, 2005 1:45 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Boost.Thread memory leak reported
Sohail Somani wrote:
Any dynamically allocated memory that isn't freed is a Bad Thing imho.
Because?
Was it forgotten? Was it intentional? If the assumption is it will always be freed, then there is no ambiguity.
Seems clearly intentional to me - the allocation is invoked by call_once and is a textbook implementation of a thread safe singleton. A "this is not a leak" comment may help.
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Elisha Berns wrote:
Well, since I started this whole thing, I have to ask... If it's a singleton, doesn't it need to clean itself up? If the assumption is that the OS will clean it up when the process is killed, isn't that a bit shaky as far as sound assumptions go?
It's a perfectly sound assumption for today's OSes, even if you rely on them to clean up kernel resources (such as a Win32 HANDLE) and not just memory. Manually releasing everything with program lifetime duration just makes the process exit more slowly (but it keeps simple leak detectors happy). In addition, there is no good place to put the delete. You have to play games with atexit and then deal with the possibily of resurrection (singleton access after it's been deleted.)
I think a comment is fine. Just a question, might be even stupid, since I don't understand the whole thread lib design rationales. But how about putting the threadmon_mutex singleton in a scoped_ptr? Christian
Peter, I don't pretend to understand the design of Boost.Thread, so I can't and won't argue this point in terms of *correct design*, it would be presumptuous of me, to say the least. That said, I can comment on your two points below. For one, not all OSes are reliable. Windows 98 is unreliable as far as releasing/freeing resources from processes that are shut down. It has a poor tolerance for any kind of kernel handles that aren't shut down by the process itself. My firsthand experience with this issue dates back about two and a half years ago when I was hired as a consultant to figure out/fix problems with a Windows application that would intermittently hang or lock up the machine. Even forcibly shutting down the offending process did not fix the problem(s). The company couldn't diagnose the problem because they developed it on Windows 2000 and only a handful of customers ran it on Windows 98. When they initially reported the problem they failed to report that they were running it on Windows 98! Windows 2000 never displayed the offensive behavior, so they were at a loss to explain to their customers what was going on. And indeed Windows 2000 has a much, much higher tolerance for cleaning up leaky resources in a process. Although that application leaked both handles to GDI objects and memory, the point for me is simply that Windows 98 is an unreliable OS. Now, Microsoft OSes have gotten much more stable and reliable over the years, but if you don't have to rely on the OS, which may be flaky, why rely on the OS? Presumably you will respond that nobody in their right mind would write software for Windows 98 at this point and this is such a marginal 'edge case' that it really is of no concern. Well, unfortunately, there are still hundreds of millions of Windows 98 machines out there, and a lot of current, cutting edge software is STILL being written to run on those machines. Purportedly Microsoft tried to kill support for Windows 98 about 2 years ago until they realized how big the problem really is (is not was). Secondly, how many microseconds does it actually take to free the memory allocated by the threadmon mutex? Even if we are talking milliseconds here, which isn't likely, some application, somewhere would have to kill a few hundred if not a few thousand processes in order to notice an actual performance hit when shutting down. How many apps need to do such a thing? And what's more, if the OS needs to clean up some unfreed memory won't it take more or less the same number of microseconds as the application itself to clean it up? So that point doesn't seem to weigh in real strongly here. Well, I'm sure you get my drift... But your point about how it will only complicate the design of the library sounds like the most reasonable one here. But surely there must some relatively simple way to do this? Is there a way to have it deleted by a mechanism called at file scope when its object code is being unloaded? Anyways, I think boost needs to assume less about how reliable OSes are, and not at all if Boost.Thread runs on Windows 98 also. Elisha
Elisha Berns wrote:
Well, since I started this whole thing, I have to ask... If it's a singleton, doesn't it need to clean itself up? If the assumption is that the OS will clean it up when the process is killed, isn't that
bit shaky as far as sound assumptions go?
It's a perfectly sound assumption for today's OSes, even if you rely on them to clean up kernel resources (such as a Win32 HANDLE) and not just memory. Manually releasing everything with program lifetime duration just makes the process exit more slowly (but it keeps simple leak detectors happy).
In addition, there is no good place to put the delete. You have to
a play
games with atexit and then deal with the possibily of resurrection (singleton access after it's been deleted.)
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Elisha Berns wrote: [...]
Secondly, how many microseconds does it actually take to free the memory allocated by the threadmon mutex?
It's not a question of microseconds. The "thread safe singleton" pattern solves the initialization order problem. You want a particular object to be available for the lifetime of the program, as if it was initialized before everything else and destroyed after everything else. C++ doesn't guarantee a particular initialization order for file scope objects in different translation units, so we have to resort to local statics or call_once. I haven't studied the threadmon code in detail, though. It might be possible to destroy the threadmon mutex safely at some point during shutdown. My original point was that this is a legitimate coding idiom and not a memory leak. It might be a resource leak on Windows 9x; I no longer keep Win9x around so I can't check.
On Aug 29, 2005, at 3:47 PM, Peter Dimov wrote:
I haven't studied the threadmon code in detail, though. It might be possible to destroy the threadmon mutex safely at some point during shutdown.
I haven't studied threadmon either. But from the sounds of it, you may want to consider a "phoenix" singleton, nicely documented in Andrei Alexandrescu's MC++D. One hitch to get a working phoenix singleton is to have a working atexit function (and I mean really working correctly). There was a lwg dr report on it: http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#3 which has now received TC status, meaning it is a normative part of the C++03 standard. I have no information on how well the various platforms conform to the standard in this regard (it can be tested for though). The idea is fairly simple. The singleton factory keeps state that tells it whether the singleton is constructed or not. When the singleton gets constructed, it registers a "destructor" with atexit. If after the atexit runs the destructor, the singleton is needed again, then it recreates itself, and re-registers with atexit. A correctly working atexit will properly run this newly registered function as it continues to process the atexit chain. -Howard
Howard Hinnant wrote:
On Aug 29, 2005, at 3:47 PM, Peter Dimov wrote:
I haven't studied the threadmon code in detail, though. It might be possible to destroy the threadmon mutex safely at some point during shutdown.
I haven't studied threadmon either. But from the sounds of it, you may want to consider a "phoenix" singleton, nicely documented in Andrei Alexandrescu's MC++D. One hitch to get a working phoenix singleton is to have a working atexit function (and I mean really working correctly). There was a lwg dr report on it:
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#3
which has now received TC status, meaning it is a normative part of the C++03 standard. I have no information on how well the various platforms conform to the standard in this regard (it can be tested for though).
The idea is fairly simple. The singleton factory keeps state that tells it whether the singleton is constructed or not. When the singleton gets constructed, it registers a "destructor" with atexit. If after the atexit runs the destructor, the singleton is needed again, then it recreates itself, and re-registers with atexit. A correctly working atexit will properly run this newly registered function as it continues to process the atexit chain.
Interestingly a more recent report from Plauger - but still open - recommends a contradictory advice (at least as far as I understand it), see http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#462 Greetings from Bremen, Daniel Krügler
On Aug 30, 2005, at 3:34 AM, Daniel Krügler wrote:
Howard Hinnant wrote:
On Aug 29, 2005, at 3:47 PM, Peter Dimov wrote:
I haven't studied the threadmon code in detail, though. It might be possible to destroy the threadmon mutex safely at some point during shutdown.
I haven't studied threadmon either. But from the sounds of it, you may want to consider a "phoenix" singleton, nicely documented in Andrei Alexandrescu's MC++D. One hitch to get a working phoenix singleton is to have a working atexit function (and I mean really working correctly). There was a lwg dr report on it:
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#3
which has now received TC status, meaning it is a normative part of the C++03 standard. I have no information on how well the various platforms conform to the standard in this regard (it can be tested for though).
The idea is fairly simple. The singleton factory keeps state that tells it whether the singleton is constructed or not. When the singleton gets constructed, it registers a "destructor" with atexit. If after the atexit runs the destructor, the singleton is needed again, then it recreates itself, and re-registers with atexit. A correctly working atexit will properly run this newly registered function as it continues to process the atexit chain.
Interestingly a more recent report from Plauger - but still open - recommends a contradictory advice (at least as far as I understand it), see
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#462
<nod> The notes (in italic) are from Lillehammer this Spring. I was one of the voices leaning towards NAD. To be fair, getting this right depends on cooperation between the C++ compiler, the C library and the runtime library, which is not an easy cooperation to manage. I.e. it is a logistical problem, not a technical one. On the positive side also note that the "Itanium C++ ABI" correctly addresses this issue. http://www.codesourcery.com/cxx-abi/abi.html#dso-dtor (at least as far as I can tell) -Howard
It's not a question of microseconds. The "thread safe singleton"
Peter Dimov wrote: pattern
solves the initialization order problem. You want a particular object to be available for the lifetime of the program, as if it was initialized before everything else and destroyed after everything else.
C++ doesn't guarantee a particular initialization order for file scope objects in different translation units, so we have to resort to local statics or call_once.
I hear you, I hear the point. I don't have a good solution to this, but I hope someone does!
participants (7)
-
Christian Henning
-
Daniel Krügler
-
Elisha Berns
-
Howard Hinnant
-
Peter Dimov
-
Rich Cook
-
Sohail Somani