Hello,
I am sorry if this is a stupid question, or simply inappropriatie for
this list. I am currently camping with this bug for way too long, so I
decided to ask this mailinglist.
I am currently in the process of finding out where a segfault is, and
how to fix it. Now, I am using the boost regex library to parse my
regexes, and for some reason when multiple threads use this regex
library, it seems to mess up and generate segfaults (it works perfectly
with only one thread). Since it also doesn't always occur on the same
place, and doesn't even occur /all/ the times, I have a strong feeling
this has something to do with thread safety.
On the Boost website is was able to find out that the boost regex
library should be thread safe when BOOST_HAS_THREADS is defined; I've
tested this, and this worked.
( http://www.boost.org/libs/regex/doc/thread_safety.html )
However, I also was able to find a mailinglist message that provides
some instructions on how to make certain boost regex functions
thread-safe :
( http://lists.boost.org/MailArchives/boost/msg59110.php )
Now, I am confused; is the boost regex library thread-safe or not ? The
solution provided in the mailinglist message ( move the regex_replace ()
function inside its own scope ) can't be applied here... :(
I've posted the stacktrace when the problem occurs and the source of the
code 'that matters' below. It also occurs in another function, I've
posted the other function too. I hope anyone has any experience with this,
or is able to help.
Thanks in advance!
Regards,
Leon Mergen
One stacktrace:
---
#0 std::string::compare(char const*) const (this=0x0, __s=0x40338b81 "C") at /root/gcc-3.3.2/i686-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:257
#1 0x4004b101 in boost::c_regex_traits<char>::update() () from /usr/local/lib/libboost_regex-gcc-1_31.so.1.31.0
#2 0x40084cd9 in boost::reg_expression
On Wed, Jan 26, 2005 at 04:16:25PM +0100, Leon Mergen wrote:
I've posted the stacktrace when the problem occurs and the source of the code 'that matters' below. It also occurs in another function, I've posted the other function too. I hope anyone has any experience with this, or is able to help.
As an addition (sorry for the double-post), I also receive assertion
faults for the regex library:
---
normserv: /usr/local/include/boost-1_31/boost/regex/v4/match_results.hpp:252: void boost::match_results
I've posted the stacktrace when the problem occurs and the source of the code 'that matters' below. It also occurs in another function, I've posted the other function too. I hope anyone has any experience with this, or is able to help.
As an addition (sorry for the double-post), I also receive assertion faults for the regex library:
That's more worrying, do you have a test case that reproduces the issue (preferable a single threaded one!) Thanks, John.
On Thu, Jan 27, 2005 at 10:59:03AM -0000, John Maddock wrote:
I've posted the stacktrace when the problem occurs and the source of the code 'that matters' below. It also occurs in another function, I've posted the other function too. I hope anyone has any experience with this, or is able to help.
As an addition (sorry for the double-post), I also receive assertion faults for the regex library:
That's more worrying, do you have a test case that reproduces the issue (preferable a single threaded one!)
Hello John, Well, the problem is, I've only seen this fault a few times; it's certainly not a consequent bug (as in, I could run the program three times and the problem only occured once), *and* as far as I am aware these problems only occur after a few minutes when two threads are doing heavy processing (think about around 50 calls a second to this function)... ... since I've built-in the mutexes I don't have this problem at all anymore; however, if I see this problem again and it's reproducable, I'll cut it down to a test case and put up the source code. Thanks for your help, Regards, Leon Mergen
... since I've built-in the mutexes I don't have this problem at all anymore; however, if I see this problem again and it's reproducable, I'll cut it down to a test case and put up the source code.
Sounds like it may be race condition related again: if one thread tries to use a regex that is actually in the process of being constructed by another thread, then it's sub-expression count may suddenly change, leading to your assertion. Just guessing at this point, though... John.
Leon Mergen wrote:
Hello,
I am sorry if this is a stupid question, or simply inappropriatie for this list. I am currently camping with this bug for way too long, so I decided to ask this mailinglist.
I am currently in the process of finding out where a segfault is, and how to fix it. Now, I am using the boost regex library to parse my regexes, and for some reason when multiple threads use this regex library, it seems to mess up and generate segfaults (it works perfectly with only one thread). Since it also doesn't always occur on the same place, and doesn't even occur /all/ the times, I have a strong feeling this has something to do with thread safety.
On the Boost website is was able to find out that the boost regex library should be thread safe when BOOST_HAS_THREADS is defined; I've tested this, and this worked. ( http://www.boost.org/libs/regex/doc/thread_safety.html )
However, I also was able to find a mailinglist message that provides some instructions on how to make certain boost regex functions thread-safe : ( http://lists.boost.org/MailArchives/boost/msg59110.php ) <snip>
That is not specifically about thread-safety in Boost.Regex; the initialisation of local static variables is generally not done in a thread-safe way even by implementations that are intended to support multithreaded programs, and this goes for variables of any type. I can tell you for certain, though, that std::string is not thread-safe in libstdc++ (see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=10350). That might be the source of the problem. Ben.
On Wed, Jan 26, 2005 at 08:14:36PM +0000, Ben Hutchings wrote:
That is not specifically about thread-safety in Boost.Regex; the initialisation of local static variables is generally not done in a thread-safe way even by implementations that are intended to support multithreaded programs, and this goes for variables of any type.
I can tell you for certain, though, that std::string is not thread-safe in libstdc++ (see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=10350). That might be the source of the problem.
Hi Ben, Ok, I will look into it tomorrow when I'm back at work. Thanks for your reply. Regards, Leon Mergen
On Wed, Jan 26, 2005 at 08:14:36PM +0000, Ben Hutchings wrote:
That is not specifically about thread-safety in Boost.Regex; the initialisation of local static variables is generally not done in a thread-safe way even by implementations that are intended to support multithreaded programs, and this goes for variables of any type.
Hi Ben, Ok, I've now completely put both functions in a boost::mutex scoped lock and it doesn't generate any segmentation faults anymore; so basically, it works now. For some reason I have the feeling this whole problem could be solved *much* more elegant, but since I have a deadline in 1.5 weeks I think this will do for now... I have found this page explaining in detail why the initialisation is thread-unsafe, in case anyone is interrested: http://weblogs.asp.net/oldnewthing/archive/2004/03/08/85901.aspx Thanks for your help, Leon Mergen
I am sorry if this is a stupid question, or simply inappropriatie for this list. I am currently camping with this bug for way too long, so I decided to ask this mailinglist.
I am currently in the process of finding out where a segfault is, and how to fix it. Now, I am using the boost regex library to parse my regexes, and for some reason when multiple threads use this regex library, it seems to mess up and generate segfaults (it works perfectly with only one thread). Since it also doesn't always occur on the same place, and doesn't even occur /all/ the times, I have a strong feeling this has something to do with thread safety.
On the Boost website is was able to find out that the boost regex library should be thread safe when BOOST_HAS_THREADS is defined; I've tested this, and this worked. ( http://www.boost.org/libs/regex/doc/thread_safety.html )
However, I also was able to find a mailinglist message that provides some instructions on how to make certain boost regex functions thread-safe : ( http://lists.boost.org/MailArchives/boost/msg59110.php )
Now, I am confused; is the boost regex library thread-safe or not ? The solution provided in the mailinglist message ( move the regex_replace () function inside its own scope ) can't be applied here... :(
I've posted the stacktrace when the problem occurs and the source of the code 'that matters' below. It also occurs in another function, I've posted the other function too. I hope anyone has any experience with this, or is able to help.
Regex is thread safe in following sense: given a const regex object, you can safely share that object between multiple threads. However, your backtrace indicates that the problem occurs during regex construction: this is a situation *you* have to deal with (no matter what the data type), the thread safety guarantees only kick in once the object has been constructed, it's up to you to ensure that no race condition occurs during the construction. Your second link above indicates a couple of methods you could use to achieve this. Regards, John.
On Thu, Jan 27, 2005 at 10:53:31AM -0000, John Maddock wrote:
Regex is thread safe in following sense: given a const regex object, you can safely share that object between multiple threads.
However, your backtrace indicates that the problem occurs during regex construction: this is a situation *you* have to deal with (no matter what the data type), the thread safety guarantees only kick in once the object has been constructed, it's up to you to ensure that no race condition occurs during the construction. Your second link above indicates a couple of methods you could use to achieve this.
Hello John, You indeed are correct, and I also came to the conclusion this is something serious I didn't take into account at all; it is not at all boost-related and I apologise for posting these off-topic questions at this mailinglist. Thanks for your replies, Leon Mergen
John Maddock wrote:
Regex is thread safe in following sense: given a const regex object, you can safely share that object between multiple threads.
However, your backtrace indicates that the problem occurs during regex construction: this is a situation *you* have to deal with (no matter what the data type), the thread safety guarantees only kick in once the object has been constructed, it's up to you to ensure that no race condition occurs during the construction. Your second link above indicates a couple of methods you could use to achieve this.
Sorry for jumping into this discussion, since I am not very knowledgable in the use of the regex library. But the buzzwords: "construction", "thread" and "local static" alerted me. There was a bug in the spirit library that seemingly had the same attributes attached. A function contained a local static object. If this is the case in regex library too this should be changed, since such usage is not thread safe. If this is not the case, please simply ignore my post since it does not apply. Roland
A function contained a local static object. If this is the case in regex library too this should be changed, since such usage is not thread safe. If this is not the case, please simply ignore my post since it does not apply.
The local static is in user code, not the library (the library does use some global state, but it's all carefully mutexed...) John.
participants (4)
-
Ben Hutchings
-
John Maddock
-
Leon Mergen
-
Roland Schwarz