[smart_ptr] Conflict with Maya?
Does anyone know of any evidence that the memory management in smart_ptr might be incompatible with Maya, in particular, in gcc 3.2.3? My supervisor, who is unfamiliar with Boost's smart_ptr library, but is familiar with Maya, tells me that Maya has its own memory management, and that Boost's smart pointers might somehow interfere with this. I've used Boost's smart pointers for long enough that I really, really, really don't want to go back to the horrors of having to delete memory that I allocate on the heap and spending hours tracking down memory leaks if I get it wrong. I'd really rather not do that, especially as it's a potential memory leak that I'm trying to fix (Maya plug-ins are created using a bare "new" which doesn't seem to require a corresponding delete, according to the examples I have seen). So someone please tell me that is couldn't possibly be true. Thanks.
Paul Giaccone wrote:
Does anyone know of any evidence that the memory management in smart_ptr might be incompatible with Maya, in particular, in gcc 3.2.3?
So someone please tell me that is couldn't possibly be true.
PS: I'm posting a similar question to HighEnd's MEL board.
Paul Giaccone wrote:
Paul Giaccone wrote:
Does anyone know of any evidence that the memory management in smart_ptr might be incompatible with Maya, in particular, in gcc 3.2.3?
PPS: I am also using lexical_cast and gregorian. Having to replace these with alternatives would be even more painful.
Paul Giaccone wrote:
Does anyone know of any evidence that the memory management in smart_ptr might be incompatible with Maya, in particular, in gcc 3.2.3?
My supervisor, who is unfamiliar with Boost's smart_ptr library, but is familiar with Maya, tells me that Maya has its own memory management, and that Boost's smart pointers might somehow interfere with this.
When you create a shared_ptr: shared_ptr<X> px( new X ); and then destroy all of its copies, it behaves "as if" you've invoked "delete" at the same context where you created px. So if you've overloaded new and delete it will use them. I'm not sure how this applies to your particular case. My passing experience with the Maya SDK is that its classes are well behaved and manage their own memory, so there might be no need to use smart pointers.
Peter Dimov wrote:
Paul Giaccone wrote:
Does anyone know of any evidence that the memory management in smart_ptr might be incompatible with Maya, in particular, in gcc 3.2.3?
My supervisor, who is unfamiliar with Boost's smart_ptr library, but is familiar with Maya, tells me that Maya has its own memory management, and that Boost's smart pointers might somehow interfere with this.
When you create a shared_ptr:
shared_ptr<X> px( new X );
and then destroy all of its copies, it behaves "as if" you've invoked "delete" at the same context where you created px. So if you've overloaded new and delete it will use them. I'm not sure how this applies to your particular case. My passing experience with the Maya SDK is that its classes are well behaved and manage their own memory, so there might be no need to use smart pointers.
Thanks. Of course, there is never any need to use smart pointers, but to be clear, I'm using them for my own variables internal to the node, not for the variables representing the input and output attributes that are to be set from within Maya (specifically, from a MEL script). I have 2D arrays and other more complex data structures, and using smart pointers makes it much easier to manage these. Here are some (disguised) extracts from my code: 1. The creator function, required by Maya. Note the absence of a smart pointer. I tried wrapping this in a shared_ptr and returning .get() but this caused a crash, if I remember rightly. void* MyNode::creator(void) { return new MyNode(); } 2. The destructor for MyNode: ~MyNode(void) { } This is the one that bothers me. All the input and output attributes that will be written to and read from my MEL script are set up using Maya's generic MObject type. All the other member data in MyNode needed for processing within the node that are not of simple types (that is, all the members that are arrays or more complex data structures) are created using smart pointers.
Paul Giaccone wrote:
1. The creator function, required by Maya. Note the absence of a smart pointer. I tried wrapping this in a shared_ptr and returning .get() but this caused a crash, if I remember rightly.
void* MyNode::creator(void) { return new MyNode(); }
This is, I think, exactly as it should be. Maya expects to take ownership of the returned pointer and will delete it when it decides that it's no longer needed. But I'm not an expert on that; you probably need to ask in a Maya SDK forum to be sure.
Peter Dimov wrote:
Paul Giaccone wrote:
1. The creator function, required by Maya. Note the absence of a smart pointer. I tried wrapping this in a shared_ptr and returning .get() but this caused a crash, if I remember rightly.
void* MyNode::creator(void) { return new MyNode(); }
This is, I think, exactly as it should be. Maya expects to take ownership of the returned pointer and will delete it when it decides that it's no longer needed. But I'm not an expert on that; you probably need to ask in a Maya SDK forum to be sure.
A bit of research and feedback from HighEnd 3D's Maya API forum suggests that this is the case - you create the node and Maya takes responsibility deleting it when necessary. Apparently, Maya owns the node and deletes it itself when it feels like it, not when the programmer says so. If I've understood correctly, this means that any memory allocated on the heap in MyNode must be done using bare pointers rather than smart pointers, so that Maya can delete it when it wants, not when the smart pointers say the memory can be released. MyNode and any objects that it allocates on the heap still need to have destructors to deallocate their contents, but it is up to Maya when the destructor for MyNode is called. Not an ideal state of affairs (once you've experienced smart pointers, you never want to go back to the inconvenience of having to work out where the all the deletes need to go), but then I've found that Maya is hardly the most programmer-friendly system, IMO.
Paul Giaccone wrote:
Peter Dimov wrote:
Paul Giaccone wrote:
1. The creator function, required by Maya. Note the absence of a
smart
pointer. I tried wrapping this in a shared_ptr and returning .get() but this caused a crash, if I remember rightly.
void* MyNode::creator(void) { return new MyNode(); }
This is, I think, exactly as it should be. Maya expects to take ownership >>of the returned pointer and will delete it when it decides that it's no longer needed. But I'm not an expert on that; you probably need to ask in a Maya SDK forum to be sure.
A bit of research and feedback from HighEnd 3D's Maya API forum suggests that this is the case - you create the node and Maya takes responsibility deleting it when necessary. Apparently, Maya owns the node and deletes it itself when it feels like it, not when the programmer says so.
If I've understood correctly, this means that any memory allocated on the heap in MyNode must be done using bare pointers rather than smart pointers, so that Maya can delete it when it wants, not when the smart pointers say the memory can be released. MyNode and any objects that it
allocates on the heap still need to have destructors to deallocate their contents, but it is up to Maya when the destructor for MyNode is called.
Not an ideal state of affairs (once you've experienced smart pointers, you never want to go back to the inconvenience of having to work out where the all the deletes need to go), but then I've found that Maya is
hardly the most programmer-friendly system, IMO.
First, let me say that I am not familiar with Maya, so I may not know what I am talking about. That said, are there still some objects you need to delete manually? I think you can still use smart pointers for those, and save the raw pointers for objects you pass back to Maya (like the pointer returned from MyNode::creator). The only problem I can see with this is if you have an object that Maya MIGHT delete for you, or that you might need to delete yourself. If you get to deal with that kind of ugliness, maybe you can use a custom deleter that checks some sort of flag before deleting the object. If you can put this flag in the object itself, then it is a simple matter of setting it before you give the object to Maya. If you can't, then it would probably be easier to use raw pointers for that object.
On 4/19/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
If I've understood correctly, this means that any memory allocated on the heap in MyNode must be done using bare pointers rather than smart pointers, so that Maya can delete it when it wants, not when the smart pointers say the memory can be released. MyNode and any objects that it allocates on the heap still need to have destructors to deallocate their contents, but it is up to Maya when the destructor for MyNode is called.
I don't think this is the case. MyNode's destructor will get called when Maya determines it's time to delete it, and any smart_ptr member variables will decrement their reference count. That's the ideal state of affairs, since it works as designed. The only difference is that MyNode pointers are created by a create function, and Maya cleans them up. You cannot attempt to manage MyNode pointers, since Maya determines their lifetime, however, you can (and should) manage anything not managed by Maya. --Michael Fawcett
Michael Fawcett wrote:
On 4/19/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
If I've understood correctly, this means that any memory allocated on the heap in MyNode must be done using bare pointers rather than smart pointers, so that Maya can delete it when it wants, not when the smart pointers say the memory can be released. MyNode and any objects that it allocates on the heap still need to have destructors to deallocate their contents, but it is up to Maya when the destructor for MyNode is called.
I don't think this is the case. MyNode's destructor will get called when Maya determines it's time to delete it, and any smart_ptr member variables will decrement their reference count. That's the ideal state of affairs, since it works as designed. The only difference is that MyNode pointers are created by a create function, and Maya cleans them up. You cannot attempt to manage MyNode pointers, since Maya determines their lifetime, however, you can (and should) manage anything not managed by Maya.
Well, I hope that's true, because I'm fiddling around with delete[], etc, and my program is leaking memory like a sieve. Like in all those washing powder commercials, there's no way I want the old brand back :) It looks like I might have been doing something else wrong after all that would cause a memory leak - copying pointers to arrays in a copy constructor instead of allocating memory and then copying the array elements.
Paul Giaccone wrote:
Michael Fawcett wrote:
On 4/19/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
If I've understood correctly, this means that any memory allocated on the heap in MyNode must be done using bare pointers rather than smart pointers, so that Maya can delete it when it wants, not when the smart pointers say the memory can be released. MyNode and any objects that it allocates on the heap still need to have destructors to deallocate their contents, but it is up to Maya when the destructor for MyNode is called.
I don't think this is the case. MyNode's destructor will get called when Maya determines it's time to delete it, and any smart_ptr member variables will decrement their reference count. That's the ideal state of affairs, since it works as designed. The only difference is that MyNode pointers are created by a create function, and Maya cleans them up. You cannot attempt to manage MyNode pointers, since Maya determines their lifetime, however, you can (and should) manage anything not managed by Maya.
It looks like I might have been doing something else wrong after all that would cause a memory leak - copying pointers to arrays in a copy constructor instead of allocating memory and then copying the array elements.
Hm, that made no difference... Maya is still hanging with a segmentation fault when delete() is being called (not by me, incidentally) after 24 out of 48 frames have been processed, as if it is deciding it doesn't want the node after that length of time. It still works fine in Windows.
On 4/19/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
Paul Giaccone wrote:
Hm, that made no difference... Maya is still hanging with a segmentation fault when delete() is being called (not by me, incidentally) after 24 out of 48 frames have been processed, as if it is deciding it doesn't want the node after that length of time. It still works fine in Windows.
I don't think it has anything to do with Boost. I specifically remember using smart_ptrs (and other Boost libraries) in conjunction with the Maya SDK, but I no longer have Maya installed so I can't code up a quick test for you. I imagine the problem is with the usage of the smart_ptrs, or lies elsewhere. Can you maybe post a small sample of your usage? --Michael Fawcett
Michael Fawcett wrote:
On 4/19/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
Paul Giaccone wrote:
Hm, that made no difference... Maya is still hanging with a segmentation fault when delete() is being called (not by me, incidentally) after 24 out of 48 frames have been processed, as if it is deciding it doesn't want the node after that length of time. It still works fine in Windows.
I don't think it has anything to do with Boost. I specifically remember using smart_ptrs (and other Boost libraries) in conjunction with the Maya SDK, but I no longer have Maya installed so I can't code up a quick test for you.
I imagine the problem is with the usage of the smart_ptrs, or lies elsewhere. Can you maybe post a small sample of your usage?
Hm, it's hard to cut it down to anything small, but I'll try to give just the minimum here. Here's the creation of the node: void* MyNode::creator(void) { return new MyNode(); } Here's the constructor: MyNode::MyNode(void) : success(false), error_message(""), info(0) { position.second.resize(NumPositionParameters); orientation.reset(new double[NumOrientationParameters]); previous_position_parameter.resize(NumPositionParameters); for (unsigned int parameter = 0; parameter != NumPositionParameters; ++parameter) { previous_position_parameter[parameter] = 0.0; } } where the following are declared in the header file (as well as lots of other non-pointer variables): boost::scoped_ptr<Info> info; std::pair<double, std::valarray<double> > position; boost::shared_array<double> orientation; std::valarray<double> previous_position_parameter; Info is a class that contains further shared_arrays of simple types. The variable info is initialised in MyNode::initialize() because the sizes of the arrays are not known until that point. MyNode::compute() calls a stand-alone function, passing in some of the above variables. Within this function are the following definitions: MyModel model; simplex<MyModel> simplex_method(model); The class "simplex" requires the classes it makes use of to have a default constructor that does not allocate any memory for objects pointed to by class members. The memory is allocated when the simplex class calls operator=, which I have defined for each class that simplex uses. I have used smart pointers and memory is allocated using reset() in each operator=() function. The only destructor I have is for the MyNode class, which resizes the valarrays to 0, although these only contain doubles, so there isn't really anything to free. I don't know how helpful this is, but thanks for anything you might spot that I have not. Paul
On 4/20/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
<code snip>
I don't see anything really wrong off hand.
boost::scoped_ptr<Info> info; std::pair<double, std::valarray<double> > position; boost::shared_array<double> orientation; std::valarray<double> previous_position_parameter;
Why is orientation a shared_array when MyNode is not copyable? Shouldn't orientation be a scoped_array, or alternatively, info be a shared_ptr? I didn't see the copy-constructor or assignment operator, so maybe you are manually doing a deep copy of info?
Info is a class that contains further shared_arrays of simple types. The variable info is initialised in MyNode::initialize() because the sizes of the arrays are not known until that point.
I didn't see initialize listed, so I can't comment.
I don't know how helpful this is, but thanks for anything you might spot that I have not.
Unfortunately I doubt I have been very helpful. Have you had any luck on the MEL forums? --Michael Fawcett
Michael Fawcett wrote:
On 4/20/07, Paul Giaccone <paulg@cinesite.co.uk> wrote:
<code snip>
I don't see anything really wrong off hand.
boost::scoped_ptr<Info> info; std::pair<double, std::valarray<double> > position; boost::shared_array<double> orientation; std::valarray<double> previous_position_parameter;
Why is orientation a shared_array when MyNode is not copyable? Shouldn't orientation be a scoped_array, or alternatively, info be a shared_ptr? I didn't see the copy-constructor or assignment operator, so maybe you are manually doing a deep copy of info?
Hm, interesting point. The variable info is a scoped_ptr because it is used only in MyNode::compute() - it is a wrapper around information that is passed in MyNode::compute() to the (stand-alone) function that does all the work. The variable orientation is also passed to this function, hence it needs to be a shared_array. So it is not that I am copying MyNode or thinking that it will be copied. (I take it that MPxNode, from which MyNode is inherited publicly, is non-copyable, then?) There is therefore no copy constructor or operator= for MyNode, nor for Info, as it is its contents that are passed to the function that does any copying. The copy constructors and operator=() functions do deep copies, yes, with their own reset()s for any smart pointers that are members, and then explicit per-element copying.
Info is a class that contains further shared_arrays of simple types. The variable info is initialised in MyNode::initialize() because the sizes of the arrays are not known until that point.
I didn't see initialize listed, so I can't comment.
My mistake - it's initialised in readData(): MStatus MyNode::readData(void) { //read other attributes here handle = data.inputValue(maya_variable, &stat); variable = handle.asDouble(); const int num_frames = 1; //info is required for only one frame info.reset(new Info(num_frames, variable)); return MS::kSuccess; }
I don't know how helpful this is, but thanks for anything you might spot that I have not.
Unfortunately I doubt I have been very helpful. Have you had any luck on the MEL forums?
Thanks for your help. I have had some feedback on the HighEnd 3D Maya API forum which suggests that you shouldn't use smart pointers: "Well inside your plugin your free to use memory as you wish. However the fundamental rule about maya api is that *maya owns everything and you own none of it*." and "[I]n general when you program with maya you don't store pointers at all, you just ask for a handle to a mdatablock which returns you a usable data handle which acts as a temporary pointer to your data. You then flush the pointer and ask it again when you come to next cycle of compute. This si akin to using smart pointers, maya just calls them datahandles. "(remember you have no data at all you have just pointers that point int a space that does not stay constant) "However the smart pointers dont really work like you expect, you do need to unallocated them when you no longer use them, they will offcourse in normal apps get terminated on exit and if you run the same strand of code later thet will do flush eventually. But maya wont, it just terminates al code that the smart pointers are using for possible cleanup. "So basically you never really get to allocate memory anyway, maya does it for you and incidentally frees it for you too. As long as you use a data block as your intended. (if you don't then maya wont save the info). So it looks like these datablocks are the way to go. I'll have a look at using those. One of my colleagues is using Boost in his Maya plug-in. Currently it is crashing in Linux too, but that might be for a different reason. Thanks for your help - it looks like I might have found the answer now, namely, "don't use smart pointers in Maya; use datablocks instead".
For what I've learned, apparently the creation of the node goes into the undo queue, and this queue has a limited length. Once the node reaches the other end of the queue, its destructor is called and any dynamically memory allocated is therefore deallocated, making the node is unusable. If memory has been allocated using Maya's own method (MDataBlocks) then this memory is still accessible. The memory is only deleted when the node itself reaches the end of its lifetime, and this is handled by Maya. I'm not quite sure if this is it (I haven't quite got it completely clear in my head). In any case, the crash in Linux occurs as soon as the program returns from the first call to the node to compute its results, so I'm not convinced that undo queue even comes into it. I hope there isn't something deeper and more subtle going on.
Assuming your issue is related to the OpenMaya API and isn't a general programming error... Maya's really good at managing its object lifetimes...you just have to let it do its job. Some quick rules (a few of which I've learned the hard way :D): - Don't store MObject instances across invocations of any of your plugin's methods - Maya can invalidate these from time to time and using an invalid MObject is pretty much the same as using a dangling pointer. Check the OpenMaya docs for more info on when Maya can decide to invalidate an MObject (I forget what all the cases are). - When a creator function returns a pointer to an MPx* object to Maya, that object belongs to Maya. Don't delete it. - Don't ever store an MPx* pointer you've given to Maya. If one custom node needs to reference another, create an attribute (like the "message" attribute Maya uses for light linking) and link them that way: - If Maya doesn't know that node A references node B it can delete node B at a whim (or allow the user or some custom script to delete it), leaving node A with a dangling pointer. If the connection were made via an attribute then node A would find the missing connection next time it tries to use B and could respond appropriately. - Also, if node A decides to delete it's pointer to B, then Maya is left with a dangling pointer which will blow things up nicely next time Maya touches that part of the DG. - Types like MPoint, MVector (etc) are close enough to being POD that you can do with these pretty much whatever you want. Just be consistent. Maya will never give you a pointer to one of its internal data arrays so you're pretty safe when it comes to messing anything up that way. - M*Array is pretty much equivalent to std::vector. The same rules apply. - If you have a bunch of data that you want to pass from one object to another use a datablock and let Maya pass the data around the DG. Nodes should never look directly at each other's data (except in special cases like light/object linking where a "message" type connection exists to ensure that object is in a valid state before it gets queried), let Maya pass that info around the DG for you. My guess is that you're either trying to use an MObject after Maya has invalidated it (in which case you need to make sure you don't keep MObject instances around too long - get a new node reference from a representation that doesn't expire each time you need the object), or you're keeping a pointer to an MPx* type node and either you or Maya is deleting it prematurely (in which case you need to replace that pointer with a proper Maya attribute and connection). Not really boost/smart pointer issues, but you might want to check to see if you've got a smart pointer to a smart pointer to something you shouldn't be managing (or something like that), or if a long-lived container has MObjects somewhere in its data elements. Phill
participants (2)
-
Andrew Holden -
Michael Fawcett -
Paul Giaccone -
Peter Dimov -
Phill Djonov