On 07/12/2013 03:13 AM, Larry Evans wrote:

Hi Borislav.

It looks interesting and fulfills a purpose somewhat like that of Steven's TypeErasure library:

http://steven_watanabe.users.sourceforge.net/type_erasure/libs/type_erasure/...

Except it may be more dynamic. For example, the Mixin example here:

http://ibob.github.io/boost.mixin/boost_mixin/introduction.html

can add or remove methods at runtime; however, IIUC, the TypeErasure library cannot. I'd guess then that the TypeErasure method calls would be faster than those of the Mixin methods calls.

Is that about right, Borislav or Steven, or am I missing something?

-regards, Larry

I don't think there are many similarities there. The type erasure library allows you to view existing types as a set of concepts. Boost.Mixin allows you to compose types. That is the main difference. And yes, Boost.Mixin allows you to change the object's composition (and effectively interface) at runtime. I'm guessing here, but I think the type erasure method calls are not slower than virtual calls. Since it uses function pointers, I guess they're about as fast as virtual. Boost.Mixin's method calls, however, are slower than virtual calls. With compiler optimizations they reach 200-250% the speed of a regular virtual call and in terms of speed can be compared to a boost/std::function call (although they're much faster than std::function in "debug" mode). The only similarity between the two libraries, that I see, is that both allow you to have information hiding by having the interface be physically separated from the implementation. -- Borislav

I'm still highly interested in this library but will not have time to try it until a few months at least. I didn't finish to read the current documentation but I have some questions: 1. is the current version usable? I mean even with some bugs, are the main features usable already? 2. did you avoid using a smart pointer in the introduction example for a specific reason? if not, I think it would be better to use one, just to avoid people copying this style of code. Joel Lamotte

On 07/12/2013 09:03 PM, Klaim - Joël Lamotte wrote:

1. is the current version usable? I mean even with some bugs, are the main features usable already?

I think so. I don't plan on adding new features until the formal submission to Boost. I may work on the reflection part, but I'm not sure I'll have time for it. We actually use the library in the company I work for a project that's under development, so the main features are actually used in production code and there don't seem to be any bugs.

2. did you avoid using a smart pointer in the introduction example for a specific reason? if not, I think it would be better to use one, just to avoid people copying this style of code.

Well I didn't want to overburden the reader with the notion of a smart pointer. The messages' first argument is `boost::mixin::object*` so using a smart pointer would also require the constant use of `ptr.get()` to call messages. This makes me think, though. I'll have to think about making the first argument a reference to an object instead a pointer to an object, or allow both (with the cost of an additional copying of the actual arguments).

3. is there a way to go through all the instances of the same type? for example: mutate(o) .add() .add();

Assuming there is more than one instance composed of book_data, is there a way to list or just apply a function to all currently instanciated book_data?

If not, I think a way to do it would be useful in high performance cases (to update all the components of the same type in one pass, avoiding cache misses and enabling some possible concurrent update setup). But as it's not the core feature I think it would be a nice update.

The library doesn't provide such a feature. There are many axes by which the memory is accessed, so completely avoiding cache misses is impossible. Indeed, there are ways to improve on the cache locality on this or that axis, but that will be at the cost of worsening it on another. Internally the object separately instantiates the mixins that comprise it and it effectively has a list of pointers to its mixins. If you mutate that object, removing or adding mixins, all of its other mixins, that are already instantiated, won't be touched. To keep all the mixins in a single buffer will require either dealing with big gaps and potentially badly fragmented memory, or moving the mixin data in the memory, which can be slow and WILL break any pointers that reference it (especially bad for classes that have self referencing in some way, like, say google::hash_map/hash_set). A mixed approach CAN be introduced, but that will require the library users to have deep knowledge of its internal workings in order to choose how their mixins will behave, and that might alienate some users. To deal with the problem of updating a huge list of objects of the same internal type (or memory problems in general) two things are introduced in the library: * Object mutators - that speed up mutation by remembering what needs to be done and greatly improve on the performance of the naive `mutate(obj).add.add.add` * Memory allocators that can be set for all mixin allocations or also for the allocations of specific mixins Still it is the user's job to maintain lists of the objects by whatever criteria they choose (like object that have `book_data` in them). Sure, the library could easily store all kinds of lists by all kinds of criteria, but I fear that this will be memory wasted, since the users know best what and WHETHER to specifically maintain. -- Borislav

On 07/13/2013 12:31 PM, Klaim - Joël Lamotte wrote:

From what I understand, the allocator might be enough to allow the user of the library to set up a pool or dynarray or non-ressizing vector of components of the same type, which would be the feature I'm asking for. I'm not sure though. Would it be problematic to allow the user to provide a factory function for the component on addition, so that there is a simple way to make sure the component is created the way we want?

Something like:

mutate(o).add( [&]{ return world_components.create() ); // template parametter types implicitely inferred // the added data_book is allocated in an array (or stable vector or pool or...) // here add would take a unique_pointer or similar - or maybe a second function for deletion - or maybe an object providing both, like an alocator but dynamic.

Unfortunately allowing the user to maintain their own pool of pre-created mixins is not possible (well it may be possible with some arrangements, but it's not likely that I'll address this in the near future). The main problem with this is the need for `bm_this`. bm_this is a macro that gets you the object from which this mixins is a part of. In order for this to work the memory that's internally allocated for a mixin is `sizeof(object*)+sizeof(ActualMixinType)` bytes and then the constructor is called for the second piece of memory. Essentially bm_this expands to in the sense of `(object*)(this-Ptr_bytes)`. There is no other non-intrusive way to do this. I do mean to address the constructor problem, though, but again, not soon. I will allow the users to choose which constructor is called for their mixins.

Which then would allow something like:

world_components.for_all( [&]( book_data& data ){ data.update( info ); } ); // go through the array of book_data to update them all

I'm not sure what this is supposed to do. You could do something like for(auto o : objects_with_book_data) // your own array { o->get()->update(info); } You could also have a message called update (handled by not necessarily only book_data) and code: for(auto o : objects) { update(o, info); }

What I fear is that the allocator would not be enough to allow different pool/list/whatever of these components to be defined, for different "contexts". I have a case where I need different "worlds" to exists in parallel and components are gathered in type-specific (stable) arrays.

The allocator works like this. When an object is created or mutated it first allocates the array of mixin pointers for its mixins, then it allocates memory for each new mixin (plus object*) and arranges them in this array. You're allowed to set a global allocator that takes care for all these allocations, and you're also allowed to assign specific allocators per mixin type. For example book_data_allocator, which will take care to allocate memory only for mixins of type book_data. So if your programs often mutates a huge group of objects by adding and removing specific mixin to them it's a good idea to provide a specific allocator for that mixin type and pre-allocate the entire chunk memory. As for your worlds example you could easily set the allocator to all rendering mixins to be A, and to all physics mixins to be B. -- Borislav

On 07/13/2013 02:46 PM, Klaim - Joël Lamotte wrote:

To make to make it clearer: are allocators allowed to have a state, in your library?

Because it needs to know in which "world" pool to allocate. This assume that the different worlds are potentially not updated in the same thread or at the same time (which happen a lot in client/server systems). Also it allow updating only the world you want to update when it is acive, while keeping all the differnt worlds in memory - reducing efforts updates when the "player" in a game can be in only one world at a time. Basically, if I have to allocate, say, all the physics components in the same global array, then I'll have a hard time updating only those that correspond to a specific world, because if there is a lot of worlds and a lot of physics objects, I'll have to go through all of them just to update those of one world.

If I can set dynamically the allocator object, then that indeed solves all I mentionned.

The allocators in the library are classes of your own, that derive from a regular interface parent (class with pure virtual methods). They can have a state, and you can manage their instances. A piece of code like this is possible: // this has been set as the book_data allocator my_book_data_allocator.prepare_book_datas(objects.size()); for(o : objects) { o->add(); }; To have all book_datas consecutive in memory, the need to move them within the buffer is needed (unless you allocate them ALL once and forget about them altogether). I could probably address this in some way. I'll have to think about it. I'll also think if I can do anything to make this easier for the user, but at a first glance, apart from documenting thoroughly how the mixin allocations work, I don't think I'll be able to help. You understand that if the user knows that their allocators provide memory that's to be used like this |object*...their own type|, they can safely manipulate the chunk in any way they wish. -- Borislav

On 07/13/2013 02:46 PM, Klaim - Joël Lamotte wrote:

Most games don't have more than one world running in memory at the same time and not all games have tons of elements to update quickly and not all games have to run on ARM efficiently. It just appears that mines does. :D

Ooops. I forgot to address this. I now understand what you mean by two worlds. I'll think about what can be done in the library to accommodate for such a case. For now it won't be possible to separate the logic between two sets of objects. The library provides something that's vaguely similar to this: domains. The domain is a set of mixins and messages that don't mix. You could add the same class and same messages to many domains and thus you'll be able to have your two worlds, but you won't be able to have N worlds, because you'll have to have the code BOOST_DEFINE_MIXIN_IN_DOMAIN(domain_a, some_mixin, features & a_allocator); BOOST_DEFINE_MIXIN_IN_DOMAIN(domain_b, some_mixin, features & b_allocator); ... But if the number of worlds is determined at runtime, this won't help you, plus managing will be a real nightmare with all the same internally generated names. Still I think I'll be able to do something in this regard. I'll think about it. -- Borislav

Just to clarify, when I'm talking about allocators having state, what I expect is to be able to do something similar to the following. Note that this is just an example, there are tons of variants of this: template< class ComponentType > class ComponentPool { public: template< typename... Args > ComponentType* create( Args... args ) { if( auto free_slot = find_free_slot() ) { *free_slot = make_optional<ComponentType>( args... ); // or something similar return free_slot.get_ptr(); } return nullptr; } void destroy( ComponentType& component ) { auto slot = find_slot( component ); assert( slot ); *slot = none_t; // reset the slot } template< typename Func > void for_all_living_components( Func&& f ) { for( ComponentSlot& slot : m_components ) { if( !slot ) continue; f( slot.get() ); } } private: using ComponentSlot = boost::optional<ComponentType>; // to know if the component memory is constructed or not. std::array< ComponentSlot, MAX_COMPONENTS_PER_WORLD > m_components; ComponentSlot* find_free_slot() { for( auto& slot : m_components ) if( !slot ) return &slot; return nullptr; } ComponentSlot* find_slot( ComponentType& component ); }; class ComponentSystem { public: class Allocator; Allocator allocator(); // proivde an allocator for component that will use this; template< class ComponentType > void declare_component_type() { auto find_it = m_pool_index.find( typeid(ComponentType) ); if( find_it == end(m_pool_index ) ) { auto new_pool = std::make_unique(); m_pool_index.insert( std::make_pair( typeid(ComponentType), std::move(new_pool) ) ); } } template< class ComponentType > ComponentPool<ComponentType>* find_pool() { auto find_it = m_pool_index.find( typeid(ComponentType) ); if( find_it != end(m_pool_index ) ) { return find_it->second.get(); } return nullptr; } private: struct PoolSlot{}; template< class ComponentType > struct PoolSlotSpecific { CompomentPool<ComponentType> pool; }; boost::flat_map< std::type_index, std::unique_ptr<PoolSlot> > m_pool_index; }; class World { public: using Entity = boost::mixin::object< ComponentSystem::Allocator >; World() { m_component_system.declare_component_type<PhysicsData>(); m_component_system.declare_component_type<BrainData>(); m_component_system.declare_component_type<GraphicData>(); m_component_system.declare_component_type<AudioData>(); } void update() { const now = m_clock.now(); // here we assmue that the data don't communicate or do it through thread-safe work queues/messages auto physics_update = async( /*some_executor,*/ [&] { m_component_system.find_pool( typeid(PhysicsData) )->for_all_living_components( [&]( PhysicsData& data ) { data.update( now ); } ); }; auto brain_update = async( /*some_executor,*/ [&] { m_component_system.find_pool( typeid(BrainData) )->for_all_living_components( [&]( BrainData& data ) { data.update( now ); } ); }; when_all( physics_update, brain_update ); // for some reasons, cannot be updated in parallel m_component_system.find_pool( typeid(GraphicData) )->for_all_living_components( [&]( GraphicData& data ) { data.update( now ); } ); m_component_system.find_pool( typeid(AudioData) )->for_all_living_components( [&]( AudioData& data ) { data.update( now ); } ); } Entity& create_entity( EntityInfo info ) { m_entities.emplace_back(); auto& new_entity = m_entities.back( m_component_system.allocator() ); // here we pass an allocator which will use this specific instance of component system. using boost::mixin; mutate( *new_entity ) .add<PhysicsData>( info ) .add<BrainData>( info ) .add<GraphicData>( info ) .add<AudioData>( info ) ; return *new_entity; } private: Clock m_clock; // some kind of clock implementation, which could be or not real time (often it's a virtual time) ComponentSystem m_component_system; boost::stable_vector<Entity> m_entities; // or whatever stable way to list them }; class Universe { public: void update() { parallel_for( auto world_idx : m_awake_worlds_indice ) { m_world[world_idx].update(); } } private: boost::stable_vector<World> m_worlds; std::vector m_awake_worlds_indice; }; ------------------------- This is a raw and a bit complex example but I hope it can clarify my request. Joel Lamotte