   Component system: data dragons
  There is always lots of talk about component systems but no much on how the data is managed.  I am having two problems with my system that keep rearing thier ugly heads.  So here is a small example of whas going on.//-----------------------------------------------------------------------class S_PointLight{private:floatm_fRadius;public:virtual voidsetRadius( float val ) { m_fRadius = val; }floatgetRadius() const { m_fRadius = val; }};//-----------------------------------------------------------------------class DM_Com{voidsetObject( DM_Object* pObject );DM_Object*getObject() const;// Also common interface for stuff like events, serialize, etc};//-----------------------------------------------------------------------class DM_ComPointLight : public DM_Com, public S_PointLight{public:// I hate this.voidsetRadius( float val ) { markDirty( DirtyRadiusNetFlag ); S_PointLight::setRadius( val ); }}; The S_PointLight comes from my render system (Simon).  Simon doesn't know anything about the DM_ComPointLight.  Only the DM system uses objects and components.   I have clung onto this separation since I might want lights that don't have the extra weight of being a component.  I am suffering from two problems.  First, if the light radius changes, I need to notify some network clients about the change. The problem is that I tend to write wrapper stuff to do this and these wrappers are prone to error and just makes for more code to manage.  Plus you have an unenforced policy to not change the values directly.  It would be better if I could use metadata someplace that would define variables as replicate, serialize, etc.  My system takes to long to add stuff now.  The other problem is with my object schemas.  My system is like Dungeon Siege in that it flattens out the inheritance tree.  Now I build my world and run the game only to find out that my torches aren't bright enough or the particle system isn't configured right.  If my system kept the inheritance tree intact and referenced it in a flyweight like manner, I could just change the schemas and reload.  The problem is how do you do inheritance of components that don't use generic properties?  My current system has schemas being defined with static fields and then flags determine which fields are valid.  class DM_ComPointLightSchema{private:u32   m_HasFlags;float m_fRadius;S_Color4f  m_Color;};This is really ugly and doesn't save much memory.  Gets worse with arrays and maps.  So currently I have a schema for each component that is define in a lua file and then a I wrote a code generate that makes a schema class simular to above.  I have thought about making the code generator make a DM_ComPointLightData class that manages all the access to the data but I it doesn't fix problems with data inheritance.  I would still be stuck with schemas that have flags that show which fields are valid.   The following will solve both problems but I am really afraid of the cost.class DM_Value{// basiclly a varriant// also has flags like READ_ONLY, INSTANCE, REPLICATE etc// can serialize itself.};class DM_Com{private:std::map< std::string, DM_Value >m_mapVlues;// hash or guidprotected:virtual voidsetProperty( const std::string& id, const DM_Value& v );virtual const DM_ValuegetProperty( const std::string& id ) const;};class DM_ComPointLight : public DM_Com{};  So now each com will be able to serialize itself without extra code and handle whether data needs to be sent to network clients and such.  Also the data inheritance would be easier.  I feel it will make the code less prone to error and easier to maitain but at what price?  What are people doing for this stuff?  I read the thread about an outboard system but there isn't much mention about these two problems.   
  Since C++ doesn't support reflection, you're stuck with one of these compromises.- Don't use string keys would be first advice. It may not be best though- If you're not afraid of extreme template magic, look into Type Lists. It results in ugly (to put it mildly) code, but it can be made almost 100% efficient. With lots of different properties, the symbol names can explode, but under recent compilers, don't have negative impact on release executable. Debug builds however can get messy.- Look into different hash tables, and different allocators. Here you need to balance flexibility with scalability. Cache locality can also become a problem. Can you used fixed allocations, can you prevent collisions, and similar.- Using LUA as primary interface is a useful solution. You intercept modifications on the C++-Lua boundary, and insert whatever needed. This is akin to AOP, and is reasonably efficient. This approach works well with all other solutions, so it's always desirable for user-friendly interface- Use code-generator/macros to create light-weight properties:struct Foo {  void setBaz(const Whatever &x);  Whatever getBaz() const;}- Expand upon the previous advice with C++ operator overloading.template < class T, class Parent >struct Proxy {  Proxy(Parent * ptr, T * ptr);  // assignment operator  // the rest of syntactic sugar};struct Foo {  Proxy<int, Foo> baz() {    return Proxy<int, Foo>(this, &x);  }private:  int x;}...Foo foo;// with sufficient syntactic sugarfoo.baz() = 10;std::cout << foo.baz();This is the cleanest compromise you can get between properties and overhead. Compilers optimize away Proxy completely, and the only difference between this an real properties is the extra () after property name. Macros are useful to generate them though....Long story short - until C++ supports properties or introspection/reflection natively, it comes down to a long list of compromises.Using such models introduces per-value-instance overhead. Most systems use a small number of properties, so it's not a problem.But there is no silver bullet. You just encountered the tip of the ice-berg.Lua solution is by far the cleanest (and reasonably efficient), if you don't require much access to properties from C++ code. I ended up doing that myself.   ;Quote:Original post by AntheusLong story short - until C++ supports properties or introspection/reflection natively, it comes down to a long list of compromises.Using such models introduces per-value-instance overhead. Most systems use a small number of properties, so it's not a problem.But there is no silver bullet. You just encountered the tip of the ice-berg.Lua solution is by far the cleanest (and reasonably efficient), if you don't require much access to properties from C++ code. I ended up doing that myself. Is your suggestion to write components in lua or just having all the data in lua and use the lua C API to access?  I have felt so many times that all this tech is already in lua and python but the problem is that the tools aren't as mature. Also you point about debug builds suffering from template magic is very valid since I spend 90% of the time running debug with asserts and what not.  I have seen on two projects where the debug build is no longer maintained because it ran to slow.;Quote:Original post by Tesshu Is your suggestion to write components in lua or just having all the data in lua and use the lua C API to access? Write logic in Lua, have only the most performance sensitive operations in C++. Your objects are exposed to Lua via C/C++ binding, so that you get rich, OO representation in scripts.Quote: Also you point about debug builds suffering from template magic is very valid since I spend 90% of the time running debug with asserts and what not.  I have seen on two projects where the debug build is no longer maintained because it ran to slow.Not slow. Physical size goes up, when symbol names can balloon to hudrends, even thousands of characters.;Quote:Original post by AntheusQuote:Tesshu Is your suggestion to write components in lua or just having all the data in lua and use the lua C API to access? Write logic in Lua, have only the most performance sensitive operations in C++. Your objects are exposed to Lua via C/C++ binding, so that you get rich, OO representation in scripts. Yeah, I really maulled this over last night.  I think I am going to go with extending my code generator.  Instead of generating code that interfaces with my components I am going to generate a data class for each component that can handle all special requirements.  This will also allow me to decouple the data from the game so it's easier to manage in XSI or whatever you use (Maya. Max etc).I really love lua and it feels like I am re-inventing the wheel but I am afraid of the lack of a debugger on par with VC.  I picked up a book on C# since I suspect it might be good for this stuff (next project).  Thanks for the suggestion though. BTW, how is it working out for you, is it a small hobby project or pretty large?Quote:Original post by AntheusQuote:TesshuAlso you point about debug builds suffering from template magic is very valid since I spend 90% of the time running debug with asserts and what not.  I have seen on two projects where the debug build is no longer maintained because it ran to slow.Not slow. Physical size goes up, when symbol names can balloon to hudrends, even thousands of characters. Well I have found STL stuff to be a complete dog in debug. I think it's because it dosen't inline.  I guess YMMV.