   My reflection system, at SiCrane's request
  SiCrane asked me to make a thread about my reflection system and explain why i use void* pointers to data.First of all, the system doesn't support templated classes. I wanted to keep it as simple as possible and didnt see a good reason to write all the extra code needed to support templated classes.ok, now to the code... Given a class:class cWidgetBase{public:void setPosition( float px, float py ) { mLocalPos.x = px; mLocalPos.y = py; }void setSize( float sx, float sy ) { mSize.set ( sx, sy ); }cVec2mSize;cVec2mLocalPos;cVec2mTextPos;cVec4mColourStatic;cVec4mBorderColourStatic;cVec4  *mColour;cVec4  *mBorderColour;cStringmText;};We can expose it to the reflection system using the following macros:REFLECT_CLASS ( cWidgetBase );REFLECT_CLASS_METHOD ( cWidgetBase, setPosition, Position X, Position Y, , , );REFLECT_CLASS_METHOD ( cWidgetBase, setSize, Sixe X, Size Y, , , );REFLECT_CLASS_MEMBER ( mSize,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mLocalPos,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mText,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mTextPos,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mColour,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mBorderColour,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mColourStatic,CMV_DEFAULT );REFLECT_CLASS_MEMBER ( mBorderColourStatic, CMV_DEFAULT );END_CLASS ( cWidgetBase );The above code block does not lie within any function scope, its anywhere (i put it near the top) in the cpp file for cWidgetBase. The class is then registerd with the reflection system before the crt calls your main function. Note that member types are automatically detected, as are function paramaters and types. Desctiptions of the paramaters can be given optionally.Once the application has started up, you can create an instance in a particular section by calling:cReflectedInstance *cReflectionSystem::createInstance( const cString className, const cString instanceName, const cString SectionName, const unsigned int ArraySize );cReflectedInstance gives you access to the new object via a void* pointer and the type information. It acts as a generic interface to a reflected object. The reflection system maintains a database of all instances that have been created.You can get access to a member by callingvoid getInstanceMember(const cInstance *instance, const cClassMember *member, cInstance &outInst, unsigned int index = 0) const;cInstance  (outInst) gives you the same interface as cReflectedInstance (cReflectedInstance inherits from cInstance) for non reflected instances. ie instances not stored in the name database.I said that it doesnt support templated classes for simplicity, however it does support my own container classes (cArray, cList, etc.) which are templated. Reflected items of these types are reflected as the above members are, and the container type is also automatically detected and handled by the system. Members can be pointers, as can the type contained in the containers, or statically sized arrays. These are all handled automatically. Static arrays, pointers to arrays and cList/cArray member types are all indexed the same way via getInstanceMember.When i want to serialize everything, i call cReflectionSystem::Serialize which recursively saves out all of the classes members, keeping track of pointers to other objects so that when you deserialise them, all the pointers are automatically fixed up.My lua integration piggy-backs off the reflection system using lua (new)index handlers, allowing me direct access to all instances, members and methods. This is all completely type safe.The reflection system has a lot more features than the basics that i have given here, but i think thats a good starting point.http://img34.imageshack.us/img34/4430/reflection.gifThe screenshot is of my reflection viewer. The objects that you see are the objects used to create the viewer that you're viewing them in. The functionality is scripted in lua. The viewer allows you to edit all member data, link objects up to members using drag and drop, and modify the dynamic container based members. Again, this is all completely type safe.Note how there is no specialised code for any data types. The serialisation happens automatically with out having to write some function that serialises a particular object type.sorry about the spelling mistakes, its nearly 1h30 here so im pretty tired...   
  This is one of typical approaches to this problem.I did however try to point out a rather important aspect that this particular implementation doesn't seem to allow, namely opaque or third-party classes.Your system seems to be closely tied to memory allocation as well, but I personally never encountered a need for true complete state serialization. Just pointing a particular instance and doing deep traversal proved sufficient. This is in part to my preference to encapsulate the ownership, so all instances will be owned by top-level Application object, and all others could then probably be traversed by starting from there.For opaque classes it is usually necessary to support getter/setter functionality. This allows use of, among other things, standard containers. A generic vector would, for example,  be fully identified with a tuple (T[], size, capacity, allocator*). This turns out to be adequate to express it regardless of internal implementation, since all those members are available from existing instance, and can be used to either construct, or initialize an existing one.Usually, this can be achieved by having non-POD type encapsulating access to virtual (not the polymorphism one) member. vector::size()==Size. This is akin to what MVS debugger does, and is quite convenient.For actual manipulation I prefer a hybrid approach. I manually support all built-in types (fundamental types and arrays thereof, as well as standard containers - for better or worse, they are standard) as first-order serialization primitives. Anything else can then be built on top of that.For opaque types, a surrogate class takes care of getting/setting the values.While I never required full state and auto tracking, and find it somewhat too intrusive considering it is incapable of supporting external types, I can imagine it could come in handy at some point.Quote:My lua integration piggy-backs off the reflection system using lua (new)index handlers, allowing me direct access to all instances, members and methods. This is all completely type safe.This is where I have different design perspective. While I am in capacity to transfer deep state to and from Lua scripts, I am very selective about doing it. Scripting for me is something that is supposed to add value, not duplicate what native code does. So scripts contain high-level event handlers and make use of closures, but mostly operate on pass-by-value types, while actual state does not pass native/script boundary.So instead of dealing with full state, script is basically something like this:function on_think(id, x, y) {  travel_to(x+10, y+10, on_arrive, on_stuck)};It's intentionally a very high-level, conceptual code that doesn't have much access to innards of state. Some select query functions are exposed. This is intentional design choice, not a limitation.I also don't like opaque types, the interface I use is based on templates. Syntax is something like this:Reflection<SomeType>::member_by_name("foo");Reflection<SomeType>::from(some_instance).set("foo", 10);All of which is compile-time type-safe.However, the almost exclusively used method of reflecting is completely hidden, and looks something like this:struct Visitor {  template < class T > void operator()(const char * name, T & value);};... Visitor v;SomeType t;visit(v, t);This visits all members, invoking the handler for each one. Only implementation of handler defines what happens. It can be used for network, file database serialization, to query members, find individual ones, or more. Script interface is generated in same way.My personal preference is to have as much type safety as possible, so this particular design is completely compile-time type-safe. There is no polymorphism as such, since at any time, the type of instance being visited is known.   ;  Since i can reflect methods as well as members, i dont think that opaque or third party classes will be a problem. I still cant support standard containers though, because i have no interest in allowing reflection of templated classes, its just too much extra code for little gain. I could however reflect instantiated templated classes, but still have no interest to do so.Memory allocation goes through a memory manager which tacks it. This can be given a custom allocator if need be.My system manually handles built in types, as does yours. But its handled in a way that allows me to easyly add a new built in type handler by means of a "type converter", which is a function pair that convert the type to/from a string, and stored in a hash table keyed with the type name. Strings are what get serialised (im using xml). Type converters can be supplied that convert any types you want.Type converters are also supplied to convert to/from lua data types which are reflected as special data objects (opaque). All the runtime type safety is handled in the type converters.The reason i chose to expose everything to the script is because i wanted to use lua to code the functionality of the widgets used to build the game editor.I have a templated interface similar to your templated one that casts a cInstance (or even a void* pointer) to the template paramater type so you can have fun with it. But all the typesafety is done at runtime, since a cInstance or void* pointer can point to any data type, the only way to do the safety check it at runtime. And yes, void* pointers are safety checked.   ;Quote:First of all, the system doesn't support templated classes. I wanted to keep it as simple as possible and didnt see a good reason to write all the extra code needed to support templated classes.This statement reflects (ahem :) ) a lack of understanding about templates. You don't "support templated classes"; you use templates to support arbitrary classes. If you write code like template <typename T> /* blah blah blah */, the code will not generally care whether T, in turn, is a template type. It only cares, for example, that when you write "T x;", that T is default-constructible.As such, it does not generally take large amounts of extra code to take advantage of templating. In many cases, the only extra code is "template <typename T>".   ;  Actually, I asked about your serialization system. What exactly happens when you serialize one of your containers?   ;Quote:Original post by thefriesSince i can reflect methods as well as members, i dont think that opaque or third party classes will be a problem. I still cant support standard containers though, because i have no interest in allowing reflection of templated classes, its just too much extra code for little gain. I could however reflect instantiated templated classes, but still have no interest to do so.This is unusual, since supporting templated classes is nothing more than extending relevant functions to take templated parameters. Something like:template < class T, class C, class A > void handle(std::basic_string<T,C,A> &);Quote:Memory allocation goes through a memory manager which tacks it. This can be given a custom allocator if need be.I avoid such issues since I consider that full memory managed system isn't possible for C++ code, and there exist managed languages which support all of that robustly.Quote:I have a templated interface similar to your templated one that casts a cInstance (or even a void* pointer) to the template paramater type so you can have fun with it. But all the typesafety is done at runtime, since a cInstance or void* pointer can point to any data type, the only way to do the safety check it at runtime. And yes, void* pointers are safety checked.I never found particular need for this. Reflection will always be used in well defined context. If there is however a need for RTTI, then there are two cases.When dealing with custom classes, I provide appropriate virtual method that handles concrete serialization.When dealing with opaque classes, the visitor system needs to decide what and how to handle them. Consider being given a Texture class, which is some COM object. It could be interface, multiply inherited, or something worse. I have no hope of serializing or in other way manipulating that, unless there exists some adequately functional serialization mechanism in there already.There is another issue which needs to be addressed. In trivial case, supporting serialization via direct member access is simple and efficient. But often, it isn't possible to do that. Consider this:struct Server {  Socket s;};Serializing internal state of socket is pointless. Instead, serialization routines need to be provided that can both "read" and "write" opaque type. Something like:void write(Archive & a, Socket & s) {  a.write(s.getAddress());  a.write(s.getPort());};void read(Archive & a, Socket & s) {  Address a  = a.read();  Port p = a.read();  try {    s->connect(a, p);  } catch (exception) {    throw serialization_exception();  }};Hopefully, the reason for this approach is clear. Full-state serialization is tricky since it's not about memory layout. For example, by using proper pointer types (wrapper around index instead of *) it is possible to do memory dump, that can be restored at any point.The above support for case-by-case opaque types allows true full-state persistence that can even handle transient resources (sockets, 3D stuff, third-party APIs, ...).Just some food for thought.   ;Quote:Original post by ZahlmanQuote:First of all, the system doesn't support templated classes.This statement reflects (ahem :) ) a lack of understanding about templates. You don't "support templated classes"; you use templates to support arbitrary classes. ...His statement refers to his system, not to the templates as such.I think that thefries is using the macros to recursively build static descriptor tree for types, that can be later used to serialize objects. And the problem with templated classes here is that each specialization is basically a different type, so he'd have to use the template parameter types to distinguish between the specializations in his database. So he would have to write something like  template<typename T>  REFLECT_CLASS ( cContainer, T );  ...That can be done, but writing it using macros is clearly pain, so if he doesn't need it .. he doesn't need it [smile];  Or he could just write REFLECT_CLASS((cContainer<Type>));;Quote:Original post by DeyjaOr he could just write REFLECT_CLASS((cContainer<Type>));Yes but he'd have to define it for every specialization again. I suppose he can write it this way now, but it's not what one would call 'support of templated classes'   ;  Why not? They're supported the same as any other class.   