C++ Reflection: Type MetaData: Part 3 – Improvements

In our last article we learned how to store information about a classes’s members, however there are a couple key improvements that need to be brought to the MetaData system before moving on.

The first issue is with our RemQual struct. In the previous article we had support for stripping off qualifiers such as *, const or &. We even had support for stripping off an R-value reference. However, the RemQual struct had no support for a pointer to a pointer. It is weird that RemQual would behave differently than RemQual, and so on. To solve this issue we can cycle down, at compile time, the type through the RemQual struct recursively, until a type arrives at the base RemQual definition. Here’s an example:

As you can see, this differs a bit from our previous implementation. The way it works is by passing in a single type to the RemQual struct via typename T. Then, the templating matches the type provided with one of the overloads and feeds the type back into the RemQual struct with less qualifiers. This acts as some sort of compile-time “recursive” qualifier stripping mechanism; I’m afraid I don’t know what to properly call this technique. This is useful for finding out what the “base type” of any given type.

It should be noted that the example code above does not strip pointer qualifiers off of a type. This is to allow the MetaData system to properly provide MetaData instances of pointer types; which is necessary to reflect pointer meta.

It should be noted that in order to support pointer meta, the RemQual struct will need to be modified so it does not strip off the * qualifier. This actually applies to any qualifier you do not wish to have stripped.

There’s one last “improvement” one could make to the RemQual struct that I’m aware of. I don’t actually consider this an improvement, but more of a feature or decision. There comes a time when the user of a MetaData system may want to write a tidbit of code like the following:

Say the user wants to send a message object from one place to another. Imagine this message object can take three parameters of any type, and the reflection system can help the constructor of the message figure out the types of the data at run-time (how to actually implement features like this will be covered once Variants and RefVariants are introduced). This means that the message can take three parameters of any type and then take them as payload to deliver elsewhere.

However, there’s a subtle problem with the “Message ID” in particular. Param1 and Param2 are assumed to be POD types like float or int, however “Message ID” is a const char * string literal. My understanding of string literals in C++ is that they are of the type: const char[ x ], x being the number of characters in the literal. This poses a problem for our templated MetaCreator, in that every value of x will create a new MetaData instance, as the templating treats each individual value of x as an entire new type. Now how can RemQual handle this? It gets increasingly difficult to actually manage Variants and RefVariant constructors for string literals for reasons explained here, though this will be tackled in a later article.

There are two methods of handling string literals that I am aware of; the first is to make use of some form of a light-weight wrapper. A small wrapper object can contain a const char * data member, and perhaps an unsigned integer to represent the length, and any number of utility functions for common string operations (concat, copy, compare, etc). The use of such a wrapper would look like:

The S would be the class type of the wrapper itself, and the constructor would take a const char *. This would require every place in code that handles a string literal to make use of the S wrapper. This can be quite annoying, but has great performance benefits compared to std::string, especially when some reference counting is used to handle the heap allocated const char * data member holding the string data in order to avoid unnecessary copying. Here’s an example skeleton class for such an S wrapper:

As I mentioned before, I found this to be rather annoying; I want my dev team and myself to be able to freely pass along a string literal anywhere and have MetaData handle the type properly. In order to do this, a very ugly and crazy solution was devised. There’s a need to create a RemQual struct for every [ ] type for all values of x. This isn’t possible. However, it is possible to overload RemQual for a few values of x, at least enough to cover any realistic use of a string literal within C++ code. Observe:

The macro ARRAY_OVERLOAD creates a RemQual overload with a value of x. The __COUNTER__ macro (though not standard) increments by one each time the macro is used. This allows for copy/pasting of the ARRAY_OVERLOAD macro, which will generate a lot of RemQual overloads. I created a file with enough overloads to cover any realistically sized string literal. As an alternative to the __COUNTER__ macro, __LINE__ can be used instead, however I imagine it might be difficult to ensure you have one definition per line without any gaps. As far as I know, __COUNTER__ is supported on both GNU and MSVC++.

Not only will the ARRAY_OVERLOAD cover types of string literals, but it will also cover types with array brackets [ ] of any type passed to RemQual.

The second issue is the ability to properly reflect private data members. There are three solutions to reflecting private data that I am aware of. The first is to try to grant access to the MetaData system by specifying that the MetaCreator of the type in question is a friend class. I never really liked the idea of this solution and haven’t actually tried it for myself, and so I can’t really comment on the idea any further than this.

The next possible solution is to make use of properties. A property is a set of three things: a gettor; a settor; a member. The gettor and settor provide access to the private member stored within the class. The user can then specify gettors and/or settors from the ADD_MEMBER macro. I haven’t implemented this method myself, but would definitely like if I find the time to create such a system. This solution is by far the most elegant of the three choices that I’m presenting. Here’s a link to some information on creating some gettor and settor support for a MetaData system like the one in this article series. This can potentially allow a MetaData system to reflect class definitions that the user does not have source code access to, so long as the external class has gettor and settor definitions that are compatible with the property reflection.

The last solution is arguably more messy, but it’s easier to implement and works perfectly fine. I chose to implement this method in my own project because of how little time it took to set up a working system. Like I said earlier, if I have time I’d like to add property support, though right now I simply have more important things to finish.

The idea of the last solution is to paste a small macro inside of your class definitions. This small macro then pastes some code within the class itself, and this code grants access to any private data member by using the NullCast pointer trick. This means that in order to reflect private data, you must have source code access to the class in question in order to place your macro. Here’s what the new macros might look like, but be warned it gets pretty hectic:

The META_DATA macro is to be placed within a class, it places a couple declarations for NullCast, AddMember and RegisterMetaData. The DEFINE_META macro is modified to provide definitions for the method declarations created by the META_DATA macro. This allows the NullCast to retrieve the type to cast to from the DEFINE_META’s TYPE parameter. Since AddMember method is within the class itself, it can now have proper access to private data within the class. The AddMember definition within the class then forwards the information it gathers to the AddMember function within the MetaCreator.

In order for the DEFINE_META api to remain the same as before, the META_DATA macro creates a RegisterMetaData declaration within the class itself. This allows the ADD_MEMBER macro to not need to user to supply to type of class to operate upon. This might be a little confusing, but imagine trying to refactor the macros above. Is the RegisterMetaData macro even required to be placed into the class itself? Can’t the RegisterMetaData function within the MetaCreator call AddMember on the class type itself? The problem with this is that the ADD_MEMBER macro would require the user to supply the type to the macro like this:

This would be yet another thing the user of the MetaData system would be required to perform, thus cluttering the API. I find that by keeping the system as simple as possible is more beneficial than factoring out the definition of RegisterMetaData from the META_DATA macro.

Here’s an example usage of the new META_DATA and DEFINE_META macros:

The only additional step required here is for the user to remember to place the META_DATA macro within the class definition. The rest of the API remains as intuitive as before.

Here’s a link to a compileable (in VS2010) example showing everything I’ve talked about in the MetaData series thus far. The next article in this series will likely be in creating the Variant type for PODs.


8 thoughts on “C++ Reflection: Type MetaData: Part 3 – Improvements

  1. Brad

    Can you possibly clarify something for me?

    It looks like you are basically using the offsetof trick to access data members in the classes which are registered with the meta system. According to the C++ standard the offsetof trick is only guaranteed to work on POD (Plain Old Data). I think you mention that we are only considering POD right at the end… but the part that confuses me is your Sprite example uses a class which isn’t POD! (the example Sprite has a non-default constructor and a virtual member function, making it non-trivial and not a standard layout class)

    So I guess my question is:  Does this only work for POD data types, or am I missing some critical (and awesome) detail which makes it work for other types as well?

    1. Randy Gaul Post author

      The standard only guarantees (afaik) that their own offsetof macro works on POD data types. It pretty much has to work for anything with a data member otherwise pointers in general won’t work, despite what the standard says. You can also use a similar trick when converting pointers from base to derived types involving multiple inheritance.

  2. Brad

    As far as I know the issue is that for non-POD types the distance from the this pointer to a member’s address could vary based on the instance of the class. That is why member function pointers can be larger than non-member function pointers. (last time I checked with MSVC, member function pointers can vary from 4 to 20 bytes in size because the “pointer” is actually a struct that the compiler uses to locate the member at runtime. http://blogs.msdn.com/b/oldnewthing/archive/2004/02/09/70002.aspx)
    In practice this only appears to be an issue if you want to know the offset from a derived type to a base type’s member.(http://stackoverflow.com/a/1130035, is fairly illustrative)

    So for your Sprite example, I guess it should work as long as you only try to ADD_MEMBER for members of the Sprite and _not_ for members of the Component it inherits from.

    Conclusion: Technically, calculating the byte offset for a member of a non-POD type is in undefined-behavior-land, but in practice it works. And it isn’t worth wasting time fixing something which isn’t broken.

    Also, thanks for the response! I have been enjoying your articles on C++ reflection, and the Game Engine Architecture Club channel on youtube.

    1. Randy Gaul Post author

      Hi there! You’re right in that member function pointers can be obnoxiously large and this is due to the implementation of multiple inheritance. As far as I can see the complexity arises when you need to upcast types that have virtual functionality defined in a derived type. I’m sure you can imagine why this would be weird to deal with in terms of the compiler handling both a run-time function pointer along with the this pointer offset from the derived type.

      So getting the member of any class will work so long as the member is within itself or anything it inherited from. You’re right (at least from what I remember) that this is *technically* undefined, but any basic understanding of how pointers work will show that this will always be safe: the syntax of p->member will always jump forward in memory an appropriate amount to dereference the member. If you have a type comprised of multiple bases care would have be taken, and casting to the derived type would be necessary to let the C++ compiler apply appropriate pointer offsets. Does this make sense?

      I’m glad you like the content :)

    1. Randy Gaul Post author

      Functionally probably not much of a difference. std::remove_cv implementations will probably differ from compiler to compiler. RemQual itself is pretty small so pulling in all of type_traits header is probably a bit unnecessary.

  3. Indi



Leave a Reply

Your email address will not be published. Required fields are marked *