Accessing struct fields using reflection without producing garbage

One of the good things we’ve made in Academia is its serialization or also known as saving/loading. We were able to create a generic writer and reader that can write/read any type of instance. Such framework has not changed much from introduction until release. This is possible using reflection. The following is the gist of how this is done:

Type type = typeof(SomeClass);
SomeClass instance = new SomeClass();
PropertyInfo[] properties = type.GetProperties(BindingFlags.Public | BindingFlags.Instance);

for(int i = 0; i < properties.Length; ++i) {
    PropertyInfo property = properties[i];
    
    // This is how to get the value
    object value = property.GetGetMethod().Invoke(instance, null);
    
    // We can do anything with value.
    // We can cast it then write it.
    if (property.PropertyType == typeof(int)) {
        int intValue = (int) value;
        // Write intValue to XML or JSON
    }
    
    // This is how to read a value unto a property
    if (property.PropertyType == typeof(int)) {
        int newValue = 111; // Say you got this from an external source like XML or JSON
        property.GetSetMethod().Invoke(instance, new object[] {
            newValue
        });
    }
}

It’s easy to make a serialization framework out of this. You can simply maintain a dictionary of writers or readers using the Type as the key. This way, you can just iterate on the properties, check if there’s a writer or reader on the dictionary then invoke that. You can even create custom attributes so you can specify which properties you want to be written or read.

We can’t use this for our current project, however, as we are using DOTS with almost pure ECS. For those who are not aware, only a subset of C# called High Performance C# (HPC#) can be used for DOTS if you want to leverage the Job System and Burst compiler. Reference types are not allowed so classes are out. We can only use structs and thus, we can’t reuse the serialization framework we made in Academia as it will incur garbage due to frequent boxing. Here’s an example:

// Sample struct to test
private struct Data {
    public int intValue;
}

[Test]
public void SetValueUsingBoxing() {
    Type type = typeof(Data);
    FieldInfo intField = type.GetField("intValue");
    Data data = new Data();
    
    // Boxing (garbage here as struct is wrapped into a reference type)
    object dataAsObject = data;
    intField.SetValue(dataAsObject, 1);
    
    data = (Data) dataAsObject;
    
    Debug.Log($"intValue: {data.intValue}");
    Assert.IsTrue(data.intValue == 1);
}

Since the method FieldInfo.SetValue() requires an object parameter, the struct instance data has to be converted to an object first. Boxing occurs here. It incurs garbage because the value type will be wrapped into a reference type. Something like this is going on in the background:

object dataAsObject = new object(data);

So we explored other options.

The solution

After some trial and errors and asking through forums, the easiest way to do this is to use pointers. Pointers are a concept from C and C++ but they can also be used in C# with the exception that you wrap them under an unsafe scope. Here’s a simple test code to do this:

[Test]
public unsafe void SetValueUsingPointers() {
    Type type = typeof(Data);
    FieldInfo intField = type.GetField("intValue");
    int fieldOffset = UnsafeUtility.GetFieldOffset(intField);
    
    const int testValue = 111;
    
    Data data = new Data();
    
    // This is how to set a value into a struct member using pointers
    void* address = UnsafeUtility.AddressOf(ref data);
    byte* addressAsByte = (byte*) address;
    byte* intFieldAddress = addressAsByte + fieldOffset;
    *(int*) intFieldAddress = testValue;
    
    Debug.Log($"intValue: {data.intValue}");
    Assert.IsTrue(data.intValue == testValue);

    int intValue = (int)intField.GetValue(data);
    Debug.Log($"Read value: {intValue}");
    Assert.IsTrue(intValue == testValue);
}

Using the same struct Data as the test struct, we get its FieldInfo for its member intValue. We then use UnsafeUtility.GetFieldOffset() to get the memory offset of the field. UnsafeUtility is part of the library Unity.Collections which is used frequently when using DOTS.

The next part is we create an instance of Data and try to set its intValue member using pointer magic. We use UnsafeUtility.AddressOf() to get the address of the struct. We then cast it to byte* (pointer to a byte) which is held by addressAsByte. The fieldOffset is then added to addressAsByte to get the address of the intField member. The line *(int*) intFieldAddress means to cast it into an int pointer (int*) then dereference that pointer which is the asterisk before (int*), thus *(int*). What we end up with is a reference to the intValue field that we can then assign a value into.

The next lines after setting the value are assertions to see that testValue has indeed been set to data.intValue. Try running this test and it should pass.

From here, it’s pretty easy to imagine a serialization framework that can accept any type. Let me show you the gist of our generic serializer that writes to XML:

public unsafe class UnmanagedSerializer {
    private readonly Type type;
    private readonly FieldInfo[] fieldInfos;

    private delegate void FieldWriter(XmlWriter writer, FieldInfo field, void* address);

    private readonly Dictionary<Type, FieldWriter> writerMap = new Dictionary<Type, FieldWriter>();

    public UnmanagedSerializer(Type type) {
        this.type = type;
        this.fieldInfos = this.type.GetFields(BindingFlags.Instance | BindingFlags.Public);

        // Populate writerMap
        AddWriter(typeof(int), IntWriter);
        // Writers to other types are added here
    }

    private void AddWriter(Type type, FieldWriter writer) {
        this.writerMap[type] = writer;
    }

    public void Write(XmlWriter writer, void* address) {
        writer.WriteStartElement(this.type.Name);
        WriteFields(writer, address);
        writer.WriteEndElement();
    }

    public void WriteFields(XmlWriter writer, void* address) {
        for (int i = 0; i < this.fieldInfos.Length; ++i) {
            FieldInfo fieldInfo = this.fieldInfos[i];
            // Must have no ExcludePersist attribute
            object[] attributes = fieldInfo.GetCustomAttributes(typeof(ExcludePersist), false);
            if (attributes.Length > 0) {
                // There's an ExcludePersist attribute. We don't write this value.
                continue;
            }
            if (this.writerMap.TryGetValue(fieldInfo.FieldType, out FieldWriter fieldWriter)) {
                fieldWriter.Invoke(writer, fieldInfo, address);
            }
        }
    }

    private static void IntWriter(XmlWriter writer, FieldInfo field, void* address) {
        int value = GetValue<int>(field, address);
        writer.SafeWriteAttributeString(field.Name, value.ToString());
    }

    // Other writer methods are not included for brevity
    
    private static T GetValue<T>(FieldInfo field, void* address) where T : unmanaged {
        int fieldOffset = UnsafeUtility.GetFieldOffset(field);
        byte* addressAsByte = (byte*) address;
        byte* fieldAddress = addressAsByte + fieldOffset;
        return *(T*)fieldAddress;
    }
}

The main idea here is we maintain a Dictionary of delegates called FieldWriter using Type as the key. An example of a FieldWriter here is the method IntWriter(). It uses a utility method GetValue<T>() which uses pointers to get the field value.

The constructor accepts a Type which then caches its array of FieldInfo of fields that are instance (non static) and public. The dictionary of writers is also prepared here. In the actual code, there are more writers added for different types like for float, bool, byte, uint, and other special types that our game has.

After instantiation of the class, the user of the code may then call either Write() or WriteFields() passing the address of the struct that they want to write. Note here that in WriteFields(), we also use a special case where fields with [ExcludePersist] attribute will not be written. ExcludePersist is just written as:

public class ExcludePersist : Attribute {
}

// Usage
struct SomeData {
    public int willBeSerialized;

    [ExcludePersist]
    public int willNotBeSerialized;
}

Here’s a sample usage of this class using Data as the test type:

public unsafe void UnmanagedSerializerTest() {
    Data data = new Data();
    data.intValue = 111;
    
    UnmanagedSerializer serializer = new UnmanagedSerializer(typeof(Data));

    string filePath = Path.Combine(Application.dataPath, "Game/Data/Test/PersistenctTest.xml");
    if (!File.Exists(filePath)) {
        File.Create(filePath);
    }
    
    XmlWriterSettings writerSettings = new XmlWriterSettings() {
        Indent = true
    };
    
    using (FileStream stream = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite, 1024,
        FileOptions.WriteThrough)) {
        using (XmlWriter writer = XmlWriter.Create(stream, writerSettings) ?? throw new InvalidOperationException()) {
            writer.WriteStartDocument();
            serializer.Write(writer, UnsafeUtility.AddressOf(ref data));
            writer.WriteEndDocument();
            
            writer.Flush();
        }
    }
}

The reader counterpart which we aptly named UnmanagedDeserializer uses the same concept only that the delegate is now accepting an XML node that then reads the value from the node and set the value to the address of the field. For example:

private static void IntReader(SimpleXmlNode node, FieldInfo field, void* address) {
    int value = node.GetAttributeAsInt(field.Name);
    SetValue(value, field, address);
}

private static void SetValue<T>(T value, FieldInfo field, void* address) where T : unmanaged {
    int fieldOffset = UnsafeUtility.GetFieldOffset(field);
    byte* addressAsByte = (byte*) address;
    byte* fieldAddress = addressAsByte + fieldOffset;
    *(T*) fieldAddress = value;
}

One scary thing about this is that it can write to member variables marked as readonly. This is good for our use case however because we don’t have to write special code for struct with readonly members.

That’s all I have for now. If you like my posts, please subscribe to my mailing list.