Storing game data as JSON in C#: The how and why

Serialization of Game Data

Who is this for?#

An intemediate programmer who is knowledgable of OOP principles, looking to learn how to and best practices for JSON serialization.

What is serialization?#

Broadly, data exists in two main forms, data while the game is running, memory, and data that exists even when the game is not, in storage. The process of taking data that exists in memory and converting it to storable data is called serialization and the opposite deserialization.

serialization

Why bother?#

Most data about a game does not need to be stored while the game is not running, but there are stuations where there is an advantage to storing data. For example, you may want to save a game state, making it so that your data is moddable by your players or allowing content to be downloaded piecemeal in larger projects. It also makes it so that non-technical members of your team can create content via a content management system or modding tools.

Why JSON?#

Serialized data can really be in any format, an archaic way was to store them as binary, a string of 1s and 0s representing your data. But a useful format is one that is human readable as this means you can modify your data with a text editor. JSON is a common format for storing data in a human readable way. I have specifically chosen to use JSON as it is well supported by the extremely popular NuGet package NewtonSoft.Json.

Lets take an example. Here is a model for Game Data, it contains two values, lives and score.

public class GameData 
{
    public int Lives {get; set;}
    public int Score {get; set;}
}

A JSON representation of this model may look like this.

{
  "lives": 3,
  "score": 400
}

This very closely matches the code representation of the model, making it much easy for someone to open the file and modify the values.

NewtonSoft.Json#

NewtonSoft.Json is an excellent package for JSON serialization. It solves a lot of the common issues surrounding serialization in C#. Consult the documentation for further details, but I will cover the most common usage and problems.

Serializing with NewtonSoft.Json#

This is the process of taking a model and turning it into a JSON string. It will automatically map your values to JSON members and vice versa, handling features like inheritance, generics and lists automatically.

GameData gameData = new GameData();
gameData.Lives = 3;
gameData.Score = 400;
string jsonString = JsonConvert.SerializeObject(myData);

Deserializing with NewtonSoft.Json#

This is the process of taking a JSON string (in this case loaded from a file) and turning it into a model. Because you pass the type of the target model it knows what model the data should be mapped to.

string examplePath = @"C:/path/to/example.json";
using (var streamReader = new StreamReader(examplePath))
{
    string fileContent = streamReader.ReadToEnd();
    GameData gameData = JsonConvert.DeserializeObject<GameData>(fileContent);
}

Common Problems#

A member in my JSON file doesn’t match my model!#

Sometimes, especially if the JSON is being generated somewhere outside your program, the members of the JSON file don’t match the fields of your model. By default NewtonSoft.Json will convert your field names from what C# usually uses, Pascal Case (e.g. GameData), to what JSON usually uses, Camel Case (e.g. gameData). However sometimes there’s a difference in naming, you can resolve this easily with the JsonProperty attribute. If you don’t know what an attribute is, its a bit like a note you attach to a field, class, ect.

{
   "lvs": 2,
   "score": 400
}

This JSON doesn’t match the naming convention, we will need to use the JsonProperty attribute to resolve this.

public class GameData 
{
    [JsonProperty("lvs")]
    public int Lives {get; set;}
    public int Score {get; set;}
}

Essentially, we have left a note for the de/serializer that “Lives” maps to “lvs”.

I’m using a base class member that stores derived types!#

This is a tricky problem to solve in C#, luckily there’s a couple of solutions. To paint a picture of the problem let’s look at an example.

Here is our base class.

public abstract class AbstractAbility 
{
  public string Name { get; set; }
}

And here is an example of a derived type.

public class DamagingAbility : AbstractAbility 
{
  public float Damage { get; set; }
}

Let’s take this as an example.

public class Character 
{
  public string Name { get; set; }
  public AbstractAbility Ability { get; set; }
}

Having an AbstractAbility on the Character allows us to assign any type that derives AbstractAbility, which allows our code to be more flexible (for more explaination as to why we would want this see this article on SOLID principles). But this poses an issue for our serializer, how will it know which type it is meant to be if it could be many different ones?

TypeNameHandling#

NewtonSoft.Json offers a few solutions. The first one is TypeNameHandling, allowing you to change the settings of the serializer to always record the type information, sounds ideal but there’s a few problems with it. First you have two options if you wish to use the feature “All” which adds the $type field to every object that is serialized with its type or “Auto” which only does it when there’s ambiguity.

The disadvantage of “All” is that every object will have an extra field, this can amount to a huge amount of extra data, a lot of the time unnecessary. As for “Auto” the issue is more subtle, because it only uses the $type field when it is ambigious a refactor to the code could cause a compatibility issue with older stored data, as it changes from being ambigious to not or vice versa.

Another larger issue is that if you change the name of a type, or its namespace it will become incompatible.

{
  "$type": "Example.Models.GameData, Example.Models",
  "lives": 3,
  "score": 400
}

The $type field has the fully qualified namespace and class name as well as the assembly, this means that if you choose to change any of those three qualities you will need to correct all the data by alternative means or it will become incompatible.

Next is about human readiblity, part of the reason we are choosing to store our data as JSON is so it can be read and modified by humans. Someone who does not know the internals of your codebase would struggle to create valid JSON files.

There are also other quirks, like NewtonSoft.Json requiring $type to be the first field in the JSON for it to work, which is not a normal consideration of JSON.

JsonConverter#

Seems quite dire then, however there is an alternative but it requires more development and forethought. You can create custom JsonConverters, these allow you to customise the way that NewtonSoft.Json binds JSON members to your fields and is a very powerful tool.

First we need something on the base class to differentiate a derived type from another, the most intuitive thing to use is an enum.

    public enum EffectType
    {
        Resource,
        Event
    }

    [JsonConverter(typeof(EffectConverter))]
    public abstract class AbstractEffect
    {
        public EffectType EffectType { get; set; }
    }

Like before we added an attribute, this attribute decorates the class and lets NewtonSoft.Json to use a different JsonConverter than the default when de/serializing AbstractEffect, in this case EffectConverter which will be explained below.

Here is an example of a derived type.

public class ResourceEffect : AbstractEffect
{
    public string ResourceId { get; set; }
    public float Increment { get; set; }
}

Note the absence of the attribute for this class, if we are directly serializing or deserialzing a ResourceEffect we know what type it is, so we don’t need to use a custom JsonConverter.

Here is an example of a valid JSON.

{
    "effectType": 0,
    "resourceId": "resource_gold",
    "increment": 20
}

Here is the full file for the EffectConverter, fragments of it will be explained further below.

public class EffectSpecifiedConcreteClassConverter : DefaultContractResolver
{
    protected override JsonConverter ResolveContractConverter(Type objectType)
    {
        if (typeof(AbstractEffect).IsAssignableFrom(objectType) && !objectType.IsAbstract)
            return null;
        return base.ResolveContractConverter(objectType);
    }
}

public class EffectConverter : JsonConverter
{
    static JsonSerializerSettings SpecifiedSubclassConversion = new JsonSerializerSettings() { ContractResolver = new EffectSpecifiedConcreteClassConverter() };
    static StringEnumConverter StringEnumConverter = new StringEnumConverter();

    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(AbstractEffect));
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        JObject jo = JObject.Load(reader);
        EffectType effectType = jo["effectType"].Value<EffectType>();
        switch (effectType)
        {
            case EffectType.Resource:
                return JsonConvert.DeserializeObject<ResourceEffect>(jo.ToString(), SpecifiedSubclassConversion);
            default:
                throw new Exception();
        }
        throw new NotImplementedException();
    }

    public override bool CanWrite
    {
        get { return false; }
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException(); 
    }
}

The main override I want to focus on is ReadJson. I will break down this function step by step.

Interestingly we need to know what type it is before we can read it in its entirety, so in order to get the effect type we need to read that single member.

JObject jo = JObject.Load(reader);
EffectType effectType = (EffectType)jo["effectType"].Value<int>();

Then we can deserialize it as the appropriate type.

switch (effectType)
{
    case EffectType.Resource:
        return JsonConvert.DeserializeObject<ResourceEffect>(jo.ToString(), SpecifiedSubclassConversion);
    case EffectType.Event:
        return JsonConvert.DeserializeObject<EventEffect>(jo.ToString(), SpecifiedSubclassConversion);
    default:
        throw new Exception();
}

This is much more maintainable, we can freely move or rename the types, the only thing that has to be maintained is the enum.

Bonus Points!

We can improve this even further, as the effectType JSON member being a number doesn’t mean much to a human. Let’s add another custom JsonConverter to the enum, this time its a built-in one with NewtonSoft.Json.

[JsonConverter(typeof(StringEnumConverter))]
public enum EffectType
{
  Resource,
  Event
}

This JsonConverter converts enums to a string representation instead of a number, this means we can have our JSON look like this instead, much better!

{
    "effectType": "resource",
    "resourceId": "resource_gold",
    "increment": 20
}

All we need to do to the EffectConverter now is replace the lines that get the EffectType with this.

JObject jo = JObject.Load(reader);
string effectString = jo["effectType"].Value<string>();
EffectType effectType = (EffectType)Enum.Parse(typeof(EffectType), effectString, true);

This also improves maintainability, because now the order of the items in an enum do not matter and only thier name needs to be maintained.