Wednesday, September 9, 2020

JSON Parsing in Unreal Engine 4

 If your game has any networked features at all, you are most likely using JSON. It's a convenient format for moving data around in general. 


However, Unreal Engine's native JSON libraries are quite verbose and can be difficult to use. As is typical with Unreal Engine, the documentation is sparse. When I was first getting familiar with UE4 JSON, I found myself cross-referencing many third-party guides, docs, examples, and forums. As a disclaimer, I am fairly new to Unreal Engine and the gaming industry in general (my background is in embedded systems), but I've found experienced game developers understandably struggling with the UE4 JSON libraries. I've seen developers feel UE4 JSON is such a pain that they'll even use third-party C++ libraries for JSON parsing, which then opens up the door for tedious conversions of FString to and from std::string which you will spend hours on debugging and maintaining, and learn many painful lessons about string encoding along the way. That's not even accounting for the time and risks involved in adding any third-party dependency. 

Examples of string conversions required to support a third-party C++ JSON library in Unreal Engine:

std::string(TCHAR_TO_ANSI(*MyFString()))
ANSI_TO_TCHAR(MyThirdPartyJson.dump().c_str())


I've also seen developers try to write their own UE4 JSON convenience functions, but these often get coupled into whatever the specific developer wants to do with the JSON and are not general-purpose. 

My goal here is to try to save other developers the pain I experienced with using Unreal Engine 4's native JSON libraries (and the pain caused by trying to get around them) by sharing this guide to help you get started more quickly. I hope the Unreal Engine community in general will benefit by increasing our efficiency and confidence to work with JSON natively in Unreal Engine, especially with the rise in mobile and networked gaming, and as more people of all experience levels use Unreal Engine for game development. 

Setup

Of course you need to include the header:

#include "Json.h"


Less obvious is the fact that you need to include the "Json" dependency in the Build.cs of any plugins or modules where you want to use UE4 JSON:

        PublicDependencyModuleNames.AddRange(new string[] {

            "Core",

            // ... your other dependencies

            "Json",

        });

Serialization

Serialization and deserialization code in UE4 is quite verbose. I think this is what scares a lot of people away from using Unreal Engine's native JSON libraries. Below is example code for serializing a JSON object in UE4:

JSON Object

    TSharedPtr<FJsonObject> MyJson = MakeShareable(new FJsonObject);

    MyJson->SetStringField("MyStringFieldKey", FString("MyStringFieldValue"));

    MyJson->SetNumberField("MyNumberFieldKey", 42);

    

    FString MyJsonFString;

    TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&MyJsonFString);

    FJsonSerializer::Serialize(MyJson.ToSharedRef(), Writer);


Why does the TJsonWriter have to be a separate class which we have to manually create every time?  

Rama only knows. 

(That's a funny joke because Rama is a notable Unreal Engine developer as well as the name of a major deity in Hinduism.)

Serialization of JSON arrays works in a similar way, although you don't need the .ToSharedRef():

JSON Array

    TArray<FName> MyFNameArray;

    TArray<TSharedPtr<FJsonValue>> MyJsonArray;

    // Put some data in the JSON array

    for (auto& Element : MyFNameArray)

    {

     MyJsonArray.Add(MakeShareable(new FJsonValueString(Element.ToString())));

    }

    

    FString MyJsonFString;

    TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&MyJsonFString);

    FJsonSerializer::Serialize(MyJsonArray, Writer);

Deserialization

As mentioned earlier, the deserialization code is quite verbose:

JSON Object

    FString MyJsonFString; // An FString containing a serialized JSON string


    TSharedPtr<FJsonObject> MyJson = MakeShareable(new FJsonObject);

    TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(&MyJsonFString);

    

    if (!FJsonSerializer::Deserialize(Reader, MyJson))

    {

        // The deserialization failed, handle this case

    }

    else

    {

        int MyNumberFieldValue = MyJson->GetNumberField("MyNumberFieldKey");

    }


But this pattern also sanity checks the deserialization which is useful. We have a TJsonReader instead of a TJsonWriter, so it reads from the input string instead of writing to it.

Deserialization of JSON arrays works exactly the same:

JSON Array

    FString MyJsonFString; // An FString containing a serialized JSON string


    TArray<TSharedPtr<FJsonValue>> MyJsonArray;

    TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(&MyJsonFString);

    

    if (!FJsonSerializer::Deserialize(MyJsonArray, MyJson))

    {

        // The deserialization failed, handle this case

    }

    else

    {

        // Can use or iterate through MyJsonArray

    }

Iteration

Iterating through a JSON object is fairly simple, if poorly-documented. Unreal Engine often demands its users to figure things out for themselves. Now you won't have to 😉.

For a JSON object, you have to first get the Values field which contains a TMap<FString, TSharedPtr<FJsonValue>> of all the data in the JSON object. Then you can iterate through this TMap as normal.

JSON Object

    TSharedPtr<FJsonObject> MyJson = MakeShareable(new FJsonObject);

    TMap<FString, TSharedPtr<FJsonValue>> JsonValues = MyJson->Values;


    for (const auto& Element : JsonValues)

    {

        FString FieldName = Element.Key;

        TSharedPtr<FJsonValue> FieldValue = Element.Value;

    }

A JSON array in UE4 is just a TArray<TSharedPtr<FJsonValue>>which can be iterated through as a normal TArray.

JSON Array

    TArray<TSharedPtr<FJsonValue>> MyJsonArray;


    for (const auto& Element : MyJsonArray)

    {

        FString MyStringElement = Element->AsString();

    }



In conclusion, I hope I've explained the benefit of using Unreal Engine's native JSON libraries rather than trying to go around them. I hope this blog post can be a helpful resource for anyone trying to use JSON in Unreal Engine. In the long term I believe we will all benefit as users and developers of Unreal Engine and provide optimal experiences for our players if we try to work with UE4's native formats as much as possible.

9 comments:

  1. I have my Json inside Result FString but if I use &Result here:

    TSharedRef> Reader = TJsonReaderFactory<>::Create(&Result);

    compiler fails ...but If I remove the & it compiles but I dont get the data.


    FString Result;
    TSharedPtr JsonValue;
    TSharedPtr MyJson = MakeShareable(new FJsonObject);
    TSharedRef> Reader = TJsonReaderFactory<>::Create(Result);
    if (FJsonSerializer::Deserialize(Reader,MyJson))
    {

    avatarId = MyJson->GetNumberField("avatarId");
    birthDate = MyJson->GetStringField("birthDate");
    gender = MyJson->GetNumberField("gender");
    playerID = MyJson->GetStringField("id");
    nickname = MyJson->GetStringField("nickname");
    }

    ReplyDelete
  2. This post saved me so much time and grief! Thank you so much!

    ReplyDelete
  3. Thank you for putting this together? @Eldany, this code works for me:

    FString DrifterJSON = "";
    TSharedPtr MyJson = MakeShareable(new FJsonObject);
    TArray> MyJsonArray;
    TSharedRef> Reader = TJsonReaderFactory<>::Create(DrifterJSON);

    ReplyDelete
    Replies
    1. Sorry I meant to put a ! not a ? lol. This helps tremendously.

      Delete
    2. One other note: When iterating through the FJsonObject, I had to get the "key" and "values" from the Element object like so:

      Element.Key
      Element.Value

      The Key and Value are not functions, they are properties.

      Delete
    3. Awesome thanks for the catch! I've updated the example

      Delete
  4. Compiles, but fails during linking. Can you help? Are you on Discord maybe?

    ReplyDelete
    Replies
    1. Hey @qus, did you add "Json" in your build.cs as mentioned in the guide?

      Delete

JSON Parsing in Unreal Engine 4

 If your game has any networked features at all, you are most likely using JSON. It's a convenient format for moving data around in gene...