15 minutes
System.Text.Json snakecase

The latest versions of .NET now include support for
snake_case
out of the box. See https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonnamingpolicy?view=net-8.0. I’m leaving this page here because it was a useful exercise, but the content is effectively irrelevant now that first-class support forsnake_case
serialization is available inSystem.Text.Json
.
Sample code
The sample code for this article can be found here https://github.com/michaelrosedev/snakecase_json.
.NET Core 3.0 Released
In my day job I spend a lot of time working with JSON APIs, and performance is always a huge factor when developing new functionality. I have used a number of different JSON libraries in the past, but in the end always tend to default to Newtonsoft.Json.
Recently .NET Core 3.0 was released, and brings with it the new System.Text.Json
APIs, with a focus on performance and throughput (https://docs.microsoft.com/en-us/dotnet/api/system.text.json?view=netcore-3.0).
Snakecase
One of the key use-cases I use Newtonsoft.Json
for is to accept and send snakecase JSON, e.g.:
{
"id": 10,
"some_property": "value",
"nested_object": {
"order_date": "2019-07-30T23:59:59+02:00",
"visible": true
}
}
This JSON is produced from an object such as this:
public class SomeObject {
public string Id { get; set; }
public string SomeProperty { get; set; }
public NestedObject NestedObject { get; set; }
}
public class NestedObject {
public DateTimeOffset OrderDate { get; set; }
public bool Visible { get; set; }
}
As System.Text.Json
promises performance improvements, I decided to experiment and see if I could manage to get it to work with snake case JSON.
To determine if this is a viable use-case, I’m going to write some unit tests and some benchmarks.
Before we begin
Make sure that you’ve got the latest .NET Core 3.0 installed. To check this, you can run:
dotnet --version
Creating some objects to test
The first thing I’m going to do it to create a new empty solution. I’ll then add a new class library to the solution that will contain the DTOs that I want to use to test serialization and deserialization.
Now I’m going to add some simple classes that we can use later.
The top-level object is a Booking
. This doesn’t really have any real-world meaning, but it is an object that we can populate with properties I’m most interested in when serializing, such as DateTimeOffset
, string
, and nested complex objects:
using System;
namespace Sample.Contracts
{
public class Booking
{
public string Id { get; set; }
public DateTimeOffset BookingDate { get; set; }
public string Title { get; set; }
public bool Premium { get; set; }
public Member Member { get; set; }
public Price Price { get; set; }
}
}
The Member
and Price
objects have the following contents:
namespace Sample.Contracts
{
public class Member
{
public string Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string EmailAddress { get; set; }
}
}
namespace Sample.Contracts
{
public class Price
{
public decimal Value { get; set; }
public string Currency { get; set; }
}
}
We can now write some simple unit tests to make sure that it is possible to serialize and deserialize these objects using snake case JSON.
Unit Tests
I’ve added a sample unit test project to my solution. I tend to gravitate towards NUnit
for unit tests, but that’s just a personal preference.
Note: I think TDD is a great approach, and I’m going to lay out some of the steps here that I followed when using this approach. For the purposes of what I wanted to test, I would say that the tests included need to be significantly extended and more complex. I just wanted to prove a very simple use-case.
First we’ll create a test to serialize an object to JSON and verify that the OrderDate
property has been named order_date
in the resulting JSON. This is as far as I’m going for now with this test, but in a real-world solution this wouldn’t be sufficient.
First test:
[Test]
public void CanSerializeObjectToJson() {
var booking = new Booking();
var json = JsonSerializer.Serialize(booking);
Assert.That(
json.Contains("booking_date"),
Is.EqualTo(true)
);
}
Now, if we run this test, it will fail. The default naming case of System.Text.Json
is camel case, so the assertion fails.
To make this test pass, we’re going to need to make the serializer return snake case property names.
Converting camel case to snake_case
There are plenty of ways to convert a string such as ThisString
to snake case (this_string
). Later in this article I’ll show the benchmark comparisons for 2 different techniques I found using a quick Google search. For now,I’ll demonstrate the simple method used.
To begin with, we need some unit tests (TDD, remember!). I’ve created a few different tests:
[Test]
public void SimpleStringCanBeConvertedToSnakeCase()
{
const string Input = "Id";
const string ExpectedOutput = "id";
var result = Input.ToSnakeCase();
Assert.That(
result,
Is.EqualTo(ExpectedOutput)
);
}
In this first test, I simply want to establish that a simple property name (Id
) is converted to its lowercase representation (id
).
Next, I want to test that a string with two words in it is converted to the appropriate snake case format, i.e. MyName
is converted to my_name
:
[Test]
public void MultiWordStringCanBeConvertedToSnakeCase()
{
const string Input = "MyName";
const string ExpectedOutput = "my_name";
var result = Input.ToSnakeCase();
Assert.That(
result,
Is.EqualTo(ExpectedOutput)
);
}
Next, I’m going to test a slightly more complex property name:
[Test]
public void TripleWordStringCanBeConvertedToSnakeCase()
{
const string Input = "ExpectedDeliveryDate";
const string ExpectedOutput = "expected_delivery_date";
var result = Input.ToSnakeCase();
Assert.That(
result,
Is.EqualTo(ExpectedOutput)
);
}
The final two tests I’m adding are some simple “defensive programming” style tests - make sure that there’s no Exception
thrown if the string I’m trying to convert is null
, or if it is an empty string:
[Test]
public void NullStringDoesNotCauseAnException()
{
const string Input = (string) null;
Assert.DoesNotThrow(() => { Input.ToSnakeCase(); });
}
[Test]
public void EmptyStringDoesNotCauseAnException()
{
var input = string.Empty;
Assert.DoesNotThrow(() => { input.ToSnakeCase(); });
}
Now that we’ve got some tests, we can create the implementation to satisfy the tests:
using System;
using System.Linq;
namespace Sample.Serialization
{
/// <summary>
/// Copied from https://gist.github.com/vkobel/d7302c0076c64c95ef4b
/// </summary>
public static class ExtensionMethods {
public static string ToSnakeCase(this string str)
{
return string.Concat(
str.Select(
(x, i) => i > 0 && char.IsUpper(x)
? "_" + x
: x.ToString()
)
).ToLower();
}
}
}
The basic premise of this method is:
- For each character in the string:
- If the character is upper case, append
_{char}
to the result; - Otherwise just append the character to the result
- If the character is upper case, append
- Return the whole result to lowercase
It works, but it’s not very efficient. We’ll see some potentially more efficient code later.
Note: I put this code together using TDD and didn’t follow the exact steps laid out here (i.e. I didn’t write all the tests, then just write the code). However, I assume you don’t want to read this article all week, so I’ve cut things down to be more brief.
Now we are able to convert a CamelCase
string to snake_case
, but how do we wire this up for the new System.Text.Json
methods?
System.Text.Json.JsonNamingPolicy
The implementation is actually very straightforward. Any call to Serialize
or Deserialize<T>
takes an optional JsonSerializerOptions
parameter.
JsonSerializerOptions
itself contains a PropertyNamingPolicy
property, so we simply need to create a new JsonNamingPolicy
which will allow us to handle snake case.
Let’s begin with the tests again:
using NUnit.Framework;
namespace Sample.Serialization.Tests
{
[TestFixture]
public class SnakeCaseNamingPolicyTests
{
private SnakeCaseNamingPolicy _sut;
[SetUp]
public void _Setup()
{
_sut = new SnakeCaseNamingPolicy();
}
[Test]
public void NullStringDoesNotCauseAnException()
{
Assert.DoesNotThrow(() => { _sut.ConvertName((string) null); });
}
[Test]
public void EmptyStringDoesNotCauseAnException()
{
Assert.DoesNotThrow(() => { _sut.ConvertName(string.Empty); });
}
[TestCase("Id", "id")]
[TestCase("Forename", "forename")]
[TestCase("PeopleCarrier", "people_carrier")]
[TestCase("MultiWordString", "multi_word_string")]
public void InputStringIsReturnedInSnakeCase(string input, string expectedOutput)
{
var result = _sut.ConvertName(input);
Assert.That(
result,
Is.EqualTo(expectedOutput)
);
}
}
}
I’ve tried to steer clear of testing the implementation of the snake case extension method again, but to ensure that we are getting the correct results from the
SnakeCaseNamingPolicy
we do have some tests that verify that the output is as expected.
The code isn’t going to compile for these tests, because SnakeCaseNamingPolicy
doesn’t yet exist. It’s an extremely simple implementation:
using System.Text.Json;
namespace Sample.Serialization
{
public class SnakeCaseNamingPolicy : JsonNamingPolicy
{
public override string ConvertName(string name)
{
return name.ToSnakeCase();
}
}
}
We just need to implement the ConvertName(string name)
method, and the implementation itself simply requires us to call our new ToSnakeCase()
extension method on the provided name
parameter.
Now that the implementation exists, our test should compile and they should all pass.
Wiring it all up for our test
Now that we have a SnakeCaseNamingPolicy
, we can update our unit test.
The first change is to add a new [SetUp]
method to the test. This is going to initialise an instance of JsonSerializationOptions
where we define the PropertyNamingPolicy
:
private JsonSerializerOptions _options;
[SetUp]
public void Setup()
{
_options = new JsonSerializerOptions
{
PropertyNamingPolicy = new Serialization.SnakeCaseNamingPolicy()
};
}
Now I want to make sure that we have an actual object to test. In the first pass at this test, we simply created a new Booking
without populating its properties. Now we’re going to make sure that we have a fully-populated instance:
[Test]
public void CanSerializeObjectToJson()
{
var booking = new Booking
{
BookingDate = new DateTimeOffset(2019, 06, 23, 22, 00, 00, TimeSpan.FromHours(1)),
Id = "af43ea6f-b3ff-4640-9a9a-dbfc7544a4a4",
Title = "Sample Booking",
Premium = false,
Price = new Price
{
Value = 9.99M,
Currency = "GBP"
},
Member = new Member
{
EmailAddress = "[email protected]",
FirstName = "William",
LastName = "McDowell",
Id = "7ce13464-a9df-4630-a50b-7fdd8a3661c4"
}
};
// rest of the implementation here...
}
Finally, we want to ensure that we’re passing the new booking
and the _options
to the serialize method:
var json = JsonSerializer.Serialize(booking, _options);
Assert.That(
json.Contains("booking_date"),
Is.EqualTo(true)
);
The full final unit test is shown below:
using System;
using System.Text.Json;
using NUnit.Framework;
using Sample.Contracts;
namespace Sample.Tests
{
public class Tests
{
private JsonSerializerOptions _options;
[SetUp]
public void Setup()
{
_options = new JsonSerializerOptions
{
PropertyNamingPolicy = new Serialization.SnakeCaseNamingPolicy()
};
}
[Test]
public void CanSerializeObjectToJson()
{
var booking = new Booking
{
BookingDate = new DateTimeOffset(2019, 06, 23, 22, 00, 00, TimeSpan.FromHours(1)),
Id = "af43ea6f-b3ff-4640-9a9a-dbfc7544a4a4",
Title = "Sample Booking",
Premium = false,
Price = new Price
{
Value = 9.99M,
Currency = "GBP"
},
Member = new Member
{
EmailAddress = "[email protected]",
FirstName = "William",
LastName = "McDowell",
Id = "7ce13464-a9df-4630-a50b-7fdd8a3661c4"
}
};
var json = JsonSerializer.Serialize(booking, _options);
Assert.That(
json.Contains("booking_date"),
Is.EqualTo(true)
);
}
}
}
This demonstrates that the following JSON is produced:
{
"id": "af43ea6f-b3ff-4640-9a9a-dbfc7544a4a4",
"booking_date": "2019-06-23T22:00:00+01:00",
"title": "Sample Booking",
"premium": false,
"member": {
"id": "7ce13464-a9df-4630-a50b-7fdd8a3661c4",
"first_name": "William",
"last_name": "McDowell",
"email_address": "[email protected]"
},
"price": {
"value": 9.99,
"currency": "GBP"
}
}
That produces exactly the JSON that we’re looking for.
Deserializing snake_case JSON
Now that we know it’s possible to produce JSON in snake case, we also want to ensure that we can deserialize JSON into a Booking
object.
The JsonNamingPolicy
in .NET Core 3.0 appears to be used for both serialisation and deserialisation, so there is no code change required to make deserialisation work. Let’s add a quick unit test to verify this.
Note: The following unit test is another over-simplified example; it demonstrates successful deserialisation of a single property, not the whole object.
[Test]
public void CanDeserializeJsonToObject()
{
const string JsonBooking = @"{
""id"": ""4f9ca774-81b9-4296-a35e-b31b96cedfb7"",
""title"": ""Sample Booking"",
""booking_date"": ""2019-05-06T16:45:00+02:00"",
""premium"": true,
""price"": {
""value"": 12.95,
""currency"": ""EUR""
},
""member"": {
""id"": ""64cc7df1-5635-44d1-bfbc-00289abd3603"",
""first_name"": ""Jessica"",
""last_name"": ""Smithsson"",
""email_address"": ""[email protected]""
}
}";
var booking = JsonSerializer.Deserialize<Booking>(JsonBooking, _options);
Assert.That(
booking.Title,
Is.EqualTo("Sample Booking")
);
}
This test should pass immediately with no code changes required.
Testing Performance
Now that we know that our code works, and we can serialise to and deserialise from snake case JSON, I want to see how the code performs compared to Newtonsoft.JSON
. Let’s add some simple benchmarks.
Before we start
I’ve added a new Sample.Benchmarks
console app to my solution and installed BenchmarkDotNet
:
dotnet add package BenchmarkDotNet --version 0.11.5
I’ve also added Newtonsoft.Json
and made sure that the version of .NET Core that I am using is 3.0
. This allows me to write code to do the equivalent conversion using both Newtonsoft.Json
and System.Text.Json
.
Writing the benchmark
I want the benchmark to be simple, but to represent a real-world use-case. I’m going to test both serialising and deserialising, and I’m going to use the same Booking
object as the unit tests above.
In the benchmark class, I’m going to add a populated Booking
instance for serialisation tests, and a string representation of a serialised Booking
to test deserialisation:
[MemoryDiagnoser]
public class Benchmarks
{
private const string JsonBooking = @"{
""id"": ""4f9ca774-81b9-4296-a35e-b31b96cedfb7"",
""title"": ""Sample Booking"",
""booking_date"": ""2019-05-06T16:45:00+02:00"",
""premium"": true,
""price"": {
""value"": 12.95,
""currency"": ""EUR""
},
""member"": {
""id"": ""64cc7df1-5635-44d1-bfbc-00289abd3603"",
""first_name"": ""Jessica"",
""last_name"": ""Smithsson"",
""email_address"": ""[email protected]""
}
}";
private readonly Booking _booking = new Booking
{
BookingDate = new DateTimeOffset(2019, 06, 23, 22, 00, 00, TimeSpan.FromHours(1)),
Id = "af43ea6f-b3ff-4640-9a9a-dbfc7544a4a4",
Title = "Sample Booking",
Premium = false,
Price = new Price
{
Value = 9.99M,
Currency = "GBP"
},
Member = new Member
{
EmailAddress = "[email protected]",
FirstName = "William",
LastName = "McDowell",
Id = "7ce13464-a9df-4630-a50b-7fdd8a3661c4"
}
};
}
Newtonsoft.Json version
We want to make sure that we configure Newtonsoft.Json
with a snake case naming strategy. This is already built in to the library; we just need to configure it:
private static readonly DefaultContractResolver ContractResolver = new DefaultContractResolver
{
NamingStrategy = new SnakeCaseNamingStrategy()
};
private readonly JsonSerializerSettings _jsonSerializerSettings = new JsonSerializerSettings
{
ContractResolver = ContractResolver
};
The code above gives us a JsonSerializerSettings
that we can pass to our serialize and deserialize methods that use Newtonsoft.Json
:
[Benchmark]
public string SerializeWithNewtonsoft()
{
var result = JsonConvert.SerializeObject(_booking, Formatting.Indented, _jsonSerializerSettings);
return result;
}
[Benchmark]
public Booking DeserializeWithNewtonsoft()
{
var result = JsonConvert.DeserializeObject<Booking>(JsonBooking, _jsonSerializerSettings);
return result;
}
System.Text.Json version
The System.Text.Json
version is very similar, we want a shared JsonSerializerOptions
so that we can configure our JsonNamingPolicy
to be snake case:
private readonly JsonSerializerOptions _options = new JsonSerializerOptions
{
PropertyNamingPolicy = new Serialization.SnakeCaseNamingPolicy()
};
Now we can add the equivalent benchmarks:
[Benchmark]
public string SerializeWithSystemTextJson()
{
var result = System.Text.Json.JsonSerializer.Serialize(_booking, _options);
return result;
}
[Benchmark]
public Booking DeserializeWithSystemTextJson()
{
var result = System.Text.Json.JsonSerializer.Deserialize<Booking>(JsonBooking, _options);
return result;
}
Now let’s run the benchmarks and see the result:
BenchmarkDotNet=v0.11.5, OS=macOS Mojave 10.14.6 (18G95) [Darwin 18.7.0]
Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.0.100
[Host] : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), 64bit RyuJIT
DefaultJob : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), 64bit RyuJIT
Method | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|
Serialize with Newtonsoft | 3.483 us | 0.0717 us | 0.1433 us | 0.6561 | 0.0038 | - | 3.02 KB |
Serialize with System.Text.Json | 2.351 us | 0.0202 us | 0.0179 us | 0.3471 | - | - | 1.6 KB |
Deserialize with Newtonsoft | 4.782 us | 0.0320 us | 0.0299 us | 0.7324 | - | - | 3.37 KB |
Deserialize with System.Text.Json | 3.470 us | 0.0673 us | 0.0899 us | 0.3319 | - | - | 1.53 KB |
So, System.Text.Json
shows itself (for this very limited use-case) to be faster, and to allocate less memory.
Serialisation with System.Text.Json
had a mean of 2.351 us
compared to 3.483 us
, and improvement of ~38.8%
. That doesn’t seem like a massive improvement, but on a system with say 1,000,000 requests per day that could be a huge saving in time spent serialising objects to JSON.
Deserialisation shows a similar improvement. System.Text.Json
had a mean of 3.470 us
compared to 4.782 us
, and improvement of ~31.79%
.
However, I did mention earlier that the code we used to convert a string to snake case was not as efficient as it could be. Another version I found on Stack Overflow uses the new Span
to reduce allocations, and looked a little more efficient.
I added a new ToSnakeCaseSpan
extension method with this implementation and then re-ran the benchmarks.
Note: I actually added some benchmarks specifically to cover the different snake case extension method implementations too, but this post is getting long enough…check the sample code in GitHub to see those benchmarks.
The new implementation (with some tidying):
public static string ToSnakeCase(this string str) {
if (str == null)
{
return string.Empty;
}
var upperCaseLength = str.Count(t => t >= 'A' && t <= 'Z' && t != str[0]);
var bufferSize = str.Length + upperCaseLength;
Span<char> buffer = new char[bufferSize];
var bufferPosition = 0;
var namePosition = 0;
while (bufferPosition < buffer.Length)
{
if (namePosition > 0 && str[namePosition] >= 'A' && str[namePosition] <= 'Z')
{
buffer[bufferPosition] = '_';
buffer[bufferPosition + 1] = str[namePosition];
bufferPosition += 2;
namePosition++;
continue;
}
buffer[bufferPosition] = str[namePosition];
bufferPosition++;
namePosition++;
}
return new string(buffer).ToLower();
}
In order to view the whole set of results, including the previous snake case implementation and the new one, I added another JsonNamingPolicy
to my solution, SnakeCaseNamingPolicySpan
. This allows me to add another couple of benchmarks.
First, I added a new version of JsonSerializerOptions
to the benchmark class:
private readonly JsonSerializerOptions _spanOptions = new JsonSerializerOptions
{
PropertyNamingPolicy = new Serialization.SnakeCaseNamingPolicySpan()
};
Then I added new serialise and deserialise benchmarks:
[Benchmark]
public string SerializeWithSystemTextJsonSpan()
{
var result = System.Text.Json.JsonSerializer.Serialize(_booking, _spanOptions);
return result;
}
[Benchmark]
public Booking DeserializeWithSystemTextJsonSpan()
{
var result = System.Text.Json.JsonSerializer.Deserialize<Booking>(JsonBooking, _spanOptions);
return result;
}
Then I re-ran the benchmarks to see what performs best:
BenchmarkDotNet=v0.11.5, OS=macOS Mojave 10.14.6 (18G95) [Darwin 18.7.0]
Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=3.0.100
[Host] : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), 64bit RyuJIT
DefaultJob : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), 64bit RyuJIT
Method | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|
SerializeWithNewtonsoft | 3.213 us | 0.0319 us | 0.0299 us | 0.6561 | 0.0038 | - | 3.02 KB |
SerializeWithSystemTextJson | 2.318 us | 0.0161 us | 0.0143 us | 0.3471 | - | - | 1.6 KB |
SerializeWithSystemTextJsonSpan | 2.468 us | 0.0470 us | 0.0560 us | 0.3471 | - | - | 1.6 KB |
DeserializeWithNewtonsoft | 4.793 us | 0.0917 us | 0.0982 us | 0.7324 | 0.0076 | - | 3.37 KB |
DeserializeWithSystemTextJson | 3.401 us | 0.0284 us | 0.0266 us | 0.3319 | - | - | 1.53 KB |
DeserializeWithSystemTextJsonSpan | 3.370 us | 0.0360 us | 0.0336 us | 0.3319 | - | - | 1.53 KB |
This was surprising, I had expected the Span
-based version to perform better or to allocate less, but it didn’t!
I’ll need to look into that more.
Conclusion
Using System.Text.Json
is very straightforward. For this very trivial use-case it performs over 30% better than Newtonsoft.Json
. However, before I start advocating that we all drop what we’re doing and immediately switch to System.Text.Json
for all systems, I’ll need to do a little more in-depth analysis and experimentation.
Next Steps
I hear very good things about nuecc’s Utf8Json, so next I am going to include some samples for that library and compare them to Newtonsoft.Json
and System.Text.Json
. I’ve also experimented with Kevin Montrose’s Jil library in the past, so I’m going to take that for a spin too. Watch this space.
3116 Words
2019-09-08 00:00