Issue with custom data type serialization/deserialization

[yongkansage]

@danielcweber what is required to support custom data type in the data model? We got the following exception in ReadJson of custom data type (Date) converter due to reader is null. Can you please take a look to see if there is anything obvious missing? Thanks.

---

public class NewtonsoftDateConverter : Newtonsoft.Json.JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return typeof(Date).IsAssignableFrom(objectType);
    }

    public override object ReadJson(Newtonsoft.Json.JsonReader reader, Type objectType, object existingValue, Newtonsoft.Json.JsonSerializer serializer)
    {
        // Not relaxed. Used by model to validate input strictly.
        return new Date(reader.Value.ToString());
    }

    public override void WriteJson(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer)
    {
        writer.WriteValue(value.ToString());
        writer.Flush();
    }
}

---

System.NullReferenceException: Object reference not set to an instance of an object.
at ....Models.NewtonsoftDateConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in ....\Models\Date.cs:line 228
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Populate(JsonReader reader, Object target)
at Newtonsoft.Json.JsonSerializer.PopulateInternal(JsonReader reader, Object target)
at Newtonsoft.Json.JsonSerializer.Populate(JsonReader reader, Object target)
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_0(JToken jToken, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 235 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JToken , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_1(JToken jToken, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 252
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JToken , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_2(JToken jToken, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 256 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_9(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 342
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_10(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 365 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_11(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 378
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_12(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 385 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_13(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 390
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_14(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 422 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.TryDeserialize[TSerialized](TSerialized serializedData, Type fragmentType, IGremlinQueryEnvironment environment) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 44 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_13(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 390
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 , IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_14(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 422 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JToken , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.TryDeserialize[TSerialized](TSerialized serializedData, Type fragmentType, IGremlinQueryEnvironment environment) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 44 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_17(JArray jArray, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 466
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 , IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JToken , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.TryDeserialize[TSerialized](TSerialized serializedData, Type fragmentType, IGremlinQueryEnvironment environment) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 44
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_12(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 383 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.<AddNewtonsoftJson>b__5_13(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 390
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.<Override>b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 , IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55
at lambda_method(Closure , JObject , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer )
at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.<>c.b__5_14(JObject jObject, Type type, IGremlinQueryEnvironment env, Func5 overridden, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 422 at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.<>c__DisplayClass6_01.b__0(TSerialized fragment, Type type, IGremlinQueryEnvironment env, Func5 _, IGremlinQueryFragmentDeserializer recurse) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 55 at lambda_method(Closure , Object , Type , IGremlinQueryEnvironment , IGremlinQueryFragmentDeserializer ) at ExRam.Gremlinq.Core.GremlinQueryFragmentDeserializer.GremlinQueryFragmentDeserializerImpl.TryDeserialize[TSerialized](TSerialized serializedData, Type fragmentType, IGremlinQueryEnvironment environment) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryFragmentDeserializer.cs:line 44 at ExRam.Gremlinq.Core.GremlinQueryExecutionResultDeserializer.GremlinQueryExecutionResultDeserializerImpl.Deserialize[TElement](Object executionResult, IGremlinQueryEnvironment environment) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Deserialization\GremlinQueryExecutionResultDeserializer.cs:line 21 at ExRam.Gremlinq.Core.GremlinQuery6.<ExRam.Gremlinq.Core.IGremlinQueryBase.ToAsyncEnumerable>b__208_0(Object executionResult) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Queries\GremlinQuery.explicit.cs:line 232
at System.Linq.AsyncEnumerable.SelectManyAsyncIterator2.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/SelectMany.cs:line 463 at System.Linq.AsyncIteratorBase1.MoveNextAsync() in //Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75 at System.Linq.AsyncEnumerable.<TryGetFirst>g__Core|291_0[TSource](IAsyncEnumerable1 source, CancellationToken cancellationToken) in //Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 119
at System.Linq.AsyncEnumerable.g__Core|291_0[TSource](IAsyncEnumerable1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 121 at System.Linq.AsyncEnumerable.<FirstOrDefaultAsync>g__Core|287_0[TSource](IAsyncEnumerable1 source, CancellationToken cancellationToken) in //Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 32
at ExRam.Gremlinq.Core.GremlinQueryExtensions.FirstOrDefaultAsync[TElement](IGremlinQueryBase`1 query, CancellationToken ct) in ....\ExRam.Gremlinq\src\ExRam.Gremlinq.Core\Extensions\GremlinQueryExtensions.cs:line 21

[danielcweber]

Custom Newtonsoft JsonConverters are not supported by Gremlinq and if these things work, it's coincidence and a hack. Instead, check out these sources for serialization/deserialization of custom types:

ConfigureSerializer in a unit test

ConfigureDeserializer for TimeSpans

[yongkansage]

Will you please share a code snippet to illustrate that? Thanks.

[danielcweber]

https://github.com/ExRam/ExRam.Gremlinq/blob/ff147e9ee4f07e6962b06cf3cfa7ff5376f5f813/test/ExRam.Gremlinq.Core.Tests/GremlinQueryTest.cs#L83

[yongkansage]

Got it. Thanks! Any plan to write some documentation for this library in the future?

[danielcweber]

Any plan to write some documentation for this library in the future?

Yes of course!

[vv-myst]

Hi @danielcweber , thanks for the wonderful library. it has been quite helpful. on the topic of serialization/deserialzation, do you plan to support attribute based Newtonsoft Json Converters in the newer version (12.0.0 currently in preview)? I see that you have extracted all Newtonsoft related stuff into a separate nuget package, hence the question.

I managed to get it working with the links you provided above, but i would like to avoid rewriting/transforming existing JsonConverters if possible. If you already do support this in the preview version of the package, then can you please point me in the right direction so that i use my existing custom Json Converters written using Newtonsoft.json? Thanks for your help in advance :-)