Making IChangeSet and Change serialization friendly

[andersstorhaug]

Currently the IChangeSet interfaces implement IEnumerable which isn't handled well by serializers, and the implementations derive from List which give Count and Capacity which as far as I can tell are superfluous when consuming.

Also, the IOptional type used in the Change structs is not serialization friendly.

This, along with some other constraints, makes serialization of change sets somewhat difficult.

My use-case is that I'm experimenting with using DynamicData with Akka.NET remoting in a cluster. For instance, an actor can maintain a SourceCache and expose the underlying IObservable with a remote stream reference, with filters potentially being applied at the source to reduce network I/O. And of course, additional filters/operations can be applied downstream (sorting, binding, etc).

So far, this appears to work quite well with some workarounds for serialization.

I should note that I have previously done something similar in a production application, where we convert DynamicData change-sets (List/Cache) to JSON patches for use in real-time D3 charts in a web frontend.

[glennawatson]

What serializer are you using, almost all the ones I seen have native support for List's

[andersstorhaug]

With Newtonsoft.Json for instance the issue is that a ChangeSet derives from List but also has properties (Adds, Removes etc).

And then Change has the Previous property which, understandably, throw when there is no value.

[andersstorhaug]

With all of this being said, it's always possible to use custom serialization or transform to another type for serialization/deserialization. The latter is what I'm doing now, for compatibility with the built-in serializer for Akka.NET.

Was just thinking that it might be reasonable to modify the underlying types for serialization purposes to reduce the GC cost when doing this. In any case, not raising this as an issue because I know that this is probably not a core concern for the project.

[andersstorhaug]

Would it be worthwhile to attempt a PR to this end?

Some thoughts:

1. It seems that IChangeSet count properties (Adds, Moves etc) are primarily used by tests, and perhaps these can be moved to extension methods
2. On Change<,> and ItemChange<> structs, Previous could now potentially be a nullable reference type

These are API-breaking changes, though (1) should theoretically have minimal impact, while (2) may affect those that have written custom operators.

[nh43de]

I'm a big fan - this would make things easier and provide a standard set of interfaces for getting these streams out of process. I've wanted to do similar things with e.g. Websockets, writing changes to a table, queue, etc. IMO this could be massively useful

Happy to help