Using serialize/deserialize and #statebox{}

[rpt]

I'm using statebox + statebox_riak and I want to serialize/deserialize the statebox record to JSON, so that it's not ETF and can be read in other languages (i.e. Python Riak client is crashing on ETF binaries:). The problem is that the statebox record is not in a header file and I cannot import it to use it directly. Looking at the accessors (value/1, last_modified/1) and constructors (new/1, new/2) from statebox.erl I don't see a way to actually (de)construct the whole statebox record to (de)serialize it. I need to (de)serialize the queue field, right? Am I missing anything?

For now I just copied the record to my module, which is bad. Here's some code:

-spec serialize(statebox:statebox()) -> binary().
serialize(#statebox{value = Dict,
                    queue = Events,
                    last_modified = Ts}) ->
    Value = orddict:fetch(key, Dict),
    Queue = [serialize_event(E) || E <- Events],
    Json = [{<<"value">>, Value},
            {<<"queue">>, Queue},
            {<<"last_modified">>, Ts}],
    jsx:encode(Json).

If this is really a "bug", I would be happy to make a pull request, just let me know if you prefer to move the record into a header file or add queue/1 and new/3.

-spec serialize(statebox:statebox()) -> binary().
serialize(Statebox) ->
    Value = orddict:fetch(key, statebox:value(Statebox)),
    Queue = [serialize_event(E) || E <- statebox:queue(Statebox)],
    Ts = statebox:last_modified(Statebox),
    Json = [{<<"value">>, Value},
            {<<"queue">>, Queue},
            {<<"last_modified">>, Ts}],
    jsx:encode(Json).

Cheers!

[etrepum]

Why not just use base64 encoding or fix the python riak client? There are libraries for reading ETF in other languages, including Python.

[etrepum]

That said, cross-language interop is not a goal of statebox and you'd have to re-implement it basically in its entirety to correctly use anything serialized with it, since consistency is achieved on read. Given these constraints, ETF is the best serialization as it is fast and compact and requires no transcoding to get the right data structures. Funs are used in the queue, so you will probably have a bad time making this work in JSON.

[rpt]

Thanks for the suggestions. Still, IF I want to do a custom serialization, I can't because the API won't let me, that's my point.

[etrepum]

There's nothing truly opaque about records in Erlang. They're just tuples. The API isn't stopping you from doing anything. Sure, it doesn't make it easy for you to do what you think you want to do, but exposing that record definition is going to encourage people to use it, which isn't a good thing. I'd consider adding some kind of to_proplist / from_proplist or something like that, but I'm not sure the use case is really there.