vNext - Support lossless option

[MangelMaxime]

Original issue: thoth-org/Thoth.Json.Giraffe#15

Hum, I guess the problem comes from the fact you are using 'T option option.

And we kind of erase the option type to a really simple representation:

* If `Some ...`, it outputs directly the value

* If `None`, it uses `null` or the absence of the value/property field.

So when nested several option we lose some information

open Fable.Core
open Thoth.Json

let someValue : string option= Some "Maxime"
let noneValue : string option = None
let someSomeValue : string option option = Some (Some "Maxime")
let someNoneValue : string option option = Some None
let deeplyNestedValue : string option option option option option = Some (Some (Some (Some None)))

JS.console.log(Encode.Auto.toString(4, someValue)) // "Maxime"
JS.console.log(Encode.Auto.toString(4, noneValue)) // null
JS.console.log(Encode.Auto.toString(4, someSomeValue)) // "Maxime"
JS.console.log(Encode.Auto.toString(4, someNoneValue)) // null
JS.console.log(Encode.Auto.toString(4, deeplyNestedValue)) // null

I am working on Thoth.Json 5 which is already doing some changes to how it represents some types perhaps we should make a custom representation for option type in order to retain the information. It will increase the JSON size but avoid losing information.

Right now, unless you copy/paste/adapt the code of the Auto modules you can't use it to solve your problem. However, you should be able to write your own Encode.option and Decode.option to have the desired behaviour I think.

Prototype:

There is a lot of code and I didn't focus on making it pretty just wanted to provide some hint for a potential solution. I think by using some helpers etc. it could look much better ^^

open Fable.Core
open Thoth.Json


// Standard behaviour from Thoth.Json Auto modules
module Standard =
    let someValue : string option= Some "Maxime"
    let noneValue : string option = None
    let someSomeValue : string option option = Some (Some "Maxime")
    let someNoneValue : string option option = Some None
    let deeplyNestedValue : string option option option option option = Some (Some (Some (Some None)))

    JS.console.log(Encode.Auto.toString(4, someValue)) // "Maxime"
    JS.console.log(Encode.Auto.toString(4, noneValue)) // null
    JS.console.log(Encode.Auto.toString(4, someSomeValue)) // "Maxime"
    JS.console.log(Encode.Auto.toString(4, someNoneValue)) // null
    JS.console.log(Encode.Auto.toString(4, deeplyNestedValue)) // null

module Custom =

    let log x = JS.console.log x

    module Encode =

        let losslessOption (encoder : 'a -> JsonValue) =
            fun value ->
                match value with
                | Some value ->
                    Encode.object 
                        [
                            "$type$", Encode.string "option"
                            "$state$", Encode.string "Some"
                            "$value$", encoder value
                        ]

                | None ->
                    Encode.object 
                        [
                            "$type$", Encode.string "option"
                            "$state$", Encode.string "None"
                        ]

    module Decode =
        
        let losslessOption (decoder : Decoder<'value>) : Decoder<'value option> =
            Decode.field "$type$" Decode.string
            |> Decode.andThen (fun typ ->
                match typ with
                | "option" ->
                    Decode.field "$state$" Decode.string
                    |> Decode.andThen (fun state ->
                        match state with
                        | "Some" ->
                            Decode.field "$value$" decoder |> Decode.map Some

                        | "None" ->
                            Decode.succeed None
                        
                        | invalid ->
                            "Expected an object with a field `$state$` set to `Some` or `None` but instead got `" + invalid + "`"
                            |> Decode.fail 
                    )

                | invalid ->
                    "Expected an object with a field `$type$` set to `option` but instead got `" + invalid + "`"
                    |> Decode.fail 
            )

    let someValue : string option= Some "Maxime"
    let noneValue : string option = None
    let someSomeValue : string option option = Some (Some "Maxime")
    let someNoneValue : string option option = Some None
    let deeplyNestedValue : string option option option option option = Some (Some (Some (Some None)))

    Encode.toString 4 (Encode.losslessOption Encode.string someValue)
    |> log

    Encode.toString 4 (Encode.losslessOption Encode.string noneValue)
    |> log

    Encode.toString 4 (Encode.losslessOption (Encode.losslessOption Encode.string) someSomeValue)
    |> log

    Encode.toString 4 (Encode.losslessOption (Encode.losslessOption Encode.string) someNoneValue)
    |> log

    Encode.toString 4 (Encode.losslessOption (Encode.losslessOption (Encode.losslessOption (Encode.losslessOption (Encode.losslessOption Encode.string)))) deeplyNestedValue)
    |> log

    match Decode.fromString (Decode.losslessOption Decode.string) (Encode.toString 4 (Encode.losslessOption Encode.string someValue)) with
    | Ok value ->
        match value with
        | Some value ->
            printfn "Got a Some ... %A" value
        
        | None ->
            printfn "Got a None"

    | Error err ->
        JS.console.error err

    match Decode.fromString (Decode.losslessOption Decode.string) (Encode.toString 4 (Encode.losslessOption Encode.string noneValue)) with
    | Ok value ->
        match value with
        | Some value ->
            printfn "Got a Some ... %A" value
        
        | None ->
            printfn "Got a None"

    | Error err ->
        JS.console.error err

    match Decode.fromString (Decode.losslessOption (Decode.losslessOption Decode.string)) (Encode.toString 4 (Encode.losslessOption (Encode.losslessOption Encode.string) someSomeValue)) with
    | Ok value ->
        match value with
        | Some (Some value) ->
            printfn "Got a Some (Some %A)" value
        
        | Some None ->
            printfn "Got a Some None"

        | None ->
            printfn "Got a None"

    | Error err ->
        JS.console.error err

REPL demo

[joprice]

A related issue I hit is being able to distinguish at least the top level presence of a field for things like a patch api where you want a ternary: present / absent / null, to allow distinguishing a field that should be unset versus one that is not being passed. I'm currently using a type Clearable that emulates this in my custom myriad generator.

[njlr]

A good search term is "skippable" e.g. https://github.com/Tarmil/FSharp.SystemTextJson/blob/master/docs/Format.md#skippable