Track issues with adjoint

[ordabayevy]

This is to track issues related to using adjoint.

• test_adjoint.py::test_sequential_sum_product_adjoint: xfail_param(MarkovProduct, reason="mysteriously doubles adjoint values?")
MarkovProduct is evaluated again at actual.reduce(sum_op) line:

    with AdjointTape() as actual_tape:
        actual = impl(sum_op, prod_op, trans, time, {"prev": "curr"})
        actual = actual.reduce(sum_op)

It might be due to " .reduce binds variables, which triggers alpha-renaming and reinterprets actual?" from slack discussion with @eb8680

• MarkovProduct is not expanded by lazy (or any other Funsor not defined in adjoint_ops). A temporary solution (Adjoint markovproduct #492 ) might be to define an adjoint for MarkovProduct. Still work in progress.
• TraceEnum_ELBO uses AdjointTape to calculate expectations. Under eager evaluation nan gradients appear in more than half of the tests as caught by Catch nan values in TraceEnum_ELBO grads pyro#2782:

test_enum_funsor.py::test_elbo_plate_plate[2-2] FAILED                   [  1%]
test_enum_funsor.py::test_elbo_enumerate_1[1] FAILED                     [  3%]
test_enum_funsor.py::test_elbo_enumerate_1[10] FAILED                    [  5%]
test_enum_funsor.py::test_elbo_enumerate_2[1] FAILED                     [  7%]
test_enum_funsor.py::test_elbo_enumerate_2[10] FAILED                    [  9%]
test_enum_funsor.py::test_elbo_enumerate_3[1] FAILED                     [ 11%]
test_enum_funsor.py::test_elbo_enumerate_3[10] FAILED                    [ 13%]
test_enum_funsor.py::test_elbo_enumerate_plate_1[batch-1] FAILED         [ 15%]
test_enum_funsor.py::test_elbo_enumerate_plate_1[batch-10] FAILED        [ 16%]
test_enum_funsor.py::test_elbo_enumerate_plate_1[masked-1] FAILED        [ 18%]
test_enum_funsor.py::test_elbo_enumerate_plate_1[masked-10] FAILED       [ 20%]
test_enum_funsor.py::test_elbo_enumerate_plate_2[batch-1] FAILED         [ 22%]
test_enum_funsor.py::test_elbo_enumerate_plate_2[batch-10] FAILED        [ 24%]
test_enum_funsor.py::test_elbo_enumerate_plate_2[masked-1] FAILED        [ 26%]
test_enum_funsor.py::test_elbo_enumerate_plate_2[masked-10] FAILED       [ 28%]
test_enum_funsor.py::test_elbo_enumerate_plate_3[batch-1] FAILED         [ 30%]
test_enum_funsor.py::test_elbo_enumerate_plate_3[batch-10] FAILED        [ 32%]
test_enum_funsor.py::test_elbo_enumerate_plate_3[masked-1] FAILED        [ 33%]
test_enum_funsor.py::test_elbo_enumerate_plate_3[masked-10] FAILED       [ 35%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[False-True-1] PASSED    [ 37%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[False-True-10] PASSED   [ 39%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[True-False-1] PASSED    [ 41%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[True-False-10] PASSED   [ 43%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[True-True-1] PASSED     [ 45%]
test_enum_funsor.py::test_elbo_enumerate_plate_4[True-True-10] PASSED    [ 47%]
test_enum_funsor.py::test_elbo_enumerate_plate_5 XFAIL                   [ 49%]
test_enum_funsor.py::test_elbo_enumerate_plate_6[parallel] FAILED        [ 50%]
test_enum_funsor.py::test_elbo_enumerate_plate_6[sequential] PASSED      [ 52%]
test_enum_funsor.py::test_elbo_enumerate_plate_7[1] FAILED               [ 54%]
test_enum_funsor.py::test_elbo_enumerate_plate_7[10] FAILED              [ 56%]
test_enum_funsor.py::test_elbo_enumerate_plates_1[1] PASSED              [ 58%]
test_enum_funsor.py::test_elbo_enumerate_plates_1[10] PASSED             [ 60%]
test_enum_funsor.py::test_elbo_enumerate_plates_2[1] PASSED              [ 62%]
test_enum_funsor.py::test_elbo_enumerate_plates_2[10] PASSED             [ 64%]
test_enum_funsor.py::test_elbo_enumerate_plates_3[1] PASSED              [ 66%]
test_enum_funsor.py::test_elbo_enumerate_plates_3[10] PASSED             [ 67%]
test_enum_funsor.py::test_elbo_enumerate_plates_4[1] PASSED              [ 69%]
test_enum_funsor.py::test_elbo_enumerate_plates_4[10] PASSED             [ 71%]
test_enum_funsor.py::test_elbo_enumerate_plates_5[1] PASSED              [ 73%]
test_enum_funsor.py::test_elbo_enumerate_plates_5[10] PASSED             [ 75%]
test_enum_funsor.py::test_elbo_enumerate_plates_6[1] PASSED              [ 77%]
test_enum_funsor.py::test_elbo_enumerate_plates_6[10] PASSED             [ 79%]
test_enum_funsor.py::test_elbo_enumerate_plates_7[1] PASSED              [ 81%]
test_enum_funsor.py::test_elbo_enumerate_plates_7[10] PASSED             [ 83%]
test_enum_funsor.py::test_elbo_enumerate_plates_8[False-False-1-1] FAILED [ 84%]
test_enum_funsor.py::test_elbo_enumerate_plates_8[False-True-1-1] XFAIL  [ 86%]
test_enum_funsor.py::test_elbo_enumerate_plates_8[True-False-1-1] FAILED [ 88%]
test_enum_funsor.py::test_elbo_enumerate_plates_8[True-True-1-1] XFAIL   [ 90%]
test_enum_funsor.py::test_elbo_enumerate_plate_9 FAILED                  [ 92%]
test_enum_funsor.py::test_elbo_enumerate_plate_10 FAILED                 [ 94%]
test_enum_funsor.py::test_elbo_enumerate_plate_11 FAILED                 [ 96%]
test_enum_funsor.py::test_elbo_enumerate_plate_12 FAILED                 [ 98%]
test_enum_funsor.py::test_elbo_enumerate_plate_13 FAILED                 [100%]