Boosting BBVI Tutorial

[lorenzkuhn]

boosting_bbvi_tutorial.ipynb contains my draft of the boosting BBVI tutorial for the Pyro documentation. Please review it - I'd be happy about any kind of feedback and suggestions.

I used pylint to check for PEP8 compliance. I've prioritised consistency with existing tutorials over being compliant with PEP8 in cases where there were conflict between these two.
I've fixed the import order with isort as recommended in the Pyro contribution guidelines .

I don' think there's a need to review the other files in this PR at this point.

[TNU-yaoy]

I made some changes mostly to the text and committed them to a new branch (see #14). Below are two additional comments:

1. you are using 3 different naming conventions for the latent and observed variables: z and x (intro), mu and y (model description), loc and obs (code). It might make sense to unify this if it is not too much work.

2. can we get away with not replicating the observations 10 times?

[gideonite]

Sorry for taking so long to get to this.

There are some code things to be cleaned up: mainly removing commented out code and refactoring plotting code.

More importantly, have we been able to factor out the relbo in a modular fashion?

[gideonite]

Generally want to separate plotting code from the algorithm itself.

[sharrison5]

This is great, thanks!

A collection of fairly random comments on the notebook primarily:

• Do we need to set max_plate_nesting?
• Maybe stick with the p, q notation in the Variational Inference section, and stick with s for the boosting component?
• Any mechanism for attribution etc - not clear at first glance if all the examples are contributed by the Pyro team?
• Agree with Yu re the data*10 statement - if we do need to keep it, perhaps just mention it as an aside?
  Again, looks really good to me. Will let you know if I come across anything else, but this seems basically ready to try and submit AFAIC.

[lorenzkuhn]

I made some changes mostly to the text and committed them to a new branch (see #14). Below are two additional comments:

Thanks for the review, @TNU-yaoy. Your changes to the notebook look good to me; I've merged them into this branch.

1. you are using 3 different naming conventions for the latent and observed variables: z and x (intro), mu and y (model description), loc and obs (code). It might make sense to unify this if it is not too much work.

Good point, I've changed it to be z and x everywhere.

2. can we get away with not replicating the observations 10 times?

The approximation we get is quite a bit better if we use more data. Instead of replicating the list ten times, a cleaner way would be to draw some samples from N(4, 0.1) to get the observations. I've updated the notebook in this way, what do you think about that?

[TNU-yaoy]

looks good to me

[lorenzkuhn]

This is great, thanks!

Hey @sharrison5, thanks a lot for having a look at the tutorial!

A collection of fairly random comments on the notebook primarily:

• Do we need to set max_plate_nesting?

I think we don't strictly need to, i.e. Pyro will try to infer it if it's not explicitly given.

• Maybe stick with the p, q notation in the Variational Inference section, and stick with s for the boosting component?

Makes sense, I've changed this.

• Any mechanism for attribution etc - not clear at first glance if all the examples are contributed by the Pyro team?

Hmm, I didn't find any information on this on the Pyro repo or in the documentation. Maybe there's no such mechanism but that might be negotiable.

• Agree with Yu re the data*10 statement - if we do need to keep it, perhaps just mention it as an aside?

Fixed this by creating observations by sampling from N(4, 0.1).

Again, looks really good to me. Will let you know if I come across anything else, but this seems basically ready to try and submit AFAIC.