Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Learn white_vec parameter of AutoGaussian guide #2946

Merged
merged 38 commits into from
Dec 14, 2021
Merged

Learn white_vec parameter of AutoGaussian guide #2946

merged 38 commits into from
Dec 14, 2021

Conversation

fritzo
Copy link
Member

@fritzo fritzo commented Oct 14, 2021

Addresses #2813
Blocked by pyro-ppl/funsor#553

This adds learnable parameters white_vec to each Gaussian factor in the AutoGaussian guide, so the total set of factors is: loc, scale, white_vec, and prec_sqrt. The parameters (white_vec, prec_sqrt) are important because they are closer to the natural parameters than (loc, scale), and should in theory lead to faster learning. The scale parameter is important because (i) being on a log scale it can quickly adapt to large or small magnitudes, and (ii) it performs row-size scaling of the precision matrix and leads to faster learning (see #2127, #2924). It appears that the loc parameter is also required.

Tested

  • strengthened a dispatch test
  • pass existing tests

@fritzo fritzo mentioned this pull request Oct 20, 2021
3 tasks
@fritzo fritzo mentioned this pull request Oct 22, 2021
6 tasks
@fritzo fritzo added this to the 1.8 release milestone Nov 10, 2021
@fritzo fritzo mentioned this pull request Nov 10, 2021
6 tasks
@fritzo fritzo removed the Blocked label Nov 18, 2021
@fritzo fritzo marked this pull request as ready for review December 13, 2021 23:13
@fritzo fritzo requested a review from eb8680 December 13, 2021 23:13
@fritzo
Copy link
Member Author

fritzo commented Dec 13, 2021

@eb8680 this isn't super high priority, but it would be nice to get it in for the first release of AutoGaussian. Nothing has changed since we discussed this a month or two ago.

Copy link
Member

@eb8680 eb8680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, merging

@eb8680 eb8680 merged commit 1877116 into dev Dec 14, 2021
@eb8680 eb8680 deleted the learn-white-vec branch December 14, 2021 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants