Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nearest correlation matrix function #656

Merged
merged 1 commit into from
Dec 2, 2024

Conversation

tommyod
Copy link
Contributor

@tommyod tommyod commented Nov 26, 2024

  • Implements the H-weighted (elementwise weighting) nearest correlation matrix in function nearest_correlation_matrix()
  • Remove old function _nearest_positive_definite()
  • Add back diagonal check == 1.0 in Iman Conover, since it now passes for inputs from this new function (whereas before it failed on output from _nearest_positive_definite())
  • Replace usage in the main function

I am unsure about these lines. I added them as an extra fail-safe, but adding some slack on the constraint X >> 0 (X must be in the cone of positive symmetric definite matrices) by adjusting it to X - eps * I >> 0 seems to work well. So the code below is never triggered on my random test cases. That being said, if it triggers it could save a user from an unfortunate situation in the future. Any thoughts on this would be appreciated. If you are agnostic, then I say we keep it - it makes a the code more robust at a not-too-high cost. I have no strong opinion however.

  # We might get small eigenvalues due to numerics. Attempt to fix this by
  # recursively calling the solver with smaller values of epsilon. This is
  # an extra fail-safe that is very rarely triggered on actual data.
  is_symmetric = np.allclose(X, X.T)
  is_PD = np.linalg.eig(X).eigenvalues.min() > 0
  if not (is_symmetric and is_PD) and (eps > 1e-14):
      if verbose:
          print(f"Recursively calling solver with eps := {eps} / 10")
      return nearest_correlation_matrix(G, weights=H, eps=eps / 10, verbose=verbose)

  return X

@tommyod tommyod marked this pull request as draft November 26, 2024 08:48
@tommyod tommyod closed this Nov 29, 2024
@tommyod tommyod force-pushed the nearest_correlation_matrix branch from 4d3c60b to e946978 Compare November 29, 2024 14:50
@tommyod tommyod reopened this Nov 29, 2024
@tommyod tommyod marked this pull request as ready for review November 29, 2024 14:58
@dafeda
Copy link
Contributor

dafeda commented Dec 2, 2024

This looks great.
As for the recursion, I don't mind it at all, but should we consider adding max recursion depth somehow?

- remove function _is_positive_definite()
- add symmetry and diagonal 1.0 to tests
- update usage. replace _nearest_positive_definite() with nearest_correlation_matrix()
- add back correlation matrix diagonal check in iman conover
- add function nearest_correlation_matrix and tests
@tommyod tommyod force-pushed the nearest_correlation_matrix branch from 72cbf63 to f2cc08b Compare December 2, 2024 10:04
@dafeda dafeda merged commit a6c3ef1 into equinor:main Dec 2, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants