Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling around optional attrs #3151

Closed
wants to merge 2 commits into from

Conversation

mpolson64
Copy link
Contributor

Summary:
After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:

  1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
  2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
  3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Differential Revision: D66833556

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Dec 5, 2024
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

@codecov-commenter
Copy link

codecov-commenter commented Dec 5, 2024

Codecov Report

Attention: Patch coverage is 95.34884% with 6 lines in your changes missing coverage. Please review.

Project coverage is 95.84%. Comparing base (ceb07f5) to head (3cdc660).

Files with missing lines Patch % Lines
ax/preview/api/client.py 93.65% 4 Missing ⚠️
ax/service/utils/best_point_mixin.py 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3151      +/-   ##
==========================================
- Coverage   95.84%   95.84%   -0.01%     
==========================================
  Files         509      509              
  Lines       51354    51425      +71     
==========================================
+ Hits        49222    49290      +68     
- Misses       2132     2135       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 6, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 6, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 9, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 9, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 9, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

Summary:

Implement "best point" functionality in Client. A few notes:
* Renamed get_best_trial --> get_best_arm: this is more accurate to what it is actually doing (calculating the best in sample point) and renaming will allow us to continue using this method when we create BatchClient
* If GS is not on a predictive step we return {} for the prediction term in both get_best_arm and get_pareto_frontier as well as log an error

Reviewed By: lena-kashtelyan

Differential Revision: D66702545
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D66833556

mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
mpolson64 added a commit to mpolson64/Ax that referenced this pull request Dec 18, 2024
Summary:

After disccusion offline with lena-kashtelyan we decided that _none_throws_experiment() is a little verbose and it would be easier for developers to have a simple property with a short name to access attributes that may or may not be set on the Client. That being said we wanted to still keep them private as to not imply that it was supported behavior for users to call these functions and rely on their outputs' structure not changing.

With that in mind we're making the following changes:
1. All optional attrs are prefixed with _maybe ex. _experiment becomes _maybe_experiment
2. All optional attrs get a @/property where the maybe is dropped and none_throws is called
3. _generation_strategy and _early_stopping_strategy get special methods that either return either return the existing attr or instantiate some default, call client.set, and return. This is important so a user that is fine with defaults will not have to call client.configure_generation_strategy(GenerationStrategyConfig()) manually

Reviewed By: lena-kashtelyan

Differential Revision: D66833556
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in de80239.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants