Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transition criteria to GenerationNode #1887

Closed
wants to merge 1 commit into from

Conversation

mgarrard
Copy link
Contributor

@mgarrard mgarrard commented Oct 3, 2023

Summary:
In this diff we do a few things:
(1) Create a TransitionCriterion class:

  • this class will subsume the CompletionCriterion class and will be a bit more flexible
  • it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:

  • minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
  • num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion

(5) clean up compeletion_criterion class bc we don't need it anymore

In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1

Reviewed By: lena-kashtelyan

Differential Revision: D49509997

@facebook-github-bot facebook-github-bot added CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported labels Oct 3, 2023
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

mgarrard pushed a commit to mgarrard/Ax that referenced this pull request Oct 3, 2023
Summary:

In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion

(5) clean up compeletion_criterion class bc we don't need it anymore

In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

crasanders pushed a commit to crasanders/aepsych that referenced this pull request Oct 3, 2023
Summary:
X-link: facebook/Ax#1887

In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion

(5) clean up compeletion_criterion class bc we don't need it anymore

In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
mgarrard pushed a commit to mgarrard/Ax that referenced this pull request Oct 3, 2023
Summary:
X-link: facebookresearch/aepsych#320


In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion


In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1
(6) clean up compeletion_criterion class once new ax release can be pinned to aepsych version

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

mgarrard pushed a commit to mgarrard/Ax that referenced this pull request Oct 4, 2023
Summary:
X-link: facebookresearch/aepsych#320


In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion


In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1
(6) clean up compeletion_criterion class once new ax release can be pinned to aepsych version

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

@codecov-commenter
Copy link

codecov-commenter commented Oct 4, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (994fc89) 94.58% compared to head (80438a9) 94.59%.

❗ Current head 80438a9 differs from pull request most recent head e0a2ff3. Consider uploading reports for the commit e0a2ff3 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1887   +/-   ##
=======================================
  Coverage   94.58%   94.59%           
=======================================
  Files         458      460    +2     
  Lines       43522    43610   +88     
=======================================
+ Hits        41166    41253   +87     
- Misses       2356     2357    +1     
Files Coverage Δ
ax/modelbridge/completion_criterion.py 100.00% <100.00%> (+7.69%) ⬆️
ax/modelbridge/generation_node.py 97.28% <100.00%> (+0.09%) ⬆️
ax/modelbridge/tests/test_completion_criterion.py 100.00% <ø> (ø)
ax/modelbridge/tests/test_transition_criterion.py 100.00% <100.00%> (ø)
ax/storage/json_store/encoders.py 97.74% <100.00%> (ø)
ax/storage/json_store/registry.py 100.00% <100.00%> (ø)
ax/utils/testing/modeling_stubs.py 96.82% <100.00%> (ø)
ax/storage/json_store/decoder.py 87.77% <94.11%> (+0.44%) ⬆️
ax/modelbridge/transition_criterion.py 94.73% <94.73%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mgarrard pushed a commit to mgarrard/Ax that referenced this pull request Oct 4, 2023
Summary:
X-link: facebookresearch/aepsych#320


In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion


In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1
(6) clean up compeletion_criterion class once new ax release can be pinned to aepsych version

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

Summary:
X-link: facebookresearch/aepsych#320


In this diff we do a few things:
(1) Create a TransitionCriterion class:
- this class will subsume the CompletionCriterion class and will be a bit more flexible
- it has the same child classes + maximumtrialsinstatus subclass, we may add more subclasses or fields later as we further test

(2) Create an list of transitioncriterion from the generationstep class:
- minimum_trials_observed can be taken care of by the more flexible MinimumTrialsInStatus class
- num_trials and enforce_num_trials can be taken care of by the more flexible MaximumTrialsInStatus class

(3) adds a doc string to GenNode class - tangential but easy

(4) updates the type of completion_criteria of GenerationStep from CompletionCriterion to TransitionCriterion


In following diffs we will:
(1) add transition criterion to the repr string + some of the other fields that havent made it yet
(2) begin moving the functions related to completing the step up to node and leveraging the transition criterion for checks instead of indexes -- this is where we may need to add additional fields to transitioncriterion
(3) add doc strings to everywhere in teh GenNode class
(4) add additional unit tests to MaxTrials to bring coverage to 100%
(5) skip max trial criterion addition if numtrials == -1
(6) clean up compeletion_criterion class once new ax release can be pinned to aepsych version

Reviewed By: lena-kashtelyan

Differential Revision: D49509997
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D49509997

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 031fa35.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants