-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Library Reorganization for v0.3.0 release #243
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WIP type coverage for core modules
This was referenced Dec 27, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update: The first round of changes around types is now complete (from #244) and ready to merge into
main
from this WIP-v0.3 branch. The changes were both substantial in terms of LOC but have relatively minor impact to the overall workflow, so it makes sense to merge them intomain
now, close some issues, and the progress to the next coherent batch on the road to v0.3.0..from_image_properties
class method #233. Removed convenience classmethodsfrom_image_properties
from across the code base. The recommended workflow is to create a :class:mpol.coordinates.GridCoords
object and pass that to instantiate these objects as needed, rather than passingcell_size
andnpix
separately. For nearly all but trivially short workflows, this simplifies the number of variables the user needs to keep track and pass around revealing the central role of the :class:mpol.coordinates.GridCoords
object and its useful attributes for image extent, visibility extent, etc. Most importantly, this significantly reduces the size of the codebase and the burden to maintain, test, and document multiple entry points to keynn.modules
. We removedfrom_image_properties
frommpol.datasets.GriddedDataset
mpol.datasets.Dartboard
mpol.fourier.NuFFT
mpol.fourier.NuFFTCached
mpol.fourier.FourierCube
mpol.gridding.GridderBase
mpol.gridding.DataAverager
mpol.gridding.DirtyImager
mpol.images.BaseCube
mpol.images.ImageCube
nn.Parameter
from ImageCube #246 Make thepassthrough
behaviour of :class:mpol.images.ImageCube
the default and removed this parameter entirely. Previously, it was possible to have :class:mpol.images.ImageCube
act as a layer withnn.Parameter
s. This functionality has effectively been replaced since the introduction of :class:mpol.images.BaseCube
which provides a more useful way to parameterize pixel values. If a one-to-one mapping (including negative pixels) fromnn.Parameter
s to output tensor is desired, then one can specifypixel_mapping=lambda x : x
when instantiating :class:mpol.images.BaseCube
.mpol.utils.log_stretch
.The remaining work will be re-raised in a new PR
Massive WIP branch to collect several library improvement efforts leading up to v0.3.0 release. These will most likely involve a large and coordinated change to the structure of the codebase, so I'm grouping them together so that they land as a single PR rather than incremental and uncoordinated changes to
main
.These changes are grouped thematically by their aim to improve the quality of MPoL as a PyTorch library
torch.tensor
ahead ofnumpy.array
, thinking about memory locations of arrays during optimization loops) should yield speed and stability improvementsvisread
(pure numpy visibility manipulations and plotting) or anotherMPoL-dev
package (as 'porcelain', to use Git's terminology).The proposed changes under consideration are now tracked by the "Architecture + Design" GitHub project board on MPoL-dev (available internally). But here is a first assessment of planned approach:
Coverage, bug-fix, and 'foundational' changes
disallow_untyped_defs
to prevent regressions..from_image_properties
class method #233 mypy error with forward references.torch.tensor
andnp.array
, which will beget further architectural redesign.log_likelihood
loss incorrect and other loss names misleading #237torch.nn
#131mpol.utils.convert_baselines
andmpol.utils.broadcast_and_convert_baselines
, since this functionality now exists invisread
(mpol.utils.convert_baselines and mpol.utils.broadcast_and_convert_baselines duplicate visread functionality #227).Changes to introduce Stochastic Gradient Descent workflow
TensorDataset
and aDataLoader
Redesign UVDataset with Pytorch idioms in mind #162model.forward
more like ML libraries? #188)Further documentation changes
.py
file inexamples/
(actual workflows following officialpytorch/examples
. E.g.,examples
or a newMPoL-dev
package implementing 'plumbing' or tutorials.Note this PR supersedes #242 after we renamed the branch from v0.2.1 to v0.3.0.