Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging #10

Merged
merged 4 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@ It is 3D global output from the mid-top CAM model, on the original model grid.

However, the demo data here is one very small part of the CAM output due to storage limit of Github. NN trained on this Demodata will not work.

# Installing

Clone this repo and enter it.\
Then run:
```
pip install .
```
to install the neccessary dependencies.\
It is recommended this is done from inside a virtual environment.

# data loader
load 3D CAM data and reshaping them to the NN input.
Expand Down
File renamed without changes.
File renamed without changes.
Empty file added newCAM_emulation/__init__.py
Empty file.
File renamed without changes.
6 changes: 3 additions & 3 deletions train.py → newCAM_emulation/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ def early_stop(self, validation_loss):


## load mean and std for normalization
fm = np.load('Demodata/mean_demo_sub.npz')
fs = np.load('Demodata/std_demo_sub.npz')
fm = np.load('../Demodata/mean_demo_sub.npz')
fs = np.load('../Demodata/std_demo_sub.npz')

Um = fm['U']
Vm = fm['V']
Expand Down Expand Up @@ -92,7 +92,7 @@ def early_stop(self, validation_loss):
if (iter > 1):
model.load_state_dict(torch.load('conv_torch.pth'))
print ('data loader iteration',iter)
filename = './Demodata/newCAM_demo_sub_' + str(iter).zfill(1) + '.nc'
filename = '../Demodata/newCAM_demo_sub_' + str(iter).zfill(1) + '.nc'
print('working on: ', filename)

F = nc.Dataset(filename)
Expand Down
61 changes: 61 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
[build-system]
requires = ["setuptools >= 61"]
build-backend = "setuptools.build_meta"

[project]
name = "newCAM_emulation"
version = "0.0.0"
description = "PyTorch Net to emulate the gravity wave drag in CAM"
authors = [
{ name="Qiang Sun", email="[email protected]" },
]
readme = "README.md"
license = {file = "LICENSE"}
requires-python = ">=3.9"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 3 - Alpha",
"Natural Language :: English",
"Programming Language :: Python :: 3",
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Typing :: Typed',
"Operating System :: OS Independent",
]

dependencies = [
"numpy>=1.20.0",
"torch",
"torchvision",
"scipy",
"matplotlib",
"xarray",
"netcdf4",
]

[project.optional-dependencies]
lint = [
"black>=24.1.0",
"pylint",
# "mypy>=1.0.0",
# "pytest>=7.2.0",
# "pytest-mock",
"pydocstyle",
]

[project.urls]
"Homepage" = "https://github.com/DataWaveProject/newCAM_emulation"
"Bug Tracker" = "https://github.com/DataWaveProject/newCAM_emulation/issues"

[tool.setuptools]
# By default, include-package-data is true in pyproject.toml, so you do
# NOT have to specify this line.
include-package-data = true

[tool.setuptools.packages.find]
where = ["."] # list of folders that contain the packages (["."] by default)
include = ["newCAM_emulation*"] # package names should match these glob patterns (["*"] by default)
exclude = ["Demodata/*"] # exclude packages matching these glob patterns (empty by default)
namespaces = false # to disable scanning PEP 420 namespaces (true by default)