Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long overdue refactoring for quality of life and performance improvements. #140

Merged
merged 213 commits into from
Aug 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
213 commits
Select commit Hold shift + click to select a range
fed23fe
Checkpoint
n8mellis Jul 14, 2023
462eaf4
Checkpoint
n8mellis Jul 14, 2023
19b73a4
Started new build service in Rust.
n8mellis Jul 15, 2023
3221e72
Checkpoint
n8mellis Jul 15, 2023
a647905
Moved examples folder to root directory.
n8mellis Jul 17, 2023
d999770
Checkpoint
n8mellis Jul 18, 2023
dc5d70d
Checkpoint
n8mellis Jul 18, 2023
5390537
Refactored package
n8mellis Jul 19, 2023
45eb3be
Changed protobuf v3.x requirement.
n8mellis Jul 19, 2023
2b38d50
Cleanup, refactoring, and started adding tests.
n8mellis Jul 19, 2023
fc46b92
Checkpoint
n8mellis Jul 19, 2023
bb96a2f
Update type definitions.
n8mellis Jul 19, 2023
242334e
Fixed error in pyproject.toml.
n8mellis Jul 20, 2023
84208ed
Refactoring and added local Docker build.
n8mellis Jul 20, 2023
a6cec52
Added remote builder on client side.
n8mellis Jul 20, 2023
845987c
Implemented deprecated publish method.
n8mellis Jul 20, 2023
160b581
Set remote build server version.
n8mellis Jul 20, 2023
a2c3a7a
Don't suffix the model version on the image name.
n8mellis Jul 20, 2023
1fc6510
Starting work on new remote build server.
n8mellis Jul 20, 2023
462dd1d
Checkpoint
n8mellis Jul 20, 2023
f0158ce
Added init method to ModelMetadata.
n8mellis Jul 21, 2023
4d0f8ad
Fixed recursion and cyclic import issues.
n8mellis Jul 21, 2023
0ce2424
Fixed pyproject.toml.
n8mellis Jul 24, 2023
0ec547d
Checkpoint
n8mellis Jul 24, 2023
8504681
Checkpoint
n8mellis Jul 24, 2023
c0e4826
Checkpoint
n8mellis Jul 24, 2023
7a34daf
Add prepare_context args to DockerBuilder.
n8mellis Jul 25, 2023
5f00e3d
Improve response from build_image.
n8mellis Jul 25, 2023
e2ce3da
Improve build_image logging.
n8mellis Jul 25, 2023
c1bb6bc
Fix legacy predict function signatures.
n8mellis Jul 25, 2023
7d2bf1d
Flush docker build message to terminal immediately.
n8mellis Jul 25, 2023
2ded88b
Add support for Tilt
n8mellis Jul 25, 2023
889e83a
Forgot to add Tiltfile.
n8mellis Jul 25, 2023
43fa29f
Added logging support for build server.
n8mellis Jul 25, 2023
4b03319
Make port a constant.
n8mellis Jul 25, 2023
32186de
Minor fix.
n8mellis Jul 25, 2023
172af55
Renamed job_routes.rs to jobs.rs.
n8mellis Jul 26, 2023
557dbf6
Ignore .DS_Store files.
n8mellis Jul 26, 2023
6de870c
Moved build server into servers directory.
n8mellis Jul 26, 2023
a8d7369
Added NumpyEncoder to new library.
n8mellis Jul 26, 2023
a86d41f
added vscode and notebook-specific files to ignore
bmunday3 Jul 26, 2023
14efbe9
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
bmunday3 Jul 26, 2023
5d11304
Checkpoint
n8mellis Jul 27, 2023
8f81061
It works!!!
n8mellis Jul 28, 2023
8023523
Some refactoring
n8mellis Jul 31, 2023
bf70d3d
Everything works!
n8mellis Jul 31, 2023
c72f024
First attempt at making the metadata class easier to work with.
n8mellis Aug 1, 2023
36634c3
Ignore scratch directory.
n8mellis Aug 1, 2023
d0e8856
Add types to function parameters.
n8mellis Aug 1, 2023
88411d1
updated current examples with public Chassis service URL and final st…
bmunday3 Aug 2, 2023
e9d678e
Tested legacy examples to verify backwards compatibility of 1.5 changes
bmunday3 Aug 2, 2023
136c651
TF example that raises cloudpickle & saved model pickle conflict
bmunday3 Aug 2, 2023
b422a91
Ignoring TF model files
bmunday3 Aug 2, 2023
ba591c8
Set destination registry via Helm values.
n8mellis Aug 2, 2023
19856d4
Add support for setting a registry path prefix via Helm values.
n8mellis Aug 2, 2023
3261d9d
Use BuildResponse for remote builds.
n8mellis Aug 2, 2023
f1f888e
Enable setting whether the static registry is insecure or not in the …
n8mellis Aug 2, 2023
c01ea7f
Added ability to set builder resources via Helm values.
n8mellis Aug 2, 2023
d008c27
Added auto job cleanup and ability to specify insecure registry from …
n8mellis Aug 3, 2023
809fde7
Fixed an issue with the Helm chart.
n8mellis Aug 3, 2023
34ecf17
Quote result so environment variable is a string.
n8mellis Aug 3, 2023
a027d6b
Fix Tilt errors.
n8mellis Aug 3, 2023
aa7486e
Update publish method with metadata changes.
n8mellis Aug 3, 2023
26f3c20
Support registry credentials and require registry url.
n8mellis Aug 7, 2023
f934776
Call webhooks when job completes.
n8mellis Aug 7, 2023
c6515bd
Use proper REST conventions for routes.
n8mellis Aug 7, 2023
81960d0
renamed legacy examples
bmunday3 Aug 7, 2023
3a2e3c9
Added object detection examples from transformers library
bmunday3 Aug 7, 2023
d145bc4
Added ultralytics example but issue with underlying cv2 dependency
bmunday3 Aug 7, 2023
094b6d7
Properly sanitize docker tags.
n8mellis Aug 7, 2023
a225c42
Removed extraneous print.
n8mellis Aug 7, 2023
9c21ebb
The Shutdown route now actually shuts down the server.
n8mellis Aug 7, 2023
b458c4f
Make Shutdown route actually work.
n8mellis Aug 7, 2023
b0874d7
Add validation steps before a container is built.
n8mellis Aug 7, 2023
4bb776d
Cleaned up some unused imports.
n8mellis Aug 7, 2023
95eccf0
First step toward multi-platform builds.
n8mellis Aug 7, 2023
6f2fca8
Reorganizing, removal of old tutorials, rewriting some pages to refle…
bmunday3 Aug 8, 2023
ca7e44d
Modified getting started guide
bmunday3 Aug 8, 2023
5e674c6
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 8, 2023
4fcf010
Fix bug.
n8mellis Aug 8, 2023
63d0412
Add support for building a GPU container.
n8mellis Aug 8, 2023
8d74ed5
Add a warning about Python version when using CUDA.
n8mellis Aug 8, 2023
5e9b497
Add ability to install arbitrary apt packages.
n8mellis Aug 8, 2023
93fcb82
Perform requirements substitutions before building container.
n8mellis Aug 8, 2023
9cca8bd
First stab at re-implementing the kserve server.
n8mellis Aug 8, 2023
d9948a4
Reconfigured navigation
bmunday3 Aug 9, 2023
32ffca9
modified requirements for mkdocs
bmunday3 Aug 9, 2023
f3bd045
decreased minimum python version supported to 3.8
bmunday3 Aug 9, 2023
e395ea4
Added docstrings to chassisml package and chassis.builder package
bmunday3 Aug 9, 2023
e13d725
Added docstrings to public facing ModelMetadata methods
bmunday3 Aug 9, 2023
b10e2cf
Added docstrings to ModelRunner
bmunday3 Aug 9, 2023
5615a68
Created pre-baked quickstart model example
bmunday3 Aug 9, 2023
b9cf389
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 9, 2023
7c595ac
KServe works!
n8mellis Aug 9, 2023
54734fa
Draft v1.5 beta release notes
caradoxical Aug 9, 2023
43b7586
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
bmunday3 Aug 9, 2023
0326199
Added an OMI client.
n8mellis Aug 9, 2023
1c3008f
Modified getting started guide - more still to come
bmunday3 Aug 9, 2023
c48a0b8
Made the OMIClient run method more convenient.
n8mellis Aug 9, 2023
9b6c192
Created quickstart option for pip install that includes dependencies …
bmunday3 Aug 9, 2023
3cc4fd1
Tweaked example
bmunday3 Aug 9, 2023
e788f04
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 9, 2023
feef67f
Made OMI client synchronous.
n8mellis Aug 9, 2023
86a05d2
Forgot to remove a line.
n8mellis Aug 9, 2023
2a63f96
Restructuring getting started section
bmunday3 Aug 9, 2023
c9da284
Fixed typo in model creation
bmunday3 Aug 10, 2023
abe1171
Added model build and inference examples for quickstart model
bmunday3 Aug 10, 2023
08d4f31
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 10, 2023
3d3e3e0
Modified Getting Started section and corresponding example code
bmunday3 Aug 10, 2023
cac72a8
Updated interfaces reference guide
bmunday3 Aug 10, 2023
d3bba3a
Add remote build status check functions to RemoteBuilder.
n8mellis Aug 10, 2023
b2ee316
Update docstrings for RemoteBuilder
n8mellis Aug 10, 2023
b78e3e8
Added docstrings to OMIClient
n8mellis Aug 10, 2023
7402a67
Use Python 3.8 compatible typings.
n8mellis Aug 10, 2023
c6899c6
More Python 3.8 compatible typings.
n8mellis Aug 10, 2023
b4e2fac
WIP re-implementing deprecated ChassisClient functions.
n8mellis Aug 10, 2023
7c9010f
Use future annotations.
n8mellis Aug 10, 2023
50c81a6
Add function to OMIClient to test a container.
n8mellis Aug 10, 2023
2fd85fa
Update deprecated `docker_infer` to use new implementation.
n8mellis Aug 10, 2023
eca9bbd
Attempt to auto-register the guides package to pickle by value.
n8mellis Aug 11, 2023
c2bd846
Tweaks to homepage
caradoxical Aug 11, 2023
73f6367
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
caradoxical Aug 11, 2023
788c816
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 11, 2023
1bc175d
Better deprecation warnings.
n8mellis Aug 11, 2023
2f110ac
Improved docs and deprecation warnings.
n8mellis Aug 11, 2023
8671ecd
changed DigitsSampleData data format to expected array of dictionary …
bmunday3 Aug 11, 2023
0bfb96d
changed DigitsSampleData data format to expected array of dictionary …
bmunday3 Aug 11, 2023
a1809df
Fixed typo - should be dict not set
bmunday3 Aug 11, 2023
0eee37c
Added diffusers example
Aug 11, 2023
4a958bc
Removed obsolete log line.
n8mellis Aug 11, 2023
255332d
Updated inference code with latest DigitsSampleData
bmunday3 Aug 11, 2023
ecc3ead
Documentation updates.
n8mellis Aug 11, 2023
f808883
Documentation update.
n8mellis Aug 11, 2023
004cdc5
Removed the now-obsolete files.
n8mellis Aug 11, 2023
8b9b5e9
Documentation updates.
n8mellis Aug 11, 2023
81a132a
Updated sample inference code
bmunday3 Aug 11, 2023
43efbfc
Updated build code
bmunday3 Aug 11, 2023
73ad995
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 11, 2023
df4c650
Added plumbing to connect docstrings to reference page
bmunday3 Aug 11, 2023
dd377f5
Documentation update.
n8mellis Aug 11, 2023
7a46cae
Update mkdocs to auto-generate SDK documentation.
n8mellis Aug 11, 2023
65c5830
Put SDK docs under `references/sdk`.
n8mellis Aug 11, 2023
9e44454
Second pass at getting started guide and corresponding example code
bmunday3 Aug 12, 2023
21ad6a2
Don’t require minor version requirement for pyenv.
n8mellis Aug 12, 2023
58ac747
Updated docs and code cleanup.
n8mellis Aug 12, 2023
0132fd4
Docs and cleanup.
n8mellis Aug 12, 2023
ea742d4
More docs edits and revisions
caradoxical Aug 13, 2023
3cc42ea
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
caradoxical Aug 13, 2023
eea7f78
Add editorconfig.
n8mellis Aug 14, 2023
7965658
Update Helm chart with one for new server.
n8mellis Aug 14, 2023
73c6217
Move old README to new location.
n8mellis Aug 14, 2023
87ecde7
Move old README to new location.
n8mellis Aug 14, 2023
a8fef94
Removed extraneous file.
n8mellis Aug 14, 2023
9d56f4c
Moved old tests folder to a more appropriate location.
n8mellis Aug 14, 2023
9ec3154
Removed extraneous test file.
n8mellis Aug 14, 2023
ffb6454
Lint suggestions.
n8mellis Aug 14, 2023
56f68ac
Fixed getting-started links
caradoxical Aug 14, 2023
9e2397c
Modified remote service installation guide, reorganized some docs str…
bmunday3 Aug 14, 2023
c7a60f1
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
bmunday3 Aug 14, 2023
7a3f332
Merge branch 'nathan/overhaul' into brad/docs-revamp
bmunday3 Aug 14, 2023
b60c19d
Removed auto expand option for sidebar
bmunday3 Aug 14, 2023
e2993bf
Set up automated testing with tox.
n8mellis Aug 14, 2023
a0b529a
Added diffusers how to guide & corrected typo in full chassis workflo…
bmunday3 Aug 14, 2023
50f908b
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
bmunday3 Aug 14, 2023
64e8685
Updates to interfaces
caradoxical Aug 14, 2023
b003175
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
bmunday3 Aug 15, 2023
c85cfb5
Updates to "get involved"
caradoxical Aug 15, 2023
2f3642d
Futzed around with mkdocstrings
caradoxical Aug 15, 2023
ea6fcd6
reordered frameworks section
bmunday3 Aug 15, 2023
8f14457
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
bmunday3 Aug 15, 2023
36343e1
Added chassis basics
caradoxical Aug 15, 2023
59d73fe
Merge branch 'brad/docs-revamp' of https://github.com/modzy/chassis i…
caradoxical Aug 15, 2023
9c73caf
All tests, lints, and types pass!
n8mellis Aug 15, 2023
d8099ac
Rename `chassis.t` package.
n8mellis Aug 15, 2023
fe3aa38
Add `py.typed` marker file.
n8mellis Aug 15, 2023
f76564f
Don't build docs automatically when running `tox`.
n8mellis Aug 15, 2023
13422cf
Fixed broken docs link.
n8mellis Aug 15, 2023
da47c65
HTML fix to homepage
caradoxical Aug 15, 2023
f94babe
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
caradoxical Aug 15, 2023
e157b7c
Light content edits
caradoxical Aug 16, 2023
da33453
Added transformers framework guide and added sample input/output to d…
bmunday3 Aug 17, 2023
afff4ba
small tweaks to reference index and interfaces pages
bmunday3 Aug 17, 2023
9a5f1fd
Edited transformers guide and corresponding python example script
bmunday3 Aug 17, 2023
c75f059
Added print statement to guide to view test results
bmunday3 Aug 17, 2023
70e6da6
Moved torch example
bmunday3 Aug 17, 2023
0f8c89b
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
bmunday3 Aug 17, 2023
2e11f6f
Added torch guide and tweaked corresponding example python script
bmunday3 Aug 17, 2023
8cac25b
Small stylistic changes and torch installation tip
bmunday3 Aug 17, 2023
2afbc67
Forgot to update one reference to the old "typing" package.
n8mellis Aug 17, 2023
08dc940
Updates to the reference section
caradoxical Aug 17, 2023
365f084
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
caradoxical Aug 17, 2023
99ab959
Optimize `torch` imports
n8mellis Aug 21, 2023
4ed1d54
Removed unused GitHub Action.
n8mellis Aug 24, 2023
d60c0e0
Updated examples to be cleaned up
bmunday3 Aug 24, 2023
53d9d3a
Merge branch 'nathan/overhaul' of https://github.com/modzy/chassis in…
bmunday3 Aug 24, 2023
bb8417e
Updated chassis url for legacy examples
bmunday3 Aug 24, 2023
e466f43
deleted legacy examples folder
bmunday3 Aug 24, 2023
b85b47a
Reorganized examples
bmunday3 Aug 24, 2023
5446e00
Test new build automation
n8mellis Aug 24, 2023
78a67d6
Try using single quotes
n8mellis Aug 24, 2023
c29ae75
Added missing `-e` from `tox` commands.
n8mellis Aug 24, 2023
b1844d9
Use explicit strings for python version.
n8mellis Aug 24, 2023
4ab00bf
Comment out types test for now.
n8mellis Aug 24, 2023
360b5b8
Remove requirements.txt from root.
n8mellis Aug 24, 2023
e8bef8d
Fix python version as a string.
n8mellis Aug 24, 2023
04982d1
Enable `latest` tag for build server image.
n8mellis Aug 24, 2023
deae87d
Merge remote-tracking branch 'origin/main' into nathan/overhaul
n8mellis Aug 24, 2023
67cfa75
Add `tox` as a dependency to build the docs.
n8mellis Aug 24, 2023
72955bb
Attempt to use a Docker image cache to speed up build.
n8mellis Aug 24, 2023
169af04
Remove `nathan/overhaul` as a default build branch.
n8mellis Aug 24, 2023
330b9ff
Update PR build automation.
n8mellis Aug 24, 2023
d402117
Use Docker buildx.
n8mellis Aug 24, 2023
3a73f30
Update build step name.
n8mellis Aug 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
16 changes: 16 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Editor configuration, see http://editorconfig.org
root = true

[*]
charset = utf-8
indent_style = space
indent_size = 4
insert_final_newline = true
trim_trailing_whitespace = true
# max_line_length = 100

[*.yaml]
indent_size = 2

[*.yml]
indent_size = 2
141 changes: 73 additions & 68 deletions .github/workflows/CI_PR.yml
Original file line number Diff line number Diff line change
@@ -1,96 +1,101 @@
# This is a basic workflow to help you get started with Actions

name: CI_PR

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
# push:
# branches: [ main ]
# tags: v*

pull_request:

# Allows you to run this workflow manually from the Actions tab
pull_request:
branches:
- main
- release-v*

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

lint:
name: Run Python lint and type checks
runs-on: ubuntu-latest

steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Setup Repo
uses: actions/checkout@v2
- name: Checkout
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
submodules: recursive
# Semantic version range syntax or exact version of a Python version
python-version: 3.9
cache: pip
cache-dependency-path: |
packages/chassisml/pyproject.toml

- name: Setup Environment Variable
- name: Install dependencies
run: |
export CHASSIS_URL=${{secrets.CHASSIS_URL}}
export DOCKER_USER=${{secrets.DOCKER_USER}}
export DOCKER_PASS=${{secrets.DOCKER_PASS}}
export MODZY_BASE_URL=${{secrets.MODZY_BASE_URL}}
export MODZY_API_KEY=${{secrets.MODZY_API_KEY}}

- name: Set up Python 3.8
uses: actions/setup-python@v2
python -m pip install --upgrade pip
pip install ./packages/chassisml[test]

- name: Lint
run: tox -e lint

# - name: Verify Types
# run: tox -e type

tests:
name: Run tests for ${{ matrix.os }}/${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
include:
- python-version: "3.8"
tox-name: py38
- python-version: "3.9"
tox-name: py39
- python-version: "3.10"
tox-name: py310
- python-version: "3.11"
tox-name: py311
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
# Semantic version range syntax or exact version of a Python version
python-version: 3.8
python-version: "${{ matrix.python-version }}"
cache: pip
cache-dependency-path: |
packages/chassisml/pyproject.toml

- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
pip install ./packages/chassisml[test]

- name: Run tests
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --extend-ignore E303,E501,E712,E111,F821,E225 --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
tests:
tox -e ${{ matrix.tox-name }}

pip:
name: Build SDK
needs: [lint, tests]
runs-on: ubuntu-latest
if: true
env:
CHASSIS_URL: ${{secrets.CHASSIS_URL}}
DOCKER_USER: ${{secrets.DOCKER_USER}}
DOCKER_PASS: ${{secrets.DOCKER_PASS}}
MODZY_BASE_URL: ${{secrets.MODZY_BASE_URL}}
MODZY_API_KEY: ${{secrets.MODZY_API_KEY}}

steps:
# new
- uses: actions/checkout@v2
with:
submodules: recursive
steps:
- uses: actions/checkout@v3

- name: Set up Python 3.8
uses: actions/setup-python@v2
- name: Set up Python
uses: actions/setup-python@v4
with:
# Semantic version range syntax or exact version of a Python version
python-version: '3.8'
python-version: 3.9
cache: pip
cache-dependency-path: |
packages/chassisml/pyproject.toml

- name: Upgrade pip
- name: Install pypa/build
run: |
# install pip=>20.1 to use "pip cache dir"
python3 -m pip install --upgrade pip
- name: Get pip cache dir
id: pip-cache
run: echo "::set-output name=dir::$(pip cache dir)"

- name: Cache dependencies
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: python3 -m pip install -r ./tests/requirements.txt
pip install build

- name: Test with Chassis specific testing scheme
- name: Build distribution
run: |
python -m tests.test --ci True

python3 -m build --sdist --wheel --outdir dist/ packages/chassisml
Loading