Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routes/scouter #53

Merged
merged 168 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
711c45d
ongoing
thorrester Sep 17, 2024
be4a450
add get profile route
thorrester Sep 18, 2024
76c6749
add tests
thorrester Sep 18, 2024
0fce62e
add drift values
thorrester Sep 18, 2024
a527177
formatting
thorrester Sep 18, 2024
beda667
lints
thorrester Sep 18, 2024
73e8ca1
add feature filter
thorrester Sep 18, 2024
c3f3b63
add monitoring folder
thorrester Sep 18, 2024
7ae0c22
add update routes
thorrester Sep 18, 2024
3744e53
update profile logic
thorrester Sep 18, 2024
bc3260f
call driftprofile from modelcard
thorrester Sep 18, 2024
d6f8599
load drift profile logic
thorrester Sep 18, 2024
de367e9
test for loading
thorrester Sep 18, 2024
ab166e8
updating metadata for updating and drift profile
thorrester Sep 18, 2024
9b1b317
check for existing
thorrester Sep 18, 2024
157cee4
lints
thorrester Sep 18, 2024
cb11e00
conform to new templateresponse standards
thorrester Sep 18, 2024
42b53a4
conform to new templateresponse standards
thorrester Sep 18, 2024
34f2b6f
client update logic
thorrester Sep 18, 2024
81bb07f
healthcheck test
thorrester Sep 18, 2024
4b959df
healthcheck test
thorrester Sep 18, 2024
233d700
formatting
thorrester Sep 18, 2024
79d0636
add server side logic
thorrester Sep 18, 2024
f75030b
test update with drift profile
thorrester Sep 18, 2024
42e6af5
format
thorrester Sep 18, 2024
5a5810c
typescript tests for update and get profile
thorrester Sep 18, 2024
0757b1d
typescript tests for update and get profile
thorrester Sep 18, 2024
f38c800
add drift values typescript test
thorrester Sep 18, 2024
0f313e7
edits to chart
thorrester Sep 19, 2024
72ceb52
edits to chart
thorrester Sep 19, 2024
630259f
chart
thorrester Sep 19, 2024
32c5af1
chart
thorrester Sep 19, 2024
7ca5623
ui route for feature distribution
thorrester Sep 19, 2024
b38397d
working on monitoring
thorrester Sep 20, 2024
a3451bf
updates
thorrester Sep 20, 2024
5c2d7b0
8890
thorrester Sep 20, 2024
68d5192
update hamburger
thorrester Sep 20, 2024
137de4e
item dropdown
thorrester Sep 20, 2024
2499932
time rang
thorrester Sep 20, 2024
2dece6b
add tests
thorrester Sep 20, 2024
a919221
formatting
thorrester Sep 25, 2024
f968b68
update
thorrester Sep 25, 2024
97e216d
renaming
thorrester Sep 25, 2024
5152dbd
add moitor alert pathing
thorrester Sep 25, 2024
a3095d1
add get alert route
thorrester Sep 25, 2024
d490eed
add side scroll
thorrester Sep 25, 2024
677bb6b
test
thorrester Sep 25, 2024
3e3218b
Update Profile.svelte
thorrester Sep 25, 2024
e539bf8
working on updating profile
thorrester Sep 26, 2024
0bde705
working on updating profile
thorrester Sep 26, 2024
f71320f
working on updating profile
thorrester Sep 26, 2024
344286a
working on updating profile
thorrester Sep 26, 2024
c8cd74f
working on updating profile
thorrester Sep 26, 2024
94fa1a6
working on updating profile
thorrester Sep 26, 2024
a0c11e8
adding reactive statements for hiding
thorrester Sep 26, 2024
3d10763
adding reactive statements for hiding
thorrester Sep 26, 2024
227ed79
dispatch
thorrester Sep 26, 2024
7219b96
update opsml alert routes
thorrester Sep 26, 2024
153bc74
add update logic
thorrester Sep 26, 2024
920fabd
misc error
thorrester Sep 26, 2024
bcbd2d9
add model monitoring page test
thorrester Sep 26, 2024
4736748
add model monitoring page test
thorrester Sep 26, 2024
8320430
update homepage bg colors
thorrester Sep 26, 2024
422e5b6
update homepage bg colors
thorrester Sep 26, 2024
14b59df
default for no alerts
thorrester Sep 26, 2024
8df8557
add alert metrics caller
thorrester Sep 26, 2024
7777e54
add opsml component
thorrester Sep 27, 2024
f7c9b13
viz logic
thorrester Sep 27, 2024
38b03c1
viz logic
thorrester Sep 27, 2024
5fea9bb
graphs
thorrester Sep 27, 2024
f84302b
redoing layout
thorrester Sep 27, 2024
b8fe141
redoing layout
thorrester Sep 27, 2024
beb5411
separate divs
thorrester Sep 27, 2024
9daa0a0
separate divs
thorrester Sep 27, 2024
38e62a9
auto width
thorrester Sep 27, 2024
1a010af
format alert chart
thorrester Sep 27, 2024
0f1e6b2
modular for different types
thorrester Sep 27, 2024
38477bb
modular for different types
thorrester Sep 27, 2024
083ce31
update sizing
thorrester Sep 27, 2024
2756831
bar chart for alert history
thorrester Sep 27, 2024
7ce178c
formatting
thorrester Sep 27, 2024
695030f
check targets
thorrester Sep 27, 2024
aafbdb4
unused
thorrester Sep 27, 2024
fd72500
update routes
thorrester Sep 27, 2024
c87b4b7
formatting
thorrester Sep 30, 2024
c666ac9
8890
thorrester Sep 30, 2024
dbf9033
test types
thorrester Oct 7, 2024
7b3b9e7
test types
thorrester Oct 7, 2024
562df48
update
thorrester Oct 7, 2024
505e1c6
format
thorrester Oct 7, 2024
e48b336
lints
thorrester Oct 7, 2024
7babd5c
format
thorrester Oct 7, 2024
33dedc3
update metadata
thorrester Oct 7, 2024
9da7946
formatting
thorrester Oct 7, 2024
ba124da
laoding
thorrester Oct 7, 2024
d1204a1
updating types
thorrester Oct 9, 2024
23aca87
update insert drift profile
thorrester Oct 9, 2024
bd07ee2
update typing
thorrester Oct 9, 2024
52c2976
Create scouter.py
thorrester Oct 9, 2024
f96916b
add client and server functionality
thorrester Oct 9, 2024
7be3b79
update client and server
thorrester Oct 9, 2024
7d48647
imports
thorrester Oct 9, 2024
64b6291
formatting
thorrester Oct 9, 2024
072c9dc
add files
thorrester Oct 9, 2024
f60b523
fix scouter typing
thorrester Oct 9, 2024
1b0682f
update testing
thorrester Oct 9, 2024
eed8f6b
update testing
thorrester Oct 9, 2024
e5b4650
update server=side test
thorrester Oct 9, 2024
e4939e5
add update test
thorrester Oct 9, 2024
f17a1a1
consistency with scouter server
thorrester Oct 9, 2024
95e2a8e
update ui for monitoring
thorrester Oct 10, 2024
1d49d0f
update ui for monitoring
thorrester Oct 10, 2024
0f59eb4
move alerts around
thorrester Oct 11, 2024
ffe430e
some parts are working
thorrester Oct 11, 2024
bc9e0b8
some parts are working
thorrester Oct 11, 2024
bc29545
working
thorrester Oct 11, 2024
b3a52c7
observability route
thorrester Oct 11, 2024
221a079
typing
thorrester Oct 11, 2024
9b83d47
update tests
thorrester Oct 11, 2024
46259d5
add alert metric test
thorrester Oct 11, 2024
d4373e7
add alert metric test
thorrester Oct 11, 2024
9492f11
add observability metric route
thorrester Oct 11, 2024
4bbf905
update metrics
thorrester Oct 11, 2024
9216f07
update routes
thorrester Oct 11, 2024
9ab39fe
add observability visualizations
thorrester Oct 11, 2024
05bfcaf
add observability visualizations
thorrester Oct 11, 2024
b60510b
adding accordian for observability
thorrester Oct 11, 2024
c3f9ca1
adding requests viz
thorrester Oct 12, 2024
6340736
observability viz
thorrester Oct 14, 2024
fb8fb23
move things around to minimize loading
thorrester Oct 14, 2024
f354bf3
move things around to minimize loading
thorrester Oct 14, 2024
8b3388e
remove features
thorrester Oct 15, 2024
1380566
add arg for data profile
thorrester Oct 15, 2024
e973223
dynamically show profile tab
thorrester Oct 15, 2024
3f83f30
update pathing
thorrester Oct 15, 2024
b4a1119
update pathing
thorrester Oct 16, 2024
78209f2
add data label annotation
thorrester Oct 16, 2024
b26364c
clean up deps and use pnpm
thorrester Oct 16, 2024
665e303
formatting
thorrester Oct 16, 2024
3b311ea
formatting
thorrester Oct 16, 2024
32f30fa
numeric view
thorrester Oct 16, 2024
8c2afc7
numeric view
thorrester Oct 16, 2024
c54c74c
numeric view
thorrester Oct 16, 2024
d2c5c79
simplify
thorrester Oct 16, 2024
3c08b8e
update viz
thorrester Oct 16, 2024
1dac610
update viz
thorrester Oct 16, 2024
e322143
update scouter for computing correlations
thorrester Oct 17, 2024
45f6596
data profile correlation viz
thorrester Oct 17, 2024
64cdbf2
dynamic tabs
thorrester Oct 17, 2024
c2a9388
testing service
thorrester Oct 18, 2024
3a1073d
move service to another working branch
thorrester Oct 18, 2024
f8c9187
lints
thorrester Oct 18, 2024
267ff4f
tests
thorrester Oct 18, 2024
1cf4f9a
formatting
thorrester Oct 18, 2024
03abffb
Update +data_profile.test.ts
thorrester Oct 18, 2024
960a02f
add tests for scouter
thorrester Oct 18, 2024
34f8efd
formatting and tests
thorrester Oct 18, 2024
3f9a91c
Change for pnpm
thorrester Oct 18, 2024
4382ee8
add pnpm setup
thorrester Oct 18, 2024
283a48f
undeeded
thorrester Oct 18, 2024
0202671
naming
thorrester Oct 18, 2024
2a9c5a0
Update SPCStats.svelte
thorrester Oct 18, 2024
2925ab3
test
thorrester Oct 18, 2024
fd57067
change name
thorrester Oct 18, 2024
7b2162d
naming
thorrester Oct 18, 2024
7453e95
naming
thorrester Oct 18, 2024
96c521b
Update SpcMonitoringUI.svelte
thorrester Oct 18, 2024
af409d8
Update +layout.svelte
thorrester Oct 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 33 additions & 15 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,24 @@ jobs:
- name: Git clone the repository
uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
pnpm install
pnpm run build

- name: configure aws credentials
uses: aws-actions/configure-aws-credentials@v3
Expand Down Expand Up @@ -81,18 +87,24 @@ jobs:
with:
fetch-depth: 0

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
pnpm install
pnpm run build

- name: "Az CLI login"
uses: azure/login@v2
Expand Down Expand Up @@ -131,18 +143,24 @@ jobs:
with:
fetch-depth: 0

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
pnpm install
pnpm run build

# identity token doesn't work with python storage client and pre-signed urls
#- name: "Authenticate with Google Cloud"
Expand Down
90 changes: 60 additions & 30 deletions .github/workflows/lint-unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,19 +22,25 @@ jobs:
with:
fetch-depth: 0

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
npm run lints
pnpm install
pnpm run build
pnpm run test

- name: Set up uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
Expand Down Expand Up @@ -62,19 +68,25 @@ jobs:
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
npm run test
pnpm install
pnpm run build
pnpm run test

- name: Set up uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
Expand Down Expand Up @@ -109,19 +121,25 @@ jobs:
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
npm run test
pnpm install
pnpm run build
pnpm run test

- name: Set up uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
Expand Down Expand Up @@ -169,19 +187,25 @@ jobs:
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
npm run test
pnpm install
pnpm run build
pnpm run test

- name: Set up uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
Expand Down Expand Up @@ -217,19 +241,25 @@ jobs:
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
name: Install pnpm
with:
version: 9
run_install: false

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: "npm"
cache-dependency-path: "opsml/app/static/package-lock.json"
cache: "pnpm"
cache-dependency-path: "opsml/app/static/pnpm-lock.yaml"

- name: Build App for testing
run: |
cd opsml/app/static
npm install
npm run build --if-present
npm run test
pnpm install
pnpm run build
pnpm run test

- name: Set up uv
run: irm https://astral.sh/uv/install.ps1 | iex
Expand Down
72 changes: 36 additions & 36 deletions .github/workflows/release-candidate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,43 +7,43 @@ on:

jobs:
Unit-Lint-Release-Candidate:
environment: pypi
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11",]
poetry-version: ["1.7.1"]
environment: pypi
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11"]
poetry-version: ["1.7.1"]

steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry==${{ matrix.poetry-version }}
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry==${{ matrix.poetry-version }}

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"

- run : |
sudo swapoff -a
sudo rm -f /swapfile
sudo apt clean
make setup.project
pip install -U twine
make lints.ci
make test.unit
make setup.project
if [[ "v$(poetry version --short)" == *"rc"* ]];
then
poetry build
twine check --strict dist/*
else
exit 1
fi
- run: |
sudo swapoff -a
sudo rm -f /swapfile
sudo apt clean
make setup.project
pip install -U twine
make lints.ci
make test.unit
make setup.project
if [[ "v$(poetry version --short)" == *"rc"* ]];
then
poetry build
twine check --strict dist/*
else
exit 1
fi

- name: upload to pypi
run: twine upload dist/*
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
- name: upload to pypi
run: twine upload dist/*
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
Loading
Loading