Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance ci env #9

Closed
wants to merge 32 commits into from
Closed

Enhance ci env #9

wants to merge 32 commits into from

Conversation

sivanantha321
Copy link
Owner

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Type of changes
Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Feature/Issue validation/testing:

Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A

  • Test B

  • Logs

Special notes for your reviewer:

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

  • Have you added unit/e2e tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

Release note:

@sivanantha321 sivanantha321 force-pushed the enhance-ci-env branch 25 times, most recently from 1923acf to ce387d0 Compare February 13, 2024 05:57
@sivanantha321 sivanantha321 force-pushed the enhance-ci-env branch 2 times, most recently from d3b4fcd to f060d7c Compare February 14, 2024 11:09
sivanantha321 and others added 2 commits February 14, 2024 13:39
@sivanantha321
Fix parameters value type conversion by pydantic (kserve#3430)
4237c1b 
* Fix parameters value type conversion

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Add test

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Fix datatype check in infer type

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Fix datatype check in infer type

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

---------

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@israel-hdez
Fix Raw Logger E2E (kserve#3434)
d3ec987 
This is a small fix to the test_raw_logger.py/test_kserve_logger E2E test.

This E2E test was creating a `message-dumper-raw` that was dependant on a Serverless setup, leading to the E2E to fail on an environment where Istio nor KNative are available.

The fix is to add the right annotation to `message-dumper-raw` to deploy it in Raw mode, and adjust the logger URL of the `isvc-logger-raw` to point to the Kubernetes Service, rather than the KNative service.

Signed-off-by: Edgar Hernández <[email protected]>
israel-hdez and others added 28 commits February 17, 2024 18:01
@israel-hdez
Reduce E2Es dependency on CI environment (kserve#3435)
9a0a94f 
Some code of the E2Es assume the environment is GitHub, because it is referring to GitHub-specific variables.

This PR focuses on references to the `kserve/image-transformer` image. This image is built in the CI flow and made available to the runner, so that a pull from an external registry is not needed.

The references to this image are changed to an environment variable that is more agnostic to the runner, in an effort to make E2Es more compatible in other environments. The e2e-test.yml is modified to declare the new variable.

Signed-off-by: Edgar Hernández <[email protected]>
@gavrissh @yuzisun
Implement support for vllm as alternative backend (kserve#3415)
1433e95 
* Initial commit to support vllm as alternative backend

Signed-off-by: Gavrish Prabhu <[email protected]>

* include minor fixes and readme changes

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix poetry lock issues

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix lint issues

Signed-off-by: Gavrish Prabhu <[email protected]>

* use_vllm support True as default

Signed-off-by: Gavrish Prabhu <[email protected]>

* refactor code and fix review comments

Signed-off-by: Gavrish Prabhu <[email protected]>

* build failure - fix tests and install vllm part of dockerfile

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix poetry lock issue

Signed-off-by: Gavrish Prabhu <[email protected]>

* include string constants

Signed-off-by: Gavrish Prabhu <[email protected]>

* linting fix

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix review comments

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix tests

Signed-off-by: Gavrish Prabhu <[email protected]>

* fix review comments

Signed-off-by: Gavrish Prabhu <[email protected]>

* add support in vllm  for locally downloaded models

Signed-off-by: Gavrish Prabhu <[email protected]>

* Update Readme

Signed-off-by: Gavrish Prabhu <[email protected]>

* Update Readme

Signed-off-by: Gavrish Prabhu <[email protected]>

* Update python/huggingfaceserver/README.md

Signed-off-by: Dan Sun <[email protected]>

---------

Signed-off-by: Gavrish Prabhu <[email protected]>
Signed-off-by: Dan Sun <[email protected]>
Co-authored-by: Dan Sun <[email protected]>
@sivanantha321
Expose qpext aggregate metrics port on container (kserve#3291)
00cbf60 
Expose qpext aggregate metrics port

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Python vulnerability fixes (kserve#3441)
6253e58 
* Bump paddlepaddle to 2.6.0

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump transformers to version 4.37.2

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump cryptography to version 42.0.2

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump fastapi to version 0.109.2

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump pillow to version 10.2.0

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump aiohttp to version 3.9.3

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Revert fastapi bump

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Bump ray serve to 2.9.2

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Pin alibi to >=0.9.4

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

* Fix alibi version

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>

---------

Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@yuzisun
Fix dup metrics aggr port (kserve#3447)
64cdc6b 
Signed-off-by: Dan Sun <[email protected]>
@terrytangyuan
feat: Allow to disable ingress creation for raw deployment mode (kser…
40385dc 
…ve#3436)

* feat: Allow to disable ingress creation for raw deployment mode

Signed-off-by: Yuan Tang <[email protected]>

* test

Signed-off-by: Yuan Tang <[email protected]>

* Add configs

Signed-off-by: Yuan Tang <[email protected]>

* codegen

Signed-off-by: Yuan Tang <[email protected]>

* python docs

Signed-off-by: Yuan Tang <[email protected]>

* update helm docs

Signed-off-by: Yuan Tang <[email protected]>

* undo changes in pkg/apis/serving/v1beta1/zz_generated.deepcopy.go

Signed-off-by: Yuan Tang <[email protected]>

* remove unnecessary changes

Signed-off-by: Yuan Tang <[email protected]>

* Empty-Commit

Signed-off-by: Yuan Tang <[email protected]>

* remove hf change

Signed-off-by: Yuan Tang <[email protected]>

* Update config/configmap

Signed-off-by: Yuan Tang <[email protected]>

* fix chart syntax

Signed-off-by: Yuan Tang <[email protected]>

* Empty-Commit

Signed-off-by: Yuan Tang <[email protected]>

---------

Signed-off-by: Yuan Tang <[email protected]>
@terrytangyuan
build: Skip already downloaded Istio in quick_install.sh (kserve#3364)
0fc353e 
* build: Skip already downloaded Istio in quick_install.sh

Signed-off-by: Yuan Tang <[email protected]>

* Update quick_install.sh

Signed-off-by: Yuan Tang <[email protected]>

---------

Signed-off-by: Yuan Tang <[email protected]>
@terrytangyuan
fix: HuggingFace predictor should not be recognized as multi-model se…
5116ce4 
…rver (kserve#3449)

* fix: HuggingFace predictor should not be recognized as multi-model server

Signed-off-by: Yuan Tang <[email protected]>

* add

Signed-off-by: Yuan Tang <[email protected]>

---------

Signed-off-by: Yuan Tang <[email protected]>
@yuzisun
Fix: bugs for huggingface runtime template (kserve#3448)
5046f3a 
* Do not inject model agent for huggingface runtime

Signed-off-by: Dan Sun <[email protected]>

* Add option to return token type ids

Signed-off-by: Dan Sun <[email protected]>

* Add option to return token type ids

Signed-off-by: Dan Sun <[email protected]>

* Default model dir

Signed-off-by: Dan Sun <[email protected]>

---------

Signed-off-by: Dan Sun <[email protected]>
@kevinmingtarja
Fix: Add padding and truncation in huggingface tokenizer (kserve#3450)
2368ba9 
Add padding and truncation in huggingface tokenizer

Signed-off-by: Kevin Mingtarja <[email protected]>
Signed-off-by: Kevin Mingtarja <[email protected]>
@yuzisun
Fix: vllm backend does not work with model_dir for huggingface runtime (
610a919 
kserve#3456)

* Fix model dir does not work with vllm

Signed-off-by: Dan Sun <[email protected]>

* Fix extra comma

Signed-off-by: Dan Sun <[email protected]>

---------

Signed-off-by: Dan Sun <[email protected]>
@sivanantha321
Increase workers to 4
53c65b1 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
merge fast and slow
01fb072 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Increase workers to 6 and merge explainer, transformer test
c9890f7 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Separate graph tests
fdec677 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Change artifact upload/download action
99bcb1f 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Use docker-container driver
f20bab9 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Update all e2e tests
c22ace9 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Remove knative installation in raw tests
13ab466 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Fix raw logger test
e2600b2 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Wait for model to be ready in mlserver e2e tests
93bba37 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Cleanup markers
f9e702d 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Clean up workflow and upgrade actions
a3b09df 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Use worksteal dist type for pytest
ce2b1ef 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
fix typo
fae0781 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Prettify pod log printing in status check
970f8c8 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Add inference graph pod logs for debugging in status check
ab6fbc6 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
@sivanantha321
Reorder Go docker files for better layer caching
5d83a10 
Signed-off-by: Sivanantham Chinnaiyan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@sivanantha321 @israel-hdez @gavrissh @yuzisun @terrytangyuan @kevinmingtarja