Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add tls support for apiserver http/grpc #40

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

HumairAK
Copy link

@HumairAK HumairAK commented May 27, 2024

@openshift-ci openshift-ci bot requested review from amadhusu and VaniHaripriya May 27, 2024 21:36
@dsp-developers
Copy link

A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

@dsp-developers
Copy link

An OCP cluster where you are logged in as cluster admin is required.

The Data Science Pipelines team recommends testing this using the Data Science Pipelines Operator. Check here for more information on using the DSPO.

To use and deploy a DSP stack with these images (assuming the DSPO is deployed), first save the following YAML to a file named dspa.pr-40.yaml:

apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1
kind: DataSciencePipelinesApplication
metadata:
  name: pr-40
spec:
  dspVersion: v2
  apiServer:
    image: "quay.io/opendatahub/ds-pipelines-api-server:pr-40"
    argoDriverImage: "quay.io/opendatahub/ds-pipelines-driver:pr-40"
    argoLauncherImage: "quay.io/opendatahub/ds-pipelines-launcher:pr-40"
  persistenceAgent:
    image: "quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40"
  scheduledWorkflow:
    image: "quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40"
  mlmd:  
    deploy: true  # Optional component
    grpc:
      image: "quay.io/opendatahub/mlmd-grpc-server:latest"
    envoy:
      image: "registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2"
  mlpipelineUI:
    deploy: true  # Optional component 
    image: "quay.io/opendatahub/ds-pipelines-frontend:pr-40"
  objectStorage:
    minio:
      deploy: true
      image: 'quay.io/opendatahub/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance'

Then run the following:

cd $(mktemp -d)
git clone [email protected]:opendatahub-io/data-science-pipelines.git
cd data-science-pipelines/
git fetch origin pull/40/head
git checkout -b pullrequest 03d741462224bad67faa17c71259fc4f9d5b025d
oc apply -f dspa.pr-40.yaml

More instructions here on how to deploy and test a Data Science Pipelines Application.

@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 1 commits between dadfb38393ce96cb7dc1eac60428d66f0e3f1933...957880ae24860fd8e230a2af00eb3d7a1258011f

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

backend/src/common/util/service.go Show resolved Hide resolved
backend/src/common/util/service.go Outdated Show resolved Hide resolved
frontend/server/utils.ts Show resolved Hide resolved
backend/src/apiserver/main.go Outdated Show resolved Hide resolved
backend/src/apiserver/main.go Outdated Show resolved Hide resolved
@HumairAK HumairAK changed the title UPSTREAM: <carry>: add tls support for apiserver http/grpc feat: add tls support for apiserver http/grpc Jul 12, 2024
@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 0 commits between f5ab037f8a6efd179b7a47cf623c997f43e05907...671048688e3df283e5e2038b825f8da7bc5884f0

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

@DharmitD
Copy link
Member

would additionally suggest adding a unit test here to verify this functionality.

@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 0 commits between f5ab037f8a6efd179b7a47cf623c997f43e05907...cd5eb38ec9f78706302d4d6c7cebee808bddb1a8

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

@HumairAK
Copy link
Author

would additionally suggest adding a unit test here to verify this functionality.

testing tls connections via unit testing is going to be non trivial, I think this is best reserved for integration testing

@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 0 commits between f5ab037f8a6efd179b7a47cf623c997f43e05907...5aee6197f373daa767a3c02372c2628364643e18

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

@HumairAK HumairAK requested review from gregsheremeta and DharmitD and removed request for amadhusu July 14, 2024 20:52
@gregsheremeta
Copy link

/lgtm

just for my own learning, why'd you flip the flag from bool to string?

@HumairAK
Copy link
Author

HumairAK commented Jul 15, 2024

just for my own learning, why'd you flip the flag from bool to string?

The reason is because flag.bool requires you to be very precise in how you declare the boolean it has to be like --my_flag=true, if you do something like --my_flag false that would be interpreted as true. This felt very error prone to me, I'd rather just parse the string and throw an error if conversion fails. [1]

at some point we should just do away with golang default flags and use viper

@gregsheremeta
Copy link

I can tag this with approve once @VaniHaripriya verifies opendatahub-io/data-science-pipelines-operator#656

@VaniHaripriya
Copy link

/verified

@VaniHaripriya VaniHaripriya added the verified For deploying/executing changes and verifyign results. label Jul 17, 2024
@gregsheremeta
Copy link

/lgtm
/approve

Copy link

openshift-ci bot commented Jul 17, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gregsheremeta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gregsheremeta
Copy link

oops just noticed the conflict

make mlpipeline server url scheme configurable
add tls handling for PA and ui
remove local grpc client tls.

Signed-off-by: Humair Khan <[email protected]>
@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 1 commits between 9faafad2fe36119ce5aa8ad9456a4b4e6e71dbad...74886eef57095a0e0d21932628bd6eb76a0e359e

@gregsheremeta
Copy link

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Jul 17, 2024
@HumairAK HumairAK merged commit 7d7659c into opendatahub-io:master Jul 17, 2024
2 of 3 checks passed
@dsp-developers
Copy link

Commit Checker results:

**NOTE**: These are the results of the commit checker scans. 
If these are not commits from upstream kfp, then please ensure
you adhere to the commit checker formatting
commitchecker verson unknown
Validating 0 commits between 7d7659c9fb731b1610570e2dd3d056d025e1600e...74886eef57095a0e0d21932628bd6eb76a0e359e

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-40
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-40
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-40
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-40
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-40
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-40

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm verified For deploying/executing changes and verifyign results.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants