OCPBUGS-41892: Single Watch on GWAPI CRD #1165

Thealisyed · 2024-11-04T13:47:38Z

A single CRD watch was added with a predicate to filter CRDs based on the gatewayapiv1.GroupName. This ensures that only relevant CRDs are watched, reducing unnecessary logging when the Gateway API feature is not enabled.

openshift-ci-robot · 2024-11-04T13:47:44Z

@Thealisyed: This pull request references Jira Issue OCPBUGS-41892, which is invalid:

expected the bug to target the "4.18.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

A single CRD watch was added with a predicate to filter CRDs based on the gatewayapiv1.GroupName. This ensures that only relevant CRDs are watched, reducing unnecessary logging when the Gateway API feature is not enabled.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Thealisyed · 2024-11-04T13:48:38Z

/jira refresh

openshift-ci-robot · 2024-11-04T13:48:45Z

@Thealisyed: This pull request references Jira Issue OCPBUGS-41892, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.18.0) matches configured target version for branch (4.18.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @lihongan

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Thealisyed · 2024-11-04T21:07:47Z

/test e2e-aws-ovn-single-node

Thealisyed · 2024-11-05T09:22:47Z

/retest-required

pkg/operator/controller/gatewayapi/controller.go

Miciah

(Whoops, I've had this review pending, waiting for me to submit it, for 5 days...)

Is it true that the existing watch only generates reconciliation requests when the Gateway API feature is not enabled?

In any case, please add a little more detail in the commit message. For example:

gatewayapi: Use single watch for CRDs

Replace the multiple watches for Gateway API CRDs with a single CRD watch, with
a predicate to filter CRDs based on the gatewayapiv1.GroupName.  This ensures
that only relevant CRDs are watched, reducing unnecessary reconciliation
requests and log messages.

This commit fixes OCPBUGS-41892.

https://issues.redhat.com/browse/OCPBUGS-41892

pkg/operator/controller/gatewayapi/controller.go

candita · 2024-11-13T17:02:36Z

/assign
/assign @Miciah

pkg/operator/controller/gatewayapi/controller.go

candita · 2024-11-13T19:20:01Z

@Thealisyed please investigate the failures in the CI tests, especially e2e-aws-gatewayapi, which is the GWAPI test. I ran a retest to see if the issue with imagestreams had gone away.

candita · 2024-11-13T19:20:52Z

/retest

pkg/operator/controller/gatewayapi/controller.go

candita · 2024-11-25T17:30:40Z

@Thealisyed please note you will need to squash your three commits before merge.

pkg/operator/controller/gatewayapi/controller.go

candita · 2024-11-25T21:56:35Z

Process did not finish before 4h0m0s timeout

/test e2e-aws-ovn-serial

Replace the multiple watches for Gateway API CRDs with a single CRD watch, with a predicate to filter CRDs based on the gatewayapiv1.GroupName. This ensures that only relevant CRDs are watched, reducing unnecessary reconciliation requests and log messages. This commit fixes OCPBUGS-41892. https://issues.redhat.com/browse/OCPBUGS-41892

Thealisyed · 2024-11-28T14:09:17Z

/retest-required

Thealisyed · 2024-12-05T09:48:53Z

/retest-required

Thealisyed · 2024-12-05T15:41:39Z

/test e2e-aws-operator

candita · 2024-12-09T18:38:08Z

@Thealisyed - before you test again, do a little analysis on the failures, and write them up like this. Then the next time they fail, if this failure keeps happening, you'll need to figure or get help to figure out why.

Two unit test failures:

TestAll/serial/TestRouterCompressionOperation
router_compression_test.go:245: failed to create service openshift-ingress/hello-pod-bpcnc: Internal error occurred: failed to allocate a serviceIP: rpc error: code = Unavailable desc = error reading from server: read tcp 10.0.89.66:43566->10.0.18.210:2379: read: connection reset by peer

TestAll/serial/TestCanaryWithMTLS
canary_test.go:343: Polling for canary success condition...
canary_test.go:346: Failed to get IngressController: Get "https://api.ci-op-ki1llnl8-9e7c5.origin-ci-int-aws.dev.rhcloud.com:6443/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default": http2: client connection lost
canary_test.go:361: Polling terminated with unexpected error: Get "https://api.ci-op-ki1llnl8-9e7c5.origin-ci-int-aws.dev.rhcloud.com:6443/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default": http2: client connection lost

/test e2e-aws-operator-techpreview

Thealisyed · 2024-12-12T12:00:07Z

"Operator "console-operator\ + cluster-samples-operator" produces more watch requests than expected [for both]
seems unrelated to changes will test again and observe

/retest

candita · 2024-12-19T20:53:08Z

@Thealisyed, in the e2e-aws-gatewayapi test logs, where the featuregate is on, I'm seeing some errors that should be investigated: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-ingress-operator/1165/pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-gatewayapi/1862083335728140288/artifacts/e2e-aws-gatewayapi/gather-extra/artifacts/pods/openshift-ingress-operator_ingress-operator-58d85999bb-l6cpv_ingress-operator.log

Can you check if this happens on a cluster without your changes? If so, please open a new bug that points to the logs. If not, we need to find out what's in this PR that would cause the errors.

Specifically, lots of these:

2024-11-28T11:51:57.292Z ERROR operator.init controller/controller.go:263 Reconciler error {"controller": "gatewayclass_controller", "object": {"name":"openshift-default"}, "namespace": "", "name": "openshift-default", "reconcileID": "e18f9589-f527-4802-87aa-bc6ebb8240fd", "error": "failed to create ServiceMeshControlPlane openshift-ingress/openshift-gateway: Internal error occurred: failed calling webhook "smcp.mutation.maistra.io": failed to call webhook: Post "https://istio-operator-service.openshift-operators.svc:443/mutate-smcp?timeout=10s\": no endpoints available for service "istio-operator-service"", "errorCauses": [{"error": "failed to create ServiceMeshControlPlane openshift-ingress/openshift-gateway: Internal error occurred: failed calling webhook "smcp.mutation.maistra.io": failed to call webhook: Post "https://istio-operator-service.openshift-operators.svc:443/mutate-smcp?timeout=10s": no endpoints available for service "istio-operator-service""}]}

and then eventually it is able to create the servicemeshcontrolplane:

2024-11-28T11:52:17.801Z INFO operator.gatewayclass_controller gatewayclass/servicemeshcontrolplane.go:46 created ServiceMeshControlPlane {"namespace": "openshift-ingress", "name": "openshift-gateway"}

candita · 2024-12-19T21:05:18Z

Hold for a check on logs
/hold
Otherwise
/lgtm

rhamini3 · 2024-12-19T21:44:16Z

Pre merge verified, Gateway logs does not flood the cluster-ingress-operator logs

% oc logs ingress-operator-747dd58b69-xwpsk -n openshift-ingress-operator | grep gateway
Defaulted container "ingress-operator" out of: ingress-operator, kube-rbac-proxy
2024-12-18T17:00:21.604Z	INFO	operator.init	controller/controller.go:231	Starting EventSource	{"controller": "gatewayapi_controller", "source": "kind source: *v1.FeatureGate"}
2024-12-18T17:00:21.605Z	INFO	operator.init	controller/controller.go:231	Starting EventSource	{"controller": "gatewayapi_controller", "source": "kind source: *v1.CustomResourceDefinition"}
2024-12-18T17:00:21.605Z	INFO	operator.init	controller/controller.go:231	Starting Controller	{"controller": "gatewayapi_controller"}
2024-12-18T17:00:22.130Z	INFO	operator.init	controller/controller.go:231	Starting workers	{"controller": "gatewayapi_controller", "worker count": 1}
2024-12-18T17:00:22.130Z	INFO	operator.gatewayapi_controller	controller/controller.go:116	reconciling	{"request": {"name":"cluster"}}

/label qe-approved

openshift-ci-robot · 2024-12-19T21:44:24Z

@Thealisyed: This pull request references Jira Issue OCPBUGS-41892, which is invalid:

expected the bug to target either version "4.19." or "openshift-4.19.", but it targets "4.18.0" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

A single CRD watch was added with a predicate to filter CRDs based on the gatewayapiv1.GroupName. This ensures that only relevant CRDs are watched, reducing unnecessary logging when the Gateway API feature is not enabled.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Miciah · 2024-12-20T21:37:45Z

/jira refresh

openshift-ci-robot · 2024-12-20T21:37:51Z

@Miciah: This pull request references Jira Issue OCPBUGS-41892, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.19.0) matches configured target version for branch (4.19.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2025-01-01T00:00:59Z

/jira refresh

The requirements for Jira bugs have changed (Jira issues linked to PRs on main branch need to target different OCP), recalculating validity.

openshift-ci-robot · 2025-01-01T00:01:30Z

@openshift-bot: This pull request references Jira Issue OCPBUGS-41892, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.19.0) matches configured target version for branch (4.19.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

In response to this:

/jira refresh

The requirements for Jira bugs have changed (Jira issues linked to PRs on main branch need to target different OCP), recalculating validity.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Miciah · 2025-01-02T13:30:52Z

@Thealisyed, in the e2e-aws-gatewayapi test logs, where the featuregate is on, I'm seeing some errors that should be investigated: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-ingress-operator/1165/pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-gatewayapi/1862083335728140288/artifacts/e2e-aws-gatewayapi/gather-extra/artifacts/pods/openshift-ingress-operator_ingress-operator-58d85999bb-l6cpv_ingress-operator.log

Can you check if this happens on a cluster without your changes? If so, please open a new bug that points to the logs. If not, we need to find out what's in this PR that would cause the errors.

Nudge! @Thealisyed, the PR is passing CI and has @candita's /lgtm. You can pick any arbitrary job run for another PR from https://prow.ci.openshift.org/job-history/gs/test-platform-results/pr-logs/directory/pull-ci-openshift-cluster-ingress-operator-master-e2e-aws-gatewayapi and compare its log output with the e2e-aws-gatewayapi log output for this PR.

Thealisyed · 2025-01-02T14:15:34Z

Thanks @Miciah, I can see this log error output is also apparent in other job runs I'll file a bug for this!

Miciah · 2025-01-02T15:45:07Z

/hold cancel
per #1165 (comment).

openshift-ci-robot · 2025-01-06T18:57:27Z

/retest-required

Remaining retests: 0 against base HEAD 4ebbf69 and 2 for PR HEAD f3551d0 in total

openshift-ci · 2025-01-06T21:03:40Z

@Thealisyed: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2025-01-06T21:08:28Z

@Thealisyed: Jira Issue OCPBUGS-41892: All pull requests linked via external trackers have merged:

openshift/cluster-ingress-operator#1165

Jira Issue OCPBUGS-41892 has been moved to the MODIFIED state.

In response to this:

A single CRD watch was added with a predicate to filter CRDs based on the gatewayapiv1.GroupName. This ensures that only relevant CRDs are watched, reducing unnecessary logging when the Gateway API feature is not enabled.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 4, 2024

openshift-ci bot requested review from Miciah and miheer November 4, 2024 13:48

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Nov 4, 2024

openshift-ci bot requested a review from lihongan November 4, 2024 13:48

Thealisyed changed the title ~~[WIP] OCPBUGS-41892: Single Watch on GWAPI CRD~~ OCPBUGS-41892: Single Watch on GWAPI CRD Nov 6, 2024

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 6, 2024

candita reviewed Nov 9, 2024

View reviewed changes

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

Miciah reviewed Nov 12, 2024

View reviewed changes

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

Miciah added the priority/backlog Higher priority than priority/awaiting-more-evidence. label Nov 13, 2024

openshift-ci bot assigned candita and Miciah Nov 13, 2024

candita reviewed Nov 13, 2024

View reviewed changes

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

candita reviewed Nov 13, 2024

View reviewed changes

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

candita reviewed Nov 25, 2024

View reviewed changes

pkg/operator/controller/gatewayapi/controller.go Outdated Show resolved Hide resolved

Thealisyed force-pushed the watcher-fix branch from 328eeeb to 56367cc Compare November 26, 2024 13:19

Thealisyed force-pushed the watcher-fix branch from 56367cc to f3551d0 Compare November 28, 2024 10:36

openshift-ci bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. labels Dec 19, 2024

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Dec 19, 2024

openshift-ci-robot added jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. and removed jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Dec 19, 2024

openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Dec 20, 2024

openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Dec 20, 2024

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 2, 2025

openshift-merge-bot bot merged commit 7732edf into openshift:master Jan 6, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-41892: Single Watch on GWAPI CRD #1165

OCPBUGS-41892: Single Watch on GWAPI CRD #1165

Thealisyed commented Nov 4, 2024

openshift-ci-robot commented Nov 4, 2024

Thealisyed commented Nov 4, 2024

openshift-ci-robot commented Nov 4, 2024

Thealisyed commented Nov 4, 2024

Thealisyed commented Nov 5, 2024

Miciah left a comment •

edited

Loading

candita commented Nov 13, 2024

candita commented Nov 13, 2024 •

edited

Loading

candita commented Nov 13, 2024

candita commented Nov 25, 2024

candita commented Nov 25, 2024

Thealisyed commented Nov 28, 2024

Thealisyed commented Dec 5, 2024

Thealisyed commented Dec 5, 2024

candita commented Dec 9, 2024

Thealisyed commented Dec 12, 2024

candita commented Dec 19, 2024

candita commented Dec 19, 2024

rhamini3 commented Dec 19, 2024 •

edited

Loading

openshift-ci-robot commented Dec 19, 2024

Miciah commented Dec 20, 2024

openshift-ci-robot commented Dec 20, 2024

openshift-bot commented Jan 1, 2025

openshift-ci-robot commented Jan 1, 2025

Miciah commented Jan 2, 2025

Thealisyed commented Jan 2, 2025 •

edited

Loading

Miciah commented Jan 2, 2025

openshift-ci-robot commented Jan 6, 2025

openshift-ci bot commented Jan 6, 2025

openshift-ci-robot commented Jan 6, 2025

OCPBUGS-41892: Single Watch on GWAPI CRD #1165

OCPBUGS-41892: Single Watch on GWAPI CRD #1165

Conversation

Thealisyed commented Nov 4, 2024

openshift-ci-robot commented Nov 4, 2024

Thealisyed commented Nov 4, 2024

openshift-ci-robot commented Nov 4, 2024

Thealisyed commented Nov 4, 2024

Thealisyed commented Nov 5, 2024

Miciah left a comment • edited Loading

Choose a reason for hiding this comment

candita commented Nov 13, 2024

candita commented Nov 13, 2024 • edited Loading

candita commented Nov 13, 2024

candita commented Nov 25, 2024

candita commented Nov 25, 2024

Thealisyed commented Nov 28, 2024

Thealisyed commented Dec 5, 2024

Thealisyed commented Dec 5, 2024

candita commented Dec 9, 2024

Thealisyed commented Dec 12, 2024

candita commented Dec 19, 2024

candita commented Dec 19, 2024

rhamini3 commented Dec 19, 2024 • edited Loading

openshift-ci-robot commented Dec 19, 2024

Miciah commented Dec 20, 2024

openshift-ci-robot commented Dec 20, 2024

openshift-bot commented Jan 1, 2025

openshift-ci-robot commented Jan 1, 2025

Miciah commented Jan 2, 2025

Thealisyed commented Jan 2, 2025 • edited Loading

Miciah commented Jan 2, 2025

openshift-ci-robot commented Jan 6, 2025

openshift-ci bot commented Jan 6, 2025

openshift-ci-robot commented Jan 6, 2025

Miciah left a comment •

edited

Loading

candita commented Nov 13, 2024 •

edited

Loading

rhamini3 commented Dec 19, 2024 •

edited

Loading

Thealisyed commented Jan 2, 2025 •

edited

Loading