statistics: handle deleted tables correctly in the PQ #57649

Rustin170506 · 2024-11-23T08:38:16Z

What problem does this PR solve?

Issue Number: close #57648

Problem Summary:

What changed and how does it work?

Correctly skip the deleted table. Sorry for overlooking this.
Make sure we don't write to nil runningJobs and mustRetryJobs because the worker might access them again after the queue is closed (we don't stop the worker at the same time because we don't want to block the etcd election process). Accessing it is ok because we will reinitialize the queue next time. So we will take care of the failed jobs next time.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: Rustin170506 <[email protected]>

Rustin170506

🔢 Self-check (PR reviewed by myself and ready for feedback.)

Rustin170506 · 2024-11-23T08:52:08Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -57,7 +57,7 @@ func (f *AnalysisJobFactory) CreateNonPartitionedTableAnalysisJob(
 	tblInfo *model.TableInfo,
 	tblStats *statistics.Table,
 ) AnalysisJob {
-	if !tblStats.IsEligibleForAnalysis() {
+	if tblStats == nil || !tblStats.IsEligibleForAnalysis() {


We didn't encounter any issues here, just double-checking.

Rustin170506 · 2024-11-23T08:52:13Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -92,7 +92,7 @@ func (f *AnalysisJobFactory) CreateStaticPartitionAnalysisJob(
 	partitionID int64,
 	partitionStats *statistics.Table,
 ) AnalysisJob {
-	if !partitionStats.IsEligibleForAnalysis() {
+	if partitionStats == nil || !partitionStats.IsEligibleForAnalysis() {


Rustin170506 · 2024-11-23T08:52:18Z

pkg/statistics/handle/autoanalyze/priorityqueue/analysis_job_factory.go

@@ -128,7 +128,7 @@ func (f *AnalysisJobFactory) CreateDynamicPartitionedTableAnalysisJob(
 	globalTblStats *statistics.Table,
 	partitionStats map[PartitionIDAndName]*statistics.Table,
 ) AnalysisJob {
-	if !globalTblStats.IsEligibleForAnalysis() {
+	if globalTblStats == nil || !globalTblStats.IsEligibleForAnalysis() {


pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

Rustin170506 · 2024-11-23T08:53:13Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

 				err := pq.syncFields.inner.delete(job)
 				if err != nil {
 					statslogutil.StatsLogger().Error("Failed to delete job from priority queue",
 						zap.Error(err),
 						zap.String("job", job.String()),
 					)
 				}
+				continue


Fix the [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"]

Rustin170506 · 2024-11-23T08:53:24Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue_test.go

@@ -608,3 +609,47 @@ func TestPQCanBeClosedAndReInitialized(t *testing.T) {
 	// Check if the priority queue is initialized.
 	require.True(t, pq.IsInitialized())
 }
+
+func TestPQHandlesTableDeletionGracefully(t *testing.T) {


Test case for: [category=stats] [recover="runtime error: invalid memory address or nil pointer dereference"]

Signed-off-by: Rustin170506 <[email protected]>

codecov · 2024-11-23T09:07:49Z

Codecov Report

Attention: Patch coverage is 66.66667% with 7 lines in your changes missing coverage. Please review.

Project coverage is 73.7750%. Comparing base (6e22b8c) to head (fda8e18).
Report is 6 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #57649        +/-   ##
================================================
+ Coverage   72.7907%   73.7750%   +0.9843%     
================================================
  Files          1676       1707        +31     
  Lines        463750     472989      +9239     
================================================
+ Hits         337567     348948     +11381     
+ Misses       105323     102632      -2691     
- Partials      20860      21409       +549

Flag	Coverage Δ
integration	`46.2980% <9.5238%> (?)`
unit	`72.3658% <66.6666%> (+0.1772%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.7673% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`45.6200% <ø> (+0.1882%)`	⬆️

---- 🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles

Rustin170506

🔢 Self-check (PR reviewed by myself and ready for feedback.)

Rustin170506 · 2024-11-23T09:08:10Z

pkg/statistics/handle/autoanalyze/priorityqueue/queue.go

+		// We allow running jobs to complete normally rather than stopping them, so this nil check
+		// is expected when jobs finish after the switch. Failed jobs will be handled by the next
+		// initialization, so we can safely ignore them here.
+		if pq.syncFields.runningJobs == nil || pq.syncFields.mustRetryJobs == nil {


For the ["Auto analyze job execution failed"] [category=stats] [recover="assignment to entry in nil map"].

Rustin170506 · 2024-11-23T09:21:03Z

/retest

elsa0520

LGTM

ti-chi-bot · 2024-11-25T07:28:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: elsa0520, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/statistics/OWNERS~~ [elsa0520,qw4990]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-11-25T07:28:42Z

[LGTM Timeline notifier]

Timeline:

2024-11-25 06:12:03.012541066 +0000 UTC m=+444110.632195595: ☑️ agreed by qw4990.
2024-11-25 07:28:41.441354769 +0000 UTC m=+448709.061009284: ☑️ agreed by elsa0520.

Rustin170506 · 2024-11-25T08:14:17Z

/retest

ti-chi-bot · 2024-11-25T08:36:26Z

In response to a cherrypick label: new pull request created to branch release-8.5: #57674.

close #57648

statistics: handle deleted tables correctly in the PQ

009fbf2

Signed-off-by: Rustin170506 <[email protected]>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 23, 2024

Rustin170506 added 2 commits November 23, 2024 16:46

fix: do not block etcd

1a7bc2f

Signed-off-by: Rustin170506 <[email protected]>

refactor: better code

51eef0d

Signed-off-by: Rustin170506 <[email protected]>

Rustin170506 commented Nov 23, 2024

View reviewed changes

Rustin170506 added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Nov 23, 2024

refactor: better way to fix the issue

fda8e18

Signed-off-by: Rustin170506 <[email protected]>

Rustin170506 commented Nov 23, 2024

View reviewed changes

Rustin170506 requested review from qw4990 and elsa0520 November 25, 2024 02:13

qw4990 approved these changes Nov 25, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 25, 2024

elsa0520 approved these changes Nov 25, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 25, 2024

ti-chi-bot bot merged commit 702c4f2 into pingcap:master Nov 25, 2024
25 checks passed

ti-chi-bot mentioned this pull request Nov 25, 2024

statistics: handle deleted tables correctly in the PQ (#57649) #57674

Merged

13 tasks

ti-chi-bot bot pushed a commit that referenced this pull request Nov 25, 2024

statistics: handle deleted tables correctly in the PQ (#57649) (#57674)

ed39273

close #57648

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

statistics: handle deleted tables correctly in the PQ #57649

statistics: handle deleted tables correctly in the PQ #57649

Rustin170506 commented Nov 23, 2024 •

edited

Loading

Rustin170506 left a comment

Rustin170506 Nov 23, 2024

Rustin170506 Nov 23, 2024

Rustin170506 Nov 23, 2024

Rustin170506 Nov 23, 2024

Rustin170506 Nov 23, 2024

codecov bot commented Nov 23, 2024 •

edited

Loading

Rustin170506 left a comment

Rustin170506 Nov 23, 2024

Rustin170506 commented Nov 23, 2024

elsa0520 left a comment

ti-chi-bot bot commented Nov 25, 2024

ti-chi-bot bot commented Nov 25, 2024

Rustin170506 commented Nov 25, 2024

ti-chi-bot commented Nov 25, 2024

statistics: handle deleted tables correctly in the PQ #57649

statistics: handle deleted tables correctly in the PQ #57649

Conversation

Rustin170506 commented Nov 23, 2024 • edited Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

Rustin170506 left a comment

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

codecov bot commented Nov 23, 2024 • edited Loading

Codecov Report

Rustin170506 left a comment

Choose a reason for hiding this comment

Rustin170506 Nov 23, 2024

Choose a reason for hiding this comment

Rustin170506 commented Nov 23, 2024

elsa0520 left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Nov 25, 2024

ti-chi-bot bot commented Nov 25, 2024

[LGTM Timeline notifier]

Rustin170506 commented Nov 25, 2024

ti-chi-bot commented Nov 25, 2024

Rustin170506 commented Nov 23, 2024 •

edited

Loading

codecov bot commented Nov 23, 2024 •

edited

Loading