Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Remove the need for external tools for managing ElasticSearch #6291

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Rohanraj123
Copy link
Contributor

Which problem is this PR solving?

Description of the changes

  • I just added index-cleaner tool into jaeger binary. In furthur commits I will add other two as well. Also Documentation and tests will be added in furthur commits.

How was this change tested?

  • Pending

Checklist

Copy link

codecov bot commented Dec 2, 2024

Codecov Report

Attention: Patch coverage is 3.04878% with 159 lines in your changes missing coverage. Please review.

Project coverage is 95.44%. Comparing base (39e3bfb) to head (2199d38).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
pkg/es/config/config.go 3.33% 145 Missing ⚠️
cmd/jaeger/internal/index_cleaner.go 0.00% 14 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6291      +/-   ##
==========================================
- Coverage   96.20%   95.44%   -0.77%     
==========================================
  Files         356      357       +1     
  Lines       20416    20596     +180     
==========================================
+ Hits        19642    19658      +16     
- Misses        585      748     +163     
- Partials      189      190       +1     
Flag Coverage Δ
badger_v1 8.77% <0.00%> (-0.13%) ⬇️
badger_v2 1.60% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v1-manual 14.63% <0.00%> (-0.22%) ⬇️
cassandra-4.x-v2-auto 1.54% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v2-manual 1.54% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v1-manual 14.63% <0.00%> (-0.22%) ⬇️
cassandra-5.x-v2-auto 1.54% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v2-manual 1.54% <0.00%> (-0.04%) ⬇️
elasticsearch-6.x-v1 18.34% <0.00%> (-0.27%) ⬇️
elasticsearch-7.x-v1 18.42% <0.00%> (-0.26%) ⬇️
elasticsearch-8.x-v1 18.57% <0.00%> (-0.28%) ⬇️
elasticsearch-8.x-v2 ?
grpc_v1 10.22% <0.00%> (-0.15%) ⬇️
grpc_v2 7.75% <0.00%> (-0.15%) ⬇️
kafka-v1 8.46% <0.00%> (-0.13%) ⬇️
kafka-v2 1.60% <0.00%> (-0.04%) ⬇️
memory_v2 1.60% <0.00%> (-0.04%) ⬇️
opensearch-1.x-v1 18.45% <0.00%> (-0.28%) ⬇️
opensearch-2.x-v1 18.46% <0.00%> (-0.27%) ⬇️
opensearch-2.x-v2 ?
tailsampling-processor 0.44% <0.00%> (-0.01%) ⬇️
unittests 94.36% <3.04%> (-0.76%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


// Validation function for Cleaner struct
func (c *Cleaner) Validate() error {
if c.Frequency <= 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validation can be declared in the field tags

type Cleaner struct {
Enabled bool `mapstructure:"enabled"`
Frequency time.Duration `mapstructure:"frequency"`
IncludeArchive bool `mapstructure:"include_archive"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in v2 archive storage is a separate instance, this property won't make sense


// ---- ES-specific config ----
// Cleaner holds the configuration for the Elasticsearch index Cleaner.
Cleaner Cleaner `mapstructure:"cleaner"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Cleaner Cleaner `mapstructure:"cleaner"`
IndexCleaner IndexCleaner `mapstructure:"index_cleaner"`

@@ -585,3 +608,198 @@ func (c *Configuration) Validate() error {
_, err := govalidator.ValidateStruct(c)
return err
}

// Run starts the cleaner that runs periodically based on the Frequency field.
func (cfg *Configuration) RunCleaner() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config package is for configuration, not for business logic. This should be somewhere in plugin/storage/es

}

// deleteOldIndices deletes indices older than the configured MaxSpanAge.
func (cfg *Configuration) deleteOldIndices() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't this logic already exist in in cmd/es-index-cleaner? Why do we need to duplicate it?

@@ -24,6 +24,8 @@ func main() {
command,
)

internal.StartCleaner(v)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's enabled by config option, no need to "start" anything expicitly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants