[BUG] Incorrect Top N value displayed in the configurations page

[deshsidd]

What is the bug?

Incorrect Top N value displayed in the configurations page in certain situations. Please find how to reproduce the bug in the following section.

How can one reproduce the bug?

1. Start with a clean cluster with no prior settings.

{
    "persistent": {},
    "transient": {}
}

2. Configure Top N for all metric types but only include window_size and top_n_size for latency:

{
    "persistent": {
        "search.insights.top_queries.latency.enabled" : true,
        "search.insights.top_queries.cpu.enabled" : true,
        "search.insights.top_queries.memory.enabled" : true,
        "search.insights.top_queries.group_by" : "none",
        "search.insights.top_queries.latency.window_size" : "1m",
        "search.insights.top_queries.latency.top_n_size" : 5
    }
}

3. Configuration for latency is displayed properly:
   
4. Look at cluster settings for top_n_size for cpu and memory:

_cluster/settings?include_defaults

Relevant part of the default settings showing top_n_size for memory and cpu is 3:

"search": {
            "default_search_timeout": "-1",
            "max_aggregation_rewrite_filters": "3000",
            "max_open_pit_context": "300",
            "insights": {
                "top_queries": {
                    "cpu": {
                        "top_n_size": "3",
                        "window_size": "60s"
                    },
                    "max_groups_excluding_topn": "100",
                    "memory": {
                        "top_n_size": "3",
                        "window_size": "60s"
                    },
                    "grouping": {
                        "attributes": {
                            "field_type": "true",
                            "field_name": "true"
                        }
                    }
                }
            },

5. UI is displaying top N for cpu and memory as 5:
   
   

What is the expected behavior?

Top n size in settings should match what is displayed on the UI

What is your host/environment?

Operating system, version.
macos

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

To fix this we should take into account the default settings when making a call to get cluster settings

[dblock]

[Catch All Triage - 1, 2, 3]