[Data Explorer] Merge to main

[ashwin-pc]

Description

This PR brings the Data Explorer branch into main so that future development can happen on mainline. It primarily implements the data explorer framework and Discover 2.0 which is built on top of it. It also supports the legacy Discover with a user can use to toggle back to the older experience. There are still a few missing features in Discover 2.0 that need to be implemented but those changes will be raised against main.

Note: Merging the feature/data-explorer branch to main had a lot of merge conflicts that were confusing the github diff even after they were resolved, so the approach I've taken here is to take a new branch off main and merge feature/data-explorer into it and then raise a PR from the newly created branch. Due to this the commit history has been squashed but the diff should be accurate. Open to suggestions on a better way of doing this though.

Issues Resolved

Screenshot

Screen.Recording.2023-08-18.at.5.25.56.PM.mov

Note: The toggle in the video changing to a button is no longer present and was an artifact of an unmerged PR being cached. The PR has been merged and the functionality works as expected.

Testing the changes

Launch OSD and use Discover. Discover 2.0 should have feature parity with the legacy app.

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4806 (6db1271) into main (0188087) will increase coverage by 0.15%.
The diff coverage is 66.40%.

@@            Coverage Diff             @@
##             main    #4806      +/-   ##
==========================================
+ Coverage   66.30%   66.46%   +0.15%     
==========================================
  Files        3322     3390      +68     
  Lines       63933    64742     +809     
  Branches    10121    10312     +191     
==========================================
+ Hits        42393    43028     +635     
- Misses      19051    19170     +119     
- Partials     2489     2544      +55     

Flag	Coverage Δ
Linux_1	34.91% <12.63%> (+0.07%)	⬆️
Linux_2	55.15% <ø> (ø)
Linux_3	44.13% <70.00%> (+0.75%)	⬆️
Linux_4	35.09% <ø> (-0.01%)	⬇️
Windows_1	34.92% <12.63%> (+0.07%)	⬆️
Windows_2	55.12% <ø> (ø)
Windows_3	44.14% <70.00%> (+0.75%)	⬆️
Windows_4	35.09% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...on/components/doc_viewer/doc_viewer_render_tab.tsx	80.00% <ø> (ø)
...ion/components/loading_spinner/loading_spinner.tsx	100.00% <ø> (ø)
...tion/components/sidebar/discover_field_details.tsx	80.00% <ø> (ø)
...mponents/skip_bottom_button/skip_bottom_button.tsx	100.00% <ø> (ø)
...blic/application/components/table/table_helper.tsx	100.00% <ø> (ø)
...public/application/doc_views/doc_views_registry.ts	0.00% <ø> (ø)
src/plugins/discover/public/build_services.ts	0.00% <ø> (ø)
src/plugins/discover/public/index.ts	0.00% <ø> (ø)
src/plugins/discover/public/migrate_state.ts	0.00% <0.00%> (ø)
.../discover/public/opensearch_dashboards_services.ts	58.82% <ø> (-7.85%)	⬇️
... and 111 more

... and 44 files with indirect coverage changes

[ashwin-pc]

All commits have signoffs. the squashed PR however is causing DCO to struggle but it does have a signoff too.

[AMoo-Miki]

Is this always true? irrespective of the branding assets being loaded?

[ashwin-pc]

Yes this is the same calc i used for VisBuilder. This works because the height of the top nav is fixed.

[AMoo-Miki]

Wouldn't this be more user-friendly if it said "The index pattern patten* was not found" and maybe suggest that they recheck their spelling?

[ashwin-pc]

The if condition here simply checks if there exists an index pattern so if this warning is shown, there exists no index pattern to show.

[AMoo-Miki]

These guys are asynchronously updating rootState. We could end up with some overwriting each other, producing unexpected and different results. Are we good with that?

[ashwin-pc]

Hmm, good call. I need this because saved searches also specify the index pattern that we need on page load and that's in the root state. I am worried about the non deterministic nature of the load though even if the overwrite is fine with me. Any suggestions on how to improve this? I do want to avoid serializing this if possible because it increases page load time. This can also be improved in future l, but I do see the problem

[ananzh]

We have not implemented this. I will create an issue.

[ashwin-pc]

Cypress test fix in opensearch-project/opensearch-dashboards-functional-test#803. Will merge after this PR is merged

[AMoo-Miki]

nit: This line is unnecessary.

[AMoo-Miki]

Suggested change

	if (!currentView) return;
	dispatch(setView(currentView?.id));
	if (!currentView?.id) return;
	dispatch(setView(currentView.id));

[AMoo-Miki]

Should this be

Suggested change

	if (!view.ui) {
	if (!view.ui?.defaults) {

or a detailed check of it being a function or a simple object? Should it also check id?

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-4806-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4f909f9aafb729248d8ccc71c52096d5c15b223a
# Push it to GitHub
git push --set-upstream origin backport/backport-4806-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4806-to-2.x.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-4806-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4f909f9aafb729248d8ccc71c52096d5c15b223a
# Push it to GitHub
git push --set-upstream origin backport/backport-4806-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4806-to-2.x.