[RFC] Hosting of OpenSearch Dashboards Extensions

[kavilla]

Background

OpenSearch Dashboards will first look into the local file path directory of /extensions. It will recursively look through each folder for each extension manifest and then read the information. If properly formatted, the extension gets returned as a discovered extension. Then the server will also handle discovered extensions, invoke it’s configurations (if it has any), look for its public, bootstrap that code, and then when the server started it hosts the bootstrapped code under the server route of the extension.

Current state:

• Core
  • The main runtime responsible for managing the lifecycle of the application and all its main services
• Packages
  • A set of static utilities that can be imported and used throughout the application
• Plugins
  • All other major functionality within OpenSearch Dashboards (for example the home plugin, navigation plugin)
  • Built and installed in before runtime
  • Installed in same process
  • Interacts with OpenSearch Dashboards core via wrapper like the plugin API.
• Extensions
  • Similar to plugins

Plugins are the current way to extend and customize the core functionality of OpenSearch Dashboards. They do not need to be a part of the Dashboards repository. Extensions are the vNext way developers can extend functionality.

These makeup some of the core applications and services within it. Plugins and extensions can consist of either client side code (public), server side code (server), or both, and common (for sharing between public and server). Plugins and extensions can also interact with each other and core from both client and server. Plugins and extensions currently contain a manifest file that describes a set of properties, both required and optional that core system can use to load and initialize correctly.

Proposal

• Utilize OpenSearch to store the information related to installed extensions
  • OpenSearch stores into a system index the manifests for installed extensions
  • PUT .osd_extensions (NOTE: tentative. Could use .kibana)

      {
        "mappings": {
          "properties": {
            "extensionId": { "type": "text" },
            "version": { "type": "text"},
            "opensearchDashboardsVersion": { "type": "text" },
            "configPath": { "type" : "text" },
            "requiredExtensions": { "type": "text" },
            "optionalExtensions": { "type": "text" },
            "requiredBundles": { "type": "text" },
            "requiredPlugins": { "type": "text" },
            "optionalPlugins": { "type": "text" },
            "ui": { "type": "boolean" },
            "server": { "type": "boolean" },
            "extraPublicDirs": { "type": "text" },
            "path": { "type": "text" }
          }
        }
      }
  

• OpenSearch Dashboards server starts up and hits this index and discovers extensions
• OpenSearch Dashboards server hosts routes from extensions to the bundled JS code
• User navigates to route and the bundled JS code is mounted to the React DOM

vNext

• Within the SDK, the extension server logic enables the power to be hosted outside the OpenSearch Dashboards main thread as a Node API
  • Enforcing all interactions to be via HTTP requests
• Therefore the public code does not need to be bootstrapped on OpenSearch Dashboards start, it will already would have been bootstrapped before and hosted at a route that is stored in the system index
• OpenSearch Dashboards accesses the route and then allow the already production use code to mount the public code
• Extension server provides a config API
• Extension server provides dependencies/capabilities API

Benefits

• Ensure interactions with core repo APIs
• Utilizes OpenSearch to store metadata so the data is replicated to all nodes within a cluster solving the problem with multiple nodes