Finish clenaup on planning/external-events doc

docs/planning/external-events.mdx

@@ -18,7 +18,6 @@ import timelineEditorFull from './assets/timeline_and_editor_full.png';
 
 # External Events
 
-A new construct in Aerie is that of External Events. In this document, we will discuss what EEs are, what their containing entities (External Sources) are, their grouping mechanisms (Derivation Groups), as well as how they manifest in the UI and in the backend.
 
 An **External Event** is temporal data used to inform a mission planner's decision making process. By nature, these events are *external* to Aerie itself and represent temporal occurrences that cannot be influenced or interacted with directly, and when uploaded to Aerie are static and immutable.
 
@@ -27,6 +26,8 @@ Examples of External Events include:
 * Orbital events
 * Weather forecasts
 
+In this document, we will discuss what External Events are, what their containing entities (External Sources) are, their grouping mechanisms (Derivation Groups), as well as how they manifest in the UI and in the backend.
+
 ## Concepts
 
 ### Data Constructs
@@ -492,7 +493,6 @@ The External Source Manager page handles the uploading & inspection of External
   <img alt="Figure 8: External Source Manager" src={externalSourceManager} />
   <figcaption><i>Figure 8: External Source Manager</i></figcaption>
 </figure>
-**TODO: UPDATE THIS IMAGE**
 
 #### Uploading an External Source
 The main view allows users to upload External Sources. Most of these fields autofill and are immutable once parsed - except the Derivation Group which will be autofilled in the style of `{external_source_type} Default`, but can be edited by the user before upload.
@@ -583,113 +583,6 @@ As a final note on the frontend, it is worth briefly detailing what parts of the
 - _External Event Type filters_ - the External Event Type filters selected in the timeline editor,
 - _External Event options_ - the options in the timeline editor filter, including `Group By` and `Bin Size`.
 
-## Backend
-For the sake of thoroughness and also for the aid of future development, we detail here all major changes made to the backend. These include changes to the database (the 2 new migrations will be discussed), changes to Hasura (including the new mutations that have been created/are in use), as well as some detailing of how said mutations are currently used in the frontend, as examples.
-
-### PostgreSQL Changes
-The introduction of External Events introduces to Aerie the following tables:
-
-**External Sources.** This is a table that links the uploaded file to information about the External Source itself, namely the fields discussed in [Concepts](#concepts). It also references the External Source Type table and Derivation Group tables (which are discussed later) to identify what it is a part of. The tuple of `(key, derivation_group_id)`, uniquely distinguish a file, as they are namespaced (`key`) under their External Source Type, but are distinguished further should they share names by their Derivation Group (`derivation_group_id`), which acts as a subset of the External Source Type. That being said, the table itself has a serial key as a primary key for ease of foreign key references.
-
-**External Events.** This is a table that details with each row an individual instance of an External Event, with all the fields discussed in [Concepts](#concepts). It has links to its External Source, as well as to an External Event Type. Logically, the tuple of (`key, event_type_id, source_id`) uniquely identify a row, as events are namespaced (`key`) within their External Event Type (`event_type_id`), and are distinguished should they share a name by the source they come from (`source_id`). Again there is a serial `id` primary key for ease of use.
-
-**External Source Type.** This table actually formally identifies different types of External Sources. Presently, each type bears only a name, but should future qualities and quantities be associated with different types (i.e like descriptions, some SQL-representation of External Source Type-specific JSON Schemae, or even versioning), that would be handled here. This table is only referenced by External Source, and by an id instead of a name for ease of use.
-
-**External Event Type.** Much like External Source Type above, this table formally identifies different types of External Events. As there is not much functionality associated with Event Types beyond identification, the only non-id field in this table is again, a name, though that could change going forwards.
-
-**External Source Event Types.** This table maps External Sources to their contained External Event Types. Eventually, this table _should_ become obsolete as a more formal tie between External Source Types and External Event Types becomes defined, at the JSON Schema level and/or at the SQL level. Since it was useful in the UI to point out what External Event Types were included in a given External Source (for External Source inspection and for filtering purposes), but there did not exist an easy way to get this information, a table was created for it. What this means is that it is entirely possible for two External Sources of the same External Source Type to have contained in it different sets of External Event Types, which when a stricter schema is imposed, should ultimately not be possible.
-
-**Derivation Group.** This table represents individual Derivation Groups, which are collections of External Sources against which the aforementioned [Derivation](#derivation) operation is run. There is not too much to a Derivation Group outside of the operation itself (just a name and an External Source Type, to identify and guarantee External Sources of the same type are added to a group); this table is simply referenced by External Sources and references an External Source Type. There is also a constraint in place that ensures that External Sources linked to this group must bear the same External Source Type.
-
-**Plan Derivation Group.** This table associates plans and Derivation Groups. Whenever a new association is created (or removed) in the External Source Manager modal, this table is what gets updated. It is from the entries in this table that the relevant External Event Types in the timeline editor filter are found as well.
-
-**Derived Events (_View_).** The [Derivation](#derivation) operation is actually handled at the database level, through a view. This way, even if Aerie were to be used without the official UI (for example, via a CLI tool or a customized UI), users wouldn't need to reimplement this operation. This view looks at all Derivation Groups individually, the sources within them, and then produces the correct derived events as output. Each row in this table is a derived event from each and every External Source and Derivation Group, making it a direct subset of the External Event table.
-
-**Derivation Group Comp (_View_).** This view was created for the sake of simplification, much like the External Source Event Types table (though it likely will not soon become obsolete). Previously, several Hasura requests and heavyweight susbscriptions were required to get the number of External Events, the different External Sources associated with Derivation Groups, and the associated External Event Types (quanities that do make an appearance at least once in the plan part of the UI). As such, to simplify things, a view was instead created.
-
-### Hasura Changes
-Metadata was added to Hasura to ensure that all tables and views described above had mutations and queries available to access them. We discuss here the mutations that we made use of, whose uses can be found in `gql.ts` and `effects.ts`, as well as in `stores/external-source.ts` and `stores/external-event.ts`.
-
-
-#### Mutations
-_`mutation CreateExternalEventType`_
-
-This is a mutation invoked to create a new External Event Type. It is invoked whenever a source with a new External Event Type is uploaded. It can be found in `effects.ts`, under `createExternalEventType()`, which is used by `ExternalSourceManager.svelte` during the file upload process.
-
-
-_`mutation CreateExternalSource`_
-
-This is a mutation invoked to create a new External Source. It is invoked anytime a source is to be uploaded, from `createExternalSource()` in `effects.ts`. That method is invoked by `ExternalSourceManager.svelte` during the file upload process.
-
-
-_`mutation CreateExternalSource`_
-
-This mutation associates External Sources and their contained Event Types. It is invoked anytime a source is to be uploaded, from `createExternalSourceEventTypeLink()` in `effects.ts`. That method is invoked by `ExternalSourceManager.svelte` during the file upload process.
-
-_`mutation CreateExternalSourceType`_
-
-This is a mutation invoked to create a new External Source Type. It is invoked whenever a source with a new External Source Type is uploaded. It can be found in `effects.ts`, under `createExternalSourceType()`, which is used by `ExternalSourceManager.svelte` during the file upload process.
-
-_`mutation CreateDerivationGroup`_
-
-This is a mutation invoked to create a new Derivation Group. Anytime a source is uploaded under a new Derivation Group name (that is, a new name for its External Source Type, as two Derivation Groups across different External Source Types can share a name but be distinct), this mutation is to be invoked. It can be found in `effects.ts`, under `createDerivationGroup()`, which is used by `ExternalSourceManager.svelte` during the file upload process.
-
-_`mutation CreatePlanDerivationGroup`_
-
-This is a mutation invoked to create an association between a plan and a Derivation Group. Any time a Derivation Group is selected in the `Manage Derivation Groups` modal in the plan view, this mutation is invoked (see `insertDerivationGroupForPlan()` in `effects.ts`, invoked in  `ManagePlanDerivationGroupsModal.svelte`).
-
-_`mutation DeleteExternalSource`_
-
-This mutation is invoked on deletion of an External Source. It removes the External Source, all External Events in it, the entry in the ExternalSourceEventType table, as well as the uploaded file. It is performed in `deleteExternalSource()` in `effects.ts`, which can be invoked either via the source inspector or the table in `ExternalSourceManager.svelte`.
-
-_`mutation DeletePlanDerivationGroup`_
-
-This is a mutation invoked to remove an association between a plan and a Derivation Group. Any time a Derivation Group is deselected in the `Manage Derivation Groups` modal in the plan view, this mutation is invoked (see `deleteDerivationGroupForPlan()` in `effects.ts`, invoked in  `ManagePlanDerivationGroupsModal.svelte`).
-
-#### Queries
-
-_`query GetExternalEvents`_
-
-This is a query to get all External Events associated with a given External Source. It is used in `effects.ts` in `getExternalEvents()`, a function invoked in `ExternalSourceManager.svelte` to get the External Events associated with the selected/inspected External Source.
-
-_`query GetExternalEventTypesBySource`_
-
-This query gets all External Event Types associated wiht a given source, which is used both for the sake of deletion (in cleaning up extra External Event Types), as well as inspection (in displaying the External Event Types associated with a given External Source). This may eventually become deprecated as strictness is imposed on what External Event Types are allowable by External Source Type, information that may be stored elsewhere.
-
-_`query GetExternalSourceMetadata`_
-
-This query, despite its name, is actually used to get the metadata associated with an External Source. This gets its own query as having a subscription on External Sources also pull all metadata with it could become very heavyweight, especially since we only access metadata one External Source at a time, which is on (necessarily singular) source inspection. It mainfests in `effects.ts` under `getExternalSourceMetadata()`, which finds use in `ExternalSourceManager.svelte`, when users inspect individual External Sources by selecting them from the table.
-
-_`query GetPlanEventTypes`_
-
-This query is used to get the External Event Types associated with a plan. Invoked in the `getExternalEventTypes()` effect of `effects.ts`, it is used in the plan view (under its `page.ts`), where it is invoked to get all the ExternalEventTypes that might be valid on page load (the subscriptions described later do not load on page load, so this simply allows everything to be visible without waiting for the stores to possibly catch up).
-
-#### Subscriptions
-
-_`subscription SubExternalEventTypes`_
-
-This subscription is used in `stores/external-event.ts` to grab all currently uploaded/used External Event Types currently in Aerie. They are used to allow for easy association of External Events, who simply have an `external_event_type_id`, with their External Event Type names (see `externalEventWithTypeName` store).
-
-_`subscription SubExternalSources`_
-
-This subscription is used in `stores/external-source.ts` to grab all External Sources (minus their metadata) currently in Aerie. They are stored as an array of `ExternalSourceSlim` structures. THis stores is used across `ExternalSourceManager.svelte`, `ExternalSourcePanelEntry.svelte`, and `ManagePlanDerivationGroupsModal.svelte`, after being transformed into a derived store `externalSourceWithResolvedNames`.
-
-_`subscription SubExternalSourceTypes`_
-
-This subscription is used in `stores/external-sources.ts` to grab all External Source Types currently in Aerie, which are used like the `SubExternalEventTypes` subscription to create a store for easy association between External Sources and External Source Type names. This transformation is done in the derived store `externalSourceWithResolvedNames`.
-
-_`subscription SubDerivationGroups`_
-
-This subscription is used in `stores/external-sources.ts` to get the list of all Derivation Groups as they appear in the transformed view [Derivation Group Comp](#postgresql-changes). It is parsed (there are JSON strings in the raw version) into the `derivationGroups` derived store, which is used in `ExternalSourceManager.svelte`, `ExternalSourcesPanel.svelte`, `ManagePlanDerivationGroupsModal.svelte`, and `TimelineEditorPanel.svelte`.
-
-_`subscription SubPlanExternalSource`_
-
-This subscription is used in `stores/external-sources.ts` to get the list of all plan/Derivation Group links, a store used in `ExternalSourceManager.svelte`, `ExternalSourcesPanel.svelte`, and `TimelineEditorPanel.svelte`.
-
-_`subscription SubExternalSourceEventType`_
-
-This subscription is used in `stores/external-sources.ts` to get the list of all External Source/External Event Type links, which is used in `ExternalSourceManager.svelte` in the upload process (to avoid duplicating entries on source upload).
-
 ## What Remains
 Here, we discuss briefly everything that is not currently implemented but that we do plan to in the future.