SPL Annotations

SPL annotations are used to preserve information about the actual application (implemented in Python/Java/Scala) in the SPL graph.

An operator can have multiple @spl_note annotations, the approach is to use the id value to indicate what the note represents. id values used start with __spl_ as a unofficial claimed namespace. text is usually a representation of a JSON object unless it is clear only a single value is ever needed.

Examples are shown as SPL code, thus text values are shown as SPL string literals.

Source location

An operator may be annotated with an @spl_note that provides information about the applications source the operator corresponds to.

• id=__spl_sourcelocation
• text=String representation of a single JSON object or array of JSON objects containing these properties. Note each property is optional.
  • file: source file name
  • class: class name
  • method : method name - Method or function call the api.method was invoked from.
  • line: line number in source file - this is line that the api method was invoked
  • api.method: api method - The API method used by the application, e.g. map, publish, etc.

Example

@spl_note(id="__spl_sourcelocation", text="{\"file\":\"quick.py\",\"method\":\"main\",\"line\":15,\"topology.method\":\"source\"}")

Stream type

An operator may be annotated with multiple @spl_note that provide information about native type of its output ports.

Note that only a subset of output ports might have this annotation.

• id=__spl_nativeType_output_N - where N is the numeric index (starting at 0) of the output port.
• text=native type name.

Example:

@spl_note(id="__spl_nativeType_output_0", text="class vwap.VWapT")

Layout hints

Single annotation for an operator invocation that provides graph layout hints for visualization tools in order to have the flow graph better represent the application the user developed.

• id=__spl_layout
• text= String representation of a single JSON object, top-level attributes are optional.
  • hidden - Boolean indicating the operator invocation is to be hidden (true==hidden). Typically because it is inserted into the graph as an implementation detail, e.g. hash adder.
  • kind - String - Operator kind to use instead of SPL kind (e.g. ParDo) - optional, display defaults to SPL operator kind
  • group - JSON object used to group operators together as a single entity (as a single node in the graph). If group exists then it should contain all its attributes and they are expected to be the same for a given group id.
    • id - String - Group identifier, not displayed. All operators with this id can be grouped together as a single operator.
    • name - String - Name to display as operator invocation name.
    • kind - String - Operator kind to use when operator is part of a group (e.g. Publish) and is being displayed as a group.
  • names: Json object mapping physical names to logical names (strings) . Logical names are provided by the user, physical names are SPL identifiers (e.g. stream names)). If an SPL operator invocation name or stream name exists in this map then the original name should be displayed, otherwise the actual SPL identifier is used.

Note a layout can contain any mix of the top level attributes, e.g. kind and names.

Example:

# Hidden operator
@spl_note(id="__spl_layout", text='{"hidden":true}')

# Grouped operator with simplified name to be shown when not grouped.
@spl_note(id="__spl_layout", text='{"kind":"Filter", "group": {"id":"xzy1", "kind": "SmartFilter", name="OutOfRangeTemps"}}')

# Info about names that were remapped (physical representation of the chinese characters is made up)
@spl_note(id="__spl_layout", text='{"names": {"Buses_3":"Buses", "uX1234uX9221", "总线"}}')