SPL Annotations

SPL annotations are used to preserve information about the actual application (implemented in Python/Java/Scala) in the SPL graph.

An operator invocation can have multiple @spl_note annotations, the approach is to use the id value to indicate what the note represents. id values used start with __spl_ as a unofficial claimed namespace. text is usually a representation of a JSON object unless it is clear only a single value is ever needed.

Examples are shown as SPL code, thus text values are shown as SPL string literals.

Source location

An operator may be annotated with an @spl_note that provides information about the applications source the operator invocation corresponds to.

• id=__spl_sourcelocation
• text=String representation of a single JSON object or array of JSON objects containing these properties. Note each property is optional.
  • file: source file name
  • class: class name
  • method : method name - Method or function call the api.method was invoked from.
  • line: line number in source file - this is line that the api method was invoked
  • api.method: api method - The API method used by the application, e.g. map, publish, etc.

Example:

@spl_note(id="__spl_sourcelocation", text="{\"file\":\"quick.py\",\"method\":\"main\",\"line\":15,\"api.method\":\"source\"}")

Stream type DO NOT USE

WILL BE CHANGED - DO NOT USE

An operator may be annotated with multiple @spl_note that provide information about native type of its output ports. ~~

Note that only a subset of output ports might have this annotation.

• id=__spl_nativeType_output_N - where N is the numeric index (starting at 0) of the output port.
• text=native type name.

Example:

@spl_note(id="__spl_nativeType_output_0", text="class vwap.VWapT")

Layout hints

Single annotation for an operator invocation that provides graph layout hints for visualization tools in order to have the flow graph better represent the application the user developed.

• id=__spl_layout
• text= String representation of a single JSON object, top-level attributes are optional.
  • hidden - Boolean indicating the operator invocation is to be hidden (true==hidden). Typically because it is inserted into the graph as an implementation detail, e.g. hash adder. Optional, when not present operator should be displayed.
  • kind - String - Operator kind to use instead of SPL kind (e.g. ParDo) - optional, display defaults to SPL operator kind
  • group - JSON object used to group operators together as a single entity (as a single node in the graph) - optional. If group exists then it should contain all its attributes and they are expected to be the same for a given group id.
    • id - String - Group identifier, not displayed. All operators with this id can be grouped together as a single operator. ids are unique within an app.
    • name - String - Name to display as operator invocation name.
    • kind - String - Operator kind to use when operator is part of a group (e.g. Publish) and is being displayed as a group.
  • names: Json object mapping SPL names to application names (strings) - optional. Application names are provided by the user, SPL names are SPL identifiers in the running job (e.g. operator logical, port or stream names). A user's application (e.g. in Streams Designer or Python Application API) can name items using values that cannot be represented as an SPL identifier (e.g. containing Unicode characters or a SPL/C reserved keyword). If an SPL operator invocation logical name or one of its stream names exists in this map then the user's application name should be displayed, otherwise the actual SPL identifier is used.

Note a layout can contain any mix of the top level attributes, e.g. kind and names.

Note a layout hint can be applied to an SPL composite invocation

• As a group - meaning the composite is to be shown as a single node. The composite may be grouped with other invocations.
• As hidden, meaning all the operators in the composite (and the composite itself) are hidden.

Example:

# Hidden operator
@spl_note(id="__spl_layout", text='{"hidden":true}')

# Grouped operator with simplified name to be shown when not grouped.
@spl_note(id="__spl_layout", text='{"kind":"Filter", "group": {"id":"xzy1", "kind":"SmartFilter", "name":"OutOfRangeTemps"}}')

# Info about names that were remapped (physical representation of the chinese characters is made up)
@spl_note(id="__spl_layout", text='{"names": {"Buses_3":"Buses", "uX1234uX9221":"总线"}}')