diff --git a/README.md b/README.md index 2bb8204..13c86c0 100644 --- a/README.md +++ b/README.md @@ -19,12 +19,12 @@ For information on Siddhi and i ## Latest API Docs -Latest API Docs is 5.0.1. +Latest API Docs is 5.0.2. ## Features -* akslack *(Stream Processor)*

This stream processor extension performs reordering of an event stream which is out of order.
 It implements the AQ-K-Slack based out-of-order handling algorithm which is originally described in
'http://dl.acm.org/citation.cfm?doid=2675743.2771828'.

-* kslack *(Stream Processor)*

This stream processor extension performs reordering of an out-of-order event stream.
 It implements the K-Slack based out-of-order handling algorithm which is originally described in
'https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf'.)

+* akslack *(Stream Processor)*

Stream processor performs reordering of out-of-order events optimized for a givenparameter using [AQ-K-Slack algorithm](http://dl.acm.org/citation.cfm?doid=2675743.2771828). This is best for reordering events on attributes those are used for aggregations.data .

+* kslack *(Stream Processor)*

Stream processor performs reordering of out-of-order events using [K-Slack algorithm](https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf).

## Dependencies diff --git a/docs/api/5.0.2.md b/docs/api/5.0.2.md new file mode 100644 index 0000000..aa87f84 --- /dev/null +++ b/docs/api/5.0.2.md @@ -0,0 +1,177 @@ +# API Docs - v5.0.2 + +!!! Info "Tested Siddhi Core version: *5.0.2*" + It could also support other Siddhi Core minor versions. + +## Reorder + +### akslack *(Stream Processor)* +

Stream processor performs reordering of out-of-order events optimized for a givenparameter using [AQ-K-Slack algorithm](http://dl.acm.org/citation.cfm?doid=2675743.2771828). This is best for reordering events on attributes those are used for aggregations.data .

+Syntax + +``` +reorder:akslack( timestamp, correlation.field) +reorder:akslack( timestamp, correlation.field, batch.size) +reorder:akslack( timestamp, correlation.field, batch.size, timeout) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k, discard.late.arrival) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k, discard.late.arrival, error.threshold, confidence.level) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
timestampThe event timestamp on which the events should be ordered.LONGNoYes
correlation.fieldBy monitoring the changes in this field Alpha K-Slack dynamically optimises its behavior. This field is used to calculate the runtime window coverage threshold, which represents the upper limit set for unsuccessfully handled late arrivals.INT
FLOAT
LONG
DOUBLE
NoYes
batch.sizeThe parameter 'batch.size' denotes the number of events that should be considered in the calculation of an alpha value. This should be greater than or equal to 15.`10,000`LONGYesNo
timeoutA timeout value in milliseconds, where the buffered events who are older than the given timeout period get flushed every second.`-1` (timeout is infinite)LONGYesNo
max.kThe maximum K-Slack window threshold ('K' parameter).`9,223,372,036,854,775,807` (The maximum Long value)LONGYesNo
discard.late.arrivalIf set to true the processor would discarded the out-of-order events arriving later than the K-Slack window, and in otherwise it allows the late arrivals to proceed.falseBOOLYesNo
error.thresholdThe error threshold to be applied in Alpha K-Slack algorithm. `0.03` (3%)DOUBLEYesNo
confidence.levelThe confidence level to be applied in Alpha K-Slack algorithm.`0.95` (95%)DOUBLEYesNo
+ +Examples +EXAMPLE 1 +``` +define stream StockStream (eventTime long, symbol string, volume long); + +@info(name = 'query1') +from StockStream#reorder:akslack(eventTime, volume, 20)#window.time(5 min) +select eventTime, symbol, sum(volume) as total +insert into OutputStream; +``` +

The query reorders events based on the 'eventTime' attribute value and optimises for aggregating 'volume' attribute considering last 20 events.

+ +### kslack *(Stream Processor)* +

Stream processor performs reordering of out-of-order events using [K-Slack algorithm](https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf).

+Syntax + +``` +reorder:kslack( timestamp) +reorder:kslack( timestamp, timeout) +reorder:kslack( timestamp, discard.late.arrival) +reorder:kslack( timestamp, timeout, max.k) +reorder:kslack( timestamp, timeout, discard.late.arrival) +reorder:kslack( timestamp, timeout, max.k, discard.late.arrival) +``` + +QUERY PARAMETERS + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
timestampThe event timestamp on which the events should be ordered.LONGNoYes
timeoutA timeout value in milliseconds, where the buffered events who are older than the given timeout period get flushed every second.`-1` (timeout is infinite)LONGYesNo
max.kThe maximum K-Slack window threshold ('K' parameter).`9,223,372,036,854,775,807` (The maximum Long value)LONGYesNo
discard.late.arrivalIf set to true the processor would discarded the out-of-order events arriving later than the K-Slack window, and in otherwise it allows the late arrivals to proceed.falseBOOLYesNo
+ +Examples +EXAMPLE 1 +``` +define stream StockStream (eventTime long, symbol string, volume long); + +@info(name = 'query1') +from StockStream#reorder:kslack(eventTime, 5000) +select eventTime, symbol, volume +insert into OutputStream; +``` +

The query reorders events based on the 'eventTime' attribute value, and it forcefully flushes all the events who have arrived older than the given 'timeout' value (5000 milliseconds) every second.

+ diff --git a/docs/api/latest.md b/docs/api/latest.md index 3449fe1..aa87f84 100644 --- a/docs/api/latest.md +++ b/docs/api/latest.md @@ -1,17 +1,21 @@ -# API Docs - v5.0.1 +# API Docs - v5.0.2 -!!! Info "Tested Siddhi Core version: *5.0.0*" +!!! Info "Tested Siddhi Core version: *5.0.2*" It could also support other Siddhi Core minor versions. ## Reorder -### akslack *(Stream Processor)* - -

This stream processor extension performs reordering of an event stream which is out of order.
 It implements the AQ-K-Slack based out-of-order handling algorithm which is originally described in
'http://dl.acm.org/citation.cfm?doid=2675743.2771828'.

- +### akslack *(Stream Processor)* +

Stream processor performs reordering of out-of-order events optimized for a givenparameter using [AQ-K-Slack algorithm](http://dl.acm.org/citation.cfm?doid=2675743.2771828). This is best for reordering events on attributes those are used for aggregations.data .

Syntax + ``` -reorder:akslack( timestamp, correlation.field, batch.size, timer.timeout, max.k, discard.flag, error.threshold, confidence.level) +reorder:akslack( timestamp, correlation.field) +reorder:akslack( timestamp, correlation.field, batch.size) +reorder:akslack( timestamp, correlation.field, batch.size, timeout) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k, discard.late.arrival) +reorder:akslack( timestamp, correlation.field, batch.size, timeout, max.k, discard.late.arrival, error.threshold, confidence.level) ``` QUERY PARAMETERS @@ -26,47 +30,47 @@ reorder:akslack( timestamp, correlation.field, timestamp - The attribute used for ordering the events. + The event timestamp on which the events should be ordered. LONG No - No + Yes correlation.field - This corresponds to the data field of which the accuracy directly gets affected, by the adaptive operation of the Alpha K-Slack extension. This field is used to calculate the runtime window coverage threshold which is an upper limit set for the unsuccessfully handled late arrivals. + By monitoring the changes in this field Alpha K-Slack dynamically optimises its behavior. This field is used to calculate the runtime window coverage threshold, which represents the upper limit set for unsuccessfully handled late arrivals. INT
FLOAT
LONG
DOUBLE No - No + Yes batch.size - The parameter 'batch.size' denotes the number of events that should be considered in the calculation of an alpha value. It should be a value which should be greater than or equal to fifteen. - 10,000 + The parameter 'batch.size' denotes the number of events that should be considered in the calculation of an alpha value. This should be greater than or equal to 15. + `10,000` LONG Yes No - timer.timeout - This corresponds to a fixed time out value in milliseconds, which is set at the beginning of the process. Once the time out value expires, the extension drains out all the events that are buffered within the reorder extension. The time out has been implemented internally using a timer. The events buffered within the extension are released each time the timer ticks. - -1 (timeout is infinite) + timeout + A timeout value in milliseconds, where the buffered events who are older than the given timeout period get flushed every second. + `-1` (timeout is infinite) LONG Yes No max.k - This is the maximum threshold value for 'K' parameter in the Alpha K-Slack algorithm. - 9,223,372,036,854,775,807 (The maximum Long value) + The maximum K-Slack window threshold ('K' parameter). + `9,223,372,036,854,775,807` (The maximum Long value) LONG Yes No - discard.flag - This indicates whether the out-of-order events which appear after the expiration of the Alpha K-slack window should be discarded or not. When this value is set to 'true', the events are discarded. + discard.late.arrival + If set to true the processor would discarded the out-of-order events arriving later than the K-Slack window, and in otherwise it allows the late arrivals to proceed. false BOOL Yes @@ -74,88 +78,45 @@ reorder:akslack( timestamp, correlation.field, error.threshold - The error threshold to be applied in Alpha K-Slack algorithm. This parameter must be defined simultaneously with 'confidenceLevel'. - 0.03 (3%) + The error threshold to be applied in Alpha K-Slack algorithm. + `0.03` (3%) DOUBLE Yes No confidence.level - The confidence level to be applied in Alpha K-Slack algorithm. This parameter must be defined simultaneously with 'errorThreshold'. - 0.95 (95%) + The confidence level to be applied in Alpha K-Slack algorithm. + `0.95` (95%) DOUBLE Yes No -Extra Return Attributes - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NameDescriptionPossible Types
beta0The timestamp based on which the reordering is performed.LONG
beta1An upper limit value assigned for the unsuccessfully handled late arrivals.DOUBLE
beta2The number of events that should be considered in the calculation of an alpha value.LONG
beta3The fixed time-out value in milliseconds assigned for flushing all the events buffered inside the extension.LONG
beta4The maximum threshold value assigned for 'K' parameter.LONG
beta5The flag set to indicate whether the out-of-order events which arrive after buffer eviction are to be discarded or not.BOOL
beta6The error threshold value set for Alpha K-Slack algorithm.DOUBLE
beta7The confidence level set for the Alpha K-Slack algorithm.DOUBLE
Examples EXAMPLE 1 ``` -define stream InputStream (eventtt long,data double); +define stream StockStream (eventTime long, symbol string, volume long); + @info(name = 'query1') -from InputStream#reorder:akslack(eventtt, data, 20) -select eventtt, data +from StockStream#reorder:akslack(eventTime, volume, 20)#window.time(5 min) +select eventTime, symbol, sum(volume) as total insert into OutputStream; ``` -

This query performs reordering based on the 'eventtt' attribute values. In this example, 20 represents the batch size.

- -### kslack *(Stream Processor)* - -

This stream processor extension performs reordering of an out-of-order event stream.
 It implements the K-Slack based out-of-order handling algorithm which is originally described in
'https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf'.)

+

The query reorders events based on the 'eventTime' attribute value and optimises for aggregating 'volume' attribute considering last 20 events.

+### kslack *(Stream Processor)* +

Stream processor performs reordering of out-of-order events using [K-Slack algorithm](https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf).

Syntax + ``` -reorder:kslack( timestamp, timer.timeout, max.k, discard.flag) +reorder:kslack( timestamp) +reorder:kslack( timestamp, timeout) +reorder:kslack( timestamp, discard.late.arrival) +reorder:kslack( timestamp, timeout, max.k) +reorder:kslack( timestamp, timeout, discard.late.arrival) +reorder:kslack( timestamp, timeout, max.k, discard.late.arrival) ``` QUERY PARAMETERS @@ -170,31 +131,31 @@ reorder:kslack( timestamp, timer.timeout, max.k, disc timestamp - This is the attribute used for ordering the events. + The event timestamp on which the events should be ordered. LONG No - No + Yes - timer.timeout - This corresponds to a fixed time-out value in milliseconds, which is set at the beginning of the process. Once the time-out value expires, the extension drains out all the events that are buffered within the reorder extension. The time-out has been implemented internally using a timer. The events buffered within the extension are released each time the timer ticks. - -1 (timeout is infinite) + timeout + A timeout value in milliseconds, where the buffered events who are older than the given timeout period get flushed every second. + `-1` (timeout is infinite) LONG Yes No max.k - The maximum threshold value for 'K' parameter in the K-Slack algorithm. - 9,223,372,036,854,775,807 (The maximum Long value) + The maximum K-Slack window threshold ('K' parameter). + `9,223,372,036,854,775,807` (The maximum Long value) LONG Yes No - discard.flag - This indicates whether the out-of-order events which appear after the expiration of the K-slack window should be discarded or not. When this value is set to 'true', the events would get discarded. + discard.late.arrival + If set to true the processor would discarded the out-of-order events arriving later than the K-Slack window, and in otherwise it allows the late arrivals to proceed. false BOOL Yes @@ -205,11 +166,12 @@ reorder:kslack( timestamp, timer.timeout, max.k, disc Examples EXAMPLE 1 ``` -define stream InputStream (eventtt long, price long, volume long); +define stream StockStream (eventTime long, symbol string, volume long); + @info(name = 'query1') -from InputStream#reorder:kslack(eventtt, 1000) -select eventtt, price, volume +from StockStream#reorder:kslack(eventTime, 5000) +select eventTime, symbol, volume insert into OutputStream; ``` -

This query performs reordering based on the 'eventtt' attribute values. In this example, the timeout value is set to 1000 milliseconds

+

The query reorders events based on the 'eventTime' attribute value, and it forcefully flushes all the events who have arrived older than the given 'timeout' value (5000 milliseconds) every second.

diff --git a/docs/index.md b/docs/index.md index 2bb8204..13c86c0 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,12 +19,12 @@ For information on Siddhi and i ## Latest API Docs -Latest API Docs is 5.0.1. +Latest API Docs is 5.0.2. ## Features -* akslack *(Stream Processor)*

This stream processor extension performs reordering of an event stream which is out of order.
 It implements the AQ-K-Slack based out-of-order handling algorithm which is originally described in
'http://dl.acm.org/citation.cfm?doid=2675743.2771828'.

-* kslack *(Stream Processor)*

This stream processor extension performs reordering of an out-of-order event stream.
 It implements the K-Slack based out-of-order handling algorithm which is originally described in
'https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf'.)

+* akslack *(Stream Processor)*

Stream processor performs reordering of out-of-order events optimized for a givenparameter using [AQ-K-Slack algorithm](http://dl.acm.org/citation.cfm?doid=2675743.2771828). This is best for reordering events on attributes those are used for aggregations.data .

+* kslack *(Stream Processor)*

Stream processor performs reordering of out-of-order events using [K-Slack algorithm](https://www2.informatik.uni-erlangen.de/publication/download/IPDPS2013.pdf).

## Dependencies diff --git a/mkdocs.yml b/mkdocs.yml index b0acf34..6d2c128 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -34,6 +34,7 @@ pages: - Information: index.md - API Docs: - latest: api/latest.md + - 5.0.2: api/5.0.2.md - 5.0.1: api/5.0.1.md - 5.0.0: api/5.0.0.md - 4.2.2: api/4.2.2.md