TimeInterval windows

The SPL language supports timeInterval windows which collect tuples into window panes specified by event-time intervals.

In contrast to tumbling and sliding windows, which keep all the tuples in a single storage area, a timeInterval window can manage several window panes. Each window pane is associated with a fixed interval defined over the event-time domain and each tuple inserted into the window has a timestamp which represents the tuple's event-time.
Note:
  • TimeInterval windows process tuples based on their assigned event-time value.
  • Time-based window policies specify eviction and trigger for tumbling and sliding windows based on the system time.

In the following examples a tuple is indicated with a timestamp value of n as T(n). A window is defined by the following SPL window clause:

window InputPort0 : timeInterval, intervalDuration(10.0), creationPeriod(5.0), discardAge(15.0);

A window pane for each of the following intervals is created:


[0, ..., 10)  
      [5, ..., 15) 
           [10, ..., 20)
                 [15, ..., 25)

--------------------------------> event-time

Each tuple that is inserted into the window is assigned to the intervals its event-time belongs to. Tuples may be assigned to several window panes, for example, a tuple with a timestamp value of T(7) is assigned to window panes [0 .. 10) and [5 .. 15).

Note: Intervals are closed at the left and open at the right. T(0) belongs to interval [0 .. 10) but T(10) does not belong to this interval.

If a timeInterval window is partitioned, each of its window panes is partitioned.

Defining windows

TimeInterval windows are represented by the TimeIntervalWindow template class, while window panes are represented by the TimeIntervalWindowPane template class. Both classes extend from the Window class and have the same template parameters as Window. Unlike tumbling and sliding windows, a timeInterval window is only a manager for the TimeIntervalWindowPane instances associated with the respective time intervals.
Note: Attempts to access the TimeIntervalWindow storage throw an exception.

TimeInterval windows are configured using a TimeIntervalWindowOptions object. The following example shows how to create window options for intervalDuration(10.0), creationPeriod(5.0), discardAge(15.0) in C++:

TimeIntervalWindowOptions().
    setIntervalDuration(timestamp(10.0)).
    setCreationPeriod(timestamp(5.0)).
    setDiscardAge(timestamp(15.0));

TimeInterval window constructors take three parameters, a reference to the operator that contains the window, an index to the input port that this window is connected to, and the window options: 

TimeIntervalWindow(*this, 0,
    SPL::TimeIntervalWindowOptions().
        setIntervalDuration(timestamp(10.0)).
        setCreationPeriod(timestamp(5.0)).
        setDiscardAge(timestamp(15.0)));

Inserting tuples

For details refer to Inserting tuples. If the event-time intervals overlap, a tuple is inserted into several window panes. Clients must create windows of shared pointers to tuples (for example TimeIntervalWindow<streams_boost::shared_ptr<Tuple> >) to avoid deleting the same tuple multiple times when the window pane gets destructed.

Watermark notifications

The SPL runtime sends a notification to the window when the operator that contains the window advances its watermark. A TimeIntervalWindowPane triggers a windowTriggerEvent when the operator's watermark becomes larger than or equal to the end of the event-time interval and it triggers again on receiving a watermark if tuples have been inserted since the last trigger. Only partitions, which have been updated since the previous trigger event, trigger again.

Window events

Clients register handlers with the TimeIntervalWindow, but events are triggered by individual window panes. Clients can register handlers for the following events:

beforeTupleInsertionEvent
Triggered before a tuple is assigned to the window pane.
afterTupleInsertionEvent
Triggered after a tuple is assigned to the window pane.
windowTriggerEvent
Triggered every time when a window pane triggers.
windowInitialFullEvent
Triggered immediately after the window pane's state transitions from Incomplete to Complete.
windowCloseEvent
Triggered immediately after the window pane's state transitions from Complete to Closed.

Execution order of events

The executions of the window events are set off by the insert calls made on the window object, similar to the tumbling and sliding windows.

  • beforeTupleInsertionEvent and afterTupleInsertionEvent are set off by tuple insert.
  • windowTriggerEvent, windowInitialFullEvent, windowCloseEvent are set off by Watermark notifications. When a watermark first passes the end of a window pane's interval, the pane first triggers a windowInitialFullEvent, followed by a windowTriggerEvent.

Partition and tuple eviction

There are no tuple or partition eviction policies for timeInterval windows. As tuples age beyond the discardAge value, the corresponding window panes get closed and deleted. If the C++ window template is instantiated using streams_boost::shared_ptr<Tuple>, tuples are deleted when no more panes reference them.

Data access

A window event handler can access tuples stored in a window pane similarly to data access in a tumbling and sliding window. Here is a fragment from a sample trigger event handler:


void MY_OPERATOR::onWindowTriggerEvent(
        WindowEventType::WindowType & window, 
        WindowEventType::PartitionType const & partition) 
{
    using namespace SPL::Functions::EventTime;

    // Access the pane's interval endpoint
    SPL::TriggerInfoProvider * p = dynamic_cast<SPL::TriggerInfoProvider *>(&window);
    if (p != NULL) {
        SPL::int64 const intervalEnd2(int64TicksFromTimestamp(
            p->getEndTime(), Milliseconds));
        SPLAPPTRC(L_DEBUG, "Interval end is " << intervalEnd2 << " millis" ,
            SPL_OPER_DBG);
    }

    . . .
    // Iterate through the tuples of the partition
    WindowType::DataType & data = window.getWindowData(partition);
    WindowType::DataType::const_iterator it;
    for (it = data.begin(); it != data.end(); it++) {
        IPort0Type const & tuple = **it;
        // access tuple attributes
        . . . 
    }
}