Operator Kalman

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.teracloud.streams.timeseries/op$com.teracloud.streams.timeseries.modeling$Kalman.svg

The Kalman operator runs an adaptive filter on a time series and can used for tracking, smoothing, adaptation.

The Kalman filter algorithm is an adaptive system that tracks data movement. The operator continuously re-estimates the model by using the current time series input value after it eliminates any noisy interference.

You can provide control values to influence how the Kalman operator re-estimates the model. By using these control values, the Kalman operator can reduce noise and other inaccuracies in the data and predict values that are more accurate. For example, in GPS devices, where the Kalman algorithm is widely used to track vehicle movement, the control values can help maximize noise reduction.

The Kalman filter algorithm is a univariate algorithm that operates on scalar or vector time series. In the case of a vector time series, the operator spawns independent algorithm instances for each dimension of the timeseries vector.

Behavior in a consistent region

  • The operator cannot be the start of a consistent region. An error occurs when you compile your streams processing application.

Exceptions

The Kalman operator throws an exception in the following cases:

  • The stateVector or controlDimension parameter value is 0.
Examples

Summary

Ports
This operator has 2 input ports and 1 output port.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 15 parameters.

Required: inputTimeSeries

Optional: controlDimension, controlMatrix, initialCovariance, initialState, inputControl, inputTimestamp, observationMatrix, observationNoiseCovariance, parameterFile, partitionBy, stateNoiseCovariance, stateTransitionMatrix, stateVectorDimension, threshold

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Always - Operator always provides a single threaded execution context.

Input Ports

Ports (0)

This port consumes data for training and scoring against the model. The inputTimeSeries parameter specifies the name of the attribute on this port that contains the time series data. The accepted data types are float64 and list<float64>.

Properties

Ports (1)

This port consumes tuples to initialize the Kalman algorithm. When a tuple is received on this port, the initialization parameter values are calculated and are used to initialize, or reinitialize, the Kalman algorithm. The parameters that are used as part of the initialization process are: initialState, initialCovariance, stateTransitionMatrix, controlMatrix and observationMatrix.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Output Functions
KalmanCOF
<any T> T predictedTimeSeries()

This function returns the values of predicted time series data. The values are either a vector time series (list<float64> data type) or a univariate time series (float64 data type), depending on the data type of the input time series.

list <float64> predictedCovariance()

This function returns the covariance of the predicted state. The return type is a list<float64> value (only diagonal elements).

<any T> T predictedTimestamp()

This function returns predicted timestamp values. The return type is the same as data type of the input timestamp. An output timestamp is produced only if the input timestamp is regular. Otherwise, a timestamp object with value 0 is returned and a message is logged. The message indicates that the timestamp is not predicted because the time series is irregular.

list <float64> predictedState()

This function returns the predicted state of the model.

list<float64> estimatedState()

This function returns the current estimated state of the model for the current observation.

list<float64> estimatedCovariance()

This custom function returns the estimated covariance for the current observation. The return type is a list<float64> value (only diagonal elements).

list<float64> estimatedInnovationCovariance()

This custom function returns the estimated covariance of the innovation for the current observation.

list<float64> estimatedInnovation()

This function returns the estimated innovations for the current observation.

<any T> T estimatedFlags()

This custom function returns the estimated flags for anomaly detection, which are either a list<boolean> or boolean depending of the type of the input time series (list or scalar). A flag value of true indicates an anomaly and a flag value of false indicates normalcy.

<any T> T AsIs()

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

Ports (0)

This port submits a tuple that contains the next predicted value as determined by the Kalman algorithm. This port submits a tuple each time a prediction is made. Custom output functions are used to assigned data to the output tuple attributes. The output tuple attributes whose assignments are not specified are assigned from input attributes.

Properties

Parameters

This operator supports 15 parameters.

Required: inputTimeSeries

Optional: controlDimension, controlMatrix, initialCovariance, initialState, inputControl, inputTimestamp, observationMatrix, observationNoiseCovariance, parameterFile, partitionBy, stateNoiseCovariance, stateTransitionMatrix, stateVectorDimension, threshold

controlDimension

Specifies the dimension of the control vector. The default value is 1u.

Properties

controlMatrix

Specifies the name of the attribute that contains the control to state matrix. If the state has dimension n and the control has dimension q the control matrix has dimension nxq.

Properties

initialCovariance

Specifies the value of the initial covariance in the model. If this parameter value is not specified, the Kalman operator does not consider covariance while calculating the prediction values.

Properties

initialState

Specifies the value of the initial state of the model that includes inaccuracies introduced due to noise or other errors. The supported types are float64 and list<float64>. If this value is not specified, the first value in the input time series is used as an initial value to initialize the model.

Properties

inputControl

Specifies the name of the attribute that contains the control values. The Kalman operator uses these control values to filter noises and give more accurate results. The supported type is list<list<float64>>.

Properties

inputTimeSeries

Specifies the name of the attribute that contains the time series data in the input tuple. The supported data types are float64 and list<float64>.

Properties

inputTimestamp

Specifies the name of the attribute that contains the time series data in the input tuple. The supported data types are timestamp and uint64.

Properties

observationMatrix

Specifies the name of the attribute that contains the state of the observation matrix. If the state has dimension n and the observation has dimension m, the observation matrix has dimension mxn.

Properties

observationNoiseCovariance
Specifies the observed noise covariance. The following data types are supported:
  • list<list<float64>> for specifying all the elements in the matrix.
  • list<float64> for specifying the values of only the diagonal elements in the matrix. The rest of the elements in the matrix are assumed to be 0.
  • Single float64 value if all diagonal elements in the matrix share the same value. The rest of the matrix elements are assumed to be 0.
Properties

parameterFile

Specifies the files that contain the Kalman filter parameters. The values that are contained in the parameter file override the values that are specified as parameter values for the operator. The format of the parameter file is:

ControlDimension:n,ObserveDimension:n

Properties

partitionBy

Specifies the name of the attribute that contains the key values that are associated with the time series values in the input tuple.

Properties

stateNoiseCovariance
Specifies the value for the initial state covariance. The following data types are supported:
  • list<list<float64>> for specifying all the elements in the matrix.
  • list<float64> for specifying the values of only the diagonal elements in the matrix. The rest of the elements in the matrix are assumed to be 0.
  • Single float64 value if all diagonal elements in the matrix share the same value. The rest of the matrix elements are assumed to be 0.
Properties

stateTransitionMatrix

Specifies the value for the state transition matrix. This is an nxn matrix where n has the same value as stateDimension parameter.

Properties

stateVectorDimension

Specifies the dimension of the internal state vector. The default value is 1u.

Properties

threshold

Specifies the threshold for the anomaly detection. It determines the tolerable margin around the variance of the prediction. The default value is 1.5f.

Properties

Code Templates

Kalman

stream<${schema}> ${outputStream} = Kalman(${inputStream}) 
{
	param
		inputTimeSeries:		${timeSeriesExpression};

	output
		${outputStream}: ${outputExpression};
}

Libraries

No description for library.
Library Name: tsatapi
Library Path: ../../../impl/lib
Include Path: ../../../impl/include
No description for library.
Library Name: tstimelag
Library Path: ../../../impl/lib
Include Path: ../../../impl/include
No description for library.
Library Name: boost_serialization