Operators: Toolkits

Operators

ARIMA2: The ARIMA2 operator implements the autoregressive integrated moving average (ARIMA) modeling algorithm.
ASN1Encode: The ASN1Encode operator creates ASN.1 DER-encoded data from XML documents that fit to the provided ASN.1 specification.
ASN1Parse: The ASN1Parse operator parses a binary data stream that contains ASN.1-encoded data, extracts parts of the data, and sends the data as tuples to downstream operators.
Aggregate: The Aggregate operator is used to compute user-specified aggregations over tuples that are gathered in a window.
AnomalyDetector: The AnomalyDetector operator can detect anomalous subsequences in an incoming data stream.
AutoForecaster2: The AutoForecaster2 operator is a forecasting operator, which detects the algorithm that best suits the input time series data in real time and forecasts future time series.
AvroToJSON: This operator converts binary Avro messages and optionally message keys into a JSON string.
BATS: The BATS operator is a forecasting operator, which can be used to do long-term forecasting of regular time series with complex seasonality by using the BATS algorithms.
BWListTagger: The BWListTagger operator loads lists of domains and IP addresses into a lookup table.
Barrier: The Barrier operator is used to synchronize tuples from two or more streams.
Beacon: The Beacon operator is a utility source that generates tuples on an ad hoc basis.
BloomFilter: The BloomFilter operator detects duplicate tuples in a memory-efficient way.
BoundedAnomalyDetector: The BoundedAnomalyDetector operator detects anomalys (outliers) in a timeseries.
Buffered: Connect subscriber to publisher using a buffer.
CSVParse: The CSVParse operator parses an input line of comma-separated values (CSV) that is received in an rstring type input tuple attribute, splits the line into fields (also called items), and assigns the field values to attributes in the output port schema.
CharacterTransform: The CharacterTransform operator is used to convert from one encoding in blob to another encoding in blob.
Compress: The Compress operator is used to compress data in a blob and generate blob output.
ControlNotificationSource: The ControlNotificationSource operator provides a connector between control events and data streams by turning control notifications into tuples.
CrossCorrelate2: In signal processing, cross-correlation is a measure of similarity of two time series as a function of a time-lag applied to one of them.
CrossCorrelateMulti: In signal processing, cross-correlation is a measure of similarity of two time series as a function of a time-lag applied to one of them.
CrossDataCenterFailover
Custom: The Custom operator is a special logic-related operator that can receive and send any number of streams and does not do anything by itself.
DHCPMessageParser: DHCPMessageParser is an operator for the Streams product that parses individual fields of DHCP messages received in input tuples, and emits tuples containing message data.
DNSMessageParser: DNSMessageParser is an operator for the Streams product that parses individual fields of DNS messages received in input tuples, and emits tuples containing message data.
DNSPacketDPDKSource: DNSPacketDPDKSource is an operator for the Streams product that receives network packets from a supported ethernet adapter.
DNSPacketFileSource: DNSPacketFileSource is an operator for the Streams product that reads prerecorded DNS packets from 'packet capture (PCAP)' files, parses the network headers and the fields of the DNS message, and emits tuples containing packet data.
DNSPacketLiveSource: DNSPacketLiveSource is an operator for the Streams product that captures live DNS packets from an ethernet interface, parses their network headers and the fields of the DNS message, and emits tuples containing packet data.
DNSTunneling: The DNSTunneling operator analyzes DNS response traffic and reports suspicious behaviour that may indicate the presence of DNS tunneling in the network.
DPSAux: The DPSAux operator's purpose is to support primitive Java operators that desire to use the DPS toolkit Java APIs.
DSPFilter2: The Digital Signal Processing (DSP) filter operator performs a digital filtering operation on an input time series.
DSPFilterFinite: The Digital Signal Processing (DSP) filter operator performs a digital filtering operation on an input time series.
DWT2: The DWT2 operator applies a discrete wavelet transform (DWT) on a vector time series.
DeDuplicate: The DeDuplicate operator suppresses duplicate tuples that are seen within a specified time period.
Decompress: The Decompress operator decompresses data in blob input and generates blob output that contains the decompressed data.
Delay: The Delay operator is used to delay a stream by an amount while it keeps the inter-arrival times of tuples and punctuation intact.
Deserializer: De-Serialization operator.
Direct: Directly connect subscriber to publisher.
DirectoryScan: The DirectoryScan operator watches a directory, and generates file names on the output, one for each file that is found in the directory.
DirectoryWatch: The DirectoryWatch operator monitors configured directories and reports changes in them.
Distribution: The Distribution operator calculates the quartile distribution for an input time series.
DomainProfiling: The DomainProfiling operator analyzes DNS response traffic and reports whether or not the behaviour of the domain is suspicious.
DpsGetTTLWithReconnect: Uses the dpsGetTTL function to read key-value pairs from DPS (of any type).
DpsPutTTLWithReconnect: Uses the dpsPutTTL function to write key-value pairs to Redis (of any type).
DynamicFilter: The DynamicFilter is a version of the Filter operator that can decide at run time which input tuples are passed through, based on control input it receives.
ElasticsearchIndex: The ElasticsearchIndex operator receives incoming tuples and stores the tuple attributes name-value pairs as JSON documents in a specified index of an Elasticsearch database.
EndpointSink: The EndpointSink operator enables data exchange between a Streams job and external clients.
EndpointSource: The EndpointSource operator enables data exchange between a Streams job and external clients.
ExceptionCatcher: DEPRECATED: The com.teracloud.streams.teda.ExceptionCatcher operator is deprecated and the @catch annotation should be used instead. The deprecated operator might be removed in a future release.
Export: The Export operator sends a stream from the current application, making it available to Import operators of applications that are running in the same streaming middleware instance.
ExtractDomain: This operator extracts the domain+tld portion of a fqdn in a tuple, and sets a pre-existing field in the tuple to that value.
FFT: The FFT operator applies a transformation of a time series from time domain into frequency domain.
FMPFilter: The FMPFilter operator is an adaptive faded-memory polynomial filter.
FTPCommand: The FTPCommand operator can run an ftp command at a remote host by using the FTP, FTPS, or SFTP protocols.
FTPPutFile: The FTPPutFile operator can put a local file on a remote host by using FTP, FTPS, or SFTP protocols.
FTPReader: The FTPReader operator is a source operator that can scan a directory at a remote host or receive files from a remote host by using the FTP, FTPS, or SFTP protocols.
FileSink: The FileSink operator writes tuples to a file.
FileSource: The FileSource operator reads data from a file and produces tuples as a result.
Filter: The Filter operator removes tuples from a stream by passing along only those tuples that satisfy a user-specified condition.
FilteredSubscribe: Filtered subscribe to a topic.
FlightPathEncounter: This operator tracks flying objects and calculates possible encounters between the objects in the future.
Format: The Format operator is similar to the FileSink, TCPSink, and UDPSink operators, in that it converts SPL tuples into formatted data.
FunctionEvaluator: The FunctionEvaluator operator applies a function to each value in a time series.
Functor: The Functor operator is used to transform input tuples into output ones, and optionally filter them as in a Filter operator.
GAMLearner: The GAMLearner operator applies the generalized additive model (GAM) algorithm to categorical or continuous time series data.
GAMScorer: The GAMScorer operator applies a generalized additive model to score the input time series values.
GMM: The GMM operator uses a Gaussian mixture model to estimate the probability density function (a smoothed histogram) of a time series.
Gate: The Gate operator is used to control the rate at which tuples are passed through.
Generator: The Generator operator generates a sine, triangular, sawtooth, or a pulse train representation of a time series.
Geofence: The Geofence operator is used to manage user-defined geographical regions called geofences and provide information as entities move in and out of of these regions.
GeometryMapMatcher: The GeometryMapMatcher operator calculates edges of a road network that are within reach of a given area of interest (AOI).
HBASEDelete: The HBASEDelete operator deletes an entry, an entire row, a columnFamily in a row, or a columnFamily, columnQualifier pair in a row from an HBase table.
HBASEGet: The HBASEGet operator gets tuples from an HBase table and puts the result in the output stream attribute specified in the outAttrName parameter.
HBASEIncrement: The HBASEIncrement operator increments the specified HBase entry.
HBASEPut: The HBASEPut operator puts tuples into an HBase table.
HBASEScan: The HBASEScan operator scans an HBase table.
HDFS2DirectoryScan: The HDFS2DirectoryScan operator scans a Hadoop Distributed File System directory for new or modified files.
HDFS2FileCopy: The HDFS2FileCopy operator copies files from a HDFS file system to the local disk and also in the opposite direction from a local disk to the HDFS file system.
HDFS2FileSink: The HDFS2FileSink operator writes files to a Hadoop Distributed File System.
HDFS2FileSource: The HDFS2FileSource operator reads files from a Hadoop Distributed File System (HDFS)
HTTPBLOBInjection: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the EndpointSource operator from the SPL toolkit instead. Embeds a Jetty web server to allow HTTP or HTTPS POST requests to submit a tuple on its output ports. Each output port corresponds to a unique URL comprising the operator name and the port index.
HTTPGetJSONContent: This operator is deprecated. Use HTTPRequest operator instead.
HTTPGetStream: This operator is deprecated. Use HTTPRequest operator instead.
HTTPGetXMLContent: Periodically connects to an HTTP endpoint to GET XML content as a single tuple.
HTTPJSONInjection: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the EndpointSource operator from the SPL toolkit instead. Embeds a Jetty web server to allow HTTP or HTTPS POST requests with mime type application/json to submit a tuple on its output ports. Each output port corresponds to a unique URL comprising the operator name and the port index.
HTTPParallelRequest: The operator HTTPParallelRequest issues HTTP requests in parallel asynchronously from the input stream.
HTTPPost: This operator is deprecated. Use HTTPRequest operator instead.
HTTPRequest: Issue an HTTP request of the specified method for each input tuple.
HTTPRequestProcess: Operator accepts a web request and generates corresponding response.
HTTPTupleInjection: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the EndpointSource operator from the SPL toolkit instead. Embeds a Jetty web server to allow HTTP or HTTPS POST requests to submit a tuple on its output ports. Each output port corresponds to a unique URL comprising the operator name and the port index.
HTTPTupleView: REST HTTP or HTTPS API to view tuples from windowed input ports.
HTTPXMLInjection: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the EndpointSource operator from the SPL toolkit instead. Embeds a Jetty web server to allow HTTP or HTTPS POST requests to submit a tuple on its output ports. Each output port corresponds to a unique URL comprising the operator name and the port index.
HTTPXMLView: REST API to view tuples from input ports.
Hangout: The Hangout operator is used to detect a basic pattern about a moving entity's location called a hangout.
HoltWinters2: The HoltWinters2 operator is a forecasting operator, which uses the Holt-Winters algorithm to do long-term forecasting.
HoltWinters3: The HoltWinters3 operator is a forecasting operator, which can be used to do long-term forecasting of seasonal regular time series by using different variants of the Holt-Winters algorithms.
HostProfiling: The HostProfiling operator analyzes DNS response traffic and reports whether or not the behaviour of the hosts are suspicious.
HttpPost: This operator posts/sends the incoming tuple's text or binary content to a HTTP or HTTPS persistent (Keep-Alive) or non-persistent endpoint specified in the operator parameter named url.
IPASNEnricher: This operator enriches IPv4 and IPv6 addresses with ASN data.
IPAddressLocation: IPAddressLocation is an operator for the Streams product that finds the geographical location of IP addresses received in input tuples, based on the subnets they are part of, and emits output tuples containing the country, state or province, city, latitude, and longitude of the subnets.
IPFIXMessageParser: IPFIXMessageParser is an operator for the Streams product that parses individual fields of IPFIX messages received in input tuples, and emits tuples containing message data.
IPFilter: This operator filters IPv4 addresses based on a list of IPv4 addresses input in CIDR format.
IPSpatialEnricher: This operator enriches IPv4 and IPv6 addresses with geospatial data.
Import: The Import operator receives tuples from streams that are made available by Export operators of applications that are running in the same streaming middleware instance.
IncrementalInterpolate: The IncrementalInterpolate operator calculates missing values in a time series.
InetSource: The InetSource operator periodically retrieves data from network servers and generates a stream from those contents.
JDBCRun: The JDBCRun operator runs a user-defined SQL statement that is based on an input tuple.
JMSSink: The JMSSink operator creates messages from Streams tuples and writes the messages to a WebSphere MQ or an Apache Active MQ queue or topic.
JMSSource: The JMSSource operator reads data from a WebSphere MQ or an Apache Active MQ queue or a topic and creates tuples from the read data.
JSONToAvro: This operator converts JSON strings into binary Avro messages.
JSONToTuple: Converts JSON strings into SPL tuples.
JavaOp: The JavaOp operator is used to call out to operators implemented in Java by using the Java Operator API.
JobControlPlane: The JobControlPlane composite operator must be added to any application with a consistent region.
Join: The Join operator is used to correlate tuples from two streams that are based on user-specified match predicates and window configurations.
KMeansClustering: Cluster analysis is a popular technique used to find natural grouping of a set of objects.
KafkaConsumer: The KafkaConsumer operator is used to consume messages from Kafka topics.
KafkaProducer: The KafkaProducer operator is used to produce messages on Kafka topics.
Kalman: The Kalman operator runs an adaptive filter on a time series and can used for tracking, smoothing, adaptation.
LPC: The Linear Predictive Coding (LPC) operator uses an autoregressive (AR) model to predict values in a time series.
MQTTSink: The MQTTSink operator creates a message for every tuple it receives on its input port and publishes the message to an MQTT server.
MQTTSource: The MQTTSource operator subscribes to MQTT topics and receives messages when they are published to subscribed topics.
MapStore: The MapStore operator stores geometry data for the edge network in a shared map store, which the PointMapMatcher operator can access.
MatchRegex: The MatchRegex operator matches a regular expression pattern over the sequence of input tuples to detect composite events.
MessageHubConsumer: The MessageHubConsumer operator is used to consume records from the IBM Event Streams cloud service.
MessageHubProducer: The MessageHubProducer operator is used to consume records from the IBM Event Streams cloud service.
MetricsSink: The MetricsSink operator reads tuples and populates metrics with the values.
NetflowMessageParser: NetflowMessageParser is an operator for the Streams product that parses individual fields of Netflow messages received in input tuples, and emits tuples containing message data.
Normalize: The Normalize operator incrementally estimates the means and variance and can normalize the time series to zero means and unit variance.
OSMCorrelator: The OSMCorrelator correlates information read by the OSMXMLGeometrySource to map generated edge IDs back to the original Way ID from the OpenStreetMaps map (.osm file).
OSMPointMatcher: The OSMPointMapMatcher composite analyzes time based object location data to heuristically project points on a network of roads specified by an OpenStreetMaps XML map file.
OSMXMLGeometrySource: OSMXMLGeometrySource reads in an OSM XML map file (.osm file) and generates geometry tuples for consumption by the PointMapMatcher and PathMapMatcher operators.
ObjectStorageScan: Scans for specified key name pattern on a object storage.
ObjectStorageSink: Operator writes objects to S3 compliant object storage.
ObjectStorageSource: Operator reads objects from S3 compliant object storage.
PMMLScoring: The PMMLScoring operator scores tuple data it receives on the first input port against a previously loaded PMML model.
PSAX: The PSAX operator is capable of providing a symbolic representation of real-valued time series data.
PacketContentAssembler: PacketContentAssembler is an operator for the Streams product that reassembles application flows (such as SMTP, FTP, HTTP, and SIP) and files (such as GIF, JPEG, HTML, and and PDF) from raw network packets received in input tuples, and emits tuples containing the reassembled content.
PacketDPDKSource: PacketDPDKSource is an operator for the Streams product that receives network packets from a supported ethernet adapter.
PacketFileSink: PacketFileSink is an operator for the Streams product that writes network packets into 'packet capture (PCAP)' files.
PacketFileSource: PacketFileSource is an operator for the Streams product that reads prerecorded network packets from 'packet capture (PCAP)' files, parses the network headers, and emits tuples containing packet data.
PacketLiveSource: PacketLiveSource is an operator for the Streams product that captures live network packets from an ethernet interface, parses their network headers, and emits tuples containing packet data.
Pair: The Pair operator is used to pair tuples from two or more streams.
Parse: The Parse operator is similar to the FileSource, TCPSource, and UDPSource operators, in that it transforms input data in a raw form into well-structured SPL tuples.
PointMapMatcher: The PointMapMatcher operator analyzes time based object location data to heuristically project points on a network of edges, as the original data may be imprecise.
PredictiveBlocklisting: The PredictiveBlocklisting operator analyzes the feature vector produced by the PredictiveBlocklistingFE operator using an SPSS model.
PredictiveBlocklistingFE: The PredictiveBlocklistingFE operator ingests DNS response traffic and outputs a feature vector that is used by the PredictiveBlocklisting operator.
Publish: Publish a stream to a topic.
Punctor: The Punctor operator is used to transform input tuples into output ones and add window punctuation to the output.
PythonOp: The PythonOp operator is used to call out to operators implemented in Python by using the Python Operator API.
QRadarSink: This operator allows Streams applications to send syslog messages to a QRadar host.
RLSFilter: The Recursive Least Squares (RLS) is linear regression estimation algorithm that learns to predict a target time series, given inputs.
RScript: The RScript operator maps input tuple attributes to objects that can be used in R commands.
RabbitMQSink: This operator acts as a RabbitMQ producer, sending messages to a RabbitMQ broker.
RabbitMQSource: This operator acts as a RabbitMQ consumer, pulling messages from a RabbitMQ broker.
ReSample: The ReSample operator changes the sampling rate of a time series.
ReadMail: The ReadMail operator reads e-mails from an imap server and ingests one tuple for each received e-mail.
RegionMatch: The RegionMatch operator stores geographical regions (also called Geofences) together with a set of attributes per region.
ReplayableStart: The ReplayableStart operator forwards tuples and punctuations from its input port to its output port.
RtpDecode: The RtpDecode Operator converts an input blob with 8 bit mylaw encoded PCM samples into uncompressed 16 bit samples.
S3ObjectStorageScan: Operator scans for specified key name pattern on a S3 compliant object storage.
S3ObjectStorageSink: Operator writes objects to S3 compliant object storage.
S3ObjectStorageSource: Operator reads objects from S3 compliant object storage.
STD2: The Seasonal Trend Decomposition (STD) operator uses the Loess algorithm to decompose an input time series into three components: the season, the trend and the residuals.
ScheduledBeacon: The ScheduledBeacon operator is a utility source that generates tuples on-the-ﬂy.
SendMail: This operator sends out an e-mail, when a tuple arrives at the input port.
Serializer: Serialization operator.
SimpleExtractDomain: Composite to fill TLD of ExtractDomain operator with files scanned in tldsDir
Sort: The Sort operator is used to order tuples that are based on user-specified ordering expressions and window configurations.
SpatialGridIndex: The SpatialGridIndex operator stores objects and their location.
SpatialRouter: The SpatialRouter operator is used to split an input stream that contains geographical data into two or more output streams, based on the geographic location specified by each tuple.
Split: The Split operator is used to split a stream into one or more output streams, which are based on a user-specified split condition.
StructureParse: The StructureParse operator parses a binary data stream, which contains fixed-size structures; therefore, the data is passed in using a blob attribute.
Subscribe: Subscribe to topics using a topic filter.
Switch: The Switch operator is used to temporarily stop tuples from flowing.
TCPSink: The TCPSink operator writes data to a TCP socket in the form of tuples.
TCPSource: The TCPSource operator reads data from a TCP socket and creates tuples out of it.
TSWindowing: The TSWindowing operator can be used to isolate a portion of the signal in a specified duration.
TextFilter: This operator filters text (e.g. domain names) based on a list of text input.
ThreadedSplit: The ThreadedSplit operator splits tuples across multiple output ports to improve concurrency.
Throttle: The Throttle operator is used to pace a stream to make it flow at a specified rate.
TupleToAvro: This operator converts Streams tuples into binary Avro messages.
TupleToJSON: Converts incoming tuples to JSON Strings.
UDPSink: The UDPSink operator writes data to a UDP socket in the form of tuples (and punctuation).
UDPSource: The UDPSource operator reads data from a UDP socket and creates tuples and punctuation out of it.
Union: The Union operator merges the streams that are connected to multiple input ports into a single stream.
VAR2: The VAR2 operator tracks data movement and predicts the next expected time series by using a multivariate autoregressive model.
WebContext: Embeds a Jetty web server to provide HTTP or HTTPS REST access to files defined by the context and contextResourceBase parameters.
WebSocketInject: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the WebSocketSource operator from the com.teracloud.streams.websocket toolkit instead. Operator recieves messages from WebSocket clients and generates a tuple which is sent to streams.
WebSocketSend: NOTE: Deprecated as of version 5.0.0 (Streams version 7.2). Use the WebSocketSink operator from the com.teracloud.streams.websocket toolkit instead. Operator transmits tuples received on the input port via WebSocket protocol to connected clients.
WebSocketSendReceive: The client-based WebSocketSendReceive operator is designed to serve a dual purpose of simultaneously sending data to and receiving data from a remote WebSocket server.
WebSocketSink: The server-based WebSocketSink operator is designed to send data to the remote WebSocket clients that are connected at any given time to the WebSocket server running inside of this operator.
WebSocketSource: The server-based WebSocketSource operator is designed to receive text (plain text, JSON or XML) or binary data from the remote WebSocket and HTTP clients.
XMLParse: The XMLParse operator accepts a single input stream and generates tuples as a result.