Home
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Toolkits
A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
Toolkits
Toolkit com.teracloud.streams.teda 4.0.1
Developing TEDA applications
Configuring Solutions
Configuring ITE Applications
Data Processing

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Learn about different Teracloud Streams tools and features to write, compile, run, and test streams applications.
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
- APIs
  Application programming interfaces (APIs) provide functions that simplify applications development.
- Job configuration overlays
  You can specify or change configuration parameters when you use the submitjob or updateoperators operations. The settings control submission constraints, host constraints and locations, and operator status. Parameters are captured and updated in the configuration JSON file. When you change the values in the job configuration overlay file, the changes override any previously set values.
- Commands
  Teracloud® Streams provides a number of commands you use to perform tasks.
- Metrics
  Teracloud® Streams provides metrics to help evaluate the health of Teracloud Streams services, to aid in diagnosing performance issues, and to analyze throughput of requests. You can use the streamtool checkinstancemetrics, streamtool checkdomainmetrics, streamtool getdomainmetrics, and streamtool checkresourcemetrics commands to view the metrics data.
- Operator model
  An operator model is an XML document that describes the basic properties of a primitive operator.
- Streams Processing Language (SPL)
  Streams Processing Language (SPL) is a distributed data flow composition language that is used in Teracloud® Streams. SPL has primitive types, program structures, and definitions that are tailored for streaming data.
- SPL Compiler
  The SPL compiler is an incremental compiler that generates artifacts needed to run your stream application. These artifacts include, for example, C++ code for several entities in your SPL program and an application bundle file.
- Toolkits
  A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
  - Specialized toolkit requirements and restrictions
    Several of the specialized toolkits require RPMs that are not required by the Teracloud® Streams product. There are also toolkit restrictions on some platforms.
  - Toolkits
- SPLDOC markup
  You can use SPLDOC markup in the descriptions associated with the SPL artifacts in a toolkit. The spl-make-doc command generates HTML documentation from the marked up artifacts.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Data Processing

Grouping with tuple deduplication: If you want to group your tuples and get rid of duplicates in your data, you can configure your application easily to do so.

Grouping with custom correlation: If you want to group your tuples and implement your own business logic to correlate the data tuples, you can configure your application easily to do so.

Grouping with custom correlation and tuple deduplication: If you want to group your tuples, implement your own business logic to correlate the data tuples, and get rid of duplicates in your data, you can configure your application easily to do so.

Processing files without group processing (Variant A): When your data does not require record deduplication or group processing, you use the framework in its most simple configuration.

Processing files with file content based group processing (Variant B): When you must process files that contain group information as part of the tuple contents, use a configuration that bases the group split on file contents, and each tuple determines the context.

Processing files with file name based group processing (Variant C): When you have a use case where the names of files contain information to determine the group, you configure the framework to send scanned files to the group-specific processing chains.

Enabling data enrichment: When you want to enrich your incoming data with already known data, you must enable the lookup function in your application.

Using checkpointing for custom logic: If you want to preserve the state of your customizations or the deduplication over a restart of your application, you must enable checkpointing for the custom or deduplication logic or both.

Cleaning up data: The framework provides parameters to define regular automated clean-up cycles to get rid of old data, and thus spare resources.

Tapping internal data streams: If your use case needs to process the data on an extra path apart from the standard data flow, the framework provides two taps that you enable by parameters.

Rejecting tuples by using your own implementation: If you must have a special business logic to reject data records, you enable a composite to implement your rejection logic.