Home
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
Developing toolkits and operators
You can create custom toolkits and operators for your applications.
Developing toolkits
Toolkits contain the building blocks for stream applications. In addition, you can add specialized toolkits or create your own toolkits.
Toolkit structure
A toolkit is organized as a directory in the file system and contains two kinds of artifacts, which are authored and generated. A toolkit information model file (info.xml) contains information about the toolkit, including its name.
Primitive operator artifacts
Each primitive operator is stored in a directory whose name follows the format ([:alnum:]+\.)*[:alnum:]+ where the last part ([:alnum:]+) is the operator name.

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
- Development concepts
  Development of stream applications consists of several components such as operators, streams, tuples, Streams Processing Language, toolkits, and more.
- Writing stream applications
  Teracloud® Streams provides features and functions to help you write your applications to fulfill your business needs.
- Enabling Streams data exchange
  Teracloud® Streams provides a data exchange REST API for inserting and retrieving Streams job data. You can add one or more endpoint operators to your Streams application to enable Streams data exchange. Providing a standard REST interface for Streams data exchange eases the integration of Streams data with other data services and externally hosted applications.
- Compiling stream applications
  The Streams Processing Language (SPL) Compiler is called sc and is included with the Teracloud® Streams installation package. You compile SPL to create files to run on your Teracloud Streams instance.
- Performance improvements for stream applications
  You can improve the processing of your stream application by following these guidelines and tips.
- Debugging stream applications
  The Streams Processing Language (SPL) Streams Debugger (sdb) provides a command line-based interactive debugger for debugging stream applications.
- Developing toolkits and operators
  You can create custom toolkits and operators for your applications.
  - Developing toolkits
    Toolkits contain the building blocks for stream applications. In addition, you can add specialized toolkits or create your own toolkits.
    - Application bundle files
      An application bundle file is a single, relocatable file that contains all the artifacts that are needed to run your application. You submit a application bundle file to run on an instance.
    - Toolkit structure
      A toolkit is organized as a directory in the file system and contains two kinds of artifacts, which are authored and generated. A toolkit information model file (info.xml) contains information about the toolkit, including its name.
      - Top-level toolkit artifacts
        Two files contain toolkit metadata, the toolkit.xml file and the info.xml file. Both files are located under the toolkit root directory.
      - Primitive operator artifacts
        Each primitive operator is stored in a directory whose name follows the format ([:alnum:]+\.)*[:alnum:]+ where the last part ([:alnum:]+) is the operator name.
      - Composite operator artifacts
        Composite operators are defined in .spl ﬁles. These files are in directories that correspond to namespaces.
      - Native function artifacts
        Toolkits can contain C++ native functions and Java™ native functions. Artifacts of native functions are in the native.function directory.
      - SPL function artifacts
        SPL functions are defined in .spl ﬁles that are located within directories that correspond to namespaces.
      - Globalization artifacts
        The globalization artifacts are in the impl/nl directory of the toolkit.
    - Creating toolkits
      A toolkit is created by organizing all the artifacts that are related to operators (composite and primitive) and functions (native or SPL) into namespace directories. Then, running the spl-make-toolkit program to index the toolkit.
    - Native functions
      Native functions are written in C++ or Java™ and are invoked from SPL by using the same syntax that invokes an SPL function.
    - Supporting multiple versions with a single toolkit
      Toolkits can support multiple versions of Teracloud® Streams with a minimum number of source files, if you follow the guidelines provided.
  - Developing operators
    You can develop many stream applications with only the operators available in the SPL standard toolkit or in the specialized toolkits. If these toolkits do not provide all of the function you need to implement your application, you can develop new operators as required.
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Primitive operator artifacts

Each primitive operator is stored in a directory whose name follows the format ([:alnum:]+\.)*[:alnum:]+ where the last part ([:alnum:]+) is the operator name.

For instance, for an operator with name Delay, some valid directory names are Delay, spl.utility.Delay, and spl.Delay. Any prefix that comes before the operator name is taken as the namespace, excluding the trailing period. Furthermore, additional parent directories of the form ([:alnum:]+\.)*[:alnum:]+ can appear and are taken as part of the namespace. For instance, all the following operator directory names are associated with the spl.utility namespace spl/utility.Delay, spl.utility/Delay, spl/utility/Delay, and spl.utility.Delay. This flexibility allows the developers to decide on the number of levels they want to manage in the directory structure. For example, com.ibm.spl/file/adapter is preferable to both com/ibm/spl/file/adapter (too deep) and com.ibm.spl.file.adapter (no hierarchy). For a namespace, only one path is allowed per toolkit. It is an error to use both spl/utility and spl.utility in the same toolkit.

The operator directory contains three types of artifacts.

The operator model is an XML file that describes syntactic and semantic properties of the operator. It is authored by the developer and is mandatory. See Developing C++ primitive operators for details. The operator model is named as <oper-name>.xml, such as Delay.xml.
The code generator templates are the header and the implementation files that contain the operator logic. They are authored by the developer and are mandatory. They are named as <oper-name>_h.cgt and <oper-name>_cpp.cgt, such as Delay_h.cgt and Delay_cpp.cgt.
The code generators are the code generators, in the form of Perl modules, which are derived from the code generator templates. They are created either as part of the toolkit indexing by the spl-make-toolkit program or by calling the spl-make-operator script as described in Developing C++ primitive operators. They are named <oper-name>_h.pm and <oper-name>_cpp.pm, such as Delay_h.pm and Delay_cpp.pm.