Home
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
Developing toolkits and operators
You can create custom toolkits and operators for your applications.
Developing toolkits
Toolkits contain the building blocks for stream applications. In addition, you can add specialized toolkits or create your own toolkits.
Creating toolkits
A toolkit is created by organizing all the artifacts that are related to operators (composite and primitive) and functions (native or SPL) into namespace directories. Then, running the spl-make-toolkit program to index the toolkit.
Globalizing a toolkit
You can load and format string resources for different locales using the tools that are provided.
Creating a unique API per message
To generate a unique API per message, you must tag the <trans-unit> elements in the resource file with the extraData attribute. This attribute represents the name of a C++ macro or Perl subroutine.

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
- Development concepts
  Development of stream applications consists of several components such as operators, streams, tuples, Streams Processing Language, toolkits, and more.
- Writing stream applications
  Teracloud® Streams provides features and functions to help you write your applications to fulfill your business needs.
- Enabling Streams data exchange
  Teracloud® Streams provides a data exchange REST API for inserting and retrieving Streams job data. You can add one or more endpoint operators to your Streams application to enable Streams data exchange. Providing a standard REST interface for Streams data exchange eases the integration of Streams data with other data services and externally hosted applications.
- Compiling stream applications
  The Streams Processing Language (SPL) Compiler is called sc and is included with the Teracloud® Streams installation package. You compile SPL to create files to run on your Teracloud Streams instance.
- Performance improvements for stream applications
  You can improve the processing of your stream application by following these guidelines and tips.
- Debugging stream applications
  The Streams Processing Language (SPL) Streams Debugger (sdb) provides a command line-based interactive debugger for debugging stream applications.
- Developing toolkits and operators
  You can create custom toolkits and operators for your applications.
  - Developing toolkits
    Toolkits contain the building blocks for stream applications. In addition, you can add specialized toolkits or create your own toolkits.
    - Application bundle files
      An application bundle file is a single, relocatable file that contains all the artifacts that are needed to run your application. You submit a application bundle file to run on an instance.
    - Toolkit structure
      A toolkit is organized as a directory in the file system and contains two kinds of artifacts, which are authored and generated. A toolkit information model file (info.xml) contains information about the toolkit, including its name.
    - Creating toolkits
      A toolkit is created by organizing all the artifacts that are related to operators (composite and primitive) and functions (native or SPL) into namespace directories. Then, running the spl-make-toolkit program to index the toolkit.
      - Toolkit information model file
        The toolkit information model is an optional file called info.xml. It contains the name, description, and version information about the toolkit. It also contains any dependencies that might exist on other toolkits, in the form of a list of toolkit names and versions or range of versions required.
      - Creating toolkit artifacts for use on multiple platforms
        Teracloud® Streams supports a varied set of hardware architectures and Linux™ distributions such as Red Hat Enterprise Linux (RHEL). If you plan to deploy a toolkit on multiple platforms, you can create toolkit artifacts that simplify the management of the various C++ libraries. These examples show how to maintain a common infrastructure that works across multiple Linux distributions and hardware platforms.
      - Mixed-mode SPL/Perl sources
        You often must parameterize application composition when you create parallel subgraphs, conditional segments, and variable length pipelines. The SPL compiler provides a Perl-based preprocessor for this purpose. SPL files that are authored with the preprocessor directives are named with the .splmm extension and are called mixed-mode SPL programs.
      - Globalizing a toolkit
        You can load and format string resources for different locales using the tools that are provided.
        The <resources> element
        To create a toolkit that loads and formats string resources for different locales, you must add a <resources> element to the toolkit information model file (info.xml).
        Creating and updating a resource file
        To load and format strings for different locales, you must create and update the resource file for the toolkit with translatable strings.
        Accessing resources from toolkit artifacts
        You can access resources, locale-sensitive data, from toolkit artifacts.
        Creating a unique API per message
        To generate a unique API per message, you must tag the <trans-unit> elements in the resource file with the extraData attribute. This attribute represents the name of a C++ macro or Perl subroutine.
        Substituting values in a message
        A substitution value is a value that is substituted into the message at format time to replace a specific placeholder.
      - Versioning guidelines
        Use a version scheme for toolkits that you develop. The toolkit version might be synced up with the version of a wrapped library, or it might depend on the feature set provided by the toolkit.
      - Packaging a toolkit
        You package a toolkit to allow other SPL application developers to access the operators and functions that are defined in the toolkit.
      - Installing a toolkit
        You install SPL toolkits by copying the toolkit directory to a location on a file system accessible to where Teracloud® Streams is installed.
      - Directing the compiler to the toolkit directory
        To use the operators and functions that are defined in a toolkit, you first need to point the SPL compiler to the toolkit root directory.
      - Documenting your SPL toolkits
        You can use SPLDOC markup and the spl-make-doc command to generate rich text documentation for your SPL toolkits.
    - Native functions
      Native functions are written in C++ or Java™ and are invoked from SPL by using the same syntax that invokes an SPL function.
    - Supporting multiple versions with a single toolkit
      Toolkits can support multiple versions of Teracloud® Streams with a minimum number of source files, if you follow the guidelines provided.
  - Developing operators
    You can develop many stream applications with only the operators available in the SPL standard toolkit or in the specialized toolkits. If these toolkits do not provide all of the function you need to implement your application, you can develop new operators as required.
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Creating a unique API per message

To generate a unique API per message, you must tag the <trans-unit> elements in the resource file with the extraData attribute. This attribute represents the name of a C++ macro or Perl subroutine.

Procedure

To generate a unique API per message, tag the <trans-unit> elements in the resource file with the extraData attribute.

Example

The following is an example of a definition.

#define MY_TOOLKIT_MESSAGE_1(p0, p1) \
  (::SPL::FormattableMessage2<RuntimeMessageFormatter,int64_t,int64_t>("MyToolkit", "MyResource", "en_US", "MSG0001", "Tuple "{0,number,integer}" value is "{1,number,integer}".", p0, p1))

You can then use the MY_TOOLKIT_MESSAGE_1 macro in your SPL application as follows:

std::string str = MY_TOOLKIT_MESSAGE_1 (1, 2);

A comparable construct is created in Perl that can be used with some existing helper functions to access the string. For example:

sub MY_TOOLKIT_MESSAGE_1($$)
{
  my $defaultText = <<STOP;
  English/default: Tuple "{0,number,integer}" value is "{1,number,integer}".
  STOP
  return SPL::Helper::SPLFormattedMessage($toolkitRoot, "MyToolkit", "MyResource", "en_US", "MSG0001", \$defaultText, @_);
}

my $string = MY_TOOLKIT_MESSAGE_1 (1, 2);

The default text is part of the message API so that it can be used if the resource bundle cannot be found at message load time. The default text is in the language in the XLIFF file that is specified by the default attribute of the <lang> element. If the PE is running in the locale that is specified by the default attribute, then the default text is used so that resource loading does not occur.