Home
Developing
Learn about different Teracloud Streams tools and features to write, compile, run, and test streams applications.
Developing custom operators
Create custom operators if shipped toolkits do not provide the necessary logic or behavior needed for your stream applications.
Developing primitive operators
You develop new primitive operators by implementing them in the C++ or Java™ language. You must define an operator model that describes the syntax and semantics of the operator and implement the operator logic by creating the header and implementation code generator templates.
Developing C++ primitive operators
The SPL language supports writing two styles of primitive operators, namely: generic operators and non-generic operators.
Advanced code generation
When you create generic operators, you can use advanced code generation techniques for more operator customization and control.
Generating C++ code from an SPL expression tree
An SPL expression tree represents the SPL expression for a parameter or output expression in a form that can be easily traversed in a C++ primitive operator. To use SPL expression trees, the context of the primitive operator must include an <splExpressionTree [param="true|false"] [output="true|false"] [cppCode="true|false"]/> element.

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Learn about different Teracloud Streams tools and features to write, compile, run, and test streams applications.
- Development concepts
  Development of stream applications consists of several components such as operators, streams, tuples, Streams Processing Language, toolkits, and more.
- Developing stream applications
  Try out tutorials and explore details about stream application development, SPL features, and best practices.
- Developing native functions
  Extend SPL's computational capabilities by creating native functions written in C++ or Java.
- Developing custom operators
  Create custom operators if shipped toolkits do not provide the necessary logic or behavior needed for your stream applications.
  - Types of operators
    When developing new operators, developers need to choose the kind of operator to implement based on the task they want to accomplish, the programming language they want to use and the trade-offs they are willing to make between generality and ease of implementation.
  - Primitive operator artifacts
    Each primitive operator is stored in a directory whose name follows the format ([:alnum:]+\.)*[:alnum:]+ where the last part ([:alnum:]+) is the operator name.
  - Developing primitive operators
    You develop new primitive operators by implementing them in the C++ or Java™ language. You must define an operator model that describes the syntax and semantics of the operator and implement the operator logic by creating the header and implementation code generator templates.
    - Developing C++ primitive operators
      The SPL language supports writing two styles of primitive operators, namely: generic operators and non-generic operators.
      - Operator implementation
        You use skeleton template files that are generated by the spl-make-operator script as the basis for operator implementations.
      - Advanced operator implementation
        Advanced operator implementation techniques provide additional runtime operator control.
      - Advanced code generation
        When you create generic operators, you can use advanced code generation techniques for more operator customization and control.
        Code sharing
        To share code between different operators in a toolkit, use one of the supported sharing methods.
        Error reporting
        SPL code generation APIs provide four different families of functions that are related to error reporting. These functions are the print, warn, error, and exit family of functions that are supplied by the SPL::CodeGen module.
        Type inspection
        The SPL code generation APIs include various type inspection functions, which are supplied by the SPL::CodeGen::Type module.
        Checking schemas
        Some primitive operators require that the output port has the same schema as the input port. To comply with this requirement, you must check the schema.
        Helper class generation
        When you develop generic operators, you often need to emit a class, which contains member variables with a list of types. The SPL code generation framework provides an emitClass function within the SPL::CodeGen module for this purpose.
        Parameter helpers
        When you write generic operators, you often must obtain the list of types and expressions that are associated with the parameter expressions. The SPL code generation framework provides helper functions within the SPL::CodeGen module to simplify this task.
        Output tuple helpers
        During code generation for generic operators, you often must initialize or assign an output tuple that is based on output attribute assignments that are captured in the operator instance model. SPL provides helper functions within the SPL::CodeGen module simplify this task.
        Window handling
        The SPL code generation framework provides code generation APIs to ease the common tasks that are associated with window handling.
        Accessing paths at compile time
        The SPL code generation APIs provide the methods to access special paths at code generation time.
        Generating C++ code from an SPL expression tree
        An SPL expression tree represents the SPL expression for a parameter or output expression in a form that can be easily traversed in a C++ primitive operator. To use SPL expression trees, the context of the primitive operator must include an <splExpressionTree [param="true|false"] [output="true|false"] [cppCode="true|false"]/> element.
    - Developing Java™ primitive operators
      To create a Java™ primitive operator inside an SPL toolkit, you must provide an operator model. The process is similar to how you provide operator models for C++ primitive operators.
    - Developing Python primitive operators
      To create a Python primitive operator inside an SPL toolkit, you must provide an operator model. The process is similar to how you provide operator models for C++ primitive operators.
  - Creating C++ libraries for operators and native functions
    To enable operators and native functions, you must compile with the appropriate include file paths and options and link any shared libraries against the correct runtime libraries.
  - Best practices
    Use these tips to write operators that perform effectively in their application, and in other applications.
  - Performance recommendations
    These tips help you to improve performance of your primitive operators in C++.
- Developing custom toolkits
  Bundle and reuse custom functions and operators across several stream applications by creating custom toolkits.
- Working with files
  Applications and operators can read and write to files in several ways. If relative paths are used, the data directory is used as the root of relative paths. To enable logic to access the runtime application or toolkit directory hierarchies, language-specific functions can be used.
- Enabling Streams data exchange
  Teracloud® Streams provides a data exchange REST API for inserting and retrieving tuples within a job to easily integrate with other data services and external applications. Stream applications can enable the data exchange feature by using one or more Endpoint operators.
- Debugging stream applications
  Debug stream applications using the interactive, command line-based Streams Debugger (sdb).
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Generating C++ code from an SPL expression tree

An SPL expression tree represents the SPL expression for a parameter or output expression in a form that can be easily traversed in a C++ primitive operator. To use SPL expression trees, the context of the primitive operator must include an <splExpressionTree [param="true|false"] [output="true|false"] [cppCode="true|false"]/> element.

Each node in an SPL expression tree can be examined to determine the SPL type, the expression kind, and the C++ code for the node (if cppCode="true" is specified). For each expression kind, the associated information can be returned. An example is the right side and left side expressions for a binary operator, or the identifier in an identifier expression.

A subclass of ExpressionTreeVisitor (ExpressionTreeCppGenVisitor) is provided to walk the SPL expression tree and use the cppCode fields to regenerate the C++ code. To use ExpressionTreeCppGenVisitor, the SPL expression tree must be created with <splExpressionTreeMode cppCode="true"> in the operator model for the primitive operator. This code causes the generated C++ code for the SPL expression to be placed in the SPL expression tree in a form that can be used to generate code.

For example,

my $g = SPL::Operator::Instance::ExpressionTreeCppGenVisitor->new();
$g->visit ($expn->getSPLExpressionTree())
my $cppCode = $g->getCppCode();
# $cppCode contains a C++ expression that will evaluate to the expression.

To generate C++ code, replacing references to custom output functions, see the following code:

package Gen;
use base qw(SPL::Operator::Instance::ExpressionTreeCppGenVisitor);
sub visit_CallExpression($$)
{
    my ($self, $expn) = @_;
    my $fcnName = $expn->getFunctionName();
    if ($fcnName =~ /^MatchRegex::(.*)/) {
        $fcnName = $1;
        if ($fcnName eq 'Count') {
            $self->append('(CEP$COUNT+pmatch._Count)');
        } elsif ($fcnName eq "Any") {
            		# gen different code here
	 }
    	return;
    }

   $self->SUPER::visit_CallExpression($expn);
}

The call $self->append(string) is used to generate the code to represent the expression. The generated code must have the same C++ type as the replaced code. To parse a subexpression, you can use:

my $subExpn = $self->new();
$subExpn->visit($a);
$self->append($subExpn->getCppCode());