Home
Developing
Learn about different Teracloud Streams tools and features to write, compile, run, and test streams applications.
Developing custom operators
Create custom operators if shipped toolkits do not provide the necessary logic or behavior needed for your stream applications.
Performance recommendations
These tips help you to improve performance of your primitive operators in C++.
Multi-threading considerations
The following topic describes situations when operators can run in multi-threaded context and the related performance considerations.

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Learn about different Teracloud Streams tools and features to write, compile, run, and test streams applications.
- Development concepts
  Development of stream applications consists of several components such as operators, streams, tuples, Streams Processing Language, toolkits, and more.
- Developing stream applications
  Try out tutorials and explore details about stream application development, SPL features, and best practices.
- Developing native functions
  Extend SPL's computational capabilities by creating native functions written in C++ or Java.
- Developing custom operators
  Create custom operators if shipped toolkits do not provide the necessary logic or behavior needed for your stream applications.
  - Types of operators
    When developing new operators, developers need to choose the kind of operator to implement based on the task they want to accomplish, the programming language they want to use and the trade-offs they are willing to make between generality and ease of implementation.
  - Primitive operator artifacts
    Each primitive operator is stored in a directory whose name follows the format ([:alnum:]+\.)*[:alnum:]+ where the last part ([:alnum:]+) is the operator name.
  - Developing primitive operators
    You develop new primitive operators by implementing them in the C++ or Java™ language. You must define an operator model that describes the syntax and semantics of the operator and implement the operator logic by creating the header and implementation code generator templates.
  - Creating C++ libraries for operators and native functions
    To enable operators and native functions, you must compile with the appropriate include file paths and options and link any shared libraries against the correct runtime libraries.
  - Best practices
    Use these tips to write operators that perform effectively in their application, and in other applications.
  - Performance recommendations
    These tips help you to improve performance of your primitive operators in C++.
    - Data allocation
      Here are some tips for allocating data with performance in mind.
    - Copying data from C-style interface
      If you write a primitive operator in C++, use this code as an example of how to copy data to C-style interfaces.
    - Passing data to C-style interface
      If you write a primitive operator in C++, use this code as an example of how to pass data to C-style interfaces.
    - Copying tuples
      Here are some performance considerations when you copy tuples that are of the same or different types.
    - Multi-threading considerations
      The following topic describes situations when operators can run in multi-threaded context and the related performance considerations.
    - Fusing multiple Java™ operators into a single PE
      If multiple Java™ primitive operators and JavaOp operator invocations have compatible vmArg parameter lists, you can fuse them together into a single processing element (PE) or run them in stand-alone mode.
- Developing custom toolkits
  Bundle and reuse custom functions and operators across several stream applications by creating custom toolkits.
- Working with files
  Applications and operators can read and write to files in several ways. If relative paths are used, the data directory is used as the root of relative paths. To enable logic to access the runtime application or toolkit directory hierarchies, language-specific functions can be used.
- Enabling Streams data exchange
  Teracloud® Streams provides a data exchange REST API for inserting and retrieving tuples within a job to easily integrate with other data services and external applications. Stream applications can enable the data exchange feature by using one or more Endpoint operators.
- Debugging stream applications
  Debug stream applications using the interactive, command line-based Streams Debugger (sdb).
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Multi-threading considerations

The following topic describes situations when operators can run in multi-threaded context and the related performance considerations.

Single threaded context in Operator's model

The providesSingleThreadedContext element is used to enable the Teracloud® Streams instance to avoid unnecessary thread synchronization.

Set providesSingleThreadedContext to Always when:

It does not perform concurrent submit calls unless its process methods are called concurrently
Its submit calls complete before the process call that triggered the submission completes

The SPL compiler ensures that the shared state variables declared and modified within the logic clause of an operator invocation are safely accessed during concurrent process methods. When you set providesSingleThreadedContext to Always, the operator code determines whether locking is required to serialize access to the state variables.

An example of an operator with a single threaded context is the Filter operator.

Unless its process method is being called concurrently, the Filter operator does not make concurrent submit calls. Its submit calls are triggered by incoming tuples.
When it receives a tuple from a process call, it makes a submit call if the received tuple passes the filter condition, and that submit call completes before the process call that triggered it is complete.
As a result, all instances of a Filter operator provide a single threaded context and the setting Always is appropriate.

Managing thread concurrency in operator defined shared states

Minimize the amount of sharing as much as possible
- Consider the use of thread local or stack local variables if that helps
If operators must share state information, then consider trade-offs between the following methods:
- Volatile variables
- Atomic operations (by compiler intrinsics)
- Spin-locks
- Mutex
Be careful about false sharing
- Use padding to avoid data that is accessed by different threads, accessing the same cache line.

Queues shared across threads

Common use case is producer / consumer queues
Consider the use of lock free queues. These queues are almost always architecture-specific.