Home
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Toolkits
A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
Toolkits
Toolkit com.teracloud.streams.teda 4.0.0
Developing applications
Configuring Solutions
Configuring Lookup Manager Applications
Working with enrichment data from CSV files
Initializing Lookup Manager data from CSV files

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
- APIs
  Application programming interfaces (APIs) provide functions that simplify applications development.
- Job configuration overlays
  You can specify or change configuration parameters when you use the submitjob or updateoperators operations. The settings control submission constraints, host constraints and locations, and operator status. Parameters are captured and updated in the configuration JSON file. When you change the values in the job configuration overlay file, the changes override any previously set values.
- Commands
  Teracloud® Streams provides a number of commands you use to perform tasks.
- Operator model
  An operator model is an XML document that describes the basic properties of a primitive operator.
- Streams Processing Language (SPL)
  Streams Processing Language (SPL) is a distributed data flow composition language that is used in Teracloud® Streams. SPL has primitive types, program structures, and definitions that are tailored for streaming data.
- Toolkits
  A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
  - Specialized toolkit requirements and restrictions
    Several of the specialized toolkits require RPMs that are not required by the Teracloud® Streams product. There are also toolkit restrictions on some platforms.
  - Toolkits
- SPLDOC markup
  You can use SPLDOC markup in the descriptions associated with the SPL artifacts in a toolkit. The spl-make-doc command generates HTML documentation from the marked up artifacts.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Initializing Lookup Manager data from CSV files

When you use at least one ITE application together with the Lookup Manager, which provides enrichment data, you must first initialize the Lookup Manager. In the case described here you are using CSV files as the data source.

Before you begin

Compile the Lookup Manager by using a LookupMgrCustomizing.xml file that contains all data source definitions and all segments with appropriate ITE applications.

In your command file you use the init command.

Start the ITE and Lookup Manager applications by submitting them to an IBM Streams instance.

About this task

Initialize the Lookup Manager’s enrichment data from CSV files.

Procedure

Create your CSV enrichment data files and move them to the directory configured with lm.file.directory
Create the command file that contains the init command and affected segments. Move the file into the directory that is configured with lm.commandsDirectory. Remember to use the ‘.cmd’ extension.
Monitor the states of your jobs with the Streams Console
When the applications are in RUN state, initialization finished successfully and you can use your applications

The following figure shows an example command file:

Lookup Manager application – initializing from CSV files

If you run a command and the DataSource-Name.csv or DataSource-Name.del.csv input CSV files are not provided, the command fails and the process moves the command file to the failed folder. In this case, provide the required CSV files in the file directory and move the command file into the commands directory again. The value of the Name attribute in the DataSource element is unique. Column indexes are defined in the LookupMgrCustomizing.xml file. With these definitions, you map the data in the column of the CSV file to SPL tuple attributes in the data source stream. Indexing begins with zero and the CSV file must have a constant number of comma-separated columns. The CSV parser handles attribute-type mapping automatically.