Home
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
Toolkits
A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
Toolkits
Toolkit com.teracloud.streams.teda 4.0.0
Developing applications
Configuring Solutions
Configuring ITE Applications
Data ingestion
Sorting input files by special file time

Welcome
Learn about the core capabilities of Teracloud® Streams, its architecture, and key concepts.
Installing
Use this information to install, upgrade, and uninstall the Teracloud® Streams product.
Configuring
Create a basic or an enterprise domain which is a single point for configuring and managing common resources, security, and instances.
Administering
Administer the product by using the Teracloud® Streams graphical user interface, APIs, or the streamtool command-line interface.
Developing
Develop stream applications with the Streams Processing Language (SPL), Java, and Python.
Troubleshooting
Resolve problems with Teracloud® Streams using the troubleshooting tools provided with the product as well as the resources offered by Teracloud Support.
Reference
Find details on the SPL language, toolkits, APIs, commands, and more.
- APIs
  Application programming interfaces (APIs) provide functions that simplify applications development.
- Job configuration overlays
  You can specify or change configuration parameters when you use the submitjob or updateoperators operations. The settings control submission constraints, host constraints and locations, and operator status. Parameters are captured and updated in the configuration JSON file. When you change the values in the job configuration overlay file, the changes override any previously set values.
- Commands
  Teracloud® Streams provides a number of commands you use to perform tasks.
- Operator model
  An operator model is an XML document that describes the basic properties of a primitive operator.
- Streams Processing Language (SPL)
  Streams Processing Language (SPL) is a distributed data flow composition language that is used in Teracloud® Streams. SPL has primitive types, program structures, and definitions that are tailored for streaming data.
- Toolkits
  A toolkit is a set of SPL artifacts, which are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications.
  - Specialized toolkit requirements and restrictions
    Several of the specialized toolkits require RPMs that are not required by the Teracloud® Streams product. There are also toolkit restrictions on some platforms.
  - Toolkits
- SPLDOC markup
  You can use SPLDOC markup in the descriptions associated with the SPL artifacts in a toolkit. The spl-make-doc command generates HTML documentation from the marked up artifacts.
Glossary
Use this glossary to find terms and definitions for Teracloud® Streams.

Sorting input files by special file time

When your business logic relies on a certain order to read the input files, you configure scanning of the directory to report the identified files in a sorted sequence. For example, this sorting is needed in automated tests where current application results are automatically compared to expected results.

About this task

Enable sorting by special file time in ascending order by using parameters. Here special file time means that the date and time information is encoded in the file name.

Procedure

In the file <PathToYourApplication>/config/config.cfg, find the ite.ingest.directoryScan.sort parameter description
To enable sorting in ascending order, select the ascending value to the parameter as follows: ite.ingest.directoryScan.sort=ascending
In the file <PathToYourApplication>/config/config.cfg, find the ite.ingest.directoryScan.sort.attribute parameter description
To enable sorting by special file time, set the parameter as follows: ite.ingest.directoryScan.sort.attribute=time
In the file <PathToYourApplication>/config/config.cfg, find the ite.ingest.directoryScan.specialFileTime parameter description
To enable special file time processing, set the parameter to on as follows: ite.ingest.directoryScan.specialFileTime=on
In the file <PathToYourApplication>/config/config.cfg, find ite.ingest.directoryScan.specialFileTime.regexp parameter description
Set the parameter to a regular expression, which extracts the date and time information from the file name, for example: ite.ingest.directoryScan.specialFileTime.regexp=.*_([0-9]{8}).txt$
In the file <PathToYourApplication>/config/config.cfg, find the ite.ingest.directoryScan.specialFileTime.format parameter description
Set the parameter to a date format string to interpret the cut out date and time information, for example: ite.ingest.directoryScan.specialFileTime.format=YYYYMMDD

For the latter two parameters, you can assign a comma-separated list of regular expressions and appropriate format strings to support many different file names.