Enabling Streams data exchange

Teracloud® Streams provides a data exchange REST API for inserting and retrieving Streams job data. You can add one or more endpoint operators to your Streams application to enable Streams data exchange. Providing a standard REST interface for Streams data exchange eases the integration of Streams data with other data services and externally hosted applications.

Enabling a Streams application for data exchange

To enable data exchange capability for a Streams application, add one or more endpoint operators to your Streams application. The endpoint operators are as follows:

  • EndpointSource – Generates tuples from data that is provided on a REST API POST call.
  • EndpointSink – Returns Streams data in response to a REST API GET call.

You can also optionally update the Streams application source code to provide customized content that is included with the dynamically generated data exchange API documentation.

  • You can use annotations to provide customized documentation content. The following annotations are supported:
    Note: These annotations are not required to enable data exchange with a Streams application; rather, they can be used to provide more helpful API documentation.

Initiating data exchange support by submitting a job

A Streams data service is started as part of instance startup. The data service detects when jobs are submitted for applications that contain endpoint operators. It then provides the REST API support for adding data to and retrieving data from those jobs.

The data exchange REST API documentation is dynamically generated when a job containing one or more endpoint operators is submitted. The documentation is available as part of the API interface, enabling programmatic discovery of the supported API endpoints. The API documentation is provided in standard OpenAPI format and is available in both JSON and YAML. This content can be imported into tools that generate client code from OpenAPI documentation.

Accessing the Streams data service

Run streamtool getinstanceurl --data to determine the top-level data service REST URL. This API returns the list of running data exchange jobs. The information returned for each data exchange job includes:

  • The set of data exchange inject endpoint APIs.
  • The set of data exchange inspect endpoint APIs.
  • The dynamically generated API documentation in JSON and YAML formats.
  • A link to launch a Swagger UI viewer for the generated documentation. This interactive viewer can be used for viewing and trying out the data exchange API.