Enabling Streams data exchange

Teracloud® Streams provides data exchange REST and WebSocket APIs, for inserting tuples into and retrieving tuples from Streams jobs, to easily integrate with external applications. Streams applications can enable the data exchange feature by using one or more Endpoint operators.

Enabling a stream application for data exchange

To enable data exchange capability for a Streams application, add one or more Endpoint operators to your application. The Endpoint operators are as follows:

  • EndpointSource – Generates tuples from data that is provided on an HTTP POST call or through a WebSocket message.
  • EndpointSink – Returns Streams tuple data in response to an HTTP GET call or as a message sent over a WebSocket.

You can also optionally update the Streams application source code to provide customized content that is included with the dynamically generated data exchange API documentation.

You can use annotations to provide customized documentation content. The following annotations are supported:
Note: These annotations are not required to enable data exchange with a Streams job; rather, they can be used to provide more helpful API documentation.

Initiating data exchange support by submitting a job

A Streams data service is started as part of instance startup. The data service detects when jobs are submitted for applications that contain Endpoint operators. It then provides the REST HTTP and WebSocket API support for adding data to and retrieving data from those jobs.

The data exchange REST and WebSocket API documentation is dynamically generated when a job containing one or more Endpoint operators is submitted. The documentation is available as part of the API interface, enabling programmatic discovery of the supported API endpoints.

  • The REST API documentation is provided in standard OpenAPI format and is available in both JSON and YAML.
  • The WebSocket API documentation is provided in standard AsyncAPI format and is also available in both JSON and YAML. This content can be imported into tools that generate client code from OpenAPI or AsyncAPI documentation.

Accessing the Streams data service

Run streamtool getinstanceurl --data to determine the top-level data service REST URL. This API returns the list of running data exchange jobs. The information returned for each data exchange job includes:

  • The set of data exchange inject endpoint URIs – both HTTP and WebSocket.
  • The set of data exchange inspect endpoint URIs – both HTTP and WebSocket.
  • The dynamically generated API documentation, including tuple data format information, in JSON and YAML formats.
  • Links to launch a Swagger UI viewer for the generated documentation. This interactive viewer can be used for viewing the REST and WebSocket API documentation. It can also be used to try out exchanging data with a Streams job using the REST API.