Operator fusion in parallel regions

Fusion of operators into PEs happens at submission time, after the parallel transformations have been done. Fusion can be controlled either manually, through the use of placement configs such as partitionColocation, or automatically by letting the Teracloud® Streams runtime decide how best to fuse operators.

Teracloud® Streams provides some intrinsic functions to help with manual fusion. The byChannel() intrinsic function, when used with the partitionColocation config, can be used to fuse operators together with other operators in the same parallel channel. The byReplica() intrinsic function fuses cousin operators from a given parallel region.

In addition, the fusionType parameter defined at submission time gives direction to the Teracloud® Streams run time about how to fuse operators from parallel regions.

Note: These rules do not apply if you select the legacy fusion scheme to submit your application.
Valid values for the fusionType parameter are as follows:
  • noChannelInfluence: Do not treat parallel regions differently. Inclusion in a parallel region does not have impact on how the operators are fused. This is the default behavior.
  • channelIsolation: Operators within a channel are fused into a PE only with operators from the same channel. Operators outside the parallel region or from other channels in the same region are fused into different PEs. One or more PEs for a channel can be created depending on the fusion constraints.

    You can explicitly collocate an operator in a channel with an operator outside the region or from a different channel in the same region. A separate PE is created if all of the explicitly colocated operators are not members of the same channel.

  • channelExlocation: Operators within a channel are fused with operators from the same channel or with operators from outside the region. Operators from other channels in the same region are not fused into the same PEs. One or more PEs for a channel can be created depending on the fusion constraints.

    You can explicitly collocate an operator in a channel with an operator outside the region or from a different channel in the same region. A separate PE is created if all of the explicitly colocated operators are not members of the same channel.

    If you use this value, you cannot change the width of a parallel region while the job is running. You would need to change the fusionType parameter to channelIsolation or to noChannelInfluence, and then resubmit the job.

Any fusing specified manually is always respected. For more information about the fusionType parameter, see Specifying how operators are fused when you submit a job.