Creating partition co-location or ex-location groups
You can specify how the operators in an application are grouped together into processing elements (PEs). You can group operators together in a PE (also called a partition), or prevent operators from being grouped with other operators.
About this task
Creating partition co-location groups can enable related operators to communicate more efficiently, while creating partition ex-location groups can prevent resources from being overloaded.
You can specify partition location constraints from the Teracloud® Streams
Console or the streamtool submitjob command. However, if you use the
streamtool submitjob command, you must use a job configuration
overlay file to specify these settings. A job configuration overlay file is optional
if you use Teracloud® Streams
Console.
To create a partition co-location group, you must create one or more LocationGroup entries within the partitionColocationGroups parameter. Each LocationGroup entry defines a partition co-location group.
All of the members of a partition co-location group are placed in the same PE at run time. When you create a partition co-location group, consider the requirements of the operators. For example, you might want to place operators with complementary requirements in a partition co-location group.
To create a partition ex-location group, create one or more LocationGroup entries within the partitionExlocationGroups parameter. Each LocationGroup entry defines a partition ex-location group.
- Operators that consume a lot of system resources are not placed in the same PE
- Operators in an HA environment run on different hosts to reduce the likelihood that multiple operators will be impacted if a host goes down
- name
- The name to use for the group. If the name matches the name of a group that is defined in the
application bundle file, this group replaces the group in the bundle.
If you do not specify a name, Teracloud® Streams generates a name.
- members
- Specify a list of the operators that you want to include in the group. (Operators can be in more
than one
group.)To specify which operators belong to the group, you can specify the name of a single operator or a group of operators:
- To specify a single operator, specify the name of the existing operator instance in the application after user-defined-parallelism (UDP) transformations.
- To specify a group of operators, use an asterisk (*) as a
wildcard character. You can specify 0 or more wildcard
characters. For example:
- Enter myParallelRegion.myOperator[*] to include all of the channels of the parallel region
- Enter MyComposite.firstComposite.* to include all of the operators in the composite region
If you do not specify a list of operators and there is a location group with the same name in the application bundle file, the group is removed from the application bundle file.