Changing parallel region widths in a running job

The streamtool updateoperators command allows parallel region widths in a running job to be updated. This operation adds operators to, or removes operators from, the running job. As a result, operators are re-fused and PEs are restarted.

Increasing the parallel region width can improve application throughput, but can use more resources. You can make more resources available to the parallel region when you change its width. Alternatively, you can decrease the parallel region width to reduce the required resources. Resources that are not needed are released automatically.

When changing the width of a parallel region, the operators inside the parallel region are considered target operators. If the operation increases the width of a parallel region, then new operators that are created will also be considered target operators.

Collateral operators are operators that have a relationship with the target operators:

  • Fused in the same PE as a target operator
  • Fused into the same PE as other collateral operators
  • Connect with the parallel region with a splitter or a merger

Both target and collateral operators are fused into new PEs and restarted.

Restrictions:
  • Only one parallel region width can be modified at a time.
  • You cannot modify the parallel region width of a running job when the original job was submitted with the parallel region fusion (fusionType) parameter set to channelExlocation (prevent fusion across channels). You must resubmit the job with the fusionType parameter set to noChannelInfluence (default) or to channelIsolation.
  • You must have 'add' and 'delete' authority for the appropriate jobgroup_name instance object. By default, the DomainAdministrator and InstanceAdministrator roles have this authority. The user who submits the job also has this authority.
  • You can add resources only if the instance's resource allocation mode is scoped to the job (instance.applicationResourceAllocationMode=job).
  • Checkpointing is not supported in the parallel region, including collateral operators.
  • Consistent regions are not supported in the parallel region, including collateral operators.
Caveats:
  • Since target and collateral operators inside of PEs are stopped and restarted, this operation can result in the loss of tuples and operator state.
  • Which operators are in each PE can change. The PE ID might not be consistent for the lifetime of an operator. You can use the streamtool capturestate command to determine the PE metrics or the streamtool lspes --long command to see the PE IDs.
  • The operation time varies, depending on the number of operators that are affected.
  • Confirm with the application developer that changing the parallel region width will not interfere with any settings in the running application. For example, if the region includes sink operators that are matched with an externally partitioned system.
  • You can change the parallel width at any level of a nested parallel region, but the change will apply to all parallel replicas at that level. For more information, see Nested parallel regions.

You can change the parallel region width of a running job in the following ways:

  • Run the streamtool updateoperators command, specifying the name and the new width in the --parallelRegionWidth parameter. If more resources are required, see information about the --addJobResources parameter in the streamtool updateoperators reference.
  • Create a job configuration overlay (JCO) file that specifies the regionName and newWidth fields in the configInstructions.adjustmentSection.targetParallelRegion object. If more resources are required, add a value to the numberOfResourcesToBeAdded field. Then, run the streamtool updateoperators command with the --jobConfig parameter set to the JCO file. For information about job configuration overlays, see Job configuration overlays and the Job configuration overlays reference.
  • Streams Console: The action can be launched from multiple locations. One way is from the Application Dashboard. Open the Streams Graph and select the job. Open the job properties by hovering your mouse over the information (i) icon, and then click Modify Parallel Region.