Intra-PE load splitting

You can split the computation load within a processing element (PE) by using user-defined parallelism or multiple threads within a PE.

Using the parallel annotation

Use the @parallel annotation and user-defined parallelism if you want to split the load by automatically replicating specific subgraphs of an application. When operators from outside of the parallel region are fused with operators inside of the parallel region, the operators are replicated inside the PE and threaded ports are inserted. For more information, see User-defined parallelism.

Using the ThreadedSplit operator

Use the ThreadedSplit operator if you do not need to control the split output.

This graphic shows a ThreadedSplit operator for non-content based, cyclic assignment.

Using the Split operator

Use the Split operator to feed operators with threaded input ports if you want to control the split output.

This graphic shows a Split operator for operator-configured, potentially content-based assignment.

Generate the replicated parallel code by using mixed-mode code in a composite operator. For example:
namespace sample;

composite SplitParallelizer() {
  param    
    operator $source;  
    operator $body;   
    type $type;
  graph
    stream<$type> Src = $source() {} 

  (stream<$type> Splitted0   
    <%for(my $i=1; $i<4; ++$i) {%>      // example of 4 way parallelization
      ,stream<$type> Splitted<%=$i%> 
  <%}%>) = ThreadedSplit(Src) {}  

  <%for(my $i=0; $i<4; ++$i) {%>   
    () as Sink<%=$i%> = $body(Splitted<%=$i%>)   
  <%}%>
} 
namespace sample;
 
composite MySource(output stream<MyType> Out) {
  graph
    // source operator here 
}

composite MyLoopBody (input In) {
  graph
    // series of operators here 
}

composite Main {
  () as Sink = SplitParallelizer() {  
  param   
    source : sample::MySource;
    body   : sample::MyLoopBody; 
    type   : MyType; 
   } 
}
You can ensure that operators are placed in the same PE by specifying partition collocation constraints. For example:
config placement : partitionColocation("myPartColocationId");

When multiple hosts are available, you can parallelize across hosts by using a Split operator.