Job configuration overlay examples

You can use a job configuration overlay file to save reusable job submission parameters or to change a running job with an updateoperators operation.

Although the job configuration overlay JSON file can have many entries, these examples show short JSON files to demonstrate distinct purposes.

Important: Refer to the Job configuration overlays reference to confirm whether the parameters apply to the submitjob operation or to the updateoperators operation.

Example 1: Set the fusion type for operators in a parallel region

{
  "jobConfigOverlays": [
    {
      "deploymentConfig": {
        "parallelRegionConfig": {
            "fusionType": "channelIsolation"
          }
      }
    }
  ]
}

Example 2: Change the width of multiple parallel regions

You can change the width of multiple parallel regions at once in a submitjoboperation.

{
  "jobConfigOverlays": [
    {
      "configInstructions": {
        "adjustmentSection":  [
          {
            "targetParallelRegion": {
              "regionName": "OuterA",
              "newWidth": 1
            }
          },
          {
            "targetParallelRegion": {
              "regionName": "OuterA.MiddleB",
              "newWidth": 2
            }
          },
          {
            "targetParallelRegion": {
              "regionName": "OuterSpaceA",
              "newWidth": 3
            }
          }
        ]
      }
    }
  ]
}

Example 3: Change the parallel region width and add tagged and untagged resources

One subsection that specifies the region, width, and tag-less resource addition and another subsection for each tagged resource addition.

{
  "jobConfigOverlays": [
    {
      "configInstructions": {
        "adjustmentSection": [
          {
            "targetParallelRegion": {
              "regionName": "PreVwap",
              "newWidth": 6
            },
            "numberOfResourcesToBeAdded": 1
          },
          {
            "numberOfResourcesToBeAdded": 1,
            "tagsForAddedResources": [
              "t1"
            ]
          },
          {
            "numberOfResourcesToBeAdded": 1,
            "tagsForAddedResources": [
              "t2"
            ]
          }
        ]
      },
      "operationConfig": {
        "forcePeStopped": true,
        "enableAutomaticAugmentation": true
      }
    }
   ]
}

Example 4: Use the wildcard character in parallel region names or in operator names

You can change the width of a parallel region by specifying the region name with the wildcard (*) character.

{
  "jobConfigOverlays": [
    {
      "configInstructions": {
        "adjustmentSection":  [
          {
            "targetParallelRegion": {
              "regionName": "Link*_3*",
              "newWidth": 43
            }
          }
        ]
      }
    }
  ]
}

You can change the operator placement configuration for multiple operators by specifying the operators with the wildcard (*) character.

{
  "jobConfigOverlays": [
    {
      "operatorConfigs": {
        "partitionColocationGroups": [
          {
            "groupName": "pclg1",
            "members": ["*Beat*"]
          }
        ],
        "hostColocationGroups": [
          {
            "groupName": "hclg1",
            "members": ["Link_1_*"]
          }
        ],
        "poolPlacementGroups": [
          {
            "name": "redpool",
            "tags": [ "red" ],
            "numberOfResourcesToRequest": 1,
            "operatorsInPool": [
              { "nameSpec": "*Beat*"}
            ]
          },
          {
            "name": "bluepool",
            "numberOfResourcesToRequest": 2,
            "tags": [ "blue" ],
            "operatorsInPool": [
              { "nameSpec": "Link_*"}
            ]
          }
        ]
      }
    }
  ]
}

Example 5: Override the partition relational placements from the application bundle

The partition placement that is specified in the SPL application is cleared. A new partition specification is applied to the operators Link*.

{
  "jobConfigOverlays": [
    {
      "operatorConfigs": {
        "operators": [
          {
            "nameSpec": "Link*",
            "partitionIsolation": true
          }
        ]
      },
      "configInstructions": {
        "ignoreBundleRelationalPlacements": "PARTITION"
      }
    }
  ]
}

Example 6: Set all operators to "restartable," but not "relocateable"

{
  "jobConfigOverlays": [
    {
      "operatorConfigs": {
        "operators": [
          {
            "nameSpec": "*",
            "relocatable": false,
            "restartable": true
          }
        ]
      }
    }
  ]
}

Example 7: Override the instance property setting for "jobResourceSharing" for a submitjob operation

{
  "jobConfigOverlays": [
    {
      "deploymentConfig": {
        "jobResourceSharing": "sameUser"
      }
    }
  ]
}

Example 8: Use submission-time parameters for a submitjob operation

Use this example with the sample SPL application, "Compress."

{
  "jobConfigOverlays": [
    {
      "jobConfig": {
        "submissionParameters": [
          {
            "name": "in",
            "value": "Makefile"
          },
          {
            "name": "out",
            "value": "Makefile.gz"
          }
        ],
        "dataDirectory": "<path-to-the-application-directory>"
      }
    }
  ]
}