Previewing a job before you deploy it

Before you deploy an application to your runtime environment, you can run a preview to see how the job will be deployed.

About this task

You can preview the job from Streams Console or by running the streamtool submitjob command with the --preview option. When you preview the job, you can determine which submission-time configuration parameters need to be adjusted to optimize the job.

Specifically, you can see the following information when you preview a job:

The operators that will be fused into each processing element (PE)
The hosts where each PE is placed
The tagging requirements for each PE

The preview also provides detailed error messages when fusion or placement constraints cannot be met.

The preview takes the following information into account:

The settings in the application bundle file
The configuration parameters on the instance where you plan to deploy the application
The submission-time configuration parameters that are specified in a job configuration overlay file or in the interface
Other optional values that you specify

If you preview a job through streamtool submitjob --preview, a job configuration overlay file is automatically generated. (The default file name is appName_config.json, where appName is the name of your application). If you want to change the configuration of your application during the preview, you can provide an input configuration file through the -g option.

If you preview a job from Streams Console, a job configuration overlay file is automatically generated with the default name jobPreviewResult.json. If you want to change the configuration of your application during the preview, the system defaults to using the file jobConfig.json.

Independent of how you generate the preview results file, it contains the job configuration settings. These settings are derived from the configuration values that are internal to the application bundle, from settings from the configuration overlay that was specified at submission time, and from settings specified by the user at submission time. The results file also contains the placement results from this specific Streams instance host configuration. This results file can be modified and used as input for subsequent calls to submitjob when you preview or submit a job.

Note: If dynamic resource allocation is enabled, Teracloud® Streams doesn't allocate any resources when it previews a job. Instead, it assumes that resources are successfully acquired and references temporary resources, which are discarded when the preview is complete. When the job is submitted, it might not acquire the same resources as the preview did.

Procedure

To preview a job, modify the results file and preview the updated settings from the interactive streamtool interface, perform the following steps:

Run the streamtool submitjob command with the --preview option:
```
streamtool submitjob --preview myApp.sab
```

Review the resulting JSON file, myApp_config.json.

In the following sample, the application configuration calls for host tags of red, green, and blue.

{  
  "jobConfigOverlays": [    
    {      
      "jobConfig": {        
        "jobGroup": "default",        
        "submissionParameters": [
          { "name": "streams::pe.display", "value": "server_name:port_number" }
        ]      
      },
      "deploymentConfig": {        
        "placementScheme": "balancedInstance",        
        "fusionScheme": "automatic"      
      },
      "operatorConfigs": {        
        "operators": [          
          { "nameSpec": "BargainIndex" },          
          { "nameSpec": "PreVwap" },          
          { "nameSpec": "Vwap", "hostIsolation": true}        
       ],        
       "poolPlacementGroups": [
       {            
         "name": "myPool",            
           "operatorsInPool": [
             { "nameSpec": "BargainIndex" },              
             { "nameSpec": "Vwap" },
             { "nameSpec": "PreVwap" }            
           ],            
           "tags": [ "red", "green", "blue" ]          
        }        
        ]      
      }    
    }  
  ],
    "results": {    
      "placementResults": [      
        {        
          "peId": 0,
          "names": [ "BargainIndex", "PreVwap", "Vwap"],        
          "tags": [ "red", "green", "blue" ],        
          "hostPlacement": "HostA"     
       }    
     ]
   }
   }

Update the job configuration overlay file to optimize the deployment.
In this example, the instance where you plan to deploy the application has host tags rack1, rack2, and rack3. These tags do not match the host tags specified in the application bundle file, so you must modify them to match the tags on the running instance.
Remember: You do not need to specify all possible options in the job configuration overlay file. You can specify only the items that you want to modify.
In this example, you want to change only the tags in the pool placement group, so your job configuration overlay file will have the following format:
```
{  
  "jobConfigOverlays": [    
    {      
      "operatorConfigs": {        
        "poolPlacementGroups": [          
          {            
            "name": "myPool",            
            "operatorsInPool": [              
              { "nameSpec": "BargainIndex" },
              { "nameSpec": "Vwap" },              
              { "nameSpec": "PreVwap" }            
            ],            
            "tags": [ "rack1", "rack2", "rack3" ]
          }        
        ]      
      }    
    }
  ]
  }
```
Preview the job deployment again with the updated job configuration overlay file:
```
streamtool submitjob --preview -g myUpdatedJobConfig.json myApp.sab
```

What to do next

You can modify the job configuration overlay file and preview the results until you are satisfied that your configuration will result in a good deployment in your environment. After you are satisfied, submit the job with the job configuration overlay file.

For example, using the streamtool interface, run the following command:

streamtool submitjob -g myUpdatedJobConfig.json myApp.sab