Configuring submission-time parameters to optimize job deployment

You can specify submission-time parameters for a job before you deploy a stream application to your environment. Typically, configuring the parameters enables you to improve the run-time performance of the application without recompiling it.

About this task

For example, you can deploy a stream application on different instances that have different loads and different processing capabilities. The default settings from the application bundle file might be optimized for a very specific deployment environment. If you deploy the application in a different environment, you might need to override the default behavior to:
  • Ensure that the application is less likely to overload a particular resource
  • Facilitate communication between related operators
  • Ensure that processing elements (PEs) are placed on appropriate resources

You can configure submission-time parameters from the Teracloud® Streams Console or by using the streamtool submitjob command.

Additionally, you can create a job configuration overlay, a JSON file that contains name-value pairs for the submission-time parameters that you want to use. The job configuration overlay enables you to define, save, and distribute the submission-time parameters. For more information about creating and using job configuring overlay files, see Job configuration overlays.

There are several types of submission-time parameters that you can configure:
Job properties
Job properties enable you to specify information about the job, such as a job name or data directory to use for the application. These properties typically do not impact the performance of a job.
Deployment configuration properties
Teracloud® Streams uses default deployment settings that should work for most applications. However, if you want to refine the behavior of your application or if the default settings don't work in your environment, you can specify the following information:
  • How operators are fused when you submit a job (fusion scheme)
  • How PEs are distributed to the resources in the instance when you run the application (placement scheme)
  • How to use threads in processing elements to execute operators in parallel (threading model)
Operator configuration properties
If you need to adjust how operators behave or how they are deployed in the runtime environment, you can specify the following information:
  • The properties of individual operators, such as whether the operator runs on a declared host or if the operator is restartable
  • The host pools that the operators are members of
  • The operators that should run on the same host at run time (host co-location)
  • The operators that cannot run on the same host (host ex-location)
  • The operators that should be placed in the same PE at run time (partition co-location)
  • The operators that cannot be place in the same PE at run time (partition ex-location)
Override instructions
You may override application-wide settings in the application bundle file, such as ignoring relational constraints or overriding the threaded port settings.
Remember: You can specify these submission time parameters in a job configuration overlay file or from the interface from which you submit the job. Alternatively, you can also use a job configuration overlay file and then tweak settings manually.

Before you submit a job, you can preview how the submission-time configuration parameters that you specify will impact the runtime behavior of the application. For more information, see Previewing a job before you deploy it.

See the following topics for more information about the submission-time parameters: