streamtool previewsubmitjob

previewsubmitjob [-d,--domain-id <did>] [-i,--instance-id <instance>] [--override <override-list>] [-g,--jobConfig <file-name>] [-P,--P <parameter-name>] ... [-C,--config <config-setting>] ... [-J,--jobgroup <jobgroup-name>] [-q,--out_jobConfig <file-name>] [--jobname <job-name>] [-U,--User <user>] [-h,--help] [--trace <level>] [-v,--verbose <level>] [--zkconnect {<host>:<port>},... | --embeddedzk] <sab-pathname>

The streamtool previewsubmitjob command previews the submission of a job. The command returns information that is useful for users who want to alter the configuration of a job when they submit it. This command is deprecated and might become obsolete. Use the streamtool submitjob --preview command instead.

Authority

You must have add authority for the appropriate jobgroup_name instance object or else submit permission for the job group. By default, the DomainAdministrator, InstanceAdministrator, and InstanceUser roles have this authority.

You must have add authority for the jobs-override instance object to use the --override ResourceLoadProtection option. By default, the DomainAdministrator and InstanceAdministrator roles have this authority.

For more information about access control lists, see the streamtool getacl and streamtool lsjobpermission commands.

Description

The command retrieves information about the operator constraints that are in effect and the operator sets that make up the processing elements (PEs). It does not submit the job.

An application can define optional or required submission parameters that can be used to control its behavior. If you do not provide required submission parameters when you preview the job, the submission fails. If you specify a parameter name that is not known to the application, the command generates a warning and the parameter is ignored.

You can optionally specify an application configuration setting by using the -C, --config option. For example, you can use this option to set the location of the data directory:

streamtool previewsubmitjob -C data-directory=/myDataDirectory myappl.sab

The command generates a configuration set that contains information about tagging requirements, operator fusion sets, and operator constraints. The configuration is stored in the applicationBundleName_config.json file. For the sample.Vwap application, for example, the configuration is stored in sample.Vwap_config.json. To specify a different file, use the --out_jobConfig option. You can use this output file as input for the streamtool submitjob command.

-C,--config <config-setting>

Specifies the name of one of the following application configuration settings and its value:

data-directory

Specifies the location of the data directory. For example: /myDataDirectory.

fusionScheme

Specifies the scheme that is used to determine how to fuse the operators in a stream processing application into partitions. By default, the command uses the instance.fusionScheme instance property value.

Valid values are automatic (default), legacy, and manual. When you use the automatic value, the system determines the most efficient the number of PEs to assign to a job.

Using the legacy value prompts operator fusion to behave as it did before Streams Version 4.2. Typically, each operator is fused into a separate PE.

With the manual fusion scheme, you can specify the target number of PEs to use.

fusionTargetPeCount

Specifies the target number of processing elements to use when fusing operators. This parameter can be specified only when fusionScheme is set to manual.

parallelRegionConfig

Specifies how the operators in the channels in parallel regions are fused into PEs.

• channelIsolation: Operators within a channel are fused into a PE only with operators from the same channel. Operators outside the parallel region or from other channels in the same region are fused into different PEs. One or more PEs for a channel can be created depending on the fusion constraints.

  You can explicitly colocate an operator in a channel with an operator outside the region or from a different channel in the same region. A separate PE is created if all of the explicitly colocated operators are not members of the same channel.

• channelExlocation: Operators within a channel are fused with operators from the same channel or with operators from outside the region. Operators from other channels in the same region are not fused into the same PEs. One or more PEs for a channel can be created depending on the fusion constraints.

  You can explicitly colocate an operator in a channel with an operator outside the region or from a different channel in the same region. A separate PE is created if all of the explicitly colocated operators are not members of the same channel.

  If you use this value, you cannot change the width of a parallel region while the job is running. You would need to resubmit the job with the parallelRegionConfig parameter set to channelIsolation or to noChannelInfluence.

• noChannelInfluence: (default) Inclusion in this parallelRegion has no impact on the fusion process.

placementScheme

Specifies a scheme to determine which PEs get placed on which hosts. Valid values are balancedInstance (default), balancedJob, and legacy.

When you use the balancedInstance value, the system distributes PEs across candidate hosts proportionally to the number of cores on the hosts. This method takes into account PE placements from previous jobs.

Using the balancedJob value means that PEs are distributed across candidate hosts proportionally to the number of cores on the hosts, but PE placements from previous jobs are ignored.

When you use the legacy value, the estimated load of PEs to be placed is matched with existing ldavg numbers of hosts that are adjusted for the number of cores in hosts.

preloadApplicationBundles

Specifies whether to preload the application bundle files onto all resources in the instance, even if not currently needed there. Valid values are true and false. By default, the command uses the instance.preloadApplicationBundles instance property value. Preloading the application bundle files can improve the performance in situations where a PE is relocated to a new resource that was not already hosting a PE from that application.

dynamicThreadingElastic

Specifies whether elasticity is on or off for dynamicThreading. Valid values are true for on, and false for off. This setting can be used only if instance.threadingModel is set to dynamic, or the application configuration threadingModel is set to dynamic.

dynamicThreadingThreadCount

Specifies the number of threads for scheduled ports to use. Inherits its value from instance.dynamicThreadingThreadCount, if set. Valid values are integers greater than or equal to 1. This setting can be used only if instance.threadingModel is set to dynamic, or the application configuration threadingModel is set to dynamic.

threadingModel

Specifies the threading model to use as the default for each PE in the job. Inherits its value from instance.threadingModel, if set. This default can be overridden by @threading annotations in the SPL code. Valid values are manual, automatic, scheduledPorts, allThreadedPorts, and default.

tracing

Specifies the PE trace setting. The following valid levels are listed in order of increasing verbosity, which is to say that the first level in the list generates the least amount of information:

• off
• error
• warn
• info
• debug
• trace

By default, the trace setting is error unless you specify otherwise in the stream processing application.

For example:

--config
tracing=off,preloadApplicationBundles=true

.

-d,--domain-id <did>

Specifies the domain identifier.

If you do not specify this option, Streams uses the domain name that is set in the STREAMS_DOMAIN_ID environment variable. By default, that domain name is StreamsDomain. If you are using the interactive streamtool interface, it uses the name of the active domain for the current streamtool session or else it prompts you for the domain name.

The active domain for the current streamtool session is set every time that you successfully run a streamtool command with a -d or --domain-id option. Alternatively, you can run the streamtool domain command in the interactive interface.

--embeddedzk

Specifies to use the embedded copy of ZooKeeper. This option is not supported within the interactive streamtool interface.

If you are not using the interactive streamtool interface and you do not specify either this option or the --zkconnect option, Streams uses the ZooKeeper connection that is associated with the active domain or the domain that is specified in the --domain-id option. Streams determines which connection maps to the domain by using cached information about the domains. In this scenario, if the domain identifier is not unique in the Streams configuration cache, the command fails.

-g,--jobConfig <file-name>

Specifies the name of an external file that defines a job configuration overlay. You can use a job configuration overlay to set the job configuration when the job is submitted or to change the configuration of a running job.

-h,--help

Specifies to show the command syntax.

-i,--instance-id <instance>

Specifies the instance identifier.

If you do not specify this option, Streams uses the instance identifier that is set in the STREAMS_INSTANCE_ID environment variable. By default, that instance identifier is StreamsInstance. If you are using the interactive streamtool interface, it tries to use an instance ID that you specified in a previous command. If no such value is found, the command uses the STREAMS_INSTANCE_ID environment variable. Alternatively, you can run the streamtool instance command in the interactive interface.

-J,--jobgroup <jobgroup-name>

Specifies the job group. If you do not specify this option, the command uses the following job group: default.

--jobname <job-name>

Specifies the name of the job.

--override <override-list>

Specifies to override the instance.resourceLoadProtectionEnabled instance property for the job and treat it as false. The resource load protection property prevents Streams from allocating processing elements (PEs) to an overloaded resource. A resource is overloaded when the values for any one of the metrics for CPU utilization, memory utilization, or network bandwidth utilization exceed their thresholds. For example:

streamtool submitjob --override ResourceLoadProtection myJob.sab

-P,--P <parameter-name>

Specifies a submission-time parameter and value for the job. You can specify this option multiple times in the command.

-q,--out_jobConfig <file-name>

Specifies the name of the output file in which the command writes the operator configuration information.

--trace <level>

Specifies the trace setting. The following valid levels are listed in order of increasing verbosity, which is to say that the first level in the list generates the least amount of information:

• off
• error
• warn
• info
• debug
• trace

The default value is off.

-U,--User <user>

Specifies an Streams user ID that has authority to run the command.

-v,--verbose <level>

Specifies to provide more detailed command output. The verbosity level can be 0-3, where 0 disables detailed reporting and each increment provides more detailed output.

--zkconnect <{<host>:<port>},...>

The name of one or more host and port pairs that specify the configured ZooKeeper servers. This option is not supported within the interactive streamtool interface.

If you are not using the interactive streamtool interface and you do not specify this option, Streams tries to use:

1. The --embeddedzk option
2. The value from the STREAMS_ZKCONNECT environment variable
3. A ZooKeeper connection string that is derived from cached information about the current domain.

sab-pathname

Specifies the path name for the application bundle file. If you do not specify an absolute path, the command seeks the file in the directory where you ran the command. Alternatively, you can specify the path name for the application description language (ADL) file if the application bundle file exists in the same directory.

[streamtool <bsmith@StreamsDomain.StreamsInstance>] 
previewSubmitJob -q myAppConfig.json myApp.sab -C fusionTargetPeCount=10 -P numUdpChannel=8