streamsx.ec

Access to the Streams execution context.

Overview

This module (streamsx.ec) provides access to the execution context when Python code is running in a Streams application.

A Streams application runs distributed or standalone.

Distributed

Distributed is used when an application is submitted to a Streams distributed instance.

With distributed a running application is a job that contains one or more processing elements (PEs). A PE corresponds to a Linux operating system process. The PEs in a job may be distributed across the resources (hosts) in the Streams instance.

Standalone

Standalone is a mode where the complete application is run as a single PE (process) outside of a Streams instance.

Standalone is typically used for ad-hoc testing of an application.

Application log and trace

Streams provides application trace and log services.

Application log

The Streams application log service is for application logging, where logging is defined as the recording of serviceability information pertaining to application or operator events. The purpose of logging is to provide an administrator with enough information to do problem determination for items they can potentially control. In general, very few events are logged in the normal running scenario of an application or operator. Events pertinent to the failure or partial failure of application runtime scenarios should be logged.

When running in distributed or standalone the com.ibm.streams.log logger has a handler that records messages to the Streams application log service. The level of the logger and its handler are set to the configured application log level at PE start up.

This logger and handler discard any message with level below INFO (20).

Python application code can log a message suitable for an administrator by using the com.ibm.streams.log logger or a child logger that has logger.propagate evaulating to True. Example of logging a file exception:

try:
    import numpy
except ImportError as e:
    logging.getLogger('com.ibm.streams.log').error(e)
    raise

Application code must not modify the com.ibm.streams.log logger, if additional handlers or different levels are required a child logger should be used.

Application trace

The Streams application trace service is for application tracing, where tracing is defined as the recording of application or operator internal events and data. The purpose of tracing is to allow application or operator developers to debug their applications or operators.

When running in distributed or standalone the root logger has a handler that records messages to the Streams application trace service. The level of the logger and its handler are set to the configured application trace level at PE start up.

Python application code can trace a message using the root logger or a child logger that has logger.propagate evaulating to True. Example of logging a trace message:

trace = logging.getLogger(__name__)

...

    trace.info("Threshold set to %f", val)

Any existing logging performed by modules will automatically become Streams trace messages if the application is using the logging package.

Application code must not modify the root logger, if additional handlers or different levels are required a child logger should be used.

Execution Context

This module (streamsx.ec) provides access to the execution context when Python code is running in a Streams application.

Access is only supported when running:

Streams 4.2 or later

This module may be used by Python functions or classes used in a Topology or decorated SPL operators.

Most functionality is only available when a Python class is being invoked in a Streams application.

Changed in version 1.9: Support for Python 2.7

Module contents

Functions

`channel`	Return the parallel region global channel number obj is executing in.
`domain_id`	Return the instance identifier.
`get_application_configuration`	Get a named application configuration.
`get_application_directory`	Get the application directory.
`get_submission_time_value`	Gets the value of a submission time parameter.
`instance_id`	Return the instance identifier.
`is_active`	Tests if code is active within a Streams execution context.
`is_standalone`	Is the execution context standalone.
`job_id`	Return the job identifier.
`local_channel`	Return the parallel region local channel number obj is executing in.
`local_max_channels`	Return the local maximum number of channels for the parallel region obj is executing in.
`max_channels`	Return the global maximum number of channels for the parallel region obj is executing in.
`pe_id`	Return the PE identifier.
`shutdown`	Return the processing element (PE) shutdown event.

Classes

`CustomMetric`	Create a custom metric.
`MetricKind`	Enumeration for the kind of a metric.