Operator CharacterTransform

The CharacterTransform operator is used to convert from one encoding in blob to another encoding in blob.

Checkpointed data

When the CharacterTransform operator is checkpointed, logic state variables (if present) are saved in checkpoint.

Behavior in a consistent region

The CharacterTransform operator can be an operator within the reachability graph of a consistent region. It cannot be the start of a consistent region. In a consistent region, a CharacterTransform operator stores its state when a checkpoint is taken. When the region is reset, the operator restores the state from the checkpoint.

Checkpointing behavior in an autonomous region

When the CharacterTransform operator is in an autonomous region and configured with config checkpoint : periodic(T) clause, a background thread in SPL Runtime checkpoints the operator every T seconds, and such periodic checkpointing activity is asynchronous to tuple processing. Upon restart, the operator restores its state from the last checkpoint.

When the CharacterTransform operator is in an autonomous region and configured with config checkpoint : operatorDriven clause, no checkpoint is taken at runtime. Upon restart, the operator restores to its initial state.

Such checkpointing behavior is subject to change in the future.

Examples

This example uses the CharacterTransform operator.

composite Main2 {
  graph
   stream<blob b> Src = FileSource() {
      param file      : "in";
            format    : block;
            blockSize : 4096u;
    }
    stream<blob b> TSrc = CharacterTransform (Src){
      output TSrc : b = Convert ("857", "UTF-8", b);
    }
    stream<ustring u, rstring r> PSrc = Parse (TSrc) {
      param format : txt;
    }
}

// This example is equivalent to the following SPL program:
composite Main {
  graph
    stream<ustring u, rstring r> PSrc = FileSource() {
      param file     : "in";
            format   : txt;
            encoding : "857";
     }
}

Summary

Ports

This operator has 1 input port and 1 output port.

Windowing

This operator does not accept any windowing configurations.

Parameters

This operator does not support parameters.

Metrics

This operator does not report any metrics.

Properties

Implementation

C++

Threading

Always - Operator always provides a single threaded execution context.

Input Ports

Ports (0)

The CharacterTransform operator is configurable with a single input port, which ingests tuples containing data to be transformed. The input tuple can contain attributes of any type. The attributes to be encoded must be of type blob.

Properties

• Optional: false

• ControlPort: false
• TupleMutationAllowed: false
• WindowingMode: NonWindowed
• WindowPunctuationInputMode: Oblivious

Output Ports

Assignments

This operator allows any SPL expression of the correct type to be assigned to output attributes. Attributes not assigned in the output clause will be automatically assigned from the attributes of the input ports that have the same name and type. If there is no such input attribute, an error is reported at compile-time.

Output Functions

TransformFunctions

blob Convert (rstring fromEncoding, rstring toEncoding, blob data)

Converts the characters in data from encoding fromEncoding to encoding toEncoding. All other output attributes are filled in from input attributes if they are not specified on the output clause. For Source or Sink purposes, either fromEncoding or toEncoding are usually UTF-8 for compatibility with the Source or Sink encoding parameter.

<any T> T AsIs(T v)

Returns the input argument.

Ports (0)

The CharacterTransform operator is configurable with a single output port, which produces tuples containing transformed data.

Properties

• Optional: false

• TupleMutationAllowed: true
• WindowPunctuationOutputMode: Preserving

Code Templates

CharacterTransform to UFT8

(stream<${schema}> ${outputStream} = CharacterTransform(${inputStream})   {
            output
            ${outputStream} : ${outputAttribute} = Encode (${sourceEncoding}, "UTF8", ${value});
        }
      

CharacterTransform from UFT8

(stream<${schema}> ${outputStream} = CharacterTransform(${inputStream})   {
            output
            ${outputStream} : ${outputAttribute} = Encode ("UTF8", ${targetEncoding}, ${value});
        }
      

Libraries

spl-std-tk-lib

Library Name: streams-stdtk-runtime

Include Path: ../../../impl/include