Upgrading of variant B and C with new custom-specific types
The new deduplication features and the improved handling of the custom context requires an upgrade of custom-specific types. The application that implements variant B or C, it requires new custom-specific types from <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl file.
If the application implements the Variant A, then you don't need to follow the described procedure.
Procedure
- Open the <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl file in an editor, for example:
gedit <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl
a. Add the new ContextCheckpointStreamType type that specifies attributes from the new ContextContainer's DataForContextDataProcessor_* stream that is required to build your custom context. You find the sample in <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file.
- Open the sample <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file in an editor, for example:
gedit <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm
-
Find and copy the lines of the ContextCheckpointStreamType type specification from the <namespace>.streams.sample::TypesCustom composite:
/** * The type of output tuples written to checkpoint files * Add here the attributes from the ContextContainer * DataForContextDataProcessor_* stream that are * required to build your custom context. * This type is used for InRecoveryStream. */ static ContextCheckpointStreamType = tuple< // ------------------------------------------------ // custom code begin // ------------------------------------------------ rstring anyAttribute // ------------------------------------------------ // custom code end // ------------------------------------------------ >;
-
Open the custom SPL file <YourProjectFolder>/<namespace>/streams.custom/TypesCustom.spl file in an editor, for example:
gedit <YourProjectFolder>/<namespace>/streams.custom/TypesCustom.spl
-
Paste the code block to the composite in the <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl file.
- If you use the custom context, then specify your checkpoint attributes in the custom code begin/end block.
For example, the demoapp:
/** * The type of output tuples written to checkpoint files * Add here the attributes from the ContextContainer * DataForContextDataProcessor_* stream that are * required to build your custom context. * This type is used for InRecoveryStream. */ static ContextCheckpointStreamType = tuple< // ------------------------------------------------ // custom code begin // ------------------------------------------------ uint8 cdrRecordType, rstring cdrCallingImsi, rstring cdrCalledImsi, rstring cdrCauseForTermination, uint8 cdrCallType, uint64 cdrSamMczDuration, rstring cdrCallReferenceTime // ------------------------------------------------ // custom code end // ------------------------------------------------ >;
b. Add the specification of the PartitionIdType type that the BloomFilter operator requires. You find the sample in <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file.
- Find and copy the lines of the PartitionIdType type specification from the <namespace>.streams.sample::TypesCustom composite:
/** * Extension of transformer output stream for partitioned deduplication. * You must define the type matching the partitioning requirements of your project. * The allowed SPL types (see reference of the BloomFilter description): * int8, int16, int32, int64 * uint8, uint16, uint32, uint64 * float32, float64, decimal32, decimal64, decimal128 * rstring, ustring * timestamp */ static PartitionIdType = uint8; // example of partitionId type definition
-
Paste the code block to the <namespace>.streams.custom::TypesCustom composite in the <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl file.
- If you use the partitioned deduplication (ite.businessLogic.group.deduplication.partitioning=on), then specify the PartitionIdType by using the one of the allowed SPL types.
c. Add the specification of new values rrRecordOutdated and rrInvalidAttribute of the rejectReason enum type definition that the BloomFilter operator requires. You find the sample in <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file.
- Find and copy the marked lines of the rejectReason type specification from the <namespace>.streams.sample::TypesCustom composite:
/** * The custom reject reasons. * Value is applied for rejected records at RecordValidator or Transformer */ static rejectReason = enum { rrDefault, rrInvalidLine, rrRecordDuplicate, rrTableDuplicate, rrInvalidAttribute, // <- new value marked rrRecordOutdated // <- new value marked };
-
Paste the lines to the rejectReason enum type specification of the <namespace>.streams.custom::TypesCustom composite in the <YourProjectFolder>/<namespace>.streams.custom/TypesCustom.spl file.
For example, the demoapp:
/** * The custom reject reasons. * Value is applied for rejected records at RecordValidator or Transformer */ static rejectReason = enum { rrDefault, rrInvalidLine, rrRecordDuplicate, rrTableDuplicate, rrInvalidAttribute, rrRecordOutdated, // ------------------------------------------------ // custom code begin // ------------------------------------------------ // add custom reject codes here rrLookupFailed // ------------------------------------------------ // custom code end // ------------------------------------------------ };
- Open the sample <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file in an editor, for example:
-
Close the sample <YourProjectFolder>/<namespace>/streams/sample/TypesCustom.splmm file.
-
Save and close the sample <YourProjectFolder>/<namespace>/streams.custom/TypesCustom.spl file.