Header with version information

Use case:

Telecommunications network elements produce two different call detail records (CDRs). There are two different versions of the network elements, each producing slightly different (extended) CDRs. The version information is available only in a header record.

Requirement:

Either the header structure never changes its layout, or a newer version of the header structure can be unambiguously detected by using a condition. As a result, the version information can be extracted from the header structure.

Solution:

  • The version field of the header is stored in the version variable.
  • Version checks are implemented in groups.
  • Event A version 0 and event B version 0 belong to version 0 and are assigned to the v0 group.
  • Event A version 1 and event B version 1 extend their version 0 counterparts and belong to version 1 and are assigned to the v1 group.
  • If a trailer structure exists, it can reset the version variable to reduce the risk that corrupted consecutive headers or trailers do not lead to wrong results.

For more information, review the etc/structure.xml structure document in the teda.sample.StructureParse.Versioning sample application. 


<?xml version="1.0" encoding="UTF-8"?>
<structures
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.ibm.com/software/data/infosphere/streams/parser StructureParseStructure.xsd"
	xmlns="http://www.ibm.com/software/data/infosphere/streams/parser"
>
	<!--
		The variable is required to store the value of the field that holds the
		version information.
	-->
	<variable name="version" type="uint16" value="99"/>
	<!--
		All structures that belong to version 0 are grouped.
	-->
	<group name="v0">
		<condition>
			<cmp op="equal">
				<variable name="version"/>
				<value>0</value>
			</cmp>
		</condition>
	</group>
	<!--
		All structures that belong to version 1 are grouped.
	-->
	<group name="v1">
		<condition>
			<cmp op="equal">
				<variable name="version"/>
				<value>1</value>
			</cmp>
		</condition>
	</group>
	<!--
		The header
		The version information is stored in the above declared variable.
	-->
	<structure name="header">
		<condition>
			<logicOp op="and">
				<cmp op="equal">
					<field name="RECORD_TYPE"/>
					<value>0</value> <!--header -->
				</cmp>
				<in>
					<field name="VERSION"/> <!-- support two versions -->
					<value>0</value>
					<value>1</value>
				</in>
			</logicOp>
		</condition>
		<action>
			<store field="VERSION" as="version"/>
			<submit/>
		</action>
		<field name="RECORD_TYPE" type="uint8"/>
		<field name="VERSION" type="uint16"/>
	</structure>
	<!--
		Event A, version 0.
	-->
	<structure name="eventA_0" group="v0">
		<condition>
			<cmp op="equal">
				<field name="RECORD_TYPE"/>
				<value>1</value> <!-- event A -->
			</cmp>
		</condition>
		<field name="RECORD_TYPE" type="uint8"/>
		<field name="ORIG_ADDR" type="rstring" size="3"/>
		<field name="DEST_ADDR" type="rstring" size="3"/>
	</structure>
	<!--
		Event A, version 1.
		It is derived from event A, version 0, and extended by additional fields.
	-->
	<structure name="eventA_1" group="v1">
		<condition>
			<cmp op="equal">
				<field name="RECORD_TYPE"/>
				<value>1</value> <!-- event A -->
			</cmp>
		</condition>
		<base name="eventA_0"/>
		<field name="IMSI_ADDR" type="rstring" size="3"/>
	</structure>
	<!--
		Event B, version 0.
	-->
	<structure name="eventB_0" group="v0">
		<condition>
			<cmp op="equal">
				<field name="RECORD_TYPE"/>
				<value>2</value> <!-- event B -->
			</cmp>
		</condition>
		<field name="RECORD_TYPE" type="uint8"/>
		<field name="SMS_CENTRE" type="rstring" size="3"/>
	</structure>
	<!--
		Event B, version 1.
		It is derived from event B, version 0, and extended by additional fields.
	-->
	<structure name="eventB_1" group="v1">
		<condition>
			<cmp op="equal">
				<field name="RECORD_TYPE"/>
				<value>2</value><!-- event B -->
			</cmp>
		</condition>
		<base name="eventB_0"/>
		<field name="IMSI_ADDR" type="rstring" size="3"/>
	</structure>
	<!--
		The trailer
		The above declared variable, which stores version information, is reset.
	-->
	<structure name="trailer">
		<condition>
			<logicOp op="and">
				<cmp op="equal">
					<field name="RECORD_TYPE"/>
					<value>3</value> <!-- trailer -->
				</cmp>
				<in>
					<variable name="version"/>
					<value>0</value>
					<value>1</value>
				</in>
			</logicOp>
		</condition>
		<action>
			<set value="99" as="version"/>
			<submit/>
		</action>
		<field name="RECORD_TYPE" type="uint8"/>
	</structure>
	<!--
		Synchronize in case of failures.
	-->
	<sync>
		<structure name="header"/>
	</sync>
</structures>