Operator IPSpatialEnricher

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.teracloud.streams.network/op$com.teracloud.streams.network.enrichment$IPSpatialEnricher.svg

This operator enriches IPv4 and IPv6 addresses with geospatial data. Only the MaxMind© GeoIP2 (or GeoLite2) City database is supported.

In order to use this operator, you must download the MaxMind© GeoIP2 (or GeoLite2) City database from the MaxMind© website. These files can then be loaded directly into the operator using the blocksIPv4File, blocksIPv6File and locationFile parameters. To dynamically update the operator whenever a new version of the database is downloaded, a DirectoryScan operator can be connected to the control input port (input port 1) of this operator. Whenever a new version of the database is downloaded and extracted into the directory being scanned, the DirectoryScan operator will submit the file paths to the IPSpatialEnricher operator. The operator will parse the database files and update it's internal tables.

Summary

Ports
This operator has 2 input ports and 1 output port.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 4 parameters.

Required: inputIPAttr

Optional: blocksIPv4File, blocksIPv6File, locationFile

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Never - Operator never provides a single threaded execution context.

Input Ports

Ports (0)

Ingests tuples containing IP addresses. The inputIPAttr parameter specifies the attribute containing the IP addresses. Both IPv4 and IPv6 addresses are supported.

Properties

Ports (1)

Control port that ingests a file path pointing to a MaxMind GeoIP2 (or GeoLite2) database CSV files. The operator determines whether the incoming file path refers to a "locations" file, "IPv4 blocks" file or "IPv6 blocks" file based on the name of the file. The expected file names are:

  • For the "locations" file: GeoIP2-City-Locations-en.csv or GeoLite2-City-Locations-en.csv
  • For IPv4 "blocks" file: GeoIP2-City-Blocks-IPv4.csv or GeoLite2-City-Blocks-IPv4.csv
  • For IPv6 "blocks" file: GeoIP2-City-Blocks-IPv6.csv or GeoLite2-City-Blocks-IPv6.csv

This control port can be used to dynamically update the operator's internal database. Each time a tuple is received containing a path to one of the files listed above, the operator will update it's internal table with the data in the file.

This input port expects a tuple containing a single attribute of type rstring.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Output Functions
EnrichFunctions
<any T> T AsIs()

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

<any T> T getIPLocationData()

Returns the location data associated with the IP address(es). The base return type is com.teracloud.streams.network.enrichment::IPLocation_t.

The return type of this function depends on the type of input attribute specified by the inputIPAttr parameter.

  • If the input attribute has type rstring, then the return type of this function is IPLocation_t.
  • If the input attribute has type list<rstring>, then the return type of this function is list<IPLocation_t>.
  • If the input attribute has type list<list<rstring>>, then the return type of this function is list<list<IPLocation_t>>.

<any T> T getRegisteredCountryData()

Returns the registered country data associated with the IP address(es).

The return type of this function depends on the type of input attribute specified by the inputIPAttr parameter.

  • If the input attribute has type rstring, then the return type of this function is IPLocation_t.
  • If the input attribute has type list<rstring>, then the return type of this function is list<IPLocation_t>.
  • If the input attribute has type list<list<rstring>>, then the return type of this function is list<list<IPLocation_t>>.

Note: The MaxMind© GeoIP2 database does not contain data for the latitude, longitude and postal_code information for registered country data. Therefore, the postal_code attribute will contain an empty string, and the latitude and longitude attributes will be populated with the value of -999.99.

<any T> T getRepresentedCountryData()

Returns the represented country data associated with the IP address(es).

The return type of this function depends on the type of input attribute specified by the inputIPAttr parameter.

  • If the input attribute has type rstring, then the return type of this function is IPLocation_t.
  • If the input attribute has type list<rstring>, then the return type of this function is list<IPLocation_t>.
  • If the input attribute has type list<list<rstring>>, then the return type of this function is list<list<IPLocation_t>>.

Note: The MaxMind© GeoIP2 database does not contain data for the latitude, longitude and postal_code information for represented country data. Therefore, the postal_code attribute will contain an empty string, and the latitude and longitude attributes will be populated with the value of -999.99.

Ports (0)

Submits a tuple for each input tuple received on input port 0. Custom output functions are used to specify which attribute should contain the enrichment data. The output tuple attributes whose assignments are not specified are assigned from input attributes.

Properties

Parameters

Required: inputIPAttr

Optional: blocksIPv4File, blocksIPv6File, locationFile

blocksIPv4File

Specifies the file path of the MaxMind IPv4 GeoIP2 (or GeoLite2) "blocks" file. Only the CSV-version of the database is supported. This parameter should not be used if input port 1 is defined.

Properties

blocksIPv6File

Specifies the file path of the MaxMind IPv6 GeoIP2 (or GeoLite2) "blocks" file. Only the CSV-version of the database is supported. This parameter should not be used if input port 1 is defined.

Properties

inputIPAttr

Specifies the input attribute containing the IP address (or IP addresses). The attribute can contain either IPv4 or IPv6 addresses. The supported data types for this attribute are rstring, list<rstring> and list<list<rstring>>. In the case where the input attributes refers to a type of list<rstring> or list<list<rstring>>, the list can contain a mixture of both IPv4 and IPv6 addresses.

Properties

locationFile

Specifies the file path of the MaxMind GeoIP2 (or GeoLite2) "locations" file. Only the English CSV-version of the database is supported. This parameter should not be used if input port 1 is defined.

Properties