Building Streams applications with DPDK operators

Setting environment variables

DPDK utilizes pkg-config for applications to easily build and link to DPDK libraries. If DPDK is installed system-wide, then no action needs to be performed. Otherwise, pkg-config needs to be informed about the custom DPDK installation.

To verify pkg-config uses the correct DPDK installation, run the following: 

pkgconf --cflags libdpdk
If the libdpdk.pc file cannot be found or the location is not expected, run the following to direct pkg-config to the correct location: 

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:<path-to-dpdk-install-dir>/lib64/pkgconfig

Try the pkgconf command again to verify the DPDK install can be found.

To ensure Streams applications know where to find non-system-wide DPDK libraries, users must either:
  1. Configure ld.so.conf[.d] and run ldconfig, or
  2. Set the LD_LIBRARY_PATH environment variable
For example, to run standlone applications, LD_LIBRARY_PATH may need to be set to find the DPDK libraries: 

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path-to-dpdk-install-dir>/lib64

Gathering system information for Streams operators

To configure DPDK operators, you will need to know how many processors the machine has, and how they are distributed amoung the machine's NUMA nodes. Use this command to display that information: 


lscpu

You will also need the PCI bus address of the adapter, the Linux interface names of the adapter's ports, and the NUMA node number the adapter is connected to.

For example, the following DPDK-provided user tool can be used to find the PCI bus address and Linux interface name ethernet adapters on a system: 


dpdk-devbind.py --status

To determine the NUMA node number the adapter is connected to, check the interface or PCI device enter either of these commands at a prompt: 


cat /sys/class/net/$LINUX_INTERFACE/device/numa_node
cat /sys/devices/pciXXXX:YY/XXXX:YY:ZZ/numa_node

Building the Streams DPDK 'glue' library

The network toolkit contains C source code for a 'glue' library for DPDK operators to integrate with DPDK.

IMPORTANT: The default glue code contains configuration and settings specific to tested ethernet adapters. Users may need to modify the code for their needs. For example, hardware offloads may not be supported for certain adapters. See the streams_source/init.c file for potential changes.

If you are the owner of the Streams installation:
  1. Navigate to $STREAMS_INSTALL/toolkits/com.teracloud.streams.network/impl/src/source/dpdk
  2. Compile the code using make
If you are not able to compile the glue inside the Streams installation:
  1. Copy the network toolkit to a new location. For example, run the following command: 
    
cp -r $STREAMS_INSTALL/toolkits/com.teracloud.streams.network custom-network-toolkit
  2. Navigate to impl/src/source/dpdk in the copied toolkit
  3. Compile the code using make
  4. Specify your copied toolkit path in the -t parameter of sc when compiling or set the location in STREAMS_SPLPATH environment variable.

Once the glue library is built, Streams applications that reference the network toolkit will need to be built to include the glue library in their SABs.

Next steps

In order to run Streams applications that utilize DPDK operators, the standalone executable need to be given Linux capabilities or the Streams domain and instance need to be configured to pass the necessary capabilities to the PE when it launches. For instructions on how to do this, see Running applications with operators requiring Linux capabilities.