Dedicated Interface Containers

This page describes the steps necessary to set up a perfSONAR Docker container that directly uses on of the host's physical interfaces.

This has been tested on 100 Gb/s interfaces and shows negligible loss of performance over bare metal.

Network

The container will be bound to physical interface eth1 and use IPv4 block 192.0.2.0/24 and IPv6 block 2001:db8::/32.

NOTE: Docker does not currently support IPv4 /31s but should in release 21.

docker network create -d macvlan \
                      --ipv6 \
                      --subnet=192.0.2.0/24 \
                      --gateway=192.0.2.1 \
                      --subnet=2001:db8::/32 \
                      --gateway=2001:db8::1 \
                      -o parent=eth1 \
                      host-eth1

Container

docker run -d --restart unless-stopped \
              --name=my-container \
              --hostname=my-container.example.net \
              --cpuset-cpus="0,2,4,6,8,10,12,14" \     # See note below.
              --memory=32g \
              --memory-swap=32g \
              --memory-swappiness=0 \
              --network=host-eth1 \
              --ip=192.0.2.2 \
              --ip6=2001:db8::2 \
              perfsonar/testpoint:v4.4.0

Selecting a CPU Set

The CPU set will depend on the hardware where the container is deployed and should be selected based on their bus proximity to the network interface. For example:

# Find eth1's NUMA node
$ cat /sys/class/net/eth1/device/numa_node
0

# List the system hardware by NUMA node
$ numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
node 0 size: 31966 MB
node 0 free: 2477 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
node 1 size: 32227 MB
node 1 free: 59 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10

The cores closest to eth1 are 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 and are the ones that should be used in the container's cpuset-cpus parameter.