neigh: enable garbage collection

VIPs and floating ips that move between differnet interfaces might stay
for very long times cached incorrectly in the neighbor table until the
garbage collection kicks in. by default a STALE (so an entry that used
to have an active connection but now doesn't anymore) gets garbage
collected after gc_stale_timeout, but *only* if there are more than
gc_thresh1 STALE entries in total. The default of 128 means that one has
to accumulate 128 stale entries (or trigger a forced cache flush) until
this is happening, which for small/low traffic clouds can take an
eternity.

chef/cookbooks/network/templates/default/sysctl_10gbe.conf.erb

@@ -6,6 +6,14 @@
 net.ipv4.ip_local_reserved_ports = 35357
 # Increase system IP port range to allow for more concurrent connections
 net.ipv4.ip_local_port_range = 27018 64999
+# ensure STALE neighbor entries expire from the cache, otherwise
+# VIPs of an OpenStack service or the floating IP of a VM
+# might not become reachable
+# gc_thresh1 is the lower threshold that needs to be reached before
+# stale entries are getting garbage collected. The default of 128 means
+# small clouds never get garbage collection and consequently suffer
+# from reachability issues
+net.ipv4.neigh.default.gc_thresh1 = 0
 
 #- from https://fasterdata.es.net/host-tuning/linux/
 # allow testing with buffers up to 128MB