Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPv6 address support in node addresses #561

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,11 @@ You might also need to provide your DigitalOcean access token in
the cloud controller to start, but in that case, you will not be able to
validate integration with DigitalOcean API.

The `DO_IP_ADDR_FAMILIES` is used to configure the required IP families and the
order in which the addresses should be populated in nodes status. The accepted values
are one of the `"ipv4", "ipv6"` or a comma-separated list of multiple IP address
families. IPv4 is the default, if not set or empty.

Please note that if you use a Kubernetes cluster created on DigitalOcean, there
will be a cloud controller manager running in the cluster already, so your local
one will compete for API access with it.
Expand Down
18 changes: 18 additions & 0 deletions cloud-controller-manager/do/cloud.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const (
publicAccessFirewallTagsEnv string = "PUBLIC_ACCESS_FIREWALL_TAGS"
regionEnv string = "REGION"
doAPIRateLimitQPSEnv string = "DO_API_RATE_LIMIT_QPS"
doIPAddrFamiliesEnv string = "DO_IP_ADDR_FAMILIES"
)

var version string
Expand Down Expand Up @@ -158,6 +159,11 @@ func newCloud() (cloudprovider.Interface, error) {
addr = fmt.Sprintf("%s:%s", addrHost, addrPort)
}

ipf, set := os.LookupEnv(doIPAddrFamiliesEnv)
if set && !validateAndSetIPFamilies(ipf) {
return nil, fmt.Errorf("invalid value set for environment variable %q", doIPAddrFamiliesEnv)
}

return &cloud{
client: doClient,
instances: newInstances(resources, region),
Expand Down Expand Up @@ -280,3 +286,15 @@ func (c *cloud) ScrubDNS(nameservers, searches []string) (nsOut, srchOut []strin
func (c *cloud) HasClusterID() bool {
return false
}

func validateAndSetIPFamilies(ipf string) bool {
for _, v := range strings.Split(ipf, ",") {
ipf := strings.TrimSpace(v)
if ipf == ipv4Family || ipf == ipv6Family {
ipFamilies = append(ipFamilies, IPFamily(ipf))
} else {
return false
}
}
return true
}
60 changes: 51 additions & 9 deletions cloud-controller-manager/do/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@ import (
v1 "k8s.io/api/core/v1"
)

type IPFamily string

var ipFamilies []IPFamily

const (
ipv4Family = "ipv4"
ipv6Family = "ipv6"
)

// apiResultsPerPage is the maximum page size that DigitalOcean's api supports.
const apiResultsPerPage = 200

Expand Down Expand Up @@ -129,17 +138,50 @@ func nodeAddresses(droplet *godo.Droplet) ([]v1.NodeAddress, error) {
var addresses []v1.NodeAddress
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeHostName, Address: droplet.Name})

privateIP, err := droplet.PrivateIPv4()
if err != nil || privateIP == "" {
return nil, fmt.Errorf("could not get private ip: %v", err)
// default case when DO_IP_ADDR_FAMILIES is not set
if ipFamilies == nil {
addr, err := discoverAddress(droplet, ipv4Family)
if err != nil {
return nil, fmt.Errorf("could not get addresses for %s : %v", ipv4Family, err)
}
addresses = append(addresses, addr...)
} else {
for _, i := range ipFamilies {
addr, err := discoverAddress(droplet, i)
if err != nil {
return nil, fmt.Errorf("could not get addresses for %s : %v", i, err)
}
addresses = append(addresses, addr...)
}
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: privateIP})

publicIP, err := droplet.PublicIPv4()
if err != nil || publicIP == "" {
return nil, fmt.Errorf("could not get public ip: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIP})
return addresses, nil
}

func discoverAddress(droplet *godo.Droplet, family IPFamily) ([]v1.NodeAddress, error) {
var addresses []v1.NodeAddress

switch family {
case ipv4Family:
privateIP, err := droplet.PrivateIPv4()
if err != nil || privateIP == "" {
return nil, fmt.Errorf("could not get private ip: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: privateIP})

publicIP, err := droplet.PublicIPv4()
if err != nil || publicIP == "" {
return nil, fmt.Errorf("could not get public ip: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIP})
return addresses, nil
case ipv6Family:
publicIPv6, err := droplet.PublicIPv6()
if err != nil || publicIPv6 == "" {
return nil, fmt.Errorf("could not get public ipv6: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIPv6})
return addresses, nil
}
return addresses, nil
}
20 changes: 18 additions & 2 deletions cloud-controller-manager/do/droplets_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,12 @@ func newFakeDroplet() *godo.Droplet {
Type: "public",
},
},
V6: []godo.NetworkV6{
{
IPAddress: "2a01::10",
Type: "public",
},
},
},
Region: &godo.Region{
Name: "test-region",
Expand All @@ -143,6 +149,12 @@ func newFakeShutdownDroplet() *godo.Droplet {
Type: "public",
},
},
V6: []godo.NetworkV6{
{
IPAddress: "2a01::10",
Type: "public",
},
},
},
Region: &godo.Region{
Name: "test-region",
Expand Down Expand Up @@ -186,8 +198,12 @@ func TestNodeAddresses(t *testing.T) {
Type: v1.NodeExternalIP,
Address: "99.99.99.99",
},
{
Type: v1.NodeExternalIP,
Address: "2a01::10",
},
}

ipFamilies = []IPFamily{ipv4Family, ipv6Family}
addresses, err := instances.NodeAddresses(context.TODO(), "test-droplet")

if !reflect.DeepEqual(addresses, expectedAddresses) {
Expand Down Expand Up @@ -223,7 +239,7 @@ func TestNodeAddressesByProviderID(t *testing.T) {
Address: "99.99.99.99",
},
}

ipFamilies = []IPFamily{ipv4Family}
addresses, err := instances.NodeAddressesByProviderID(context.TODO(), "digitalocean://123")

if !reflect.DeepEqual(addresses, expectedAddresses) {
Expand Down
4 changes: 3 additions & 1 deletion docs/controllers/node/examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ status:
type: InternalIP
- address: 138.197.174.81
type: ExternalIP
- address: 2a03:b0c0:3:d0::e68:a001
type: ExternalIP
allocatable:
cpu: "4"
memory: 6012700Ki
Expand All @@ -80,7 +82,7 @@ status:

DigitalOcean cloud controller manager has made the cluster aware of the size of the node, in this case c-4 (4 core high CPU droplet). It has also assigned the node
a failure domain which the scheduler can use for region failovers. Note also that the correct addresses were assigned to the node. The `InternalIP` now represents
the private IP of the droplet, and the `ExternalIP` is it's public IP.
the private IP of the droplet, and the `ExternalIP` is it's public IP. The order and IP families depends on the env variable `DO_IP_ADDR_FAMILIES`.

## Node clean up

Expand Down
2 changes: 2 additions & 0 deletions docs/example-manifests/cloud-controller-manager.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ spec:
secretKeyRef:
name: digitalocean
key: access-token
- name: DO_IP_ADDR_FAMILIES
value: ipv4,ipv6

---
apiVersion: v1
Expand Down