Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPv6 address support in node addresses #561

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ package-hosted directory like this:

```bash
cd cloud-controller-manager/cmd/digitalocean-cloud-controller-manager
REGION=fra1 DO_ACCESS_TOKEN=your_access_token go run main.go \
REGION=fra1 DO_ACCESS_TOKEN=your_access_token DO_IP_ADDR_FAMILIES=ipv4 go run main.go \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd omit the usage from this example since it's not strictly needed when running locally.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

--kubeconfig <path to your kubeconfig file> \
--leader-elect=false --v=5 --cloud-provider=digitalocean
```
Expand All @@ -79,6 +79,11 @@ You might also need to provide your DigitalOcean access token in
the cloud controller to start, but in that case, you will not be able to
validate integration with DigitalOcean API.

The `DO_IP_ADDR_FAMILIES` is used to configure the required IP familes and the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: families

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

order in which address should be populated in nodes status. The accepted values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
order in which address should be populated in nodes status. The accepted values
order in which the addresses should be populated in nodes status. The accepted values

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

are one of the `{"", "ipv4", "ipv6", "ipv4,ipv6", "ipv6,ipv4"}`.IPv4 is the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have curly braces here? It almost looks like JSON but we don't require JSON, do we?

Can we rephrase this in a more user friendly manner, e.g., say one of "ipv4", "ipv6", or a comma-separated list of multiple IP address families?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

default, if not set or empty.

Please note that if you use a Kubernetes cluster created on DigitalOcean, there
will be a cloud controller manager running in the cluster already, so your local
one will compete for API access with it.
Expand Down
20 changes: 20 additions & 0 deletions cloud-controller-manager/do/cloud.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,14 @@ const (
publicAccessFirewallTagsEnv string = "PUBLIC_ACCESS_FIREWALL_TAGS"
regionEnv string = "REGION"
doAPIRateLimitQPSEnv string = "DO_API_RATE_LIMIT_QPS"
doIPAddrFamiliesEnv string = "DO_IP_ADDR_FAMILIES"
)

var version string

var ipFamilyOpts = []string{"", "ipv4", "ipv6", "ipv4,ipv6", "ipv6,ipv4"}
sachintiptur marked this conversation as resolved.
Show resolved Hide resolved
var ipFamilies string

type tokenSource struct {
AccessToken string
}
Expand Down Expand Up @@ -158,6 +162,13 @@ func newCloud() (cloudprovider.Interface, error) {
addr = fmt.Sprintf("%s:%s", addrHost, addrPort)
}

// var ipFamilies string
ipf, set := os.LookupEnv(doIPAddrFamiliesEnv)
ipFamilies = ipf
if set && !containsString(ipFamilyOpts, ipFamilies) {
sachintiptur marked this conversation as resolved.
Show resolved Hide resolved
return nil, fmt.Errorf("invalid value set for environment variable %q", doIPAddrFamiliesEnv)
}

return &cloud{
client: doClient,
instances: newInstances(resources, region),
Expand Down Expand Up @@ -280,3 +291,12 @@ func (c *cloud) ScrubDNS(nameservers, searches []string) (nsOut, srchOut []strin
func (c *cloud) HasClusterID() bool {
return false
}

func containsString(vals []string, val string) bool {
for _, v := range vals {
if v == val {
return true
}
}
return false
}
41 changes: 33 additions & 8 deletions cloud-controller-manager/do/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"context"
"errors"
"fmt"
"strings"

"github.com/digitalocean/godo"
v1 "k8s.io/api/core/v1"
Expand Down Expand Up @@ -129,17 +130,41 @@ func nodeAddresses(droplet *godo.Droplet) ([]v1.NodeAddress, error) {
var addresses []v1.NodeAddress
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeHostName, Address: droplet.Name})

privateIP, err := droplet.PrivateIPv4()
if err != nil || privateIP == "" {
return nil, fmt.Errorf("could not get private ip: %v", err)
for _, i := range strings.Split(ipFamilies, ",") {
addr, err := discoverAddress(droplet, i)
if err != nil {
return nil, fmt.Errorf("could not get addresses for %s : %v", i, err)
}
addresses = append(addresses, addr...)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: privateIP})

publicIP, err := droplet.PublicIPv4()
if err != nil || publicIP == "" {
return nil, fmt.Errorf("could not get public ip: %v", err)
return addresses, nil
}

func discoverAddress(droplet *godo.Droplet, family string) ([]v1.NodeAddress, error) {
var addresses []v1.NodeAddress
if family == "ipv4" || family == "" {
sachintiptur marked this conversation as resolved.
Show resolved Hide resolved
privateIP, err := droplet.PrivateIPv4()
if err != nil || privateIP == "" {
return nil, fmt.Errorf("could not get private ip: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeInternalIP, Address: privateIP})

publicIP, err := droplet.PublicIPv4()
if err != nil || publicIP == "" {
return nil, fmt.Errorf("could not get public ip: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIP})
return addresses, nil
}
if family == "ipv6" {
publicIPv6, err := droplet.PublicIPv6()
if err != nil || publicIPv6 == "" {
return nil, fmt.Errorf("could not get public ipv6: %v", err)
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIPv6})
return addresses, nil
}
addresses = append(addresses, v1.NodeAddress{Type: v1.NodeExternalIP, Address: publicIP})

return addresses, nil
}
20 changes: 20 additions & 0 deletions cloud-controller-manager/do/droplets_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,12 @@ func newFakeDroplet() *godo.Droplet {
Type: "public",
},
},
V6: []godo.NetworkV6{
{
IPAddress: "2a01::10",
Type: "public",
},
},
},
Region: &godo.Region{
Name: "test-region",
Expand All @@ -143,6 +149,12 @@ func newFakeShutdownDroplet() *godo.Droplet {
Type: "public",
},
},
V6: []godo.NetworkV6{
{
IPAddress: "2a01::10",
Type: "public",
},
},
},
Region: &godo.Region{
Name: "test-region",
Expand Down Expand Up @@ -186,6 +198,10 @@ func TestNodeAddresses(t *testing.T) {
Type: v1.NodeExternalIP,
Address: "99.99.99.99",
},
{
Type: "public",
Address: "2a01::10",
},
}

addresses, err := instances.NodeAddresses(context.TODO(), "test-droplet")
Expand Down Expand Up @@ -222,6 +238,10 @@ func TestNodeAddressesByProviderID(t *testing.T) {
Type: v1.NodeExternalIP,
Address: "99.99.99.99",
},
{
Type: "public",
Address: "2a01::10",
},
}

addresses, err := instances.NodeAddressesByProviderID(context.TODO(), "digitalocean://123")
Expand Down
4 changes: 3 additions & 1 deletion docs/controllers/node/examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ status:
type: InternalIP
- address: 138.197.174.81
type: ExternalIP
- address: 2a03:b0c0:3:d0::e68:a001
type: ExternalIP
allocatable:
cpu: "4"
memory: 6012700Ki
Expand All @@ -80,7 +82,7 @@ status:

DigitalOcean cloud controller manager has made the cluster aware of the size of the node, in this case c-4 (4 core high CPU droplet). It has also assigned the node
a failure domain which the scheduler can use for region failovers. Note also that the correct addresses were assigned to the node. The `InternalIP` now represents
the private IP of the droplet, and the `ExternalIP` is it's public IP.
the private IP of the droplet, and the `ExternalIP` is it's public IP. The order and IP families depends on the env variable `DO_IP_ADDR_FAMILIES`.

## Node clean up

Expand Down
2 changes: 2 additions & 0 deletions docs/example-manifests/cloud-controller-manager.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ spec:
secretKeyRef:
name: digitalocean
key: access-token
- name: DO_IP_ADDR_FAMILIES
value: ipv4,ipv6

---
apiVersion: v1
Expand Down