My workaround is 'cn': get_hostname(ip) or ip,

To clarify. It's really not exactly clear under which circumstances this happens.
I observed it with the Octavia charm. (All the other units don't have ipv6 so it probably was never triggered elsewhere)

The code around where it happens looks totally benign. It's basically the tls-certificates interface layer plain vanilla.

What is utterly strange however is, that the issue is triggered or becomes visible only during juju agent upgrades or when the model is migrated to a different controller. I have no clue what environmental factors are triggered by that.

If you reboot the instance the bug goes away. But the v6 address stays.

I just traced the issue down to this line and not matter what triggers it, a list with None as key can't be good. And nothing in the tls-certificate layer looks in any way out of place.

Before this vault only had one request. After the juju upgrade-model and this fix to charmhelpers we have two.

before
juju show-unit vault/0

      octavia/1:
        in-scope: true
        data:
          cert_requests: '{"juju-52da4d-22-lxd-21.example.com": {"sans":
            ["100.115.0.168", "100.115.0.56"]}}'
          certificate_name: 91a7d86f-0e14-405d-a722-7c3e557b3dbb
          common_name: octavia.example.com
          egress-subnets: 100.115.0.168/32
          ingress-address: 100.115.0.168
          private-address: 100.115.0.168
          sans: '["100.115.0.168", "100.115.0.56", "octavia.example.com"]'
          unit_name: octavia_1

after
juju show-unit vault/0

      octavia/1:
        in-scope: true
        data:
          cert_requests: '{"fc00:4248:fae8:2d6d:f816:3eff:fe27:ec30": {"sans": ["fc00:4248:fae8:2d6d:f816:3eff:fe27:ec30"]},
            "juju-52da4d-22-lxd-21.example.com": {"sans": ["100.115.0.168",
            "100.115.0.56"]}}'
          certificate_name: 91a7d86f-0e14-405d-a722-7c3e557b3dbb
          common_name: octavia.example.com
          egress-subnets: 100.115.0.168/32
          ingress-address: 100.115.0.168
          private-address: 100.115.0.168
          sans: '["100.115.0.168", "100.115.0.56", "octavia.example.com"]'
          unit_name: octavia_1

Having ipv6 addresses floating around in vault caused more issues.

Am I really the only one seeing this?

What's special here is that we only use IPv6 for the amphoras. Perhaps that is unusual?

Also I found no way of re-building my charm based on a released version (with build.lock) and my patched charmhelpers. I had to manually patch the zip files. I really need something long term for that.

@fnordahl have you seen this one before?