Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux: Update libvirt image to debian/bookworm64 and clab image to python:3.11-alpine #1623

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 152 additions & 0 deletions netsim/ansible/templates/initial/linux/debian.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
echo -n 'Starting initial config ' && date
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you just copied the "ubuntu.j2" file, added "get netplan" and removed a few things. So when we figure out something needs to be changed and we fix it in one of the files everyone knows it has to be fixed in the other file as well, right?

There's a reason Jinja2 has "include" functionality.


# Set persistent hostname
hostnamectl set-hostname {{ inventory_hostname }}

# Update APT and install netplan
apt-get update -qq
apt-get install -qq nplan

# (Overwrite resolved config to remove DNS stuff)
cat <<SCRIPT >/etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: networkd
ethernets:
eth0:
dhcp4: true
dhcp6: false
optional: true
SCRIPT
netplan apply

# cat <<SCRIPT > /etc/systemd/resolved.conf
# [Resolve]
# DNS=
# FallbackDNS=
# Domains=
# DNSOverTLS=no
# Cache=yes
# DNSStubListener=yes
# SCRIPT
# systemctl restart systemd-resolved

{% if netlab_net_tools|default(False) %}
#
# Install net-tools (arp, route...)
#
if which arp; then
echo "net-tools already installed"
else
apt-get install -qq net-tools
fi
if which traceroute; then
echo "traceroute already installed"
else
apt-get install -qq traceroute
fi
{% endif %}
{% if netlab_lldp_enable|default(False) %}
#
# Enable LLDP
#
if systemctl is-active --quiet lldpd.service; then
echo "LLDP already installed"
else
apt-get install -qq lldpd
fi

cat <<CONFIG >/etc/lldpd.d/system.conf
configure lldp tx-interval 30
configure lldp tx-hold 3
configure system interface pattern *,!eth0,eth*
CONFIG
systemctl enable lldpd
systemctl restart lldpd
{% endif %}

# Sysctl settings: IPv4/IPv6 forwarding, IPv6 LLA
#
{% set pkt_fwd = "1" if role|default("host") == "router" else "0" %}
cat <<SCRIPT > /etc/sysctl.d/10-netsim.conf
net.ipv4.ip_forward={{ pkt_fwd }}
net.ipv6.conf.all.forwarding={{ pkt_fwd }}

{% if loopback.ipv6 is defined %}
net.ipv6.conf.lo.disable_ipv6=0
{% endif %}
{% for l in interfaces|default([]) %}
{% if l.ipv6 is defined %}
net.ipv6.conf.{{ l.ifname }}.disable_ipv6=0
{% endif %}
{% endfor %}

SCRIPT
sysctl -p /etc/sysctl.d/10-netsim.conf

# Loopback addressing
{% if loopback.ipv4 is defined or loopback.ipv6 is defined %}
cat <<SCRIPT > /etc/netplan/02-loopback.yaml
network:
version: 2
renderer: networkd
ethernets:
lo:
addresses:
{% if 'ipv4' in loopback %}
- {{ loopback.ipv4 }}
{% endif %}
{% if 'ipv6' in loopback %}
- {{ loopback.ipv6 }}
{% endif %}
SCRIPT
{% endif %}

# Interface addressing
{% for l in interfaces|default([]) if (l.ipv4 is defined or l.ipv6 is defined or l.dhcp is defined)%}
cat <<SCRIPT > /etc/netplan/03-eth-{{ l.ifname }}.yaml
network:
version: 2
renderer: networkd
ethernets:
{{ l.ifname }}:
{% if l.dhcp.client.ipv4|default(False) %}
dhcp4: true
{% endif %}
{% if l.dhcp.client.ipv6|default(False) %}
dhcp6: true
{% endif %}
{% for af in ('ipv4','ipv6') if af in l %}
{% if loop.first %}
addresses:
{% endif %}
- {{ l[af] }}
{% endfor %}
{% if l.mtu is defined %}
mtu: {{ l.mtu }}
{% endif %}
SCRIPT
{% endfor %}

# Add routes to IPv4 address pools pointing to the first neighbor on the first link
{% for ifdata in interfaces|default([]) if ifdata.gateway is defined %}
cat <<SCRIPT > /etc/netplan/04-routes-{{ ifdata.ifname }}.yaml
network:
version: 2
renderer: networkd
ethernets:
{{ ifdata.ifname }}:
routes:
{% for name,pool in pools.items()|default({}) %}
{% for af,pfx in pool.items() if af == 'ipv4' and name != 'mgmt' and name != 'router_id' %}
- to: {{ pfx }}
via: {{ ifdata.gateway.ipv4|ipaddr('address') }}
{% endfor %}
{% endfor %}
SCRIPT
{% endfor %}

echo -n 'Starting netplan generate ' && date
netplan generate
echo -n 'Starting netplan apply ' && date
nohup netplan apply &
4 changes: 3 additions & 1 deletion netsim/cli/connect.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@ def ssh_connect(
if data.netlab_ssh_args:
c_args.extend(data.netlab_ssh_args.split(' '))

if data.ansible_ssh_pass:
if data.ansible_ssh_private_key_file:
c_args.extend(['-i', strings.eval_format(data.ansible_ssh_private_key_file,{'name': data.host})])
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's wrong with passing the whole node data, and if you need the 'name' parameter, you can adjust the 'adjust_inventory_host' call to retain it. This is yet another example of the "let's fix my immediate problem" mentality.

elif data.ansible_ssh_pass:
c_args = ['sshpass','-p',data.ansible_ssh_pass ] + c_args

if data.ansible_port:
Expand Down
11 changes: 5 additions & 6 deletions netsim/devices/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,23 @@ features:
server: true
relay: true
libvirt:
image: generic/ubuntu2004
image: debian/bookworm64 # generic/ubuntu2004
group_vars:
netlab_linux_distro: ubuntu
netlab_linux_distro: debian
virtualbox:
image: generic/ubuntu2004
image: debian/bookworm64 # generic/ubuntu2004
group_vars:
netlab_linux_distro: ubuntu
netlab_linux_distro: debian
group_vars:
ansible_network_os: linux
ansible_connection: paramiko
ansible_user: vagrant
ansible_ssh_pass: vagrant
docker_shell: sh -il
ansible_python_interpreter: auto_silent
netlab_lldp_enable: False
netlab_net_tools: False
clab:
image: python:3.9-alpine
image: python:3.11-alpine # Matches Python version in debian/bookworm64
mtu: 1500
kmods:
node:
Expand Down
6 changes: 3 additions & 3 deletions netsim/outputs/ansible.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@

forwarded_port_name = { 'ssh': 'ansible_port', }

def copy_provider_inventory(host: Box, p_data: Box) -> None:
def copy_provider_inventory(host: Box, p_data: Box, node: Box) -> None:
if 'inventory' in p_data:
for k,v in p_data.inventory.items():
host[k] = v
host[k] = strings.eval_format(v,node)

if 'inventory_port_map' in p_data and 'forwarded' in p_data:
for k,v in p_data.inventory_port_map.items():
Expand All @@ -42,7 +42,7 @@ def provider_inventory_settings(host: Box, node: Box, topology: Box) -> None:
node_provider = devices.get_provider(node,topology)
p_data = defaults.providers[node_provider]
if p_data:
copy_provider_inventory(host,p_data)
copy_provider_inventory(host,p_data,node)

if 'provider' in node: # Is the node using a secondary provider?
copy_device_provider_group_vars(host,node,topology)
Expand Down
3 changes: 3 additions & 0 deletions netsim/providers/libvirt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,6 @@ attributes:
uplink: str
global:
providers:

inventory:
ansible_ssh_private_key_file: .vagrant/machines/{ name }/libvirt/private_key
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is set for all Vagrant devices and probably breaks every single device that uses Vagrant default key (as in: most of them, I did a spot check on Cisco IOSv). Awesome. Just awesome.

Would you once in a while stop for a microsecond, think about the wider implications of your changes, and make the minimum amount of changes necessary?

1 change: 1 addition & 0 deletions netsim/templates/provider/libvirt/Vagrantfile.j2
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
VAGRANT_COMMAND = ARGV[0]

Vagrant.configure("2") do |config|
config.ssh.insert_key = true
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have 21 Vagrant boxes and most of them are not Linux and thus cannot have their SSH key replaced, and you go ahead and change the system default? Congratulations, great thinking.

FWIW, according to Vagrant documentation, the "true" value is equal to the default behavior, so this was completely unnecessary, but great job anyway.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured it's the system default, so this only makes explicit what's already happening. However, Vagrant has an elaborate merging process for its settings - see https://developer.hashicorp.com/vagrant/docs/vagrantfile - and so we need to make sure it's set to true for the private key file to exist in the location we expect it to, in case users may override system defaults. I experimented with setting this to false

config.vm.provider :libvirt do |libvirt|
{% if addressing.mgmt._network|default(False) %}
libvirt.management_network_name = "{{ addressing.mgmt._network }}"
Expand Down
8 changes: 4 additions & 4 deletions tests/topology/expected/anycast-gateway.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ nodes:
h1:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 4
interfaces:
Expand Down Expand Up @@ -284,7 +284,7 @@ nodes:
h2:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 5
interfaces:
Expand Down Expand Up @@ -341,7 +341,7 @@ nodes:
h3:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 6
interfaces:
Expand Down Expand Up @@ -381,7 +381,7 @@ nodes:
af:
ipv4: true
ipv6: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 13
interfaces:
Expand Down
8 changes: 4 additions & 4 deletions tests/topology/expected/components.yml
Original file line number Diff line number Diff line change
Expand Up @@ -657,7 +657,7 @@ nodes:
pod_1_l1_srv:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/srv/hosts:/etc/hosts
Expand Down Expand Up @@ -815,7 +815,7 @@ nodes:
pod_1_l2_srv:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/srv/hosts:/etc/hosts
Expand Down Expand Up @@ -1229,7 +1229,7 @@ nodes:
pod_2_l1_srv:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/srv/hosts:/etc/hosts
Expand Down Expand Up @@ -1387,7 +1387,7 @@ nodes:
pod_2_l2_srv:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/srv/hosts:/etc/hosts
Expand Down
4 changes: 2 additions & 2 deletions tests/topology/expected/device-node-defaults.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ nodes:
h1:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/h1/hosts:/etc/hosts
Expand Down Expand Up @@ -83,7 +83,7 @@ nodes:
h2:
af:
ipv4: true
box: python:3.9-alpine
box: python:3.11-alpine
clab:
binds:
- clab_files/h2/hosts:/etc/hosts
Expand Down
2 changes: 1 addition & 1 deletion tests/topology/expected/dhcp-server-on-segment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ nodes:
h1:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 1
interfaces:
Expand Down
6 changes: 3 additions & 3 deletions tests/topology/expected/dhcp-vlan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ nodes:
h1:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 3
interfaces:
Expand Down Expand Up @@ -280,7 +280,7 @@ nodes:
h2:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 4
interfaces:
Expand Down Expand Up @@ -317,7 +317,7 @@ nodes:
h3:
af:
ipv6: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 5
interfaces:
Expand Down
8 changes: 4 additions & 4 deletions tests/topology/expected/evpn-asymmetric-irb-ospf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ nodes:
h1:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 3
interfaces:
Expand Down Expand Up @@ -195,7 +195,7 @@ nodes:
h2:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 4
interfaces:
Expand Down Expand Up @@ -225,7 +225,7 @@ nodes:
h3:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 5
interfaces:
Expand Down Expand Up @@ -253,7 +253,7 @@ nodes:
h4:
af:
ipv4: true
box: generic/ubuntu2004
box: debian/bookworm64
device: linux
id: 6
interfaces:
Expand Down
Loading
Loading