SROS VRF implementation

[fluffytrlolz]

I am having some issues implementing the VRF module for SROS within netlab and would appreciate any feedback. I have slowly been expanding the use cases of netlab while using clab as the provider. I can't get netlab to recognize and deploy VRFs on SROS. I don't know why it is skipping the playbook vrf task deployment. I've tried a few variations of the topology file to address this. Here's an example of the topology file I'm implementing:

defaults:
  device: sros
  devices.sros:
    clab.image: vrnetlab/vr-sros:21.10.R3
    clab.node.license: license-21.txt
  provider: clab

vrfs:
  red:

bgp.as: 65001
mpls.ldp: True
mpls.vpn: True
module: [isis,bgp,mpls,vrf]

nodes:
  r1:
  r2:

links: [r1-r2]

Here is the output I see from the verbose output of the "netlab up" command:

TASK [Find configuration deployment deploy_script for vrf] *********************
task path: /usr/local/lib/python3.10/dist-packages/netsim/ansible/tasks/deploy-module.yml:44
skipping: [r2] => changed=false 
  skip_reason: Conditional result was False
skipping: [r3] => changed=false 
  skip_reason: Conditional result was False
skipping: [r4] => changed=false 
  skip_reason: Conditional result was False

Thanks!

[ipspace]

I tried to reproduce your sample topology as well as I could (I don't have a SR OS container or license), and it looks like the VRF module is not used at all because the VRFs are not used anywhere.

netlab transformation code tries to optimize the configuration and resources, and sometimes removes unused components. For example, if you use OSPF module on a device but the device has no links on which it could run OSPF, the OSPF module is silently removed from that device.

Using your topology, I ran netlab create followed by netlab inspect --node r1, and the modules used by the device (the module variable) were ISIS, BGP, and MPLS.

In theory, it should be enough to have loopbacks in VRFs to make them work, but that's not how netlab works at the moment (I opened #1092 to have that fixed). Adding stub links to the VRF should get you the expected results:

vrfs:
  red:
    links: [ r1, r2 ]

[fluffytrlolz]

I tested the local copy and have the following results:

Scenario 1
In this scenario I attempted to use the new "vrf.loopback = True" parameter and removed the stub loopback. This did not deploy the VRFs or setup the vpn-ipv4 families for bgp (because vrfs were not deployed).

Topology.yml

defaults:
  device: sros
  devices.sros:
    clab.image: vrnetlab/vr-sros:21.10.R3
    clab.node.license: license-21.txt
  provider: clab

vrfs:
  red:

bgp.as: 65001
mpls.ldp: True
mpls.vpn: True
vrf.loopback: True
module: [isis,bgp,mpls,vrf]

nodes:
  r1:
  r2:

links: [r1-r2]

Inspect node 1 output

af:                                                                                                                                                       
  ipv4: true                                                                                                                                              
ansible_connection: paramiko_ssh                                                                                                                          
ansible_host: clab-netlab-r1                                                                                                                              
ansible_network_os: sros                                                                                                                                  
ansible_ssh_pass: admin                                                                                                                                   
ansible_user: admin                                                                                                                                       
bgp:                                                                                                                                                      
  advertise_loopback: true                                                                                                                                
  as: 65001                                                                                                                                               
  community:                                                                                                                                              
    ebgp:                                                                                                                                                 
    - standard                                                                                                                                            
    ibgp:                                                                                                                                                 
    - standard                                                                                                                                            
    - extended                                                                                                                                            
    ibgp_localas:                                                                                                                                         
    - standard                                                                                                                                            
    - extended                                                                                                                                            
  ipv4: true                                                                                                                                              
  neighbors:                                                                                                                                              
  - activate:                                                                                                                                             
      ipv4: true                                                                                                                                          
    as: 65001                                                                                                                                             
    ipv4: 10.0.0.2                                                                                                                                        
    name: r2                                                                                                                                              
    type: ibgp                                                                                                                                            
  next_hop_self: true                                                                                                                                     
  router_id: 10.0.0.1                                                                                                                                     
box: vrnetlab/vr-sros:21.10.R3                                                                                                                            
clab:                                                                                                                                                     
  kind: vr-sros                                                                                                                                           
  license: license-21.txt                                                                                                                                 
  type: sr-1                                                                                                                                              
hostname: clab-netlab-r1                                                                                                                                  
id: 1                                                                                                                                                     
interfaces:                                                                                                                                               
- clab:                                                                                                                                                   
    name: eth1                                                                                                                                            
  ifindex: 1                                                                                                                                              
  ifname: 1/1/c1                                                                                                                                          
  ipv4: 10.1.0.1/30                                                                                                                                       
  isis:                                                                                                                                                   
    network_type: point-to-point                                                                                                                          
    passive: false                                                                                                                                        
  ldp:                                                                                                                                                    
    passive: false                                                                                                                                        
  linkindex: 1                                                                                                                                            
  mtu: 1500                                                                                                                                               
  name: r1 -> r2                                                                                                                                          
  neighbors:                                                                                                                                              
  - ifname: 1/1/c1                                                                                                                                        
    ipv4: 10.1.0.2/30                                                                                                                                     
    node: r2                                                                                                                                              
  type: p2p                                                                                                                                               
isis:                                                                                                                                                     
  af:                                                                                                                                                     
    ipv4: true                                                                                                                                            
  area: '49.0001'                                                                                                                                         
  type: level-2                                                                                                                                           
ldp:                                                                                                                                                      
  router_id: 10.0.0.1                                                                                                                                     
loopback:                                                                                                                                                 
  ifindex: 0                                                                                                                                              
  ifname: loopback.0                                                                                                                                      
  ipv4: 10.0.0.1/32                                                                                                                                       
  neighbors: []                                                                                                                                           
  type: loopback                                                                                                                                          
  virtual_interface: true                                                                                                                                 
mgmt:                                                                                                                                                     
  ifname: A/1                                                                                                                                             
  ipv4: 192.168.121.101                                                                                                                                   
  mac: 08:4f:a9:00:00:01                                                                                                                                  
module:                                                                                                                                                   
- bgp                                                                                                                                                     
- isis                                                                                                                                                    
- mpls                                                                                                                                                    
mpls:                                                                                                                                                     
  vpn: {}                                                                                                                                                 
mtu: 1500                                                                                                                                                 
netlab_console_connection: ssh                                                                                                                            
sros_grpc_port: 57400                                                                                                                                     
sros_use_openconfig: false                                                                                                                                
vrf:                                                                                                                                                      
  as: 65000                                                                                                                                               
  loopback: true  

Scenario 2
In this scenario, I kept the original "stub loopback" and it successfully created the VRFs and deployed the vpn-ipv4 families with the BGP groups.

Topology.yml

defaults:
  device: sros
  devices.sros:
    clab.image: vrnetlab/vr-sros:21.10.R3
    clab.node.license: license-21.txt
  provider: clab



vrfs:
  red:
    links: [r1,r2]

bgp.as: 65001
mpls.ldp: True
mpls.vpn: True
#vrf.loopback: True
module: [isis,bgp,mpls,vrf]

nodes:
  r1:
  r2:

links: [r1-r2]

Inspect Node 1 Output

af:                                                                                                                                                       
  ipv4: true                                                                                                                                              
  vpnv4: true                                                                                                                                             
ansible_connection: paramiko_ssh                                                                                                                          
ansible_host: clab-netlab-r1                                                                                                                              
ansible_network_os: sros                                                                                                                                  
ansible_ssh_pass: admin                                                                                                                                   
ansible_user: admin                                                                                                                                       
bgp:                                                                                                                                                      
  advertise_loopback: true                                                                                                                                
  as: 65001                                                                                                                                               
  community:                                                                                                                                              
    ebgp:                                                                                                                                                 
    - standard                                                                                                                                            
    ibgp:                                                                                                                                                 
    - standard                                                                                                                                            
    - extended                                                                                                                                            
    ibgp_localas:                                                                                                                                         
    - standard                                                                                                                                            
    - extended                                                                                                                                            
  ipv4: true                                                                                                                                              
  neighbors:                                                                                                                                              
  - activate:                                                                                                                                             
      ipv4: true                                                                                                                                          
    as: 65001                                                                                                                                             
    ipv4: 10.0.0.2                                                                                                                                        
    name: r2                                                                                                                                              
    type: ibgp                                                                                                                                            
    vpnv4: 10.0.0.2                                                                                                                                       
  next_hop_self: true                                                                                                                                     
  router_id: 10.0.0.1                                                                                                                                     
box: vrnetlab/vr-sros:21.10.R3                                                                                                                            
clab:                                                                                                                                                     
  kind: vr-sros                                                                                                                                           
  license: license-21.txt                                                                                                                                 
  type: sr-1                                                                                                                                              
hostname: clab-netlab-r1                                                                                                                                  
id: 1                                                                                                                                                     
interfaces:                                                                                                                                               
- clab:                                                                                                                                                   
    name: eth1                                                                                                                                            
  ifindex: 1                                                                                                                                              
  ifname: 1/1/c1                                                                                                                                          
  ipv4: 10.1.0.1/30                                                                                                                                       
  isis:                                                                                                                                                   
    network_type: point-to-point                                                                                                                          
    passive: false                                                                                                                                        
  ldp:                                                                                                                                                    
    passive: false                                                                                                                                        
  linkindex: 1                                                                                                                                            
  mtu: 1500                                                                                                                                               
  name: r1 -> r2                                                                                                                                          
  neighbors:                                                                                                                                              
  - ifname: 1/1/c1                                                                                                                                        
    ipv4: 10.1.0.2/30                                                                                                                                     
    node: r2                                                                                                                                              
  type: p2p                                                                                                                                               
- bgp:                                                                                                                                                    
    advertise: true                                                                                                                                       
  bridge: netlab_2                                                                                                                                        
  clab:                                                                                                                                                   
    name: eth2                                                                                                                                            
  ifindex: 2                                                                                                                                              
  ifname: 1/1/c2                                                                                                                                          
  ipv4: 172.16.0.1/24                                                                                                                                     
  linkindex: 2                                                                                                                                            
  mtu: 1500                                                                                                                                               
  name: r1 -> stub                                                                                                                                        
  neighbors: []                                                                                                                                           
  type: stub                                                                                                                                              
  vrf: red                                                                                                                                                
isis:                                                                                                                                                     
  af:                                                                                                                                                     
    ipv4: true                                                                                                                                            
  area: '49.0001'                                                                                                                                         
  type: level-2                                                                                                                                           
ldp:                                                                                                                                                      
  router_id: 10.0.0.1                                                                                                                                     
loopback:                                                                                                                                                 
  ifindex: 0                                                                                                                                              
  ifname: loopback.0                                                                                                                                      
  ipv4: 10.0.0.1/32                                                                                                                                       
  neighbors: []                                                                                                                                           
  type: loopback                                                                                                                                          
  virtual_interface: true                                                                                                                                 
mgmt:                                                                                                                                                     
  ifname: A/1                                                                                                                                             
  ipv4: 192.168.121.101                                                                                                                                   
  mac: 08:4f:a9:00:00:01                                                                                                                                  
module:                                                                                                                                                   
- bgp                                                                                                                                                     
- isis                                                                                                                                                    
- mpls                                                                                                                                                    
- vrf                                                                                                                                                     
mpls:                                                                                                                                                     
  vpn:                                                                                                                                                    
    ipv4:                                                                                                                                                 
    - ibgp                                                                                                                                                
mtu: 1500                                                                                                                                                 
netlab_console_connection: ssh                                                                                                                            
sros_grpc_port: 57400                                                                                                                                     
sros_use_openconfig: false                                                                                                                                
vrf:                                                                                                                                                      
  as: 65000                                                                                                                                               
vrfs:                                                                                                                                                     
  red:                                                                                                                                                    
    af:                                                                                                                                                   
      ipv4: true                                                                                                                                          
    export:                                                                                                                                               
    - '65001:1'                                                                                                                                           
    id: 1                                                                                                                                                 
    import:                                                                                                                                               
    - '65001:1'                                                                                                                                           
    rd: '65001:1'                                                                                                                                         
    vrfidx: 100 

[ipspace]

Thanks a million for the feedback. I haven't fixed scenario #1 yet (see #1092), it's nice to hear scenario #2 is now working.

[fluffytrlolz]

No problem, thank you for this fantastic tool! I'm just getting started with netlab and excited about all that it offers. I was in the process of slowly developing my own templates for clab deployments (big fan and user of clab for a while), when I heard about this tool on packet pushers that did exactly what I was after. The netlab + clab implementation has been super helpful to quickly stand up, test, and tinker with various networking scenarios.

[ipspace]

Scenario #1 should be fixed in be5743c (#1109)

[fluffytrlolz]

I pulled the latest repo down today and ran 2 tests:

Test 1

I declared the single red VRF. I got a warning message on the creation (shown below), and no vrfs were created on the nodes. With this warning message I went ahead and ran Test 2

[WARNING] vrf: Node r1 uses no VRFs, removing 'vrf' from node modules
[HINT]    A globally-defined VRF is not used on a node unless that node is attached to a VRF link, or has the
          VRF listed in the node 'vrfs' dictionary and uses VRF loopbacks.
[WARNING] vrf: Node r2 uses no VRFs, removing 'vrf' from node modules
[CREATED] provider configuration file: clab.yml
[CREATED] transformed topology dump in YAML format in netlab.snapshot.yml
[GROUPS]  group_vars for all
[GROUPS]  group_vars for modules
[GROUPS]  group_vars for sros
[HOSTS]   host_vars for r1
[HOSTS]   host_vars for r2
[CREATED] minimized Ansible inventory hosts.yml
[CREATED] Ansible configuration file: ansible.cfg

Topology.yml

defaults:
  device: sros
  devices.sros:
    clab.image: vrnetlab/vr-sros:21.10.R3
    clab.node.license: license-21.txt
  provider: clab

vrfs:
  red:

bgp.as: 65001
mpls.ldp: True
mpls.vpn: True
vrf.loopback: True
module: [isis,bgp,mpls,vrf]

nodes:
  r1:
  r2:

links: [r1-r2]

Test 2

I nested the vrfs definition under each node as well as kept the vrfs global definition. Once I kept the global and nested the vrfs under each node, it successfully created the vrfs.

When I didn't have the global definition it complained

[FATAL]   Initial vrf_id value is not set, get_next_id failed

Topology.yml

defaults:
  device: sros
  devices.sros:
    clab.image: vrnetlab/vr-sros:21.10.R3
    clab.node.license: license-21.txt
  provider: clab

vrfs:
  red:

bgp.as: 65001
mpls.ldp: True
mpls.vpn: True
vrf.loopback: True
module: [isis,bgp,mpls,vrf]

nodes:
  r1:
    vrfs:
      red:
  r2:
    vrfs:
      red:
links: [r1-r2]

Is this the expected result of these tests and usage? I thought that if I declared the VRF parameters globally, each node would inherit unless I somehow stated not to include... However, this may be a misinterpretation of the tool on my part and I need to define the VRF globally and then specifically say what nodes are to use the specific VRFs. Either use case makes sense to me, and I'm thinking I misinterpreted the use of the tool. Thanks for the bug fix!

[ipspace]

I declared the single red VRF. I got a warning message on the creation (shown below), and no vrfs were created on the nodes.

As expected. Thank you.

I nested the vrfs definition under each node as well as kept the vrfs global definition. Once I kept the global and nested the vrfs under each node, it successfully created the vrfs.

Great to hear that, we squashed this one ;)

When I didn't have the global definition it complained

[FATAL]   Initial vrf_id value is not set, get_next_id failed

And you uncovered another VRF bug (#1110). Thank you!

Is this the expected result of these tests and usage?

Yes.

I thought that if I declared the VRF parameters globally, each node would inherit unless I somehow stated not to include...

We took the opposite approach: we're not copying VRFs into nodes unless they are needed/required. While that's now mentioned in the documentation (not published yet, but the source code is in the repo), I will write a blog post to emphasize it.

I need to define the VRF globally and then specifically say what nodes are to use the specific VRFs.

You don't have to specify that a node uses a VRF if it has an interface (link) in that VRF. If you want to have "fully virtual" VRFs then you have to specify them.