[BGP]FRR doesn't relay EVPN Encapsulation Extended Communities (ET: 8) to eBGP neighbors

[nitinbhat]

Description

Summary

When acting as a route reflector, FRR does not advertise EVPN Extended Communities containing Encapsulation Type (ET:8, VXLAN) to eBGP EVPN peers, even when correctly configured with send-community all at the neighbor level. This results in downstream devices being unable to properly interpret the encapsulation type for type-5 EVPN routes and defaulting to MPLS instead of VXLAN.

Environment

• FRR Version: 10.3.1_git (confirmed current behavior)
• Topology: Leaf-Spine (Route Reflector scenario)
• Leaf (OcNOS): 192.168.3.3 (eBGP with FRR spine)
• Spine (FRR): 192.168.3.250 (eBGP with leafs, RR for EVPN)

Steps to Reproduce

1. Set up a BGP EVPN leaf-spine fabric with FRR as route reflector:

   • leafs and spine running EVPN, all configured with send-community all on all EVPN neighbors

2. Learn type-5 prefix routes with ET:8 (VXLAN) in extended community from upstream leafs

3. Check advertised routes from the spine to a leaf (eBGP):

   show bgp l2vpn evpn neighbors <leaf-ip> advertised-routes
   

4. No extended communities (including ET:8) are present in advertisements, even though received routes have ET:8.

Expected Behavior

• FRR should relay all received extended communities (including encapsulation ET:8) to downstream eBGP peers when send-community all is set.
• Downstream EVPN receivers (like OcNOS) should be able to interpret VXLAN encapsulation correctly.

Actual Behavior

• Extended Community (ET: 8) is present in received routes:

  Route Distinguisher: 192.168.11.2:3
   *>  [5]:[0]:[24]:[192.168.11.0]
       100.65.0.2               0             0 64512 ?
       RT:64512:100 ET: 8 Rmac:aa: bb:cc:00:00:65
  

• But when advertising to 192.168.3.3 (OcNOS or any other frr leaf):

  show bgp l2vpn evpn neighbors 192.168.3.3 advertised-routes
  Route Distinguisher: 192.168.11.2:3
   *> [5]:[0]:[24]:[192.168.11.0]
                                      0 64512 ?
  (No Extended Community line)
  

• A permissive route-map does not resolve the issue.

Impact

• Interoperability breaks for VXLAN EVPN overlays when FRR is used as route reflector.
• Devices downstream may default to MPLS (incorrect) because Extended Community is missing.

Workarounds Tried

• send-community all at neighbor and global BGP level
• Adding a route-map (permit any) on the RR neighbor
• Restarting BGP sessions
• Upstream routes are confirmed to include ET:8, but not advertised to neighbors

Request

Please investigate and advise:

• Is this a known bug in FRR 10.3.1?
• Are there configuration steps required to ensure FRR relays EVPN encapsulation extended communities to eBGP peers?
• Is there a version where this is resolved?

Let us know if further logs or PCAPs are required.

Version

FRRouting 10.3.1_git (spine) on Linux(5.10.0-22-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--sbindir=/usr/lib/frr' '--libdir=/usr/lib' '--enable-rpki' '--enable-vtysh' '--enable-multipath=64' '--enable-vty-group=frrvty' '--enable-user=frr' '--enable-group=frr' '--enable-pcre2posix' '--enable-scripting' 'CC=gcc' 'CXX=g++'

How to reproduce

[Leaf1 (AS 64512)] ----eBGP---- [Spine/RR (AS 64612)] ----eBGP---- [Leaf2 (AS 2202)]
VXLAN EVPN FRR 10.3.1 OcNOS/FRR

###Spine (FRR 10.3.1 - Route Reflector):

router bgp 64612
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 neighbor 192.168.1.1 remote-as 64512
 neighbor 192.168.3.3 remote-as 2202
 neighbor 192.168.1.1 send-community all
 neighbor 192.168.3.3 send-community all
 ! 
 address-family l2vpn evpn
  neighbor 192.168.1.1 activate
  neighbor 192.168.3.3 activate
  advertise-all-vni
 exit-address-family
exit

###Leaf1 (Originating EVPN routes with VXLAN):

router bgp 64512
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 neighbor 192.168.3.250 remote-as 64612
 neighbor 192.168.3.250 send-community extended
 ! 
 address-family l2vpn evpn
  neighbor 192.168.3.250 activate
  advertise-all-vni
  advertise ipv4 unicast
 exit-address-family
exit
! 
interface vxlan1
 vxlan id 100
 vxlan local-tunnelip 100. 64.0.1
exit

Expected behavior

Encapsulation should be relayed to other peers as received

Actual behavior

On spines

show bgp l2vpn evpn

Route Distinguisher: 192.168.10.2:3
 *>  [5]:[0]:[24]:[192.168.10.0]
     100.64.0.1               0             0 64512 ?
     RT:64512:100 ET: 8 Rmac:aa: bb:cc:00:00:64

show bgp l2vpn evpn neighbors 192.168.3.3 advertised-routes

Route Distinguisher: 192.168.10.2:3
 *> [5]:[0]:[24]:[192.168.10.0]
                                    0 64512 ?
(No Extended Community - ET:8 is missing!)

Attempted workaround with route map

route-map EVPN_OUT permit 10
exit
router bgp 64612
 address-family l2vpn evpn
  neighbor 192.168.3.3 route-map EVPN_OUT out
 exit-address-family
exit
clear bgp l2vpn evpn 192.168.3.3 soft out

Additional context

No response

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.

[ton31337]

Just to double check: is this a regression and was working fine before 10.3 (e.g. 10.2)? Or is this just a "visibility issue" (not showing in advertised routes)?

[nitinbhat]

I did not check it on previous versions. I tried it on 10.3.1 and also latest (10.5.0) Behaviour remains same

[ton31337]

Ok, I will verify.

[ton31337]

Are you saying that you don't see just the ECs with show bgp l2vpn evpn neighbors 192.168.3.3 advertised-routes or actually on the receiving router?

[ton31337]

And I'm confused with your topology a bit. Could you give us the full configurations?

[nitinbhat]

I don't see ECs with VXLAN encapsulation sent from spine to leafs and also not seen with show bgp l2vpn evpn neighbors 192.168.3.3 advertised-routes

This is the flow

• There are few hosts connected to leaf1 and leaf2
• leaf1 and leaf2 will send type-5 routes to spines
• Spine acts as route-reflector and sends type-5 routes received from leaf2 to leaf1 and vice versa
• When spine relays this type-5 routes, it won't include ECs with VXLAN encapsulation (but leaf1 and leaf2 have sent it to spine)

Spine

debug zebra events
    debug zebra vxlan
    debug bgp zebra
    debug zebra events
    debug zebra nht
    debug zebra kernel
    debug zebra rib
    debug zebra nexthop
    debug bgp neighbor-events
    debug bgp updates
    debug bgp keepalives
    debug bgp nht
    debug bgp zebra
    debug bfd network
    debug bfd peer
    debug bfd zebra

    router bgp 64612
    no bgp ebgp-requires-policy
    no bgp network import-check
    no bgp default ipv4-unicast

    neighbor 192.168.1.1 remote-as 64512
    neighbor 192.168.3.3 remote-as 2202


    address-family ipv4 unicast
      neighbor 192.168.1.1 activate
      neighbor 192.168.3.3 activate
    exit-address-family
    !
    address-family l2vpn evpn
      neighbor 192.168.1.1 activate
      neighbor 192.168.3.3 activate
      advertise-all-vni
      advertise-svi-ip
    exit-address-family

    exit

Leaf-1

debug zebra events
    debug zebra vxlan
    debug bgp zebra
    debug zebra events
    debug zebra nht
    debug zebra kernel
    debug zebra rib
    debug zebra nexthop
    debug bgp neighbor-events
    debug bgp updates
    debug bgp keepalives
    debug bgp nht
    debug bgp zebra
    debug bfd network
    debug bfd peer
    debug bfd zebra
    !
    interface eth2
    !
    interface eth3
    !
    vrf red
      vni 100
    exit-vrf
    !
    vrf blue
      vni 200
    exit-vrf
    !
    router bgp 64512
    no bgp ebgp-requires-policy
    no bgp network import-check
    no bgp default ipv4-unicast

    neighbor 192.168.1.250 remote-as 64612
    neighbor 192.168.1.250 allowas-in origin
    !
    address-family ipv4 unicast
      neighbor 192.168.1.250 activate
      network 100.64.0.1/32
      redistribute connected
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      neighbor 192.168.1.250 activate
      neighbor 192.168.1.250 allowas-in origin
      advertise-all-vni
      advertise-svi-ip
      advertise ipv4 unicast
    exit-address-family
    exit
    !
    router bgp 64512 vrf blue
    !
    address-family ipv4 unicast
      redistribute connected
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      advertise ipv4 unicast
    exit-address-family
    exit
    !
    router bgp 64512 vrf red
    !
    address-family ipv4 unicast
      redistribute connected
    exit-address-family
    !
    address-family ipv6 unicast
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      advertise ipv4 unicast
      advertise ipv6 unicast
    exit-address-family
    exit
    !

Leaf2

debug zebra events
    debug zebra vxlan
    debug bgp zebra
    debug zebra events
    debug zebra nht
    debug zebra kernel
    debug zebra rib
    debug zebra nexthop
    debug bgp neighbor-events
    debug bgp updates
    debug bgp keepalives
    debug bgp nht
    debug bgp zebra
    debug bfd network
    debug bfd peer
    debug bfd zebra
    !
    interface eth2
    !
    interface eth3
    !
    vrf red
      vni 100
    exit-vrf
    !
    vrf blue
      vni 200
    exit-vrf
    !
    router bgp 2202
    no bgp ebgp-requires-policy
    no bgp network import-check
    no bgp default ipv4-unicast

    neighbor 192.168.3.250 remote-as 64612
    neighbor 192.168.3.250 allowas-in origin
    !
    address-family ipv4 unicast
      neighbor 192.168.3.250 activate
      network 100.63.0.3/32
      redistribute connected
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      neighbor 192.168.3.250 activate
      neighbor 192.168.3.250 allowas-in origin
      advertise-all-vni
      advertise-svi-ip
      advertise ipv4 unicast
    exit-address-family
    exit
    !
    router bgp 2202 vrf blue
    !
    address-family ipv4 unicast
      redistribute connected
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      advertise ipv4 unicast
    exit-address-family
    exit
    !
    router bgp 2202 vrf red
    !
    address-family ipv4 unicast
      redistribute connected
    exit-address-family
    !
    address-family ipv6 unicast
      redistribute static
    exit-address-family
    !
    address-family l2vpn evpn
      advertise ipv4 unicast
      advertise ipv6 unicast
    exit-address-family
    exit
    !

Note: Spine configuration was modified to include

neighbor 192.168.1.1 send-community all
neighbor 192.168.3.3 send-community all

which didn't help

[nitinbhat]

Please let me know if you need more information on this issue