Overview on VXLAN in the Fabric Network- Cisco ACI

Today we will discuss on the topic of VXLAN. A lot of people are confused why and where we are generally using the VXLAN. Before the VXLAN Overlay there are traditional VLAN in the network for layer 2 segment supported.

Some of the short-comes which actually VLAN have but addressed into VXLAN deployments

  • VXLAN provides greater scalability in the number of Layer 2 segments supported. Whereas VLANs are limited to just over 4000 segments, VXLAN can scale (through the use of a 24-bit ID) to up to 16 million individual segments
  • VXLAN allows extension of Layer 2 across Layer 3 boundaries through the use of MAC address in User Datagram Protocol (MAC-in-UDP) encapsulation. 

VXLAN Tunnel Endpoints
So first of all about VXLAN, VXLAN uses tunnel from source to destination and the VTEP is the virtual or the physical device that maps end devices to VXLAN segments and also VTEP doing encapsulation and de-capsulation.

As per the concept of the VTEP, It has two interfaces which will actually create the VXLAN tunnel between the segments and these are:
  • One on the local LAN segment, used to connect directly to end devices.
  • The other on the IP transport network, used to encapsulate Layer 2 frames into UDP packets and send them over the transport network. 


Fig 1.1- VTEP source and Destination

So if we are talking about the concept of the Cisco ACI, the feature VXLAN uses from source to destination. The infrastructure in Cisco ACI is fabric infra where each leaf switch is act as VTEP in the network.

VXLAN also allows the separation of location from identity. So if i am talking about the basic architecture of the Cisco ACI and the SDA platform, there is underlay IP network where you have all the IP boxes and over it there is protocol used named as LISP ( location identifier and separation Protocol ) which generally a bridge between the underlay and overlay network. 

Fig 1.2- Spine-Leaf Architecture Sample


An overlay technology such as VXLAN separates these functions and creates two name spaces: one for the identity, and another to signify where that endpoint resides.In the case of Cisco ACI, the endpoint’s IP address is the identifier, and a VTEP address designates the location (leaf) of an endpoint in the network.Cisco ACI uses a dedicated VRF and a subinterface of the uplinks as the infrastructure to carry VXLAN traffic. In Cisco ACI terminology, the transport infrastructure for VXLAN traffic is known as Overlay-1.

Regarding the Overlay-1 transport infra, the Overlay-1 VRF contains /32 routes to each VTEP, vPC virtual IP address, APIC and spine proxy IP address.

So you now understand the concept of VTEP here in the Cisco ACI infrastructure as they act as a physical endpoints in the spine-leaf architecture. With the physical tunnel points the spine switch can also be act as a proxy-TEP and is the unicast address in all spine to look into mapping database. Each VTEP address exists as a loopback on the Overlay-1 VRF. The fabric is also represented by a fabric loopback TEP (FTEP), used to encapsulate traffic in VXLAN to a vSwitch VTEP if present. Cisco ACI defines a unique FTEP address that is identical on all leaf nodes to allow mobility of downstream VTEP devices.

Now our further question, What protocol is there in the Leaf-Spine ACI network ?
  • Well in the ACI infrastructure, Fabric network is used and for the control plane inside the fabric, IS-IS protocol is used on the sub-interfaces to maintain infra reachability.
  • COOP protocol- Council of Oracles protocol runs on Proxy-TEP loopback to sync and ensure consistency of the endpoint database.
  • MP-BGP is run on the PTEP loopback to advertise external WAN routes throughout the fabric. 
  • VXLAN tunnels to PTEPs of other leaf and spine proxy TEPs.
We will further discuss about the architecture, traffic flow and some of the queries which actually comes whenever we thought about the ACI infrastructure and the next strategy of the Cisco in the market which includes SDA.