Cisco Nexus: Why Enable & Recommended vPC Peer-Gateway
In this article, I am going to talk about the requirement of the peer-gateway in the nexus environment. We will explain the requirement with the help of the example and topology as shown below.
 |
Fig 1.1- vPC Peer-Gateway Logical Topology
|
Let us suppose, in the above-mentioned topology that:
Host-1 in vlan 10 - IP 10.1.10.3
Host-2 in vlan 20 - IP 10.1.20.4
When host-1 and host-2 are assigned with FHRP VIP (10.1.x.250) as the default gateway there is no need for peer-gateway.Initiate ping from host-1 in vlan 10 to host-2 in vlan 20. Host-1 will use MAC of vlan 10 VMAC as Destination MAC.
As per vPC implementation either of the N7K irrespective of FHRP Active/Standby can forward data traffic as both will have Gateway 'G' bit set for FHRP VIP MAC Address as below
vPC peer-gateway disabled:
N7K1(config)# sh mac add vlan 10
G 10 0000.0c07.ac0a static - F F sup-eth1(R)
G 10 6c9c.ed40.28c1 static - F F sup-eth1(R)
* 10 6c9c.ed4a.9141 static - F F vPC Peer-Link
N7K1(config)# sh mac add vlan 20
G 20 0000.0c07.ac14 static - F F sup-eth1(R)
G 20 6c9c.ed40.28c1 static - F F sup-eth1(R)
* 20 6c9c.ed4a.9141 static - F F vPC Peer-Link
N7K2(config)# sh mac add vlan 10
G 10 0000.0c07.ac0a static - F F sup-eth1(R)
* 10 6c9c.ed40.28c1 static - F F vPC Peer-Link
G 10 6c9c.ed4a.9141 static - F F sup-eth1(R)
N7K2(config)# sh mac add vlan 20
G 20 0000.0c07.ac14 static - F F sup-eth1(R)
* 20 6c9c.ed40.28c1 static - F F vPC Peer-Link
G 20 6c9c.ed4a.9141 static - F F sup-eth1(R)
If you have host-1 assigned with default gateway IP of N7K1 vlan 10 SVI (10.1.10.1).
Initiate ping from host-1 in vlan 10 to host-2 in vlan 20 and assume that packet is first received by N7K2 on vPC 10.
Host-1 will use MAC of N7K1 as Destination MAC.
N7K2 will forward the packet to N7K1 via peer-link where the inter-vlan routing occurs.
N7K1 has to forward the packet to host-2 via vPC 20, where vPC loop prevention mechanism kicks in and eventually the packet will get dropped.
vPC loop-prevention states that if a frame is received on a vPC port-channel and crosses the vPC peer-link then it cannot be forwarded out to another vPC port-channel, the exception to this rule being orphan ports and vPC port-channels with only one side UP.
Now enable peer-gateway under vPC domain
N7K(config)# vpc domain 10
N7K(config-vpc-domain)# peer-gateway
How Peer-Gateway solves the issue?
Each N7K in a vPC sets Gateway bit 'G' for its peer N7K MAC.
Now, even if N7K2 receives the packet with Destination MAC of the packet being N7K1, N7K2 will perform inter-vlan routing and forward the packet towards host-2 through vPC 20 PO, thus to avoid the packet to cross the peer-link and get dropped.
N7K1# sh mac add vlan 10
G 10 0000.0c07.ac0a static - F F sup-eth1(R)
G 10 6c9c.ed40.28c1 static - F F sup-eth1(R)
G 10 6c9c.ed4a.9141 static - F F sup-eth1(R)
N7K1# sh mac add vlan 20
G 20 0000.0c07.ac14 static - F F sup-eth1(R)
G 20 6c9c.ed40.28c1 static - F F sup-eth1(R)
G 20 6c9c.ed4a.9141 static - F F sup-eth1(R)
N7K2# sh mac add vlan 10
G 10 0000.0c07.ac0a static - F F sup-eth1(R)
G 10 6c9c.ed40.28c1 static - F F sup-eth1(R)
G 10 6c9c.ed4a.9141 static - F F sup-eth1(R)
N7K2# sh mac add vlan 20
G 20 0000.0c07.ac14 static - F F sup-eth1(R)
G 20 6c9c.ed40.28c1 static - F F sup-eth1(R)
G 20 6c9c.ed4a.9141 static - F F sup-eth1(R)
Why would anyone use one of N7Ks interface IP as default gateway instead of FHRP VIP?
The above scenario has been quoted just for example, however it has been to be noted that some storage vendor servers though been assigned with FHRP VIP as default-gateway, when sending packets to default gateway to get routed, uses one of the Gateway's interface MAC (N7K that is HSRP Active) as its destination MAC instead of using FHRP VMAC as its destination MAC.
In the above topology assume N7K1 is the HSRP Active. If Host-1 in vlan 10 is from that particular storage vendor then it will use N7K1's interface MAC 6c9c.ed40.28c1 instead of HSRP VMAC 0000.0c07.ac0a.
In Summary
Have peer-gateway enabled for all vPC vlans to avoid issues caused due to storage servers of some vendors.
Disable peer-gateway for a vlan when SVI for that vlan is down one of the vPC pairs to avoid drops of some inter vlan routed traffic. This is irrespective of whether peer-link is configured on M1 or F1 modules.
In case of peer-link configured on F1 modules, with peer-gateway enabled and SVI on one of the vlan is down on one of the vPC pairs, then it may look as if layer-2 traffic is impacted when actually its not. Only traffic to SVI IP of N7K and that bound to be process switched are impacted.
