Home / Datacenter / Introduction to Cisco Nexus 7000 vPC Auto-Recovery Feature

Introduction to Cisco Nexus 7000 vPC Auto-Recovery Feature

Today I am going to talk about the vPC Auto-Recovery feature which is generally used in many of the datacenter environment. Before we start with the configuration part of it ,first i would like to discuss on the basics of the vPC Auto-Recovery feature and we required this feature.

Why do we need vPC Auto-Recovery?
There are two main reasons for this vPC enhancement:
In a data center outage or power outage, both vPC peers that are comprised of Nexus 7000 switches are off. Occasionally, only one of the peers can be restored. Since the other Nexus 7000 is still off, the vPC peer-link and the vPC peer-keepalive link are also off. 

In this scenario, the vPC does not come on even for the Nexus 7000 which is already on. All vPC configurations have to be removed from the port-channel on that Nexus 7000 to cause the port-channel to work. When the other Nexus 7000 comes on, you have to again make configuration changes to include the vPC configuration for all the vPCs. In Release 5.0(2) and later, you can configure the reload restore command under the vPC domain configuration to address this problem.

For some reason, the vPC peer-link goes off. Since the vPC peer-keepalive is still on, the vPC secondary peer device turns all its vPC member ports off due to dual-active detection. 

Hence all the traffic goes through the vPC primary switch. For some reason, the vPC primary switch also goes off. This switch problem black holes the traffic since the vPCs on the secondary peer device are still off because it detected dual-active detection before the vPC primary switch went off.

How does auto-recovery really work?
The assumption is that vPC auto-recovery is configured and saved to the startup configuration on both switches S1 and S2.

Case-1:A power outage shuts off both Nexus 7000 vPC peers simultaneously and only one switch is able to come on.

Fig 1.1- vPC Scenario-1
  • S1 and S2 are both on. vPC is formed correctly with peer-link and peer-keepalive on.
  • Both S1 and S2 power off simultaneously.
  • Now only one switch is able to power on. For example, S2 is the only switch which powers on.
  • S2 waits for the vPC auto-recovery timeout (the default is 240 seconds which can be configured with the auto-recovery reload-delay x command, where x is 240-3600 seconds) in order to verify if either vPC peer-link or peer-keepalive status powers on. If any of these links is on (peer-link or peer-keepalive status), auto-recovery is not triggered.
  • After the timeout, if both links are still off (peer-link as well as peer-keepalive status), vPC auto-recovery enables and S2 becomes primary and initiates in order to power on its local vPC. Since there are no peers, the consistency check is bypassed.
  • Now S1 comes on. At this time, S2 retains its primary role and S1 takes a secondary role, a consistency check is performed, and appropriate actions are taken.
Case-2: vPC peer-link powers off first and then the primary vPC peer powers off.

Fig 1.2- vPC Scenario-2
  • S1 and S2 are both on and vPC is formed correctly with peer-link and peer-keepalive on.
  • For some reason, vPC peer-link goes off first.
  • Since vPC peer-keepalive is still on, it detects dual-active detection. The vPC secondary S2 turns off all its local vPCs.
  • Now the vPC primary S1 goes off or reloads.
  • This outage also turns off the vPC peer-keepalive link.
  • S2 waits for three consecutive peer-keepalive messages to be lost. For some reason, either the vPC peer-link comes on or S2 receives a peer-keepalive message, and auto-recovery does not enable.
  • However, if the peer-link remains off and three consecutive peer-keepalive messages are lost, vPC auto-recovery enables.
  • S2 assumes the role of primary and enables its local vPC which bypasses the consistency check.
  • When S1 completes the reload, S2 retains its role of primary and S1 becomes secondary, a consistency check is performed, and appropriate actions are taken.
Should I enable vPC auto-recovery?
It is a good practice to enable auto-recovery in your vPC environment.
There is a slight chance that the vPC auto-recovery feature might create a dual-active scenario. For example, if you first lost the peer-link and then you lost the peer-keepalive, you will have dual-active scenario.

In this situation, each vPC member port continues to advertise the same Link Aggregation Control Protocol ID that it did before the dual-active failure.

A vPC topology intrinsically protects from loops in case of dual-active scenarios. In a worst case scenario, there are duplicate frames. Despite this, as a loop-prevention mechanism, each switch forwards Bridge Protocol Data Units (BPDUs) with the same BPDU Bridge ID as prior to the vPC dual-active failure.

While not intuitive, it is still possible and desirable to continue to forward traffic from the access layer to the aggregation layer without drops for current traffic flows, provided that the Address Resolution Protocol (ARP) tables are already populated on both Cisco Nexus 7000 Series peers for all needed hosts.

If new MAC addresses need to be learned by the ARP table, issues might arise. The issues arise because the ARP response from the server might be hashed to one Cisco Nexus 7000 Series device and not to the other, which makes it impossible for the traffic to flow correctly.

Suppose, however, that before the failure in the situation just described, traffic was equally distributed to both Cisco Nexus 7000 Series devices by a correct PortChannel and by an Equal Cost Multipath (ECMP) configuration. In this case, server-to-server and client-to-server traffic continues with the caveat that single-attached hosts connected directly to the Cisco Nexus 7000 Series will not be able to communicate (for the lack of the peer link). 

Also, new MAC addresses learned on one Cisco Nexus 7000 Series cannot be learned on the peer, because this would cause the return traffic that arrives on the peer Cisco Nexus 7000 Series device to flood.