Thursday, May 31, 2012

Cisco Nexus 5000/2000 Install (Part 3): FEX Configuration Notes

Cisco Nexus 5000/2000 Install (Part 3): FEX Notes

Before we get to the notes on the FEXs, Im adding these links below as an afterthought to this series.  This Nexus install was a four part series, this page being part three.  Here are the links to the others if you are interested.
For the notes on the install of the 5000s/2000s and how to configure a redundant setup, click here.
For the notes on the vPC connections, click here.
For some config examples without explanations, click here.

Here are some notes about the FEXs (2232PP) that I learned on this install.  One thing I wanted to point out is that when the FEX boots up, or the N5K boots up (whichever is the case), the FEXs take some time to come 'online'.  In this case below, Ill be showing two FEXs coming back online after a N5K reboots.  Here are the 'show fex' outputs that I saw while waiting on them to come up:

N5K-2# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
100        FEX0100             Connected    N2K-C2232PP-10GE   SSIXXXXXX1
101        FEX0101             Connected    N2K-C2232PP-10GE   SSIXXXXXX2

After a few minutes:
N5K-2# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
100        FEX0100       Online Sequence    N2K-C2232PP-10GE   SSIXXXXXX1
101        FEX0101             Connected    N2K-C2232PP-10GE   SSIXXXXXX2

A little more waiting:
2009 Apr  1 20:52:51 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 100 is online
2009 Apr  1 20:52:51 N5K-2 %$ VDC-1 %$ %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-100 On-line
2009 Apr  1 20:52:52 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 100 is online
N5K-2# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
100        FEX0100                Online    N2K-C2232PP-10GE   SSIXXXXXX1
101        FEX0101             Connected    N2K-C2232PP-10GE   SSIXXXXXX2

More waiting:
N5K-2# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
100        FEX0100                Online    N2K-C2232PP-10GE   SSIXXXXXX1
101        FEX0101       Online Sequence    N2K-C2232PP-10GE   SSIXXXXXX2

And we arrive:
2009 Apr  1 20:53:34 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
2009 Apr  1 20:53:34 N5K-2 %$ VDC-1 %$ %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-101 On-line
2009 Apr  1 20:53:37 N5K-2 %$ VDC-1 %$ %PFMA-2-FEX_STATUS: Fex 101 is online
N5K-2# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
100        FEX0100                Online    N2K-C2232PP-10GE   SSIXXXXXX1
101        FEX0101                Online    N2K-C2232PP-10GE   SSIXXXXXX2

I just found this interesting.  Be patient, these take time to come 'online'.