FEX

FEX (Fabric EXtender): 

  • Nexus 2000 Series 

  • It acts as a module or a line card, but it’s basically a different or separate device/hardware 

  • It is Brainless device which does not have its own CPU or control Plane. It can’t be configured without the help of Parent switch 

  • It does not have any console or Aux port for connecting to workstation 

  • It has to be managed and configured from Parent switch. Parent can be either 5000 or 7000 

  • It is mostly being used with Nexus 5000 devices because Nexus 5000 does not have additional Line cards. Not all of the models of Nexus 7000 support FEX 

  • Maximum 12 FEX devices can be connected to a Single Parent Switch 

 

  • Fex device will have two kind of ports 

    1. Fabric Ports (Back Panel Ports) – connect to Parent Switch(s) 

    2. Front Panel Ports – connect Servers of workstations (host based devices) 

  • Once FEX is connected, It will appear as an additional Line Card on Parent Switch 

  • Front Panel Ports of FEX can’t be used to connect Switches because  

    • BPDUGuard is enabled by default 

    • Portfast is also enabled.  

    • If we connect Switch, port goes to error disabled state 

  • No numbering on Fabric Ports 

 

  • We do not have to install any NXOS image separately on FEX i.e. Nexus 2000 series device. Actually it downloads the NXOS automatically from Parent Switch once it is connected properly 

  • No Local Switching – It means, if two devices want to communicate within the same VLAN, connected on the FEX port, that traffic will go over Parent Switch even though they are connected locally to same FEX 

 

  • When we first connect fex, it takes around 10 mins to install the image and get discovered 

 

#conf t 

#feature fex 

 

#int eth1/2 

#switchport mode fex-fabric Considering FEX is connected to eth1/2 

#no shut 

#end 

 

#show interface status | i eth1/2 state will be FEX not connected 

 

#show fex To see the Fex models with serial numbers, with no FEX number 

State will be in Discovered initially 

 

  • There are two ways to configure Switchports connected to FEX 

    1. Static Pinning 

    2. Port Channel 

  • Static Pinning means configure the interfaces and associate FEX on that, we are binding each port with FEX. Port channel will have multiple ports logically bundled 

 

#conf t 

#fex 100 This is the logical representation of FEX Port 

#pinning max-links 2 Change in max-links will cause outage/traffic disruptions (how many links between FEX and Nexus), max-links can more than 1 only in Static Pinning 

#end 

#int eth1/2 

#fex associate 100 

#end 

 

#show run fex To see the fex config 

 

#show fex To see the Fex models with serial numbers, with FEX number 100 

State will change from Discovered –> Online Sequence –> Online, once we configure Fex number and associate with the interface 

#show fex 100 details To see more details about FEX 100 config and with ports up or down 

 

#show interface status 

Eth1/2 will be in connected state 

Eth100/1/1 – Eth100/1/32 will be added (logical ports), Eth100 is Fex port number 

 

Now we can configure interfaces as well 

 

#conf t 

#interface eth100/1/2 

#swithport mode trunk 

#end 

 

  • If FEX connection is lost somehow- then we do not have to worry about losing configuration because Parent Switch (Nexus Switch) stores it’s configuration. FEX number is retained once it reconnects 

    #conf t 

    #interface eth1/2 

    #shut Bringing the interface (fex connected) down 

    #end 

     

    #show fex State will change from Online –> Offline 

    #show run interface eth100/1/2 Interface will not be available (Invalid range) 

     

    #conf t 

    #interface eth1/2 

    #unshut Bringing the interface (fex connected) up 

    #end 

     

    #show fex State will change from Offline –> Online, it will take some time 

    #show run interface eth100/1/2 Interface will be available now, once fex is online 

    #show fex 100 version To see the software and hardware details 

     

  • So configuration is retained whenever Fex device goes down or Fabric connectivity is lost 

 

  • One FEX can be connected to multiple Parent Switches as well 

  • Suggestion is always to use port channel at Parent Switch end for connecting FEX. Even if one of the ports inside port channel flaps somehow – It wouldn’t have any impact to FEX 

 

  • If we use Static Pinning without Port channel, then we have to Map front panel ports. 

  • Because Front Panel ports associated with Back Panel ports 

  • So if we have two Parent switch ports eth1/2 (port 2) and eth1/3 (port 3) being used for connecting FEX (Fabric Ports/Back Panel Ports)  

  • and if FEX has 32 ports in front panel, then first half of the ports will be mapped to port 2 and next 16 with port 3 

  • So if port 2 goes down then first 16 ports will remain down. That’s the problem with static pinning with physical interfaces 

  • If we want to switch the non-working ports in case of fabric port (connected to port 2 in this case) failure 

    #fex pinning redistribute 100 Now, if port 2 goes down, port 3 will take ownership to run first 16 ports 

 

 

FEX Config with Port-channels: 

  • if we use port channel for bundling fabric ports- we need to configure max-links and has to set it to 1 

     

    #conf t 

    #fex 101 This is the logical representation of FEX Port 

    #pinning max-links 1 

 

  • If we do not configure max-link to 1, then we will get error message “ERROR: FEX max-links not configured to one”. It appears while associating fex to port channel 

  • LACP is not supported when we bundle fabric links which connect to FEX 

 

#int eth1/3 

#switchport mode access  

#channel-group 23 mode on It should be ON and not ACTIVE because LACP is not supported, even lacp feature is enabled 

#no shut 

#end 

 

#int po 23 

#switchport mode fex-fabric 

#fex associate 101 

#no shut 

 

#show fex To see the Fex models with serial numbers, with FEX number 101 

State will change from Discovered –> Online Sequence –> Online, once we configure Fex number and associate with the interface 

#show fex 101 details To see more details about FEX 101 config and with ports up or down 

#show fex 101 version To see the software and hardware details 

 

#show port-channel summary 

Po23 will have eth1/3 

 

  • No. of links in Port channel should be either 1, 2 or 4 but NOT 3. That’s the rule while using Port channel for FEX 

  • We can’t mix port channel and individual ports while connecting a single FEX. It has to be either port channel or individual links 

  • To check the detail about the model, cpu, serial number of fex being used: 

     

  • If we have 8 Fabric port links of 10G capacity (Total 80G) & we have 32 front ports on FEX – Then, Fabric ports will handle 40Gbps Tx and 40Gbps Rx. Reason is that FEX depends 100% on Parent switch for sending and receiving Traffic


error: Content is protected !!