Times Ain't Like They Used To Be...

'I tell ya what', I'm not as young as I used to be. It used to be that I could work hard, physical work all day and all night long without breaking my stride. Now I'm afraid I just get tired. I have not been this tired in a very long time. This past week and weekend, I've moved my family belongings into storage while we closed on the sale of our house today. After eleven years, I'm really glad to be moving on from that house. Its just time for a change and I welcome it. Enough for now. Its time to rest.

Cisco ASA: How To Send Traffic To Your IPS Module

A lot of people think that when you put the Cisco IPS module in the ASA, that is all you really have to do to get it going.  Well, not so.  You have to tell the traffic to go to the IPS module and then through the backplane of the IPS module.  Thats at least a two step process just to get traffic there and back, without any other configuration to it.  First, get your traffic to go to the IPS module:

access-list IPS-ACL permit ip any any

class-map IPS-Class
 description IPS Module
 match IPS-ACL

policy-map global_policy
 class inspection_default
class IPS-Class
  ips inline fail-open

Next, go into the IME for the IPS module, and select the check box as shown below:

Off Sunday..

Sorry folks, been moving from my house today. Its been a long weekend for sure. I've read very little this weekend, but I am reminded through this move that sometimes life changes take some time. You have to go through some hard times to get to the glorious end.

Cisco ASA: How To Reset The IPS Module Password

I came to a customer that needed some IPS help in their ASA.  They happened to not know the password to get into the IPS module.  So, I needed to reset it.  Here is what I did, in the ASA, to reset the IPS module.  It does reset the IPS module when you do this, so if you have a 'fail close' scenario, keep that in mind.
ASA(config)# hw-module module 1 password-reset

Reset the password on module in slot 1? [confirm]  <-- hit enter
ASA(config)#

ACL Explanations Of 'In' And 'Out' When Applying To An Interface

I was asked recently to explain how the 'in' and 'out' works when applying an ACL to an interface.  I thought I would demonstrate this with a visual.  Red is for outgoing to the interface, green for incoming to the interface.  Everything is from the perspective of the interface.  So act like you are standing 'in the interface' and face in the direction the interface is pointing, away from the ASA.  Then think about incoming and outgoing.  Take a look at the drawing below.

Cisco ASA 8.3 And Later: Allowing FTP Through The Firewall To An FTP Server

I had a consultant ask me to allow FTP in so he could send a large file over to a server on our network.  All he needed to do was start the FTP and walk away.  Ok, here is how I did this on the ASA, 8.3 and after code:
ASA# config t
ASA(config)# object network obj-10.10.10.150
ASA(config-network-object)# host 10.10.10.150
ASA(config-network-object)# nat (inside,outside) static 5.5.5.50
ASA(config-network-object)#exit
ASA(config)# access-list outsideIN permit tcp any host 10.10.10.150 eq ftp 
ASA(config)#exit

Cisco CME: How To Know If Someone Is Actively Using An Analog Line

I have always liked Cisco CME (unless its on a UC500).  Its always been a very reliable phone system for my customers that have it (again, unless its on the UC500 platform).  Run CME on a real Cisco router and you have a winner.

Sometimes I have to troubleshoot calls coming in on an analog line.  So its important to be able to figure out what is happening when you are calling in.  OR, if you just need to be able to see when no one is one an external call, for whatever reason, there is a good way to check it out without going all around the office to verify no one is on the phone.

Below, in this capture, you will see in yellow that there is an incoming call coming in.  No one has answered the phone just yet.  Its still ringing.  Use the 'show voice call summary' command:

UC520DrU#sh voice call sum

PORT           CODEC     VAD VTSP STATE            VPM STATE

============== ========= === ==================== ======================

0/0/0         -          -  -                                              FXSLS_ONHOOK

0/0/1         -          -  -                                             FXSLS_ONHOOK

0/0/2         -          -  -                                             FXSLS_ONHOOK

0/0/3         -          -  -                                             FXSLS_ONHOOK

0/1/0         -          -  -                                             FXOLS_ONHOOK

0/1/1         -          -  -                                             FXOLS_ONHOOK

0/1/2         g711ulaw   n             S_ALERTING      FXOLS_PROCEEDING

0/1/3         -          -  -                                             FXOLS_ONHOOK

Below, in orange, you can use the same command as above to see that the call actually did connect.

UC520DrU#sh voice call sum

PORT           CODEC     VAD VTSP STATE            VPM STATE

============== ========= === ==================== ======================

0/0/0         -          -  -                                               FXSLS_ONHOOK

0/0/1         -          -  -                                               FXSLS_ONHOOK

0/0/2         -          -  -                                              FXSLS_ONHOOK

0/0/3         -          -  -                                             FXSLS_ONHOOK

0/1/0         -          -  -                                             FXOLS_ONHOOK

0/1/1         -          -  -                                             FXOLS_ONHOOK

0/1/2         g711ulaw   n              S_CONNECT     FXOLS_CONNECT

0/1/3         -          -  -                                             FXOLS_ONHOOK

Thats a very useful command when needing to know if someone is on an external call on an analog line.

Brocade FCX Replaces Cisco 3560

I have a customer that was experiencing some pretty major delays on the network.  They initially purchased, some time back, all Cisco 3560s for a VoIP network. This worked fine at the time and has been pretty solid. But, as the network grew and demands changed, the core switch got really overburdened. CPU was at 70 to 90 percent most of the time.  I put this FCX648S in and now they are happy.

More John...

Hi all. I have planned on dedicating each Sunday post on this blog to the Lord. However, I am in the process of moving and this is taking a lot of my time this week. So I am going to have to make this one short (and from my phone).
I love how John just mentions things right in the middle of writing.  John 3:24 mentions how this event he is talking about (baptism) was before he was put into prison. This was just a side note to you, from John, and its not really part of the story he is telling you at that time. He is just telling you a 'when' in John's ministry (before he was put in prison).  Interestingly, I can't find in the Book of John where John talks about him being put in prison. Although he clearly remembers that event when he WAS put in prison.  I just find that interesting.

Cisco ASA: Both PRE-8.3 And 8.3 And Later Site To Site VPN Template Configs

I think it has occurred to me that I have not written just a simple site to site VPN template for both PRE-8.3 and POST-8.3 versions of the Cisco ASA.  Or, at least, I didnt find it on my blog.  So, since I just did this, Ill post it here.

8.2 AND BELOW CONFIG:
PHASE I SA:
crypto isakmp policy 10
 authentication pre-share
 encryption aes-256
 hash sha
 group 2
 lifetime 86400

TUNNEL GROUP FOR PHASE I TUNNEL SETUP:
tunnel-group 22.33.44.55 type ipsec-l2l
tunnel-group 22.33.44.55 ipsec-attributes
 pre-shared-key VPNkey$

PHASE II SA:
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

PHASE II TUNNEL SETUP:
crypto map outside_map 20 match address FarEnd
crypto map outside_map 20 set peer 22.33.44.55
crypto map outside_map 20 set transform-set ESP-3DES-SHA

NO NAT APPLICATION:
nat (inside) 0 access-list inside_nat0_outbound

NONAT ACL:
access-list inside_nat0_outbound extended permit ip 10.0.7.0 255.255.255.0 host 192.168.5.5

INTERESTING TRAFFIC ACL: 
access-list FarEnd extended permit ip 10.0.7.0 255.255.255.0 host 192.168.5.5

APPLY CRYPTO MAP TO THE OUTSIDE INTERFACE:
crypto map outside_map interface outside




8.3 AND LATER CONFIG:
DESTINATION OBJECT:
object network object-10.0.7.0
subnet 10.0.7.0 255.255.255.0

SOURCE OBJECT:
object network obj-192.168.5.5-01
 host 192.168.5.5

INTERESTING TRAFFIC ACL:
access-list FarSide extended permit ip host 192.168.5.5 10.0.7.0 255.255.255.0

NONAT NAT STATEMENT:
nat (dmz,outside) source static obj-192.168.5.5-01 obj-192.168.5.5-01 destination static obj-10.0.7.0 obj-10.0.7.0 no-proxy-arp route-lookup

PHASE I SA:
crypto ikev1 policy 11
 authentication pre-share
 encryption aes-256
 hash sha
 group 2
 lifetime 86400

TUNNEL GROUP FOR PHASE I TUNNEL SETUP:
tunnel-group 66.77.88.99 type ipsec-l2l
tunnel-group 66.77.88.99 ipsec-attributes
 pre-shared-key VPNkey$

PHASE II SA:
crypto ipsec ikev1 transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

PHASE II TUNNEL SETUP:
crypto map outside_map 130 match address FarSide
crypto map outside_map 130 set peer 66.77.88.99
crypto map outside_map 130 set transform-set ESP-3DES-SHA

APPLY CRYPTO MAP TO THE OUTSIDE INTERFACE:
crypto map outside_map interface outside

Cisco Switch: "AAA: Warning authentication list "default" is not defined for LOGIN"

Have you ever seen this message before when trying to deal with logins for a switch?  I have.  You have to go define what "default" is in the system.  Here is what you do to fix this on the Cisco 3750X:

Switch(config-line)#line con 0
Switch(config-line)#login authen default
AAA: Warning authentication list "default" is not defined for LOGIN.

Switch(config-line)#exit
Switch(config)#aaa new-model
Switch(config)#aaa authentication login default local
Switch(config-line)#line con 0
Switch(config-line)#login authen default
Switch(config-line)#exit

ShoreTel: VoiceMessage Control Install

It never seems to fail that when a new IE comes along, there is a new trouble.  For me, this trouble mainly seems to revolve around ShoreTel and the previous versions.  I usually do run IE when it comes to my voice administrative duties, either for Cisco or ShoreTel.  They just seem to program the phone systems for IE for some reason.  Whatever reason it is, I wish they would rethink it.  Either way, with ShoreTel, the magic IE (for me) seems to be IE8.  So I have an old laptop with that on it and I seem to be able to do most things without any problems.
Now, with that said, I seemed to have problems getting this 'voicemessage control' thing to install when I needed to make a change to the AA.  If I did what it wanted me to do, which was click on the install dropdown that comes up when you go to the AA page, all it would let me do was click 'ok' to leave the page, only to log in again.  In which case, it never let me install anything.  So I guess I have to do this manually, which is a pain.
So, here are the steps I took to resolve this issue, so I could record a new AA:
1.  Web browse to http://serverIPaddress/shorewaredirector/voicemessage.ocx .
2.  Download that to c: (or wherever you want it).
3.  Close out IE.
4.  Go to start --> run and type in the box 'regsvr32 C:\voicemessage.ocx' .

Once it has registered, you should be in good shape.

Cisco ASA ASDM Problem... Can Not Reach The ASDM

I got a call from a customer that said they could not reach the ASDM when trying to get in and do some management of the firewall.  I got in and looked and noticed nothing really unusual.  HTTP server was enabled and had and IP range on the inside of the network that was allowed to access it via HTTP.  However, what I did see was that the image on disk0: was not the same name as the configured image name:
ASA5520# show disk0:
--#--  --length--  -----date/time------  path
   80  18927088    Jun 07 2012 03:20:34  asdm-649.bin
   10  4096        Jul 16 2011 03:15:52  crypto_archive
   81  15261696    Aug 01 2011 06:48:32  asa824-k8.bin
   83  4096        Sep 10 2011 03:52:16  tmp

Whereas the configuration stated the following:

asdm image disk0:/asdm-642.bin

Ah, the difference on one number of the version made the difference.  Once I configured the correct version (649), all started working as the customer expected.

Cisco Nexus: "%ARP-3-DUP_SRCIP_PROBE: arp [5622] Duplicate address Detected. Probe packet received from"

This was a little odd, but I found this message in a pair of Nexus 7000 logs:
%ARP-3-DUP_SRCIP_PROBE:  arp [5622]  Duplicate address Detected. Probe  packet received from

I found that this is common when connecting to other Cisco gear like 3750-Xs, etc.  See the below topology for this problem I was having (this is generic, but basically was it was):

What appears to be happening is that the 3750Xs are sending out some type of check.  Not sure exactly what at this point, as Im still researching it, but it sends these messages for device tracking.  The 6509E passes that traffic right on over to the Nexus (at layer 2, not 3).  So the Nexus doesnt like that for some reason and logs it.  TAC says its a compatibility thing, but Im not sure still.  
So here is what I did to fix it, on the 3750Xs, not the 6509E.  Nothing was done on the 6509E, only on the 3750X uplinks TO the 6509E.
3750X#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
3750X(config)#int gi1/1/4
3750X(config-if)#ip device tracking max 0
3750X(config-if)#end
3750X#wr

I did this on both the 3750Xs and this stopped the log entries on both Nexus devices.  I still need to find the reason this happens, but this is the fix so far.

For a 3850 switch you also have to configure this:
3850(config)# interface gi1/0/1
3850(config-if)# NMSP attach suppress 
3850(config-if)# end
3850# wr mem

A Change For Sunday Posts...

I think I have decided that on each Sunday for this blog, I will do something Bible related.   Maybe just thoughts or maybe just a verse I'm reading or just whatever comes to mind for that particular Sunday post.  We will see how it goes.
Tonight I'm reading in John chapter 2.  I find something very interesting about when Jesus turns over the tables and drives out the animals in the temple.  Keep in mind, John, the author of this book, was a disciple of Jesus.  Read John 2:12-22.  What I find interesting is that at the beginning of this book, John mentions verse 21 and 22. Go read it.  John, who was there with Jesus as a disciple, refers to something he recalls from memory, at the beginning of the book.   I think this is all the more evidence that Jesus actually did raise Himself from the dead.  He has not even gotten to that part of the story yet (which would be chapter 19 and 20 in John) when he mentions Him being raised from the dead.  Its like he is writing this all from his real memory of the events of what happened.  But then again, I guess he really was writing from his memory, since he was there with Jesus.  I just think it is interesting that he inserted these two verses after Jesus' discussion with the Jews.  Because John was a real life witness to Jesus' resurrection, I guess John just had to say what he knew after recalling that event of what happened in the temple.  Interesting for sure.

Getting An Idea Of How The Fiber Patch Panel Is Connected

Looking above, there are three 'pairs' on this fiber run.  Theoretically, you could cross any pair, but I think just about everyone would do this like the above picture.  You plug in your fiber patch cables (at the top) and you use pair 1 for the first connection, pair 2 for the second, and so on.  You get the idea, but this picture is looking inside the patch panel.  This is what the cable guy would be doing before you get your equipment in place.

Cisco CUCM: "EXT3-fs error (device sd(8,6)) in start_transaction: Journal has aborted"

This has bit me a few times with this one phone system here in the last couple of days.  I get this:

I couldnt figure out what was going on with this, so I called TAC, who referenced me to the following bug:

7828-H3 server goes down with Journal Aborted error
CSCsv49493
Description
Symptom:
Phone services will go down, and server will only be semi-responsive. Local console access will show the following error constantly scrolling across the screen.
EXT3-fs error (device sd(8,6)) in start_transaction: Journal has aborted

Conditions:
During normal operation services will go down. Reboot will bring services back up for a while, anywhere between a couple hours and a couple days. Seen most frequently on
MCS7828-H3-K9/BE but has been reported on MCS7825-H2-IPC1 and MCS7825-H3. This defect is for the MCS7828-H3 servers only.

Workaround:
Shut down the server, and remove the first hard drive until a final fix is available.
If server still fails, try switching to the other drive. Watch during boot up for any errors which might indicate hardware failure (SMART errors in particular).
If server stills fails on 2nd drive, leave one drive in, and reinstall CUCM.

Further Information
This issue was tracked to a firmware problem on the hard drives. A firmware update is available on cisco.com. Please download the readme and iso for 7828h3-hddfwupdate-v11 under the MCS 7828-H3 Unified Communications Manager Appliance. 
Direct link:
http://www.cisco.com/cisco/software/release.html?mdfid=281021974&flowid=6679&softwareid=283046733&release=2.8%281%29&relind=AVAILABLE&rellifecycle=&reltype=latest

Alternatively you can find the download under the MCS 7828-H3 section, "7828h3-hddfwupdate-v11.iso" is the correct file name.

So, at the moment, I have one drive out until I can get the download.

GBICs: How To Physically Check Your GBICs To Verify They Look OK

Im not a fan of looking straight into a laser, but sometimes you have to verify that the GBIC has the TX laser coming out of it.  I usually will tilt my head to the side and ease my head in to see the laser.  I suspect that looking directly into the laser is not healthy for your eyes, so one way of doing this is by using your smartphone to take a picture of the GBIC.  That way, you can see if the GBIC appears to be working properly or not.  Im just needing to check to see the laser is coming out, nothing else.  The phone/picture thing works well.

Brocade Switch: How To Do Policy Based Routing With Route-Maps

I have a customer that wanted to divide up how the users accessed the Internet.  They wanted some users to go out the 10Meg Internet, while others go out the 3Meg Internet circuit, while still the rest goes out the simple cable provider Internet.  Below is the topology (generically speaking) and below that is how I routed certain IPs across different Internets.  Keep in mind, if one Internet fails, you have to manually move over the traffic to another Internet circuit for them to have access.

ip route 0.0.0.0 0.0.0.0 192.168.1.1

access-list 101 deny ip 10.1.1.0 0.0.0.255 10.0.0.0 0.255.255.255
access-list 101 deny ip 10.1.1.0 0.0.0.255 192.168.0.0 0.0.255.255
access-list 101 deny ip 10.1.1.0 0.0.0.255 172.16.0.0 0.0.255.255
access-list 101 permit ip host 10.1.1.153 any
access-list 101 permit ip host 10.1.1.96 any
access-list 101 permit ip host 10.1.1.250 any
access-list 101 permit ip host 10.1.1.203 any
access-list 101 permit ip host 10.1.1.248 any
access-list 101 permit ip host 10.1.1.247 any
access-list 101 permit ip host 10.1.1.246 any

access-list 102 deny ip 10.1.1.0 0.0.0.255 10.0.0.0 0.255.255.255
access-list 102 deny ip 10.1.1.0 0.0.0.255 192.168.0.0 0.0.255.255
access-list 102 deny ip 10.1.1.0 0.0.0.255 172.16.0.0 0.0.255.255
access-list 102 permit ip host 10.1.1.245 any
access-list 102 permit ip host 10.1.1.244 any
access-list 102 permit ip host 10.1.1.243 any
access-list 102 permit ip host 10.1.1.241 any
access-list 102 permit ip host 10.1.1.240 any
access-list 102 permit ip host 10.1.1.239 any
access-list 102 deny ip any any
!
route-map  Internet permit  10
 match ip address  101
 set ip next-hop 10.30.1.1

route-map  Internet permit  20
 match ip address  102
 set ip next-hop 10.1.10.1

interface ve 5
ip policy route-map Internet

Brocade Switch: How To Configure SSH And Disable Telnet On The FCX and ICX Series Switches

I personally think its a good idea, from a security standpoint, to enable SSH and to disable telnet.  Why?  Because passwords sent over a telnet session are in clear text, whereas with SSH, they are encrypted.  Dont believe that telnet is not secure?   Take a look at this post I did some time back. 

I have to tell you that I thought about this after I read one of Javier's posts at his blog.  It was a reminder of what I already knew, but he shows you how to enable SSH as well, and I thought about the security aspects of this and about how many attacks actually come from the inside of the network.  It was this post:  Enabling SSH on the ICX6610.  Check out Javier's blog.

So how do you enable SSH and disable telnet on a Brocade switch?  Ill show you the config:
ENABLING SSH AND A USERNAME
Switch#config t
Switch(config)#crypto key generate rsa
Switch(config)#username shanekillen pri 0 password passphrase
Switch(config)#aaa authentication login default local

DISABLING TELNET

Switch(config)#no telnet server

Check Point: How To KNOW The Version Of OS And Check Point Software On Your Firewall In CLI

I have been asked this sever times before on how to know for sure what version of OS and Check Point software you are running.  Its pretty easy from CLI.
For finding out the OS version:
[Expert@CP]# sh ver
This is Check Point SecurePlatform Pro R76 Build 468

For finding out the Check Point version:
[Expert@CP]# fw ver
This is Check Point VPN-1(TM) & FireWall-1(R) R76 - Build 380
[Expert@CP]#

Cisco ASA: NAT Statement Order Matters, Again

I ran into this problem again tonight.  What a pain.  I think its time for a firewall assessment.
http://www.shanekillen.com/2013/06/cisco-asa-nat-statement-order-does.html

How Long Does It Take To Complete A VPN?

How long does it take to create a site to site VPN from start to finish?  According to an major ISP in the US, 12 days.  Thats right, 12 days.  I guess I have a different view of how long it takes.

Rogue DHCP Servers: What A Pain

One of my customers today experienced this rogue DHCP server that was handing out chaos on the network.  No one really knew what was going on except that they 'couldn't do anything'.  So when I got onsite and noticed that they had a funny IP address, from a 'server' (or whatever) not meant to be a DHCP server.  Sadly, I could not find this device.  I found where it was 'supposed' to be, via the switch ARP tables, but never could actually find the device.  So, I blocked all access to the device on that VLAN.  What is a guy to do in that case when you just cant find the problem device???  Well, that was my solution.  Probably not the best thing, but that was all I could do with the time I had available.  I just wrote out a simple ACL and applied it to the VLAN interface.  Maybe the customer can find that device in the next day or so.

Cisco Nexus: How To Install A 10 Gig FEX On A 5548UP

I really like the data center products.  The Nexus is pretty cool in my opinion.  I had a customer ask me to get another 10 gig FEX ready for them for the next day.  So I went in and did just that.  Two 5548UPs and adding a N2K-C2232PP-10GE.  So I got the fiber in place and wrote out a template on what I would put in on the 5548UPs.  Just FYI, I usually do write a template first.  Its just my personal preference.

Just a note here on air flow.  I had an engineer that the air flow comes from the top of the unit.  He was right, so dont stack these without some space in between them.

The model of the FEX:

Back to the FEX notes.  So when, on the 5548UPs, I do a 'show fex', I get the following (before connecting the new fex):
N5K-1# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
14  FEX-14                Online    N2K-C2232PP-10GE  SSIxxxxxxxx
15  FEX-15                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
16  FEX-16               Online   N2K-C2248TP-E-1GE  SSIxxxxxxxx
18  FEX-18                Online   N2K-C2248TP-E-1GE   SSIxxxxxxxx
19  FEX-19                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
N5K-1#

Now, for my template for adding a FEX in.  This goes on the 5548UP.  Not just one, but both of them.  We are doing a redundant topology.
n5K-1# config t
Enter configuration commands, one per line.  End with CNTL/Z.
N5K-1(config)#
N5K-1(config)# fex 13
N5K-1(config-fex)#   pinning max-links 1
Change in Max-links will cause traffic disruption.
N5K-1(config-fex)#   description "FEX-13"
N5K-1(config-fex)# exit
N5K-1(config)# interface port-channel13
N5K-1(config-if)#   description **FEX-13**
N5K-1(config-if)#   switchport mode fex-fabric
N5K-1(config-if)#   fex associate 13
N5K-1(config-if)#   vpc 13
N5K-1(config-if)# interface Ethernet2/1
N5K-1(config-if)#   description **FEX-13**
N5K-1(config-if)#   switchport mode fex-fabric
N5K-1(config-if)#   fex associate 13
N5K-1(config-if)#   channel-group 13

Once I got the commands in, I do a 'show fex' to see if the 5548UP can see the FEX:
N5K-1# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
14  FEX-14                Online    N2K-C2232PP-10GE  SSIxxxxxxxx
15  FEX-15                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
16  FEX-16               Online   N2K-C2248TP-E-1GE  SSIxxxxxxxx
18  FEX-18                Online   N2K-C2248TP-E-1GE   SSIxxxxxxxx
19  FEX-19                Online    N2K-C2232PP-10GE   SSIxxxxxxxx

---       --------             Connected    N2K-C2232PP-10GE   SSIxxxxxxxx

It says above it is connected.  That is good.  Its going to take a little time now for it to come online.  I then moved to the other 5548UP and put in the same config above.
Monitoring the status:
N5K-1# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
14  FEX-14                Online    N2K-C2232PP-10GE  SSIxxxxxxxx
15  FEX-15                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
16  FEX-16               Online   N2K-C2248TP-E-1GE  SSIxxxxxxxx
18  FEX-18                Online   N2K-C2248TP-E-1GE   SSIxxxxxxxx
19  FEX-19                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
13 FEX-13        Image Download    N2K-C2232PP-10GE   SSIxxxxxxxx

Notice the 'image download' status above.
When I look at the details, just for fun, I see this:
N5K-1# sh fex 13 det
FEX: 13 Description: FEX-13   state: Image Download
  FEX version: 4.2(1)N1(1) [Switch version: 5.1(3)N2(1)]
  FEX Interim version: 4.2(1)N1(0.002)
  Switch Interim version: 5.1(3)N2(1)
  Module Sw Gen: 21  [Switch Sw Gen: 21]
  post level: complete
  Pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth2/16
  FCoE Admin: false
  FCoE Oper: true
  FCoE FEX AA Configured: false
  Fabric interface state:
    Po183 - Interface Up. State: Active
    Eth2/16 - Interface Up. State: Active
  Fex Port        State  Fabric Port
Logs:
02/27/2014 18:21:23.196783: Module register received
02/27/2014 18:21:23.197741: Image Version Mismatch
02/27/2014 18:21:23.204499: Registration response sent
02/27/2014 18:21:23.204876: Requesting satellite to download image
N5K-1#

When the image has downloaded, the FEX reboots:
N5K-1# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
14  FEX-14                Online    N2K-C2232PP-10GE  SSIxxxxxxxx
15  FEX-15                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
16  FEX-16               Online   N2K-C2248TP-E-1GE  SSIxxxxxxxx
18  FEX-18                Online   N2K-C2248TP-E-1GE   SSIxxxxxxxx
19  FEX-19                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
13 FEX-13               Offline    N2K-C2232PP-10GE   SSIxxxxxxxx

When the FEX comes back up, you finally see it come online:
N5K-1# sh fex
  FEX         FEX           FEX                       FEX
Number    Description      State            Model            Serial
------------------------------------------------------------------------
14  FEX-14                Online    N2K-C2232PP-10GE  SSIxxxxxxxx
15  FEX-15                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
16  FEX-16               Online   N2K-C2248TP-E-1GE  SSIxxxxxxxx
18  FEX-18                Online   N2K-C2248TP-E-1GE   SSIxxxxxxxx
19  FEX-19                Online    N2K-C2232PP-10GE   SSIxxxxxxxx
13 FEX-13               Online    N2K-C2232PP-10GE   SSIxxxxxxxx

By the way, below is the fiber module I put in the 10 gig FEX, in case you need to know the part number:

Palo Alto: Failed To Install 5.0.0 - Upgrading Content Requires Version Of 320 Or Greater And Found 255-1052...

Long error message.  I was trying to upgrade the PanOS to version 5.0.0 from 4.1.X.  However, it obviously wouldn't let me do it.  See the full error message below.

So how did I overcome this?  I had to go do the dynamic updates for 4.1.X first.  Once I did this and got it updated, THEN I could go and do the upgrade to 5.0.X.  Happy to be over this hump.