Thank you Orion!!!!

Got enough information to noodle with it a bit more to check it all out!

I ended up removing the trunk group from our Allied Telesys AT-9000/52 switch, turned off auto negotiation on the connected ports, and moved it to mode=balance-alb and miimon=100 (missed in config before).  

It's odd that it works for the other servers being trunked via 802.3ad, but I think the problem comes in that the switch only supports 12 LACP groups and we probably hit that barrier by trying to add this server.  Investigating further….

So here's what my config looks like now….

[root@tape ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 
DEVICE=bond0
IPADDR=10.1.16.20
NETWORK=10.1.16.0
NETMASK=255.255.224.0
GATEWAY=
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
BONDING_OPTS="mode=6 miimon=100"

[root@tape ~]# cat /etc/sysconfig/network-scripts/route-bond0 
default via 172.16.0.1 dev eth0
10.1.0.0/19 via 10.1.16.20 dev bond0

And logs are reporting back…..

[root@tape ~]# dmesg | grep -Fi bond
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
bonding: bond0 is being created...
bonding: cannot add bond bond0; already exists
bonding: Bond creation failed.
bonding: bond0: setting mode to balance-alb (6).
bonding: bond0: Setting MII monitoring interval to 100.
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth2.
bonding: bond0: enslaving eth2 as an active interface with a down link.
bonding: bond0: Adding slave eth3.
bonding: bond0: enslaving eth3 as an active interface with a down link.
bonding: bond0: Adding slave eth4.
bonding: bond0: enslaving eth4 as an active interface with a down link.
bonding: bond0: Adding slave eth5.
bonding: bond0: enslaving eth5 as an active interface with a down link.
bond0: link status definitely up for interface eth2, 1000 Mbps full duplex.
bonding: bond0: making interface eth2 the new active one.
bonding: bond0: first active interface up!
bond0: link status definitely up for interface eth3, 1000 Mbps full duplex.
ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
bond0: link status definitely up for interface eth4, 1000 Mbps full duplex.
bond0: link status definitely up for interface eth5, 1000 Mbps full duplex.
bond0: no IPv6 routers present

[root@tape ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:21:2f:a8
Slave queue ID: 0

Slave Interface: eth3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:21:2f:a9
Slave queue ID: 0

Slave Interface: eth4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:21:2f:ac
Slave queue ID: 0

Slave Interface: eth5
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:21:2f:ad
Slave queue ID: 0