SAN overview

There are 3 main high level components that makes SAN:

  • Storage arrays – these are the arrays of disk from various vendors. They offer various level of redundancy, connection ports, protocol support
  • SAN Switches – storage area network devices that connect the servers and their remote disks.
  • HBA (Host Bus Adapters) – The NICs to connect to the SAN. HBA runs native FC, many times with FC offload future to improve performance.

Storage Arrays

These are the physical disk enclosures which offers the block level storage to servers. Block level means that the server thinks he is communicating with the disk locally connected. Dont confuse with NAS which is network attached storage communicating via CIFS/SMB NFS protocol over regular ethernet networks. NAS offers file level storage, so the OS sees regular files and folders remotely mapped, as a remote disk for example.

Can be JBOD or RAID. JBOD combine disk together and presents them as one big disk. RAID is for redundancy purposes.

Dont confuse also with DAS which is direct attached storage – disk locally on server.

Storage arrays typically supports 3 interfaces.

  • FC (Fiber Channel) this is most common
  • iSCSI – typical for low to mid range arrays
  • FCoE – not much common but vendors started to integrating also FCoE because of capability to run as 100 gig solution. Regular FC can run only on 1,2,4,8,16 GB.

SAN Switches

Purpose of SAN Switches is to connect servers to Storage Arrays. We call in SAN environment the server as initiator and storage array (LUN disk) as target.

Like IP Routers, SAN switches can do protocol conversion. For example FC to FCOE, FC to iSCSI or FC to FCIP. Depends on the connectivity – same like in LAN world, you can connect ethernet LAN to mpls WAN 😉

You can meet in SAN world with something called Directories. Directories are just high-end SAN switches. Something like Nexus7K. They just offer higher port density, higher redundancy, performance,… (power, supervisors, line cards,… you know the concept)

HBA

NIC cards for SAN. Connects the servers and storage arrays to SAN switches. There are typically three types of HBAs:

  • FC HBA. SAN traffic only. 1/2/4/8/16 GB mode only
  • iSCSI HBA – this is ethernet NIC with iSCSI offload
  • FCoE CNA – 10 GB LAN plus FCoE HW offload. They offer much more! Check the vendor docs. For example TCP offload, VN-Link, etc.

Nexus support for SAN

In Nexus 5548UP or 5596UP can run as 1/10 GigE or 1/2/4/8 Native FC. FC ports on Nexus starts at end of module and count down. For example 32,31,30,… Changing the mode from Ethernet to FC causes the N5K to reboot. You need to replace SFPs to be able to run FC on the switch. This is because the physical layer for ethernet is different for FC. The SFP for ethernet world cannot be used. You can create FC port channel in Nexus. If you do that the number cannot be same as number from regular ethernet port channel.

Nexus 7000 is just Ethernet switch without the FC supports. It support the FCoE though, but it must be in the separate VDC for storage. Also FCoE is supported only on F cards. M cards doesnt support that.