Storage / SAN / NAS

Jarle BjørgeengenUniversity of Oslo / USIT

October 18, 2011

I I’m available in room PS223 on Fridays.I .. except those weeks I have lectures other weekdays.I .. like this week.

I Discuss topics related to:I StorageI PerformanceI Unix/LinuxI Configuration mgmtI Virtualization / CloudI Etc.....

I E-mail: jarle.bjorgeengen@usit.uio.no

Outline

About USIT

About data storage

SAN introduction

NAS introduction

Types of NAS

Key points about USIT

I Approximately 50 000 file, print, mail and web-app users inUiO, with different privileges .

I Dev. and op. of the FS/Studentweb application used bymost universities and colleges

I Dev. and op. of Cerebrum, the glue that ties together allperson/user/machine information.

I Dev. and op. of the national authentication service "Moria".I Op. of mail and file backends for Classfronter for the

Nordic CountriesI Data-storage for LHC in Cern.I Op. of HPC clusters for research number-crunching.

About data storage

About data storage

I Computers need storageI Early days, punch cardsI ..then disk media (magnetical/optical)

I increasing storage capacity and performanceI decreasing physical sizeI Now magnetic high density hard disks dominant (still)

I Applications need performance (directly affect app. perf.)I Applications create workloadI Different applications create different workload types.I Storage needs to satisfy vastly varying workloads

simultaneously.I Cost reduction require resource sharing (d.t consolidation)I Resource sharing introduce further risk (Why ? )

Storage Area Network - SAN

QoS bridgeQoS bridge

QoS bridgeQoS bridge

QoS bridgeConsumers

Shared physical resources

SAN

Virtual disks

Centralized storage pool

Storage Area Network - SAN

I Flexible, sharable pool of block storage.I Disk virtualization.I Used for consolidation (centralization of resources).I Clusters need shared disksI Shared disk introduces risk (Why ?)I Physically located outside of server.I Interconnected through a network media (With switches).I A protocol for block access at the top (SCSI/ATA...)

Storage Area Network - SAN

I Uses RAID for disk redundancy and performance.I Varying degree of component redundancy (cache,

controller, buses, etc.)I You get what you pay for.I Cost increase exponentially when approaching 100%

uptime and keeping performance .I Intelligent applications can compensate for errors, hence

cheaper less reliable storage is possible (Hello Google)I Cost of downtime vs. cost of insurance against downtime

(redundancy)I Cost vs. performance vs. availability.

Typical (FC) SAN layout - simplified

AR

RAY

1

AR

RAY

2FC switches

hosts

ZONE A

Storage Area Network - SAN ?

I Slices of disk (virtual disks/LUNs)I Host "sees" it as local disk (/dev/sda , /dev/sdb and so on

in Linux)I Limit access between initatiors

I Login process in iSCSII Present only to WWN1, WWN2, and so onI Zones ín FC switches. Which WWN’s can see each other

I Risks introduced by lack of access control ?I 2 or more hosts can see the same virtual disk

I The hosts need to behave. (Coordinate writing – ClusterSW)

Host OS considerations ?

I Stable and working driver for HBAI Used to be a support/certification nightmareI Now HBA-vendors make drivers available upstream

I Multiple paths (several approaches)I Built into FC driver (only failover)I Separate MP driver on top (dm-multipath)I dm-multipath mostly used now. Fleksible and works well.I Storage vendors push their own drivers and agents.I Advantages / disadvantages ?

Hot topics in storage ?

I SSD is used increasinglyI Intelligent cachingI Automatic tiering

I Usually 3-4 tiersI SSD, FC (15k), SAS(10k), SATA (7.5k)

I Different approaches regardingI Estimation of what needs to be movedI Granularity of workload profiling

I Distributed network file systems for linear scaleability incapacity and performance

I Appliance bundling (Oracle Exadata, EMC Vblock, etc. )I Thin provisioning

I Thin write ?I FS / Application awareness ?

Sub-LUN tiering / Autotiering

Network Attached Storage - NAS

I Collect term for accessing files over an IP-network.I Using NAS involves "mounting" of remote filesystems.I ... and user authentication / authorization.I Typical usage:

I Making home directories available across many machines.I Group collaboration on file level.I File Archive (WORM/Policy-based retention)

I Untypical usage:I Shared storage for clustering (SAN is typical for that)

Types of NAS: NFS

I NFS (Network File System) developed by SUNI Utilizes IP (TCP or UDP)I Heavily based on RPC (Remote procedure calls)I Available on any Unix/LinuxI Version 2,3 and 4I V2 is old, unsecure, UDP and synchronous writes only.I V3 supports asynchronous writes and TCP in addition.

Also unsecure.I V4 an IETF standard, secure, only TCP, has

implementations for windows and support kerberos auth. 1

I V4 consolidates a number of protocols.

1http://www.nuug.no/aktiviteter/20100413-kerberos/

NFS server / client

I Server has 3 daemonsI Mountd - authorization / rejection of client mount requestsI Nfsd - data transferI Lockd - file-locking (Advisory locking. What does it mean ?)

I Client sends mount request to serverI If allowed, the client operates on it like any local FS.I NB: User ID’s must match.I client root user is mapped to "nobody" on server. (Why ?)I can be turned off with no_root_squash option.

NFS server / client

Configuring NFS: Server

I Install nfs-common, nfs-serverI Start daemons (/etc/init.d/nfs-server start)I Edit /etc/exports (man 5 exports)I exportfs -a (check with exportfs or showmount)I Statistics - nfsstat

Configuring NFS: Client

I Install nfs-common,nfs-client,portmapper.I mount server:/exported/fs /local/mount/point or ..I edit fstab and run mount /local/mount/point or mount -aI Check with df , mount, and try file operations (ls,touch,cat,

vi, cp, rm, mv)I Verify identical user ID’s in /etc/passwd or use centralized

UID lookup (LDAP) (Other options ? )I Not working ?

I Portmapper running on client ?I Any firewalls in between , or local? (iptables -L on both)I SELinux / AppArmorI Log files on serverI tcpdump

Instances of NAS: SMB and CIFS

I Server Message Block protocolI CIFS = SMB (Network file access anyway)I Originates from IBM. Modified by Microsoft.I Commonly used for integrating Windows and Linux env.I SAMBA for Unix/Linux

I Client and serverI Can mount windows sharesI Can Serve files to win clients.I Similar to AD when combined with MIT Kereberos and

OpenLDAP.I Built in file/print service (file share) for Windows server.I Many other implementations

(NetAPP,FreeNAS,Veritas,EMC,etc....)

SAMBA Server

I Server provide 5 basic servicesI File sharingI Printer sharingI Authentication / authorizationI Name resolution (through WINS)I Service announcement

I Behavior is defined in smb.confI Parallel user/pw database, managed by smbpasswd

SAMBA Client

I Windows: net use X: \\server\share

I Linux/Unix: smbmount / mount -t cifs / fstab mount -aI Must authenticate: use credentials=<file> (mode 700)I Samba utilities:

I Smbstatus - Info about smbd connections.I Smbclient - display observed shares on server.I Smbtar - backup of shares.I SWAT - Graphical (web-gui) config of SAMBAI inetd/xinetd service. Listens on port 901 default

I Local/remote UID does not matter.

Configuring SAMBA

I Install SAMBA server on server machineI Edit smb.conf (man 5 smb.conf)I Testparm to check syntax.I Start/reload service.I Mount filesystems on client(s)