8/14/2019 Introduction to Lustre

1/19

Lustre File System

chris

8/14/2019 Introduction to Lustre

2/19

Outlines

What is lustre How does it works

Features

Performance

8/14/2019 Introduction to Lustre

3/19

What is Lustre?

Cluster File Systems, Inc. (CFS) is the companythat originally developed the Lustre distributed filesystem.

On September 12, 2007, Sun Microsystems, Inc.signed an agreement to acquire CFS.[1] Theacquisition was completed on October 1, 2007.

Lustre is a shared disk file system, generally usedfor large scale cluster computing.

The name Lustre is a portmanteau of Linux andcluster. The project aims to provide a file systemfor clusters of tens of thousands of nodes withpetabytes of storage capacity, withoutcompromising speed or security. Lustre is

available under the GNU GPL.

http://en.wikipedia.org/wiki/Sun_Microsystems,_Inc.http://en.wikipedia.org/wiki/Sun_Microsystems,_Inc.

8/14/2019 Introduction to Lustre

4/19

Support Platform

8/14/2019 Introduction to Lustre

5/19

8/14/2019 Introduction to Lustre

6/19

Lustre Components

MGS The Management Server (MGS) definesconfiguration information for all Lustrefilesystems at a site.

Each Lustre target contacts the MGS toprovide information, and Lustre clientscontact the MGS to retrieve information.

The MGS can provide live updates to the

configuration of targets and clients. TheMGS requires its own disk for storage.However, there is a provision that allows theMGS to share a disk ("co-locate") with asingle MDT.

8/14/2019 Introduction to Lustre

7/19

Lustre Components

MDTThe MDT provides back-end storage for

metadata for a single filesystem. The

Metadata Server (MDS) provides the

network request handling for one or morelocal MDTs.

The metdata managed by the MDT consists of

the file hierarchy ("namespace"), along withfile attributes such as permissions andreferences to the data objects stored on theOSTs.

8/14/2019 Introduction to Lustre

8/19

Lustre Components

OSS and OSTs An OST provides back-end storage for file object

data (effectively, chunks of user files). Typically,multiple OSTs provide access to different filechunks.

The MDT tracks the location of the chunks. On anode serving OSTs, an Object Storage Server(OSS) component provides the network request

handling for one or more local OSTs.

8/14/2019 Introduction to Lustre

9/19

Lustre Configuration

1. Create Lustre format file system onMGS/OSS mkfs.lustre --fsname={whatever}

--{--ost/--mdt/--mgs} --mgsnode={hostname@tcp0}

/dev/sda

2. Mount lustre filesystem on MGS and OSS mount t lustre /dev/sda /mnt/test

mount mgs/mdt at first

3. Mount lustre filesystem on clients mount t lustre {hostname@tcp0}:/{fsname}

/mnt/client_FS

8/14/2019 Introduction to Lustre

10/19

Lustre Configuration

8/14/2019 Introduction to Lustre

11/19

Lustre Failover mechanism

Active/active Active/passive Heartbeat STONITH (Shoot The Other Node In The Head

OSS-1

OSS-1-

backup

Heartbeat msg

STONITHpoweroff

8/14/2019 Introduction to Lustre

12/19

Lustre Failover mechanism

In Lustre, a disk failure is an unrecoverableerror. For this reason, you must have reliableback-end storage with RAID.

Backup device

dd if=/dev/{original} of=/dev/{new} bs=4kconv=sync,noerror

8/14/2019 Introduction to Lustre

13/19

Features

Kerberos

ACL Bonding

Live Add/Remove OSTs

SNMP

Quota File Stripe size

Using POSIX Standard

8/14/2019 Introduction to Lustre

14/19

Performance

Read/write files with different kernel io scheduler

8/14/2019 Introduction to Lustre

15/19

Performance

Create files with different Kernel io scheduler

8/14/2019 Introduction to Lustre

16/19

Performance

Multi-clients

8/14/2019 Introduction to Lustre

17/19

Performance

Lustre vs GPFS

8/14/2019 Introduction to Lustre

18/19

Reference

1.

Sun Microsystems Expands High Performance C, press release, September 12, 2007.

2. Lustre 1.6 Operations Manual

3. http://wiki.lustre.org/index.php?

title=Main_Page

http://www.sun.com/aboutsun/pr/2007-09/sunflash.20070912.2.xmlhttp://www.sun.com/aboutsun/pr/2007-09/sunflash.20070912.2.xml

8/14/2019 Introduction to Lustre

19/19

Any Questions?