HP/UX Logical Volume Manager notes

Configure a brand new volume group and logical volumes from scratch (without using SAM)

o Find out which devices you have

ioscan -fn

o Find out the size of the disk you want to allocate

diskinfo -v /dev/rdsk/cXtXdX

o Create a directory for the new volume group (with a character special file called group).

mkdir /dev/vgXX

mknod /dev/vgXX/group c 64 0xNN0000Note: The minor number for the group file should be unique among all the volume groups on the system. It has the format 0xNN0000, where NN runs from 00 to 09. The maximum value of NN is controlled by the kernel tunable parameter maxvgs. Convention: NN is the same as the volume group number XX above.

o Initialize the disk using pvcreate

pvcreate /dev/rdsk/cXtXdX

o Create the physical volume group consisting of 1 or many disks

vgcreate /dev/vgXX /dev/dsk/cXtXdX [disk2] [disk3] ...Note 1: Use the block device notation here, not the character device.Note 2: You can use the -s option to specify the physical extent size (PE SIZE)Note 3: A volume group can consist of multiple disks.

o Create a (zero size) logical volume

lvcreate -n lvol1 /dev/vgXXNote: by not allocating the size here, the door is open to create mirrored logical volume using lvextend where the mirror is on a different disk drive.

o Extend the logical volume to the desired size

lvextend -L [Megs] /dev/vgXX/lvol1 /dev/dsk/cXtXd0

o Create a mirror on a different disk drive

lvextend -m 1 /dev/vgXX/lvol1 /dev/dsk/cXtXd1

There is alternate way to create and extend a mirrored logical volume all in one step.

o lvcreate -n lvol2 -L 1024 -m 1 -D y -s g /dev/vg01

Logical volume name = lvol2

Size = 1 gig (1024 meg)

-m 1 = One mirror copy

-D y = Turn on distributed allocation (distribute evenly between physical volumes)

-s g = Set a strict allocation policy. Mirrors of a logical extent cannot share the same physical volume.

Prepare logical volumes for use with Informix

o Note: make sure you use the character device here (rlvol1)

o chmod 660 /dev/vgXX/rlvol1

o chown informix:informix /dev/vgXX/rlvol1

o ln -s /dev/vgXX/rlvol1 /RawSpaces/chunk1

Run the hp_disk_table1.sh to interrogate /etc/lvmtab to map hardware addresses to device names on HP/UX

o Output looks like:sdisk card instance 2 SCSI target 6 SCSI LUN 0 section 0 at address 0/0/2/1.6.0 /dev/dsk/c2t6d0sdisk card instance 1 SCSI target 6 SCSI LUN 0 section 0 at address 0/0/2/0.6.0 /dev/dsk/c1t6d0

Here's an example of the whole thing in a script. This will need to be edited before use.

Note: When using 2 Gb logical volumes with a size of 2,048 Mb, create Informix chunks that are 2,097,150 Kb.mkdir /dev/vg04

mknod /dev/vg04/group c 64 0x040000ls -ls /dev/vg04

pvcreate /dev/rdsk/c5t1d3

vgcreate /dev/vg04 /dev/dsk/c5t1d3

lvcreate -n root_dbs /dev/vg04lvcreate -n physical_dbs /dev/vg04lvcreate -n logical_dbs /dev/vg04lvcreate -n chunk1 /dev/vg04lvcreate -n chunk2 /dev/vg04lvcreate -n chunk3 /dev/vg04lvcreate -n chunk4 /dev/vg04lvcreate -n chunk5 /dev/vg04lvcreate -n chunk6 /dev/vg04lvcreate -n chunk7 /dev/vg04

lvextend -L 2048 /dev/vg04/root_dbs /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/physical_dbs /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/logical_dbs /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk1 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk2 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk3 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk4 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk5 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk6 /dev/dsk/c5t1d3lvextend -L 2048 /dev/vg04/chunk7 /dev/dsk/c5t1d3

chmod 660 /dev/vg04/rroot_dbschmod 660 /dev/vg04/rphysical_dbschmod 660 /dev/vg04/rlogical_dbschmod 660 /dev/vg04/rchunk1chmod 660 /dev/vg04/rchunk2chmod 660 /dev/vg04/rchunk3chmod 660 /dev/vg04/rchunk4chmod 660 /dev/vg04/rchunk5chmod 660 /dev/vg04/rchunk6chmod 660 /dev/vg04/rchunk7

chown informix:informix /dev/vg04/rroot_dbschown informix:informix /dev/vg04/rphysical_dbschown informix:informix /dev/vg04/rlogical_dbschown informix:informix /dev/vg04/rchunk1chown informix:informix /dev/vg04/rchunk2chown informix:informix /dev/vg04/rchunk3chown informix:informix /dev/vg04/rchunk4chown informix:informix /dev/vg04/rchunk5

chown informix:informix /dev/vg04/rchunk6chown informix:informix /dev/vg04/rchunk7

ln -s /dev/vg04/rroot_dbs /RawSpaces/root_dbsln -s /dev/vg04/rphysical_dbs /RawSpaces/physical_dbsln -s /dev/vg04/rlogical_dbs /RawSpaces/logical_dbsln -s /dev/vg04/rchunk1 /RawSpaces/chunk1ln -s /dev/vg04/rchunk2 /RawSpaces/chunk2ln -s /dev/vg04/rchunk3 /RawSpaces/chunk3ln -s /dev/vg04/rchunk4 /RawSpaces/chunk4ln -s /dev/vg04/rchunk5 /RawSpaces/chunk5ln -s /dev/vg04/rchunk6 /RawSpaces/chunk6ln -s /dev/vg04/rchunk7 /RawSpaces/chunk7

mirror_stripe.ksh Script to do mirroring and striping at the same time on HP-UX.mirror_stripe_config.ksh Config file used with above script.

Change the size of /usr, /var, /tmp filesystems

If you want to change the size of a Unix filesystem that is always mounted and used by the operating system, it is a problem because you can only change the size of a filesystem that is un-mounted. This is how you get around that problem.

The following example is how I increased the size of /usr (on /dev/vg00/lvol7) from 400 MG to 700 MG.

Make sure you have free PE's in that volume group.o vgdisplay -v

Backup the system. At least make a tar tape of /usr.

Shutdown to single user mode. When prompted, press any key to interrupt the boot and go into single user mode.

o shutdown -y now

Boot from the primary device and invoke ISL.

o bo pri isl

Interact with ISL ? - respond: Y

Boot into single user mode.

o hpux -is

Extend the logical volume that holds the filesystem.

o /sbin/lvextend -L 700 /dev/vg00/lvol7

o Note: 700 is the desired end result size, NOT just the amount to increase.

o Note: If you want to run 'bdf', it is located in /usr/bin.

Make sure /usr is not mounted. If it is mounted, mount and unmount /usr to get it to be unmounted. I know this should be unneccessary - but I had to do it!

o mount (to see if /usr is mounted)

o mount /dev/vg00/lvol7 /usr (to mount /usr)

o umount /dev/vg00/lvol7 (to un-mount /usr)

o mount (to see if /usr is still mounted)

Extend the filesystem. Don't forget to use the character device below.

o /sbin/extendfs -F hfs /dev/vg00/rlvol7

Reboot back to multi-user mode.

o shutdown -r now

Check the size of the new /usr filesystem

o bdf

Logical Volume Manager (LVM) using HP-UX

Glossary

Term

Journaled File System (JFS) Unix filesystem that uses a journaled log to record changes to the filesystem.

Logical Extents Logical extents are used to create the logical volume - they are the same size as physical extents

Logical Interchange Format (LIF) HP Storage format used for interchange of files between HP systems.

Logical Volumes Allocates the space inside a volume group, the logical volume gets the file system

Mirroring Mapping a logical extent to more than one physical extent.

Physical Extent (PE) Unit used for allocating disk space to logical volumes, default size is 4MB.

Physical Partitions The physical volume is organized into blocks called physical partitions (same as physical extent??)

Physical Volume Physical disk that is being managed by LVM

Striping A logical volume that is configured to write across more than one disk in a stripe.

Volume A device used for storage - either an entire disk or a partition.

Volume Group Contain one or more physical volumes, a physical volume can only be in one volume group (pool of partitions)

Commands

Command

lvcreate Command used to create logical volumes

lvdisplay -v /dev/<VG>/<LV> Command used to display information about logical volumes

pvchange Changes the attributes of existing physical volumes.

pvcreate /dev/rdsk/cxtxdxpvcreate -f /dev/rdsk/cxtxdx

Creates a physical volume by creating the LVM data structures on the physical disk called physical extents.creation of the PV - potentially destroying old PV information.

pvdisplay /dev/dsk/cxtxdxpvdisplay -v /dev/dsk/cxtxdx

Command used to display information about physical volumes

vgchange Command used to change information about the volume group, such as to activate an inactive vg

vgcreate Command used to create the volume groups

vgdisplay [-v] [VG_NAME} Command used to display information about volume groups, -v is the verbose option which will list lvs and pvs.

vgexport

vgimport

Files

File

/dev/dsk/cxtxdx Block device for disk

/dev/rdsk/cxtxdx Character device for disk

/dev/<volumegroup>/group Character device file (created with the mknod command) that allows the lvm kernel and the lvm commands to communicate

/etc/lvmrc Script that starts each volume group based upon the contents of /etc/lvmtab

/etc/lvmtab This file has the device file associated with each disk in a volume group.

/etc/vpath.cfg If IBM SDD is being used this file contains the mapping of vpath to the corresponding disk-target-lun.command.

Procedures

Filesystem Procedures:

Create a new filesystem:(A new filesystem will require a new logical volume)

1. Create the Logical Volume (Procedure Below)2. Create the File System

o Using newfs:

1. newfs -F <filesystem_type> <path_to_lvol_devicefile>

2. newfs -F vxfs /dev/vg02/lvol1

o Using mkfs:

1. mkfs -F <FILESYSTEM TYPE> -o bsize=<OPTIONS>,<OPTIONS> <PATH TO LV>

1. -F is filesystem type, such as vxfs for the Veritas File System or JFS

2. -o are options, such as bsize (Block Size) and largefiles.

3. Path to LV is of the form /dev/VGNAME/LVNAME

2. For example:  "mkfs -F vxfs -o bsize=8192,largefiles /dev/bgbill/lvbill"

1. Create the Mount Point

0. mkdir -p <full path to mountpoint>

1. -p will make any subdirectories in the path that do not already exist

2. Set the proper ownership and permissions on the mount point

0. chown owner:group /<mount point>

1. chmod XXX /<mount point>

3. Mount the File System

0. Edit /etc/fstab using vi and add the proper entry

2. <LV PATH> <MOUNTPOINT PATH> <FILESYSTEM> <OPTIONS> <BACKUP FREQUENCY> <PASS NUMBER>

3. For example:  "/dev/vgbill/lvbill  /maindir/subdir/share vxfs rw,suid,largefiles,delaylog,datainlog 0 2"

1. "mount /<mountpoint>" to mount the file system

2. Test

0. Do a "bdf <mount point>".    This command will show if the file system is mounted, and it's size.

1. Do an "ls -ld <mount point>" to verify that the proper permissions are there.

2. Have the user test the filesystem, and report back with any problems

Extending a VxFS Filesystem (Online JFS)1. Verify that there is enough space in the volume group

o Check the volume group for enough space

vgdisplay <VG Name>

o Check the logical volume to determine if mirroring or striping is in effect

lvdisplay /dev/vg_name/lv_name

o If there is not enough space then the volume group must be extended - see procedure below.

2. First lvextend the logical volume - 2 Options for lvextend:

1. lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME

2. lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME

1. fsadm -F vxfs  -b <new size of filesystem in K> <MOUNT POINT>

o NOTE:  extendfs will NOT work with Online JFS, you must use fsadm

o NOTE:  new size of the filesystem = <# of LEs> * <LE Size in MB> * 1024

o NOTE:  If lvextend -L was used simply multipy size in MB by 1024

o NOTE:  If this FAILS with errno 28, then the filesystem is 100% full and must be reduced to less than 100%.

o NOTE:  If this FAILS with "write failure at block XXXXXXXX : No such device or address" then the new size may be too big, use a smaller number

Extending a VxFS Filesystem (No Online JFS)1. Verify that there is enough space in the volume group

o vgdisplay <VG Name>

o If there is not enough space then the volume group must be extended - see procedure below.

2. First lvextend the logical volume - 2 Options for lvextend:

1. lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME

2. lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME

2. Unmount the filesystem

3. extendfs -F vxfs /dev/<VG_Name>/r<LV_Name>

o Note:  extendfs uses the raw logical volume name (ie rlvol1)

How to determine if you have Online JFS swlist -l fileset | grep -i advanced

How to determine the filesystem type grep <mount point> /etc/fstab Results should show the filesystem type (vxfs or hfs) in the return string

Logical Volume Procedures:

Creating a Logical Volume:1. Determine if an existing volume group has enough spare capacity for a new

logical volume.1. Is mirroring or striping required (if not known, then see if other logical

volumes in that volume group use it)?

1. vgdisplay   <This will list existing volume groups

1. Spare capacity is indicated by the number of free PEs

2. Multiply the "Free PE" by "PE Size (MBytes)" to get spare capacity in MB of the volume group

Note:  If mirroring or striping is required then there must be enough space available on multiple disks

3. Is there a volume group with enough spare capacity?

11 If yes, the continue to "Create the Logical Volume"

11 If no, then the Volume Group must be extended or a new Volume Group must be created - see procedures below.

2. Create the logical volume:

o lvcreate -1 <size in LEs> -n <LV Name> -r N <VG Directory path> OR

1. -l is size in Logical Extents (same as PE size)

2. -n is the name of the Logical Volume

3. -r N disable bad block allocation

o lvcreate -L <size in MBs> -n <LV Name> -r N <VG Directory path>

1. lvcreate -L 100 /dev/vg01

Logical volume will be 100MB in size

Name of the logical volume default name will be lvolx

2. lvcreate -L 100 -n oracle /dev/vg01

This creates a logical volume called oracle of 100 MB

2. Verify the lv was created properly "vgdisplay -v <volume group>"

Removing a Logical Volume Unmount the filesystem

o umount /<Path to Mount Point>

Remove the volume group

o lvremove /dev/<VG_NAME>/<LV_NAME>

Clean up

o Delete the mount point

o Remove the mount from /etc/fstab

Troubleshooting (just an idea, not sure if this really works??)ONCTRA01# lvchange -a n /dev/vg_wte_test/lv_wte_testLogical volume "/dev/vg_wte_test/lv_wte_test" has been successfully changed.Volume Group configuration for /dev/vg_wte_test has been saved in /etc/lvmconf/vg_wte_test.conf

Unmirroring a Logical Volume lvreduce -m 0 /dev/<VG_NAME>/<LV_NAME>

Volume Group Procedures:

Create a Volume Group1. Determine the list of disk devices that will make up the volume group

(procedure below)2. Create PV disks for each of the disks (procedure below)

3. Create the volume group directory

1. mkdir /dev/vg01

4. Create the group file

1. Determine the minor number to use

1. ls -l /dev/*/group | sort +5

2. Choose the next unused number

1. mknod /dev/vol_group_directory/group c Major_number Minor_number

1. mknod /dev/vg01/group c 64 0x010000

2. The group file is used to allow communications between the lvm commands and the lvm kernel

2. Create the volume group

1. vgcreate -l 255 -p 64 -s 16 /dev/<vg id>/dev/dsk/<disk id>  <-- create the volume group with initial disk devices

1. -l sets the maximum number of logical volumes for the volume group

2. -p sets the maximum number of physical volumes for the volume group

3. -s sets the physical extent size

4. /dev/<vg id> is the volume group directory

5. /dev/dsk/diskid are the physical volumes to be added to the volume group

6. vgcreate /dev/vg01 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0

2. vgextend /dev/vgxx /dev/dsk/diskid  <-- adds additional disk devices to the volume group

Extending a Volume Group A volume group is extended by adding additional PVs to it with vgextend

o vgextend /dev/vgxx /dev/dsk/diskid

Note:  If there are no available PVs, then disk space (LUNS) must be added to the system and PVs must be created - see procedures below.

Activating a Volume Group1. vgchange -a y /dev/<vg name>2. mount -a  ==>  This may indicate that a filesystem check needs to be run (fsck)

3. fsck -m /dev/<vg name>/<lv name>  ==> this is a sanity check, it is very quick, but may indicate that a complete fsck needs to be run

4. fsck /dev/<vg name>/<lv name>  ==> this is the full file system check

Deactivating a Volume Group1. umount all of the logical volumes in the volume group

o Determine all of the mounted logical volumes by "bdf | grep vg_name"

2. vgchange -a n /dev/<vg name>

Adding mirroring to the root vg - vg001. pvcreate -B /dev/rdisk/c2t2d0

o Create the PV on the second disk so that LVM can manage it

o -B makes it a bootable volume

2. mkboot /dev/rdsk/c2t2d0

o Installs the boot files on the second disk

3. mkboot -a "hpux-lq (;0)/stand/vmunix" /dev/rdsk/c2t2d0

o Creates an autoboot file on the disk, with information on how to boot

4. vgextend /dev/vg00 /dev/dsk/c2t2d0

o extends the volume group to include this second disk.

5. vgdisplay -v vg00

o To verify that the second disk is now part of the volume group

6. lvlnboot -v

o To verify that the system thinks this disk can boot

7. lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c2t2d0

o This extends the logical volume to the mirrored disk, effectively creating the mirrored copy.

o This command needs to be run for each logical volume in the volume group

Removing a Volume Group

To remove a volume group using vgreduce and vgremove: Remove all Logical Volumes (see procedure above) remove all disks except on using vgreduce <VG_NAME> </dev/dsk/DISK>

o This will need to be done for each disk in the volume group, except for the last disk

remove the final disk in the volume group with vgremove <VG_NAME>

To remove a volume group using vgexport: Deactive the volume group vgexport -m <mapfile> -v -f <devicefile> <VG_NAME>

PV Procedures:

How to recognize a newly added LUN:1. Check for the new hardware

o ioscan -fnC disk | more  <-- the new devices will have a hardware path, but no device file associated with it.

2. Create the device file for the hardware path

o insf

3. If using vpaths then create the vpath association

o /opt/IBMdpo/bin/cfgvpath

o /opt/IBMdpo may not be the path of the sdd software, "whereis cfgvpath" may need to be run to find it if it is not in path.

4. Verify the new devices/vpaths are there:

o ioscan -fnC disk

o strings /etc/vpath.cfg

o /opt/IBMdpo/bin/showvpath

5. Create the PV

o For each disk device (not vpath) issue a "pvcreate /dev/rdsk/cxtxdx"

o Note for vpaths this information can be found in /etc/vpath.cfg

How to create Physical Volumes: 3 Methods of Using pvcreate:

o pvcreate /dev/rdsk/cxtxdx

Note this command uses the "r" version of the device

o pvcreate -f /dev/rdsk/cxtxdx

Note:  the -f option will overwrite existing pv data on the disk device

Note this command uses the "r" version of the device

o pvcreate -B /dev/rdsk/cxtxdx

Makes the PV boot capable

Note this command uses the "r" version of the device

How to determine available disks to be used in a Volume Group:1. "ioscan -funC disk"  will list all of the disk devices

1. Some of these devices will be allocated, some will not

2. "vgdisplay -v" will list all of the PVs and their devices for all of the existing volume groups

1. This is a list of the devices that are in use

3. Any devices that are on the ioscan, but are NOT on the vgdisplay are available for use

1. Possible strategy to automate this process using sed, awk and grep

1. create a file that has all of the disks that can be used (in this example HITACHI)

1. cat wte_disk_ioscan | sed 1,2d | grep -v -e TOSHIBA -e c2t0d0 | xargs -n 10 | grep HITACHI | grep -vi subsystem > wte_hitachi_disk

1. cat will output the file that contains the disk ioscan (ioscan -fnC disk) 

2. sed is used to delete the first 2 header lines of the file.

3. Next grep is used to print any lines that DO NOT include TOSHIBA or c2t0d0

4. xargs is used to group the output into groups of 10

5. Next grep finds all of the lines with HITACHI in them

6. All of the lines that have HITACHI in them are saved to a file

1. Refine the ioscan of HITACHI disks to include just the disk devices, sorted - this is a list of all HITACHI disks on the system

1. awk '{ print $9 }' wte_hitachi_disk | sort -u > wte_hitachi_sorted_u

1. awk prints just the 9th field of the file

2. sort - u sorts the file and surpresses any duplicates

3. This is saved to a sorted file

2. Print a list of all the disks that are currently being used (a list of PVs)

1. vgdisplay -v | grep  "PV Name" > wte_pvdisk_used

1. vgdisplay -v prints a verbose listing of all volume groups

2. grep only prints lines that contain PV Name

3. The list of PVs is saved to a file

3. Refine the list of disks that are being used

1. awk '{ print $3 }' wte_pvdisk_used | sort -u > wte_pvdisk_sorted_u

1. awk prints on the 3rd field (the disk device)

2. sort will sort the list, surpressing any duplicate entries

3. the results are saved to a file

4. Compare the 2 files - the list of all Hitachi disks on the system with the list of all disks being used

1. diff wte_hitachi_sorted_u wte_pvdisk_sorted_u

1. diff compares the 2 files and prints out any differences.  The difference will be a disk that the system sees, but that is not being used by LVM

How to remove PVs Identify the hardware path of the disk to remove

o ioscan -fnC disk

Remove the special device file

o rmsf -H <HW Path from ioscan>

How to perform an non-destructive test of a disk:dd if=/dev/rdsk/cxxxxxx of=/dev/null bs=1024k