(Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and...
Transcript of (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and...
![Page 1: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/1.jpg)
(Re)Introduction to High-Performance Computing (HPC)
![Page 2: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/2.jpg)
HPC cluster structure
![Page 3: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/3.jpg)
HPC cluster components
Nodes: Individual computers in the cluster
Cores (threads): individual processing units available within each CPU of each Node
e.g. a “Node” with eight “quad”-core CPUs = 32 cores for that node.
Shared disk: storage that can be shared (and accessed) by all nodes
![Page 4: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/4.jpg)
High-Performance Computing
Serial Multithreaded & Serial
Multithreaded and Parallel
For 3 samples
![Page 5: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/5.jpg)
Research Computing https://rc.hms.harvard.edu/
!5
Introduction to High Performance Computing and O2 for New Users
HMS Research Computing
(Slides courtesy of Kris Holton at HMS-RC)
![Page 6: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/6.jpg)
Research Computing https://rc.hms.harvard.edu/
!6
O2 Tech Specs• 11000 cores • 32, 28, or 20 cores per node • 256-160GB RAM (memory) per node (8-9GB/core) • 756GB RAM (high memory nodes) • 40 GPUs (V100, K80, M40) • CentOS 7 Linux • SLURM job scheduler
![Page 7: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/7.jpg)
Research Computing https://rc.hms.harvard.edu/
2-Factor Authentication
▪ For logins using WiFi networks other than HMS Private/Harvard Secure
▪ Easiest: download Duo app to phone
▪ Similar to the setup for Harvard Key logins
▪ Setup details at:https://wiki.rc.hms.harvard.edu:8443/display/O2/Two+Factor+Authentication+on+O2
![Page 8: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/8.jpg)
The Job Scheduler, SLURM
Choosing the proper resources for your job with the appropriate Slurm arguments
![Page 9: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/9.jpg)
Research Computing https://rc.hms.harvard.edu/
!9
Submitting JobsIn an “interactive session”, programs can be run directly, however your computer will have to remain connected to the cluster for the duration of this run.
mfk8@login-1:~$ srun --pty -p interactive -t 0-8:00 —mem 6G /bin/bash
mfk8@compute-a:~$ bowtie2 -c 4 hg19 file1_1.fq
![Page 10: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/10.jpg)
Research Computing https://rc.hms.harvard.edu/
!10
Submitting JobsWhat if you wanted to run the program, close your computer and come back later to check on it? A script with the required commands can be submitted to O2 (SLURM) using the
sbatch command.
mfk8@login-1:~$ sbatch mybowtiejob.sh
OR
mfk8@compute-a:~$ sbatch mybowtiejob.sh
![Page 11: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/11.jpg)
Research Computing https://rc.hms.harvard.edu/
!11
Creating a job submission script#! /bin/sh
#SBATCH -p short#SBATCH –t 0-03:00 #aim for 125% over
#SBATCH –c 4 #number of cores/threads, match with what is used in script
#SBATCH --mem=8G
#SBATCH –o %j.out
#SBATCH –e %j.err
#SBATCH -J bowtie2_run1 #job name
#SBATCH —mail-type=ALL #notify about job completion or errors
#SBATCH –[email protected]
module load gcc/6.2.0 module load bowtie2/2.2.9
bowtie -c 4 hg19 file1_1.fq
Save script as myJobScript.run and run it as follows from login or compute node:
$ sbatch myJobScript.run
![Page 12: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/12.jpg)
Research Computing https://rc.hms.harvard.edu/
!12
Partitions -p
Partition Priority Max Runtime Max Cores Limits
short 12 12 hours 20medium 6 5 days 20long 4 30 days 20interactive 14 12 hours 20 2 job limitpriority 14 30 days 20 2 job limitmpi 12 5 days 640 20 core minhighmem 12 5 days 8 750Ggpu 120 GPU hours 20 cputransfer 5 days 4
![Page 13: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/13.jpg)
Research Computing https://rc.hms.harvard.edu/
!13
Job Priority• Dynamically assigned • Factors contributing: Age, Fairshare, Partition, QOS, Nice • Fairshare: 0-1 scale
![Page 14: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/14.jpg)
Managing jobs and getting information about submitted/running jobs
![Page 15: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/15.jpg)
Research Computing https://rc.hms.harvard.edu/
!15
Job Monitoring• $ O2squeue #HMS wrapper• $ squeue –u eCommons
Detailed job info: • $ scontrol show jobid <jobid>
Completed job statistics: • $ O2sacct #HMS wrapper
![Page 16: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/16.jpg)
Research Computing https://rc.hms.harvard.edu/
!16
Cancelling Jobs• $ scancel <jobid>• $ scancel --name JOBNAME
![Page 17: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/17.jpg)
Filesystems and storage
![Page 18: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/18.jpg)
Filesystems and storage
![Page 19: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/19.jpg)
Research Computing https://rc.hms.harvard.edu/
!19
O2 Primary Storage
O2 Cluster• 11000+ cores• SLURM batch system
Your computer
/home• /home/user_id• quota: 100GB per user• Backup: extra copy & snapshots: • daily to 14 days, weekly up to 60
days
/n/data1, /n/data2, /n/groups• /n/data1/institution/dept/lab/
your_dir• quota: expandable• Backup: extra copy &
snapshots:• daily to 14 days, weekly up to
60 days
![Page 20: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/20.jpg)
Research Computing https://rc.hms.harvard.edu/
!20
Temporary “Scratch” storage
• /n/scratch2 • For data only needed temporarily during analyses. • Each account can use up to 10 TB and 1 million files/directories
º Lustre --> a high-performance parallel file system running on DDN Storage. º More than 1 PB of total shared disk space. º No backups! Files are automatically deleted after unaccessed for 30 days, to save space. º More info at: http://hmsrc.me/O2docs
![Page 21: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/21.jpg)
Research Computing https://rc.hms.harvard.edu/
!21
Checking storage usage▪ For /n/data1, /n/data2, /n/groups
› $ quota› Breaks down per user, directory
▪ For /n/scratch2› $ lfs quota –h /n/scratch2 › 1 million files/folders, 10TB limit
![Page 22: (Re)Introduction to High-Performance Computing (HPC)Introduction to High Performance Computing and O2 for New Users HMS Research Computing (Slides courtesy of Kris Holton at HMS-RC)](https://reader033.fdocuments.us/reader033/viewer/2022042110/5e8b731a933c364d4d7814c0/html5/thumbnails/22.jpg)
Research Computing https://rc.hms.harvard.edu/
!22
For more direction• http://hmsrc.me/O2docs • http://rc.hms.harvard.edu • RC Office Hours: Wed 1-3p Gordon Hall 500 • [email protected]