Linux Cluster and Distributed Resource Manager
-
Upload
hong-changbum -
Category
Technology
-
view
2.024 -
download
2
description
Transcript of Linux Cluster and Distributed Resource Manager
Linux Cluster and Distributed Resource
Manager
Center for Genome Science, NIH, KCDCHong Chang Bum
Linux Cluster
• 3 Linux Cluster Machine
• KHAN
• KGENE
• BIO
KHAN
• Total 94 Nodes (1 Master + 94 work Node)
• 64Bit PowerPC 720 4-way, 16GB RAM
• 64Bit PowerPC 770 2-way, 2GB RAM
• User Space: /home1 1.6TB
• Scratch Space: /home2/scratch 935GB
• Software: /home1/biosoftware
• EIGNSTRAT, merlin, phase, plink, R ...
DRM
• Distribute Resource Manager
• Job Scheduler
• IBM LoadLeveler, PBS, OpenPBS, Torqueue, Grid Engine, Sun Grid Engine(SGE), LSF
• Job Distribute, Job Status, Queue status, Node Status, User Accounting
1,000 Serial Job
• 1,000 Data Sets
• input.1, input.2... input.1000
• program -i input.(1~1000) -o output.(1~1000)
• 1,000 Job Submit Scripts
• 1,000 times Queue Submit Commands
Job Array - LoadLeveler
• Large Data sets: 1,000(input.1 ~ input.1000)
1,000 times??
Using Script
1,000 Command Script
1,000 Job Submit Command
1,000 Command Script
1,000 Job Submit Command
Job Status
Using R Scripts• Interactive Program
• Need to use R to analyze your data
Convert Bash Shell Script??
--quiet, -q Don’t print startup message--no-save Don’t save it
Using I/O redirection Form
R and LoadLeveler