CHTC working w/ Chemistry Dept
description
Transcript of CHTC working w/ Chemistry Dept
![Page 1: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/1.jpg)
CHTC working w/ Chemistry Dept
![Page 2: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/2.jpg)
Gromacs
![Page 3: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/3.jpg)
30+ day runtime
• Too long, even as HTC
• Step one – compile with SSE support– 10x improvement
• Just a Gromacs compile-time option– Hand-coded assembly, not gcc option
![Page 4: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/4.jpg)
3 days still too long…
• Gromacs also support MPI
• CHTC doesn’t have infiniband
• What do to?
![Page 5: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/5.jpg)
Whole machine jobs
• Submit file magic to claim all 8 slotsuniverse = vanilla
requirements = (CAN_RUN_WHOLE_MACHINE =?= TRUE)
+RequiresWholeMachine=trueexecutable = some job
arguments = arguments
should_transfer_files = yes
when_to_transfer_output = on_exit
transfer_input_files = inputs
queue
![Page 6: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/6.jpg)
MPI on Whole machine jobs
universe = vanilla
requirements = (CAN_RUN_WHOLE_MACHINE =?= TRUE)
+RequiresWholeMachine=true
executable = mpiexec
arguments = -np 8 real_exeshould_transfer_files = yes
when_to_transfer_output = on_exit
transfer_input_files = real_exequeue
Whole machine mpi submit file
Condor Motto:
If you want it,
Bring it yourself
![Page 7: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/7.jpg)
Advantages
• Condor is parallel agnostic:– MPICH, OpenMPI, pthreads, fork, etc.
• High-bandwith memory transport
• Easy to debug– Ssh-to-job still works
• Access to all machine’s memory
![Page 8: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/8.jpg)
Disadvantages
• Still need to debug parallel program– helps if others already have
• Fewer full-machine slots– Currently 30, more coming
• Lots of competition for 30 machines
![Page 9: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/9.jpg)
30 machines not enough: OSG to the rescue
![Page 10: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/10.jpg)
Glideing-in MPI to OSG
CHTCPool
CHTCschedd
Jobqueue
Glide-inPool
![Page 11: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/11.jpg)
OSG Submit File
universe = vanilla
requirements = (CAN_RUN_WHOLE_MACHINE =?= TRUE)
+RequiresWholeMachine=true
+WantGlidein = trueexecutable = some job
arguments = arguments
should_transfer_files = yes
when_to_transfer_output = on_exit
transfer_input_files = inputs
queue
![Page 12: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/12.jpg)
Restrictions of glidein
• More diverse hardware resources– No pre-staged, some AMD machines…
• Less guaranteed memory (1.7 Gb)
• Preemption at any time– Restartable or checkpointable a big plus– (Gromacs, others has checkpointing!)
![Page 13: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/13.jpg)
Computational Results
![Page 14: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/14.jpg)
Real Results
• Iron-Catalyzed Oxidation Intermediates Captured in A DNA Repair Monooxygenase, C. Yi, G. Jia, G. Hou, Q. Dai, G. Zheng, X. Jian, C. G. Yang, Q. Cui, and C. He, {\it Science}, Submitted
![Page 15: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/15.jpg)
Real Results
Disruption and formation of surface salt bridges are coupled to DNA binding in integration host factor (IHF): acomputational analysis, L. Ma, M. T. Record, Jr., N. Sundlass, R. T. Raines and Q. Cui, {\it J. Mol. Biol.}, Submitted
![Page 16: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/16.jpg)
Real Results
• An implicit solvent model for SCC-DFTB with Charge-Dependent Radii, G. Hou, X. Zhu and Q. Cui, {\it J. Chem. Theo. Comp.}, Submitted
•
![Page 17: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/17.jpg)
Real Results
• Sequence-dependent interaction of $\beta$-peptides with membranes, J. Mondal, X. Zhu, Q. Cui and A. Yethiraj, {\it J. Am. Chem. Soc.}, Submitted
•
![Page 18: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/18.jpg)
Real Results
• A new coarse-grained model for water: The importance of electrostatic interactions, Z. Wu, Q. Cui and A. Yethiraj, {\it J. Phys. Chem. B} Submitted
•
![Page 19: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/19.jpg)
Real Results
• How does bone sialoprotein promote the nucleation of hydroxyapatite? A molecular dynamics study using model peptides of different conformations, Y. Yang, Q. Cui, and N. Sahai, {\it Langmuir}, Submitted
•
![Page 20: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/20.jpg)
Real Results
• Preferential interactions between small solutes and the protein backbone: A computational analysis, L. Ma, L. Pegram, M. T. Record, Jr., Q. Cui, {\it Biochem.}, 49, 1954-1962 (2010)
•
![Page 21: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/21.jpg)
Real Results
• Establishing effective simulation protocols for $\beta$- and$\alpha/\beta$-peptides. III. Molecular Mechanical (MM) model for a non-cyclic $\beta$-residue, X. Zhu, P. K\"onig, M. Hoffman, A. Yethiraj and Q. Cui, {\it J. Comp. Chem.}, In press (DOI: 10.1002/jcc.21493)
•
![Page 22: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/22.jpg)
Real Results
• Curvature Generation and Pressure Profile in Membrane with lysolipids: Insights from coarse-grained simulations, J. Yoo and Q. Cui, {\it Biophys. J.} 97, 2267-2276 (2009)
![Page 23: CHTC working w/ Chemistry Dept](https://reader036.fdocuments.us/reader036/viewer/2022062301/56815294550346895dc0ba58/html5/thumbnails/23.jpg)
Summary
• CHTC can deliver a lot of CPU, if jobs are “shaped” appropriately
• CHTC supports whole machine jobs
• Consider “small mpi”, or – High Throughput of High performance
• OSG Record: 50,000 CPU hours in a day!– (this could be you!)