SDSC RP Update

SAN DIEGO SUPERCOMPUTER CENTER

at the UNIVERSITY OF CALIFORNIA; SAN DIEGO

SDSC RP Update

October 21, 2010



SDSC RP - Big Picture• Staffing

• Bill Young (TSRI) – New SDSC HPC Cluster Platforms Manager• Dave Hart (SDSC) – New NCAR User Services Manager

• Resources• Dash

• In production for start-up allocations, ~15 projects now computing• User Guide published: www.sdsc.edu/us/resources/dash/index.html (also linked

from TG website)• Gordon

• On schedule for deployment 3Q11• *Trestles (new “DCL” award system)

• *EOT Programs• Events

• SDSC 25th Anniversary – Oct 12• *Grand Challenges in Data-Intensive Discovery- October 26-29• SC’10 – Nov 13-19 (stop by the SDSC booth 2239)

http://www.sdsc.edu/us/resources/dash/index.html



Trestles - Big Picture• Proposed as a high-productivity system for modest-

scale/gateway users • Tailored to user/science needs • Establish policies and queues to maintain good throughput times –

continuous tuning of allocations/utilization/scheduling• Also plan a queue for long-running jobs

• Will be operated in parallel with Gordon• Nominal schedule

• 10/1/10 – Two test nodes• 11/1/10 – Onsite build of production system starts• 12/1/10 – Friendly users begin• 01/1/11 –Full production• Operations till Dec 2013



Trestles - System DescriptionSystem Component Configuration

AMD MAGNY-COURS COMPUTE NODESockets 4Cores 32Clock Speed 2.4 GHzFlop Speed 307 Gflop/sMemory capacity 64 GBMemory bandwidth 171 GB/sSTREAM Triad bandwidth 100 GB/sFlash memory (SSD) 120 GB

FULL SYSTEMTotal compute nodes 324Total compute cores 10,368Peak performance 100 Tflop/sTotal memory 20.7 TBTotal memory bandwidth 55.4 TB/sTotal flash memory 39 TB

QDR INFINIBAND INTERCONNECTTopology Fat treeLink bandwidth 8 GB/s (bidrectional)Peak bisection bandwidth 5.2 TB/s (bidirectional)MPI latency 1.3 us

DISK I/O SUBSYSTEMFile systems NFS, LustreStorage capacity (usable) 150 TB: Dec 2010

2PB : June 20114PB: July 2012

I/O bandwidth 50 GB/s



Trestles - Configuring for productivity• Plan to allocate ~70% of the theoretically available SUs

• The common practice is to allocate 80% (or more), which can result in high wait times. A 70% maximum will allow the scheduler to better satisfy a mix of scheduling modes, provide an opportunity to assess the impact of the other changes, and may be revised upward as we collect more data.

• Cap allocation per project at 1.5M SUs/year (~2.5% of annual total).• Because projects with large percentage allocations dominate resource consumption and

drive up queue waits, this cap not only creates room for more users, but also should reduce queue waits across the board.

• Allow new users to request up to 50,000 SUs in startup allocations, and front-load the SUs offered during the first few allocations cycles.• This should foster expanding the user community from the outset and accelerate the

frequently slow start-up for new systems. • Configure the job queues and resource schedulers for lower

expansion factors and generally faster turnaround.• System data and user feedback will be used to make adjustments to these in order to

achieve the optimum performance for each of the job classes



Right now we’re ahead of schedule with HW in this week



SDSC Summer High School Research Program

• Six-week paid internships to help increase awareness of the life science industry and related fields of research to students in the San Diego region

• Students gained exposure to career options, hands-on laboratory experience, work readiness skills, and mentoring by a company or research scientists

• The success of the program has led to plans for future internships

• 23 students participated at SDSC• Ended with a student poster

presentation and reception August 12



Student Poster Session

SDSC RP Update

Documents

Transcript of SDSC RP Update