Research Experience and Plans - Fermilab
Transcript of Research Experience and Plans - Fermilab
Overview• Third postdoc talk (previous: Feb 2020, Sep 2018)• Received PhD in May 2018 from the University of Notre Dame• Joined Fermilab in June 2018 as part of the Scientific Computing Division (SCD)
• Supervised by Nhan Tran• Mentored by Anadi Canepa• Analysis guided by Bo Jayatilaka• Technical project guided by Giuseppe Cerati
• Technical project: vectorized Kalman Filter (mkFit) project• Analysis: DM searches in LPC-DM group
February2021 2
DarkMatterSearches– LPC-DM• One of the leaders of the LPC-DM analysis group
(https://lpc-dm.github.io/), along with Bo, Matteo, and Doug• Collaborating with Cornell, Nebraska, Kyungpook, SPRACE,
UC Riverside and others
• Since last year: focus on leading LLP searches• iDM approaching preapproval
February2021 4
Ongoing projects within EXO LLP• Self-interacting dark matter (SIDM)• Inelastic dark matter (iDM)
Ongoing projects within EXO MET+X• Dark Higgs• Mono-top (not directly involved)
DarkHiggs• Search for dark Higgs model: spin-1 Z’ and
a spin-0 dark Higgs (hs)• Final states: monojet, mono-V, mono-hs(bb)• Mono-hs expected dominate sensitivity
• Collaborators: F. Dolek (Cukurova), S. Dogra, J. Lee, J. Hong, C.S. Moon (Kyungpook), T. Tomei (SPRACE)• Select hs→ bb with DeepAK15 tagger• Model main backgrounds using jet mass
sidebands + control regions• Fit is now working; plan to move
quickly to preapproval after finishingdocumentation
February2021 5
InelasticDarkMatter• Search for dark matter excited state χ2 →
ground state χ1 and a displaced lepton jet• Final state: lepton jet, aligned with
MET, back-to-back with an ISR jet• Based on arXiv:1508.03050 from
E. Izaguirre, G. Krnjaic, B. Shuve• First search at collider• Timeline: preapproval in the next month• Collaborators: A. Frankenthal (Princeton),
T. Reid, P. Wittich, J. Alexander (Cornell)• CADI: EXO-20-0120
February2021
Theoretical Models
08/10/2018 Yangyang Cheng | LPC-DM Meeting 2
• Astrophysical observations & DM-baryon density suggest a more complex dark
sector with dark matter interactions
• Mediators between the dark and visible worlds: Higgs-portal, vector-portal(Z’, A’…)
• Feeble couplings & no definite mass scaleÆ long-lived particles
Self-interacting Dark Matter
Cosmic connection
Collider production displaced
lepton jet
Darkonium
Inelastic Dark Matter
DM coannihilation
Excited state decay (through off-shell dark photon)
Collider Signature
Self-Interacting Dark Matter Inelastic Dark Matter
08/10/2018 Yangyang Cheng | LPC-DM Meeting 3
• Two-body decay of DM bound state• m_pseudoscalar = 2m_DM• m_darkphoton << m_DMÆ two displaced lepton jets back-to-back
mediator (on-shell): prompt decay
mediator (off-shell): displacement
ISR jet for triggering
• Heavier (“excited”) DM* has longer lifetime, decays to lighter (“ground”) DM, emitting a soft lepton pair
• O(MeV) – O(10GeV) DM; O(10%) mass splitting• m_darkphoton ~ m_DM (set to 3x m_DM) Æ one displaced lepton jet collimated with Etmiss
6
Analysisstrategy• Trigger on MET, since muons are soft (< 30 GeV)• Use displaced standalone (dSA) muons to extend
sensitivity for longer lifetimes• Match dSA muons with global muons if possible,
which have improved pt and vertex resolution• Derived scale factors for dSA ID and reco, which
are now officially supported as part of the muon POG
• Data-driven ABCD background estimation using Δφ and displacement of di-muon vertex• Sensitivity to exclude parameter space with 10%
mass splitting
February2021 7
iDM 2.0• Jose Monroy, Sam Bright-Thonney (Cornell) interested in iDM 2.0 • Adding electron final state, probe smaller mass splittings• Explore low pt electron electron reconstruction developed for b parking algorithm
• Collaborated with Christian last summer with our two virtual summer students
February2021 8
Self-interactingdarkmatter• Search for dark matter resonance Bps decaying
to long-lived dark photons Zd• Final state: two displaced lepton-jets• Require 1 lepton-jet to decay to 2μ for
trigger. Other can decay to 2μ or 2e• Based on arXiv:1811.05999 from Y. Tsai,
T. Xu, H. Yu• Timeline: approval in summer 2021• Collaborators: W. Si, G. Hanson (UC
Riverside), D. Claes, J. Castaneda (Nebraska)
February2021 9
SIDManalysisstrategy• 2018 only due to trigger constraints
• Large sensitivity increase possible in Run 3!
• Cluster PF muons, dSA muons, electrons, and photons using anti-kt algorithm• 2μ2e and 4μ final states
• ABCD method with Δφ between lepton-jets and lepton-jet isolation• Validation method: control region with
single muon as a proxy for a lepton-jet
February2021 10
MotivationTracking• Essential for all CMS physics (jets, MET,
PU mitigation, particle flow …) • Most computationally complex and time-
consuming part of reconstruction • Exponential increase in time/event with
respect to increased PUComputing• CPU frequency and single-thread performance
no longer exponentially increasing• Number of transistors and number of logical
cores still increasingConclusion: Need parallel algorithms
February2021
Figure 1. CPU time per eventversus instantaneous luminosity,for both full reconstruction andthe dominant tracking portion.Simulated data with pile-up of25 primary interactions perevent (PU25) corresponds to thedata taken during 2012, whilepile-up of 140 (PU140)corresponds to the low end ofestimates for the HL-LHC era.
as Intels Xeon Phi and NVIDIA general-purpose graphics processing units (GPGPUs). In thisinvestigation we have followed a staged approach, starting with Intel Xeon and Xeon Phi KnightsCorner (KNC) architectures, an idealized detector geometry, and a series of simpler “warm-up”exercises such as track fitting. This simplified problem domain was used to develop our tools,techniques, and understanding of the issues scaling track finding to many cores. The warm-upexercises let us develop useful components while also allowing the physicists to become familiarwith the computational tools and techniques, while the computational experts learned about theproblem domain. Armed with the results of those initial investigations, we are now addressingmore realistic detector geometries and event content, as well as adding new platforms. Thispaper gives an overview of our progress to date and assesses the e↵ectiveness of our stagedapproach.
2. Kalman Filter Tracking
Our targets for parallel processing are track reconstruction and fitting algorithms based on theKalman Filter [3] (KF). KF-based tracking algorithms are widely used to incorporate estimatesof multiple scattering directly into the trajectory of the particle. Other algorithms, such asHough Transforms and Cellular Automata [4][5], are more naturally parallelized. However,these are not the main algorithms in use at the LHC today. The LHC experiments have anextensive understanding of the physics performance of KF algorithms; they have proven to berobust and perform well in the di�cult experimental environment of the LHC.
KF tracking proceeds in three main stages: seeding, building, and fitting. Seeding providesthe initial estimate of the track parameters based on a few hits in the innermost regions of thedetector; seeding is currently out of scope for our project. Track building projects the trackcandidate outwards to collect additional hits, using the KF to estimate which hits represent themost likely continuation of the track candidate. Track building is the most time consuming step,as it requires branching to explore multiple candidate tracks per seed after finding compatiblehits on a given layer. When a complete track has been reconstructed, a final fit using the KF isperformed to provide the best estimate of the track’s parameters.
To take full advantage of parallel architectures, we need to exploit two types of parallelism:vectorization and parallelization. Vector operations perform a single instruction on multiple data(SIMD) at the same time, in lockstep. In tracking, branching to explore multiple candidates per
12
VectorizedKalmanFilterProject(mkFit)• Mission: Implement parallelized and
vectorized Kalman filter tracking algorithm • Collaboration with USCD, Princeton, and
Cornell; SciDAC-4 project U. Oregon• Focus on track building • Wrote new “Matriplex” library to perform
SIMD (single-instruction multiple data) operations on small matrices
• Test algorithm on highly parallel CPU architectures. Primarily using Intel Xeon or Xeon Phi• Current goal: prepare mkFit for use in
offline CMS reconstruction in Run 3
February2021
Achieved same track building efficiency as CMSSW, tested on
TTBar PU 50 sample
13
mkFit withinCMSSW• mkFit is integrated within CMSSW as an
external package• Can be run in place of the nominal track
building iteration 0• Achieved > 6x speedup with respect to
CMSSW• Caveat: CMSSW is compiled with gcc,
mkFit is compiled with icc• Includes all overheads, duplicate removal,
seed cleaning, data conversion, etc• Track fitting now takes more time than
track building• Work ongoing to extend mkFit to other
iterations
February2021 14
Recentaccomplishments• mkFit paper published in JINST
• “Speeding up particle track reconstruction using a parallel Kalman filter algorithm”, S. Lantz et. al., JINST 15 (2020) 09, doi 10.1088/1748-0221/15/09/p09030, arXiv:2006.00071
• Published under CMS “limited authorship” rules
• Main task in last year: various physics improvements, collectively referred to as Pandora’s Box• Necessary to reproduce full
performance of current CMS tracking
• Material effects, parametric magnetic field, invalid modules, multiple hits per layer from overlapping modules
• Requires re-tuning of track building parameters to optimize performance
February2021 15
11
12
13
14
15
16
17
18
0.225 0.250 0.275 0.300
Bin 5: Low pt, 1.7 < eta < 2.5
Fake rate
Num
ber
of la
yers
Previous performanceCurrent head of develNew proposed tuning
Codeportabilitystudies• Most current HPCs and all three future exascale HPCs rely on different GPUs
• Need code that can efficiently on CPUs and a variety of GPUs• Continued collaboration with Future Technologies Group at Oak Ridge
National Laboratory • Pre-covid, spent 25% of my time at ORNL
• Testing performance of one function from the mkFit algorithm• Propagation of track parameters to a
specific z location (“p2z”)• Implement using a variety of parallelization
strategies for CPUs and GPUs, including OpenMP, OpenACC, Eigen, and Kokkos
• Collaborating with HEP CCE-PPS (Matti, Martin) on related test kernel (“p2r”)
• Plan to submit paper to CS journal February2021 16
PresentationsandOutreach• LHC Physics Conference, virtual, May 2020
• Long-lived particle searches in CMS• Seminar: “Preparing CMS for the HL-LHC and the future of computing”
• Presented seminars at MIT, Harvard/Brandeis, SLAC, Brookhaven, UMass Amherst (scheduled)
• Colloquium at TRIUMF• QuarkNet masterclasses, 2021 Expanding Your Horizons workshop• Lecturer for Saturday Morning Physics• Developed and taught QuarkNet Summer Session for Teachers, a six-week
course for high school teachers• Official mentor for a summer intern in 2020
• Virtual internship with UChicago student based in Hungary• Unofficially mentored several other students
18February2021
Otherroles
• LPC Distinguished Researcher in 2020 and 2021• On the organizing committee for the LPC Physics Forum
• Convener, HEP Software Foundation (HSF) Data Analysis Working Group • Started in November 2021• Organized a series of three meetings on analysis metadata and how it is handled
across different HEP experiments• Future meeting topics: columnar and declarative analysis, improving analysis-
related training across experiments, initiating more discussions with the nuclear physics and neutrino physics communities
• ARC member for “Search for HH->bbγγ” (HIG-19-018) and “Search for dark matter in the dark Higgs to WW MET+diplepton final state” (EXO-20-013)
February2021 19
Jobapplications• Goal: professor at a college or university with a focus on teaching
• Establish CMS research program at a Primarily Undergraduate Institution (PUI) as a CMS “affiliate” member
• Last year: applied to several PUIs and a few research institutions• One interview at a research institution, 0 at PUIs
• Second round of applications has been much more successful so far• Adjusted application materials based on feedback from others at PUIs (Julie
Hogan, Matt Bellis, faculty at my undergrad)• Added teaching experience through QuarkNet course• Four virtual “on-campus” interviews
• Two job offers!
February2021 20
ChallengesforPUIapplications• Emphasizing that I’m not a “flight risk”
• Highlight that this is my actual goal, not a backup plan• Heard from several schools that my letters did a good job reinforcing this
• Mentoring undergrads in research through Fermilab internship programs• Gives confidence that my research program will actually work• Asked my references to point out my mentoring experience in letters
• Get creative with teaching experience • Considered teaching a course as an adjunct at a local community college, instead
made my own course through QuarkNet
• Roles outside CMS might be viewed more favorably• eg HSF convenership• Difficult to explain how CMS works and what my individual contributions are to the
collaboration
February2021 21
PUIinterviews• Teaching demos
• Often the deciding factor in who gets the offer• Either choosing my own topic or on an assigned topic
• Research talk at the level of sophomore undergraduates • Also used to evaluate teaching – outreach experience helped here!
• Common questions:• Why do you want to be here specifically/at a PUI in general?• How will you involve undergrads in research? • What courses do you want to teach? Any ideas for new courses?• What active learning techniques do you use in the classroom?• How will you promote an inclusive/equitable environment?
February2021 22