How to Make MPI Awesome: A Proposal for MPI Sessions
-
Upload
insidehpc -
Category
Technology
-
view
447 -
download
3
Transcript of How to Make MPI Awesome: A Proposal for MPI Sessions
![Page 1: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/1.jpg)
HowtomakeMPIAwesome:MPISessions
WesleyBland,IntelRyanGrant,SandiaNa=onalLaboratory
DanHolmes,EdinburghParallelCompu=ngCenterKathrynMohror,LawrenceLivermoreLaboratoryMar=nSchulz,LawrenceLivermoreLaboratory
AnthonySkjellum,AuburnUniversityJeffSquyres,CiscoSystems,Inc.
^
more
Follow-ontoJeff’scrazythoughtsdiscussedatEuroMPI2015inBordeaux,France
![Page 2: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/2.jpg)
Whatwewant
• FixMPI-3.1limita=ons:– Cannotini=alizeMPIfromdifferenten==eswithinaprocesswithoutaprioriknowledge/coordina=on
– Cannotini=alizeMPImorethanonce– Cannotre-ini=alizeMPIa\erithasbeenfinalized– CannotseterrorbehaviorofMPIini=aliza=on
![Page 3: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/3.jpg)
Whatwewant• Anythread(e.g.,library)canuseMPIany=meitwants• Buts=llbeabletototallycleanupMPIif/whendesired
MPIProcess
// Library 3MPI_Init(…);
// Library 4MPI_Init(…);
// Library 5MPI_Init(…);
// Library 6MPI_Init(…);// Library 7
MPI_Init(…);
// Library 8MPI_Init(…);
// Library 9MPI_Init(…);
// Library 10MPI_Init(…);
// Library 11MPI_Init(…);
// Library 12MPI_Init(…);// Library 1
MPI_Init(…);// Library 2MPI_Init(…);
![Page 4: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/4.jpg)
Howdowegetthosethings?
![Page 5: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/5.jpg)
Newconcept:“session”
• AlocalhandletotheMPIlibrary– Implementa=onintent:lightweight/usesveryfewresources
– Canalsocachesomelocalstate
• Canhavemul=plesessionsinanMPIprocess– MPI_Session_init(…,&session);– MPI_Session_finalize(…,&session);
![Page 6: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/6.jpg)
MPISession
MPIProcess
oceanlibrary
MPI_SESSION_INIT(…)
atmospherelibrary
MPI_SESSION_INIT(…)
MPIlibrary
![Page 7: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/7.jpg)
MPISession
MPIProcess
oceanlibrary
atmospherelibrary
MPIlibrary
oceansession
atmos-pheresession
UniquehandlestotheunderlyingMPIlibrary
![Page 8: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/8.jpg)
MPISession
MPIProcess
oceanlibrary
atmospherelibrary
MPIlibrary
oceanErrorsreturn
atmos-phereErrorsabort
Uniqueerrorhandlers,info,localstate,etc.
![Page 9: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/9.jpg)
Great.Ihaveasession.Nowwhat?
![Page 10: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/10.jpg)
Fairwarning
• TheMPIrun=mehaslong-sincebeenabastardstepchild– Barelyacknowledgedinthestandard
– Mainlyintheformofnon-norma=vesugges=ons
• It’s&metochangethat
![Page 11: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/11.jpg)
Workwiththerun=me
• Generalscheme:– Querytheunderlyingrun-=mesystem• Geta“set”ofprocesses
– Determinetheprocessesyouwant• CreateanMPI_Group
– Createacommunicatorwithjustthoseprocesses• CreateanMPI_Comm
Queryrun=meforsetofprocesses
MPI_Group
MPI_Comm
MPI_Session
![Page 12: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/12.jpg)
Run=meexposessetsofprocesses
• Setsareiden=fiedbystringname• Twosetsaremandated– “mpi://WORLD”– “mpi://SELF”
• Otherimplementa=on-definedsetscanbereported,too
![Page 13: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/13.jpg)
Examplesofsets
MPIprocess0 MPIprocess1 MPIprocess2 MPIprocess3
mpi://WORLD
![Page 14: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/14.jpg)
Examplesofsets
MPIprocess0 MPIprocess1 MPIprocess2 MPIprocess3
mpi://WORLD
arch://x86_64
![Page 15: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/15.jpg)
Examplesofsets
MPIprocess0 MPIprocess1 MPIprocess2 MPIprocess3
mpi://WORLD
job://12942
arch://x86_64
![Page 16: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/16.jpg)
Examplesofsets
MPIprocess0 MPIprocess1 MPIprocess2 MPIprocess3
loca=on://rack/17 loca=on://rack/23
![Page 17: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/17.jpg)
Well,thatallsoundsgreat.
…butwhocallsMPI_INIT?
AndwhatsessiondoesMPI_COMM_WORLD/
MPI_COMM_SELFbelongto?
![Page 18: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/18.jpg)
Newconcept:nolongerrequireMPI_INIT/MPI_FINALIZE
![Page 19: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/19.jpg)
Theoveralltheme
• JustuseMPIfunc=onswheneveryouwant– Ini=aliza=onessen=allybecomesanimplementa=ondetail
• Finaliza=onwilloccurwheneveralluser-definedhandlesaredestroyed
![Page 20: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/20.jpg)
Exampleint main() { // Create a datatype – initializes MPI MPI_Type_contiguous(2, MPI_INT, &mytype);
Thecrea=onofthefirstuser-definedMPIobjectini=alizesMPI
Ini=aliza=oncanbealocalac=on!
![Page 21: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/21.jpg)
Exampleint main() { // Create a datatype – initializes MPI MPI_Type_contiguous(2, MPI_INT, &mytype); // Free the datatype – finalizes MPI MPI_Type_free(&mytype);}
Thedestruc=onofthelastuser-definedMPIobjectfinalizesMPI.
![Page 22: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/22.jpg)
Exampleint main() { // Create a datatype – initializes MPI MPI_Type_contiguous(2, MPI_INT, &mytype); // Free the datatype – finalizes MPI MPI_Type_free(&mytype);
// Re-initialize MPI! MPI_Type_dup(MPI_INT, &mytype);
Wecanalsore-ini=alizeMPI!(it’stransparenttotheuser–sowhynot?)
![Page 23: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/23.jpg)
Keyinsight:SplitMPIAPIsintotwosets
Performancedoesn’tma/er(asmuch)
• Func=onsthatcreate/query/destroy:– MPI_Comm– MPI_File– MPI_Win– MPI_Info– MPI_Op– MPI_Errhandler– MPI_Datatype– MPI_Group– MPI_Session– Arributes– Processes
• MPI_T
Performanceabsolutelyma/ers
• Pointtopoint• Collec=ves• I/O• RMA• Test/Wait• Handlelanguagexfer
![Page 24: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/24.jpg)
Keyinsight:SplitMPIAPIsintotwosets
Performancedoesn’tma/er(asmuch)
• Func=onsthatcreate/query/destroy:– MPI_Comm– MPI_File– MPI_Win– MPI_Info– MPI_Op– MPI_Errhandler– MPI_Datatype– MPI_Group– MPI_Session– Arributes– Processes
• MPI_T
Performanceabsolutelyma/ers
• Pointtopoint• Collec=ves• I/O• RMA• Test/Wait• Handlelanguagexfer
EnsurethatMPIisini=alized(and/orfinalized)bythese
func=ons
Thesefunc=onss=llcan’tbeusedunlessMPIis
ini=alized
![Page 25: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/25.jpg)
Keyinsight:SplitMPIAPIsintotwosets
Performancedoesn’tma/er(asmuch)
• Func=onsthatcreate/query/destroy:– MPI_Comm– MPI_File– MPI_Win– MPI_Info– MPI_Op– MPI_Errhandler– MPI_Datatype– MPI_Group– MPI_Session– Arributes– Processes
• MPI_T
Performanceabsolutelyma/ers
• Pointtopoint• Collec=ves• I/O• RMA• Test/Wait• Handlelanguagexfer
Thesefunc=onsinit/finalizeMPItransparently
Thesefunc=onscan’tbecalledwithoutahandlecreatedfrom
thele\-handcolumn
![Page 26: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/26.jpg)
Waitaminute–WhataboutMPI_COMM_WORLD?
int main() { // Can’t I do this? MPI_Send(…, MPI_COMM_WORLD);
Thiswouldbecallinga“performancemarers”
func=onbeforea“performancedoesn’tmarer”
func=on
I.e.,MPIhasnotini=alizedyet
![Page 27: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/27.jpg)
Waitaminute–WhataboutMPI_COMM_WORLD?
int main() { // This is valid MPI_Init(NULL, NULL); MPI_Send(…, MPI_COMM_WORLD);
Re-defineMPI_INITandMPI_FINALIZE:constructoranddestructorfor
MPI_COMM_WORLDandMPI_COMM_SELF
![Page 28: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/28.jpg)
INITandFINALIZE
• INIT/FINALIZEcreateanimplicitsession– YoucannotextractanMPI_SessionhandlefortheimplicitsessioncreatedbyMPI_INIT[_THREAD]
• Yes,youcanuseINIT/FINALIZEinthesameMPIprocessasothersessions
![Page 29: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/29.jpg)
(Abbreviated)Summary
• MPIsession:aunitofisola=on– Allow“ocean”and“atmosphere”scenarios
• Definesomeinterac=onswiththerun=me– Queryprocesssets,makegroups,makecommunicators
• NolongerrequireMPI_INIT/FINALIZE– Keyinsight:cannotinvoke“performancemarers”func=onswithoutahandle
![Page 30: How to Make MPI Awesome: A Proposal for MPI Sessions](https://reader031.fdocuments.us/reader031/viewer/2022021920/586f72791a28ab10258b5421/html5/thumbnails/30.jpg)
Moreideasanddetailsinthefullslidedeck
hrp://blogs.cisco.com/performance/“MPISessions:
AproposaltotheMPIForum”BlogentryfromMarch2,2016