Programming GPUs with OpenACC · GPU Hackathon 2017-OpenACC CPU-GPU memory model 3 PCIe...

GPU Hackathon 2017- OpenACC

Programming GPUs with OpenACC

1

SaberFekiComputationalScientistLead

SupercomputingCoreLaboratory,KAUST [email protected]


GPU architecture

2


CPU-GPU memory model

3

PCIeInterconnect16X- 8GB/s(gen2)and15.75GB/s(gen3),verythinpipe!KeplerK402,880cudacores1.48Tflops/s


GPU programming

4


OpenACC, the standard

• ByNVIDIA,CRAY,PGIandCAPS• ThestandardwasannouncedinNov2011atSC11conference• http://www.openacc-standard.org• OpenACC2.0releasedinsummer2013• Now,20+partnersfromacademiaandindustry

5


OpenACC advantages

• Easy:Directivesaretheeasypathtoacceleratecomputeintensiveapplications

• Open:OpenACCisanopenGPUdirectivesstandard,makingGPUprogrammingstraightforwardandportableacrossparallelandmulti-coreprocessors

• Powerful:GPUDirectivesallowcompleteaccesstothemassiveparallelpowerofaGPU

6


PGI and CAPS compilers study (I)

7

S.Feki,A.Al-Jarro,H.Bağcı.PortinganExplicitTimeDomainVolumeIntegralEquationSolveronGPUswithOpenACC,IEEEAntennasandPropagationMagazine,July,2014

#pragma acc kernels {for ( l = 0 ; l < nt; ++l) { // time loop#pragma acc loop independent collapse (3)

for (int i = 0; i < n; ++i){ for (int j = 0; j < n; ++j){

for (int k = 0; k < n; ++k){B[i][j][k] = B[i]][j][k] + ....

}}

}

#pragma acc loop independent collapse (3)for (int i = 0; i < n; ++i){

for (int j = 0; j < n; ++j){for (int k = 0; k < n; ++k){

B[i][j][k] = B[i]][j][k] + ....}

}}

} // end time loop }

#pragma acc datafor ( l = 0 ; l < nt; ++l) { // time loop#pragma acc kernels#pragma acc loop independent gang

for (int i = 0; i < n; ++i){ #pragma acc loop independent gang,vector

for (int j = 0; j < n; ++j){#pragma acc loop independent gang,vector


} } }#pragma acc kernels#pragma acc loop independent gang

for (int i = 0; i < n; ++i){ #pragma acc loop independent gang,vector

for (int j = 0; j < n; ++j){#pragma acc loop independent gang,vector


} } }} // end time loopCAPS PGI


PGI and CAPS compilers study (II)

8

0

5

10

15

20

25

30

35

6 11 25 32 41 56 77 113 176

Speedup

Numberofdegreesoffreedom(X1000)

CAPSPGI

S.Feki,A.Al-Jarro,H.Bağcı.PortinganExplicitTimeDomainVolumeIntegralEquationSolveronGPUswithOpenACC,IEEEAntennasandPropagationMagazine,July,2014


Directive syntax

• Fortran!$accdirective[clause[,]clause]…]…oftenpairedwithamatchingenddirective!$accenddirective• C#pragmaaccdirective[clause[,]clause]…]Oftenfollowedbyastructuredcodeblock

9


kernels: Your first OpenACC Directive

• Eachloopexecutedasaseparatekernel (aparallelfunctionthatrunsontheGPU)

!$acc kernelsdo i=1,n

a(i) = 0.0 b(i) = 1.0c(i) = 2.0

end dodo i=1,na(i) = b(i) + c(i)

end do !$acc end kernels

10


Compile and run

• C:pgcc–acc[-Minfo=accel]–osaxpy_accsaxpy.c• Fortran:pgf90–acc[-Minfo=accel]–osaxpy_accsaxpy.f90• Compileroutput:[sfeki@c4hdnsaxpy]$pgcc-acc-ta=nvidia-Minfo=accel-osaxpysaxpy.csaxpy:

5,Generatingpresent_or_copyin(x[0:n])Generatingpresent_or_copy(y[0:n])Generatingcomputecapability1.0binaryGeneratingcomputecapability2.0binary

6,LoopisparallelizableAcceleratorkernelgenerated6,#pragmaaccloopgang,vector(128)/*blockIdx.xthreadIdx.x*/CC1.0:8registers;48shared,0constant,0localmemorybytesCC2.0:12registers;0shared,64constant,0localmemorybytes

11


SAXPY example, revisited

12


Jacobi Iteration: C code

13


Jacobi Iteration: OpenACC code

14


PGI Accelerator Compiler output

15


Performance

16

CPU:[email protected]

GPU:NVIDIATeslaM2070


What went wrong ?

17


Excessive data transfer

18


Another way of detecting it: NVIDIA Profiler

• Usenvprof forprofilingtheGPUapplication:

• UseNVVPGUI:NVIDIAVisualProfiler:

19


Data construct

• Fortran!$accdata[clause…]structuredblock

!$accenddata• C#pragmaaccdata[clause…]{structuredblock}• Managedatamovement.Dataregionsmaybenested• GeneralClausesif(condition)async(expression)

20


Data clauses

• copy (list)AllocatesmemoryonGPUandcopiesdatafromhosttoGPUwhenenteringregionandcopiesdatatothehostwhenexitingregion.

• copyin (list)AllocatesmemoryonGPUandcopiesdatafromhosttoGPUwhenenteringregion.

• copyout (list)AllocatesmemoryonGPUandcopiesdatatothehostwhenexitingregion.

• create (list)AllocatesmemoryonGPUbutdoesnotcopy.• present (list)DataisalreadypresentonGPUfromanother

containingdataregion.• andpresent_or_copy[in|out],present_or_create,deviceptr.

21


Array shaping

• Compilersometimescannotdeterminesizeofarrays• Mustspecifyexplicitlyusingdataclausesandarray“shape”• C#pragmaaccdatacopyin(a[0:size]),copyout(b[s/4:3*s/4])

• Fortran!$accdatacopyin(a(1:size)),copyout(b(s/4:3*s/4))• Note:dataclausescanbeusedondata,kernelsorparallel

22


Jacobi Iteration: OpenACC C Code, Revisited

23


Performance numbers

24


New NVIDIA profiles

25


CUDA Kernels

• Threadsaregroupedintoblocks• Blocks aregroupedintoagrid• Akernel isexecutedasagridofblocksofthreads

26


Thread blocks• Threadblocksallowcooperation– Cooperativelyload/storeblocksofmemorythattheyalluse

– Shareresultswitheachotherorcooperatetoproduceasingleresult

– Synchronizewitheachother• Threadblocksallowscalability– Blockscanexecuteinanyorder,concurrentlyorsequentially

– Thisindependencebetweenblocksgivesscalability:• AkernelscalesacrossanynumberofSMs

27


Mapping OpenACC to CUDA I

• TheOpenACCexecutionmodelhasthreelevels:gang,worker,andvector

• AllowsmappingtoanarchitecturethatisacollectionofProcessingElements(PEs)

• OneormorePEspernode• EachPEismulti-threaded• Eachthreadcanexecutevectorinstructions

• Tile pragmainOpenACC2.0

28


Mapping OpenACC to CUDA II

• ForGPUs,themappingisimplementation-dependent.Somepossibilities:– gang==block,worker==warp,andvector==threadsofawarp– omit“worker”andjusthavegang==block,vector==threadsofablock

• Dependsonwhatthecompilerthinksisthebestmappingfortheproblem

• ...Butexplicitlyspecifyingthatagivenloopshouldmaptogangs,workers,and/orvectorsisoptionalanyway– Furtherspecifyingthenumberofgangs/workers/vectorsisalsooptional

– Sowhydoit?Totunethecodetofitaparticulartargetarchitectureinastraightforwardandeasilyre-tunedway.

29


OpenACC loop directive and clauses

#pragmaacc kernelsloopfor(int i=0;i<n;++i)y[i]+=a*x[i];Useswhatevermappingtothreadsandblocksthecompilerchooses.Perhaps16blocks,256threadseach#pragma acc kernelsloopgang(100),vector(128)for(int i=0;i<n;++i)y[i]+=a*x[i];100threadblocks,eachwith128threads,eachthreadexecutesoneiterationoftheloop,usingkernels#pragma acc parallelnum_gangs(100),vector_length(128){#pragma acc loopgang,vectorfor(int i=0;i<n;++i)y[i]+=a*x[i];

}100threadblocks,eachwith128threads,eachthreadexecutesoneiterationoftheloop,usingparallel

30


Mapping OpenACC to CUDA threads and blocks

31

• Nestedloopsgeneratemulti-dimensionalblocksandgrids:#pragmaacckernelsloopgang(100),vector(16)for(…)

#pragmaaccloopgang(200),vector(32)for(…)

16threadtallblock

100blockstall(row/Y

direction)

and32threadwide

200blockswide(column/Xdirection)


Other clauses for loop directive

32

#pragmaaccloop[cluases]

•independent:forindependentloops•seq:forsequentialexecutionoftheloop•Reduction:forreductionoperationsuchasmin,max,etc…


Jacobi example … again

33

WithKernelsanddatadirectives



34

Afteraddingloopdirectivewithgangandvectorclauses


An opportunity for Auto-tuning

• Gangandvectorvaluescanbeauto-tunedfortheapplication,targetingtheavailableacceleratordevice

35

2.37

1.68

1.83

1.44

1.15

1.49

1.67

2.54

1.171.22

1.321.24 1.20

1.10

1.331.25 1.27 1.26

1.00

1.20

1.40

1.60

1.80

2.00

2.20

2.40

2.60

Performan

ceSpe

edup

ProblemSizes

S.Siddiqui,F.Al-Zayer,S.Feki.HistoricLearningApproachforAuto-tuningOpenACCAcceleratedScientificApplications,iWAPT2014,Eugene,Oregon,USA


An opportunity for Auto-tuning

36

Input code annotated with OpenACC

#pragma acc kernels#pragma acc loop independentfor (x = 4 ; x < nx-4; x++) {

#pragma acc loop independentfor (y = 4; y < ny-4; y++) {

#pragma acc loop independentfor (z = 4; k < nz-4; z++) {U[x][y][z] = c1*V[x]][y][z] + ....

}}

}

Accelerator Specification

Automatic code generator

#pragma acc kernels#pragma acc loop independent gang(a),vector(b)for (x = 4 ; x < nx-4; x++) {

#pragma acc loop independent gang(c)for (y = 4; y < ny-4; y++) {

#pragma acc loop independent vector(d)for (z = 4; k < nz-4; z++) {U[x][y][z] = c1*V[x]][y][z] + ....

}}

}

#pragma acc kernels#pragma acc loop independent gang(a)for (x = 4 ; x < nx-4; x++) {

#pragma acc loop independent gang(b),vector(c)for (y = 4; y < ny-4; y++) {

#pragma acc loop independent vector(d)for (z = 4; k < nz-4; z++) {U[x][y][z] = c1*V[x]][y][z] + ....

}}

}

#pragma acc kernels#pragma acc loop independent gang(a),vector(b)for (x = 4 ; x < nx-4; x++) {

#pragma acc loop independent vector(c)for (y = 4; y < ny-4; y++) {

#pragma acc loop independent gang(d),vector(e)for (z = 4; k < nz-4; z++) {U[x][y][z] = c1*V[x]][y][z] + ....

}}

}

#pragma acc kernels#pragma acc loop independent for (x = 4 ; x < nx-4; x++) { #pragma acc loop independent gang(a),vector(b)

for (y = 4; y < ny-4; y++) {#pragma acc loop independent gang(c),vector(d)

for (z = 4; k < nz-4; z++) {U[x][y][z] = c1*V[x]][y][z] + ....

}}

}

Runtime evaluation and selection

Database



37

• Whichotheroptimizationwecanfurtherdo?

– RestructuringthecodewillenhancebothCPUandGPUversion– Hint:reducememoryoperations


OpenACC Runtime Library

38

• InC:#include“openacc.h”• InFortran:#include‘openacc_lib.h’ oruseopenacc• Contains:– Prototypesofallroutines– Definitionofdatatypesusedintheseroutinesincludingenumerationtypedescribingtypesofaccelerators

GPU Hackathon 2017- OpenACC 39

OpenACC Runtime Library Definitions

• openacc_version withavalueyyyymm (yearandmonthoftheopenacc version)

• acc_device_t :typeofacceleratordevice– acc_device_none– acc_device_default– acc_device_host– acc_device_not_host


• acc_get_num_devices:returnsthenumberofdevicesofthegiventypeattachedtothehost

• acc_set_device_type:tellswhichtypeofdevicetousewhenexecutinganacceleratorparallelorkernelregion.

• acc_get_device_type:tellswhichtypeofdevicetobeusedforthenextacceleratedregion

• acc_set_device_num:specifywhichdevicetouse• acc_get_device_num:returnsthedevicenumberofthe

specifieddevicetypethatwillbeusedtorunthenextacceleratorparallelorkernelsregion

40

OpenACC Runtime Library Routines I


OpenACC Runtime Library Routines II

• acc_init:initializetheruntime,canbeusedtoisolatetheinitializationcostfromthecomputationcost

• acc_shutdown:shutdowntheconnectiontothedeviceandfreeanyallocatedresources

• acc_malloc:allocatememoryontheacceleratordevice• acc_free:freesmemoryontheacceleratordevice

41


OpenACC Runtime Library Routines: use case

• PortinganMPIcodetomultipleGPUs.• Exampleinrunningon8nodes,with4GPUseach,i.e.32MPI

processes

• acc_init()• acc_set_device_num( rank%4)

• Eachnoderuns4MPIprocesses,eachofthemisoffloadingcomputekernelstoaseparateGPU

42

S.Feki,A.Al-Jarro,H.Bağcı.MultipleGPUsElectromagneticsSimulationsusingMPIandOpenACC,PosterinGPUTechnologyConference,SanJose,California,USA,March24-27,2014


OpenACC and CUDA libraries

43


GPU accelerated libraries

44


Sharing data with libraries

• CUDAlibrariesandOpenACCbothoperateondevicearrays• OpenACCprovidesmechanismsforinteroperabilitywith

librarycalls– deviceptr dataclause– host_data construct

• Note:samemechanismsusefulforinteroperabilitywithcustomCUDAC/C++/Fortrancode

45


deviceptr Data Clause

deviceptr(list)Declaresthatthepointersinlistrefertodevicepointersthatneednotbeallocatedormovedbetweenthehostanddeviceforthispointer.Example:• C#pragmaacc datadeviceptr(d_input)• Fortran$!acc datadeviceptr(d_input)

46


host_data Construct

• Makestheaddressofdevicedataavailableonthehost.• deviceptr(list)Tellsthecompilertousethedeviceaddress

foranyvariableinlist.Variablesinthelistmustbepresentindevicememoryduetodataregionsthatcontainthisconstruct

• Example• C#pragmaacc host_data use_device(d_input)• Fortran$!acc host_data use_device(d_input)

47


Summary on device pointers

• Usedeviceptr dataclausetopasspre-allocateddevicedatatoOpenACCregionsandloops

• Usehost_data togetdeviceaddressforpointersinsideaccdataregions

• ThesametechniquesshownherecanbeusedtosharedevicedatabetweenOpenACCloopsand– YourcustomCUDAC/C++/Fortran/etc.devicecode– AnyCUDALibrarythatusesCUDAdevicepointers

48GPU Hackathon 2017- OpenACC

GPU Hackathon 2017- OpenACC 49

Thanks !


Programming GPUs with OpenACC · GPU Hackathon 2017-OpenACC CPU-GPU memory model 3 PCIe...

Documents

Transcript of Programming GPUs with OpenACC · GPU Hackathon 2017-OpenACC CPU-GPU memory model 3 PCIe...