10/7/16

1

ConcurrentsystemsLecture1:Introductiontoconcurrency,threads,

andmutualexclusion

Michaelmas 2016Dr RobertN.M.Watson(WiththankstoDr StevenHand)

1

Concurrentanddistributedsystems

• Onecourse,twoparts– 8lecturesonconcurrentsystems(Michaelmas term)– 8furtherlecturesofdistributedsystems(Lentterm)

• Similarinterestsandconcerns:– Scalability givenparallelismanddistributedsystems– Masklocalordistributedcommunicationslatency– Importanceinobserving(orenforcing)executionorders– Correctness inthepresenceofconcurrency(+debugging)

• Importantdifferences– Underlyingprimitives:sharedmemoryvs.messagepassing– Distributedsystemsexperiencecommunicationsfailure– Distributedsystems(may)experienceunboundedlatency– (Further)difficultyofdistributedtime

2

10/7/16

2

Concurrentsystemscourseoutline1. Introductiontoconcurrency,threads,and

mutualexclusion2. Moremutualexclusion,semaphores,producer-

consumer,andMRSW3. CCR,monitors,concurrencyinpractice4. Safetyandliveness5. Concurrencywithoutshareddata;transactions6. Furthertransactions7. Crashrecovery;lockfreeprogramming;TM8. Concurrentsystemscasestudy

3

Recommendedreading

• “OperatingSystems,ConcurrentandDistributedSoftwareDesign“,JeanBaconandTimHarris,Addison-Wesley2003

• “ModernOperatingSystems”,(3rd Ed),AndrewTannenbaum,Prentice-Hall2007

• “JavaConcurrencyinPractice”,BrianGoetzandothers,Addison-Wesley2006

4

10/7/16

3

Whatisconcurrency?• Computersappeartodomanythingsatonce

– e.g.runningmultipleprogramsonyourlaptop– e.g.writingbackdatabufferedinmemorytotheharddiskwhile

theprogram(s)continuetoexecute• Inthefirstcase,thismayactuallybeanillusion

– e.g.processestimesharingasingleCPU• Inthesecond,thereistrueparallelism

– e.g.DirectMemoryAccess(DMA)transfersdatabetweenmemoryandI/Odevices(e.g.,NIC,SATA)atthesametimeastheCPUexecutescode

– e.g.,twoCPUsexecutecodeatthesametime• Inbothcases,wehaveaconcurrency

– manythingsareoccurring“atthesametime”

5

Inthiscoursewewill• Investigatethewaysinwhichconcurrencycanoccurina

computersystem– processes,threads,interrupts,hardware

• Considerhowtocontrolconcurrency– mutualexclusion(locks,semaphores),condition

synchronization,lock-freeprogramming• Learnaboutdeadlock,livelock,priorityinversion

– Andprevention,avoidance,detection,recovery• Seehowabstractioncanprovidesupportforcorrect&

fault-tolerantconcurrentexecution– transactions,serialisability,concurrencycontrol

• Exploreadetailedconcurrentsoftwarecasestudy• Nextterm,extendtheseideastodistributedsystems

6

10/7/16

4

Recall:Processesandthreads• Processes areinstancesofprogramsinexecution

– OSunitofprotection&resourceallocation– Hasavirtualaddressspace;andoneormorethreads

• Threads areentitiesmanagedbythescheduler– Representsanindividualexecutioncontext– Athreadcontrolblock(TCB)holdsthesavedcontext(registers,

includingstackpointer),schedulerinfo,etc• Threadsrunintheaddressspacesoftheirprocess

– (andsometimesinthekerneladdressspace)• ContextswitchesoccurwhentheOSsavesthestateofone

threadandrestoresthestateofanother– Ifaswitchisbetweenthreadsindifferentprocesses,then

processstateisalsoswitched– e.g.,theaddressspace

7

Multiplethreadswithinaprocess• Asingle-threadedprocesshas

code,aheap,astack,registers• Additionalthreadshavetheir

ownregistersandstacks– Per-threadprogramcounters($pc)

allowexecutionflowstodiffer– Per-threadstackpointers($sp)

allowlocalvariablestodiffer

• Heapandcode(+globals)aresharedbetweenallthreads

• Accesstoanotherthread’sstackissometimespossible– butdeeplydiscouraged! 8

Code

Processaddressspace

Heap

Thread1registers

$pc

$t0

$sp

$a0$a1

Stack

Thread2registers

$pc

$t0

$sp

$a0$a1

Stack

10/7/16

5

ConcurrencywithasingleCPU(1)

• Process/OSconcurrency– ProcessXrunsforawhile(untilblocks orinterrupted)– OSrunsforawhile(e.g.doessomeTCPprocessing)– ProcessXresumeswhereitleftoff…

• Inter-processconcurrency– ProcessXrunsforawhile;thenOS;thenProcessY;thenOS;thenProcessZ;etc

• Intra-processconcurrency– ProcessXhasmultiplethreadsX1,X2,X3,…– X1runsforawhile;thenX3;thenX1;thenX2;then…

9

ConcurrencywithasingleCPU(2)

• WithjustoneCPU,canthinkofconcurrencyasinterleaving ofdifferentexecutions,e.g.

Proc(A) OS Proc(B) Proc(C) Proc(A)OS Proc(B) OS OS

time

timerinterrupt diskinterrupt systemcall pagefault

• Exactlywhereexecutionisinterruptedandresumedis notusuallyknowninadvance…• this makesconcurrencychallenging!

• Generallyshouldassumeworstcasebehavior10Non-deterministicorsocomplexastobeunpredictable

10/7/16

6

Concurrencywithmultipleprocessors

• ManymodernsystemshavemultipleCPUs– Andevenifdon’t,haveotherprocessingelements

• Hencethingscanoccurinparallel,e.g.Proc(A) OS Proc(B) Proc(C)

Proc(A)

OS Proc(B) OS OS

time

CPU0

CPU1 Proc(A)OS Proc(D)Proc(C) OS

• NoticethattheOSrunsonbothCPUs:tricky!• MoregenerallycanhavedifferentthreadsofthesameprocessexecutingondifferentCPUstoo

11

OS

Whatdoesthiscodedo?

12

void main(void) {threadid_t threads[NUMTHREADS]; // Thread IDsint i; // Counter

for (i = 0; i < NUMTHREADS; i++)threads[i] = thread_create(threadfn, i);

for (i = 0; i < NUMTHREADS; i++)thread_join(threads[i]);

}

void threadfn(int threadnum) {sleep(rand(2));printf(“%s %d\n”, threadstr, threadnum);

}

Whatorderscouldtheprintfs runin?

#define NUMTHREADS 4char *threadstr = “Thread”;

Globalvariablesaresharedbyallthreads

main() iscalledonceatstartup

Eachthreadhasitsownlocalvariables

Additionalthreadsarestartedexplicitly

10/7/16

7

Possibleorderingsofthisprogram

• Whatordercouldtheprintf()soccurin?• Twosourcesofnon-determinisminexample:– Programnon-determinism:Programrandomlysleeps0or1secondsbeforeprinting

– Threadschedulingnon-determinism:Arbitraryorderforunprioritised,concurrentwakeups,preemptions

• Thereare4!(factorial)validpermutations– Assumingprintf()isindivisible– Isprintf()indivisible?Maybe.

• Evenmorepotentialtimings ofprintf()s

13

1:N- user-levelthreading• Kernel onlyknowsabout(and

schedules)processes• Userspace implementsthreads,

contextswitching,scheduling– E.g.,intheJVMorathreadinglibrary

• Advantages– Lightweightcreation/terminationand

contextswitch;application-specificscheduling;OSindependence

• Disadvantages– Awkwardtohandleblockingsystem

callsorpagefaults,preemption;cannotusemultipleCPUs

• Veryearly1990s!

14

Kernel

P1 P2

CPU1 CPU2

P1T1 T2

T3

10/7/16

8

1:1- kernel-levelthreading• Kernel providesthreadsdirectly

– Bydefault,aprocesshasonethread…– …butcancreatemoreviasystemcalls

• Kernelimplementsthreads,threadcontextswitching,scheduling,etc.

• Userspace threadlibrary1:1mapsuserthreadsintokernelthreads

• Advantages:– Handlespreemption,blockingsystemcalls– StraightforwardtoutilizemultipleCPUs

• Disadvantages:– Higheroverhead(traptokernel);less

flexible;lessportable• Threadingmodelofchoiceacrossall

majorOSestoday– Windows,Linux,MacOSX,FreeBSD,

Solaris,…

15

Kernel

P1

CPU1 CPU2

P1 T1T2

T3

Kernel

P1

CPU1 CPU2

P1T1

T2T3

T2

M:N- hybridthreading• Bestofbothworlds?

– M:Nthreads,scheduleractivations,…• Kernel exposesasmallernumber(M)of

activations – typically1:1withCPUs• Userspace schedulesalargernumber(N)of

threads ontoavailableactivations– Kernelupcallswhenathreadblocks,returning

theactivationtouserspace– Kernelupcallswhenablockedthreadwakes

up,userspace schedulesitonanactivation– Kernelcontrolsmaximumparallelismby

limitingnumberofactivations• FeatureremovedfrommostOSes– why?• UsedinVirtualMachineMonitors(VMMs)

– EachVirtualCPU(VCPU)isanactivation• M:Nreappearsinconcurrencyframeworks

– E.g.,Apple’sGrandCentralDispatch(GCD)

16

10/7/16

9

Advantagesofconcurrency

• AllowsustooverlapcomputationandI/Oonasinglemachine

• Cansimplifycodestructuringand/orimproveresponsiveness– e.g.onethreadredrawstheGUI,anotherhandlesuserinput,andanothercomputesgamelogic

– e.g.onethreadperHTTPrequest– e.g.backgroundGCthreadinJVM/CLR

• Enablestheseamless(?!)useofmultipleCPUs–greaterperformancethroughparallelprocessing

17

Concurrentsystems• Ingeneral,havesomenumberofprocesses…– …eachwithsomenumberofthreads…– …runningonsomenumberofcomputers…– …eachwithsomenumberofCPUs.

• Forthishalfofthecoursewe’llfocusonasinglecomputerrunningamulti-threadedprocess– mostproblems&solutionsgeneralizetomultipleprocesses,CPUs,andmachines,butmorecomplex

– (we’lllookatdistributedsystemsinLentterm)• Challenge:threadswillaccesssharedresourcesconcurrentlyviatheircommonaddressspace

18

10/7/16

10

Example:HousematesBuyingBeer

• Thread1(person1)1. Lookinfridge2. Ifnobeer,gobuybeer3. Putbeerinfridge

• Inmostcases,thisworksjustfine…• Butifbothpeoplelook(step1)beforeeitherrefillsthe

fridge(step3)…we’ll endupwithtoomuchbeer!• Obviouslymoreworryingif“lookinfridge”is“check

reactor”,and“buybeer”is“togglesafetysystem” ;-)

• Thread2(person2)1. Lookinfridge2. Ifnobeer,gobuybeer3. Putbeerinfridge

19

Solution#1:LeaveaNote• Thread1(person1)

1. Lookinfridge2. Ifnobeer&nonote

1. Leavenoteonfridge2. Gobuybeer3. Putbeerinfridge4. Removenote

• Thread2(person2)1. Lookinfridge2. Ifnobeer &nonote

1. Leavenoteonfridge2. Gobuybeer3. Putbeerinfridge4. Removenote

• Probablyworksforhumanbeings…• Butcomputersarestooopid!

• Canyouseetheproblem?

20

10/7/16

11

Non-Solution#1:LeaveaNote// thread 1beer = checkFridge();if(!beer) {

if(!note) {note = 1;buyBeer();note = 0;

}}

// thread 2beer = checkFridge();if(!beer) {

if(!note) {note = 1;buyBeer();note = 0;

}}

• Easiertoseewithpseudo-code…21

Non-Solution#1:LeaveaNote// thread 1beer = checkFridge();if(!beer) {

if(!note) {

note = 1;buyBeer();note = 0;

}}

// thread 2

beer = checkFridge();if(!beer) {

if(!note) {note = 1;buyBeer();note = 0;

}}

• Easiertoseewithpseudo-code…

contextswitch

contextswitch

22

10/7/16

12

Non-Solution#1:LeaveaNote

• Ofcoursethiswon’thappenallthetime– Needthreadstointerleaveinthejusttherightway(orjustthewrongway;-)

• Unfortunatelycodethatis‘mostlycorrect’ismuchworsethancodethatis‘mostlywrong’!– Difficulttocatchintesting,asoccursrarely–Mayevengoawaywhenrunningunderdebugger• e.g.onlycontextswitchesthreadswhentheyblock• (suchbugsaresometimescalledHeisenbugs)

23

CriticalSections&MutualExclusion

• Thehigh-levelproblemhereisthatwehavetwothreadstryingtosolvethesameproblem– BothexecutebuyBeer()concurrently– Ideallywantonlyonethreaddoingthatatatime

• Wecallthiscodeacriticalsection– apieceofcodewhichshouldneverbeconcurrentlyexecutedbymorethanonethread

• Ensuringthisinvolvesmutualexclusion– Ifonethreadisexecutingwithinacriticalsection,allotherthreadsareprohibitedfromenteringit

24

10/7/16

13

AchievingMutualExclusion

• Onewayistoletonlyonethreadeverexecuteaparticularcriticalsection– e.g.anominatedbeerbuyer– butthisrestrictsconcurrency

• Alternativelyour(broken)solution#1wastryingtoprovidemutualexclusionviathenote– Leavinganotemeans“I’minthecriticalsection”;– Removingthenotemeans“I’mdone”– But,aswesaw,itdidn’twork;-)

• Thiswasbecausewecouldexperienceacontextswitchbetweenreading‘note’,andsettingit

25

Non-Solution#1:LeaveaNote

26

// thread 1beer = checkFridge();if(!beer) {

if(!note) {

note = 1;buyBeer();note = 0;

}}

// thread 2

beer = checkFridge();if(!beer) {

if(!note) {note = 1;buyBeer();note = 0;

}}

contextswitch

contextswitch

Wedecidetoenterthecriticalsectionhere…

Butonlymarkthefacthere…

Theseproblemsarereferredtoasraceconditionsinwhichmultiple

threadsracewithoneanotherduringconflictingaccesstosharedresources

10/7/16

14

Atomicity• Whatwewantisforthecheckingofnoteandthe(conditional)settingofnotetohappenwithoutanyotherthreadbeinginvolved– Wedon’tcareifanotherthreadreadsitafterwe’redone;orsetsitbeforewestartourcheck

– Butoncewestartourcheck,wewanttocontinuewithoutanyinterruption

• Ifasequenceofoperations(e.g.read-and-set)occurasifoneoperation,wecallthematomic– Sinceindivisible fromthepointofviewoftheprogram

• Anatomicread-and-set operationissufficientforustoimplementacorrectbeerprogram

27

Solution#2:AtomicNote// thread 1beer = checkFridge();if(!beer) {

if(read-and-set(note)) {buyBeer();note = 0;

}}

// thread 2beer = checkFridge();if(!beer) {

if(read-and-set(note)) {buyBeer();note = 0;

}}

• read-and-set(&address)atomically checksthevalueinmemoryandiff itiszero,setsittoone– returns1iff thevaluewaschangedfrom0->1

• Thispreventsthebehaviorwesawbefore,andissufficienttoimplementacorrectprogram…– althoughthisisnotthatprogram:-)

28

10/7/16

15

Non-Solution#2:AtomicNote// thread 1beer = checkFridge();if(!beer) {

if(read-and-set(note)) {buyBeer();note = 0;

}}

// thread 2

beer = checkFridge();if(!beer) {

if(read-and-set(note)) {buyBeer();note = 0;

}}

• Ourcriticalsectiondoesn’tcoverenough!

contextswitch

contextswitch

29

Generalmutualexclusion

• Wewouldliketheabilitytodefinearegionofcodeasacriticalsectione.g.

// thread 1ENTER_CS();beer = checkFridge();if(!beer)

buyBeer();LEAVE_CS();

// thread 2ENTER_CS();beer = checkFridge();if(!beer)

buyBeer();LEAVE_CS();

• Thisshouldwork…• …providingthatourimplementationofENTER_CS()/LEAVE_CS()iscorrect

30

10/7/16

16

Implementingmutualexclusion• Oneoptionistopreventcontextswitches– e.g.disableinterrupts(forkernelthreads),orsetanin-memoryflag(foruserthreads)

• ENTER_CS()=“disablecontextswitches”;LEAVE_CS()=“re-enablecontextswitches”

• Canworkbut:– Ratherbruteforce(stopsallotherthreads,notjustthosewhowanttoenterthecriticalsection)

– Potentiallyunsafe(ifdisableinterruptsandthensleepwaitingforatimerinterrupt;-)

– Anddoesn’tworkacrossmultipleCPUs

31

Implementingmutualexclusion

• Associateamutualexclusionlockwitheachcriticalsection,e.g.avariableL– (mustensureusecorrectlockvariable!)ENTER_CS()=“LOCK(L)”LEAVE_CS()=“UNLOCK(L)”

• CanimplementLOCK()usingread-and-set():

LOCK(L) { while(!read-and-set(L))

; // do nothing}

UNLOCK(L) { L = 0;

}

32

10/7/16

17

Solution#3:mutualexclusionlocks// thread 1LOCK(fridgeLock); beer = checkFridge();if(!beer)

buyBeer();UNLOCK(fridgeLock);

// thread 2LOCK(fridgeLock); beer = checkFridge();if(!beer)

buyBeer();UNLOCK(fridgeLock);

• Thisis– finally!– acorrectprogram• Stillnotperfect

– Lockmightbeheldforquitealongtime(e.g.imagineanotherpersonwantingtogetthemilk!)

– WaitingthreadswasteCPUtime(orworse)– Contention occurswhenconsumershavetowaitforlocks

• Mutualexclusionlocksoftenknownasmutexes– Butwewillpreferthistermforsleepable locks– seeLecture2– Sothinkoftheaboveasaspinlock

33

Summary+nexttime• Definitionofaconcurrentsystem• Originsofconcurrencywithinacomputer• Processesandthreads• Challenge:concurrentaccesstosharedresources• Criticalsections,mutualexclusion,raceconditions,andatomicity• Mutualexclusionlocks(mutexes)

• Nexttime:– Moreonmutualexclusion– Hardwaresupportformutualexclusion– Semaphoresformutualexclusion,processsynchronisation,and

resourceallocation– Producer-consumerrelationships.

34