2017 SPTCC - Lock-free algorithms for Kotlin...
Transcript of 2017 SPTCC - Lock-free algorithms for Kotlin...
Lock-freealgorithmsforKotlincoroutines
ItisallaboutscalabilityPresentedatSPTCC2017
/RomanElizarov@JetBrains
Speaker:RomanElizarov
• 16+yearsexperience• Previouslydevelopedhigh-perftradingsoftware@Devexperts• Teachconcurrent&[email protected]• Chiefjudge@NortheasternEuropeanRegionofACMICPC• NowworkonKotlin@JetBrains
Agenda
• Kotlincoroutinesoverview&motivationforlock-freealgorithms• Lock-freedoublylinkedlist• Lock-freemulti-wordcompare-and-swap• Combiningthemtogetmorecomplexatomicoperations (withoutSTM)
Kotlinbasicfacts
• KotlinisaJVMlanguagedevelopedbyJetBrains• Generalpurposeandstatically-typed• Object-orientedandfunctionalparadigms• OpensourceunderApache2.0• Reachedversion1.0in2016• Compatibilitycommitment• Nowatversion1.1
• OfficiallysupportedbyGoogleonAndroid
BlockingthreadsKotlin fun postItem(item: Item) {
val token = requestToken()val post = submitPost(token, item)processPost(post)
}
fun postItem(item: Item) {requestToken { token ->
submitPost(token, item) { post -> processPost(post)
}}
}
CallbacksKotlin
fun postItem(item: Item) {requestToken()
.thenCompose { token ->submitPost(token, item)
}.thenAccept { post ->
processPost(post) }
}
Futures/Promises/RxKotlin
CoroutinesKotlin fun postItem(item: Item) {
launch(CommonPool) {val token = requestToken()val post = submitPost(token, item)processPost(post)
}}
CSP&Actormodels
• Astyle ofprogrammingformodernsystems• Lotsofconcurrenttasks/jobs• Waitingmostofthetime• Communicatingallthetime
Sharedatabycommunicating
Kotlincoroutinesprimitives
• Jobs/Deferreds (futures)• join/await
• Channels• send&receive• synchronous&bufferedchannels
• Select/alternatives• Atomicallywaitonmultipleevents
• Cancellation• Parent-childhierarchies
Implementationchallenges
• Coroutinesarelikelight-weightthreads• Allthelow-level scheduling&communicationmechanismshavetoscale tolotsofcoroutines
Buildingblocks
• Single-wordCAS(that’sallwehaveonJVM)• Automaticmemorymanagement(GC)• Practical lock-freealgorithms• Lock-FreeandPracticalDoublyLinkedList-BasedDeques UsingSingle-WordCompare-and-SwapbySundell andTsigas• APracticalMulti-WordCompare-and-SwapOperationbyTimothyL.Harris,KeirFraserandIanA.Pratt.
Doublylinkedlist
SN
1N
PS
PH T
sentinel sentinel
Usesamenodeinpractice
next linksformlogicallistcontentsprev linksareauxiliary
Doublylinkedlist(remove1)
SN
1N
PS
PH T
Markremovednode’snext link
Usewrapper objectformarkinpractice
Cachewrappersinpointed-tonodes
CAS
RetryremoveonCASfailure
1
2
Don’tuseAtomicMarkableReference
CAS
Doublylinkedlist(remove2)
SN
1N
PS
PH T
Markremovednode’sprev link
RetrymarkingonCASfailure
”finishremove”
Nodestates
Initnext:Okprev:Ok
prev.next:--next.prev:--
Insert1next:Okprev:Ok
prev.next:menext.prev:--
Insert2next:Okprev:Ok
prev.next:menext.prev:me
Remove1next:Remprev:Ok
prev.next:menext.prev:me
Remove2next:Remprev:Rem
prev.next:menext.prev:me
Remove3next:Remprev:Rem
prev.next:++next.prev:me
Remove4next:Remprev:Rem
prev.next:++next.prev:++
helpremove
correctprev
correctprev
1 2 3
4 5 6 7
Concurrentremove&insert(3)
SN
1N
PS
PH T
2N
P
detectwrongprev(t.prev.next -- removed)do“correctprev”R1
R1
I2
Takeaways
• Akindofalgo youneedapaperfor• Hardtoimprovew/owritinganotherpaper• Goodnews:stresstestsuncovermostimpl bugs• Badnews:whenstresstestfails,youuptolonghours• Morebadnews:hardtofindbugsthatviolatelock-freedomness ofalgorithm
Summary:whatwecando
• Insertitems(attheendofthequeue)• Removeitems(atthefrontofthequeue)• Traversethelist• Removeitemsatarbitrarylocations• InO(1)
Linearizability
• Insertlast• LinearizesatCASofnext
• Removefirst/arbitrary• Success– atCASofnext• Fail– atreadofhead.next
Moreaboutalgorithm
• Sundell &Tsigas algo supportsdeque operations• CanPushLeft &PopRight
• PopLeft issimple– readhead.next &remove• Butcannotlinearizethemallatcas points• PushLeft,PushRight,PopRight - Ok• PopLeft linearizesathead.next read(!!!)
Summaryofimpl notes
• UseGC(dropallmemorymanagementdetails)• Mergehead&tailintoasinglesentinelnode• Emptylistisjustoneobject(prev &nextontoitself)• Oneitem+=oneobject
• Reuse“removemark”objects• One-elementlistsreuseofptrs tosentinelallthetime
• Encapsulate!
SN
P1
N
PQ
Practicaluse-case:synchronouschannels
val channel = Channel<Int>()
// coroutine #1for (x in 1..5) {
channel.send(x * x) }
// coroutine #2repeat(5) {
println(channel.receive()) }
1
2
3
Senderswait
Sender#1H Sender#2 Sender#3 T
Moresenders
Incomingreceivers
Receiverremovesfirstifitisasendernode
Senderinsertslastifitisnotareceivernode
Receiverswait
Receiver#1H Receiver#2 Receiver#3 T
Morereceivers
Incomingsenders
Senderremovesfirstifitisareceivernode
Receiverinsertslastifitisnotasendernode
Sendfunctionsketch
fun send(element: T) {while (true) {
// try to add sender, unless prev is receiverif (enqueueSend(element)) break// try to remove first receiverval receiver = removeFirstReceiver()if (receiver != null) {
receiver.resume(element) // resume receiver break
}}
}
1
2
3
4
Channeluse-caserecap
• Usesinsert/removeopsconditionalontail/headnode• Canabort(cancel)waittoreceive/sendatanytimebyusingremove• Fullremoval-- nogarbageisleft
• Prettyefficientinpractice• Oneitemlists– one“garbage”object
Use-case:selectexpression
val channel1 = Channel<Int>()val channel2 = Channel<Int>()
select {channel1.onReceive { e -> ... }channel2.onReceive { e -> ... }
}
Impl summary:register(2)
Selectstatus:NS
Channel1Queue
Channel2Queue
Addnodetochannel1queueifnotselected(NS)yet
N1
Impl summary:register(3)
Selectstatus:NS
Channel1Queue
Channel2Queue
Addnodetochannel2queueifnotselected(NS)yet
N1 N2
Impl summary:select(resume)
Selectstatus:S
Channel1Queue
Channel2Queue
N1
Makeselectedand removenodefromqueue
Impl summary:cleanuprest
Selectstatus:S
Channel1Queue
Channel2Queue
Removenon-selectedwaitersfromqueue
DCSSspecinpseudo-code
A B
fun <A,B> dcss(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B) =
atomic { if (a.value == expectA && b.value == expectB) {
a.value = updateA}
}
1
2
34
DCSS:prepare
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A B
CASptr todescriptorifa.value == expectA
expectA expectB
updateA
DCSS:readb.value
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A B
CASptr todescriptorifa.value == expectA
expectA expectB
updateA
DCSS:complete(whensuccess)
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateACAStoupdatedvalueifastillpointstodescriptor
DCSS:complete(whenfail)
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA !expectB
updateACAStooriginalvalueifastillpointstodescriptor
DCSS:States
InitA:???
(desc created)
A:descAwasexpectA
prepok
A:???Awas!expectA
prepfail
onetread
1 2
A:updateABwasexpectB
success
A:expectABwas!expectB
4 5
fail
Anyotherthreadencounteringdescriptorhelpscomplete
Originatorcannotlearnwhatwastheoutcome
Lock-freealgorithmwithoutloops!
3
Caveats
• A&Blocationsmustbetotallyordered• orriskstack-overflowwhilehelping
• Onewaytolookatit:RestrictedDCSS(RDCSS)
DCSSMod:learnoutcome
A B
fun <A,B> dcssMod(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B): Boolean =
atomic { if (a.value == expectA && b.value == expectB) {
a.value = updateAtrue
} else false
}
DCSSMod:init descriptor
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
Consensus
DCSSMod:prepare
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
DCSSMod:readb.value
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
DCSSMod:reachconsensus
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateA
Outcome:SUCCESS
CAS(UNDECIDED,DECISION)
DCSSMod:complete
DCSSDescriptor(a,expectA,updateA,
b,expectB)
A BexpectA expectB
updateA
Outcome:SUCCESS
DCSSMod:States
InitA:???
Outcome:UND(desc created)
A:descOutcome:UNDAwasexpectA
prepok
A:???Outcome:FAILAwas!expectA
prepfail
onetread
1 2A:desc
Outcome:SUCCBwasexpectB
success
A:descOutcome:FAIL
6
fail
A:expectA
A:updateA5
7
Stillnoloops!
3
4
CASNspecinpseudo-code
A B
fun <A,B> cas2(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B, updateB: B): Boolean =
atomic { if (a.value == expectA && b.value == expectB) {
a.value = updateA b.value = updateB
true} else
false}
1
2
3
4
5
Fortwowords,forsimplicity
CASN:init descriptor
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
updateB
CASN:prepare(1)
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
updateBCAS
CASN:prepare(2)
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:UNDECIDED
updateB
UseDCSStoupdateBifOutcome==UNDECIDED
DCSS
CASN:decide
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:SUCCESS
updateB
CASoutcome
CASN:complete(1)
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:SUCCESS
updateBCAS
CASN:complete(2)
DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)
A BexpectA expectB
updateA
Outcome:SUCCESS
updateBCAS
CASN:States
A:???B:???O:UND
A:descB:???O:UND
A:descB:descO:UND
A:updateAB:descO:SUCC
InitA:updateAB:updateBO:SUCC
1 2 3
5
A:descB:descO:SUCC
4
6A:???B:???O:FAIL
A!=expectA
A:descB:???O:FAIL
B !=expectB
onetread
A:expectAB:???O:FAIL
7 8
9
DCSS
PreventsfromgoingbackinthisSM
descriptorisknowntoother(helping)threads
ItiseasytocombinemultipleoperationswithDCSS/CASNthatlinearizeonaCASwithadescriptorparametersthatare
knowninadvance
OperationDescriptor
Aref:???expectA:SentinelupdateA:Node#2
…Outcome:UNDECIDED
Doublylinkedlist:insert(0)
SN
1N
PS
PH T
2N
P
CAShere
WeknowexpectedvalueforCASinadvanceWeknowupdatedvalueforCASinadvance
???canfillinAbeforeCAS&updateonretry
DCSShereisneeded(always!)
Doublylinkedlist:insert(1)
SN
1N
PS
PH T
2N
P
OperationDescriptor
Aref:???expectA:SentinelupdateA:Node#2
…Outcome:UNDECIDED
DCSSDescriptor
affectednode:#1operationref
Doublylinkedlist:insert(2)
SN
1N
PS
PH T
2N
P
OperationDescriptor DCSSDescriptor
Helpersareaboundtostumbleuponthesamedescriptor
CAScanonlysucceedonlastnode
Competinginsertswillcomplete(help)usfirst
affectednode:#1operationref
Aref:???expectA:SentinelupdateA:Node#2
…Outcome:UNDECIDED
Doublylinkedlist:insert(3)
SN
1N
PS
PH T
2N
P
OperationDescriptor
desc isupdatedafter successfulDCSS
Aref:Node#1expectA:SentinelupdateA:Node#2
…Outcome:UNDECIDED
DCSSDescriptor
affectednode:#1operationref
Doublylinkedlist:insert(4)
SN
1N
PS
PH T
2N
P
OperationDescriptorStayspointeduntiloperationoutcomeisdecided
Aref:Node#1expectA:SentinelupdateA:Node#2
…Outcome:UNDECIDED
Doublylinkedlist:remove(0)
SN
1N
PS
PH T
R1
CAShere
2N
P
OperationDescriptor
Aref:???expectA:???
updateA:Rem[???]…
Outcome:UNDECIDED
BothnotknowninadvanceDeterministicf(expectA)
Doublylinkedlist:remove(1)
SN
1N
PS
PH T
R1
2N
P
OperationDescriptor
Aref:???expectA:???
updateA:Rem[???]…
Outcome:UNDECIDED
DCSSDescriptor
affectednode:#1
operationrefoldvalue:#2
Doublylinkedlist:remove(2)
SN
1N
PS
PH T
R1
2N
P
OperationDescriptor
Aref:???expectA:???
updateA:Rem[???]…
Outcome:UNDECIDED
ItlockswhatnodewearetoremoveCannotchangew/oremovalof#1Wedon’tsupportPushLeft!!!
DCSSDescriptor
affectednode:#1
operationrefoldvalue:#2
Doublylinkedlist:remove(3)
SN
1N
PS
PH T
R1
2N
P
OperationDescriptor
Aref:Node#1expectA:Node#2updateA:Rem[#2]
…Outcome:UNDECIDED
desc isupdatedafter successfulDCSS
DCSSDescriptor
affectednode:#1
operationrefoldvalue:#2
Doublylinkedlist:remove(4)
SN
1N
PS
PH T
R1
2N
P
OperationDescriptor
Aref:Node#1expectA:Node#2updateA:Rem[#2]
…Outcome:UNDECIDED
Stayspointeduntiloperationoutcomeisdecided
Closingnotes
• AllwecareaboutisCAS thatlinearizesoperation• Subsequentupdatesarehelpermoves• Invokeregularhelp/correctfunctions
• PerfectalgorithmtocombinewithoptionalHardwareTransactionalMemory(HTM)
References
• Kotlinlanguage• http://kotlinlang.org
• Kotlincoroutinessupportlibrary• http://github.com/kotlin/kotlinx.coroutines