Orchestrating Tasks from the Cloud with Groovy and AWS SWF
-
Upload
spring-io -
Category
Technology
-
view
719 -
download
1
description
Transcript of Orchestrating Tasks from the Cloud with Groovy and AWS SWF
![Page 1: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/1.jpg)
ORCHESTRATING TASKSFROM THE CLOUD
WITH GROOVY AND AWS SWFCreated by ClayMcCoy
![Page 2: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/2.jpg)
ABOUT MESr. Software Engineer @ NetflixEngineering Tools TeamContributes to Asgardwww.claymccoy.com@ClayMcCoy
![Page 3: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/3.jpg)
ASGARDWeb app written in GrailsUsed for deployment to AWSOpen sourced on Github
![Page 4: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/4.jpg)
ASGARD NEEDED A WORKFLOW SERVICETask coordinationLong lived tasksDistributed tasks
![Page 5: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/5.jpg)
AMAZON SWF"Amazon Simple Workflow (Amazon SWF) is a task
coordination and state management service for cloudapplications. With Amazon SWF, you can stop writing
complex glue-code and state machinery and invest more in thebusiness logic that makes your applications unique."
![Page 6: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/6.jpg)
AMAZON SWF CONCEPTSdomainstasklistsdecision tasksactivity taskslong pollingtimeoutsheartbeatsworkflow execution
![Page 7: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/7.jpg)
SHOW AWS API
![Page 8: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/8.jpg)
AMAZON FLOW FRAMEWORKhigher level APIremoves boilerplatedecider logic looks like typical JavapromisespollersAspectJ and generated code
![Page 9: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/9.jpg)
SHOW FLOW EXAMPLE
![Page 10: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/10.jpg)
GROOVY IS GOOD AT HIDING BOILERPLATECODE
no AspectJ or generated codestill uses core Flow objectstypesafe calls to schedule activities and start workflows
![Page 11: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/11.jpg)
Abstract away worflow concernsinterface Workflow<A> { Promise<Void> status(String message)
AndPromise allPromises(Promise... promises) OrPromise anyPromises(Promise... promises)
A getActivities()
<T> Promise<T> waitFor(Promise promise, Closure<Promise<T>> work) <T> Promise<T> promiseFor(T value)
<T> DoTry<T> doTry(Closure<Promise<T>> work) <T> Promise<T> retry(RetryPolicy retryPolicy, Closure<Promise<T>> work)
Promise<Void> timer(long delaySeconds)}
![Page 12: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/12.jpg)
SHOW SWF AND LOCAL IMPLEMENTATIONS
![Page 13: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/13.jpg)
NOW FOR AN EXAMPLE
![Page 14: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/14.jpg)
![Page 15: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/15.jpg)
DECIDER INTERFACE@Workflow@WorkflowRegistrationOptions( defaultExecutionStartToCloseTimeoutSeconds = 60L)interface BayAreaTripWorkflow {
@Execute(version = '1.0') void start(String name, Collection<BayAreaLocation> previouslyVisited)
@GetState List<String> getLogHistory()}
![Page 16: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/16.jpg)
ACTIVITIES INTERFACE@Activities(version = "1.0")@ActivityRegistrationOptions(defaultTaskScheduleToStartTimeoutSeconds = -1L, defaultTaskStartToCloseTimeoutSeconds = 300L)interface BayAreaTripActivities {
![Page 17: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/17.jpg)
MORE ACTIVITIES INTERFACEinterface BayAreaTripActivities {
@ActivityRegistrationOptions(defaultTaskScheduleToStartTimeoutSeconds = -1L, defaultTaskStartToCloseTimeoutSeconds = 86400L) boolean askYesNoQuestion(String question)
String goTo(String name, BayAreaLocation location)
@ActivityRegistrationOptions(defaultTaskScheduleToStartTimeoutSeconds = -1L, defaultTaskStartToCloseTimeoutSeconds = 300L, defaultTaskHeartbeatTimeoutSeconds = String hike(String somewhere)
String enjoy(String something)
String win(String game)}
![Page 18: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/18.jpg)
![Page 19: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/19.jpg)
![Page 20: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/20.jpg)
DECIDER IMPLEMENTATIONPromise<BayAreaLocation> destinationPromise = doTry {if (!previouslyVisited.contains(BayAreaLocation.GoldenGateBridge)) { return promiseFor(BayAreaLocation.GoldenGateBridge)}if (!previouslyVisited.contains(BayAreaLocation.Redwoods)) { return promiseFor(BayAreaLocation.Redwoods)} else { waitFor(activities.askYesNoQuestion('Do you like roller coasters?')) { boolean isThrillSeeker -> if (isThrillSeeker) { return promiseFor(BayAreaLocation.Boardwalk) } promiseFor(BayAreaLocation.Monterey) }}}.result
![Page 21: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/21.jpg)
ACTIVITIES IMPLEMENTATION@ManualActivityCompletionboolean askYesNoQuestion(String question) { sendManualActivityCompletionInfo(activity.taskToken, activity.workflowExecution) true // does not matter what is returned here because // the result will be supplied manually}
![Page 22: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/22.jpg)
![Page 23: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/23.jpg)
![Page 24: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/24.jpg)
DECIDER IMPLEMENTATIONwaitFor(activities.goTo(name, destination)) { status it waitFor(activities.hike('across the bridge')) { status it }}
![Page 25: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/25.jpg)
ACTIVITIES IMPLEMENTATIONString goTo(String name, BayAreaLocation location) { "${name} went to ${location}."}
String hike(String somewhere) { int stepsTaken = 0 int totalStepsForHike = hikeNameToLengthInSteps[somewhere] ?: 100 while (stepsTaken < totalStepsForHike) { activity.recordHeartbeat("Took ${++stepsTaken} steps.") } "And hiked ${somewhere}."}
![Page 26: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/26.jpg)
![Page 27: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/27.jpg)
![Page 28: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/28.jpg)
DECIDER IMPLEMENTATIONwaitFor(activities.goTo(name, destination)) {status itstatus 'And stretched for 10 seconds before hiking.'waitFor(timer(10)) { DoTry<String> hiking = doTry { promiseFor(activities.hike('through redwoods')) } DoTry<Void> countDown = cancellableTimer(30) waitFor(anyPromises(countDown.getResult(), hiking.getResult())) { if (hiking.getResult().isReady()) { countDown.cancel(null) status "${hiking.getResult().get()}" } else { hiking.cancel(null) status 'And ran out of time when hiking.' } }}
![Page 29: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/29.jpg)
![Page 30: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/30.jpg)
![Page 31: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/31.jpg)
DECIDER IMPLEMENTATIONint numberOfTokensGiven = 3int numberOfTokens = numberOfTokensGivenRetryPolicy retryPolicy = new ExponentialRetryPolicy(60). withMaximumAttempts(numberOfTokens).withExceptionsToRetry([IllegalStateException])DoTry<String> tryToWin = doTry { retry(retryPolicy) { if (numberOfTokens <= 0) { null } numberOfTokens-- return promiseFor(activities.win('a carnival game')) }} withCatch { Throwable e -> status "${e.message} ${numberOfTokensGiven} times."}waitFor(tryToWin.result) { status it if (numberOfTokens > 0) { waitFor(activities.enjoy('a roller coaster')) { status it } } Promise.Void()}
![Page 32: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/32.jpg)
ACTIVITIES IMPLEMENTATIONString enjoy(String something) { "And enjoyed ${something}."}
String win(String game) { if (isWinner()) { return "And won ${game}." } else { throw new IllegalStateException("And lost ${game}.") }}
![Page 33: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/33.jpg)
![Page 34: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/34.jpg)
![Page 35: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/35.jpg)
DECIDER IMPLEMENTATIONPromise<String> eating = promiseFor(activities.enjoy('eating seafood'))Promise<String> watching = promiseFor(activities.enjoy('watching sea lions'))waitFor(allPromises(eating, watching)) { status "${eating.get()} ${watching.get()}" doTry { promiseFor(activities.enjoy('looking for sea glass on the beach')) } withCatch { Throwable t -> status t.message promiseFor(activities.enjoy('the aquarium')) } withFinally { String result -> status result waitFor(activities.enjoy('the 17-Mile Drive')) { status it } } Promise.Void()}
![Page 36: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/36.jpg)
ACTIVITIES IMPLEMENTATIONString enjoy(String something) { "And enjoyed ${something}."}
String win(String game) { if (isWinner()) { return "And won ${game}." } else { throw new IllegalStateException("And lost ${game}.") }}
![Page 37: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/37.jpg)
SHOW UNIT TESTS
![Page 38: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/38.jpg)
SHOW WORFLOW EXECUTION HISTORY
![Page 39: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/39.jpg)
SHOW ASGARD'S USE OF SWF
![Page 40: Orchestrating Tasks from the Cloud with Groovy and AWS SWF](https://reader033.fdocuments.us/reader033/viewer/2022052903/5579ace7d8b42ac1148b4edf/html5/thumbnails/40.jpg)
QUESTIONS