Building Open Source Communities @ AWS Serverless · • Obsessed about serverless developer tools...

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Sanath Kumar Ramesh

Senior Software Engineer, AWS Serverless

Building Open Source

Communities

@ AWS Serverless

☁️ ️ @sanathkr_ @sanathkr


About Me

• Obsessed about serverless developer tools since 2016

• Dev Lead for open source serverless developer tools: SAM and SAM CLI

• Open source enthusiast

• Previously: • Full stack developer at startups

• Windows Kernel Engineer at Microsoft

• Designed multicore processors at grad school

Sanath Kumar Ramesh,

Software Engineer, AWS Serverless

@sanathkr_

@sanathkr


Happy 20th Birthday Open Source!


Open Source @ AWS

Grow

Communities Increase

Contributions Improve

Code

aws.github.io


Motivation for Open Source

We are motivated to collaborate by many of our partners and customers

Open source projects help us to innovate

Scaling open source projects as AWS services meets customer needs

Open data helps customers build innovative services


Open Source Code

Contributions


gvsbuild

cbmc

go lang

apache joshua

apache mxnet chro

miu

m

wing

air

flow

mqtt

lantram

tupl openfire moses

s3_file

ela

sticsearc

h

xen

nett

y

beansta

lker

react date

range p

icker

pig

re

dis

linux k

ern

el

apache hive

hue

docker

apache bigtop

libgxps apache s

park

apache oozie

libsoup

joshua decoder

presto

apache hbase exoplayer

gpyopt

glib

go-d

ockerc

lient

apache phoenix

boto

flow

clo

ud-i

nit

rdoc

glib-networking

grp

c-g

o

chromium vim

red h

at

rails

scikit-fuzzy

fast align

ca

lifo

rniu

m

logsta

sh

-input-

udp

pygresql

eclipse paho

apache zeppelin

qemu

co

nta

ine

rd

cnn

catwatch

ios-webkit-debug-proxy

celery dynamodb backend

rosette

ya

rn

boto 3

aw

s iot devk

it

hana

mariadb-connector-j

appiu

madb

go

ogle

gua

va

elastalert

jackson d

ata

bin

d

esp-open-rtos

kenlm

dove

cot

hapro

xy

goa

tsung

mshadow

um

l-utilities

appiu

m-c

hro

medri

ver

github-p

lugin

pyzmq

statsd backend

lom

bo

k

ca

pyba

ra

apache t

om

cat

gerrit-check

mysql workbench

dynamic-object

angula

r to

oltip

s

apache commons lang

appium-gulp-plugin

ruby-simple-queue

go-s

tats

d-c

lient

ldns

sentry

facebook w

ebdri

ver

agent

little

pro

xy

bixie

troposphere

git-b

igsto

re

fop

dja

ngo

-debug-t

oolb

ar

bandit

elephas

akka

grpc

parq

uet-

mr

rspec-retry

glisten

wordpress

file

queue

oci-fetch

zipper

pkg errors

jcom

mander

aws-scala

cm

is_5

java

dpdk

rebouncer

parquet-python

libfuse appiu

m r

em

ote

debugger

leve

ldb

mvel

we

bdri

ve

rage

nt

yield-java

immutables.org

cocoapods

assertj-core

appiu

m-d

ot-

exe

safe

-eva

l

xcpretty

ews java api

str

eet

addre

ss

wyche

pro

of

mqtt js

appiu

m

fabri

c

scapy

react-

hot-

loader

sys-proctable

httpcom

ponents

httpclient

apache p

ig

sync

word segmentation

treeextra

ultipart-parser

tez

wired tiger

cmock

go-winio

postgresql

apache commons dbutils

gradle

dtc

ore

text

libarc

hiv

e

p5-p

roto

col-

http2

react-native

apache ant

open fire

appium-android-bootstrap

iputils

gulp-typescript

dynamodb-milkhatx

kin

esalite

yubico

apache commons io

presto hadoop apache 2

ansible

nginx

2016


gvsbuild

cbmc

go l

ang

apache joshua

apache mxnet

wing

air

flow

mqtt

lantram

tupl

moses

s3_file

ela

sticsearc

h

xen

nett

y

beansta

lker

react date

range p

icker

pig

re

dis

lin

ux k

ern

el

apache hive

hue

docker

apache bigtop

libgxps

apache s

park

apache oozie

libsoup

joshua decoder presto

apache hbase

exoplayer

gpyo

pt

glib

go-d

ockerc

lient

apache phoenix

boto

flow

clo

ud-i

nit

rdoc

glib-networking

grp

c-g

o

chromium vim

red h

at

rails

scikit-fuzzy

fast align

logsta

sh

-input-

udp

pygresql

eclipse paho

apache zeppelin

qemu

co

nta

ine

rd

cnn

catwatch



rosette

yarn boto 3

aw

s iot devk

it

hana

mariadb-connector-j

appiu

madb

go

ogle

gua

va

elastalert

jackson

data

bin

d

esp-open-rtos

kenlm

dove

cot

hapro

xy

goa

tsung

mshadow

um

l-utilities

appiu

m-c

hro

medri

ver

github

-plu

gin

pyzmq

statsd backend

lom

bok

ca

pyba

ra

apache t

om

cat

gerrit-check

mysql workbench

dynamic-object

angula

r to

oltip

s

apache commons lang

appium-gulp-plugin

ruby-simple-queue

go-s

tats

d-c

lient

ldns

sentry

facebook w

ebdri

ver

agent

little

pro

xy

bixie

troposphere

git-b

igsto

re

fop

dja

ng

o-d

ebug-t

oolb

ar

bandit

ele

phas

akka

grpc

parq

uet-

mr

rspec-retry

glisten

word

pre

ss

file

queue

oci-fetch

zipper

pkg e

rrors

jcom

mander

aws-scala

cm

is_5

java

dpdk

rebouncer

parquet-python

libfuse

appiu

m r

em

ote

debugger

leve

ldb

mvel w

ebdri

ve

rage

nt

yield-java

immutables.org

cocoapods

assertj-core

appiu

m-d

ot-

exe

safe

-eva

l

xcpretty

ews java api

str

eet

addre

ss

wyche

pro

of

mqtt js

appiu

m

fabri

c

scapy

react-

hot-

loader

sys-proctable

httpcom

ponents

httpclient

apache p

ig

sync

word segmentation

treeextra

ultipart-parser

tez

wired tiger

cmock

go-winio


gradle

dtc

ore

text

libarc

hiv

e

p5-protocol-http2

react-native

apache a

nt

open fire


iputils

gulp-typescript

dynamodb-milkhatx

kin

esalite

yubico

apache commons io


ansible

nginx

chro

miu

m

openfire

nodejs

linux (arch) ja

nus g

rap

h

win

g

wingtk

open mpi

repolinter

deep visualization toolbox

kubernetes

flin

k

tacacs p

lus

glib

-openssl

sparklingml

openjd

k

tercom

react-draft-wysiwyg

incubator mxnet

mxnet notebooks

libm

oon

gpy

moby

oss-attribution-generator

licensee

glib

seabios

om

pi

linux (

xen d

rivers

)

linux (kvm)

postgresql

tslint-eslint-rules

cni

dmlc

conta

inern

etw

ork

ing

gbm

jackson-d

ata

form

ats

-bin

ary

tinyxm

l12

peew

ee

netlink

definitely

typed

phoenix

pynam

odb

jgi

t

rgp

gpyopt

tabular

sm

ack

sqoop

firefox

postcss-extract-animations

irate

irate

astrum

gatsby

cerebro

mithriljs

keras

geckodriver

wireshark

cmake

gtk

+

tslint

behave

bro

appiu

m-u

iauto

mato

r2-s

erv

er

linux xfs

ngin

x

para

mz

pydeepgp

rust

em

run

hyp

oth

esis

-pyt

hon

linux (pm)

intellij-community

xenbits

apache (http server)

typescript-json-schema

mysql

389ds

conda-build

spring framework

yarn

git lfs

gse

ttin

gs-t

est

git-l

fs

ex_aws

web socket sharp

conda

liblo

uis

eslint

bencode-go

emscripten

react-

native

-sw

ipeout

kappa

apache h

adoop

yarn

macports

pars

imonio

us

coreos

ublockorigin

vowpal wabbit

airlift

wycheproof

s2n

linux (blkfront)

kotlin

homebrew

linux (fbdev)

pyaad

libkv

cura

tor

libc

allure

ts-test

slight.alexa

lombok

linux-pci

imm

uta

ble

assig

n

apache lucene

react-int

dyn

am

ic-o

bje

ct

jscolor

chart

.js

rigor

linux

(nvm

e)

s3fs

apache beam

aalto-x

ml

deniswernert/udev

koji

ntpsec

lmdbja

va

apache hadoop

react-spinkit redux b

in

webhooks

corenlp

papaparse

counterfeiter

go-b

trfs

jruby

cairo

sequel pidgin

sockeye

rollbar

typescript-fsa

alks-cli

gucum

be

r

libjson

autovpn

json11

moto

r509-ocsp-responder

bam

bam

boogie

man pkcs7pad

flask-sqlalchemy

ca

lifo

rniu

m

multi-module-mave

tensorb

oard

fort

une s

erv

er

linux-n

vme-c

li

security

monk

ey

functional-streams-for-scala

spark-redshift

pla

ntu

ml-syn

tax

multi-module-maven-release-plugin

2017


gvsbuild

cbmc

go l

ang

apache joshua

apache mxnet

wing

air

flow

mqtt

lantram

tupl

moses

s3_file

ela

sticsearc

h

xen

netty

beansta

lker

react date

range p

icker

pig

re

dis

linu

x k

ern

el

apache hive

hue

docker

apache bigtop

libgxps

ap

ach

e s

pa

rk

apache oozie

libsoup

joshua decoder presto

apache hbase

exoplayer

gpyo

pt

glib

go-d

ockerc

lient

apache phoenix

boto

flow

clo

ud-i

nit

rdoc

glib-networking

grp

c-g

o

chromium vim

red h

at

rails

scikit-fuzzy

fast align

logsta

sh

-input-

udp

pygresql

eclipse paho

apache zeppelin

qemu

co

nta

ine

rd

cnn

catwatch



rosette

yarn boto 3

aw

s iot devk

it

hana

mariadb-connector-j

appiu

madb

go

ogle

gua

va

elastalert

jackson

data

bin

d

esp-open-rtos

kenlm

dove

cot

hapro

xy

goa

tsung

mshadow

um

l-utilities

appiu

m-c

hro

medri

ver

github

-plu

gin

pyzmq

statsd backend

lom

bok

ca

pyba

ra

apache t

om

cat

gerrit-check

mysql workbench

dynamic-object

angula

r to

oltip

s

apache commons lang

appium-gulp-plugin

ruby-simple-queue

go-s

tats

d-c

lient

ldns

sentry

facebook w

ebdri

ver

agent

little

pro

xy

bixie

troposphere

git-b

igsto

re

fop

dja

ng

o-d

ebug-t

oolb

ar

bandit

ele

phas

akka

grpc

parq

uet-

mr

rspec-retry

glisten

word

pre

ss

file

queue

oci-fetch

zipper

pkg e

rrors

jcom

mander

aws-scala

cm

is_5

java

dpdk

rebouncer

parquet-python

libfuse

appiu

m r

em

ote

debugger

leve

ldb

mvel w

ebdri

ve

rage

nt

yield-java

immutables.org

cocoapods

assertj-core

appiu

m-d

ot-

exe

safe

-eva

l

xcpretty

ews java api

str

eet

addre

ss

wyche

pro

of

mqtt js

appiu

m

fabri

c

scapy

react-

hot-

loader

sys-proctable

httpcom

ponents

httpclient

apache p

ig

sync

word segmentation

treeextra

ultipart-parser

tez

wired tiger

cmock

go-winio


gradle

dtc

ore

text

libarc

hiv

e

p5-protocol-http2

react-native

apache a

nt

open fire


iputils

gulp-typescript

dynamodb-milkhatx

kin

esalite

yubico

apache commons io


ansible

nginx

ch

rom

ium

openfire

nodejs

linux (arch) ja

nus g

rap

h

win

g

wingtk

open mpi

repolinter

deep visualization toolbox

kubernetes flin

k

tacacs p

lus

glib

-openssl

sparklingml

openjd

k

tercom

react-draft-wysiwyg

incubator mxnet

mxnet notebooks

libm

oon

gpy

moby

oss-attribution-generator

licensee

glib

seabios

om

pi

lin

ux

(xe

n d

rive

rs)

linux (kvm)

postgresql

tslint-eslint-rules

cn

i

dmlc

conta

inern

etw

ork

ing

gbm

jackson-d

ata

form

ats

-bin

ary

tinyxm

l12

pe

ew

ee

netlink

definitely

typed

phoenix

pyna

mo

db

jgi

t

rgp

gpyopt

tabular

sm

ack

sqoop

firefox

postcss-extract-animations

irate

irate

astrum

gatsby

cerebro

mithriljs

keras

geckodriver

wireshark

cmake

gtk

+

tslint

be

ha

ve

bro

appiu

m-u

iauto

mato

r2-s

erv

er

linux xfs

ngin

x

para

mz

pydeepgp

rust

em

run

hyp

oth

esis

-pyt

hon

linux (pm)

intellij-community

xenbits

apache (http server)

typescript-json-schema

mysql

389ds

conda-build

spring framework

yarn

git lfs

gse

ttin

gs-t

est

git-l

fs

ex_aws

web socket sharp

conda

liblo

uis

eslint

bencode-go

emscripten

react-

native

-sw

ipeout

kappa

apache h

adoop

yarn

macports

pars

imonio

us

coreos

ublockorigin

vowpal wabbit

airlift

wycheproof

s2n

linux (blkfront)

kotlin

homebrew

linux (fbdev)

pyaad

libkv

cura

tor

libc

allure

ts-test

slight.alexa

lombok

linux-pci

imm

uta

ble

assig

n

apache lucene

react-int

dyn

am

ic-o

bje

ct

jscolor

chart

.js

rigor

linux

(nvm

e)

s3fs

apache beam

aalto-x

ml

deniswernert/udev

koji

ntpsec

lmdbja

va

apache hadoop

react-spinkit redux b

in

webhooks

corenlp

papaparse

counterfeiter

go-b

trfs

jruby

cairo

sequel pidgin

sockeye

rollbar

typescript-fsa

alks-cli

gucum

be

r

libjson

autovpn

json11

moto

r509-ocsp-responder

ba

m b

am

bo

ogie

ma

n

pkcs7pad flask-sqlalchemy

ca

lifo

rniu

m

multi-module-mave

tensorb

oard

fort

une s

erv

er

linux-n

vme-c

li

security

monk

ey

functional-streams-for-scala spark-redshift

pla

ntu

ml-syn

tax

multi-module-maven-release-plugin

2018

ndbench

openjml

tinycorelinux

xen (devel)

nnvm

tvm

llvm

libuavcan

gdk-pixbuf graphql-go

cynical

authenticator

protobuf

scipy

apache activemq

tacacs_plus graphql-java

clearlydefined

cryptography

sparkmagic

packer

ub

un

tu

xen devel libfabric

jupyter notebook

go-git

arrow

jest

tele

po

rt

co

nvo

lutio

na

l-po

se

-ma

chin

es-t

ensorf

low

irqba

lance

focus-android

miniredis

jdk

spack

apa

che

incuba

tor

air

flo

w

nim

bus-jo

se

-jw

t

packr

mosquito

apache jena

rusqlite

webkit

kcov

jawk

ffmpeg

cargo-kcov

quic

kty

pe

sw

agge

r co

de

ge

n

bundler

linux-f

sde

ve

l apache httpcomponents core

recrud

netty reactive streams

txthings

go

leve

ldb s

eunje

on

jose

pysnm

p

retool

ope

n q

ua

ntu

m s

afe

am

ph

tml

liboqs

oauth2

ko

tlin

po

et

kuromoji

diaporama

ba

dssl.co

m

xcode-install

un

ico

de

cld

r

kpatch graal

foolbox

apache commons-compress

goics

linux kexec

glide jedis apache solr istio selenium

notebook

pyinvestigate

hypothesis

lsp

-in

tellij-s

pa

ce

ma

cs

boringssl

cyrus-sasl

openssl

jepsen

jetty.project


Open Source @ AWS Serverless


No servers to provision

or manage

Scales with usage

Never pay for idle Availability and fault

tolerance built in

Serverless means…


Serverless Cloud

App

Logic as Configuration (API, User Auth,

Workflows)

Static Configuration (DB, Storage, Permissions)

Business Logic as Code (Java, Python, JS etc)

Standard Cloud App

Static Configuration (DB, Storage, Permissions)

Business Logic +

API + Auth + Workflow +

Plumbing

(Java, Python, JS etc)


Open Source Serverless Development Tools

Chalice Serverless

Express Serverless Java

Container Zappa

and many more..

AWS

SAM


Open Source Tools From AWS

Chalice Serverless

Express Serverless Java

Container Zappa

and many more..

AWS

SAM



AWS Serverless

Appilcation Model

github.com/awslabs/serverless

-application-model



AWS Serverless

Appilcation Model

github.com/DataDog/dd-aws-lambda-

functions/tree/master/Log github.com/alexa/skill-sample-nodejs-fact

github.com/awslabs/serverless

-application-model



Build Web & Mobile

Apps with Serverless

Backend



Chalice Python Microservices

Framework



Serverless

Express Run NodeJs Apps on

AWS Lambda



Serverless

Java

Container

Wrapper to run Java

Spring apps on AWS

Lambda


Open Source @ AWS SAM


Meet

SAM!


From: https://github.com/awslabs/aws-serverless-samfarm/blob/master/api/saml.yaml

<-THIS

BECOMES THIS->

AWS SAM Template


AWS SAM Toolset

SAM Implementation SAM CLI

github.com/awslabs/serverless-application-model github.com/awslabs/aws-sam-cli


October 2016 – Open Specification


April 2018 – Open Source Implementation


Our Journey


Open Specification on GitHub


Is Open Sourcing possible?


SAM is a global service


SAM is integrated to other services


What if…


What if someone stole our secret sauce?


What if someone added malicious code?


What if someone wrote poor quality code?


Can we…


Can we run an open source product as

service?


Can we keep some features secret?


Do we have the right team to manage open

source?


Build Communities First,

Software Next


Open Communication


Open Development


Open Priorities


DevOps – Deployment Automation

Github

Internal

Deployment

Pipeline


DevOps – Metrics & Goals

Business Metrics Community Metrics Operational Metrics

Downloads/week Num Issues, PRs Cost of install

Downloads by Python

Version

Time to first response First action success

rate

Downloads by SAM

CLI version

Time to close PRs Latency of commands

Community vs Internal

PRs

Error rate

Age of Issues


Support Community with Best Practices

• Security reviews on every release

• Deployment pipeline, automation & safe-guards

• Automated Unit & Integration Testing

• Two “LGTM” Code Reviews

• Public design discussions


Incredible Community!


Feedback Welcome

[email protected]

github.com/awslabs/serverless-application-model

github.com/awslabs/aws-sam-cli


Thank You ️ ️ Questions?

@sanathkr_

@sanathkr

[email protected]

Building Open Source Communities @ AWS Serverless · • Obsessed about serverless developer tools...

Documents

Transcript of Building Open Source Communities @ AWS Serverless · • Obsessed about serverless developer tools...