The new stack isn’t a stack: Fragmentation and terraforming the service layer

Donnie Berkholz, Ph.D.Research Director — Development, DevOps, & IT Ops

The new stack? An infinite array of possible stacks.

2

Open source Software acquisition became fast and free.

3

Why the divergence? Around 2005, suddenly things changed. More VC deals, smaller deal size.

4

Chris Tacy

Cloud Amazon changed. Cloud brought hardware acquisition cost and time near zero. Anyone can run a supercomputer for an hour. This enabled low-‐cost funding models like Y Combinator, amortizing risk across more startups for the same investment size.

5

Chris Tacy

The API economy The new mode for access and transfer of data has become APIs.

6

Developer experience is not just a thing, it’s the thing

7

http://redmonk.com/dberkholz/2012/04/18/adoption-‐of-‐software-‐is-‐a-‐funnel/

Polyglot programming There’s no obvious choice for the right language, based on community adoption.

8

http://redmonk.com/dberkholz/2014/05/02/github-‐language-‐trends-‐and-‐the-‐fragmenting-‐landscape/

Polyglot databases

9 https://orchestrate.io/blog/2013/09/11/11polyglot-‐persistence-‐and-‐nosql-‐more-‐flexibility-‐more-‐complexity/

Are we nearing peak fragmentation of languages? The top 5 languages become less popular every year, and they’re all equalizing. The #1 language used to be >30% of open-‐source code; now it’s 10%. Middle-‐tier languages are growing more popular as well, further equalizing usage and fragmenting language choice. But — this change is slowing. 10

http://redmonk.com/dberkholz/2015/04/02/are-‐we-‐nearing-‐peak-‐fragmentation/

Are we nearing peak fragmentation of languages? Less-‐used languages increase in popularity every year.

11

http://redmonk.com/dberkholz/2014/05/02/github-‐language-‐trends-‐and-‐the-‐fragmenting-‐landscape/

It’s not just languages; frameworks have the same problem

12

http://redmonk.com/dberkholz/2015/04/03/react-‐and-‐polymer-‐arising-‐among-‐javascript-‐mv-‐frameworks/

The UNIX philosophy and the accordion hypothesis Different layers of the stack expand and contract over time. We can’t cope with everything being fragmented all the time.

13

https://www.flickr.com/photos/gammaman/7274558212

Fragmentation drives microservices —different tools for each service

14

Loosely coupled services Business-‐defined separations. Bounded context based on cross-‐organizational empathy. Steve Yegge memo — Amazon must be SOA, or you’re fired.

15

Loosely coupled teams “One of the biggest changes is that we no longer have an official ‘architecture’ team. Instead, we have made ‘architecture’ an ‘ingredient’ on each of our teams.” – Lauri Apple, Gilt Groupe, 14 Nov 2014 DevOps is how you build and run microservices.

16 http://tech.gilt.com/post/102628539834/making-‐architecture-‐work-‐in-‐microservice

Example: Netflix (﴾yawn)﴿ Amazing stuff, great open-‐source code But hard for the rest of the world to envision becoming like them Any others?

17

Example: Software AG •  Launch a multitenant, elastically scalable, componentized cloud integration

platform •  Dynamically launch and run 100s of different demos in AWS •  Their own customers running hotel kiosks, retail POS

18 http://www.softwareag.com/blog/reality_check/index.php/integration-‐insights/microservices-‐fan-‐help-‐adopt/

Example: eBay ca. 2006

19 www.addsimplicity.com/downloads/eBaySDForum2006-‐11-‐29.pdf

Example: eBay ca. 2006

20 www.addsimplicity.com/downloads/eBaySDForum2006-‐11-‐29.pdf

Example: Ctrip (﴾Chinese travel site)﴿

21

http://www.slideshare.net/yang75108/micro-‐service-‐architecture-‐c-‐trip-‐v11

Example: Hailo

22 http://www.slideshare.net/nathariel/scaling-‐microservices-‐architecture-‐on-‐aws

Example: REA (﴾realestate.com.au)﴿

23 http://techblog.realestate.com.au/a-‐microservices-‐implementation-‐retrospective/

Example: Karma

24

https://blog.yourkarma.com/building-‐microservices-‐at-‐karma

What happens when everybody has microservices?

25

What’s next? Tying all the services together

26

Providers

•  The usual IaaS suspects: AWS, DO, OS, etc. •  +Heroku. PaaS? Huh? •  +Mailgun, CloudFlare, DNSimple, DNSMadeEasy.

So now we’re in SaaS land.

Terraform lets you control, orchestrate, and share configuration data across low-‐ and high-‐level services.

27

A trivial example See how the IP below is a variable from above?

28

A Heroku app Add-‐ons, app config, and DNS

29

Service layer – “easy.” But not.

Mixing on-‐prem code and external APIs is the biggest thing since Twinkies

30

Example: Discourse

31 https://hashicorp.com/blog/terraform-‐discourse.html

Application layer – the future

32 https://hashicorp.com/blog/terraform-‐custom-‐providers.html

JS on AWS

33

Look ma, no servers!

34 http://lg.io/2015/05/16/the-‐future-‐is-‐now-‐and-‐its-‐using-‐aws-‐lambda.html

Conclusions •  Technology is increasingly fragmented.

•  Microservices will have heterogeneous technology

stacks.

•  Tooling will need to integrate internal + external microservices.

•  Developers will build apps using only browser-‐side code and microservice orchestration — no servers to manage.

35

Thank you!Donnie Berkholz@dberkholzdonnie.berkholz@451research.com

Some images from this presentation are Creative-‐Commons licensed. https://creativecommons.org/licenses/by-‐sa/2.0/https://creativecommons.org/licenses/by/2.0/

37