Download - Apache Flink · 2016-07-18 · Apache Flink Flink Core Features - General purpose data processing for clusters - Compatibility: Kafka, Hadoop YARN, HDFS, … - Fully pipelined native

Transcript
Page 1: Apache Flink · 2016-07-18 · Apache Flink Flink Core Features - General purpose data processing for clusters - Compatibility: Kafka, Hadoop YARN, HDFS, … - Fully pipelined native

Apache Flink

Flink Core Features - General purpose data processing for clusters

- Compatibility: Kafka, Hadoop YARN, HDFS, …

- Fully pipelined native streaming runtime

- Built-in program optimizations

- Stateful operators and UDF support

- A single system for batch & stream processing

Platform Overview and Related Systems

AnOpenSourcePla/ormforDistributedStreamandBatchDataProcessing

The Flink Modular System Stack Flink in the Analytics Ecosystem

Flink History and Community

Flink Success Stories / Use Cases

Stay Tuned!

MailingList:[email protected] flink.apache.org/blog @ApacheFlink

WhydidZalandochoseFlink?•  Flinkprocesseseventstreamsathigh

throughputswithconsistentlylowlatencies.•  Flinkprovidesanefficient,easytouse,key/

valuebasedstate.•  Flinkisatruestream-processingframework:

EventsareprocessedoneataEmeandeacheventhasitsownEmewindow.

•  StreamimperfecEonslikeout-of-ordereventsareeasilyhandledusingtheframework’seventEmeprocessingsupport.

•  FlinksupportwasperceivedtobebeHerthanSpark’s.ZalandoisabletocontactFlinkdevelopersdirectlyandtheyareeagertoimproveFlinkandaddressourissues.

Source:hHps://tech.zalando.de/blog/apache-showdown-flink-vs.-spark/

FormoreinformaEonvisit:hHp://flink.apache.orghHp://www.dima.tu-berlin.dehHp://data-arEsans.com

2008 2009 2010

201120122014

2015

2016

1st Flink Forward Conference

2nd Flink Forward ConferenceSeptember 2016flink-forward.org

Proposal for Stratosphere I

Spinning Fast Ite-rative Dataflowspaper published

Founded

Grant Award Start of Stratosphere I

Nephele / PACTs paper published

Proposal for Stratosphere II

Grant Award Start of Stratosphere II

Flink Incubator Project

Stratosphere System-paper published

Initial vision for a big data analytics platform

FlinkTop Level Project

21 Meetups Worldwide 186 Contributors6326 Members20 Cities12 Countries

Flink Community GroupsAcross Europe

2010

2012

2015 2015

20162016

Berlin 758Paris 500Madrid 384Stockholm 313Brussels 279London 190