Apache Flink · 2016-07-18 · Apache Flink Flink Core Features - General purpose data processing...
1
Apache Flink Flink Core Features - General purpose data processing for clusters - Compatibility: Kafka, Hadoop YARN, HDFS, … - Fully pipelined native streaming runtime - Built-in program optimizations - Stateful operators and UDF support - A single system for batch & stream processing Platform Overview and Related Systems An Open Source Pla/orm for Distributed Stream and Batch Data Processing The Flink Modular System Stack Flink in the Analytics Ecosystem Flink History and Community Flink Success Stories / Use Cases Stay Tuned! Mailing List: news@flink.apache.org flink.apache.org/blog @ApacheFlink Why did Zalando chose Flink? • Flink processes event streams at high throughputs with consistently low latencies. • Flink provides an efficient, easy to use, key/ value based state. • Flink is a true stream-processing framework: Events are processed one at a Eme and each event has its own Eme window. • Stream imperfecEons like out-of-order events are easily handled using the framework’s event Eme processing support. • Flink support was perceived to be beHer than Spark’s. Zalando is able to contact Flink developers directly and they are eager to improve Flink and address our issues. Source: hHps://tech.zalando.de/blog/apache-showdown-flink-vs.-spark/ For more informaEon visit: hHp://flink.apache.org hHp://www.dima.tu-berlin.de hHp://data-arEsans.com 2008 2009 2010 2011 2012 2014 2015 2016 1st Flink Forward Conference 2nd Flink Forward Conference September 2016 flink-forward.org Proposal for Stratosphere I Spinning Fast Ite- rative Dataflows paper published Founded Grant Award Start of Stratosphere I Nephele / PACTs paper published Proposal for Stratosphere II Grant Award Start of Stratosphere II Flink Incubator Project Stratosphere System - paper published Initial vision for a big data analytics platform Flink Top Level Project 21 Meetups Worldwide 186 Contributors 6326 Members 20 Cities 12 Countries Flink Community Groups Across Europe 2010 2012 2015 2015 2016 2016 Berlin 758 Paris 500 Madrid 384 Stockholm 313 Brussels 279 London 190
Transcript of Apache Flink · 2016-07-18 · Apache Flink Flink Core Features - General purpose data processing...
Apache Flink
Flink Core Features - General purpose data processing for clusters
- Compatibility: Kafka, Hadoop YARN, HDFS, …
- Fully pipelined native streaming runtime
- Built-in program optimizations
- Stateful operators and UDF support
- A single system for batch & stream processing
Platform Overview and Related Systems
AnOpenSourcePla/ormforDistributedStreamandBatchDataProcessing
The Flink Modular System Stack Flink in the Analytics Ecosystem
Flink History and Community
Flink Success Stories / Use Cases
Stay Tuned!
MailingList:[email protected] flink.apache.org/blog @ApacheFlink
WhydidZalandochoseFlink?• Flinkprocesseseventstreamsathigh
throughputswithconsistentlylowlatencies.• Flinkprovidesanefficient,easytouse,key/
valuebasedstate.• Flinkisatruestream-processingframework:
EventsareprocessedoneataEmeandeacheventhasitsownEmewindow.
• StreamimperfecEonslikeout-of-ordereventsareeasilyhandledusingtheframework’seventEmeprocessingsupport.
• FlinksupportwasperceivedtobebeHerthanSpark’s.ZalandoisabletocontactFlinkdevelopersdirectlyandtheyareeagertoimproveFlinkandaddressourissues.
Source:hHps://tech.zalando.de/blog/apache-showdown-flink-vs.-spark/
FormoreinformaEonvisit:hHp://flink.apache.orghHp://www.dima.tu-berlin.dehHp://data-arEsans.com
2008 2009 2010
201120122014
2015
2016
1st Flink Forward Conference
2nd Flink Forward ConferenceSeptember 2016flink-forward.org
Proposal for Stratosphere I
Spinning Fast Ite-rative Dataflowspaper published
Founded
Grant Award Start of Stratosphere I
Nephele / PACTs paper published
Proposal for Stratosphere II
Grant Award Start of Stratosphere II
Flink Incubator Project
Stratosphere System-paper published
Initial vision for a big data analytics platform
FlinkTop Level Project
21 Meetups Worldwide 186 Contributors6326 Members20 Cities12 Countries
Flink Community GroupsAcross Europe
2010
2012
2015 2015
20162016
Berlin 758Paris 500Madrid 384Stockholm 313Brussels 279London 190
