Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in...
Transcript of Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in...
![Page 1: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/1.jpg)
Building a database startup in ChinaTrends in Tech and Business
By Dongxu Huang
![Page 2: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/2.jpg)
Part I - Intro to PingCAP
![Page 3: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/3.jpg)
OLTP & OLAP
● OLTP (Online transaction processing)○ Oracle / SQL Server / IBM DB2○ MySQL / PostgreSQL
● OLAP (Online analytical processing)○ SAP HANA / Apache Spark / Pivotal
Greenplum○ Hadoop Hive / Apache Impala / Apache
Kudu / Presto / Druid○ ...
ETL
![Page 4: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/4.jpg)
What are the pain points?
● Data is growing at a faster rate than ever before○ Trend: AI / Data mining○ Distributed systems become mainstream
● Traditional OLTP databases are no longer sufficient to meet the needs of companies in the big data era
○ Scalability with strong consistency
● OLTP and OLAP are separate to each other○ You can’t do real-time analyze because of the existence of ETL○ And writing & maintaining ETL jobs is very hard and boring
● People are moving to cloud, maintaining infrastructure is painful● SQL never dies
○ Legacy codes / applications, they already depend on SQL databases.○ Everybody knows SQL, and it’s developer-friendly.
![Page 5: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/5.jpg)
PingCAP: A Chinese-born Database Company
● Founded in 2015 by 3 infrastructure engineers
● What’s the story?
![Page 6: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/6.jpg)
Part II - What’s TiDB?
![Page 7: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/7.jpg)
The Wishlist
● Scalability○ Scaling the capacity/thoughput with the cluster size○ Elasticly scaling out for hyper growth
● ACID semantics with steady transaction latency● High Availability● OLAP without interfering the OLTP workload
![Page 8: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/8.jpg)
![Page 9: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/9.jpg)
Expectation
![Page 10: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/10.jpg)
Reality
![Page 11: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/11.jpg)
VS
![Page 12: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/12.jpg)
VS
![Page 13: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/13.jpg)
PingCAP.com
TiDB platform
● NewSQL: the best features of both RDBMS and NoSQL
○ Full-featured SQL
■ MySQL compatibility
○ ACID compliance
○ HA with strong consistency
○ Elastic scalability
● HTAP
○ Serve both OLTP & Real-time OLAP
HTAP
![Page 14: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/14.jpg)
PingCAP.com
TiDB Architecture
TiDB
TiDB
Worker
Spark Driver
TiKV Cluster (Storage)
Metadata
TiKV TiKV
TiKV
MySQL Clients
Syncer
Data location
Job
TiSpark
DistSQL API
TiKV
TiDB
TSO/Data location
Worker
Worker
Spark Cluster
TiDB Cluster
TiDB
... ......
DistSQL API
PD
PD
PD Cluster
TiKV TiKVTiDB
PD
![Page 15: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/15.jpg)
PingCAP.com
Components
● TiDB (tidb-server)
● TiKV (tikv-server)
● Placement Driver (PD)
● TiSpark
● Tools (syncer / TiDB-Lightning / {tikv,pd}-ctl)
● TiDB-operator for Kubernetes
![Page 16: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/16.jpg)
PingCAP.com
TiDB (tidb-server)
● Stateless SQL layer
○ Client can connect to any
existing tidb-server instance
○ TiDB *will not* re-shuffle the
data across different
tidb-servers
● Full-featured SQL Layer
○ Speak MySQL wire protocol
■ Why not reusing MySQL?
○ Homemade parser & lexer
○ RBO & CBO
○ Secondary index support
○ DML & DDL
SQL AST Logical Plan
OptimizedLogical Plan
Cost Model
SelectedPhysical Plan
TiKV TiKV TiKV
tidb-server
Statistics
TiKV TiKVTiKV
TiKV Cluster
![Page 17: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/17.jpg)
PingCAP.com
TiKV (tikv-server)
● The storage layer for TiDB
● Distributed Key-Value store
○ Support ACID Transactions○ Replicate logs by Raft○ Range partitioning
■ Split / merge
dynamically○ Support coprocessor for
SQL operators pushdown
TiKV TiKV
TiKV TiKV
TiKV TiKV
PD PD
PD
Placement Driver
TiKV TiKV
TiKV TiKV
TiKV TiKV
TiKV Nodes
Client
Metadata
Dataflow
![Page 18: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/18.jpg)
PingCAP.com
TiKV (tikv-server) - Physical stack
Highly layered
TiKV
API (gRPC)
Transaction
MVCC
Multi-Raft (gRPC)
RocksDB
Raw KV API
Transactional KV API
![Page 19: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/19.jpg)
PingCAP.com
TiKV (tikv-server) - Logical view (1/2)● Stores Key-Value pairs
● Infinite sorted (in byte-order) Key-Value map
● Key space is split into regions (Range-based) dynamically, like HBase
● Metadata: [start_key, end_key)
● Each region has multiple replicas (default 3) across different physical nodes, data is replicated by Raft
● All regions in the same node share the same RocksDB instance
TiKV Key Space
[ start_key, end_key)
(-∞, +∞)Sorted Map
96 MB
![Page 20: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/20.jpg)
PingCAP.com
TiKV (tikv-server) - Logical view (2/2)
Region 1
Region 2
Region 3
Region 1
Region 2
Region 3
Region 1
Region 2
Region 3
Raft Group 1
Raft Group 2
Raft Group 3
A - D
D - H
H - K
Key Space
...
...
TiKV A TiKV B TiKV C
![Page 21: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/21.jpg)
PingCAP.com
TiKV (tikv-server) - Region split & merge
Region ARegion ARegion B
Region A
Region A
Region B
Split
Region ARegion A
Region B
MergeNode 2Node 1
Region splitting and merging affect all replicas of one region. The correctness and consistency are guaranteed by Raft.
![Page 22: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/22.jpg)
PingCAP.com
TiKV (tikv-server) - Scaling & Rebalancing
Region 1
Region 3
Region 1Region 2
Region 1*
Region 2 Region 2Region 3Region 3
Node A
Node B
Node C
Node D
![Page 23: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/23.jpg)
PingCAP.com
TiKV (tikv-server) - Scaling & Rebalancing
Region 1
Region 3
Region 1^
Region 2Region 1*
Region 2Region 2
Region 3Region 3
Node A
Node B
Node E1) Transfer leadership of region 1 from Node A to Node B
Node C
Node D
![Page 24: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/24.jpg)
PingCAP.com
TiKV (tikv-server) - Scaling & Rebalancing
Region 1
Region 3
Region 1*
Region 2
Region 2Region 2
Region 3
Region 1
Region 3
Node A
Node B
2) Add Replica to Node E
Node C
Node D
Node E
Region 1
![Page 25: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/25.jpg)
PingCAP.com
TiKV (tikv-server) - Scaling & Rebalancing
Region 1
Region 3
Region 1*
Region 2
Region 2Region 2
Region 3
Region 1
Region 3
Node A
Node B
3) Remove Replica from Node A
Node C
Node D
Node E
![Page 26: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/26.jpg)
Part III - What has changed?
![Page 27: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/27.jpg)
What has changed?
Software!
![Page 28: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/28.jpg)
What has changed?
Hardware!
![Page 29: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/29.jpg)
What has changed?
● Data type○ Hot data: Need for speed!○ Warm data: The source of
truth○ Cold data: Archive
● Warm data architecture○ the missing part of modern
data processing stack
![Page 30: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/30.jpg)
Let’s say we have an application like this...
SELECT COUNT(DISTINCT t1.BuyerID)FROM Orders_USA t1, Orders_China t2WHERE t1.BuyerID = t2.BuyerID;
![Page 31: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/31.jpg)
In the old days...
Hot
Cold
![Page 32: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/32.jpg)
How many replicas do we need?
![Page 33: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/33.jpg)
![Page 34: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/34.jpg)
Part IV - Tech trends
![Page 35: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/35.jpg)
Let’s say we want to build a new database in 2010s...
![Page 36: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/36.jpg)
Log is the new database
● Fewer I/O● Smaller network
packets
HyPer AWS Aurora
![Page 37: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/37.jpg)
VS
Log is the new database
Traditional RDBMS vs TiDB
![Page 38: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/38.jpg)
Vectorized
SELECT SUM(C4) FROM R;
VS
![Page 39: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/39.jpg)
Vectorized: Challenges
● Limitation of the Volcano Model○ Tuple at a time
● Poor cache utilization● Virtual function call overhead
○ next()● How to keep data fresh?
![Page 40: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/40.jpg)
SELECTId,Name,Age, (Age-30)*50 AS
BonusFROM PeopleWHERE Age > 30
Vectorized: From Tuple to Chunk
![Page 41: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/41.jpg)
Workload Isolation
● What’s the real trade-off?● How to keep data fresh?
○ Raft○ MVCC○ MemStore
● TiFlash
![Page 42: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/42.jpg)
Workload Isolation
![Page 43: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/43.jpg)
SIMD
void plus( uint32_t * dest, uint32_t * src, size_t n) { for i in 0..n { dest[i] += src[i]; }}
![Page 44: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/44.jpg)
SIMD
void plus( uint32_t * dest, uint32_t * src, size_t n) { while ... { _mm_add_epi32(&dest[i], &src[i]); i += 4; }}
![Page 45: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/45.jpg)
Dynamic Data placement
VS
![Page 46: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/46.jpg)
Dynamic Data placement
● Flexible, no need to expose sharding details to users. Application development becomes simple and flexible
● Aware of the workload changes and response in real-time● Logical partitioning based on business trait is more intuitive● Challenges:
○ How to find hot spots in time○ How to adjust the replica strategy more flexibly
■ Number of replicas, replica data placement, data structure of replica
○ Could AI help us?
![Page 47: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/47.jpg)
Storage and Computing Seperation
● What are we talking about when we talk about Storage Computing Separation?
○ Q: Is TiDB a Storage-Compute-Separation architecture? ● Pros:
○ The physical resources required by the storage and computing layers are different, separation is good for resource scheduling.
■ Stateless access layer (Like TiDB-Server instances, handling connections) is more convenient to expand on demand.
■ Operation and maintenance friendly, components can be upgraded on demand.
![Page 48: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/48.jpg)
Everything is Pluggable
● Computing○ TiDB SQL○ Spark SQL
● Storage○ Local storage
■ TiFlash■ ...
○ Multi-model data source■ Unistore■ ...
![Page 49: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/49.jpg)
Distributed Transaction
● 2PC is still the only option○ Timestamp is the best thing we got
for now● Challenges:
○ Reduce round-trips■ Is it necessary to assign 2
timestamps for each transaction?
■ Is PD the only place to get timestamp?
![Page 50: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/50.jpg)
Distributed Transaction: What we can do
● Follower Read (WIP)○ Scale-out read performance without
sacrificing consistency● Optimizing Percolator model (DONE)
○ OptimizedCommitTS.tla● Use wall clock like HLC?
![Page 51: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/51.jpg)
Cloud-Native Architecture
● DB is DB, NOTHING MORE, NOTHING LESS● Putting cluster scheduling, resource allocation,
and tenant isolation outside of the database kernel
● Integrating with the user's infrastructure○ Kubernetes is winning
![Page 52: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/52.jpg)
Part V - Business trends
![Page 53: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/53.jpg)
What’s happening in China
野蛮生长
![Page 54: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/54.jpg)
What’s happening in China
![Page 55: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/55.jpg)
What’s happening in China
![Page 56: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/56.jpg)
What’s happening in China
![Page 57: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/57.jpg)
What’s happening in China
对新技术赋能业务的期望更高
![Page 58: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/58.jpg)
![Page 59: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/59.jpg)
真实的两个故事,来自:TiDB 的一个在某二线城市的客户TiDB 的一个行业巨头客户
![Page 60: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/60.jpg)
What’s happening in China
传统行业互联网转型过程中的阵痛带来的机会
![Page 61: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/61.jpg)
![Page 62: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/62.jpg)
What’s happening in China
基础软件人才储备逐渐变强
![Page 63: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/63.jpg)
What’s happening in China
一些核心场景(银行核心系统)敢于使用国产技术
![Page 64: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/64.jpg)
![Page 65: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/65.jpg)
What’s happening in China
PingCAP 路径:开源(互联网/社区)<-> 商业化
![Page 66: Building a database startup in China - Cloudinary · 2019-02-27 · Building a database startup in China Trends in Tech and Business By Dongxu Huang. Part I - Intro to PingCAP. ...](https://reader030.fdocuments.us/reader030/viewer/2022040609/5ec999176fea8a618a6844e0/html5/thumbnails/66.jpg)
Thank You !