There and Back Again As Quick As a Flash: Tuning ... · Hibernate (MySQL) MongoDB (net) DB4O...
Transcript of There and Back Again As Quick As a Flash: Tuning ... · Hibernate (MySQL) MongoDB (net) DB4O...
There and Back AgainAs Quick As a Flash:
Tuning Performance in CDOStefan Winkler
Independent Software Developer and IT ConsultantCDO Committer
@xpomul on Twitter
Note: Extended version of these slides available on the ECE page of this talk!
1Dienstag, 23. Oktober 12
CDO in a nutshell
Server
Client
Client
Client
Storage
2Dienstag, 23. Oktober 12
CDO in a nutshell
Server
Client
Client
Client
Storage
3Dienstag, 23. Oktober 12
CDO in a nutshell
Server
Client
Client
Client
Storage
CDOID
attribute
reference
CDOID
3Dienstag, 23. Oktober 12
CDO in a nutshell
Server
Client
Client
Client
Storage
CDOID
attribute
reference
CDOID
CDOID
3Dienstag, 23. Oktober 12
CDO in a nutshell
Server
Client
Client
Client
Storage
CDOID
attribute
reference
CDOID
3Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
CDOID
attribute
reference
CDOID
3Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
attribute
reference
CDOID
attribute
reference
4Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
attribute
reference
CDOID
CDOID
attribute
reference
4Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
attribute
reference
CDOIDCDOID
attribute
reference
4Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
attribute
reference
CDOIDCDOID
attribute
reference
4Dienstag, 23. Oktober 12
Storage
CDO in a nutshell
Server
Client
Client
Client
Storage
attribute
reference
CDOID
CDOID
attribute
reference
4Dienstag, 23. Oktober 12
Layers
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
5Dienstag, 23. Oktober 12
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
6Dienstag, 23. Oktober 12
Choose the Right Storage!
MEM
Lissome
DB (h2 in-proc)
DB (h2 net)
DB (MySQL)
Hibernate (MySQL)
MongoDB (net)
DB4O (in-proc)
0 200.000 400.000 600.000
1.701
519.063
22.314
9.381
4.269
1.486
1.475
1.121
time to read 10,000 objects (msec)
0 2.500 5.000 7.500 10.000
2.268
1.503
10.151
6.686
5.546
721
431
283
time to write 10,000 objects (msec)
7Dienstag, 23. Oktober 12
Choose the Right Storage!
Store Usage Scenarios Pros Cons
MEMStore prototyping, testing supports all CDO features
not persistent
LissomeStore custom CDO file-based store, optimized for performance
performance, full support planned
custom file format (no external access)
DBStore common/default solution supports all CDO features
DB selection and tuning adds to complexity
HibernateStore custom DB schema required custom DB schema and mapping
only supports subset of CDO features, slower
MongoDB Store No SQL DB wanted,fast writing speeds required
fast writing speeds slow reading speeds,limited commit size
DB4OStore No SQL DB wanted,fast reading speeds required
fast reading speeds slower writing speedsdual licence
ObjectivityStore OODB with commercial support needed
performance ?(not measured)
commercial licence, performance not measured
8Dienstag, 23. Oktober 12
Choose the Right Database!
h2 (net)
MySQL (net)
PostgreSQL (net)
0 2.500 5.000 7.500 10.000
4.231
9.381
4.269
time to read 10,000 objects
5.000 5.833 6.667 7.500
6.606
6.686
5.546
time to write 10,000 objects (msec)
9Dienstag, 23. Oktober 12
Choose the Right Database!
h2 (net)
MySQL (net)
PostgreSQL (net)
0 2.500 5.000 7.500 10.000
4.231
9.381
4.269
time to read 10,000 objects
5.000 5.833 6.667 7.500
6.606
6.686
5.546
time to write 10,000 objects (msec)
Standard Configurations !!!
10Dienstag, 23. Oktober 12
Choose the Right Database!
h2 (net)
MySQL (net)
PostgreSQL (net)
0 2.500 5.000 7.500 10.000
4.231
9.381
4.269
time to read 10,000 objects
5.000 5.833 6.667 7.500
6.606
6.686
5.546
time to write 10,000 objects (msec)
Standard Configurations !!!
... and tweak it!
10Dienstag, 23. Oktober 12
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
11Dienstag, 23. Oktober 12
Choose the Right Repository Mode!
Time
v1 v2 v3 v4 v5 v6 v7
Audit Mode
12Dienstag, 23. Oktober 12
Choose the Right Repository Mode!
Time
v1 v2 v3 v4 v5 v6 v7
Non-Audit Mode
13Dienstag, 23. Oktober 12
Choose the Right Repository Mode!
Time
0:v1 v2 v3 v4 v5 v6Branching Mode
1:v1
2:v1
1:v2
2:v2 2:v3
14Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
•Audit:
• plain vs. range-based list mapping
•Branching:
• plain vs. range-based list mapping
• copy-lazy vs. copy-on-write list branching
15Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
datesum
Orderdatesum
Productdatesum
Productnumberprice
Product
OrderOrderOrderOrder
id version date sum
1 2 3/29/12 102.34
ProductProductProductProduct
id version number price
2 2 1-423-4 39.12
3 2 1-485-3 60.71
4 2 1-584-2 2.51
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id version index value
1 2 0 2
1 2 1 3
1 2 2 4
• non-audit repo mode
• no historic („audit“) views
• writes deltas
16Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
OrderOrderOrderOrder
id version date sum
1 2 3/29/12 102.34
ProductProductProductProduct
id version number price
2 2 1-423-4 39.12
3 2 1-485-3 60.71
4 2 1-584-2 2.51
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id version index value
1 2 0 2
1 2 1 3
1 2 2 4
• audit repo mode
• classic
• always writes complete list
• range-based
• writes deltas
17Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
OrderOrderOrderOrder
id version date sum
1 2 3/29/12 102.34
ProductProductProductProduct
id version number price
2 2 1-423-4 39.12
3 2 1-485-3 60.71
4 2 1-584-2 2.51
• audit repo mode
• classic
• always writes complete list
• range-based
• writes deltas
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id from to index value
1 1 null 0 2
1 1 1 1 4
1 2 null 1 3
1 1 1 2 3
1 2 null 2 4
17Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
• branching repo mode
• classic vs. range-based
• copy-lazy (default) vs. copy-on-branch
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 1 1 null 0 2
1 1 1 1 1 4
1 1 2 null 1 3
1 1 1 1 2 3
1 1 2 null 2 4
18Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
• branching repo mode
• classic vs. range-based
• copy-lazy (default) vs. copy-on-branch
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 1 1 null 0 2
1 1 1 1 1 4
1 1 2 null 1 3
1 1 1 1 2 3
1 1 2 null 2 4
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 2 1 null 1 5
18Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
• branching repo mode
• classic vs. range-based
• copy-lazy (default) vs. copy-on-branch
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 1 1 null 0 2
1 1 1 1 1 4
1 1 2 null 1 3
1 1 1 1 2 3
1 1 2 null 2 4
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 2 1 null 1 5
18Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
• branching repo mode
• classic vs. range-based
• copy-lazy (default) vs. copy-on-branch
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 1 1 null 0 2
1 1 1 1 1 4
1 1 2 null 1 3
1 1 1 1 2 3
1 1 2 null 2 4
18Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
• branching repo mode
• classic vs. range-based
• copy-lazy (default) vs. copy-on-branch
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 1 1 null 0 2
1 1 1 1 1 4
1 1 2 null 1 3
1 1 1 1 2 3
1 1 2 null 2 4
Order_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_ListOrder_Product_List
id branch from to index value
1 2 1 null 0 2
1 2 1 null 1 5
1 2 1 null 2 4
18Dienstag, 23. Oktober 12
Choose the Right DBStore Mapping!
0
2.250
4.500
6.750
9.000
non-audit audit branching branching w/ ranges
721
1.4911.481
688
4.581
1.4731.483
736
8.941
1.5061.442
681
time to insert 100 objects at beginningtime to insert 100 objects in the middletime to insert 100 objects at the end
19Dienstag, 23. Oktober 12
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
20Dienstag, 23. Oktober 12
Choose the Right Transport Layer!
CDO Client
CDOServer
CDO Client
CDOServer
Computer 1 Computer 2
TCP (true client/server)
TCP (same computer)
21Dienstag, 23. Oktober 12
Choose the Right Transport Layer!
CDO Client
CDOServer
CDO Client
CDOServer
CDO Client
CDOServer
Computer 1 Computer 2
TCP (true client/server)
TCP (same computer)
JVM (same JVM, direct calls)
21Dienstag, 23. Oktober 12
A Word On Caching
• Revision Caches in CDO use Soft references
• Caches grow until memory limit is reached
• View Cache can be configured to use strong or weak references instead
CDO Client
CDOServer
ViewCache
(Soft Ref)
SessionCache
(Soft Ref)
RepositoryCache
(Soft Ref)
view.options().setCacheReferenceType(ReferenceType.STRONG);
22Dienstag, 23. Oktober 12
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
23Dienstag, 23. Oktober 12
Choose the Right Model Configuration!
Native
Reflective
Legacy
0 3.000 6.000 9.000 12.000
11.302
8.882
8.581
time to read 100,000 objects
0 2.500 5.000 7.500
4.460
3.076
2.802
time to write 100,000 objects (msec)
24Dienstag, 23. Oktober 12
AppLogic
CDO Client
EMFModel
CDOServer
IStoreCode
Storage
25Dienstag, 23. Oktober 12
Make Use of CDO Loading Policies!
• Lazy Collection Loading
session.options().setCollectionLoadingPolicy( CDOUtil.createCollectionLoadingPolicy( /* initial = */ 5, /* onResolve = */ 5));
• Explicit Prefetching
CDOObject fromCDO = getMyObjectFromCDO();fromCDO.cdoPrefetch(/* depth = */ 3);
• Prefetching by Policy
view.options().setRevisionPrefetchingPolicy( CDOUtil.createRevisionPrefetchingPolicy( /* size = */ 5));
26Dienstag, 23. Oktober 12
Leftovers
• Things I did not talk about
• unordered collections
• store queries (use with caution!)
• LOBs & streaming
27Dienstag, 23. Oktober 12
Summary• Choose the Right Storage!
• Choose the Right Database!
• Choose the Right Repository Mode!
• Choose the Right DBStore Mapping!
• Choose the Right Transport Layer!
• Choose the Right Model Configuration!
• Make Use of CDO Loading Policies!
More of CDO: Now that I‘ve got my model - where is my Application? Thu 10:30-11:30
Newsgroup: eclipse.tools.emf
28Dienstag, 23. Oktober 12