2015 09-02 - transaction log prototype
30
David Heath Publishing platform team Government Digital Service GDS Publishing platform team
-
Upload
david-heath -
Category
Software
-
view
83 -
download
1
Transcript of 2015 09-02 - transaction log prototype
David HeathPublishing platform teamGovernment Digital Service
GDSPublishing platform team
Transaction log prototype
2GDSPublishing platform team
Why?
3GDSPublishing platform team
Complexity
4GDSPublishing platform team
many bugs...
5GDSPublishing platform team
* many bugs and issues which we’ve experienced have been related to that
It's a master copyof every change or transaction
6GDSPublishing platform team
It’s an audit log
7GDSPublishing platform team
a feature we’d really like to have
Future-proof
8GDSPublishing platform team
if we have a log of every operation, it’s easy to adapt to possible future requirements, because we can easily add new “derived representations”
What is a derived representation?
9GDSPublishing platform team
Bank account
10GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
Balance: £50
Bank account
11GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
Balance: £45.01
Bank account
12GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
3/9/2015 Cheque 003 £20
Balance: £65.01
Bank account
13GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
3/9/2015 Cheque 003 £20
Balance: £65.01
Derived representation
In publishing…
GDSPublishing platform team
Event Log
In publishing…
GDSPublishing platform team
Event Log
Current documents
In publishing…
GDSPublishing platform team
Event Log
Current documents
History of published documents
In publishing…
GDSPublishing platform team
Event Log
Current documents
History of published documentsEditorial history and comments
GDSPublishing platform team
Event Log
Current documents
History of published documents
Editorial history and comments
Admin elasticsearch index
Message on message queue (asynchronous update)
(separate service/data store)
(Synchronously updated)Publishing API
* Derived representations may be in separate systems* Some things need to be updated synchronously, those would need to be in the same data store and updated using a database transaction to guarantee consistency.
What we did in the prototype
19GDSPublishing platform team
What we did in the prototype• event log
20GDSPublishing platform team
What we did in the prototype• event log• commands (for modification)
21GDSPublishing platform team
What we did in the prototype• event log• commands (for modification)• queries (read from derived
representation)
22GDSPublishing platform team
What we did in the prototype• event log• commands (for modification)• queries (read from derived
representation)• wired up to policy publisher
23GDSPublishing platform team
24GDSPublishing platform team
we used postgresql because it’s awesome- real transactions- JSON data type
Show me some code!
25GDSPublishing platform team
Remaining questions
26GDSPublishing platform team
• conceptual model of editing/workflow
Remaining questions
27GDSPublishing platform team
• conceptual model of editing/workflow
• how much validation?• use schemas?
Remaining questions
28GDSPublishing platform team
• conceptual model of editing/workflow
• how much validation?• use schemas?
• write performance
Remaining questions
29GDSPublishing platform team
• conceptual model of editing/workflow
• how much validation?• use schemas?
• write performance• replay, snapshotting etc
David HeathPublishing platform teamGovernment Digital Service
GDSPublishing platform team
