A tale of 3 databases
-
Upload
chris-skardon -
Category
Technology
-
view
155 -
download
0
Transcript of A tale of 3 databases
![Page 2: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/2.jpg)
Before We Begin
![Page 3: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/3.jpg)
Contents• Prologue• Chapter 1 – SQL• Chapter 2 – RavenDB• Chapter 3 – Neo4j• Epilogue
![Page 4: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/4.jpg)
Prologue
![Page 5: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/5.jpg)
What is Tournr?• Competition running site• Any competition, • Sport (running, swimming) • Game (scrabble, chess)• Other…
• Help organisers and competitors have a permanent record• Unified Ranking
![Page 6: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/6.jpg)
Chapter 1SQL
![Page 7: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/7.jpg)
WISAThe traditional entry point for a .NET developer
• Windows• Internet Information Services (IIS)• MS SQL Server 2012• Entity Framework (ORM)
• Asp.NET
![Page 8: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/8.jpg)
The Good• Lots of common knowledge• SQL is well-known• Standard• Libraries• Language• Skills
• Cheap!
![Page 9: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/9.jpg)
INTERLUDE – The Generalist• The Swiss-Army Knife of development• Good at lots of things• But you’re not going to want to cut a
tree down with it
• I can write SQL• But I’m no expert
![Page 10: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/10.jpg)
The Bad• Changes to the model involve writing SQL• Lots of SQL
• Running scripts against DBs, • then resetting, • then…
• Slooooooow turnaround• Feels so 2000
![Page 11: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/11.jpg)
6 Months Later…• Adding features to Tournr was becoming a bit onerous• Lookup Tables to more Lookup Tables• Playing around with RavenDB• *PING* Maybe a good idea for Tournr?
![Page 12: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/12.jpg)
Chapter 2Document DB (RavenDB)
![Page 13: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/13.jpg)
WISA to WIRA• Windows / IIS / RavenDB / Asp.NET• Took about a month, maybe a smidge more• This is partly due to a lack of users – so migration not an issue
• Threw away a lot of code (this is good btw)
![Page 14: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/14.jpg)
The Good• RavenDB has 1st class .NET integration• Really – it’s awesome• Supports LINQ
• Adding new features – very quick• Tournr would not exist without RavenDB• Simple to setup (dev wise at least)• Embeddable • Testing against an actual DB instance
• Dynamic Indexing
![Page 15: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/15.jpg)
Example
using (var session = RavenSession) {
var tournament = session.Load<Tournament>(id);
tournament.Name = "New Name"; session.SaveChanges(); }
![Page 16: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/16.jpg)
INTERLUDE – Disclaimer• Struggled with TECH X eh? That’s because:• You’re doing it wrong…• You don’t understand…• You’re being vindictive…• <insert your own reason here>
• Probably totally correct (except the vindictive one), and this is really MY experiences!
![Page 17: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/17.jpg)
The Bad• The documentation is sucky• Niche DB
• Getting help is sometimes hard• Especially for edge cases• Hiring
• Forced into a way of architecting your code• Ayende’s way or the Highway
• Manual linking of documents• Documents grow• Hosting
![Page 18: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/18.jpg)
A year or so later…• Tournr was doing well• Adding features was going OK, but…• Beginning to hit the wall of the design• Duplicates appearing in the documents (by design obvs)
• Drawing out a new feature – looked like a graph• But I’m not using a Graph DB• I could though right?
![Page 19: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/19.jpg)
Chapter 3Graph DB (Neo4j)
![Page 20: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/20.jpg)
Am I being foolish?• A question for every day• Negatives:• Switching DBs - you can’t add new stuff whilst doing it• I have users this time
• Positives:• I’ve been using Neo4j for 3 years or so now, so have
experience• Fits the model better
![Page 21: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/21.jpg)
JUST DO IT!
![Page 22: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/22.jpg)
WIRA to WINA• Windows / IIS / Neo4j / Asp.NET• Nuget -> Neo4jClient• Actually:• Azure Web Apps / GrapheneDB / Asp.NET• Is AWAGA as good?
![Page 23: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/23.jpg)
Membership• Important Quick Win• MVC3 Membership -> MVC5 Identity• A pain in itself• The only ‘architectural’ change aside from DB changes• Why?
• No existing Neo4j Membership implementation• An existing Neo4j Identity implementation
• Switched over, and it worked• No-one more surprised than me (seriously)
![Page 24: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/24.jpg)
The “Process”1. Remove RavenDB2. Break build entirely3. Fix build4. Start putting Model 1 into place5. Decide to go with Model 26. Pull out Model 1 code7. Replace with Model 2 code8. Repeat
![Page 25: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/25.jpg)
The ‘Models’• Evolved over time• First model pretty much thrown out
• Getting to market was priority• Ease of writing queries / code over correctness
![Page 26: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/26.jpg)
Model 1
USER PERSONALDETAILS: HAS_PERSONAL_DETAILS
![Page 27: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/27.jpg)
Model 2
USER
PersonalDetails {get;}Email { get; }
![Page 28: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/28.jpg)
Points of change – New stuff• Custom Json Serializers• Not needed before
• Cypher Queries• All DB access was changed• Single biggest area of changes
![Page 29: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/29.jpg)
Points of change – Removed stuff• Controller DB Access code• Ravens approach not really appropriate for Neo4j
![Page 30: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/30.jpg)
So… It compiles – job done!• Looks like it’s working…• Uh oh – problems• Missing / Broken functionality• Performance is SHOCKINGLY bad• 30+ seconds to load a page• Don’t worry Neo4j fans! it’s my fault…
![Page 31: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/31.jpg)
Problems you say… why?• Change of mindset required• First initial code change – simple, and dumb• Multiple queries = SLOW• Always try to get the most out of one query• Literally – if a controller action required 2 calls, make that 1
![Page 32: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/32.jpg)
The Good• Quick & Easy to install• Good modelling • Adding new features quick• Documentation• Cypher
MATCH (t:Tournament)-[:HAS_COMPETITOR]->(c:Competitor)WHERE t.Name = ‘Awesome Comp #1’ RETURN c
![Page 33: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/33.jpg)
The Bad• Niche DB• Support• Hiring
• Cypher can cause muddles• Hosting
![Page 34: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/34.jpg)
EpilogueOr “Was it worth it?”
![Page 35: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/35.jpg)
Model
Comp
750m
User
1.5Km
IS_OWNER
IS_COMPETING
IS_COMPETINGHAS_EVENT
HAS_EVENT
To
![Page 36: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/36.jpg)
Code - Max
IL Cyclomatic ComplexityMax LOC per MethodMethods per Type
![Page 37: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/37.jpg)
Code - Averages
IL Cyclomatic ComplexityLOC per MethodMethods per Type
![Page 38: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/38.jpg)
Future Development• With Model – easier to ‘see’• Some frustrations over how to do some things
![Page 39: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/39.jpg)
Appendix 1Or “Where the book analogy shows signs of being tired”
![Page 40: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/40.jpg)
What should you take from this?• Converting from SQL / N.E.Other Database• Totally Doable
• Is it easy?• Yes and no
• Is it worth it?• Depends on your use-case• Does a graph model provide a closer analogy to your work
space?• Would I do it again?• Yes
![Page 41: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/41.jpg)
Further Reading
![Page 42: A tale of 3 databases](https://reader031.fdocuments.us/reader031/viewer/2022030211/58a1a05c1a28abe6468b4873/html5/thumbnails/42.jpg)
Links!• RavenDB – http://ravendb.net/• Neo4j – http://neo4j.org/• Tournr – https://www.tournr.com/• XClave - http://www.xclave.co.uk/