MongoDB as a Data Warehouse: Time Series and Device History Data (Medtronic)

Post on 22-Apr-2015

984 views 1 download

description

 

Transcript of MongoDB as a Data Warehouse: Time Series and Device History Data (Medtronic)

MongoDB at Medtronic

Jeff LemmermanMatt Chimento

1

Medtronic Energy and Component Center

*Disclaimer: We are not currently managing any Device Patient Data in MongoDB

| MDT Confidential2

Agenda

• Intro• Challenge• Approach• Opportunities• Takeaways

3

Medtronic Energy and Component Center• MECC est. 1976

• Manufacture components for devices

• Census – 1200 Employees

• Plant Size – 190,000 Square Feet40,000 Manufacturing15,000 R&D Labs38,000 Office97,000 Common, Support, Warehouse

| MDT Confidential4

Every 3 Seconds…

Where does our data come from?

6

Where stored?

Big Data: Volume, Variety, Velocity

| MDT Confidential7

Without MongoDB

| MDT Confidential8

Data Management Vision

| MDT Confidential9

Reporting and Analytics VarietyOperations Dashboards SPC Charts

Component Summary Time Series

| MDT Confidential10

Component Dataset In Excel

11

How Is Data Retrieved?

12

Loading Data Into Central Repository

13

Component Dataset in RDBMS

14

Ideal Future State

Medtronic Confidential

15

Loading Components Into MongoDB

16

Ideal Future State

Medtronic Confidential

| MDT Confidential17

| MDT Confidential18

Simple, Fast Queries – Complete History

19

Medtronic Confidential

Component Time Series Data

20

Collect and Store Raw Time Series Data

Medtronic Confidential

Our experience with MongoDB

• Consulting/Training has been excellent• Support agreement has been under-utilized

– Emails for security updates etc. are prompt– Release cycle is frequent

• MongoDB Monitoring Service– Potential concerns storing db stats externally– MMS can now be hosted locally

• MongoDB Certification now available– Udacity course, “Data Wrangling with MongoDB”

Gaps

• Enterprise acceptance of “new” approach• Integration with off-the-shelf reporting and

analytics tools• User interface for managing MongoDB cluster• Developer familiarity with JSON and MongoDB• 21 CFR Part 11 Compliance – Audit Tracing

Thank You

Questions?

23

| MDT Confidential24

Key Features• Data stored as documents (JSON-like BSON)

– Flexible-schema– In schema design, think about optimizing for read vs. storage

• Full CRUD support (Create, Read, Update, Delete)– Atomic in-place updates– Ad-hoc queries: Equality, RegEx, Ranges, Geospatial

• Secondary indexes• Replication – redundancy, failover• Sharding – partitioning for read/write scalability• Terminology

– Collection = Table– Index = Index– Document = Row– Column = Field– Joining = Embedding & Linking

26

Creating Components

• .insert() will always try to create new document• .save() if _id already exists will update • If document doesn’t have _id field it is added

27

Reading Components

28

Reading ComponentsReturns Null

29

Updating Components

• $set keyword used for partial updates• Without $set keyword entire document is replaced• {multi : true} to update multiple documents

30

Deleting ComponentsWorks like .find()

Drops collection

Drops database

31

Creating Components – Add()

32

Reading Components

33

Updating Components – Save()

Save sends entire document back to server

34

Updating Components – Update()

Update only sends changes

35

Deleting Components

Needed to add reference to Repo class

36

Automapping

37