Normalizing with Ember Data 1.0b

Jeremy Gillick

or

True Facts of Using Data in Ember

I’m Jeremy

http://mozmonkey.com

https://github.com/jgillick/

https://linkedin.com/in/jgillick

I work at Nest

We love Emberdepending on the day

Ember Data is GreatExcept when data feeds don’t conform

Serializers connect Raw Data to Ember Data

{ … }

JSONSerializer

Ember Data

Let’s talk about data

Ember prefers side loading to nested JSON

But why?

For example{! "posts": [! {! "id": 5,! "title":You won't believe what was hiding in this kid's locker",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "spam-me@please.com"! }! }! ]!}

{! "posts": [! {! "id": 6,! "title": "New Study: Apricots May Help Cure Glaucoma",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "spam-me@please.com"! }! },! {! "id": 5,! "title": "You won't believe what was hiding in this kid's locker",! "body": "...",! "author": {! "name": "Jeremy Gillick",! "role": "Author",! "email": "spam-me@please.com"! }! }! ]!}

For example

Redundant, adds feed bloat and which one is the source of truth?

This is better{! "posts": [! {! "id": 4,! "title": "New Study: Apricots May Help Cure Glaucoma",! "body": "...",! "author": 42! },! {! "id": 5,! "title": "You won't believe what was hiding in this kid's locker",! "body": "...",! "author": 42! }! ],! "users": [! {! "id": 42,! "name": "Jeremy Gillick",! "role": "Author",! "email": "spam-me@please.com"! }! ]!}

Ember Data Expects{! "modelOneRecord": {! ...! }! "modelTwoRecords": [! { ... },! { ... }! ],! "modelThreeRecords": [! { ... },! { ... }! ]!}

No further nesting is allowed

Ember Data Expects

{! "posts": [! ...! ],!! "users": [! …! ]!}

App.Post records

App.User records

Not all JSON APIs will be flat

A nested world{! "products": [! {! "name": "Robot",! "description": "A robot may not injure a human being or...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", "black", "#E1563F"]! }! ]! }! ]!}

Ember Data can’t process that

{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

{! "products": [! {! "id": "product-1",! "name": "Robot",! "description": “...”,! "price": "price-1",! "size": "dimension-1",! "options": [! “options-1”! ]! }! ],! "prices": [! {! "id": "price-1",! "value": 59.99,! "currency": "USD"! } ! ]! "dimensions": [ … ],! "options": [ … ]!}!!

Flatten that feed

How do we do this?With a custom Ember Data Serializer!

Two common ways• Create ProductSerializer that manually converts the

JSON

• A lot of very specific code that you’ll have to repeat for all nested JSON payloads.

• Build a generic serializer that automatically flattens nested JSON objects

• Good, generic, DRY

Defining the model{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Product = DS.Model.extend({! name: DS.attr('string'),! description: DS.attr('string'),! price: DS.belongsTo('Price'),! size: DS.belongsTo('Dimension'),! options: DS.hasMany('Option')!});!!App.Price = DS.Model.extend({! value: DS.attr('number'),! currency: DS.attr('string')!});!!App.Dimension = DS.Model.extend({! height: DS.attr('number'),! width: DS.attr('number'),! depth: DS.attr('number')!});!!App.Option = DS.Model.extend({! name: DS.attr('string'),! values: DS.attr()!});

Steps

• Loop through all root JSON properties

• Determine which model they represent

• Get all the relationships for that model

• Side load any of those relationships

{! "products": [! {! "name": "Robot",! "description": "...",! "price": {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Product

Relationships• price • size • option

Side load

$$$ Profit $$$

JS Methodsextract: function(store, type, payload, id, requestType) { ... }

processRelationships: function(store, type, payload, hash) { ... }

sideloadRecord: function(store, type, payload, hash) { ... }

Create a Serializer/**! Deserialize a nested JSON payload into a flat object! with sideloaded relationships that Ember Data can import.!*/!App.NestedSerializer = DS.RESTSerializer.extend({!! /**! (overloaded method)! Deserialize a JSON payload from the server.!! @method normalizePayload! @param {Object} payload! @return {Object} the normalized payload! */! extract: function(store, type, payload, id, requestType) {! return this._super(store, type, payload, id, requestType);! }!!});

{! "products": [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! return this._super(store, type, payload, id, requestType);!}

{! "products": [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! ! }, this);!! return this._super(store, type, payload, id, requestType);!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container!! ! ! ! .lookupFactory('model:' + key.singularize());!! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": [! {! ...! }! ]!}

{! "products": [! {! ...! }! ]!}

product

Singularize

container.lookup(‘model:product’)

App.Product

"products"

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container!! ! ! ! .lookupFactory('model:' + key.singularize());!! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": [! {! ...! }! ]!}

{! "products": ! [! {! ...! }! ]!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container! .lookupFactory('model:' + key.singularize()),! hash = payload[key];!! }, this);!! return this._super(store, type, payload, id, requestType);!}

extract: function(store, type, payload, id, requestType) {! var rootKeys = Ember.keys(payload);!! // Loop through root properties and process their relationships! rootKeys.forEach(function(key){! var type = store.container! .lookupFactory('model:' + key.singularize()),! hash = payload[key];!! // Sideload embedded relationships of this model hash! if (type) {! this.processRelationships(store, type, payload, hash);! }! }, this);!! return this._super(store, type, payload, id, requestType);!}

{! "products": ! [! {! ...! }! ]!}

/**! Process nested relationships on a single hash record!! @method extractRelationships! @param {DS.Store} store! @param {DS.Model} type! @param {Object} payload The entire payload! @param {Object} hash The hash for the record being processed! @return {Object} The updated hash object!*/!processRelationships: function(store, type, payload, hash) {!!},

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! return hash;!},

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! }!! return hash;!},

{! "products": [! {! ...! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! ! }, this);! }!! return hash;!},

!App.Product.eachRelationship(function(key, relationship) {! !!}, this);!

App.Product = DS.Model.extend({! name: DS.attr('string'),! description: DS.attr('string'),! price: DS.belongsTo('Price'),! size: DS.belongsTo('Dimension'),! options: DS.hasMany('Option')!});

key = 'price'! relationship = {! "type": App.Price,! "kind": "belongsTo",! ...! }

key = 'size'! relationship = {! "type": App.Dimension,! "kind": "belongsTo",! ...! }

key = 'options'! relationship = {! "type": App.Option,! "kind": "hasMany",! ...! }

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key]; // The hash for this relationship! ! }, this);! }!! return hash;!},

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key], // The hash for this relationship! relType = relationship.type; // The model for this relationship!! }, this);! }!! return hash;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

App.Price

processRelationships: function(store, type, payload, hash) {!! // If hash is an array, process each item in the array! if (hash instanceof Array) {! hash.forEach(function(item, i){! hash[i] = this.processRelationships(store, type, payload, item);! }, this);! }!! else {!! // Find all relationships in this model! type.eachRelationship(function(key, relationship) {! var related = hash[key], ! relType = relationship.type;!! hash[key] = this.sideloadRecord(store, relType, payload, related);! ! }, this);! }!! return hash;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

/**! Sideload a record hash to the payload!! @method sideloadRecord! @param {DS.Store} store! @param {DS.Model} type! @param {Object} payload The entire payload! @param {Object} hash The record hash object! @return {Object} The ID of the record(s) sideloaded!*/!sideloadRecord: function(store, type, payload, hash) {! !},

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! ! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! ! }! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ]!}

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! ]!}

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! id = this.generateID(store, type, hash);! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! ]!}

Every record needs an ID

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! ! // Sideload, if it's not already sideloaded! if (sideloadArr.findBy('id', id) === undefined){! sideloadArr.push(hash);! payload[sideLoadkey] = sideloadArr;! }! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": ! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! },! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": “generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

sideloadRecord: function(store, type, payload, hash) {! var id, sideLoadkey, sideloadArr, serializer;!! // If hash is an array, sideload each item in the array! if (hash instanceof Array) {! id = [];! hash.forEach(function(item, i){! id[i] = this.sideloadRecord(store, type, payload, item);! }, this);! }! // Sideload record! else if (typeof hash === 'object') {! sideLoadkey = type.typeKey.pluralize(); ! sideloadArr = payload[sideLoadkey] || [];! ! // Sideload, if it's not already sideloaded! if (sideloadArr.findBy('id', id) === undefined){! sideloadArr.push(hash);! payload[sideLoadkey] = sideloadArr;! }! }!! return id;!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

processRelationships: function(store, type, payload, hash) {! ...! hash[key] = this.sideloadRecord(store, relType, payload, related);! ...!},

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": "generated-2",! "options": [! “generated-3”! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ],! "dimensions": [{! "id": "generated-2",! "height": 24,! "width": 12,! "depth": 14! }],! "options": [ ! {! "id": "generated-3",! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]!}

{! "products": [! {! "name": "Robot",! "description": "...",! "price": "generated-1",! "size": {! "height": 24,! "width": 12,! "depth": 14! },! "options": [! {! "name": "Color",! "values": ["silver", ! "black", ! "#E1563F"]! }! ]! }! ],! "prices": [! {! "id": "generated-1",! "value": 59.99,! "currency": "USD"! }! ]!}

Apply the Serializer

App.ApplicationSerializer = App.NestedSerializer;

App.ProductSerializer = App.NestedSerializer.extend({});

- OR -

Now for a demo

http://emberjs.jsbin.com/neriyi/edit

http://emberjs.jsbin.com/neriyi/edit

Questions?

http://www.slideshare.net/JeremyGillick/normalizing-data