Elective in Software and Services(Complementi di software e servizi per la società dell'informazione)

Section Information VisualizationNumber of credits : 3Tutor: Marco Angelini (angelini@dis.uniroma1.it)

4.3 – D3: Data, Geographical & Linked VisualizationsA.A. 2013/14

RECAP

A.A. 2013/14

• What have we seen so far:

Graphical elements

(SVG shapes)(d3 primitives)

(complex shapes)

(Layouts)

Data Interaction/Behaviors

(binding)(scales)

(specific Layouts representation)

(Transitions)(Listeners)

(Events)(Behaviors)

RECAP

A.A. 2013/14

• What we still have to see:

Data manipulation Geographic visualizations

Coordinated views

OUTLINE

• Recap

• Manipulating data

• Geo visualization

• Coordinated views

• Deploy

A.A. 2013/14

Manipulating Data

A.A. 2013/14

• In D3 you can use/import your preferred choice of data representation, but at low level layer the D3's canonical representation of data is an array

• plain JavaScript offers 3 main types of methods for manipulating arrays:

Mutator

array.reverse

array.shift

array.sort

array.splice

array.unshift

Accessor

array.concat

array.join

array.slice

array.indexOf

array.lastIndexOf

Iteration

array.filter

array.forEach

array.every

array.map

array.some

array.reduce

Manipulating Data

A.A. 2013/14

• On top of that, d3 library provides its own functions for data manipulation

Ordering: d3.ascending(a,b), d3.descending(a,b)

d3.min, d3.max, d3.extent

Statistical values: d3.sum(array[, accessor])

d3.mean(array[, accessor])

d3.median(array[, accessor])

d3.quantile(numbers, p)

d3.shuffle(array)

Manipulating Data: objects

A.A. 2013/14

D3 provides several operators for converting associative arrays to standard indexed arrays:

# d3.keys(object)Returns an array containing the property names of the specified

object (an associative array)

# d3.values(object)Returns an array containing the property values of the specified

object (an associative array)

# d3.entries(object)Returns an array containing the property keys and values of the

specified object (an associative array). Each entry is an object with a key and value attribute, such as {key: "foo", value: 42}

• In all cases, the order of the returned array is undefined!

Manipulating Data: objects

A.A. 2013/14

Example:• Original data set:

• d3.keys(dati):

• d3.values(dati):

Manipulating Data: objects

A.A. 2013/14

• d3.entries(dati):

Manipulating Data: operators

A.A. 2013/14

# d3.split(array[, function])Splits the specified array into multiple arrays at breakpoints

identified by the specified function

# d3.merge(arrays)Merges the specified arrays into a single array. Similar to the built-

in array concat method, but more convenient when you have an array of arrays

# d3.range([start, ]stop[, step])Generates an array containing an arithmetic progression

# d3.permute(array, indexes)Returns a permutation of the specified array using the specified

array of indexes

# d3.zip(arrays…)Returns an array of arrays, where the ith array contains the ith

element from each of the argument arrays

Manipulating Data: nest

A.A. 2013/14

• Nesting allows elements in an array to be grouped into a hierarchical tree structure

• The resulting output is a tree rather than a plain array

• The d3.nest() operator doesn’t affect the original array passed to it, but operates on a copy and produces a new object as result.

• Moreover, the nest returned object is reusable, and does not retain any references to the data that is nested.

Manipulating Data: nest

A.A. 2013/14

Example:

• Original data of our visualization, imported by default function:

d3.csv(“data.csv”,function(csv)

Manipulating Data: nest

A.A. 2013/14

Let’s apply the nest operator on continent property:

…and what we will obtain is a NEW ARRAY with original elements set in place according to the resulting hierarchy:

Manipulating Data: nest

A.A. 2013/14

• Now, the new formatted data will be referenced with:

First we reference the States:

….and then for each of them we reference the countries:

Manipulating Data: rollup

A.A. 2013/14

# nest.rollup(function): Specifies a rollup function to be applied on each group of leaf elements.

• In our case:

… and the result is a new array with aggregated values according to the function passed to the rollup operator

OUTLINE

• Recap

• Manipulating data

• Geo visualization

• Coordinated views

• Deploy

A.A. 2013/14

GEO VISUALIZATION

A.A. 2013/14

• Geographic visualizations are one of the most used tools in visualizing the demographics, current poll results, election results, sentiment analysis and so on….

• Really easy to understand (everyone know geography)

• Useful in the scenario of coordinated views

• D3 provides primitives for easing the work of the developers on geo visualizations

• Based on GeoJson, a standard way of representing geographic features in JavaScript

GeoJSON 1.0 specification: http://www.geojson.org/geojson-spec.html

GEO VISUALIZATION: maps

A.A. 2013/14

• Maps can be made of two ways:

1. Prebuilt SVG paths

Simply search on the web for the SVG path of the country you want to represent and attach it statically to the draw-area of your visualization

• Cool & simple, but limited in scope (you can’t add in location data)

GEO VISUALIZATION: maps

A.A. 2013/14

2. Geo paths/Geo projectionsGeo paths generate non basic shapes, using either a geometry or GeoJson object to connect the dots

Geo Projections can be added onto geo paths to connect the geo data to different built in types of map projections.

Types of map projections built in includes:

Mercator - flat map of the world

Albers - conic map projection where distortion slightly occurs for shape and size, but not as much for parellel lines

Azimuthal - equidistant and equal-area projections

…along with a whole projection plugin of extra projections you can add in!

GEO VISUALIZATION: maps

A.A. 2013/14

Example: Azimuthal

GEO VISUALIZATION: maps

A.A. 2013/14

Example:

• Us-state.json is a GeoJson file that contains the respective coordinates, in the form of an object, of the 50 US states.

GEO VISUALIZATION

A.A. 2013/14

Marking Coordinates:

When working with maps, marks are left to note important locations (cities, landmarks, traps).

With the projection system in place on our map, all we have to do is add in marker with our coordinates.

• The coordinates are expressed in terms of longitudes and latitudes.• …so you can choose to cable them in a static file or retrieve them

dynamically from web resource

GEO VISUALIZATION

A.A. 2013/14

(click on the image to start example)

GEO VISUALIZATION: election’12

A.A. 2013/14

USA TODAY APLLICATION for USA ELECTION’12

http://i.usatoday.net/news/graphics/elections-2012-app/index.html#map/president

GEO VISUALIZATION: resources

A.A. 2013/14

• Shapefiles : http://geocommons.com/ http://www.diva-gis.org/gData

http://www.naturalearthdata.com

• GIS-GeoJSON converter: http://converter.mygeodata.eu/

(online)

• Static maps (SVG format): http://d-maps.com/

OUTLINE

• Recap

• Manipulating data

• Geo visualization

• Coordinated views

• Deploy

A.A. 2013/14

COORDINATED VIEWS

A.A. 2013/14

• Usually a visualization is really good at pointing a particular aspect of the domain of analysis considered….

• …but the analyst can be interested in

studying MORE than one aspect of that domain• ….or more PROPERTIES in the context of the same analysis process

COORDINATED VIEWS: data

A.A. 2013/14

• d3 library doesn’t provide any explicit mechanism for managing multiples coordinated views….

• …but the foundation of the library is the main principle on which multiple coordinated visualizations are obtained: DATA

• By allowing all the visualizations to be built and able to modify the same shared dataset you can obtain the coordination and coherence among visualizations

• Clearly that doesn’t mean that all the visualizations represent the same data (you can assign different fields of the data set to different visualizations, with their intersection void)

COORDINATED VIEWS: data

A.A. 2013/14

General schema:

DATA

Visualization B

Visualization A

redraw()redraw()action()

The action invoked in Visualization B modify the shared data

After data modification, the redraw() functions of both visualizations are re-invoked

COORDINATED VIEWS: layout

A.A. 2013/14

• For what concerning the layout of the multi coordinated views, you can assign a different SVG AREA in the html page to a particular visualizations

• In this way, you will obtain:

• Portability of the different visualizations taken as stand-alone components

• Visbility among visualizations in the same page

• Scalability: you can add and/or remove different visualizations from the page

• A series of function called by listeners will be in charge for updating the different visualizations

• …always after the data has changed and fully loaded (or you will have inconsistency)

COORD. VIEWS: Example

A.A. 2013/14

Multi-coordinated views (1 line chart +1 time chart)

COORD. VIEWS: Example

A.A. 2013/14

var contextXScale = d3.time.scale() .range([0, contextWidth]) .domain(charts[0].xScale.domain()); var contextAxis = d3.svg.axis() .scale(contextXScale) .tickSize(contextHeight) .tickPadding(-10) .orient("bottom"); var contextArea = d3.svg.area() .interpolate("monotone") .x(function(d) { return contextXScale(d.date); }) .y0(contextHeight) .y1(0); var brush = d3.svg.brush() .x(contextXScale) .on("brush", onBrush);

Creates x-scale for the brushing graph

Creates x-axis for the brushing graph

Creates the brushing area and attaches to it the previous components

Attach to the brushing area the “brush” behavior listener

COORD. VIEWS: Example

A.A. 2013/14

function onBrush(){ /* this will return a date range to pass into the chart object */ var b = brush.empty() ? contextXScale.domain() : brush.extent(); for (var i = 0; i < countriesCount; i++){ charts[i].showOnly(b); }}

• charts[i] contains the references to the single areas of the main chart, each one tied to a particular country.

COORD. VIEWS: Example

A.A. 2013/14

Chart.prototype.showOnly = function(b){ this.xScale.domain(b); (1) this.chartContainer.select("path").data([this.chartData]).attr("d", this.area); (2) this.chartContainer.select(".x.axis.top").call(this.xAxisTop); (3) this.chartContainer.select(".x.axis.bottom").call(this.xAxisBottom); (3)}• Coordinate view is achieved by passing from the brush area a time

frame (represented by the variable b) to the main one

• b represents an initial and a final date; with it we update the new domain (1), the path representing the trends (2), and the x and y axes (3)

COORD. VIEWS: 4 views

A.A. 2013/14

Diabetes dashboard (focus+ stacked bars+ 2 time vis)

http://diabetesvis.herokuapp.com/diabetes/dashboard

COORD. VIEWS: Example

A.A. 2013/14

Occupy Wall Street (Geo + frequency)

http://jensfinnas.com/dataist/ows/

COORD. VIEWS: Example 2

A.A. 2013/14

http://keminglabs.com/ukuni/

OUTLINE

• Recap

• Manipulating data

• Geo visualization

• Coordinated views

• Deploy

A.A. 2013/14

DEPLOY

A.A. 2012/13

• Two modalities available for testing and deploying your project

Hot Deploy

• Put all files (data+html+libraries+scripts) in the same directory (or subdirectories tree)

• Load the index page into the browser

• Works only with Firefox

Web Application

• Needs a web application server (Tomcat, Glassfish, Jboss) on which deploying the application)

• Create a war for the project containing all the files and libraries needed

• Works with all Browsers

DEPLOY

A.A. 2013/14

• Be consistent with the design paradigm choice

• Bundle with the homework a document in which you clearly specify how to deploy your application

What about data…

A.A. 2013/14

• For the assignment, you’re encouraged to choose a data-set of your choice, something in which you are interested, and perform some analysis on it…..

• …just in case you’re short on data of interest, you can find a good collection of data in the Gapminder site at:

• Some of the datasets are still xlsx, so you will need to convert the file to CSV or JSON in order to import it in d3

http://www.gapminder.org/data/