Mashups: The new breed of Web Applications

Umut Orhan

Outline

• The term “Mashup”

• Mashups Genres

• Technologies behind Mashups

• Emerging Technical and Social Challenges

History - Information Technology Spend “had” been growing nicely

1964: S/360 debuts

1971: First Intel Micro

1981: IBM PC

1994: Netscape Navigator

2000: Dot-com collapse

Dot.com Collapse - Is it the end?

Age of Information and Telecommunications

Irruption

Innovation

The Industrial Revolution

Age of Steam and Railways

Age of Steel, Electricityand Heavy Engineering

Age of Oil, Automobilesand Mass Production

Frenzy Synergy Maturity

Panic1797

Depression1893

Dot.comCollaps

e

• Formation of Mfg. industry• Repeal of Corn Laws opening

trade

• Standards on gauge, time• Catalog sales companies • Economies of scale

Current period ofInstitutional Adjustment

• Separation of savings, investment banks

• FDIC, SEC

• Build-out of Interstate highways

• IMF, World Bank, BIS

1

2

3

4

5

Panic1847

1771

1829

1875

1908

1971

1873

1920

1974

1829

Source: “Technological Revolutions and Financial Capital, Carlota Perez, 2002

Deployment

Crash1929

Crash

Just the beginning of a “golden” age

Mashups are…• … an exciting genre of interactive Web

applications that– Retrieve content from external data

sources– Create entirely new and innovative

services– Hallmark of the Web 2.0– Run-time integration

The term Mashup

• Borrowed from the pop music scene– Vocal and instrumental sounds from different

source tracks are mixed to form a new “mashup” song

• An unusual or innovative composition of content (often from unrelated data sources)– Made for human (rather than computerized)

consumption

Current StatusFrom

programmableweb.com on 1/12/2006

What might a Mashup look like?

• The ChicagoCrime.org Web site– A mapping mashup– One of the first popular mashups– Mashes crime data from the Chicago

Police Department’s online database with cartography from Google Maps

– The concept and the presentation are simple and the composition of crime and map data is visually powerful

Outline

• The term “Mashup”

• Mashups Genres

• Technologies behind Mashups

• Emerging Technical and Social Challenges

Mashups

• A Mashup web site is characterized by the way in which it spreads roots across the Web– Building upon the content and functionality

retrieved from external data source– Mostly the data source lay outside of the

organizational boundaries

Mashup Genres

• Mapping mashups

• Video and photo mashups

• Search and Shopping mashups

• News mashups

Mapping Mashups• Locational information presented graphically

using maps in a specified context• Google Maps API opened the floodgates;

mash all sort of data from nuclear disasters to Boston’s CowParade cows onto maps

• Other APIs– Microsoft (Virtual Earth)– Yahoo (Yahoo Maps)– AOL (MapQuest)

Video and Photo Mashups• The emergence of photo/video hosting and

social networking sites such as Flickr with APIs and YouTube has led to a variety of interesting mashups.

• Metadata associated with the hosted images and videos– Who took the picture– What it is a picture of– Where and when it was taken– So on…

Search and Shopping Mashups

• Exist long before the term Mashup was coined– Combinations of B2B technologies or

screen-scraping to aggregate comparative price data

• eBay and Amazon have released APIs for programmatically accessing their content

News Mashups• News sources such as BBC and Reuters

have used syndication technologies like RSS and Atom since 2002.

• Personalized newspaper by Syndication feed mashups

• Doggdot.us, combines feeds from the techie-oriented news sources Digg.com, Slashdot.org and Del.icio.us

Outline

• The term “Mashup”

• Mashups Genres

• Technologies behind Mashups

• Emerging Technical and Social Challenges

The architecture

• A mashup architecture comprimes of 3 logically and physically disjoint components– API/Content Provider– Mashup Site– Client’s Web Browser

API/Content Exposition

• Web Protocols– REST Services– SOAP Web Services– RSS/Atom

• Screen Scraping

Mashup Site

• Mashup is hosted by a web site– Mashup logic resides in this site– Execution is distributed over the web

• The Mashup Site can be implemented and deployed similarly to traditional Web application

Client-Side Processing• Mashed content can be generated directly within the

client’s browser through client-side scripting (JavaScript) or applets.

• Rich Internet Applications• Less overhead on behalf of the mashup server

– Data can be retrieved directly from the content provider• More seamless user-experience

– Pages can request updates for portions of their content without having to refresh the entire page

• Client’s Web Browser is where the application is rendered graphically and where the user interaction takes place

AJAX• Asynchronous JavaScript and XML• Web application model rather than a specific

technology• Comprises;

– XHTML, CSS and XSL for presentation– DOM API exposed by API for dynamic display and

interaction– Asynchronous data exchange, XML or JSON data– JavaScript

• Aims to create a smooth, cohesive Web experience with reducing server-side overloads

Web Protocols• As part of the SOA paradigm, these

protocols are platform neutral to interact with any remote services

• Exchanging XML based messages in general

• Messages are typically conveyed over standard transports such as HTTP and SMTP

Web Protocols: SOAP• Simple Object Access Protocol• Fundamental technology behind Web

Services• SOAP APIs for Web Services described by

WSDL documents– What operations a service expose– The format for the messages that it accepts

(using XML Schema)– How to address the service

Web Protocols: REST

• Representational State Transfer

• Similar to document-literal style of SOAP

• Identification of resources by URIs

• Supports less operations (POST, GET, PUT, DELETE)

Screen Scraping

• Scraping is the process of using software tools to parse and analyze content that was originally written for human consumption in order to extract semantic data structures representative of that information that can be used and manipulated programmatically.

Semantic Web and RDF• To create web infrastructure that augments

data with metadata to give it meaning for automation, integration, reasoning and re-use

• Resource Description Framework (RDF) provides methodologies to establish syntactic structures that describe data

• RDF allows creation of hierarchical structures of knowledge that can be searched and formally reasoned about, ontologies

Semantic Web and RDF

Reasoner

animalType: Carnivore

animalType: Carnivore

animalType:

animalType:

eats?

eats?

lives in

lives in lives in

lives in

eats

eats

RSS and ATOM• XML-based syndication formats• Syndication implies that a Web site that wants to

distribute content creates an RSS or ATOM document and registers the document with an RSS publisher.

• RSS/ATOM-enabled client can check the publisher’s feed for new content and react to it in an appropriate manner.

• Syndication technologies are useful for mashups that aggregate event-based or update-driven content

Outline

• The term “Mashup”

• Mashups Genres

• Technologies behind Mashups

• Emerging Technical and Social Challenges

Data Integration Challenges• According to surveys; number one enterprise IT

concern today is data integration within the enterprise virtual organization

• Virtual Organization: a composition of federated business units, each contained within its own administrative domain

• Mashup developers are faced with challenges of deriving shared semantic meaning between heterogeneous data sets.

• Mashup developers’ challenges are similar to integration challenges faced by enterprise IT.

• Translator/Mapper systems between different data models must be designed.

Data Pollution

• Many mashups solicit public user input– it can be quite powerful because it enables

open contribution and best-of-breed data evolution

– it can be subject to inconsistent, incorrect or intentionally misleading data entry

Social Challenges

• Protection of intellectual property and consumer privacy versus fair-use and the free flow of information

In general…

• Publishing and Discovering External Data Services other than SOAP-based Web Services

• Semantically annotation of data services

• Security leaks (e.g. openness to injection attacks)

In general…• Client-side processing is not suitable for

handling complex queries such as “Show me the average purchase price for real estate bought by actors who have co-starred in movies with Brad Pitt”

• Interoperability among different external data providers

• Standard formats for persistent data and exchanged messages

Thank you for your attention…