1

SMA 5506Database Technology

Summer 2005

Session L18

Web Aggregators and Aggregation Research:

Prof. Stuart Madnick ( smadnick@mit.edu )

© S. Madnick & M. Siegel, 2005

2

Multiple Perspectives

Strategy and Policy

Aggregators

Global Financial Account Aggregation

Application

eCommerce & Financial Services

Technology

Web wrapper & Context mediator

2

3

Overview: Aggregation – Focus on Information Re-use and Re-purposing

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

4

Aggregator Definition

Transparently collects information

with or without prior source arrangement

resolves contextual differences

Performs analysis

not merely gather

Represents user

acts as agent for user

3

5

Aggregator Types & Examples

Investment analysis, policy analysis

Department calendars

Complementary

Financial account aggregation

Customer Relationship Management (CRM)

Relationship

Shop for best prices (e.g., books)

Benchmarking divisions

Comparison

ExternalInternalSources

Types

6

www.amazon.com

4

7

www.books.com *

* Acquired by Barnes & Nobles

8

www.mysimon.com

5

9

www.evenbetter.com *

* Now called DealTime.com Comment: Need for Mega-Aggregators

10

Mega - Aggregators

Note: multiple book price aggregators

• using differing sources

• giving differing results !

What about an aggregator of aggregators

• A Mega- Aggregator … more later …

Aggregators not just about prices ...

6

11

www.usair.com

12

www.maxmiles.com

7

13

What is MaxMiles

You give it all your frequent flyer info, it . . .

Does mileage management for you . . .

Gathers all your frequent flyer info

Sends a personalized report via email

Updates reports on web site daily

Searches for missing mileage credits

Analyzes most current milage offers

14

From US Airways site"Access to this information is subject to the rules in the Dividend Miles Membership Guide. In addition, by using this site to access your Dividend Miles account, you agree that you will use this site in a manner consistent with the Dividend Miles Membership Guide and you further agree not to allow access to this siteto any third party by revealing your access code to any third party for any reason. Failure to comply with the foregoing restrictions on the use of this site shall be grounds for the termination of your access to this site and/or your membership in the Dividend Miles program."

8

15

“Power of Attorney” on MaxMiles Site

16

Real time Tibco

Complementary AggregatorIntegrated Spreadsheet Sources

InternalExternal

9

17

Everyone is impacted:If you are not an Aggregator,

you are an Aggregatee

Books.com, MySimon.com, EvenBetter.comBarnes and Noble, Borders, Amazon, . . .

MaxMilesUnited, American, TWA, US Air, . . .

Universal Financial Aggregator (UFA)your banks, your brokers, your billers , . . .

NOTE: Aggregatees -> Aggregators

18

Overview

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

10

19

Global Comparison Aggregator

http://soursop.mit.edu/global

20

Best global online price

11

21

Global Price Comparison

USMexico

BrazilUK

FranceGermany

SpainItaly

NetherlandsDenmark

NorwaySweden

Japan500

1000

1500

2000

Pric

e of

DC

R-I

P5

($)

N= 52 22 22 18 12 8 2 3 3 6 6 16 2

22

Mega-Aggregator: Using Regional Aggregators

US Sweden France UK

12

23

Global Complementary AggregationQ: How did CO2 emissions

(total, per GDP, per capita) change over time

(between 1990 and 2000) in Yugoslavia?

User 1: YUG as a geographic region bounded before the breakupUser 2: YUG as a legal autonomous state

Related effort: - Laboratory for Information Globalization and Harmonization Technologies (LIGHT)

24

2.07162SVN

2.0608.7MKD

4.5266.9HRV

3.913.6BIH

10.61627.823.7698.3YUG

PopGDPPopGDPCountry

20001990

SITTolarSVNSlovenia

MKDDenarMKDMacedonia

HRKKunaHRVCroatia

BAMMarkaBIHBosnia and Herzegovia

YUNNew Yug. Dinar

YUGYugoslavia

CurCodeCurrencyCodeCountry

225.93SVNUSD

64.757MKDUSD

8.089HRVUSD

2.086BIHUSD

67.26710.5YUGUSD

20001990ToFrom

3981SVN

3378MKD

5405HRV

1279BIH

1548035604YUG

20001990Country

CO2 Emission

4560

282

1.28

104.8

29523

2000

2800

535

1.5

66.5

35604

1990

User 2User 1

11002800GDP/Capita

640535CO2/GDP

1.461.5CO2/capita

24.266.5GDP

1548035604CO2

20001990Country

GDP in billions local currency; Population in millions

In 1000 tons per year

Total CO2 in 1000 tons per year; GDP in billions USD; CO2/Capita in tons per person; CO2/GDP in tons per million USD; GDP/Capita in USD per person

World Bank’s World Dev. Indicator DB; UN Statistic Division; Statistics Bureaus

OAK Ridge’s CDIAC DB; WRI; GSSD; EPAs Olsen (Web)

Many sources needed:Meanings in sources & users might differ

13

25

Overview

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

26

Emerging Technologies

Web wrappers

Extract selected information from web

Allows web to be treated as large database

Context mediators

Resolve semantic (meaning) differences

Enable meaningful aggregation

& comparison

14

27

Overview

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

28

User

UFA

CitigroupCredit Cards

FidelityBrokerage

FleetBanking

BancOneCredit Cards

Concept of a Universal Financial Aggregator ( UFA )

15

29

COntext INterchange Project (context2.mit.edu/coin)

30

Universal Financial Aggregator (UFA) Demonstration

16

31

Aggregated Report produced ( top part )

32

Aggregated Report produced ( bottom part )

17

33

“Aggregation . . . The ‘killer app’ in the e-finance revolution”

http://www.tfconferences.com/conferences/AAG01/index.html

34

Current UFA Offerings (examples)

Account Aggregation Technology Suppliers• Yodlee• VerticalOne• eBalance

Account Aggregation Offerings• Chase, Fleet, Citibank (myCiti.com), . . .• Merrill-Lynch, Fidelity Investments, . . .• Yahoo, AOL, . . .

18

35

Aggregatee

Aggregatee

Aggregator

Commercial Financial Account Aggregators

36

FI #2

FI #3

FI #1

User

UFA

First Call

Excel

Advisors

Wizards

Agents

Account Analyst

Asset Allocation

Financial Sources

Other Internet Sources

Intranet Sourcesand applications

Real Value: “After Aggregation” Services

UserInformation

19

37

Advisors: Stock holdings re-aggregate

38

Wizards: Money Market (MMA) example

20

39

MMA is also an aggregator, of MM rate sources

Better rates

40

Multi- Mega- Aggregators !

The Universal Financial Aggregator (UFA) :

Aggregates your financial information

several ways: totals and stock holdings

Also uses the Money Market Aggregator

which draws on multiple MM sources

including sources that are, in fact,

aggregators of some MM sources

21

41

UFA Goals

Consumers and Financial Institutions:

• Can do what they do better

• Can do things they could not do before

• Redefine the use, role, and relationship of financial service

Business-to-Business examples also

42

Overview

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

22

43

Strategy, Legal, Cultural, and Policy Issues Regarding Aggregation

Relationships between Aggregators and AggregateesProtection of investment in data vs re-use

EU Database Directive4 proposed bills in US

Protection of consumer privacyLiability, consumer protection, jurisdiction

44

www.intershipper.net

23

45

www.iship.com

46

Aggregator “states” example

Independent Agent ( BITS, Inc. ]

Independent Aggregator

Intershipper

iShip

Dependent Aggregator/Dominant ( UPS ]

Dependent Aggregator / Consortium

Partial Collaborative Aggregator

24

47

Database Re-Use Lawsuits and Claims

Example CaseseBay vs. Bidder’s Edge

HomeStore.com vs. Bargain Network

First Union vs. PayTrust

mySimon vs. Priceman

Register.com vs. Verio

Ticketmaster vs. Tickets.com

Principles Used• Copyright infringement

• Trespass to chattels

• Breach of contract

• Misappropriation

• False advertising• Violation of the Computer

Fraud and Abuse Act• . . .

48

“ Trespass to chattels ”eBay vs. Bidder’s Edge

Injunction on May 24, 2000

Ticketmaster vs. Tickets.com

“it is hard to see how entering a publicly available web site could be called a trespass”

“ Copyright infringement ”Protects original works of authorship

Usually not applicable to data re-use

25

49

Battle over Database Re-Use -continued

New Paradigm – “sweat of the brow”Collections of Information Antipiracy Act, H.R.354 (1999) – Coble bill

Consumer and Investor Access to Information Act of 1999, H.R. Rep.1858 (1999) – Bliley bill

Two other failed bills in 1996, 1997

EU Database Directive

50

International Differences

Decentralized

Self- regulation

No DB protection law

Opt-out for privacy

Centralized

Specific, strict regulation

DB Directive

Opt-in for privacy

U.S. E.U.

Reciprocity provision• Not protected if home country does not have similar law for DB

protection

• No data flow to countries without similar privacy protection law

Harmonization is needed• Safe Harbor Agreement between US and EU

26

51

Overview

Aggregator definition, types & examples

Emerging Global Aggregation

Enabling Technologies

New Emerging Aggregators

Strategy, Legal, Cultural, and Policy Issues

Summary

52

Key Global Aggregation Insights & ImplicationsInsights• Everyone impacted

• Aggs not limited to price comparison

• Aggs can be combined and aggregated

• Aggs create new info spaces

• After-agg even more important

• New technologies emerging

• Business models and market conditions continue to change

Implications• Be part of strategy planning

• Identify new Agg applications and business opportunities

• Entirely new types of applications can be defined

• Who will own this space

• This must be exploited

• Exploit these technologies

• Must be prepared for changes

27

53

Current Related Projects ( partial list )• Aggregation strategies (e.g., Open aggregation)• Regional & Global Aggregation: Cultural & legal• Multi-enterprise eBusiness: Trust & relationships

• Financial services/Internet support for SME’s• Future Financial Services (e.g., Wireless aggregator)

• Context Mediation & Web data extraction *• Data Quality & What XML will and will not do• Internal & Industry data standards efforts• Other new technologies (e.g., Mobile aggregation)

* Patents awarded in 1999

54

For Further Information and ContactsMIT Research:

– http://context2.mit.edu and http://context2.mit.edu/aggregation

– http://web.mit.edu/tdqm

MIT Technology Licensed Software:– http://web.mit.edu/smadnick/www/patents.html

Email Contact: – Stuart Madnick at smadnick@mit.edu– Michael Siegel at msiegel@mit.edu

Research funded, in part, by Fleet Bank, Merrill Lynch, Suruga Bank, BancoSantander (BSCH), Citibank, DoD Advanced Projects Agency (DARPA), MITRE Corporation, PricewaterhouseCoopers, Singapore-MIT Alliance (SMA), MIT-Malaysia University of Science and Technology (MUST) Collaboration