Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing...

58
Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States See http://creativecommons.org/licenses/by-nc-sa/3.0/us/ for details CS 451/651 (Fall 2018) Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo October 16, 2018 These slides are available at http://lintool.github.io/bigdata-2018f/

Transcript of Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing...

Page 1: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Data-Intensive Distributed Computing

Part 5: Analyzing Relational Data (1/3)

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United StatesSee http://creativecommons.org/licenses/by-nc-sa/3.0/us/ for details

CS 451/651 (Fall 2018)

Jimmy LinDavid R. Cheriton School of Computer Science

University of Waterloo

October 16, 2018

These slides are available at http://lintool.github.io/bigdata-2018f/

Page 2: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Structure of the Course

“Core” framework features and algorithm design

Anal

yzin

gTe

xt

Anal

yzin

gG

raph

s

Anal

yzin

gRe

latio

nal D

ata

Data

Min

ing

Page 3: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Evolution of Enterprise Architectures

Next two sessions: techniques, algorithms, and optimizations for relational processing

Page 4: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

MonolithicApplication

users

Page 5: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Page 6: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Source: Wikipedia

Page 7: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

database

Why is this a good idea?

Page 8: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

An organization should retain data that result from carrying out its mission and exploit those data to generate insights that benefit the organization, for example, market analysis, strategic planning, decision making, etc.

Duh!?

Business Intelligence

Page 9: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

database

BI tools

analysts

Page 10: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

database

BI tools

analysts

Why is myapplication so slow?

Why does my analysis take so long?

Page 11: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Database Workloads

OLTP (online transaction processing)Typical applications: e-commerce, banking, airline reservations

User facing: real-time, low latency, highly-concurrentTasks: relatively small set of “standard” transactional queries

Data access pattern: random reads, updates, writes (small amounts of data)

OLAP (online analytical processing)Typical applications: business intelligence, data mining

Back-end processing: batch workloads, less concurrencyTasks: complex analytical queries, often ad hoc

Data access pattern: table scans, large amounts of data per query

Page 12: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

OLTP and OLAP Together?

Downsides of co-existing OLTP and OLAP workloadsPoor memory management

Conflicting data access patternsVariable latency

Solution?

users and analysts

Page 13: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Source: Wikipedia (Warehouse)

Build a data warehouse!

Page 14: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

OLTP database for user-facing transactions

OLAP database for data warehousing

What’s special about OLTP vs. OLAP?

Page 15: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Customer Billing

OrderInventory

OrderLine

A Simple OLTP Schema

Page 16: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Dim_Customer

Dim_Date

Dim_ProductFact_Sales

Dim_Store

Stars and snowflakes, oh my!

A Simple OLAP Schema

Page 17: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

ETL

TransformData cleaning and integrity checking

Schema conversionField transformations

When does ETL happen?

Extract

Load

Page 18: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

My data is a day old… Meh.

Page 19: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

Frontend

Backend

users

Frontend

Backend

external APIs

OLTP database

OLTP database

Page 20: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What do you actually do?

Dashboards

Report generation

Ad hoc analyses

Page 21: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

store

prod

uct

time

slice and dice

Common operations

roll up/drill down

pivot

OLAP Cubes

Page 22: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

OLAP Cubes: Challenges

Fundamentally, lots of joins, group-bys and aggregationsHow to take advantage of schema structure to avoid repeated work?

Cube materializationRealistic to materialize the entire cube?If not, how/when/what to materialize?

Page 23: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

Frontend

Backend

users

Frontend

Backend

external APIs

OLTP database

OLTP database

Page 24: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Fast forward…

Page 25: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

“On the first day of logging the Facebook clickstream, more than 400 gigabytes of data was collected. The load, index, and aggregation processes for this data set really taxed the Oracle data warehouse. Even after significant tuning, we were unable to aggregate a day of clickstream data in less than 24 hours.”

Jeff Hammerbacher, Information Platforms and the Rise of the Data Scientist. In, Beautiful Data, O’Reilly, 2009.

Page 26: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

Facebook context?

Page 27: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

“OLTP”

Adding friendsUpdating profilesLikes, comments…

Feed rankingFriend recommendationDemographic analysis…

Page 28: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

analysts

ETL(Extract, Transform, and Load)

“OLTP” PHP/MySQL

data scientists✗Hadoop

or ELT?

Page 29: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What’s changed?

Dropping cost of disksCheaper to store everything than to figure out what to throw away

5 MB hard drive in 1956

Page 30: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What’s changed?

Dropping cost of disksCheaper to store everything than to figure out what to throw away

Rise of social media and user-generated contentLarge increase in data volume

Growing maturity of data mining techniquesDemonstrates value of data analytics

Types of data collectedFrom data that’s obviously valuable to data whose value is less apparent

Page 31: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

a useful service

analyze user behavior to extract insights

transform insights into action

$(hopefully)

Google. Facebook. Twitter. Amazon. Uber.

Virtuous Product Cycle

Page 32: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What do you actually do?

Dashboards

Report generation

Ad hoc analyses“Descriptive”“Predictive”

Data products

Page 33: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

a useful service

analyze user behavior to extract insights

transform insights into action

$(hopefully)

Google. Facebook. Twitter. Amazon. Uber.

data sciencedata products

Virtuous Product Cycle

Page 34: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

“On the first day of logging the Facebook clickstream, more than 400 gigabytes of data was collected. The load, index, and aggregation processes for this data set really taxed the Oracle data warehouse. Even after significant tuning, we were unable to aggregate a day of clickstream data in less than 24 hours.”

Jeff Hammerbacher, Information Platforms and the Rise of the Data Scientist. In, Beautiful Data, O’Reilly, 2009.

Page 35: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

data scientists

ETL(Extract, Transform, and Load)

“OLTP”

Hadoop

Page 36: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

ETL(Extract, Transform, and Load)

Hadoopreduce

reduce

reduce

map

map

map

…

…

reducereduce

reduce

mapmap

map

…

…

reducereducereduce

mapmapmap…

…

reduce

reduce

reduce

map

map

map

…

…

Wait, so why not use a database to begin with?

The Irony…

“OLTP”

data scientists

Page 37: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Why not just use a database?

Scalability. Cost.

SQL is awesome

Page 38: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Databases are great…If your data has structure (and you know what the structure is)

If you know what queries you’re going to run ahead of timeIf your data is reasonably clean

Databases are not so great…If your data has little structure (or you don’t know the structure)

If you don’t know what you’re looking forIf your data is messy and noisy

Page 39: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

“there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are unknown unknowns – the ones we don't know we don't know…” – Donald Rumsfeld

Source: Wikipedia

Page 40: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Databases are great…If your data has structure (and you know what the structure is)

If you know what queries you’re going to run ahead of timeIf your data is reasonably clean

Databases are not so great…If your data has little structure (or you don’t know the structure)

If you don’t know what you’re looking forIf your data is messy and noisy

Known unknowns!

Unknown unknowns!

Page 41: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Don’t need to know the schema ahead of time

Many analyses are better formulated imperatively

Raw scans are the most common operations

Much faster data ingest rate

Advantages of Hadoop dataflow languages

Page 42: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What do you actually do?

Dashboards

Report generation

Ad hoc analyses“Descriptive”“Predictive”

Data products

Which are known unknowns and

unknown unknowns?

Page 43: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

Frontend

Backend

users

Frontend

Backend

external APIs

OLTP database

OLTP database

Page 44: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Frontend

Backend

users

Frontend

Backend

external APIs

“Traditional”BI tools

SQL on Hadoop

Othertools

Data Warehouse“Data Lake”

data scientists

OLTP database

ETL(Extract, Transform, and Load)

OLTP database

OLTP database

Page 45: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Twitter’s data warehousing architecture (circa 2012)

Page 46: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

circa ~2010~150 people total

~60 Hadoop nodes~6 people use analytics stack daily

circa ~2012~1400 people total

10s of Ks of Hadoop nodes, multiple DCs10s of PBs total Hadoop DW capacity

~100 TB ingest dailydozens of teams use Hadoop daily

10s of Ks of Hadoop jobs daily

Page 47: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

How does ETL actually happen?

Twitter’s data warehousing architecture (circa 2012)

Page 48: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Scribe Daemons(Production Hosts)

Main HadoopDW

Main Datacenter

Staging Hadoop Cluster

HDFS

ScribeAggregators

Scribe Daemons(Production Hosts)

Datacenter

Staging Hadoop Cluster

HDFS

ScribeAggregators

Scribe Daemons(Production Hosts)

Datacenter

Staging Hadoop Cluster

HDFS

ScribeAggregators

Importing Log Data

Page 49: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

What’s Next?Two developing trends…

Page 50: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

database

BI tools

analysts

Page 51: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

BI tools

analysts

ETL(Extract, Transform, and Load)

Data Warehouse

OLTP database

Frontend

Backend

users

Frontend

Backend

external APIs

OLTP database

OLTP database

Page 52: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Frontend

Backend

users

Frontend

Backend

external APIs

“Traditional”BI tools

SQL on Hadoop

Othertools

Data Warehouse“Data Lake”

data scientists

OLTP database

ETL(Extract, Transform, and Load)

OLTP database

OLTP database

My data is a day old… I refuse to

accept that!

Page 53: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

ETLOLAPOLTP

What if you didn’t have to do this?

Page 54: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

HTAP

Hybrid Transactional/Analytical Processing (HTAP)

Coming back full circle?

Page 55: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Frontend

Backend

users

Frontend

Backend

external APIs

“Traditional”BI tools

SQL on Hadoop

Othertools

Data Warehouse“Data Lake”

data scientists

OLTP database

ETL(Extract, Transform, and Load)

OLTP database

OLTP database

Page 56: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Frontend

Backend

users

Frontend

Backend

external APIs

“Traditional”BI tools

SQL on Hadoop

Othertools

Data Warehouse“Data Lake”

data scientists

HTAP database

ETL(Extract, Transform, and Load)

HTAP database

HTAP database

Analyticstools

data scientists

Analyticstools

data scientists

Page 57: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Frontend

Backend

users

Frontend

Backend

users

Frontend

Backend

external APIs

“Traditional”BI tools

SQL on Hadoop

Othertools

Data Warehouse“Data Lake”

data scientists

ETL(Extract, Transform, and Load)

Everything In the cloud!

IaaS / Load balance aaS

OLTP database

OLTP database

OLTP database

DBaaS (e.g., RDS)

DBaaS (e.g., RedShift)

S3

“Cloudified” tools

ELT aaS

Page 58: Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 5: Analyzing Relational Data (1/3) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Source: Wikipedia (Japanese rock garden)


Data-Intensive Distributed Computing › ~cs451 › slides › big-data-part0… · Comparison Function Index online offline Abstract IR Architecture. one ... delta compression, gap

Data-Intensive Distributed Computing › ~cs451 › slides › big-data-part0… · Comparison Function Index online offline Abstract IR Architecture. one ... delta compression, gap

Building an Encrypted, Distributed, and Searchable Key ...qian/papers/AsiaCCS16.pdf · for data-intensive computing and cloud-based applications. Among those distributed data stores,

Building an Encrypted, Distributed, and Searchable Key ...qian/papers/AsiaCCS16.pdf · for data-intensive computing and cloud-based applications. Among those distributed data stores,

Distributed Cluster Computing Platforms. Outline What is the purpose of Data Intensive Super Computing? MapReduce Pregel Dryad Spark/Shark Distributed.

Distributed Cluster Computing Platforms. Outline What is the purpose of Data Intensive Super Computing? MapReduce Pregel Dryad Spark/Shark Distributed.

Distributed Energy-Efficient Scheduling for Data-Intensive ...xqin/pubs/ipccc08.pdf · effective, have been commonly applied in distributed data centers [15][12]. ... of massive data

Distributed Energy-Efficient Scheduling for Data-Intensive ...xqin/pubs/ipccc08.pdf · effective, have been commonly applied in distributed data centers [15][12]. ... of massive data

Integrated e-Infrastructure for Distributed, Data-driven, Data- intensive High Performance Computing: Biomedical Requirements Peter V Coveney Centre for.

Integrated e-Infrastructure for Distributed, Data-driven, Data- intensive High Performance Computing: Biomedical Requirements Peter V Coveney Centre for.

DiscFinder: A Data-Intensive Scalable Cluster Finder for ...jclopez/ref/discfinder-tr.pdf · DiscFinder is a scalable, distributed, data-intensive group finder for analyzing observation

DiscFinder: A Data-Intensive Scalable Cluster Finder for ...jclopez/ref/discfinder-tr.pdf · DiscFinder is a scalable, distributed, data-intensive group finder for analyzing observation

NetLogger Using NetLogger for Distributed Systems Performance Analysis of the BaBar Data Analysis System Data Intensive Distributed Computing Group Lawrence.

NetLogger Using NetLogger for Distributed Systems Performance Analysis of the BaBar Data Analysis System Data Intensive Distributed Computing Group Lawrence.

WANalytics: Geo-Distributed Analytics for a Data Intensive ...

WANalytics: Geo-Distributed Analytics for a Data Intensive ...

Greener Big Data: Optimizing Data Exchange and Power ... · fashion. In particular, the rise of the cloud and distributed data-intensive (\big data") applications puts pressure on

Greener Big Data: Optimizing Data Exchange and Power ... · fashion. In particular, the rise of the cloud and distributed data-intensive (\big data") applications puts pressure on

Hadoop: A Framework for Data- Intensive Distributed Computing

Hadoop: A Framework for Data- Intensive Distributed Computing

Foundations of data-intensive science: Technology and practice for high throughput, widely distributed, data management and analysis systems William Johnston.

Foundations of data-intensive science: Technology and practice for high throughput, widely distributed, data management and analysis systems William Johnston.

Hadoop: A Framework for Data- Intensive Distributed Computingcs561/s12/Lectures/6/Hadoop.pdf · Hadoop: A Framework for Data-Intensive Distributed Computing CS561-Spring 2012 WPI,

Hadoop: A Framework for Data- Intensive Distributed Computingcs561/s12/Lectures/6/Hadoop.pdf · Hadoop: A Framework for Data-Intensive Distributed Computing CS561-Spring 2012 WPI,

A Parallel R Framework - Data-Intensive Distributed ...datasys.cs.iit.edu/events/DataCloud2013/datacloud_haolin.pdf · R List – Most general data structure • Collection to store

A Parallel R Framework - Data-Intensive Distributed ...datasys.cs.iit.edu/events/DataCloud2013/datacloud_haolin.pdf · R List – Most general data structure • Collection to store

CHAIO: Enabling HPC Applications on Data-Intensive File ...thakur/papers/chaio.pdf · Keywords-high-perfomrance computing, MapReduce, data-intensive, distributed file system I. INTRODUCTION

CHAIO: Enabling HPC Applications on Data-Intensive File ...thakur/papers/chaio.pdf · Keywords-high-perfomrance computing, MapReduce, data-intensive, distributed file system I. INTRODUCTION

Data-Intensive Distributed Computing - RoegiestData-Intensive Distributed Computing Part 8: Analyzing Graphs, Redux (1/2) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Data-Intensive Distributed Computing - RoegiestData-Intensive Distributed Computing Part 8: Analyzing Graphs, Redux (1/2) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 6: Data Mining (1/4) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

Data-Intensive Distributed Computing · Data-Intensive Distributed Computing Part 6: Data Mining (1/4) This work is licensed under a Creative Commons Attribution-Noncommercial-Share

The Quest for Scalable Support of Data-Intensive …people.cs.uchicago.edu/~iraicu/publications/2009_HPDC09...The Quest for Scalable Support of Data-Intensive Workloads in Distributed

The Quest for Scalable Support of Data-Intensive …people.cs.uchicago.edu/~iraicu/publications/2009_HPDC09...The Quest for Scalable Support of Data-Intensive Workloads in Distributed

Intensive Data Management in Parallel Systems: A …ranger.uta.edu/~iahmad/journal-papers/[J25] Intensive...Distributed and Parallel Databases, 7, 383–414 (1999) c 1999 Kluwer Academic

Intensive Data Management in Parallel Systems: A …ranger.uta.edu/~iahmad/journal-papers/[J25] Intensive...Distributed and Parallel Databases, 7, 383–414 (1999) c 1999 Kluwer Academic

Aspects of Data-Intensive Cloud Computing€¦ · Aspects of Data-Intensive Cloud Computing Sebastian Frischbier and Ilia Petrov Databases and Distributed Systems Group Technische

Aspects of Data-Intensive Cloud Computing€¦ · Aspects of Data-Intensive Cloud Computing Sebastian Frischbier and Ilia Petrov Databases and Distributed Systems Group Technische

Nebula: Distributed Edge Cloud for Data-Intensive …dcsg.cs.umn.edu/Projects/Nebula/posters/Nebula-Poster...Nebula for aggregation and decomposition. Expand the range of data-intensive

Nebula: Distributed Edge Cloud for Data-Intensive …dcsg.cs.umn.edu/Projects/Nebula/posters/Nebula-Poster...Nebula for aggregation and decomposition. Expand the range of data-intensive