NIST Big Data PWG Standardization Activitieshfoxwell/Symposium/Presentation_Chang.pdf · NIST Big...

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016 1

NIST Big Data PWG & Standardization Activities

Wo Chang, NISTDig i ta l Data Adv i so r

N IST B ig Data Pub l i c Work ing Group , Co ‐Cha i rI SO/ IEC J TC 1/WG 9 Work ing Group on B ig Data , Convenor

wchang@ni st . gov

J anua r y 7 , 2016

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016

NIST Big Data Public Working Group - Goal

2

Develop a secured reference architecture that is vendor-neutral, technology- and infrastructure-agnostic to enable any stakeholders (data scientists, researchers, etc.) to perform analytics processing for their given data sources without worrying about the underlying computing environment.

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016 3

Internet of Things

Analytics Engine

Social Media

Electronic Health Record Life Science

Others…

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016 4

Data Sources‐ Sensors‐ Simulations‐ Modeling‐ Etc.

NIST / ISO Big Data Standards – A Big Roadmap

Data Consumers‐ End users‐ Repositories ‐ Systems‐ Etc.

Data Scientist

BDRA InterfaceResource Management/Monitoring, Analytics Libraries, etc.

BDRA Ecosystem Components

Computing Resources

AnalyticsResources

Distributed File System ServicesInfrastructure Services

Database ServicesData Sources ServicesSupport Infrastructure

Value-added Content ServicesSecurity and Privacy Services

Visualization & BI ServicesAnalytics Services

Analytics Application


NIST Big Data Standardization Activities

5

Approaches to Establish Interoperable Ecosystem

NIST Big Data Public Working Group (NBD-PWG)

ISO/IEC JTC 1/WG 9 Working Group on Big Data, with collaborations:

ISO/IEC JTC 1/WG 11 – MPEG

ISO/TC 69 – Applications of Statistical Methods

ISO/TC 204 – Intelligent Transportation

Work Across Academic, Industry, and Standards to achieve interoperability

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016


5 Subgroups (July 2013 – now):

1. Definitions & Taxonomies

2. UC & Requirements

3. Security & Privacy

4. Reference Architecture

5. Standards Roadmap

V1 (high-level RA components and descriptions) Big Data Interoperability Framework:Released on September 16, 2015:http://bigdatawg.nist.gov/V1_output_docs.php

6


NIST SP1500-1: Definitions

NIST SP1500-1: Definitions

NIST SP1500-2: Taxonomies

NIST SP1500-2: Taxonomies

NIST SP1500-3: Use Cases &

Requirements

NIST SP1500-3: Use Cases &

Requirements

NIST SP1500-4: Security &

Privacy

NIST SP1500-4: Security &

Privacy

NIST SP1500-5: Architecture

Survey – White Paper

NIST SP1500-5: Architecture

Survey – White Paper

NIST SP1500-6: Reference

Architecture

NIST SP1500-6: Reference

Architecture

NIST SP1500-7: Standards Roadmap

NIST SP1500-7: Standards Roadmap

GMU Big Data Symposium, NIST Big Data Stds Activities, Wo Chang, Jan. 7, 2016


7


7CODATA Big Data Workshop, Wo Chang, NIST/ITL, June 9, 2014

Vendors Big Data architectures



8

V2 focuses on interface between NBD-RA components through use cases by:

Analyze activities diagrams

Analyze functional diagrams

Apply DevOps on small scale implementations

Goals:

Aggregate low-level interactions into high-level general interfaces

Produce set of white papers to demo how NBD-RA can be used




9

Selection of use cases: (a) available of datasets and (b) available of analytics codes

Fingerprints Matching Human and Face Detection from Video

Twitter Feeds Spatial Big Data/GIS Healthcare Payment Fraud

• Data warehousing• Global Cities

• Earth Science• Life Science

• IoT• Others…




10

NISTGlobal City

TeamsChallenges NIST

Cyber-PhysicalSystems

PWG

IEEEInternet of

Things (IoT)

NISTCloud PWG

NIST Big Data

PWG

ISOSTANDARDSSCS/WGS

JTC 1/WG10Working Group

OnInternet of

Things (IoT)

JTC 1/WG9Working Group

On Big Data

JTC 1/SC32Data

Management andInterchange

JTC 1/SC38Cloud Computing

and Distributed Platforms

JTC 1/SC27Security and

Privacy

NISTPUBLIC

WORKINGGROUPS

Explore collaboration by working with industry, academic and governments to harmonize analytic ecosystems


ISO/IEC Big Data Standardization Activities

11

ISO/IEC JTC 1/WG 9 Working Group on Big Data

130+ from 21 NBs: Australia, Austria, Brazil, Canada, China, Finland, France, Germany, Ireland, Italy, Japan, Korea, Luxembourg, Netherlands, Norway, Russian Federation, Spain, Singapore, Sweden, UK, US

Current Projects

• ISO/IEC 20546 Information technology – Big data – Overview and vocabulary

• ISO/IEC 20547 Information Technology – Big data Reference architecture (5 Parts)

Part 1: (TR) Framework and Application Process Part 2: (TR) Use Cases and Derived Requirements Part 3: (IS) Reference Architecture Part 4: (IS) Security and Privacy Fabric Part 5: (TR) Standards Roadmap

ISO/IEC Liaisons: SC 6/WG 7, SC 27, SC 29, SC 32, SC 36, SC 38, SC 39, ISO/TC 69, ISO/TC 204, ITU-T SG13



12

Identify and characterize existing multimedia Big Data deployment

Identify Big Media use cases

Identify MPEG tools relevant for Big Media

ISO/IEC JTC 1/SC 29/WG 11 (MPEG) on Big Media

Create AHG between SC 29/WG11 and WG9 to



13

ISO/TC69 – Applications of Statistical Methods

Apply standard statistical methodologies (CRISP, SEMMA, etc.)

Create AHG between TC69, WG9, and NIST Big Data PWG to:

Explore new Big Data statistical methods

Identify use cases (healthcare fraud, live twitter feeds, etc.)

Implement use cases using best practice Big Data computing ecosystem

Document findings

Standardize new Big Data statistical methodologies



14

ISO/TC204 – Intelligent Transportation

Apply standard statistical methodologies (CRISP, SEMMA, etc.)

Create AHG between TC204 and WG9 to

Review SDOs in the Big Data area particularly architecture models, semantic definitions, metadata issues and APIs

Identify “Big Data topics” needed for transport data exchange and external data sources; gather and / or generate use cases related to big data topics for ITS

Examine TC204 work that support the Big Data areas and identify the gaps to fit into the foundation / architecture currently under development by SDOs (e.g., ISO/IEC, IEEE, SAE)

Examine security, privacy, ownership, and usage issues related to Big Data ITS applications

Recommend future work items (if any) to be developed by TC204

Recommend liaisons with SDOs for which collaboration is needed


Work Across Academic, Industry, and Standards to achieve interoperability

15

Research(‐ Google log file

‐ academic)

Development

Deployment

Community/Industry/Product

BigTable(2002)

• Hadoop• Hbase, Hive

• Others…

• Cloudera Impala• IBM BigInsights

• Hortonworks HDP• Others…

Custom

er‐based

Standards D

evelop

ment

Tradition

al

Standards D

evelop

ment Focuses/Activities

• Functional Research• Apply Research• Experimental• Testbed• Best Practices• Consortium• Industry Practices/ Standards

Focuses/Activities• Functional Research• Apply Research• Experimental• Testbed• Best Practices• Consortium• Industry Practices/ Standards

Focuses/Activities• Functional Research• Apply Research• Experimental• Testbed• Best Practices• Consortium• Industry Practices/ Standards Ac

adem

ic and

R&D Labs

NIST Big Data PWG Standardization Activitieshfoxwell/Symposium/Presentation_Chang.pdf · NIST Big...

Documents

Transcript of NIST Big Data PWG Standardization Activitieshfoxwell/Symposium/Presentation_Chang.pdf · NIST Big...