Hadoop Security

84
Agenda : Securing Hadoop Deployments to Enterprise Compliance Regulations Q & A @sureshmandava

Transcript of Hadoop Security

Page 1: Hadoop Security

Agenda :

Securing Hadoop Deployments to Enterprise Compliance Regulations

Q & A

@sureshmandava

Page 2: Hadoop Security

Welcome to BigData Cloud Architects Meetup Group

August 9, 2014

Bi-Weekly, Saturday 2:00p - 4:00p

Exciting Topic Every Week, Please Register @ http://www.meetup.com/BigData-Cloud-Meetup/

2:00 - 2:15 : Socializing 2:15 - 2:30 : Meetup Introduction

2:30 - 3:45 : Topic of the week 3:45 - 4:15 : Wrap UP

Started in Jun 2013

+1 Year

Page 3: Hadoop Security

Suresh Mandava

BigData/Cloud Security Principal, CSC

24+ Years of IT Experience in Infrastructure, Platform and Data Management in Mission Critical Enterprise Environments

Founder/CTO for Network Security and a Cloud IaaS Product Companies

Page 4: Hadoop Security

Apply Common Sense, to BigDATA

SmartDATA

Page 5: Hadoop Security
Page 6: Hadoop Security

Business News

Page 7: Hadoop Security

Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing

Google shows off Mesa, a super-fast data warehouse that runs across data centers

Mesa is an ACID-compliant database

Page 8: Hadoop Security

BigTable does not provide the necessary atomicity required by Mesa applications.

While Megastore, Spanner, and F1 (all three are intended for online transaction processing) do provide

strong consistency across geo-replicated data, they do not support the peak update throughput needed by

clients of Mesa.

However, Mesa does leverage BigTable and the Paxos technology underlying Spanner for metadata storage

and maintenance.

Page 9: Hadoop Security

Elon Musk tweeted some scary thoughts about artificial intelligence over the weekend, but he might have oversold the threat.

Still, there’s plenty to worry about with advanced algorithms, which is why we need good rules about how data is used and shared.

A diagram of DARPA’s proposed deep learning system.

Page 10: Hadoop Security

Responsive Web

Web Components

www.polymer-project.org

Page 11: Hadoop Security

Technology News

Page 12: Hadoop Security

Cloudera Enterprise 5.1 is Now Available

Cloudera Enterprise’s newest release contains important new security and performance features.

Page 13: Hadoop Security

Hadoop PIG 0.13

Page 14: Hadoop Security

Oink is a gateway server to Apache Pig/Hadoop providing a REST API.

Built at eBay, it was open-sourced this week. The main design goals include governance, scalability, and change management.

Page 15: Hadoop Security

Spark and Tez, out of phase

Spark and Tez, which in most contexts likely wouldn’t be compared, suddenly find themselves competitors.

Both of them pave the way for MapReduce’s diminished influence, and for interactive Hadoop to move to the mainstream.

Page 16: Hadoop Security
Page 17: Hadoop Security

Agenda :

Securing Hadoop Deployments to Enterprise Compliance Regulations

Q & A

@sureshmandava

Page 18: Hadoop Security

Data is to information society what fuel was to the industrial economy: the critical resource powering the innovations that

people rely on.Just as countries’ fuel repositories need protection and security

because they can come under attack, so do companies’ big data repositories.

“Companies, markets, and countries are increasingly under attack from cyber-criminals.They need to get much better at protecting [and securing] themselves”

Page 19: Hadoop Security

How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did

As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction”

score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.

Page 20: Hadoop Security

Target CEO Fired - Can You Be Fired If Your Company Is Hacked?

Target’s CEO Gregg Steinhafel, a 35-year employee of the company with the last six at the helm, resigned in light of the recent holiday-

season credit-card security breach that affected 40 million customers.

Page 21: Hadoop Security
Page 22: Hadoop Security
Page 23: Hadoop Security
Page 24: Hadoop Security
Page 25: Hadoop Security
Page 26: Hadoop Security
Page 27: Hadoop Security
Page 28: Hadoop Security
Page 29: Hadoop Security
Page 30: Hadoop Security
Page 31: Hadoop Security
Page 32: Hadoop Security
Page 33: Hadoop Security
Page 34: Hadoop Security
Page 35: Hadoop Security
Page 36: Hadoop Security
Page 37: Hadoop Security
Page 38: Hadoop Security
Page 39: Hadoop Security
Page 40: Hadoop Security
Page 41: Hadoop Security
Page 42: Hadoop Security
Page 43: Hadoop Security
Page 44: Hadoop Security
Page 45: Hadoop Security
Page 46: Hadoop Security
Page 47: Hadoop Security
Page 48: Hadoop Security
Page 49: Hadoop Security
Page 50: Hadoop Security
Page 51: Hadoop Security
Page 52: Hadoop Security
Page 53: Hadoop Security
Page 54: Hadoop Security
Page 55: Hadoop Security
Page 56: Hadoop Security
Page 57: Hadoop Security
Page 58: Hadoop Security
Page 59: Hadoop Security
Page 60: Hadoop Security
Page 61: Hadoop Security
Page 62: Hadoop Security
Page 63: Hadoop Security
Page 64: Hadoop Security
Page 65: Hadoop Security
Page 66: Hadoop Security

Why LUKS?

• compatibility via standardization, • secure against low entropy attacks, • support for multiple keys, • effective passphrase revocation, • free

Page 67: Hadoop Security
Page 68: Hadoop Security
Page 69: Hadoop Security
Page 70: Hadoop Security
Page 71: Hadoop Security
Page 72: Hadoop Security
Page 73: Hadoop Security
Page 74: Hadoop Security
Page 75: Hadoop Security
Page 76: Hadoop Security
Page 77: Hadoop Security
Page 78: Hadoop Security
Page 79: Hadoop Security
Page 80: Hadoop Security
Page 81: Hadoop Security

Next SessionHadoop Compliance Security

August 9th 2:00pm-4:00pm

Page 82: Hadoop Security

Feel free to reach me out.

@sureshmandava [email protected]

Page 83: Hadoop Security
Page 84: Hadoop Security

Enter Feedback @ Group Reviews

Thank you