Analyze billions of records on Salesforce App Cloud with BigObject

35
Unleash the Potential of Big Data BigObject Webinar December 2015 Jonathan Bruce Director Product Management, Big Data – AppCloud @jonbruce Pat Patterson Developer Evangelist Architect @metadaddy

Transcript of Analyze billions of records on Salesforce App Cloud with BigObject

Page 1: Analyze billions of records on Salesforce App Cloud with BigObject

Unleash the Potential of Big DataBigObject WebinarDecember 2015

Jonathan BruceDirector Product Management, Big Data – AppCloud@jonbruce

Pat PattersonDeveloper Evangelist Architect@metadaddy

Page 2: Analyze billions of records on Salesforce App Cloud with BigObject

Forward Looking StatementSafe harbor statement under the Private Securities Litigation Reform Act of 1995:This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

Page 3: Analyze billions of records on Salesforce App Cloud with BigObject

“By now, you’ve heard the message — turn more data into business insights — and your business has seen the potential in big data to deliver it. By Q2 2016, 65% of data and analytics decision-makers say their firms will have implemented big data solutions, and some part of the 26% who were interested with no immediate plans in 2015 will move into early implementation phases”

“The Integration Imperative of Digital Experiences, Forrester Research, August 2015”

2015 is about Big Data Execution

Page 4: Analyze billions of records on Salesforce App Cloud with BigObject

CustomersCreatingMassiveAmountsofData

90%of the world’s data created in

last 12 months

Posts

Leads

Apps

Profiles

Usage

Events

VisitsProducts

Repairs

Cases

Page 5: Analyze billions of records on Salesforce App Cloud with BigObject

Big Data to Become the Primary Driver for Data Archive

“By 2016, 75% of structured data archiving will incorporate support for Big Data analytics and by 2017, archiving support of Big Data Analytics will surpass archiving for compliance as the primary use case for structured data archiving” - Gartner

Source: “Magic Quadrant Structured Data Archive Application & Retirement” - Gartner

Page 6: Analyze billions of records on Salesforce App Cloud with BigObject

Today: Understanding Your Customer

ERP HCM SCM Logs

1. Acquire Data

2. Wrangle Data(Cleanse, Augment, Transform, Join)

Data Lake / EDW

4. Take ActionCustomer Success Platform

3. AnalyzeWave

Firewall

Page 7: Analyze billions of records on Salesforce App Cloud with BigObject

Future: Understanding Your Customer

ERP HCM SCM Logs

4. Take ActionSalesforce Apps

3. AnalyzeWave

Firewall

2. Wrangle DataAsync SOQL

1. Acquire DataBigObjects

Wav

e To

olin

g

Page 8: Analyze billions of records on Salesforce App Cloud with BigObject

The Data-Driven App Cloud

AsyncSOQL

Wave Data Enrichment

Sales Service Marketing AppCloud ISV

1 Data Acquisition

2 Reduce Data

3 Visualize Analyze, Understand

4 Take Action

DataBigObject, SObject, External Objects

Page 9: Analyze billions of records on Salesforce App Cloud with BigObject

Customer 360

Data Retention

Event Capture

BigObject Top Use Cases

Page 10: Analyze billions of records on Salesforce App Cloud with BigObject

BigObjects let you store manage billions of records natively

BigObject

BigObject Storage and Services

SOQL Async SOQLAPI Bulk API

Data ArchiveNear-line storage for policy driven data retention

Object query language Resilient query

Page 11: Analyze billions of records on Salesforce App Cloud with BigObject

BigObjects means 100s of billions of records on AppCloud

Based on Apache HBase, powered by Apache Phoenix

• BigObject is the AppCloud abstraction of our trusted linear scalable a NOSQL data store

• Fully available across all production & sandbox instances

Data persistence optimized for high-volume data• Geared for 1, 10, 100s of billions of records• Targeted Immutable data – think archive data,

events, external data, historical data, loyalty, logs….• No Transactions, Eventual Consistency

High-volume storage for AppCloud - reliable, highly-available & secure

BigObject

Page 12: Analyze billions of records on Salesforce App Cloud with BigObject

BigObjects brings familiar look and feel

Familiar, object-based development model• Familiar and new data types – string,

number, date, JSON (in future)• Relationships - lookup, master-detail• Exposed in SOAP, REST, Bulk, and Metadata

APIs

Data scale means new services and different contracts• Synchronous & Asynchronous SOQL • Reports, Dashboard & Search not

supported

High throughput Ingress & Egress• New Bulk API Implementation geared for

1billion record/day ingest

High-volume storage for AppCloud - reliable, highly-available & secure

BigObject

Page 13: Analyze billions of records on Salesforce App Cloud with BigObject

BigObject and SOQL

WhyConstrainedSOQL?• #Field Audit Trail mandatesbyFieldHistoryType,

ParentId&CreatedDate,andinthatorder• CustomBigObjectcanfilteronIDonly

Future: CustomKeyPrefixes&SecondaryIndexes• Newdesignpatternaroundcustomkeyprefixesthat

requiresthinkingthruaccesspatterns• Enablescomplex&outoforderfilters

Data scale means highly targeted SOQL only

Page 14: Analyze billions of records on Salesforce App Cloud with BigObject

Introducing: Asynchronous SOQL

Based on Apache Hadoop, powered by Salesforce

Aggregation, Filtering of Across All Records• SOQL-style queries across any object• Works across SObject, BigObject and External

Objects• Seamless relationships to SObjects extending

the customer data models

Exposed via RESTful API • New async-query end-point to issues

queries• Capable Multiple parallel queries• Directed ResultSet Targets• Target field(s) in SObject, BigObject and

External Objects

Reliable, Resilient & Federated SOQL

Page 15: Analyze billions of records on Salesforce App Cloud with BigObject

Source Query

Anatomy of a Asynchronous SOQL SOQL Source, Mapping and Object Destination{

"query": "SELECT End_Time__c,Start_Time__c,Uber_Type__c,

Rider__r.FirstName, Rider__r.LastName, Rider__r.Email

FROM Rider_Record1__b WHERE Star_Rating__c = '5'",

"targetObject": “Rider_Reduced__b",

"targetFieldMap": {

"End_Time__c": "End_Time__c",

"Start_Time__c": "Start_Time__c",

"Uber_Type__c": ”Car_Type__c",

"Rider__r.FirstName": "First_Name__c",

"Rider__r.LastName": "Last_Name__c",

"Rider__r.Email": "Rider_Email__c"

}

}

Target Object

Field Mappings

Page 16: Analyze billions of records on Salesforce App Cloud with BigObject

Asynchronous SOQL

Support for aggregate functions: count(field), avg(), sum(), more to follow

SOQL Syntax group-by, like, having (coming Spring ‘16)

Where ClauseComparison operators for TEXT and NUMBER fields only

=, !=, <, <=, >, >-, >=Logical operators

AND, OR

Relationship QueriesChild-to-parent relationships are supported using the dot notation. SELECT, WHERE and GROUP BY clauses Single-level Only

SOQL Coverage To Date

Page 17: Analyze billions of records on Salesforce App Cloud with BigObject

Introducing: Asynchronous SOQL

New Limits• New query & result size limits model• Number of Parallel queries

Future Enhancements include• Apex driven SOQL queries supported in the

future• Additional grouping & aggregates – let us

know!

Reliable, Resilient & Federated SOQL

Page 18: Analyze billions of records on Salesforce App Cloud with BigObject

2: Data Reduction & Processing

SObjects

BigObjects

Wave Data Sets

External Objects

Files

Archive Objects

SObjects

BigObjects

Wave Data Sets

External Objects

Files

Archive Objects

Parallel Processing JobsMulti-tenancy Hadoop, resource management, scheduling, job monitoring and management

SOQL API and Apex interface for data filtering, grouping and aggregates

Data Sources

Data Processing

Data Targets

Async SOQL

Page 19: Analyze billions of records on Salesforce App Cloud with BigObject

3: Wave Driven Data Exploration, Insights & Analysis CloudSpeed&ScaleCompletecustomerinsights

acrossyourbusiness

Search-BasedQueryKey-value-pairandinvertedindexesforspeed&flexibility

DataDrivenConversationViadesktopormobilewhilefullycollaborative

Page 20: Analyze billions of records on Salesforce App Cloud with BigObject

4: Data Driven Action on InsightsFastActiononYourInsightsCompletecustomerinsightsacrossyourbusiness

Enrichforce.comObjectsApplyinsightstoyourcoreforce.comentities

IntelligentProcessesTurninsightsintoactionbydrivingbusinessprocesswithnewcustomerKPIsandmetrics

Page 21: Analyze billions of records on Salesforce App Cloud with BigObject

Demo & Code ExamplesDataArchive,Customer360&Reports&DashboardswithBigObjectdata

Page 22: Analyze billions of records on Salesforce App Cloud with BigObject

Active PilotsStillacceptingparticipants!

Page 23: Analyze billions of records on Salesforce App Cloud with BigObject

BigObject Pilot Current - Future

VF and LightningList Views - Related lists and SFX coming

Available Data TypesNumber, Text, Datetime & Lookup – Related Lists, Large Text field, JSON & System

API UXCreate today - Full CRUD for MetaData; CRD for Data

Setup UXView only now - full UX coming during 2016

2ndPILOTWinter‘16

Page 24: Analyze billions of records on Salesforce App Cloud with BigObject

AsyncSOQL NewPILOTWinter‘16

FiltersAdded Having, Datetime filters- Filters on Date Time Literals, Booleans, nulls, multi-select picklists, ….

Relationships1-level WHERE< SELECT & GROUPBY– 2/3 level relationships, sub-selects,…..

Clauses- Limit, OffSet, Group By, Type Of

Setup UXView only now - full UX coming during 2016

Current - Future

Page 25: Analyze billions of records on Salesforce App Cloud with BigObject

Call To Action!Stillacceptingparticipants!

Page 26: Analyze billions of records on Salesforce App Cloud with BigObject

Data Archive

Engage!

BigObject

AsyncSOQL

Engage in All PilotsEach of these products have an active pilot, apply with your AE today to participate.

Pilot participation is free!

Make your Voice HeardBigObject Chatter GroupTwitter - find me at @jonbruce

Build Out Your Use CasesLeverage our Implementation Want your voice heard?

Page 27: Analyze billions of records on Salesforce App Cloud with BigObject

AppendixSubtitleplaceholder

Page 28: Analyze billions of records on Salesforce App Cloud with BigObject

Salesforce ShieldSubtitleplaceholder

Page 29: Analyze billions of records on Salesforce App Cloud with BigObject

Now,YouCanEnhanceRecordRetentionCapabilitiesKeeptrackofyourdataevolutionandlifecycle

3x20FIELDS PER OBJECTFIELDS PER OBJECT

Out-of-the-Box FieldAuditTrail

Applies to all editions: Contact Manager, Group, Professional, Enterprise, Performance, Unlimited, Developer

SOQL

BulkAPI

Page 30: Analyze billions of records on Salesforce App Cloud with BigObject

MaximizeAuditabilityWithoutSacrificingPerformanceRetainarchivedfieldhistorydataupto10years,independentoffieldhistorytracking

1810

MONTHS

YEARS

SALESFORCE FIELD AUDIT TRAIL

1DAY

FIELD HISTORY

accountscontacts leads

opportunities

cases

customobject

customobject

customobject

customobject

customobject

customobject

customobject

inspection

warehouse inventory

positionjob

review

badge

workstation

workstation

userstory patient medicationsbudgets

expenses

members

points

transactions

OUT-OF-THE BOX CUSTOM POLICYDATA ARCHIVE

bug

applicant

candidate

Page 31: Analyze billions of records on Salesforce App Cloud with BigObject

Field Audit Trail in a NutshellSinglenormalizeddatasetforeasierquery,storageandaccessibility

Page 32: Analyze billions of records on Salesforce App Cloud with BigObject

Field Audit Trail - Long Range Road Map

NewFieldTypesMulti-selectPicklists,LargeTextFields

LongerRetentionRates10,15even30years

WaveIntegrationVisualizationforFIeldAuditTrail

Page 33: Analyze billions of records on Salesforce App Cloud with BigObject

thank y u

Page 34: Analyze billions of records on Salesforce App Cloud with BigObject

BigObject & Thunder (IoT)Subtitleplaceholder

Page 35: Analyze billions of records on Salesforce App Cloud with BigObject

Better Understand and Engage with your CustomerBigObjects

• Enriched customer profile - high-volume data storage and management for historical customer data

• Self-service data curation – tools to prepare needed for better decision making

• Intelligent processes – turn insights into action by driving business process with new customer KPIs and metrics

IoT Cloud• Connect with Everything –

connect every device, website and app to gather relevant data

• Build Smart Real-time Logic – use data and insights to make smarter, more personalized actions at IoT scale

• Engage Proactively – use the power of Salesforce to reach out to your customers & employees 1:1 whereever they are