Analyze billions of records on Salesforce App Cloud with BigObject
-
Upload
salesforce-developers -
Category
Technology
-
view
5.865 -
download
2
Transcript of Analyze billions of records on Salesforce App Cloud with BigObject
Unleash the Potential of Big DataBigObject WebinarDecember 2015
Jonathan BruceDirector Product Management, Big Data – AppCloud@jonbruce
Pat PattersonDeveloper Evangelist Architect@metadaddy
Forward Looking StatementSafe harbor statement under the Private Securities Litigation Reform Act of 1995:This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
“By now, you’ve heard the message — turn more data into business insights — and your business has seen the potential in big data to deliver it. By Q2 2016, 65% of data and analytics decision-makers say their firms will have implemented big data solutions, and some part of the 26% who were interested with no immediate plans in 2015 will move into early implementation phases”
“The Integration Imperative of Digital Experiences, Forrester Research, August 2015”
2015 is about Big Data Execution
CustomersCreatingMassiveAmountsofData
90%of the world’s data created in
last 12 months
Posts
Leads
Apps
Profiles
Usage
Events
VisitsProducts
Repairs
Cases
Big Data to Become the Primary Driver for Data Archive
“By 2016, 75% of structured data archiving will incorporate support for Big Data analytics and by 2017, archiving support of Big Data Analytics will surpass archiving for compliance as the primary use case for structured data archiving” - Gartner
Source: “Magic Quadrant Structured Data Archive Application & Retirement” - Gartner
Today: Understanding Your Customer
ERP HCM SCM Logs
1. Acquire Data
2. Wrangle Data(Cleanse, Augment, Transform, Join)
Data Lake / EDW
4. Take ActionCustomer Success Platform
3. AnalyzeWave
Firewall
Future: Understanding Your Customer
ERP HCM SCM Logs
4. Take ActionSalesforce Apps
3. AnalyzeWave
Firewall
2. Wrangle DataAsync SOQL
1. Acquire DataBigObjects
Wav
e To
olin
g
The Data-Driven App Cloud
AsyncSOQL
Wave Data Enrichment
Sales Service Marketing AppCloud ISV
1 Data Acquisition
2 Reduce Data
3 Visualize Analyze, Understand
4 Take Action
DataBigObject, SObject, External Objects
Customer 360
Data Retention
Event Capture
BigObject Top Use Cases
BigObjects let you store manage billions of records natively
BigObject
BigObject Storage and Services
SOQL Async SOQLAPI Bulk API
Data ArchiveNear-line storage for policy driven data retention
Object query language Resilient query
BigObjects means 100s of billions of records on AppCloud
Based on Apache HBase, powered by Apache Phoenix
• BigObject is the AppCloud abstraction of our trusted linear scalable a NOSQL data store
• Fully available across all production & sandbox instances
Data persistence optimized for high-volume data• Geared for 1, 10, 100s of billions of records• Targeted Immutable data – think archive data,
events, external data, historical data, loyalty, logs….• No Transactions, Eventual Consistency
High-volume storage for AppCloud - reliable, highly-available & secure
BigObject
BigObjects brings familiar look and feel
Familiar, object-based development model• Familiar and new data types – string,
number, date, JSON (in future)• Relationships - lookup, master-detail• Exposed in SOAP, REST, Bulk, and Metadata
APIs
Data scale means new services and different contracts• Synchronous & Asynchronous SOQL • Reports, Dashboard & Search not
supported
High throughput Ingress & Egress• New Bulk API Implementation geared for
1billion record/day ingest
High-volume storage for AppCloud - reliable, highly-available & secure
BigObject
BigObject and SOQL
WhyConstrainedSOQL?• #Field Audit Trail mandatesbyFieldHistoryType,
ParentId&CreatedDate,andinthatorder• CustomBigObjectcanfilteronIDonly
Future: CustomKeyPrefixes&SecondaryIndexes• Newdesignpatternaroundcustomkeyprefixesthat
requiresthinkingthruaccesspatterns• Enablescomplex&outoforderfilters
Data scale means highly targeted SOQL only
Introducing: Asynchronous SOQL
Based on Apache Hadoop, powered by Salesforce
Aggregation, Filtering of Across All Records• SOQL-style queries across any object• Works across SObject, BigObject and External
Objects• Seamless relationships to SObjects extending
the customer data models
Exposed via RESTful API • New async-query end-point to issues
queries• Capable Multiple parallel queries• Directed ResultSet Targets• Target field(s) in SObject, BigObject and
External Objects
Reliable, Resilient & Federated SOQL
Source Query
Anatomy of a Asynchronous SOQL SOQL Source, Mapping and Object Destination{
"query": "SELECT End_Time__c,Start_Time__c,Uber_Type__c,
Rider__r.FirstName, Rider__r.LastName, Rider__r.Email
FROM Rider_Record1__b WHERE Star_Rating__c = '5'",
"targetObject": “Rider_Reduced__b",
"targetFieldMap": {
"End_Time__c": "End_Time__c",
"Start_Time__c": "Start_Time__c",
"Uber_Type__c": ”Car_Type__c",
"Rider__r.FirstName": "First_Name__c",
"Rider__r.LastName": "Last_Name__c",
"Rider__r.Email": "Rider_Email__c"
}
}
Target Object
Field Mappings
Asynchronous SOQL
Support for aggregate functions: count(field), avg(), sum(), more to follow
SOQL Syntax group-by, like, having (coming Spring ‘16)
Where ClauseComparison operators for TEXT and NUMBER fields only
=, !=, <, <=, >, >-, >=Logical operators
AND, OR
Relationship QueriesChild-to-parent relationships are supported using the dot notation. SELECT, WHERE and GROUP BY clauses Single-level Only
SOQL Coverage To Date
Introducing: Asynchronous SOQL
New Limits• New query & result size limits model• Number of Parallel queries
Future Enhancements include• Apex driven SOQL queries supported in the
future• Additional grouping & aggregates – let us
know!
Reliable, Resilient & Federated SOQL
2: Data Reduction & Processing
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
Parallel Processing JobsMulti-tenancy Hadoop, resource management, scheduling, job monitoring and management
SOQL API and Apex interface for data filtering, grouping and aggregates
Data Sources
Data Processing
Data Targets
Async SOQL
3: Wave Driven Data Exploration, Insights & Analysis CloudSpeed&ScaleCompletecustomerinsights
acrossyourbusiness
Search-BasedQueryKey-value-pairandinvertedindexesforspeed&flexibility
DataDrivenConversationViadesktopormobilewhilefullycollaborative
4: Data Driven Action on InsightsFastActiononYourInsightsCompletecustomerinsightsacrossyourbusiness
Enrichforce.comObjectsApplyinsightstoyourcoreforce.comentities
IntelligentProcessesTurninsightsintoactionbydrivingbusinessprocesswithnewcustomerKPIsandmetrics
Demo & Code ExamplesDataArchive,Customer360&Reports&DashboardswithBigObjectdata
Active PilotsStillacceptingparticipants!
BigObject Pilot Current - Future
VF and LightningList Views - Related lists and SFX coming
Available Data TypesNumber, Text, Datetime & Lookup – Related Lists, Large Text field, JSON & System
API UXCreate today - Full CRUD for MetaData; CRD for Data
Setup UXView only now - full UX coming during 2016
2ndPILOTWinter‘16
AsyncSOQL NewPILOTWinter‘16
FiltersAdded Having, Datetime filters- Filters on Date Time Literals, Booleans, nulls, multi-select picklists, ….
Relationships1-level WHERE< SELECT & GROUPBY– 2/3 level relationships, sub-selects,…..
Clauses- Limit, OffSet, Group By, Type Of
Setup UXView only now - full UX coming during 2016
Current - Future
Call To Action!Stillacceptingparticipants!
Data Archive
Engage!
BigObject
AsyncSOQL
Engage in All PilotsEach of these products have an active pilot, apply with your AE today to participate.
Pilot participation is free!
Make your Voice HeardBigObject Chatter GroupTwitter - find me at @jonbruce
Build Out Your Use CasesLeverage our Implementation Want your voice heard?
AppendixSubtitleplaceholder
Salesforce ShieldSubtitleplaceholder
Now,YouCanEnhanceRecordRetentionCapabilitiesKeeptrackofyourdataevolutionandlifecycle
3x20FIELDS PER OBJECTFIELDS PER OBJECT
Out-of-the-Box FieldAuditTrail
Applies to all editions: Contact Manager, Group, Professional, Enterprise, Performance, Unlimited, Developer
SOQL
BulkAPI
MaximizeAuditabilityWithoutSacrificingPerformanceRetainarchivedfieldhistorydataupto10years,independentoffieldhistorytracking
1810
MONTHS
YEARS
SALESFORCE FIELD AUDIT TRAIL
1DAY
FIELD HISTORY
accountscontacts leads
opportunities
cases
customobject
customobject
customobject
customobject
customobject
customobject
customobject
inspection
warehouse inventory
positionjob
review
badge
workstation
workstation
userstory patient medicationsbudgets
expenses
members
points
transactions
OUT-OF-THE BOX CUSTOM POLICYDATA ARCHIVE
bug
applicant
candidate
Field Audit Trail in a NutshellSinglenormalizeddatasetforeasierquery,storageandaccessibility
Field Audit Trail - Long Range Road Map
NewFieldTypesMulti-selectPicklists,LargeTextFields
LongerRetentionRates10,15even30years
WaveIntegrationVisualizationforFIeldAuditTrail
thank y u
BigObject & Thunder (IoT)Subtitleplaceholder
Better Understand and Engage with your CustomerBigObjects
• Enriched customer profile - high-volume data storage and management for historical customer data
• Self-service data curation – tools to prepare needed for better decision making
• Intelligent processes – turn insights into action by driving business process with new customer KPIs and metrics
IoT Cloud• Connect with Everything –
connect every device, website and app to gather relevant data
• Build Smart Real-time Logic – use data and insights to make smarter, more personalized actions at IoT scale
• Engage Proactively – use the power of Salesforce to reach out to your customers & employees 1:1 whereever they are