Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... ·...

13
APR. 2017 Data production will be 44 times greater in 2020 than it was in 2009. Source: IDC R Systems Inc. W H I T E P A P E R THOUGHT LEADERSHIP Achieve Analytics Sophistication Become Analytics-Driven

Transcript of Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... ·...

Page 1: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

APR. 2017Data production will be 44 times greater in 2020 than it was in 2009.

Source: IDC

R Systems Inc.

W H I T E P A P E R

THOUGHT LEADERSHIP

Achieve Analytics Sophistication

Become Analytics-Driven

Page 2: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

Table of Contents

1. Introduction

2. Analytics-Driven Organization

3. Analytics Maturity

4. Analytics Sophistication

5. Analytics Skill Gap

6. References

7. About R Systems

2.1. Big Data Revolution

2.2. Resurgence of Machine Learning and AI

2.3. Analytics as Competitive Advantage

1

1

3

3

4

4

6

8

10

11

Thought Leadership Whitepaper | analytics.rsystems.com

By 2020, data-driven organizations are set toachieve additional $65Bn. in productivity bene�ts.

Page 3: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

In today’s world, a successful enterprise has to achieve competencies in data and analytics to remain competitive and relevant. An organization can position itself ahead of disruptions in its industry by leveraging all relevant data which is at its disposal and by applying a variety of analytics on top of it. This is a mandate for many classic organizations who are under constant threat of disruptions in their own markets by newcomers or agile competitors. Although nothing can save an obsolete product or business model, analytics is fundamental for both innovating new products and enhancing existing products, as well as, their reach to the right customers, in the right way.

The use of Machine Learning (ML) and advanced analytics to create value from corporate or public data is nothing new [1]. Leveraging these technologies for commercial use started in the late ‘80s/ early ‘90s, when predictive analytics and ML techniques had become very popular. Today, we collectively cover both these aspects under “data science1 .”

1. Introduction

2. Analytics-Driven Organization

1 of 11

In the last three decades, the genuine interest in treating data as an asset2 and advanced uses of data for analytical decision making has been the driving force for many of the technology innovations. Figure1 is an illustration of the goal that has survived the test of time which is to collect and process all types of data in granular detail and in conjunction with each other to gain valuable novel insights.

As is typical, such interests started out of speci�c industries that realized the need which were then led by the innovators (technology enthusiasts) and early adopters (visionaries).

In Figure 1, the technologies involved have been constantly changing at a very fast pace. E.g., due to the exponential growth in volume/ velocity of data and data variety due to internet connectivity and proliferation of web products, data technologies have been under both evolutionary and revolutionary changes. Analytics tools and related technologies have also been continuously improving at a great pace with the most signi�cant change being the abundance and easy availability of open-source analytics tools/ software. In parallel, for classic and new companies, the possibilities to innovate/ improve their products and services by leveraging analytics has signi�cantly increased. This has created amazing new market opportunities. The use of analytics for product innovation and improvement is the norm today. In short, the general theme of evolution from detailed/ granular data of all kinds to intelligent business decisions has stayed unchanged in the past 30 years, while the technologies, processes and implementation details on achieving it have been in constant change.

For many classic organizations, coping- with constant changes in technologies and applications has become their main challenge towards becoming a data and analytics-driven organization.

At the same time, this has brought in big opportunities for many players as well. In early years, data warehousing and data marts were of a signi�cant focus with IT-led reporting and BI. On the advanced analytics end, the focus was on �nding rewarding use cases and right algorithms for commercial use. By mid ‘2000, the focus shifted to cost-e�ective data infrastructure, to cope up with the exponential growth of data and business-led BI, with an emphasis on visualization and self- service. On the advanced analytics front, with signi�cant breakthroughs in the processing speed, open-source tools and easy availability of all kinds of data, it became possible to deliver more con�dently on the promises of Machine Learning and AI.

This whitepaper focuses on the Two main breakthrough changes in the recent years:

Big Data Revolution and ML/ AI

1 We de�ne data science as “the practice of extracting insights from data using a multitude of disciplines and technologies for the purpose of creating new data products and services or improving the existing ones [1].” Based on this de�nition, data science is nothing new but the term itself and the recent level of interest in it.

2 As you have heard, data is the new Oil.

Nowadays, data science is an umbrella term that spans across data mining, advanced and predictive analytics, statistical modeling, decision science, knowledge discovery in databases (KDD), ML, pattern recognition and AI to make better decisions based on a vast amount of data of all kinds. Data science gained traction when there were real business interests in speci�c industries about advanced analysis of data for business decision making.

For over two decades, Data Science has been the cornerstone for a lot of applications. For e.g., Real-time Fraud Detection in the Finance vertical and the Recommendation Systems in the Retail vertical. However, due to the popularity of web products since early ‘2000, data science today has become widely acceptable across industries. This whitepaper explains di�erent

progressive levels in an enterprise’s journey to become analytics driven

Page 4: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

By 2018, more than half of large organizations globally will compete using advanced analytics and proprietary algorithms Source: Gartner

2 of 11Thought Leadership Whitepaper | analytics.rsystems.com

In the last three decades, the genuine interest in treating data as an asset2 and advanced uses of data for analytical decision making has been the driving force for many of the technology innovations. Figure1 is an illustration of the goal that has survived the test of time which is to collect and process all types of data in granular detail and in conjunction with each other to gain valuable novel insights.

As is typical, such interests started out of speci�c industries that realized the need which were then led by the innovators (technology enthusiasts) and early adopters (visionaries).

In Figure 1, the technologies involved have been constantly changing at a very fast pace. E.g., due to the exponential growth in volume/ velocity of data and data variety due to internet connectivity and proliferation of web products, data technologies have been under both evolutionary and revolutionary changes. Analytics tools and related technologies have also been continuously improving at a great pace with the most signi�cant change being the abundance and easy availability of open-source analytics tools/ software. In parallel, for classic and new companies, the possibilities to innovate/ improve their products and services by leveraging analytics has signi�cantly increased. This has created amazing new market opportunities. The use of analytics for product innovation and improvement is the norm today. In short, the general theme of evolution from detailed/ granular data of all kinds to intelligent business decisions has stayed unchanged in the past 30 years, while the technologies, processes and implementation details on achieving it have been in constant change.

For many classic organizations, coping- with constant changes in technologies and applications has become their main challenge towards becoming a data and analytics-driven organization.

At the same time, this has brought in big opportunities for many players as well. In early years, data warehousing and data marts were of a signi�cant focus with IT-led reporting and BI. On the advanced analytics end, the focus was on �nding rewarding use cases and right algorithms for commercial use. By mid ‘2000, the focus shifted to cost-e�ective data infrastructure, to cope up with the exponential growth of data and business-led BI, with an emphasis on visualization and self- service. On the advanced analytics front, with signi�cant breakthroughs in the processing speed, open-source tools and easy availability of all kinds of data, it became possible to deliver more con�dently on the promises of Machine Learning and AI.

This whitepaper focuses on the Two main breakthrough changes in the recent years:

Big Data Revolution and ML/ AI

Page 5: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

3 of 11

2.1 Big Data Revolution

2.2 Resurgence of Machine Learning and AI

For a moment, we �rst consider one of these signi�cant technology changes that are the big data revolution and its drastic impact on organizational data infrastructure. In early to mid ‘2010, big data was at its hype. So many companies were experimenting or planning to explore Hadoop ecosystem. At the time, top technical challenges with Hadoop were the speed of processing, ETL and data preparation, useage of advanced analytics and real-time embedding of advanced analytics in operational �ows[4,5] In 2015, per Gartner, investment in big data going strong, where 75% of all com-panies were either investing or were planning to invest in big data in the next two years[7]. By 2016, big data maturity survey[6] indicated that 75% of the respondents already had at least one Hadoop cluster in some shape or form, whereas 73% had it in production. The survey also established that big data was still growing fast, data governance was a growing concern ahead of security and access. It also revealed big data cloud option was a preferred choice for partial/ full data storage or development for many organizations3 . It also indicated that 95% of respondents have achieved positive value from their investments in big data, or are expecting that they will. Given that all the investments in data infrastructure, were from a workload’s

Earlier we mentioned a few successful commercial uses of ML in early ‘90s, in �nancial (real-time fraud detection using neural networks[10]) and retail (recommendation systems) verticals that are still in use. Since then, there have been many applications of ML used everyday, but still go unnoticed. In recent years, one very interesting development is the resurgence of AI, which comprise of ML. We are no longer concerned whether computers (or robots) can ever display intelligent behavior, but rather how intelligent they may get as witnessed by recent AI successes like Watson in Jeopardy! and Google DeepMind AlphaGo4 are the evidence of the possibilities in AI. There have been di�erent approaches to AI. The most successful approach, that typically outperforms others, involves ML applied to large datasets.

For some of these applications, the results are so human-like that many companies are looking to hire more data scientists, collecting more data and applying latest ML & AI technologies to improve their existing decision-making processes or innovate new data products.

New analytical tools for making decisions, like Watson, are creating entirely new opportunities. With the digitization of world commerce, the emergence of big data and the advancement of analytical technologies, organizations now have amazing opportunities to di�erentiate themselves through analytics. Per the study, “Analytics: The Widening Divide,” by MIT Sloan Management, most companies have seized these opportunities.

standpoint, BI had now gone ahead of ETL which is an indication of putting data to analytics use. BI is the fastest way to show the value of data and justify investments in any new data technology. Focusing on BI doesn’t diminish advanced analytics’ value, that typically brings higher business value (ROI) although it has a longer turnaround time and higher risks.

4 AlphaGo uses Deep Learning via convolutional neural networks trained to predict moves that expert players would make, using a dataset of 30 million di�erent positions from real games. The real world applications are much harder than games but ML/ AI have recently proven their success on many new real-world use cases.

3 In 2014, there was fear of Cloud among many enterprise companies. That mindset has quickly changed. Today, every enterprise is either using the cloud or in process of moving there. This trend is witnessed across many industries like �nancial, retail, insurance, healthcare, retail, and even government.

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

Page 6: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

4 of 11

3. Analytics Maturity

Earlier we mentioned a few successful commercial uses of ML in early ‘90s, in �nancial (real-time fraud detection using neural networks[10]) and retail (recommendation systems) verticals that are still in use. Since then, there have been many applications of ML used everyday, but still go unnoticed. In recent years, one very interesting development is the resurgence of AI, which comprise of ML. We are no longer concerned whether computers (or robots) can ever display intelligent behavior, but rather how intelligent they may get as witnessed by recent AI successes like Watson in Jeopardy! and Google DeepMind AlphaGo4 are the evidence of the possibilities in AI. There have been di�erent approaches to AI. The most successful approach, that typically outperforms others, involves ML applied to large datasets.

For some of these applications, the results are so human-like that many companies are looking to hire more data scientists, collecting more data and applying latest ML & AI technologies to improve their existing decision-making processes or innovate new data products.

New analytical tools for making decisions, like Watson, are creating entirely new opportunities. With the digitization of world commerce, the emergence of big data and the advancement of analytical technologies, organizations now have amazing opportunities to di�erentiate themselves through analytics. Per the study, “Analytics: The Widening Divide,” by MIT Sloan Management, most companies have seized these opportunities.

2.3 Analytics as Competitive Advantage

For more than 20 years, organizations that embraced analytics enjoyed a clear-cut competitive advantage over their peers. As the importance of analytics grew and success stories publicized, analytics thus gained more ground across industries (typically the laggards). A 2011 MIT Sloan Management survey[2] indicated that at the time 58% of executives globally, across all industries considered analytics as a competitive advantage, which is a very signi�cant jump from what they perceived a year before. However, by 2016, increased market adoption of analytics had leveled the playing �eld and made it harder for companies to keep their competitive edge using analytics while the optimism about analytics’ potential had stayed strong[3].

Our �ve progressive levels of analytics maturity (Figure 2) shows the �nal, most sophisticated level named as “Transformed”. Such companies are data native hence regard analytics an integral part of everything, thus use analytics strategically daily and at all levels. Reference[9] below, elaborates each of these levels further.

Thought Leadership Whitepaper | analytics.rsystems.com

Figure 1: From detailed granular data to analytics-driven decisions

Decisions

Advanced Analytics

Business Intelligence

Data Types (Structured, Semi-structured, Unstructured)

Data Infrastructure

Many enterprises remain in the early stages of their analytics initiatives and strategically consider it as an engine of innovation. Almost every classic organization is realizing the journey through analytics maturity requires a lot of hard work and commitment due to constant changes in technologies involved and the pressures of new competition. This commitment does touch many aspects of organizational behavior, from revamping the data management and analytics skills/ tools, to changing cultural norms[3].

Figure 1 illustrates the succession of steps to go, from Raw Data (variety of detailed operational data, third party data, etc.) to Intelligent Decisions, for achieving analytics maturity. Companies can optimize their decisions based on speci�c analysis performed on the data and dependent on the speci�c business use cases. In classic organizations, these competencies will take time to evolve and mature. An organization should �rst commit to the importance of decisions that are analytics-driven. This requires gradual alignment in culture, along with a clear technical roadmap. For a classic organization, achieving analytics maturity is a journey across multiple dimensions. Organizations will not progress through analytics maturity levels solely based on “data readiness/ infrastructure” and employing “analytics tools/ techniques”. Other dimensions like “culture”, “top leadership”, “people/ skillsets”, “business processes”, “nature of industry/ market”, competitive map, etc. need to be taken into consideration to make this journey successful.

For brevity, in this paper we have merged some of these dimensions and laid emphasis on three dimensions:

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

Page 7: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

5 of 11

Analytics Sophistication (skill level, expertise, techniques and tools)

Data Management Sophistication (data readiness, infrastructure, skills, experience and tools)

Culture (competitive landscape, leadership, processes and industry)

Our �ve progressive levels of analytics maturity (Figure 2) shows the �nal, most sophisticated level named as “Transformed”. Such companies are data native hence regard analytics an integral part of everything, thus use analytics strategically daily and at all levels. Reference[9] below, elaborates each of these levels further.

Figure 1: From detailed granular data to analytics-driven decisions

Figure 1 illustrates the succession of steps to go, from Raw Data (variety of detailed operational data, third party data, etc.) to Intelligent Decisions, for achieving analytics maturity. Companies can optimize their decisions based on speci�c analysis performed on the data and dependent on the speci�c business use cases. In classic organizations, these competencies will take time to evolve and mature. An organization should �rst commit to the importance of decisions that are analytics-driven. This requires gradual alignment in culture, along with a clear technical roadmap. For a classic organization, achieving analytics maturity is a journey across multiple dimensions. Organizations will not progress through analytics maturity levels solely based on “data readiness/ infrastructure” and employing “analytics tools/ techniques”. Other dimensions like “culture”, “top leadership”, “people/ skillsets”, “business processes”, “nature of industry/ market”, competitive map, etc. need to be taken into consideration to make this journey successful.

For brevity, in this paper we have merged some of these dimensions and laid emphasis on three dimensions:

Culture plays a very crucial role as it in�uences leadership, processes and people across the organization and at all levels. Irrelevant of a company’s sophistication across the data and analytics dimensions, if the culture does not promote and reward the analytics-driven decisions, higher levels of analytics maturity may not be achieved. Leadership in a company can initiate an analytics-driven culture and thinking down the ranks. Changing established processes, mindset and thought process in an organization takes time but it is instrumental to get everybody on board on analytics and data initiatives. However, mostly, this change may be initiated even from the mid-level of an organization and then cascade up to the top and the rest of the organization. In either scenario, an analytics-driven thinking and culture is

Figure 2: Five progressive levels of analytics maturity

Transformed

Beginner

Advanced

Motivated

Experienced

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

essential for a successful implementation of data and analytics competencies which help companies to move towards an analytics-driven decisions whether tactical or strategical. Many times the culture change is forced upon a company by market dynamics and competitive landscape in the industry which they operates in. In such situation, companies look up to the leaders and innovators in their industry or similar industries.

Page 8: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

6 of 11Thought Leadership Whitepaper | analytics.rsystems.com

4. Analytics Sophistication

Analytics can only perform well if an organization stands over a solid data foundation and infrastructure, which is designed and built for various analysis. Highlighting the importance of data management, infrastructure and its readiness here is not needed but it is su�cient enough to say that the most matured organizations are continuously making it easier for all types of users (data scientists and analysts) to access relevant quality data they need. In an analytically matured company, data scientists and analysts should never be concerned with core data management tasks such as data collection, ingestion, storage, access, documentation quality, governance and integration (raw data quality). Data engineering/ architecture teams provide these services across the company and work with other data scientists to collect and transform new sources of data for the use of analytics.

5 In this document, we may use the term “Data Science” and “Data Mining” interchangeably with data science obviously preferred.

Culture plays a very crucial role as it in�uences leadership, processes and people across the organization and at all levels. Irrelevant of a company’s sophistication across the data and analytics dimensions, if the culture does not promote and reward the analytics-driven decisions, higher levels of analytics maturity may not be achieved. Leadership in a company can initiate an analytics-driven culture and thinking down the ranks. Changing established processes, mindset and thought process in an organization takes time but it is instrumental to get everybody on board on analytics and data initiatives. However, mostly, this change may be initiated even from the mid-level of an organization and then cascade up to the top and the rest of the organization. In either scenario, an analytics-driven thinking and culture is

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

essential for a successful implementation of data and analytics competencies which help companies to move towards an analytics-driven decisions whether tactical or strategical. Many times the culture change is forced upon a company by market dynamics and competitive landscape in the industry which they operates in. In such situation, companies look up to the leaders and innovators in their industry or similar industries.

Page 9: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

What is the best that can happen given constraints?

What is the trend?

What will happen?

What actions are needed?

What is happening overall?

What exactly is the problem?

What is happening?

What exactly is the problem?

AD-HOC REPORTS

OLAP DRILL DOWN

What happened speci�cally?

What happened?

STD. REPORTS

PA, ML, AI

FORECASTING

STATISTICAL ANALYSIS

ALERTS

DASHBOARD &

VISUALIZE

OPTIMIZATION

Advanced Analytics:Predictive & Proactive

Basic Analytics:Descriptive & Reactive

Prescriptive

AN

ALY

TIC

S SO

PHIS

TIC

ATIO

N =

CO

MPE

TITI

VE

AD

VAN

TAG

E

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

used interchangeably and covers the full spectrum. The term Analytics is often considered ‘business-friendly’, whereas, Data Science is still viewed like a technical term. However, both these terms achieve the same objective of extracting value from data via analysis. From a technical standpoint, the term “Data Science” is regarded to be more focused on covering the high-end of the analytics spectrum.

To be successful at any level of analytics sophistication, a company must have an outstanding level of expertise in data management. Only then an enterprise

can start its evolution from a low-end analytics like standard reports towards adhoc reporting, simple alerts, OLAP, real-time dashboards (monitoring KPIs) and interactive visualization. Business and data analysts usually operate in this range. A data analyst must have a su�cient business domain knowledge and a good understanding of relevant data along with common analytical skills. To perform data analysis tasks, business/ data analysts often deal with an aggregated data, mostly from a single data source and a handful of variables at anytime.

Figure 3: Competitive advantage has direct correlation with analytics sophistication. Here we measure sophistication only by examples of analytics deployed from less complex basic analytics to more sophisticatedalgorithmic techniques (advanced analytics) 7 of 11

Page 10: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

8 of 11Thought Leadership Whitepaper | analytics.rsystems.com

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

5. Analytics Skill GapData Science is a multi-disciplinary �eld that ideally requires a strong technical knowledge of ML, pattern recognition, programming, statistics, mathematics, probability, modeling, databases, and computing. Given the existing silos in our higher education system, companies had to hire and train data scientists. In recent years, there have been many data science programs (degreed or non-degreed) which are introduced in di�erent settings (academic/ non-academic), but still data science can be best learned on the job, assuming that the companies have experienced data scientists onboard who can mentor the new hires. However, due to lack of data science talent, high emphasis has been put on the some of the data science techniques and other stand-alone/ cloud-based tools like Python (Pandas, Scikit-learn, etc.), R, SAS, SPSS, SQL, Azure ML, H2O, Hadoop ecosystem (Hive, Spark, HBase, etc.).

Data scientists always deal with large granular data, in all shapes and forms and from a variety of sources. They have to deal with a huge number of variables and employ advanced sets of algorithms. A data scientist always demands more data. “All data in raw form” is always preferred, but it may not always be practical to store and analyze all that data[1].

Aside from deep knowledge of the data science “black arts” and above mentioned techniques, data scientists should possess many non-technical skills, which are often over-looked. The ten non-technical skills

outlined `below are important (the �rst seven are the Key ones) to be successful in data science tasks in the real world:

01. Problem solving ability02. Ability to question the work of self and others as well 03. Passion for data (the more, the better)04. Attention to detail and an ability to validate own work in multiple ways05. Statistical thinking (one who knows when to reason deterministically and when probabilistically)06. Passion for exploration and discovery (quick learner from failures)07. Ability to devise optimal and creative ways to experiment new (�nding novel insights is cumbersome. One can never �nd a sure way to �nd it)08. Presentation ability (written and oral)09. Business acumen10. Ability to simplify complex concepts to be able to easily explain it to others

While one interested in data science can often learn the technical skills and evolve, the non-technical skills however, may not necessarily be as straightforward to grasp. Organizations often face these challenges as they attempt to create, build and evolve a data science team.

Page 11: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

9 of 11

Figure 4: Transition to advanced analytics always requires changes to the skillsets and tools.

Figure 5: Analytics sophistication and big data. Transition from large to big data for both traditional BI and Predictive Analytics/ Data Mining requires new skills development and tools.

Di�erent skill set (both in terms of training and experience)

Di�erent Data Infrastructure (only sometimes)

Scienti�c Mindset

Di�erent Analysis Toolset

Longer Project Cycles

Di�erent Data Transformations

01

03

05

0706

04

02

Most Granular/ Detailed Use of Data

High Performance Data Mining(HPDM)

Data Volume Big DataLarge

Traditional Data Mining | Predictive Analytics

Traditional BI

“Big Data” Mining

Big Data A

nalytics(BD

A)

Ana

lytic

sSo

phis

ticat

ion

Basi

cAd

vanc

ed

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

Data Science is a multi-disciplinary �eld that ideally requires a strong technical knowledge of ML, pattern recognition, programming, statistics, mathematics, probability, modeling, databases, and computing. Given the existing silos in our higher education system, companies had to hire and train data scientists. In recent years, there have been many data science programs (degreed or non-degreed) which are introduced in di�erent settings (academic/ non-academic), but still data science can be best learned on the job, assuming that the companies have experienced data scientists onboard who can mentor the new hires. However, due to lack of data science talent, high emphasis has been put on the some of the data science techniques and other stand-alone/ cloud-based tools like Python (Pandas, Scikit-learn, etc.), R, SAS, SPSS, SQL, Azure ML, H2O, Hadoop ecosystem (Hive, Spark, HBase, etc.).

Data scientists always deal with large granular data, in all shapes and forms and from a variety of sources. They have to deal with a huge number of variables and employ advanced sets of algorithms. A data scientist always demands more data. “All data in raw form” is always preferred, but it may not always be practical to store and analyze all that data[1].

Aside from deep knowledge of the data science “black arts” and above mentioned techniques, data scientists should possess many non-technical skills, which are often over-looked. The ten non-technical skills

outlined `below are important (the �rst seven are the Key ones) to be successful in data science tasks in the real world:

01. Problem solving ability02. Ability to question the work of self and others as well 03. Passion for data (the more, the better)04. Attention to detail and an ability to validate own work in multiple ways05. Statistical thinking (one who knows when to reason deterministically and when probabilistically)06. Passion for exploration and discovery (quick learner from failures)07. Ability to devise optimal and creative ways to experiment new (�nding novel insights is cumbersome. One can never �nd a sure way to �nd it)08. Presentation ability (written and oral)09. Business acumen10. Ability to simplify complex concepts to be able to easily explain it to others

While one interested in data science can often learn the technical skills and evolve, the non-technical skills however, may not necessarily be as straightforward to grasp. Organizations often face these challenges as they attempt to create, build and evolve a data science team.

Page 12: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

FFC483

Insights-driven businesses will earn over $1.2 Trillion annually, by 2020

Source: Forrester

10 of 11Thought Leadership Whitepaper | analytics.rsystems.com

[1] Hassibi Khosrow - 2014 High Performance Data Mining and Big Data Analytics: The Story of Insight from Big Data (Available on Amazon)

[2] MIT Sloan Management Review Research Report: 2011 Analytics The Widening Divide

[3] MIT Sloan Management Review Research Report: 2016 Beyond The Hype: The Hard Work Behind Analytics Success

[4] Rogers Shawn John Myers and Barry Devlin: 2013 Operationalizing the Buzz: Big Data 2013 Research Enterprise Management Research (EMA)

[5] Graham Bradley and M R Rangaswami 2013 "Do you Hadoop? A Survey of Big Data Practitioners" Sand Hill Group - October 29 (Accessed March 31, 2017) sandhill.com

[6] Hortonworks & AtScale 2016: 2016 Big Data Maturity Survey

[7]Gartner.com 2015 "Survey Analysis: Practical Challenges Mount as Big Data Moves to Mainstream

[8] Ernst & Young (EY) 2015: Becoming an Analytics Driven Organization to Create Value

[9] Analytics Maturity Assessment 2017 (Accessed April 30, 2017)

http:⁄⁄analytics.rsystems.com⁄data-analytics-consulting⁄analytics-maturity-assessment⁄

[10] Hassibi Khosrow 2001 "Detecting Payment Card Fraud with Neural Networks" In Business Applications of Neural Networks: The State-of-the-Art of Real-World Applications by Paulo J G Lisboa Alfredo Vellido and Bill Edisbury, 141-158. World Scientific

http:⁄⁄bit.ly⁄IDC-estimates-big-data

http:⁄⁄bit.ly⁄IDC-press-release

http:⁄⁄bit.ly⁄Gartner-press-release

http:⁄⁄bit.ly⁄Forrester-predictions

REFERENCES:

Analytics sophistication is one of the dimension which is considered for the assessment of analytics maturity. One key issue that always comes up in data science5, BI, or other data processing tasks is the precise de�nition of the term “analytics,” and what does it really mean. Analytics covers a vast spectrum and is often found to be a source of confusion for clients evaluating various vendors’ capabilities. Figure 3 is an illustration of an enterprise evolution in analytics[1]. It plots the analytics sophistication in terms of simple and familiar analytics practices and its direct correlation with competitive advantage. The analytics spectrum is basically divided into Two - Basic and Advanced. The divisions are based on at least two vital di�erentiating factors.

Firstly, advanced analytics needs moresophisticated algorithms, tools and skillsets to achieve its goals. Secondly, the business value/ insights obtained from Advanced analytics is also higher, since it has a predictive, proactive and prescriptive (3Ps) nature. This means the capabilities it provides can impact future outcomes in a smart manner[1].

Basic analytics looks at the past data and requires simpler processing. It is both reactive and descriptive in nature.In the business world, both these terms (Analytics and Data Science) are often

As an enterprise recognizes the value of its data through the analytics processes, it will continue to improve upon them to maximize their business value. However, there is a saturation point beyond which, the marginal contribution achieved from the basic analytics practices quickly diminishes[1]. Precisely at this point, a disruptive change has to take place in an organization’s vision and culture in order for it to transition into a higher form of analytics. In Figure 3, this transition point is illustrated using a yellow lightning bolt while Figure 4, outlines some key factors required for this disruptive transition. At this side of the spectrum, a company becomes more proactive in employing a data-driven predictive and prescriptive processes. Statistical Analysis, Machine Learning (ML), Predictive Analytics (PL),Forecasting, Arti�cial Intelligence (AI) and mathematical optimization (under real-world constraints) are examples of Advanced Analytics techniques. Data scientists typically operate in this range.It is worthy to mention that when an organization transitions from large data to big data, there would be disruptive changes required as well. The enterprise will clearly need di�erent mindset and thought process in its approach along with various tools and techniques to cope up with big data. The traditional BI and advanced analytics skillset also needs to improve and evolve to address the challenges of the big data. [Figure 5]

Page 13: Achieve Analytics Sophistication - R Systemsanalytics.rsystems.com/wp-content/uploads/dlm... · challenge towards becoming a data and analytics-driven organization. ... the Recommendation

Executive Contacts:

11 of 11Thought Leadership Whitepaper | analytics.rsystems.com

ABOUT R SYSTEMS

R Systems Analytics helps clients uncover actionable insights to drive competitive advantage and capture business value. We help organizations integrate and operationalize data analytics solutions, enabling them to gain visibility into previously opaque or hard to measure processes. This empowers our clients to make smarter business decisions.

Our team of data experts, consultants and data scientists leverage proven analytics methodologies, tools and best practices to de�ne the right analytics solutions for you, that solve complex business challenges/ speci�c use cases and drive future growth.

© 2016 R Systems International Limited. All Rights Reserved. All content/information present here is the exclusive property of R Systems International Ltd. The content/information contained here is correct at the time of publishing. No material from here may be copied, modi�ed, reproduced, republished, uploaded, transmitted, posted or distributed in any form without prior written permission from R Systems International Ltd. Unauthorized use of the content/ information appearing here may violate copyright, trademark and other applicable laws, and could result in criminal or civil penalties

Khosrow Hassibi, PHDChief Data Scientist

[email protected]

Je� Johnstone Director – Client Services, Analytics

Je�[email protected]

W H I T E P A P E R

ANALYTICS SOPHISTICATION