Measuring Knowledge Management – Metricizing

Intangibles

Syndicate Room, NITIE20th Feb’09

Alakh AsthanaTwitter: alakh_asthanaalakh.asthana@gmail.com

For things to get better, we have to get better

For things to change, we have to change

Manage change or change will manage you

Metricizing KM

Greater Visibility over the years

Corporate reality – Tangible or Intangible

Believe It or not

Time Warner has only 6.49% of its value attributable to tangibles

For Oracle Corporation, tangibles account for only 4% of its value

General Electric (worth over US$450 billion), tangibles account for less than 11% of its value

Source: http://www.standardsinstitute.org/background/background.html

Measure the Treasure

Is there a clear ‘Measure’ on Knowledge or K-Sharing?

Create indicators, metrics, level etc.

Have multiple objectives – comparison between units, employees and senior management (WIFFIM)

Map objectives of the metrics closer to the business goals

Check Historical data

Preliminary Tools – A Q&A checklist

Preliminary Tools – Impact:Popularity

Critical to Business

Popularity / Penetration

High Cs & High Ps

Less Cs & Less Ps

Ask an Expert

Messenger Service

Documentation

Explicit Document repository

Content maintenance

BPI / Idea Sharing

CoPs

Informal Blogs and Forums

High Cs & Less Ps

Less Cs & High Ps

Answer the question why your website exists, then identify the 2 or 3 metrics that are critical to goal conversion

– Avinash Kaushik, Google Analytics Evangelist

What to measure?

Activity Metrics• Knowledge asset capture over time (Jan’08 to Mar’08)

• Resource utilization of the KM team members

• Number of users / Portal visits

• Asset usage across locations

• Searches / Downloads

• Submissions / discussions / publications

• Collaborative sessionQuality Interaction Sharing Collaboration

Real Metrics!

• Time Saved

• Productivity improvements

• Quality improvements

• Reduced risk

• Brand value

• Higher employee satisfaction

• Higher customer satisfaction

• Intellectual property usage / awareness

Combination of ‘Activity’ and ‘Real’

• Number of times a document is read – Utility of KM Repository

• Successful queries – Failed queries

• Content accessibility – Time saved

• Correlations metrics – Content supply and demand

• Documentation quality – User experience

Input Metrics

• Reviewer rating

• User rating

• Qualitative feedback

• Popularity

• Frequency / recency of download

Technology

• Databases • OLAP

• Business intelligence • Dashboards (monthly, quarterly) • User authentication techniques

• Survey tools

What can’t be measured• Direct measures or positive correlations

E.g. Effect of primary education on national GDP

Since 1950s, both the atmospheric CO2 and crime levels have increased sharply. Hence, atmospheric CO2 causes crime

• True value or impact of KM

“Not everything that can be counted counts, and not everything that counts can be counted”Albert Einstein

To Summarize / Tips

• Don’t claim all the credit

• Start measuring early in the life of the program and continue to measure often

• Match the measure to the audience

• Measures are best "designed-in" to KM projects, not added later

• Keep it simple. Focus on a few critical measures - Don't create measurement schemes that are more trouble than they are worth—

too time-consuming, too expensive, too hard to understand• Derive actionable metrics (don’t touch what you cant change!) • Don't raise unrealistic expectations about ROI - ROI is still primarily captured indirectly

Measurement Tips• The best output measures are those already used by the

organization and widely understood by managers and individual contributors

- Avoid developing complicated ones

• Stories are powerful indicators of success and promotional tools– Not a replacement for "hard dollar" measures, but useful to demonstrate

progress to managers and to drive knowledge-sharing behavior throughout the organization

Jumpstart is a NSE listed KPO with stellar growth since 5 years, from an employee strength of 7 in the first financial year – to 820 in the fourth, serving clients in three verticals – QMS, eCommerce and Standardization

Being a KPO, they are highly K-dependent; a KM team was established to capture existing knowledge. Process transitions being the backbone of every function, documentation framework was established

Problem: Documentation was highly critical to the business, however an overhead for the operations team

Quality of the documents was mediocre and was regularly battered by BD teams and PDC (Proximity Development Centers)

A documentation specialist was hired by the KM team. Ops team were delighted

Are more specialists needed? (hiring a specialist is expensive)

How do we identify which vertical is spending more? Where does the 80 / 20 lie?

What is the current quality of the documents? Who is better and why?

How do we stem the fire fighting? Is there scope for automation?

Case Study # 1

The Solution

An online tracker was launched to manage the workflow, with uploads

‘FIFO’ was implemented

Three status were maintained – WIP, Release and Declined along with time

Declined stated a reason by the Specialist

The ‘comments & feedback’ tab gave reasons for decline

Release was open to review by BD and PDC teams

BD and PDC had authority to decline as well

Tangible Results

1) A calculation of WIP gave an idea about resource utilization

2) More No. of rejects – BAD (and vice versa)

3) Few verticals like Standardization had more documents

4) A FTE was later hired for standardization (a win for the KM team)

5) Constant track and monitoring

Failure to spend the time on this "non value-added effort" can cost an executive dearly later on when

they're unable to make a strong business case.

- Robert Kaplan

Thank you!

Connect to me on:http://in.linkedin.com/in/alakhasthana Twitter: alakh_asthanaalakh.asthana@gmail.com