Learning with the Toolmakers Using Systemic M&E …...Learning with the Toolmakers Using Systemic...
Transcript of Learning with the Toolmakers Using Systemic M&E …...Learning with the Toolmakers Using Systemic...
Learning with the Toolmakers
Using Systemic M&E Tools in Feed The Future
Uganda: Network MappingTuesday, July 14, 2015, 9:30 AM EDT
Eric DerksLeanne Rasmussen Lucho Osorio
A few take-aways from the first webinar
• Patterns of behaviour and relationships
• Action – reaction
• Learning, experimentation, adaptation
• Good questions
Systemic M&E: Network Analysis• Recap• Why Network Analysis• What we expect it to show• Methodology• Various Metrics• Implications for the project
Systemic Change
Ag-Inputs Firms FarmersImpacts
Facilitative Interventions
Customer-oriented growth strategies
Low yields & high production costs
Basic trading, arms-length relations, prevalence of counterfeits
Low input use, often misapplied
Proper/ higher use of quality inputs
High yields & lower production costs
Theory of Change and Intervention Strategy
Intervention Scheme
A R
ISC
OA
AR
R
R
O
O
O SC
SC
SCI
I I
I
II
II
I
I
I
GAPS
Intervention Scheme
M&E Scheme
Co
mp
on
ents
Performance Mgmt
Near-term Short-term Medium-term
Results Assessment
Attribution Assessment
A R
I
SCO
A
A
R
R
R
O
O
OSC
SC
SC
I
I I
I
II
I
I
I
I
I
Intervention Scheme
A R O SC ISCOR
• After Action Rws• Cost-Share Assess• Formal / informal
coms• Etc.
• Progress M&E• USAID PMP • Investigations• Adoption Survey
• Network Analysis• SenseMaker ®• USAID PMP• Investigations
• Mobile phone surveys
• Investigations
Transactional Network Analysis: Agro-Inputs
Network structure:• Changes over time• Differences between
businesses & locations
S
W
R
WW
S SS
R
RR
R
Suppliers/Importers: seed, CPP, fertilizers
Wholesalers
Retailers
Different patterns of behavior
“Collect data first…investigate later”
Find the differences
See the needle move
Changes we expected to see in network structures?
Sustainability(durability) Window of
Viability
Diversity + Interconnectivity
Greater resilienceIncreasing stagnation
Greater efficiencyIncreasing brittleness
Changes we expected to see:• Fewer actors• More specialization (esp. functions)• More integration• More cooperative relationships
S. Goerner, B. Lietaer, R. Ulanowicz “Quantifying economic sustainability: Implications for free-enterprise theory, policy and practice”, Ecological Economics, 2009
Other expected changes• Growth• Fewer stock-outs
Data collection process…
Interviews with 250+ wholesalersin 26 district town centres
Wholesalers
Who did you buy from? (Suppliers)
Strongest relationships, technical info
Who did you sell to?(Retailers)
Strongest relationships, technical info
Structures of social networks vs. transactional networks…
SNA: many connections in many directions, “web”
TNA: Direction of product flows; “hub and spoke”
Metric 1: Number of actors in the distribution chain
Suppliers114
Kampala/Kenya
Wholesalers300
26 districts
Rural retailers 1232+
from wholesalers
At least 30 newly mentioned in second season; high rate of new entry
No evidence yet of any businesses exiting market; anyone can trade in
agro-inputs
Many more who buy directly from Kampala
Metric 2: Percent of relationships that are rated highly
1st
season2nd
season
Target firms 21.8% 41.0%
Non-target firms 32.4% 28.4%
Percent of suppliers that wholesalers regularly relied on for product knowledge….
1st
season2nd
season
Target firms 44.7% 37.0%
Non-target firms
46.2% 37.3%
Percent of their suppliers that wholesalers had a ‘strong’ relationship with….
First season 2014: 53 target firms and 165 non-target firms; Second season 2014: 45 target firms and 196 non-target firms
To answer, Why? Hypotheses and investigations….
“I started asking for it from suppliers.”
“The supplier started doing trainings.”
General seasonal trend?
Confirm after next data collection.
Metric 3: “Churn rate” of relationships
Transactional relationships between suppliers and wholesalers
Assumption: as the system improves, the # or % of consistent connections will go up.
Total number transactional relationships between suppliers and wholesalers
1510
Number of relationships that were consistent for Season 1 and Season 2 833 (55%)
Number of relationships that were new for Season 2 677 (45%)
Investigation: Why do wholesalers add or drop suppliers?
Top reasons to add:
• New product/company came on the market
• Products now available/back in stock
• Products now demanded by customers
• Company now delivers
Top reasons to drop:
• Stopped delivering/do not deliver
• Products unavailable/out of stock
• Products not demanded by customers
• Company raised prices
Or: “Oops, we forgot to mention them…”
Implications?Relationships are about buying cheap and easy, and selling quickly
Lack of records means wholesalers don’t monitor relationships well
In Sum: What has network analysis done for us?
In mid-term project assessment; network data showed expected changes are not happening; need to consider new options became apparent
• Network structure tells us that incentives are stacked against enterprise-level change, due to continued over-crowding and nature of relationships
• Competitive pressures for poor performers to exit is not materializing.
• How else can the system get bad businesses to exit?
• Pointing us to new areas of intervention
Gives us numbers to tell us how the system is changing. • Less about making graphics, more about comparing metrics
over time.• Pace and direction of change in the system.
Thank you!
Upcoming webinar:Using Systemic M&E Tools in Feed The Future Uganda: Sensemaker ®
July 28, 2015, 9:30 AM EDT
Previous webinar:Using Systemic M&E Tools in Feed The Future Uganda
June 26, 2015, 9:30 AM EDT
Learn more here:http://www.seepnetwork.org/learning-with-the-toolmakers-pages-20747.php