Version, September 2015 Mid-term evaluation of the AIMS ...€¦ · Version, September 2015...

www.technopolis-group.com

Version, September 2015

Mid-term evaluation of the AIMS-IDRC/DFID program 2010-2017

Appendices to the final report

Mid-term evaluation of the AIMS-IDRC/DFID program 2010-2017 Appendices to the final report

technopolis |group| September 2015

Francie Sadeski

Rebecca Allinson

Soheir Dani

Matthias Ploeg

Marina Svetachova

Flore Vaucelle

Stijn Zegel

i

Table of Contents Mid-term evaluation terms of reference ................................................................................. 5 Appendix A

Evaluation questions and methods ....................................................................................... 13 Appendix B

Evaluation questions, indicators and data sources (included IDRC/DFID indicators) ........ 15 Appendix C

Cross reference table of DFID indicators with the MTE evaluation indicators .................... 23 Appendix D

Cross referenced table of IDRC indicators with the MTE evaluation indicators ................... 24 Appendix E

List of interviewees ................................................................................................................ 25 Appendix F

Interviews topic guide ........................................................................................................... 31 Appendix G

Survey questionnaire ......................................................................................................... 35 Recommendations made by previous assessments per topic and issue ......................... 36 Survey and counterfactual analysis .................................................................................. 38 Bibliometric analysis ............................................................................................................ 66 Appendix K

IDRC and DFID completed indicators by pillars ................................................................. 80 Appendix L

SWOT ................................................................................................................................... 84 Appendix M

Figures Figure 1 AIMS admission requirements .................................................................................................................... 38 Figure 2 Acceptance rates throughout the year .......................................................................................................... 39 Figure 3 Acceptance rates throughout the year .......................................................................................................... 39 Figure 4 Overview of where non-admitted applicants end up in the first year after they were not selected / turned down the offer for AIMS ................................................................................................................................ 40 Figure 5 Overview of the reasons why non-admitted applicants did not participate in AIMS .......................... 41 Figure 6 How did applicants first learn about AIMS? .............................................................................................. 41 Figure 7 Age distribution of alumni and applicants when applying to AIMS ........................................................... 42 Figure 8 Current age distribution of alumni .............................................................................................................. 42 Figure 9 Years of relevant work experience of alumni prior to attending AIMS ....................................................... 42 Figure 10 Occupation type prior to applying to AIMS ................................................................................................ 43 Figure 11 Highest academic qualification prior to attending AIMS ........................................................................... 43 Figure 12 Overview of whether non-admitted applicants, knowing what they know now of AIMS, would apply again to AIMS ............................................................................................................................................................... 43 Figure 13 Overview of whether non-admitted applicants, knowing what they know now of AIMS, would recommend AIMS to a friend, colleague or peer ......................................................................................................... 44 Figure 14 Assessment of course elements of AIMS by alumni .................................................................................... 44 Figure 15 Number of graduated and not graduates AIMS alumni .............................................................................. 45 Figure 16 Number of graduated students, by centre, by gender, by start year till 2013 ............................................ 45 Figure 17 Nationality of AIMS graduates (up to and including 2013) ........................................................................ 46

ii

Figure 18 Assessment of the contribution of AIMS to the level of development of skills by alumni ........................ 48 Figure 19 Assessment of the contribution of AIMS to increase of focus on Africa’s challenges by alumni ............... 49 Figure 20 Highest degree (finished or currently engaged) ......................................................................................... 49 Figure 21 Alumni Master degrees: Finished and currently engaged ......................................................................... 50 Figure 22 Key figures on academic output ................................................................................................................. 50 Figure 23 Current occupation type by cohort (alumni and non participants), based on survey sample .................. 51 Figure 24 Use of skills by alumni in their current occupation .................................................................................... 52 Figure 25 Thematic area of employment ..................................................................................................................... 52 Figure 26 Number of alumni that indicate to have been (or currently are) mathematics teachers, tutors or lecturers since attending AIMS. .................................................................................................................................. 53 Figure 27 Number of non-admitted applicants that indicate to have been (or currently are) mathematics teachers, tutors or lecturers since applying to AIMS. ................................................................................................................. 53 Figure 28 Current organisation (survey) ..................................................................................................................... 54 Figure 29 Current location of AIMS alumni ................................................................................................................ 54 Figure 30 Gross Income development ........................................................................................................................ 55 Figure 31 Assessment of the monthly gross income of alumni in case they would not have participated in AIMS. . 56 Figure 32 The extent to which alumni and non-admitted applicants focus on Africa’s challenges in their current occupation .................................................................................................................................................................... 56 Figure 33 View of alumni on the current top 3 challenges for Africa ......................................................................... 57 Figure 34 Official awards received by alumni and non-admitted applicants ............................................................. 57 Figure 35 Companies started by alumni and non-admitted applicants ..................................................................... 58 Figure 36 Closely involvement in patent applications by alumni and non-admitted applicants .............................. 58 Figure 37 Closely involved in the launch of any products or services by alumni and non-admitted applicants ....... 59 Figure 38 Overview of sectors in which alumni and non-admitted applicants want to continue their career. ......... 59 Figure 39 Assessment of alumni and non-admitted applicants about the career opportunities in Africa for Science, Technology, Engineering and Mathematics (STEM). ................................................................................................ 60 Figure 40 Overview of where alumni and non-admitted applicants will look for future career opportunities. ...... 60 Figure 41 Assessment of the contribution of AIMS to get in contact with people that are currently important in the professional network of alumni ................................................................................................................................... 61 Figure 42 Assessment of alumni about the impact of AIMS on their career opportunities ....................................... 61 Figure 43 Assessment of alumni about the level of success in their career without AIMS; Assessment of alumni about the amount of time to achieve the same in their career without AIMS ............................................................ 62 Figure 44 Overview of alumni and their participation in Post-AIMS Research Opportunities. (Multiple answers allowed) ........................................................................................................................................................................ 63 Figure 45 Likelihood that alumni would have been able to arrange a different source of funding without this/these research opportunity/ies ............................................................................................................................................. 63 Figure 46 Consequences of hypothetically not having this/these research opportunity/ies ..................................... 63 Figure 47 Model analysis: Presence of significant effects (analysis through ordened logits, probit or conventional regression where appropriate) ..................................................................................................................................... 64 Figure 48 Overview of the number of people that alumni and non-admitted applicants are responsible for in their current occupation ....................................................................................................................................................... 65 Figure 49 Overview of the ranges of the monthly gross income in USD of alumni and non-admitted applicants of their current occupation .............................................................................................................................................. 65

iii

Figure 50 AIMS publications per year (non-cumulative) .......................................................................................... 68 Figure 51 Top 10 researchers that have worked at AIMS ........................................................................................... 68 Figure 52 Key citation figures ..................................................................................................................................... 69 Figure 53 Total number of citations and citations per paper ..................................................................................... 69 Figure 54 Top 5 cited papers ....................................................................................................................................... 70 Figure 55 Impact factors for the top 5 journals with most AIMS publications .......................................................... 70 Figure 56 The average Source Normalized Impact per Paper (SNIP) over 2004 – 2014* ........................................ 71 Figure 57 Network analysis of co-publications ........................................................................................................... 71 Figure 58 Top 10 institutes in terms of co-publications ............................................................................................. 72 Figure 59 Top 5 African institutes in terms of co-publications .................................................................................. 72 Figure 60 Nature Index ranking .................................................................................................................................. 73 Figure 61 Key figures .................................................................................................................................................... 74 Figure 62 Alumni publications per year ...................................................................................................................... 75 Figure 63 Top 10 alumni researchers .......................................................................................................................... 75 Figure 64 Number of alumni with per AIMS year and h-index .................................................................................. 76 Figure 65 Key citation figures ...................................................................................................................................... 76 Figure 66 Total number of citations and citations per paper - alumni ...................................................................... 77 Figure 67 Top 5 cited papers ........................................................................................................................................ 77 Figure 68 Impact factors for the top 5 journals with most the alumni publications ................................................. 78 Figure 69 The average Source Normalized Impact per Paper (SNIP) over 2004 – 2014 .......................................... 78 Figure 70 Network analysis of co-publications ........................................................................................................... 79 Figure 71 Top 15 institutes in terms of co-publications .............................................................................................. 79

5

Mid-term evaluation terms of reference Appendix A

Term%s%of% Referenc%e%

Ind%e%p%e%n%d%e%n%t%M%id% T%erm%E%valuati%o%n%o%f%t%he% A%I%M%S%:%IDR%C%:%DFI%D%Pro%gram% %

1.0 INT%R%O%D%U%C%T%ION

The% African% Institute% for%Mathematical% Sciences% (AIMS),% was% established% in% 2003% in% Cape% Town,% South%

Africa.%It%is%an%innovative%post:graduate%centre%for%education,%research%and%public%engagement,%providing%

advanced,%broadly%applicable%mathematical%skills%to%talented%students%from%all%over%Africa.%AIMS%offers%a%

number% of% educational% programs% highlighted% by% its% one:year% Master’s% in% Mathematical% Sciences.% The%

Master’s%program%trains%hundreds%of%talented%women%and%men%African%scholars%each%year,%and%prepares%

them% for% leadership%careers% in%academia,%government%and% industry.% Research% is% also%an% integral%part%of%

the% AIMS% work.% Since% its% launch,% AIMS% has% won% global% recognition% as% a% centre% of% excellence% for%

postgraduate%education%and%research.%Building%on%its%success,%AIMS% launched%the%Next%Einstein% Initiative%

(NEI)%in%2008%to%build%a%critical%mass%of%scientific%and%technical%talent%in%Africa,%capable%of%driving%progress%

across%the%continent.%

AIMS:NEI%key%objectives%are%to:%

• Promote%mathematics%and%science%in%Africa;%

• Recruit%and%train%talented%women%and%men%students%and%teachers%in%mathematical%sciences;%and%

• Build%capacity%for%African%initiatives%in%education,%research%and%technology.%

In% July% 2010,% the% Government% of% Canada% contributed% CA$20% million% to% the% African% Institute% for%

Mathematical%Sciences%(AIMS).%The%grant,%to%be%administered%by%the%International%Development%Research%

Center%(IDRC),%is%to%support%the%AIMS%centre%in%South%Africa;%and%establish%three%new%centres%in%Senegal,%

Ghana% and% Cameroon% by% December% 20151

.% In% November% 2012,% the% UK% Department% for% International%

Development% (DFID)% entered% into% a% match% funding% arrangement% with% IDRC,% contributing% a% sum% of%

approximately%£18,160,000,% to% complement%and%build%on%Canada’s%contribution% to%AIMS:NEI2

.%The%DFID%

funds%were%geared%towards%the%establishment%and%initial%operating%costs%of%two%additional%AIMS%centres%

in%Tanzania%and%Benin;%provide%consolidation%funding%for%the%centres%in%South%Africa,%Senegal,%Ghana%and%

Cameroon;% and% contribute% to% the% enhancement% of% the% AIMS% curriculum% and% learning% environment% in%

order% to% facilitate% career% opportunities% within% relevant% sectors% and% industries% in% Africa.% As% part% of% the%

agreement,% IDRC%provided%a% further% contribution%of%CA%$2%million% to%build% the% research% capacity%of% the%

AIMS%network.%To% facilitate%efficient%management%and%oversight%of%the%Program,% IDRC% is% responsible% for%

managing%DFID’s%contribution.%

The%goal%of% the%AIMS:IDRC:DFID%Program% is% to%provide%advanced%training% in%applied%mathematics% to%top%

African%students%enabling%them%to%pursue%high%quality%post%graduate%studies%and%eventually%contribute%as%

future%leaders%to%the%further%economic,%political%and%educational%advancement%of%the%African%continent.%

1" Canada’s"contribution"was"originally"meant"to"support"the"establishment"of"four"new"centres;"however"by"mutual"agreement,"plans"to"establish"a"centre"in"Nigeria"were"eliminated"from"the"Objectives."

2" The"grant"agreement"between"AIMS"and"IDRC"was"signed"in"May"2013."

2

The&specific&objectives&of&the&program&are&to:&&

1. support&existing&AIMS&Centres;&&

2. establish&new&AIMS&Centres;&&

3. update&and&implement&a&monitoring&and&evaluation&plan;&&

4. support&and&strengthen&AIMS:NEI's&Secretariat;&&

5. develop&a&common&set&of&administrative&and&operating&procedures&across&the&AIMS&network;&&

6. form&a&unified&learning&network&with&an&appropriate&balance&between&a&central&approach&and&

local&variation;&&

7. enhance&post&graduate&opportunities&for&AIMS&centre&graduates,&including&complementing&AIMS:

NEI's&curriculum&with&employability,&entrepreneurship&and&business&skills&modules;&and&&

8. improve&the&financial&stability&(sustainability)&of&AIMS:NEI.&&

&

In& keeping& with& the& terms& of& the& grant& agreement,& AIMS,& in& collaboration& with& IDRC& and& DFID,& has&

commissioned&a&Mid:term&evaluation&(MTE)&of&the&Program.&&

&

The&MTE&is&important&and&timely&as&it&is&an&opportunity&for&AIMS&to&clarify&and&assess&the&initiatives&under&

this&Program,&as&well&as&highlight&areas&of&programmatic&strength&and&opportunities&for&change,&especially&

now&that&the&organization&is&growing&and&working&to&advance&its&theory&of&change&(ToC).&&

&

The&MTE&will& serve& learning& and& accountability& purposes& for& AIMS:NEI,& IDRC& and&DFID&who&will& be& the&

primary&users&of& the&evaluation.& Essentially,& it&will& seek& to& identify:& progress& against&objectives,&what& is&

going&well,&highlight&early&results,&what&are&the&challenges,&what&can&be&improved,&and&what&are&the&key&

recommendations&to&ensure&progress&towards&the&objectives&of&the&Program.&&

&&

2 .0& & & EVALUATION&OBJECTIVES&&&SCOPE&

The&MTE&will&be&guided&by&general&OECD:DAC&criteria&and&will&have&the&following&objectives:&&

• Determine& the& progress& towards& achieving& the& Program’s& objectives& and& intended& outcomes&

specific&to:&

o AIMS&operational&plans&and&mechanisms&made&possible&by&the&Program;&&

o AIMS&academic&programs&specific&to&areas&supported&by&the&Program;&and&&

o AIMS&research&component&specific&to&areas&supported&by&the&Program.&

• Identify&best&practices,&opportunities,&lessons&and&corrective&actions&needed&for&the&next&phase&of&

implementation&and&to&ensure&the&realization&of&the&expected&results.&

&

2 .1&Evaluation&Scope&

The& investigation&will& focus&on& the&areas&of&AIMS&operation,& academic& and& research&work& that& is& being&

funded&under&the&AIMS:IDRC:DFID&Program.&The&mid:term&evaluation&will&focus&on&the&AIMS&operations&

in&South&Africa&(both&the&Centre&and&the&Secretariat),&and&centers&in&Ghana,&Senegal&and&Cameroon.&At&all&

four& locations,& the& evaluation& will& assess& AIMS’& operations& (administration& and& finance& arrangements,&

systems& etc.),& research& and& academic& work.& & Specifically,& the& MTE& will& review& of& the& AIMS& Master’s&

Program,&AIMS&Industry&Initiative&and&the&Research&program.&Also,&there&will&be&a&global&survey&of&AIMS&

students&(current&and&graduated)&to&assess&and&document&the&perceptions&of&AIMS’s&direct&beneficiaries.&

&

3

The&mid:term&evaluation&will&seek&to&capture&information&from&all&levels&of&stakeholders&including:&&

• Selected& AIMS& senior& management& team& (Secretariat& and& Centers)& and& staff& in,& UK,& Germany,&

Canada3

;&

• AIMS&professors&and&tutors&(regional&and&international,&etc.);&

• AIMS&students&and&alumni&(Alumni&survey);&

• AIMS&partners&in&government,&targeted&sector&agencies,&institutions&and&companies;&

• AIMS&researchers&and&scientists&and&academia;&

• Employers&of&AIMS&graduates&to&date&(representative&sample);&

• Management&and&program&staff&at&IDRC&and&DFID.&

&

2.2&Evaluation&Methodology&and&Approaches&

The& methodology& should& combine& a& wide& range& of& methods& (e.g.& quantitative,& qualitative,& and&

participatory),&tools&and&information&sources&to&allow&triangulation&of&information&and&ensure&impartiality.&

In&particular,&there&should&be&an&extensive&review&of&all&relevant&documents;&a&representative&sample&of&

key&stakeholders&should&be&consulted,&their&opinions&and&feedback&considered&in&order&to&determine&the&

scope&and&quality&of&the&outputs&and&ensure&a&comprehensive&understanding&of&diverse&perspectives&on&

issues,& performance& and& results.& The& approaches& should& assess& the& extent& to& which& each& of& the&

implemented& outputs& fulfilled& their& initial& objectives.& It& is& expected& that& the& analysis& will& focus& on& the&

immediate&outcome& level& results& and&will& comment&on& the&extent& to&which& the&Program& is&progressing&

towards&higher&level&outcomes&and&impacts.&To&measure&this&progress,&the&evaluation&will&be&informed&by&

the&DFID& Logframe& and& IDRC’s& Schedule& of& Performance& Indicators& (SPIs).& Finally,& the& analysis&must& be&

informed&by&gender:disaggregated&data.&

&

The&methodology&should&be&presented&in&the&inception&report.&It&will&include&an&evaluation&matrix&aligned&

to&the&key&evaluation&questions&and&data&collection&tool&to&guide&the&desk&review&and&field&work;&outline&

the& evaluation& team’s& overall& work& plan& including& site& visits& schedules& and& division& of& responsibilities&

between& evaluation& team& members.& The& methodology& will& ensure& that& all& four& centres/countries& are&

considered&in&the&assessment.&To&allow&aggregation&of&findings&across&the&countries,&the&team&will&use&the&

same&data&tools&in&each&country&visit.&

&&

&

2 .3&Evaluation&Management&

The& evaluation&will& be&managed& by& AIMS:NEI.& AIMS’s& Senior& Evaluation& Adviser&will& be& responsible& for&

hiring&the&evaluation&team;&preparing&and&managing&all&evaluation&related&contracts&in&collaboration&with&

the&Executive&Vice&President;&and&monitoring&the&evaluation&process&against&the&deliverables&articulated&

in&the&Terms&of&References&(TORS).&

An& Advisory& Committee& comprised& of& AIMS& Senior& Evaluation& Adviser,& one& member& of& AIMS& Senior&

Management& team,&and&representatives&of& IDRC&and&DFID.&The&Advisory&Committee&will&be& responsible&

for&reviewing&and&approving&the&MTE&TORs,&the&Inception&report&and&the&draft&Evaluation&report.&

3

&The&Canada,&UK&and&Germany&locations&are&AIMS&fundraising&and&advancement&offices.&

"

4

3.0& & & KEY&EVALUATION&QUESTIONS&

&

Key&questions&have&been&developed&to&guide&this&evaluation&based&on&initial&desk&review&and&discussions&

with&AIMS&Program&management&team.&They&will&be&further&detailed&in&a&matrix&of&evaluation&questions&

that&will&be&prepared&by&the&evaluation&team&during&the&inception&phase.&

&

3 .1&Program&Relevance& &

This&section&will&focus&on&the&extent&to&which&the&activities&and&outputs&of&the&AIMS:IDRC:DFID&Program&

are&consistent&with&its&goal,&objectives,&intended&outcomes&and&impacts.&&

&

i. Are"the"activities"and"outputs"of"the"Program"consistent"with"the"overall"goal"and"the"attainment"of"its"objectives?"

ii. Are& the& activities& and& outputs& of& the& program& consistent& with& the& intended& outcomes,& and&

impacts?&

&

3.2&Effect iveness&

What"evidence" is" there" to" indicate" the"Program" is"advancing" towards" the"achievement"of" its"goal"and"objectives?&Specifically:&&

I. To&what&extent&has&AIMS&expansion&progressed&as&planned&at&the&onset&of&the&Program?&

II. To&what&extent&has&a&common&set&of&administrative&and&operating&procedures&and&policies&been&

developed& and& implemented& across& the& AIMS& network?& How& have& they& contributed& to& AIMS&

growth&and&success&over&the&period?&

III. To&what&extent&has&a&unified& learning&network&been&formed?&How&effective&are&AIMS&academic&

programs&specific&to&recruitment,&enrollment,&retention,&graduation,&and&student&performance?&

IV. To& what& extent& has& the& Program& been& able& to& enhance& post:graduate& opportunities& for& AIMS&

alumni&through&the&Industry&Initiative,&small&research&grants&and&research&Chairs?&What&could&be&

done&differently&to&enhance&post:graduate&opportunities?&&

V. How&effective&has&the&curriculum&enhancement&with&employability&skills&been&and&to&what&extent&

has&this&component&supported&or&enhanced&student&and&/or&alumni&ability&to&get&internships&and&

permanent&job&placements?&

VI. To&what&extent&has&AIMS&institutional&capacity&been&strengthened?&&

VII. How&effective&has&the&Monitoring&and&Evaluation&system&now&in&place&supporting&results:oriented&

thinking,&reporting&and&strategic&and&institutional&decision:making?&

VIII. How&and& to&what& extent&has& the&Program&contributed& to& the&provision&of& advanced& training& in&

applied&mathematics& to& top& African& students?&How&does& AIMS& compare& to& other& programs?& Is&

there& any& evidence& that& AIMS& alumni& are& pursuing& high& quality& post:graduate& studies& and& that&

they&will&eventually&contribute&to&the&further&economic,&political&and&educational&advancement&of&

the&African&continent?&&

&

3 .3&Eff ic iency&

The&MTE&will&explore&the&efficiency&of&the&AIMS&operations&specific&to&the&expansion&of&the&centers,&the&

human&resources&additions&and&updates&implemented,&the&quality&of&the&systems&and&processes&

introduced&to&execute&the&outputs&and&enhance&performance.&&

&

5

i. &&&How&and&to&what&extent&does&the&Program&present&Value&for&Money?&&

3.4&Sustainabi l i ty & &

These&questions&will&examine&the&extent&to&which&sustainability&mechanisms&have&been&introduced.&

&

i. How& and& to& what& extent& are& AIMS& students,& alumni,& donors,& and& governments& promoting& the&

sustainability&of&AIMS’&activities&after&the&end&of&the&Program?&&&

&

ii. How& and& to& what& extent& has& the& financial& stability& of& the& AIMS:NEI& been& improved?& How&

successful&has&AIMS&been&in&securing&funding?&How&diversified&are&the&sources&of&funding&secured&

by&AIMS?&How&can&the&organization’s&financial&stability&(sustainability)&be&improved?&&

&

4 .0& & & EVALUATION&PHASES& &

& &

4 .1& Inception&phase&

This&phase&is&expected&to&last&for&one&month&and&is&meant&to&ensure&that&the&evaluation&team&is&fully&

prepared&before&undertaking&the&site&visits.&It&includes:&

i. Desk&Review&of&existing&documents&including:&relevant&policies&and&strategies,&proposals,&

assessments,&monitoring/&data,&progress&or&donor&reports,&etc.&

ii. Inception&briefing&at&AIMS:NEI&Secretariat&in&South&Africa,&with&internal&stakeholders&and&

reference&groups.&

iii. Consultation&with&key&external&stakeholders.&&

iv. Drafting&of&inception&report&including&evaluation&matrix,&methodology&and&data&collection&

tools&

v. Review&and&sign&off&of&inception&report&including&revised&MTE&TOR,&plan,&methodology,&tools&

by&the&Evaluation&Advisory&Committee&

vi. Consultation&with&AIMS&Senior&management&team&to&determine&the&persons&to&be&met&at&

country&level&

vii. Finalization&of&logistics&for&field&visits&

&

&

4 .2&Data&Col lect ion&(F ie ld&Mission&phase)&

This"phase"is"expected"to"last"six"to"seven"weeks."

i. Visits&to&the&four&countries&(South&Africa,&Senegal,&Ghana,&and&Cameroon)&where&AIMS&centres&have&

been&opened&before&20144

.&Seven&days&minimum&per&country.&The&evaluation&team&is&expected&to&

split&in&two&groups&each&covering&two&countries.&

ii. Each&field&visit&will&include:&

: Initial& introduction& meeting:& AIMS& Centre& Management& and& selected& program& staff,& key&

stakeholders&and&partners.&

: Discussions,& interviews,& surveys,& focus& groups& etc.& with& key& stakeholders& including& AIMS&

staff,&students,&professors,&researchers,&academia,&government&officials,&employers&to&elicit&

feedback&on&the&Program&

: AIMS&Alumni&survey&

4"A"fifth"AIMS"centre"opened"in"2014"in"Tanzania."Since"it"is"so"recent,"it"will"not"be"part"of"this"evaluation.""

6

: Debriefing&of&AIMS&Centre&staff&and&key&stakeholders&on&main& findings&at& the&end&of&each&

country&visit.&&

&

4 .3&Report ing&phase&(Preparation, &Submission&and&Approval & for &Dissemination)&

This"phase"is"expected"to"last"for"five"weeks"and"is"meant"to"share"emerging"findings"with"evaluation"stakeholders"as"a"means"to"validating"data"collection"insights"and"detecting"evidence"gaps"as"part"of"the"consolidation"of"conclusions"for"report"writing."

i. General&debriefing&sessions&at&AIMS&HQ&SA&with&1)&Senior&Management&and&staff,&2)&Program&

teams&3)&reference&groups;&and&4)&IDRC&and&DFID&representatives;&and&5)&other&key&stakeholders,&

including&contributors&from&other&centers&via&teleconference&

ii. Submission&of&first&draft&evaluation&report&

iii. Review&of&draft&report&by&the&Evaluation&Advisory&Committee&for&content&accuracy&and&quality&

iv. Incorporation&of&comments&and&revision&of&the&report&

v. Submission&of&summary&report&

vi. Submission&of&final&draft&report&with&supporting&data,&appendices,&etc.&&

&

4 .4&Evaluation&Learning&Workshop&

Evaluation& learning& workshop& will& be& planned& and& facilitated& by& AIMS&Monitoring,& Evaluation& and&

Learning& team.& The& purpose& of& the& workshop& will& be& to& share& the& findings& and& lessons& of& the&

evaluation.&The&evaluation&team&leader&may&contribute&to&the&exercise.&

&

5 .0& & & Evaluation&Ethics &and&Standards&

&

Evaluators& will& ensure& that& appropriate& international& development& evaluation& ethical& standards& and&

guidelines& will& be& observed& in& the& implementation& of& the& evaluation.& Quality& will& be& assessed& on& the&

extent&to&which&the&evaluation&demonstrates&that&it&has&fulfilled&its&purpose,&and&will&use&internationally&

recognized&evaluation&standards.&&&

6 .0& & & EVALUATION&DELIVERABLES&

&

The&team&leader&is&responsible&for&the&timely&submission&of&the&deliverables&to&the&Evaluation&Advisory&

Committee.&&

&

1)&& Inception& report & will& expand& on& the& key& issues& including& evaluation& scope,& questions,& key&

informant/&stakeholder&list.&It&will&also&present&the&evaluation&matrix,&the&methodology;&including&strategy&

for&the&Alumni&Survey;&and&will&propose&data&collection&tools.&

&

2)& Aide:mémoire& of& key& findings& will& be& prepared& at& the& end& of& each& country& visit.& The& aide:

mémoires&will& follow& the& same& format& to& facilitate& analysis,& comparability& and& aggregation& of& findings.&

They& will& be& used& to& support& the& debriefing& with& key& stakeholders.& An& aide:mémoire& consolidating&

findings&from&the&four&centres&will&be&prepared&for&the&debriefing&in&South&Africa&upon&completion&of&the&

field&visits.&A&total&of&five&aide&memoires&will&be&prepared&and&shared&for&review&and&approval.&&

&

3)& Evaluation& report & will& build& on& the& findings& of& the& desk& review& and& country& visits.& Findings&

should& be& evidence:based& and& relevant& to& the& evaluation& objectives& and& questions.& There& should& be& a&

7

logical&flow&from&findings&to&conclusions&and&from&conclusions&to&recommendations.&The&first&draft&will&be&

submitted&within&30&days&of& the&end&of& the& field& trips.&Final& report& submitted&will& include& the& feedback&

from& the&Advisory& Committee& and& other& relevant& AIMS& staff.& The& submission&will& include& all& data& files,&

annexes&and&presentations.&&

&

The&final&evaluation&report&will&be&a&maximum&of&30&pages&including&an&Executive&Summary.&The&executive&

summary&of&the&evaluation&report&will&be&presented&to&the&AIMS&Senior&Management&team&and&donors&as&

the&summary&evaluation&report.&&

7&.0& & TIMELINE&

The&table&below&outlines&the&timeline&for&the&deliverables&envisaged&for&the&evaluation.&The&specific&

details&are&subject&for&negotiation&with&the&evaluation&team&leader&to&ensure&timely&completion&of&the&

evaluation&and&delivery&of&the&evaluation&report.&&

&

Evaluation&Act iv it ies &

and&Del iverables&

T imeline&

2015&

January& February& March& & Apri l & & May& June& July& &

Inception&Phase&

• Initial&Desk&Review&and&Interview&with&Key&

Stakeholders&

• Update&TORs&,&Evaluation&Work&plan&&&Methodology&

• Develop&Evaluation&Tools&&• Submit&Inception&report&for&

review&and&approval&&

& & & & & & & & & & & & & & & & x& x& x& x& & & & & & & & & & & & &

Data&Col lect ion&

• Missions&to&4&AIMS&Centres&

• Interviews&with&boundary&partners&

• Alumni&Survey&Executed&

• Submission&of&Aide:mémoires&

from&each&centre&

& & & & & & & & & & & & & & & & & & x& x& x& x& x& x& & & & & & & & &

Debrief &with&key&

stakeholders&

& & & & & & & & & & & & & & & & & & & & & & x& x& & & & & & & & &

Data&Synthesis &and&Report&

Writ ing&

& & & & & & & & & & & & & & & & & & & & & & x& x& x& x& & & & & & &

Report&submission&and&

approval &

• Submit&first&draft&evaluation&

report&&

• Summary&Report&prepared&

& & & & & & & & & & & & & & & & & & & & & & & & & x& x& & & & & &

&

&

&

&

&

8

8.0& & & EVALUATION&TEAM&COMPOSITION&AND&SELECTION&CRITERIA&

&

To& ensure& the& independence& of& the& evaluation& and& the& credibility& of& the& findings,& a& team& of& external&

consultants,&identified&through&a&transparent&selection&process,&will&conduct&the&evaluation.&The&team&will&

comprise& three& professionals& with& an& appropriate& balance& of& expertise& in& evaluation& methodologies,&

relevant& subject:matter& or& technical& expertise& and& practical& experience.& & The& evaluation& team& will& be&

supported&by&an&Administration&and&Logistics&officer.&&

&

The& team& leader& will& report& to& AIMS& Senior& M&E& Advisor.& He/she& should& have& strong& evaluation&

experience&in&international&development&programmes,&as&well&as&excellent&analytical,&team&management&

and& communication& skills& (verbal& and& written).& He/she& should& be& able& to& conceptualize& complex&

evaluations&and&design&the&subsequent&evaluation&approach&and&methodology.&He/she&should&be&fluent&in&

English&and&French.&&

&

The&other&two&evaluators&should&be&area/subject&experts&in&Higher&Education&programming,&in&particular&

STEM&education&programmes&and&International&Development.&One&of&the&evaluators&must&have&extensive&

experience& living&and&working& in&Africa&and&have&evaluated&programmes&and&or&policies&associated&with&

Africa’s&development&priorities.&

&

Team&members&will& report& to& the& Evaluation& Team& Leader& and& are& responsible& for& delivering& inputs& as&

agreed.&

&

&

9.0$Evaluation$Period &&

The$evaluation$will$be$conducted$over$a$28month$period$(May$18June$30,$2015).&$10.0$Appl icat ion$Procedure: &&

All$ interested,$ qualified$ applicants$ are$ to$ forward$ the$ letters$ of$ interest$ and$ CVS$ which$ must$include:$ $their$ availability,$ relevant$ experience$ and$ experience$ evaluating$ either/or$ both$ Science,$Technology,$ Engineering$ and$ Mathematics$ (STEM)$ higher$ education$ programmes;$ and$ development$programmes$in$Africa;$willingness$to$travel$to$AIMS$centers$in$Africa;$and$at$least$two$samples$of$relevant$evaluations.$$The$Applicants$are$also$invited$to$propose$at$least$one$other$evaluator$who$can$complement$their$skills$in$conducting$this$evaluation.&&

$A l l $ letters $ of $ interest $ are$ to$ be$ submitted$ to$ AIMS$ MEL$ Off ice: $email : [email protected]$with$ the$ subject $ l ine$ AIMS8IDRC8DFID$ MTE.$ Submission$deadl ine$ is $5 $pm$EST$on$10$Apri l $2015. &&

13

Evaluation questions and methods Appendix B

Eval

uatio

n pi

llar

Evaluation questions Gen

eral

des

k R

esea

rch

Logi

cal f

ram

ewor

k an

alys

is

Act

ivity

map

ping

Inte

rnal

int

ervi

ews

Exte

rnal

inte

rvie

ws

Alu

mni

surv

ey

Bibl

iom

etri

cs

Are project objectives realistic and consistent with national / African and international (donor) objectives?

��

Does AIMS respond to the needs of the target population?

��

Is the AIMs design appropriate for achieving the project’s core objectives? Is the model design relevant and appropriate to the specific context?

��

How coherent are the objectives in terms of how they fit in with the policies, programs and projects undertaken by the government and other development partners?

��

Did the project benefit from available knowledge (for example, the experience of other similar projects in the area or in the country) during its design and implementation?

��

Has design been retrofitted? ��

What are the main factors that contributed to a positive or less positive assessment of relevance?

��

Eff

ecti

ven

ess

To what extent have the objectives of AIMS and its components been attained both in quantitative and in qualitative terms? Is it likely that so far unattained objectives may be accomplished in full/in part?

��

To what extent did the AIMS team implement the initially planned activities?

��

What are the drivers and barriers to achieve objectives?

��

Eff

icie

nc Are the results achieved adequate for the budget spent? ��

Was the allocation of resources among different aspects appropriate to achieve the results?

��

14

Eva

luat

ion

pil

lar

Evaluation questions Gen

eral

des

k R

esea

rch

Logi

cal f

ram

ewor

k an

alys

is

Act

ivity

map

ping

Inte

rnal

int

ervi

ews

Exte

rnal

inte

rvie

ws

Alu

mni

surv

ey

Bibl

iom

etri

cs

Is the management structure efficient for the achievement of the expected outcomes?

��

What factors help account for project efficiency performance?

��

What are the administrative costs per beneficiary and how do they compare with other similar institutes?

��

Impa

ct

What is the impact of AIMS activities on productivity and growth in Africa?

��

What is the impact of AIMS activities on African (Scientific, Economic, Political) leadership?

��

What is the impact of AIMS on mathematical and scientific research excellence in Africa?

��

What is the impact of AIMS activities policy and innovation in Africa?

��

What is the impact of AIMS activities on interest for and skills in Maths and Sciences in Africa?

��

What is the impact of AIMS activities on STEM policies in Africa?

��

What is the impact of AIMS activities on AIMS as a flagship initiative to STEM educational reform in Africa?

��

Sust

ain

abil

ity

Was a specific strategy or approach prepared and agreed upon by key partners to ensure sustainability?

��

What are the chances that benefits generated will continue what factors militate in favour of or against maintaining benefit?

��

To what extent were sustainability mechanisms introduced?

��

What can be learned from best practices and worst cases for the next phase of the implementation?

��

What recommendations can be made based on the MTE to AIMS?

��

15

Evaluation questions, indicators and data sources (included IDRC/DFID indicators) Appendix C

Top

ic AIMS evaluation questions Indicator Data Source

Rel

evan

ce


How coherent are the objectives in terms of how they fit in with the policies, programs and projects undertaken by the government and other development partners?

% national, African and international strategies/policies/programs and projects aligned

% of respondents who agree that AIMS objectives fit with other policies and programs undertaken

% of respondents who agree there is a need to promote maths and science in Africa

% of respondents who indicate a lack of educational and research opportunities

Review of national, African and international (donor) strategies and policies

Amount of funds/budgets available for STEM in Africa

Interviews with national HEI stakeholders and international donors

Does AIMS respond to the needs of the target population? % of respondents who agree that AIMS promote successfully maths and science in Africa, increases opportunities (jobs, educational and research)

Interviews with staff, students, policy makers and industry representatives

Online surveys for alumni

Is the AIMS design relevant?

- Is the AIMs design appropriate for achieving the project’s core objectives? Is the model design relevant and appropriate to the specific context?

- Has design been retrofitted through time?

% of respondents who agree that AIMS design is appropriate

Examples of retrofit in design

Interviews with staff, students, policy makers, lecturers & tutors, academic supervisors of AIMS alumni and industry

Online surveys for alumni


% of respondents who confirm interactions with other initiatives

% of respondents (not part of AIMS initiative) who agree that AIMS is part of the broader community of practice for mathematical sciences in Africa

No of projects from which AIMS benefited from

Review of documentation

Interviews with staff, donors, and external stakeholders

Did project objectives remain relevant over the period of time required for implementation?

% respondents who agree that project objectives remained relevant

Interviews with staff, donors, external stakeholders

What are the main factors that contributed to a positive or less positive assessment of relevance?

Examples Interviews with staff, donors, external stakeholders

Top


How does AIMS compare to other programs?

Eff

ecti

ven

ess

To what extent did the AIMS team implement the initially planned activities? Where activities implemented on time?

All activities implementation indicators, and notably`

Cost per student (IDRC 3.4.1.) N° of centers (DFID Output 1.1.) N° of bursaries for AIMS students (DFID Output 2.3)

Overview of program planning

Reporting data

To what extent have the intended outputs of AIMS and its components been attained both in quantitative and in qualitative terms?

Education

- To what extent has a unified learning network been formed?

- To what extent did AIMS contributed to the training of skilled graduates

- How effective are AIMS academic programs specific to recruitment, enrolment, retention, graduation and student performance?

- How and to what extent has the Program contributed to the provision of advanced training in applied Mathematics to top African students?

- To what extent has the program been able to enhance postgraduate opportunities for AIMS alumni through the Industry Initiative?

- How effective has the curriculum enhancement with employability skills been and to what extent has it supported student and/or alumni ability to get internships and permanent job placements

Research

- To what extent has the Program been able to enhance postgraduate opportunities for AIMS alumni through small research grants and research Chairs? What could be done differently to enhance post-graduate opportunities?

-

N° of skilled graduates (IDRC 3.1.1)

N° of applicants/N° of dropouts

N° of completions

Skills level on the 5 formative areas

N° of bursaries post AIMS (IDRC 100.1.2 etc)

Time to completion

N° of interns (in industry) (DFID Output 2.5)

Time of internship (during or after the course)

Quality of internship (use of math skills)

N° of research projects completed by type

N° of applications/success rate (DFID Output 3.2; IDRC 3.3.2)

Alumni data base

Reporting data

Alumni survey

Reporting data

Alumni Survey

Top


Outreach

- To what extent has the Program been able to enhance qualification of teachers? and engagement of public?

Organisation

- To what extent has a common set of administrative and operating procedures and policies been developed and implemented across the AIMS network

- How have administrative and operating procedures and policies contributed to AIMS growth and success over the period?

- How have administrative and operating procedures and policies contributed to AIMS growth and success over the period?

- What factors account for success or failure in opening new Centers ?

N° of teachers trained

Type of activities per center

Quality of the offer per center

N° of conferences/ public lectures

N° of workshops

N° of attendees (IDRC 3.3.1; DFID Output 3.1)

Type of attendees (policy/practitioners)

N° of events spearheaded by AIMS)

Turnover of staff - retention / ability of staff

Gender ratio (Output 1.2)

Efficient IT systems

% of key positions filled

Management systems performance of staff (DFID Output 4.1)

Budget linked to annual planning

Overall overhead

Ratio of admin staff

Interaction between the centers

Does staff have an understanding of the AIMS values

Quality of M&E data/ assessment of the M&E system/lessons learned (IDRC 3.5; DFID Output 5.2)

Financial stability of AIMS and diversification of funding sources (DFID Output 4.2) ; M&E training

Initiatives that went cool: Benin, Ethiopia Initiatives in: Morocco, Rwanda, Nigeria

Existing evaluations

Center interviews

Reporting data

Internal interviews with management and staff

Annual reports

Interviews with external stakeholders: DFID, IDRC

Interviews with Board members

Top


To what extent have the intended outcomes of AIMS and its components been attained both in quantitative and in qualitative terms?

Education

- Did the Program contribute to better employment opportunities for graduates?

- Did the Program contribute to an increase in skilled staff available in Africa?

- Is there any evidence that AIMS alumni are pursuing high quality post-graduate studies and that they will eventually contribute to the further economic, political and educational advancement of the African continent?

N° of graduates going to jobs that requires skills acquired at AIMS (6 months after) (DFID Output 2.3)

Type of job / location of the job

Level of entry

Wage premium estimate (wage monthly income 5 years after graduation) (IDRC 3.2.3)

Career prospects

Employment disaggregated by academia/industry (IDRC 2.2.)

Employment disaggregated by sectors (priority areas) (IDRC 2.1.)

N° of grads into entrepreneurship - disaggregated by sector (IDRC 2.3.)

N° of grads going to further education opportunities: type of course/level of course (flow chart) (DFID Output 2.4)

N° of grads going to research (PhD / being researchers) - by universities - by area of research (IDRC 2.4.)

Use of skills in job/research post AIMS: in 5 formative areas (daily/never)

N° of publications (IDRC 1.5)

Perception of companies of interns skills (IDRC 3.1.1; DFID Output 2)

Assessment/perception by donors/companies who hire the AIMS graduates

Alumni survey

Interview with donors: Google, DAAD + other companies

Scopus

Reporting data

Top


Research

- Did the Program contribute to scientific outputs/publications from scientists of the African continent?

- Did the Program contribute to innovation related outputs?

Outreach

- Did the Program improve education opportunities for children in Maths and Sciences?

- Did the Program contribute to an increased awareness in Math and Sciences at the political and economic level?

Organisation

- To what extent AIMS has visibility and is a flag initiative for Maths excellence in Africa

N° publications by type (scheduled, actual) (IDRC 1.4; DFID Output 5.1)

N° of citations N°

collaborations

N° of established network/ partners (DFID 4.3)

N° of co-publications

N° of exchange visits

N° of visitors to center

N° of patents from research projects

N° of innovation products

N° of children exposed to trained teachers (IDRC 3.3.1)

Acknowledgement of importance of Math and Science by the partners & policy makers

N° of newsletter subscription

Quality assessment of visibility of results/activities

Scopus

Interview with research leaders

Interview with Ministry of education

Interview with partners

Interview with policy makers

Interview with partners

Alumni survey

Interview with donors (AU, NEPAD)

Is it likely that so far unattained objectives may be accomplished in full/in part ?

Opinion on achievements Interviews

What are the drivers and barriers to achieve objectives? Examples of drivers and barriers Interviews

Are the results achieved adequate for the budget spent?

Was the allocation of resources among different aspects appropriate to

Costs of (overhead)

the different management structures Reporting data

Top


achieve the results?

Is the management structure efficient for the achievement of the expected outcomes?


How does the organisation present value for money?

Financial benchmarking of costs for other and similar educational programs

Judgement on efficiency [qualitative]

What factors help account for project efficiency performance? Opinion on factors accounting for project efficiency performance

Reporting data

Internal interviews

Donors interviews

Top


Impa

ct

What is the impact of AIMS activities on:

- Productivity and Growth?

- African (Scientific, Economic, Political) leadership?

- Mathematical and scientific research excellence in Africa?

- on Policy and innovation in Africa?

- on interest in Africa for and skills in Math & Sciences in Africa?

- on STEM policies in Africa?

- on AIMS as a flagship initiative for STEM educational reform in Africa?

N° AIMS alumni who created an innovation with solution to development challenges (IDRC 1.1.)

Change in GDP (DFID Impact 1)

Critical mass and critical thinking, social capital, knowledge careers, publications

How skills help for productivity? Are there better carrier prospect? Employer satisfied with type of skills?

N° of leaders based on titles/salaries/ #people responsible for/awards (DFID Outcome 1)

% of AIMS alumni in high level leadership positions in and out (IDRC 1.2)

% awards (IDRC 1.3)

N° of citation in quality journals by field

Publications and patents at country level (DFID Impact 3)

N° policies changes coming from AIMS research

Economic impact from innovation from AIMS

Acknowledgement

% of enrolment in STEM Master’s at country (DFID Impact 2)

More applicants at secondary school, grad and post

Internal Interviews

Alumni survey

Interviews with government

Case study examples

Scopus

Data base of publications

Researcher interviews

Univ. Partners interviews

Policy makers

Donors: AU NEPAD, AfDB, UNESCO, Google, DAAD

Top


grad level

Changes to STEM policies

N° of followers

N° of centers based on AIMS model

Sust

ain

abil

ity


Explanation of the approach to sustainability Internal interviews

DFID, IDRC interviews

What are the chances that benefits generated will continue what factors militate in favour of or against maintaining benefits?

Opinion on sustainability of benefits Internal interviews


To what extent were sustainability mechanisms introduced - How and to what extent are AIMS students, alumni, donors, and

governments promoting the sustainability of AIMS’s activities after the end of the Program?

- How and to what extent has the financial stability of the AIMS- NEI been improved? How successful has AIMS been in securing funding? How diversified are the sources of funding secured by AIMS? How can the organisation’s financial stability be improved?

Opinion on sustainability mechanisms Internal interviews


What can be learned from best practices and worst cases for the next phase of the implementation?

SWOT analysis Internal interviews


What recommendations can be made based on the MTE to AIMS? Opinion on future improvements Interviews

Survey

Cross reference table of DFID indicators with the MTE evaluation indicators Appendix D

DFID indicator reference

Result type

AIMS pillar

Result

Indicators

Output 1.1. Activities N° of centres

Output 1.2 Outputs Organisation Well-ran organisation (overall/centres) Gender ratio

Output 2.3. Activities N° of bursaries for AIMS students

Output 2.3

Outcomes

Education

Better employment opportunities for graduates N° of graduates going to jobs that requires skills acquired at AIMS (6 months after)

Output 2.4.

Outcomes

Education

Better employment opportunities for graduates

N° of grads going to further education opportunities: type of course/level of course (flow chart)

Output 2.5 Outputs Education Interns N° of interns (in industry)

Output 3.1 Outputs Outreach Public reached through public engagement N° of attendees

Output 3.2

Outputs

Education

Finished research projects N° of applications/success rate (- evolution? DFID indicator)

Output 4.1 Outputs Organisation Well-ran organisation (overall/centres) Management systems performance of staff

Output 4.2.

Outputs

Organisation

Well-ran organisation (overall/centres) Financial stability of AIMS and diversification of funding sources

Output 4.3 Outcomes Research Scientifics outputs/publications N° of established network/ partners

Output 5.1. Outcomes Research Scientifics outputs/publications N° publications by type (scheduled, actual)

Output 5.2.

Outputs

Organisation

Well-an organisation (overall/centres) Quality of M&E data/ assessment of the M&E system/lessons learned

Outcome 1

Impacts

Education

New leaders and example

N° of leaders based on titles/salaries/ #people responsible for/awards

Outcome 2 Outcomes Education Increase of skilled staff available in Africa Perception of companies of interns skills

Impact 1

Impacts

Education Productivity and growth (includes alumni impact on economy)

Change in GDP

Impact 2

Impacts

Outreach

Increased interest for and skills in Math & Sciences

% of enrolment in STEM Master’s at country

Impact 3 Impacts Research Scientific impact Publications and patents at country level

Cross referenced table of IDRC indicators with the MTE evaluation indicators Appendix E

IDRC indicator reference

Result type

AIMS pillar

Result

Indicators

1.1.

Impacts

Education Productivity and growth (includes alumni impact on economy)

N° AIMS alumni who created an innovation with solution to development challenges

1.2. Impacts Education New leaders and example % of AIMS alumni in high level leadership positions in and out

1.3. Impacts Education New leaders and example % awards

1.4. Outcomes Research Scientifics outputs/publications N° publications by type (scheduled, actual)

1.5. Outcomes Education Increase of skilled staff available in Africa N° of publications

2.1. Outcomes Education Better employment opportunities for graduates Employment disaggregated by sectors (priority areas)

2.2. Outcomes Education Better employment opportunities for graduates Employment disaggregated by academia/industry

2.3. Outcomes Education Better employment opportunities for graduates N° of grads into entrepreneurship - disaggregated by sector

2.4.

Outcomes

Education

Better employment opportunities for graduates N° of grads going to research (PhD / being researchers) - by universities - by area of research

3.1.1. Outputs Education Skilled Graduates F/M N° of skilled graduates

3.2.1. Outcomes Education Increase of skilled staff available in Africa Perception of companies of interns skills

3.2.3

Outcomes

Education

Better employment opportunities for graduates Wage premium estimate (wage monthly income 5 years after graduation) (ps= data exists in last survey)

3.3.1. Outcomes Outreach Improved education opportunities for children N° of children exposed to trained teachers

3.3.1. Outcomes Outreach Improved education opportunities for children N° of attendees

3.3.2 Outputs Research Finished research projects N° of applications/success rate (- evolution? DFID indicator)

3.4.1 Activities All Cost per student

3.5. Outputs Organisation Well-ran organisation (overall/centres) Quality of M&E data/ assessment of the M&E syste

100.1.2 etc Outputs Education Skilled Graduates F/M N° of bursaries post AIMS Source: Technopolis (2015), from 2014 IDRC indicators list

List of interviewees Appendix F

AIMS SECRETARIAT KEY INFORMANTS

Name Position Entity Date

1. Managing Director NEF AIMS- NEI Germany 01/07/15

2. Senior Financial Operations Analyst AIMS Global Secretariat 22/07/15

3. M&E and Learning Manager AIMS Global Secretariat 29/05/15

4. International Academic Advisor AIMS-Canada 02/07/15

5. Executive Vice President AIMS Global Secretariat 29/05/15

6. Director, Network Operations AIMS Global Secretariat 08/07/15 7. Program Finance Manager/Executive Officer AIMS-Canada 06/07/15 8. Director, AIMS-UK AIMS-UK 24/06/15 9. Director, Gender and Inclusion AIMS Global Secretariat 08/07/15 10. Sr. Grants & Compliance Manager AIMS Global Secretariat 29/05/15 11. Chief Financial Officer AIMS-Canada 22/07/15 12. Director, AIMS Industry Initiative AIMS Global Secretariat 13/07/15 13. Communications Director AIMS Global Secretariat 22/07/15 14. Research Manager AIMS Global Secretariat 01/07/15 15. IT Manager AIMS Global Secretariat 22/07/15 16. The CEO and President AIMS Global Secretariat 11/07/15 17. HR Consultant AIMS Global Secretariat 08/07/15

INTERNATIONAL BOARD OF DIRECTORS KEY INFORMANTS

Name Position Date

1.

Department of Applied Mathematics and Theoretical Physics, Cambridge University, former Director of the Isaac Newton Institute for Mathematical Sciences and President, International Union for Theoretical and Applied Mechanics

24/06/15

2. Chairman of Canada's Science, Technology and Innovation Council (STIC) 06/07/15

3.

Director of the Perimeter Institute for Theoretical Physics and former Chair of Mathematical Physics, Cambridge University

13/07/15

AIMS ALUMNI EMPLOYERS KEY INFORMANTS

Name of organisation Contact person Position Date

1. Head of Marketing 09/07/15

2. Founder & Partner 22/06/15

3. Programme Manager, Web Index 26/06/15

AIMS DONORS

Name of organisation Contact person Position Date

1. IDRC

Director Technology and Innovation 09/07/15

Program Management Officer 08/07/15

2. DFID Program Manager 06/08/15

3. Robert Bosch Stiftung Foundation Senior Vice President of Health and Science 24/06/15

4. DAAD Head of Section Central and West Africa, Germany 25/06/15 5. Alexander von Humboldt-Stiftung /

Foundation

Division: Physics, Engineering, Mathematics 05/07/15

OTHER KEY PARTNERS KEY INFORMANTS

Name of organisation Contact person Position Date/ time

1.

Head and Advisor of NEPAD Science, Technology and Innovation Hub

03/07/15

SPECIAL INTEREST GROUPS KEY INFORMANTS

Name of organisation Contact person Date/time

1. Canadian High Commission in SA

06/07/15

2. Cameroon High Commission in Canada 08/07/15

SOUTH AFRICA KEY INFORMANTS

Category Name/Group Title Date

1. Tutors 3 27/05/15

2. Students 10 27/05/15

3. Staff Academic Manager (AIMSSEC) 28/05/15

Academic Director 27/05/15

Administration and Research Manager 27/05/15

HR and Finance Manager 27/05/15

Facilities & Logistics Manager 27/05/15

IT Manager for AIMS SA 27/05/15

4. Head/rep researchers Director, AIMS South Africa 27/05/15

Arete Junior Research Chair 27/05/15

SARChI Chair AIMS/Stellenbosch University 27/05/15

Senior Resident Researcher 27/05/15

Senior Resident Researcher 27/05/15

5. University of Cambridge, UK Professor of Fluid Dynamics, Institute of Theoretical Geophysics 19/06/15

6. University of Oxford, UK Professor of Pure Mathematics 22/06/15

7. Universite de Paris Sud XI, France 30/06/15

8. National Research Foundation Executive Director Human and Infrastructure Capacity Development (HICD)

17/07/15

9. DST 17/07/15

SENEGAL KEY INFORMANTS


1. Tutors 5 09/06/15

2. Students 10 09/06/15

3. Staff President 09/06/15

Executive Director 09/06/15

Finance Manager 09/06/15

Academic Manager 15/06/15

IT manager 09/06/15

Research Chair 09/06/15

4. Board Chair Board Chair 11/06/15

5. Université Cheikh Anta Diop de Dakar Faculty of Sciences and Technics 10/06/15

Former Head of Department of Mathematics and Computer Sciences and new Director of the Doctoral Schools

10/06/15

6. Université Gaston Berger de Saint Louis Prof of statistics, mathematics, probability 10/06/15

7. Université Assane Seck de Ziguinchor Chef Service Coopération et Recherche 10/06/15

8. Université Paris IX Professor, the Laboratoire de physique théorique 15/06/15

9. Ikagel Fish Processing Company Président Directeur Général 09/06/15

10. Ministry of Higher Education Deputy Minister of Higher Education 10/06/15

11. Special Advisor Strategic Advisor/Consultant 15/06/15

12. Former president of AIMS Senegal Former president of AIMS Senegal 10/06/15

GHANA KEY INFORMANTS


1. Tutors 3 10/06/15

2. Students 8 20/06/15

3. Staff Academic Manager 20/06/15

IT Manager 20/06/15

Finance Officer 20/06/15

Chief Operating Officer 20/06/15

Research Chair 21/06/15

Academic Director 21/06/15

4. President The President 20/06/15

5. Board Chair Member, Board of Trustees 21/06/15

6. Kwame Nkrumah University of Science and Technology, Ghana Doctor of Mathematics 20/06/15

7. University of Ghana, Ghana Doctor of Mathematics 20/06/15

8. University of Cape Coast, Ghana Doctor of Mathematics 20/06/15

9. Durham University, UK Vice-Chancellor 25/06/15

CAMEROON KEY INFORMANTS

Category Name /Group Title Date

1. Tutors 4 10/06/15

2. Students 10 10/06/15

3. Staff Academic Director 10/06/15

Executive Director 09/06/15

Facilities and Logistics Manager 09/06/15

Programme manager 09/06/15

Outreach and admin officer 10/06/15

IT Manager 10/06/15

Finance & HR Manager 10/06/15

4. University of Buea HOD, Maths 09/06/15

5. University of Douala

HOD, Maths 09/06/15

HOD, Physics 09/06/15

6. University of Kassel, Germany Head of Research group 24/06/15

7. University of Applied Sciences, Giessen-Germany Former Rector 19/06/15

8. ECOBANK Branch Manager 10/06/15

9. Minister of Higher Education Inspector General of Academics 10/06/15

Interviews topic guide Appendix G

Introduction G.1 AIMS invited the Technopolis Group to conduct and evaluation of its IDRC/DFID program. The evaluation takes place from May 2015 to July 2015 and focuses on two components:

• An evaluation of the progress towards achieving the Program’s objectives and intended outcomes specific to both the three programmatic pillars of AIMS, namely the academic, research and outreach pillars, and AIMS operational plans and mechanisms

• The documentation of best practices, opportunities, lessons and corrective actions needed for the next phase of implementation and to ensure the realization of the expected results.

Individual responses will remain confidential and only reported at the aggregate level. If specific examples are considered to be of added value to the overall evaluation, the evaluation team will explicitly ask for permission to use.

The interviews is structured around themes, each consisting of a number of in depth questions.

The themes are:

• Implementation of activities

• Governance and management structure

• Funding arrangements

• Educational activities

• Research activities

• Communication and outreach activities

• Monitoring and evaluation

• Employability of former AIMS students

• Impacts regarding a.o.:

− Productivity and growth in Africa

− Leadership, engagement to solve African challenges

− Excellency of African Mathematical Research;

− Awareness of Maths and Sciences importance

− STEM policies

− Policies

− AIMS as a flagship for STEM education reform

• The SWOT of AIMS

• Best practice

• Recommendations

Background and context G.2

Eva

luat

ion

pil

lar

Evaluation questions

Inte

rnal

int

ervi

ews

Boa

rd o

f Dir

ecto

rs

Part

neri

ng u

nive

rsiti

es

Mai

n D

onor

s

Fund

ers a

nd P

artn

ers

Em

ploy

ers

Spec

ial i

nter

est g

roup

s

Rel

evan

ce


��

Does AIMS respond to the needs of the target population? ��

Is the AIMs design appropriate for achieving the project’s core objectives? Is the model design relevant and appropriate to the specific context?

��

How coherent are the objectives in terms of how they fit in with the policies, programmes and projects undertaken by the government and other development partners?

��


��

Has design been retrofitted? �� What are the main factors that contributed to a positive or less positive assessment of relevance?

��

Overview of Activities G.3

Eva

luat

ion

pil

lar

Evaluation questions Inte

rnal

int

ervi

ews

Boa

rd o

f Dir

ecto

rs

Part

neri

ng u

nive

rsiti

es

Mai

n D

onor

s

Fund

ers a

nd P

artn

ers

Em

ploy

ers

Spec

ial i

nter

est g

roup

s

Act

ivit

ies

Education �� Research �� Public engagement �� Organisation ��

Effectiveness, Efficiency, Impacts G.4

Eva

luat

ion

pil

lar


rnal

int

ervi

ews

Boa

rd o

f Dir

ecto

rs

Part

neri

ng u

nive

rsiti

es

Mai

n D

onor

s

Fund

ers a

nd P

artn

ers

Em

ploy

ers

Spec

ial i

nter

est g

roup

s

Eff

ecti

ven

ess To what extent have the objectives of AIMS and its

components been attained both in quantitative and in qualitative terms? Is it likely that so far unattained objectives may be accomplished in full/in part?

��

To what extent did the AIMS team implement the initially planned activities?

��

What are the drivers and barriers to achieve objectives? ��

Eff

icie

ncy

Are the results achieved adequate for the budget spent? �� Is the management structure efficient for the achievement of the expected outcomes?

��

What factors help account for project efficiency performance? ��


��

Impa

ct

What is the impact of AIMS activities on productivity and growth in Africa?

��

What is the impact of AIMS activities on African (Scientific, Economic, Political) leadership?

��

What is the impact of AIMS on mathematical and scientific research excellence in Africa?

��

What is the impact of AIMS activities policy and innovation in Africa?

��

What is the impact of AIMS activities on interest for and skills in Maths and Sciences in Africa?

��

What is the impact of AIMS activities on STEM policies in Africa?

��

What is the impact of AIMS activities on AIMS as a flagship initiative to STEM educational reform in Africa?

��

SWOT, value for money and sustainability G.5

Eva

luat

ion

pil

lar


rnal

int

ervi

ews

Boa

rd o

f Dir

ecto

rs

Part

neri

ng u

nive

rsiti

es

Mai

n D

onor

s

Fund

ers a

nd P

artn

ers

Em

ploy

ers

Spec

ial i

nter

est g

roup

s

Sust

ain

abil

ity


��

What are the chances that benefits generated will continue what factors militate in favour of or against maintaining benefit?

��

To what extend were sustainability mechanisms introduced? ��

Are there positions for AIMS graduates in the industry or academia?

��

What is your assessment of AIMS SWOT? ��

What is AIMS value for money? ��

Conclusions and Recommendations G.6 What can be learned from best practices and worst cases for the next phase of the implementation?

What recommendations can be made based on the MTE to AIMS?

Survey questionnaire

Introduction

AIMS Alumni Survey

Dear AIMS alumni,

This survey comes to you following earlier messages you would have received from Dr Rosita Yocgo and Trust Chibawara at the

AIMS-NEI Global Secretariat in Cape Town. This electronic survey is part of the current independent mid-term review of the

Canadian and British government funding to AIMS, which began in 2010 and runs through to 2020. This funding built on everything

and everyone from 2003!

We are Technopolis Group and through this survey we want to obtain your input and recommendations.

We are therefore very interested to know what you are currently doing, but also to look back at your time at AIMS and reflect on

what you especially appreciated and what perhaps could use further improvement.

Your contributions are very important for this evaluation, which will help AIMS to keep improving continuously. As an

encouragement, AIMS has made available a fantastic prize: two all-expenses paid volunteer positions (one male, one female) at

the Next Einstein Forum (NEF), the first ever global science gathering in Africa, on 8-10 March 2016 in Dakar, Senegal. You must

complete the survey by the deadline in order to be considered.

We appreciate that some of these questions might appear repetitive of the questions posed in the AIMS alumni updates. However,

this survey is meant to fill in the gaps in the AIMS alumni database and to provide further details around your career path,

complementing the information that exists already.

All your answers will be held confidentially. Note that the prize will not depend on any of your answers, and we ask you to answer

the questions open and truthfully.

The survey will take about 15 to 20 minutes. If you have any questions, feel free to contact us at

[email protected].

Français: La questionnaire sera en Anglais. Cependant, vous pouvez répondre aux questions ouvertes en Français si vous voulez.

Background

AIMS Alumni Survey

First name

Last name

Personal information

Date of birth

DD

/

MM

/

YYYY

Please enter your date of birth

How did you first learn about AIMS?

Alumni of AIMS

Professor

Internet

Relative

Other (please specify)

On this page we will ask you to select options from lists that describe your occupation prior to

AIMS.

Your occupation is defined as the activity that took up most of your time, for example a job or

studying.

Please select the options that describe best that occupation.

- In the first list you can select your type of occupation (student, intern, employee, etc.)

- In the second list you can select the level of your occupation (education/employment levels).

Please choose a level relating to your selected occupation type.

- In the third list you can select the skill domain of your selected occupation. Please select the

skill domain that is most appropriate.

- In the fourth list you can select the area/sector of application of your selected occupation.

Please select the area/sector of application that is most appropriate.

Occupation, experience and education prior to AIMS

AIMS Alumni Survey

Occupation type

Level

(choose within your occupation type) Skill domain

First

occupation

prior

to AIMS

Could you please indicate your occupation directly prior to AIMS?

Start year End year Country

Occupation

prior

to AIMS

Please let us know when you started and ended this occupation and in which country it took place?

How many years of relevant work experience did you have prior to attending AIMS?

Zero or less then one year

One or more, but less than three

Three or more, but less than five

Five or more, but less than ten

Ten or more

Level of the qualifiation Scientific focus

Highest

academic

qualification

prior to

attending

AIMS

Could you please provide insight in your highest academic qualifications prior to attending AIMS?

(In case of multiple degrees of the same level, please provide information about the qualification closest to

attending AIMS)

Please name the institution where you obtained this degree

Skills prior to attending AIMS

AIMS Alumni Survey

Very weak Weak Average Strong Very strong

Mathematical knowledge and skills

Computing knowledge and skills

Scientific knowledge and skills

Communication skills

Research and analytical skills

Attitudes and values (e.g. intercultural team-work, goal setting)

Innovation and entrepreneurship

Please indicate your level of development of the skills in these areas, prior to attending AIMS:

The AIMS program

AIMS Alumni Survey

Very poor Poor Satisfactory Good Excellent

Lecturers

Tutors

Facilities and organisation

Support from other students

Interaction with other students

Supervision during research projects

Teaching methods

The format of the course

Support services / administration

Please rate the quality/level of the following course elements:

Not at all

To a small

extent

To some

extent

To a great

extent








Please indicate to what extent attending AIMS contributed to your development in the following areas:

To what extent do you feel that AIMS increased your focus on Africa's challenges?

Not at all To a small extent To some extent To a great extent

To what extent did AIMS help you get in contact with people that are now important in your professional

network?


AIMS internship program

AIMS Alumni Survey

Have you participated in an internship program facilitated by AIMS?

Yes, during AIMS

Yes, after AIMS

No

AIMS internship program

AIMS Alumni Survey

Please select the sector of your internship:

Not at all

To a small

extent

To some

extent

To a great

extent








Please indicate to what extent attending the AIMS internship program contributed to your development

in the following areas:

Please indicate to what extent the AIMS internship program helped you in finding a job:


Were you hired at the organisation where you did the internship?

Yes

No

We are very interested in learning more about your experience during your internship, feel free to share

you experience below.

Post-AIMS Research Opportunities

AIMS Alumni Survey

Following graduation from AIMS, did you receive any of the research opportunities listed below?

(Multiple answers allowed)

Post-AIMS bursary

DAAD Fellowship

AIMS Alumni Small Research Grant

None of the above

Post-AIMS Research Opportunities

AIMS Alumni Survey

Would you have continued your studies without this/these research opportunity/ies?

(And without other funding)

Yes

Yes, but at a later point in time

Probably not

Definitely not

Without this/these research opportunity/ies, what would have been the likelihood that you would have

been able to arrange a different source of funding:

Very unlikely Unlikely Likely Very likely

We are very interested in learning more about how you benefited from this/these research

opportunity/ies, feel free to share this below.

On the following pages we will ask you to select options from lists that describe your

occupations between AIMS and now.


studying.

- Please only describe substantial steps in your career path (where you were for at least 6

months)

- Start with the occupation directly after AIMS (so including any other MSc or PhD courses)

- Continue with any subsequent steps in your career

- Always select the category that best describes your position

- Fill in your current position last

- You career includes both educational steps and professional steps

- Each step will be covered on its own page

- At the end of each page you will be asked if this is your current occupation. If you answer no,

you will get another career page, if you answer yes, you will continue with the next part of the

survey.

Career path after AIMS

AIMS Alumni Survey

Please select the options that describe best the occupation.


- In the second list you can select the level of your occupation (education/employment

levels).Please select a level under the header of your selected occupation.





Career path after AIMS - First occupation after AIMS

AIMS Alumni Survey

Occupation type

Level


First

occupation

after AIMS

Could you please indicate what was your first occupation after AIMS?

Please give the name of the Organisation (University/Institute/Company/Other organisation):


First

occupation

after AIMS

Could you please indicate the start year, end year and country of this occupation?

Is the above described occupation your current occupation?

Yes, this is my current occupation

No, I have since moved on to a different occupation

Please select the options that describe best the occupation.


- In the second list you can select the level of your occupation (education/employment

levels).Please select a level under the header of your selected occupation.





NOTE: This is the last step possible to fill in, please make sure you fill in your current position.

Career path after AIMS - Tenth occupation after AIMS

AIMS Alumni Survey

Occupation type

Level


Current occupation

Could you please indicate what is your CURRENT occupation?

Please give the name of the Organisation (University/Institute/Company/Other organisation):


Current

occupation

Could you please indicate the start year, end year and country of this occupation?

Current position

AIMS Alumni Survey

Challenges

Challenge 1:

Challenge 2:

Challenge 3:

In your perspective, what are Africa's top 3 challenges?

To what extent do you focus on Africa's challenges in your current occupation?


If you focus on Africa's challenges, could you briefly describe how your work contributes towards solving

them?

How many people are you responsible for in your current occupation?

Zero

One to two

Three to five

Six to ten

Eleven to twenty

Twenty-one to thirty

Thirty-one to forty

Forty-one to fifty

More than fifty

Please indicate the range of your monthly gross income in USD during your current occupation (before

taxes and deductions)

Less than $500

Between $501 and $1000





More than $5000

To what extent do you work together with other AIMS alumni?


If you recently had significant collaboration with other AIMS alumni, could you give an example of how

you cooperated and what you have achieved?

Skills in current career

AIMS Alumni Survey

On daily

basis

Twice a

week

Once a

week

Twice a

month

Once a

month

Less than

once a

month Never






Attitudes and values (e.g.

intercultural team-work, goal setting)


Please indicate how often you use the following skills in your current occupation:

Career achievements

AIMS Alumni Survey

Please name your company(ies)/venture(s) here:

Have you started a company/venture, since attending AIMS?

No (or not yet) Yes, one company/venture Yes, more than one company/venture

Have you received any official awards, since attending AIMS?

(Including fellowships, but excluding scholarships and bursaries)

No (or not yet) Yes, one Yes, two Yes, three or more

Have you been closely involved in any patent applications, since attending AIMS?

(If yes, how many?)


Have you been closely involved in the launch of any products or services, since attending AIMS?

(If yes, how many?)


To what extent do these products or services contribute towards solving an African development

challenge?

Not at all To a small extent To some extent To a great extent Was not involved in products or services

If you answered yes to any of the above questions, please briefly describe the nature (e.g. sector,

development challenge and country) of your achievement(s):

How many academic papers have you written that were published in an international peer-reviewed

scientific journal, since attending AIMS?

Zero One Two Three to five Five to ten More than ten

Have you been or are you currently a mathematics teacher, tutor or lecturer? If yes, please select the

level of education that you taught at.

Primary education Secondary

education

Higher education Professional

education

I am currently not,

and have not

been, a mathematic

s teacher, tutor or

lecturer

Current skills

AIMS Alumni Survey









Please indicate your current level of development of the skills in these areas:

Impact of AIMS

AIMS Alumni Survey

Please indicate to what extent attending AIMS prepared you for your further professional/academic

career:


Please indicate to what extent attending AIMS increased your ability to gain and maintain employment


Without AIMS, would you have been able to achieve the same level of success in your career?

I would have achieved the same without AIMS I would have achieved somewhat less without AIMS

I would have achieved very little without AIMS

Without AIMS, would it have taken longer to achieve the same level of success in your career?

It would have taken the same time without AIMS It would have taken somewhat more time without AIMS

It would have taken much more time without AIMS

Without AIMS, please indicate whether your monthly gross income would be:

To what extent do you feel AIMS will be important for advancing your career in the future?


Future career prospects

AIMS Alumni Survey

Please select the sector in which you would like to continue your career:

Private sector

Public sector

Academia

Non-Governmental Organisations including civil society

None of these

Please describe how you envision your future career.

How do you feel about the career opportunities in Africa for Science, Technology, Engineering and

Mathematics (STEM).?

There are a lot of career opportunities within Africa

There are some career opportunities within Africa

There are very few career opportunities within Africa

There are no career opportunities within Africa

Not applicable as I am no longer pursuing a career in STEM

Will you look outside of Africa for future career opportunities?

I will look only within Africa

I will look both within and outside Africa

I will look only outside Africa

Please briefly explain your choice above

If you are currently living outside of Africa, what would convince you to return to Africa?

Strengths and weaknesses

AIMS Alumni Survey

Strength 1:

Strength 2:

Strength 3:

Please list three strengths of AIMS

Weakness 1:

Weakness 2:

Weakness 3:

Please list three weaknesses of AIMS

Statements

AIMS Alumni Survey

Strongly disagree Somewhat disagree Somewhat agree Strongly agree

I do not

know

AIMS should have stronger links

to African universities for post-AIMS

research opportunities

AIMS should have stronger links to

international universities for post-AIMS

research opportunities

AIMS should have stronger links to the

private and public sector for post-AIMS

career opportunities

AIMS should be extended to a two-year

research Master’s programme

AIMS should offer an additional Master’s

programme for more advanced students

and with a greater focus on research

AIMS should offer a six-month internship

in industry

AIMS should offer a six-month internship

in research

AIMS could introduce fees for the

Master’s programme

More teacher training in primary and

secondary education is important for my

country

Pan-Africanism is one of my core values

I value pan-Africanims more as a result of

AIMS

Community service is very important

Since AIMS I have been actively

participating in community service

The world needs more women in science

I value the importance of more women in

science more as a result of AIMS

Please indicate if you agree with the following statements:

Recommendations

AIMS Alumni Survey

If you have any feedback on topics related to AIMS that were not part of the survey, please share

themhere:

If you have any recommendations for AIMS, please share them below:

If you have any suggestions for how AIMS can improve its Alumni programme, please share them

below:

Why should you be selected for the all-expenses paid volunteer position at the Next Einstein Forum

(NEF), the first ever global science gathering in Africa, on 8-10 March 2016 in Dakar, Senegal?

You have reached the end of this survey.

Thank you very much for your contribution.

End

AIMS Alumni Survey

Introduction

AIMS Non-participant Survey

Dear AIMS applicant,

You are receiving this survey as a former applicant to an AIMS centre. However, unfortunately AIMS could not offer you a position

despite your qualifications, or you were admitted, but chose to take another opportunity.

As part of the current mid-term review of AIMS that we are undertaking, you have been selected to complete a brief survey that will

help in assessing some key functions and results of AIMS.

We are grateful for your participation. The survey will take 5 – 10 minutes. If you have any questions, feel free to contact us

[email protected].

All your answers will be treated confidentially.

By completing this survey by the 19 July deadline, your name will be entered in a draw to win one of two Samsung Galaxy Core

Prime smartphones.

Français: La questionnaire sera en Anglais. Cependant, vous pouvez répondre aux questions ouvertes en Français si vous voulez.

Background


First name

Last name

1. Personal information

Date of birth

DD

/

MM

/

YYYY

2. Please enter your date of birth

3. How did you first learn about AIMS program?

Alumni of AIMS

Professor

Internet

Relative


4. Could you please indicate which AIMS centre was your first choice during application?

South Africa

Ghana

Senegal

Cameroon

Application procedure


Description

5. When applying to AIMS, you were...

... qualified to attend AIMS but not selected

... qualified to attend AIMS and selected but you turned down AIMS, as you accepted a different offer

... qualified to attend AIMS and selected but you turned down AIMS, as it was not your preferred centre

... qualified to attend AIMS and selected but you turned down AIMS, for a different reason (please describe below)

6. Knowing what you know of AIMS today, would you apply again?

Yes

No

I do not know

7. Knowing what you know of AIMS today, would you recommend AIMS to a friend, colleague or peer?

Yes

No

Don't know

Accepted other offer



8. Please describe what best describes the offer you accepted instead of AIMS

An Master in Africa with scholarship

An Master in Africa without scholarship

An Master outside Africa with scholarship

An Master outside Africa without scholarship

A PhD in Africa with scholarship

A PhD in Africa without scholarship

A PhD outside Africa with scholarship

A PhD outside Africa without scholarship

A job in Africa

A job outside Africa

9. Could you describe the offer in more detail?

Instead of AIMS


10. Please indicate what best describes what you did immediately in the first year after you were not

selected / turned down the offer for AIMS?

I started a Master in my home country with scholarship

I started a Master in my home country without scholarship

I started a Master somewhere else in Africa with scholarship

I started a Master somewhere else in Africa without scholarship

I started a Master somewhere else in Africa with scholarship

I started a PhD

I started a job while applying for other educational opportunities

I started a job without applying for other educational opportunities

I started applying for other opportunities but had no job

On this page we will ask you to select options from lists that describe your occupation prior to

your application to AIMS.


studying.









Occupation, experience and education prior to your application to AIMS


Occupation type

Level


First

occupation

prior

to AIMS

11. Could you please indicate your occupation directly prior to your application to AIMS?


Occupation

prior

to AIMS

12. Please let us know when you started and ended this occupation and in which country it took place?

13. How many years of relevant work experience did you have prior to your application to AIMS?





Ten or more


Highest

academic

qualification

prior to your

application

to AIMS

14. Could you please provide insight in your highest academic qualifications prior to your application to

AIMS?

(In case of multiple degrees of the same level, please provide information about the qualification closest to

attending AIMS)

15. Please provide details on your highest academic qualifications

Skills prior to your application to AIMS










16. Please indicate your level of development of the skills in these areas, prior to your application to

AIMS:

Highest current academic qualification



Highest

current

academic

qualification

17. Could you please provide insight in your highest academic qualifications?

(In case of multiple degrees of the same level, please provide information about the most recent

qualification)

18. Please name the institution where you obtained this degree

On this page we will ask you to select options from lists that describe your current occupation.


studying.









Current occupation


Occupation type

Level


Current

occupation

19. Could you please indicate what is your current occupation?

20. Please give the name of the Organisation (University/Institute/Company/Other organisation):


Current

occupation

21. Could you please indicate the start year, end year and country of this occupation?

Current position


22. To what extent do you focus on Africa's challenges in your current occupation?


23. How many people are you responsible for in your current occupation?

Zero

One to two

Three to five

Six to ten

Eleven to twenty


Thirty-one to forty

Forty-one to fifty

More than fifty

24. Please indicate the range of your monthly gross income (before taxes and deductions) in USD

during your current occupation:

Less than $500






More than $5000

Skills in current position


On daily

basis

Twice a

week

Once a

week

Twice a

month

Once a

month

Less than

once a

month Never






Attitudes and values (e.g.

intercultural team-work, goal setting)


25. Please indicate how often you use the following skills in your current occupation:

Career achievements


Please name your company(ies)/venture(s) here:

26. Have you started a company/venture, since your application to AIMS?

No (or not yet) Yes, one company/venture Yes, more than one company/venture

27. Have you received any official awards, since your application to AIMS?

(Including fellowships, but excluding scholarships and bursaries)


28. Have you been closely involved in any patent applications, since your application to AIMS?

(If yes, how many?)


29. Have you been closely involved in the launch of any products or services, since your application to

AIMS?

(If yes, how many?)


30. How many academic papers have you written that were published in an international peer-reviewed

scientific journal, since attending AIMS?

Zero One Two Three to five Five to ten More than ten

31. Have you been or are you currently a mathematics teacher, tutor or lecturer? Please tick the boxes

on which type of education you taught at.

Primary education

Secondary education

Higher education

Professional education

None of the above

Current skills










32. Please indicate your current level of development of the skills in these areas:

Future career prospects


33. Please select the sector in which you would like to continue your career:

Private sector

Public sector

Academia

Non-Governmental Organisations including civil society

None of these

34. How do you feel about the career opportunities within Africa for Science, Technology, Engineering

and Mathematics (STEM).

There are a lot of career opportunities within Africa

There are some career opportunities within Africa

There are a few career opportunities within Africa

There are no career opportunities within Africa

Not applicable as I am no longer pursuing a career in STEM

35. Will you be looking outside of Africa for future career opportunities?

No, I will only look within Africa

Yes, I will look both within and outside Africa

Yes, I will only look outside Africa

36. Please indicate why you chose the option above

Statements


Completely

disagree

Partially

disagree Partially agree

Completely

agree I do not know

Providing mathematics teacher training at primary

and secondary level is very important

Pan-Africanism is one of my core values

Community service is very important

The world needs more women in science

37. Please indicate if you agree with the following statements:

Recommendations


38. Please feel free to provide any other feedback or suggestions on AIMS or mathematics education in

Africa here:

You have reached the end of this survey.

Your name has been entered into the prize competition.

Thank you very much for your contribution.

End


Recommendations made by previous assessments per topic and issue

Topics Issues International panel review 2010 Δ External independent evaluation 2012 Δ

Intervention logic & strategy AIMS model

AIMS is a unique teaching and learning model in Africa. It is well advanced in the creation of a truly Pan-African centre of research with an international reputation. We encourage AIMS to continue along this path, remaining true to its founding principles of creativity, innovation and flexibility.

+ Strengthen the integration the three programs: AIMSSEC, + Academic (PGD) and Research to demonstrate their /- complementarity in achieving strong consolidated results for AIMS-SA. Strategically involve African lecturers and scientists to build capacity in Africa and increase the sustainability and + replicability of the AIMS model across Africa. Invest, strengthen and expand the co-lecturer program

+/- +

Intervention logic & strategy

Gender Good practices

Build on the existing progress for the admission of female+ students into the PGD program and strengthen the gender /- equity model within AIMS-SA. This broader and comprehensive approach would enable the development and implementation of internal good practices that complement and enhance current efforts. Additionally, it would guarantee the same opportunities of access to and success at AIMS-SA for men and women at all level

+/-

M&E System

Expand the on-going evaluation and monitoring system that + is tied to the strategic plan and logic model for the entire /- institution (academic, research and outreach). Enhance measurements and metrics with increased use of Information Communication Technology (ICT) to improve efficiency in information, data management and reach in all areas of AIMS-SA

+/-

Academic AIMS Diploma

The core AIMS Diploma Program has been a great success and should be maintained and continue to be suitably funded. In order to encourage the participation of South African students there will be a new intake of students at the beginning of the calendar year 2012. We support this initiative and recommend that the DOHE provide adequate funding

+ Consider developing and incorporating in the curriculum + some modules on Africa’s development challenges, business and employability skills. Consider undertaking a detailed assessment of what it will take to convert the current PGD program into a full master’s program. This will increase AIMS–SA’s relevance at multiple levels, its effectiveness and efficiency, and increased ability for its graduates to enter and contribute to the labour market earlier than may currently be the case

+

Academic Links with other universities

Visiting AIMS lecturers and research visitors to also + travel to other universities. This should be a specific program, which should be included in the portfolio of a dedicated administrative staff member. This will serve as an advertising campaign for AIMS throughout the

+

Increase collaboration and strengthen partnerships with similar institutions across Africa to increase reach and impact

+

Topics Issues International panel review 2010 Δ External independent evaluation 2012 Δ

country and establish ties with universities and students.

Organization of student exchanges between AIMS and South African universities in both directions

+

Research Activities

A specific SARChI research chair be awarded to an appropriate person, selected in one of themes of research supported by AIMS, who would be physically based at Muizenberg.

Strong efforts, including financial incentives from the government and universities, be made to create joint appointments with South African universities.

An expanded associate program to enable South African academics to spend time at the AIMS research centre This could be done with a series of MOUs with South African universities and/or a program of visiting fellow to which researchers can apply.

+ + +

N.a

Public engagement Activities

There is a crisis in secondary school education in mathematics and the sciences in South Africa due to the lack of well-trained teachers. In this regard we recommend that the AIMSSEC outreach activities be properly funded and expanded

Strengthen and broaden outreach as a core pillar and fundamental basis for developing Mathematical Sciences in Africa. Involve Alumni strategically to contribute in building a strong community of practice for mathematical sciences in Africa

+/-

Organisation Governance

For historical reasons AIMS has been integrated with universities in the Western Cape, to the mutual benefit of all concerned. In order to strengthen the ties with the rest of the country we recommend that AIMS appoint a distinguished academic from each of the other research intensive universities in South Africa to the governing Council

Organisation Recognition AIMS be designated a national centre of excellence Source: Technopolis (2015), from various assessments of AIMS activities

Survey and counterfactual analysis

Survey response rates and representativeness J.1 As part of the survey analysis we have carried out an analysis of how this survey is representative for the entire population of AIMS graduates. The total response rate was 52.1% (392/753). Women and men responded in roughly equal proportions. Older cohorts show a significantly lower response rate compared more recent graduates, which is not unexpected but does impact the further analysis since the cohort is likely to be positively associated with various outcome and impact indicators. However, if controlled for the cohort, the overall response can be classified as generally representative of the wider group of AIMS graduates.

Student admission analysis and counterfactual design J.2 The administration of the selection of students is managed by the AIMS Secretariat who acts as the coordinating body for the network of AIMS centres. AIMS has recently changed its selection system, which was well received by the executive committees in charge of the student selection.

In 2014-2015 the network received over 2600 applications. A team of 8 evaluators including two members of partner universities implement the selection at each centre. AIMS is working towards increasing the number of external evaluators involved in the selection process. Applicants need to satisfy admission requirements listed in Figure 1. In addition to these requirements the Secretariat consider the following criteria when selecting applicants:

• Having a balance in countries including meeting targets set by local partner authorities in terms of number of local students accepted

• Having a balance in Gender • Discipline and age of candidates • Excellence: • Educational history including grades obtained for previous degrees • Scholarships, prizes, distinctions, awards or other honours received • References • Language skills • Work experience • Leadership skills • Social engagement, in particular the willingness to give back to Africa, to serve the

development of Africa • Personal goals and motivations when applying for the program • Relevant skills (e.g. computing), interests and activities

Figure 1 AIMS admission requirements

• The program is currently open to students of African nationality.

• Applicants should hold, or anticipate completing by the start date of the AIMS program, a four-year university degree in mathematics (or corresponding), or any science or engineering subject with substantial mathematics component.

• An applicant's record should demonstrate strong aptitude in mathematics.

• Each applicant will be required to provide the names and contact information of two referees, who we will contact on behalf of the applicant. Note that it is the responsibility of the applicant to ensure that the referees do submit their references. Applications are not complete and will not be considered unless both referees have submitted their reports.

• Applications are considered incomplete until ALL relevant academic transcripts and certificates have been received, scanned as .jpg, .jpeg, .pdf

Source: http://www.nexteinstein.org

The number of applications has been increasing throughout the years almost exponentially, doubling every ±4-5 years. Even though the number of accepted students has also tripled, as new centres are opening up, this means the acceptance rate has decreased from 34% in 2003-2005 to 18% in 2012-2013. The average acceptance rates throughout the years is 21%

Figure 2 Acceptance rates throughout the year Cohort (application

year) Total applicants Accepted Acceptance rate

2003-2005 327 110 34%

2006-2008 667 144 22%

2009-2011 872 201 24%

2012-2013 1673 298 18% Source: AIMS database

When testing for various background characteristics influencing the likelihood of students’ admission rates,1 we find several background characteristics to be of significant influence.2 The average acceptance rate is 21.3%. The largest effect is gender, women having an almost 15% point higher likelihood of being accepted (36%). Acceptance rates also decrease throughout the years due to much higher application numbers, as is shown in the table above. Existence of prior degrees seem to have no significant effect on the likelihood of acceptance. Older students are somewhat less likely to be accepted. In terms of country of origin, various interesting observations can be made. South Africa (50%), Madagascar (48%) Lesotho (47%) Senegal (38%) all show very high admission rates. Nigeria (13%), and Mauretania (12%) have very low admissions rates, the first of which can be partly explained by the fact that the absolute number of applications from Nigeria is very high (706, 20% of total). AIMS strives for a balance between nationalities among its students body.

Interestingly, higher self-assessment of skill-levels before application has a significant negative effect on participation. Whereas applicants give themselves a total of 22.8 (out of 35 skill ‘points’) across all seven skill categories, rejected applicants indicate an average of 28.3 (out of 35). Since interviews with assessment committees at AIMS indicate no policy of rejecting overqualified candidates (at least not on a substantial scale), and it is unlikely assessment committees would make such collective errors in judging candidates profiles, the most likely explanation is that AIMS graduates benchmark their pre-AIMS skills to their post-AIMS level, whereas many rejected applicants have no such point of reference. This could point to an effect of ‘better knowing what you do not know’ as a result of AIMS

Figure 3 Acceptance rates throughout the year Cohort (application year)

Total applicants

Accepted Acceptance rate

2003-2005 327 110 34%

2006-2008 667 144 22%

2009-2011 872 201 24%

2012-2013 1673 298 18% Source: Alumni survey 2015

It is important to note that of the accepted candidates, many already had significant prior education before applying to AIMS:

1 Note that the year 2014-2015 has been left out of all analyses 2 This analysis is based on a probit-estimation with a total sample of 757 participants and non-participants

• 59% BSc Degree (minimum requirement) • 8% Postgraduate diploma • 23% Taught Master’s • 10% Research Master’s • 1% PhD Roughly 1 out of every 5 valid applicants is accepted and enters the AIMS Master’s program. In most cases, rejected applicants either start a job while looking for other educational opportunities (37%) or start a Master’s in their home country without a scholarship (24%). Interestingly, a substantial percentage (8%) started a PhD instead (see Figure 7 below). However, only 7% of rejected applicants continued a degree abroad, indicating that AIMS represents a rather unique possibility for many students to gain an educational experience outside of their home country in Africa. Interviews with current students also showed that the large majority would not have been able to afford a Master’s abroad. In fact, the fully-funded nature of AIMS, made available to a relatively large number of students annually makes it an extremely attractive program. In Ghana, for example, this has led to a very large share of all mathematics and related disciplines Bachelor graduates applying to AIMS, even if they are possibly not particularly interested in Mathematics as a discipline. In the range of options available to undergraduates, such as finding a good job (difficult), continuing a Master’s degree in their exact field of interest (difficult without funding), a fully-funded degree in Mathematics is a highly attractive option.

Figure 4 Overview of where non-admitted applicants end up in the first year after they were not selected / turned down the offer for AIMS Non-admitted

applicants

%

I started a Master in my home country with scholarship 39 8,69%

I started a Master in my home country without scholarship 107 23,83%

I started a Master somewhere else in Africa with scholarship 22 4,90%

I started a Master somewhere else in Africa without scholarship 9 2,00%

I started a Master somewhere else in Africa with scholarship 2 0,45%

I started a PhD 34 7,57%

I started a job while applying for other educational opportunities 168 37,42%

I started a job without applying for other educational opportunities 35 7,80%

I started applying for other opportunities but had no job 33 7,35%

Total sample 449 100,00% Source: Non-admitted applicant survey 2015

Around 10% of those who did not participate were actually admitted but turned down their offer because they were accepted elsewhere (more than half of them being outside of Africa, for a Master’s or a PhD with a scholarship), or because of family reasons, military obligations or missing documentation. Women are more likely to turn down admission offers. The reason for rejection is further detailed in the table below.

Figure 5 Overview of the reasons why non-admitted applicants did not participate in AIMS “When applying to AIMS, you were qualified to attend AIMS

… ” Non-

admitted applicants

%

Female

Male

… but not selected 436 88,08% 73,44% 90,19%

... and selected but you turned down AIMS, as you accepted a different offer

24

4,85%

12,50%

3,74%

... and selected but you turned down AIMS, as it was not your preferred centre 6 1,21% 4,69% 0,70%

... and selected but you turned down AIMS, for a different reason (please describe)

29

5,86%

9,38%

5,37%

Total sample 495

100,00% 64 428 Source: Non-admitted applicant survey 2015

Applicants learnt of AIMS through various sources (see Figure 11 below). Those who were successfully admitted were much were likely to have heard of AIMS through alumni of AIMS, whereas rejected applicants more often found AIMS on the internet. This suggests a strong network effect of AIMS. 93% of non-admitted applications would recommend AIMS to a friend, colleague or peer

Figure 6 How did applicants first learn about AIMS? Applicants Alumni

Alumni

Non- admitted

Female

Male

Cameroon

Ghana

Senegal

South Africa

Alumni of AIMS

208

123

50,42%

55,64%

45,83%

51,02%

58,33%

54,37%

Professor 95 96 28,57% 22,93% 25,00% 26,53% 16,67% 26,19%

Internet 27 157 3,36% 8,65% 16,67% 4,08% 6,67% 6,75%

Relative 16 60 5,88% 3,38% 4,17% 4,08% 8,33% 3,17%


39

64

11,76%

9,40%

8,33%

14,29%

10,00%

9,52%

Total sample 3

85

500

119

266

24

49

60

252

Source: Alumni survey & Non-admitted applicant survey 2015

The analysis above also feeds into a decision regarding the counterfactual assessment model. Since the most useful explanatory variables for acceptance are more categorical or even binary variables, we propose to use simple counterfactual comparison instead of methods such as propensity score matching or regression discontinuity design, since we do not have access to reliable predictors of endogenous variables that distort the measurement of effect size of participation (mainly the intrinsic quality of the student), as no systematic scoring of applicants was carried out and the survey self-assessment questions suffer from the methodological problems discussed above. While we will control for background variables (gender, cohort, and center where relevant), this means that the analysis should be seen as a comparative analysis rather than a precise measurement of effect size.3 This also means that any effects (differences between participants and applicants on outcome and impact variables) are likely to have an upward bias and will be therefore overestimated.

3 We would suggest that for future M&E purposes AIMS could consider to introduce a standardised scoring mechanism of applications, which would strengthen the possibilities for evaluations using a counterfactual design

Figure 7 Age distribution of alumni and applicants when applying to AIMS

Source: Technopolis Group 2015, based on Alumni survey and AIMS database of non-admitted applicants

Figure 8 Current age distribution of alumni

Source: Technopolis Group 2015, based on Alumni survey

Figure 9 Years of relevant work experience of alumni prior to attending AIMS

Applicants Alumni

Alumni Not admitted Female Male Cameroon Ghana Senegal

South Africa


163 131 61,82% 38,15% 60,87% 36,96% 56,36% 42,98%


117 142 25,45% 35,74% 21,74% 45,65% 23,64% 33,19%


55 89 6,36% 19,28% 13,04% 10,87% 12,73% 17,02%


22 72 6,36% 6,02% 4,35% 6,52% 7,27% 5,96%

Ten or more 2 11 0,00% 0,80% 0,00% 0,00% 0,00% 0,85%

Total sample 359 445 110 249 23 46 55 235

Source: Alumni survey & Non-admitted applicant survey

010

2030

4050

Frqu

ency

of r

espo

nse

20 25 30 35 40Age

(N = 380)Q: What is your date of birth?

050

100

150

200

250

Frqu

ency

of r

espo

nse

10 20 30 40 50 60Age


Alumni Non admitted applicants

010

2030

4050

Frqu

ency

of r

espo

nse

20 30 40 50Age


Figure 10 Occupation type prior to applying to AIMS

Applicants Alumni

Alumni Non-admitted Female Male Cameroon Ghana Senegal

South Africa

Employee (Public, private, NGOs)

53 125 10,53% 15,95% 8,70% 12,50% 7,27% 16,73%

Employee (Academic)

110 149 21,05% 33,46% 39,13% 37,50% 20,00% 29,39%

Self-employed 3 5 0,00% 1,17% 0,00% 2,08% 1,82% 0,41%

Intern 15 20 2,63% 4,67% 8,70% 2,08% 3,64% 4,08%

Voluntary work (unpaid)

6 14 0,88% 1,95% 0,00% 6,25% 0,00% 1,22%

Student 173 117 61,40% 40,08% 39,13% 35,42% 67,27% 44,90%

Unemployed 11 19 3,51% 2,72% 4,35% 4,17% 0,00% 3,27%

Total sample 371 449 114 257 23 48 55 245


Figure 11 Highest academic qualification prior to attending AIMS

Applicants Alumni

Alumni Non-admitted Female Male Cameroon Ghana Senegal

South Africa

Undergraduate (BSc. / Licence) 218 285 65,22% 55,64% 52,17% 72,92% 37,50% 61,22%

Postgraduate diploma 27 33 10,43% 5,84% 4,35% 2,08% 7,14% 8,57%

Taught Master’s (MSc. / Master 1 et 2) 84 82 17,39% 24,90% 30,43% 20,83% 32,14% 20,00%

Research Master’s (MPhil/MSc. / Master 2 recherche) 38 42 6,96% 11,67% 13,04% 4,17% 21,43% 8,57%

Doctoral Degree (PhD) 5 6 0,00% 1,95% 0,00% 0,00% 1,79% 1,63%

Total sample 372 448 115 257 23 48 56 245


Figure 12 Overview of whether non-admitted applicants, knowing what they know now of AIMS, would apply again to AIMS

Non-admitted applicants %

Yes 331 67,83%

No 74 15,16%

I do not know 83 17,01%

Total sample 488 100,00%

Source: Non-admitted applicant survey

Figure 13 Overview of whether non-admitted applicants, knowing what they know now of AIMS, would recommend AIMS to a friend, colleague or peer

Non-admitted applicants %

Yes 456 92,87%

No 14 2,85%

I do not know 21 4,28%


Source: Non-admitted applicant survey

Student assessment of quality of training J.3 As part of this mid-term evaluation, graduates were asked to reflect on the quality of key AIMS course elements and give an assessment. The figure below presenting the results show that alumni are overall extremely positive about virtually every aspect of their AIMS experience, with every aspect showing >75% good or higher. The quality of lecturers stands out especially with 72% rating this as excellent. Students are slightly less enthusiastic about the support from lecturers during research projects, support of administration and the format of the course. The least praise is given to the quality of the tutors, which is rated excellent by only 26% of alumni while 23% gives assessments of ‘satisfactory’ or lower.

Figure 14 Assessment of course elements of AIMS by alumni

Source: Alumni survey

Students’ results J.4 Probably the most important direct result of the AIMS Master’s program is the number of graduate students. Up until 2013-2014, 748 students had graduated from AIMS, with only 5 students not graduating. This results in a graduation rate of 99.4%.

02 20 51 26

2 17 50 31

1 16 46 37

1 3 13 39 44

03 13 40 44

02 14 39 45

1 12 43 44

1 10 38 51

02 26 72

0 20 40 60 80 100Share of respondents (%)

Tutors

The format of the course

Support from other students

Support services / administration

Supervision during research projects

Facilities and organisation

Teaching methods

Interaction with other students

Lecturers

(N = 368)Q: Please rate the quality/level of the course elements

Very poor

Poor

Satisfactory

Good

Excellent

Figure 15 Number of graduated and not graduates AIMS alumni

Graduated Not graduated Total

AIMS alumni 748 5 753

Source: Technopolis Group 2015, based on AIMS database of alumni

This extremely high graduation rate is explained by the AIMS philosophy of based on AIMS’ philosophy of prioritising learning over rote learning and regurgitation during exams. Students that fail important assignments such as the research project get the opportunity to repeat and are in almost all cases eventually granted the diploma. While this policy of almost certain graduation is intentionally designed to promote inclusiveness and is a bulwark against intolerable stress levels (which are already high enough), a side effect is the reality that some students, though generally a small minority, may graduate without having reached minimum standards4. A potential risk is that the AIMS degree could lose credibility with academic institutions elsewhere.

The Figure below (Figure 16) shows the number of graduated students by centre. Only fairly recently, have new centres entered the AIMS family with Senegal first graduating students in 2011, Ghana in 2012, Cameroon in 2013 and Tanzania in 2014. Centres generally need a few years until they are at the full capacity of 50-55 students. On average, women make up 36% of graduates, slightly above the minimum criteria of 30%

Figure 16 Number of graduated students, by centre, by gender, by start year till 2013

Source: Technopolis Group 2015, based on AIMS data

The figure below (Figure 17) gives an overview of the nationality of AIMS graduates. AIMS has had graduates from the large majority of African countries, with only a few countries (Sierra Leone, Guinea, Guinea-Bissau, the Gambia, Tunisia, South-Sudan,5 Djibouti, Gabon, Côte d’Ivoire, Equatorial Guinea and Namibia6) not represented in the AIMS alumni group. Countries with an AIMS centre generally have high participation rates, as do relatively populous countries such as Ethiopia, Nigeria, DRC and Sudan. Currently, Anglophone Africa is best 4 This is issue is compounded by the fact that many assignments are collaborative, where weak students can benefit from others or even copy material. Some students also alerted us that solutions were widely available through contact with students of previous years. 5 In practice students from South Sudan have participated under their Sudanese nationality before the split 6 The course in 2013-2014 also has students from Côte d’Ivoire, Guinea and Namibia participating

10

22

23

38

8

18

5

29

21

37

12

24

12

28

13

35

18

38

020

4060

Num

ber o

f gra

duat

ed st

uden

ts

2011 2012 2013

CameroonGhana

Senegal

South Africa

CameroonGhana

Senegal

South Africa

CameroonGhana

Senegal

South Africa

Female Male

represented, while Francophone and especially Lusophone representation is still lagging. The Maghreb countries are also somewhat underrepresented. However, the background of students is sufficiently diverse to rightfully deserve the accolade of ‘pan-African’

Figure 17 Nationality of AIMS graduates (up to and including 2013)

Source, AIMS database, Technopolis analysis 2015

Knowledge and skills

Naturally, education is not just about graduation numbers but also about the level of knowledge and skills that students acquire during their AIMS Master degree.

AIMS has formulated five core skill areas in its Graduate Profile document:

• Mathematical, computing and scientific knowledge and skills • Communications • Research and analytical skills • Attitudes and values • Innovation and entrepreneurship The following section will discuss the outcomes of acquisition in these knowledge and skill areas during the AIMS Master’s program.

Whereas the progress on skills is generally considered very good, some interviewees did indicate that there is a risk that not all students’ knowledge of mathematical concepts was equally developed. According to the tutors, there is a risk that a substantial share of students do not fully comprehend the material of the short courses after they finish but are rather pragmatically focused on dealing with all the assignment deadlines due to the high pressure on the students. This factor is compounded by the fact that students co-operate a lot on assignments, which is good for building teamwork skills, but makes individual assessments (without tests) at times difficult.

(50,100](30,50](20,30](15,20](10,15](5,10][1,5]No data

Examiners of research projects (essays) indicated that while there are some essays of very high quality, a number of students are instructed to carry out revisions to their essays. However, examiners never get to see the final versions of these papers7 and therefore cannot verify whether minimum standards are met. Overall, though, examiners were impressed with the level of students at the end of the year.

Overall, students, tutors and other stakeholders are very positive about the level of skills and knowledge development of the students during their 10 months at AIMS. Several students called the course a ‘transformative experience’ (or similar expressions), and both tutors and lecturers/examiners were impressed by the level of progress during those 10 months. In particular, students’ improvement is strong in the area of applied mathematical skills (especially in the area of programming and software use) and improvements in language and communication skills (particularly for the non-English speakers). The profile pillar around attitudes and values was also generally seen as very successful, students appreciated and grew in an intercultural environment and received a lot of exposure to not just a wide variety of topics but also to an international environment of excellence.

Results were more mixed in the area of research and analytical skills. While students and others did indicate that their analytical skills improve in terms of problem- solving and general analytical skills, the value added in terms of specific research skills was more limited. The research phase (essay) is relatively short, and students’ work on relatively fixed and guided topics. Whereas many students appreciated that this was a good introduction to research, not all would feel prepared to immediately continue in to a research career (e.g. PhD). However, local lecturers and examiners still indicated that there were cases of quite excellent essays that would compare well against the work of local students following an equivalent program. The innovation & entrepreneurship pillar seems currently less developed and relatively disconnected from the other skill pillars.

There were other notable inputs into the discussion on student outcomes. In general the partner universities were very satisfied with the quality of student outcome at AIMS and highlight AIMS “leaves people with a lot of skills”.

The electronic survey confirms the results of the focus groups and interviews (see Figure 18 below).

7 AIMS South Africa does publish the final versions of projects online

Figure 18 Assessment of the contribution of AIMS to the level of development of skills by alumni

Source: Alumni survey 2015

Cultural outcomes

There are many types of cultural outcomes which are seen in AIMS graduates:

• Multicultural acceptance and integration • New learning cultures • Increasing importance of African and development related issues An AIMS Master’s program, for many, provides the first opportunities for to live and work with students from other African countries, as well as being exposed to international lecturers. In Senegal and Cameroon, the students also work in bilingual environments and leave AIMS with a good grasp of both English and French (culturally and linguistically). There is also religious and gender integration. Therefore it is important to have the right level of personal and social support throughout their time at AIMS.

The commendable learning culture is something that was raised throughout the conversations with students, tutors and AIMS staff. The learning model at AIMS is very different from that of other higher education institutions in Africa. It is also different to many other international universities. The vision for AIMS is one where the centres will produce a new generation of leaders, critical thinkers and scientifically excellent graduates. A broad transmission of the enthusiasm for this vision is a big success of AIMS, as evidenced by the interviews with students and partners.

Finally, the AIMS model includes the need to ensure that African related issues are explored and developed. There is a strong recognition amongst students of the importance of working on African related issues, as evidenced in their research presentations and through the focus groups, but also by the results of the electronic survey (see below).

5 18 43 34

1 4 37 57

04 34 62

1 3 30 66

2 33 64

02 29 69

12 26 72



Communication knowledge

Attitudes and values





(N = 369)Q: Contribution of AIMS to the level of development of skills

Not at all

To a small extent

To some extent

To a great extent

Figure 19 Assessment of the contribution of AIMS to increase of focus on Africa’s challenges by alumni

Alumni %

To a great extent 222 60,33%

To some extent 127 34,51%

To a small extent 16 4,35%

Not at all 3 0,82%

Total sample 368 368


Internships

While the internship programs of AIMS are currently relatively limited in terms of student numbers, the electronic survey still investigated in more detail what the effects of these internships have been. Out of the 371 alumni survey responses in our survey:

• 17 alumni indicated participated in an internship program, facilitated by AIMS, during AIMS • 24 alumni indicated participated in an internship program, facilitated by AIMS, after AIMS • 330 alumni did not participate in an internship program Alumni that participated were overall very positive about the contribution of their internship to the set of skills that were discussed in full in 2.3.1.2. Around 5% indicated that the internship program did not contribute at all, around 10% indicated that it contributed “to a small extent”, around 30% indicated “some extent” and 55% “to a great extent”. Alumni were a bit less positive about the contribution of the internship towards finding a job, around 30% indicated “not at all”/”to a small extent”, 40% “to some extent”, and 30% “to a great extent”. Twelve out of 38 alumni indicated being hired by the organisation where they had an internship. There is not necessarily a direct correlation between “being hired” and “being positive about the contribution towards finding a job” as most alumni that were not hired indicated a contribution of “to some extent”. Alumni that did get hired are however, as expected, positive about the contribution.

Career

A key output of AIMS is skilled graduates, something discussed in the previous sections. Beyond the direct control of AIMS but an important signifier of the effectiveness of AIMS is the subsequent academic and professional career of AIMS graduates. In this section several aspects of alumni’s career are discussed in greater detail.

The figure below (Figure 20) shows the highest degree (finished or currently engaged) of all current alumni and rejected applicants that responded to the electronic survey (51%). A substantial share of the AIMS graduates have finished or are currently engaged in a Master’s program (25%) while the largest share (41%) are currently engaged or have already finished a PhD. Women are slightly more likely have a Master’s as the highest degree and less likely to do a PhD, although the difference is relatively small. Interestingly, a slight majority of rejected applicants also finish a Master’s, but they are much less likely to pursue or have finished a PhD (5% vs. 41%).

Figure 20 Highest degree (finished or currently engaged)

Non- Participants Alumni Men (alumni) Women (alumni)

Undergraduate degree

• 35% (154) NA NA NA

Post graduate diploma

• 8% (33) NA NA NA

AIMS degree NA • 34% (251) • 33% (171) • 35% (80)

Non- Participants Alumni Men (alumni) Women (alumni)

(Post-)AIMS Master’s degree

• 52% () • 25% (189) • 23% (121) • 30% (68)

PhD • 5% (23) 41% (309) 44% (227) 36% (82)

Source: Electronic Survey 2015

Many stakeholders pointed towards the tendency for African students to ‘collect’ Master’s’ degrees as continued education is a relatively attractive occupation when PhD scholarships and good professional opportunities are scarce. This notion is also shown in the survey results (see Figure 21 below), more than 3 out of 5 alumni have finished or are currently engaged in a second, third or even fourth Master degree.

Figure 21 Alumni Master degrees: Finished and currently engaged

Men Women

One • 37% (191) • 40% (282)

Two • 55% (286) • 53% (123)

Three • 7% (38) • 15 (7%)

Four • 1% (4) • 0.4% (1)

Source: Electronic survey (no sig. difference between men and women) 2015

One indicator for subsequent academic and scientific achievements is an analysis of publications of alumni. Figure 22 sums the key figures of the scientific output of alumni until AIMS year 2013. There are 170 alumni with at least 1 scientific paper in the database. Women are less likely to have published scientific papers than men.

Figure 22 Key figures on academic output

Sum of publications of alumni8 587

Alumni with at least 1 paper 170

Alumni with at least 2 papers 110

Female alumni in database Male alumni in database

174 (31% of alumni) 293 (69% of alumni)

Female alumni with publications Male alumni with publication

39 (22% of female alumni) 131 (33% of male alumni)

Female alumni with at least 2 publications Male alumni with at least 2 publications


Publications by female alumni

95 (16 % of publications) 492 (84% of publications)

Source: Technopolis 2015, Scopus

The electronic alumni survey featured an extensive section on alumni’s career progression. The table below summarises the results with respect to the type of occupation alumni currently exercise, split into four cohorts in order to account for career progression throughout the years. There is no significant difference in the distribution between men and women. AIMS alumni are much more 8 Larger than number of unique publications and some publications have more than 1 alumni as author

likely to still be a student compared to rejected applicants. In total 26.3% of alumni is currently carrying out a PhD, as opposed to only 7% of non-participants. In general, AIMS alumni seem to be especially disposed towards an academic career with 70% of the first cohort (2003-2005) in academic employment and a further 11% pursuing a PhD. The share of alumni focusing on an academic career is consistently above 80% throughout the cohorts. Rejected applicants are much more likely to be engaged in a non-academic career in comparison to alumni. The focus groups with students confirmed this picture, as did interviews with employers in industry that mentioned the academic interest of AIMS graduates (see below). This trend does call into question some of the parts of the AIMS intervention logic that work towards providing African companies with skilled employees.

Figure 23 Current occupation type by cohort (alumni and non participants), based on survey sample 9

Cohort 1 2003-2005

Cohort 2 2006-2008

Cohort 3 2009-2011

Cohort 4 2012-2013

Alumni Academic employment

70% 46% 24% 21%

Non-Academic employment

18% 15% 9% 9%

Student (incl. PhD)

11% 34% 60% 58%

Other 1% 4% 7% 12%

Total N 27 47 103 187

Non-participants Academic employment

N too small 51% 37% 33%

Non-Academic employment

N too small 30% 36% 30%

Student N too small 12% 18% 23%

Other N too small 7% 9% 14%

Total N 8 43 91 241

Source: Electronic survey

When analysing the occupation advancement level achieved through alumni’s careers, a few trends can be highlighted. While it is quite early to assess final career outcomes as even the first cohorts are still only 10 years into their careers, a number of intermediate observations can be made. In terms of alumni reaching leadership positions (one of the key impact criteria of the AIMS intervention logic) out of the first cohort, 3 alumni (12.5%) have made it to assistant professor. This is the highest academic position present in the sample. Four alumni have a position of senior lecturer/researcher. In total nine alumni (3%) have a manager position in a non-academic organisation or company. For non-applicants, this figure is 10%, and 2% of non-applicants have reached director level. This highlights the academic/professional divergence between alumni and non-participants.

Only two alumni (1%, both from the most recent cohort) are full-time company owners. This figure is similar for non-participants. Various respondents have company activities ‘on the side’. In general, there is no significant difference between women and men (also given the low total number of senior / leadership positions).

Not every occupation is directly related to the skills taught in the AIMS Master’s program. However, the figure below shows that over 90% still use mathematics on a daily basis, while 80% still use their knowledge of computer programming. In general, most skills taught at AIMS are still used on a daily basis, except for innovation and entrepreneurship, which is considered much less relevant in terms of daily utility by most alumni.

9 Note that the risk of bias is high, especially for non-participants from earlier years given the small response percentage.

Figure 24 Use of skills by alumni in their current occupation


Thematic areas

Looking at the thematic areas where alumni focus on in their careers, we find that women amongst alumni are significantly more likely to be in the health care thematic area, men are more likely to be in education. The differences between alumni and non-participants are not significant. In general, education is by far the most popular occupation with more than half of alumni working in this domain.

Figure 25 Thematic area of employment

Non – participants Alumni

Administrative and financial 6% 2%

Agriculture & Mining 2% 0%

Education 50% 57%

Environment 3% 2%

Governance, security 1% 0.3%

Health care 5% 10%

ICT 11% 5%

Infrastructure & Transport 2% 1%

Manufacturing 3% 1%

Other 18% 23%

Social Affairs 0.2% 0.3%

Source: Electronic survey 2015

84 7 5 1111

90 4 11212

80 5 5 2 2 3 2

87 4 4 1211

70 10 9 3 4 21

81 8 5 2112

29 12 11 10 5 19 14


Communication knowledge




Attitudes and values



(N = 343)Q: Use of skills in current occupation

Daily basis

Twice a week

Once a week

Twice a month

Once a month

Less than once a month

Never

Even though both alumni and non-participants focus heavily on education, the two graphs below show clearly that alumni focus mostly on higher education (65% of alumni that teach or have taught) whereas non-participants are equally as likely to teach at university as at secondary education.

Figure 26 Number of alumni that indicate to have been (or currently are) mathematics teachers, tutors or lecturers since attending AIMS.


Figure 27 Number of non-admitted applicants that indicate to have been (or currently are) mathematics teachers, tutors or lecturers since applying to AIMS.


Organisation

Alumni were also asked to report on where they are currently working. The table below provides an overview of responses that occurred multiple times. Given the long history of AIMS South Africa, it is

76

28

217

60

7

0 50 100 150 200Share of respondents (%)

(N = 330) (Multiple answers allowed)Q: On what level of education have you been or are you currently a mathematics teacher, tutor or lecturer?

Primary education

Secondary education

Higher education

Professional eudcation

None of these

73

35

184

189

24


(N = 413) (Multiple answers allowed)Q: On what level of education have you been or are you currently a mathematics teacher, tutor or lecturer?

Primary education

Secondary education

Higher education

Professional eudcation

None of these

not surprising that the South African Universities are the most present in this overview of current employers.

Figure 28 Current organisation (survey)

Popular organisation (academic) Organisations (non-academic)

Popular • University of Cape Town (15) • Stellenbosch University (15) • AIMS (±10) • University of Pretoria (6) • University of the Western Cape (5) • CPUT (4) • University of Khartoum (4) • University of KwaZulu Natal (4) • University of Ottawa (4) Notable mentions • Berlin Mathematical School • Chinese Academy of Sciences Institute of Mathematics • INFN Rome • Eindhoven University of Technology • Royal Melbourne Institute of Technology Total N: 333

Notable mentions • Vodacom DRC • Airbus Optronics SA • Benin Electrical Distribution Company • FedEx Europe • Ghana meteorological agency • Schlumberger Oilfield UK • Thales


Current location

Since AIMS intends to build African skills in order to contribute to Africa’s development, the issue of international migration and brain drain is particularly pertinent.

The table below presents an overview of the current location of AIMS alumni, separated by graduation cohort. When comparing AIMS alumni with non-participants, it is clear that AIMS graduates are much more international, both within and outside Africa. In total 30% of alumni is still outside Africa 10 years into their post-AIMS career. In general, of those who state they are abroad, the following countries are popular:

• South Africa (49%) • Canada (9%) • Germany (7%) • USA (5%) There is no difference between men and women among participants.

Figure 29 Current location of AIMS alumni

Cohort 1 2003-2005

Cohort 2 2006-2008

Cohort 3 2009-2011

Cohort 4 2012-2013

Alumni In home country 30% 21% 17% 40%

In other African country

39% 43% 50% 37%

In Europe 21% 18% 17% 12%

In other non- 9% 20% 16% 10%

Cohort 1 2003-2005

Cohort 2 2006-2008

Cohort 3 2009-2011

Cohort 4 2012-2013

African country

Total N 23 47 95 173

Non-participants In home country Too small N 60% 65% 77%

In other African country

Too small N 31% 24% 15%

In Europe Too small N 3% 7% 4%

In other non-African country

Too small N 5% 3% 4%

Total N 8 38 89 268


Gross Income of Alumni

One measure of career success is the monthly gross income that alumni receive. While a very imperfect measure, as it is self-reported and should ideally be adjusted by country, it is tangible and permits a direct counterfactual analysis with non-participants.

There is no difference between women and men in terms of their level of gross income nor of the growth rate of their incomes. Furthermore, participants start with a lower income than non-participants (±400 USD per month after the finishing of the program), most likely due to the fact that many rejected applicants start working instead of studying. However, later on every additional year brings an additional $50 – $595 a month for graduates only, meaning that AIMS graduates quickly outperform non-participants. This pattern is visualised in the figure below.

Alumni were also asked to self-report what their own assessment of salary difference would be with and without participation. This confirms the trends that graduates have higher incomes than non-participants, as a large majority notes a substantial expected difference.

Figure 30 Gross Income development


500

1000

1500

2000

2500

3000

Mon

thly

gro

ss in

com

e ($

)

2 4 6 8 10 12Years since application

Participants Rejected Applicants

Figure 31 Assessment of the monthly gross income of alumni in case they would not have participated in AIMS.

Alumni %

... the same 52 16,51%

... 1% to 10% lower 39 12,38%

... 11% to 20% lower 23 7,30%

... 21% to 30% lower 25 7,94%

... 31% to 40% lower 18 5,71%

... 41% to 50% lower 32 10,16%

... more than 50% lower 30 9,52%

Not applicable 96 30,48%



Focus on Africa’s challenges in current occupation

The electronic survey also inquired about the extent to which alumni are working currently on Africa’s main developmental challenges. As the table below shows, only a very small share of alumni indicate that their work does not relate to Africa’s challenges (12%), and a large share (30%) indicate that their work is related to Africa’s challenges to a great extent. Interestingly, non-participants self-report to be significantly more focused on Africa’s challenges.

Figure 32 The extent to which alumni and non-admitted applicants focus on Africa’s challenges in their current occupation

Applicants

Alumni % Non-admitted %

To a great extent 106 30,46% 218 52,91%

To some extent 142 40,80% 137 33,25%

To a small extent 58 16,67% 32 7,77%

Not at all 42 12,07% 25 6,07%

Total sample 348 100% 412 100%


Alumni were also asked to indicate what they consider are the most important challenges for Africa. The figure below shows that alumni see Education, Governance and Poverty reduction as the most important challenges.

Figure 33 View of alumni on the current top 3 challenges for Africa


Awards received

The table below shows that quite a high share of alumni have received specific awards (29%), mostly being academic honours or specific fellowships (standard academic grants are excluded).

Figure 34 Official awards received by alumni and non-admitted applicants

Applicants


One 66 19,19% 79 19,17%

Two 21 6,10% 16 3,88%

Three or more 11 3,20% 14 3,40%

Total sample 344 100% 412 100%


A number of examples are given below:

• Schlumberger Foundation Faculty for the Future Fellowship in order for me to pursue a PhD in mathematics at University of Glasgow

• PostDoc fellowship from FWO (Flemish science foundation) Belgium & twice the best paper award • Singapore International Graduate Award • A merit award from Stellenbosch University for my PhD studies. An exchange scholarship

program at the University of Pittsburgh, sponsored by the Benter Foundation of Pittsburgh, Pennsylvania.

• Faculty for the Future Fellowship (by Schlumberger), Fellowship Funds Incorporated (Queensland

010

020

030

0#

men

tione

d by

resp

onde

nts

Education

Governance

Poverty reductionHealth

Security and conflict

Economic growth

Population growth

Environmental issues

Gender equality

(N = 352)Q: In your perspective, what is Africa's top 3 challenges?

Graduate Women)

Companies founded

Given its objective of stimulating entrepreneurship, the establishment of companies is an important indicator for effectiveness. Around 8% of alumni were involved with the establishment of a company or organisation (as opposed to 16% of non participants).

As already noted during the occupation analysis above, only 2 alumni are full-time company/organisation owner, most alumni have an organisation or consultancy on the side.

Figure 35 Companies started by alumni and non-admitted applicants

Applicants


One company 25 7,25% 55 13,68%

More than one company 2 0,58% 10 2,49%

Total sample 345 100% 402 100%


The list of examples also shows that a large share of organisations are non-commercial and very closely related to mathematics education and academia.

Some examples are below:

• Communication networks, the product is a software that enables video transmission over low bandwidth networks, which characterise most of the communication links used across the continent

• Lesotho Association for the Advancement of Science and Engineering Association • Tanzania Students and Scholars Foundation Limited - www.tssfl.com • Digit Inspire Foundation (https://www.facebook.com/DigitInspire) • Network Software and Technologies (Nsat, www.nsatechnologies.net ) and online Media

(www.aucongo.cd) Patent applications The table below shows that a small share of alumni are involved in patent applications at their work or research. Around 9% of alumni were involved in patent applications, although the degree of involvement cannot be independently verified.

Figure 36 Closely involvement in patent applications by alumni and non-admitted applicants

Applicants


One 15 4,41% 54 13,11%

Two 6 1,76% 12 2,91%

Three or more 11 3,24% 15 3,64%

Total sample 340 100,00% 412 100,00%


Products and services

The regular introduction of innovations (the launch of substantially new product or services) is an important proxy for a productive and modern economy. The results below show that around 10% of alumni are involved in launching products and services in their current occupation. Of the 34 products/services mentioned, half are indicated to be greatly focused on African development challenges, another 10 are focused to some extent on these challenges.

Figure 37 Closely involved in the launch of any products or services by alumni and non-admitted applicants

Applicants


One 18 5,25% 82 19,90%

Two 9 2,62% 17 4,13%

Three or more 7 2,04% 24 5,83%

Total sample 343 100,00% 412 100,00%


Future career perspectives

Since most alumni’s careers have only just started, looking at expectations for future careers is a good way to look at expected future impact.

The table below shows that roughly two-thirds of alumni want to continue their career in academia, and around 19% want to continue in the private sector. This is similar to the proportion in non-admitted applicants. This shows that roughly 10% of alumni are expected to shift from academia to other sectors in the future, career possibilities permitting.

Figure 38 Overview of sectors in which alumni and non-admitted applicants want to continue their career.

Applicants


Private sector 63 18,58% 59 14,53%

Public sector 20 5,90% 49 12,07%

Academia 233 68,73% 271 66,75%

NGOs including civil society 18 5,31% 26 6,40%

None of these 5 1,47% 1 0,25%

Total sample 339 100,00% 406 100,00%


Survey respondents were also asked to indicate whether they are optimistic regarding career possibilities in Africa.

Students are relatively divided regarding the opportunities, with 33% of alumni very positive, while 36% are quite negative. Somewhat surprisingly, non-admitted applicants are significantly more optimistic than alumni.

Figure 39 Assessment of alumni and non-admitted applicants about the career opportunities in Africa for Science, Technology, Engineering and Mathematics (STEM).

“There are … … within Africa”

Applicants


… a lot of career opportunities … 111 32,84% 213 52,72%

… some career opportunities … 103 30,47% 92 22,77%

… very few career opportunities … 120 35,50% 83 20,54%

… no career opportunities … 1 0,30% 4 0,99%

Not applicable (no longer pursuing a career in STEM) 3 0,89% 12 2,97%

Total sample 338 100,00% 404 100,00%


The table below shows that around 19% of alumni are focused only on Africa for their future career, while 79% would look both inside and outside Africa. Very few alumni (2%) are only focused in opportunities outside Africa.

Figure 40 Overview of where alumni and non-admitted applicants will look for future career opportunities.

“I will look only …” Applicants


… within Africa 66 19,47% 103 25,43%

… within and outside Africa 266 78,47% 292 72,10%

… outside Africa 7 2,06% 10 2,47%

Total sample 339 100,00% 405 100,00%


Alumni were asked to reflect on what would convince them to stay in Africa (or convince them to come back). Reasons most mentioned include:

• Job security, availability, payment and career development • Safety, living security, governmental stability and democracy • Being able to make a (bigger) difference in Africa (then elsewhere) • The feeling to be obligated to make a difference in Africa • Personal reasons like “home”, “family” etc.

Other outcomes: an integrated learning community

One key intended outcome of AIMS is to set up a lasting and integrating learning community that stays with alumni during their entire career.

As can be seen in the table below, AIMS contacts are quite important for alumni, with a majority indicating that other AIMS alumni or relations known through AIMS are important for their current professional network. While this decreases for older cohorts, the figures remain relatively high. In total 40% of alumni currently still work together with alumni to some or even great extent.

Figure 41 Assessment of the contribution of AIMS to get in contact with people that are currently important in the professional network of alumni

Alumni % 2003-2005 2006-2008 2009-2011 2012- 2013

To a great extent 256 69,75% 55,56% 54,35% 67,65% 76,56%

To some extent 86 23,43% 40,74% 34,78% 24,51% 17,71%

To a small extent 18 4,90% 3,70% 4,35% 6,86% 4,17%

Not at all 7 1,91% 0,00% 6,52% 0,98% 1,56%

Total sample 367 100,00% 27 46 102 192


Figure 42 Assessment of alumni about the impact of AIMS on their career opportunities

Source: Technopolis (2015), Alumni survey

The figure below provides an overview of effects that were found to be significant in a comparative model with non-participants. As can be seen, participants outperform on income and on the number of published papers, but other indicators show no significant effect. The analysis of career achievement also showed that AIMS are more likely to successfully continue an academic career. More qualitatively, alumni generally indicate that AIMS has been a strong boost to their academic or professional career, with less than 10% of alumni indicating that they would have achieved the same without AIMS. This leads to the conclusion that AIMS has a strong additional effect of delivering skilled mathematic graduates that are better placed to pursue academic careers. The evidence on non-academic careers is currently too weak to draw any meaningful conclusions.

2 9 40 49

1 6 36 58

1 3 24 73


... increased your ability to gainand maintain employment

... will be importnet for advancingyour career in the future

... prepared you for you furtherprofessional/academic career

(N = 344)Q: Please indicate to what extent AIMS ...

Not at all

To a small extent

To some extent

To a great extent

Figure 43 Assessment of alumni about the level of success in their career without AIMS; Assessment of alumni about the amount of time to achieve the same in their career without AIMS


Research

In total 26.3% of alumni are currently carrying out a PhD, compared to only 7% of non-participants.

In general, AIMS alumni seem to be especially disposed towards an academic career with 70% of the first cohort (2003-2005) in academic employment and a further 11% pursuing a PhD. The share of alumni focusing on an academic career is consistently above 80% throughout the cohorts. Rejected applicants are much more likely be engaged in a non-academic career in comparison to alumni.

The effect of post-AIMS research bursaries is quite clear: 59% of alumni who did continue with research carriers obtained AIMS facilitated bursaries.

8 52 40


Level of success

(N = 340)Q: Without AIMS, would you have been able to achieve the same level of succes in your career?

The same without AIMSSomewhat less without AIMS

Very little without AIMS

9 49 42


Time to achieve same level of success

(N = 338)Q: Without AIMS, would it have taken longer to achieve the same level of success in your career?

The same time without AIMSSomewhat more time without AIMS

Much more time without AIMS

Figure 44 Overview of alumni and their participation in Post-AIMS Research Opportunities. (Multiple answers allowed)


Figure 45 Likelihood that alumni would have been able to arrange a different source of funding without this/these research opportunity/ies

Number of alumni

%

Very unlikely 34 16,75%

Unlikely 75 36,95%

Likely 86 42,36%

Very likely 8 3,94%

Total 203 100%


Figure 46 Consequences of hypothetically not having this/these research opportunity/ies

Number of alumni

%

Would have continued their studies 23 11,27%

Would have continued their studies, but at a later point in time 76 37,25%

Probably would not have continued their studies 74 36,27%

Definitely would not have continued their studies 31 15,20%

Total 204 100%


166

13

40

183


(N = 370) (Multiple answers allowed)Q: Following graduation from AIMS, did you receive any of these research opportunities?

Post-AIMS bursary

DAAD Fellowship

AIMS Alumni Small Research Grant

None of these research opportunities

It is clear that AIMS bursaries are an important addition to the existing bursaries’ system. The rate of AIMS students pursuing a PhD is proof of the assistance given by AIMS bursaries: looking at the 2009-2011 cohort10 for instance, of those with bursaries, 49% are currently pursuing a PhD or have finished one as opposed to 38% of other alumni.

Counterfactual analysis table J.5 Whereas simply binary comparison between participants and non-participants has been used for various analyses of career paths, a number of more (semi-) quantitative variables can be used in a counterfactual model where outcomes are controlled for by gender and years since graduation, as well as various derived terms. The key results are presented below. Again we must stress the fact that due to sample control difficulties the results should be treated with care. All information in the table originates from the survey and is, as such, self-reported.

Figure 47 Model analysis: Presence of significant effects (analysis through ordened logits, probit or conventional regression where appropriate)

Variable Participant Gender Years since application11

Gross income (see also analysis below)

• Lower start, higher growth rate over time

• Positive effect for participants

Staff responsibility • Lower for women (18 vs 26 for men)

Skill use • Mathematical skills less used by women

• Research skill use increases over time

Number of published papers

• Participants start with less publications but catch up later (av. 1.2 articles for alumni, 0.7 for non-participants)

• Number of papers increases over time

Focus on African challenges in current occupation

• Lower for participants • Lower for women

Current skill levels • Lower mathematical skills for women

• Lower innovation/entrepreneurship skills for women

Awards • Lower for participants • Lower for recent applicants/alumni, afterwards higher

Companies founded • Lower for participants

Innovations

Source: Technopolis 2015

10 Alumni survey 2015 11 Includes analysis of squared and interaction terms

Supporting tables J.6

Figure 48 Overview of the number of people that alumni and non-admitted applicants are responsible for in their current occupation

Applicants Alumni

Alumni Non- admitted Female Male 2003-2005

2006-2008 2009-2011 2012-2013

Zero 136 82 55.10% 34.75% 21.74% 45.45% 39.58% 42.69%

One to two 37 40 7.14% 12.71% 17.39% 6.82% 9.38% 12.28%

Three to five 51 60 10.20% 17.37% 8.70% 11.36% 18.75% 15.20%

Six to ten 15 44 4.08% 4.66% 8.70% 6.82% 4.17% 3.51%

Eleven to twenty

9 24 2.04% 2.97% 8.70% 2.27% 4.17% 1.17%


8 23 3.06% 2.12% 4.35% 0.00% 3.13% 2.34%

Thirty-one to forty

5 12 1.02% 1.69% 0.00% 2.27% 1.04% 1.75%

Forty-one to fifty

5 6 1.02% 1.69% 0.00% 0.00% 1.04% 2.34%

More than fifty

68 115 16.33% 22.03% 30.43% 25.00% 18.75% 18.71%

Total sample 334 406 98 236 23 44 96 171


Figure 49 Overview of the ranges of the monthly gross income in USD of alumni and non-admitted applicants of their current occupation

Applicants Alumni


2006-2008 2009-2011 2012-2013

Less than $500 121 214 39.58% 35.32% 0.00% 13.95% 26.09% 53.53%

Between $501 and $1000 88 95 26.04% 26.81% 15.38% 20.93% 25.00% 30.59%

Between $1001 and $1500 27 36 8.33% 8.09% 3.85% 9.30% 7.61% 8.82%

Between $1501 and $2000 39 27 11.46% 11.91% 26.92% 16.28% 19.57% 4.12%

Between $2001 and $2500 18 11 4.17% 5.96% 7.69% 13.95% 7.61% 1.76%

Between $2501 and $5000 33 17 9.38% 10.21% 38.46% 20.93% 13.04% 1.18%

More than $5000 5 2 1.04% 1.70% 7.69% 4.65% 1.09% 0.00%

Applicants Alumni


2006-2008 2009-2011 2012-2013

Total sample 331 402 96 235 26 43 92 170


Bibliometric analysis Appendix K

Methodology K.1 Bibliometric analysis (or bibliometrics) can basically be defined as the analysis of the number of research papers published for example by a researcher, a research team, an institution or country, or in which scientific disciplines they are active. It can also be an analysis in terms of the impact of the publications – how often they are cited by others. There are different indicators used in bibliometrics to capture scientific output and quality:

• Publication counts is the most basic of the bibliometric methods and measures the total research output. It is the main indicator for productivity but does not tell anything about the quality of the research (even though academic journals rely on quite rigorous filtering processes through peer review). Most publications provide only – if any –small contributions to the body of scientific knowledge, whereas a few influential papers provide significant contributions.

• Citation counts address questions of quality, influence and transfer of knowledge. It assumes that the most cited publications contain eminent research findings, which contribute the most to science. However, there are many factors challenging this assumption: citation cartels (i.e. a group of researchers cites predominantly work of the cartel members), self-citations (which we exclude from the count), review articles (which receive a high number of citations because of their usefulness as a summary) or negative references (i.e. if a researcher has published an article which states the opposite of the mainstream of a given school of thought. Most likely, he or she will be cited by the mainstream researchers). Still, one of the main purposes of citation analysis is to serve as a proxy for the contribution to scientific progress.

• The h-index (Hirsh-index) is an index that quantifies both the actual scientific productivity and the citations counts (the apparent scientific impact). The index is based on the set of a scientist’s most cited papers and the number of citations that these papers have received in other people’s publications. An h-index indicates the number of papers that have at least that same number of citations. The index can also be applied to the productivity and impact of a group of scientists, such as a department or university or country.

• Another indication of scientific impact is the impact of the journals the papers are published in. There are different indicators for the impact of journals. A common indicator is the average number of citations per publication in a given period. This indicator is easy to understand and to calculate. However, as it does not take into account that citation behavior differs between subject fields, and is thus is an unreliable indicator to compare journals across fields. Therefore, we use the Source Normalized Impact per Paper (SNIP) which takes into account several characteristics of the source (the journal) in which the paper is published. In particular, it takes into account the subject field of a source which it determines using the set of documents citing that source. It then considers the subject field’s frequency at which authors cite other papers in their reference lists, the speed at which citation impact matures and the extent to which the database used in the assessment covers the field’s literature. SNIP a comparative indicator; a SNIP value that is higher than one means that the journal has an above average SNIP for its field. A SNIP that is lower than one means that the journal has a below average SNIP for its field. If SNIP is equal to 1, the journal is absolutely average for its field.

• Network analysis provides insight in collaboration patterns of the researchers and alumni of AIMS. An analysis of the network indicates the level of researchers and institutes the AIMS works together.

There are many different databases available for bibliometric purposes. Technopolis Group used

SciVerse Scopus scientific abstract and citation database.12 This database, comparable to the Thomson Reuters ISI database, currently contains almost 22,000 journal titles with more than 53 million articles.

The next sections present two bibliometric analyses: one of the scientific output of AIMS’ researchers and one of the scientific output of AIMS’ educational program alumni.

Bibliometric analysis for AIMS K.2 The bibliometric analysis below is based on two databases of AIMS publications originated from two sources:

• All papers in Scopus13 that have an author (co-authors included) with an affiliation with AIMS at the time of publishing. Scopus identified 210 papers.

• All papers from 2009 until May 2015 of Non-Alumni Network Members of AIMS, provided by AIMS to Technopolis, which were also in the Scopus database. The complete list counted 15314 publications and of these, 129 publications were in the Scopus database. Of the 129, 25 publications were not in the first database (publications Scopus identified as having an AIMS affiliation). These mostly include papers from visiting researchers. These 25 publications were added to the main publications database for the bibliometric analysis.

As such, the analysis below is based on 235 papers. These can include papers from AIMS alumni who have worked at AIMS. In addition to these papers, there are 14 publications provided by AIMS that were not included as no data in Scopus was found. These mostly included non-peer reviewed publications in university or online journals and blogs.

The analysis of AIMS scientific output is based on the output of all the AIMS centres, and does not group articles per centre. An analysis per centre is not feasible, as researchers affiliated with the South African Centre have published the vast majority of the articles. According to the publications list received from AIMS, each other centre has one researcher that published during his/her work at AIMS. These three researchers together account for 11.6 % of the publications (0.6% from Ghana, 0.6% from Cameroon and 10.4% from Senegal). In addition, researchers may be affiliated with more than one centre but only have one affiliation (South Africa) mentioned in the published paper.

AIMS’ scientific output K.2.1 The scientific output of AIMS in terms of number of publications is rising, which corresponds to the growth AIMS has experienced in the recent years. The figure below illustrates this growth from 2010 until 2014. During this period the output has grown with an average factor of 1.8 per year. Between 20% to 30% of the scientific output is from visiting researchers.

12 http://www.scopus.com/ 13 Retrieved June 2015 14 The list contained 157 publications with 4 duplicates.

Figure 50 AIMS publications per year (non-cumulative)


The papers published consist of 90% of published articles in journals; the other 10% include conference papers, articles in press, notes and reviews. The three main subject areas of papers are Physics and Astronomy (34%), Earth and Planetary Sciences (21%) and Mathematics (16%).15

Figure 51 sums the top 10 researchers that have worked at AIMS since 200916 in terms of their h-index. The number of publications, citations and h-index are based upon their total research output during their career. The top 10 are all male researchers and affiliated with AIMS in South Africa.

Figure 51 Top 10 researchers that have worked at AIMS

Name (* still at AIMS) Nationality Position Number of publications

Number of citations

h-index

Romeel Davé* American Research Chair 143 7911 48

Bruce Bassett* South African Senior Resident

Researcher 98 8446 39

Martin Kunz* Swiss/German? Visiting Professor 149 3352 32

Roberto Trotta Italian Research Fellow 72 2127 28

Douw G. Steyn South African Senior Visiting


Delfim F.M. Torres Portugese Researcher 171 2044 24

Cang Hui Chinese Research Chair 95 1122 18

Ignacy Sawicki Polish Visiting


Alan F. Beardon British Senior Resident

Researcher 65 280 9

Ronald I. Becker South African Senior Resident

Researcher 32 195 9


15 Papers can have multiple subject areas 16 List of researchers since 2009 is based on the Non-Alumni Network Members with publication provided by AIMS

Scientific impact of AIMS K.2.2 There are two main indicators for the scientific impact of an institution: the number citations of its publications and the impact of the journals it publishes in.

Citations

Figure 52 shows the key citations figures. As the output of papers by AIMS has been rising, so has the cumulative number of citations AIMS’ papers have received (1624 on the 30th of June 2015). The indicator of interest though is the average number of citations per paper, which is 7 citations per paper. Another key indicator is the h-index, which is 19. This indicates that AIMS researchers have published 19 documents that have been cited at least 19 times. Figure 53 shows how the average number of citations per paper has evolved over time. This indicator has a cumulative character as it takes the average of all proceeding years. The graph shows a relatively high number of citations per paper in the years up to 2009, a low in 2011 and a slightly increasing trend after 2011. The relative high citation rate in the first years can be explained by the relative low number but successful publications during this period.

Figure 52 Key citation figures

Number of citations Citations per paper Percentage with no citations

Percentage with no citations until 2014

h-index

1624 7 50% 33% 19


Figure 53 Total number of citations and citations per paper


Figure 54 sums the top 5 cited papers. It shows that the most successful papers were published in 2004-2006 and in 2014, which corresponds to the graph above.

Figure 54 Top 5 cited papers AIMS researcher

Paper #

M. Kunz Ade, P. A. R., Aghanim, N., Armitage-Caplan, C., Arnaud, M., Ashdown, M., Atrio-Barandela, F., ... & Davies, R. D. (2014). Planck 2013 results. XVI. Cosmological parameters. Astronomy & Astrophysics, 571, A16.

304

P.G. Ferreira

Skordis, C., Mota, D. F., Ferreira, P. G., & Boehm, C. (2006). Large scale structure in Bekenstein’s theory of relativistic modified Newtonian dynamics. Physical Review Letters, 96(1), 011301.

113

P.G. Ferreira; K. Moodley

Dunkley, J., Bucher, M., Ferreira, P. G., Moodley, K., & Skordis, C. (2005). Fast and reliable Markov chain Monte Carlo technique for cosmological parameter estimation. Monthly Notices of the Royal Astronomical Society, 356(3), 925-936.

105

M. Kunz Ade, P. A. R., Aghanim, N., Armitage-Caplan, C., Arnaud, M., Ashdown, M., Atrio-Barandela, F., ... & De Bernardis, P. (2014). Planck 2013 results. XXII. Constraints on inflation. Astronomy & Astrophysics, 571, A22.

103

P.G. Ferreira

Zlosnik, T. G., Ferreira, P. G., & Starkman, G. D. (2006). Vector-tensor nature of Bekenstein’s relativistic theory of modified gravity. Physical Review D, 74(4), 044037.

47


Journals

The Source Normalized Impact per Paper (SNIP) is a comparative indicator of the impact of journals. SNIP is the ratio of a source's average citation count per paper and the citation potential of its subject field. A SNIP value that is higher than one means that the journal has an above average SNIP for its field. A SNIP that is lower than one means that the journal has a below average SNIP for its field. If SNIP is equal to 1, the journal is absolutely average for its field.

Figure 55 shows the SNIP for the journals most published in since its inception. AIMS has published 111 papers in these journals which constitutes 47% of all publications. The SNIP factor in the table represents the average SNIP factor of the journal from 2004 until 2014. The AIMS mostly publishes in journals with an above average impact factor.

Figure 55 Impact factors for the top 5 journals with most AIMS publications

Journal Number of publication

SNIP 2004 - 2014

SNIP 2014

Astronomy and Astrophysics

50

1.22 0.95

Astrophysical Journal 17 1.53 1.19

Journal of Cosmology and Astroparticle Physics 15 0.88 0.97

Physical Review D - Particles, Fields, Gravitation and Cosmology

15 1.36 1.16

Monthly Notices of the Royal Astronomical Society 14 1.35 1.33


The table above represents the average SNIP factor of the journals from 2004 until 2014 and from 2014, and does not take into account the SNIP factor of journals at the moment of AIMS publication. Figure 56 illustrates the average SNIP per publication per year (the solid red line). The figure indicates that the average SNIP was at a relative high level in the first couple of years, at a low in 2009 and slightly higher than 1 thereafter. However, the average SNIP before 2010 is based on only a few publications, as the grey line indicates. The high SNIP in 2004 – 2006 is due to several publications in the high impact journal Physical Review Letters. The low SNIP in 2009 is due to the publication in the Journal of Bionanoscience, which had a SNIP of 0.012. The only other publication in 2009 was published in a journal with an unknown SNIP factor for that year. The overall average SNIP factor per publication is 1.2, which indicates that AIMS publishes in journals that perform slightly better than other journals in similar fields.

Figure 56 The average Source Normalized Impact per Paper (SNIP) over 2004 – 2014*

* Publication in journals with a SNIP of zero were excluded Source: Technopolis 2015, Scopus

Network analysis K.2.3 A network analysis maps the countries and institutions the AIMS researchers co-publish its papers with.

Figure 57 illustrates the countries the AIMS co-publishes papers with. The figure shows that the AIMS works mostly together with institutes in South Africa, US, Canada and Europe. In Europe the AIMS mostly co-published with researchers from the UK, Switzerland, Spain, Germany and Italy.

Figure 57 Network analysis of co-publications


The country network analysis maps per paper the countries the authors are situated. However, if AIMS

(100,300](50,100](25,50](10,25](5,10][1,5]No data

co-publishes one paper with multiple authors from a country, the analysis does not incorporate this. Therefore it is also important to map the institutions the AIMS co-published with. Figure 58 shows the top 10 institutes in terms of co-publication.

Figure 58 Top 10 institutes in terms of co-publications

Institute Number of co-publications Country

Université de Genève 70 Switzerland

Niels Bohr Institute 53 Denmark

University of California, Santa Barbara 52 United States

University of Cambridge 52 United Kingdom

Princeton University 51 United States

Imperial College London 51 United Kingdom

Lawrence Berkeley National Laboratory 49 United States

Helsingin Yliopisto 48 Finland

CEA Saclay 48 France

CNRS Centre National de la Recherche Scientifique 47 France


In addition to the above top 10 institutes AIMS co-publishes with, AIMS works together with many African institutes as well. Figure 59 shows the top 5 African institutes in terms of co-publications, which are all South African institutes. The only other African institute that is not in South Africa that AIMS co-published with frequently (17 papers) is Universite de Yaounde I in Cameroon.

Figure 59 Top 5 African institutes in terms of co-publications

Institute Number of c0-publications Country

University of KwaZulu-Natal 41 South Africa

University of Cape Town 33 South Africa

Universiteit Stellenbosch 32 South Africa

South African Astronomical Observatory 26 South Africa

University of the Western Cape 26 South Africa


Benchmark with other African institutes K.2.4 There are different indicators that can be compared among universities, research institutes and research departments to understand how well AIMS is doing. First of all the SNIP factor of the journals AIMS’ researchers publish in, is an indicator of the impact of research. The average SNIP factor of 1.2 indicates that AIMS is doing slightly better than the average in its field worldwide.

In addition to the SNIP factors, there are several published benchmarks that indicate AIMS’ performance in relation to other African institutes. Nature Index is the most interesting for AIMS as it benchmarks on article output in the natural sciences.

Nature index17

The Nature Index, compiled by the Nature Publishing Group (NPG), is a database of author affiliation information collated from research articles published in an independently selected group of 68 high-quality natural sciences journals. The 68 journals represent less than 1% of the journals covering natural sciences, but account for more than 30% of the total citations. This index is thus good

17 See http://www.natureindex.com/

indication of AIMS’ research performance in the natural sciences.

Figure 60 shows the top 10 institutes (academic as well as government and NGO) in Africa according to their articles output in the 68 high-quality journals in the last 12 months (1 June 2014 – 31 May 2015).

Article output is counted in three ways18:

• Article count (AC): where a count of one is assigned to an institution or country if one or more authors of the research article are from that institution or country, regardless of how many co-authors there are from outside that institution or country.

• Fractional count (FC): that takes into account the percentage of authors from that institution (or country) and the number of affiliated institutions per article. For calculation of the FC, all authors are considered to have contributed equally to the article. The maximum combined FC for any article is 1.0.

• Weighted fractional count (WFC): a modified version of FC in which fractional counts for articles from specialist astronomy and astrophysics journals have been down weighted. These journals encompass a much larger proportion of the total publication output of these fields than any other field covered by the Nature Index. The WFC allows ordering of institutions and countries so as not to give undue emphasis to these fields. The weighting is achieved by multiplying the fractional count from these astronomy and astrophysics journals by a factor of 0.2. This down weighting is in proportion to an approximation of the level to which astronomy and astrophysics articles are overrepresented compared to the total publication output of other fields covered by the Nature Index.

The index is ranked according to the article account (AC), in which AIMS ranks 10th in Africa. The fractional count and the weighted fractional account are relatively low. This indicates that AIMS’ papers in high quality journals are often written with many co-authors from other institutes and that a large share of the articles has been published in specialised astronomy and astrophysics journals.

Figure 60 Nature Index ranking

Rank Institution AC FC WFC

1. University of Cape Town (UCT) 113 23.2 14.74

2. University of KwaZulu-Natal (UKZN) 67 11.45 10.2

3. South African Astronomical Observatory (SAAO) 66 11.94 2.74

4. University of the Western Cape (UWC) 56 10.02 2.76

5. University of the Witwatersrand, Johannesburg (Wits University) 54 10.39 9.23

6. SKA South Africa 41 3.4 0.78

7. Rhodes University (RU) 32 2.88 1.3

8. Stellenbosch University (SU) 31 11.89 11.63

9. North-West University (NWU) 29 9 1.95

10. African Institute for Mathematical Sciences (AIMS) 21 1.17 0.67

Analysis for AIMS Alumni K.3 For the bibliometric analysis for the AIMS Alumni three main sources were used:

• Publications in Scopus that could be matched to the Alumni name (170 alumni) • Publications from the Alumni online profiles (74 alumni)

18 Cited from http://www.natureindex.com/faq#methodology3

• Publications provided by AIMS (91 alumni) All alumni until AIMS-year 2013 were analysed, amounting to 560 alumni. Using their full name and known publications from the online profiles and those provided by AIMS, we matched 170 alumni to articles in Scopus. Scopus identified 574 unique19 publications from these 170 alumni.

In addition to these publications, some alumni have published scientific work outside the scope of Scopus, e.g. on online blogs. From the online profiles and publications provided by AIMS we can distil that at least 23 alumni have published papers that are not in the Scopus database. These mostly included papers published in online open sources. The analysis below is based on publications in the Scopus database.

Scientific output of alumni K.3.1 Figure 61 sums the key figures of the scientific output of alumni until AIMS year 2013.

Figure 61 Key figures

Sum of publications per alumni20 587

Alumni with at least 1 paper 170

Alumni with at least 2 papers 110

Female alumni in database Male alumni in database

174 (31% of alumni) 293 (69% of alumni)

Female alumni with publications Male alumni with publication


Female alumni with at least 2 publications Male alumni with at least 2 publications


Publications by female alumni

95 (16 % of publications) 492 (84% of publications)


The scientific output of alumni in terms of number of publications is rising each year, which is not surprising as the number of alumni and their years of experiences rise as well. Figure 67 illustrates the number of publications from 2004 until 2014. In addition, in 2015 (until June 2015) another 60 papers were published. The total number of published papers (until June 2015) amounts to 574.

19 Some publications were written by more than 1 alumni 20 Larger than number of unique publications and some publications have more than 1 alumni as author

Figure 62 Alumni publications per year


The papers published consist of 75% of articles in journals (including articles in press), 22% of conference papers. The other 3% include notes, book chapters and reviews. The five main subject areas of papers are Mathematics (19%), Physics and Astronomy (19%), Computer Science (11%), Engineering (96%) and Medicine (9%).21 Figure 63 sums the top 10 alumni according to their h-index and number of citations. The number of publications, citations and h-index are based upon their total research output during their career. One the alumni, Mazandu Gaston Kuzamunu, currently is affiliated with AIMS. All alumni in the top 10 are male.

Figure 63 Top 10 alumni researchers

Name Affiliation Publications Citations h- index

Osalusi Emmanuel Heriot-Watt University, International Centre for Island Technology, Edinburg, UK 12 169 8

Abdussalam Shehu Shuaibu,

Abdus Salam International Centre for Theoretical Physics, Triestem Italy 12 285 7

Ndeffo Martial Loth Mbah Yale University, Center for Infectious Disease Modeling and Analysis, New Haven, US 20 99 7

Mazandu Gaston Kuzamunu

African Institute for Mathematical Sciences, Muizenberg, South Africa 17 82 6

Mabiala Justin Cyclotron Institute, College Station, US 29 72 6

Okeke Onyekwelu Uzodinma

Harvard University, Department of Physics, Cambridge, US 6 93 4

Hamdouni Yamen N/A 10 92 4

Johnstone-Robertson Simon Peter

University of Melbourne, Parkville, Australia 6 85 4

Akofor Earnest Syracuse University, Department of Electrical Engineering and Computer Science, US 9 79 4

Worku Dawit Solomon

University of Cape Town, UCT-CERN Research, Centre and Department of Physics, Cape Town, South Africa 4 71 4

Source: Technopolis 2015, Scopus 21 Papers can have multiple subject areas

Most researchers have an h-index of 1 or 0 (no publications and/or no citations), which is not surprising as most alumni have recently graduated. Figure 64 shows the number of alumni per study years for each h-index up to the maximum (8) in the sample.

Figure 64 Number of alumni with per AIMS year and h-index


Scientific impact of alumni K.3.2 There are two main indicators for the scientific impact of an institution: the number of citations of its publications and the impact of the journals it publishes in.

Citations

Figure 65 shows the key citations figures. The cumulative number of citations on the 30th of June 2015 is 1663. The average number of citations per paper is 2.9. However, more than half of the papers have not received any citations. This is not surprising as many papers have been published as conference papers and/or have been published recently. If papers with zero citations are excluded, the average citations per paper amounts to 6.9 per paper. Another key indicator is the h-index, which is 19. Thus, 19 documents of alumni have been cited at least 19 times. Figure 66 shows how the average number of citations per paper has evolved over time. This indicator has a cumulative character as it takes the average of all proceeding years. The graph shows an increasing number of citations per paper. Thus, the number of cumulative citations if rising faster than the number of cumulative papers.

Figure 65 Key citation figures

Number of citations Citations per paper Percentage with no citations

Percentage with no citations until 2014

h-index

1663 2.9 58% 53% 19


0 10 20 30 40 50

2004-2005

2006-2007

2008-2009

2010-2011

2012-2013

Number of researchers

1 or no citations 2 to 3 4 to 5 6 to 8

Figure 66 Total number of citations and citations per paper - alumni


Figure 67 sums the top 5 cited papers. It shows that Shehu S.Abdussalam published many successful papers.

Figure 67 Top 5 cited papers Alumni Paper reference # G. Mbianda Puckett, A. J. R., Brash, E. J., Jones, M. K., Luo, W., Meziane, M., Pentchev, L.,

... & Huber, G. M. (2010). Recoil polarization measurements of the proton electromagnetic form factor ratio to Q 2= 8.5 GeV 2. Physical review letters, 104(24), 242301. 112

Tendai Mugwagwa

Den Braber, I., Mugwagwa, T., Vrisekoop, N., Westera, L., Mögling, R., de Boer, A. B., ... & Tesselaar, K. (2012). Maintenance of peripheral naive T cells is sustained by thymus output in mice but not humans. Immunity, 36(2), 288-297. 71

Shehu S.Abdussalam

Conlon, J. P., Abdussalam, S. S., Quevedo, F., & Suruliz, K. (2007). Soft SUSY breaking terms for chiral matter in IIB string compactifications. Journal of High Energy Physics, 2007(01), 032. 54

Shehu S.Abdussalam

Feroz, F., Allanach, B. C., Hobson, M., AbdusSalam, S. S., Trotta, R., & Weber, A. M. (2008). Bayesian selection of sign µ within mSUGRA in global fits including WMAP5 results. Journal of High Energy Physics, 2008(10), 064. 53

Shehu S.Abdussalam

Feroz, F., Allanach, B. C., Hobson, M., AbdusSalam, S. S., Trotta, R., & Weber, A. M. (2008). Bayesian selection of sign µ within mSUGRA in global fits including WMAP5 results. Journal of High Energy Physics, 2008(10), 064. 50


Journals

The Source Normalized Impact per Paper (SNIP) is a comparative indicator of the impact of journals. SNIP is the ratio of a source's average citation count per paper and the citation potential of its subject field. A SNIP value that is higher than one means that the journal has an above average SNIP for its field. A SNIP that is lower than one means that the journal has a below average SNIP for its field. If SNIP is equal to 1, the journal is absolutely average for its field.

Figure 68 shows the SNIP for the journals most published in since the first publications in 2004. The alumni have published 48 papers in these journals which constitutes 8.4% of all alumni publications. The SNIP factor in the table represents the average SNIP factor of the journal from 2004 until 2014. Four of the five journals most published in have a slightly higher impact factor than the average in

their field.

Figure 68 Impact factors for the top 5 journals with most the alumni publications

Journal Number of publication

SNIP 2004 - 2014

SNIP 2014

Physical Review C - Nuclear Physics 13 1.64 1.649

PLoS ONE 10 0.74 1.034

Journal of Physics: Conference Series 9 0.24 0.266

Journal of High Energy Physics 8 1.09 1.048

Physical Review D - Particles, Fields, Gravitation and Cosmology 8 1.36 1.159


The table above represents the average SNIP factor of the journals from 2004 until 2014 and from 2014, and does not take into account the SNIP factor of journals at the moment of AIMS publication. Figure 69 illustrates the average SNIP per publication per year (the solid red line). The figure indicates that until 2009 alumni published in slightly below average quality journals (with an exception of 2005) and from 2010 onwards in journals with a slightly above average impact factor. The overall average SNIP factor per publication is 1.1, which indicates that AIMS publishes in journals that perform slightly better than other journals in similar fields.

Figure 69 The average Source Normalized Impact per Paper (SNIP) over 2004 – 201422


Network analysis K.3.3 A network analysis maps the countries and institutions alumni co-publish its papers with.

Figure 70 illustrates the countries the alumni co-publishes papers with. The figure shows that the alumni work mostly together with researchers from institutes in South Africa, US, Canada and Europe. In Europe the alumni mostly co-published with researchers from the UK, Italy, France, Belgium and

22 Publications with an unknown SNIP or a SNIP of zero were excluded (142 publications), of which 58 were published in 2015.

Germany. Besides South Africa, the alumni co-publish with researchers from other African countries; mainly from Nigeria, Algeria, Ghana, Sudan and Tanzania.

Figure 70 Network analysis of co-publications


The country network analysis maps per paper the countries the authors are situated. However, if alumni co-publish one paper with multiple authors from a country, the analysis does not incorporate this. Therefore it is also important to map the institutions the alumni co-published with. Figure 71 shows the top 15 institutes in terms of co-publication. It shows that the universities the alumni co-publish with most often are predominantly in South Africa.

Figure 71 Top 15 institutes in terms of co-publications

Institute Number of c0-publications Country

University of Cape Town 83 South Africa

Universiteit Stellenbosch 61 South Africa

University of KwaZulu-Natal 41 South Africa

University of the Western Cape 38 South Africa

University of Witwatersrand 36 South Africa

Katholieke Universiteit Leuven 23 Belgium

Heriot-Watt University 21 United Kingdom

Cyclotron Institute 20 United States

Texas A and M University 19 United States

The Council for Scientific and Industrial Research 17 South Africa

Ithemba Laboratory for Accelerator-Based Sciences 16 South Africa

University of Cambridge 15 United Kingdom

INFN - Laboratori Nazionali del Sud 15 Italy

Yale University 14 United States

Khartoum University 14 Sudan


(100,300](50,100](25,50](10,25](5,10][1,5]No data

IDRC and DFID completed indicators by pillars Appendix L

Training Pillar L.1 Legend

✓ = roughly on track - = below expectation + = above expectation


Result type Result Indicators23 Target 2014/ 2015

Value / assessment (up to 2014)

1.1. Impacts Productivity and growth (includes alumni impact on economy)


9 34 alumni were involved with innovations24. (±)

1.2. Impacts New leaders and example

% of AIMS alumni in high level leadership positions in and out

13% <1% (-)

1.3. Impacts New leaders and example

% awards 10% 29% (+)

2.1. Outcomes Better employment opportunities for graduates

Working on Africa’s challenges in education or employment

95% 72% / 88% (✓ )


Employment in industry 15% 10% (-)


N° of grads into entrepreneurship - disaggregated by sector

8% 1% (full-time), 8% (part-time) (-)


N° of grads going to research (PhD / being researchers)25

50% in 5 years After 5 years 60% is in a PhD (+)

3.1.1. Outputs Skilled Graduates F/M

N° of skilled graduates 1200 within five years Currently 748 graduates, 36% women.

3.2.1. Outcomes Increase of skilled staff available in Africa

Perception of companies of interns skills

NA NA


Result type Result Indicators Target Value

23 For disaggegation, see relevant sections in the chapter above 24 Self assessment of alumni 25 This is identified as academic employment at a university or a PhD. It is difficult to distinguish between teaching at university and research as it is usually a mix.


Result type Result Indicators Target Value

Output 2.3 Outcomes Better employment opportunities for graduates

N° of graduates going to jobs that requires skills acquired at AIMS (6 months after)

128 195 (+)

Output 2.4. Outcomes Better employment opportunities for graduates

N° of grads going to further education opportunities: type of course/level of course (flow chart)

336 496 (+)

Output 2.5 Outputs Interns N° of interns (in industry) 5% 11% (+)

Outcome 1 Impacts New leaders and example

Based on titles 3 NA, Sample does not contain examples (±)

Outcome 2 (adapted)

Outcomes Increase of skilled staff available in Africa

Perception of companies of AIMS graduates skills

85% Overall satisfactory, but also criticism (✓)

Impact 1 Impacts Productivity and growth (includes alumni impact on economy)

Change in GDP 4.5% annual growth 2014: South Africa 1.4% (-) Cameroon: 5% (+) Ghana: 4.5% (+), Tanzania 7.2%

Impacts Productivity and growth (includes alumni impact on economy)


9 34 alumni were involved with innovations26. (±)

Research pillar L.2 IDRC indicator reference

Result type Result Indicators Target 2014/ 2015


1.4.27 Outcome Scientific outputs/ Publications

N° publications by type (scheduled, actual)

400 191 by the research team and 587 by alumni (+)




3. Impacts Scientific impact Publications and patents at country level

Publications: 31,500 Patents: 200

N/A

5.1.28 Output Scientific outputs/ Publications

N° publications by type (scheduled, actual)

400 191 by the research team and 587 by alumni (+)

26 Self assessment of alumni 27 This indicator does not relate only to the Research pillar but also to alumni activities in research 28 This indicator does not relate only to the Research pillar but also to alumni activities in research

Public engagement pillar L.3




3.3.1. Outcome Improved education opportunities for children

N° of attendees to events (public lectures or teachers training)

Attendees - 1,600 (SA - 1000, CA - 200, GH - 200 & SE - 200)

4582 (+)




2 Impact Increased interest for and skills in Math & Sciences

% of enrolment in STEM Master’s at country

Engineering: BEN - 4.9% Men & 2.7% Women; CMR - 6.9% Men & 2.1% Women; GHA - 10.5% Men & 3.7% Women; TZA - 5.3% Men & 3.2% Women

Science: BEN - 10.3% Men & 6.1% Women; CMR - 24.3% Men & 12.5% Women; GHA - 12% Men & 7% Women; TZA - 7% Men & 4.2% Women

N/A

3.1 Output Public reached through public engagement

N° of attendees at events Attendees - 1,600 (SA - 1000, CA - 200, GH - 200 & SE - 200)

Participants - 176 (SA - 44, CA - 44, GH - 44 & SE - 44)

Attendees: 4522 (+)

Trained teachers: 60 (-)

Organisational development pillar L.4 IDRC indicator reference



3.4.1 Outcome Well-ran organisation

Cost per student $19,500 $20,000 to $38,000 (✓-)

3.5.1 Outcome Well-ran organisation

Quality of the M&E data/assessment of the M&E system

59%-67% N/A




1.1 Output New Centres N° of Centres accepting applications

5 5 (✓)

1.2 Output New Centres Gender ratio Ratio of women:men involved in decision-making positions 1:3 – Sec; 2:5– SA 1:3 – SE; 2:1 – GH 1:2 – CA

Ratio of women:men not involved in decision-making positions in AIMS 1:1 – Sec 2:5– SA 4:11 – SE 1:10 – GH 1:2- CA 1:2 -TA

N/A

4.1 Output Well-ran organisation

Management systems performance of staff

25% N/A


Financial stability of AIMS and diversification of funding sources

27.9 million (increase of 1 new major funder (providing +$5 million) per year)

N/A


Number of partnerships that contribute to AIMS achieving results and opportunities to influence policy at country and pan-African level

Government Partners 4 - SA, 2-GH, 3-SE, 2-CA, 1-TA Industry Partners 3 - SA, 3-GH, 3-SE, 2-CA, 1-TA

Academic Partners 7 - SA, 6-GH, 7-SE, 5-CA, 3-TA

Policy Advocacy Activities 3-SA, 3-GH, 3-SE, 2-CA, 1-TA

Government Partners 4 – SA, 2-GH, 3-SE, 2-CA (✓+) Industry Partners 5 - SA, 0-GH, 4-SE, 2-CA (+-)

SWOT Appendix M

Strengths Weaknesses

• AIMS is built on a strong vision and core values that translate to motivated and dedicated staff, lecturers, tutors and students

• The AIMS leadership is visionary, inspiring and enthusiastic, and has been successful in translating this to funding for its programs

• A 24-hour learning environment where the students live together and interact with lecturers, tutors and researchers gives students a unique transformative formation experience

• The Secretariat-Centre model allows for the necessary local ownership and flexibility needed to succeed in different countries

• AIMS-NEI secretariat is a professional, driven and flexible organisation

• AIMS offers a high-quality course with exposure to international expertise that builds relevant skills with students

• Program quality is recognised by students, lecturers, partners and governments

• The focus on ‘future-ready’ skills and high computer literacy of students is appreciated by all respondents

• Innovative model which is very appropriate to the needs of African students

• AIMS has strong partnerships with local and international universities and a large network

• AIMS alumni are very successful in getting admitted to further education and a large share get a PhD position at a reputable university

• The teacher-training program is very important as it tackles important needs

• Increasingly strong brand name and rising number of applications as a result

• Generally strong connections with national governments with (various levels of) concrete support

• A professional organisation in terms of financial management, HR and other operational aspects

• Communication between secretariat and centres not always optimal; some centre staff not equally engaged

• HR issues, in particular delayed recruitment of talented individuals, is placing strain on both the secretariat and centres

• The AIMS degree is not considered equivalent of a full research Master’s in Mathematics and students often need to complete another Master degree before a PhD

• Relatively limited engagement with industry

• Entrepreneurship and innovation skills development is less focused on, a majority of students choose academia

• A strong dependency on multiple donors makes governance and independent strategy setting more difficult

• Suboptimal facilities in terms of teaching facilities and enough student dorms in some countries (especially Ghana) could limit further growth

• Extracurricular activities are not adequately developed across centres

• Low level of synergy between the main pillars

• Clarity of mission and purpose is reduced with the growing number of activities with different objectives

• Financial model relies heavily on donors even though dependency is going down

• Limited local funding for education and especially the research pillar outside of South Africa

• Some staff is overburdened which could lead to higher turnover

• Maintaining the quality of the program as AIMS grows can be challenging

• Recruitment of good directors for the new centres is very challenging

Opportunities Threats

• AIMS is directly spot-on the trends towards more skill-based education, increasing attention for STEM and the renewed economic growth in Africa

• Offering increased research opportunities in Africa with the creation of a research centre in Cameroon that can serve a pilot for the other centres

• There is a potential for even better placement of alumni in good programs by further building on links with international partners

• AIMS has recently been recognised by the African Union as a key initiative for Africa’s future

• Current harmonisation of processes and quality standards across the centres will help to maintain quality as AIMS grows

• There is a strong interest from various national governments to support the teacher training activities of AIMS

• Development of an AIMS PhD program in maths could be relevant for countries where local PhD programs have limited resources (e.g. Ghana). AIMS has a strong position to move into this area should it want to

• Newly appointed secretariat proposal manager should improve access to international funding

• An ever increasing alumni body should increase the possibilities of showcasing success and tapping into alumni networks

• Better leverage of international connections could help alumni to find funded PhD positions after AIMS

• Operating in a pan-African context has specific challenges that have an effect on AIMS-NEI as well

• Divergence in curriculum design (in particular testing) may lead to divergence of AIMS centre models

• Governance problems at AIMS entities management level may destabilise the organisation

• Growing popularity of AIMS due to its reputation but also its fully funded nature (which is unique) could put the admission system under strain

• There is a threat that local academic partners may increasingly see AIMS as a rival in getting the best students (or funding)

• In South Africa it has been challenging to get a larger cohort of students from South Africa due to a lack of applications from this country

• Having the best students leaving home countries for Europe or South Africa because of the lack of opportunities in their home countries (brain drain)

• The large increase in AIMS graduates could overwhelm the postgraduate opportunities in African universities

• The availability of international excellent lecturers may not keep pace with the strong growth of AIMS

• A strong growth in the number and scope of activities stretches the organisational and leadership capacity of AIMS

• Growth in number of centres puts the governance model under strain, as board members currently already have multiple positions • Risk of future organisational conflict due to dual governance structures (local board as well as AIMS-NEI supervision) • Mathematical sciences, particularly research are a low priority for many African countries

technopolis |group| United Kingdom 3 Pavilion Buildings Brighton BN1 1EE United Kingdom T +44 1273 204320 E [email protected] www.technopolis-group.com

Version, September 2015 Mid-term evaluation of the AIMS ...€¦ · Version, September 2015...

Documents

Transcript of Version, September 2015 Mid-term evaluation of the AIMS ...€¦ · Version, September 2015...