September 2011Alternative system Alternative System for Admission into Engineering Programmes Report...

September 2011 Alternative system

Alternative System for Admission into Engineering Programmes

Report of the CommitteePresented to the Council of

Indian Institutes of TechnologyA proposal for consideration


Acknowledgement

Prof Acharya and his Committee members Dr BK Gairola; Sri V Joshi; Sri H Bhartia; Sri M

Tuli Directors of all IITs 2063 people who participated in opinion poll Director and Experts from ISI and their

students Colleagues from MHRD, DST and NIC Chairpersons of school boards and Chairman and members of the IIT Council Some Media representatives and all those

who helped in the exercise


Decision of the Council in its 41st meeting

Prof Acharya committee had been commissioned to study the present system of conducting examinations for admission into engineering progammes of the country and suggest alternatives

The committee presented its interim report in the 41st meeting

There was unanimity that the present system required a change as proposed

The committee was enlarged to address the issue of the need to recognize diversity of learning


Major findings of Acharya Committee

Screening based on normalized Board scores at Standard X and/or Standard XII and Multiple Choice examination replacing the two stage JEE from 2006.

Entry barrier to be raised to 60% in the +2 examinations.

Factors, other than the Standard XII marks and AIR based on PCM testing, such as raw intelligence, logical reasoning, aptitude, comprehension and general knowledge need to be considered.

Need to factor in school performance more significantly into the selection process.


Major findings of Acharya Committee

Decision based on one time test needs to be re-examined. Opportunities to improve must be built in.

Students must be relieved of the pressure of multiple JEEs.

Influence of coaching for JEE needs to be minimized. Urban-rural and gender bias has to be eliminated or

at least minimized. The objective type of examination lends itself to

undue influence of coaching. The conventional pen and paper examination with well designed long and problem solving oriented questions should be revived by keeping numbers in any JEE within reasonable limits.


Work of the present committee

1. Study of Acharya committee work2. Made Non formal survey among hundreds of school

students, parents, employers, faculty and media person3. Met and decided on General approaches4. Held with Chairmen and nominees of all school boards5. Collected data on scores of school board examinations6. Wrote down a philosophy document and placed in public

domain7. Carried out survey of public opinion poll8. Enrolled ISI for carrying out exploring statistical methods

for normalization of school board scores 9. Met with Directors and Senior Faculties of IITs10. Prepared a draft report for comments and advice of the

Council


Consultation with states and School boards

First stage Gained access to data on scores on school

board examinations through formal mechanisms

Enrolled participation in development of NTS Second Stage

Assistance in harmonization and electronic access to data

Building trust and development of a process with safeguards and integrity


Public participation through opinion poll

On-line opinion survey among the people of India For multi parametric grading system as

against single test models of JEE Screening out as against selection

strategies With Responder profile, opinion polls,

suggestions for alternative national test systems; risk mitigation strategies

Survey time slot open for three weeks


Responder profile to the poll

2063 participated 59% students; 8% teachers, 5.5% parents, 23.4%

non-teaching professionals, less than 1% coaching

~80% of respondents had taken entrance examinations in their lives

1220 students had participated 80% students were from engineering stream and

95% of them had taken entrance examination ~160 teachers had participated

~90% of them are engaged in tertiary education


Analysis of the current system of admission into engineering

960 of 2063 commented on the multiplicity of entrance examinations with different sociological implications 715 agreed with the view expressed

947 of 2063 commented on need for reform 85% voted for reform in admission

system


Inputs for reforms fo admission systems

1. Reduction in the number of examinations to one

2. Testing knowledge intensity, alignment to the 12th class syllabus

3. Reduction of dependency on coaching and pressure on students

4. Emphasis on aptitude….. (More than 90%)5. Transparency in processes6. Removal of negative marking7. Online processes8. Multiple centers, better scheduling


Responses to reform directions

Factor-in performance in school boards 66% in favor and 34% not in favor

45 of those disfavor fear that board examinations do not assessing capability and 30% fear non-uniformity

620 responded to question on Indian equivalent of SAT type 73% voted in favor of aptitude type test

646 responded to types of tests 70% prefer a mix of aptitude and advanced type

tests 629 responded questions on test features

Dominant support is for high-filter type and SAT type tests


On summary

More than 85% supported the concept of a single entrance test for admission into engineering programmes and voted for reforms

70 % voted in favor of one test with provisions for testing both aptitude and advanced knowledge

66% of people favor factoring in school board scores

Of 34% People who disfavor fear primarily the problem of non-uniformity. This could be addressed.


Some Important suggestions for the committee from opinion poll

National Test in place of multiple competitive examination is generally welcomed

Concerns expressed about the process integrity and fairness of testing

methodology Normalization methodologies across school

boards Multiple chances for candidates for

improving scores A single screening examination with a mix

of aptitude (like SAT type) and advanced (like JEE type)


Study for normalization of scores of school board examinations

Committee gained access to some relevant past data with a view to examine Stability of scores of the same school board

over time Potentials for normalization of scores across

various boards Enrolled ISI into carrying out statistical

studies for normalization of board scores


Work carried out Indian Statistical Institute

For exploring normalization methodologies for school board scores


Pilot testing

Selected four boards for pilot testing Central Board, TN Board, WB Board, ISE Evaluation years 3-4 years for each board

Variations in Density of population CBSE (5-6lakhs), TN(5.6-7.3 lakhs), WB (3.0 -4.6

lakhs, ICSE 25000-56000 Evaluation of stability of scores over time for

the same board Potentials for mapping the profiles of several

boards onto one selected board through monotone transformations


Models tested

Select a percentile score (P) for all boards and determine the scores (X1) for P across boards

MappingModel 1: Y1= {Xn – X1}/{Xm-X1}Model 2: Y2 = Xn/X1

where Xn , X1 , Xm are scores obtained by any candidate, marks corresponding to percentile P, and maximum scores obtained by any candidate in each board. Y1 will range from 0 to 1.0; while Y2 will be ratios in the range from 1.0 to Xm/X1


Observed Relationships of Scores

For the four boards over timeModel 1: Y1= {Xn – X1}/{Xm-X1}Where aggregate score percentages are used

percentile rank

no

rma

lize

d s

core

50 60 70 80 90 100

0.0

0.2

0.4

0.6

0.8

1.0

CBSE 2007CBSE 2008CBSE 2009ICSE 2007ICSE 2008ICSE 2009ICSE 2010

TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009

Normalized score vs. percentile rank: cutoff 50 %

percentile rank

no

rma

lize

d s

core

60 70 80 90 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


percentile rank

no

rma

lize

d s

core

75 80 85 90 95 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


percentile rank

no

rma

lize

d s

core

85 90 95 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


aggregate score as fraction of max score

pe

rce

ntil

e r

an

k

0.5 0.6 0.7 0.8 0.9 1.0

50

60

70

80

90

10

0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009

Percentile rank vs. aggregate score: cutoff 50 %


pe

rce

ntil

e r

an

k

0.6 0.7 0.8 0.9 1.0

60

70

80

90

10

0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009



pe

rce

ntil

e r

an

k

0.6 0.7 0.8 0.9 1.0

75

80

85

90

95

10

0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009



pe

rce

ntil

e r

an

k

0.7 0.8 0.9 1.0

85

90

95

10

0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


percentile rank

no

rma

lize

d s

core

50 60 70 80 90 100

0.0

0.2

0.4

0.6

0.8

1.0

CBSE 2007ICSE 2007ICSE 2008ICSE 2009ICSE 2010

TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009

Normalized score vs. percentile rank for PCMB: cutoff 50 %

percentile rank

no

rma

lize

d s

core

60 70 80 90 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


percentile rank

no

rma

lize

d s

core

75 80 85 90 95 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009


percentile rank

no

rma

lize

d s

core

85 90 95 100

0.0

0.2

0.4

0.6

0.8

1.0


TN 2007TN 2008TN 2009TN 2010WB 2007WB 2008WB 2009



Standardized (Normalized) score

Normalized percentile rank

Percentile rank of student – 75

100 – 75X 100.=


Some general conclusions emanating from the ISI study

Percentile scores are relatively stable over the periods studied for each board when aggregate percentage scores are analyzed

It is possible to normalize percentile ranks across boards

Correlations of normalized percentile ranks against percentile ranks of various boards map on to common linear relationship


percentile rank

no

rma

lize

d p

erc

en

tile

ra

nk

75 80 85 90 95 100

02

04

06

08

01

00

Normalized percentile rank vs. percentile rank: cutoff 75%

Transformations onto one theoretical board

ISI Experts are certain that this transformation will be the same for all boards for all years


It appears that for normalization of school board scores

A statistical method is feasible after all. ISI may be encouraged to develop the methodology further and extend it to all boards and reconstruct past scenario for present IIT and NIT students over the last four years for revalidation of the method


A pilot test among a select group of students: A suggestion

A group of statistically significant number of some volunteers from current student population may be enrolled into a pilot test for evaluating the utility of different models and suggestions. This pilot test has to be designed with care and confidentiality based on informed consent of all involved. The merit or otherwise of the approach will be discussed within the committee before decisions are taken


Summary of work done so far

Opinion poll reveals support for reforms and favor single examination with provisions for both aptitude and advanced

while making provisions for factoring-in scholastic performance in school board examinations

ISI study presents a methodology for scientific normalization of scores across school boards Percentile scores are stable over each board and it

is possible to carry out monotone transform board scores and accomplish normalization across boards

Weighting options for school and entrance tests’ performance are considered


Two approaches considered Approach 1

weighing consistency of performance in school board examinations and employ them for testing ability to write solutions and

One objective screening test with two sections; one for testing the aptitude and the other advanced knowledge in domain areas.

Approach 2 weighing consistency of performance in

school board examinations and employ them for testing ability to write solutions and

one objective aptitude test based on multiple choices and computer based correction systems


Considerations of six different options

Option 1: Deployment of Scores as criteria based on class XII performance only Equal weighting of school board scores A1and A2

Equal weighting of aptitude scores A4 and advanced scores A5

Normalized score = {A1 + A 2+A4 +A5 }/4 Option 2: Deployment of Scores as criteria

based on class XII performance only Equal Weighting of board score A3 which is {A1+

A2}/2 Equal Weighting of Aptitude scores A4 and A5

Normalized score ={A3 +A 4+A5}/3



Option 3: Deployment of Scores as criteria based on consistency of performance at class X and Class XII levels as well as in National Level Aptitude and Advanced Tests Equal weighting for aggregate as well as subject performance at

class X and Class XII levels where ) 0.1X (normalized score at class X in aggregate + normalized score at class X in subjects of choice + normalized score at class XII + normalized score at class XII in subjects of choice)

One third weighting of aptitude score 0.3 A4 One third weighting of advanced score A5 Normalized score =0.1{ Normalized aggregate class X +

normalized class X subject score + Normalized class XII aggregate + Normalized class XII subject score} + 0.3 A3and 0.3 A5

Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks Specify a Cut-off normalized percentile rank score for school

performance say as 80 or 85 percentile rank 50% weighting of National Level Aptitude score A4 for candidates

passing the cut off of percentile rank 50% weighting of National Level Advanced Score A5 for candidates

passing the Normalized score = 0.5 A4 +0.5A5



Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system according freedom for the individual institutions to select mixing proportions within a pre-specified guideline

Option 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system whereNormalized score =0.5 A2+0.5A4


Recommended preferences of the committee Option 2: Deployment of Scores as criteria based on

class XII performance Option 6: Equal weighting of School Board

performance as subject score and National Level Aptitude Test as objective test system 0.5 A2+0.5A4

Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system with freedom for the individual institutions to select mixing proportions within a pre-specified guideline

Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks; Specify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank; and rank by 0.5 A4 +0.5A5


A Suggestion

A committee of experts from engineering institutions could be assigned the task of interacting with ISI Group for internalization of methodology for normalization of board scores

IITs could be assigned the task of setting up a question paper for National Screening Test based on objective examination models and conduct the examinations for the year 2012-13

For Aptitude examination like SAT, we may take the help of NTS or any other global agency


Thank you For the patience and waiting

We recommend normalization of school board scores for factoring-in based on ISI inputsSingle National level test to cover aptitude and advanced or aptitude alone andJudicious mix of school and national test performance

September 2011Alternative system Alternative System for Admission into Engineering Programmes Report...

Documents

Transcript of September 2011Alternative system Alternative System for Admission into Engineering Programmes Report...