An Analysis of 2015 HSC Examination Results for Selected ...

52 Bangladesh Education Journal

Decentralization of PECE can be undertaken initially on a trial basis in a limited number of upazilas. The pilot upazilas could include a mix of high and low performing upazilas, urban and rural combinations, and upazilas that differ based on socio-economic and learning status t. Provided it is undertaken carefully, this trial can lay the foundation for future decentralization of the entire primary education system in Bangladesh. The reform needs to be slowly and carefully planned, engaging all primary education stakeholders at both the local and central levels.

References:

1. Ahmed, M. (2015). “Primary Education: Learners or Just Test-Takers?” The Daily Star, August 19, 2015. http://www.thedailystar.net/op-ed/primary-education-learners-or-just-test-takers-128776

2. CAMPE (2014). Whiter Grade V Examination? An assessment of primary education completion examination in Bangladesh. Education Watch 2014, Dhaka: Campaign For Popular Education.

3. CAMPE (2008) State of Primary Education in Bangladesh. Progress Made, Challenges Remained. Education Watch 2008, Dhaka: Campaign For Popular Education

4. DPE (2012). Bangladesh Primary Education Annual Sector Performance Report 2012, Dhaka: Directorate of Primary Education

5. DPE (2014a). Bangladesh Primary Education Annual Sector Performance Report 2012, Dhaka: Directorate of Primary Education.

6. DPE (2014b). Primary Education Completion Examination 2014, Dhaka: Directorate of Primary Education.

7. Kingdon, G., Little, A., Aslam, M., Rawal, S., Moe, T., Patrinos, H., Beteille, T., Banerji. R., Parton, B. & Sharma S. (2014). A rigorous review of the political economy of education systems in developing countries, Institute of Education, University of London.

8. Ministry of Education (2010). National Education Policy (2010). Dhaka: Ministry of Education. The Government of the People’s Republic of Bangladesh.

9. USAID (2004). Time to Learn: Teachers’ and Students’ use of time in government primary schools in Bangladesh (2004). Basic Education Policy Support, USAID, Washington DC: June 2004

Abstract

The results from the 2015 Higher Secondary Certificate (HSC) examinations give some indication of how the new Creative Questions (CQ) and the accompanying modified Multiple Choice Questions (MCQ) are being treated across the various Boards of Intermediate and Secondary Education (BISE). This analysis is based on data on 8 HSC subjects on which tests were conducted by three BISEs. It reveals the discrepancies in test results for different subjects within and among Boards which raise questions about validity and reliability of the tests. It is recommended that, at the least, grade point (GP) and grade point average (GPA) should be calculated on the basis of standardised scores rather than raw scores. Continuing evaluation and analysis of the tests are needed; particularly, regular item analysis of questions in order to improve validity and reliability of tests.

Key Words. Secondary Education Public Examinations, Standardisation, Multiple Choice Questions, Creative Questions, Bangladesh Secondary Education.

1. BackgroundExaminations in Bangladesh at the higher secondary certificate (HSC) level are conducted by 8 examination boards (BISE). Each board covers the same range of subjects, but currently sets its own question papers. In most subjects there are two papers or parts – one described as Creative Questions (CQ) for which each question has 4 sub-parts leading to a maximum of 10 marks. There is a choice with more questions than students need to answer. Marks are awarded to the 4 sub-parts based on a hierarchical model derived from Bloom’s taxonomy of learning domains designed to encourage higher order learning of analysis, evaluation and application of knowledge, rather than only knowing and recalling facts. The CQ papers are marked by examiners who are expected to follow a marking scheme created by the question setters. The other paper is a collection of multiple choice questions (MCQ), also based on the same hierarchical model as for the CQ paper, but seeking to have a wider content coverage. The MCQ papers are marked by machine, the Optical Marking Reader (OMR).

An Analysis of 2015 HSC Examination Results forSelected Subjects of Three Examination Boards

By the BEDU team led by:Robiul Kabir Chowdhury, Jack Holbrook, Chowdhury Mufad Ahmed,

*Md. Ali Ahsan, Ahmed Obaidus Sattar Bhuiya

* In addition to the above named, the Bangladesh Examination Development Unit (BEDU) team contributing to the study included: Shamima Akhter, Lipika Rani Saha, Ranjit Kuma Sarkar, Salma Akter, Md. Khalid Hossain,, Muhammad Saiful Islam, Muhammad Aslam Khaled, Md. Shamsul Huda, and Mohammad Nasir Uddin.

54 Bangladesh Education Journal Bangladesh Education Journal 55

The marks obtained in each paper are combined (proportionately for a total of 100 for a particular subject). This is then converted to grades based on the following formula: A+ = > 80 marks; A = 70-79; A- = 60-69; B = 50-59; C = 40-49; D = 33-39; F = <33 These are recorded as grade points (GP) with A+ =5 and so on, and F=0. Each Board follows the same formula and declares for each student a combined result as a grade point average (GPA).

BEDU, the Bangladesh Examinations Development Unit, as part of its job of improving the examination system and engaging in needed research, has undertaken this analysis. The main purpose of this study is to examine the validity and reliability of the examinations, recognising that examinees’ marks for each subject are indicated in terms of a grade point (GP) based on raw scores for each paper without taking account of variability of papers for different subjects set by the various BISE and the reliability of marking by the examiners appointed by each BISE (Ministry of Education, n.d.)

2. Study QuestionsThe report is in three parts which deal with: (a) comparison of scores of the CQ and MCQ papers, (b) effect of applying a standardisation formula on scores, and (c) comparison of percentages of high GPA for selected subjects in three Boards (NFER, n. d.).This study, therefore, addresses the following research questions:

i. How suitable is the marking of question papers undertaken by examiners compared to marks obtained by a computerised optical marking record (OMR) across the various BISEs?

ii. Are the results at HSC level, by each BISE, equivalent? If not, is it possible to ensure equivalency of results from all BISEs?

iii. Can meaningful grade point and grade point averages be established to ensure an equivalent standard of examination results from all BISEs?

3. Comparison of CQ and MCQ Scores

This section compares:

• Mark distributions for a selection of HSC subject papers from three BISEs

• Mean scores per subject based on marks awarded for CQ and MCQ papers, and

• Practical examination marks.

Discussion of FindingsMarks obtained by students vary greatly with distinct troughs and peaks for CQ papers. The peaks are seen at the point of boundaries for grade point, such as a score of 80 for GP5 or letter grade A. However, marks for MCQ papers are distributed more evenly. MCQ items are machine marked; there is no examiner influence in this case. It can be reasonably argued that machine marking is more objective and, therefore, has eliminated peaks at the grade point boundaries.

Comparison of CQ and MCQ results

The difference can be clearly seen in graphical representation (Figure 1). The graph for CQ and MCQ marks distribution for Sociology papers set by Barisal BISE illustrates the point.

Frequency

The marks obtained in each paper are combined (proportionately for a total of 100 for a particular subject). This is then converted to grades based on the following formula: A+ = > 80 marks; A = 70-79; A- = 60-69; B = 50-59; C = 40-49; D = 33-39; F = <33 These are recorded as grade points (GP) with A+ =5 and so on, and F=0. Each Board follows the same formula and declares for each student a combined result as a grade point average (GPA).

BEDU, the Bangladesh Examinations Development Unit, as part of its job of improving the examination system and engaging in needed research, has undertaken this analysis. The main purpose of this study is to examine the validity and reliability of the examinations, recognising that examinees’ marks for each subject are indicated in terms of a grade point (GP) based on raw scores for each paper without taking account of variability of papers for different subjects set by the various BISE and the reliability of marking by the examiners appointed by each BISE (Ministry of Education, n.d.)

2. Study QuestionsThe report is in three parts which deal with: (a) comparison of scores of the CQ and MCQ papers, (b) effect of applying a standardisation formula on scores, and (c) comparison of percentages of high GPA for selected subjects in three Boards (NFER, n. d.).This study, therefore, addresses the following research questions:

i. How suitable is the marking of question papers undertaken by examiners compared to marks obtained by a computerised optical marking record (OMR) across the various BISEs?

ii. Are the results at HSC level, by each BISE, equivalent? If not, is it possible to ensure equivalency of results from all BISEs?

iii. Can meaningful grade point and grade point averages be established to ensure an equivalent standard of examination results from all BISEs?

3. Comparison of CQ and MCQ Scores

This section compares:

• Mark distributions for a selection of HSC subject papers from three BISEs

• Mean scores per subject based on marks awarded for CQ and MCQ papers, and

• Practical examination marks.

Discussion of FindingsMarks obtained by students vary greatly with distinct troughs and peaks for CQ papers. The peaks are seen at the point of boundaries for grade point, such as a score of 80 for GP5 or letter grade A. However, marks for MCQ papers are distributed more evenly. MCQ items are machine marked; there is no examiner influence in this case. It can be reasonably argued that machine marking is more objective and, therefore, has eliminated peaks at the grade point boundaries.

Comparison of CQ and MCQ results

The difference can be clearly seen in graphical representation (Figure 1). The graph for CQ and MCQ marks distribution for Sociology papers set by Barisal BISE illustrates the point.

Frequency

The difference in CQ and MCQ marks distribution points to undue and damaging examiner interference that appears to ignore the marking scheme. The effect is somewhat mitigated when the CQ marks and MCQ marks are combined (since there is no peaks and troughs in MCQ marks) to arrive at a total that is converted to grades.

There is a wide range of mean scores across Boards for the same subject, as well as across subjects for both CQ and MCQ papers.

The following table shows the range of CQ mean scores per subject across Boards (expressed as a percentage), plus the range of mean scores across subjects and across Boards.

The table shows that there is a wide discrepancy for most subjects for mean scores within each Board and between different Boards for the same subject. In some cases the mean percentage score is quite high indicating an easy paper (leading to poor discrimination between the very able and not so able examinees).

It also points to poor validity of the question papers in maintaining a meaningful and uniform standard of results and in setting suitable questions, based on the CQ model (as indicated by the low standard deviation of mark distribution). It also suggests poor reliability of the marking, raising marks for some candidates while penalising others.

Comparison of Mean Scores

Table 2 considers the mean percentage scores per Board per subject for the MCQ papers, expressed in percentages of maximum possible mark.

Subject Code Subject Barisal (B)

Mean % Mean % Mean % Mean Scores

Comilla (C) Sylhet (S) Range of

117 Sociology I 61.1 46.4 55.7 46.4 – 61.1

118 Sociology II 68.7 52.7 58.6 52.7 – 68.7

123 Psychology I 69.9 64.2 41.5 41.5 – 69.9

124 Psychology II 64.5 60.3 34.8 34.8 – 64.5

176 Chemistry I 57.0 51.7 60.2 51.7 – 60.2

177 Chemistry II 55.5 53.9 55.8 53.9 – 55.8

178 Biology I 54.7 52.8 64.0 52.8 – 64.0

179 Biology II 62.4 52.5 65.9 52.5 - 65.9

253 Accounting I 48.2 50.4 48.6 48.2 – 50.4

254 Accounting II 62.7 47.6 50.9 47.6 – 62.7

265 Maths I 66.9 59.7 51.5 51.7 – 66.9

266 Maths II 68.9 54.9 58.3 54.9 – 68.9

269 Civics I 60.3 49.5 59.3 49.5 – 60.3

270 Civics II 51.3 50.3 54.9 50.3 – 54.9

304 History I 58.1 49.5 55.2 49.5 – 58.1

305 History 2 63.0 53.0 53.4 53.0 – 63.0

Range 48.2 – 68.9 47.6 – 59.7 48.6 – 65.9

(subject) (253 266) (254 265) (253 179)

Table 1: Range of mean scores for CQ papers of 8 subjects in three BISEs

Subject Subject Barisal (B) Code Mean % Mean % Mean % Mean Scores

117 Sociology I 61.1 49.2 51.4 49.2 - 61.1C B

118 Sociology II 65.8 51.8 47.8 47.8 - 65.8S B

176 Chemistry I 74.9 62.9 52.9 52.9 – 74.9S B

177 Chemistry II 73.6 63.5 56.5 56.5 – 73.6S B

178 Biology I 66.2 66.0 56.7 56.7 – 66.2S B

179 Biology II 72.3 63.3 58.8 58.8 – 72.3S B

253 Accounting I 71.1 50.6 55.9 55.9 – 71.1S B

254 Accounting II 70.5 51.7 47.7 47.7 – 80.6S B

269 Civics I 58.9 58.3 59.6 58.3 59.6C S

270 Civics II 66.2 71.1 74.3 66.2 - 74.3B S

304 History I 71.2 64.5 66.4 64.5 – 71.2C B

305 History II 59.1 53.3 56.2 53.3 – 59.1C B

Range 58.9 – 74.9 49.2 – 71.1 47.7 – 74.3269 176 117 270 254 270

Table 2: Range of mean scores for MCQ papers of 8 subjects in three BISEs

The difference in CQ and MCQ marks distribution points to undue and damaging examiner interference that appears to ignore the marking scheme. The effect is somewhat mitigated when the CQ marks and MCQ marks are combined (since there is no peaks and troughs in MCQ marks) to arrive at a total that is converted to grades.

There is a wide range of mean scores across Boards for the same subject, as well as across subjects for both CQ and MCQ papers.

The following table shows the range of CQ mean scores per subject across Boards (expressed as a percentage), plus the range of mean scores across subjects and across Boards.

The table shows that there is a wide discrepancy for most subjects for mean scores within each Board and between different Boards for the same subject. In some cases the mean percentage score is quite high indicating an easy paper (leading to poor discrimination between the very able and not so able examinees).

It also points to poor validity of the question papers in maintaining a meaningful and uniform standard of results and in setting suitable questions, based on the CQ model (as indicated by the low standard deviation of mark distribution). It also suggests poor reliability of the marking, raising marks for some candidates while penalising others.

Comparison of Mean Scores

Table 2 considers the mean percentage scores per Board per subject for the MCQ papers, expressed in percentages of maximum possible mark.

Subject Code Subject Barisal (B)

Mean % Mean % Mean % Mean Scores

117 Sociology I 61.1 46.4 55.7 46.4 – 61.1

118 Sociology II 68.7 52.7 58.6 52.7 – 68.7

123 Psychology I 69.9 64.2 41.5 41.5 – 69.9

124 Psychology II 64.5 60.3 34.8 34.8 – 64.5

176 Chemistry I 57.0 51.7 60.2 51.7 – 60.2

177 Chemistry II 55.5 53.9 55.8 53.9 – 55.8

178 Biology I 54.7 52.8 64.0 52.8 – 64.0

179 Biology II 62.4 52.5 65.9 52.5 - 65.9

253 Accounting I 48.2 50.4 48.6 48.2 – 50.4

254 Accounting II 62.7 47.6 50.9 47.6 – 62.7

265 Maths I 66.9 59.7 51.5 51.7 – 66.9

266 Maths II 68.9 54.9 58.3 54.9 – 68.9

269 Civics I 60.3 49.5 59.3 49.5 – 60.3

270 Civics II 51.3 50.3 54.9 50.3 – 54.9

304 History I 58.1 49.5 55.2 49.5 – 58.1

305 History 2 63.0 53.0 53.4 53.0 – 63.0

Range 48.2 – 68.9 47.6 – 59.7 48.6 – 65.9

(subject) (253 266) (254 265) (253 179)

Table 1: Range of mean scores for CQ papers of 8 subjects in three BISEs

Subject Subject Barisal (B) Code Mean % Mean % Mean % Mean Scores

117 Sociology I 61.1 49.2 51.4 49.2 - 61.1C B

118 Sociology II 65.8 51.8 47.8 47.8 - 65.8S B

176 Chemistry I 74.9 62.9 52.9 52.9 – 74.9S B

177 Chemistry II 73.6 63.5 56.5 56.5 – 73.6S B

178 Biology I 66.2 66.0 56.7 56.7 – 66.2S B

179 Biology II 72.3 63.3 58.8 58.8 – 72.3S B

253 Accounting I 71.1 50.6 55.9 55.9 – 71.1S B

254 Accounting II 70.5 51.7 47.7 47.7 – 80.6S B

269 Civics I 58.9 58.3 59.6 58.3 59.6C S

270 Civics II 66.2 71.1 74.3 66.2 - 74.3B S

304 History I 71.2 64.5 66.4 64.5 – 71.2C B

305 History II 59.1 53.3 56.2 53.3 – 59.1C B

Range 58.9 – 74.9 49.2 – 71.1 47.7 – 74.3269 176 117 270 254 270

Table 2: Range of mean scores for MCQ papers of 8 subjects in three BISEs

Table 2 illustrates, similar to CQ papers, that there is also wide discrepancy between mean scores per Board for the same subject and between subjects. The results again point to poor validity of the test items in maintaining a meaningful standard, based on the model intended, as indicated by the low standard deviation for scores..

A comparison of means for CQ and MCQ papers may show the level of difficulty of the two types of questions. However, results show no general pattern across subjects and Boards. .

A more detailed comparison of MCQ and CQ scores means is shown in Table 3.

The MCQ mean score is often higher than the CQ score, as seen in Table 3. The MCQ mean score is excessively high for some subjects in some Boards (This is in contrast with the SSC results where the MCQ mean score is more often lower than the CQ mean score, not shown in this paper.) Especially high mean scores suggest the MCQ papers may be poorly set and not following a suitable assessment model.

It is necessary to undertake item analysis for each paper to have a better understanding of validity and reliability across subjects and across Boards. This is a task to which BEDU needs to give attention and develop its capacity for this purpose.

Practical examination Scores

Practical examinations are conducted for certain subjects as part of the public examination, where applicable. The practical examination scores all cluster around the maximum possible mark. The practical examination results, therefore, are totally unreliable as a measure of learning achievement. All the practical mark does is to boost the mean overall score for each student taking that subject, irrespective of the Board. The mean scores are shown in Table 4.

The mean scores of all practical examinations are exceptionally high, having a very low standard deviation. There is virtually no discrimination seen among examinees. It denotes that teachers are seemingly more interested in the examination results than in educating their students in meaningful ways.The practical scores boosting the overall scores give an undue advantage to the students having practical subjects, especially, the science stream students.

4. Standardization of examination results

One of the objectives of the study is to ascertain whether the results at HSC level from the different BISEs maintain a comparable standard across Boards and subjects. This study, conducted by BEDU, uses relevant data from Barisal, Comilla and Sylhet Boards.

Table 3: Comparison of the range of mean scores for CQ and MCQ papers

Subject Code SubjectRange of

CQ Mean ScoresRange of

MCQ Mean scores

Sociology I

Sociology II

Psychology I

Psychology II

Chemistry I

Chemistry II

Biology I

Biology II

Accounting I

Accounting II

Maths I

Maths II

Civics I

Civics II

History I

History 2

46.4 – 61.1 C B

52.7 – 68.7 C B

41.5 – 69.9 S B

34.8 – 64.5 S B

51.7 – 60.2 C S

53.9 – 55.8 C S

52.8 – 64.0 C S

52.5 - 65.9 C S

48.2 – 50.4 B C

47.6 – 62.7 C B

51.7 – 66.9 S B

54.9 – 68.9 C B

49.5 – 60.3 C B

50.3 – 54.9 C S

49.5 – 58.1 C B

53.0 – 63.0 C B

49.2 -61.1 C B

47.8 - 65.8 S B

52.9 – 74.9 S B

56.5 – 73.6 S B

56.7 – 66.2 S B

58.8 – 72.3 S B

55.9 – 71.1 S B

47.7 – 80.6 S B

58.3 – 59.6 C S

66.2 – 74.3 B S

64.5 – 71.2 C B

53.3 – 59.1 C B

Note: NA – No MCQ tests used.

Table 4: Mean scores for practical examinations in 8 papers for 3 BISEs.

Subject CodeBarisal Mean (%)

Examination Board

Comilla Mean (%) Sylhet Mean (%)

24.74 (99.0 %)

23.75 (95.0 %)

23.79 (95.2 %)

24.12 (96.5 %)

24.40 (97.6 %)

24.41 (97.6 %)

24.59 (98.4 %)

24.61 (98.4 %)

23.77 (95.1 %)

23.99 (96.0 %)

24.02 (96.1 %)

24.17 (96.7 %)

24.31 (97.2 %)

23.33 (93.3 %)

23.73 (94.9 %)

22.82 (91.3 %)

23.02 (92.1 %)

22.92 (91.7 %)

23.13 (92.5 %)

23.25 (93.0 %)

23.31 (93.2 %)

Table 2 illustrates, similar to CQ papers, that there is also wide discrepancy between mean scores per Board for the same subject and between subjects. The results again point to poor validity of the test items in maintaining a meaningful standard, based on the model intended, as indicated by the low standard deviation for scores..

A comparison of means for CQ and MCQ papers may show the level of difficulty of the two types of questions. However, results show no general pattern across subjects and Boards. .

A more detailed comparison of MCQ and CQ scores means is shown in Table 3.

The MCQ mean score is often higher than the CQ score, as seen in Table 3. The MCQ mean score is excessively high for some subjects in some Boards (This is in contrast with the SSC results where the MCQ mean score is more often lower than the CQ mean score, not shown in this paper.) Especially high mean scores suggest the MCQ papers may be poorly set and not following a suitable assessment model.

It is necessary to undertake item analysis for each paper to have a better understanding of validity and reliability across subjects and across Boards. This is a task to which BEDU needs to give attention and develop its capacity for this purpose.

Practical examination Scores

Practical examinations are conducted for certain subjects as part of the public examination, where applicable. The practical examination scores all cluster around the maximum possible mark. The practical examination results, therefore, are totally unreliable as a measure of learning achievement. All the practical mark does is to boost the mean overall score for each student taking that subject, irrespective of the Board. The mean scores are shown in Table 4.

The mean scores of all practical examinations are exceptionally high, having a very low standard deviation. There is virtually no discrimination seen among examinees. It denotes that teachers are seemingly more interested in the examination results than in educating their students in meaningful ways.The practical scores boosting the overall scores give an undue advantage to the students having practical subjects, especially, the science stream students.

4. Standardization of examination results

One of the objectives of the study is to ascertain whether the results at HSC level from the different BISEs maintain a comparable standard across Boards and subjects. This study, conducted by BEDU, uses relevant data from Barisal, Comilla and Sylhet Boards.

Table 3: Comparison of the range of mean scores for CQ and MCQ papers

Subject Code SubjectRange of

CQ Mean ScoresRange of

MCQ Mean scores

Sociology I

Sociology II

Psychology I

Psychology II

Chemistry I

Chemistry II

Biology I

Biology II

Accounting I

Accounting II

Maths I

Maths II

Civics I

Civics II

History I

History 2

46.4 – 61.1 C B

52.7 – 68.7 C B

41.5 – 69.9 S B

34.8 – 64.5 S B

51.7 – 60.2 C S

53.9 – 55.8 C S

52.8 – 64.0 C S

52.5 - 65.9 C S

48.2 – 50.4 B C

47.6 – 62.7 C B

51.7 – 66.9 S B

54.9 – 68.9 C B

49.5 – 60.3 C B

50.3 – 54.9 C S

49.5 – 58.1 C B

53.0 – 63.0 C B

49.2 -61.1 C B

47.8 - 65.8 S B

52.9 – 74.9 S B

56.5 – 73.6 S B

56.7 – 66.2 S B

58.8 – 72.3 S B

55.9 – 71.1 S B

47.7 – 80.6 S B

58.3 – 59.6 C S

66.2 – 74.3 B S

64.5 – 71.2 C B

53.3 – 59.1 C B

Note: NA – No MCQ tests used.

Table 4: Mean scores for practical examinations in 8 papers for 3 BISEs.

Subject CodeBarisal Mean (%)

Examination Board

Comilla Mean (%) Sylhet Mean (%)

24.74 (99.0 %)

23.75 (95.0 %)

23.79 (95.2 %)

24.12 (96.5 %)

24.40 (97.6 %)

24.41 (97.6 %)

24.59 (98.4 %)

24.61 (98.4 %)

23.77 (95.1 %)

23.99 (96.0 %)

24.02 (96.1 %)

24.17 (96.7 %)

24.31 (97.2 %)

23.33 (93.3 %)

23.73 (94.9 %)

22.82 (91.3 %)

23.02 (92.1 %)

22.92 (91.7 %)

23.13 (92.5 %)

23.25 (93.0 %)

23.31 (93.2 %)

It is noteworthy that the results at HSC level are published on the basis of the raw score overlooking the nature and pattern of the scores of the respective subjects. This leads to non-comparable results across boards, subjects and years. The average marks obtained in the subjects of the Science stream having the advantage of practical marks are higher than that of Commerce and Humanities subjects. As a consequence, fewer numbers of students from the latter streams obtain GPA 5 compared to the science students. The commerce and humanities students are thus placed at a disadvantaged position when their performance is assessed for higher education admission, job markets and other purposes (NFER, nd.)

This section focuses on publishing the results which would be comparable across boards, years and subjects. For this purpose, two steps were necessary:

• Collecting raw scores from HSC examination subjects in order to use these to create standardised scores.

• Comparing raw scores and standardized scores for selected subjects from three Boards.

The research question for this section, therefore, is: Are the results at HSC level of different BISEs of an equivalent standard? If not, is it possible to establish comparable examination results from all BISEs?

Methodology

• HSC examination results for 2015 were obtained from three BISEs.

• The results were analysed to determine the distribution of examinees’ marks in the different subject papers.

• The distribution of marks from total raw score, adding CQ and MCQ scores, and the distribution of marks from standardized scores were derived.

• The distribution of marks based on raw scores and standardised scores were checked against a normal distribution curve, which is predicted to represent the expected mark distribution for the large numbers taking each examination.

• The resulting graphs were analysed in order to derive answers to the research question posed.

Discussion of Findings -- Comparison of raw score and standardized score

Graphs of raw score and standardized score of Barisal in Chemistry (Figure 2.1) show an almost normal distribution in contrast to what was seen above (Figure 1) for Sociology in the same Board. This suggests a difference in scores and marking pattern for science and other subjects. There may also be differences in standard and criteria in constructing questions.

60 80 100

Histogram

Mean = 72.7Std.Dev. = 9.68N = 5,344

60 80 100

Tscore 176

Histogram

Mean = 65.96Std.Dev. = 10.04N = 5,344

ndFigure 2.1: Barisal Board HSC 2015 Chemistry Code 177 2 paper.

Note the graphs for the standardized scores are similar in appearance to those from the unstandardized (raw) scores. This is not so for all subjects and all Boards, which suggests problems in the quality of setting questions and marking standard.

The percentages of GP A+ across boards are different for different subjects and for different Boards as determined from the raw scores (not shown). On the other hand, the percentage of GP A+ across subjects and Boards are likely to be similar for standardized score (approximately about 10%). (Figure -2.2 for Chemistry paper, Barisal)

Figure 2.2: A+ mark distribution based on raw scores and standardised scores

gp 176A+ A- B C DA

gp 176

gp 176A+ A- B C DA F

Tgp 176

gp 176

0.299 0.075

A+ (26.38%)Chemistry Paper 1, Barisal Raw Scores A+ (10.36%) Chemistry Paper 1, BarisalStandardised Scores

It is noteworthy that the results at HSC level are published on the basis of the raw score overlooking the nature and pattern of the scores of the respective subjects. This leads to non-comparable results across boards, subjects and years. The average marks obtained in the subjects of the Science stream having the advantage of practical marks are higher than that of Commerce and Humanities subjects. As a consequence, fewer numbers of students from the latter streams obtain GPA 5 compared to the science students. The commerce and humanities students are thus placed at a disadvantaged position when their performance is assessed for higher education admission, job markets and other purposes (NFER, nd.)

This section focuses on publishing the results which would be comparable across boards, years and subjects. For this purpose, two steps were necessary:

• Collecting raw scores from HSC examination subjects in order to use these to create standardised scores.

• Comparing raw scores and standardized scores for selected subjects from three Boards.

The research question for this section, therefore, is: Are the results at HSC level of different BISEs of an equivalent standard? If not, is it possible to establish comparable examination results from all BISEs?

Methodology

• HSC examination results for 2015 were obtained from three BISEs.

• The results were analysed to determine the distribution of examinees’ marks in the different subject papers.

• The distribution of marks from total raw score, adding CQ and MCQ scores, and the distribution of marks from standardized scores were derived.

• The distribution of marks based on raw scores and standardised scores were checked against a normal distribution curve, which is predicted to represent the expected mark distribution for the large numbers taking each examination.

• The resulting graphs were analysed in order to derive answers to the research question posed.

Discussion of Findings -- Comparison of raw score and standardized score

Graphs of raw score and standardized score of Barisal in Chemistry (Figure 2.1) show an almost normal distribution in contrast to what was seen above (Figure 1) for Sociology in the same Board. This suggests a difference in scores and marking pattern for science and other subjects. There may also be differences in standard and criteria in constructing questions.

60 80 100

Histogram

Mean = 72.7Std.Dev. = 9.68N = 5,344

60 80 100

Tscore 176

Histogram

Mean = 65.96Std.Dev. = 10.04N = 5,344

ndFigure 2.1: Barisal Board HSC 2015 Chemistry Code 177 2 paper.

Note the graphs for the standardized scores are similar in appearance to those from the unstandardized (raw) scores. This is not so for all subjects and all Boards, which suggests problems in the quality of setting questions and marking standard.

The percentages of GP A+ across boards are different for different subjects and for different Boards as determined from the raw scores (not shown). On the other hand, the percentage of GP A+ across subjects and Boards are likely to be similar for standardized score (approximately about 10%). (Figure -2.2 for Chemistry paper, Barisal)

Figure 2.2: A+ mark distribution based on raw scores and standardised scores

gp 176A+ A- B C DA

gp 176

gp 176A+ A- B C DA F

Tgp 176

gp 176

0.299 0.075

A+ (26.38%)Chemistry Paper 1, Barisal Raw Scores A+ (10.36%) Chemistry Paper 1, BarisalStandardised Scores

In some cases, the total raw score graphs show peaks rather than a smooth distribution of marks. The peaks occur from marking irregularities by examiners, which are not fully eliminated when marks from the CQ and MCQ papers are combined.

When both the graphs of raw and standardized scores do not follow a normal distribution, this indicates weaknesses in setting the question paper and perhaps marking the papers by examiners.

Figure 2.3 showing practical examination results peaking at the point of maximum score indicates faulty and rather useless testing and grading.

5. Making Grade Point Averages (GPA) equivalent across BoardsThis section explores whether grade points and grade point averages for different Boards can be equivalent.

Methodology

• HSC examination results for 2015 were obtained from different BISEs.

• The results were analysed to determine the distribution of GP for student scores in different subject papers. Graphs (histograms) were plotted for the percentages of GP (Grade Point) students attained in each examination paper for total raw score and for standardized score.

Figure 2.3: Chemistry, Code 176, Practical Test Scores, Barisal Board

p_17615 20 25

Histogram

Mean = 23.74Std. Dev. = 1.752N = 5,344

• The distribution of GP based on raw scores and standardized score are compared.

• The distribution of GPs were checked against a normal distribution, predicted to represent the expected GP distribution for the large student numbers taking each examination.

Findings

Table 5 shows the combined raw scores for CQ and MCQ papers per subject across boards, plus the range of mean scores by subject and by Board.

Table 5: Combined CQ and MCQ raw score means and their range

SubjectCode

Examination Board

Barisal (B)Mean %

Comilla (C)Mean %

Sylhet (S)Mean %

Range ofMean Scores

Range of mean scores

across subjects

46.9 – 67.6

(117 177)

56.3 – 73.5

(253 179)

49.3 – 69.1 (254 179)

46.9 – 59.8

51.4 – 66.1

65.3 – 72.7

65.0 - 71.6

67.0 – 68.4

66.6 – 73.5

49.4 – 56.3

48.1 – 64.8

53.1 – 59.7

57.3 – 63.0

55.5- 63.4

53.1- 61.5

In some cases, the total raw score graphs show peaks rather than a smooth distribution of marks. The peaks occur from marking irregularities by examiners, which are not fully eliminated when marks from the CQ and MCQ papers are combined.

When both the graphs of raw and standardized scores do not follow a normal distribution, this indicates weaknesses in setting the question paper and perhaps marking the papers by examiners.

Figure 2.3 showing practical examination results peaking at the point of maximum score indicates faulty and rather useless testing and grading.

5. Making Grade Point Averages (GPA) equivalent across BoardsThis section explores whether grade points and grade point averages for different Boards can be equivalent.

Methodology

• HSC examination results for 2015 were obtained from different BISEs.

• The results were analysed to determine the distribution of GP for student scores in different subject papers. Graphs (histograms) were plotted for the percentages of GP (Grade Point) students attained in each examination paper for total raw score and for standardized score.

Figure 2.3: Chemistry, Code 176, Practical Test Scores, Barisal Board

p_17615 20 25

Histogram

Mean = 23.74Std. Dev. = 1.752N = 5,344

• The distribution of GP based on raw scores and standardized score are compared.

• The distribution of GPs were checked against a normal distribution, predicted to represent the expected GP distribution for the large student numbers taking each examination.

Findings

Table 5 shows the combined raw scores for CQ and MCQ papers per subject across boards, plus the range of mean scores by subject and by Board.

Table 5: Combined CQ and MCQ raw score means and their range

SubjectCode

Examination Board

Barisal (B)Mean %

Comilla (C)Mean %

Sylhet (S)Mean %

Range ofMean Scores

Range of mean scores

across subjects

46.9 – 67.6

(117 177)

56.3 – 73.5

(253 179)

49.3 – 69.1 (254 179)

46.9 – 59.8

51.4 – 66.1

65.3 – 72.7

65.0 - 71.6

67.0 – 68.4

66.6 – 73.5

49.4 – 56.3

48.1 – 64.8

53.1 – 59.7

57.3 – 63.0

55.5- 63.4

53.1- 61.5

Table 5 shows:

1. There is a wide discrepancy between mean scores based on raw student marks for different Boards for the same subject in many cases.

2. There is a wide discrepancy between mean scores based on raw marks per subject for the same board in many cases.

3. The lower mean scores were for humanities (e.g., Sociology, 1st paper). while science mean scores were higher (e.g., Biology 2nd paper); the mean of all science subjects are higher than for other streams.

4. Mean scores for the Comilla Board were lower on average, while mean scores for Barisal Board were higher, showing another variability of mean student marls across Boards.

The results point to poor validity of the question papers in terms of maintaining a meaningful and uniform standard in setting suitable questions.

Table 6 shows the standardized scores per subject across Boards, plus the range of mean scores by subjects and by Boards.Table 6 illustrates the effect of standardization of scores. The results based on standardised marks point out that through standardization the variation of mean scores across the subjects and across the Boards can be minimised.

The findings presented in Table 7 show the percentage of A+ distribution for raw GP and standardized GP per subject across Boards. (Also see Figure 2.2 above.)

Analysis of Table 7 data indicates that on applying standardization, the percentage of A+ is similar and comes close to 10% for each subject across each Board. It also shows:

1. For raw scores, there is a very high discrepancy in the percentage of A+ within the same subject across Boards.

2. For raw scores, there is a very high discrepancy in the percentage of A+ within the same Board across subjects.

3. The percentage of A+ tends to be lower for humanities subjects but much higher for science subjects.

Discussion - Comparison of raw score and standardized score distribution

The results highlight a lack of common standards among the Examination Boards arising from setting questions and marking papers at different standards by each Board. Applying standardisation to raw scores and calculating and publishing results based on standardization of scores can contribute to establishing common standards in results across academic streams and Boards. It is important to note that the benchmark of 10% for A+ was an assumption made by BEDU and was not derived from any psychometric analysis of measurement. It was chosen to represent a meaningful standard, which could be used to equate grade points across Boards, based on a normal distribution assumption. In practice, however, exactly 10% was not

Table 6: Mean standardised scores for subjects across Boards and their range.

SubjectCode

Examination Board

BarisalMean %

ComillaMean %

SylhetMean %

Range ofMean Scores

63.8 – 64.0

62.9 – 67.0

64.0 – 66.0

62.5 – 66.0

64.0 – 65.0

63.5 – 64.0

58.0 – 61.5

59.0 –60.0

63.0 – 65.8

Range of mean scores across

subjects

50.0 – 68.9

59.0 – 67.3

56.0 – 68.0

61.9 – 63.0

58.0 – 59.7

62.1 – 65.0

always possible to achieve. As the 10% benchmark was arbitrary, the selected A+ percentage could be modified to relate to other grade point average cut-off still approximating a normal distribution. The A+ % thus represents a selected standard, but once chosen it needs to be applied across Boards and across examination years without changing it frequently. Thus, comparison could be made per subject, across different cohorts of examinees and over time. One important criterion of judging the quality of the examination would be approximation of results to a normal distribution.