AI Structured Summary
Not yet generated for this judgment
Judgment
JUDGMENTTAG-JUDGMENT
Sanjeev Sachdeva, J
Petitioners, who are all in-service candidates and had appeared in the NEET – Post Graduate 2024 (‘NEET-PG’ for short) examination, seek a direction to the Respondents to redraw the State Merit list for the in-service candidates.
The dispute pertains to the State Merit List for the State of Madhya Pradesh. As per the Petitioners who are in-service candidates of the State of Madhya Pradesh (hereinafter referred to as ‘in-service candidates’), their merit in the All India List is higher than some of the other in-service candidates but in the State List they have been shown lower in the Merit.
NEET-PG examination was initially scheduled to be held on 23.06.2024 in a single shift, however the same was postponed by the Ministry of Health and Family Welfare on 22.06.2024. To ensure additional measures of security and to maintain sanctity of the examination, it was decided to conduct the examination at limited test centres. Accordingly, it was notified on 05.07.2024 that the NEET-PG examination would be held on 11.08.2024 in two shifts.
National Board of Examinations in Medical Sciences (NBEMS for short) which conducted the test, in its Reply has contended that the qualifying criteria for NEET-PG as prescribed in the Post Graduate Medical Education Regulations notified by the National Medical Commission is based on percentile. For General/EWS the eligibility criteria is 50th percentile; for UR PwD, it is 45th percentile and for SC/ST/OBC (including PwD of SC/ST/OBC) it is 40th percentile.
In view of the above eligibility criteria, NBEMS notified on 09.08.2024 that the percentile based normalisation process, which was being used by All India Institute of Medical Sciences, New Delhi for its various examinations conducted in more than one shift, would be adopted.
As per the NBEMS, to eliminate bias in distribution of candidates, equal numbers of candidates were randomly allotted different groups (shifts). Further, with a large population of examinees spread over the entire country, possibility of bias becomes remote. The overall merit/ranking is based on the percentile score of the ‘Raw score’ obtained by the candidates in their respective shift and tie-breaking criteria as per the Information Bulletin for NEET-PG – 2024. Percentiles upto 7 decimal places for the raw score are calculated and the percentile score for the raw scores for both the shifts are merged and arranged so as to derive an overall merit list/ranking. The percentile score is the relative performance of all those who have appeared for the examination.
As per the reply filed by NBEMS, All India NEET – PG 2024 rank was based on percentile obtained as per the notified normalisation process. It is contended that several States/Union Territories in India have a provision to award additional incentivised marks in NEET – PG to their in-service candidates who have served in rural/hard posting. The incentivised marks in percentage of the raw scores obtained differ from State to State ranging from 0.263% to 35%.
In the State of Madhya Pradesh, in-service candidates who have served in rural/hard posting are entitled to 10%, 20% and 30% incentivised marks in percentage of their raw scores based on the period of completed service in rural/hard area. Candidates who have completed one year are entitled to 10%, and those who have completed two years, 20% and those more than three years, 30% incentivised marks of their raw scores.
Both, Dr. Subodh Kumar, Executive Director NBEMS and Dr. N.I. Iboyaima Mangang, Additional Director (Medical) NBEMS explained the methodology adopted for preparation of the All India ranking and the State ranking. They also submitted a State list and a note of their submissions. As per them, both the list and the note of submissions contain certain data which is confidential and not in public domain. Consequently, we are not mentioning the names of candidates or their raw scores in the order. The State List and note of submissions, after reference, are being kept in a sealed cover, with a direction to the Registry not to open the sealed cover except with the leave of the Court.
Further, they submitted that as the examination was held in two different shifts, the standard practise is to adopt a normalisation process so as to equalise the scores of the two shifts. They stated that the overall merit/ranking is based on percentile of raw score obtained by the candidates in their respective shifts and ‘tie breaking’ criteria as mentioned in the information bulletin for NEET-PG 2024. Percentile (up to 7 decimal places) for the raw score is to be calculated. The percentile (not percentage) for the raw score for both the shifts is merged and arranged so as to derive an overall Merit/Ranking. The percentile ranges from 100 to 0. The percentile indicates the percentage of candidates that have scored equal to or below (same or lower raw scores) that particular percentile in a shift of that examination. The topper (highest scorer) of each group gets the same percentile of 100. The marks scored between the highest and lowest scores are converted to appropriate percentile. The formula used to calculate percentile is:
Let ‘T’ denote the raw marks obtained by a candidate:
No. of candidates from group with
Total Percentile (TP) = 100 × total marks ≤ T
No. of candidates in the group/shift
It is further clarified that in Shift 1, a total of 107959 candidates appeared. The percentile of the candidate who scored the highest raw score in Shift 1 was taken as 100. The percentile of the candidate who scored the second highest raw score in Shift 1 was calculated as 100 × 107958 ÷ 107959 = 99.9990737. The percentile of the candidate who scored the third highest raw score was calculated as 100 × 107957 ÷ 107959 = 99.9981474 and so on.
Similarly, in Shift 2, a total of 108134 candidates appeared. The percentile of the candidate who scored the highest raw score in Shift 2 was taken as 100. The percentile of the candidate who scored the second highest raw score in shift 2 was calculated as 100 × 108133 ÷ 108134 = 99.9990752. The percentile of the candidate who scored the third highest raw score in shift 2 was calculated as 100 × 108132 ÷ 108134 = 99.9981504 and so on.
Said exercise was carried out for normalisation of the scores and then the percentiles were merged and the All India ranking obtained. For the toppers and the ‘tie breaker’ percentiles, the ‘tie-breaking’ criteria as per the Information Bulletin was applied. After merger, the result looked as under:
Roll No.
Candidate
NEET-PG
Percentile
NEET-PG 2024
Published rank
Shift
********
A
100
1
Shift 1
********
B
100
2
Shift 2
********
C
99.9990752
3
Shift 2
********
D
99.9990737
4
Shift 1
********
E
99.9981504
5
Shift 2
********
F
99.9981474
6
Shift 1
********
As per Dr. Subodh Kumar, and Dr. N.I. Iboyaima Mangang, since the State Counselling Authorities do not have all the information like raw scores, shifts of candidates, tie breaking data etc. NBEMS was entrusted to provide the State specific merit list based on percentage of incentive marks to be awarded to the candidates and data of eligible candidates provided by the concerned States/UT.
State specific percentile and state specific rank including inter-se merit for a particular state has been freshly prepared by NBEMS after awarding incentivised/bonus marks obtained by the concerned candidates. It is contended that bonus marks an only be added to the raw score and not to percentile. As per the note submitted,following methodology is stated to have been used:
“1. In the master result data of NEET-PG 2024 containing data of 216093 candidates, all eligible candidates of a state were identified and incentivised marks as per the criteria given by the State was awarded to all such eligible candidates for the respective State, provided by the respective State counselling authority.
Total score in NEET-PG 2024 after awarding incentivised marks to the applicable candidate were taken as their final total score without disturbing the raw scores of other candidates.
Fresh percentile was generated for both the shifts including all the 216093 candidates who appeared in NEET-PG 2024 and were merged and arranged in decreasing order of percentile calculated on the basis of applicable incentivised score. This was done on an all India basis to have a large denominator to prevent sample and selection bias and to maintain uniformity across all states.
The state specific list was prepared by extracting the data of all the registered candidates as per the list provided by the respective State Counselling authorities, from the list prepared in step 3 as above.
NEET-PG 2024 State Specific Rank was generated using the tie breaking criteria as mentioned in the information bulletin.
It may be noted that normalisation was done only once.”
We may at this juncture note that the methodology that was adopted for drawing up the State Specific Rank as submitted by NBEMS above, was never notified.What had been notified was only the methodology for drawing up the All India Merit List.
The Note further states that
“1. It is pertinent to note that unlike the percentage score that reflects how well the candidate did in the exam itself. The percentile reflects how well he did in comparison to other candidates. Therefore, any increase in raw scores of a candidate in a particular shift be awarding the incentivisedmarks, which are REAL or ACTUAL marks considered for the purpose of result generation and preparation of State specific merit list, would affect the percentile of not only the candidates getting incentivised marks but the other candidates of the same shift and hence the overall merit position as percentile are the relative performance of all those who appear for the examination……
Hence, the new State specific percentile and rank of candidates, including their state inter-se merit prepared after the award of additional state specific incentivised marks, cannot be compared with All India NEET-PG 2024 percentile score and rank, including their All India inter-se merit which was prepared without award of any state specific incentivised marks.
Both All India merit list and State Specific merit lists for NEET-PG 2024 have been prepared by NBEMS and further vetted by result team experts of Alms, New Delhi including the process.
Hypothetically, if the benefit of awarding incentivised marks in percentage of marks obtained by the candidates, are provided to the percentile score in NEET-PG 2024, the following situation would emerge:
a. All the TOP 20 State merit list will be filled by the candidates who have been provided incentivized marks.
b. All these toppers would have percentile score more than 100, which is not possible/feasible
c. The first non-incentivized candidate will appear at State rank of 23 instead of the current State Rank no 2.
d. Most of the toppers will eventually be filled by candidates who have been provided incentivized marks.”
What NBEMS has done is that they have prepared the All India Merit List adopting the normalisation process and a comparative merit list is prepared by applying the percentile method. For preparation of the State List, they have gone back to the raw scores pre normalisation and added the incentivised marks to the raw scores and then applied the normalisation process.
The Hypothesis mentioned in their note is fallacious. The hypothesis begins with a presumption that 30% or 20% or 10% addition is being sought by the petitioners to be done on the percentile, which is not the case. Further, NBEMS has granted incentivised marks on the raw scores prior to normalisation, which is where the error has crept in. Candidates have to be granted incentivised marks on their normalised scores to create a level playing field.
Normalisation is a statistical process that equalises scores across multiple examination shifts. It is used when the examinations are conducted in multiple shifts for the same syllabus. The difficulty level can vary between sessions. Normalisation ensures that all candidates have an equal opportunity by adjusting their raw scores across shifts.
If one were to take a very basic example. Suppose the degree of difficulty between two shifts is 10%. Shift 1 being easier by 10% than Shift 2. Then to normalise the result, the raw scores of Shift 1 would be adjusted by 10% to bring them at par with scores of Shift 2. One such normalisation method is percentile system as has been adopted by the NBEMS. If the topper of Shift 1 obtained say 550 marks and topper of Shift 2 obtains 500 marks then with degree of difficulty being 10% both marks would be respectively taken as 100 percentile. If both the candidates were to get the incentive of 30%, then the raw score of topper of Shift 1 become 715 (550 + 165 i.e. 30% of 550) and the raw score of topper of Shift 2 becomes 650(500 + 150 i.e. 30% of 500). Candidate in Shift 1 gets a benefit of 165 marks whereas the candidate in Shift 2 gets a benefit of 150 marks. In fact the candidate of Shift one gets a benefit of not 30% but 33% i.e. 10% more than the candidate of Shift 2. In shift 1 the benefit is compounded to 33% instead of 30%.
If one were to make a similar calculation in the present case. By taking the highest raw score of Shift 2 and then dividing it by the highest score of Shift 1 and multiplying it by 100. The degree of difficulty comes to about 98.3263598% i.e. more by approximately 1.6736402%(100 – 98.3263598) between Shift 1 and Shift 2.
If the incentivised marks were awarded after normalisation of raw scores, then the relative advantage to candidates would be equal i.e. 30%, 20% or 10% as the case may be. But if the incentivised marks were awarded before normalisation of raw scores, then the relative advantage to candidates would not be equal but would be compounded by the difference in the degree of difficulty of the two shifts as has happened in this case. By taking into account the difference of the raw scores of the toppers, the degree of difficulty is approximately 1.6736402% between Shift 1 and Shift 2, the real advantage is approximately 31.6735402% and not 30%.
Since, NBEMS has added 30% incentive to the raw scores pre normalisation, it has led to a compounded benefit to candidates of one of the Shifts of approximately 1.6736401% i.e. an incentive of approximately 31.6736401% instead of 30% and similarly 21.6736401% and 10.6736401%respectively. In an examination where the result is calculated upto 7 decimals, this makes a world of a difference.
To demonstrate the anomaly caused by adopting this grant of incentive marks on raw scores and not on normalised scores while preparing the State List, following examples may be seen. Candidates who are higher in comparative merit in their All India ranking have become lower in merit to the same individual in the State Merit List. The list is only illustrative. There are several more such instances that have not been extracted for the sake of brevity.
Candidate
Shift
All India Rank
% of incentive mark
State Specific Rank
A1
Shift 01
1196
Nil
29
A2
Shift 02
1174
Nil
30
B1
Shift 01
4852
Nil
150
B2
Shift 02
4694
Nil
151
C1
Shift 01
44693
30%
195
C2
Shift 02
42959
30%
216
D1
Shift 01
47806
30%
270
D2
Shift 02
45970
30%
292
E1
Shift 01
49826
30%
309
E2
Shift 02
47210
30%
322
E2.1
Shift 02
48007
30%
340
F1
Shift 01
36691
20%
327
F2
Shift 02
35742
20%
349
G1
Shift 01
55059
30%
441
G1.1
Shift 01
55489
30%
458
G1.2
Shift 01
56410
30%
479
G1.3
Shift 01
56999
30%
503
G2
Shift 02
53373
30%
516
H1
Shift 01
54059
20%
821
H2
Shift 02
52123
20%
858
I1
Shift 01
55967
20%
908
I2
Shift 02
53886
20%
931
I2.1
Shift 02
54380
20%
946
J1
Shift 01
62720
20%
1123
J2
Shift 02
59915
20%
1131
K1
Shift 01
79948
30%
1261
K1.1
Shift 01
80429
30%
1276
K2
Shift 02
77562
30%
1316
K2.1
Shift 02
77975
30%
1331
L1
Shift 01
84015
30%
1390
L1.1
Shift 01
84051
30%
1391
L2
Shift 02
79945
30%
1398
L2.1
Shift 02
81046
30%
1434
M1
Shift 01
85219
30%
1439
M2
Shift 02
81879
30%
1470
M2.1
Shift 02
82240
30%
1484
NBEMS in its note has given an explanation regarding candidate A1 and A2 as under:
“EXPLANATION REGARDING STATE SPECIFIC RANKS OF A1 (29) &A2 (30)
A1 appeared in Shift 1 of NEET-PG 2024 and scored **** out of 800 with All India Merit of 1196 and percentile of 99.4627590.
A2 appeared in Shift 2 of NEET-PG 2024 and scored **** out of 800 with All India Merit of 1174 and percentile of 99.4645532.
As per the Data provided by the MP State Counselling Authority, 5 candidates who appeared in the same shift with A2 i.e. Shift 2 scored above his position after awarding incentivised marks, who earlier scored below him in All India NEET-PG 2024, which affected his State specific percentile which is based on the percentile obtained by the candidate in his or her shift and then later merged together to create a common merit position for the State of Madhya Pradesh.
Similarly, as per the Data provided by the MP State Counselling Authority, only 1 candidate scored above the position of A1 in Shift 1 after awarding incentivised marks, who earlier scored below her in All India NEET-PG 2024.
Hence, the inter-se merit position of A1 (29) &A2(30) in the State Merit list for MPcannot be compared with that of the All India Merit List.”
With regard to candidate C1 and C2 the explanation given is as under:
“EXPLANATION REGARDING STATE SPECIFIC RANKS OF C1 (195) &C2 (216)
C1 appeared in Shift 1 of NEET-PG 2024 and scored **** out of 800 with All India Merit of 44639 and percentile of 79.3819876.
C2 appeared in Shift 2 of NEET-PG 2024 and scored **** out of 800 with All India Merit of 42959 and percentile of 80.1329832.
Despite C1 scoring more in raw score in Shift 1 in the exam than C2 in Shift 2, C1 had a lower All India merit list due to the normalization process explained above.
Both C1 and C2, as per MP State Policy, were to be provided with additional incentivized of 30% to the marks obtained by them.
After addition of the incentivized marks, C1 now got final score of **** and C2 got a final score of ****. Since the actual raw score of C1 was more than that of C2, C1 got more benefit in actual marks, with the same 30% incentivised marks, in comparison to that of C2.
Therefore, awarding the same 30% increase in marks obtained do not necessary translate to the same increase in marks due to the difference in their actual raw scores.
Further, in NEET-PG 2024 due to the large volume of candidates appearing in the exam, there is clubbing of hundreds of candidates with the same score in a shift having the same percentile scores. The difference of 1680 ranks in All India merit list between C2 and C1 with a mere difference of 0.7509955 in percentile score should be seen in this light.”
The explanations highlight the fallacy in the method adopted by the NBEMS. As per NBEMS, overall merit position as percentile is the relative performance of all those who appeared for the examination. So,as per the All India Merit List A1 and C1 in their respective relative performance to A2 and C2 have performed better. However, in the State Merit list A2 and C2 are shown to have performed better. This is clearly not reconcilable.
Neither A1 nor A2 got benefit of the incentivised marks and yet their ranking has changed between the two lists. In the case of C1 and C2, the contention is that awarding the same 30% increase in marks obtained do not necessary translate to the same increase in marks due to the difference in their actual raw scores. This explanation clearly demonstrates the error. If addition on incentivised marks is done to raw scores before normalisation the incentive would not be 30% for both candidates but compounded by the degree of difficulty for one shift than the other shift. If the incentive had been given after normalisation, then both the candidates would have got benefit of 30% and there would have been no shift of the rank between similarly situated candidates.
Since, as per the reply filed by NBEMS, rankings are based on relative performance. It is not discernible as to how and why relative performance can change between two candidates by merely placing them in different lists.
The methodology of percentile as contended by the NBEMS does not reflect the substantive merit of a candidate but a comparative merit of the candidate. In this view, it completely defies logic as to how a candidate who has scored higher in comparison to another candidate in the All India rank list has scored lower in the State List in comparison to the same candidate. The comparison of scores has to be in relation to the examination and not a list. It is unfathomable that a candidate in the same examination is higher in one list and lower in another list vis a vis the same candidate.If the ranking in the All India List merit list is based on relative performance, then the relative performance of two candidates cannot change when they are separated and placed in the State List.
In so far as the normalisation of the result by using the formula that has been adopted by AIIMS, New Delhi is concerned, there is no challenge to the same. Petitioners have not impugned either the methodology adopted for declaring the All India NEET – PG 2024 result or the result as such. To the All India result there is no challenge. Evenotherwise such a challenge could not be raised as the methodology was clearly notified to all the candidates prior to the examination.The dispute pertains only to the State Merit List for the State of Madhya Pradesh. We may note that there was no notification of the method that has been adopted for the preparation of the State Merit List.
Reliance placed by learned Senior Counsel for the NBEMS on the judgment of the Supreme Court in Sunil Kumar & others versus Bihar Public service Commission (2016) 2 SCC 495 to contend that only experts bodies can decide appropriate methodology on the basis of facts specific to particular examination
does not further the case of the Respondents in as much as there is no challenge to the adoption of the percentile method for normalisation and preparation of the All India Merit list. The challenge is only to the preparation for the State Merit List, which as we have noticed hereinabove is clearly erroneous.
In view of the above, the State Merit List for the NEET-PG 2024 examination for the State of Madhya Pradesh cannot be sustained and is accordingly quashed. National Board of Examinations in Medical Sciences is directed to prepare the State Merit List afresh by awarding the incentivised marks to the in-service candidates, not on their raw scores but on their normalised scores. The exercise be carried out as expeditiously as possible. The Writ Petition is allowed in the above terms. There shall be no orders as to costs.