Ofqual head Glenys Stacey is looking at what went wrong with some exam papers this summer The exams watchdog claims that a lapse in due process was "an issue in more than one of the errors" in this year's public examinations.Ofqual is investigating mistakes by the five exam boards in England, Wales and Northern Ireland which led to 10 errors in exam papers.
Ofqual mandates that a scrutineer must work through each question paper first.
Referring to initial findings, Ofqual said: "It does look on the face of it as if someone hasn't done their job."
An exam paper is first written by a principal examiner. It is then revised and rewritten with any changes implemented.
The paper is then discussed by an examining team in detail, for one to two days, before being typeset and made available to the team for proofreading and checking.
A scrutineer is then meant to sit the paper, as if he or she was a candidate.
These appear to be rigorous procedures.
Failure of processHowever, the nature and number of mistakes this summer has prompted many to ask whether there has been a fundamental break-down in the examinations system.
Continue reading the main storyWe do expect to find examples here of where people simply haven't done their jobs properly, because it's an issue in more than one of the errors that we're talking about”End Quote Glenys Stacey Chief executive, Ofqual The mistakes range from incorrectly labelled diagrams in geography to impossible to answer maths and physics questions, and a biology multiple choice question which contained only the wrong answers.
Exam board employees are contractually not allowed to give interviews to the media.
This stems from the necessity of not revealing the contents of exam papers, but in practice it stops examiners discussing any element of the workings of their organisations.
However, one principal examiner who has worked for two of England's three boards (OCR, AQA and Edexcel) for 20 years agreed to speak to the BBC anonymously:
"To give the example of the biology paper, the multiple choice question with the correct answer missing: I can see how the process could let that happen up to a certain point, because the focus is going to be on the so-called distracters, the wrong answers in the multiple choice, because these must be potentially feasible."
"But it should never have got past the scrutiny process," she added.
"If someone does the paper, then the fact that the right answer isn't there will immediately be apparent. And the only thing I can think happened there is that the scrutiny process wasn't done properly."The BBC has spoken to a number of principal and chief examiners who say that scrutineers are not always adhering to the rules.
One Edexcel employee said "I know of those who have done a close proof read and that's it".
Mark Davies, director of studies at St Albans School and a former principal examiner, says this is also the case for the OCR awarding organisation.
Asked specifically whether a scrutineer was meant to sit the paper as a candidate would, he replied: "No, they look through the paper and they review it".
'Quality control'Ofqual has confirmed to the BBC that it will look at what the examination boards expect of scrutineers in its ongoing inquiry.
Chief executive Glenys Stacey insisted that there is a clear requirement that they "work the paper", or carry out a dry run under controlled conditions to ensure that all questions are possible within the given time.
"Preliminary evidence would suggest that that hasn't worked, and it hasn't worked well in every case and we do want to have a look at that very closely indeed," she said.
The measurement mistake in this A-level physics exam is just one of many in this year's test papers "We do expect to find examples here of where people simply haven't done their jobs properly because it's an issue in more than one of the errors that we're talking about."
OCR spokesperson Rebecca Birkett-Smith told the BBC that OCR "follows Ofqual's Code of Practice on the preparing of question papers".
Speaking to BBC Radio 4's The Report, Ms Stacey revealed that other weaknesses in the exam-setting process have also emerged, including the making of late changes to a paper before it is sent out to schools and colleges.
"Papers are honed up over a period of time. But if there are changes to papers we want to make sure that quality controls checks happen and that things come out right.
"A second area that we've identified is the arrangement for the printing of papers because we've seen a number of shortcomings there."
Ofqual hopes to publish interim findings in October, and a final report by December.
You can hear the full investigation on the The Report on Thursday 14 July at 2000 BST on BBC Radio 4. Listen again via the BBC iPlayer or download the podcast.
No comments:
Post a Comment