'Slips to blame' for exam errors

14 July 2011 Last updated at 00:28 GMT By Hannah Barnes Radio 4's The Report Ofqual head Glenys Stacey is looking at what went wrong with some exam papers this summer The exams watchdog claims that a lapse in due process was "an issue in more than one of the errors" in this year's public examinations.

Ofqual is investigating mistakes by the five exam boards in England, Wales and Northern Ireland which led to 10 errors in exam papers.

Ofqual mandates that a scrutineer must work through each question paper first.

Referring to initial findings, Ofqual said: "It does look on the face of it as if someone hasn't done their job."

An exam paper is first written by a principal examiner. It is then revised and rewritten with any changes implemented.

The paper is then discussed by an examining team in detail, for one to two days, before being typeset and made available to the team for proofreading and checking.

A scrutineer is then meant to sit the paper, as if he or she was a candidate.

These appear to be rigorous procedures.

Failure of process

However, the nature and number of mistakes this summer has prompted many to ask whether there has been a fundamental break-down in the examinations system.

Continue reading the main story

We do expect to find examples here of where people simply haven't done their jobs properly, because it's an issue in more than one of the errors that we're talking about”

End Quote Glenys Stacey Chief executive, Ofqual The mistakes range from incorrectly labelled diagrams in geography to impossible to answer maths and physics questions, and a biology multiple choice question which contained only the wrong answers.

Exam board employees are contractually not allowed to give interviews to the media.

This stems from the necessity of not revealing the contents of exam papers, but in practice it stops examiners discussing any element of the workings of their organisations.

However, one principal examiner who has worked for two of England's three boards (OCR, AQA and Edexcel) for 20 years agreed to speak to the BBC anonymously:

"To give the example of the biology paper, the multiple choice question with the correct answer missing: I can see how the process could let that happen up to a certain point, because the focus is going to be on the so-called distracters, the wrong answers in the multiple choice, because these must be potentially feasible."

"But it should never have got past the scrutiny process," she added.

"If someone does the paper, then the fact that the right answer isn't there will immediately be apparent. And the only thing I can think happened there is that the scrutiny process wasn't done properly."

The BBC has spoken to a number of principal and chief examiners who say that scrutineers are not always adhering to the rules.

One Edexcel employee said "I know of those who have done a close proof read and that's it".

Mark Davies, director of studies at St Albans School and a former principal examiner, says this is also the case for the OCR awarding organisation.

Asked specifically whether a scrutineer was meant to sit the paper as a candidate would, he replied: "No, they look through the paper and they review it".

'Quality control'

Ofqual has confirmed to the BBC that it will look at what the examination boards expect of scrutineers in its ongoing inquiry.

Chief executive Glenys Stacey insisted that there is a clear requirement that they "work the paper", or carry out a dry run under controlled conditions to ensure that all questions are possible within the given time.

"Preliminary evidence would suggest that that hasn't worked, and it hasn't worked well in every case and we do want to have a look at that very closely indeed," she said.

The measurement mistake in this A-level physics exam is just one of many in this year's test papers

"We do expect to find examples here of where people simply haven't done their jobs properly because it's an issue in more than one of the errors that we're talking about."

OCR spokesperson Rebecca Birkett-Smith told the BBC that OCR "follows Ofqual's Code of Practice on the preparing of question papers".

Speaking to BBC Radio 4's The Report, Ms Stacey revealed that other weaknesses in the exam-setting process have also emerged, including the making of late changes to a paper before it is sent out to schools and colleges.

"Papers are honed up over a period of time. But if there are changes to papers we want to make sure that quality controls checks happen and that things come out right.

"A second area that we've identified is the arrangement for the printing of papers because we've seen a number of shortcomings there."

Ofqual hopes to publish interim findings in October, and a final report by December.

You can hear the full investigation on the The Report on Thursday 14 July at 2000 BST on BBC Radio 4. Listen again via the BBC iPlayer or download the podcast.

View the original article here

Inquiry into exam errors launched

1 July 2011 Last updated at 08:09 GMT Ofqual waited for the exam season to end before beginning its inquiry Exams regulator Ofqual has launched an inquiry into a string of errors in this summer's exam papers.

The regulator said it would be trying to find the root causes of the 10 blunders that have emerged over the past few weeks.

Three exam boards in England and one in Northern Ireland have experienced problems with errors in their papers.

Deputy chair of Ofqual, Sandra Burslem, said the body had made it clear that exam paper errors were unacceptable.

She said: "Ofqual's priorities during the exam season were to make sure the awarding organisations did everything possible to prevent further errors and to make sure that, where errors have occurred, the marking of papers neither unfairly advantages or disadvantages the candidates involved.

"Now that the taking of exams is over we turn our attention to an inquiry."

Held to account

She said the inquiry would hold the exam boards to account for their mistakes by finding out what had caused the errors and how they could be put right.

It would also look at the need for any necessary improvements and arrangements for risk management. And it would set out any decisions on further regulatory action related to the errors.

She added: "The regulators will not hesitate to take regulatory action as necessary at any stage to protect the interests of students."

So far 10 errors have been identified on exam papers from three exam boards in England, OCR, AQA and Edexcel, and one board in Northern Ireland, CCEA.

Ofqual said it would continue to work to ensure that any candidates faced with exam errors were not unfairly disadvantaged or advantaged as a result.

National Union of Students vice-president for further education Toni Pearce welcomed the inquiry, saying: "It is vital that students are able to have full confidence in the exam paper in front of them.

'Tiny proportion'

"We have heard from many students worried about the added stress of wondering if a difficult question is in fact unanswerable.

"It is unacceptable for exam boards to let even one error slip through and they should be forced to compensate every student who feels they have been disadvantaged by exam board error."

Dr Jim Sinclair, director of the Joint Council for Qualifications, which represents exam boards, said the council welcomed the inquiry and would continue to work with the regulators to identify the causes of the errors.

"This inquiry builds on the rigorous investigations awarding bodies are carrying out within their own organisations.

"The examination system is huge, and the questions and papers that contained errors - although unacceptable - represent a tiny proportion of the total number. Each year awarding bodies set more than 60,000 questions."

He said awarding bodies deeply regretted the errors that occurred and apologised to students who took the papers.

The Schools Minister Nick Gibb said it was right that the watchdog should take a "tough line" with the exam bodies.

"These are important exams and students affected will understandably be concerned about these mistakes," he said.

"We expect exam boards to carry out their clear responsibility to ensure exam papers are free from errors, and to make sure that students are not disadvantaged by these failures."

View the original article here

Exam errors not acceptable - PM

22 June 2011 Last updated at 13:34 GMT Errors this year have included impossible questions A series of at least nine errors in this year's GCSE, A-level and AS-level exam papers is "not an acceptable situation", Prime Minister David Cameron has said.

He said England's exams watchdog Ofqual would take the "toughest possible action" to stop further errors.

Ofqual said it was meeting with exam board heads and would launch an inquiry this summer.

Three more mistakes emerged on Tuesday, in AQA and OCR exam papers.

Both exam boards apologised for the mistakes, which affected OCR's A-level physics and GCSE Latin papers, and AQA's GCSE maths foundation paper.

Speaking at prime minister's questions, Mr Cameron said he had discussed the issue with Education Secretary Michael Gove, who had spoken to Ofqual.

The qualifications regulator was "taking the toughest possible action to root out this failure and make sure it doesn't happen again", he said.

In a statement on Wednesday morning, Ofqual said it was "unacceptable" that candidates had faced fresh errors after exam boards had provided written promises that additional quality checks would take place after the first wave of errors.

"Our concerns are serious enough that our director of regulation is meeting with the chief executive of OCR today to understand how these recent errors occurred and to make sure, so far as possible, that there will be no more avoidable errors," the statement said.

'Angry'

Regulators would be meeting on Thursday with heads of other exam boards and would "again ask for confirmation that there will be no more avoidable errors", Ofqual said.

When the exams are over, full details of an inquiry will be announced, it said.

"We want to understand how this pattern of errors occurred, how errors remained after additional checks had been made and what needs to be done to prevent any recurrence of such avoidable errors."

On Tuesday, a spokesman for the Department for Education said the latest errors were "completely unacceptable" and that Mr Gove was "angry".

"He has said repeatedly that the exam system is discredited and action must be taken," the spokesman said.

In the Latin GCSE error, a paper taken by up to 8,000 pupils contained three errors, representing 14 out of 50 marks.

An error in a question in the A-level physics paper, taken on Tuesday morning, meant that pupils were asked to answer a question which included incorrect units of measurement.

And a printing error on a maths GCSE foundation paper, also taken by pupils on Tuesday morning, had questions from a previous paper accidentally printed in the middle of it.

An OCR spokesman said the exam board was "deeply unhappy" about the latest errors. It said there would be an investigation, and jobs could be lost as a result.

AQA said it was sorry that the error had caused some students distress. It said the batches of papers that were checked were all fine, and it was investigating with its printer how the problem had arisen.

Six other errors in previous papers taken this year have included a multiple-choice question where all the answers were incorrect, and a question where the information needed to make a calculation was missing from the paper.

These appeared in papers produced by the Edexcel exam board, and the Northern Ireland CCEA board, as well as AQA and OCR, and have affected exams taken in England, Wales and Northern Ireland.

View the original article here

Three more errors in exam papers

21 June 2011 Last updated at 11:45 GMT By Hannah Richardson BBC News education reporter These latest exam papers errors follow the discovery of six previous mistakes Three more mistakes have been found in GCSE and A-level exam papers being taken by pupils in the summer exam season.

A-level physics and a GCSE Latin paper from the OCR exam board and an AQA maths GCSE foundation paper are the latest to contain errors.

After a series of earlier errors, the Ofqual exams watchdog warned exam boards against any more mistakes.

Both the AQA and OCR boards have apologised for the errors.

Earlier this month, Ofqual reassured candidates that it was addressing the problem of errors in exam papers - after pupils had been confronted with unanswerable questions.

But three more errors have emerged, including in two papers taken on Tuesday.

Ofqual now says it will "now be seeking explanations as to why these errors still occurred".

Printing error

An OCR spokesman said the exam board was "deeply unhappy" about the latest errors and there would be an investigation and jobs could be lost as a result.

The Latin GCSE error, in a paper taken by up to 8,000 pupils, contained three errors which represented 14 out of 50 marks.

Examiners will now have to decide how to mark the paper in a way that will treat candidates fairly.

An error in a question in the A-level physics paper, taken on Tuesday morning, used the wrong measurements.

A candidate taking the OCR exam paper told the BBC that pupils "were given a few minutes extra in our exam when the mistake was spotted and it caused some disruption".

There was also printing error in a maths GCSE foundation paper, also taken by pupils on Tuesday morning, set by the AQA exam board.

Some of the papers, given to 31,659 students, had questions from a previous paper accidentally printed in the middle of it, AQA said.

It estimates that the printing error affects only 2,500 of the total.

AQA said it was sorry that the error had caused some students distress.

It added: "From the information we have at this stage, it seems that some of the papers contained questions from the March 2011 paper.

"We understand the problem version of the paper begins and ends with June questions, but has March questions in the middle of it."

AQA said in some of the papers one of the questions had another from a previous paper printed in the middle of it - making it difficult for candidates to follow. It said 80% of the June paper was wrongly made up from questions from the earlier paper.

It added: "We have told schools that students should attempt the paper as it is and we will consider the most appropriate action to protect students' interests, when we have a full understanding of the extent of the problem.

"The batches of papers that we checked as part of our quality assurance process are all fine and we are in the process of investigating with our printers how this problem has arisen.

"As with any problem of this nature, our top priority is to protect the interests of students and we will ensure that no student is disadvantaged by this printing error."

A fortnight ago AQA had to apologise for errors in a geography AS Level paper, a business studies paper and a computing paper.

And it is not the only exam board that has run into difficulties with papers this year.

The exams watchdog, Ofqual, said on June 9 that it was investigating six errors. Five were AS-levels and one was a GCSE.

View the original article here

Teachers complain of exam errors

16 June 2011 Last updated at 15:59 GMT By Angela Harrison Education correspondent, BBC News Students involved will sit their other ICT exam later this month ICT teachers have complained they were told to "mark right answers as wrong" because of mistakes by an exam board.

They have complained to England's exams watchdog Ofqual about several errors on a guide they were given for marking course work for an AS-level exam.

The exam board involved, OCR, says students will not suffer as a result and that there were no mistakes on the paper itself.

There have been several errors on exam papers this summer.

Ofqual last week said the series of mistakes was unacceptable and disappointing.

This latest case involves the marking of course work, known as a "structured task".

Students have to complete a number of tasks under exam conditions, such as creating websites and databases.

The course work accounts for 40% of the AS-level.

It is marked by students' teachers against a guide or "mark scheme" given by the exam body which tells them what they can award marks for. The work is then sent to the exam board where a sample from each school is checked.

'No excuses'

Documents seen by the BBC show the exam board accepts there were four mistakes on the marking scheme - though some teachers have said there were as many as six.

When teachers pointed out the errors to the exam board, the tutors say they were told to follow the marking scheme and that adjustments would be made by exam board moderators.

The exam bodies normally check a sample of papers from each school or exam centre.

Continue reading the main story

There should be a system where they are checked and double-checked - not just for us but for the children”

End Quote Lynne Head ICT teacher Lynne Head, an ICT teacher from Hemel Hempstead in Hertfordshire, said: "It's so unprofessional. It's become a farce. There should not be any excuse for any of these mistakes.

"There should be a system where they are checked and double-checked - not just for us but for the children."

She said some of her students had been upset when she showed them their marks - even though she explained that these could well rise once the exam board had re-checked them.

"They were crestfallen, saying, 'All that work and I only got a C,'" she said. "I feel so sorry for them."

Former ICT examiner Paul Long, who runs an online support group for ICT teachers, told the BBC: "When the mark scheme was published, there were four serious errors in the answers given. If OCR had issued an errata notice [a correction], then this could have been resolved.

"However, they told teachers to mark correct answers as wrong. This would be like in a maths exam if a child had written '5 + 5 =10', but the mark scheme said '11' and teachers were told to mark '10' as wrong."

Mr Long says the process the exam board is putting in place to manage the errors is "completely flawed".

He said: "They will only see a small percentage of work from candidates, and both those candidates who get the answers right, and those who get them wrong, will be treated exactly the same."

'Rare mistakes'

A spokeswoman for OCR confirmed there had been errors but said these would "not impact on students" because measures were being put in place to deal with the problem.

"We are able to adjust for that," she said.

"It is rare that exam boards make these errors. But we are aware of them and we are dealing with them."

An ICT teacher from Leicestershire has made a formal complaint to Ofqual about the assessment of the AS-level, saying he speaks on behalf of "many colleagues in numerous schools throughout the UK".

Ofqual - England's exams watchdog - has confirmed it is looking at the complaint. It is already investigating errors on six of this summer's papers.

It has published figures for the number of errors in recent years - although it warns that they are not directly comparable because Ofqual was created last year and its predessor bodies might have recorded errors in a different way.

Last week it suggested the number for this year was roughly in line with previous years.

But the published figures show no incidents recorded for last year or in 2009, with three in 2008 and one in 2006 and again in 2007.

View the original article here

Students hit by more exam errors

9 June 2011 Last updated at 15:12 GMT By Angela Harrison Education correspondent, BBC News Student Well Edwin Li Ping Wah: "I spent 15 minutes on that question"

England's exams watchdog is investigating a total of six mistakes in exams taken by tens of thousands of students.

The latest are in a geography AS-level paper and a business studies GCSE.

Ofqual has said the series of errors in this year's public exams are "disappointing and unacceptable".

It has ordered urgent extra checks by exam bodies amid complaints from students and teachers.

The National Union of Students is calling for an urgent inquiry.

More than 90,000 teenagers took the papers concerned.

The six exam papers being investigated by Ofqual cover A-level, AS-level and GCSEs taken in England, Wales and Northern Ireland.

A seventh has been identified by the Northern Ireland exam board and regulator.

Ofqual says five are AS-levels (which can count towards an A-level) and one is a GCSE.

On Thursday afternoon, this was confirmed as being the CCEA business studies GCSE, taken by 3,400 students in Northern Ireland and 500 in England.

Students hit by more exam errors

In the geography paper, a diagram showing the flow of a river was mislabelled in the first question.

A head of geography in the south of England told the BBC News website: "The question was impossible to answer correctly in my opinion.

"They should cut that question from the paper. Being the first question, it really threw some of the students.

"I contacted the exam board but have had no response to the questions I asked."

More errors

The AQA exam body has confirmed the mistake and apologised for it, saying all students would be given full marks for the question. It says the question carried four marks out of 50.

It also said there had been a problem with a diagram on its computing AS-level paper on 7 June, saying: "An arrow in a diagram was shorter than it should have been, which may have confused some candidates".

A spokeswoman said: "We are very sorry for these mistakes and will ensure that no students will be disadvantaged as a result."

The errors affect tens of thousands of students, many of whom will be relying on the results for their university applications.

Student Edwin Li Ping was affected by a mistake in his biology AS-level, set by the Edexcel exam body.

"For the first 15 minutes I looked at that question when I should have spent one minute on it," he told the BBC.

He wants to study medicine and says he "would really like to get that A".

Compensation call

The National Union of Students has written to the Education Secretary Michael Gove to call for an independent inquiry.

It says students whose university places are affected, or who are forced to take re-sits as a result of the errors, should be given compensation.

Shane Chowen, from the NUS, said the errors had become a "deeply concerning" trend.

"Those students who have been confronted with unanswerable questions may have had their confidence knocked and their performance in the rest of the exam affected," he said.

The head of Ofqual has written to all the exam boards on behalf of the exam regulators in England, Wales and Northern Ireland asking them to carry out extra checks on exam papers.

Protecting students

Ofqual's chief executive Glenys Stacey told the exam bodies to make sure measures were in place to deal with any errors that occurred.

Continue reading the main story

We take instances like this very seriously - I am calling on awarding organisations to take steps now to protect students from further disruption and anxiety”

End Quote Glenys Stacey Chief Executive, Ofqual "While the vast majority of question papers taken so far have been free from error, there have been a number of question papers that have included errors," she said.

"We take instances like this very seriously. I am calling on awarding organisations to take steps now to protect students from further disruption and anxiety."

The watchdog says in general the exam bodies have a good track record.

The exam boards involved in the other confirmed mistakes say measures are in place which will ensure candidates do not suffer.

The errors on the other AS-level papers involved:

A maths question which was impossible to answer because not enough information was given (OCR) A business studies question which did not give enough information (AQA) A multiple-choice biology question which gave several possible answers but not the correct one (Edexcel)

The marks at stake varied from just one out of a total of 80 in the biology exam paper, to the maths question, which was worth 11% of the total marks on the paper.

AS-levels are qualifications in their own right but are also taken by teenagers as the first stage of their A-levels.

Some of those who messaged the BBC News website said they were worried the mistakes could cost them a place at university.

'Thousands let down'

Natalie Edwards, from Stevenage, told the BBC: "The exam was hard enough without having mistakes on questions that are designed for people to be able to pick up the easier marks. I couldn't work out why I wasn't getting any of the answers in the paper, which caused me to get stressed."

Sarah Streiber, from Wales, said she took both the business and maths papers in which errors appeared, and was hoping to go to university before increased tuition fees are introduced in 2012.

"And now I have potentially lost that chance! Thank you very much to the examiners who have let thousands of us down," she said.

Jim Sinclair: "We would like to to reassure students procedures are in place to cope with errors"

The exam bodies have released a statement as a group, saying nearly all of the tens of thousands of examination questions published each year are error-free.

They add that extra checks are being carried out.

Director of the Joint Council for Qualifications Jim Sinclair said: "Awarding bodies are aware that a small number of questions in this year's exam papers have contained errors and understand the distress this may have caused students.

"Students and parents should be assured that no one will be disadvantaged as a result of these mistakes. Examiners marking the papers are aware of the incidents and will make careful adjustments so that all students receive the marks they deserve."

Ofqual has powers to name exam bodies which make errors, and to direct them to take certain actions to rectify the situation.

Its ultimate sanction is to withdraw recognition from an awarding body - effectively removing its powers to set and mark public exams.

Northern Ireland's Council for the Curriculum, Examinations & Assesment (CCEA) said there was also a problem with its further maths AS-level paper, which was taken by 122 students in Northern Ireland.

In this case, it said: "An error had been identified as part of CCEA's quality procedures and the paper was sent to the printers for correction. During this process a new error was introduced by the printer.

"This related to one part of the last question on the paper and was worth four marks out of a total of 75 marks."

View the original article here