Forums » BMJ » Student BMJ » Are multiple choice exam questions the best way to identify the best candidates?
|
Are multiple choice exam questions the best way to identify the best candidates?
posted at 11/10/2011 2:36 PM BST
on bmj.com
|
Posts: 1
First: 11/10/2011 Last: 11/10/2011 |
With thousands of applicants worldwide currently filling in their forms to apply for a foundation job in the UK, the question remains how to reliably distinguish between so many high scoring, high achieving, and competitive candidates? Currently the form allows candidates to score a maximum of 100 points; 50 of which come from five white space questions, which are short answer questions around scenarios commonly encountered by junior doctors. Candidates must not only answer the question in less than 200 words but also demonstrate they meet the criteria laid out in the UK Foundation Programme Office person specification. Whilst allowing a degree of individual expression in a process otherwise devoid of personality, the questions have left candidates resorting to plagiarism, collusion, and paying to go on courses coaching students in how to answer the questions to score maximum marks. Concerns about such unfair practices have led to the white space questions being scrapped next year, in favour of a situational judgement test. This consists of a one hour invigilated test, comprising 30 multiple choice questions around how you would react to different situations junior doctors commonly find themselves in. This method means students cannot prepare in advance, minimising the potential for cheating amongst applicants. But, is a multiple choice exam really better at distinguishing the best candidates from the rest? |
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 11/10/2011 2:48 PM BST
on bmj.com
|
Posts: 614
First: 13/1/2009 Last: 10/5/2012 |
I think the answer is not that simple... it depends on how the MCQ test is done, whether they just test your ability to memorize useless facts, or if they actually make you think and relate things. Because real medicine is about thinking, reasoning and putting pieces together, not memorizing. The former is by far a terrible way of selecting candidates, albeit a very prevalent one.. |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 12/10/2011 4:57 PM BST
on bmj.com
|
Posts: 38
First: 10/5/2010 Last: 20/4/2012 |
I think the best possible way to assess candidates and for job-matching is interview. Some deaneries across the UK, ie Yorkshire and Humber interview every potential candidate as part of the application. A standardised interview would ensure that candidates are talked to, learnt about, and assessed objectively by more than one person. Additionally, case-scenarios and vignettes can be worked out with the assessor directly examining how the interviewee is reacting. It seems odd that certain deaneries interview candidates whereas others don't. And surely this system directs where some people apply and where they don't, which can lead to bias. In what we're working with - the SJT will be a better indicator than the white space questions. It's more personalised, less open to cheating/outside assistance, and surely more relevant in terms of shaping up a junior doctor. |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 12:08 AM BST
on bmj.com
|
Posts: 1317
First: 7/3/2009 Last: 15/5/2012 |
Multiple choice exam is not the best way. However it is convenient to check such an exam and to give grades without going into trouble reading the written answers. And another thing: once you go wrong with the numbers of the questions, the whole exam could turn ourt a disaster, not because you didn't answer correctly, simply because you mixed up number 10 for example with number 11 and the whole thing got moved ahead one question,,,
I find the multiple choice method "lazy"... When you allow a person to express his/her thoughts that gives you a much better picture about the candidate.And an interview is important as well. |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 2:06 AM BST
on bmj.com
|
Posts: 2071
First: 10/3/2009 Last: 16/5/2012 |
They do not test linguistic expression or creativity. Grammatically correct, succinct letters are my life-blood. Medicolegal letters require perfection. I find many graduates' English expression wanting. Good communication with patients and other doctors is an essential element in the art of medicine which is not just a science.
|
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 11:46 AM BST
on bmj.com
|
|
*Moderator*
Posts: 622
First: 7/4/2011 Last: 16/5/2012 |
How can multiple choice exam questions really identify the best candidates? As Odysseus comments - there's no trace of the logic or method behind the answers given! The decisions that doctors have to make are not always so black or white. You could just guess your way through a multiple choice exam and pass with flying colours. Does that make you a credible candidate? The linearity of logic and reasoning is integral to making decisions. The answer is important but the route to it is just as valuable in my opinion. |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 1:24 PM BST
on bmj.com
|
Posts: 470
First: 12/7/2010 Last: 14/4/2012 |
I have to disagree with both Matthew and Odysseus here. The greatest strength of multiple choice questions is that they are objective, they won't be subjected to the examiners biases. Either you know the answer or you don't, what's the need for literary maneuverings? The idea is to test the medical knowledge of the examinees, not their literary acumen.
Multiple choice questions do not always have to be yes/no. You can give multiple correct answers to a questions and the examinee has to identify them all. Similarly if you frame the question correctly, you can obviously get the examinee to apply logic.
As for communication skills - judege them separately. Why attempt to the judge the communication skill of a student from his anatomy answer sheet? |
|
|
Re: Reply to Post
posted at 13/10/2011 1:57 PM BST
on bmj.com
|
|
*Moderator*
Posts: 622
First: 7/4/2011 Last: 16/5/2012 |
Ah too true Deb_D Examiners can be very subjective, so even if you wrote a detailed and brilliant answer, it still does not secure you a great mark. Marking multiple choice questions does not require much effort on the examiners side -which is good if you have an examiner that is not completely objective - but are these types of questions nuanced enough to really understand how you might behave or react in a particular situation? |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 3:32 PM BST
on bmj.com
|
|
Posts: 38
First: 10/5/2010 Last: 20/4/2012 |
Surely, with multiple choice exam questions, there is also the risk that one can simply guess the right answer without actually knowing whether that's the right answer or not. Therefore, although it may be objective, you may not be testing doctors' knowledge, rather their ability to make an educated guess. When you're on a ward and have to make an emergency decision, no one gives you five possible options, you have to provide a solution from core knowledge. There's no buzz words or triggers in such a situation, hence MCQ questions have little practical application. |
|
|
Re: Are multiple choice exam questions the best way to identify the best candidates?
posted at 13/10/2011 4:38 PM BST
on bmj.com
|
|
Posts: 470
First: 12/7/2010 Last: 14/4/2012 |
Yes there is always a chance of guessing, and that's what is, a guess. A simple check routinely done here in India is to employ negative marking. You give 1 mark for each correct answer and deduct 1 mark for each wrong answer. It stops all the guessing, at least those who are intelligent enough don't guess and those who are foolish enough to guess end up with negative marks. In this example, if you guess there will be a 25% chance that you will be rewarded and a 75% chance of penalty. Increase the number of options or the number of marks deducted and there will be still heavier penalty for any guesswork.
It entirely possible to set up multiple choice questions that test the decision making ability of the examinees. Sure it's a lot of work for the question setter but it's not impossible.
|
Forums » BMJ » Student BMJ » Are multiple choice exam questions the best way to identify the best candidates?
