On opening my emails this morning I was greeted by yet more email from various organisations offering me practice papers and guidance on how to score highly in the forthcoming situational judgement test (SJT), the new replacement for ‘white space’ questions in foundation programme applications.  This is in addition to the full page adverts in the Student BMJ promoting books and online services.

According to the Improving Selection to the Foundation Programme (ISFP) website, the “what you should do” format of the questions was specifically designed to avoid students receiving coaching, citing the GMC’s emphasis on probity.  It would appear that these resources aim to undermine this, with one book offering to “provide hints and tips to maximise your score and win your chosen Foundation Programme”.  It is ironic that the same book also claims to follow GMC guidelines, presumably excluding those surrounding probity.

Given the 50% weighting of the test in foundation applications, it is little wonder that students are apprehensive of what to expect.  Some of the resources on the market are free but many come at a cost, and this raises a number of questions.

If these resources live up to their claims, will students who do not purchase them be at a disadvantage?  Will students who use these resources perform better as doctors than those who don’t, or will they simply gain a few extra points on their SJT (if at all)?  Does reading a book replace clinical experience in learning how to manage everyday situations?  And if, by your final days in medical school, you are relying on a book to tell you how to behave, should you be graduating as a doctor at all? 

The burgeoning market of ‘how to pass exam’ resources is not a new phenomenon. Take the UKCAT, which was introduced to assess aptitude instead of knowledge.  A quick search on Amazon reveals a dizzying array of books offering to help candidates “get into medical school”.  Other examples include the courses aimed to help students pass their finals.  Last month’s Student BMJ had a full page spread advertising a course which claimed a “99.7% pass rate for previous attendees”.  This is an impressive statistic which will entice many students, but does not prove any cause-effect relationship between attending the course and passing exams. Could it be, for example, that the type of student who is likely to attend such courses is also the kind of student who would do well regardless?

These examples all have one thing in common: they are all ‘high stakes’ exams, and this market preys on students who feel they will fail or do badly if they do not use these resources.  Many students do well in exams without such coaching, and until the specific merits of these resources can be quantified, they should not target medical students quite so aggressively.

Thank you nb238 for and interesting and thought provoking post.

I’ve been involved as an Editor in providing such revision material for situational judgement questions for a reputable revision provider. In my opinion, revision providers carry a great responsibility to ensure that candidates are given accurate information and questions of the highest standard possible to enable them to practice the format and familiarise themselves with the style of the exam. This test is quite “different” from the more factual tests that the candidates are used to sitting and, in my experience, there is a lot of understandable concern amongst candidates about what to expect and a desire to practice similar style questions before the exam.

In my opinion, revision providers have a number of roles. Firstly, to enable candidates to practice the style of questions encountered and familiarise themselves with the format. This seems to be particularly important with the “rank order” style questions. Some candidates seem to struggle with this style of question where there is a need to prioritise less favourable options. Secondly, we have a role in educating and passing on a wealth of experience with our colleagues. Our questions (and answers) are compiled by a panel of experienced clinicians. The scenarios we use in our questions are often taken from decades of real life clinical encounters and the answers are compiled by referencing good medical practice and the opinion of an experienced panel. Doctors have always discussed difficult cases with more senior colleagues and our aim with this has been to provide a similar environment where the answers to our SJ questions correlate with the opinions of an expert panel. To provide such a resource costs time, money and a lot of effort and I would be concerned about the validity of a free resource. I would also have concerns about the validity of the source material from some providers – with much the same scepticism that I would view product reviews from an anonymous source on the Internet.

As a final and personal point, I also feel that integrity plays a part here. As a practising clinician, I regularly come into contact with candidates who have been through the assessment and have seen “real” SJ questions. Under no circumstances would I ever participate in any activity that undermined the examination itself. My aim as Editor has always been to provide questions that accurately reflect the content and style of any exam or assessment without copying any remembered questions. I would like to think that any candidates who have used my own resource have emerged better educated and more confident in their abilities to sit such an assessment having shared in the wealth of experience provided by our writing panel.