Challenging our teaching in evidence-based medicine. A risk worth taking

Only those who will risk going too far can possibly find out how far one can go.

“Is there a need? Will people come?” Questions I worried over when deliberating putting together the first qualification in teaching evidence-based health care (TEBHC). I had some answers to the first question but the second was an unknown. It could have stopped us in our tracks. I’m glad we took the risk.

Between February 3 – 7 we had the first run of a new module (developing practice in medical education) with 5 postgraduate students signed up to the certificate in TEBHC and one stand-alone student. The stand-alone is now keen to complete the certificate.

Being an educator in EBHC creates a dilemma. We teach others the skills needed to ensure their practice is informed by the best-available evidence. But being an educator is a profession. How then do we ensure our practice is “evidence-based”? And what does “evidence-based” mean in the context of education and teaching? These were some of the questions I was interested in and wondered if others had similar. The reality was I became much more interested in questions I hadn’t even considered. And that for me is perhaps the greatest thing about learning.

As teachers of EBM we were all familiar with CATs – or at least we thought we were. For us CATs meant “Critically Appraised Topics”. What we didn’t know is that teaching and education also has CATs – Classroom Assessment Techniques. Examples include “The muddiest point” – a simple CAT to help assess where students are having difficulties and consists of asking students to jot down a quick response to one question: “What was the muddiest point in [the lecture, discussion, homework assignment, film, etc.]?” The term “muddiest” means “most unclear” or “most confusing.”

Throughout the week I was mindful of bringing up the point of how much our prior knowledge in our field acted as both a help and hinderance when looking at the evidence-base in teaching and education. This was particularly relevant for a new session I delivered on how to find and appraise evidence in education. It brought us to Best Evidence Medical Education (BEME). They provide systematic reviews of education interventions and practices, aiming to raise the standards of evidence in medical education which is great to see. When we looked at a few BEME reviews through an EBM lease (think Cochrane, GRADE etc), we found some issues, particularly with details as to the methods used to assess the quality of included studies. Some in our group were surprised by this – so much so that they now want to systematically assess all the reviews in BEME and help improve this element of their reviews.

One of the most hotly debated topics was around learning styles. Did you know that evidence to support their existence is pretty weak? And even more so regarding the evidence that adjusting teaching style and delivery to match with different learning styles? Finding this out was a “threshold concept” moment for many. Finding out about “threshold concepts” was a threshold concept (TC) for me.

What is a TC? An example might help here. Let’s say tomorrow you are told that the earth does not actually orbit the sun and the ancient Greeks had it right all along. And this new truth is now what physics teachers have to teach to their learners. This new learning would be a threshold concept for all physics teachers when they encounter it for the first time.

Formally, TCs are “The theoretical framework in higher education to describe ways of thinking and reasoning that are unique to a profession and enable the learner to “think like” and become a professional”. There are four key components of a TC:

1. Transformational – The irreversible reconfiguration of patterns of understanding, which results from the engagement, has effects on the patterns of practical action that follow. Once one has seen the “world” anew, the way in which one thinks and acts has changed.
2. Integrative – Crossing the threshold brings new connections and patterns in the focal area of study into view through the new conceptual lens (“It’s all coming together now”)
3. Irreversible – The change in understanding associated with threshold concepts is usually not reversible. Once new connections and patterns have been discerned, a retreat into earlier patterns of understanding cannot easily be achieved, although the concept itself might be superseded by even more sophisticated, alternative conceptualizations.
4. Troublesome – For some threshold concepts, troublesomeness arises because of the way core concepts are bound together in “an underlying game” to create a “disciplinary way of knowing” that may be imperceptible to novice students. Knowledge that may seem simple to the ‘expert’ professional may really become a barrier to the learner’s progression.

(Find out more about TCs here).

This got me thinking how evidence-based medicine must have been a TC for many when it was first introduced, particularly and more so for medical educators than students and trainees. An exert from Dave Sackett’s last interview provides some striking evidence for this, where in Chapter III-5: How did Britain respond to your introduction of EBM?, Dave states:

“…the response of medical students (Bob Phillips and his classmates put on their own EBM Workshop), post-graduate trainees (the Chair of Medicine told me that growing numbers of applicants for our house jobs listed working with me as their motivation), and individual clinicians throughout Britain and Europe was wonderfully enthusiastic and supportive. But the initial response of the British medical ‘establishment’ was so negative, condescending, and dismissive that I was often miserable for the 1st year and a half of my time there.”

“The establishment considered EBM an affront to their omniscience and authority and dismissed it as both ‘old hat’ (“everybody’s already doing it”) and a “dangerous innovation, perpetuated by the arrogant to serve cost cutters and suppress clinical freedom.” It was labelled “impossible to practice,” “could be conducted only from ivory towers,” “cookbook medicine,” and “restricted to randomised trials and meta-analyses.” Rather than take me on, a fellow-professor right in Oxford published nasty criticisms of 2 of our younger advocates in a ‘Socratic dissent’ [68,69]. Another Oxford colleague, learning that I planned to videotape my forthcoming Grand Round on ‘Observer Variation in Evaluating a Patients with Dysphagia,’ wrote a letter of protest to the Chair of Medicine, stating that I was attempting to destroy the tradition of confidentiality in Oxford’s most cherished hall.”

Thanks to the perseverance of Dave Sackett (and others), the principles of EBM and EBHC are now less likely to be TCs for many of our learners. Our thoughts turned to the possible TCs within EBM/EBHC. This was tricky, but some emerged. The ongoing struggle of applying evidence of average effects in groups of people to individuals was one. Indeed the subject was being discussed on Twitter the day I wrote this! Other potential TCs included what a successful shared decision making (SDM) process looks like, when SDM should and shouldn’t take place, and how to successfully teach our learners these concepts? Teaching of these elements is not present to the same degree as for other steps of evidence-based practice in the majority of curricula programmes and studies of EBM/EBHC educational interventions, suggesting they may indeed be TCs medical educators need to embrace.

Had we not taken the risk to put together and run the programme in Teaching EBHC, I may never have had the opportunity to experience this learning and share some of it as I am now with you. I may never have been inspired to lead a series in the BMJ EBM journal all about considering “What next for education in evidence-based health care”. Borrowing the words of T.S. Elliot: “Only those who will risk going too far can possibly find out how far one can go.” How glad I am we took the risk.

We will take these discussions forward to this year’s EBMLive conference taking place in Toronto 6-8 July. I hope you can join us.

Competing interests: DN has received funding from the NHS NIHR SPCR programme and the Royal College of General Practitioners.

Disclaimer: The views expressed in this article represent the views of the author and not necessarily those of the host institution, the NHS, the NIHR, or the Department of Health.

David Nunan is a Senior Researcher and Director of Postgraduate Certificate in Teaching Evidence-Based Health Care. He is based at the Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford.

You can follow him on Twitter @dnunan79