The dog ate my homework: Education’s evidence-excuses echo-chamber

Image source: ChatGPT I am not the first to write about the fact that education has a fraught and not very pretty history when it comes to generating, critiquing, and applying rigorous evidence in order to maximise student academic and wellbeing outcomes. Examples of this commentary can be found here, here and here. Way back in 1996, David Hargreaves observed in a Teacher Training Agency Annual Lecture in London, that “Teaching is not at present a research-based profession. I have no doubt that if it were, teaching would be more effective and more satisfying”. In 2000, Dr Louisa Moats commented that “Unfortunately, lack of rigor and respect for evidence in reading education are reinforced by the passivity of education leaders who feel that any idea that can muster a vigorous advocate is legitimate and deserves to be aired”. Dr Moats was discussing reading instruction, but her words are no less applicable to other curriculum areas, most notably mathematics instruction. Of course, there’s a bias from me right there in my opening paragraph – the idea that education exists to maximise student outcomes, academically and psychosocially. So yes, let’s be clear that that is, for me, a strong ethical and civic responsibility for those who work in education, across the university, policy, research, and classroom practice levels. And yes, I am of the view that transmission of knowledge as well teaching students how to apply said knowledge, is core business for schools. For some, that immediately casts me as “neoliberal” in my political views. I’m not now, nor have I ever been, a member of any political party but if my position that accountability matters as much in education as it does in other professional fields makes me neoliberal, I’ll take that over being what I’ve termed previously, paleoprogressive. On social media platforms, I see some astonishing approaches to evidence, sadly, often from other education academics, but also from education consultants whose steady-as-she-goes operating models cleverly create illusions of change while simultaneously minimising it (often with a great deal of fee-for-service “busy work” in the middle). Meanwhile, classroom teachers are making it their business to learn the language of evidence, and often do so in their own time, via teacher-led platforms such as researchED around the globe. Others engage in further study and/or join closed communities of practice on platforms such as Facebook. They are not waiting for those in the academy to take the lead on prioritising evidence-based practice, and sadly, that’s probably just as well. I have blogged previously about the fact that much change is occurring on the metaphorical forest floor (classrooms), while the tall trees, whose branches thrive in rarefied air (education academics) are somewhere between oblivious and hostile to these efforts. “Evidence-based medicine” is a term that entered clinical parlance as recently as the 1990s, though it had antecedents that go back much further than that, such as the US Flexner Report in the early 20th century. Critically, it was a call to medical practitioners (and rapidly, by extension, all health professionals) to move beyond blind and often uncritical faith in practitioner judgement, beliefs, and established practice, to a recognition that independent research might offer different, more effective, sometimes more efficient approaches, to patient care. Evidence-based practice in health is not aboutso-called "cookbook" practice (aka “one-size fits all approaches)practitioners blindly following research findingsreplacing/overriding professional judgementignoring the context of the individualpretending that endless resources are available. Medicine, like education is an interface of science, craft, and the complex vagaries of human nature and human behaviour. Claims by some in education that “it’s easy in medicine, you just do a clinical trial and find the answer” betray a phenomenal ignorance of the psychological, socio-cultural, cognitive, linguistic, and economic factors that introduce enormous variability into clinical trials and have to be taken account of in research analysis. Education is not special. Like health, it deals with human beings in all their complex and unpredictable glory. Like health, too, its practitioners have ethical obligations to (a) do no harm and (b) deliver effective services that improve the human condition. The difference, however, lies in the misconceptions and misrepresentations of research that are perpetuated in education university lecture theatres, and by extension in classrooms around the globe. Education has enabled excessive permeability on ideas and practices that have low or no empirical evidence to support them (e.g., learning styles, left-brain-right-brain learners, Brain Gym, whole language, coloured lenses for struggling readers, etc etc - e.g., see here) while paradoxically resisting approaches that do have substantial evidence to support them (explicit reading instruction being a case in point). I’ve compiled a list below of what I see most commonly on social media and have quoted to me by teachers who are grappling with change processes at school and sector-levels. These all amount to poorly conceptualised excuses as to why education should be given an evidence leave-pass. They should not go unchecked. Line of argument against evidence-based practice My response Teaching is too complex to be studied scientifically Complexity does not take systematic investigation off the table. Researchers in medicine, allied health, economics, psychology, nursing, criminology, and public health all study highly complex humans and human systems. They do so using research methodologies and tools that are appropriate for the questions being asked. They then triangulate findings across research methodologies and findings to find the best direction of travel, based on current evidence, knowing that this will need to change as the evidence evolves. This is not the same as "choose your own adventure". Research in education is not about achieving “perfect certainty” – that’s an unhelpful straw man argument. Rather, it should seek to identify practices that are more likely than alternatives to improve outcomes, for the majority of students. Education research, like health research, should allow practitioners to play the probabilities in measured, careful ways. Every child is different, therefore findings based on aggregated data are meaningless Children are individuals, yes, as are patients. However, there are more similarities than there are differences between humans with respect to how they learn (see for example, the work of Dan Willingham and Stanislas Dehaene). Human variation exists in every field where evidence is used; this is not unique to education. Research identifies tendencies and probabilities within and across populations. In so doing, it is drawing on amounts of data that extend way beyond the bounds of individual practitioner experience. The late Daniel Kahneman, author of Thinking, Fast and Slow reminded us that “The confidence that individuals have in their beliefs depends mostly on the quality of the story they can tell about what they see, even if they see little.” Experience is more important than research evidence Experience is of course valuable, but it can also reinforce misconceptions and create blind spots where new evidence is concerned, especially if this challenges existing beliefs and practices. People are often poor judges of cause and effect, partly because they do not have clean line of sight of all of the factors in play. This is one reason that we are all prone to various forms of cognitive bias (see here and here): mental heuristics that have a place in everyday thought-processes, but need to be kept in check to allow us to verify assumptions and revise beliefs and practices in response to new evidence, even when this creates cognitive dissonance or ideological discomfort. Many educational practices that “felt” effective to teachers for a number of decades were later shown to have little or inadequate impact on student learning. Emily Hanford’s award-winning Sold a Story podcast laid this bare in relation to the pernicious harm done to tens of thousands of children, around the globe, by balanced literacy and its antecedent, whole language. Teachers used these approaches in good faith and because they had not been presented with alternatives. In many cases, though, when alternatives are made available, teachers seize them with enthusiasm. Evidence-based practice threatens teacher professionalism and autonomy Professions that we hold in high esteem (e.g., medicine, aviation) actually have very low levels of autonomy, and high levels of accountability. I have blogged about this aspect of professionalism previously. There is a double-standard in the fact that folk in education expect evidence-based decision-making when they are the patient/client/passenger in a transaction but don’t see that the rest of the community expects the same rights as “consumers” of education. What threatens teacher professionalism (and workforce longevity) is the creeping realisation that they have not been adequately equipped for classroom practice, using effective approaches for positive behaviour support and instructional success. Evidence-based practice does not inherently require pre-prepared lesson plans but it’s difficult to understand opposition to practice that reduces variability and promotes consistency within and between classrooms. High variability and high quality cannot co-exist. Schools and sectors need to choose one. In most professional fields, expertise involves integrating evidence with professional judgement. This can occur when teachers are well-equipped by their initial teacher preparation to critique and use research evidence. The application of evidence does not remove the need for teacher decision-making. It enhances and refines teacher judgement and allows teachers to apply their pattern-recognition skills in decisions across the day. Classroom practices should be dictated by the best interests of the students, not the personal preferences/ideologies of the teacher. Think about how this principle plays out in medicine, nursing, and aviation. When teacher unions (mis)play the autonomy card, they are working against the best interests of their members. Randomised controlled trials are not applicable / ethical in education. Randomised controlled trials (RCTs) are sometimes described as the “gold standard” for establishing cause-and effect with respect to whether an intervention works. I won’t provide a tutorial on RCTs here as readers can easily learn more from online sources. Suffice to say that when they are well-designed, RCTs give us the greatest clarity about the efficacy of a teaching or intervention approach, as they control for the interference of many variables that can sit between cause (the teaching or intervention) and effect (student outcomes). It is important to distinguish between efficacy (how well an approach works under well-controlled experimental conditions) and effectiveness (how well it performs “in the wild” of the real world). If an RCT supports efficacy, then the next step is an effectiveness trial. A common misconception is that when RCTs are done in schools, some students receive “something” while those in the control arm receive “nothing”. This is incorrect. Those in the control arm may receive business-as-usual teaching, or they may receive another intervention that is considered worthy of testing. In some cases, there is a cross-over design, where students are shifted from one arm of the study to the other, and so are exposed to more than one approach. RCTs, like all research carried out in schools, are subject to rigorous review by human research ethics committees. For a fascinating account of how RCTs have transformed our world, across health, criminology, education and economics, have a read of Andrew Leigh’s Randomistas. Schools are no more or less complex as research environments than hospitals, community health settings, prisons or rehabilitation facilities. All must grapple with the vagaries of human nature and variation in all its guises. Research changes all the time, so teachers can’t rely on it Revision in light of new evidence is a strength of science, not a weakness. Science is by definition self-correcting. The story of stomach ulcers is a good example of this. While once thought to be due to stress, ground-breaking and paradigm-shifting research by Australian investigators, Marshall and Warren showed that bacterial infection, not stress, is the culprit, and management was overhauled at scale, via revised clinical care pathways that all practitioners were expected to adopt, and did so, because that it is their ethical responsibility. The fact that some findings evolve / are modified over time does not mean that all findings are equally uncertain. It means that we need to be open to calibration and in some cases, de-implementation of current practices, as the evidence evolves. It’s insulting to talk about a “hierarchy of evidence”. All methodologies are equally valid. This is an unsophisticated play for egalitarianism in the face of both common sense and Research Methods 101 principles. Research methodologies should be primarily selected for their goodness-of-fit with the question(s) under investigation. In many cases, researchers are faced with choices, e.g., if they are asking “What is the lived experience of teachers who have been abused and bullied by dissatisfied parents?” they may elect to carry out qualitative, in-depth interviews, or they may elect to construct a survey that is framed in such a way th