Making Sense of Research: What's good, what's not and how to tell the difference

Author(s) McEwan, Elaine K. and McEwan, Patrick J.
Publisher Thousand Oaks, CA: Corwin Press
Published 2003
Pages 174
Price 25.00
ISBN 0761977082
Reviewed by David Turner
University of Glamorgan
Review published 1 December 2004

This volume, like its authors, brings together the experience of leadership in schools and the commitment to academic study. The authors' purpose is to provide a book giving a framework that practising teachers and school administrators can use to identify academic research that can be practically useful. The premise of the volume is that there is important guidance to be found in the research literature on policy issues which have direct impact upon conduct of schools. In order for that research knowledge to be usefully available, teachers must be able to sift, select and evaluate research so that they can base school policies on sound premises. The alternative is that teachers will be blown off course by the latest fashion from academia.

Making Sense of Research shows its North American heritage most clearly in the assumption that the ideal design for all research is the scientific experiment, and probably the double blind random drug trial. In this way educational treatments could be assigned to random groups and the effect of those treatments evaluated with absolute clarity and without ambiguity.

It is against that standard which all educational research is measured and, not surprisingly, found somewhat wanting. Notwithstanding that general disposition, the book does have much that it is interesting to say about quasi experiments, para-experimental design and the use of comparative study in what might be described as natural experiments. In the process the authors lead the reader through a sequence of questions about any possible educational innovation - Does it work? How does it work? Is it worthwhile? Will it work for me? And, Is it working for me? Each question is illustrated with extracts from the literature on specific topics to show how good research literature may inform policy decisions. The whole process of planning for school administrators is drawn together in a parable of educational planning within an idealised school.

In view of the North American origins of this book there is one rather peculiar anomaly. The authors cite Zvi Griliches (1985) commentary that: "If the data were perfect, collected from well-designed randomised experiments, there would hardly be room for a separate field of econometrics". This quote is used to reinforce the point that where randomised experiments have been used, statistics are hardly necessary to calculate the result.(p.59)

However, having admitted that the bulk of good educational research only approximates to randomised experiments, and therefore, presumably, making space for the need for statistical methods, the authors deliberately eschew the use or discussion of those statistical methods. Correlation studies warrant no more discussion than two pages and that discussion does not really address the question of statistical interpretation. Even so it may perhaps be noted that space devoted to correlation studies amounts to more than that devoted to interviewing and analysing documents put together.

Although attempting to span the space between academia and school-based policy makers, this book falls rather awkwardly between those two stools. It is not quite rigorous or systematic enough to help a researcher decide what makes good research and what does not, and yet it remains slightly too academic to provide a firm framework within which school managers would be able to sift and select from available research literature. The attempt to make evidence-based policy available to decision makers and policy makers at a local level is clearly a worthy goal, and if this book helps to improve the operation of any institutions one can only applaud that outcome. It is hard to imagine, though, that there are large numbers of school policy-makers who are so naive or incompetent in their sifting of research data that they will be much improved by reading this book. On reflection, there may after all be an important market for it.