🔸Do a critical appraisal🔸
If you are a research student, you'll run into an assignment like this one sooner or later.
But what is it really asking you to do? 🤔 The guidebooks will offer you clear, crisp definitions of what a critical appraisal is, like this one:
❝The use of explicit, transparent methods to assess the data in published research, by systematically considering such factors as validity, adherence to reporting standards, methods, conclusion and generalisability❞ (from Systematic Approaches to a Successful Literature Review, pg. 304).
Excellent! Now you know that you need to be explicit, transparent, and systematic. You need to assess the data in the research that has been published in your field. And there are any number of checklists 📋 you can consult that will help you zero in on what to look for (Are the conclusions warranted by the data? Does the analysis stand up? Does the study measure what it says it will measure? Have the methods been robustly justified and applied?).
But how do you actually do your critical appraisal? When you are sitting with a stack of studies 📚, you very naturally go through them one by one. You read the first study, go through your checklist of questions, spot some areas that don't quite stand up to scrutiny, and write that up, perhaps along lines like these: Smith and Jones 2018 put forward a number of valuable insights, but their case study approach does not warrant the conclusions made. And then you pick up the next study, and so on.
There is great value in this, as you are honing your critical eye, 🧐 learning about methods and methodology, and building your mental database of all the ways that your topic has thus far been studied. All of these things are a necessary part of your learning as a research student.
➡️ But here is something crucial in writing critical appraisals that seldom gets mentioned -- going through studies one by one and writing out separate evaluations for each study is only your starting point. It is a necessary step in your critical appraisal, but not the final step.
If you simply put each evaluation in your chapter, you wind up with a series of separate appraisals. Each one might be wonderfully insightful, and each one might show that you really know your stuff.
➡️ But your purpose in a critical appraisal is not to pick holes in previous studies to show that you know your stuff. If you are writing your literature review chapter, your purpose is to show why your research question needs to be asked. If you are writing your methodology and methods chapter, your purpose is to show that your methodology and methods are an appropriate and fitting way to answer your question.
The one-by-one appraisal process you started with will not establish either of these things. What you need now is the next step, which is to identify patterns and trends in the studies you've just read and assessed. You need to critically assess the body of work in your field, not separate studies one at a time.
➡️ And how do you do that? Try this: once you've finished step one, go back through that material and start extracting some lists (in tables, in a spreadsheet, on note cards -- whatever works for you). Record key information, such as all the research questions that were asked, all of the methods used, all of the weaknesses that you found, all of the strengths that you found, all of the conclusions and findings. Then start sorting this information, putting like with like, looking for contrasts, and so on. What are the overall patterns? Do you find that most work in your field is qualitative rather than quantitative, for example? Or maybe you find that work in your field is dominated by a specific approach to analysis while others have been left untried? Or maybe work in your field has stemmed from some pioneering papers whose assumptions now need to be revisited? In this way, you move from evaluating single studies in isolation to using those evaluations as evidence in your argument, either establishing why your research question should be asked, or establishing why your methods are justified.
And now you are on your way to what a critical appraisal should be 😊
*Booth, A., Sutton, A., and Papaioannou, D. 2016. Systematic Approaches to a Successful Literature Review. 2nd ed. London: Sage.
Susan Mandala, PhD is an independent writing and language consultant and founder of Writing Works Consulting. She is dedicated to helping everyone she works with achieve greater professional success through more effective writing, thinking, and use of language.
She specialises in dissertation literature review workshops for doctoral candidates and academics new to research, so if you are a research manager or research administrator in higher education and looking to develop research support in your institution, contact Susan to discuss a workshop or short course.
🟢 Hope on to her website and contact her from there: www.writingworksconsulting.co.uk
🟢 Connect on LinkedIn and drop her a message: www.linkedin.com/in/susan-mandala-phd-6a94b7290
