I came across an interesting example of a literature review the other day and thought it was worth sharing. The review was in a paper by Glowacki and colleagues (2020)*, who were reporting on a study to see if a series of grant writing workshops run with academic staff had actually increased the number of successful grant applications.
What caught my eye was that the review was systematised. So what is that and why is it important if you are grappling with a literature review, or helping someone else grapple with one? To answer that, I need to back up a few steps and explain systematic review studies. If you are doing a qualitative systematic review study, or its quantitative counterpart, a meta-analysis, you are essentially doing a study of studies. That is, you have a research question you want to answer, but instead of interviewing people or running an experiment or analysing a corpus of newspaper articles (or social media posts or whatever), you locate the relevant studies on your question and analyse the patterns and trends in the body of previous findings in order to find your answer. An example is a recent-ish systematic review by Rogers and colleagues (2016)**, who wanted to know, now that we have considerable body of work on critical discourse analysis in educational research, what we have actually found out and what the implications of this are for future research. To answer these questions, they located 257 previous articles that undertook critical discourse anlayses in educational matters and plotted the major patterns and trends that emerged in that body of work.
Systematic review studies are different from literature reviews. While a systematic review study is a piece of research in its own right that sets out to answer a question, the literature review is not a piece of research in its own right; rather, it supports a piece of research by setting out why the research question needs to be asked. That is why your dissertation literature review is a chapter in your dissertation or, if you are writing a journal article reporting on a research project, the literature review is section in that article. The literature review is not the research project itself -- it supports the research by establishing why the research question needs to be asked.
Systematic reviews are very serious about the way they find the previous articles that will form their dataset, the body of articles that will be analysed in order to answer the research question. Protocols have been developed for how searches are to be done and how the final body of articles are to be delimited in order to a) be sure the answer to the question is based on the best available evidence to date; and b) avoid confirmation bias, cherry-picking articles that skew the answer in one direction or another.
While your dissertation literature review is not a systematic review study, are the rigorous search protocols from these studies usefully applied? This question is covered in an excellent book, Systematic Approaches to a Successful Literature Review, by Booth and colleagues***. They note that even if you are not doing a fully systematic review study, your literature review can still be systematised to various degrees. For example, what databases did you search? Do you have a list of those? What search strings did you use to search them, and how did you develop and pilot them? From the potentially thousands of studies you might have located, how did you decide which of those studies would make the final cut for your dissertation literature review? If you also hand-searched for references, how did you do that? You may or may not proceed this way -- that is a conversation for you and your supervisor. And if you do proceed this way, you may or may not write all of your procedures up in your actual literature review, the way Glowacki and colleagues did (another question for your supervisor). You might instead put it in an appendix, or keep a record for yourself, in case an examiner asks you about your literature review. To return to our starting question, should your review be systematised? Maybe, maybe not. But it is worth thinking about.
* Glowacki, S., Nims, J.K., & Liggit, P. 2020. 'Determining the Impact of Grant Writing Workshops on Faculty Learning'. The Journal of Research Administration. 51 (2): 58-83.
** Rogers, R., Schaenen, I., Schott, C., O'Brien, K., Trigos-Carillo, L., Starkey, K., and Chasteen, C.C. 2016. 'Critical Discourse Analysis in Education: A Review of the Literature, 2004-2012'. Review of Educational Research. 86 (4): 1192-1226.
*** Booth, A., Sutton, A., & Papaioannou, D., 2016. Systematic Approaches to a Successful Literature Review. 2nd ed. London: SAGE.
No comments:
Post a Comment