Reverse Engineering a Review

An exercise I often use with my students is to reverse engineer a literature review to work out how they are put together.  This helps to get students thinking about what authors actually have to do in order to turn reading notes into a review.  The analogy here is with creative writing: if you were a novice creative writer and you wanted to work out how to write great characters, you would not just read novels with great characters but study them carefully to see how the authors of those works created them. So as novice literature reviewers, I tell my students, train yourselves to learn from more experienced writers. 

As I guide my students through this process so that they can eventually do it more independently, one text I often use for this purpose is an article on English in multilingual advertising by An H. Kuppens (2010), which I have on my list of favorite reviews. Using the following short quote from the Introduction, I ask my students to speculate about what the author had to do to get this seemingly simple observation on the page.  

One notable exception is the research of English in advertising, which, after the publication of a few early studies (e.g. Haarman 1984, 1989; Masavisut et al. 1986; Bhatia 1987; Takashi 1990; Mueller 1992; Cheshire and Moser 1994; Myers 1994) has seen a striking expansion of research during the last decade (e.g. Griffin 1997, 2001; Martin 1998, 2002a, 2002b, 2007; Meinhof 1998; Gerritsen et al. 1999, 2000, 2007; Kelly-Holmes 2000, 2005; Piller 2000, 2001, 2003) . . . [8 more studies are listed, which I omit here for the sake of brevity] (Kuppens 20210, p. 115).

After a thoughtful silence, the students usually say something along the lines of Well, first the author would have had to read the articles.  Excellent, I say, but let's go back even further than that. Working together, we create a hypothetical to-do list. First, the author would have to have had the idea to work on multilingual advertising and would then have started reading on that topic.  At some point, that reading would have become more focused.  The author would then have had to locate studies on English in multilingual advertising, decide which ones were worth a closer read, and then start to make some sense of them as a body of work. At this point, I typically direct the students to the two blocks of references (the first beginning with Haarman 1984 and ending with Myers 1994, and the second beginning with Griffin 1997). Why are these references listed together like this?  What do they have in common?  There generally follows another thoughtful silence, longer this time, as the students look, really look -- perhaps for the first time -- at the references and consider why they are there.  There is no quote, so why are they given? And why so many?  Gradually, after several readings, the students notice that the first string of references are all examples of studies the author has decided are 'early', and the second, longer string represents what the author saw as the 'striking expansion' of studies that followed. The references support decisions the author has made about important trends in the body of previous work on multilingual advertising.  

When I first started doing this exercise, I was surprised that the students often found it a bit difficult.  I had designed it as a straightforward warm-up question, as the answers seemed obvious to me.  But the students did not find it straightforward or obvious -- it took some time for them to answer, and even then the responses were hesitant.  Why was this? I have noticed over the years that block references like this are typically absent from student work. Quotes from single sources are cited one at a time in support of claims made and the references are given, but I almost never see multiple sources in a block in student work, either at undergraduate or MA level.  

As a result of doing this exercise with multiple groups of students, I now have some insight as to why this might be.  Block references such as those Kuppens has given represent the results of synthesizing -- reading a body of work, coming to grips with what each study says, looking for patterns that hold across the whole body of work, and then naming those patterns in a relevant way -- deciding, that is, why they matter to the argument you are making.  It is in the noticing and naming of patterns in previous work that your original, critical voice as an academic author begins to emerge.  It is in and through this process that you offer your synthesis of how the multitude of previous papers fit together, what all the previous findings ultimately add up to.  The students, accustomed to finding quotes that confirmed what they already believed, were not practiced in this kind of synthesis, and so they did not notice syntheses in the articles they were reading, and because they did not notice them, what they read did not inform their own writing skills.  I realized that if I wanted my students to be able to write actual literature reviews in their literature review chapters, as opposed to serial descriptions of studies they had read, I would need to teach them how to synthesize -- which is why I started developing these reverse engineering exercises.  It is curious that the skill of synthesizing does not have more prominence in research methods curricula, at least as far as I am aware, and it remains mysterious in how-to guides.  Does anyone else find that?  Let me know in the comments 😊