A Practical Guide to Conducting a Systematic Literature Search
When I was in graduate school, I was frequently tasked with writing a review for class or conducting a literature search as part of my RA duties. Even though I generally knew how to conduct a literature search, I didn’t know the exact, step-by-step technique I should use. After a lot of trial and error, studying the MARS and JARS standards, and a meta-analysis course taught by Dr. Dana Joseph, I have put together a how-to guide for conducting a systematic literature search.
DR. LAUREN R. LOCKLEAR, Assistant professor of management
Steps for conducting A Systematic Literature Search
1. Choose your research question. This includes the topic like “gratitude” but also the specific relationships you are interested in. For example, “the well-being outcomes of gratitude.” This will help you with search terms, inclusion criteria, etc. Don’t start your lit search without knowing your specific direction/RQ—it is very likely that your literature search will be incorrect.
2. Create/choose your search terms. These should be cultivated from prior literature reviews and from common terms in the literature. This search should be exhaustive, so use all words that are relevant. For example “gratitude” and “appreciation” and “grateful” and “thankful” and “appreciat*”
Search terms use Boolean phrasing. Become familiar with this. When you do a search OR makes it bigger whereas AND narrows the search. For example, “gratitude OR appreciation” will return all articles that have either gratitude or appreciation, but “gratitude AND appreciation” will return only articles that include both.
Become friends with the asterisk (*). In Boolean searches, it tells the search engine to pull anything that begins with the letters preceding the *. For example, the search term attitud*, will pull results from keywords attitude, attitudes, attitudinal, etc.
3. Conduct the literature search. Choose ~three electronic databases with articles and topics relevant to your content area. If you aren’t sure, look at the description of the databases. However, searching electronic databases is necessary but not sufficient. You will also need to search the following:
• Electronic databases searched (*asterisks include favorites for OB/psych research)
PsycInfo*
PsycArticles
ScienceDirect
ERIC
Web of Science/SSCI*
ABI Inform*
Academic Search Complete*
Medline
Dissertation Abstracts
Etc.
• Conference proceedings/abstracts (e.g., SIOP, AOM)
• Dissertations & Theses (ProQuest is a database for dissertations and theses)
• Hand searches of literature (can use google scholar for this)
• Listservs (OBNet, HRNet, RMNet, LDRNet, etc.)
• Relevant authors in the area (caution: may interfere with blind review)
Listservs and authors can be contacted when the review is almost complete so as not to alert others to your review
• Reference sections of articles/Review articles
• Unpublished manuscripts
• Test manuals
• Articles that have cited seminal papers on the topic
• Note: Sometimes you can HIRE a reference librarian
• Record your search as you go. Eventually, you will need to write a methods section detailing your literature search strategy. But, metas and reviews take time, so in a couple years you might not remember. DOCUMENT! Also, it is common for enough time to pass that you will need to update your literature search, so recording can help you easily update your search in the future. Record:
• ALL information for each search conducted:
• What database was searched?
• Who did the search?
• When was the search conducted?
• What time period did the search cover?
• What were the search terms (don’t just record the search terms, but also any use of “AND”, “OR”, etc.)
• What type of search term was it (e.g., keyword/abstract/title/full text)?
• How many results did the search pull?
• How many pdfs could not be located?
• All of the references for all of the papers found in a search (use database reference export functions, use the “CITE” button in Google Scholar)
• Any conversations with authors about unpublished papers
• What papers could and could not be found through interlibrary loan
• How do I do this?
• Make three excel files:
• A search term tracker (records the exact boolean phrasing and terms you searched in the databases etc)
• A search tracker (records the exact searches you completed: what, who, where, when)
• An interlibrary loan tracker (records the articles you requested through ILL because you couldn’t get them elsewhere)
4. Download your literature search results into Endnote (or other reference management software). Using the .ris export function from the electronic databases, download all articles that your searches returned. Add other articles from your searches like conference proceedings, dissertations, etc.
5. Remove all duplicates. Endnote does this automatically (tip: when you select find duplicates, Endnote wants to reconcile each one individually. If you press the X button, it will have all duplicates highlighted and you can drag the duplicates in the trash or a duplicates folder. Remember to record the number of duplicates). Once you’ve removed duplicates, you will likely have a corpus of thousands of articles. Don’t despair—the final sample that ends up being coded is typically between 2%-8% of your initial pool of articles.
6. Inclusion and Exclusion Criteria. Before you can winnow down the corpus of articles, you need to decide the boundaries of your review. To do so, you will use your inclusion/exclusion criteria. These criteria are important guidelines for what articles make it into the review and which articles do not. These criteria should be directly tied to your RQ and not be unnecessarily narrow. Your criteria might include:
constructs of interest (e.g., must include gratitude or appreciation)
participants (e.g., adults, employees, non-clinical samples)
study design (e.g., empirical, qualitative)
type of publication (e.g., peer-reviewed journal article, original primary research)
7. Excluding Articles via Title and Abstract Scan. Now that you know your inclusion criteria, you can begin to winnow your article pool. Winnowing is a two-step process. The first step includes scanning based on title and abstract only. You will use the title and abstract to determine whether you think the article should be included in your sample. Be liberal here, because step 2—the full text scan—can be more conservative. In both steps of winnowing, it is very important to stay organized because you will need to make an inclusion/exclusion flow chart of your literature review, as seen in Appelbaum et al. (2018) (called a PRISMA diagram). Therefore, you need to know how many articles you excluded at each step and for what reason. Therefore, I make folders in Endnote and sort articles into folders.
In Endnote, make a folder for the articles that meet all inclusion criteria. This will be your “keep” pile.
Make a folder for each of your unmet inclusion criteria. For example, “not empirical” or “no focal construct”
As you find articles that do not meet your criteria for one reason or another, place them into the appropriate folder. If they do meet all criteria, put them in the include/keep folder.
Note: By sorting each of the articles into folders, you will have a count of all the decision you made that you can easily integrate into your JARS literature search flowchart. Make sure you record these final numbers somewhere safe—you’ll need it for your method section later. (e.g., 300 articles excluded because they did not use employee samples).
8. Exclude Articles via Full Text Review. The title and abstract scan should shrink your pool of articles significantly, leaving (usually, in my experience) a few hundred to a thousand articles. From this step, I export the “keep” pile into a NEW endnote library and again create folders for my inclusion criteria and a keep folder. This step differs from the prior because we are going to scan the full .pdf text of the article, not just the title and abstract. This means that you need all of the .pdfs. Endnote can partially automate this process (References > Find full text). However, I usually have only a 25%-50% success rate. The other pdfs have to be retrieved the old fashion way. Download and save each of the articles into your endnote library or into a folder (preferably in the cloud so you don’t lose the articles and so you don’t take up all the space on your computer). Now that you have the full texts, review each for your inclusion criteria. Be very thorough here—are there code-able effect sizes (if your review is quantitative), are the variables of interest measured, is the population of interest studied? Doing this step well will make your life easier in the future . Again, record all inclusion and exclusion decisions for your future methods section. Once you’ve scanned the full texts of all articles, the “keep” folder holds your final body of articles that will make up your review.
9. Write up your literature search methods sections. Now that you’ve completed the search and the sorting, write up the methods section while it is fresh in your brain! Seriously, this will save future you a lot of time. Use the MARS standards to make sure you include everything needed and are fully transparent. Go ahead and create your PRISMA diagram too.
10. Start coding your articles. Now that you have your final sample of articles that will be in your review, you can start coding!
Notes: Endnote is a finicky program. Learn its intricacies so you don’t lose your work or corrupt your entire library. For example, I always save endnote libraries as an .enlp where the data are compressed with the library.
Acknowledgements: I would like to acknowledge Dr. Dana Joseph for teaching me much of the information in this document through her seminar on meta-analysis and mentorship!
References and Sources:
Meta-Analytic Reporting Standards https://apastyle.apa.org/jars/quant-table-9.pdf
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3-25. http://dx.doi.org/10.1037/amp0000191