Literature Review
Continuing our series of articles on research methods, presented in the article “How to choose the most suitable research method for your study?”, let’s start with one of the research methods most applied by the academic community, the Systematic Literature Review.
A systematic literature review is one of the types of literature review that aims to summarize all existing information about a phenomenon in an impartial and complete manner.
In contrast to the non-systematic process, the systematic review is done in a formal and meticulous manner. This means that we must follow the plan defined in the review protocol which, among other things, establishes a well-defined sequence of steps.
Due to this thoroughness, one of the advantages of a systematic review of the literature is that it allows other researchers to make future updates to the review, if they follow the same set of steps established in the protocol.
We can summarize the stages of a literature review into three phases: Planning the Review, Conducting the Review, and Reporting the Review. We will detail them below.
Planning the Literature Review
The planning phase must be carried out with great care, as any mistakes that happen will impact the other phases and your review may be totally compromised. The main output of this phase is the review protocol.
To carry out the planning you must follow the steps below:
- Identify the need for literature review. For this you have to reflect and try to answer the following questions:
- Is the research area mature enough, with a lot of published studies?
- Are there any recent reviews on the same topic?
- Define the research questions you intend to answer with the review.
- A critical part of the review, these questions are used to build strings (keywords) to search for articles in the bibliographic databases.
- They determine what information will be extracted from the articles that will be analyzed, which we call primary studies.
- Once defined in the planning phase, these issues cannot be changed in the following phases.
- We recommend that an expert on the subject of the review review the issues still in the protocol construction phase.
- Previous literature reviews in the same area, or in similar areas, may help to define these issues.
- Create the review protocol
- In this protocol, the entire step-by-step procedure for reviewing is defined. The steps must be well defined and followed to the letter to reduce the possibility of errors in the execution of the review.
- All review authors must participate in the development of the protocol.
- It is important to simulate the protocol execution, which will help to find possible errors in each of the steps. During simulation, survey questions can still be changed.
- Validate the protocol
- A fundamental step that must be done with the help of the advisor or specialist in the area.
- This step is very important as this protocol will determine how all research will be conducted.
- IMPORTANT: You should only proceed to the next phase when you are sure your protocol is ready. Repeat the planning steps as many times as necessary to be sure. Remember that any error in planning could compromise your entire research.
Conducting the Literature Review
The conduct of the review must strictly follow the protocol that was developed. Below are the steps to conduct the review.
Search for primary studies
- There are three strategies and search for primary studies, they are:
- Manual Search: when we visit the websites and/or conference proceedings and journals in search of articles on the researched topic.
- Automatic Search: when we visit digital libraries to search for articles according to a certain keyword or set of them, which we call search strings. Example of digital libraries: Google Scholar, Citeseer Library, Scopus, Science Direct, PubMed, Scielo, CAPES Journal Portal, BVS, etc.
- Snow-Balling: when we analyze the reference list of articles in search of new studies
.
- Search strategies can be used individually or in combination. To ensure the completeness of your review, we advise that you combine the three strategies and utilize more than one digital library.
- IMPORTANT: Each digital library’s search engine works in a specific way. In this way, you will have to elaborate your search strings according to each of the libraries.
Select primary studies:
- At this stage you will have to select which studies will be considered in your review. That is, which ones will be analyzed.
- Depending on the volume of studies captured in the previous step, we look at the title and abstract of these studies in order to eliminate irrelevant studies. It is common in some areas of research that the abstract (or abstract) has little or poor quality information. In these cases, it is necessary to read the introduction and conclusion.
- After this first filter, the full text of each article must be considered. For this, we apply inclusion and exclusion criteria defined in the protocol to obtain the final list of primary studies that will be considered in the review.
Assess the quality of primary studies
- It is important to assess the quality of primary studies to support the inclusion/exclusion process and assign weights to specific studies so that they can be considered in the data synthesis phase.
- There is no universal definition of what a quality study is, but the literature suggests that these studies have little or no bias, and that internal and external validity is maximized.
- Quality assessment is widely used in healthcare reviews.
Extract information from primary studies
- We must identify and capture information from each primary study. For this, the approach to be used is to answer each of the research questions with the information contained in these studies.
- To reduce the review bias, a data extraction form must be defined and evaluated in the protocol review phase.
- An interesting approach for when we have a large amount of primary studies is to have a researcher to extract data and another to check the data that has already been extracted.
Perform information synthesis
- Once the data has been extracted, it must be synthesized in a way that answers the research questions.
- There are several techniques for performing this synthesis, such as: narrative synthesis, meta-ethnography (meta-ethnography), grounded theory, thematic analysis/synthesis, etc.
Writing the Systematic Literature Review
Once the questions have been answered, the literature review should now be documented. That is, you will write a document about your review, which could be an article, chapter of your TCC, etc. In this context, the question arises: how will I know what to write and which sections and subsections I should consider?
The following are the main sections and subsections of a Literature Review:
- Introduction
- Related works
- Planning the review
- Identification of the topic to be researched
- Specifying Research Questions
- Construction and development of the protocol
- Protocol evaluation
- Conducting the review
- Search identification
- Selection of primary studies
- Evaluation of studies.
- Data extraction and monitoring.
- Data synthesis
- Reporting the review
- Explaining how results will be displayed
- sorting the data
- Writing the results and their interpretation
- Evaluating the results
- Threats to Study Validity
- Conclusion
- References
- Appendices
Conclusion
Finally, we recommend that you read the references and examples of systematic reviews we have selected below to learn more about this type of literature review, as this article is not intended to be a complete guide. If you have any questions, please leave us a comment below that we’ll be happy to answer.
References
Guidelines for performing Systematic Literature Reviews in Software Engineering
Systematic Review in Software Engineering
Systematic review studies: a guide for careful synthesis of the scientific evidence
