Eleni Liapi, MD, is a scientific review officer (SRO) at the National Institutes of Health (NIH) Center for Scientific Review (CSR). In her role, Dr. Liapi ensures that applications for funding submitted to the NIH receive fair, independent, expert and timely scientific reviews, free from inappropriate influences.
Dr. Liapi spoke with IR Quarterly regarding her work at the NIH, the grant application review process and how researchers should best present their applications.
Tell us about your role.
Eleni Liapi, MD: I’m an SRO and manage the Clinical and Translational Imaging Science (CTIS) scientific review groups (commonly referred to as study sections). CTIS is one of about 175 chartered scientific review groups of CSR that meets three times a year to review grant applications. CSR’s study sections span the full range of science funded by NIH. As an SRO, I’m responsible for generating a fair, expert and unbiased scientific and technical review of applications submitted to the study section. This is the initial level of NIH peer review and is independent of funding decisions.
Many applicants would think that, with such an integral role in the NIH peer review process, I would be the person to contact for feedback and specific comments on their application. However, this is not the case. As an SRO, I must remain objective and fair to all applicants and, therefore, I do not offer any scientific opinion on submitted applications. I’m the NIH point of contact before the study section meeting and can answer questions related to procedures prior to the review meeting or questions related to study section description and guidelines. Program officials (or program officers) may be contacted after the review meeting, for specific feedback, guidance and discussion of the review outcome with applicants.
Overall, I see my role as an SRO to be an interactive mix of administrative, leadership and scientific duties, all supporting the NIH mission, aiming to improve human health and the lives of all Americans.
How do you generate an unbiased review process?
EL: I start by conducting an administrative review of submitted applications, as well as a scientific assessment of the expertise needed for the review. Then I recruit members of the scientific community who will review specific applications. In order to best identify the highest impact science, diverse perspectives are critical, so I aim to assemble a panel with the expertise needed to review the applications in hand as well as a panel that is diverse on multiple dimensions, such as career stage, geographic distribution and demographics. It is also my responsibility to train the reviewers in the NIH peer review procedures, ensure policy compliance and implementation, as well as adherence to the NIH peer review guidelines.
During the review meeting, which is conducted in accordance with the Federal Advisory Committee Act (FACA), I’m the designated federal official who manages the study section meeting and guides the reviewers to ensure that NIH policies are implemented. I also work in partnership with the chair of the study section, who oversees the discussion, to ensure fair review of each application.
After the review meeting, I release priority scores and generate summary statements for both discussed and nondiscussed applications. For discussed applications, I compose the resume and summary of discussion. The top section of the summary statement, which includes a high-level overview of the discussion, is based on my notes and reflects the basis for the priority scores assigned.
Can you walk us through the review process?
EL: From the applicants’ perspective, the NIH review process starts with the submission of their application to NIH via the grants.gov website. We highly recommend that applicants submit early—at least 2 days before the due date, providing enough time to address potential errors. There is a 2-day viewing window after submission during which errors may be fixed, but the 2-day window does not extend the deadline. If no errors are found, applications enter the eRA system for further processing at NIH. There, the first stop is at the Division of Receipt and Referral (DRR), which is within the Center for Scientific Review (CSR). Applications are screened for administrative completeness, fit to a specific NIH institute or center’s interests and mission, and subsequently assigned to a study section with the expertise to evaluate the scientific and technical merit of the application.
Applicants may suggest a specific study section assignment on the Public Health Service (PHS) Assignment Request Form, and these suggestions are considered, but the assignment is the prerogative of CSR and follows published study section guidelines. Assignments are a multistep process that also involves institutes and centers, scientific review officers and review branch chiefs, who take into account applicant requests, published guidelines and other factors such as reviewers’ conflicts.
Once an application reaches a specific study section, the SRO first performs an administrative review to determine expertise needed and recruit and assign reviewers. Each application receives critiques from three assigned reviewers, regardless of whether it is discussed or not, based on specific NIH guidelines and review criteria. Approximately the top 50% of applications are discussed. At the study section meeting, assigned reviewers present each application and their reviews and discuss aspects such as consideration of the importance of the proposed research, rigor of scientific methods and how the proposed research would improve scientific knowledge and how the field will change, should the proposed aims be achieved. Following discussion of each application, all reviewers enter a score for overall impact, based on the NIH scoring guidance. As mentioned earlier, this is the first level of NIH peer review. At this stage, the funding of applications is not discussed. Funding decisions are made at the second level of review by the director of the funding institute/center.
Within 30 days after the study section meeting, applicants receive summary statements with written critiques from assigned reviewers and, if discussed, with the SRO’s resume and summary of discussion and administrative notes of special consideration. This is when program officers can be contacted for discussion and interpretation of the review results and potential guidance. Even though program officers are the best point of contact after the summary statement is released, if anyone (applicant, review staff, program staff) has a concern about the review being unfair or biased, they may contact Dr. Gabriel Fosu, CSR’s Associate Director of Diversity and Workforce Development at g.fosu_assocdir@csr.nih.gov. CSR will investigate and, if we agree, the application will be rereviewed in the same review cycle. If CSR disagrees, the normal avenue for appeals remains available to the investigator.
The summary statement is then considered in the second level of review. At this stage, the advisory council of the funding institute will consider the study section’s recommendations and determine the relevance of the proposed research to the institute’s priorities and public health needs and make a recommendation to the director of the funding institute/center.
What are the common mistakes applicants make when it comes to the submission process?
EL: In addition to noncompliance, the number one reason for withdrawal of an application by the NIH is that a funding institute or center will not accept assignment of an application. Applicants can avoid this problem by taking the following precautions:
- Carefully read the Notice of Funding Opportunity (NOFO, previously referred to as a FOA). This contains information on the review criteria, deadlines, program and review contacts and specifically lists all participating institutes and centers. If the institute or center that they are targeting is not listed, their grant application cannot be assigned to that institute and, therefore, the application will be withdrawn.
- Talk to program officials who administer research portfolios relevant to the applicants’ research, in advance. Program officials may advise applicants, attend study sections, make funding recommendations and monitor awarded grants.
- Submit early, as mentioned above. There is a 2-day “viewing” window, but this does not extend the deadline.
- Subscribe to the guide notices. This is the most important communication vehicle for the NIH. All policy changes are announced in advance through the guide, and this is a relatively effortless method to stay current with policy. The extramural NEXUS, of the Office of Extramural Research (OER), is a great source for news on grants, policies, processes and more. The OER listservs and feeds are also a must.
What makes an application stand out?
EL: While I cannot point to any particular aspect, it is important to consider the review process from the reviewers’ perspectives. It is important that applicants be familiar with the review criteria and the broad guidance that we ask reviewers to consider. CSR frames the review by asking reviewers to consider at least two big questions: “Should it be done?” and “Can it be done?”
“Should it be done” concerns the significance and importance of the proposed research and needs to be highlighted in a way that is clear and convincing to the reviewers. “Can it be done” refers to the proposed scientific methods, the expertise of the team relevant to the proposed studies, and sufficiency of institutional resources to accomplish the work. Applicants should present the proposal with sufficient clarity and details to demonstrate the ability to perform proposed research and the rigor of the strategy.
The applications that do well clearly argue the significance of the proposed work to the field and beyond and have a highly rigorous research plan.
What resources would you recommend for learning how to navigate NIH funding?
EL: The number one resource to utilize is … the NIH itself! I mentioned earlier that the SRO is the point of contact before the review meeting. For clinical and translational imaging applications, I would encourage applicants to reach out to me to discuss whether their application may be a strong fit for the CTIS study section.
There is also a great screening tool, the CSR Assisted Referral Tool, which was developed by the NIH Center for Scientific Review (CSR) to recommend potentially appropriate study sections, based on the scientific content of a user’s grant application. The assisted referral tool uses natural language processing and large-scale machine learning technology to make recommendations. It uses indexed representations of both the application text that the user enters as well as all the grant application data used to train the models.
In addition, I would encourage applicants to browse the NIH RePORTER. This is an incredible source of information, including funded project abstracts, study section assignments and the “matchmaker” for identifying program officials. As mentioned earlier, the program officers are the point of contact for applicants at any step during this process and specifically after the review meeting.
Lastly, I’m excited to share with you a recently released and very comprehensive infographic, developed by CSR, which includes resources and programs for NIH applicants.
What is the best way for applicants to reach out for further questions and guidance?
EL: We strongly recommend that applicants contact the NIH prior to submitting an application. This can be done in several ways:
- Identify the NIH institute/center that supports research in your area, then check the institute/center’s website to determine whether your idea matches any of the IC’s high-priority research areas and obtain specific information related to the institute’s funding opportunities and specific research priorities. Note that some institutes/centers publish cleared concepts well before the funding opportunities are published. However, not all concepts become funding opportunities, and this is one reason NIH encourages you to contact a program official as early as possible in the process of applying for grant funding.
- Contact a program official at the appropriate institute or center by phone or e-mail to clarify any questions you may have, such as whether your proposed research project falls within the scope of an existing funding opportunity. The program official is the NIH official responsible for the programmatic, scientific and/or technical aspects of a grant. NIH grants management staff can provide advice on business and administrative issues. Investigators are encouraged to identify a program officer and reach out to them when at the stage of crafting specific aims to make sure it fits funding priorities of the institute they are targeting. If resubmitting, it’s also wise to talk to the program officer for guidance in addressing the summary statement.
Finally, if you have a question and can’t figure out who to speak with, feel free to contact CSR at communications@csr.nih.gov and we’ll get you to the right place. We understand that NIH is huge and complex and that, sometimes, you just need to reach a person. CSR will get you to that person.
