Role of panel meetings in the peer review process
We rely on peer review panel meetings to judge the relative quality of research proposals competing for funding. The review panels are responsible for placing the proposals before it in a funding priority order. From this list, the final decision is made on funding.
Role of the panel meeting
Proposals put to the panel meeting have already been independently reviewed by experts nominated by the applicants and by EPSRC. The panel does not re-review proposals, nor does it make detailed study of proposal costings. Instead, it orders the proposals based on a relative judgment of the reviewer reports, taking account of all the assessment criteria. Research quality will always be the pre-eminent criterion in this respect, while national importance will also be a major factor in these considerations.
To assist in the prioritisation process, panel members will have been asked before the meeting to act as introducers for individual proposals. Where possible, these assignments will align with members' expertise but this may not always be possible.
The panel member's role
Panels have around ten members whose expertise reflects the area of the research proposals being considered. Membership will mostly be taken from the EPSRC college, comprising both academics and industrialists. The chair will normally be an experienced panel member, often having attended the previous panel meeting. Typically around 30% of the panel membership is drawn from previous panels to give continuity to the decision-making process.
Each panel member contributes to the panel meeting, helping it achieve its objective of ranking proposals based on their research quality and national importance, as well as Pathways to Impact, applicant(s) ability and resources and management.
The panel draws on the comments of the expert reviewers and on the applicants' responses to the reviewers' comments. The panel does not review proposals; panel members are appointed to represent the collective views of the expert reviewers and to bring the benefits of their general experience in science and engineering research. Panel members are asked to take a broad view, which covers the breadth of research included within the panel's remit.
EPSRC is placing increasing emphasis on funding and facilitating multidisciplinary/interdisciplinary research opportunities. Panel members should be aware that from time to time, they will be asked to consider proposals which cut across the remit of more than one of the council's themes. Individual reviewers may not be in a position to assess the proposal as a whole, but may choose to provide comments on the part of the proposal that relates directly to their own area of expertise.
Panel members are paid an attendance fee except for those working for a government agency or receiving a grant in aid.
What happens at peer review panels
Before the panel meeting
Meeting paperwork is added to the EPSRC Peer Review Extranet at least two weekends before the meeting. Each panel member is assigned as an introducer for a number of proposals and should prepare to act in that capacity. EPSRC will allocate introducers a number of research area rationales to read in detail.
EPSRC also asks members to submit their initial proposal gradings two to three days before the meeting. This enables EPSRC and the panel to reflect on the scores prior to the meeting and help focus discussion during the meeting.
The running order for the meeting is based on the collected pre-scores of the introducers, taking the average of the three introducer's scores. In the case of a large difference in scores, the higher will be used as its indicative position. Introducers are of course permitted to change their scoring during the panel meeting and the panel must agree the final score for ranking.
Each proposal may be considered in this running order. The order of the proposals should then facilitate the discussion, enabling comparison of proposals of a similar quality.
What happens at a panel meeting?
The EPSRC officer leading the meeting will start with an introductory briefing, highlighting important issues in the papers which have already been provided to the panel members. Any queries are dealt with.
The EPSRC theme lead will then give a contextual briefing presentation to set the proposals in the wider context of the EPSRC portfolio.
The chair starts with brief round table introductions. The panel then considers each proposal in turn. Every proposal is allocated to three introducers responsible for leading a discussion based on the input from expert reviewers (see role of introducers section).
The chair then invites other panel members to comment on the proposal and recommended score, before any decision on a final grade is agreed. During the meeting the panel will review the scoring of the proposals to accommodate any alterations in the light of later discussions. At the end the panel will be asked to agree the ordering of the proposals, including prioritising those with the same grade and identify the cut-off point on the list, below which it believes funding would not be appropriate.
The resulting output of the meeting is a rank ordered list that is signed by the chair as a formal record of the meeting.
Role of introducers at peer review panels
For each proposal, three panel members (the 'introducers') will have been nominated to act as summarisers and to lead the discussion.
The first introducer will initiate discussion, drawing on the reviewers' reports and principle investigator's (PI) response, taking care not to re-review. The second and third introducers comment on any differences of opinion they may have from the first speaker. The first two introducers should focus on quality as the primary criterion and the third introducer on the major secondary criteria of national importance. Introducers' remarks should also include consideration of the proposal's Pathways to Impact, applicant(s) ability, resources and management in the light of the reviewers' comments.
When summarising the proposal, introducers should present the collective views of the reviewers. Each introducer should also complete an introducer's assessment report form for each proposal they introduce.
Introducers should take into consideration the PI's response to the reviewers. EPSRC makes every effort to feed back reviewer comments to the PI before the meeting. In some cases, however, a late reviewer report will not have been seen by the applicant but should still be considered, whilst bearing in mind the fact that the PI will not have been able to respond to them.
Having introduced a particular proposal, the introducers will be asked to assign it a grade from 1-10 (10 high). This is to help rank proposals in priority order for funding. The marking scale has no other value outside the individual panel meeting and is not communicated to the applicant. An overall grade for the proposal is agreed by the panel.
This overall grade will then be used in determining the proposal's relative ranking. Once compiled, the panel will be asked to review the overall rankings, to revise them if necessary, before agreeing a final priority list.
The introducer's report form is intended as an ‘aide memoire' during discussions at panel meetings. It is collected at the end of the review meeting and often used to help provide feedback to the applicant.
When acting as an introducer, a panel member should refer to the form to highlight:
- Any important issues identified by the reviewers and whether the PI has addressed them.
- Any discrepancies between reviewers' comments.
- Any response from the applicant.
- Any comments on the general level of resource requested.
- Any specific feedback to the applicant.
- Whether the reviewers' comments were of sufficient quality to help in the decision.
- If the reviewer was an appropriate choice.