NOIRLab TAC Process Updates for Semester 2022B: Introducing the 2-stage Review Process

By: Verne V. Smith, Dara Norman, Nicole van der Bliek, Mia Hartman, Chadd Myers, Alfredo Zenteno

The NOIRLab 2-stage review will be implemented as part of the Time Allocation Committee (TAC) process for semester 2022B, the deadline of which will be 31 March 2022. The first stage of the NOIRLab 2-stage review is very similar to dual anonymous procedures, such as those used by STScI and the Gemini Fast-Turnaround program, with PI and co-I names separated from the proposal text at the time of the review. There are also requirements for proposal scientific content to be anonymized and for information about proposing teams to be submitted separately from the scientific text. Initial proposal grading is done in a dual anonymous fashion, while in the NOIRLab process, the second stage of the review allows for the opportunity to make adjustments in the ranked lists based on considerations concerning the proposing team and their demographics. These considerations can be taken into account in the second stage of the review in iterating on a final proposal ranking after the initial scientific merit review is completed.

NOIRLab is responsible for allocating observing time on a number of ground-based telescopes and facilities, which include the following for semester 2022A:

• the U.S. share of time on Gemini North (8.1 meters on Maunakea, including exchange time with the Subaru 8.2-meter),
• the U.S. share of time on Gemini South (8.1 meters on Cerro Pachón),
• Keck I and II (10 meters on Maunakea via the NSF’s MSIP program),
• Víctor M. Blanco 4-meter Telescope (on Cerro Tololo)
• 4.1-meter SOAR Telescope (on Cerro Pachón)
• WIYN 3.5-meter Telescope  (on Kitt Peak via the NNEXPLORE program)
• CHARA (6x1-meter interferometer on Mt. Wilson via the NSF’s MSIP program)
• Las Cumbres Observatory (1 meter and 2 meters sited around the globe via the NSF’s MSIP program)
• SMARTS Telescopes (1.5 meters and 0.9 meter on Cerro Tololo)
• SMARTS 1.5-meter Telescope/CHIRON (on Cerro Tololo via the NNEXPLORE program)
• MINERVA-Australis (on Mt. Kent in Queensland, Australia, 5x0.7-meters feeding a precision RV spectrometer via the NNEXPLORE program)

This diverse array of telescope capabilities results in NOIRLab’s receiving ~350–400 proposals per observing semester. The review of this large number of proposals is carried out by the NOIRLab TAC, which is currently composed of eight panels (covering a broad range of astrophysical expertise), each with five panelists and a non-voting chair.

NOIRLab is committed to conducting a fair review of observing proposals, based on specific reviewer instructions and criteria, although as a necessarily human endeavor, the review process is subject to influence by implicit biases. The recognition of such influences suggests that it is prudent to explore ways to mitigate such effects and to realize that the best practices for subjective evaluation may evolve. A number of groups have examined proposal acceptance outcomes from their respective TAC processes and have concluded that unconscious gender bias may be a factor (e.g., Reid, N. 2014, PASP, 126, 923; Patat, F. 2016, arXiv:1610.00920).  The NOAO TAC examined proposal acceptance statistics over 17 semesters from 2008A to 2016A. Total submissions over these semesters were consistently around 400 proposals per semester. It was found that, except in four semesters (08B, 09B, 11A, 13B), proposals by female PIs had a consistently lower than expected acceptance rate when compared to proposals by male PIs. While recognizing that there are many factors that influence reviewer decisions about the acceptance or rejection of individual proposals, commencing with semester 2014A the TAC orientation presentation given to TAC members has included these statistics and a discussion of the role that implicit bias might play in the review process.

A number of organizations that manage proposal review processes have begun to implement a dual anonymous review of proposals, in order to lessen the role of implicit bias in these reviews. In those cases, the identity of the proposer is removed from the proposal at the time of review. The NOIRLab TAC team has participated in monitoring some of these new reviews, such as HST Cycle 26, and reviewing the process as implemented by other groups (STScI Dual Anonymous workshop, NASA’s Virtual Town Hall on Dual-Anonymous Peer Review, ESO’s Guidelines), and has developed a new review process design, but with modifications for NOIRLab-specific mission objectives; this process is referred to as a “2-stage review”. More details about the 2-stage review process, along with a document describing anonymization instructions for PIs, will be posted by 1 March 2022, when the Call for Proposals for 2022B will be posted.