XDGNIRS Pipeline - Science Verification

The GNIRS XD reduction pipeline has been updated and is ready for use! We have made the pipeline available to the public via a GitLab repository and developed a cookbook describing what the pipeline does at each step along the reduction. The work involved in this project consists of the following:

• Fixed the existing pipeline version on the DR forum and updated it to be compatible with Python 3.7.11.
• Verified that the code complies with the latest version of the Gemini IRAF Virtual Machine.
• Tested the final version of the pipeline on 15 sources (2 stellar, 13 extragalactic) with data obtained from the Gemini Observation Archive and compared the reduced products to published spectra.
• Created a GitLab repository that contains the code, examples, and basic instructions on how to run it.
• Created a Cookbook in READtheDocs that contains an updated version of Rachel Mason’s original instruction manual for the pipeline.

Check out the GitLab repository and the updated documentation.

Frames taken with the GNIRS cross-dispersed mode. Credit: Rachel Mason / Gemini Observatory.

The project began by running the pipeline as is and recording all the error messages we received. Once we could not proceed any further, we began addressing the errors. The errors ranged from incorrect syntax, defunct Python libraries, or incorrectly defined variables. The pipeline contains twenty-eight Python and three csh files, so this process involved going through each file line by line to identify existing issues or incorrect syntax and ensuring that each file called is in the correct order. After successfully addressing all the error messages, we explored the pipeline’s autonomous and interactive settings to ensure they were operational.

Testing the autonomous settings involved providing the pipeline with data and verifying that it ran from start to finish without errors or any additional input from the user. If the pipeline failed, we would reperform the failed step in interactive mode to identify if there was anything wrong with the data. If nothing were wrong with the data, we would inspect the code to uncover what was happening. For the most part, the code could operate independently, but two steps failed for some data sets: performing NSTRANSFORM or S-distortion correction. Fortunately, these two steps only failed with data taken during a poor weather program, which explains why the pipeline struggled to perform these steps. We found the pipeline runs smoothly for data not taken during poor weather.

After successfully running the pipeline, we tested the pipeline on data from the Gemini Observatory Archive. In particular, we chose the brown dwarf Wise0410 because Leggett et al. 2017 used GNIRS to observe the source and included a reduced spectrum in their paper. Running the data we obtained through the pipeline resulted in a spectrum resembling the published version without needing to manually interact with the pipeline after running it. A few months into the project, a new version of the Virtual Machine was released. We downloaded the new machine, uploaded the pipeline, and reran our previous tests to ensure that the pipeline still works. With only minor edits, we successfully ran the pipeline on the latest version of the virtual machine and achieved the same results as our previous Wise0410 test.

A one-to-one comparison of the spectra reduced with the updated pipeline (blue) and the published spectra (red) that was included in Mason et al. (2015).

After verifying that the updated pipeline works on the new virtual machine, we teamed up with the Brazilian NGO head, Alberto Ardila, who has abundant experience using the GNIRS instrument. Alberto compiled a list of eight galaxies whose spectra he published; three have good quality, three have passable quality, and the last two have bad quality, as denoted on the Gemini Observation Archive. We successfully ran all eight examples through our pipeline and then met with Alberto again to discuss the pipeline’s performance. We found that all the emission/absorption lines identified in the published spectra were also present in our spectra. We even found that the pipeline handled data marked ‘bad quality’ without issues and that their reduced spectra closely resembled the published versions.

Once we verified that the pipeline produces science-level spectra, we set up a GitLab repository that contains the finished pipeline and fifteen examples (thirteen extragalactic and two stellar). Most examples contain the data we used in the reduction, the reduced spectrum, and a published version of the spectrum so that users can compare their results to ours. Some examples contain the content mentioned above and additional information on how to edit the pipeline’s defaults. For example, NGC4736 is the galaxy belonging to a poor weather condition program. Without manual intervention, the pipeline will give an error message when trying to fit emission lines and stop running, so we included additional guidance on how to adjust the pipeline so that the NSTRANSFORM step becomes interactive, as well as instructions on how to use the interactive prompt.

Results of testing the GNIRS XD pipeline on different computers. We found that when given the same dataset, the pipeline reproduced identical results for all six sources.

With the GitLab repository established, we moved on to testing the pipeline on other machines. We had many successful tests, but we wanted to ensure that users whose computers may differ from ours can achieve the same results. We ran the pipeline on six data sets and found that the pipeline could reproduce nearly identical spectra. We found that there are no visible differences between the two spectra.

In addition to updating the pipeline’s code and creating the GitLab repository, we took Rachel Mason’s original pipeline manual and created an updated READtheDocs version. This new manual contains instructions for the new features we added in this version, including an updated ability to query SIMBAD for a source’s redshift and the option to change the final spectrum’s CRTYPE between LINEAR and MULTISPEC. The Cookbook also includes a section that displays all the acceptable inputs that either allow the user to manually perform specific tasks or perform additional tasks that the pipeline does not perform by default, such as the ‘shift_to_rest’ command, which obtains the source’s redshift from SIMBAD and shifts the reduced spectrum into the rest frame. Besides documenting the new features, we removed defunct features, such as the ability to choose the GemIRAF version or the Ureka installation of IRAF/PyRAF/Python.

GNIRS usage over time at Gemini North

GNIRS usage over time. Data taken from the Gemini Observatory Archive.