From: "Arjun Dey" Sent: Monday, May 20, 2002 5:49 PM To: deveny@noao.edu Subject: shufflerun.tex \documentstyle[12pt,aaspp]{article} \tighten \input /home/dey/tex/tex.def2 \begin{document} \title{Report on the Nod-and-Shuffle T\&E Run on the Mayall Telescope - 2000 March 15 and 16, UT} \author{Arjun Dey, Roger Lynds, Tod Lauer, Buell Jannuzi, Rich Reed, Rob Seaman, Bob Marshall, Dave Mills, Jim De Veny, Tom Wolfe, Bill Ditsler, Doug Williams} \affil{\it National Optical Astronomy Observatories, 950 N. Cherry Ave., Tucson, AZ 85719} \begin{abstract} The first T\&E observing run employing the nod-and-shuffle technique with the RC Spectrograph on the KPNO 4m Mayall Telescope was successful. Using this technique, we were able to achieve accurate subtraction of strong telluric lines (as well as the fringing in the red) to better than XX\%, thus reaching the shot noise limit for spectroscopy. The overhead on the mode is large, but the new scientific capabilities enabled (increasing target density on the mask, improving sky subtraction for faint targets, spectroscopy of diffuse emission) more than compensate for the overhead. We suggest some ways of reducing the overhead. A second T\&E run in fall will be required to conduct further tests of the system before it may be declared available for general use. \end{abstract} \section{Introduction} The Nod-and-Shuffle spectroscopic observing mode was first brought to our attention by Karl Glazebrook who helped implement this mode on the LDSS++ spectrograph at the Anglo Australian Observatory (Glazebrook et al. 1998, AAO Newsletter, 84, 9; Glazebrook 1998, AAO Newsletter, 87, 11). The technique involves shuffling the charge on the CCD array (without reading it out) synchronized with nodding the telescope between object and sky in order to measure the sky nearly simultaneously with the object measurements, through the same slitlets and on the same pixels.  Subtraction of the near-simultaneous sky spectra from the object spectra results in excellent sky and fringe subtraction (even through ragged slitlets), allowing for spectroscopy near the shot-noise limit of the sky. The penalty in this mode is two-fold: the noise in the sky is increased by $\sqrt{2}$ and the observing time overhead is larger by  the factor of 2 due to the extra time spent observing the sky as well as by the time spent offsetting the telescope. The nod-and-shuffle (hereafter N\&S) mode necessitates a successful coupling between the CCD control software, the data acquisition software, and the telescope control software. In this document, we describe the modifications made to the CCD control software, ICE, and the telescope offset control software in order to implement the nod-and-shuffle mode at the Mayall and the tests carried out during the run.  We then describe possible ways in which the overhead may be reduced and outline tasks that need to be carried out before our next T\&E run. We conclude by briefly describing 3 operational modes that would greatly enhance the scientific capability of the Mayall. \section{Modifications to the Control Systems} \subsection{CCD Microcode} The microcode for the standard KPNO CCD 2901 controllers already support the feature of being able to shift charge both toward and away from the readout amplifier.  No changes were necessary to this software (maintained by OUV). Instead, changes were made to the ICE interface to enable the nod-\&-shuffle mode. \subsection{ICE User Interface} The data acquisition package (ICE at KPNO) must synchronize and manage both the instrument control (camera shutter operation), the charge shifting on the CCD, and the telescope nodding. The mountain programming group (MPG) implemented a modified version of ICE to enable the N\&S mode.  Several possible designs were discussed between the MPG and IRAF programmers, and the final choice was to fully implement a TCS--level offset function. The IRAF group (primarily Rob Seaman) was responsible for modifications to the ICE CCD data acquisition software.  The nodding/charge shuffling exposure mode requires that only the middle third of the chip be unmasked.  After each object or sky subexposure is taken, the shutter must be closed, the charge shift to one side or the other and the telescope nodded to the other offset position.  This is repeated through many exposures, back-and-forth. Given the requirement to repeat this on time scales of a minute or quicker for an hour or more at a time for each exposure, any successful implementation of charge shuffling must parametrize the most useful exposure parameters while hiding the complexity of the multiple offset exposures from the observers and the telescope operator.  New parameters added to ICE include the number of nods to include in each exposure and the number of pixels to shift for each exposure.  It is now possible to turn the N\&S mode on and off at will and to easily change the parameters with each new exposure. In addition, since preferred NOAO observing recommendations for charge shuffling are still being evaluated, access was provided to a variety of different nodding patterns (a simple `alternate' ABAB object/sky pattern versus a `bracket' pattern that begins and ends with a half-length sky subexposure; see below).  A second evaluation feature allows the on-object and on-sky exposure times to differ.  Whether these options are preserved as user parameters or are rather implemented as hardwired defaults will depend on scientific evaluation of their utility.  The added complexity of the shuffling mode also required revamping various abort procedures and other such infrastructure and the addition of several new keywords to the image headers (see table~\ref{irafheader}. A fully supported implementation of charge shuffling as a KPNO facility class instrument will require further discussions and work on both observing run preparation (to better generate slit masks) and the data reduction facilities for these new image types. \subsection{Telescope Offset Control} The basic Leaky guider software was not altered. However, Bob Marshall and Dave Mills did make changes to two routines: `PFCOM' (the TCS GWC interface) and `XTCS' (the TCS GUI).  The commands added to `PFCOM' were: \begin{tabular}{lcl}     nod-object     & --- & move the telescope to the "nod" object position\\     nod-sky        & --- & move the telescope to the "nod" sky position\\     nod-offsets?   & --- & return the values of the "nod" sky offsets\\     nod-test \{N\} & --- & move between the sky and object positions N times\\ \end{tabular} \noindent ICE uses `PFCOM' to communicate to the TCS and other subsystems. A new inspector window to setup for nodding was added to the TCS GUI. The operator uses this window to define the object and sky positions. The GUI publishes the sky offset values as GWC variables.  Also, the GUI now responds to the GWC commands for `nod-object' and `nod-sky' by turning off the guider, moving the telescope and then turning the guider back on. \section{Description of the Nod and Shuffle Mode} The N\&S mode involves synchronous nodding by the telescope and shuffling of charge on the CCD. In its current implementation at the Mayall, two guide stars are required: one for the object position and one for the off (or sky) position.  Since both the on and off positions are guided, the most stable guiding results when both stars are of comparable magnitude. The telescope is offset between these two on and off positions in such a way that both guide stars appear in the exact same position on the guide probe --- i.e., the relative position of the guide probe and the spectrograph slitmask are unchanged between the on and off positions. We experimented with two observation modes, which we called `BRACKET' and `ALTERNATE' respectively. In the BRACKET mode, the sequence of on and off observations are symmetric, beginning and ending with a ${1\over{2}}$ sky observation (i.e., the sequence is ${1\over{2}}$sky - object - sky - object - sky - ... - sky - object - sky - object - ${1\over{2}}$sky).  In the ALTERNATE mode, the sequence simply alternates between object and sky:  object - sky - object - sky - ... - sky - object - sky. We conducted three tests comparing the two modes, and found that in all cases the BRACKET mode resulted in slightly better sky subtraction that the ALTERNATE mode. The BRACKET mode observation takes place as follows. First, the telescope is set up on the on (object) position and the observation is begun. The array is reset and ICE sends a command to the telescope controller to offset to the sky position.  The guiding is stopped, the telescope offsets, the guiding is turned back on, and the new guide star is brought into the center of the guide probe reticle. As soon as the guiding is turned back on, there is a 2 sec wait time, and the telescope control computer then sends a command to ICE. ICE opens the shutter and begins exposing. When half the exposure time is elapsed, the shutter is closed, and ICE sends a command to offset the telescope to the object position. The guiding is turned off, and the telescope reverses the offset, turns on the guider, waits 2 seconds (while it is reacquiring the guide star) and then returns a command to ICE to resume the exposure this time on the object. ICE continues with a full exposure at the object position, and then repeats the offset to sky, where it continues with a full exposure at the sky position, and so on. The bracket mode ends with a half-exposure at the sky position, the shutter is closed, the telescope is offset back to the object position and the CCD array is read out. Starting the nod sequence on either the sky position or the object position works. The BRACKET sequence always ensures that the first exposure is on sky whether or not the telescope is pointed to object or sky.  \begin{figure} \plotfiddle{raw_full.eps}{5in}{-90}{80}{80}{-300}{450} \caption{The raw CCD frame produced by the Nod-and-Shuffle mode. The central set of spectra are the object spectra; the lower set, the sky spectra recorded on the same pixels as the object spectra; and the blank region at the top, the storage region for the object spectra while the sky spectra are being recorded.} \label{raw_full} \end{figure} \begin{figure} \plotfiddle{raw_sub.eps}{6.3in}{0}{80}{80}{-250}{-25} \caption{The upper panel shows a subsection of the raw CCD frame produced by the Nod-and-Shuffle mode. The lower panel shows the result of simply shifting the image by the shuffle offset and subtracting the shifted version from the original.} \label{raw_sub} \end{figure} \begin{figure} \plotfiddle{o1sub.eps}{4in}{0}{100}{100}{-290}{-300} \caption{The upper panel shows a single row of the raw CCD frame in a region containing the strong [OI]$\lambda$5577 and NaI$\lambda$5893 telluric emission lines. The lower panel shows the same region in the (unflatfielded) sky subtracted spectrum. The $\sqrt{2}\sigma$ Poisson noise from the sky is overplotted in red in the lower panel.} \label{o1sub} \end{figure} \begin{figure} \plotfiddle{ohsub.eps}{4in}{0}{100}{100}{-290}{-300} \caption{The upper panel shows a single row of the raw CCD frame in a region of strong OH telluric emission ($\approx 7180-8340$\AA). The lower panel shows the same region in the (unflatfielded) sky subtracted spectrum. The $\sqrt{2}\sigma$ Poisson noise from the sky is overplotted in red in the lower panel.} \label{ohsub} \end{figure} \section{Results of Tests} All the tests carried out in March 2000 made use of the RC Spectrograph configured with the BL181 grating set up to cover the wavelength region $\lambda\lambda$5000-9500\AA, and the GG495 order sorting filter. The spectra show strong defocussing (i.e., the pupil is noticeable) beyond $\approx$8700\AA. \subsection{Sky Subtraction} The final CCD frame consists of two sets of spectra, one set taken at the object position and one set taken at the sky position (figure~\ref{raw_full}).  The object and sky spectra are separated on the CCD image by the number of rows equal to the pixel offset used for the shuffle. We obtained data using a 700 pixel shuffle (roughly 1/3 of the total size of the array). Simply shifting the frame by 700 pixels and subtracting the shifted frame from the original data results in excellent sky subtraction, as shown in figure~\ref{raw_sub}. Figures~\ref{o1sub} and \ref{ohsub} show the effect of the N\&S subtraction on a single row in two different wavelength regions. It can be seen that subtraction is possible to the shot-noise limit of the sky, and that the residuals of the strong emission lines do not show strong `derivatives' that generally result from standard background model subtraction techniques. Experimentation with individual nod exposures of 15 sec, 30 sec, 60 sec, 120 sec, and 180 sec, suggests (as one might expect) that the shorter times sample sky variations better and result in better sky subtraction (figure~\ref{3min_30sec_comp}).  In general, we found that individual nod exposures shorter than 1 min are preferable; nod exposures as long as 3 min work, but the OH removal leaves some residuals, suggesting that the lines vary on timescales $\simgt$1min. Table~\ref{short_and_long} shows sample statistics for two BRACKET mode observations: a `short' nod exposure (30 sec $\times$ 30 nods) and a `long' nod exposure (180 sec $\times$ 5 nods). Although the statistics are limited, it can be seen that the mean sky value is closer to zero for the `short' nod compared to the `long' nod. However, the RMS values appear to be larger for the `short' nod exposure. The reason for this is unclear at present: it could be due to noise introduced by the shuffling (the `short' nod exposure has 6 times as many nods as the `long' nod exposure) and partly due to the longer total integration time (i.e., the time between the start of the observation and the readout) for the `short' nod exposure (2759 sec for `short' versus 2094 sec for `long'). A comparison of the histogram of counts in two frames: \begin{center} \begin{table} \centering \begin{tabular}{|c|c|c|c|} \hline                    & storage region & inbetween obj\&sky & bottom of frame\\ \hline   number of pixels    &  419211     &      209500        &      90400    \\ a0077 (180s x 5 nods):& 0.45+/-1.87 &      2.93+/-2.29   &   3.05+/-2.25 \\ a0078 (30s x 30 nods):& 0.49+/-1.81 &      4.38+/-2.56   &   4.09+/-2.51 \\ \hline \end{tabular} \end{table} \end{center} \begin{figure} \plottwo{histogram.ps}{histogram2.ps} \caption{Histograms of the bias-subtracted counts in three regions of the charge-shuffled CCD frame. The top, middle and bottom panels show the count distribution in the top (storage) region, middle (inbetween the object and sky spectra) region and bottom (below the sky spectra) region. The black and red lines show the count distributions in the frames a0077.fits (which had 180~sec~$\times$~5~nods) and a0078.fits (which had 30~sec~$\times$~30~nods) respectively.  } \label{histogram} \end{figure} \begin{center} \begin{table}[t] \centerline{Comparison of Short and Long Nod Exposures} \medskip \centering \begin{tabular}{lcccccccc} \hline % %                                Short                         Long %                   -----------------------------  -----------------------------   Region  &  Mean & RMS &Median& Mode & Mean& RMS &Median&Mode\\ \hline 900:1180,225:245 & 0.721& 10.60&  0.565&  0.448& 1.522&  9.23& 0.717&  2.636\\ %900:1100,225:245 & 0.744& 10.70&  0.587&  5.995& 1.451&  9.21& 0.521& -1.457\\ 700:1000,225:245 & 0.620& 10.57&  0.332&  0.348& 1.402&  9.66& 0.800& -0.982\\ 900:1100,18:38   & 0.332& 11.38& -0.252& -3.818& 1.256& 10.12& 1.198& -0.111\\ 500:900,18:38    &-0.067& 11.16& -0.286&  4.544& 0.870& 10.07& 0.591& -1.014\\ %850:1200,18:38   & 0.342& 11.09& -0.143& -4.545& 1.253&  9.67& 0.924&  0.675\\ 850:1200,137:156 & 1.628& 11.05&  1.195& -3.791& 1.878&  9.80& 1.551&  0.285\\ 1000:1350,137:156& 1.322& 10.32&  1.366&  1.620& 1.846&  8.97& 1.053&  0.177\\ \hline \end{tabular} \end{table} \end{center} Comparison of BRACKET and ALTERNATE modes. \subsection{Overhead for the Nod-and-Shuffle Mode} Currently, the overhead on the nod+shuffle mode is large: the overhead on each nod (defined as a exposure on target, offset to sky, an exposure on sky, and an offset back to the target position) is 30sec. That is, the total overhead added by each offset is 15 sec.  As mentioned above, for the best sky subtraction, the sky must be sampled on time scales of $\le$60 sec.  Hence, a 30 min exposure on target, broken up into 60s integrations, takes 75 min, not including the 2 min readout time of the CCD.  For shorter exposures and more nods, the overhead is proportionately increased. \section{Possible Modifications to Improve Observing Efficiency} The overhead might be reduced if (1) the settle time of the telescope is shortened, and (2) guiding is turned off when one is observing sky. The main part of the overhead is in reacquiring the guide star each time the telescope is offset.  At present, there is a built in 2 sec delay to allow for settling before the telescope tells ICE to resume integrating: this could be a variable parameter on the telescope operator's GUI. The settling time seems to be shorter for smaller offsets, and may be unnecessary for most offsets $<$2 arcmins. We tried offsets as large as 13 arcmin (with FORMAPS off) and the telescope behaved reasonable well. The settling time was longer (6 secs), but we never lost the star through 20 to-and-fro offsets. The settling time will likely be longer with FORMAPS on, since the telescope tries to readjust the mirror support after every offset, depending on the size and direction of the offset and the current position of the telescope. Throws larger than about 3 arcmin generally make FORMAPS reconfigure the mirror support. A new guider is critical, especially if one intends to guide in the off position during small offsets.  On 00mar16ut we worked through light cirrus and with a 10-day old moon, but were able to use ~15.5mag guide stars successfully as long as the seeing was stable. Many fields are likely to need fainter guide stars, and a high-throughput CCD guider capable of performing small offsets that keep the guide star within the camera field and guiding at more than one position within the field, would reduce the overhead. A more sensitive guider would also reduce the overhead in searching for a second guide star if a large throw is required by the nature of the observation. \section{Modifications Needed to Make this a General User Mode} \begin{itemize} \item {The N\&S mode requires a `Neat Stop'  --- i.e., an interrupt command that enables the observation to stop after the current complete nod. In the case of a BRACKET mode observation, the last sky observation must have half the exposure time length.} \item {Related to the above, We need a way of redefining the number of nods in mid-exposure, either extending / curtailing the total number of nods. } \item {A new or modified slitmask design program that \begin{itemize} \item[(1)]{packs the slits together,} \item[(2)]{maximizes the slit length to fill empty space on the masks, } \item[(3)]{extends the slits on the end to fill in the full decker length} \end{itemize} } \item {The FITS header keywords need to contain all of the data that is normally reported and also include the relevant information describing the N\&S mode (i.e., the N\&S mode (NODPAT), the exposure time per nod (NODT0001), the exposure time on sky if this is different from the object exposure time (NODT0002), the total number of nods (NODCOUNT), the total exposure time on target (SEXP0001), the total dark time (DARKTIME or TOTTIME?), the number of shuffle rows (NODPIX). Rob Seaman has already included the following keywords shown in table~\ref{irafheader}} \item{The N\&S observing strategy would benefit from a mode in which the off (sky) position is unguided. This will likely increase the observing efficiency.} \item{One of the potential problems we noted was that the slitmask position is not repeatable - i.e., once the slitmask is moved, it does not necessarily return to its original position. This suggests that the mask should not be moved after it has been aligned on source. This is a problem that needs to be addressed independent of the N\&S implementation.} \end{itemize} %%%%%%%%%%%%%%%%% %% TABLE 1 %%%%%%%%%%%%%%%%% \begin{center} \begin{table} \centering \centerline{IRAF HEADER KEYWORDS ADDED TO FITS HEADER} \medskip \begin{tabular}{lcrrl} \hline \hline DARKTIME&=&             4425.689&  /&  CCD dark time (sec) \\ NSUBEXPS&=&                    2&  /&  Number of subexposures \\ OBJT0001&=& 'object            '&  /&  Pointing type \\ SEXPOOO1&=&                1800.&  /&  Object exposure 30x60.0 (sec) \\ OBJT0002&=& 'sky               '&  /&  Pointing type \\ SEXPOOO2&=&                1800.&  /&  Sky exposure 30x60.0 (sec) \\ NODPAT  &=& 'bracket           '&  /&  Nod pattern for charge shuffling \\ NODPIX  &=&                  700&  /&  Number of pixels to shuffle \\ NODCOUNT&=&                   30&  /&  Number of full nods per exposure \\ NODT0001&=&                  60.&  /&  Object exposure per nod (sec) \\ NODT0002&=&                  60.&  /&  Sky exposure per nod (sec) \\ INITTIME&=&                1.593&  /&  software/hardware initialization (sec) \\ PREPTIME&=&                7.127&  /&  CCD preparation time (sec) \\ READTIME&=&              122.047&  /&  CCD readout time (sec) \\ TOTTIME &=&             4556.456&  /&  total time spent in task (sec) \\ \hline \hline \end{tabular} \label{irafheader} \end{table} \end{center} Things to do in the future: \begin{itemize} \item{test with a long slit (wide long slit as well)} \item{test with a standard star on one of the slitlets (or on the long       slit) for good relative spectrophotometry calibration} \item{test with about 80 microslits} \item{measure the CTE using a line lamp and a long slit. There is some       indication that the CTE results in higher noise over many shuffles,       although i am not sure of this yet.} \end{itemize} Need a mode in which we do not guide in the off position. Main thing that has to be fixed then is the handshake between the telescope and ICE. \vfill\eject \begin{appendix} \section{Observing procedure:} \noindent The observing procedure using this mode consists of the following steps: \begin{itemize} \item[{\bf (1)}]{go to setup field - align mask on setup stars} \item[{\bf (2)}]{go to target field - align check star very crudely on location          of check star slitlet} \item[{\bf (3)}]{zero the coordinates and offsets} \item[{\bf (4)}]{find the preselected guide star } \item[{\bf (5)}]{find offset guide star  by moving telescope, not guide probe          (check the field if needed to make sure it's not crowded). Record          offsets to guide star. (This step would be modified if one does not          intend to guide in the sky position.)} \item[{\bf (6)}]{return to object field position and align the slitmask by          peaking up on the check star. One could leave the guiding on for this          procedure and simply peak up using the acquisition TV and `offset          inspector'.  Once aligned, record this as the object position by          setting object position on `nod' gui control} \item[{\bf (7)}]{move to `sky' guide star and center the `sky' guide star in          the guide box by moving the telescope, not the guide probe} \item[{\bf (8)}]{set `sky' position on nod gui control} \item[{\bf (9)}]{test the offsets by using `move to object' and `move to sky'          on the nod gui} \item[{\bf (10)}]{start exposure} \end{itemize} \end{appendix} \end{document}