1. Programs & Projects
  2. Mid-Scale Observatories — Cerro Tololo Inter-American Observatory
  3. Retired Instruments
  4. ISPI
  5. ISPI Users Guide
  6. ISPI Data Reduction
  7. ISPI Data Reduction

ISPI Data Reduction

Basics

ISPI data should be flat fielded and sky (and/or "dark") subtracted. These steps can be accomplished with may standard IRAF or IDL routines. One way is to use the basic reduction tool "osiris" provided in the cirred set of IRAF scripts. A basic set of initial reduction steps would include:

  • Build flats from raw ISPI images, typically subtracting a combined "lights off" image from a combined "lights on" image. Use the cirred task med.cl (for "median combining").
  • Build a bad pixel mask from images with typical bad pixels (for example, use a lights on image and a lights off or dark image). Use the cirred task maskbad.cl.
  • Run the cirred meta script "osiris" to read in a series of images efficiently, divide each by a flat, and fix bad pixels (interpolates over bad pixels in x and y). For ISPI, choose flipy=yes. This puts N up and E to the left on an IRAF image display and is required to run the astrometric tools described below. If a bad pixel mask called "mask.fits" was created, and a flat called "flat.fits", then a typical osiris call might be:

    cl> osiris n1 n2 pre=r suf=n mask=mask.fits div=yes, dome=flat.fits

    n1 and n2 are command line params, the others can be set in the param file. Here pre is a "prefix" or image base name, suf will be the output image suffix and the images to be reduced have numbers n1 thru n2, inclusive. Suppose the prefix for input images is "r" for raw. If n1=1, and n2=3, then images r001.fits, r002.fits, r003.fits will be processed and images r001n.fits, r002n.fits, and r003n.fits will be output. Look over the cirred pages, the osiris param file, and the osiris.cl script itself for more information.

  • The flat fielded images should be sky subtracted. Separate dark subtraction is not needed as long as the images are not scaled during sky subtraction. If images are to be scaled significantly, and the dark and bias pattern is important compared to the sky level, then darks of the same exposure and coadds should be subtracted before sky subtraction (usually at the very beginning; this can be done in osiris.cl). Use the cirred script sky_sub. A typical sky_sub might be:

    cl> sky_sub 10 25 sky=kskyimage make+ sub+

    Where the image numbers work as in the osiris example. Prefixes and suffixes are set in the param file or on the command line. The two flags "make" and "sub" control how sky_sub behaves. Make will combine the indicated images into a sky frame, and sub will subtract that sky frame from the images. A set of independent skys could be combined with make+ ans sub- (to suppress subtraction) and applied to a separate set of object frames with make- (to suppress making of the sky frame) and sub+.

Astrometry

One of the most important tasks for ISPI data reduction is the assignment of a proper coordinate system and correction of image distortions. These tasks are conceptually straightforward, but often difficult to accomplish in practice (especially the latter). A preliminary set of tasks and scripts has been set up to allow for the determination of the world coordinate system keywords to be included in the image headers. A solution to higher order distortions is also available, which will allow for the registration and combination of dithered images.

Software.

The processing of ISPI images described here requires that one download and install several external packages.

  • WCSTOOLS. These are C libraries and programs by Douglas Mink at the SAO. These routines update the headers of ISPI images with FITS keywords describing the image coordinate system, its projection on the sky, etc. Install the WCSTOOLS software, and read the information about imwcs which is the program used to compute a coordinate solution and update the image headers.
  • Catalogs. WCSTOOLS needs an astrometric reference catalog. These can be existing catalogs like 2MASS or user catalogs. The best way to proceed is to download and install one or more catalogs locally. At CTIO, we have so far experimented with 2MASS, since this is very useful for near infrared observations in the Galactic plane. Other catalogs will work well for regions where the optical extinction is not high. To install the 2MASS catalog, follow the instructions given by the WCSTOOLS pages. The disk space requirement is about 40 GB.
  • A program to find image centroids in the ISPI images. This can be accomplished in IRAF (for example daofind). Other well known programs are DOPHOT, DAOPHOT, and SExtractor. Finding accurate centroids is necessary for certain options using WCSTOOLS.
  • A program to resample or interpolate images. We have used SWARP, a program by Emmanuel  Bertin.

Updating the Basic Headers for a WCS.

  • The images should be preprocessed to have a near zero rotation from some known position. For ISPI, the usual N up E left projection is easily accomplished by choosing the "flipy" flag in the osiris.cl IRAF meta script. ISPI raw images have N down and East left when displayed in IRAF. An EPOCH keyword should also be added to the header which matches the value of the ISPI EQUINOX keyword (the default is 2000.0).
  • The simplest task needed for raw ISPI images is to add appropriate world coordinate system (WCS) keywords. The program imwcs should be used to do this. There are a variety of options, but a simple call would be the following:

    > imwcs -c tmc -d t150s.dao -h 200 -n 6 -vw -p 0.303 t150s.fits

    In this example, -c tmc calls for the 2MASS catalog. The location of the catalog is specified during the build process of WCSTOOLS. -h 200 calls for 200 stars to be used in the matching between the image and catalog, and -n 6 refers to the order of the fit between image and catalog positions (6 parameters in each axis). -p 0.303 gives the initial image scale in arcseconds per pixel for ISPI; this is required since this information is not in the header. Alternately, one could add SECPIX (with a value of 0.3) to the ISPI image headers. -d t150s.dao specifies that the file "t150s.dao" be used for the input x,y positions of the stars to be matched by the catalog. This file should be sorted on brightness so the brightest stars are used. The file should be one star per line, x, y, and mag. "#'s" indicate comments and are ignored in this file, as are any fields after the first three of each line. See the IMWCS Command Line Arguments for more information. -vw sets the "verbose" flag along with a call (w flag) to write a new image (in this case t150sw.fits) which has the updated WCS. See the imwcs command line argument -o for more output options, including updating the input image header.

  • The result of IMWCS is an image updated (if desired) with a WCS which is a standard tangent plane projection on the sky. In the above example, a new image t150sw.fits was created from t150s.fits. The pixel data are not changed, only the header.
  • Once this step has been performed, use DS9 (SAO image) to display the updated image. The WCS coordinates should appear in the upper left of the image display. WCS specific IRAF scripts or any other program which is based on the FITS standard should also now be able to read and use the WCS information stored in the updated ISPI header.

Taking Distortion into Account

  • The above call to imwcs results in a new image header, but it is often desired to actually resample or interpolate the image to a new projection. This could just be to have the pixel to sky projection modified or to allow for correcting higher order distortions. The latter is the case for ISPI since these distortions will affect the combination of dithered images. ISPI exhibits distortion, so combining dithered images without correcting for this will result in a degraded PSF at positions away from the center of the image. The effect can be significant (of order several pixels change in FWHM.
  • The key to creating a plate solution which accounts for the ISPI distortions is to have an image or images with many stars across the field of view and a set of accurate catalog positions.
  • Once a matched catalog of pixel and sky coordinates is in hand, any program which fits the latter to the former and writes an appropriate solution can be used to further update the image header. In practice, our solution relies on the IRAF task "ccmap" because this task will do the fit and add the appropriate keywords to the header. This is done in the IRAF TNX system. The advantage is that the SWARP program used to resample the dithered ISPI images automatically handles the TNX system (which is not officially part of the FITS standard). TNX is a combination of the linear WCS solution plus higher order terms. The fit is a general polynomial in x and y with cross terms. Initial testing with ISPI suggests ccmap fits of 4th-6th order (in IRAF speak) with full cross terms do well.

In this image, the residuals to a linear fit are shown. The peak errors in position would be about three pixels if the distortion were not accounted for. In the next image, the residuals for a 4th order fit are shown. The RMS is less than 0.1'' which is about 1/3 of an ISPI pixel.

  • The ccmap solution is written to a database file which IRAF understands and also to the image header if the flag "update" is turned on (i.e. set to yes) and images are specified in the images parameter of the task. The image header keywords are explained in the TNX description.
  • The ccmap solution can easily be applied in SWARP. Refer to the SWARP pages for details on how to execute this program. A simple call to swarp would be:

    > swarp -c swarp.gc t150sw.fits t151sw.fits t152sw.fits t153sw.fits t154sw.fits

    In this example, the swarp config file is "swarp.gc" and five images are reinterpolated. Each image can have its own TNX solution, which in our example would be the combination of WCS and TNX keyword values produced by imwcs and ccmap. Unless the fit to each image is very good, it is advisable to use an average solution for the higher order terms (and rotation and scale in this case). This can easily be done in swarp by specifying an auxillary header file with the FITS keywords and values. Cutting and pasting from the above solution from the FITS header, a file of the form t15?sw.head was made for each image. The file has the following form and is identical for each image:

    WAT0_001= 'system=image'
    WAT1_001= 'wtype=tnx axtype=ra lngcor = "3. 4. 4. 1. -0.09223848232288634 0.095'
    WAT2_001= 'wtype=tnx axtype=dec latcor = "3. 4. 4. 1. -0.09223848232288634 0.09'
    CD1_1   =  -8.4729762794223E-5
    CD1_2   =  7.23448951262463E-7
    CD2_1   =  7.40672424953437E-7
    CD2_2   =  8.47678406227307E-5
    WAT1_002= '30461739012073 -0.08294874238596413 0.086260218248302 -3.81821493608'
    WAT1_003= '0989E-6 0.004201053595244364 0.004530606959289172 -0.722711400721411'
    WAT1_004= '9 -5.887103279160741E-4 0.006853642315098468 0.0766861807819218 0. -'
    WAT1_005= '0.005316483436707363 -0.4617939442868234 0. 0. 0.1156522538399243 0.'
    WAT2_002= '530461739012073 -0.08294874238596413 0.086260218248302 4.34379162931'
    WAT2_003= '0940E-6 -5.007423412969465E-4 6.003889063455503E-5 0.084985861670387'
    WAT2_004= '73 0.003355387441743075 0.003754664665905738 -0.3911461579926636 0. '
    WAT2_005= '0.001750656256554302 0.04350355226749533 0. 0. -0.5986827241716564 0'
    WAT1_006= ' 0. 0. "'
    WAT2_006= '. 0. 0. "'
    END

    These lines were copied directly from the fits header of the image used in our example to find the higher order solution. A tab follows the keyword "END."

  • Swarp will combine the dithered images using the individually resampled images (see the swarp documentation and the config file options). The resampling includes rotation, scale, distortion and the standard pixel to sky projection (in this case, the TAN or tangent plane projection).

    Bad pixels may be rejected in the combine process by using the "pixel weighting" capability in SWARP. A simple weighting which rejects bad pixels can be accomplished by using the "mask2.fits" file produced by maskbad.cl (see the cirred package). This file has a 1 for each good pixel and 0 for each bad one. The file is called in the swarp configuration file by specifying its name in the WEIGHT_IMAGE keyword and using WEIGHT_MAP as the weighting method (it appears that one must specify this file name for each input image. I.e, make a coma separated list for the value of WEIGHT_IMAGE with the same file name repeated to match the number of input images). See the swarp documentation for more information. A slightly more sophisticated weight map can be made by multiplying the normalized flatfield image by mask2. Finally, one can specify a unique weight mask for each image through the file "imagename.suffix.fits" where "suffix" is given in the swarp config file, and the default is "imagename.weight.fits."

End to End Processing of ISPI Images

The following is an example of how ISPI images may be fully processed using the tools discussed on this page (cirred, WCSTOOLS, SWARP, IRAF).

  1. Make flats: Use med.cl to combine lights on and off images into a domeon and domeff image, respectively. Subtract the off image from the on.
  2. Make bad pixel masks: Use maskbad.cl to create bad pixel masks.
  3. Fix flats: Use fixfits.f to fix the flat images. The resulting images will have bad pixels interpolated over. If the images are named "dFilterNamef.fits", then the osiris.cl script will apply the right flat according to the FITS header filter name where FilterName is th evalue of this header parameter (e.g. KA for the ISPI Ks filter).
  4. Run osiris.cl: The meta script which will preprocess images, divide by a flat, and add appropriate head key words.
  5. Sky subtract the images: Use sky_sub.cl to make and subtract sky frames.
  6. Add a WCS: Use do_wcs.cl to add a World Coordiante System to the images. (Preliminary script included in cirred package. Requires WCSTOOLS. Requires a catalog implementation for the WCSTOOLS.).
  7. Compute high order distortions: Use do_ccmap.cl to compute the high order distortion coefficients and add them to the image header (Preliminary script included in cirred package. Requires WCSTOOLS. Need to incorporate TNX keywords manually as described above).
  8. Shift/Combine images: Use SWARP (by Emmanuel Bertin) to reinterpolate the images and optionally shift and combine them.
  9. Make a color image: SWARP can be used to shift and orient the final J, H, and K images so they can be combined in a false color image.

Acknoledgements: We would like to acknowledge the useful comments and input of Silas Laycock regarding ISPI data reduction and astrometry. It is also a pleasure to acknowledge the very useful software and documentation of the Mink WCSTOOLS, the Bertin SWARP program, and the IRAF suite.

Please mail questions or comments to:
rblumATctio.noao.edu
nvdbliekATctio.noao.edu
bgregoryATctio.noao.edu

Updated on June 2, 2022, 11:25 am