Note: This is a rearrangement of the GLIMPSE Post-Pipeline Flowchart(pdf file).
Here we assign tasks
during each quarter (mostly fiction as we get further into the future).
Note: This document needs reworking.
Items marked in red, and surrounded by asterisks (for b/w printers) are mostly completed.
Database
*Set up Database; no interaction with data reduction at this stage (MKE, Wolff & Whitney)*.
Automation
1) *Install OPUS (pipeline software) and learn about it (Wolff & Meade.)*
2) * Run DAOPHOT from OPUS (Wolff & Babler).*
Level A - Validation
1) *Generate test images with bad pixel maps, cosmic masks (Whitney & Meade)*
2) *Write fortran programs to do dead pixel counts, noise level checks, and
update fits header accordingly (Meade & Whitney)*.
Level B - Basic Processing & Mask Propagation
1) *Compute Zodi background based on position and time (Wolfire)*.
2) *Write wrapper for Zodi subroutine (Meade)*.
Level C - Photometry on each frame
1) Apply necessary corrections from Level B for each frame before doing photometry:
replace bad pixels, cosmic rays with mean of surrounding pixels. (Whitney & Meade)
2) Zodi subtraction (using values calculated in Level B). (Meade)
3) *Photometry (DAOPHOT) (Babler)*
4) *Produce a primitive GBSC (20 sigma) 1 wavelength band (Babler)*
Level D - Mosaicing
1) *Generate test images - download MSX or WFPC data in overlapping frames;
rotate to arbitrary angle*
; modify WCS (world coordinate system).
*generate appropriate fits keyword headers. (Meade)*
2) *Initial research on mosaicing tools - Drizzle, IRAF, IDL routines
(Wolfire).*
Level E - Processing for GLIMPSE Point Source Catalog
Database
1) *Continue setting up the database. (MKE, Wolff & Whitney)*
2) *Practical interaction with Database
- read files from database, perform operations, put in database (Wolff)*
Automation
1) *Install stand-alone version of DAOPHOT (Babler & Wolff)*
2) *Add in developed tasks from previous quarter to OPUS pipeline (Wolff)*
Level A - automated checks on all frames
1) *generate necessary test data for the steps listed below (Whitney & Meade).*
2) look for pixels above and below certain thresholds (Meade)
3) *Is the AOR executed correctly? Automated checks comparing image header
information to AOR text. (Meade)*
Level B - Basic Processing & Mask Propagation
1) make our own cosmic ray mask (Esquivel)
2) make our own bad pixel flag (Meade & Whitney)
3) learn more what SSC provides in way of geometric correction instructions (Whitney)
Level C - GBSC processing
1) add photometry results for multiple observations (Babler)
2) Band merging I
- correlate source positions from different images (Babler)
Level D - Mosaicing Images
1) Continue researching Mosaic tools (Whitney & Wolfire)
2) Register and resample images to a galactic coordinate frame. (Whitney)
3) Practice combining multiple images (Whitney)
Level E - Processing for GLIMPSE Point Source Catalog
Continue building database
Settle on pipeline architecture
Automated Pipeline & interaction with database - 25% completion.
Level A - Validation
1) bad downlink data? (checksums?) look into this
2) striping
4) file integrity
Level B - Basic Processing & Mask Propagation
1) Banding correction
2) Add our own quality flags for noise
3) quality flag for pointing accuracy
4) continue automating processes
Level C - GBSC processing
1) read photometry results into database, have database create catalog.
2) start Band Merging II
- refine positions (using something like J.P's tool)
then correlate source positions from different images
3) continue automation of GBSC generation
Level D - Mosaicing Images
1) Zodi subtraction, photometry on combined images.
2) look for solar system objects
3) apply geometric corrections (flux conserving)
4) combine multiple images into larger mosaics, practice working with supermosaics
with several layers.
5) continue automating processes
Level E - Processing for GLIMPSE Point Source Catalog
1) continue automating processes
Automated Pipeline & interaction with database - 50% completion
Continue generating test data and hopefully getting IRAC test data from SSC.
Level A - Validation
1) photometry spot check
do this on central frame, first observed frame, last observed frame?
position vs. MSX/2MASS (check Jeff Percival's tool for star tracker).
fluxes vs. MSX/2MASS
image quality (psf check). (focus can change going from MIPS to IRAC due to
temperature change)
sensitivity
2) AOR completion (close-out)
develop criteria, based on developed validation steps
Accept, or Re-observe
Level B - Basic Processing & Mask Propagation
1) tune latency flags (if desirable)
SSC may provide Latent Strength Estimation, and therefore latent image
instrument teams may have latency model
Level C - GBSC processing
1) latent image correction. either throw out flagged pixels or subtract latent image.
2) Quality control on GBSC
3) continue Band Merging II
- refine positions (using something like J.P's tool)
then correlate source positions from different images
Level D - Mosaicing Images
1) continue work on supermosaic
2) apply high-pass filter on images, if desired
3) automation of mosaicking tasks 80% complete
Level E - Processing for GLIMPSE Point Source Catalog
1) put out primitive GPSC, produced by database
Automated Pipeline & interaction with database - 100% completion by Apr. 1, 2002
SSC Data Plan Review - Mar. 2002
Level A - Validation
1) Compare parameters between supertiles to watch for variations in PSF, sensitivity,
noise
2) Track survey completion
3) Automate procedures, and interaction with database
Level B - Basic Processing & Mask Propagation
1) Automate procedures, and interaction with database
Level C - GBSC processing
1) Automate procedures, and interaction with database
2) Full GBSC
Level D - Mosaicing Images
1) Automate procedures, and interaction with database
2) Full mosaicing capabilities
Level E - Processing for GLIMPSE Point Source Catalog
1) Automate procedures, and interaction with database
2) Full GPSC
Level A - Validation
1) Verify that all procedures in Level A are in place and automated.
Level B - Basic Processing & Mask Propagation
1) Verify that all procedures in Level B are in place and automated.
Level C - GBSC processing
1) Verify that all procedures in Level C are in place and automated.
Level D - Mosacing Images
1) Verify that all procedures in Level D are in place and automated.
Level E - Processing for GLIMPSE Point Source Catalog
1) Verify that all procedures in Level E are in place and automated.