'sxiflagpix' flags SXI events that may be unsuitable to use in data analysis, such as those that fall on defective or non-sensitive pixels; those that might have missing or compromised telemetry; and those for which processing has encountered certain errors. The STATUS column is updated, encoding the flagging condition for each event. This tool does not actually filter or remove events, however the STATUS column can be used for later filtering. An image in DET coordinates showing the location of unsuitable pixels is also output for later use in constructing an ARF or flat field image.
The tool uses several input files to assign STATUS: a list of bad or dead pixels or columns that don't change ("badpixfile"); a list of hot pixels, which can change on short timescales and are observation-specific ("hotpixfile"); a list of flickering pixels, which can be produced by the task searchflickpix ("flickpixfile"); and a mask file that encodes the location of CCD boundaries, segments, calibration source regions, and other regions that don't change ("maskfile"). The locations of the charge injection rows are read from the input event list header, as are settings for window mode and area discrimination.
In addition to an output event list with updated STATUS column, sxiflagpix optionally outputs a bad pixel list (specified by the parameter 'outbadpix') containing the events that have any STATUS flag set by the tool. Also output is an image (specified by the parameter 'outbadimg') in SXI DET coordinates containing a value for each pixel corresponding to its STATUS. The values in these files are, in order of increasing precedence:
0: a good pixel
1: a pixel inside the calibration source region
2: a bad pixel set by the parameter "bad_status"
-1: a pixel out of the CCD, window, or area discrimination region
For example, a pixel inside the calibration region but flagged as bad will have value 2 in the output image. Flickering pixels are not included in this map because they can depend on time. Note that the output bad pixel list contains only those pixels for which events are found in the input file, while the output image contains all pixels.
Below is a table mapping flag bit position in the STATUS column to particular flagging conditions. If the flag is set to 1, that condition is flagged. Unless otherwise noted, the STATUS values apply to the central pixel of an event. Tools other than sxiflagpix that set STATUS values are noted in parentheses. "CI" indicates "charge injection."
=========================================================================== flag description =========================================================================== 1 All bad events set by "bad_status" parameter 2 Inside the calibration source region --------------------------------------------------------------------------- 3 Out of CCD (Out of area) 4 Out of window 5 Out of area discrimination --------------------------------------------------------------------------- 6 CI row (Pixels) 7 Bad pixel from CALDB 8 Bad column from CALDB 9 Hot pixel from pre-pipeline 10 Flickering pixel --------------------------------------------------------------------------- 11 CCD boundary (Boundaries) 12 Window boundary 13 Segment boundary 14 Area discrimination boundary 15 At least one 3x3 surrounding pixel has a bad status --------------------------------------------------------------------------- 16 CI trailing row (Neighbors) 17 CI preceding row 18 Preceding/following of bad column 19 Neighbors of bad/hot pixel and bad column 20 Neighbors of flickering pixel 21 Neighbors of preceding/following of bad column 22 Neighbors of CCD/window boundary 23 Neighbors of segment boundary --------------------------------------------------------------------------- 24 (sxiphas) 3x3 info is present but 5x5 is absent (Others) 25 (sxiphas) 3x3 is absent 26 (sxipi - general) PHAS[0] < event threshold 27 (sxipi - vtevnodd) Video temperature is out of range 28 (sxipi - vtevnodd) Lack of video temp HK at time close to the event 29 (sxipi - chtrail/CTI) Correction value is negative 30 (sxipi - general) Null value by correction process --------------------------------------------------------------------------- 31 1st trailing row of the CI rows (Diagnostics) 32 1st preceding row of the CI rows 33 2nd trailing row of the CI rows 34 2nd preceding row of the CI rows 35 3rd trailing row of the CI rows 36 3rd preceding row of the CI rows --------------------------------------------------------------------------- 37-48 Reserved ===========================================================================
The special STATUS flag 1 is set for any event that has a STATUS flag listed in the "bad_status" parameter. This is to enable easy filtering syntax later in the processing, since good events can be extracted with a simple filter expression.
Several STATUS conditions apply to events where neighboring pixels are flagged. Because an event is composed of a 3x3 pixel island, the quality of the region immediately surrounding the event center must be taken into account. For example, a bad pixel in the outer ring of a 3x3 island will not report a valid pulse height, and this could result in a event with an otherwise unacceptable grade being accepted as a good event. The parameters 'npixnbr' and 'nboundnbr' can be used to set the maximum distance away from a bad pixel or boundary to flag further pixels and events. In addition, for more flexibility, sxiflagpix writes a new column PHAS_MASK to the output event list. This is a 9-element mask of the PHAS column, with each element indicating whether the corresponding pixel i in PHAS is good (PHAS_MASK[i]=0) or bad (PHAS_MASK[i]=1). PHAS_MASK can be used later in sxipi to determine how to grade the event and calculate the summed PHA, using the sxipi 'badpixopt' parameter. Since sxipi can overwrite the PHAS column, the ability to reset it to the original telemetry values is provided here with the 'copyphas' parameter. Similarly, 'resetflags' can be used to reset all the sxiflagpix STATUS flags and the elements of PHAS_MASK to zero.
Charge trailed or leaked from charge injection rows can produce spurious events in several rows immediately surrounding the injected rows, manifesting as noise at soft energies. The parameters 'citrailnbr' and 'ciprenbr' can be used to tune a compromise between lower noise and higher efficiency, since flagging additional rows reduces the effective area.
Some examples of how to use the STATUS flags in filtering and extracting events are shown below in the "Examples" section. Also see the help for 'calc_express' for proper filtering syntax.
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits"
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile=NONE badpixfile=NONE
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits" \
citrailnbr=4 ciprenbr=2
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits" \
bad_status="3:9,11:12,16"
ftcopy "output.evt[events][STATUS[1]==b0]" output_filtered.evt
ftcopy "output.evt[events][STATUS[1]==b0 && STATUS[2]==b1]" output_filtered.evtAnother way to do the same filtering, using the "bxxx" bit mask notation, which is explained fully under "calc_express". The wild card "x" indicates any value is allowed.
ftcopy \
"output.evt[events][STATUS==b01xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]" \
output_filtered.evt
searchflickpix, sxiphas, sxipi, calc_express
February 3, 2016