Skip to content

Commit

Permalink
Update calwf3_with_v1.0_PCTE.ipynb
Browse files Browse the repository at this point in the history 
mostly minor edits to update the notebook for being hosted in hst_notebooks repo
  • Loading branch information
@bjkuhn
bjkuhn authored Nov 3, 2023
1 parent 0cc1a19 commit d6abf2f
Showing 1 changed file with 50 additions and 19 deletions.
69 changes: 50 additions & 19 deletions notebooks/WFC3/calwf3_v1.0_cte/calwf3_with_v1.0_PCTE.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@
"\n",
"[Additional Resources](#resources) <br>\n",
"[About the Notebook](#about) <br>\n",
"[Citations](#cite) \n"
"[Citations](#cite) \n",
"\n"
]
},
{
Expand All @@ -61,11 +62,10 @@
"\n",
"## Introduction \n",
"\n",
"The v1.0 pixel-based Charge Transfer Efficiency (CTE) correction was first implemented into `calwf3` `v3.3` in 2016 ([Ryan et al. 2016](https://ui.adsabs.harvard.edu/abs/2016wfc..rept....1R/abstract),<br> [Anderson & Bedin 2010](https://ui.adsabs.harvard.edu/abs/2010PASP..122.1035A/abstract), [HSTCAL release notes](https://github.com/spacetelescope/hstcal/releases/tag/1.0.0)). This also marked the first time users could directly download CTE-corrected `flc & drc` <br> files from [MAST](https://mast.stsci.edu/search/hst/ui/#/). While the v1.0 correction was sufficient for many years, the degradation of CTE over time reduced the efficacy of the model <br> in treating low-level pixels. The v1.0 correction adversely impacts (overcorrects) both the image background and faint sources. In April 2021 <br> the v2.0 pixel-based CTE correction was implemented in `calwf3` `v3.6.0` ([Anderson et al. 2021](https://ui.adsabs.harvard.edu/abs/2021wfc..rept....9A/abstract), [Kuhn & Anderson 2021](https://ui.adsabs.harvard.edu/abs/2021wfc..rept....6K/abstract), [HSTCAL release <br>notes](https://github.com/spacetelescope/hstcal/releases/tag/2.7.0)). Since MAST uses the latest release of `calwf3` for calibration, any WFC3/UVIS CTE corrected data retrieved from MAST, regardless of <br> observation date, will be calibrated with the v2.0 pixel-based CTE correction. Although v1.0 pixel-based CTE-corrected `flc & drc` files are <br> no longer accessable through MAST, this notebook steps through the procedure required to calibrate WFC3/UVIS images using the v1.0 CTE <br> correction. <br>\n",
"The v1.0 pixel-based Charge Transfer Efficiency (CTE) correction was first implemented into `calwf3` `v3.3` in 2016 <br> ([Ryan et al. 2016](https://ui.adsabs.harvard.edu/abs/2016wfc..rept....1R/abstract), [Anderson & Bedin 2010](https://ui.adsabs.harvard.edu/abs/2010PASP..122.1035A/abstract), [HSTCAL release notes](https://github.com/spacetelescope/hstcal/releases/tag/1.0.0)). This also marked the first time users could directly <br> download CTE-corrected `flc & drc` files from [MAST](https://mast.stsci.edu/search/hst/ui/#/). While the v1.0 correction was sufficient for many years, the <br> degradation of CTE over time reduced the efficacy of the model in treating low-level pixels. The v1.0 correction adversely <br> impacts (overcorrects) both the image background and faint sources. In April 2021 the v2.0 pixel-based CTE correction <br> was implemented in `calwf3` `v3.6.0` ([Anderson et al. 2021](https://ui.adsabs.harvard.edu/abs/2021wfc..rept....9A/abstract), [Kuhn & Anderson 2021](https://ui.adsabs.harvard.edu/abs/2021wfc..rept....6K/abstract), [HSTCAL release notes](https://github.com/spacetelescope/hstcal/releases/tag/2.7.0)). Since <br> MAST uses the latest release of `calwf3` for calibration, any WFC3/UVIS CTE corrected data retrieved from MAST, <br> regardless of observation date, will be calibrated with the v2.0 pixel-based CTE correction. Although v1.0 pixel-based <br> CTE-corrected `flc & drc` files are no longer accessable through MAST, this notebook steps through the procedure <br> required to calibrate WFC3/UVIS images using the v1.0 CTE correction. <br>\n",
"\n",
"\n",
"One of the limiting factors of using the v1.0 CTE correction are the CTE corrected dark current reference files (`DRKCFILE`). These dark <br> reference files are delivered to MAST by the WFC3 team and use the same pixel-based CTE correction within `calwf3`. Now that we <br> have switched to the v2.0 CTE correction there is a cut off for dark current reference files that use the v1.0 correction. Observations taken <br> after February 2021 will not have\n",
"CTE corrected dark files using the v1.0 algorithm, which means **applying the v1.0 CTE correction works <br> best for observations taken between May 2009 - February 2021.** If the observation being calibrated was taken after February 2021 there <br> are two options: **1)** use the last v1.0 CTE corrected dark reference file from February 2021 or **2)** use the v2.0 CTE corrected dark with the <br> most appropriate `USEAFTER` for the science exposure's observation date. "
"One of the limiting factors of using the v1.0 CTE correction are the CTE corrected dark current reference files (`DRKCFILE`). <br> These dark reference files are delivered to MAST by the WFC3 team and use the same pixel-based CTE correction within <br> `calwf3`. Now that we have switched to the v2.0 CTE correction there is a cut off for dark current reference files that use <br> the v1.0 correction. Observations taken after February 2021 will not have CTE corrected dark files using the v1.0 algorithm, <br> which means **applying the v1.0 CTE correction works best for observations taken between May 2009 - February 2021.** <br> If the observation being calibrated was taken after February 2021 there are two options: **1)** use the last v1.0 CTE corrected <br> dark reference file from February 2021 or **2)** use the v2.0 CTE corrected dark with the most appropriate `USEAFTER` for the <br> science exposure's observation date. "
]
},
{
Expand All @@ -75,7 +75,9 @@
"<a id=\"imports\"></a>\n",
"## 1. Imports\n",
"\n",
"This notebook assumes you have created the virtual environment in [WFC3 Library's](https://github.com/spacetelescope/WFC3Library) installation instructions.\n",
"<div class=\"alert alert-block alert-warning\" style=\"color:black\" > <b> This notebook assumes you have created and activated a virtual environment using the requirements file in this notebook's repository..</b> <br> </div>\n",
"\n",
"<!-- **This notebook assumes you have created and activated a virtual environment using the requirements file in this notebook's repository.** -->\n",
"\n",
"We import: <br>\n",
" <br>\n",
Expand Down Expand Up @@ -105,15 +107,15 @@
"outputs": [],
"source": [
"import glob\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import os\n",
"import shutil\n",
"\n",
"from astropy.io import fits\n",
"from astroquery.mast import Observations\n",
"from astropy.table import Table\n",
"from astropy.time import Time\n",
"from astroquery.mast import Observations\n",
"from ginga.util import zscale\n",
"from photutils.aperture import aperture_photometry, CircularAperture, CircularAnnulus\n",
"from wfc3tools import calwf3\n",
Expand All @@ -127,7 +129,7 @@
"source": [
"<a id=\"txtfile\"></a>\n",
"## 2. Verify `archived_drkcfiles.txt` is in CWD\n",
"When you cloned/downloaded this notebook from [WFC3 Library](https://github.com/spacetelescope/WFC3Library), a .txt file should have been included. The file name is <br> `archived_drkcfiles.txt` and it is used later on in the notebook. This .txt file includes the file name, delivery date, activation <br> date, and USEAFTER date for every v1.0 CTE corrected dark reference file between May 2009 - February 2021. Below, we will use <br> this file in conjunction with the observation date of the file(s) being calibrated to pick out the most appropriate v1.0 CTE corrected <br> dark reference file(s).<br>\n",
"When you cloned/downloaded this notebook from [hst_notebooks](https://github.com/spacetelescope/hst_notebooks/), a .txt file should have been included. The file name is <br> `archived_drkcfiles.txt` and it is used later on in the notebook. This .txt file includes the file name, delivery date, activation <br> date, and USEAFTER date for every v1.0 CTE corrected dark reference file between May 2009 - February 2021. Below, we will use <br> this file in conjunction with the observation date of the file(s) being calibrated to pick out the most appropriate v1.0 CTE corrected <br> dark reference file(s).<br>\n",
"\n",
"Please make sure the file is in the current working directory before continuing."
]
Expand All @@ -149,10 +151,10 @@
"<a id=\"checkcalver\"></a>\n",
"## 3. Check that `calwf3` Version is `v3.5.2`\n",
"In April 2021, a new `calwf3` version was released that contains the v2.0 CTE-correction.\n",
"If you would like to use the v1.0 correction, your <br> current environment must be using `calwf3` versions equal to or between `3.3` - `3.5.2`. However, in order to get the best v1.0 calibrated <br>images we must use `calwf3` `v3.5.2`. This version of `calwf3` includes the recent ([~Jan 2021](https://github.com/spacetelescope/hstcal/releases/tag/2.5.0)) update that added MJD as a parameterized <br> variable for the `PHOTMODE` keyword, which enables a time-dependent photometric correction and zeropoint. If your version is `3.6.0` or higher, <br> you must downgrade the [`hstcal` package](https://github.com/spacetelescope/hstcal). The safer option, however, is to create a new environment such as: <br>\n",
"If you would like to use the v1.0 correction, your <br> current environment must be using `calwf3` versions equal to or between `3.3` - `3.5.2`. However, in order to get the best v1.0 calibrated <br>images we must use `calwf3` `v3.5.2`. This version of `calwf3` includes the recent ([~Jan 2021](https://github.com/spacetelescope/hstcal/releases/tag/2.5.0)) update that added MJD as a parameterized <br> variable for the `PHOTMODE` keyword, which enables a time-dependent photometric correction and zeropoint. If your version is `3.6.0` or higher, <br> you must downgrade the [`hstcal` package](https://github.com/spacetelescope/hstcal). The safer option, however, is to create a new environment using the requirements file provided in the [notebook's repository](https://github.com/spacetelescope/hst_notebooks/tree/main/notebooks/WFC3/calwf3_v1.0_cte): <br>\n",
"\n",
"* `$ conda config --add channels http://ssb.stsci.edu/astroconda`\n",
"* `$ conda create -n v1_PCTE hstcal==2.5.0 python=3.7 ginga stsci-hst notebook`\n",
"* `$ conda create -n v1_PCTE hstcal==2.5.0 --file requirements.txt`\n",
"\n",
"`hstcal` `v2.5.0` provides version `3.5.2` of `calwf3`, which is the last version that offers the v1.0 pixel-based CTE correction.\n"
]
Expand Down Expand Up @@ -317,7 +319,7 @@
"source": [
"# Generate table of filename, date-obs, drkc-filename, corresponding useafter\n",
"raw_dkc_tab = Table([rawfiles, obsdates, dkcfiles, uafters, active_dates],\n",
" names=('filename','date-obs','dkc-filename','dkc-useafter','dkc-activation'))\n",
" names=('filename', 'date-obs', 'dkc-filename', 'dkc-useafter', 'dkc-activation'))\n",
"# Display table\n",
"raw_dkc_tab\n"
]
Expand All @@ -341,7 +343,7 @@
"metadata": {},
"outputs": [],
"source": [
" for file in glob.glob('i*raw.fits'):\n",
"for file in glob.glob('i*raw.fits'):\n",
" # Using raw_dkc_tab from above, grab appropriate drkcfile\n",
" ctecorr_dark = 'iref$'+raw_dkc_tab[raw_dkc_tab['filename']==file]['dkc-filename'][0]\n",
" fits.setval(file, 'DRKCFILE', value = ctecorr_dark)\n"
Expand Down Expand Up @@ -372,7 +374,7 @@
"<a id=\"reinspect\"></a>\n",
"## 7. Re-Inspect Image Header \n",
"Now with the headers modified, we inspect the keywords one last time to verify the file was updated properly before we process <br> it through `calwf3`. At this point you should see:<br> \n",
"- `PCTETAB` set to `iref$zcv2057mi_cte.fits\t`<br>\n",
"- `PCTETAB` set to `iref$zcv2057mi_cte.fits`<br>\n",
"- `DRKCFILE` set to `iref$3961719li_dkc.fits`<br>"
]
},
Expand Down Expand Up @@ -619,8 +621,7 @@
"source": [
"<div style=\"text-align: left\"> <strong> <font size =\"4\">Animated GIF of the v1.0 and v2.0 FLC image subsections: </font> </strong> </div>\n",
"\n",
"<img align=\"left\" width=\"800\" height=\"800\" src=\"example/v1_v2_bkg.gif\">\n",
"\n"
"<img align=\"left\" width=\"800\" height=\"800\" src=\"example/v1_v2_bkg.gif\" alt=\"An animated gif blinking between a subsection of background sky using the v1.0 and V2.0 pixel-based CTE corrections. The v2.0 background appears smoother with less noise and pixel variations.\">\n"
]
},
{
Expand Down Expand Up @@ -886,9 +887,9 @@
"<a id=\"about\"></a>\n",
"## About this Notebook\n",
"\n",
"**Author:** Benjamin Kuhn; WFC3 Instrument Team\n",
"\n",
"**Updated on:** January 19, 2023\n",
"**Author:** Benjamin Kuhn; WFC3 Instrument Team <br>\n",
"**Created:** February 22, 2022 <br>\n",
"**Last updated on:** November 3, 2023\n",
"\n",
"<a id=\"cite\"></a>\n",
"## Citations\n",
Expand All @@ -901,6 +902,7 @@
"* [Citing `matplotlib`](https://matplotlib.org/stable/users/project/citing.html)\n",
"* [Citing `numpy`](https://numpy.org/citing-numpy/)\n",
"* [Citing `photutils`](https://photutils.readthedocs.io/en/stable/citation.html)\n",
"* [Citing `scipy`](https://scipy.org/citing-scipy/)\n",
"<br>\n",
"***\n",
"[Top of Page](#top)\n",
Expand All @@ -924,7 +926,36 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.12"
"version": "3.7.16"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
Expand Down

0 comments on commit d6abf2f

Please sign in to comment.