diff --git a/content/notebooks/aperture_photometry/aperture_photometry.ipynb b/content/notebooks/aperture_photometry/aperture_photometry.ipynb index fcfbdfe..3809824 100644 --- a/content/notebooks/aperture_photometry/aperture_photometry.ipynb +++ b/content/notebooks/aperture_photometry/aperture_photometry.ipynb @@ -32,20 +32,7 @@ "source": [ "# Kernel Information\n", "\n", - "To run this notebook, please select the \"Roman Calibration\" kernel at the top right of your window.\n", - "\n", - "# Table of Contents\n", - "- [Imports](#Imports)\n", - "- [Introduction](#Introduction)\n", - "- [Tutorial Data](#Tutorial-Data)\n", - " - [Image Data](#Image-Data)\n", - " - [Source Catalog](#Source-Catalog)\n", - "- [Forced Aperture Photometry](#Forced-Aperture-Photometry)\n", - " - [Create Apertures](#Create-Apertures)\n", - " - [Source Visualization](#Source-Visualization)\n", - " - [Aperture Photometry with Photutils](#Aperture-Photometry-with-Photutils)\n", - "- [Additional Resources](#Additional-Resources)\n", - " - [About This Notebook](#About-This-Notebook)" + "To run this notebook, please select the \"Roman Calibration\" kernel at the top right of your window." ] }, { @@ -61,6 +48,7 @@ "- *matplotlib*\n", "- *photutils* is an Astropy-affiliated package for photometry\n", "- *roman_datamodels* opens and validates WFI data files\n", + "- *os* for checking if files exist\n", "- *s3fs* streams data from Simple Storage Service (S3) buckets on Amazon Web Services (AWS)" ] }, @@ -80,6 +68,7 @@ "import numpy as np\n", "from photutils.aperture import CircularAperture, aperture_photometry\n", "import roman_datamodels as rdm\n", + "import os\n", "import s3fs" ] }, @@ -127,52 +116,12 @@ "slide_type": "slide" } }, - "source": [ - "In this tutorial, we use a Level 2 (L2; calibrated rate image) WFI data file that is the result of RomanCal processing of a Level 1 (L1; uncalibrated ramp cube) simulated file created with Roman I-Sim. If you have already worked through the tutorials \"Simulating WFI Imaging Data with Roman I-Sim\" and \"Calibrating WFI Exposures with RomanCal,\" then you may already have these files saved locally. If not, then these files are also stored in the science platform S3 bucket. For more information on how to access these data, see the Data Discovery and Access tutorial." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Stream the files from the S3 bucket if they are not in local storage\n", - "#\n", - "# Uncomment the following lines if you do not have the output files \n", - "# from the tutorials \"Simulating WFI Imaging Data with Roman I-Sim\" and\n", - "# \"Calibrating WFI Exposures with RomanCal\"\n", - "#\n", - "#" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, "source": [ "## Image Data\n", "\n", - "We load a simulated Roman WFI image in the F106 filter generated with [Roman I-Sim](https://romanisim.readthedocs.io/) from a source catalog containing Gaia stars and simulated galaxies. This image covers a single WFI detector. It is a Level 2 (L2; calibrated rate image) file, meaning that the data were processed to flag and/or correct for detector-level effects (e.g., saturation, classic non-linearity, etc.), and that the ramp was fitted into a count rate image in units of Data Numbers (DN) per second." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "## FIX THIS CELL ONCE DATA ARE IN BUCKET\n", + "In this tutorial, we use a Level 2 (L2; calibrated rate image) WFI data file that is the result of RomanCal processing of a Level 1 (L1; uncalibrated ramp cube) simulated file created with Roman I-Sim. If you have already worked through the tutorials \"Simulating WFI Imaging Data with Roman I-Sim\" and \"Calibrating WFI Exposures with RomanCal,\" then you may already have these files saved locally. If not, then these files are also stored in the science platform S3 bucket. For more information on how to access these data, see the Data Discovery and Access tutorial.\n", "\n", - "#asdf_dir_uri = 's3://roman-sci-test-data-prod-summer-beta-test/'\n", - "#fs = s3fs.S3FileSystem()\n", - "#\n", - "#asdf_file_uri = asdf_dir_uri + 'ROMANISIM/r0003201001001001004_0001_wfi01_cal.asdf'\n", - "#\n", - "#with fs.open(asdf_file_uri, 'rb') as f:\n", - "# dm = rdm.open(f)\n", - "# image = dm.data.value.copy()\n", - "# wcs = copy.deepcopy(dm.meta.wcs)" + "As a reminder, the file we are using is a L2 file meaning that the data were processed to flag and/or correct for detector-level effects (e.g., saturation, classic non-linearity, etc.), and that the ramp was fitted into a count rate image in units of Data Numbers (DN) per second." ] }, { @@ -181,10 +130,20 @@ "metadata": {}, "outputs": [], "source": [ - "## TMP FIX FOR TESTING\n", - "f = rdm.open('r0003201001001001004_0001_wfi01_cal.asdf')\n", - "image = f.data.copy()\n", - "wcs = copy.deepcopy(f.meta.wcs)" + "#Stream the files from the S3 bucket if they are not in local storage\n", + "\n", + "if os.path.exists('r0003201001001001004_0001_wfi01_f106_cal.asdf'):\n", + " f = rdm.open('r0003201001001001004_0001_wfi01_f106_cal.asdf')\n", + " image = f.data.copy()\n", + " wcs = copy.deepcopy(f.meta.wcs)\n", + "else:\n", + " fs = s3fs.S3FileSystem()\n", + " asdf_dir_uri = 's3://roman-sci-test-data-prod-summer-beta-test/AAS_WORKSHOP/'\n", + " asdf_file_uri = asdf_dir_uri + 'r0003201001001001004_0001_wfi01_f106_cal.asdf'\n", + " with fs.open(asdf_file_uri, 'rb') as f:\n", + " dm = rdm.open(f)\n", + " image = dm.data.value.copy()\n", + " wcs = copy.deepcopy(dm.meta.wcs)" ] }, { @@ -204,20 +163,16 @@ "metadata": {}, "outputs": [], "source": [ - "## FIX THIS CELL ONCE DATA ARE IN BUCKET\n", + "#Stream the files from the S3 bucket if they are not in local storage\n", "\n", - "#catalog_file_uri = asdf_dir_uri + 'ROMANISIM/CATALOGS_SCRIPTS/fullcat_101M_pared_ra0.50_dec0.50_WFI01.ecsv'\n", - "#with fs.open(catalog_file_uri, 'rb') as catalog_file_stream:\n", - "# cat = Table.read(catalog_file_stream, format='ascii.ecsv')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cat = Table.read('full_catalog.ecsv')" + "if os.path.exists('full_catalog.ecsv'):\n", + " cat = Table.read('full_catalog.ecsv')\n", + "else:\n", + " fs = s3fs.S3FileSystem()\n", + " asdf_dir_uri = 's3://roman-sci-test-data-prod-summer-beta-test/AAS_WORKSHOP/'\n", + " asdf_file_uri = asdf_dir_uri + 'full_catalog.ecsv'\n", + " with fs.open(asdf_file_uri, 'rb') as f:\n", + " cat = Table.read(f, format='ascii.ecsv')" ] }, { @@ -269,8 +224,8 @@ "print(f\"Number of sources on detector: {sum(on_detector_mask)}\")\n", "\n", "stars = cat[on_detector_mask & (cat[\"type\"] == \"PSF\")]\n", - "gals = cat[on_detector_mask & (cat[\"type\"] == \"SER\")]\n", - "print(f\"Number of stars: {len(stars)}\\nNumber of galaxies: {len(gals)}\")" + "galaxies = cat[on_detector_mask & (cat[\"type\"] == \"SER\")]\n", + "print(f\"Number of stars: {len(stars)}\\nNumber of galaxies: {len(galaxies)}\")" ] }, { @@ -288,7 +243,7 @@ "source": [ "fig, ax = plt.subplots()\n", "ax.hist(np.log10(stars[\"F106\"].value), bins=50, range=[-10, -5], log=True, label=\"Stars\")\n", - "ax.hist(np.log10(gals[\"F106\"].value), bins=50, range=[-10, -5], log=True, label=\"Galaxies\")\n", + "ax.hist(np.log10(galaxies[\"F106\"].value), bins=50, range=[-10, -5], log=True, label=\"Galaxies\")\n", "\n", "ax.set_xlabel(\"$\\log_{10}$(F106 flux [maggies])\")\n", "ax.set_ylabel(\"Number of sources\")\n", @@ -326,11 +281,9 @@ "metadata": {}, "outputs": [], "source": [ - "stars = cat[cat['type'] == 'PSF']\n", "star_positions = [(y, x) for x, y in zip(*wcs.world_to_array_index_values(stars[\"ra\"], stars[\"dec\"]))]\n", "star_apertures = CircularAperture(positions=star_positions, r=3)\n", "\n", - "galaxies = cat[cat['type'] == 'SER']\n", "galaxy_positions = [(y, x) for x, y in zip(*wcs.world_to_array_index_values(galaxies[\"ra\"], galaxies[\"dec\"]))]\n", "galaxy_apertures = CircularAperture(positions=galaxy_positions, r=5)" ] @@ -424,6 +377,13 @@ "ax.grid(alpha=0.3)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In addition to what we expected, we see a distribution of stars that have very low measured fluxes of ~20 DN/s for a wide range of input catalog fluxes. These sources were skipped in the Roman I-Sim simulation, and thus while they are in the input catalog, there is no source at that poisition in the simulated image, and thus we are measuring sky background levels." + ] + }, { "cell_type": "markdown", "metadata": {},