diff --git a/content/notebooks/data_visualization/data_visualization.ipynb b/content/notebooks/data_visualization/data_visualization.ipynb index b13b271..980fbbe 100644 --- a/content/notebooks/data_visualization/data_visualization.ipynb +++ b/content/notebooks/data_visualization/data_visualization.ipynb @@ -8,38 +8,55 @@ } }, "source": [ - "# Visualizing Data in Roman ASDF Files" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "skip" - } - }, - "source": [ - "***" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## Kernel Information\n", + "# Visualizing Data in Roman ASDF Files\n", + "***\n", + "\n", + "## Learning Goals\n", + "\n", + "This notebook is designed to walk you through opening and displaying Roman Wide-Field Instrument (WFI) `ASDF` files using `matplotlib` and `Imviz`.\n", + "\n", + "## Table of Contents\n", + "\n", + "\\- [1. Loading Data](#load_data)\n", + "\n", + "\\- [2. Displaying a Static Image](#static_im)\n", + "\n", + "\\- [3. Examining the Data Interactively using `Imviz`](#imviz)\n", + "\n", + "***\n", + "\n", + "\n", + "## 0. Introduction\n", + "In this tutorial, we demonstrate how to visualize and explore Roman WFI image data arrays. We separate this tutorial into two high-level components:\n", + "\n", + "1. Creating static images with world coordinate system (WCS) overlays; and\n", + "2. Using the `Jdaviz` tool to interactively explore WFI image data.\n", + "\n", + "We focus on how to visualize WFI Level 2 (L2) data in `ASDF` format. For WFI, L2 indicates that the data have been processed to flag and/or correct for detector-level effects (e.g., saturation, classic non-linearity, etc.) and the resultants fitted into a count rate image. Each L2 `ASDF` file contains a single WFI detector, thus a complete WFI exposure is made up of `18` L2 files. For more information WFI L2 files, please see the RDox article on [Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-Level2-CalibratedExposuresLevel2).\n", + "\n", + "***\n", + "### Kernel Information\n", "\n", "To run this notebook, please select the \"Roman Calibration\" kernel at the top right of your window.\n", "\n", - "## Imports\n", - "- *astropy.visualization.simple_norm* for automatically scaling image arrays\n", - "- *astropy.coordinates.SkyCoord* to create Python objects containing sky coordinate transforms\n", - "- *matplotlib.pyplot* for creating static image previews\n", - "- *jdaviz.Imviz* to examine Wide Field Instrument (WFI) images interactively\n", - "- *roman_datamodels* for opening Roman WFI ASDF files" + "### Imports\n", + "Here we import the required packages for our data access examples including:\n", + "\n", + "- `astropy.visualization.simple_norm` for automatically scaling image arrays\n", + "\n", + "- `astropy.coordinates.SkyCoord` to create Python objects containing sky coordinate transforms\n", + "\n", + "- `astropy.table Table` for reading catalog data\n", + "\n", + "- `matplotlib.pyplot` for creating static image previews\n", + "\n", + "- `numpy` for array operations\n", + "\n", + "- `jdaviz.Imviz` to examine WFI images interactively\n", + "\n", + "- `roman_datamodels` for opening Roman WFI `ASDF` files\n", + " \n", + "- `s3fs` for streaming in data directly from the cloud" ] }, { @@ -58,7 +75,6 @@ "from astropy.table import Table\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", - "\n", "from jdaviz import Imviz\n", "import roman_datamodels as rdm\n", "import s3fs" @@ -72,33 +88,12 @@ } }, "source": [ - "## Introduction\n", - "In this tutorial, we demonstrate how to visualize and explore Roman WFI image data arrays. We separate this tutorial into two high-level components:\n", - "1. creating static images with world coordinate system (WCS) overlays; and\n", - "2. using the Jdaviz tool to interactively explore WFI image data.\n", - "\n", - "We focus on how to visualize WFI Level 2 (L2) data in ASDF format. For WFI, L2 indicates that the data have been processed to flag and/or correct for detector-level effects (e.g., saturation, classic non-linearity, etc.) and the resultants fitted into a count rate image. Each L2 ASDF file contains a single WFI detector, thus a complete WFI exposure is made up of 18 L2 files. For more information WFI L2 files, please see the RDox article on [Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-Level2-CalibratedExposuresLevel2)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "***" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## Loading data\n", - "We use the same data file in both examples. Note that the static image portions can be used on any `numpy.ndarray` object, and the WCS axes may be optionally included with any `astropy.wcs.WCS` object. Also, while the Jdaviz example is designed to work on Roman data, Jdaviz can also work with other data and file types. A link to the Jdaviz documentation is provided in the Additional Resources section below.\n", + "***\n", + "\n", + "## 1. Loading data\n", + "We use the same data file in both examples. Note that the static image portions can be used on any `numpy.ndarray` object, and the WCS axes may be optionally included with any `astropy.wcs.WCS` object. Also, while the `Jdaviz` example is designed to work on Roman data, `Jdaviz` can also work with other data and file types. Check out the [`Jdaviz` documentation](https://jdaviz.readthedocs.io/en/stable/).\n", "\n", - "A complete explanation on how to load and work with Roman ASDF files is provided in the notebook tutorial *Working with ASDF*. We read in the data using the `roman_datamodels` package:" + "A complete explanation of how to load and work with Roman `ASDF` files is provided in the n[otebook tutorial Working with ASDF](https://github.com/spacetelescope/roman_notebooks/tree/main/content/notebooks). We read in the data using the `roman_datamodels` package:" ] }, { @@ -110,7 +105,7 @@ "asdf_dir_uri = 's3://roman-sci-test-data-prod-summer-beta-test/'\n", "fs = s3fs.S3FileSystem()\n", "\n", - "asdf_file_uri = asdf_dir_uri + 'ROMANISIM/DENSE_REGION/R0.5_DP0.5_PA0/r0000101001001001001_01101_0001_WFI16_cal.asdf'\n", + "asdf_file_uri = f'{asdf_dir_uri}ROMANISIM/DENSE_REGION/R0.5_DP0.5_PA0/r0000101001001001001_01101_0001_WFI16_cal.asdf'\n", "with fs.open(asdf_file_uri, 'rb') as f:\n", " file = rdm.open(f)" ] @@ -119,11 +114,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Static Image Display\n", + "***\n", + "\n", + "\n", + "## 2. Displaying a Static Image\n", "\n", - "### Plot an Image with Dynamic Scaling\n", + "Here we show how to use `matplotlib` and `astropy` to plot the data array from our WFI image and scale the array automatically.\n", "\n", - "Here we show how to use matplotlib and Astropy to plot the data array from our WFI image and scale the array automatically." + "We will first set the image normalization. Here we use a `log` scale with the minimum and maximum of the range based on `99%` of the pixels. Then we will use `matplotlib.pyplot.imshow()` to display the image. " ] }, { @@ -132,17 +130,20 @@ "metadata": {}, "outputs": [], "source": [ - "# Set the image normalization. Here we use a log scale \n", - "# with the minimum and maximum of the range based on \n", - "# 99% of the pixels.\n", "norm = simple_norm(file.data.value, 'asinh', percent=95)\n", "\n", "fig, ax = plt.subplots(figsize=(8, 6))\n", - "sc = ax.imshow(file.data.value, norm=norm, origin='lower')\n", + "\n", + "sc = ax.imshow(file.data.value,\n", + " norm=norm,\n", + " origin='lower')\n", + "\n", "ax.set_xlabel('X Science Axis (pixels)')\n", "ax.set_ylabel('Y Science Axis (pixels)')\n", "ax.set_title('romanisim Simulation WFI16')\n", + "\n", "plt.colorbar(sc, ax=ax)\n", + "\n", "plt.tight_layout();" ] }, @@ -150,7 +151,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can see a bright, extended source in the lower portion of the image. We isolate and examine that region a little more closely. Looking at the output image above, we can see that it is located approximately between science Y coordinates 750 – 1250 and science X coordinates 1750 – 2250." + "We can see a bright, extended source in the lower portion of the image. We isolate and examine that region a little more closely. Looking at the output image above, we can see that it is located approximately between science `Y` coordinates `750 – 1250` and science `X` coordinates `1750 – 2250`." ] }, { @@ -160,11 +161,17 @@ "outputs": [], "source": [ "fig, ax = plt.subplots(figsize=(8, 6))\n", - "sc = ax.imshow(file.data.value[750:1250, 1750:2250], norm=norm, origin='lower')\n", + "\n", + "sc = ax.imshow(file.data.value[750:1250, 1750:2250],\n", + " norm=norm,\n", + " origin='lower')\n", + "\n", "ax.set_xlabel('X Science Axis (pixels)')\n", "ax.set_ylabel('Y Science Axis (pixels)')\n", "ax.set_title('romanisim Simulation WFI16')\n", + "\n", "plt.colorbar(sc, ax=ax)\n", + "\n", "plt.tight_layout();" ] }, @@ -181,10 +188,17 @@ "metadata": {}, "outputs": [], "source": [ - "fig, ax = plt.subplots(figsize=(8, 6), subplot_kw={'projection': file.meta.wcs})\n", - "sc = ax.imshow(file.data.value, norm=norm, origin='lower')\n", + "fig, ax = plt.subplots(figsize=(8, 6), \n", + " subplot_kw={'projection': file.meta.wcs})\n", + "\n", + "sc = ax.imshow(file.data.value,\n", + " norm=norm,\n", + " origin='lower')\n", + "\n", "ax.grid(':', color='white')\n", + "\n", "plt.colorbar(sc, ax=ax)\n", + "\n", "ax.set_xlabel('Right Ascension (deg)')\n", "ax.set_ylabel('Declination (deg)');" ] @@ -193,15 +207,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Interactively Examine the Data Using Imviz\n", + "***\n", + "\n", "\n", - "### Introduction and Setup\n", + "## 3. Examining the Data Interactively using `Imviz`\n", "\n", - "We can also use Imviz, the 2-D image viewer from the Jdaviz package, to visualize and explore the 2-D arrays contained within WFI L2 ASDF files. We highly recommend that users consult the [Imviz documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html), which describes many of the features in Imviz in detail. In this tutorial, we will cover the basics to get you started.\n", + "We can also use `Imviz`, the 2-D image viewer from the Jdaviz package, to visualize and explore the 2-D arrays contained within WFI L2 `ASDF` files. We highly recommend that users consult the [`Imviz` documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html), which describes many of the features in `Imviz` in detail. In this tutorial, we will cover the basics to get you started.\n", "\n", - "**Note:** An enhancement to Cubeviz, the 3-D datacube visualization tool in Jdaviz, will soon allow interactive exploration of WFI L1 data cubes. This functionality is currently being tested and will be available in a future Jdaviz release.\n", + "**Note:** An enhancement to `Cubeviz`, the 3-D datacube visualization tool in `Jdaviz`, will soon allow interactive exploration of WFI L1 data cubes. This functionality is currently being tested and will be available in a future `Jdaviz` release.\n", "\n", - "The first cell below loads Imviz and creates a split panel to the right. You can control the height, or use a pop out window or an inline viewer. For more information, please see the [display options](https://jdaviz.readthedocs.io/en/latest/display.html) documentation." + "The first cell below loads `Imviz` and creates a split panel to the right. You can control the height, or use a pop-out window or an inline viewer. For more information, please see the [display options](https://jdaviz.readthedocs.io/en/latest/display.html) documentation." ] }, { @@ -218,11 +233,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Note:** You may need to expand the Imviz window to see all of the menubar items. You can also minimize the viewer by clicking on the tab on the far-right of your browser that says \"imviz.\" Clicking the \"x\" symbol next to it will close the tab, but clicking on the word \"imviz\" will hide the viewer until it is clicked again.\n", + "**Note:** You may need to expand the `Imviz` window to see all of the menubar items. You can also minimize the viewer by clicking on the tab on the far right of your browser that says `imviz`. Clicking the `x` symbol next to it will close the tab, but clicking on the word `imviz` will hide the viewer until it is clicked again.\n", "\n", "### Loading Data and Programmatically Adjusting Display\n", "\n", - "Next, we load the image into Imviz. By default, for Roman WFI data, Imviz only loads the data array in the viewer to improve performance. Additional arrays (e.g., the data quality array) may be loaded using the `ext` optional argument. An example demonstrating how to load the data quality array is provided in a commented line in the following cell. For more information on the arrays contained within WFI L2 files, please see the RDox article on WFI [Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-L2ScienceDataSpecifications)." + "Next, we load the image into `Imviz`. By default, for Roman WFI data, `Imviz` only loads the data array in the viewer to improve performance. Additional arrays (e.g., the data quality array) may be loaded using the `ext` optional argument. An example demonstrating how to load the data quality array is provided in a commented line in the following cell. For more information on the arrays contained within WFI L2 files, please see the RDox article on WFI [Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-L2ScienceDataSpecifications)." ] }, { @@ -245,7 +260,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The colormap, stretch, and scale limits may be adjusted interactively in the viewer by clicking the plot options icon (three vertically-stacked sliders to the far right of the teal bar above the image display, hereafter the \"tool bar\"), and expanding the \"Plot Options\" section. Note that there are additional options such as the minimum and maximum scale limits under the \"More Stretch Options\" expander.\n", + "The colormap, stretch, and scale limits may be adjusted interactively in the viewer by clicking the plot options icon (three vertically-stacked sliders to the far right of the teal bar above the image display, hereafter the `tool bar`), and expanding the `Plot Options` section. Note that there are additional options such as the minimum and maximum scale limits under the `More Stretch Options` expander.\n", "\n", "If we know the settings we want to apply, we can do so via the API as follows:" ] @@ -267,7 +282,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we will display a second image, showing the same field but slightly dithered, and we will link the two images by their WCS. The second image is dithered by ~100 arcseconds compared to the first, so sources should move by ~1000 pixels between the two images. This is easy to observe with the galaxy in the bottom-left quadrant of the first image, which is located in the bottom-middle of the second image." + "Now we will display a second image, showing the same field but slightly dithered, and we will link the two images by their WCS. The second image is dithered by `~100 arcseconds` compared to the first, so sources should move by `~1000 pixels` between the two images. This is easy to observe with the galaxy in the bottom-left quadrant of the first image, which is located in the bottom-middle of the second image." ] }, { @@ -292,9 +307,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now that the second image has been displayed, we can blink between the two using the \"b\" button. Make sure that your cursor is placed over the image to make it the active window, then blink between the images.\n", + "Now that the second image has been displayed, we can blink between the two using the `b` button. Make sure that your cursor is placed over the image to make it the active window, then blink between the images.\n", "\n", - "Similarly to our matplotlib demonstration, let's investigate a region of interest and focus on the large extended source in the lower portion of the image. In this case, we know the galaxy has science pixel coordinates of (X, Y) ~ (1189, 737) pixels or sky coordinates of (RA, Dec) ~ (0.836, 0.598) degrees. This information can be determined from the catalog used to simulate the products, or may come from interactive examintion of the image or a source detection algorithm. Regardless of how we obtained the coordinates, we can center the viewer on them using the API as shown below:" + "Similarly to our `matplotlib` demonstration, let's investigate a region of interest and focus on the large extended source in the lower portion of the image. In this case, we know the galaxy has science pixel coordinates of `(X, Y) ~ (1189, 737) pixels` or sky coordinates of `(RA, Dec) ~ (0.836, 0.598) degrees`. This information can be determined from the catalog used to simulate the products or may come from interactive examination of the image or a source detection algorithm. Regardless of how we obtained the coordinates, we can center the viewer on them using the API as shown below:" ] }, { @@ -316,12 +331,14 @@ "metadata": {}, "source": [ "We can also set the zoom level to better display the region around the extended source. The zoom level settings are such as:\n", - "* 1: real-pixel-size.\n", - "* 2: zoomed in by a factor of 2.\n", - "* 0.5: zoomed out by a factor of 2.\n", - "* 'fit' means zoomed to fit the whole image into display.\n", + "| Scale Factor | Description |\n", + "|--------------|------------------------------------------|\n", + "| 1 | Real pixel size |\n", + "| 2 | Zoomed in by a factor of 2 |\n", + "| 0.5 | Zoomed out by a factor of 2 |\n", + "| fit | Zoomed to fit the whole image into display |\n", "\n", - "In this case, we will set the zoom level to 0.4 so we can see the extended source in both images and blink between them. Recall that to blink the images, place the cursor over the viewer and press the \"b\" key on your keyboard." + "In this case, we will set the zoom level to `0.4` so we can see the extended source in both images and blink between them. Recall that to blink the images, place the cursor over the viewer and press the `b` key on your keyboard." ] }, { @@ -339,7 +356,7 @@ "source": [ "As you can see, when blinking the images in detector coordinates, the sources seem to move because the second image is slightly dithered. If we link the images by their WCS information, we see that the sources remain fixed, but the regions of the sky observed are slightly different.\n", "\n", - "**Note:** Roman WFI ASDF files use a Generalized World Coordinate System (gwcs) object in Python to store the WCS transformation. The transformation is only well-defined within a bounding box, and moving outside that bounding box produces unexpected behavior, particularly due to the polynomial terms in the transformation between pixel and sky coordinates. As a result, the current version of Imviz might have some difficulties with setting a position and zooming after linking the images by WCS." + "**Note:** Roman WFI `ASDF` files use a [Generalized World Coordinate System (gwcs)](https://gwcs.readthedocs.io/en/latest/) object in Python to store the WCS transformation. The transformation is only well-defined within a bounding box, and moving outside that bounding box produces unexpected behavior, particularly due to the polynomial terms in the transformation between pixel and sky coordinates. As a result, the current version of `Imviz` might have some difficulties with setting a position and zooming after linking the images by WCS." ] }, { @@ -356,7 +373,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "You can also save the current view to a PNG file on your cloud storage:" + "You can also save the current view to a `PNG` file on your cloud storage:" ] }, { @@ -372,7 +389,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This should save the file to your current working directory on the the science platform. If you want to download this file, you can do so by right-clicking on the file in the file browser and selecting the \"Download\" option.\n", + "This should save the file to your current working directory on the science platform. If you want to download this file, you can do so by right-clicking on the file in the file browser and selecting the `Download` option.\n", "\n", "### Overlaying Catalog Data\n", "\n", @@ -395,7 +412,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "There are a lot of sources in this file, but let's pare them down somewhat for display purposes. In this case, let's filter down to the brightest ($m_{AB}$ < 15) sources in the F158 filter. The flux columns in the table are in units of maggies, which may be converted to AB magnitudes as $m_{AB} = -2.5\\log_{10}(f)$ where $m_{AB}$ is the magnitude in AB mags and $f$ is the flux in maggies." + "There are a lot of sources in this file, but let's pare them down somewhat for display purposes. In this case, let's filter down to the brightest ($m_{AB}$ < 15) sources in the `F158` filter. The flux columns in the table are in units of maggies, which may be converted to AB magnitudes as $m_{AB} = -2.5\\log_{10}(f)$ where $m_{AB}$ is the magnitude in AB mags and $f$ is the flux in maggies." ] }, { @@ -453,37 +470,26 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "For more advanced use cases such as interactive aperture photometry or analysis of line profiles, please consult the [Imviz documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Aditional Resources\n", + "For more advanced use cases such as interactive aperture photometry or analysis of line profiles, please consult the [`Imviz` documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html).\n", + "\n", + "***\n", + "\n", + "## Additional Resources\n", + "\n", + "- [`Imviz` Documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html)\n", + "\n", + "- [Additional `JDaviz` Notebooks](https://github.com/spacetelescope/jdaviz/tree/main/notebooks)\n", + "\n", + "- [RDox WFI Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-L2ScienceDataSpecifications)\n", + "\n", + "\n", + "**For additional support, please contact the [Roman Help Desk at STScI](https://stsci.service-now.com/roman).**\n", "\n", - "- [Imviz Documentation](https://jdaviz.readthedocs.io/en/latest/imviz/index.html)\n", - "- [Additional JDaviz Notebooks](https://github.com/spacetelescope/jdaviz/tree/main/notebooks)\n", - "- [RDox WFI Data Levels and Products](https://roman-docs.stsci.edu/data-handbook-home/wfi-data-format/data-levels-and-products#DataLevelsandProducts-L2ScienceDataSpecifications)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ "## About this notebook\n", "**Author:** Tyler Desjardins, Brett Morris \n", - "**Updated On:** 2024-09-20" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "\n", + "**Updated On:** 2024-11-06\n", + "\n", "***" ] },