Tidy up of the io doc strings and API page. Deprecate toplevel io namespace.

CHANGELOG.rst

@@ -125,7 +125,7 @@ New Features
 - Adds two tutorials that demonstrate how to use LASCO data in overlaying maps (:ref:`sphx_glr_generated_gallery_plotting_lasco_overlay.py`) and how to create a custom mask for a LASCO C2 image (:ref:`sphx_glr_generated_gallery_map_lasco_mask.py`). (`#6576 <https://github.com/sunpy/sunpy/pull/6576>`__)
 - Able to run the ``sunpy`` tests doing ``python -m sunpy.tests.self_test``. (`#6600 <https://github.com/sunpy/sunpy/pull/6600>`__)
 - Able to detect gzip-compressed FITS files even if they don't have the ``.gz`` extension in the filename.
-  `~sunpy.io.detect_filetype` now looks for the right file signature while checking
+  ``sunpy.io.detect_filetype`` now looks for the right file signature while checking
   for gzipped FITS files. (`#6693 <https://github.com/sunpy/sunpy/pull/6693>`__)
 - Added ``AttrAnd`` and ``AttrOr`` to the namespace in ``sunpy.net.attrs``.
   This allows users to to avoid ``|`` or ``&`` when creating a query a larger query. (`#6708 <https://github.com/sunpy/sunpy/pull/6708>`__)
@@ -850,7 +850,7 @@ Bug Fixes
   location was a fully specified Stonyhurst heliographic coordinate. (`#5584 <https://github.com/sunpy/sunpy/pull/5584>`__)
 - `~sunpy.map.sources.XRTMap` uppercases the ``TIMESYS`` key before checking if the
   key needs to be fixed. (`#5592 <https://github.com/sunpy/sunpy/pull/5592>`__)
-- Fixed passing a URL to :func:`sunpy.io.read_file` on windows. (`#5601 <https://github.com/sunpy/sunpy/pull/5601>`__)
+- Fixed passing a URL to ``sunpy.io.read_file`` on windows. (`#5601 <https://github.com/sunpy/sunpy/pull/5601>`__)
 - Fixed a bug where the ``date`` property on `~sunpy.map.sources.HMISynopticMap` returned ``None``
   if the ``DATE-OBS`` key was present. (`#5648 <https://github.com/sunpy/sunpy/pull/5648>`__)
 
@@ -1718,7 +1718,7 @@ Improved Documentation
   The page :ref:`sunpy-topic-guide-coordinates-rotatedsunframe` has an example of a mini-gallery at the bottom. (`#4124 <https://github.com/sunpy/sunpy/pull/4124>`__)
 - Added `sunpy.visualization.colormaps.color_tables` to the docs. (`#4182 <https://github.com/sunpy/sunpy/pull/4182>`__)
 - Made minor improvements to the map histogramming example. (`#4205 <https://github.com/sunpy/sunpy/pull/4205>`__)
-- Add a warning to `sunpy.io` docs to recommend not using it for FITS (`#4208 <https://github.com/sunpy/sunpy/pull/4208>`__)
+- Add a warning to ``sunpy.io`` docs to recommend not using it for FITS (`#4208 <https://github.com/sunpy/sunpy/pull/4208>`__)
 
 
 Trivial/Internal Changes
@@ -1824,7 +1824,7 @@ Features
   objects. (`#3408 <https://github.com/sunpy/sunpy/pull/3408>`__)
 - `~sunpy.map.GenericMap` objects now have a ``.cmap`` attribute, which returns the full `~matplotlib.colors.Colormap`.
   object. (`#3412 <https://github.com/sunpy/sunpy/pull/3412>`__)
-- `sunpy.io.write_file()` now accepts `~pathlib.Path` objects as filename inputs. (`#3469 <https://github.com/sunpy/sunpy/pull/3469>`__)
+- ``sunpy.io.write_file`` now accepts `~pathlib.Path` objects as filename inputs. (`#3469 <https://github.com/sunpy/sunpy/pull/3469>`__)
 - :func:`sunpy.map.header_helper.make_fitswcs_header` now accepts a `tuple` representing the shape of an array as well as the actual array as the ``data`` argument. (`#3483 <https://github.com/sunpy/sunpy/pull/3483>`__)
 - Made a couple of module imports lazy to reduce the import time of sunpy.map by
   ~40%. (`#3495 <https://github.com/sunpy/sunpy/pull/3495>`__)

changelog/6736.feature.rst

@@ -1 +1 @@
-:func:`sunpy.io.read_file` will now try to detect the filetype based on the content and then fallback to using the file extension.
+``sunpy.io.read_file`` will now try to detect the filetype based on the content and then fallback to using the file extension.

changelog/7537.breaking.rst

@@ -0,0 +1,2 @@
+``sunpy.io.read_file`` and ``sunpy.io.write_file`` are deprecated and will be removed in the future.
+These were intended to be private functions and should not be used.

docs/dev_guide/contents/internal_api.rst

@@ -0,0 +1,15 @@
+.. _sunpy-dev-guide-internal-api:
+
+**************************
+Internal API Documentation
+**************************
+
+This page contains the documentation for the internal API of `sunpy`.
+
+.. automodapi:: sunpy.io._file_tools
+
+.. automodapi:: sunpy.io._header
+
+.. automodapi:: sunpy.io._fits
+
+.. automodapi:: sunpy.io._jp2

docs/dev_guide/index.rst

@@ -71,6 +71,7 @@ This section contains the various guidelines to be followed by anyone working on
             contents/public_api
             contents/logger
             contents/remote_data_manager
+            contents/internal_api
 
 {%else%}
 

docs/how_to/index.rst

@@ -13,13 +13,14 @@ If you're starting fresh you might want to check out the :ref:`sunpy-tutorial-in
 
    coord_components
    create_a_map
-   create_rectangle_on_map
    create_coords
    create_custom_map
    create_custom_timeseries
+   create_rectangle_on_map
    fix_map_metadata
    manipulate_grid_lines
    parse_time
+   read_asdf_file
    remote_data_manager
    search_multiple_wavelengths
    search_vso

docs/how_to/read_asdf_file.rst

@@ -0,0 +1,67 @@
+.. _sunpy-how-to-read-an-asdf-file:
+
+****************************
+Read an ASDF file into a Map
+****************************
+
+`ASDF <https://asdf-standard.readthedocs.io/en/latest/>`__ is a modern file format designed to meet the needs of the astronomy community [citation needed].
+It has deep integration with Python, sunpy, and Astropy, as well as implementations in other languages.
+It can be used to store known Python objects in a portable, well defined file format.
+It is primarily useful for storing complex Astropy and sunpy objects in a way that can be loaded back into the same form as they were saved.
+It is designed to be an archive file format, with human readable metadata and a simple on-disk layout.
+
+sunpy currently implements support for saving `Map <sunpy.map.GenericMap>` and `coordinate frame <sunpy.coordinates.frames>` objects into asdf files.
+As asdf tightly integrates into Python, saving a map to an asdf file will save the metadata, data and mask.
+In comparison, the mask is not currently saved to FITS.
+
+The following code shows to to save and load a sunpy Map to an asdf file:
+
+.. code-block:: python
+
+    >>> import asdf
+
+    >>> import sunpy.map
+    >>> from sunpy.data.sample import AIA_171_IMAGE  # doctest: +REMOTE_DATA
+
+    >>> aiamap = sunpy.map.Map(AIA_171_IMAGE)  # doctest: +REMOTE_DATA
+    >>> tree = {'amap': aiamap}  # doctest: +REMOTE_DATA
+    >>> with asdf.AsdfFile(tree) as asdf_file:  # doctest: +REMOTE_DATA
+    ...     asdf_file.write_to("sunpy_map.asdf")  # doctest: +REMOTE_DATA
+    >>> input_asdf = asdf.open("sunpy_map.asdf")  # doctest: +REMOTE_DATA
+    >>> input_asdf['amap']  # doctest: +REMOTE_DATA
+        <sunpy.map.sources.sdo.AIAMap object at ...>
+        SunPy Map
+        ---------
+        Observatory:                 SDO
+        Instrument:          AIA 3
+        Detector:            AIA
+        Measurement:                 171.0 Angstrom
+        Wavelength:          171.0 Angstrom
+        Observation Date:    2011-06-07 06:33:02
+        Exposure Time:               0.234256 s
+        Dimension:           [1024. 1024.] pix
+        Coordinate System:   helioprojective
+        Scale:                       [2.402792 2.402792] arcsec / pix
+        Reference Pixel:     [511.5 511.5] pix
+        Reference Coord:     [3.22309951 1.38578135] arcsec
+        array([[ -95.92475  ,    7.076416 ,   -1.9656711, ..., -127.96519  ,
+                -127.96519  , -127.96519  ],
+            [ -96.97533  ,   -5.1167884,    0.       , ...,  -98.924576 ,
+                -104.04137  , -127.919716 ],
+            [ -93.99607  ,    1.0189276,   -4.0757103, ...,   -5.094638 ,
+                -37.95505  , -127.87541  ],
+            ...,
+            [-128.01454  , -128.01454  , -128.01454  , ..., -128.01454  ,
+                -128.01454  , -128.01454  ],
+            [-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
+                -127.899666 , -127.899666 ],
+            [-128.03072  , -128.03072  , -128.03072  , ..., -128.03072  ,
+                -128.03072  , -128.03072  ]], dtype=float32)
+    >>> input_asdf.close()  # doctest: +REMOTE_DATA
+
+When saving a Map to ASDF, all maps are saved as a `.GenericMap` and not a specific source class.
+This comes with some trade-offs.
+If you are using custom map sources defined outside of the `sunpy` core package, and these sources are imported after asdf has been invoked for the first time (used, not just imported), then they will not be registered with the asdf converter.
+Also if the custom map subclass is not registered with `sunpy.map.Map` upon loading of the map, it will be returned as a `.GenericMap`.
+This approach has been chosen despite these limitations so that once a map is saved to an ASDF file it can always be loaded back into a map rather than the asdf library returning it as a Python dictionary.
+It also follows the philosophy of the way maps are saved and loaded in the FITS format, where the components of the Map are serialized and the way meta data is handled depends solely on the contents of the ``.meta`` attribute.

docs/reference/io.rst

@@ -1,112 +1,26 @@
-Input/output (`sunpy.io`)
-*************************
+Input/output (``sunpy.io``)
+***************************
 
+The primary focus of ``sunpy.io`` is to provide a common interface for reading and writing.
 ``sunpy.io`` contains readers for files that are commonly used in solar physics.
 
-Unified File Readers
-====================
+These include:
 
-.. automodapi:: sunpy.io
+- Public API
+  - GENX
+  - ANA
+  - NOAA SWPC Solar Region Summary (SRS)
+  - ASDF
 
-.. automodapi:: sunpy.io.ana
+The other readers are intended for use by `sunpy.map` and `sunpy.timeseries` and are not intended to be used directly by users.
 
 Special File Readers
 ====================
 
 .. automodapi:: sunpy.io.special.genx
 
-``sunpy`` has a custom reader for NOAA SWPC Solar Region Summary (SRS) files:
+.. automodapi:: sunpy.io.ana
 
 .. automodapi:: sunpy.io.special.srs
 
-asdf (Advanced Scientific Data Format)
---------------------------------------
-
-`ASDF <https://asdf-standard.readthedocs.io/en/latest/>`__ is a modern file format
-designed to meet the needs of the astronomy community. It has deep integration
-with Python, SunPy, and Astropy, as well as implementations in other languages.
-It can be used to store known Python objects in a portable, well defined file
-format. It is primarily useful for storing complex Astropy and SunPy objects in
-a way that can be loaded back into the same form as they were saved.
-It is designed to be an archive file format, with human readable metadata and a
-simple on-disk layout.
-
-sunpy currently implements support for saving `Map <sunpy.map.GenericMap>` and
-`coordinate frame <sunpy.coordinates.frames>` objects into asdf files. As asdf
-tightly integrates into Python, saving a map to an asdf file will save the
-metadata, data and mask. In comparison, the mask is not currently saved
-to FITS. The following code shows to to save and load a sunpy Map to an asdf
-file
-
-.. doctest-requires:: asdf
-
-  >>> import asdf
-  >>> import sunpy.map
-  >>> from sunpy.data.sample import AIA_171_IMAGE  # doctest: +REMOTE_DATA
-  >>> aiamap = sunpy.map.Map(AIA_171_IMAGE)  # doctest: +REMOTE_DATA
-  >>> tree = {'amap': aiamap}  # doctest: +REMOTE_DATA
-  >>> with asdf.AsdfFile(tree) as asdf_file:  # doctest: +REMOTE_DATA
-  ...     asdf_file.write_to("sunpy_map.asdf")  # doctest: +REMOTE_DATA
-  >>> input_asdf = asdf.open("sunpy_map.asdf")  # doctest: +REMOTE_DATA
-  >>> input_asdf['amap']  # doctest: +REMOTE_DATA
-    <sunpy.map.sources.sdo.AIAMap object at ...>
-    SunPy Map
-    ---------
-    Observatory:                 SDO
-    Instrument:          AIA 3
-    Detector:            AIA
-    Measurement:                 171.0 Angstrom
-    Wavelength:          171.0 Angstrom
-    Observation Date:    2011-06-07 06:33:02
-    Exposure Time:               0.234256 s
-    Dimension:           [1024. 1024.] pix
-    Coordinate System:   helioprojective
-    Scale:                       [2.402792 2.402792] arcsec / pix
-    Reference Pixel:     [511.5 511.5] pix
-    Reference Coord:     [3.22309951 1.38578135] arcsec
-    array([[ -95.92475  ,    7.076416 ,   -1.9656711, ..., -127.96519  ,
-            -127.96519  , -127.96519  ],
-           [ -96.97533  ,   -5.1167884,    0.       , ...,  -98.924576 ,
-            -104.04137  , -127.919716 ],
-           [ -93.99607  ,    1.0189276,   -4.0757103, ...,   -5.094638 ,
-             -37.95505  , -127.87541  ],
-           ...,
-           [-128.01454  , -128.01454  , -128.01454  , ..., -128.01454  ,
-            -128.01454  , -128.01454  ],
-           [-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
-            -127.899666 , -127.899666 ],
-           [-128.03072  , -128.03072  , -128.03072  , ..., -128.03072  ,
-            -128.03072  , -128.03072  ]], dtype=float32)
-   >>> input_asdf.close()  # doctest: +REMOTE_DATA
-
-When saving a Map to ASDF all maps are saved as a `.GenericMap` and not a specific source class.
-This comes with some trade-offs.
-If you are using custom map sources defined outside of the `sunpy` core package, and these sources are imported after asdf has been invoked for the first time (used, not just imported), then they will not be registered with the asdf converter.
-Also if the custom map subclass is not registered with `sunpy.map.Map` upon loading of the map, it will be returned as a `.GenericMap`.
-This approach has been chosen despite these limitations so that once a map is saved to an ASDF file it can always be loaded back into a map rather than the asdf library returning it as a Python dictionary.
-It also follows the philosophy of the way maps are saved and loaded in the FITS format, where the components of the Map are serialised and the way meta data is handled depends solely on the contents of the ``.meta`` attribute.
-
-Internal API
-============
-
-These are readers and other utilities that are used internally by ``sunpy`` to read files.
-They are not intended to be used directly by users and we do not guarantee that they will
-work for all files of a given type nor will the API be stable.
-
-.. automodapi:: sunpy.io._fits
-
-.. automodapi:: sunpy.io.header
-
-.. automodapi:: sunpy.io._jp2
-
-.. automodapi:: sunpy.io._file_tools
-
-CDF (common data format)
-------------------------
-
-CDF files are commonly used to store timeseries data observed by instruments
-taking in-situ measurements of plasmas throughout the heliosphere.
-sunpy provides support for reading in CDF files that conform to the
-`Space Physics Guidelines for CDF <https://spdf.gsfc.nasa.gov/sp_use_of_cdf.html>`_.
-
-.. automodapi:: sunpy.io._cdf
+.. automodapi:: sunpy.io.special.asdf

sunpy/io/__init__.py

@@ -1 +1,20 @@
-from sunpy.io._file_tools import *
+"""
+This subpackage contains a series of file readers and only for internal use.
+
+None of this code is intended to be used directly by a user and is not part of the sunpy public API
+and is not added to the API reference pages.
+"""
+from sunpy.io._file_tools import read_file as _read_file
+from sunpy.io._file_tools import write_file as _write_file
+from sunpy.util.decorators import deprecated
+
+
+@deprecated("6.0", message="The toplevel space of the io subpackage was never intended for public use and will be removed in the future.")
+def read_file(*args, **kwargs):
+    return _read_file(*args, **kwargs)
+
+@deprecated("6.0", message="The toplevel space of the io subpackage was never intended for public use and will be removed in the future.")
+def write_file(*args, **kwargs):
+    return _write_file(*args, **kwargs)
+
+__all__ = ['read_file', 'write_file']

sunpy/io/_file_tools.py

@@ -125,7 +125,7 @@ def read_file_header(filepath, filetype=None, **kwargs):
     """
     Reads the header from a given file.
 
-    This should always return a instance of `~sunpy.io.header.FileHeader`.
+    This should always return a instance of `~sunpy.io._header.FileHeader`.
 
     Parameters
     ----------

sunpy/io/_fits.py

@@ -1,9 +1,6 @@
 """
 This module provides a FITS file reader for internal use.
 
-We instead recommend users use the `astropy.io.fits` module,
-which provides more generic functionality to read FITS files.
-
 Notes
 -----
 1. FITS files allow comments to be attached to every value in the header.
@@ -35,7 +32,7 @@
 
 from astropy.io import fits
 
-from sunpy.io.header import FileHeader
+from sunpy.io._header import FileHeader
 from sunpy.util.exceptions import warn_metadata, warn_user
 from sunpy.util.io import HDPair
 
@@ -108,7 +105,7 @@ def get_header(afile):
     Returns
     -------
     `list`
-        A list of `sunpy.io.header.FileHeader` headers.
+        A list of `sunpy.io._header.FileHeader` headers.
     """
     if isinstance(afile, fits.HDUList):
         hdulist = afile
@@ -141,7 +138,7 @@ def format_comments_and_history(input_header):
 
     Returns
     -------
-    `sunpy.io.header.FileHeader`
+    `sunpy.io._header.FileHeader`
     """
     try:
         comment = "\n".join(input_header['COMMENT']).strip()
@@ -290,8 +287,8 @@ def extract_waveunit(header):
 
     Parameters
     ----------
-    header : `sunpy.io.header.FileHeader`
-        One `~sunpy.io.header.FileHeader` instance which was created by
+    header : `sunpy.io._header.FileHeader`
+        One `~sunpy.io._header.FileHeader` instance which was created by
         reading a FITS file. For example, `sunpy.io._fits.get_header` returns a list of
         such instances.
 

sunpy/io/_jp2.py

@@ -1,16 +1,13 @@
 """
 This module provides a JPEG 2000 file reader for internal use.
-
-It was developed to read JPEG 2000 files created by the Helioviewer Project
-and not as a general JPEG 2000 file reader.
 """
 import os
 
 # We have to use lxml as lxml can not  serialize xml from the standard library
 import lxml.etree as ET
 import numpy as np
 
-from sunpy.io.header import FileHeader
+from sunpy.io._header import FileHeader
 from sunpy.util.io import HDPair, string_is_float
 
 __all__ = ['read', 'get_header', 'write']

sunpy/io/ana.py

@@ -12,7 +12,7 @@
 """
 import os
 
-from sunpy.io.header import FileHeader
+from sunpy.io._header import FileHeader
 from sunpy.util.io import HDPair
 
 try:
@@ -78,7 +78,7 @@ def get_header(filename, debug=False):
     Returns
     -------
     `list`
-        A list of `~sunpy.io.header.FileHeader` headers.
+        A list of `~sunpy.io._header.FileHeader` headers.
 
     Examples
     --------
@@ -102,7 +102,7 @@ def write(filename, data, comments=False, compress=True, debug=False):
         Name of file to be created.
     data : `numpy.ndarray`
         The data to be stored.
-    comments : `~sunpy.io.header.FileHeader`, optional
+    comments : `~sunpy.io._header.FileHeader`, optional
         The comments to be stored as a header.
     compress : `bool`, optional
         Compress the data with `True` (the default).