Update GitHub docs

.github/workflows/documentation.yml

@@ -10,13 +10,20 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # Fetch all history and tags for setuptools_scm
       - uses: actions/setup-python@v5
         with:
           python-version: '3.9'
-      - name: Install documentation dependencies
+      - name: Install build tools
         run: |
           pip install --upgrade pip setuptools wheel setuptools_scm
-          pip install sphinx sphinx_rtd_theme myst_parser numpydoc sphinx-astropy
+      - name: Install hipercam package
+        run: |
+          pip install .
+      - name: Install documentation dependencies
+        run: |
+          pip install sphinx sphinx_rtd_theme sphinx-autoapi myst_parser numpydoc sphinx-astropy
       - name: Sphinx build
         run: |
           sphinx-build docs _build

README.rst

@@ -8,7 +8,7 @@ hipercam pipeline can also be used to reduce data from the ULTRACAM
 and ULTRASPEC high-speed cameras, and other instruments too.
 
 More information about using the hipercam package can be found in
-`the documentation <https://vikdhillon.staff.shef.ac.uk/hipercam/pipeline/index.html>`_.
+`the documentation <https://hipercam.github.io/hipercam>`_.
 
 Docker Installation
 ===================
@@ -18,9 +18,8 @@ is simple and cross-platform, but the image is large (around 2.8GB). To
 install via this route, please download the
 `dockerfile <https://raw.githubusercontent.com/HiPERCAM/hipercam/refs/heads/master/data/hipercam.dockerfile>`_
 
-+++++
 Linux
-+++++
+-----
 
 To build the docker image from the docker file, run::
 
@@ -33,9 +32,8 @@ And, to run the image, use::
 This will mount `<local-path-to-some-data>` in the virtual machine at
 `/home/hiperuser/data`. You can then run the pipeline as normal.
 
-+++++
 MacOS
-+++++
+-----
 
 Installation on Apple Silicon Macs is a little more complicated. The instructions below assume you have a modern
 MacOS/Apple Silicon system with Docker Desktop installed.
@@ -84,14 +82,14 @@ Building from Source
 You can build the package from source using the standard Python build tools.
 
 Prerequisites
-+++++++++++++
+-------------
 
 Make sure you have the required build tools installed::
 
   pip install build
 
 Basic Build Commands
-++++++++++++++++++++
+--------------------
 
 To build a source distribution (tarball)::
 
@@ -206,7 +204,7 @@ your O/S package manager. e.g. under fedora, Cython appears as
 
   trm.utils :
          generally useful routines used at a few places. Available
-	 from PyPi.
+         from PyPi.
 
   websocket-client :
          for talking to the hipercam server.
@@ -229,8 +227,7 @@ Further Information
 
 For more information see:
 
-  * `The documentation
-    <https://vikdhillon.staff.shef.ac.uk/hipercam/pipeline/index.html>`_
+  * `The documentation <https://hipercam.github.io/hipercam>`_
 
   * `The HiPERCAM pipeline github repository <https://github.com/HiPERCAM/hipercam>`_
 

docs/change_log_1.1.0_1.2.0.rst

@@ -66,7 +66,7 @@ List of changes from git, newest first, with the commit keys linked to github:
   * `e881e9bd98f65f9c1dcfacb3ae7fed85a104179b <https://github.com/HiPERCAM/hipercam/commit/e881e9bd98f65f9c1dcfacb3ae7fed85a104179b>`_ added plotting of the max counts in plog
   * `db1706632c88449e541a8f077f3454149d68802c <https://github.com/HiPERCAM/hipercam/commit/db1706632c88449e541a8f077f3454149d68802c>`_ cline now sets the default for the ignore value
   * `781022e2430bd0cda4a286b134b340afe8b04030 <https://github.com/HiPERCAM/hipercam/commit/781022e2430bd0cda4a286b134b340afe8b04030>`_ further adjustment to docs
-  * `bc713d7b9f6d1e8ca3ee10e5a9a49e504a02c032 <https://github.com/HiPERCAM/hipercam/commit/bc713d7b9f6d1e8ca3ee10e5a9a49e504a02c032>`_ utils.py strips STD_ from start of target names
+  * `bc713d7b9f6d1e8ca3ee10e5a9a49e504a02c032 <https://github.com/HiPERCAM/hipercam/commit/bc713d7b9f6d1e8ca3ee10e5a9a49e504a02c032>`_ utils.py strips `STD_` from start of target names
   * `f6f2d350386d489cbd8c88d6cd04c589cd9bd97a <https://github.com/HiPERCAM/hipercam/commit/f6f2d350386d489cbd8c88d6cd04c589cd9bd97a>`_ doc updates to describe phase II process in the main
   * `bb76ebc1d62cdc41b28d0dcb168acec252e95255 <https://github.com/HiPERCAM/hipercam/commit/bb76ebc1d62cdc41b28d0dcb168acec252e95255>`_ further tweak following change to defect file
   * `721e56070fb4a03bf5dfb511a349828c5f6f4806 <https://github.com/HiPERCAM/hipercam/commit/721e56070fb4a03bf5dfb511a349828c5f6f4806>`_ new hipercam defects file following Vik's cleaning of CCD 2

docs/commands.rst

@@ -54,7 +54,7 @@
 .. |ol-sub|      replace:: subtract two frames
 .. |ol-uls|      replace:: list the runs on the ULTRACAM server
 
-.. highlightlang:: rest
+.. highlight:: rest
 
 |hiper| commands
 ****************
@@ -449,54 +449,54 @@ containing multiple images, stored in a .fits file. ``frame`` refers to a
 single image from a run as might be extracted using |grab|. These have file
 extension '.hcm' to distinguish them, although they are also FITS-format files.
 
-.. autofunction:: hipercam.scripts.add
-.. autofunction:: hipercam.scripts.averun
-.. autofunction:: hipercam.scripts.cadd
-.. autofunction:: hipercam.scripts.cdiv
-.. autofunction:: hipercam.scripts.cmul
-.. autofunction:: hipercam.scripts.combine
-.. autofunction:: hipercam.scripts.csub
-.. autofunction:: hipercam.scripts.div
-.. autofunction:: hipercam.scripts.exploss
-.. autofunction:: hipercam.scripts.fits2hcm
-.. autofunction:: hipercam.scripts.flagcloud
-.. autofunction:: hipercam.scripts.ftargets
-.. autofunction:: hipercam.scripts.genred
-.. autofunction:: hipercam.scripts.grab
-.. autofunction:: hipercam.scripts.hfilter
-.. autofunction:: hipercam.scripts.hinfo
-.. autofunction:: hipercam.scripts.hist
-.. autofunction:: hipercam.scripts.hlog2col
-.. autofunction:: hipercam.scripts.hlog2fits
-.. autofunction:: hipercam.scripts.hls
-.. autofunction:: hipercam.scripts.hplot
-.. autofunction:: hipercam.scripts.hpackage
-.. autofunction:: hipercam.scripts.joinup
-.. autofunction:: hipercam.scripts.ltimes
-.. autofunction:: hipercam.scripts.ltrans
-.. autofunction:: hipercam.scripts.makebias
-.. autofunction:: hipercam.scripts.makedark
-.. autofunction:: hipercam.scripts.makeflat
-.. autofunction:: hipercam.scripts.makefringe
-.. autofunction:: hipercam.scripts.makemovie
-.. autofunction:: hipercam.scripts.mstats
-.. autofunction:: hipercam.scripts.mul
-.. autofunction:: hipercam.scripts.ncal
-.. autofunction:: hipercam.scripts.nrtplot
-.. autofunction:: hipercam.scripts.pbands
-.. autofunction:: hipercam.scripts.plog
-.. autofunction:: hipercam.scripts.redanal
-.. autofunction:: hipercam.scripts.reduce
-.. autofunction:: hipercam.scripts.rtplot
-.. autofunction:: hipercam.scripts.rupdate
-.. autofunction:: hipercam.scripts.setaper
-.. autofunction:: hipercam.scripts.setdefect
-.. autofunction:: hipercam.scripts.setfringe
-.. autofunction:: hipercam.scripts.shiftadd
-.. autofunction:: hipercam.scripts.splice
-.. autofunction:: hipercam.scripts.stats
-.. autofunction:: hipercam.scripts.sub
-.. autofunction:: hipercam.scripts.uls
+.. autoapifunction:: hipercam.scripts.add
+.. autoapifunction:: hipercam.scripts.averun
+.. autoapifunction:: hipercam.scripts.cadd
+.. autoapifunction:: hipercam.scripts.cdiv
+.. autoapifunction:: hipercam.scripts.cmul
+.. autoapifunction:: hipercam.scripts.combine
+.. autoapifunction:: hipercam.scripts.csub
+.. autoapifunction:: hipercam.scripts.div
+.. autoapifunction:: hipercam.scripts.exploss
+.. autoapifunction:: hipercam.scripts.fits2hcm
+.. autoapifunction:: hipercam.scripts.flagcloud
+.. autoapifunction:: hipercam.scripts.ftargets
+.. autoapifunction:: hipercam.scripts.genred
+.. autoapifunction:: hipercam.scripts.grab
+.. autoapifunction:: hipercam.scripts.hfilter
+.. autoapifunction:: hipercam.scripts.hinfo
+.. autoapifunction:: hipercam.scripts.hist
+.. autoapifunction:: hipercam.scripts.hlog2col
+.. autoapifunction:: hipercam.scripts.hlog2fits
+.. autoapifunction:: hipercam.scripts.hls
+.. autoapifunction:: hipercam.scripts.hplot
+.. autoapifunction:: hipercam.scripts.hpackage
+.. autoapifunction:: hipercam.scripts.joinup
+.. autoapifunction:: hipercam.scripts.ltimes
+.. autoapifunction:: hipercam.scripts.ltrans
+.. autoapifunction:: hipercam.scripts.makebias
+.. autoapifunction:: hipercam.scripts.makedark
+.. autoapifunction:: hipercam.scripts.makeflat
+.. autoapifunction:: hipercam.scripts.makefringe
+.. autoapifunction:: hipercam.scripts.makemovie
+.. autoapifunction:: hipercam.scripts.mstats
+.. autoapifunction:: hipercam.scripts.mul
+.. autoapifunction:: hipercam.scripts.ncal
+.. autoapifunction:: hipercam.scripts.nrtplot
+.. autoapifunction:: hipercam.scripts.pbands
+.. autoapifunction:: hipercam.scripts.plog
+.. autoapifunction:: hipercam.scripts.redanal
+.. autoapifunction:: hipercam.scripts.reduce
+.. autoapifunction:: hipercam.scripts.rtplot
+.. autoapifunction:: hipercam.scripts.rupdate
+.. autoapifunction:: hipercam.scripts.setaper
+.. autoapifunction:: hipercam.scripts.setdefect
+.. autoapifunction:: hipercam.scripts.setfringe
+.. autoapifunction:: hipercam.scripts.shiftadd
+.. autoapifunction:: hipercam.scripts.splice
+.. autoapifunction:: hipercam.scripts.stats
+.. autoapifunction:: hipercam.scripts.sub
+.. autoapifunction:: hipercam.scripts.uls
 
 .. [#f1] Several other commands (``aligntool``, ``atanalysis``, ``atbytes``,
          ``calsearch``, ``harchive``, ``hlogger``, ``hmeta``, ``logsearch``,

docs/conf.py

@@ -31,25 +31,32 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['sphinx.ext.autodoc', 'sphinx.ext.autosummary',
-              'sphinx.ext.imgmath', 'numpydoc']
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.imgmath',
+    'numpydoc',
+    'autoapi.extension',
+]
 
 # this needed for numpydoc I think
 autosummary_generate = True
 numpydoc_show_class_members = False
 numpydoc_show_inherited_class_members = False
 
+
+# Autoapi settings
+autoapi_dirs = ['../hipercam']
+autoapi_generate_api_docs = False  # Don't auto-generate, we'll write e.g. scripts in commands.rst
+
 # number figures
 numfig = True
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
 # The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-#
-# source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
+source_suffix = {'.rst': 'restructuredtext'}
 
 # The master toctree document.
 master_doc = 'index'
@@ -66,21 +73,24 @@
     # Try to get version from installed package first
     from importlib.metadata import version as _version
     release = _version('hipercam')
+    print(f"[Sphinx] Using installed package version: {release}")
 except ImportError:
     # Fallback to setuptools_scm if package not installed
     try:
         from setuptools_scm import get_version
         release = get_version(root='..', relative_to=__file__)
-    except (ImportError, LookupError):
+        print(f"[Sphinx] Using setuptools_scm version: {release}")
+    except (ImportError, LookupError) as e:
         # Final fallback to a default version
         release = "unknown"
+        print(f"[Sphinx] WARNING: Could not determine version ({e}), using: {release}")
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
 #
 # This is also used if you do content translation via gettext catalogs.
 # Usually you set "language" from the command line for these cases.
-language = None
+language = "en"
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.

docs/files.rst

@@ -2,7 +2,7 @@
 
 .. include:: globals.rst
 
-.. highlightlang:: rest
+.. highlight:: rest
 
 Useful files
 ************

docs/index.rst

@@ -9,7 +9,7 @@ The |hiper| pipeline manual
 ****************************
 
 The |hiper| reduction software, known as the "pipeline" for short,
-serves two purposes. First, it provides a means of displaying and reducing 
+serves two purposes. First, it provides a means of displaying and reducing
 |hiper|, ULTRACAM and ULTRASPEC data. Second,
 it provides an "Application Programmers Interface" (API) to allow you
 to access and manipulate the same sets of data. The first will be of
@@ -19,7 +19,7 @@ aspects. It does not cover use of the |hiper| GUI or the finding chart
 generator, hfinder, or the equivalent components for ULTRACAM and
 ULTRASPEC. Details of these for |hiper| may be found on `github
 <https://github.com/HiPERCAM>`_; and follows these links for much
-more information on 
+more information on
 `ULTRACAM <http://www.vikdhillon.staff.shef.ac.uk/ultracam/userman/userman.html>`_ and
 `ULTRASPEC on the TNT <http://www.vikdhillon.staff.shef.ac.uk/ultraspec/ultraspec_tnt.html>`_.
 

docs/installation.rst

@@ -18,7 +18,7 @@ Updating
 
 You should keep the directory 'hipercam' created by the first ``git clone``
 command because then it makes updating simple. It becomes a matter of going
-to the hipercam directory, pulling the changes from github, and 
+to the hipercam directory, pulling the changes from github, and
 re-installing::
 
   cd XXX/YYY/hipercam
@@ -85,4 +85,3 @@ HIPERCAM_ENV             Directory for storage of default parameter files
                          that it does not contain too much junk.
 ULTRACAM_DEFAULT_URL     URL of the ULTRACAM FileServer on the rack PC to access data
 =======================  ==============================================================
-

docs/makingmovies.rst

@@ -2,7 +2,7 @@
 
 .. include:: globals.rst
 
-.. highlightlang:: rest
+.. highlight:: rest
 
 Making |hiper| movies
 *********************
@@ -91,4 +91,3 @@ You can loop the movie with something like:
   ``ffmpeg -start_number 3 -i run0011_%04d.png -filter_complex loop=5:362 -c:v libx264 -pix_fmt yuv420p movie.mp4 -y``
 
 which would add 6 loops.
-

docs/phaseII.rst

@@ -2,7 +2,7 @@
 
 .. include:: globals.rst
 
-.. highlightlang:: rest
+.. highlight:: rest
 
 |hiper| Phase II
 ****************
@@ -363,4 +363,3 @@ of efficiency, then windows plus "clear" might be what you want. This is
 because you can expose for just the exposure delay by clearing the CCD
 immediately before starting the exposure. With clear enabled, the exposure time
 just equals the exposure delay, with no added extra for the readout of the masked data. e.g.
-

docs/telescope.rst

@@ -2,7 +2,7 @@
 
 .. include:: globals.rst
 
-.. highlightlang:: rest
+.. highlight:: rest
 
 At the telescope
 ****************

hipercam/hcam.py

@@ -1614,7 +1614,7 @@ def htimer(tbytes):
            in the FITS file
 
     Returns
-    --------
+    -------
 
       timestamp : (tuple)
           a tuple containing (frameCount, timeStampCount, years, day_of_year,

hipercam/hlog.py

@@ -1191,7 +1191,7 @@ def resample(
         Resamples the Timeseries onto a uniform grid.
 
         Parameters
-        -----------
+        ----------
         bin_edges : `~np.ndarray`
             The left-hand edges of the grid.
 
@@ -1336,7 +1336,7 @@ def bin(self, binsize, bitmask=None, inplace=True):
         can be used to skip points.
 
         Parameters
-        -----------
+        ----------
         binsize : int
             Number of observations to incude in every bin
 

hipercam/scripts/aligntool.py

@@ -221,8 +221,8 @@ def separateBigSmallSpots(xpos, ypos, fwhm, flux):
     Use K-Means clustering to seperate large and small starfield tube spots based on
     FWHM and flux.
 
-    Arguments
-    ---------
+    Parameters
+    ----------
     xpos, ypos : np.ndarray
         spot locations
     fwhm : np.ndarray