Solve Sphinx referencing problem (:ref: ... must be before section an…

…d not redudant)

aaanalysis/data_loader/data_loader.py

@@ -186,7 +186,7 @@ def load_dataset(name: str = "INFO",
 
     See Also
     --------
-    * Overview of all benchmarks in :ref:`1_overview_benchmarks`.
+    * Overview of all benchmarks in :ref:`t1_overview_benchmarks`.
     * Step-by-step guide in the `data loading tutorial <tutorial2_data_loader.html>`_.
 
     Examples

docs/build/html/_sources/index/tables.rst.txt

@@ -5,28 +5,35 @@
    for tables.rst, which is automatically generated based on the information here and
    in the .csv tables from the /tables directory.
 
-   To add a new table:
-   1. Save it as a .csv file in the /tables directory.
-   2. Add an entry for it in the "Overview Table" section below.
-   3. Add a new section describing it, including each column and any important data types (e.g., categories).
+   Instructions for Adding a New Table:
+   1. Store the table as a .csv file in the index/tables directory. Name it using the format tX,
+      where X is incremented based on the last entry's number.
+   2. Update the t0_mapper.xlsx with a corresponding entry for the new table.
+   3. Create a new descriptive section here that elucidates the table's columns and any
+      essential data types, such as categories.
 
    Note: Each table should include a 'Reference' column.
 
-   Ignore the warning: 'tables_template.rst: WARNING: document isn't included in any toctree.'
+   # Key Annotations for Automated Table Generation via create_tables_doc.py:
+   _XXX: A string to be stripped from the references. This prevents redundancies that may result
+         in broken links.
+   ADD-TABLE: Placeholder indicating where tables for the corresponding section should be inserted.
 ..
 
 Tables
-======================
+======
 
 .. contents::
     :local:
     :depth: 1
 
+.. _t0_mapper:
+
 Overview Table
 --------------
 All tables from the AAanalysis documentation are listed here, in chronological order based on the project history.
 
-.. _0_mapper:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -35,14 +42,16 @@ All tables from the AAanalysis documentation are listed here, in chronological o
    * - Table
      - Description
      - See Also
-   * - 1_overview_benchmarks
+   * - t1_overview_benchmarks
      - Protein benchmark datasets
      - aa.load_dataset
-   * - 2_overview_scales
+   * - t2_overview_scales
      - Amino acid scale datasets
      - aa.load_scales
 
 
+.. _t1_overview_benchmarks:
+
 Protein Benchmark Datasets
 --------------------------
 Three types of benchmark datasets are provided:
@@ -55,7 +64,7 @@ Datasets are named beginning with a classification (e.g., 'AA_LDR', 'DOM_GSEC',
 Some datasets have an additional version for positive-unlabeled (PU) learning containing only positive (1)
 and unlabeled (2) data samples, as indicated by appending '_PU' to the dataset name (e.g., 'DOM_GSEC_PU').
 
-.. _1_overview_benchmarks:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -213,11 +222,13 @@ and unlabeled (2) data samples, as indicated by appending '_PU' to the dataset n
      - 1 (substrate), 2 (unknown substrate status)
 
 
+.. _t2_overview_scales:
+
 Amino Acid Scale Datasets
 -------------------------
 Various amino acid scale datasets are provided.
 
-.. _2_overview_scales:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -252,3 +263,4 @@ Various amino acid scale datasets are provided.
      - 60
      - :ref:`Breimann23a <Breimann23a>`
 
+

docs/build/html/generated/aaanalysis.load_dataset.html

@@ -149,7 +149,7 @@ <h1>aaanalysis.load_dataset<a class="headerlink" href="#aaanalysis-load-dataset"
 </ul>
 </dd>
 <dt class="field-even">Returns<span class="colon">:</span></dt>
-<dd class="field-even"><p>Dataframe (df_seq) containing the selected sequence dataset.</p>
+<dd class="field-even"><p>Dataframe (df_seq) with the selected sequence dataset.</p>
 </dd>
 <dt class="field-odd">Return type<span class="colon">:</span></dt>
 <dd class="field-odd"><p>DataFrame</p>
@@ -158,7 +158,7 @@ <h1>aaanalysis.load_dataset<a class="headerlink" href="#aaanalysis-load-dataset"
 <div class="admonition seealso">
 <p class="admonition-title">See also</p>
 <ul class="simple">
-<li><p>Overview of all benchmarks in <span class="xref std std-ref">1_overview_benchmarks</span>.</p></li>
+<li><p>Overview of all benchmarks in <a class="reference internal" href="../index/tables.html#t1-overview-benchmarks"><span class="std std-ref">Protein Benchmark Datasets</span></a>.</p></li>
 <li><p>Step-by-step guide in the <a class="reference external" href="tutorial2_data_loader.html">data loading tutorial</a>.</p></li>
 </ul>
 </div>

docs/build/html/index/tables.html

@@ -126,9 +126,10 @@ <h1>Tables<a class="headerlink" href="#tables" title="Permalink to this heading"
 </ul>
 </nav>
 <section id="overview-table">
-<h2><a class="toc-backref" href="#id1" role="doc-backlink">Overview Table</a><a class="headerlink" href="#overview-table" title="Permalink to this heading"></a></h2>
+<span id="t0-mapper"></span><h2><a class="toc-backref" href="#id1" role="doc-backlink">Overview Table</a><a class="headerlink" href="#overview-table" title="Permalink to this heading"></a></h2>
 <p>All tables from the AAanalysis documentation are listed here, in chronological order based on the project history.</p>
-<table class="docutils align-default" id="mapper">
+<p>ADD-TABLE</p>
+<table class="docutils align-default">
 <colgroup>
 <col style="width: 33%" />
 <col style="width: 33%" />
@@ -141,19 +142,19 @@ <h2><a class="toc-backref" href="#id1" role="doc-backlink">Overview Table</a><a
 </tr>
 </thead>
 <tbody>
-<tr class="row-even"><td><p>1_overview_benchmarks</p></td>
+<tr class="row-even"><td><p>t1_overview_benchmarks</p></td>
 <td><p>Protein benchmark datasets</p></td>
 <td><p>aa.load_dataset</p></td>
 </tr>
-<tr class="row-odd"><td><p>2_overview_scales</p></td>
+<tr class="row-odd"><td><p>t2_overview_scales</p></td>
 <td><p>Amino acid scale datasets</p></td>
 <td><p>aa.load_scales</p></td>
 </tr>
 </tbody>
 </table>
 </section>
 <section id="protein-benchmark-datasets">
-<h2><a class="toc-backref" href="#id2" role="doc-backlink">Protein Benchmark Datasets</a><a class="headerlink" href="#protein-benchmark-datasets" title="Permalink to this heading"></a></h2>
+<span id="t1-overview-benchmarks"></span><h2><a class="toc-backref" href="#id2" role="doc-backlink">Protein Benchmark Datasets</a><a class="headerlink" href="#protein-benchmark-datasets" title="Permalink to this heading"></a></h2>
 <p>Three types of benchmark datasets are provided:</p>
 <ul class="simple">
 <li><p>Residue prediction (AA): Datasets used to predict specific properties of amino acid residues.</p></li>
@@ -163,7 +164,8 @@ <h2><a class="toc-backref" href="#id2" role="doc-backlink">Protein Benchmark Dat
 <p>Datasets are named beginning with a classification (e.g., ‘AA_LDR’, ‘DOM_GSEC’, ‘SEQ_AMYLO’).
 Some datasets have an additional version for positive-unlabeled (PU) learning containing only positive (1)
 and unlabeled (2) data samples, as indicated by appending ‘_PU’ to the dataset name (e.g., ‘DOM_GSEC_PU’).</p>
-<table class="docutils align-default" id="overview-benchmarks">
+<p>ADD-TABLE</p>
+<table class="docutils align-default">
 <colgroup>
 <col style="width: 10%" />
 <col style="width: 10%" />
@@ -348,9 +350,10 @@ <h2><a class="toc-backref" href="#id2" role="doc-backlink">Protein Benchmark Dat
 </table>
 </section>
 <section id="amino-acid-scale-datasets">
-<h2><a class="toc-backref" href="#id3" role="doc-backlink">Amino Acid Scale Datasets</a><a class="headerlink" href="#amino-acid-scale-datasets" title="Permalink to this heading"></a></h2>
+<span id="t2-overview-scales"></span><h2><a class="toc-backref" href="#id3" role="doc-backlink">Amino Acid Scale Datasets</a><a class="headerlink" href="#amino-acid-scale-datasets" title="Permalink to this heading"></a></h2>
 <p>Various amino acid scale datasets are provided.</p>
-<table class="docutils align-default" id="overview-scales">
+<p>ADD-TABLE</p>
+<table class="docutils align-default">
 <colgroup>
 <col style="width: 25%" />
 <col style="width: 25%" />

docs/source/conf.py

@@ -9,8 +9,12 @@
 
 sys.path.append(os.path.abspath('.'))
 
+
+# Create notebooks rst and table rst first
 from create_tables_doc import generate_table_rst
 from create_notebooks_docs import export_notebooks_to_rst
+export_notebooks_to_rst()
+generate_table_rst()
 
 # -- Path and Platform setup --------------------------------------------------
 path_source = os.path.join(os.path.dirname(__file__))
@@ -58,6 +62,7 @@
     # 'pydata_sphinx_theme',  # Theme with a focus on long-form content and optimized for _data-focused libraries
 ]
 
+
 # -- Autodoc & Numpydoc settings ----------------------------------------------
 # Autodoc settings
 # See https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#configuration
@@ -110,9 +115,9 @@
 napoleon_custom_sections = None
 
 
-# -- Juypter tutorials integration -------------------------------------------
+# -- Juypter tutorials integration --------------------------------------------
 nbsphinx_execute = 'never'
-export_notebooks_to_rst()
+
 
 # -- Intersphinx mapping -----------------------------------------------------
 intersphinx_mapping = {
@@ -172,8 +177,7 @@
      "Advanced analysis tools for researchers.", "Miscellaneous"),
 ]
 
-# Create table.rst
-generate_table_rst()
+
 
 # -- Linkcode configuration ---------------------------------------------------
 _module_path = os.path.dirname(importlib.util.find_spec("aaanalysis").origin)  # type: ignore

docs/source/create_notebooks_docs.py

@@ -37,11 +37,9 @@ def export_notebooks_to_rst():
     for filename in os.listdir(FOLDER_NOTEBOOKS):
         if filename.endswith('.ipynb') and filename not in LIST_EXCLUDE:
             full_path = os.path.join(FOLDER_NOTEBOOKS, filename)
-
             # Load the notebook
             with open(full_path, 'r') as f:
                 notebook = nbformat.read(f, as_version=4)
-
             # Export to RST
             rst_exporter = nbconvert.RSTExporter()
             rst_data, _ = rst_exporter.from_notebook_node(notebook)

docs/source/create_tables_doc.py

@@ -12,14 +12,16 @@
 FILE_REF = FOLDER_IND + "references.rst"
 FILE_TABLE_TEMPLATE = FOLDER_IND + "tables_template.rst"
 FILE_TABLE_SAVED = FOLDER_IND + "tables.rst"
-FILE_MAPPER = FOLDER_TABLES + "0_mapper.xlsx"
+FILE_MAPPER = FOLDER_TABLES + "t0_mapper.xlsx"
 LIST_TABLES = list(sorted([x for x in os.listdir(FOLDER_TABLES) if x != "0_mapper.xlsx"]))
 
 COL_MAP_TABLE = "Table"
 COL_DESCRIPTION = "Description"
 COL_REF = "Reference"
 
 COLUMN_WIDTH = 8
+STR_REMOVE = "_XXX" # Check with tables_template.rst for consistency
+STR_ADD_TABLE = "ADD-TABLE"
 
 
 # Helper Functions
@@ -89,7 +91,7 @@ def generate_table_rst():
     overview_table_rst = _convert_excel_to_rst(df_mapper)
 
     # Generate the tables and store them in a dictionary
-    tables_dict = {"0_mapper": overview_table_rst}
+    tables_dict = {"t0_mapper": overview_table_rst}
     for index, row in df_mapper.iterrows():
         table_name = row[COL_MAP_TABLE]
         df = pd.read_excel(FOLDER_TABLES + _f_xlsx(on=True, file=table_name))
@@ -101,16 +103,19 @@ def generate_table_rst():
 
     # Initialize variables
     rst_content = ""
-
+    table_name = ""
     # Loop through the lines of the template
     for line in template_lines:
-        rst_content += line
         # Check for hooks like ".. _1_overview_benchmarks:"
         match = re.search(r'\.\. _(\w+):', line)
-        if match:
-            table_marker = match.group(1)
-            if table_marker in tables_dict:
-                rst_content += "\n" + tables_dict[table_marker] + "\n"
+        if not match:
+            rst_content += line
+        else:
+            line_with_new_marker = line.replace(STR_REMOVE, "")
+            rst_content += line_with_new_marker
+            table_name = match.group(1).replace(STR_REMOVE, "")
+        if STR_ADD_TABLE in line and table_name in tables_dict:
+            rst_content += "\n" + tables_dict[table_name] + "\n"
 
     # Write the new content to the output .rst file
     with open(FILE_TABLE_SAVED, 'w') as f:

docs/source/index/tables.rst

@@ -5,28 +5,35 @@
    for tables.rst, which is automatically generated based on the information here and
    in the .csv tables from the /tables directory.
 
-   To add a new table:
-   1. Save it as a .csv file in the /tables directory.
-   2. Add an entry for it in the "Overview Table" section below.
-   3. Add a new section describing it, including each column and any important data types (e.g., categories).
+   Instructions for Adding a New Table:
+   1. Store the table as a .csv file in the index/tables directory. Name it using the format tX,
+      where X is incremented based on the last entry's number.
+   2. Update the t0_mapper.xlsx with a corresponding entry for the new table.
+   3. Create a new descriptive section here that elucidates the table's columns and any
+      essential data types, such as categories.
 
    Note: Each table should include a 'Reference' column.
 
-   Ignore the warning: 'tables_template.rst: WARNING: document isn't included in any toctree.'
+   # Key Annotations for Automated Table Generation via create_tables_doc.py:
+   _XXX: A string to be stripped from the references. This prevents redundancies that may result
+         in broken links.
+   ADD-TABLE: Placeholder indicating where tables for the corresponding section should be inserted.
 ..
 
 Tables
-======================
+======
 
 .. contents::
     :local:
     :depth: 1
 
+.. _t0_mapper:
+
 Overview Table
 --------------
 All tables from the AAanalysis documentation are listed here, in chronological order based on the project history.
 
-.. _0_mapper:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -35,14 +42,16 @@ All tables from the AAanalysis documentation are listed here, in chronological o
    * - Table
      - Description
      - See Also
-   * - 1_overview_benchmarks
+   * - t1_overview_benchmarks
      - Protein benchmark datasets
      - aa.load_dataset
-   * - 2_overview_scales
+   * - t2_overview_scales
      - Amino acid scale datasets
      - aa.load_scales
 
 
+.. _t1_overview_benchmarks:
+
 Protein Benchmark Datasets
 --------------------------
 Three types of benchmark datasets are provided:
@@ -55,7 +64,7 @@ Datasets are named beginning with a classification (e.g., 'AA_LDR', 'DOM_GSEC',
 Some datasets have an additional version for positive-unlabeled (PU) learning containing only positive (1)
 and unlabeled (2) data samples, as indicated by appending '_PU' to the dataset name (e.g., 'DOM_GSEC_PU').
 
-.. _1_overview_benchmarks:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -213,11 +222,13 @@ and unlabeled (2) data samples, as indicated by appending '_PU' to the dataset n
      - 1 (substrate), 2 (unknown substrate status)
 
 
+.. _t2_overview_scales:
+
 Amino Acid Scale Datasets
 -------------------------
 Various amino acid scale datasets are provided.
 
-.. _2_overview_scales:
+ADD-TABLE
 
 .. list-table::
    :header-rows: 1
@@ -252,3 +263,4 @@ Various amino acid scale datasets are provided.
      - 60
      - :ref:`Breimann23a <Breimann23a>`
 
+