New docs

.github/workflows/publish-docs.yml

@@ -22,26 +22,30 @@ jobs:
     - name: Set up Python
       uses: actions/setup-python@v4
 
-    - name: Install Python dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install sphinx numpydoc sphinx-autobuild sphinx-rtd-theme
-
-    - name: Install this Python package
-      run: |
-        python -m pip install .[clipboard,download,express,gel]
-
     - name: Session information
       run: |
         python --version
 
+    - name: Install Poetry
+      uses: snok/install-poetry@v1
+      with:
+        virtualenvs.prefer-active-python: true
+
+    - name: 🔩 list Poetry settings
+      run: poetry config --list
+
+    - name: 🔩 Install with Poetry
+      run: poetry install --all-extras --with docs
+
     - name: Build documentation
       run: |
         cd docs
-        make html
+        poetry run bash autogen_docs.sh
+        poetry run make html
 
     - name: Deploy to GitHub pages 🚀
-      if: github.event_name != 'pull_request'
+      # Deploy only when master changes
+      if: github.ref == 'refs/heads/master' && github.event_name == 'push'
       uses: JamesIves/[email protected]
       with:
         clean: false

.gitignore

@@ -66,6 +66,8 @@ coverage.xml
 
 # Sphinx documentation
 docs/_build/
+docs/markdown_notebooks
+docs/modules/
 
 # PyBuilder
 target/

README.md

@@ -4,16 +4,13 @@
 | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
 | [![Documentation Status](https://github.com/BjornFJohansson/pydna/actions/workflows/publish-docs.yml/badge.svg)](https://github.com/BjornFJohansson/pydna/actions/workflows/publish-docs.yml)                                                      | [![GitHub issues](https://img.shields.io/github/issues/BjornFJohansson/pydna.svg)](https://github.com/BjornFJohansson/pydna/issues)                | [![Anaconda-Server Badge2](https://anaconda.org/bjornfjohansson/pydna/badges/license.svg)](https://anaconda.org/bjornfjohansson/pydna) | [![GitHub stars](https://img.shields.io/github/stars/BjornFJohansson/pydna.svg)](https://github.com/BjornFJohansson/pydna/stargazers) |
 
+<!-- docs/index.rst-start -->
 
-Planning genetic constructs with many parts and assembly steps, such as recombinant
-metabolic pathways :petri_dish:, are often difficult to **properly** document as is evident from the poor
-state of documentation in the scientific literature :radioactive:.
-
-
-The pydna python package provide a human-readable formal descriptions of :dna: cloning and genetic assembly
-strategies in Python :snake: which allow for simulation and verification.
+Pydna is a python package that provides a human-readable formal descriptions of 🧬 cloning and genetic assembly strategies in Python 🐍 for simulation and verification.
 Pydna can be used as [executable documentation](https://en.wikipedia.org/wiki/Literate_programming) for cloning.
 
+Designing genetic constructs with many components and steps, like recombinant metabolic pathways 🧫, often makes accurate documentation difficult, as seen in the poor state of scientific literature ☢️
+
 A cloning strategy expressed in pydna is **complete**, **unambiguous** and **stable**.
 
 Pydna provides simulation of:
@@ -28,14 +25,15 @@ Pydna provides simulation of:
 - Golden gate assembly (in progress)
 
 
-Virtually any sub-cloning experiment can be described in pydna, and its execution yield
-the sequences of intermediate and final DNA molecules.
+Virtually any sub-cloning experiment can be described in pydna, and its execution yield the sequences of intermediate and final DNA molecules.
 
 Pydna has been designed with the goal of being understandable for biologists with only some basic understanding of Python.
 
 Pydna can formalize planning and sharing of cloning strategies and is especially useful for complex or combinatorial
 DNA molecule constructions.
 
+<!-- docs/index.rst-end -->
+
 ## Acknowledgement 🤝
 
 If you use pydna in your research, please reference the paper:
@@ -289,12 +287,20 @@ rec_vec =  ( lin_vector + pcr_prod ).looped()
 
 ## Installation 📦
 
+<!-- docs/installation.rst-start -->
+
 By default, pydna is installed with minimal dependencies, but there are optional dependencies for additional functionality.
 
+
+### Optional dependencies
 <details>
 <summary>Click here to see optional dependencies</summary>
 
-### `clipboard`
+<!-- I did not make these headers because they mess up the sphinx docs by being inside of a collapsible section-->
+
+_________________
+
+**clipboard**
 
 Enables the `pydna.dseqrecord.Dseqrecord.copy_gb_to_clipboard()` and `pydna.dseqrecord.Dseqrecord.copy_fasta_to_clipboard()`
 
@@ -305,7 +311,9 @@ These methods will put a copy the sequence on the clipboard in either Genbank (g
 | --------------------------------------------------- | -------------------------- |
 | [pyperclip](https://pypi.python.org/pypi/pyperclip) | copy sequence to clipboard |
 
-### `download`
+_________________
+
+**download**
 
 Pyparsing enables the `pydna.genbankfixer.gbtext_clean()` function that can automatically
 correct malformed sequence files in Genbank format. These are often found online, so this
@@ -317,15 +325,19 @@ option also installs requests to enable the  `pydna.genbankfixer.download.downlo
 | [pyparsing](https://pypi.python.org/pypi/pyparsing) | fix corrupt Genbank files with pydna.genbankfixer |
 | [requests](https://pypi.org/project/requests)       | download sequences with pydna.download            |
 
-### `express`
+_________________
+
+**express**
 
 This option enables the `pydna.utils.cai()` function and the `cai()` method
 available from subclasses of `pydna.seqrecord.SeqRecord`, such as
 `pydna.dseqrecord.Dseqrecord`.
 
 | [cai2](https://pypi.python.org/pypi/cai2)           | codon adaptation index calculations in several modules |
 
-### `gel`
+_________________
+
+**gel**
 
 Scipy, matplotlib and pillow (PIL) enable the generation of gel images. Numpy is also
 needed, but usually installed as a dependency of biopython.
@@ -337,6 +349,8 @@ needed, but usually installed as a dependency of biopython.
 | [matplotlib](http://matplotlib.org)               | “                             |
 | [pillow](https://github.com/python-pillow/Pillow) | “                             |
 
+_________________
+
 </details>
 
 ### Installing with pip 🐍
@@ -367,6 +381,8 @@ poetry remove pydna
 poetry add pydna --extras "express gel"
 ```
 
+<!-- docs/installation.rst-end -->
+
 ## Contributing and feedback 🛠️
 
 Feedback & suggestions are very welcome! Please create an issue with your question, comment or suggestion. Please include the version of pydna you are using and code to reproduce the issue if possible.
@@ -485,6 +501,8 @@ sphinx-autobuild --watch src/ docs docs/_build/html
 
 ```
 
+More info about how to contribute to the documentation can be found [here](docs/README.md)
+
 ## Release process 🚀
 
 See the [releases](https://github.com/BjornFJohansson/pydna/releases) for changes and releases.

docs/README.md

@@ -0,0 +1,91 @@
+# Documentation
+
+Documentation is built using [Sphinx](http://www.sphinx-doc.org/) from [docstrings](https://www.python.org/dev/peps/pep-0257/)
+using a GitHub [action](https://github.com/BjornFJohansson/pydna/actions/workflows/publish-docs.yml).
+The [numpy](www.numpy.org) [docstring format](https://numpy.org/doc/stable/dev/howto-docs.html#docstring-intro) is used.
+
+Below the commands to run a local sphinx server that auto-updated when files are changed.
+
+```bash
+# Install docs dependency group
+poetry install --with docs
+
+# Start the sphinx server to see docs live by default at http://127.0.0.1:8000/
+sphinx-autobuild --watch src/ docs docs/_build/html
+```
+
+When you run the sphinx server live, it not always updates things if you make relevant changes (e.g. adding new pages,
+changing the css, etc.). To force an update, you can run the command below which deletes the existing build folder and
+then rebuilds it.
+
+```bash
+rm -rf docs/_build/html && sphinx-autobuild --watch src/ docs docs/_build/html
+```
+
+## Adding new sections to the documentation
+
+You can add new sections (equivalent to "Getting started" or "Example gallery") by creating a new `.rst` or `.md` file in the `docs` folder, and then adding a reference to it in the `.. toctree::` directive in the `docs/index.rst` file.
+
+## Auto-generated files
+
+The script `autogen_docs.sh` is run in the github action before creating the docs. If you want to reproduce locally, you
+should run it from the `docs` folder
+
+```bash
+cd docs
+bash autogen_docs.sh
+```
+
+
+* It converts all notebooks in the `docs/notebooks` folder to `.md` in `docs/markdown_notebooks` (excluded from git)
+* It creates all files in `docs/modules`, which are used to generate the API reference. For instance, it will create
+  a `docs/modules/index.rst` file that starts like this:
+
+  ```rst
+  Modules
+  =======
+
+  .. toctree::
+     :maxdepth: 1
+     :caption: Modules:
+
+     pydna
+     pydna__pretty
+  ```
+  And then individual files for each module and submodule, e.g. `docs/modules/pydna.rst`.
+
+  ```
+  pydna
+  =====
+
+  .. automodule:: pydna
+      :members:
+      :undoc-members:
+      :show-inheritance:
+  ```
+
+## Text imported from README.md
+
+To avoid having to maintain the same text in multiple files, fragments of the `README.md` are imported using the directive
+`include`. For instance, in the `installation.rst` file, you can find the code below. What this does is to import the text of the README.md file between the start and end markers, which are markdown comments and therefore not rendered.
+
+```rst
+.. include:: ../README.md
+   :parser: myst_parser.sphinx_
+   :start-after: <!-- docs/installation.rst-start -->
+   :end-before: <!-- docs/installation.rst-end -->
+```
+
+## Including notebooks in the getting started and example sections
+
+You can see the example of how to do this in the `getting_started.md` file. Note that the notebooks present in the `docs/notebooks` folder will automatically be converted to markdown in the `docs/markdown_notebooks` folder. So if you have a notebook `docs/notebooks/Example_Gibson.ipynb`, it will be converted to `docs/markdown_notebooks/Example_Gibson.md` and you can use that file path to make a link to it.
+
+## Custom CSS
+
+For now, I have used css to make notebook outputs that are too long scrollable, and to add a small label `python code` to the code cells and `output` to the output cells.
+
+For further customization, you can edit the `custom.css` file.
+
+## Misc
+
+Other changes, such as changing the favicon, the css etc., can be made in the `conf.py` file. See the [sphinx docs](https://www.sphinx-doc.org/en/master/usage/configuration.html) and the [sphinx-rtd-theme](https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html) docs for more information.

docs/_static/custom.css

@@ -0,0 +1,48 @@
+/* Make output cells that are too long scrollable */
+.highlight-none {
+    position: relative;
+    margin-top: 20px;
+}
+
+/* Add a label to the top of the output cell */
+.highlight-none::before {
+    content: "output";
+    position: absolute;
+    top: 0px;
+    left: 0;
+    background-color: #f0f0f0;
+    color: #333;
+    padding: 2px 8px;
+    font-size: 12px;
+    border-radius: 3px 3px 0 0;
+    z-index: 1;
+}
+
+div.highlight-none pre {
+    max-height: 300px;
+    overflow-y: auto;
+}
+
+/* Add same label to highlight-python cells */
+.highlight-python {
+    position: relative;
+}
+
+/* Add a label to the top of the output cell */
+.highlight-python::before {
+    content: "python code";
+    position: absolute;
+    top: 0px;
+    left: 0;
+    background-color: #f0f0f0;
+    color: #333;
+    padding: 2px 8px;
+    font-size: 12px;
+    border-radius: 3px 3px 0 0;
+    z-index: 1;
+}
+
+/* Add padding to the top to fit the label */
+.rst-content div[class^="highlight"] pre {
+    padding-top: 25px;
+}

docs/autogen_docs.sh

@@ -0,0 +1,52 @@
+#!/bin/bash
+
+# Exit on error
+set -e
+
+# Loop through all notebooks in the markdown_notebooks directory
+for notebook in notebooks/*.ipynb; do
+    # Convert each notebook to markdown
+    python -m nbconvert $notebook --to markdown --output-dir markdown_notebooks
+done
+
+# Get module names
+all_modules=$(python -c "import pydna; import pkgutil; [print(name) for _, name, _ in pkgutil.iter_modules(pydna.__path__)]")
+
+# If the modules folder does not exist, create it
+mkdir -p modules
+
+# Create the root pydna.rst file
+echo "Creating pydna.rst"
+echo "pydna" > modules/pydna.rst
+echo "=====" >> modules/pydna.rst
+echo "" >> modules/pydna.rst
+echo ".. automodule:: pydna" >> modules/pydna.rst
+echo "    :members:" >> modules/pydna.rst
+echo "    :undoc-members:" >> modules/pydna.rst
+echo "    :show-inheritance:" >> modules/pydna.rst
+
+# Create the modules/index.rst file
+echo "Creating modules/index.rst"
+echo "Modules" > modules/index.rst
+echo "=======" >> modules/index.rst
+echo "" >> modules/index.rst
+echo ".. toctree::" >> modules/index.rst
+echo "   :maxdepth: 1" >> modules/index.rst
+echo "   :caption: Modules:" >> modules/index.rst
+echo "" >> modules/index.rst
+echo "   pydna" >> modules/index.rst
+
+# For each module create a rst file in ./modules
+for module in $all_modules; do
+    echo "Creating pydna_$module.rst"
+    echo "pydna.$module" > modules/pydna_$module.rst
+    echo "==========" >> modules/pydna_$module.rst
+    echo "" >> modules/pydna_$module.rst
+    echo ".. automodule:: pydna.$module" >> modules/pydna_$module.rst
+    echo "    :members:" >> modules/pydna_$module.rst
+    echo "    :undoc-members:" >> modules/pydna_$module.rst
+    echo "    :show-inheritance:" >> modules/pydna_$module.rst
+
+    # Add the module to the modules/index.rst file
+    echo "   pydna_$module" >> modules/index.rst
+done