Collapse all :glob:-discovered files into one TOC entry - python-sphinx

I've got something like this:
.. toctree::
:maxdepth: 2
:caption: Contents:
:hidden:
:glob:
docs
frontend
backend
tools/*
I'd like all the documents found in the tools directory to collapse into one expandable sidebar TOC entry. Should be possible, but I can't find a clue.
I use sphinx_rtd_theme.
To put it another way: suppose I have a very long document like this (tools.rst):
Section 1
*********
Subsection 1
============
Subsection 2
============
Subsection 3
============
How am I supposed to split it by the subsection and preserve the document structure, without resorting to includes, which don't sit well with Sphinx anyway.

source/index.rst:
.. toctree::
:maxdepth: 2
:caption: Contents:
:glob:
tools/_tools
docs
frontend
backend
tmp/*
Old source/tools/_tools.rst:
Notes on tools
**************
.. include:: black.rst
.. include:: docker.rst
.. include:: git_hooks.rst
.. include:: github_webhooks.rst
.. include:: mypy.rst
.. include:: pipm.rst
.. include:: poetry.rst
.. include:: sphinx.rst
.. include:: uvicorn.rst
New source/tools/_tools.rst:
Notes on tools
**************
.. toctree::
:maxdepth: 2
:glob:
*
The new one renders exactly like the old one, and is more versatile an a lot cleaner.

Related

How to use raw .html files in sphinx-build

Let's say I have this structure in my source folder for my sphinx documentation :
journals /
*.md files
reports /
*.html files
conf.py
index.rst
and the following code in index.rst :
.. toctree::
:maxdepth: 4
:glob:
:caption: Reports
reports/*
.. toctree::
:maxdepth: 4
:glob:
:caption: Journals
journals/*
When I generate my docs with sphinx-build, sphinx automatically builds the resulting html pages with the .md files found in my journals folder, but it ignores the .html files found in the reports folder.
What I would like is for sphinx to automatically include the .html files found in the reports folder. The resulting .html pages should still include the sphinx layout (sidebar, headers, etc.), and should not just be a pure copy of the raw .html files.
I know I could do .rst files with the following :
.. raw:: html
:file: reports/file.html
This basically does what I want. The .rst file is found by sphinx and builds a .html file that includes both sphinx layout and my source .html file.
I just don't know how to do this automatically though, as I have a lot of files in my reports file. I would prefer to avoid having to create one .rst file for each of my .html file.
Is there a way to automatize this process with sphinx-build?

How to include pages generated at build time?

I seek to make an "API Reference" page exposing genindex.html and py-modindex.html; problem is, I rather the entire page not be just clickable links to said pages, but to actually include their contents. We can do this with a README - however, README.rst exists in source/, whereas genindex.html is only in build/, so below don't work:
.. include:: modindex
.. include:: genindex
.. include:: modindex.rst
.. include:: genindex.rst
.. include:: ../build/genindex.html
Can this be accomplished?

Is there a way for having multiple wildcards?

I try to use GNU make to organize my research data, processing and visualization as recommended by the Data Science CookieCutter project. My raw data is structured like this:
.
├── data
│   ├── interim
│   │   └── cleaned
│   └── raw
│   ├── ex01
│   └── ex02
Where I keep the data of experiment 1 and 2 seperated but combine them after cleaning them. Eg data/raw/ex01/p0-c0.csv becomes data/interim/cleaned/ex01-p0-c0.hdf.
In make I use two rules like this:
data/interim/cleaned/ex01-%.hdf: data/raw/ex01/source0/%.csv
data/raw/ex01/source1/%.csv
$(PYTHON) src/data/make_dataset.py $^ $#
data_interim_cleaned_ex01: $(addprefix $(CLEANED_DIR)/ex01-, $(addsuffix .hdf, $(basename $(basename $(notdir $(wildcard data/raw/ex01/source0/*.csv))))))
This strikes me as oddly verbose (especially because I copied the block for experiment 2) and I my intuition tells me that it would be easier if there were multiple (named) wildcards. I guess regexps would help, but are not (easily) available in make.
Is there a canonical way to solve this?
The following solution isn't really a canonical make file but IMHO much of the canonical functionality of make is too hard to grasp and remember anyway. Questions like "how can I transform my set of filenames from shape X to Y" come up all the time because users do employ directory and filename structure as means to organize their projects (a very natural and logical way) and make is really badly equipped to handle such tasks programmatically.
One way is to use the usual range of command line tools like sed, the other are helper libraries like gmtt to take apart strings:
include gmtt-master/gmtt.mk
COMMON_ROOT = data/raw
COMMON_DEST = data/interim/cleaned
SOURCE = data/raw/ex01/p0-c0.csv data/raw/ex01/p1-c1.csv data/raw/ex02/p0-c0.csv data/raw/ex02/p1-c1.csv
# a pattern which separates a string into 5 parts (see below)
SEP_PATTERN = $(COMMON_ROOT)/ex*/*.csv
# use the elements (quoted variable-references '$$'!) in the new filename
OUTPUT_PATTERN = $(COMMON_DEST)/ex$$2-$$4.hdf
# glob-match tests a glob pattern on a string and returns the string cut up at the border of
# the glob elements (*,?,[] and verbatim strings). We immediately turn this into a gmtt table
# by prepending the number of columns (5) to it:
SEPARATED = 5 $(foreach fname,$(SOURCE),$(call glob-match,$(fname),$(SEP_PATTERN)))
$(info $(SEPARATED))
$(info -----------------)
$(info $(call map-tbl,$(SEPARATED),$(OUTPUT_PATTERN)$$(newline)))
Output:
$ make
5 data/raw/ex 01 / p0-c0 .csv data/raw/ex 01 / p1-c1 .csv data/raw/ex 02 / p0-c0 .csv data/raw/ex 02 / p1-c1 .csv
-----------------
data/interim/cleaned/ex01-p0-c0.hdf
data/interim/cleaned/ex01-p1-c1.hdf
data/interim/cleaned/ex02-p0-c0.hdf
data/interim/cleaned/ex02-p1-c1.hdf
make: *** Keine Ziele. Schluss.
I fear that turning the makefile into one which generates rules dynamically is inevitable, tho.
The answer is perhaps not one you are going to like, but it is to not introduce variability or repetition in you file names. There are easy or at least reasonable ways to articulate relationships in a Makefile between stem names where you add or remove a prefix (such as a directory name) or a suffix. Anything else creates complications where you end up with tortured and complex transformation rules or external helper scripts to manage the mappings, or, in the worst care, a situation where you simply have to abandon make for dependency management.
One workaround which sort of allows you to keep your cake and eat it too is to set up symlinks between your preferred, human-friendly naming conventions and the structures managed by make; but this is a crutch at best.
Another technique which may be useful to you is to touch a simple flag file to mark a complex set of dependencies as handled. Especially if there are dependencies which do not map directly to a set of input file names for another target, putting all of those behind a simple
.input-files-done: some complex depencies
touch $#
and then just depending on .input-files-done for the targets which share these dependencies can simplify your Makefile and your workflow.
But in summary, my main recommendation would be to keep file names uniform, so that you can always declare an explicit dependency from one file name to another with a simple rule.

Include directive in master doc

In my conf.py I have defined the master as index ( index.rst)
# The master toctree document.
master_doc = 'index'
When I import a list of rst files using the .. include:: list.rst where list .rst file is being created dynamically.
Project Test
================================
.. include:: list.rst
it works fine; but when I add some other rst names in the same index.rst it doesn't work, like static.rst and about.rst in the following declaration :
Project Test
================================
.. include:: list.rst
static
about
I created a new file staticList.rst and moved all the static rst names.
.. toctree::
:maxdepth: 1
static
about
and included the staticList.rst in the index.rst file, it is giving the desired result but slightly broken listing.
Project Test
================================
.. include:: list.rst
.. include:: staticList.rst

Get value from external file

I have a file called version.h that among other things contains the following line:
#define VERSION 0.4.2-b
Now, on the Sphinx documentation for this piece of software, I'd like to display this value in one of the .rst files.
How can I do this?
Thanks!
See the literalinclude directive.
In your case:
.. literalinclude:: version.h
:language: c
For the proper syntax highlighting, you can change the Pygments lexer by specifying the language attribute.

Resources