How to modify documentation built with sphinx via a script in readthedocs build - python-sphinx

Question

I am trying to run a script to modify the documentation built by sphinx hosted by Read the Docs (because some links are not properly handled). The script works when I try to build it locally, but either fails on the Read the Docs build or the changes do not get propagated to the web site.
The script I'm trying to run is super simple, it replaces some html links that are not properly converted by sphinx-markdown-tables:
#!/bin/bash
# fix_table_links.sh
FILE="_build/html/api_reference.html"
if [[ "$1" != "" ]]; then
FILE="$1"
fi
sed -E 's/a href="(.*)\.md"/a href="\1\.html"/g' -i ${FILE}
My readthedocs.yml looks like this:
# Required
version: 2
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py
# Optionally build your docs in additional formats such as PDF and ePub
formats: all
# Optionally set the version of Python and requirements required to build your docs
python:
install:
- requirements: docs/requirements.readthedocs.txt
build:
os: ubuntu-20.04
tools:
python: "3.8"
jobs:
post_build:
- echo "Running post-build commands."
- bash docs/fix_table_links.sh _readthedocs/html/api_reference.html
There are two cases:
Case 1) Using the readthedocs.yml as above, the build fails because _readthedocs/html/api_reference.html does not exist, despite this directory being the place the documentation claims will get uploaded from here. An example failure of this run is here.
Case 2) If I change the final of readthedocs.yml to bash docs/fix_table_links.sh docs/_build/html/api_reference.html, then the build passes (example here). But the links are not updated on the Read the Docs site: they still point to markdown pages rather than their corresponding HTML pages, so it must not be the version that gets uploaded to the Read the Docs web site.
Wading through documentation, I can't figure out how do this. Has anybody done this before or have a better grasp on how Read the Docs builds work? Thanks!

Answer 1

If you're willing to rewrite the script as a Python function, then you can do this super easily by connecting it as an event handler for the build-finished event.
I've done something similar in one of my own repos, except I post-process a .rst file. It's not actually used on RTD, but I can see it works in the build logs. So it should work to post-process your HTML files as well, since the build-finished event would occur after they've been generated.
First, define the script as a function in your conf.py. It needs to have app and exception as parameters.
def replace_html_links(app, exception):
with open(FILE, 'r') as f:
html = f.read()
# stuff to edit and save the html
Then either define or add to your setup function:
def setup(app):
app.connect('build-finished', replace_html_links)
And that's it!