How to pre-process source files while a Sphinx run?
Question: I have set up a Sphinx documentation for my project and would like to extract
doc strings for the source files and embed them into the final documentation.
Unfortunately, the source file's language (VHDL) is not supported by Sphinx.
There seems to be no Sphinx domain for VHDL.

So my ideas is as follows:

  * Hook into the Sphinx run and execute some Python code before Sphinx
  * The Python codes extracts text blocks from each source file (the top-most multi-line comment block) and assembles one reST file per source file, consisting of this comment block and some other reST markup.
  * All source files are listed in an `index.rst`, to generate the apropriate `.. toctree::` directive.
  * The text extraction and transformation is done recursively per source code directory.

So the main question is: **How to hook into Spinx?**

Or should I just import and run my own configuration in `conf.py`?

    
    
    #!/usr/bin/env python3
    # -*- coding: utf-8 -*-
    #
    from my_preprocessor import my_proc
    proc = my_proc()
    proc.run()
    #
    # Test documentation build configuration file, created by
    # sphinx-quickstart on Tue May 24 11:28:20 2016.
    # ....
    

I can't modify the build process files: `Makefile` and `make.bat`, because the
real build process runs on ReadTheDocs.org. RTDs only executes `conf.py`.


Answer: As noted in my previous comments and mertyildiran's answer, the official way
to hook into Sphinx for a language would be to [create an
extension](http://www.sphinx-doc.org/en/stable/extdev/index.html#dev-
extensions) to implement a new domain for VHDL.

This has already been done for many other languages - e.g. Erlang, PHP,
CoffeeScript - and APIs - e.g. HTTP REST - just to name a few from [sphinx-
contrib](https://bitbucket.org/birkenfeld/sphinx-contrib). However, that is
going to take a lot of time, which you don't have... You are therefore left
with an option to do some quick parsing yourself and then hooking that into
your Sphinx build somehow.

Since you are bypassing the official hooks, this question becomes "how do I
run my own code inside a Sphinx build?" For which, I would recommend that you
simply follow the guidance for a local extension - i.e. put it in a separate
directory, add that to your path, then import and invoke it. As noted in the
[docs](http://www.sphinx-doc.org/en/stable/config.html):

> The configuration file is executed as Python code at build time (using
> execfile(), and with the current directory set to its containing directory),
> and therefore can execute arbitrarily complex code. Sphinx then reads simple
> names from the file’s namespace as its configuration.

As a final though, this opens up the options to use 3rd party packages like
pyVhdl2Sch (with a nod again to mertyildiran's answer) to create some
schematic and then maybe write your static `rst` files around it to explain
the schematic.

