Including API documentation in docs built with Jupyter Book

The title is chock full of jargon. Let me break it down.

API stands for Application Programming Interface. An API is the user-facing part of a software package. For example, if you use the "numpy" package, "numpy.array" is an API. And APIs need to be documented. A single API can produce different results depending on the inputs so documenting the full capabilities of an API is important.

Jupyter Book is a new tool in the Scientific Python community to create and upload documentation easily. The community as a whole seems to be working towards better documenting their projects and some have chosen to go with Jupyter Book instead of relying on Sphinx.

I personally don't know why the community felt the need to move away from vanilla rst (reStructureText) and Sphinx. One of the advantages of Sphinx is the fact that API documentation for the package can be created automatically (using sphinx-apidoc) and the API docs can be included in the user-facing documentation. This is fairly common practice in the Python community in general. See API documentation in Traits and TraitsUI.

I was recently working with a package that uses Jupyter Book for their documentation needs and what stood out to me was the fact that they were hosting their API documentation on their GitHub Wiki page. They had manually documented and formatted the API (functions) in the wiki page and linked to sections of the wiki page from the Jupyter Book docs where relevant. The formatting of the documentation on the wiki page wasn't ideal so I initially wanted to improve that. But, when I took a step back to look at the big picture, the fact the documentation was linking to the GitHub wiki page felt off to me. So I decided to look into how API documentation can be linked in docs built using Jupyter Book. It's not like I have anything better to do this weekend.

In the end, the solution is non-trivial but it's definitely possible, using the "eval-rst" MyST directive.

  1. Manually create an rst file that uses the "autofunction" Sphinx directive to autogenerate the API document for a function of interest
  2. Use the "eval-rst" MyST directive to include the above rst file into the Jupyter Book markdown docs
  3. Install the package containing the function of interest in the Python environment before building the Jupyter Book documentation

It took a lot of trial and error to get here. I initially tried directly using the "autofunction" sphinx directive within a "eval-rst" MyST directive but that didn't work. Then I thought maybe I could autogenerate the API docs using Sphinx and then include those files in the documentation. This was a step in the right direction, which eventually led to the short and sweet 3 steps you see above. All in all, I think it took me around 6 hours to look into the possibilities, try out a few different things, come across on a solution and refine it to remove unnecessary pieces.

If you know of a cleaner, simpler, better solution, I would love to know!

For a concrete example, see the following Pull Request on GitHub. Now we wait and see if the maintainers of the GitHub repository are open to the changes I introduced in that PR.

Popular posts from this blog

Arxiv author affiliations using Python

So, I wanted to get author affiliation information from papers on arXiv. arXiv provides with an API to bulk query their database and get information. Following that, I look for the attribute 'arxiv:affiliation' in the html data. Here's the code - import urllib from BeautifulSoup import BeautifulStoneSoup   url = 'http://export.arxiv.org/api/query?search_query=all:astro&start=0&max_results=1000'   data = urllib.urlopen(url).read() soup = BeautifulStoneSoup(data) #print(soup.prettify()) #list = soup.findAll('arxiv:affiliation') #for i in range(len(list)): #        print list[i].contents   test = [tag.string for tag in soup.findAll('arxiv:aiffiliation')] Now, the problem I'm having is that I'm getting affiliation of all authors which I want to split into sets of affiliations of authors of a paper, which I'm stuck on at the moment. Once I get that part, I can move on to the next part of this pet project, displaying these
Read more

Farewell to Enthought

I started my professional career at Enthought , at the Pune office in India, on Feb 29, 2016. Over the course of eight long years, I spent time at the Austin office in the US, the Cambridge office in the UK, and I worked remotely from India. I was let go on March 27, 2024 . I had an unbelievably great time. I had three amazing mentors - Pankaj Pandey, Senganal Thirunavukkarasu, and Mark Dickinson. Pankaj mentored me in my first year at Enthought. I started paying attention to the craft of software after working with him. Senganal started mentoring me in my second year at Enthought, and continues to mentor me, even after the both of us have moved on from Enthought. He instilled professionalism in me, ensuring that the code I wrote produced precise scientific results, and that I tackled the known unknown aspects of a project first, to ensure timely communication about project progress. I was eager to manage people at the time and Senganal made me realize that I didn't want to manage
Read more

Elementary (particle physics), my dear Watson

Ground Control to Major Tom:  Shruti Patel checking in. Since my dear friend and fellow-physicist Rahul Poruri is homeward bound, and shall remain there for a week, I'm stepping in to continue his tradition of daily posts to Astronut . Even as I write, I'm not quite sure what this post is going to be about, so please forgive me for occasional rants and digressions.  By the way Rahul, I totally resent you for the name of this blog- I'm never going to start my own blog because there's no way I'll come up with a name as cool as Astronut! For now, I shall try and be happy with being a guest writer! Before I begin, a little bit on who I am and what I do; so that you can trust that my general views on science- particle physics in particular- are not entirely baseless. I graduated with a masters in Physics from IIT Madras in July this year, and I'm currently working towards (an eventual) PhD with the Theory Group at the Deutsches Electronen-Synchrotron (DESY)
Read more