A GitHub Code Archeology adventure

At work, I am part of a team which maintains the Enthought Tool Suite (ETS). ETS includes a number of open source Python packages which are the foundation of the desktop applications we build at work. One of those packages is enable, a 2D drawing library.

Yesterday, I came across this issue on enable regarding a python module (kiva.fonttools.sstruct) which wasn't used anywhere in the enable codebase at the moment. "... at the moment" is important here because the enable GitHub repository has been evolving for over 10 years now and we don't know if at some point in the past, kiva.fonttools.sstruct was used somewhere in enable. Before we decide on removing the python module from the codebase, we need to understand when it was added, who was using it and how and when the dependency on enable.fonttools.struct got removed from enable. Only then can we confidently remove the module without worrying about unexpected repurcussions.

So, I started digging.

The first thing to do is to look at the history of the module itself via the GitHub interface. We do that by clicking on the History button on the right.

Use the History button on the right.

 

That will now show us the history of the kiva.fonttools.sstruct Python module

The list of Git commits which modified the kiva.fonttools.sstruct python module.

 

The first commit which modified this module in this tree - which isn't actually the commit which created this module.
Click on the highlighted "<>" button to browse the GitHub repository at that commit.

Here we see the first issue. The first commit in the list of Git commits reported on GitHub is "move kiva out of enable namespace". But, if you look at that commit, you will notice that the commit simply moved `kiva.fonttools.sstruct` from one location to another. But, because we moved the module while making other significant changes in the codebase, Git wasn't able to keep the history of the kiva.fonttools.sstruct module intact.

To understand the history of kiva.fonttools.sstruct, we need to dig deeper. We can do that by looking at the Git history of the commit - 76683223b1d76d94b8557d41ea266c823153fa37. In that history, let's browse the GitHub repository and look for kiva.fonttools.sstruct in the commit c61bd1dcd993541020a90244fa8fe2d461255e99. 

Let's browse the GitHub repository at the selected commit.

This is where we run into the same problem again. When I find the kiva.fonttools.sstruct module again, which now lives in enable.kiva.fonttools.sstruct, there is only one item in the Git history. This is the same issue that we just saw - someone moved modules around in the Git repository and Git lost the history of this file. Let's go one level deeper and repeat what we just did - browse the history at the Git commit and try to find an older commit which we can explore further.

Inception - We need to go one level deeper

 

This time, we explore the GitHub repository at the commit a67d58bffd935aec385198f876645057d500d9e3. Here again, if we find the sstruct module and look at its history, we see the same problem - broken history. So, again, we go through the same process to find a commit where the git commit history for the module is intact.

Finally, we arrive at the following commit - c40a38cb216544a9da393503af47083b38f84b05. We can see that the sstruct module didn't exist in any previous commit so the module was introduced to git in this commit.

Now, if we explore the codebase to search for who uses the kiva.fonttools.sstruct module, we find that it is used by the vendorized fontTools module in kiva.

There we go. We now know when the sstruct Python module was added to the Git repository and who uses it.

Now, we need to understand why no one uses the sstruct module at the moment. This is simpler because we now just need to find out what happened to the vendorized fontTools code in kiva. If we look at the latest commit on the main branch, you will see that the vendorized fontTools code doesn't exist. So, at some point in the past, it must have been removed. We now need to find the PR where it was removed from the git repository. Searching through the merged Pull Requests on the enable GitHub repository leads me to the following PR - Remove vendored fontTools.

In that PR, the modules which use kiva.fonttools.sstruct are removed. Because no one else uses the sstruct module, it too could have been removed in the same PR but it looks like we didn't.

Finally, we have all the answers we need to resolve the original issue.

The sstruct Python module was added to the codebase to support the vendorized fontTools code. The vendorized fontTools has since been removed but we forgot to remove the sstruct module. So, we can now safely remove the module without worrying about any unexpecte repurcussions.

Popular posts from this blog

Farewell to Enthought

I started my professional career at Enthought , at the Pune office in India, on Feb 29, 2016. Over the course of eight long years, I spent time at the Austin office in the US, the Cambridge office in the UK, and I worked remotely from India. I was let go on March 27, 2024 . I had an unbelievably great time. I had three amazing mentors - Pankaj Pandey, Senganal Thirunavukkarasu, and Mark Dickinson. Pankaj mentored me in my first year at Enthought. I started paying attention to the craft of software after working with him. Senganal started mentoring me in my second year at Enthought, and continues to mentor me, even after the both of us have moved on from Enthought. He instilled professionalism in me, ensuring that the code I wrote produced precise scientific results, and that I tackled the known unknown aspects of a project first, to ensure timely communication about project progress. I was eager to manage people at the time and Senganal made me realize that I didn't want to manage ...
Read more

Arxiv author affiliations using Python

So, I wanted to get author affiliation information from papers on arXiv. arXiv provides with an API to bulk query their database and get information. Following that, I look for the attribute 'arxiv:affiliation' in the html data. Here's the code - import urllib from BeautifulSoup import BeautifulStoneSoup   url = 'http://export.arxiv.org/api/query?search_query=all:astro&start=0&max_results=1000'   data = urllib.urlopen(url).read() soup = BeautifulStoneSoup(data) #print(soup.prettify()) #list = soup.findAll('arxiv:affiliation') #for i in range(len(list)): #        print list[i].contents   test = [tag.string for tag in soup.findAll('arxiv:aiffiliation')] Now, the problem I'm having is that I'm getting affiliation of all authors which I want to split into sets of affiliations of authors of a paper, which I'm stuck on at the moment. Once I get that part, I can move on to the next part of this pet project, displaying these...
Read more

Elementary (particle physics), my dear Watson

Ground Control to Major Tom:  Shruti Patel checking in. Since my dear friend and fellow-physicist Rahul Poruri is homeward bound, and shall remain there for a week, I'm stepping in to continue his tradition of daily posts to Astronut . Even as I write, I'm not quite sure what this post is going to be about, so please forgive me for occasional rants and digressions.  By the way Rahul, I totally resent you for the name of this blog- I'm never going to start my own blog because there's no way I'll come up with a name as cool as Astronut! For now, I shall try and be happy with being a guest writer! Before I begin, a little bit on who I am and what I do; so that you can trust that my general views on science- particle physics in particular- are not entirely baseless. I graduated with a masters in Physics from IIT Madras in July this year, and I'm currently working towards (an eventual) PhD with the Theory Group at the Deutsches Electronen-Synchrotron (DESY)...
Read more