Posts

Showing posts with the label data

Dude, Where's my Train? - The Indian Railways.

I have a friend who was traveling from New Delhi to Guwahati and due to certain constraints, she had to take train number 12502. She had made further plans of traveling from Guwahati based on the assumption that she would reach Guwahati at the expected arrival time. You all know where this is going. The train was late and she had to make changes to her travel plans. We all have either known someone who went through this or personally went through this ourselves. Usually, trains run by the Indian Railways are not more than an hour late. There are ones who run perfectly on time too. And then there are also trains that are multiple hours late, sometimes even > 6! Which I don't think is acceptable. And because I had nothing better to do on a Sunday, I set about to do something about it. If you guys have read a few of my earlier blog posts, you know where this is going. I'm going to write some code that will help me automate something. Or get some data. Or make a plot or a map...

A preliminary look at my activity on Facebook.

Image
Because I have nothing better to do on a Friday night, I downloaded whatever data that Facebook had on me, which you can as well by going to this part of Facebook  and clicking on the "Download a copy of your Facebook data". It might take a couple of minutes but you'll finally get a zipped file, one that will contain a "wall.htm" file inside of it. The contents of this "wall.htm" file is what I'll constrain myself to at this moment. Here's a small part of the file to give you an idea of the kind of information available in this file. 1 2 3 4 5 6 7 8 9 10 11 12 <p> <div class= "meta" > Monday, January 11, 2016 at 1:13am UTC+05:30 </div> Rahul Poruri shared Lunarbaboon &#039; s photo. </p> <p> <div class= "meta" > Sunday, January 10, 2016 at 6:32pm UTC+05:30 </div> Rahul Poruri shared a link. </p> As you can see above, there between th...

Arxiv author affiliations using Python

So, I wanted to get author affiliation information from papers on arXiv. arXiv provides with an API to bulk query their database and get information. Following that, I look for the attribute 'arxiv:affiliation' in the html data. Here's the code - import urllib from BeautifulSoup import BeautifulStoneSoup   url = 'http://export.arxiv.org/api/query?search_query=all:astro&start=0&max_results=1000'   data = urllib.urlopen(url).read() soup = BeautifulStoneSoup(data) #print(soup.prettify()) #list = soup.findAll('arxiv:affiliation') #for i in range(len(list)): #        print list[i].contents   test = [tag.string for tag in soup.findAll('arxiv:aiffiliation')] Now, the problem I'm having is that I'm getting affiliation of all authors which I want to split into sets of affiliations of authors of a paper, which I'm stuck on at the moment. Once I get that part, I can move on to the next part of this pet project, displaying these...

Variable stars from Stripe 82.

I finally uploaded the ipython notebooks related to variable stars, the thing that I was working on over the weekend. And I also started converting a solution to the ant colony optimization problem that my and friends had worked on a couple of years to python from matlab. We had worked on it as part of a competition, that we ended up actually winning. I am a still working on it but you can access it here . I just figured out that I should look through the file containing object name and period to search for objects with small periods and then plot those light curves, to see variability.

Astronomical data - object coordinates

As part of the workshop on astronomical data analysis that I helped organize during Shaastra '15, we introduced participants to celestial coordinates and how to query for object coordinates using sql query on the SDSS sql query page . Here's a handy ipython notebook that runs through the sql query code, show you how the data set queried for looks like and how to load the data and plot it using python. It also gives two different perspectives of the same data set i.e one plot made using the equatorial coordinates and the other made using galactic coordinates. PS - if you work on the terminal regularly, I suggest you look up what the reverse-i-search is and how it can make your life on the terminal easy!

Colors of Quasars- Data Acquisition and Analysis

Image
Over the course of my project, i've had to work with multiple data sets. Two of the important data sets were the original data set used by Richards et al in their paper and a new data set that i acquired - data set whose samples are constrained by the same conditions as mentioned in the richards et al paper. Part of the original data used in the paper can be acquired from browsed from the vizier archived data , SIMBAD object list and NED object list . These data sets can be found through a simple search for the paper on SAO/NASA ADS. As mentioned, this is just a part of the original data set i.e data pertaining to only 898 of the 2625 quasars used in the original paper are available for download through these links. The location for the rest of the sample set is still elusive - maybe they were released as part of the SDSS Data Release 1 which is why they weren't specifically included in this data set. This conclusion is based on the fact that none of these 898 objects a...