An obsession

I don't have anything new to add so i am going to write about an obsession of mine. Almost an year back, I was at IIST Trivandrum working with Prof. Anand Narayanan on Quasars. It was my first serious summer project on astronomy and i was having a fun time learning about quasars and working with python and sql query. I was asked to reproduce the results of the richards et al. 2001 paper but as I eventually learnt, it was getting harder and harder to find the original data set. So, instead of reproducing the results from the original data set, I intended to extend the results to a new data set - the data for quasars from the SDSS DR9. The original data set in question was put together using quasars from the SDSS DR3 and other surveys. As i reproduced the results for the new (much larger) data set, weird artifacts started popping up in the results. As i dug deeper, I found that these artifacts were grouped in red-shift space and in coordinate space. Further, when i looked at the spectra for these objects, they looked nothing like those of quasars. Either their SNR was too low or the spectra looked more like that of late-M type stars. I kept dugging up digital sky survey and SDSS images for the portion of sky where these objects were grouped and went through my data analysis one more time to be sure i didn't make any mistakes. Eventually, I mailed Prof. Richards with my data set and doubts I had. He replied saying that the pipeline which identifies quasars from the SDSS data is (ofcourse) not perfect (DUH!) and if I wanted a data set which was only quasars, I should use either Schender et al. 2010 or Paris et al. 2014 which were checked and cleared of such artifacts which could've crept through the SDSS pipeline. I spent close to two months on this only to come at the conclusion that I should've used a more refined data set and not the crude one. I wouldn't say that the time spent was a complete waste because i got better at python. The data set is available as a github repository here and you can look at my previous blog posts on this topic from June and July 2013 for more on quasars and my work.

21:00:30 - 21:19:00

Popular posts from this blog

Animation using GNUPlot

Image
Animation using GNUPlot I've been trying to create an animation depicting a quasar spectrum moving across the 5 SDSS pass bands with respect to redshift. It is important to visualise what emission lines are moving in and out of bands to be able to understand the color-redshift plots and the changes in it. I've tried doing this using the animate function in matplotlib, python but i wasn't able to make it work - meaning i worked on it for a couple of days and then i gave up, not having found solutions for my problems on the internet. And then i came across this site, where the gunn-peterson trough and the lyman alpha forest have been depicted - in a beautiful manner. And this got me interested in using js and d3 to do the animations and make it dynamic - using sliders etc. In the meanwhile, i thought i'd look up and see if there was a way to create animations in gnuplot and whoopdedoo, what do i find but nirvana! In the image, you see 5 static curv
Read more

Arxiv author affiliations using Python

So, I wanted to get author affiliation information from papers on arXiv. arXiv provides with an API to bulk query their database and get information. Following that, I look for the attribute 'arxiv:affiliation' in the html data. Here's the code - import urllib from BeautifulSoup import BeautifulStoneSoup   url = 'http://export.arxiv.org/api/query?search_query=all:astro&start=0&max_results=1000'   data = urllib.urlopen(url).read() soup = BeautifulStoneSoup(data) #print(soup.prettify()) #list = soup.findAll('arxiv:affiliation') #for i in range(len(list)): #        print list[i].contents   test = [tag.string for tag in soup.findAll('arxiv:aiffiliation')] Now, the problem I'm having is that I'm getting affiliation of all authors which I want to split into sets of affiliations of authors of a paper, which I'm stuck on at the moment. Once I get that part, I can move on to the next part of this pet project, displaying these
Read more

Thoughts on the National Deep Tech Startup Policy

The Principal Scientific Advisor to the Government of India published a draft policy to support and nurtue the unique requirements of Deep Tech startups in India. I just finished reading it and put together a few comments in a FOSS United forum thread . I will probably read it again a week or so later and try to update my comments on the thread with any new insights/thoughts I have.
Read more