Rahul gives unsolicited advice

 I purchased the O'Reilly Humble Book Bundle two weeks back and I've just read my second book from the bundle. So far, I'm pretty happy with the purchase. I paid for the full bundle and last weekend, I read Introducing Go and today, I read 97 Things every Java programmer should know  and i'm pretty happy with both. "Introducing Go" is a short book that introduces the Go language. The part of it that I found most interesting was the last chapters, one of which introduced Goroutines i.e. coroutines in Go. That leads nicely to looking at "Using Asyncio in Python". I browsed through  Think Julia  and  Using Asyncio in Python  as well last weekend, the latter of which made my head hurt. I'll prolly take a closer look at "Using Asyncio in Python" next weekend. "97 Things every Java programmer should know" seems to be part of the "97 Things XYZ should know" series of books by O'Reilly. There were a lot of "Things...

Let me jump the gun and go straight to what I find mysterious - Running ` from __future__ import * ` raises a SyntaxError . Specifically, you will get the error - >>> from __future__ import *   File "<stdin>", line 1 SyntaxError: future feature * is not defined Now, that's weird and mysterious. For those of you who are not steeped in Python land, __future__ is part of Python's standard library. It is a library that can make the version of Python you use currently behave like a __future__ version of Python. For example, adding the line ` from __future__ import print_function ` on Python 2.7 will change print into a function (Python 3) instead of being a statement (Python 2). Now, so far, there's not much that's mysterious about the __future__ module. Sure, it's an interesting way to make one version of a programming language behave like another version. One of the __future__ module imports is barry_as_FLUFL , which is an e...

For those of you interested, here are the full set of slides I used for the talk - Talk slides . TL;DR : - Why should you port your applications from 2 to 3? - What does it mean to port your code base? - How can you port your packages/apps? Why? Python 2 end of life (EoL) is 2020. After it's EoL, there will be no more feature additions, changes or bug fix releases on the 2 or 2.7.x version of Python. You might not care about changes to the underlying Python language as long as your code keeps working. FYI any new security vulnerability will not be fixed on 2.7.x. Maybe you don't care about that either. FYI the packages you use will probably stop supporting Python 2.7.x. Some already have. Use numpy or pandas in your code base? They already stopped supporting Python 2 and all new feature releases will be Python 3 only. See here and here for more info. What? There are roughly four types of changes between Python 2 and 3 - Syntax changes. One of the most popu...

The first edition of the regional Python conference PySangaman got over yesterday. First of all, I applaud the team on their decision to hold the conference on Friday and Saturday. During the audience feedback session after the conference late Saturday, a few people mentioned that they'd prefer a Saturday/Sunday conference. This prompted one of the organizers, Shreyas, to respond with their reason for the current schedule, which is an obvious answer in hindsight. He mentioned that heading back to work immediately after two busy days of conferences leaves no room to ponder upon what was learnt. It also leaves little time for folks out of town to travel and relax before getting back to their usual routine. Moving on, there were a few amazing talks and there were a lot of amazing people. I don't think I've networked as much as I did at PySangaman. The keynote speaker's talks made me ask a lot of questions, mostly in person. A couple of talks introduced me to some very a...

I came across this really concise and approachable introduction to Recursion in Python and thinking recursively in programming. While it didn't teach me anything new, it was a good way to talk about the topic. And extra points for the visuals!

Last weekend, I gave a brief talk on Virtual Environments in Python at the April meet of the PythonPune meetup group . Yesterday, I conducted a workshop on the same topic at the May meet of the PyLadies meetup group . The slides can be found on a GitHub repository . The GitHub repo also contains slides for most other talks/workshops I gave/conducted. To give you a brief overview of the slides, I introduce what a Virtual Environment are, why we should use them and finally, how we go about using them. Along with Virtual Environments, I also introduce environment managers such as conda and edm and talk about how the environments created by them differ from virtual environments. Neither virtual environments not environment managers help understand the Python language better but they are crucial to using Python in enterprise/open-source projects. A large number of open source Python libraries suggest the use of virtual environments if you are interested in contributing.

Yesterday, I conducted a workshop on Python Packages at the May meet of the PyLadies meetup group . Slides I used for the workshop can be found on the GitHub repository . The GitHub repository also contains slides I used for other talks/workshops I gave/conducted. To give you a brief overview of the workshop, I first introduced what a Python package is, why we should use them and finally, how to go about creating a Python package. Python packages make distribution and installation of your work easy, crucial to get others to use your work. Two references I highly recommend are the sample project published by PyPA and the Packaging User Guide, also published by PyPA .

The cost of launching satellites is going down, be it in the US or in India. Smaller and easier to develop micro-satellites are the latest trend, usually developed with a specific goal in mind e.g. satellite imagery. These, and a lot of other factors, have contributed to an increase in the number of satellites hovering over earth in the recent years, a number which is bound to only keep increasing. ISRO recently put 88 (micro) satellites belonging to Planet Labs , satellites which Planet Labs will use to image the Earth everyday. ISRO and other government space organizations themselves have satellites that image the Earth and/or their respective countries on a regular basis. And a number of these organizations are releasing their data publicly. With that context, I realized a while back that daily imagery will help identify and possibly curb sand mafia, specifically in India . The boom in infrastructure, specifically housing and office construction in India, was one of the reasons...

My fascination of drones is growing day after day. Of all the areas drones can be used, I want to talk about exploration for the moment. You might be wondering, what exploration? All corners of the Earth have been explored and there's nothing else to discover. Au Contraire. Correction. The surface of the Earth has been explored, not what's underneath. NASA and other space organizations have been sending out satellites and telescopes to understand what's out there but similar efforts aren't being made to understand unexplored caves and the depths of the ocean. Two articles that I read recently drove this point home for me; one on exploring a cave formation in Uzbekistan and another on exploring the Indian Ocean's floor . The first is an account of a team of scientists' and explorers' attempt at exploring a cave formation and discovering new regions of the cave. The second is an account of how the search for the missing MH370 flight also led to a detaile...

This article on Baidu and it's bet on AI sparked a thought in my head. Well, the thought was to basically copy what Baidu is doing in China and do the same in India. I wrote earlier on the language barrier in India and how AI can help solve the problem. The crux of what Baidu is doing in China is to employ a bunch of people to translate English technical documents to Chinese (Mandarin prolly). By doing so, not only is the industry able to communicate with Chinese customers better, Baidu now has a big database of English-Chinese (Mandarin) translations. Don't you think doing the same with Indian languages would be awesome? Instead of hackathons and docathons, we could have translate-athons (?) where volunteers could translate documents into the numerous Indian languages and in the process, help in compiling an impressive English-Indian language dataset, a dataset that can now be used to train AI to better translate between English and Indian languages and maybe even betwe...

I gave a brief talk on Cython during the February meet of the PythonPune meetup group . The talk I had in mind when I had proposed the session and the one I ended up delivering at the meet were very different, mainly because I wanted to limit the scope of the talk. Initially, I wanted to dive into Cython and give numerous code examples. However, when I finally sat down the make the presentation, I chose to lay back and give an overview of why Cython is needed in the first place and other ways to speed up Python code. I talked about how Cython can be used to speedup Python routines and work with existing C/C++ code bases. I took a detour and asked the audience why Python was slow in the first place, in comparison to C/C++/Java/JavaScript. After driving the difference between interpreted, compiled and JIT-compiled languages, I introduced JIT compilers available in Python land e.g. Psyco, PyPy and Pyjion. PyPy is the most popular alternative compiler to the Python language but isn...

Natively, I speak Telugu. I've learnt to speak Hindi and English. I currently live in a city, where most speak Marathi. I used to live in a city where most spoke Tamil. And I'm not atypical. Most Indians cross state lines over the course of their lives, for work or other reasons. And the language barrier is the biggest problem while doing so. Personally, even though I lived in Chennai for over 6 years, I still can't speak or understand Tamil and Marathi might as well be Arabic to me, even after living in Pune for just over an year. (I'm clearly not trying hard enough to learn the language.) Unlike me, most people who stay for a long (1+ year) at a location learn the local language, because it makes day-to-day life easier. But what about travelers? What about those who stay for a short period of time? What about those not educated in a common language? You can argue that a majority of Indians can speak and understand Hindi. Actually, No. And no, the majority of India...

If you've been following my blog, you would've noticed my efforts to give talks on Python and get better at public speaking. Early December, I organized two workshops at SciPy India at IIT Bombay. Late December, I gave a talk at a local Python meetup. Early January, I attended the local PyLadies event. I wanted to do something similar at my almamater, IIT Madras. It also gave me a reason to go back and visit some of my classmates who are now grad students at IITM and the professor I worked with as an undergrad. Coming to the point, I gave number of sessions on Friday night and Saturday. The session on Friday night was on Git & GitHub. I used the same slides that I had during the SciPy workshop but we went further at IITM because we had internet connectivity. Towards the end, everyone had created a GitHub repository and everyone had pushed their local changes to GitHub. I didn't get time to get into what I call the advanced workflow, which involves branches and merging...

Inspired by this blog post : https://langui.sh/2016/12/09/data-driven-decisions/ , I wanted to play around with Google BigQuery myself. And the blog post is pretty awesome because it has sample queries. I mix and matched the examples mentioned on the blog post, intent on answering two questions -  1. How many people download the Pandas library on a daily basis? Actually, if you think about it, it's more of a question of how many times was the pandas library downloaded in a single day, because the same person could've downloaded multiple times. Or a bot could've. This was just a fun first query/question. 2. What is the adoption rate of different versions of the Pandas library? You might have come across similar graphs which show the adoption rate of various versions of Windows. Answering this question is actually important because the developers should have an idea of what the most popular versions are, see whether or not users are adopting new features/cha...

For some background, read my previous posts on the topic - simple queries to start understanding Pandas downloads ( https://rahulporuri.blogspot.in/2017/01/on-whos-downloading-pandas.html ),  building up queries incrementally to understand Pandas downloads ( https://rahulporuri.blogspot.in/2016/12/pandas-download-statistics-pypi-and.html ) and using Tableau Public to visualize Pandas downloads by version over the last 6 months ( https://rahulporuri.blogspot.in/2017/01/visualizing-pypi-pandas-download.html) Having visualized the total number of downloads per version per month of Pandas in the last post, we now come to the total number of downloads per month by location. The relevant query is SELECT STRFTIME_UTC_USEC( timestamp , "%Y-%m" ) AS yyyymm, country_code, COUNT ( * ) as total_downloads, FROM TABLE_DATE_RANGE( [the - psf:pypi.downloads], DATE_ADD( CURRENT_TIMESTAMP (), - 6 , "month" ), CURRENT_TIMESTAMP () ) WHERE file.pro...

Before I get to my point, let me take you on a detour. I was having breakfast on the 1st of Jan with a bunch of friends, and one of the friends' niece. The niece just finished her 8th standard and is moving to the 9th. Every single person at the dining table was from a different career direction - an MBA, a physics student turned programmer, a graduate student studying ecology, an MBBS student and an engineer turned entrepreneur. And we were all talking to her about academics in general and about specific subjects like mathematics and science. Eventually, when it was my turn to talk, I just asked her to learn how to program. No matter what she decides to become, a doctor or an engineer, learning how to program will open new doors for her. Now, let me get to my main point, which are the infinite new possibilities that arise from being a programmer. Programming is becoming an integral part of every profession and learning how to program will open doors that you didn't even kno...

This post/video is, IMO, a sister post of the previous one - https://rahulporuri.blogspot.in/2016/12/on-bias-at-work.html . The video, embedded below, talks about ethics in software design and the IT sector in general. All of us have seen these so called Dark Patterns on the internet/websites. Watch the video and you'll probably recognize a number of them. Better yet, visit the website : http://darkpatterns.org/ and you'll find more examples which couldn't be covered in the talk. Just watch the talk, I'm not going to give you a summary. What I want to talk about is the topic of Ethics in the Software Engineering world. Engineering Ethics is a course every student sits through in college. For most it is a formality. And given that now-a-days most college graduates end up working with and actively developing software, it is important to understand and lookout for ethical boundaries that we might be crossing while developing software. There is another beautiful/...

I just watched this brilliant keynote today. It's a commentary on Programmers and the software development industry/ecosystem as a whole. I am not going to give you a tl;dr version of the talk because it is a talk that I believe everyone should watch, that everyone should learn from. Instead, I am going to give my own parallel- ish views on programmers and programming. As pointed out in the talk, there are mythical creatures in the software development industry who are revered as gods. Guido Van Rossum, the creator of Python, was given the title Benevolent Dictator For Life (BDFL). People flock around the creators of popular languages or libraries. They are god-like to most programmers and are treated like gods. By which, I mean to say, we assume they don't have flaws. That they are infallible. That they are perfect. And alongside this belief in the infallibility of these Gods, we believe that they were born programmers. That programming is something that peopl...

I rehashed the workshop I conducted at SciPy last weekend into a 20-ish minute talk today at the local Python meetup group. I've been following their activities for well over 4 months now and every time it so happens that I'm traveling when they organize the meetup. Every month for the last 4 months. And finally. Finally. Today, there were no conflicts. It couldn't have happened at a better time either. I already had all of the material because I used the same slides as those I used during the workshop. Ohh, if you don't know, the workshop I conducted was on `Automated Testing using Python`. I talked about the unittest module and the mock module and I briefly mentioned the pytest and nose test runners in passing. Overall, the talk was well received. A few people came up to ask me about follow up questions, about how mock can actually help with testing a large code base and I tried my best to give them examples. I also talked to a few of the regulars and the organiz...

I had nothing better to do on a Sunday morning so I made a map. If you have been following my blog, you will know that I am trying to make sense of the Indian Railways, why the trains runs late  and if there's anyway we can learn about the cause of the delays, based on patters in delay. Towards that goal, I posted two blog posts that described how and where I was collecting my data from and a first look at the data I was gathering . Even from the preliminary look , it can be seen that there are specific routes/stations that are causing a delay along a train's route. And the delays were induced on multiple runs at the same station, meaning that it wasn't simply a one time thing. Moving on, another way to look at the problem is to understand how crowded the railway lines are. By mapping all train movement in India, we will be able to understand how crowded specific routes are and if they're crowded at a specific time of the day. By adding delay information ...