Skip to content

Everybody is wrong! (About language popularity)

February 18, 2013

PyPL has gotten a lot of attention lately as index of programming language popularity. It criticizes TIOBE, the most popular popularity index, and it has some valid criticism, mainly that it is a lagging index, it looks, amongst other things, at how many pages about the language that Google finds when you search. PyPL instead uses Google Trends to look at how many searches for a language. That does make sense. And while TIOBE searches for “<language> programming” TIOBE believes this underestimates some languages, as you don’t need to type “programming” as a qualification for PHP.

The difference in results are not just significant, but stunning. In PyPL, Objective-C doesn’t even chart, while in TIOBE it’s the third most popular language. How can this be? One of these indexes must be wildly wrong about Objective-C. And unfortunately, it’s PyPL. Because PyPL falls in the same trap it tries to avoid. Instead of underestimating PHP by looking at “PHP programming” when “PHP” will do, it completely underestimates Objective-C by searching for “Objective-C tutorial” when that’s simply not a search people uses. Objective-C has seen a massive rise in popularity that’s to Apples iOS. Nobody cares that it’s Objective-C. What they search for is “iPhone tutorial” or rather “iPhone programming tutorial” or similar.

But those search terms are too wide, because you can use other languages as well.

The solution? Probably Google hits or searches shouldn’t be used at all, or at least only be a small part of the total score. PyPL also claims that “An analysis of language tag followers on StackOverflow, or of the visits of the Wikipedia language pages result in a similar ranking of objective-c.” That’s complete bogus. If you look at how many visits the Objective-C page on Wikipedia has, or how many new questions per day there are on Stackoverflow, then Objective-C ranks as about half as popular as Python. That’s a big difference from TIOBE who ranks it as twice as popular than Python. But it’s an even bigger difference than PyPL where Python is 7 times as popular as Objective-C.

The only real conclusion to this is of course that ranking languages is hard.

About these ads

From → python

11 Comments
  1. PyPL ranks Objective-C 7 times lower than Python, not 30 times (that was for search with the programming keyword, not with the tutorial keyword).

    So, compared to Wikipedia page views, TIOBE overrates objective-C by a factor of 4, while PYPL underrates it by a factor of 2.

  2. fkm permalink

    I also wonder, if people need to do more searching for languages, where it’s much easier to make mistakes or where it’s harder to implement things, i.e. C – which also gives completely wrong numbers on popularity.

    • No, they would search for something more specific. But it might be an issue for looking at Stackoverflow as a source of popularity.

  3. PHP might be popular because of the “how do I replace PHP” search ?

  4. “ranking languages is hard” I completely agree, many people make it seem to simple and provide charts looking at which everybody quickly jumps to wrong conclusions. Also you can check out my article http://smthngsmwhr.wordpress.com/2012/11/19/measuring-popularity-of-programming-languages/ I did a quick research on the subject some time ago

  5. I believe this will be the modern replacement for PyPL: It indexes primarily on lines of code changed or updated to GitHub, and active tags on StackOverflow. Much better representation IMO.

  6. I believe this will be the modern replacement for PyPL: It indexes primarily on lines of code changed or updated to GitHub, and active tags on StackOverflow. Much better representation IMO.

    http://langpop.corger.nl/

    (sorry did not include link in last post.

  7. I think looking at lines of code on github will actually underestimate “corporate” languages like Java, as they are less likely to use github, and stackoverflow might do the same, for similar reasons. In short, having just two parameters is likely to introduce biases.

  8. There are various indices other than TIOBE :

    1. Redmonk : They consider number of projects in github for a language as well as number of questions tagged for a language in stackoverflow. But there also Objective-C is in the #9th position.

    2. O’Reilly does a study based on the computer book sales. There also Objective C is with in the first ten (I guess).

    3. JobTractor captures the job trends based on the job posted in Twitter. There for this month (Jan 2014), Objective C is in the third position.

    There are various other indices. Those also indicate that people are now (Jan 2014) really working on Objective C. I have enlisted different popular programming language indices which can be used to do a better comparative study :

    http://deathbycode.blogspot.in/2014/01/popularity-index-for-programming.html

  9. github tends to only track open source projects. Certain languages will be over represented in open source, while others under represented. Due to the fact objective-C’s dominant usage is in iOS and Mac development, one expects most of these projects are commercial in natural and would never show up in github. At the same time, due to the language being so closely tied to the rather rich libraries in iOS and OS X, one would expect fewer generic support library projects in github as well. If you wanted a do a open source project, objective-C may not be important, if you want a job, it may be very important.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,336 other followers

%d bloggers like this: