You are currently browsing the tag archive for the ‘ruby’ tag.
I compiled some programming language popularity statistics in April 2009, October 2009, October 2010, September 2011 and August 2012 . Here’s an update for February 2013:
I made a number of Google searches of the forms below and summed the results:
"implemented in <language>" "written in <language>"
Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within the same search since the number of results Google returns can vary greatly over time.
I’ve divided the table into sections based on large percentage drops from one language to the next.
|------+-----------------+------------+----------+-------+------------| | Rank | Language | # Search| Previous | Rank | Delta from | | | | Results| Rank | Delta | Apr '09 | |------+-----------------+------------+----------+-------+------------| | 1 | PHP | 52,699,000| 1 | | 3 | | 2 | C | 39,330,000| 2 | | -1 | | 3 | C++ | 26,490,000| 4 | 1 | | | 4 | Python | 22,410,000| 3 | -1 | 1 | | 5 | C# | 21,474,000| 5 | | 2 | |------+-----------------+------------+----------+-------+------------| | 6 | Perl | 11,013,000| 8 | 2 | | | 7 | Java | 10,150,000| 6 | -1 | -5 | | 8 | JavaScript | 7,340,000| 9 | 1 | 1 | |------+-----------------+------------+----------+-------+------------| | 9 | Ruby | 3,456,000| 7 | -2 | 1 | | 10 | Lisp Family (1) | 2,955,000| 10 | | -2 | | 11 | FORTRAN | 2,256,000| 11 | | N/A | | 12 | Lisp | 1,708,000| 17 | 5 | | | 13 | R | 1,305,000| 21 | 8 | N/A | | 14 | Tcl | 1,072,100| 13 | -1 | -1 | | 15 | Lua | 1,011,000| 19 | 4 | 5 | | 16 | ML Family (2) | 988,400| 16 | | -2 | | 17 | Erlang | 842,000| 18 | 1 | -1 | | 18 | COBOL | 729,200| 23 | 5 | N/A | | 19 | Haskell | 707,000| 12 | -7 | -4 | | 20 | Common Lisp | 557,000| 20 | | -2 | | 21 | OCaml | 528,000| 24 | 3 | -4 | | 22 | Prolog | 521,000| 25 | 3 | -3 | | 23 | (S)ML (3) | 496,800| 27 | 4 | 1 | | 24 | Scala | 426,100| 22 | -2 | 1 | | 25 | Scheme | 347,000| 28 | 3 | -14 | | 26 | Groovy | 320,000| 14 | -12 | N/A | |------+-----------------+------------+----------+-------+------------| | 27 | Smalltalk | 201,400| 29 | 2 | -6 | | 28 | Go | 201,200| 15 | -13 | N/A | | 29 | CoffeeScript | 182,800| 31 | 2 | N/A | | 30 | Clojure | 173,100| 30 | | -2 | | 31 | Forth | 128,800| 26 | -5 | -8 | | 32 | Caml | 102,600| 34 | 2 | -6 | | 33 | Racket | 93,500| 33 | | N/A | | 34 | Arc | 76,400| 32 | -2 | -12 | | 35 | Io | 60,200| 35 | | -8 | |------+-----------------+------------+----------+-------+------------|
(1) combines Lisp, Scheme, Common Lisp, Racket, Arc & Clojure
(2) combines OCaml, (S)ML, Caml
(3) summed separate searches for standard ml, sml & ml
I compiled some programming language popularity statistics in April 2009, October 2009, October 2010 and September 2011 . Here’s an update for August 2012:
I made a number of Google searches of the forms below and summed the results (some earlier posts averaged the results):
"implemented in <language>" "written in <language>"
Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within the same search since the number of results Google returns can vary greatly over time.
While formatting the table, I noticed there was a fairly natural break into three sections. Languages more popular than FORTRAN, languages between FORTRAN and COBOL and languages less popular than COBOL, so I highlighted the three sections 
. I’m curious to see if any languages jump into a higher or lower section next time.
I also switched from HTML to using Emacs org-mode to create a textual table since the latter is so much nicer to deal with.
Update 8/5/12: Someone on the TriFunc mailing list mentioned the omission of Groovy. I don’t feel like updating the table, but I’ll record the total number of results here to allow for a comparison next time. Groovy came in at 460,300 results which puts it above Go and in the middle section.
|----+-----------------+------------+----------+----------+------------| | | Language | # Search | Previous | Position | Delta from | | | | Results | Position | Delta | Apr '09 | |----+-----------------+------------+----------+----------+------------| | 1 | PHP | 21,120,000 | 2 | 1 | 3 | | 2 | C | 15,440,000 | 1 | -1 | -1 | | 3 | Python | 11,441,000 | 4 | 1 | 2 | | 4 | C++ | 9,788,000 | 3 | -1 | -1 | | 5 | C# | 8,319,000 | 5 | 0 | 2 | | 6 | Java | 6,020,000 | 6 | 0 | -4 | | 7 | Ruby | 4,784,000 | 9 | 2 | 3 | | 8 | Perl | 4,183,000 | 7 | -1 | -2 | | 9 | JavaScript | 3,117,000 | 8 | -1 | 0 | | 10 | Lisp Family (1) | 898,680 | 10 | 0 | -2 | |----+-----------------+------------+----------+----------+------------| | 11 | FORTRAN | 795,500 | 11 | 0 | N/A | |----+-----------------+------------+----------+----------+------------| | 12 | Haskell | 490,000 | 14 | 2 | 3 | | 13 | Tcl | 476,000 | 12 | -1 | 0 | | 14 | Go | 391,100 | N/A | N/A | N/A | | 15 | ML Family (2) | 375,780 | 17 | 2 | -1 | | 16 | Lisp | 352,000 | 13 | -3 | -4 | | 17 | Erlang | 334,000 | 15 | -2 | -1 | | 18 | Lua | 304,000 | 16 | -2 | 2 | | 19 | Common Lisp | 256,000 | 19 | 0 | -1 | | 20 | R | 221,000 | N/A | N/A | N/A | | 21 | Scala | 219,000 | 22 | 1 | 4 | |----+-----------------+------------+----------+----------+------------| | 22 | COBOL | 218,600 | 18 | -4 | N/A | |----+-----------------+------------+----------+----------+------------| | 23 | OCaml | 181,100 | 20 | -3 | -6 | | 24 | Prolog | 176,300 | 21 | -3 | -5 | | 25 | Forth | 168,700 | 27 | 2 | -2 | | 26 | (S)ML (3) | 159,700 | 26 | 0 | -2 | | 27 | Scheme | 135,500 | 23 | -4 | -16 | | 28 | Smalltalk | 114,500 | 24 | -4 | -7 | | 29 | Clojure | 74,500 | 25 | -4 | 0 | | 30 | CoffeeScript | 68,660 | N/A | N/A | N/A | | 31 | Arc | 40,990 | 30 | -1 | -9 | | 32 | Racket | 39,690 | N/A | N/A | N/A | | 33 | Caml | 34,980 | 28 | -5 | -7 | | 34 | Io | 23,110 | 29 | -5 | -7 | |----+-----------------+------------+----------+----------+------------|
(1) combines Lisp, Scheme, Common Lisp, Racket, Arc & Clojure
(2) combines OCaml, (S)ML, Caml
(3) summed separate searches for standard ml, sml & ml
The Problem
I noticed some unusual behavior with respect to YAML String serialization between my Linux production system and my Mac OSX development system.
After dumping the production database via pg_dump -O –no-acl mydb | gzip > ~/mydb.sql.gz and then restoring it on my development system via rake db:drop; rake db:create; psql mydb < mydb.sql, I noticed that a particular serialized field in my Rails app that should always be an Array of String objects occasionally contained Integers.
After a little research and experimentation, I discovered that the production Linux system would occasionally omit quotations around Strings containing only numeric digits. I haven’t analyzed the pattern fully, but here are some examples where the YAML serialization did or did not use quotes:
- “90103″
- 000080
- “000071″
- “000124″
- “000003″
- 008397
- 000408
- 000009
- 000188
- “000021″
Further investigation revealed that the Linux production system was using Syck (a “dated C implementation of YAML 1.0″) and my Mac OSX development system was using psych (a “libyaml wrapper (in Ruby core for 1.9.2)”). libyaml is a “fast C implementation of YAML 1.1. So, either the quotation rules have changed between YAML 1.0 and YAML 1.1, or there is a bug in one of the implementations (likely Syck).
The Solution
The solution for proper “future” behavior is pretty simple. Install libyaml on the Linux system as follows:
wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz tar xzf yaml-0.1.4.tar.gz cd yaml-0.1.4 ./configure make make install
I think that’s enough, but I went ahead and rebuilt my Ruby 1.9.2 just in case it needed to know about the existence of libyaml at build time.
The solution for converting my database with YAML 1.0 serialization to YAML 1.1 serialization is a bit trickier. Since the “dump” and “load” operations are matched for a particular version of YAML, it seems difficult to load the data using YAML 1.0 (thereby retaining the String type when reading an unquoted 000088) and then dump the data using YAML 1.1 (to get proper quoting of ’000088′). Further complicating this is the fact that Rails handles the serialization operations automatically.
It does appear possible to dynamically switch between syck and psyck by using the following:
YAML::ENGINE.yamler = 'syck' YAML::ENGINE.yamler = 'psych'
So, one option is to repeatedly switch to syck, read in data, switch to psych, and then write the data. <sigh>
Update:
It appears that due to the semantics of the Rails serialize function, it’s not enough to just read the model object using syck and then immediately write with psych because that doesn’t appear to be enough to cause the field to be deserialized. I had to refer to the field for each object. This is a pain because it prevents me from doing a generic loop where I can handle all model objects easily w/o reference to their specific fields.
I’ll withhold judgment for a while, but my first inclination is to consider abandoning YAML serialization for something a little more robust and portable.
Update 2:
It appears my welcome from psych is a serious memory leak. I’ve been running long running Ruby/Rails processes for years, and this is the first time I’ve experienced a failure due to an out of memory condition. There are a number of Google hits regarding the issue. After I fix the leak, I’ll begin researching alternatives to YAML serialization in Rails.
Update 3:
The number of bug reports on psych and rubygems I’ve had to wade through recently is amazing. My current solution is to remove the psych system gem and install Ruby 1.9.3p0 which required upgrading Passenger to the latest version from source to get Ruby 1.9.3 compatibility. I still had to track down a few odd errors such as “undefined method `yaml’ for #<Psych::Nodes::Stream:…>” and “invalid date format in specification: “2011-10-02 00:00:00.000000000Z”" – all because I chose to use the default Rails serialization assuming there would be no issues. Lesson learned.
See Part Five
I compiled some programming language popularity statistics in April 2009, October 2009 and October 2010 . Here’s an update for September 2011:
I made a number of Google searches of the forms below and summed the results (previous posts averaged the results):
"implemented in <language>" "written in <language>"
Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within the same search since the number of results Google returns can vary greatly over time.
| Language | Total | Prev. Position | Position Delta |
|---|---|---|---|
| C | 10,360,000 | 2 | 1 |
| PHP | 10,351,000 | 1 | -1 |
| C++ | 6,495,000 | 3 | 0 |
| Python | 5,759,000 | 5 | 1 |
| C# | 5,335,000 | 4 | -1 |
| Java | 4,890,000 | 8 | 2 |
| Perl | 3,702,000 | 6 | -1 |
| JavaScript | 3,077,000 | 7 | -1 |
| Ruby | 1,654,000 | 9 | 0 |
| Lisp Family1 | 1,022,870 | 11 | 1 |
| FORTRAN | 975,600 | 10 | -1 |
| Tcl | 594,500 | 12 | 0 |
| Lisp | 486,000 | 14 | 1 |
| Haskell | 450,500 | 16 | 2 |
| Erlang | 419,700 | 13 | -2 |
| Lua | 367,100 | 18 | 2 |
| ML Family2 | 348,400 | 17 | 0 |
| COBOL | 308,270 | 15 | -3 |
| Common Lisp | 254,900 | 19 | 0 |
| OCaml | 240,300 | 21 | 1 |
| Prolog | 224,000 | 20 | -1 |
| Scala | 203,400 | 23 | 1 |
| Scheme | 184,700 | 22 | -1 |
| Smalltalk | 129,700 | 24 | 0 |
| Clojure | 84,600 | 27 | 2 |
| (S)ML3 | 83,630 | 25 | -1 |
| Forth | 69,980 | 26 | -1 |
| Caml | 24,470 | 28 | 0 |
| Io | 17,700 | 30 | 1 |
| Arc | 12,670 | 29 | -1 |
1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml
I stumbled upon a programming challenge a company was using for recruitment purposes and thought I’d create a Haskell solution as a learning exercise. The first problem was to find the longest palindrome embedded in a text string.
The following Haskell solution seems very readable to me, but it’s a naive solution that’s inefficient. It computes an answer on my 2.6 year old Macbook Pro in under 4 seconds, but a 2x increase in text requires a 7x increase in CPU time.
I believe there are algorithms to find the longest embedded palindrome in linear time, so I may post a refinement later.
-- Find the longest palindrome in a text string. module Main where import Char text = "I'll just type in some example text here and embed a little palindrome - A man, a plan, a canal, Panama! - I expect that will be the longest palindrome found in this text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium dolor, et mattis leo turpis non est. Sed aliquet, sapien quis consequat condimentum, sem magna ornare ligula, id blandit odio nisl vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc vitae tempus purus." clean = map toLower . filter isAlpha palindrome str = str == reverse str substrings [] = [] substrings (x:xs) = substrings' (x:xs) ++ substrings xs where substrings' [] = [] substrings' (y:ys) = [y] : [ (y:s) | s <- substrings' ys ] longest [] = [] longest (x:xs) = if length x > length max then x else max where max = longest xs longest_palindrome xs = longest (filter palindrome (substrings (clean text))) main = print (longest_palindrome text)
As a comparison, I translated the program into Ruby. I program predominantly in Ruby these days, and I like it, but the Ruby version is 25 times slower (98 sec. vs. 4 sec.), and it’s 2.4 times more lines of code (31 vs. 13 – excluding the text).
A gain in runtime efficiency, expressive power and multi-core capability is very attractive!
I’m using Ruby 1.9.2 and GHC 6.12.3 on Mac OS X 10.5.8 on a 2.4 GHz Core 2 Duo w/ 4 GB RAM.
ruby 1.9.2p0 (2010-08-18 revision 29036) [i386-darwin9.8.0]
Glasgow Haskell Compiler, Version 6.12.3, for Haskell 98, stage 2 booted by GHC version 6.12.2
TEXT = <<END
I'll just type in some example text here and embed a little
palindrome - A man, a plan, a canal, Panama! - I expect that will be
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis
consequat condimentum, sem magna ornare ligula, id blandit odio nisl
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc
vitae tempus purus.
END
def clean str
str.gsub(/[^A-Za-z]/,'').downcase
end
def palindrome? str
str == str.reverse
end
def subs str
return [] if str.empty?
y = str[0,1]
subs(str[1..-1]).inject([y]) do |result, s|
result << y + s
result
end
end
def substrings str
return [] if str.empty?
subs(str) + substrings(str[1..-1])
end
def longest strs
strs.inject("") do |max, str|
max = str if str.length > max.length
max
end
end
def longest_palindrome str
longest(substrings(clean(str)).inject([]) {|result, str|
result << str if palindrome?(str)
result
})
end
puts longest_palindrome(TEXT)
Update 3/13/11 19:30
Rick’s comment (see his blog post linked to in his comment) regarding the importance of algorithm choice is certainly a valid one – a better algorithm in a slower language may win over an inferior algorithm in a faster language (for large enough datasets). However, what I’m becoming interested in is the fact that one may gain both productivity/power and runtime speed by making wise programming language choices.
In the case of an interpreted language such as Ruby, it’s helpful to “stay in C code” as much as possible. In other words, to favor built-in library routines that have been implemented in C over hand written Ruby code. As much as I like Ruby, this is one of the things that bothers me – the fact that the Ruby code written by the programmer is vastly inferior in performance to the built-in library routines.
I wasn’t planning on refining the Haskell version this soon, but after seeing Rick’s blog post response, I couldn’t resist
I should first provide some background info for context. When I wrote the original Haskell version above, I was in the middle of an online programming challenge, and my goal was simply to compute the answer in a reasonable amount of time to move on to the next challenge, so the brief, easily understandable, Haskell version worked great. Hence, my disclaimers in the post regarding the naivity and inefficiency of the solution. The Ruby code in this post was an afterthought simply to see a comparison between identical algorithms. Given that apple & apple comparison, I’m impressed with the brevity and speed of the Haskell version.
Since I’m still very much a Haskell newbie, and short on time, I found an incredible solution by Johan Jeuring. It’s a little bit longer than the Ruby version, but much, much faster. It’s so fast, I had to increase the input quite a bit to get a reasonable comparison – I replicated the original text 23 times and reversed one of the replications to make a fairly long palindrome.
Rick’s Ruby version took 169.4 seconds, Johan’s Haskell version took 0.032 seconds. In other words, Ruby takes over 5,000 times as long to compute the result. Clearly this is an apples and oranges comparison, but I fully expect that a Ruby version using an identical algorithm will take 100 times as long (or longer) to run and would be less concise. Giving up runtime performance to gain programmer power is one thing, but giving up runtime performance and power is a tough pill to swallow.
Here is a slightly modified version of Johan Jeuring’s code. He was also kind enough to provide his code here:
-- Reorganized from Johan Jeuring's solution:
module Main where
import Data.List (maximumBy,intersperse)
import Data.Char
import Data.Array
text = "I'll just type in some example text here and embed a little
palindrome - A man, a plan, a canal, Panama! - I expect that will be
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis
consequat condimentum, sem magna ornare ligula, id blandit odio nisl
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc
vitae tempus purus."
clean = map toLower . filter isAlpha
longestPalindrome input =
let inputArray = listArrayl0 input
(maxLength,pos) = maximumBy
((l,_) (l',_) -> compare l l')
(zip (palindromesAroundCentres inputArray) [0..])
in showPalindrome inputArray (maxLength,pos)
longestPalindromes m input =
let inputArray = listArrayl0 input
in concat $ intersperse "n"
$ map (showPalindrome inputArray)
$ filter ((m String
lengthLongestPalindrome = show . maximum . palindromesAroundCentres . listArrayl0
lengthLongestPalindromes :: String -> String
lengthLongestPalindromes = show . palindromesAroundCentres . listArrayl0
palindromesAroundCentres a =
let (afirst,_) = bounds a
in reverse $ extendTail a afirst 0 []
extendTail a n currentTail centres
| n > alast = finalCentres currentTail centres
(currentTail:centres)
| n-currentTail == afirst =
extendCentres a n (currentTail:centres)
centres currentTail
| a!n == a!(n-currentTail-1) =
extendTail a (n+1) (currentTail+2) centres
| otherwise =
extendCentres a n (currentTail:centres)
centres currentTail
where (afirst,alast) = bounds a
extendCentres a n centres tcentres centreDistance
| centreDistance == 0 =
extendTail a (n+1) 1 centres
| centreDistance-1 == head tcentres =
extendTail a n (head tcentres) centres
| otherwise =
extendCentres a n (min (head tcentres)
(centreDistance-1):centres)
(tail tcentres) (centreDistance-1)
finalCentres 0 _ centres = centres
finalCentres (n+1) tcentres centres =
finalCentres n
(tail tcentres)
(min (head tcentres) n:centres)
finalCentres _ _ _ = error "finalCentres: input < 0"
showPalindrome a (len,pos) =
let startpos = pos `div` 2 - len `div` 2
endpos = if odd len
then pos `div` 2 + len `div` 2
else pos `div` 2 + len `div` 2 - 1
in show [a!n|n <- [startpos .. endpos]]
listArrayl0 string = listArray (0,length string - 1) string
sampleText s = concat (replicate 8 s ++ [ "x" ] ++ [ reverse s ] ++ replicate 14 s)
main = print (longestPalindrome (clean (sampleText text)))
Here’s the relevant change to Rick’s Ruby version:
def sample_text str str * 8 + 'x' + str.reverse + str * 14 end puts clean(sample_text(TEXT)).longest_palindrome
See Part Five
I compiled some programming language popularity statistics in April 2009 and October 2009 . Here’s an update for October 2010:
I made a number of Google searches of the forms below and averaged the results:
"implemented in <language>" "written in <language>"
Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within one column since the number of results Google returns can vary greatly over time.
| Language | Apr 2009 | Oct 2009 | Oct 2010 | Position Delta |
|---|---|---|---|---|
| PHP | 680,000 | 5,083,500 | 14,096,000 | +3 |
| C | 1,905,500 | 16,975,000 | 9,675,000 | -1 |
| C++ | 699,000 | 6,270,000 | 6,510,000 | -1 |
| C# | 349,700 | 2,125,000 | 5,132,000 | +4 |
| Python | 396,000 | 3,407,000 | 5,114,500 | +1 |
| Perl | 365,500 | 3,132,500 | 4,675,000 | +1 |
| JavaScript | 102,700 | 1,163,000 | 2,120,000 | +4 |
| Java | 850,000 | 5,118,000 | 1,495,500 | -5 |
| Ruby | 99,650 | 227,000 | 1,426,000 | +13 |
| FORTRAN | 1,621,000 | 770,850 | 0 | |
| Lisp Family1 | 176,507 | 3,489,650 | 399,685 | -6 |
| Tcl | 44,800 | 382,000 | 313,400 | +5 |
| Erlang | 22,285 | 161,700 | 188,800 | +12 |
| Lisp | 61,900 | 486,500 | 174,050 | +1 |
| COBOL | 247,300 | 166,435 | +6 | |
| Haskell | 22,550 | 280,500 | 157,150 | +4 |
| ML Family2 | 29,062 | 1,003,800 | 149,005 | -5 |
| Lua | 13,065 | 131,800 | 128,150 | +9 |
| Common Lisp | 20,600 | 554,500 | 112,750 | -5 |
| Prolog | 17,750 | 390,500 | 100,000 | -4 |
| OCaml | 22,000 | 343,500 | 99,050 | -3 |
| Scheme | 86,450 | 2,100,000 | 82,650 | -13 |
| Scala | 3,570 | 66,250 | 65,950 | +6 |
| Smalltalk | 9,105 | 187,500 | 56,950 | 0 |
| (S)ML3 | 5,173 | 590,700 | 42,130 | -12 |
| Forth | 6,465 | 146,450 | 25,880 | 0 |
| Clojure | 782 | 62,200 | 23,525 | +3 |
| Caml | 1,889 | 69,600 | 7,825 | 0 |
| Arc | 6,775 | 286,500 | 6,710 | -10 |
| Io | 1,760 | 198,500 | 3,025 | -7 |
1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml
See Part Five
I compiled some programming language popularity statistics in April and mentioned I’d update the results in 6 months, so here they are:
I made a number of Google searches of the forms below and averaged the results:
"implemented in <language>" "written in <language>"
| Language | # Results Apr 09 |
# Results Oct 09 |
Position Delta |
|---|---|---|---|
| C | 1,905,500 | 16,975,000 | 0 |
| C++ | 699,000 | 6,270,000 | +1 |
| Java | 850,000 | 5,118,000 | -1 |
| PHP | 680,000 | 5,083,500 | 0 |
| Lisp Family1 | 176,507 | 3,489,650 | +3 |
| Python | 396,000 | 3,407,000 | -1 |
| Perl | 365,500 | 3,132,500 | -1 |
| C# | 349,700 | 2,125,000 | -1 |
| Scheme | 86,450 | 2,100,000 | +2 |
| FORTRAN | 1,621,000 | N/A | |
| JavaScript | 102,700 | 1,163,000 | -1 |
| ML Family2 | 29,062 | 1,003,800 | +3 |
| (S)ML3 | 5,173 | 590,700 | +12 |
| Common Lisp | 20,600 | 554,500 | +5 |
| Lisp | 61,900 | 486,500 | -2 |
| Prolog | 17,750 | 390,500 | +4 |
| Tcl | 44,800 | 382,000 | -3 |
| OCaml | 22,000 | 343,500 | 0 |
| Arc | 6,775 | 286,500 | +4 |
| Haskell | 22,550 | 280,500 | -4 |
| COBOL | 247,300 | N/A | |
| Ruby | 99,650 | 227,000 | -10 |
| Io | 1,760 | 198,500 | +6 |
| Smalltalk | 9,105 | 187,500 | -1 |
| Erlang | 22,285 | 161,700 | -7 |
| Forth | 6,465 | 146,450 | -1 |
| Lua | 13,065 | 131,800 | -5 |
| Caml | 1,889 | 69,600 | 0 |
| Scala | 3,570 | 66,250 | -2 |
| Clojure | 782 | 62,200 | 0 |
1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml
I first became interested in functional programming when I was exposed to Python, Ruby & JavaScript a number of years ago. Since then I’ve looked into Arc, Clojure, Common Lisp, Haskell, Logo, ML & Scheme. I haven’t yet determined whether I’ll be more productive in any of them than I am with Ruby for developing web applications, but I do find them quite interesting.
After bumping into a number of local programmers who expressed an interest in functional programming, I thought it might be a good time to start a local group that focused on functional programming languages, so I did a couple days ago.
TriFunc.org is a group for programmers who are interested in functional programming languages and live near the Research Triangle area of North Carolina.
If you live in the area and have an interest in functional programming languages, feel free to dive in and start participating in the Google Group discussions. Once we reach a critical mass, I expect we’ll produce a meeting schedule, etc., but that will depend on where the group wants to take this.
Background
The 2008 Programming Language Plan didn’t go as well as I hoped, so I’m regrouping for another go at it. I did make progress learning some Logo and teaching it to my daughters, and I worked through seven chapters of “Programming in Haskell” which was very enjoyable, but I also spent way too much time trying to decide which language(s) to learn without actually learning them.
I have now decided which languages I want to learn this year, so I figured I’d post this blog entry. In some respects, things haven’t changed much from last years plan, but my decisions are much less tentative which is encouraging. I’m glad to switch from information gathering to actually learning a few languages.
Motivation
When I switched from C++ to Java in 1996 and noticed a large increase in productivity; however, it didn’t occur to me to consider whether switching from Java to another language might also give me a similar boost in productivity. The job demand for Java was high during the 10 years I used it, and I was getting paid for my time vs. what I produced, so the oversight wasn’t too costly.
When I switched from Java to Ruby in 2006 I experienced a comparable, if not greater, boost in productivity as I had with the switch from C++ to Java. This time I considered the effect of programming language choice on productivity more carefully and began to wonder about the optimal programming language for me. As a small business owner, my primary consideration is productivity, not popularity or the interchangeableness of programmers.
I don’t think a perfect programming language exists in general, but I think there is an optimal language for me given my particular circumstances, the problems I want to solve, the type of software I want to develop, my way of thinking, my aesthetic tastes, etc.
The Problem
There is a cost to research programming languages to determine the optimal one, and there is a cost to switch programming languages, so the benefit of a new programming language needs to exceed those costs.
Additionally, a significant catch-22 exists in that to truly determine if a language is optimal, one must reach a level of proficiency with the language. There isn’t enough time to learn all of the candidate languages, so some filtering mechanism must be used first to narrow the choices to a reasonably small set. This filtering mechanism is theoretical by nature, but I think the subtle practicalities of a language have a significant effect on productivity.
I have a long term perspective for this task, so I don’t mind investing time researching programming languages, and I also don’t mind if I need to develop/acquire supporting libraries before being ultimately more productive than I am currently with Ruby and Rails.
The Process
I briefly entertained the idea of going about this research in a more systematic, scientific way. That might be a reasonable approach, but due to the following:
- My laziness
- The high degree of subjectivity in programming language choice
- The lack of consensus among programming language researchers
I ended up with a fairly ad-hoc approach involving examining the following areas and trying to absorb enough information to formulate a plan:
Efficiency
Programming language efficiency is becoming increasingly important to me as Moore’s law loses steam, and power & cooling issues become more prominent.
On the one hand, Ruby is near the bottom of the pack with respect to efficiency, but the productivity has been outstanding, and at the volume of transactions I’ve needed to deliver thus far, performance hasn’t been a problem.
On the other hand, if programming languages exist (and I believe they do) that have similar or greater power than Ruby but are compiled instead of interpreted, that would be an advantage.
Benchmarks are notoriously controversial with respect to the degree in which they predict performance, but I don’t think they’re irrelevant. One popular benchmark site is: Programming Language Shootout. I also created some micro-benchmarks of my own to test performance.
One negative result of Ruby being inefficient is the disparity in performance of code written in Ruby vs. library code, or extensions, written in C. This creates counter-intuitive scenarios where a piece of code that appears less efficient is actually more efficient because of the use of built-in code implemented in C. I don’t like thinking in those terms; I would much prefer that user written code in the language is much closer to the efficiency of built-in library code.
Concurrency
With the flattening of CPU MHz and proliferation of CPU cores, I think concurrency may continue to grow in importance. Although I develop mainly server side web applications which can take advantage of multiple cores by virtue of having multiple independent processes, I would prefer to have better concurrency mechanisms in the language/libraries directly.
Joy
I think I first heard joy used, in the context of programming languages, in the Ruby community – probably from Matz himself. I have to agree that programming in Ruby has been more joyful than previous programming languages.
I’m not sure exactly why this is, but I expect it has something to do with the effectiveness and productivity of Ruby, but it may be helped by the incredibly friendly Ruby community.
Fundamental Nature
My preference is for a programming language to minimize arbitrariness and to maximize orthogonality. In other words, I would prefer the language to have a minimal set of core concepts/axioms/operators/etc. that are built upon systematically.
Community
Whether my optimal language is mainstream or not is irrelevant to me, so I don’t need the programming language community to be large (and in fact, being too large is a detriment), but I do think it should be active enough that there are people available to help answer questions, write libraries, maintain/improve compilers, debuggers etc.
To this end, I spent time on various usenet groups, IRC channels and blogs. Occasionally asking questions, but mostly observing the dialog and interaction among the community members.
Education
What educational materials are available?
I tend to prefer learning from books, so searching Amazon to see what books are available, how well they’re received, etc. was helpful. Although I prefer books typically, in the exploratory stages, it’s cheaper to go through online materials, so searching the web for free PDFs was useful. The existence of a few good texts is also an indication of the vitality of community.
Productivity
Ultimately, I’m after the most productive language for me; however, I think productivity is the factor that requires the most proficiency with a language to judge, so it was the most difficult factor to research prior to learning a language. I was limited to anecdotal stories, case studies, personal testimonies, etc.
Licensing
I prefer open source licensing because I think programming languages with proprietary licensing are more likely to die.
The Final Candidates
I began with Ruby as the point of reference irrespective of Rails and other libraries. Even though Rails is an important factor in my current productivity, since I’m taking a long term view, I didn’t want to exclude a fantastic language simply because it’s lacking something that Ruby was also lacking N years ago.
Due to the cost of researching, learning and switching to a new programming language, I only considered languages that had the potential of offering a significant improvement in one or more areas without losing too much in other areas.
The candidates I eventually selected fell into two groups. Both groups have strong functional capabilities, good efficiency, and long, successful track records. I believe that even the best programming language designers make serious mistakes which can only be identified with the hindsight of years of use. It’s possible that someone is about to release a brand new language that has the potential to be the most productive for me personally; however, I’m not willing to take the risk to learn it.
The Lisp Family
Lisp has a very long track record of success and adaptability. In fact, if I was forced to program in a single programming language from now on, I would probably choose a Lisp since I think it would have the greatest likelihood of being able to adapt.
The Lisp family has dynamic typing which I’ve grown to love with Ruby, but most also allow type declarations for efficiency. Lisp also appears to have a more fundamental nature than most languages. I’ve heard it said that Lisp was more discovered than invented.
I first became interested in Lisp when I learned that Ruby was influenced by it. My interest was reinforced by some of Paul Graham’s essays & books. I like the exploratory and dynamic nature of developing Lisp – this is mostly from reading comments of others, but I’ve tasted a small part of this when evaluating Lisp code in Emacs and having it be available immediately.
Logo is first on the list simply because it’s both a Lisp and a great language for teaching children how to program, so I can kill two birds with one stone.
Scheme is next because it’s a natural sequel to Logo and I still want to work through “The Structure and Interpretation of Computer Programs”. The fact that Paul Graham chose mzcheme to create his Arc language is a good “letter of recommendation”. Scheme feels the most fundamental, simple & clean thus far.
Common Lisp is after Scheme due to its eminent practicality and completeness. It has plenty of warts, but also plenty of power and functionality. It’s been called a great, big, ball of mud – but in a good way 
Lastly, is Clojure. It’s last in the Lisp family because I want to learn Scheme & Common Lisp first so I’ll be better equipped to judge Clojure. This will also allow more time for Clojure to mature. Clojure has a focus on functional programming & concurrency that is symbiotic with the Java platform. The latter provides a quick start and access to Java libraries, the JVM infrastructure, etc., but my preference would be to not be dependent on the JVM in the long run. I’ll withhold judgment until I’ve learned it and used it for a while.
The ML Family
I’m least familiar with the ML family and with functional programming in general. I’ve spent most of my career studying and using imperative, object-oriented programming languages.
I think the ML family is worth studying because:
- Functional programming may be beneficial with respect to concurrency.
- Prog. Lang. researchers seem to be enamored with ML family.
- There are enough anecdotal testimonies of productivity to warrant further study.
Standard ML is an important functional language that differs from Haskell in that it’s impure, i.e. it allows side effects, and it’s strict, i.e. not lazy. It shares Hindly-Milner static typing with Haskell. The community seems rather tiny, and I expect that if I go with a static typed language it will likely be Haskell, but I wanted to learn Standard ML first because of its historical importance and to be able to compare an impure/strict language with a pure/nonstrict language.
Haskell is probably the most different programming language from what I’m used to. This, in and of itself, has some advantages with respect to gaining new perspectives and ideas. In some ways, it’s a language that has taken things to extremes with respect to functional purity and laziness.
The community is very active. It offers a great compiler (GHC), software transactional memory, a decent base of libraries, etc.
At this early stage, I’m skeptical of static typing, functional purity and laziness, so becoming proficient in Haskell is a great opportunity to be able to determine how I feel about those.
The Plan
Rather than go through the candidates sequentially, I’m going to try and make progress on two tracks concurrently to allow me to compare concepts from both families:
| ML Family | Lisp Family |
|---|---|
| Standard ML | Logo |
| Haskell | Scheme |
| Common Lisp | |
| Clojure |
I’m curious to find out how I feel about these languages after I’ve achieved some skill with them, but I think becoming proficient in the Lisp and ML families will be time well spent. At minimum, I’ll be better equipped to compare other languages.
See Part Five
Despite the numerous ways in existence to quantify programming language popularity, I thought I’d throw yet another one into the mix. I made a number of Google searches of the forms below and averaged the results:
"implemented in <language>" "written in <language>"
I’m very curious to see how these stats change over time, so I’ve added a calendar item to recompute them in six months. Leave a comment if you’d like to add a programming language to the list, and I’ll update this article and it will be included in the recomputation six months from now.
| Language | # Results |
|---|---|
| C | 1,905,500 |
| Java | 850,000 |
| C++ | 699,000 |
| PHP | 680,000 |
| Python | 396,000 |
| Perl | 365,500 |
| C# | 349,700 |
| Lisp Family1 | 176,507 |
| JavaScript | 102,700 |
| Ruby | 99,650 |
| Scheme | 86,450 |
| Lisp | 61,900 |
| Tcl | 44,800 |
| ML Family2 | 29,062 |
| Haskell | 22,550 |
| Erlang | 22,285 |
| OCaml | 22,000 |
| Common Lisp | 20,600 |
| Prolog | 17,750 |
| Lua | 13,065 |
| Smalltalk | 9,105 |
| Arc | 6,775 |
| Forth | 6,465 |
| (S)ML3 | 5,173 |
| Scala | 3,570 |
| Caml | 1,889 |
| Io | 1,760 |
| Clojure | 782 |
1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml
Update 4/23/09 added C#, Tcl per comment requests.
Recent Comments