Lojic Technologies

Archive for the ‘programming’ Category

Programming Language Popularity – Part Four

leave a comment »

See Part Five

I compiled some programming language popularity statistics in April 2009, October 2009 and October 2010 . Here’s an update for September 2011:

I made a number of Google searches of the forms below and summed the results (previous posts averaged the results):

"implemented in <language>"
  "written in <language>"

Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within the same search since the number of results Google returns can vary greatly over time.

Language Total Prev. Position Position Delta
C 10,360,000 2 1
PHP 10,351,000 1 -1
C++ 6,495,000 3 0
Python 5,759,000 5 1
C# 5,335,000 4 -1
 
Java 4,890,000 8 2
Perl 3,702,000 6 -1
JavaScript 3,077,000 7 -1
Ruby 1,654,000 9 0
Lisp Family1 1,022,870 11 1
 
FORTRAN 975,600 10 -1
Tcl 594,500 12 0
Lisp 486,000 14 1
Haskell 450,500 16 2
Erlang 419,700 13 -2
 
Lua 367,100 18 2
ML Family2 348,400 17 0
COBOL 308,270 15 -3
Common Lisp 254,900 19 0
OCaml 240,300 21 1
 
Prolog 224,000 20 -1
Scala 203,400 23 1
Scheme 184,700 22 -1
Smalltalk 129,700 24 0
Clojure 84,600 27 2
 
(S)ML3 83,630 25 -1
Forth 69,980 26 -1
Caml 24,470 28 0
Io 17,700 30 1
Arc 12,670 29 -1

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Written by Brian Adkins

September 22, 2011 at 10:37 am

Find Longest Substring Palindrome in Haskell

with 7 comments

I stumbled upon a programming challenge a company was using for recruitment purposes and thought I’d create a Haskell solution as a learning exercise. The first problem was to find the longest palindrome embedded in a text string.

The following Haskell solution seems very readable to me, but it’s a naive solution that’s inefficient. It computes an answer on my 2.6 year old Macbook Pro in under 4 seconds, but a 2x increase in text requires a 7x increase in CPU time.

I believe there are algorithms to find the longest embedded palindrome in linear time, so I may post a refinement later.

-- Find the longest palindrome in a text string.

module Main where
import Char

text = "I'll just type in some example text here and embed a little 
palindrome - A man, a plan, a canal, Panama! - I expect that will be 
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris 
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id 
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere 
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
 odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante 
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium 
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis 
consequat condimentum, sem magna ornare ligula, id blandit odio nisl 
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt 
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros 
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc 
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc 
vitae tempus purus."

clean = map toLower . filter isAlpha

palindrome str = str == reverse str

substrings []     = []
substrings (x:xs) = substrings' (x:xs) ++ substrings xs where
  substrings' []     = []
  substrings' (y:ys) = [y] : [ (y:s) | s <- substrings' ys ]

longest []     = []
longest (x:xs) = if length x > length max then x else max
  where max = longest xs

longest_palindrome xs =
  longest (filter palindrome (substrings (clean text)))

main = print (longest_palindrome text)

As a comparison, I translated the program into Ruby. I program predominantly in Ruby these days, and I like it, but the Ruby version is 25 times slower (98 sec. vs. 4 sec.), and it’s 2.4 times more lines of code (31 vs. 13 – excluding the text).

A gain in runtime efficiency, expressive power and multi-core capability is very attractive!

I’m using Ruby 1.9.2 and GHC 6.12.3 on Mac OS X 10.5.8 on a 2.4 GHz Core 2 Duo w/ 4 GB RAM.

ruby 1.9.2p0 (2010-08-18 revision 29036) [i386-darwin9.8.0]
Glasgow Haskell Compiler, Version 6.12.3, for Haskell 98, stage 2 booted by GHC version 6.12.2

TEXT = <<END
I'll just type in some example text here and embed a little
palindrome - A man, a plan, a canal, Panama! - I expect that will be
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
 odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis
consequat condimentum, sem magna ornare ligula, id blandit odio nisl
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc
vitae tempus purus.
END

def clean str
  str.gsub(/[^A-Za-z]/,'').downcase
end

def palindrome? str
  str == str.reverse
end

def subs str
  return [] if str.empty?
  y = str[0,1]
  subs(str[1..-1]).inject([y]) do |result, s|
    result << y + s
    result
  end
end

def substrings str
  return [] if str.empty?
  subs(str) + substrings(str[1..-1])
end

def longest strs
  strs.inject("") do |max, str|
    max = str if str.length > max.length
    max
  end
end

def longest_palindrome str
  longest(substrings(clean(str)).inject([]) {|result, str|
            result << str if palindrome?(str)
            result
          })
end

puts longest_palindrome(TEXT)

Update 3/13/11 19:30
Rick’s comment (see his blog post linked to in his comment) regarding the importance of algorithm choice is certainly a valid one – a better algorithm in a slower language may win over an inferior algorithm in a faster language (for large enough datasets). However, what I’m becoming interested in is the fact that one may gain both productivity/power and runtime speed by making wise programming language choices.

In the case of an interpreted language such as Ruby, it’s helpful to “stay in C code” as much as possible. In other words, to favor built-in library routines that have been implemented in C over hand written Ruby code. As much as I like Ruby, this is one of the things that bothers me – the fact that the Ruby code written by the programmer is vastly inferior in performance to the built-in library routines.

I wasn’t planning on refining the Haskell version this soon, but after seeing Rick’s blog post response, I couldn’t resist 🙂

I should first provide some background info for context. When I wrote the original Haskell version above, I was in the middle of an online programming challenge, and my goal was simply to compute the answer in a reasonable amount of time to move on to the next challenge, so the brief, easily understandable, Haskell version worked great. Hence, my disclaimers in the post regarding the naivity and inefficiency of the solution. The Ruby code in this post was an afterthought simply to see a comparison between identical algorithms. Given that apple & apple comparison, I’m impressed with the brevity and speed of the Haskell version.

Since I’m still very much a Haskell newbie, and short on time, I found an incredible solution by Johan Jeuring. It’s a little bit longer than the Ruby version, but much, much faster. It’s so fast, I had to increase the input quite a bit to get a reasonable comparison – I replicated the original text 23 times and reversed one of the replications to make a fairly long palindrome.

Rick’s Ruby version took 169.4 seconds, Johan’s Haskell version took 0.032 seconds. In other words, Ruby takes over 5,000 times as long to compute the result. Clearly this is an apples and oranges comparison, but I fully expect that a Ruby version using an identical algorithm will take 100 times as long (or longer) to run and would be less concise. Giving up runtime performance to gain programmer power is one thing, but giving up runtime performance and power is a tough pill to swallow.

Here is a slightly modified version of Johan Jeuring’s code. He was also kind enough to provide his code here:

-- Reorganized from Johan Jeuring's solution:
module Main where
import Data.List (maximumBy,intersperse)
import Data.Char
import Data.Array

text = "I'll just type in some example text here and embed a little 
palindrome - A man, a plan, a canal, Panama! - I expect that will be 
the longest palindrome found in this text.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Integer volutpat lorem imperdiet ante bibendum ullamcorper. Mauris 
tempor hendrerit justo at elementum. Vivamus elit magna, accumsan id 
condimentum a, luctus a ipsum. Donec fermentum, lectus at posuere 
ullamcorper, mauris lectus tincidunt nulla, ut placerat justo odio sed
 odio. Nulla blandit lorem sit amet odio varius nec vestibulum ante 
ornare. Aliquam feugiat, velit a rhoncus rutrum, turpis metus pretium 
dolor, et mattis leo turpis non est. Sed aliquet, sapien quis 
consequat condimentum, sem magna ornare ligula, id blandit odio nisl 
vitae erat. Nam vulputate tincidunt quam, non lacinia risus tincidunt 
lacinia. Aenean fermentum tristique porttitor. Nam id dolor a eros 
accumsan imperdiet. Aliquam quis nibh et dui ultricies cursus. Nunc 
et ante non sapien vehicula rutrum. Duis posuere dictum blandit. Nunc 
vitae tempus purus."

clean = map toLower . filter isAlpha

longestPalindrome input =
  let inputArray      =  listArrayl0 input
      (maxLength,pos) =  maximumBy
                            ((l,_) (l',_) -> compare l l')
                            (zip (palindromesAroundCentres inputArray) [0..])
  in showPalindrome inputArray (maxLength,pos)

longestPalindromes m input =
  let inputArray =  listArrayl0 input
  in concat $ intersperse "n"
            $ map (showPalindrome inputArray)
            $ filter ((m String
lengthLongestPalindrome = show . maximum . palindromesAroundCentres . listArrayl0

lengthLongestPalindromes :: String -> String
lengthLongestPalindromes = show . palindromesAroundCentres . listArrayl0

palindromesAroundCentres a =
  let (afirst,_) = bounds a
  in reverse $ extendTail a afirst 0 []

extendTail a n currentTail centres
  | n > alast = finalCentres currentTail centres
                   (currentTail:centres)
  | n-currentTail == afirst =
      extendCentres a n (currentTail:centres)
                    centres currentTail
  | a!n == a!(n-currentTail-1) =
      extendTail a (n+1) (currentTail+2) centres
  | otherwise =
      extendCentres a n (currentTail:centres)
                    centres currentTail
  where  (afirst,alast)  =  bounds a

extendCentres a n centres tcentres centreDistance
  | centreDistance == 0 =
      extendTail a (n+1) 1 centres
  | centreDistance-1 == head tcentres  =
      extendTail a n (head tcentres) centres
  | otherwise =
      extendCentres a n (min (head tcentres)
                    (centreDistance-1):centres)
                    (tail tcentres) (centreDistance-1)

finalCentres 0     _        centres  =  centres
finalCentres (n+1) tcentres centres  =
  finalCentres n
               (tail tcentres)
               (min (head tcentres) n:centres)
finalCentres _     _        _        =  error "finalCentres: input < 0"

showPalindrome a (len,pos) =
  let startpos = pos `div` 2 - len `div` 2
      endpos   = if odd len
                 then pos `div` 2 + len `div` 2
                 else pos `div` 2 + len `div` 2 - 1
  in show [a!n|n <- [startpos .. endpos]]

listArrayl0 string  = listArray (0,length string - 1) string

sampleText s = concat (replicate 8 s ++ [ "x" ] ++ [ reverse s ] ++ replicate 14 s)

main = print (longestPalindrome (clean (sampleText text)))

Here’s the relevant change to Rick’s Ruby version:


def sample_text str
  str * 8 + 'x' + str.reverse + str * 14
end

puts clean(sample_text(TEXT)).longest_palindrome

Written by Brian Adkins

March 13, 2011 at 1:37 am

Posted in programming

Tagged with ,

Cracker Barrel Peg Board Puzzle in Haskell

with one comment

I first wrote a program to solve the Cracker Barrel peg board puzzle (15 holes arranged in a triangle with 14 golf tees) many years ago as youth using the BASIC language. I wish I still had the source to that, because I’m pretty sure this Haskell version would kick its butt 🙂

I’m still trying to get my head around Haskell, so I expect there are many possible improvements to this program, but even so, I’m pleased with how Haskell allows me to express logic.

-- Solve the Cracker Barrel Peg Board Puzzle

module Main where

type Pos = (Int, Int)
type Move = (Pos, Pos)
type Board = [ Pos ]

isOccupied b p = elem p b
isEmpty b p    = not (isOccupied b p)
isPos (r,c)    = elem r [0..4] && elem c [0..r]

-- Possible moves for one position
positionMoves b p = [ (p, dst) | (neighbor, dst)  isPos p1 && isPos p2)
                   [ ((r + or `div` 2, c + oc `div` 2),(r + or, c + oc)) |
                     (or, oc) <- [ (-2,0), (0,2), (2,2), (2,0), (0,-2), (-2,-2) ] ]

-- Possible moves for all positions on the board
possibleMoves b = concat [ positionMoves b pos | pos  (pos /= src) && (pos /= neighbor)

-- Make moves until the goal position is met
play b p moves =
  if null nextMoves then
    if length b == 1 && head b == p then reverse moves else []
  else
    tryMoves nextMoves
  where
    nextMoves       = possibleMoves b
    tryMoves []     = []
    tryMoves (m:ms) =
      let result = play (move b m) p (m:moves)
      in if null result then tryMoves ms else result

-- Compute the initial empty position to know the goal, then solve the puzzle
solve b = let emptyPos = head [ (r,c) | r <- [0..4], c <- [0..r], isEmpty b (r,c) ]
          in play b emptyPos []

-- A sample board with the topmost hole empty
board :: Board
board = [ (1,0), (1,1),
          (2,0), (2,1), (2,2),
          (3,0), (3,1), (3,2), (3,3),
          (4,0), (4,1), (4,2), (4,3), (4,4) ]

main = print (solve board)

Written by Brian Adkins

March 10, 2011 at 2:24 pm

Posted in programming

Tagged with

Programming Language Popularity – Part Three

with one comment

See Part Five

I compiled some programming language popularity statistics in April 2009 and October 2009 . Here’s an update for October 2010:

I made a number of Google searches of the forms below and averaged the results:

"implemented in <language>"
  "written in <language>"

Naturally this is of very limited utility, and the numbers are only useful when comparing relatively within one column since the number of results Google returns can vary greatly over time.

Language Apr 2009 Oct 2009 Oct 2010 Position Delta
PHP 680,000 5,083,500 14,096,000 +3
C 1,905,500 16,975,000 9,675,000 -1
C++ 699,000 6,270,000 6,510,000 -1
C# 349,700 2,125,000 5,132,000 +4
Python 396,000 3,407,000 5,114,500 +1
Perl 365,500 3,132,500 4,675,000 +1
JavaScript 102,700 1,163,000 2,120,000 +4
Java 850,000 5,118,000 1,495,500 -5
Ruby 99,650 227,000 1,426,000 +13
FORTRAN 1,621,000 770,850 0
Lisp Family1 176,507 3,489,650 399,685 -6
Tcl 44,800 382,000 313,400 +5
Erlang 22,285 161,700 188,800 +12
Lisp 61,900 486,500 174,050 +1
COBOL 247,300 166,435 +6
Haskell 22,550 280,500 157,150 +4
ML Family2 29,062 1,003,800 149,005 -5
Lua 13,065 131,800 128,150 +9
Common Lisp 20,600 554,500 112,750 -5
Prolog 17,750 390,500 100,000 -4
OCaml 22,000 343,500 99,050 -3
Scheme 86,450 2,100,000 82,650 -13
Scala 3,570 66,250 65,950 +6
Smalltalk 9,105 187,500 56,950 0
(S)ML3 5,173 590,700 42,130 -12
Forth 6,465 146,450 25,880 0
Clojure 782 62,200 23,525 +3
Caml 1,889 69,600 7,825 0
Arc 6,775 286,500 6,710 -10
Io 1,760 198,500 3,025 -7

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Written by Brian Adkins

October 8, 2010 at 3:30 pm

Git commit message style

leave a comment »

I just learned something about how git handles multi-line commit messages. If a commit message has a blank second line, git will treat the first line as the “subject” and the 3rd and following lines as the “body”. This will allow some nice things to be done with the git log command.

For example, the following command:

git log --pretty=format:"%ai %an: %s%n%n%b%n"

Will produce output such as this:

2010-08-30 13:10:51 -0400 Brian Adkins: This is my second commit for this example.

And this is the body for the second commit.
Second line of the body.

2010-08-30 13:07:23 -0400 Brian Adkins: This text will be treated as the subject.

This text will be treated as the body. The subject should be
a short summary of the commit since you may want to show just
the subject on some git log invocations.

If you just want to see the “subject”, use:

git log --pretty=format:"%ai %an: %s%n"

Which will display as follows:

2010-08-30 13:10:51 -0400 Brian Adkins: This is my second commit for this example.

2010-08-30 13:07:23 -0400 Brian Adkins: This text will be treated as the subject.

See “git log –help” for various formatting specifications.

Thanks to Nathaniel Talbott for the chat message that prompted me to look into this.

Written by Brian Adkins

August 30, 2010 at 12:15 pm

Posted in programming

Tagged with

Switching from CarbonEmacs to Emacs.app

with one comment

Thanks to a tip from David Joyner, I discovered that the current version of Emacs will easily build an Emacs.app application for Mac OSX i.e. I no longer need to install the CarbonEmacs application. I’m thankful to Seiji Zenitani for creating CarbonEmacs, but I’m glad to be able to build the app directly from the latest Emacs source code.

Here’s what I did:

Get the source code and build it

git clone git://git.savannah.gnu.org/emacs.git
cd emacs
./configure --with-ns
make install

Copy Emacs.app to Applications

Drag nextstep/Emacs.app to Applications

Modify Emacs config files

I had several configuration items I needed to adjust.

carbon-emacs-package-add-to-path no longer exists

I used the above command to set the path to allow running programs via shell-command as follows:

(carbon-emacs-package-add-to-path "/Users/badkins/sync/bin")
(defun my-fun ()
  (interactive)
  (shell-command "my_ruby_script.rb"))

I found a comment by Alex Payne on Ola Bini’s blog that provided an equivalent for Emacs.app:

(setq path "/Users/badkins/sync/bin")
(setenv "PATH" path)

Use command for meta instead of alt/option

The newly built Emacs.app used the alt/option key for meta instead of the command key that I was used to with CarbonEmacs. The following (found on Hacker News) took care of that:

(setq ns-command-modifier 'meta)

Prior to the above, I had tried the following. It worked, but was more verbose:

(setq mac-option-key-is-meta nil)
(setq mac-command-key-is-meta t)
(setq mac-command-modifier 'meta)
(setq mac-option-modifier nil)

Allow use of command-h to minimize Emacs

After the above fix for the command key, I could no longer minimize Emacs via command-h because Emacs mapped M-h to a command. The following key binding allowed using M-h to minimize again:

(global-set-key (kbd "M-h") 'ns-do-hide-emacs)

Customize exec-path variable

I was having difficulty using the built-in vc git support. It turns out that Emacs wasn’t able to find the git command because it wasn’t in the exec-path. I customized the exec-path variable to include /opt/local/bin where git resides as follows (it’s within the custom-set-variables invocation):

 '(exec-path (quote ("/usr/bin" "/bin" "/usr/sbin" "/sbin" "/Applications/Emacs.app/Contents/MacOS/bin" "/opt/local/bin")))

Emacs is Awesome

That was all it took to get the latest Emacs up and running equivalently to CarbonEmacs for me. I’m more pleased with Emacs today than when I started using it in earnest in 2008. I’m continually learning new things to be more productive. There are just so many things that it does very well, and it’s so extensible and introspective.

Written by Brian Adkins

March 17, 2010 at 12:42 am

Posted in programming

Tagged with ,

Programming Language Popularity – Part Two

with one comment

See Part Five

I compiled some programming language popularity statistics in April and mentioned I’d update the results in 6 months, so here they are:

I made a number of Google searches of the forms below and averaged the results:

"implemented in <language>"
"written in <language>"

Language # Results
Apr 09
# Results
Oct 09
Position
Delta
C 1,905,500 16,975,000 0
C++ 699,000 6,270,000 +1
Java 850,000 5,118,000 -1
PHP 680,000 5,083,500 0
Lisp Family1 176,507 3,489,650 +3
Python 396,000 3,407,000 -1
Perl 365,500 3,132,500 -1
C# 349,700 2,125,000 -1
Scheme 86,450 2,100,000 +2
FORTRAN 1,621,000 N/A
JavaScript 102,700 1,163,000 -1
ML Family2 29,062 1,003,800 +3
(S)ML3 5,173 590,700 +12
Common Lisp 20,600 554,500 +5
Lisp 61,900 486,500 -2
Prolog 17,750 390,500 +4
Tcl 44,800 382,000 -3
OCaml 22,000 343,500 0
Arc 6,775 286,500 +4
Haskell 22,550 280,500 -4
COBOL 247,300 N/A
Ruby 99,650 227,000 -10
Io 1,760 198,500 +6
Smalltalk 9,105 187,500 -1
Erlang 22,285 161,700 -7
Forth 6,465 146,450 -1
Lua 13,065 131,800 -5
Caml 1,889 69,600 0
Scala 3,570 66,250 -2
Clojure 782 62,200 0

1 combines Lisp, Scheme, Common Lisp, Arc & Clojure
2 combines OCaml, (S)ML, Caml
3 summed separate searches for sml and ml

Written by Brian Adkins

October 24, 2009 at 10:23 am