Adopting functional programming (or “Just why did people start using Python anyway?”)
2008-10-08 / 18:30 / dave
The end of CUFP 2008 was an open discussion that focused on–surprise surprise!–how to take functional languages mainstream. The future looks pretty bright to me: as Don Syme pointed out Microsoft is serious about F# and–although it got no love at ICFP–Clojure provides a functional escape for the JVM bound.
Of course availability is not enough: people also need a reason to switch. Many people brought up the “killer app”, the functional programming Rails. One functional language already has one: Erlang. Instead of db-backed webapps, Erlang makes it “easy” to create massive backends (hee hee). Francesco Cesarini, Nick Gerakines & Mark Zweifel (slides [PDF]) and Bob Ippolito (slides) all had commercial examples. Of course process oriented concurrency, no shared state and the OTP aren’t exactly an app… maybe “killer underlying philosophy and toolkit”?
Most other commercial users were either financial quants or small startups looking for their Paul Graham advantage (see the list of abstracts). All emphasized the enhanced productivity of functional programming. The quants were an interesting case. They are small groups within huge–and conservative–companies. Functional programming snuck-in around the edges. And once you have working code…
Hence the subtitular question: why did people start using Python? I don’t know of any killer Python apps. It’s just easy to code yet feels “solid”: it’s got unit testing frameworks, test coverage tools, documentation extractors, a REPL, a package manager, etc. Knowing that smart developers are writing tools for Python is pretty reassuring. (So is having Guido work for Google.)
Haskell feels solid. GHC is easy to install and comes with an interpreter (though not quite a REPL, which still bothers me). Hackage and cabal-install handle package distribution. And there are enough developer tools: QuickCheck, HUnit, HPC, Haddock, etc. It even has an Emacs mode.
All that is to say that from my “I work for a small group within a small company and can get away with a lot” point of view, Haskell is ready. All I have to do now is decide if it’s easy… and then produce some working code.
Why did people start people using Python?
Python makes your life easier. As for why did people use it when it had no standard library, I don’t know, but I would say that the general ease of use of the Python standard library is a big boost to Python’s popularity. Right now, there’s a lot of interesting researchy-yet-systemsy stuff being written for Haskell, but it’s not quite so compelling as Python was. Python took familiar imperative and OO concepts and cut most of the crap out of the way to just let you Get Things Done, which is part of its heritage as a scripting language. Haskell also makes your code short and pretty, but it’s much harder to Get Things Done. This is a cheap shot, but most tasks that people need to do are IO related, which is less than straightforward in Haskell.
But I agree, it’s important to think about what reasons compel people to switch languages.
My bet is still there is no FP mainstream comming up and Java will stay with us at least for the next 10 years if not more (Java seems to be the first big ecosphere which innovates through open source and evolves through a 2-yearly process)
http://stephan.reposita.org/archives/2008/01/21/no-future-for-functional-programming-in-2008-scala-f-and-nu/
If Java is not an option and I should still make a bet: It’s Scala.It does offer FP, has a rather good OO syntax and is extensible (see Scala OTP, actors, STM an other ideas implemented as a library).
Peace
-stephan
For me Python doesn’t feel solid because of explicit self in methods. It’s leaky abstraction, incomplete syntax sugar, or just an ugly hack.
Compared to Python – Ruby is much easier to learn and use, and more natural IMHO. Plus it doesn’t require indentation, and functions can be passed as objects (comes close to Lisp’s code = data philosophy)
It comes with a rich standard library, a REPL, and has a nice curious and enthusiastic community around it. Tools for profiling, and automated testing freely available. The success of the RAILS framework owes a lot to Ruby’s elegance and meta programming capabilities. Ruby is so flexible that a ruby interpreter in ruby itself has been written! (Rubinius).
The only downside is the speed, but MRI (Matz ruby interpretter) has many faster alternatives one of them being JRuby (Java implementation of the MRI), I’m excited to see what’s in store for Ruby in the near future (one of the thing I hope is a faster C implementation of MRI)
“Ruby is so flexible that a ruby interpreter in ruby itself has been written! (Rubinius).”
This argument is pretty weak. C compilers are written in C and there is also a Python interpreter for Python (PyPy). Nothing special here.
As why are programmers using Python? Probably for the relatively clean syntax, easy to learn, lot of libraries, was (is?) more mature than Ruby at the time, installed by default on most Linux distribution and… *you don’t have to learn a new programming paradigm (functional programming)*
I don’t understand why people have problems with the indentation. You are (hopefully) indenting your code anyway, so why do you need curly-braces?
I agree it can be a problem if you mix tab’s and spaces, but there’s a simple solution for that – only use spaces ;-)
In my experience I can focus more on the program when I don’t have to worry about properly closing curly-braces. If you have a if-sentence in a while loop in a method in a class, it is very easy to f*ck things up.
Python is a pile of stinking crap, made of cut and paste features. Use any other language but python and live a meaningful life.
Kotonymous: PyPy is NOT a python interpreter written in python. It is a framework for JIT compilation and optimization of dynamic languages written mostly in a statically compilable subset of python. Comparing PyPy to a simple interpreter is roughly equivalent of comparing Windows Notepad to a high-end IDE as far as the scope and size of the project is considered.
(On the other hand, the comparison still holds when talking about bug counts and production times. If PyPy was an interpreter, it’d be done by now. :D)
Joel: The big difference between Python and Ruby is that one of the major design goals, some would say THE major design goal of python is readability of source. Hacking Ruby by myself is fun, but when I have to start writing a large software project with >10 coders and plenty of people leaving and coming, Ruby just isn’t an option. Python is.
kL: explicit self seems like a kludge until you realize that in python, methods are just normal functions that you happen to call trough a class. This has massive advantages, especially when doing functional programming.
class A(object):
a1 = A; a2 = A
a1.dosomething(a2) is EXACTLY the same as A.dosomething(a1,a2).
Why this matters? You can pass A.dosomething to map or reduce.
Stephan: I suggest trying clojure. It seems to be creeping from behind because it has a concurrency model that a) actually works, b) is usable in most real world situations.
@Joel “functions can be passed as objects” – well, in Python functions are objects, and methods are just functions.
@Tuna-Fish: I had a small post about Scala vs. Clojure on my blog recently:
http://stephan.reposita.org/archives/2008/10/02/scala-vs-clojure/
I think concurrency is not a real problem to solve in nowadays applications. The request-per-thread model (with worker threads working the queues) works fine for most people, and scales very well – even for synchronous IO, see e.g. Talkinator
http://mailinator.blogspot.com/2008/08/benchmarking-talkinator.html
“In my benchmarks now, on my quad-core desktop the talkinator server can push about 39000 messages per second. Keep in mind this is processed messages as in decoded, packaged, and queued for particular recipients.” (From the mailinator guy who runs the whole mailinator application on one server)
This might change, but I doubt it. The “FP is better for multi-cores” doesn’t hold with some scrutinizing.
Peace
-stephan
@Tuna-Fish: One more note: FP is fine, but not for concurrency reasons.
(and people confuse side-effect free and stateless code with FP, both can and should be used in OO languages and are not FP specific)
Peace
-stephan
I’ve added a comment twice which didn’t appear, I’ve added it here
http://stephan.reposita.org/archives/2008/10/02/scala-vs-clojure/#comment-177225
Peace
-stephan
@Stephan: comment got eaten by Akismet, apologies. I’ve restored it: comment #10.
@Dave: Ah, no problem, I guress Akismet eats comments on my blog too, which it should do :-) Thanks -stephan
A couple of corrections
First, the Python philosophy is about as far from that of functional languages as you can get while still having dynamic typing and automatic memory management. Guido goes out of his way to demonstrate that Python is a *dynamic* *multi-paradigm* language, not necessarily a functional one (there is a very good talk of his kicking around on Google video somewhere). In fact Python has relatively prosaic higher-order-function support (that was on the block for 3.0 no less…read his blog for the details). Python is foremost a practical language that does take ideology very seriously. I think it is safe to say that the Python community is generally skeptical of the purported benefits of functional programming—look at the talks for PyCon 2008: not many are on functional programming.
Second, characterizing Quants as working at huge and conservative companies is totally misleading. The words “huge” and “conservative” suggest a degree of corporate befuddlement that simply does not exist at banks and financial boutiques. Quants have been using APL for quite a while, and I’ve heard of a derivatives pricing package that was written in Perl and a default risk algorithm coded in Fortran. These languages were not chosen because they had the right buzzwords. They were chosen because they could get the job done. When I hear “huge” and “conservative” it makes me think, “oh, you want to use Java. That way you can just hire legions of unqualified programmers and not worry too much about getting mired in the pits of obfuscation.”
@David:
I agree that Python is not functional and didn’t mean to indicate that it was.
As for quants, I think in general their companies are large (e.g. Credit Suisse) and conservative. Which is to say that the majority of computer people at those institutions are maintaining the web and keeping the plumbing working, and they’re not doing that in OCaml. My point was that inside this large company there is a small group that is using “the best tool for the job.” Ta-da! FP is inside a large company.
Of course there’s no guarantee FP (or whatever other non-mainstream technology) will spread to the rest of the company, but the chance is higher than if it’s not in the door at all.
The functions-are-objects argument is a red herring IMHO. This doesn’t mean you have to have explicit self in method declaration: if
a.b()can be syntactic sugar that implies an argument, thendefinclasscould do the same.I just hate when
a.b(x)fails with error that only 1 argument is expected. This too could be handled withargs-1 if call.looks_like_method() else argsthere somewhere.Ha,huge and conservative (financial) companies : they were. Not any more methinks.
no one’s mentioned Django yet? now granted, I’ve not actually USED Rails, but I’ve done a fair bit of reading and poked at Django, and I cant see why I’d want to use Rails over Django.
@Joel: You obviously know nothing of Python. For example
@Joel: “[in ruby] functions can be passed as objects (comes close to Lisp’s code = data philosophy)”
Consider the following Python statements:
def foo(): print 5
class bar:
@staticmethod
def foo2(): print 5
a = foo
a()
b = bar
b()
c=bar.foo2
c()
d=bar().foo2
d()
e = lambda :d()
e()
First: which of those are as straightforward as in Ruby?
Second: what can you do in Ruby but not Python that you were describing as “passing functions as objects”?
I’ll answer the first question myself. Several of those are syntactically more complex in Ruby which belies your claim that: ” Compared to Python – Ruby is much easier to learn and use,” You can disprove me by rewriting the code in Ruby.
@Joel: “Plus it doesn’t require indentation,”
Every written style guide I’ve ever seen requires indentation. All quality code I’ve ever seen requires indentation. Every quality workplace I’ve seen requires indentation. So I don’t really see it as a burden.
@Joel: “It comes with a rich standard library, a REPL, and has a nice curious and enthusiastic community around it. Tools for profiling, and automated testing freely available.”
Which language are you talking about here?
@Joel: “The success of the RAILS framework owes a lot to Ruby’s elegance and meta programming capabilities.”
And what do the successes of the Django and App Engine frameworks owe to?
@Joel: “Ruby is so flexible that a ruby interpreter in ruby itself has been written! (Rubinius).”
And a compiler for C is written in C. And for Smalltalk in Smalltalk. And for Python in Python. And for C++ in C++. What does that prove really?
@Joel: “The only downside is the speed,”
No, not the only downside. Not by a nautical mile. I have no interest in language bashing so I won’t enumerate all of the other problems but they are legion. Performance is not even the only crippling problem with the MRI implementation.
[...] said I want to learn Haskell but I’ve also talked about the pains of learning new programming environments (and given an [...]