Articles / Critique of Where Perl 6 is…

Critique of Where Perl 6 is Heading

The purpose of this essay is to explain why I believe Perl 6, the way it currently seems to progress, is the wrong thing at the wrong time, and why I predict (with all the expected caveats of predicting something) that it won't be successful. I will also suggest a better alternative for the future of Perl which makes more sense at this point.

Overview of the Current Design of Perl 6

The design of Perl 6 is progressively laid out in the Perl 6 Apocalypses which are written by Larry Wall (the original creator and designer of Perl versions 1 through 5), and exemplified in the Exegeses by Damian Conway and others. There's a more progressive design which has been shared among the members of the perl6-language Mailing List, but hasn't been formulated into a specification yet, and/or has not finally been decided upon by Larry.

It is evident from the Apocalypses and Exegeses that Perl 6 will break compatibility with Perl 5 in many ways (usually deliberately). Furthermore, it incorporates a lot of features and paradigms from different languages that are not common in most languages in common use today.

Why Perl 5 is Good

Perl 5 has seen incredible popularity. There are now millions of lines of code written in Perl by thousands of programmers. (Possibly even milliards of lines of code, written by hundreds of thousands of programmers.) The so-called Perl 5 Man Pages are (or at least used to be) intended for experienced Unix system administrators and programmers who are already familiar with either Perl 4 or with the Unix toolset of shells, awk, and sed. The Web Revolution wanted something else. Many programmers who were new to Unix (or even to programming in general) learned Perl because it was the most suitable tool for the job, often using it in situations in which writing a shell script would have been better. Thousands of CGI scripts have already been written in Perl, and new ones are constantly being written. Perl has also seen wide deployment in other areas -- bio-informatics, for example, and programming UIs. MandrakeSoft has written the Mandrake Linux installer in Perl, and also many of the distribution system tools. And Mandrake is a very popular distribution.

CPAN has provided programmers with a great number of modules that can be easily installed and used to solve most tasks much more easily. CPAN's convenience and coverage is the envy of all other languages, and even accounts for some people using Perl despite liking a different language better.

One thing that has to be remembered is that Perl exhibits the Iceberg Effect: for every piece of code that is distributed (either on CPAN or otherwise), there's much more code that is kept in-house or in private. Many CPAN modules have many equivalent ad-hoc routines doing similar things written in Perl that are actively in use in many software shops and were never released to the public, nor are going to be released. (I know I wrote such things when I began programming in Perl)

Perl 5 is a modern, powerful language which can be used to write code that either incorporates many different paradigms or even emulates many other languages. As Larry Wall once noted: "You can write Perl programs that resemble sed, or awk, or C, or Lisp, or Python." (Or BASIC/Visual Basic for Applications, Pascal, a great deal of Haskell code, PHP, etc.) Most people can write a lot of Perl 5 code and won't encounter a place where they think that it is inadequate. Thus, Perl 5 is good enough not only for today, but also for the future ahead. ANSI C is a much more limited language than Perl 5, and still, many people are actively using it and writing new code in it. They also often like doing so. If ANSI C is good enough, there's no reason to assume Perl 5 isn't.

Why Perl 6 is Bad?

No one Understands Perl 6

Having read this title, you are probably thinking to yourself: "So you don't understand Perl 6. What makes you think everyone else doesn't? How do you justify this inductive thinking?" Let me tell you a little story:

One day, I met with a good friend of mine (whom I highly appreciate both as a person and as a software engineer), and I asked him if he reads the Apocalypses. He said he does, but that he doesn't understand them. This eventually made me realize that I also read them, and also did not understand many things. And neither he nor I are particularly stupid people. And here's an interesting quote from the famous Weblog "Joel on Software": "Whenever somebody gives you a spec for some new technology, if you can't understand the spec, don't worry too much. Nobody else is going to understand it, either, and it's probably not going to be important." (Read more at the link).

Granted, my editor told me that the Apocalypses are sort of Larry Wall's braindumps, notes on design notes using a lot of jargon from the Perl mailing lists and summarizing a lot of discussion that took place there in one condensed article. However, they do indicate that a lot of new things will be added that most people are not familiar with and will need a lot of time to understand because they haven't encountered them before.

It even takes Damian Conway, Allison Randal, et al. a lot of time to prepare the Exegesis after the latest Apocalypse is published. Perhaps a different, more accessible format of the Apocalypses is needed. Perhaps, eventually, such accessible documentation will be made available, teaching people new to Perl 6 or even Perl in general how to learn Perl 6. (I do hope it will fare better than the original Perl 5 man pages, which were a positively awful resource for beginning Perl programmers.)

I'm pretty sure my friend and I are not dumb people. I know people who will have even more difficulty, and they're not dumb either. Who is going to take all of these people by the hand and teach them Perl 6?

Perl 6 is not Backwards-Compatible with Perl 5

You probably think: "Ha! Backwards-Compatibility is the curse of software design. Let's break it now so we'll get something manageable." That's very nice, but backwards-compatibility is also something you need to make sure people can have some amount of confidence in writing code that makes use of the language or API. Breaking it on purpose, without a good enough reason, is bound to cause resentment among the users, who may opt to convert to something that is known to be more backwards-compatible or stay with the old version.

Perl 6 breaks a lot of compatibility on purpose. In fact, most of the Perl 5 code out there won't compile with it (unless Perl 6 will run in some backwards-compatibility mode.) Naturally and hopefully, this Perl 5 code can be re-used from within Perl 6. That's better than nothing.

As an example of breaking things on purpose, here's some Perl 5 code I wrote:

for my $t (0 .. 6)
  {
  if (abs($new[$t]-$new[$t+1]) > 3)
  {
  $is_ok = 0;
  last;
  }
  }
  if ($is_ok)
  {
  push @moves, [$i,$j];
  }

And here's the equivalent Perl 6 code:

my $is_ok = 1;
  for 0..6 -> $t {
  if abs(@new[$t] - @new[$t+1]) > 3 {
  $is_ok = 0;
  last;
  }
  }
  if $is_ok {
  push @moves: [$i, $j];
  }

Recognizable, but completely different! Furthermore, the dereferencing and method invocation in Perl 6 changes from "->" to "." (just because that's the way it's done in languages such as Java and Python), so the good old string concatenation operator is now "_".

So imagine you have a large body of Perl 5 code which you need to extend with more code. Are you going to write the new code in Perl 6? Well, if you want other Perl 5 programmers (of which there are now several hundreds of thousands) to be able to understand it without getting on their nerves, forcing them to learn a new language, forcing them to understand Perl 6, etc., then Perl 5 is the only option. I predict that most software shops are going to stay a clear mile away from Perl 6, and mandate all of their programmers to use Perl 5 exclusively.

Writing new code in Perl 6 while having to maintain old code in Perl 5 is going to be confusing, and will require the developers working for your project to know both and to actively use both. Most Unix shops just settle on either Perl, Python, or a similar Agile language, and may even have strict style and use guidelines on how to write the code in these languages. They're not going to like having a dual Perl 6/Perl 5 codebase.

While software enthusiasts, who like to write code for fun, may learn Perl 6 because many of them like to learn new languages, they will probably be told to use Perl 5 (or Python, or PHP) at their workplace. Perl (1 to 4) grew as a language to get some Unix jobs done better. Perl 5 was intended as a revamp to eliminate many of the Perl 4 shortcomings, and even to direct Perl for writing large applications. Both of these goals were achieved, but what should still be remembered is that most people use Perl for in-house use, and this is what "pays the Perl developers' rent", so to speak.

Perl 6 may be a good language to design 10 or 20 years from now, but it's certainly not one for this moment. Perl 5 is a good language today already, and if we want people to adopt a new dialect of Perl, we need to start from it, not from a completely new page.

Perl 6 is too Complex

Before we go on, I ask you to make sure you've read what Joel Spolsky has to say about the growing complexity of programming. Yes, programming is becoming more complex, and IT workers find themselves having to (at least temporarily) specialize in a certain field or limited set of fields. However, it is still important that the framework designers try to reduce unnecessary complexity.

A good example are the Unix mail servers. Sendmail used to be the most popular solution, but its configuration is incredibly complex (which is just one of its many shortcomings). As a result, a new crop of mail servers came about: qmail, Postfix, and others. These all sported an easier configuration that even mere mortals could do.

Another good example are languages like Java or PHP. These are highly successful languages, in part because the rules, grammar, and syntax of them were kept as simple as possible. While their standard libraries are very comprehensive and complex, the language as a language is very simple. (I don't suggest that Perl 5 follow this lead -- a complicated core language is one of the things that makes it sexy -- I'm just noting this fact).

The Perl 5 core language is already very complex. Part of this complexity is justified, and the rest is a result of either legacy features or things that some programmers like to use. Nevertheless, Perl 6 will be much more complex than that, too complex for the present level of Perl programmers.

When Perl 5 was created, people expected its programmers to be experienced Unix sysadmins or programmers, probably with some working knowledge of Perl 4 (the Perl 5 man pages originally reflected this). The WWW boom wanted something else, and a great many Web programmers learned it because that's what their job required, sometimes even using it for writing "shell" scripts because Perl was all they knew at the time. Further developments, like the growing popularity of Linux and the bio-informatics boom, have made Perl even more popular. Some people even learn Perl as their first real programming language.

So, Perl 6 will be too complex for them. Much too complex. If I and my friend (who now eats hardcore scientific documents for lunch, and got a reputation for himself as the local Linux kernel guru) could not understand the Perl 6 Apocalypses, I expect much less of the many everyday Perl workers, who are just using Perl to get their jobs done and could not care less about all the extraneous Perl 6 features.

A lot of what Perl 6 integrates will probably turn out to be linguistic fads that no one will want to use. Integrating everything at this point is not going to be beneficial, especially for a language like Perl 5 that aims to be a no-nonsense language for everyday use.

Analogy with the Perl 4 to Perl 5 Conversion

A question that needs to be asked is whether this is analogous to the Perl 4 to Perl 5 conversion that took place circa 1993 and also broke a lot of compatibility, and required people to adapt or even rewrite a lot of their code. There is a place for comparison, but nevertheless, the situation today is very different. Perl 4 was heavily lacking in many respects and justified a good redesign that would make it more up-to-par with the powerful languages of then and now like Common LISP, Smalltalk, C++, and Python.

Perl 5, on the other hand, can serve us well in the foreseeable future. Another thing to note is that Perl 4 was a niche language that was used by various Unix workers to write support scripts and the occasional large-scale application, but not much else. Perl 5, on the other hand, is much more commonly used. There are many more Perl 5 programmers, much more Perl 5 code out there, and much more collective knowledge and love of Perl 5. Also, the Perl "market" of programmers that can learn it is much more saturated today than it was when Perl 4 started.

The Language Formerly Known as Perl 5

Here's a better idea: Let's continue to develop Perl 5 (either using the Perl 5 backend, PONIE, or something else), making incremental improvements, and so make sure it keeps with the pace of technology. Sometimes, compatibility can be broken, or features removed, but not in one big swoop, and always informing people about it first.

We need a new name, because Perl 6 bounds us with its upper digit, leading us to eventually have Perl 5.666. I'll use the name Fifer here, simply because it is an awful name that I don't intend to use. (I'll let the Perl community pick a better name from the list of new names, one of which I'm going to advocate). Fifer would be the language formerly known as Perl 5. It would belong to the Perl Family of Languages, which includes such languages as Perl 4, Perl 5, Perl 6, Ruby, and even PHP.

10 years from now, Fifer will probably be more similar to what Perl 6 is today. However, at that point, there would be plenty of Rindolf code that is perfectly working and modern, as people will have plenty of time to gradually adapt their code.

In a way, I think this is what will happen regardless of whether people read this article or not, but I like to make this prediction clear because I think a programmer eventually needs to make such predictions if he wants to make sure his work lasts for a long time. I see Perl 6 as a dead end, and so don't intend to try to understand it further, and don't intend to learn how to program in it, unless I actually have to maintain some Perl 6 code. (which I don't expect to happen). And I suggest you do the same.

The standing issues in Perl 5 (some of the major ones will be covered shortly) can be resolved without a complete redesign of the language. While a backend change may be in order (like switching to PONIE instead of perl5), or at least a heavy revamp of the original perl5 one, overhauling the language is not needed. The time calls for a gradual evolution instead of a complete revolution.

Standing Problems in Perl 5 and their Solutions

Perl 5 is a good language, all in all. Nevertheless, it has some standing problems which should be resolved. Perl 6 is the wrong way to resolve them. Here is the list of the most major ones as I see them, with suggestions on how to resolve them. Other people may have different views.

Lack of Good Accessible Reference Documentation

The Perl 5 Man Pages are the official reference documentation for Perl 5. Nevertheless, they:

  1. Are quite disorganized. Many things are documented in very obscure places.
  2. Rely on a lot of knowledge of other technologies (many of them pretty obscure or largely superseded by Perl 5 now).
  3. Give many examples that give a common use, instead of trying to illustrate the point. Usually, they cannot be fluently read.
  4. Are generally very hard for a person without a good background in what they talk about to understand.

The Camel Book is much better in this regard, but it is not freely redistributable or modifiable, and not even available online except as part of the pay-per-month Safari, or in various illegitimate online copies. It is, thus, also inadequate.

A good core reference documentation needs to be written, especially taking into account the fact that the ones for Python or PHP are excellent. The easiest way for the community to resolve this is that the authors of the Camel Book make its text available online, preferably with its POD sources. This will not hurt the book sales much, as many people buy it to read before going to bed, or to give as a present to their co-workers, or to keep at their workplace for everyone to enjoy.

If this is not going to be done, then either the Perl 5 Core Documentation will need to be heavily revamped, or an entirely new documentation ought to be written. This will mean much more work on the part of the contributors volunteering to do it, but it is quite necessary.

Perl 6 would be a non-solution to this problem because it gives an entirely new language for which new core documentation has to be written.

(Note that I'm not referring to Perl 5 tutorials, which help a beginner get started. There are quite a lot available, and some of them are reportedly good enough.)

A More Unified Object System

The Perl 5 Object-Oriented Programming system is quite powerful, and adequate for most needs. Nevertheless, to extend it whenever possible, developers wrote a great deal of userland Perl modules on CPAN. However, it is not perfectly clear whether it is possible to make active use of several of these modules or all of them in one applications.

A good idea may be to create a new OOP system which features everything these modules have to offer. This may be possible to do in userland, while requiring people to choose between this or the original Perl 5 way of doing things. It can actually model the Perl 6 OOP framework, without requiring people to adopt Perl 6 entirely. Or it can be something completely new that attempts to be more Perl 5-ish.

Regardless, it can be done in Perl 5.

Better Garbage Collection

The current perl5 garbage collection, which doesn't work well if there are circular references, is incredibly limiting in the choice of the application design. We need a better one. I believe Parrot and PONIE will give it to us for free, but I may be wrong. In any case, perl5 can probably be tweaked to adopt such a system.

Multi-Threading

The current iThreads threading model is very resource intensive, and goes against most of the multi-threading tradition used before Perl 5 adopted it. We need a better threading model in which each thread simply shares the same memory space and variables as the other threads. This is similar to what happens with POSIX Threads on Unix, Win32 Threads on Windows, and the threading model of Java or Python. It also was the old Perl 5.005 threading model.

It is possible that the perl5 backend cannot support this very well. If Parrot (or an entirely different backend) does, then a change of backend may be in order.

Other Issues

I have outlined some of the things I found pressing in Perl 5 in the Rindolf Spec (version 0.3.5). Since then, I neglected working on the specification (much less started working on a suitable implementation), partly because I concluded that Perl 5 was already good enough for me. Nevertheless, the specification may be studied to search for nice additions to the language, all of them possible without breaking compatibility.

Other people may have other things that bother them. I am happy to be informed of any things that they feel are missing.

Conclusion

Perl 6 will probably not be widely adopted. The future of Perl lies within Perl 5 and Fifer, i.e., the language powered by perl5, PONIE, or perhaps a different backend. Any time invested in working on Perl 6 or learning about it can probably give you new insights about programming, but don't expect your efforts to have any day-to-day useful value.

People want a language that gets the job done, is fun to program in, and which they love. Different people have different ideas of what this language is, and they also use different languages for different tasks. To quote my good friend (a different one), "How can you make a programming language that is good for every purpose, if you can't even make such a screwdriver?"

Perl 6 is the wrong thing at the wrong time. Perl 5 is already good enough and will be so for the foreseeable future. We can evolve Perl 5 and gradually make it even better and more suitable for our needs, as we see these needs coming. There is no reason to break every last bit of compatibility, create an incredibly complex language which even experts find difficult to understand, and integrate every last feature from every other language in existence today, just because it is conjectured that some people may find it useful.

A gradual evolution of Perl 5 would be a better idea. Fifer will be a language of the present, instead of a very crude prediction of what new languages will look like 10 or 20 years from now. In twenty years, Fifer will probably be more similar to Perl 6. However, it will still be very different, as things in the software world don't always develop the way people expect them to. And it will also be able to run most of the Perl 5/Fifer code that has already been written (and possibly adapted to the new changes).

Let's continue to work on Perl 5 and just make it an even better language (and technology and culture) than it already is. What is for certain is that Perl 6 isn't better.

Recent comments

30 Jul 2010 15:02 Avatar dskoll

I think Shlomi Fish was on the mark. Perl 6 is shaping up to be a Chandleresque disaster and a poster-child for Second System Syndrome.

14 Jan 2010 19:49 Avatar pcg1

it's definitely vaporware.

hwoever, perl5 has gained momentum and has a unified event library (AnyEvent) async I/O (IO::AIO) and stable threading (Coro).

in the meantime, perl6/parrot have basically dropped all of their original goals. new goals: get the mark&sweep stop-the-world garbage collector working. make perl6 rpograms not crash after 100 lines. announce win when perl6 reaches the just-100-times-slower-than-perl5 mark and so on.

basically, it was all hot air.

17 Apr 2009 23:59 Avatar sehe

Erm... subjective update: 2009 now, in my humble experience there is still not much of a Perl6 that I call tangible. I'm inclined to start saying... vaporware. (Debian doesn't know anything about pugs, rakudo or other scary vegetables. Only parrot has certainly gained some audience!)

9 years in the making... i'm still happily (ab)using perl5 when the need arises (seldom).

03 Apr 2007 10:44 Avatar niravdani

OK, now talking about some real numbers
Shlomi Fish, I don't use Perl but you have pressed very sensitive nerve of many Perl advocates.

Anyhow, here are some real-time statistics from a premium job board.

Java : 1 - 30 of 16216 jobs

C# : 1 - 30 of 6625 jobs

.NET : 1 - 30 of9080 jobs

Perl : 1 - 30 of 5368 jobs

Python : 1 - 30 of 1022 jobs

Ruby: 1 - 30 of 439 jobs

Ruby on Rails : 1 - 30 of 223 jobs

TCL: 1 - 30 of 407 jobs

Well, Perl is still holding up good but who knows for how long.

23 Mar 2007 22:00 Avatar FernandoGonzalez

Re: My feeling
Yes, it does have a long way to go but consider where it's heading. Some of the things that will be incorporated into the language are still being researched and developed. Make no mistake, when it's done it'll be the most advanced language ever seen.


"Will it be complex?"


Sure, some things will be different. But as many of you have pointed out, writing some real apps with a language straightens out all the kinks in your brain, or at least, goes a long way.


Take Haskell, a very powerful language with which the Pugs perl6 interpreter is being built. Oooh, monads, type classes ... Believe me, that stuff is scary at first, especially with all the people going around whining, "What are monads? What's category theory? Do I need to study math to code?". Just write some code. No math required.
Monads: no big deal. The let you keep track of what's going on and when. Type classes: you'll love these, they're a nice generics mechanism. Believe me, you'll get it and you'll love it.


The clarity and coherency of Haskell's design will shine through after an indeterminate number of lines of code.
I suspect the same will happen to those choosing to move to perl6. Reading a spec 20 times won't make you grok any language. It's mileage on the keyboard that really counts.


"So, uh, will it be complex?"


No.

The whole point of the new language is to take what was perl5 and make it cleaner, more powerful and beautiful.


"Should I wait for perl6?"


What the Hell? What kind of self-respecting hacker waits for anything?


You've got perl5, right? If you like it, use it. If you think it should be better, contribute.


As for perl6: If it's taking too long for you, go to the pugs website and lend a hand.


Simple. If you don't contribute, don't complain. Perl can be what you want it to be.


Fernando

Screenshot

Project Spotlight

Kigo Video Converter Ultimate for Mac

A tool for converting and editing videos.

Screenshot

Project Spotlight

Kid3

An efficient tagger for MP3, Ogg/Vorbis, and FLAC files.