Some thoughts on Python

I’ve never studied Python, but I’d quite like to at some point in the future.
It seems very much like Perl with a different set of design criteria. Yeah,
there are differences, but compare with C, Lisp, COBOL, and you can see why
Perl and Python are clustered together in the “P languages” (Perl/Python/P^HRuby etc.)

Actually, that’s one of the main reasons I’ve not expended signifant effort
into learning Python – I can hurt my brain far more by trying to learn C or
Haskell. But I’m sure it would be a good thing to do at some point. Here
are a few notes on first impressions: these are just that, impressions,
and may not be factually accurate: if you want to know full and correct
details about Python, there are many excellent blogs that will clarify.

2008-09-25 Thanks for the excellent comments, I’ll fold
in corrections below!

Whitespace

This is the one wart that Perl programmers fairly consistently bring up.
I’m coming around to the idea that it’s a good idea, but I do think that:

  1. it’s not entirely necessary, as most good code is indented anyway
  2. Haskell does it better, with some excellent whitespace inferencing rules, but
    the possibility to drop back to an explitly { braces; and; semicolons }
    style if you need it

Things that you miss out on include multiline strings (2008-09-25: Bill Mill points that “”"syntax”"” does indeed allow multiline strings, thanks), and multiline well, anything,
for example lambdas, but that’s OK because…

Functional Programming

Python has likened to an easy version of Lisp, and it has the same kind of
functional goodies as Perl. At some point Guido wanted to remove (I think)
reduce, grep, map from the core language, and there was much gnashing of
teeth. I think he backed down, but the idea that he suggested it is interesting.
He also wanted to get rid of lambdas, which seems utterly crazy, as even the
Java people are now thinking about adding them. But lambdas are still
deprecated, and are considered not the “pythonic” way of doing things.
Hence they are discouraged and restricted to single lines. Hmmmm.

(2008-09-25: Bill Mill points out that map remains; reduce got moved to Standard Lib, as comprehensions already do the right thing; and grep doesn’t exist. Is that called “filter” in Python? Of course comprehensions can already to map/grep too)

On the other hand, python has list comprehensions which work equally well
on lazy generators! Very nice indeed. It also has a much fuller destructuring
bind than Perl’s (which only works on single-level lists). So some good, some
bad, and some interesting syntax and features. I doubt there’s a clear
“winner”, some more discussion on Perl/Python (via Lisp of course) in Paul
Graham’s Revenge of the Nerds
post.

Oh, and Python has continuations! With a builtin yield command. Perl has
the Coro module, which is a fantastic piece of work, but rather
complicated with confusing documentation. The yield command itself is in
Coro::Generator but it has a number of limitations, like not
being able to clone a continuation. There was a lovely post on creating
monadic “do-notation” using continuations only, and I ported it to Perl
with much swearing only to find that though it worked great for simple
monads, it failed utterly for the List monad (important as it’s
what list comprehensions are built on) precisely because of this.
Apparently the Perl internals don’t really mesh with this, so it’s
unlikely that we’ll be getting it for Perl 5 any time soon. Point to
Python.

(2008-09-25: dolio points out I’m confusing generators and coroutines.
Yup, I do that, sorry! I think one can be implemented in terms of each other (and
vice-versa?), and that some literature does also blur the distinction, but could be wrong about that too? And yes, Ruby has callcc, but apparently the Monad example I was reading can’t be easily ported to Ruby because the continuations/coroutines/thingies generated by yield aren’t clonable)

OO

Python’s OO looks serviceable enough, and method prototypes and so on are nice.
Perl is really catching up here though with projects like Moose, which
truly does deliver on being a “postmodern object system”. Sure, the syntax
could do with some work in places, but the new buzz around
Devel::Declare and Method::Signatures means that this
is coming on well too.

Will on Geekup posted some nice examples of “declarators on attributes”,
which I think are similar to lvalue tied methods, but look rather better
designed. (From Damian Conway’s talk Contextual::Return may
fix the problems Perl has with this kind of nice syntax. I say may
as with Damian’s talks it’s hard to tell whether they truly are a massive
advance in usability and functionality or crack-fueled insanity).
Actually, declarators deserve their own section:

@Declarators

These are really nice syntax for modifying subroutine declarations.
Of course, there’s nothing you can’t do in Perl, but it’s prettier
than direct symbol table hacking. And less hackish than bloody :attributes.
Still, nothing a bit of Devel::Declare or similar sugar couldn’t
fix :-)

But talking about declarators, I have to ask:

Is Python really more readable?

One of the Python slogans is something like “executable pseudocode”,
as if you can just write out readable summaries of code which are readable
but compile. From the python examples I was looking at (admittedly, these
may have been complex, with features like @declarators) I couldn’t make
head or tail of it. Now I’m not trying to suggest that Python is naturally
unreadable, or that Perl is naturally readable. But I would say that

  1. If you know language X but not language Y, then you may find X more readable than Y.
  2. This may hold even if X happens to be Perl, or Y happens to Python :-)

This may be a good time to pimp my
Readable Perl
slides up, for those people that insist on joking that the phrase is oxymoronic.

Community

Both Perl and Python communities seem to be vibrant, productive (of modules, apps,
blogs, and conferences), and to have a mix of helpful and friendly, and snobbish and
trollish. So pretty much situation normal for perhaps underrated “scripting” languages :-)

Some differences I’ve heard mentioned include (with no figures or such to back them up…):

  • Python conferences are apparently full of people talking about cool projects they’ve done.
    At Perl conferences, people often talk about syntax and technical problems and… oh, yeah, I used this to do this project but I don’t have time to talk about it now…

  • I’ve very rarely heard Perl programmers talk badly about Python. Well OK, we always whine about the whitespace, but I know that jerakeen and
    Dean
    have said generally positive things about it.
    I’ve had Python people telling me bad things about Perl, despite not knowing anything about modern Perl, which I think is fairly silly, unless they are doing it only for postmodern ironic reasons
    (which is an excuse apparently).

  • Of course this may be because Perl programmers pick on PHP and Java instead. But that’s OK because we’re being postmodern and ironic.
  • I have an (utterly uninformed) hunch that Python conferences are better attended, but that this
    may be in part due to there being fewer of them.

  • Python has Guido. Perl has @Larry (in the plural). (I’m not sure that this isn’t overrated,
    $Larry probably gets the deciding vote).

There is a constant meme that href="http://greenokapi.net/blog/2008/09/24/italian-perl-workshop-2008/#comments">Python
is somehow more “modern” than Perl.

This is almost never qualified, I remember in this interesting book on programming language design that it was mentioned that the fact that
Python’s regular expressions lived in an external module was a more modern design that Perl’s,
which had them as builtins. Perhaps Perl not building in support for declarators and list
comprehensions is more modern too? I’m sure if I say this often enough, it’ll be a good
substitute for actually making sense :-)

But Python does seem to get more buzz, with things like Google’s AppEngine supporting
Python before Perl. Hey ho. I still think that there are many more Perl jobs going,
for what that’s worth.

Modules

CPAN is a big advantage for the Perl community, and I believe that Python’s module ecosystem
is somewhat less complete. But it looks very impressive nonetheless:

Off the top of my head, there’s Twisted, Python’s equivalent to POE; Django (Catalyst);
and PyGame (possibly comparable to SDL_Perl?) as fairly high-profile complex libraries
that show, to be honest, that both languages are pretty healthy with regards to module
support.

(2008-09-25: Bill Mill points out PyPI
link, and admits it’s “still no CPAN, but it’s coming along”, which is more or less what I
was trying to suggest ;-)

So…

Python looks like a great language, with many of the same strengths of Perl. If you’ve
looked at Perl and it didn’t really click, try Python. And vice-versa. If you’ve not used
a dynamic language and want to try, then use one. Honestly, you’ll love it. (And if you
don’t, try the other one. Or Ruby. Or go crazy and learn Haskell.)

Italian Perl Workshop 2008

I went to my 3rd Italian Perl Workshop, href="http://conferences.yapceurope.org/ipw2008/">IPW2008 at the end of
last week. It seems to have been the most successful Italian conference to
date, and it certainly succeeded at being both a national workshop and an
international event. It hadn’t occurred to me before that these are actually
two orthogonal aims.

An international event

The organizers managed to pull out all the stops with sponsorship. There’s
always various random swag, books for the auction, cheap/free use of rooms from
the University. And in recent years, the conference has had just enough money
to be completely free of charge, even with its (excellent) coffee and biscuit
break. But this year, the href="http://conferences.yapceurope.org/ipw2008/call_sponsors.html">
“platinum”, “gold” and “silver” sponsors contributed enough money to pay
travel and accomodation for speakers of international calibre:

  • Tim Bunce
  • Rafaël Garcia-Suarez
  • Marcus Ramberg
  • Matt Trout

Other international attendees included Michel “XML::Twig” Rodriguez (though he
lives in nearby Lucca and spoke in Italian); Bruno (a Pole who lives in
Spain… or Amsterdam or something… I’m confused, especially as to why he
attended the Pisa workshop :-); a bevy of Norwegians from Opera’s HR
team; and another Norwegian expat who was completely unrelated; an Indian
postgrad studentessa; and me, I guess.

Hmmm, 4 Norwegians, 3 Brits, 2 French. I wouldn’t have expected quite that
many Norwegians, largely because I’d never have thought that Opera, based in
Oslo, would have been recruiting at a workshop in Italy. But it’s on their
“world tour” as several of the core Perl team for their social network are
Italian. And they really capitalised on the opportunity, sending 3 perlisti
and 2 HR, all of whom were very visible throughout, sponsored a competition
for a Wii, and hired an interview room for recruiting sessions during the
workshop. I’ll be really interested to see how successful they, and the
other recruiting companies (Wind, Dada, A-Tono) have been. It’s very positive
that Italian companies using Perl are getting involved like this.

Oh, the talks! Matt spoke about Devel::Declare, which rocks. I finally
got to see Tim Bunce’s Perl Myths talk in the flesh, and also his demo of
Devel::NYTProf which is so beautiful it makes me want to cry. Marcus
introduced Catalyst, and I missed the others, for various reasons.

A national event

There’s a danger that the focus on the exotic allure of geeks arriving by
luxurious Ryanair jet could distract from the fact that this is also the event
for Italian programmers. Having two tracks, and a general policy of not
scheduling 2 “guests” against each other worked very well here.

The first day’s Italian talks concentrated on beginner and intermediate
topics, including dakkar’s tutorial and regex theory, and Flavio Poletti on
writing IRC bots, though there was some crossover, as rgs also presented on
coding style in English. The perl.it guys are really keen on appealing to new
programmers, which is fantastic. (There was a little gnashing of teeth about
how the recent Pycon in Italy had even more attendees despite being a younger
conference.)

Not that it was all for beginners: emi spoke about Linux wifi captive
portal setup; emazep showed a fantastic UI for constructing complex DB queries,
running on Catalyst with jQuery; grubert presented a news portal prototyped in
2 months with the awesome power of CPAN; [LucaS] finally presented his
workgroup software IGSuite, yay! Cosimo spoke about scaling and the Dogpile
Effect at Opera. Sadly I missed the “GUI track” completely with Mattia Barbon,
the author of WxPerl, and nids talking about Perl/TK. And finally I had to
give an emergency talk myself to fill in a gap (went OK, trailed off towards
the end).

More info

IPW2008
slides
and
photos
are already being posted, and larsen is collating blog posts to link to from
the main perl.it page.

Two cultures of interactive fiction

One of the questions in a recent href="http://groups.google.com/group/rec.arts.int-fiction/browse_thread/thread/6d217fa01cabc0ee">survey
on rec.arts.int-fiction asked if we preferred “story” or “puzzle”
interactive fiction. Though it’s not as much of a binary choice as the wording
implied, there are two extremes of a continuum, and many people have a
preference towards one end or the other.

I’m not a very big IF player, but I did get into the competition in 2004, when
I played and voted on every z-code game and many of the others. My favourite
games were Blue Chairs and
Gamlet. Both had great writing, lots to do, and,
when they tried to do puzzles (Blue Chairs’s midsection, the second half of
Gamlet) lost it completely, requiring judicious use of the walkthrough to be
able to continue the story.

I completely bypassed the puzzle-based scifi game, All Things Devours, the
first time around: I’d found it an unappealing combination of boring, hard,
and confusing (I died repeatedly, without ever having understood what it was I
was meant to do). It turns out ATD placed 3rd, just after Blue Chairs, and
after reading some rave reviews of it, I decided to give it another shot. That
time I managed to get into the lab (previously I’d missed the buttons that
opened it) and managed to blow myself up once or twice before getting bored.

Now the premise of the game sounds interesting: you have to travel back in
time, but while you’re there, you can’t meet your future self or Bad Things
will happen. My problem was that I couldn’t even work out how to use the time
machine…

Just recently, interest in IF briefly sparked again, I decide to give ATD another go.
I’m momentarily put off by the fact that setting the “timer” doesn’t mean the
one on the time travel device, but the one on the bomb I’m carrying. Oops.
So I have to set the panel? But what to? I try random numbers between 1 and
500 and end up dying. So I resort to a walkthrough. It’s a number of seconds
(why the game can’t tell me this, given that the PC I’m controlling invented
it, I don’t know). I calculate the right number of seconds, press the button,
and vow to actually pay attention to the time reported in the status bar. Then
I start wandering around and, yes, the game is actually very clever (as I’ve
heard, but not experienced first hand up until now), you die if you see your
future self, or if that self discovers part of her world universe in an
inconsistent state. My resolve to stick at this with a map and a list of times
and object locations vanishes after dying twice.

I’m not trying to criticize ATD by the way: people whose opinions I trust have
said great things about it, and I have tried really hard (well, fairly hard,
but a number of times at least) to get it. It’s just much further inclined to
the puzzle end of things than I’m comfortable with, hey ho. If you’ve not
played ATD, then go and play it now, you might well love it (but play Blue
Chairs too!)

Oddly, I love Spider and Web, and that has some really difficult puzzles too.
Now some of them I got by myself: getting in and out of the scanweb, and
that puzzle. But others (the clattering lockpick) completely baffled
me. I don’t get the world model well enough to know when my actions will do
the right thing, especially when they’re about timing (which is an odd
thing to do without detailed visual/sensory stimulation to work with). But, in the main, though I don’t really get on with puzzles, you’re given
a gentle training in the use of the gadgets. And I loved how the game
deflates your progressive attempts to do things like learning how to shut down
the scanweb to sneak across the gun. So having to resort to a walkthrough to
work out the frustrating “boring” puzzles is actually worthwhile: because it gives
you a payoff including more clever and fun puzzles, the NPC interaction,
and the aha moments of the unreliable narration. OK, so the endgame doesn’t
work for me: yes, it’s nice to see what use the PC actually made of various
gadgets and hiding places, but now you’re working without the training wheels
of the interrogator to tell you what you “actually” did. (I’m torn between
wishing on the one hand that the dash through the facility was more “on rails”;
and on the other that I’d taken a deep breath, mapped it, and realised that the
basic principle was “look at all the places that have been signalled in
unreliable narrations” and solved the puzzle/story myself).

But no matter, the first half of Spider and Web has set things up, it’s
earnt the right to have a cruel and boring timed puzzle. I’m not sure
it quite deserves the final lab puzzle though, oh well…

The r.a.i-f thread I linked above has some more discussion about the dichotomy/continuum
of puzzle/story IF.
If you haven’t already read/played modern IF, then it’s worth looking at a number
of different styles before making an opinion on this varied form!
(And if you play IF, where do you fall in the continuum?)

Italian Perl Workshop 2008 looking tempting

(Cross posted from my use.perl blog)

The Italian Perlmongers are finalising their preparations for
IPW 2008, Pisa. I managed to get to the last 2 while I was working in Florence, and the organizers have always managed to get a great venue, coffee breaks with unusually nice biscuits, and put on a fantastic mix of talks and a lively hallway track.

Like all the national conferences, talks in previous years have been largely in the local language, with maybe a handful in English from expats and visitors. This year though, the organizers have also managed to get great sponsorship from Opera, Booking.com and others and as well as making the conference free, they’re able to sponsor some fantastic guest speakers.

Bepi has already confirmed:

  • Marcus Ramberg of Catalyst fame (speaking in English. Or possibly Norwegian :-)
  • Rafael Garcia Suarez, Perl 5.10 pumpking (speaking in English… or French — or Spanish? ;-)
  • … other speakers to be advised (Update Oops, I pre-announced one of the speakers currently in discussion with Bepi, must have misunderstood the email, apologies to all, but I do hope that does get confirmed as it’s tremendously exciting).
  • And looking at the talk schedule, it looks like Andrew Shitov (ash) from Moscow.pm and organizer of the Russian Perl Workshop will be talking on distributed programming with WWW::Page and Gearman, as part of his European tour (also in English)

All in all, there has never been a better time to go to an Italian Perl Workshop as a visitor, even if you don’t speak Italian.
It’s all looking quite tempting, though I already have YAPC::EU the month before, hmmm…

Some notes on logistics if you’re thinking of travelling to Pisa from outside Italy:

  • Pisa airport flies various low-cost routes (Ryanair, Easyjet and others) as well as some real airlines (I like the Meridiana Gatwick-Pisa flight). Some airlines fly to nearby Florence, Bologna, or Rome, and Pisa is a convenient transport hub with trains from these cities, and Europe (Paris, Vienna, Geneva).
    The conference page summarizes travel options to Pisa.

  • Some useful links for accomodation in Pisa

  • The most important tourist sites in Pisa are conveniently clumped together in the “Campo dei Miracoli”: the leaning tower, the Duomo and the Baptistery.

  • Florence is about 1h30 away by train or coach. Damn I miss Florence…

10,25,50… sequence fun

Debolaz asked in #moose about the best way to create:

a list of numbers like 10,25,50,100,250,500,1000,etc without tracking any other
state than the number itself

Nope, this isn’t
A020179
but the much more prosaic
A112024.
Debolaz wanted this sequence, or something similar to it, for the same reason
the US used it for currency – the numbers are human meaningful, and are useful
as the numbers get exponentially bigger: in his case, for the labels for a
chart.

This may not be the most interesting of sequences, but, as autarch
pointed out, there are many algorithms that work, and exploring some of them
is fun and/or instructive.

First steps

My first idea was as follows:

  • if the number is a power of 10, multiply by 2.5
  • otherwise multiply by 2

Of course, knowing whether the number is a power of 10 is the only problem here.
The most trivial way is to treat it as a string, and check if it begins with “1″.

sub seq_1 {
    my $n = shift;
    my $val = 10;
    my @ret;
    for (1..$n) {
        push @ret, $val;
        $val *= $val =~ /^1/ ?
                2.5
              : 2;
    }
    return @ret;
}

say for seq_1(10);

autarch suggested dividing by 10 until you get to 1 (or not) as an alternative.

So we were back to a state variable like the position of the element in the sequence.

Mappy

Rather than doing it recursively, you can calculate the value from each position trivially:

    map { 10 * 10**(int $_/3) * [1,2.5,5]->[$_ % 3] } 0..10

That is to say

    10    10    10    100    100    100
*    1   2.5     5      1    2.5      5
=   10    25    50    100    250    500

i.e
10 * 10 ^
     0     0     0      1      1      1
*
     1   2.5     5      1    2.5      5

Not much else to say really, though I think indexing into [1,2.5,5] with the modded value
is cute.

awwaid suggested an improvement:

    map { 10**(int $_ / 3 +2) * 2**($_ % 3 -2) } -1..9

Which hurt my head for a while: it comes from the realization that the sequence is actually

    10   100   100   100   1000   1000
/    1     4     2     1      4      2

Recursive

frodwith suggested a recursive solution with an explicit parameter:

 sub your_seq {
  my $n = shift;
  return 10 unless $n;
  my @prev = your_seq($n-1);
  return (@prev, $prev[-1] * ($n % 3 == 1 ? 2.5 : 2));
}

That has a rather odd recursion style: turning it into tail-recursive style, and then
eliminating the recursion using the techniques from
Higher Order Perl
are left as an exercise to the reader.

Is Perl an acceptable Haskell?

Finally though, it is possible to do this without any additional state, by
turning the function into a little state machine. On the first iteration, the value
is multiplied by 2.5 and sets the next iteration to be one that multiplies by 2, leaving
the next iteration to multiply by 2 but queue up the original (x2.5) function next.

Instead of actually writing a state machine, I decided I’d rather have an infinite cycle
of the 3 functions, and iterate through them. Haskell has this as a builtin, but we
have to define it in Perl

sub cycle {
    my @list = @_;
    my @curr = @list;
    return sub {
        @curr = @list unless @curr;
        return shift @curr;
        };
}

Now we can create a list of functions that multiply by 2.5, 2, 2:

cycle([ map { times($_) } @multipliers ])

Where “times” is a function that given one multiplier, returns a function that
multiplies by that number. That is to say, it’s the * operator “curried”.
And by great coincidence, I just uploaded
Sub::Curried to CPAN, so we’ll
use that:

curry times ($x,$y) { $x * $y }

We’ll use the “curry” declarator for the rest of this example, largely because of the
automatic argument unpacking magic (I love Matt Trout’s
Devel::Declare).
Now that we have a sequence, we need to write something similar to Haskell’s scanr,
that repeatedly applies a transformation, returning a list of the intermediate values.

curry scan_iterator ($it, $start) {
    my $next = $start;
    return sub {
        my $ret = $next;
        $next = $it->()->($next); # prepare next value;
        return $ret;
        };
};

And as Perl doesn’t like infinite calculations, we may as well transform a slice of this
sequences into an array:

curry iterator_to_array ($it, $count) {
    return map { $it->() } 1..$count;
};

At which point we can put the whole thing together like so:

say for iterator_to_array(
        scan_iterator(
            mk_seq( [2.5, 2, 2] )
        )->(10) # start value
    )->(12); # number of iterations

I’ve uploaded the final source code as

an example for Sub::Curried
.

Elegant? Overcomplicated? You decide!

Update:

  • Joeri Samson pointed out a thinko (multiple/power)
  • augustuss suggests the haskell version: scanl (*) 10 $ cycle [2.5,2,2]

    Much neater than my faffy version. But we can do the same in Perl (we just need to define scanl and rename iterator_to_array to “take”. Link to complete source code is updated.

    say for take 12 => scanl(times)->(10 => cycle [2.5, 2, 2] );

    Notice how (times) with no arguments is equivalent to a function reference, due to the autocurrying!

More Perl hate, and what to do about it: AUTOLOAD

A couple of interesting comments to my
Five things I hate about Perl.
Quicksilver pointed out that I’d missed the difference between arrays and lists.
Yup, that’s pretty subtle, and of course it’s bitten me a few times (though oddly
enough, I don’t have that strong a dislike of it). Luqui on the other hand pointed
out that Perl is so much more fun if you do take the plunge and use
language-mutating modules, irrespective of the fact that they’re not standard.
I do agree, though it can sometimes be difficult getting to use these modules
for work, as everyone else has to buy in to the benefits and (perceived or real)
risks (learning curve, stability, speed, etc.).

So here’s an example of something that I’d forgotten I hated until I banged my
head against it in work’s codebase: AUTOLOAD.

AUTOLOAD like other languages’ “method-missing” feature, allows you
to dynamically respond to a method invocation without having explicitly defined
the method. So, for example, someone might call get_name on an object,
and the accessor for the ‘name’ field will be automatically be generated.

Actually, that sounds pretty cool, no? The problems are really in the implementation.

  • AUTOLOAD subs are ugly. You have to do something like
    use vars '$AUTOLOAD';
    sub AUTOLOAD {
        my $sub_name = $AUTOLOAD=~/(\w+)$/;
        ...
    }
  • Autoloading a sub is slower than normal dispatch. This isn’t a big deal of course in Perl,
    as you can hack the symbol table. The first time the sub is called, you generate a coderef to deal
    with the thing, then register that sub. On subsequent invocations, AUTOLOAD will never get called.

  • …Which does lead to the point that in very many cases you can just pre-generate a whole lot
    of methods during late-compilation/early runtime.

  • If AUTOLOAD doesn’t handle a method, then what? It’s all too easy to forget to handle this case, and suddenly you have method calls that do nothing silently, an insidious and subtle bug.
  • And if it didn’t handle a method, but the next class in the inheritance chain could have… tough — it’ll never get a chance to deal with it.
  • You can’t use ‘can‘ to find out whether your object provides the method or not.

Yuck. Of course, someone has already dealt with most of the problem in a module: Ben Tilly’s
Class::AutoloadCAN. Using this module, you just declare a CAN sub, which may or may not return a coderef (the subroutine that will deal with the problem). Behind the scenes, overriding of the standard ‘can‘ meta-method and an auto-generate AUTOLOAD will automagically do the right thing. Yay!

The syntax isn’t that pleasant though. If we want to handle multiple autoloaded subs, we have to
stick them all into this CAN routine. I’d much rather do something like:

sub foo :handles(/^foo_(.*)/) {
    print "Fooing $1";
}
sub bar :handles(/^bar_(.*)/) {
    print "Bar'ing $1";
} 

So I thought, let’s use Attribute::Handlers, which lets us define these handlers. But I don’t really like Perl’s
attributes feature, and to be honest, I’m not keen on the redunancy of naming a sub and then the
regex that it handles.

I’d rather have some syntax like:

autosub (get_(.*)) {
    ...
}

and of course that suggests Matt Trout’s beautiful
Devel::Declare.

And it’s surprisingly simple* to knock up a prototype AutoSub module which implements this. From the accompanying test.pl:

autosub (^take_(.*)) {
    my($what, @args) = @_;
    return "Took a $what";
};
autosub (^get_([^_]+)_(.*)) {
    my($adj, $noun, @args) = @_;
    return "Got a $adj $noun";
};
autosub (^do_the_.*$) {
    my ($what, @args) = @_;
    return join "," => $what, @args;
};

* Where “surprisingly simple” glosses over a couple of points, noted here briefly for reference:

  1. Devel::Declare isn’t all that well documented yet. Basically, install_declarators takes 2 subs:
    1. add code at the beginning of the sub, e.g. for argument unpacking. We don’t use this, so we just return an empty string.

    2. install the subroutine reference. Other examples install this into the symbol table. We just add the sub into our list of @CANS.
  2. Yay for closures! Closing over @CANS means we don’t have to mess about with the name of the variable.
  3. Yes, I don’t really know what I’m doing with sub exporting. I think Ricardo Signes’s
    Sub::Exporter would do the right thing, certainly with the CAN sub. But I’m not sure where to hang the Devel::Declare logic with it, I’ll investigate.

  4. Oh, and there are certainly better ways of exporting the goodness of Class::AutoloadCAN into caller() than eval "package $package; use ..."
    Update: mst pointed me at export_to_level, which is a standard flag handled by Exporter and other modules. But C::AC has a custom import… The other trick he suggests works: goto &Class::AutoloadCAN::import at the end of my custom import sub.

  5. If you call a regex in list context, it returns a list of all the captures found. Unless there weren’t any captures, in which case it returns a list containing (1). This sucks, as I want to return the captured text in that case. (I guess I’d prefer it if it returned () but true, but that doesn’t exist in Perl…)
    So right now, I’m checking if $1 was defined, and using the despised $& if not.
    Suggestions?

Hello world

I’ve enjoyed using Vox.com over the last couple of months but I’m finally taking the plunge and moving the blog here to greenokapi.net/blog.

Though I’ve occasionally wanted more flexibility, the trade-off with a hosted service is that you waste no time tinkering, the only thing you can do is write. But the fact that Vox doesn’t, at time of writing, handle comments gracefully reduces the value of the blog — I’ve had some great comments and suggestions from readers, and I’d say that the feedback is one of the most useful effects of having a blog at all. Vox’s insistence on signing up in order to comment is just a barrier to feedback, but I got tired of comments like “Hey, where did my last comment go?” because the default formatter apparently doesn’t like code.

Of course, WordPress has its hatefulness too. For a code-oriented blog you need the Text Control plugin, which is the only way to prevent it from moronizing newlines and making quotes “smart”. (By default, WordPress even turns ascii smileys into horrid yellow gifs, though this at least can be switched off easily).

Five things I hate about Perl

Enough
advocacy, let's get to the nitty gritty of 5 things I hate about Perl.
(After brian d foy.)

  1. The difference between list and scalar context is subtle. One bug that bites people too
    often is this (for some value of "people", including me, far too often):

    sub myfunc { return }    my $scalar = myfunc(); # contains undef
    
        my %hash = (        key => 'value',        key => myfunc(),    );

    In scalar context, we get undef. But in list context, myfunc returns
    a list of zero elements. Guess which context hash declarations are in…

    Luckily in this case we'll get a "Odd number of elements in hash declaration" warning.
    Perl's warnings are mostly very useful and surprisingly helpful. However:

  2. Some warnings suck. Yes, some of them almost always point out an error (the void context
    error is useful: I usually find it means I've written: my $x => 1 instead of
    my $x = 1) but some are more irritating.

    When was the last time an 'uninitialized' warning had any effect on your code apart from making
    you have to sprinkle $x ||= '' throughout your code? Yet I usually restrain
    myself from adding no warnings 'uninitialized' because maybe one time in a hundred
    there's a genuine bug I need to know about.

    Similarly 'once' warnings sound useful, but in practise are usually because you've
    referred to a magic global that's used plenty of times in the library you want to use.
    For example, from List::Util's documentation,

    my $sum = reduce { $a + $b } @list;

    will warn about $main::a and $main::b being used only once. $a and $b are the canonical
    magical variables, wtf is that about? (mst suggests that these warnings are inhibited
    for sort only, which was their first use case).

  3. Argument passing. Yes, it's really flexible to be able to unpack
    @_ in any way you want but I'd like a proper way of doing it. (And yes,
    there are all kinds of solutions including Devel::Declare, but none are
    standard yet). Oh, and we don't have full destructuring bind, so yes, you can
    do ($x,$y) = ($y,$x) and my ($self, %args)=@_, but not my
    ({foo=>$foo}, [undef,undef,$bar]) = @_
    .

  4. It's a big language, with lots of syntax, several powerful minilanguages, a vast array of
    standard idioms, a large set of standard libs (including various incompatible ways of doing
    similar tasks) and a truly staggering number of 3rd party libs, CPAN, with even more ways of
    doing it. Actually, learning a big language is fine, but the bigness is one of the things
    making Perl difficult to parse. The other is the sheer amount of flexibility that Perl
    gives you to shoot yourself in the foot, change the way the language is parsed etc. Only
    Perl can parse Perl, and usually only after executing part of your code first. Yay for exploits
    on your language-aware text editor!

  5. Functional programming isn't easy or elegant. Yes, we have first class
    functions, but things like argument unpacking being ugly make it less
    convenient. Little kludges like the functions passed to map and grep taking
    $_ rather than the usual argument list @_ just add to the fun
    ($_ seems like a special-cased points-free hack, and it's far less
    consistent or common than Haskell's currying)

    I also dislike that regex substitution etc. can't be done as a non-destructive function.

Hmmm, there are probably more things, but those are the main ones. In particular
I don't hate references (yes, they take a little learning, and I'm aware that's a
stumbling block for most learners, but they are quite sensible once you do)
or OO (baroque, but quite capable, and see Moose for a cute modern take on
it).

More Countdown: laziness, Scheme, and German frogs

The Countdown code I showed you isn't really taking advantage of Haskell's laziness.
We should only have to check entries up until the point that we have enough matches
(in the current code 'take 4 $ getAnagrams') and that's good. However, we
have to generate the whole powerset first, so that we can sort it in reverse order
of length. Ideally, we'd generate the powerset breadth-first, in order of length.

OK, so generating the powerset doesn't take all that much time and this isn't a good
optimization as such, but I did think it might be fun trying to do the breadth
first powerset, as all the examples I'd seen had been depth first.

But first: an interlude to marvel at one of the scariest things I've seen this year.
A monadic definition of
powerset
.

import Control.Monad (filterM)powerset = filterM (const [True, False])

I'm not even going to attempt to twist my head around that now, but it's very beautiful,
though it's impossible to tell just by reading it what the intent of the code is.

I asked on #london.pm if anyone knew good breadth-first algorithms for powerset. Joel
helpfully pasted the following

(define (combinations set n)  (if (zero? n)    (list '())    (let ((n2 (- n 1)))      (pair-fold-right       (lambda (pr acc)         (let ((first (car pr)))           (append (map (cut cons first <>)                        (combinations (cdr pr) n2))                   acc)))       '()       set))))

(define (power-set set)  (let ((size (length set)))    (let loop ((i 0))      (if (> i size)        '()        (append (combinations set i)                (loop (+ i 1)))))))

I think it surprised Joel (and actually, it surprised me a little) that I was more or less
unable to read this at all. Yes, I know that the syntax of Lisp is incredibly simple, and
you can learn all the syntax of Scheme in 20 minutes or whatever. The noise of the
parenthesis, the cdrs etc. is just noise, you can filter it out if you look at
it calmly. But I still don't understand where the pairs are in pair-fold-right,
and cut apparently does something similar to currying, but what does that mean
in context?

To cut a long story short, I was reading this as a foreign language rather than a piece of
generic pseudocode. When I was very little, I read with my mother a picture book about
frogs, in German. With the help of the pictures and a little imagination, it was easy to
tell which word meant "frog", which meant "tree", and what the sense of the story was.
After we finished, very puffed up with just how damn clever I was, I started trying to
read it again, and got utterly confused about all these strange words like 'in'
and 'auf' and 'dem' that I just hadn't worried about the first time
around.

So… trying to see the frog for the trees, we can see that
combinations gives every set of combinations of a particular length,
and that power-set merely loops through 0..length, appending the
combinations found for that length.

We can write combinations as a variant on the original powerset function, but which refuses
to carry on once it's got enough letters:

combinations xs n = comb' xs n (length xs)

comb' xss@(x:xs) n l | n == 0 = [[]]                     | l == n = [xss]                     | otherwise = (map (x:) $ comb' xs (n-1) (l-1)) ++ (comb' xs n (l-1))comb' [] _ _ = [[]]

And powerset is easy as:

powerset xs = powerset' xs (length xs)powerset' xs l = if l < minLength                    then []                    else (combinations xs l) ++ (powerset' xs (l-1))

minLength = 3

We can now remove out the lines with sortBy and filter longEnough,
as the new definition already presents the items in the right order.

Does this make it any faster? Apparently not: as I guessed, powerset is not the
hotspot. I guess that the problem is the repeated lookups in the Data.Map — any
suggestions on how to profile the Haskell code, and better algorithms to deal with it?