text:
The Venue: I thought this was the best YAPC yet in terms of venue.1 It was a gorgeous hotel, and the rooms—I didn't stay there myself, but I visited one—were almost insanely spacious.2 The bathrooms were also fairly insane: immaculate, huge, with marble everywhere—it was like taking a leak in a palace or something. The food was good, there were plugs everywhere (a perennial problem at YAPCs), and I did not fail to get onto the hotel wifi once ... not one single time. That's fairly amazing.
If I had to come up with a complaint, I would say I had a hard time getting comfortable in most of the chairs. But, I mean, come on: at this point I'm grasping at straws. Umm ... what else? I did get cold once, but mainly I was thankful they had the A/C blastin' to combat the almost-ninety-degree heat outside.
I didn't stay at the hotel because I brought my family, met my kids' grandparents and aunt in Salt Lake City, and we rented a house. The hotel was literally half a block from a light rail station, so I could easily ride the train in for the day stuff, and I drove in for the night stuff and parked in the free parking garage under the hotel. I mean, seriously: it's hard to imagine a more perfect place.
The Buzz: As you might imagine from Larry's announcement of imminent Yuletide arrival, the buzz this year was all about Perl 6. There were several talks on the topic, and I attended at least 4 of them. Much of what was discussed I found very exciting. Over the past couple of years, I've become a bit jaded on the topic of Perl 6, but this year's YAPC revitalized me to a certain extent. Many of the talks (particularly the back-to-back double whammy of Bruce Gray and Curtis Poe) reminded me exactly why I got so excited by the damn thing in the first place.3 For the first time in several years, I'm actually excited again by the possibilities inherent in Perl 6.
It's not all sunshine and roses, however. There are still several areas of Perl 6 that disturb me:
use Perl 5 modules in a Perl 6 program), which could totally moot the previous problem, doesn't seem to be a reality yet.
I think the first two points are self-explanatory. The third one I threw out at lunch on Wednesday, and tried to explain why that mattered to me by saying that the ability to create dialects was one of the things I found most enticing about Perl 6. To which Randal Schwartz wryly replied, "to the horror of everyone who works with you." Which, on the one hand, is a fair comment. But, as I tried to explain then,4 we already have a bunch of dialects in Perl 5: from the simplicity of adding a few more functions such as List::Util and List::MoreUtils to all-encompassing efforts such as perl5i to practically standard things such as Moose to bits of insane genius such as :IO::All and IO::Prompter. And I'm not even getting into the ones I absolutely detest, such as SQL::Abstract. Although that beautifully illustrates my point: here's a dialect whose syntax is infinitely worse than what it attempts to model, but it's not because its authors were deficient in any way; that's just the best you can do with Perl 5. Now, I'm still not convinced that you wouldn't just be better off writing SQL, but I am convinced that you could do a much better job on something like SQL::Abstract with Perl 6. But we deal with the ugliness because that's the best you can do, and now it's deeply embedded in the most popular ORM for Perl and we all just accept it, 'cause, what else can you do? Crushing that sort of ugliness is what I feel is the promise of proper macros in Perl 6.
The last point ... well, as I say, I want to do some more research before I start a proper rant or anything. Just let me say that I'm one of those people who find context integral to what makes Perl Perl.5 I suppose you could accuse me of being one of those "fear of change" people—after all, I have gone on record as saying that I'm irked that we lost a perfectly good concatenation operator just because all the other languages were jumping off a bridge. But let's not forget that I'm the guy who pimped Method::Signatures, they guy thinks Moose doesn't go far enough syntactically, the guy who is still peeved that smartmatch is "experimental" ... truth is, I friggin' love change. But there's still a set of core principles that you need to hang on to no matter how much you evolve and grow.
But, overall, all the excitement over Perl 6 was pretty awesome to see, and I found it infectious.
The Keynotes: I'm not going to include the "States of the Velociraptors," since that was a series of small talks, so it didn't really feel like a keynote. That leaves us 2 keynotes, which is a 67% reduction in number of keynotes over last year, which I'll take as a mostly good thing. Of course, the number of mostly-non-Perl-related keynotes increased by 100% over last year, which seems a bit worrisome, although I have to admit that both keynotes this year were pretty interesting.6
The first was from Seth Johnson, and it was mostly about social engineering, and it was fairly entertaining. I'm not sure I got much connection back to Perl, but it wasn't a waste of my time or anything.
The second was from Phil Windley, and it was about the Internet of Things, and a language for dealing with that which happened to be implemented in Perl, but I found that a tenuous connection at best. This talk was little less entertaining, but a lot more interesting factually.7 At first it threatened to be too abstract, and then it threatened to be too detailed, but overall it provided several cool insights into what is likely the next big challenge for the Internet. Excellent bibliography at the end too.
The Extras: Once again, Bad Movie Night and Game Night were our social events of choice. The Bad Movie BOF this year was composed of this past December's live RiffTrax Santa Claus, including the bizarre short "Sugar and Spice," and a classic MST3K from the Joel days: Gamera vs Guiron. I'd seen both before, although not the live version or the short for the former, but they were still great, and we had a great time. Plus someone ordered a lot of pizza for only a $3/slice contribution, and Wendy shared some wine from Perl, Germany: both were quite tasty.
On Game Night, we played a massively insane nine-person hand of Fluxx, which I don't think made it more than twice around the table before a fellow named Peter won, mainly by taking everyone else's hands away and playing various cards which essentially just let him keep drawing and rifling through the discard pile until he finally had so many cards he couldn't help but win. I think the one or two newbies at the table never even figured out exactly how to play.
Unlike last year, these two events were on separate nights, so that was much better. However, unlike last year, Game Night was after a big dinner, which meant that we got a pretty late start, so we really only got one game in before it was late enough that we were starting to fade. Still, we had some great conversations beforehand,8 so I'm not complaining.9
The Core: The thing I probably found most refreshing was the lack of any attempt to convince ourselves that Perl is not dead. Maybe it was all the buzz about Perl 6. Maybe it was just that this YAPC was sold out.10 Whatever the case, no one felt any need to try to stamp out that meme this year, and overall I think that's a positive sign.
In fact, call me crazy, but I felt a distinct undercurrent of hope this year. It's hard to put my finger on exactly why I say this ... again, the enthusiasm about Perl 6 has got to be a big part, as is the fact that no one felt the need to remind us that we weren't dead yet. But it was more than that, somehow. Perhaps it was that I heard the absolute least amount of bitching about crappy jobs this year. Still a bit here and there, of course, but, weirdly, it seemed like a record number of people were happy with their employment.11 Maybe it was just bits here and there that came out in talks: a gentleman who gave a Lightning Talk on how he's spending his retirement hacking on the Perl core and it helps keep him feeling relevant, Stevan Little saying he's now embracing the messiness of the Perl core in his work on the p5mop, Larry's answer to the final question in the Bernard Pivot questionnaire, "If heaven exists, what would you like to hear God say when you arrive at the pearly gates?": "I gave you creativity, and you used it to inspire creativity in others."12 Things like this, just like the enthusiasm for Perl 6, are infectious: hope inspires hope.
But perhaps the most exemplary moment of this year's YAPC came in the form of dinner on the final night. You see, I mentioned last year that my first YAPC I brought zero children, my second YAPC I brought 1 child, and my third YAPC I brought 2 children, so obviously this year I was on the hook to bring 3 children. And, oddly, I did. My youngest is only 3 years old, so I didn't think she'd appreciate Bad Movie Night or Game Night as much as the other two kids. But I really did want to have her come out to meet people at least once. So I cooked up a crazy plan to have my entire family—3-year-old, 9-year-old, 16-year-old, The Mother, and The Mother's younger sister—come out and meet me at the hotel after the closing talk, whence we would venture forth and find dinner. I invited 3 of my former co-workers, all of whom had met one or more of my children before, to accompany me, and all said yes. But somewhere along the way we started accumulating more attendees. "Can anyone come?" asked one of my friends. "Sure!" I said, "as long as they don't mind putting up with my children." So they started inviting others, and soon I got into the spirit of it, and, by the time we arrived at the restaurant, we were asking for a table for 16. So I would particularly like to thank Joe, Aran, Michael, David, Krishna, Shane, Bruce, Liz, Dean, Sean, and Adam,13 several of whom I was meeting for the very first time that night, not only for excellent dinner company, but also for reminding me what an amazing community we have.14 They didn't just put up with my children, they engaged with them (and with The Mother and her sister, neither of whom have much interest in the sorts of geeky things we tend to steer the conversation towards), and were good-natured and friendly, and also didn't lynch me when it turned out that the restaurant I chose for our original destination turned out not to allow children. Often times I forget what a lucky sod I am to have fallen into this community, mostly by accident. Maybe the Java guys and the Python guys and the PHP guys have these illuminative, epiphanous moments at their conferences too. But I bet they don't. Certainly I feel like our community is special. And that, in some odd, recursive fashion, is what makes it special.
So, thanks everyone. And, as always, looking forward to next year!
The Appendix: A few quick shout-outs:
My reflections on my first YAPC are available on my Other Blog. Reflections on my second YAPC and my third YAPC are here on this one.
1 But keep in mind this is only my fourth YAPC.
2 My friend David had two balconies in his room.
3 And I needed some help remembering. I'm pretty sure that when I first started getting excited about Perl 6 my eldest child was still in diapers. Those of you who met him this year and had to look up to say "hello" can no doubt understand the full implication of this bit of trivia.
4 But mostly failed. Hopefully I can do better here: I usually express myself much more eloquently in written form.
6 To me, anyway. Your mileage may vary, as always.
7 Again, in my opinion, of course.
8 In fact, two of the "Things I Learned at YAPC" I learned in these conversations.
9 Much.
10 Okay, it only sold out the day before it started, but still.
11 I say "weirdly" because I really don't believe that corporate America—or corporate The World, for that matter—got any better at treating their employees well this year. But maybe corporate Perl, somehow, did.
12 I'm typing from memory here; please forgive me if I didn't get that exactly word-for-word.
13 Especially poor Adam, who had to suffer an hour-long (off and on) description of "Jet Pack Joyride" after asking the innocent question "whatcha playing over there?"
14 And I deeply apologize if I forgot anyone who was with us at dinner.
15 This qualifier had to be added for hopefully obvious reasons.
A while back, I decided to play with Dist::Zilla, and one of the first things I decided to do was make my own plugin bundle.1 Now, if you don't know what a plugin bundle is ... well, that's a bit above and beyond the scope of this article.2 Suffice it to say that, if you want to get the most of out DZ, you want to create your own plugin bundle. (And, if you don't want to do that, then you probably want to be using something simpler than DZ, like Dist::Milla or Minilla or Zilla::Dist.)
So I created one a long time ago but then I never did much with it. I personally don't have enough CPAN distros to juggle to make spending a lot of time fiddling with DZ a priority. But lately I've decided I want to get back into it. So I started out by installing the latest version I'd put out on CPAN.
Well, trying to install it, anyway.
Well, cpanm uses a file called
~/.cpanm/sources/http%www.cpan.org/02packages.details.txt to locate modules. If your module isn't there, it doesn't really matter where else it is. And my module wasn't there.
So I was all set to fire off an email to the module-authors mailing list asking why PAUSE wasn't indexing my module, but something was nagging me, in the back of my head ... hadn't I read something in the Perl Weekly about this recently? Why, yes ... yes, I had.3
A couple of months back, David Golden published this article explaining how PAUSE isn't using your distname to figure out your module name any more. I didn't immediately think of this because David talked about a PAUSE permissions error, whereas what I was getting out of cpanm looked more like this:
[absalom:~] cpanm --info Dist::Zilla::PluginBundle::BAREFOOT
! Finding Dist::Zilla::PluginBundle::BAREFOOT on cpanmetadb failed.
! Finding Dist::Zilla::PluginBundle::BAREFOOT () on mirror http://www.cpan.org failed.
! Couldn't find module or a distribution Dist::Zilla::PluginBundle::BAREFOOT
But I wondered if this could be related, so I went back and reread the post to see what the root of the problem was. It turns out that it shows up when you don't have a
package statement ... like when you use
class instead. Which I was doing. For my little plugin bundle, I'm currently using MooseX::Declare.4 So, yes, I'm using
class instead of
package, so that's probably my problem.
Now, I also remembered that David had done a follow-up article explaining how to fix it. That post advises us to add a
provides directive into our Makefile. Unfortunately, that doesn't help you much from the
Dist::Zilla perspective, because the Makefile is generated for you. What we need is a plugin to generate that for us, preferably by scanning our files and looking for
class declarations. And ... lo: there it is. Dist::Zilla::Plugin::MetaProvides::Class. So, if we're using DZ directly, we would add this to our
inherit_version = 1
But of course we're doing a plugin bundle, so we'll do it like so:
$self->add_plugins (
# lots of stuff here
# metadata
[ GithubMeta => { remote => $self->git_remote } ],
[ 'MetaProvides::Class' => { inherit_version => 1 } ], # <-- added this line
MetaJSON =>
# lots more stuff
And that's pretty much all there is to it.
Except ...
As you may recall, I typically like to use TDD, where feasible. Now, I hadn't really done much in the way of unit tests for my plugin bundle ... as I said above, I never did too much with it, and it was primarily just a shameless rip-off of somebody else's plugin bundle.5 But now I'd really like to figure out a way to test this new feature I want to add. Of course, testing a Dist::Zilla plugin bundle isn't the easiest thing in the world. But I had an idea: I could create a temporary directory, create just enough fake files to make it into a fake distro, then run
dzil build on it. Then I can use the files that it builds to verify various things.
Let's start with a basic test that should pass. Here it is, built up bit by bit:
use Test::Most 0.25;
Our prefix. Test::Most gives us
strict and
warnings, plus all the functions from
Test::More and several other popular test modules. This enables us to keep our boilerplate nice and simple.
use Path::Tiny qw< path cwd tempdir >;
our $tdist = tempdir( CLEANUP => 1 );
# go to our temp dir for this test
my $old = cwd;
END { chdir $old } # go back to original dir so cleanup of temp dir can happen
chdir $tdist;
Instead of using
File::Temp directly, I'm using Path::Tiny here, because it will make several other things easier as well. We make a temp dir for our faux distro, and then
chdir to it. The one tricky bit is, if we're still in the temp dir when our test ends, it can't be cleaned up, because you can't remove the current directory. So I use an
END block to make sure we go back wherever we came from.6
# create a skeletal distribution
my $tname = 'Test-Module';
my $tversion = '0.01';
path('dist.ini')->spew( <<END );
name = $tname
author = Buddy Burden <barefoot\@cpan.org>
license = Artistic_2_0
copyright_holder = Buddy Burden
version = $tversion
fake_release = 1
repository_link = none
my $lib = path( 'lib', 'Test' );
$lib->path('Module.pm')->spew( <<'END' );
package Test::Module;
# ABSTRACT: Just a module for testing
my $t = path( 't' );
$t->path('require.t')->spew( <<'END' );
use Test::Most 0.25;
require_ok( 'Test::Module' );
That's about the minimum you can get away with for a DZ distro. I add the
fake_release bit just in case I want to test releasing someday ... I definitely don't want my unit tests accidentally uploading things to PAUSE. And I need the
repository_link = none bit because I'm using the GithubMeta plugin, but I'm not creating a git repo for my fake distro.
# now build our test dist so we can have some files to test
demand_successful_command("dzil build");
chdir "$tname-$tversion";
my $meta = path('META.json')->slurp;
like $meta, qr/"version" \s* : \s* "$tversion"/x, 'version is correct in meta';
dzil build, change to the build dir it makes, suck in our
META.json file, and verify something trivial in it (in this case, the version number). Easy peasy.
Have to let the TAP harness know we successfully reached the end our test file without keeling over in the middle.
sub demand_successful_command
my ($command) = @_;
# get rid of stdout, but keep stderr
# it might aid in debugging
is system("$command >/dev/null"), 0, "command succeeded [$command]"
or done_testing, exit;
And, finally, the definition of our function to run a command: execute, toss stdout, keep stderr, verify success, bail out if not.
(You can see the full version of this first cut at my unit test on GitHub.)
So, I ran this and it worked like a charm. Now what I needed was a failing test.7 Well, at this point, that's pretty simple. We just change this:
my $lib = path( 'lib', 'Test' );
$lib->path('Module.pm')->spew( <<'END' );
package Test::Module;
# ABSTRACT: Just a module for testing
to this:
my $lib = path( 'lib', 'Test' );
$lib->path('Module.pm')->spew( <<'END' );
class Test::Module with Some::Role
# ABSTRACT: Just a module for testing
and add one line towards the end:
like $meta, qr/"provides" \s* : /x, 'contains a `provides` in meta';
And, voilà: verification of our
META.json tweak. (Again, full code for this version can be seen on GitHub.) This code runs, the test fails, I add the line to my plugin bundle that I showed you before, test passes, and bob's yer uncle.
Now, before I uploaded this to PAUSE, I decided to go ahead and build my distro and verify that the
META.json would be correct now. Except it wasn't. I spend an embarrasingly long time trying to figure out what the hell was going on there before I realized that my plugin module is self-hosting: that is, I build my DZ plugin module using my DZ plugin module. This gives us a bit of a chicken-and-egg problem; specifically, when I build the new version of my module, I'm actually using an old version of my module ... whatever the last version I had installed was. My new code is local to my git repo, but, when I build the module, it's going out to the old code, installed wherever Perl modules are installed.8 Which is fine, most of the time. But, in this case, building using the old code means that I'm not actually going to fix my problem: the
META.json I'm about to upload to PAUSE doesn't contain the
provides directive, so it's still not going to get indexed. Crap.
We can fix this, though. Build the new version locally but do not upload it to PAUSE. Now use
cpanm to install the resulting tarfile:
cpanm -n Dist-Zilla-PluginBundle-BAREFOOT-0.04.tar.gz
Now, build it again (which will now be using the new code), verify that the
META.json looks right (it does), then upload to PAUSE. Yay!
But now CPAN Testers lets us know that it hates us. Sigh.
sh: 1: dzil: not found
# Failed test 'command succeeded [dzil build]'
# at t/tdist.t line 79.
# got: '32512'
# expected: '0'
# Looks like you failed 1 test of 1.
My first thought was, how can
dzil not be found? After all,
Dist::Zilla is listed as a dependency of ours, so it has to be installed ... right? Well, yes, but that doesn't mean that
dzil necessarily has to be in the path. For some smokers, it will be. For others, it won't. Counting on being able to find it is just not a good idea, in the general case.
Briefly I considered posting something to the perl-qa mailing list asking how I can figure out where the
bin/ directory is. Maybe I could use
@INC to figure out where
Dist::Zilla is installed, and somehow reverse engineer where
dzil must be ... ? But then a much simpler solution occurred to me.
Have you ever looked at the code for dzil? It's super-simple. It's just a wrapper around the
run method of the application class. So why bother to shell out to run an external command from our Perl unit test? We can just run the application class ourselves.
Again, this is pretty simple. In our unit test, we'll need to add:
use Dist::Zilla::App;
And we'll need to change this line:
demand_successful_command("dzil build");
to this:
And define that command thusly:
sub run_dzil_command
local @ARGV = @_;
Dist::Zilla::App->run or die("failed to run dzil");
How simple can you get?
Dist::Zilla::App is expecting to find its arguments in
@ARGV, so we'll transfer
@_ there, but we use
local to make sure we only temporarily override the arguments to our unit test.9 Then we just run
run and call it a day.
One problem though: apparently
run doesn't return success when it's successful. I didn't spend any time trying to figure out why that was true; I just moved my
or die somewhere else:
chdir "$tname-$tversion" or die("failed to run dzil");
There: now, if the build fails, we won't get the build dir, therefore we can't
chdir to it, therefore failure to
chdir indicates the build didn't work. Simple enough. (Again, full version on GitHub.)
So, that was my recent adventure with
Dist::Zilla.10 Hopefully there's something in there that will be helpful to at least a few people. I'd like to tell you that this solved all my problems, but the truth of the matter is it didn't. Maybe now that I've figured out how to tell PAUSE about my
class, next time I'll tell you how to tell
Pod::Weaver about it.
As soon as I figure out how myself ...
1 Warning: Don't actually try to install my plugin bundle—it won't install for you. Still working on that. But I think the value of other people's plugin bundles is not so much in the installing as in the code perusal anyway.
2 If you need a good place to start learning about Dist::Zilla in general, try the official tutorial. Lots of good info there.
3 And this demonstrates (once again) why you too should be subscribed to the Perl Weekly. Go sign up now, if for some reason you haven't already.
4 But I'll probably switch over to Moops at some point.
5 In this case, David Golden's. Which I suppose is an odd coincidence, given it was his article that led me to find the problem here.
6 Technically, it doesn't matter where we go, since it only happens right before we exit, and the current directory won't persist past the end of our process. So I could have used
chdir '..' or even
chdir '/'. But this seemed politer/safer/more obvious. Although maybe that's just me.
7 'Cause, you know: that's how TDD works.
8 In this case, it's the
lib/ directory of a Perlbrew'ed verison of perl.
9 Likely that isn't strictly necessary, but you never know. It's one of those things that will be sheer hell to track down if it ever does cause a problem, so why risk it?
10 And dzil plugin bundles, and PAUSE, and Perl Weekly, and TDD, and CPAN Testers ...
Perl sure does seem to need a lot of scaffolding these days before one can get around to the business purpose.
And my response was that Perl had always needed a lot of scaffolding. It's just that we never used to notice, because it was all built-in.
Pretty much every Perl tutorial is going to tell you that the first line of your first Perl script should look something like this:
#! /usr/bin/perl
Now, sure: it isn't going to be that easy on Windows, or other non-Unixy systems, and even in the Unices there are going to be flavors where Perl is in
/usr/local/bin or somesuch, but that line actually works on a significant majority of the potential cases. And that's all you have to do to make your Perl program work. Perl is a compiled language, but you don't have to compile your Perl code (which is why it's commonly considered an interpreted language, even though it's technically not). Whenever you run your Perl code, it just magically compiles and runs, including finding all the executable bits and all the libraries and all the modules. You don't have to worry about compiling and linking, as you would with C++. For that matter, you didn't have to install Perl in the first place, as you might have to do with Python or Ruby. Nope, everything—all the scaffolding—is just there.
Sometimes things seem like really good ideas and aren't.
The first problem you inevitably hit when you're learning Perl is modules. CPAN is huge. It contains everything you could possibly want,1 and quite a few things you really don't. It isn't practical for Unix boxes to come with the entirety of CPAN installed. Plus CPAN is always changing: even if your favorite flavor of Unix did include all of CPAN, it would be out-of-date long before the install CD hit your laptop. So Unix distros have to pick which modules come standard. Perl itself has to choose what modules are considered "core." So you're always going to run into that situation where you need to install something from CPAN.
Happily this is not hard. The toolchain for installing CPAN modules is excellent. Oh, sure: we bitch about this part and that part constantly, but try intalling a Ruby gem sometime.2 Our shit is tight compared to that.
No, the problem is not getting it installed, but rather where to install it. See, it took us something on the order of a decade or so to figure it out, but we finally realized that installing modules into the system Perl area is a terrible idea. Modules need other modules, or upgraded versions of standard modules, and modules (or versions of modules) don't always play nice with each other. Managing all this module interaction is hard enough in a codebase of any significant size. Throwing system Perl into the mix just makes it crazy.
And on top of everything else, Perl is just so goshdarned useful for sysadmin-y things that lots of systems tools are built on it. So, if you end up breaking system Perl, sometimes you end up breaking your entire system. And, to add insult to injury, you rarely break it definitively. It often breaks in really strange ways that can be difficult to reproduce: different symptoms for different users, breaking or not depending on module load order, that sort of thing.
So we invented local::lib.3 And we started using it to solve this problem. Now we could have our own little library area, completely separate from the system Perl. For instance, here's the standard prefix that we've been using at my current company:
use strict;
use warnings;
use Cwd 'abs_path';
use File::Spec::Functions qw(catpath splitpath);
use local::lib catpath((splitpath(abs_path $0))[0, 1], '../extlib');
use lib catpath((splitpath(abs_path $0))[0, 1], '../lib');
use CE::Util::ScriptBootstrap;
As you can see, it uses only core modules (only modules that have been core since ancient times, even4), and it uses the directory of the script itself to find our non-system CPAN modules (
extlib/) and our corporate modules (
lib/). That way, if you happen to have two separate copies of the codebase (different branches, perhaps), each one finds its own set of modules.
Still, there are problems here. For one, it assumes that the script itself lives only one level down from the root of our codebase (e.g., in
bin/). What if it's two levels down, such as in
bin/util/? Well, then, you have to change the boilerplate:
use Cwd 'abs_path';
use File::Spec::Functions qw(catpath splitpath);
use local::lib catpath((splitpath(abs_path $0))[0, 1], '../../extlib');
use lib catpath((splitpath(abs_path $0))[0, 1], '../../lib');
use CE::Util::ScriptBootstrap;
And so on, the deeper you go. Then what happens if you need to move the script to a different level?
The other problem here is more subtle, although I'm sure some of you saw it right away. I'll give a bizarre error message as a hint:
File::Spec version 3.4 required--this is only version 3.33 at /var/local/CE/bin/../extlib/lib/perl5/Path/Tiny.pm line 13.
BEGIN failed--compilation aborted at /var/local/CE/bin/../extlib/lib/perl5/Path/Tiny.pm line 13.
What makes that bizarre is that, if you look at the version of File::Spec in our
extlib/, it really is 3.4. So where did 3.33 come from?
Why, from the system Perl, of course.
Because, you see, we loaded File::Spec before we used local::lib to redirect where we were pulling modules from.5 Here was one suggestion to fix the problem:
use strict;
use warnings;
use FindBin;
use lib "$FindBin::Bin/../lib";
use local::lib "$FindBin::Bin/../extlib";
use lib "$FindBin::Bin/../lib";
use CE::Util::ScriptBootstrap;
There. Pesky references to File::Spec all gone ... right? Nope. Not at all. In fact, not only does FindBin6 use File::Spec, so does local::lib itself.7
And don't forget that those "pragmas" at the top (i.e.
strict and
warnings) are just modules with a slightly different naming convention. As I discovered to my chagrin when I attempted to add a
use autodie to that list.
And on top of all that, we're still using the system Perl. Not the system Perl modules any more, but still good ol'
/usr/bin/perl, as the shebang line attests.
The shebang line ... the problem with that (as we mentioned above) is that it contains a hard-coded pathname, which may not always be where the system Perl is, depending on the system. But it gets worse if you decide you don't want to use the system Perl at all. See, system Perls have a tendency to lag behind the rest of the world: until recently, it wasn't that uncommon to have your system Perl turn out to be over a decade old.8 Lately, our beloved Perl has been getting some much needed improvements, catching up with those young whippersnappers that want to supplant us, but that does you no good if your system Perl is stuck in 2002. Unless you totally get rid of your system Perl, of course.
Enter Perlbrew (and plenv, although I'm not as familiar with that one). Now you can install whatever version of Perl strikes your fancy, and it probably won't hose your system, even. Of course that means you have to go back and change all your shebang lines. But whatever shall you change them to? Well, you can read all about the intricacies and debates on the Perlbrew web site, but the common concensus these days is this:
#! /usr/bin/env perl
which will fetch Perl from your
$PATH—that is, theoretically you'll get the same thing you'd get if you typed
perl at the command line. Again, presuming Unixoid systems, which likely encompasses recent versions of MacOS but not Windows. Of course, there are versions of Unix where
env doesn't live in
/usr/bin, but this works quite a lot of the time, and you'll find it's the most common incantation recommended these days in tutorials and blog posts.
Of course, it presumes that the right version of Perl is in the
Now, let's say I want to install Perl (via Perlbrew, or plenv, or what-have-you) into a directory in my corporate codebase. Like our
extlib/ above, but probably something more like
perl5/. I certainly can't put an absolute path in my shebang lines for my scripts, so I'm probably going to want to go with something like the
/usr/bin/env perl trick. Except ... how do I get the proper directory into the environment? In an ideal world, it's the same directory on every machine, but I learned long ago that we don't live in an ideal world. What if we need multiple installations on a single machine (hinted at above)? Then we certainly can't use the same directory for everything. There might be other reasons too (e.g. different environments, like staging or integration testing, might be more convenient if set up using different directories). So something somewhere has to set that directory—I know damn well that if I hardcode it everywhere, I will definitely live to regret that. Something has to put it in the path. Us humans can set up the environments for ourselves, but what about the web servers that run as pseudo-users? What about the cronjobs, which have little to no environments at all? What about the scripts that have to install all the directories in the first place?
One of my coworkers polled a group of devops folks somewhere, and the concensus was, use a bash script which sets up your environment and passes arguments through to the real Perl. Such a bash script would be invoked whenever you need to launch your scripts in a scenario where you can't guarantee that the environment would already be there (cronjobs, sudo, running remotely via
ssh, etc). That is, instead of
or even
perl bin/foo
now you're going to do
bin/launch bin/foo
instead.9 Okay, lovely. But it means you have to go through all your crontab files, and all your scripts that log into other machines and run other scripts, etc etc, and modify how they're all called. So it got me thinking ... if we're going to have to change all our shebang lines anyway, couldn't we just do all this in one fell swoop? I mean, couldn't we write something like this:
#! /bin/bash
exec $PERLDIR/bin/perl "$@"
and call it ... oh, I dunno, say
/usr/local/bin/invoke, and then change all our shebang lines to:
#! /usr/local/bin/invoke
The more I pondered this, the more excited I got by the possibilities. Not only does this give us one centralized location to fiddle with environment variables, but it also gives us a place to add Perl switches. We could turn on taint checks for every script at once with a 3-character change to a single file, for instance. Always load some module or other. Change it temporarily to generate profiling info. Whatever. This could be cool!
If it works ...
So first I checked to see if it should work, reliably, across enough different flavors of Unix. Meaning I Googled. A lot. And I found a page which told me that any reasonably modern version of Linux and bash should be able to handle this. I even found several examples of people doing it. Woohoo. So, next, I tried it.
And it just hung. Like, forever.
So, I added some debugging. And tried again. And promptly blew out my
screen buffer. So I tried again, a bit more cautiously:
[absalom:~] ./t.pl | head
+ echo 'args: ./t.pl'
+ exec /usr/bin/env perl ./t.pl
args: ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
args: ./t.pl
+ exec /usr/bin/env perl ./t.pl
+ echo 'args: ./t.pl'
+ exec /usr/bin/env perl ./t.pl
args: ./t.pl
+ echo 'args: ./t.pl'
What the hey?? It's like my bash script is just invoking itself, over and over again, and nothing in my Perl script (which also had debugging in it) was getting run at all. Hmmm ...
But then I remembered seeing this in the Wikipedia article about shebangs, and this in a random Stack Overflow question. Both of them go to some trouble to skip the shebang line when reprocessing the script. And, sure, neither one seems to have anything at all to do with the problem I'm trying to solve, but it's worth a shot, right?
#! /bin/bash
# have to skip the shebang line in the script we're running
# otherwise we get an infinite loop
script="$1" ; shift
exec $PERLDIR/bin/perl <(tail -n +2 "$script") "$@"
Boom. That works.
So I told my coworkers. Two of them immediately said I didn't need to do all that futzing around with skipping the first line, and one of them additionally pointed out that I was going to throw off all the error messages in our scripts by one. To which my responses were: 1) oh, yes, you really do need all that—trust me—and 2) oh, yeah ... good point. Sigh.
So, while thinking about how to change my magical incantation so it preserved the line numbers, I attempted to explain to everyone why it was necessary to skip the first line. Casting my mind back to a conversation from a few years ago, I remembered that Perl itself would pay attention to a shebang line. That has to do with being able to pass multiple switches to the Perl executable in the shebang line. In other words, the reason you can do this:
#! /usr/bin/perl -T -Ilib -w
or whatever10 probably has nothing to do with your version of Linux or bash, and only works because Perl re-executes itself with the proper switches after seeing the shebang line. So it has to be reading the shebang line, and conditionally doing something with it. So, maybe it has something to do with that, I threw out (totally grasping at straws).
My coworkers expressed doubts. What about good old
/usr/bin/env perl, they challenged? Why does that one work then?
"Uhhhhh ..." was my brilliant reply. Maybe there's a specific exception for
env? I followed that piece of infallible hypothesis with this one: "or else it might just be happy that the word
perl is in there somewhere." Immediately one of my coworkers changed the name of his script from
launch to
launch-perl or somesuch and bang! it worked.
Whoa, I said.
You mean I was really right about that? No way ...
Well, of course now I had to go find where this is documented. As it turns out, it's right there at the top of the perlrun man page:
If the
#!line does not contain the word "perl" nor the word "indir" the program named after the
#!is executed instead of the Perl interpreter. This is slightly bizarre, but it helps people on machines that don't do
#!, because they can tell a program that their SHELL is /usr/bin/perl, and Perl will then dispatch the program to the correct interpreter for them.11
So, at the end of it all, I'm not sure if this post is an interesting detective story, or a tip for avoiding an infinite loop gotcha should you ever try what I did, or just a long, meandering answer to my sysadmin's observation about Perl sure requiring a lot more scaffolding these days,12 but hopefully someone out there got something out of it. I thought the whole thing was pretty interesting, at least. And now I've shared it with you.
1 Okay, okay: not everything. As our dear language ages, we are finding more and more things that there isn't a CPAN module for yet. But you'll allow me a bit of hyperbole in service of a greater truth, I hope.
2 Not to pick on Ruby. Python can be just as much of a pain in the ass, and Node is usually even more so.
3 And by "we" I mean "mst."
4 Perl 5.004 was released in 1997, which makes it older than my oldest child. That counts as ancient times as far as I'm concerned.
5 Please note that I do not take any credit for figuring this out; smarter people than I at $work explained this to me, and now I'm passing it on to you.
6 Which you really shouldn't be using anyway.
7 Well, most versions of local::lib out there in the wild anyway. To be fair, the newer versions are more circumspect about loading File::Spec—for this very reason.
8 Yes, Perl 5.8 is nearly 13 years old, and that was standard on many distros. Probably still is, on some.
9 Let's conveniently ignore the question of how we ended up in the current directory in the first place. You can't ignore it in the real world, of course, but we can ignore it here, for now.
10 Except don't actually use
perl -w. It's inferior to
use warnings in many, many ways.
11 The "indir" bit is new as of Perl 5.16, although I can't seem to find any explanation of what it's there for. I did find the commit which changed the man page, hoping that its log message would also contain some reference to its function. But, no such luck. If anyone knows the reason for it, toss it out in the comments: I'd love to hear about it.
12 Yeah, okay, it's long and meandering either way: I know. I know.
cpanm, and either have root access on their machine or are already using
perlbrew or
plenv, and ... In other words, other Perl programmers. And, if that's your audience, then lovely. Although, even then ... what if you need a particular version of Perl, and particular versions of certain modules? It's all doable, certainly, and even moderately easy for Perl'ers of a certain experience level. But why should we limit ourselves unnecessarily?
What Sawyer X points out is that we have the technology: we have
perlbrew, and
local::lib, and
cpanm, and so on and so forth. We have ways to bundle everything you need to run a given Perl app, from the precise version of Perl to the preceise version of every dependency module, all in one place, without giving a royal shit what version of system Perl someone has, or what version of this or that module they have, or whether they can install modules or not (either because of access issues or inexperience), or any of that. We have all the pieces. We just need to put them together.
Imagine you were writing a program in, let's say, C++. You would create yourself a tarball with all the source code, and you'd include a
Makefile, and you'd put in some instructions on how to build it, and that's what you'd distribute ... right? You wouldn't try to create a binary distribution. Because it's not practical: binary packages have to be compiled for a certain OS and architecture and all that happiness. And you don't want to mess with all that. If people want binary versions of your software, someone will eventually make an RPM and a
.deb out of it, and there you go: binary distribution issues solved. Totally separate from you. Which is, I think, how it should be.
So I think that, instead of trying to come up with a way for the author to make a binary package, what should we come up with is a standard way to create a build process ... our version of the
Makefile. I mean, it could be an actual
Makefile, I suppose ... except that I utterly despise
Makefile syntax, and honestly if your language is not compiled it's rank overkill anyway. Anything a
Makefile can do, Perl can do better anyhow ... hell, even bash can do better. And of course if we use bash for most of the logic, we don't have to worry about what version of system Perl people are running, which was one of our concerns in the first place. This is how you install
perlbrew, right?
curl -L http://install.perlbrew.pl | bash
There's a simple little bash script, and you just download and feed it directly to
bash, and then it just ... works.
No reason another bash script couldn't download your app, install
perlbrew, brew up the proper Perl, and install all the exact versions of all the modules you need ... right? No reason at all, I say. So, here you go:
#! /bin/bash
set -e
set -o pipefail
# Author of the package should set these to appropriate values.
# End-user should not touch, unless you really know what you're doing.
# --or--
# Author should set a default value here:
: ${INSTALL_PACKAGE_TO:=/opt/yourproj}
# End user can override like so:
# curl -kL https://raw.githubusercontent.com/you/yourproj/master/build/install | INSTALL_PACKAGE_TO=~/opt/foo bash
# Or, not:
# curl -kL https://raw.githubusercontent.com/you/yourproj/master/build/install | bash
function die
echo "$@"
exit 1
function verify_program
if which $program 2>/dev/null
return 0
elif type $program 2>/dev/null
return 0
die "cannot locate program $program (or else can't figure how to locate things)"
function count_files
( # parends used advisedly
shopt -s nullglob
local c=("$dir"/*)
echo ${#c[@]}
if [[ $INSTALL_PACKAGE_TO ]]
install_dir=$(dirname $0)/..
if [[ ! -d $install_dir/perl5 ]]
die 'you must set $INSTALL_PACKAGE_TO before calling this script'
if [[ -e $install_dir ]]
if [[ ! -d $install_dir ]]
die "requested directory $install_dir is not a directory"
if [[ ! -w $install_dir ]]
die "requested directory $install_dir is not writeable (re-run with sudo?)"
parent_dir=$(dirname $install_dir)
if [[ ! -e $parent_dir ]]
die "can't make $install_dir because $parent_dir doesn't exist"
if [[ ! -d $parent_dir ]]
die "can't make $install_dir because $parent_dir isn't a directory"
if [[ ! -w $parent_dir ]]
die "can't make $install_dir because $parent_dir is not writeable (re-run with sudo?)"
mkdir $install_dir || die "can't make install dir (maybe disk error?)"
if [[ ! -w $install_dir ]]
die "made directory $install_dir but now can't write to it (hunh?)"
# need an absolute path
install_dir=$(perl -MCwd=realpath -e 'print realpath(shift)' $install_dir)
if [[ $(count_files $install_dir) == 0 ]]
# install directory appears to be empty
# let's fill it up
case $INSTALL_METHOD in
git-clone) verify_program git
git clone ${INSTALL_BRANCH:+ --branch $INSTALL_BRANCH} $INSTALL_URL $install_dir
download-tar) verify_program curl
verify_program tar
curl -kL $INSTALL_URL | tar xpzf - -C $install_dir
# tricky bit: if our directory contains nothing but another directory,
# move all the inner dir's files up to the top level and ditch the dir
if [[ $(count_files $install_dir) == 1 && -d $(echo $install_dir/*) ]]
( # subshell to localize the `shopt`
shopt -s dotglob
tar_dir=$(echo $install_dir/*)
mv $tar_dir/* $install_dir
rmdir $tar_dir
*) die "unknown install method: $INSTALL_METHOD"
unset PERL_CPANM_OPT
unset PERL_LOCAL_LIB_ROOT
unset PERL_MB_OPT
unset PERL_MM_OPT
unset PERL5LIB
export PERLBREW_ROOT=$perlbrew_dir
export PERLBREW_HOME=$PERLBREW_ROOT
unset PERLBREW_PERL
unset PERLBREW_VERSION
unset PERLBREW_CSHRC_VERSION
unset PERLBREW_PATH
unset PERLBREW_MANPATH
if [[ ! -x $perlbrew ]]
curl -kL http://install.perlbrew.pl | bash
source $perlbrew_dir/etc/bashrc
# let's just always do this
# that way, we're sure to have the latest version of cpanm
$perlbrew install-cpanm --force
if [[ ! -x $perlbrew_dir/perls/perl-$PERL_VERSION/bin/perl ]]
perlbrew install -n $PERL_VERSION
$perlbrew_dir/bin/cpanm -n --cpanfile $install_dir/build/cpanfile --installdeps $install_dir/build/
You would put this into a
build/ directory, along with a
cpanfile. Regular readers of The Perl Weekly1 may recall a blog post on cpanfile from over a year ago. In the comments, Reini Urban asks "Why should anyone use a 4th format to define dependencies?" To which the proper answer is, "they shouldn't." But
cpanfile isn't really for defining dependencies for modules in the way that the other 3 methods Reini cites are. What
cpanfile is good for is this exact situation: a Perl app that will be installed via some method other than the standard CPAN module toolchain. Plus
cpanm supports exact versions, which CPAN module installation doesn't (yet). Here's a simple/stupid example of a
requires 'IPC::System::Simple', '== 1.25';
requires 'Moose', '== 2.1403';
requires 'Path::Class', '== 0.32';
requires 'Const::Fast', '== 0.014';
Your main program, in the
bin/ directory, might look something like this:
#! /bin/bash
basedir=$(dirname $0)/..
export SHELL=/bin/bash # have to fool perlbrew a bit
export PERLBREW_ROOT=$basedir/perl5
export PERLBREW_HOME=$PERLBREW_ROOT
source $basedir/perl5/etc/bashrc
perl -I$basedir/lib -e '
use 5.14.0;
use warnings;
use autodie ":all"; # requires adding IPC::System::Simple to cpanfile
use App::YourProj;
' "$@"
And that's pretty much all there is to it. Now, this isn't perfect, but it almost completely works.2 It could be made even more expansible, of course—the idea of
build/ shouldn't be hardcoded in there, and maybe it should be smart enough to deal with
curl not being present and fallback to
wget—but this moderately simplistic script already supports two install methods, and doubles as an upgrade script if run from the directory it builds after initial install. It will only work for Unix-oid OSes, true, but I think it ought to work on all flavors of Linux, BSD, even OSX (probably).3 The only bits of Perl it uses are backcompatible enough to work on anything going back to 5.6 ... or even beyond. And, once installed, everything just runs self-contained, completely oblivious to any Perl verisons or Perl modules that may or may not be installed in other places.
It still wants more testing, of course, but it seems to work pretty damn well so far. It can download a tarball from GitHub (or anywhere, really), and you can get GitHub to give you a tarball of any tag you make. Or it can just clone a GitHub repo and let your user run the app right out of that. In the latter case, you can also specify which branch you want.4 And of course you can specify any Perl version you like.
I wonder if it might be worthwhile to turn this into a CPAN module ... or, better yet, a little Perl app. I could even use this script to deliver it to Perl application programmers. Sort of like how a compiler is generally used to compile itself.
Comments certainly welcome. :-)
1 And, if you aren't one of those, why not?
2 Specifically, the part that I know doesn't quite work yet is my
cpanm invocation. It still needs something, probably a
--self-contained. I'll be doing more testing.
3 And, if it doesn't work on any of those, I'll damn well make it work.
4 E.g. you could make yourself a
stable branch.
The other day I was working on my music library scripts,2 and I needed a menu for something. Now, there are oodles and oodles of modules on CPAN to help you write menus. I've looked at most of them, and tried quite a few, but long ago I settled on using the
-menu option in IO::Prompter, by the Damian. For a nice, pretty menu layout—say, something you do as a central feature for a program—it's tough to beat. It's not perfect, by any stretch, but it offers some very nice features, such as (optionally) not requiring
ENTER after a menu choice.
But that's not what I wanted in this case. What I was looking for here was a quick, compact menu ... sort of like what you get when you're interactively staging a commit in
git (that is,
git add -p, or, probably more commonly,
git add -i then choose "patch"). Specifically, the features I wanted were:
? to see what the options actually do.
So I want something quick-and-dirty, and I don't really want to spend a lot of time writing it either. I need to leverage as much as possible. Since I want to be able to respond to a single keypress (no
ENTER required), and since I know IO::Prompter does that already,5 I'll base my menu around that module. Here's a simple first cut:
func mini_menu ($choices, $prompt)
my @choices = split(//, $choices);
my $opts = join(',', @choices);
my $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
redo PROMPT unless $choices =~ /\Q$choice/;
return $choice;
(Notice we're using Method::Signatures here. I tend to do that a lot. Go figure.) So, this is pretty basic. Remember that splitting on the null pattern is just a quick way to get an array of the individual characters in a string. Then I put them back together with commas for display, call IO::Prompter's
prompt method with
-single (which is the "don't require
ENTER" part), verify that they typed one of the characters I'm expecting, then return what they typed. Note the
\Q in my verification regex: very important in case I want one of my options to be
* or
You'd call it like so:
my $choice = mini_menu( "arq" => "What do you want to do?" );
where maybe "a" means "add" and "r" means "replace" and "q" means "quit." Or whatever. Of course, the user doesn't know that, eh? We'd better add the "
? to get help" feature.
func mini_menu ($choices, $prompt, HashRef :$help)
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join(',', @choices);
my $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
if ($help and $choice eq '?')
say "$_ - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
return $choice;
That was pretty easy. We've already got an array of
@choices, so printing out what each one does is just a matter of getting that info from our caller and a quick loop through the array. And of course we add the help for "help" ourselves. This means that
? is now not a valid option for inclusion in
$choices, but we can just document that. A slightly bigger issue is that now we're having to include the choices twice: once in
$choices, and once as keys in
%$help. We could dispense with
$choices altogether and just use the
$help keys, but that would mean we'd have to make help mandatory (which I could live with) and we'd also lose the ability to control the order of our options (which I couldn't). Or else we could try to do something clever with passing in a tied hash where key order is preserved, but that adds a whole 'nother dependency, and makes calling the function a bit messier ... ah, screw it. We'll just live with the redundancy.
What else might we want? Well, I suppose we might want to use something other than comma as our option separator. Any time you're hardcoding a string in your code (even one as trivial as this one), you probably want to review that to make sure you're not locking yourself in unnecessarily. And it's trivial to fix:
func mini_menu ($choices, $prompt, HashRef :$help, :$delim = ',')
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join($delim, @choices);
my $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
if ($help and $choice eq '?')
say "$_ - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
return $choice;
(Yes, technically that hardcoded comma is still in there—we just moved it around. But the difference in intention is massive.) Nice.
Now, I said I didn't want a big, complex dispatch system for this. But how about a little tiny one, completely optional? This does add a layer of complexity, both in terms of code and in terms of interface, but I think it'll be worth it. Let's say that we'll accept an optional hashref of coderefs, and we'll call the coderef when the user picks the option. If the coderef is undefined, or if it returns 0, we'll come back to the user; otherwise, we'll just do the menu again. This will be quite helpful for the particular application I have in mind.
func mini_menu ($choices, $prompt, HashRef :$help, HashRef :$dispatch, :$delim = ',')
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join($delim, @choices);
my $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
if ($help and $choice eq '?')
say "$_ - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
if ($dispatch and $dispatch->{$choice})
if ( $dispatch->{$choice}->($choice) != 0 )
redo PROMPT;
return $choice;
There. Basic, but serviceable. What else, what else ... Oh, yeah. I can tell you from years of experience writing menus that sooner or later I'm going to want my menu to do something right before it prints the options. Before we added the dispatch system, that was something that could just be done before calling the menu function in the first place. But now that we have the option of doing some function then showing the menu again, we're gonna need a way to redo the ... whatever. Check some status, relist some things which were possibly modified by the dispatch functions, etc. Sort of a "pre-menu" function. Okay, let's jam that in there too:
func mini_menu ($choices, $prompt, HashRef :$help, HashRef :$dispatch, CodeRef :$premenu, :$delim = ',')
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join($delim, @choices);
my $choice;
$premenu->() if $premenu;
print "$prompt [$opts] ";
$choice = prompt -single;
if ($help and $choice eq '?')
say "$_ - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
if ($dispatch and $dispatch->{$choice})
if ( $dispatch->{$choice}->($choice) != 0 )
redo PROMPT;
return $choice;
Excellent. Except ... well, now we're going to call that pre-menu function again even when the user just hits
? to see the help. If the function takes any time at all to do its job, that'll annoy the user.6 Oooh, here's a clever way to avoid that: once the user hits
$choice is defined. So we can do the pre-menu thing only if
$choice is
undef. And then we'll just explicitly
undef it after dispatch.
func mini_menu ($choices, $prompt, HashRef :$help, HashRef :$dispatch, CodeRef :$premenu, :$delim = ',')
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join($delim, @choices);
my $choice;
$premenu->() if $premenu and not defined $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
$choice = "\n" if length($choice) == 0; # empty string means the user just hit ENTER
if ($help and $choice eq '?')
say "$_ - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
if ($dispatch and $dispatch->{$choice})
if ( $dispatch->{$choice}->($choice) != 0 )
undef $choice;
redo PROMPT;
return $choice;
This actually worked perfectly for my first couple of attempts of using it. Until I got a bit fancy and tried to include
SPACE and
ENTER as two of my menu options. Well, we have two problems with that:
-single mode,
IO::Prompter::prompt doesn't treat
ENTER the same as other keys. At first I thought it was trying to return a default, but that didn't turn out to be true.7 But, anyway, when you hit
ENTER, you get an empty string back. So, fine: we'll just catch that explicitly.
my %KEYNAMES = ( ' ' => 'SPACE', "\n" => 'ENTER' );
func mini_menu ($choices, $prompt, HashRef :$help, HashRef :$dispatch, CodeRef :$premenu, :$delim = ',')
my @choices = split(//, $choices);
if ($help)
push @choices, '?';
$help->{'?'} = 'print help';
my $opts = join($delim, map { $KEYNAMES{$_} // $_ } @choices);
my $choice;
$premenu->() if $premenu and not defined $choice;
print "$prompt [$opts] ";
$choice = prompt -single;
$choice = "\n" if length($choice) == 0; # empty string means the user just hit ENTER
if ($help and $choice eq '?')
say $KEYNAMES{$_} // $_, " - $help->{$_}" foreach @choices;
redo PROMPT;
redo PROMPT unless $choices =~ /\Q$choice/;
if ($dispatch and $dispatch->{$choice})
if ( $dispatch->{$choice}->($choice) != 0 )
undef $choice;
redo PROMPT;
return $choice;
Now, it could still stand a few improvements:
TAB and
mini_menu. Suggestions welcome.
$choices, the keys of
%$help, and the keys of
%$dispatch. Double icky. Still don't see a good solution though.
0 returns from the dispatch system and anything else stays in. Originally it was false and true, but that looked even worse:
if ( not $dispatch->{$choice}->($choice) ) Maybe it should be reversed ... ?
&$premenu and when not may be too clever for its own good. Anything else I could think of involved it calling it more than once place though. Although perhaps that's not so bad.
?. I personally can't imagine using anything else for the "help" feature, but someone's bound to disagree sooner or later.
BTW, here's an example of its use in one my MP3 tagging scripts:
mini_menu("ratsvc \n\$" => "What shall we do with it? ",
premenu => sub {
say '';
say $album->basename, ':';
say '';
system qq{ ls -C "$album" };
system qq{ check-picard "$album" };
help => {
r => "reset and reclean",
a => "reset artist sort order",
t => "reset title sort order",
s => "break title into title/subtitle",
v => "fix vocals frames",
c => "reset comments to v1 values",
"\n" => "check again",
' ' => "move on",
'$' => "go to a command prompt",
dispatch => {
r => sub { die qq{ RE-PROCESS "$album" }; },
a => sub { system qq{ sort-order "$album" A: }; 1 },
t => sub { system qq{ sort-order "$album" T: }; 1 },
s => sub { name_tweak qq{ subtitle "$album" }; 1 },
v => sub { system qq{ comments-from-vocals "$album" }; 1 },
c => sub { system qq{ comments-from-v1 "$album" }; 1 },
' ' => undef,
"\n" => sub { 1 },
'$' => sub { system("bash"); 1 },
In this case the
die is just a way to throw control back to the script which called this one. Everything else is mostly self-explanatory. (Knowing blogs.perl.org's tendency to mangle code snippets, I've put together the final code and its sample call into a gist.)
I'd love to get any comments and/or thoughts and/or suggestions.
1 You can see it online, right about 47:59.
2 I actually do quite a bit of work on these, although not all of them are as pretty as I'd like. I've been pondering doing another big series on them, since they embody quite a lot of things I love about Perl, from the quick-and-dirty to the opportunity for elegant class design. But I've not gotten organized enough to do it yet. Maybe one day.
3 This is one of my few complaints about IO::Prompter. With that, your first option is
a and your second option is
b and so on. You can't change that.
4 Yes, that's technically different from how the menu works in
git. But I like this way better.
5 And also since I'm already using IO::Prompter for many things in most of my scripts, so I won't need to add another dependency or another
use line.
6 Remember: in this case, the user is me. I try not to annoy myself any more than absolutely necessary.
7 Possibly that's a slight bug in IO::Prompter, or possibly it's a misunderstanding on my part. Not sure which.
I'll freely admit that I didn't contribute that much, mainly on account of me not being rich. As I'm fond of pointing out here, I'm just a regular Joe working-class programmer. But, at the same time, Perl has been very good to me throughout the years, and I'm not averse to giving a little back when I can. And, as it happens, right now I can.
Now, I know that many of you readers out there have a healthy skepticism when you think you're being sold something. Trust me: I do as well. Now, I personally think it would be a fairly uncharitable characterization to view scrottie's proposed project, as an attempt to get us viewers to pay his bills for him while he contributes to an open-source project, which is something that many of us already do for free ... but I'm sure some out there have already characterized it thus. As is their right. I personally think there's more going on here, and, while I won't be able to change the minds of the most cynical among you, I'm going to try anyway. I'm a glutton for punishment like that.
So that got me past the "here's another dude begging for money" thing. Then I took a look at some of the details. His goal is $10 grand. Now, that might sound like a lot of money at first. But, as scrottie explains:
The Goal on this project gives me about six months of modest living to work with. I can get a lot done in six months.
Now, I can't live off that amount for six months. That amount doesn't even pay my mortgage for six months, much less put food on the table for the family. I'm betting there's a lot of you out there who are in a similar situation. Perhaps Scott doesn't have kids, like I do, and I do recall that he lives a more reasonably priced part of the country than I do. Still, living off just over $1,500 a month is an impressive feat in my eyes.2
Next, there's the general concept: resurrect a Perl CMS to destroy the ever-annoying WordPress. Now that's a worthy goal. If you've ever had to deal with WordPress, or have had friends who have, you know that WordPress is one of those products that is awesome ... as long as you want to do everything the way it wants you to do it. When you start wanting to be different, it becomes a huge pain in the ass. Plus, even if you love WordPress, you gotta admit one thing: it's written in PHP. I feel about PHP the same way I felt about web forums after having used newsgroups for several years: someone wrote a much crappier version of something we'd already had for years, and now people love it? WTF???
Of course, like web forums, the popularity has nothing to do with the quality of the product. It's the delivery mechanism. PHP is easy to deploy. WordPress, for all its warts, is easy to deploy. Things written in Perl ... not so much. This is part of the challenge that we as a community are faced with. If you were at YAPC (or if you took my advice from last post and watched it online), you heard Sawyer X say the same thing in his brilliant talk "The Joy in What We Do." Sawyer X had some suggestions for tackling that problem in a general fashion. What scrottie is doing with his Kickstarter is a more specific version of the same thing.3
And this gets to the heart of why I decided, in the end, to support scrottie's project. Because I believe he has it right when he says
The best way to create demand for Perl is to have desirable, plug-in extensible applications out there written in Perl.
It's something we need to encourage. No, more than encourage: foster. And, when you foster something, you assume some financial responsibility for it. If you were to foster a child, or a dog, or an au pair, that would be a pretty significant investment, of both time and money. This ... this is easy. I get to drop some cash on Scott, and he does all the work.
Assuming he hits his goal. If he doesn't, I've lost nothing. If he does, I'm out a small amount of cash. I'll refrain from going out to eat a few times and that'll be that.
Of course, looking back on that last sentence, it occurs to me I sound like Sally Struthers or somesuch. This is bound to tweak the healthy skepticism of many of you out there. Which, as I say, is good. Being a skeptic in today's world is a basic survivial skill. But also remember that being a skeptic doesn't mean rejecting everything. It means not accepting anything without giving it a rigorous investigation first. I've given Scott's project idea what I feel is a rigorous investigation, and I hope you do too. In the end, I decided it was worth a few dollars, and I hope that you will too. But, if you don't, that's all right. I've no desire to make you feel guilty about anything (as if I even could).
All I'm saying is that I feel like doing things to foster projects which make Perl accessible again, to a modern audience, is something that falls on all of us. I wish I personally could do more to do it. But I never seem to find the time to accomplish all the things I want to. Scott is saying he has the time. He has the background, the ability, and the knowledge to do it. He just needs a little cash. That works for me.
If you read my "Perl and Me" series, and especially the final installment, you'll recall that I do accept the inevitable truth that Perl cannot last forever. Acceptance, however, is not the same as surrender. My acceptance of the fact that one day I'll have to let go of Perl doesn't mean that I have to pass up any opportunity to stave off that outcome today. I'll take that opportunity, and encourage others to do the same. If we're lucky, we'll end up with something that gets people excited about Perl again, even if it's only one market segment. That's worth a few bucks to me.
1 Probably a bit younger. Or else he just wears his years far better than I do.
2 Although, given the all-or-nothing nature of Kickstarter, I personally think he should have set a lower goal—say, $5,000 for three months—and then set stretch goals for adding more months. But that's just me.
3 What would be really awesome would be if Scott can somehow integrate Sawyer X's suggestion on Perl project installation into his update of WebGUI.
General bits and bobs: I thought the venue was pretty decent. It's definitely the best hotel showers I've ever had: excellent water pressure and never even a hint of running out of hot water. Perhaps they have on-demand heaters. I could have used more open plugs in the room, but that's a minor complaint. The wifi was pretty awful, as usual—one person even did a lightning announcement suggesting we complain about it. I didn't see the point, personally. Several hundred geeks, most with multiple devices, plus several dozen spouses, partners, and children,1 all descending on a single location for four days ... hell, that could break anybody's router. And I don't think it's particularly reasonable to ask a place that can comfortably house all those people to maintain a super-network year-round that will only really get used once a year. Just point your laptop at your cell phone instead and move on with life, I say.
As always, I met several new folks this year. A special shout-out to Kevin, Alan, and uncanny_kate, who shared former employers with me and were willing to commiserate.2 I also had a great conversation with scrottie, a game of Munchkin with two German fellows named Thomas and Alex, and got introduced to Ann.3 And I got to see plenty of other folks that I'd had fun hanging out with at previous YAPC's, including the Slashdot guys (2/3 of whom I also share former employers with), and spq_easy, who I helped interview once at a former employer. Socially, it was a ball, despite the fact that many of us geeks (myself included) can be painfully shy in many social situations.
Talk-wise, there were good and bad sides. On the bad side, I missed some of my favorite speakers, like Piers Cawley, Ingy, schwern, and chromatic. On the other hand, I got to see mst speak twice, which is always entertaining, and Sawyer X twice, who is rapidly becoming one of my new favorite speakers. If I had to pick one talk as the absolute best of the conference, Sawyer X's "The Joy in What We Do" wins it, hands down. If you choose only one YAPC video to watch because you couldn't make it this year (and you're silly enough to listen to my advice), choose this one. It was the best keynote of the conference.]]> Yes, we had multiple keynotes. Two or three a day, even. I thought that was a bit weird. My friend David Hand kept joking that we had gone beyond keynotes and were now into keychords. Almost enough for an entire keyscale, even. One of them, of course, was Larry's State of the Onion, which was pretty good, and one was mst's State of the Velociraptor, which was also pretty good, although he did have to go and recycle part of his speech from one of the few other YAPC's I'd actually already attended.4 One was a mostly non-Perl talk by a fellow named Charlie Stross, who is apparently a Hugo-award-winning scifi author, although honestly I felt the most exciting part of his Wikipedia article was finding out that he invented the githzerai and the githyanki. His talk was pretty good too, as was Mark Keating's, who is also a very entertaining speaker. But I thought the final two keynotes—Sawyer X's that I already raved about, and genehack's, which I'll come to in a moment—were the two best.
Outside the talks, I once again had a blast at Game Night and the Bad Movie BOF. I wish they hadn't both been the same night this year. At Game Night, we played Munchkin for an hour with me, my son, two former co-workers, and the two aforementioned German gentlemen. All were new to the game except my son and I. It was utterly insane (which, if you know anything about Munchkin, is not news to you), and at one point we were all sex-changed and I found myself playing with Charlene, Xena, Isabel, and Jemima.5 Once that finally finished (with Thomas coming up from behind to win it at the wire), we adjourned to Bad Movie Night, already in progress with the classic Godzilla vs Megalon, which I hadn't seen in ages. After that was over, we watched a few shorts while we waited for people to decide whether they wanted to see a whole 'nother movie or not. There was something about pork, which was strangely terrifying, and something about a "creeps machine", which was strangely not, and then something about the "spirit of music", which was just wrong. Finally, we settled into Santa and the Ice Cream Bunny, which was basically so bad as to be indescribable. Santa was definitely in it, although his sleigh was stuck on a beach for some reason, and then Tom Sawyer and Huck Finn showed up for some reason, and then we were interrupted to show scenes from an amusement park for some reason, and then we were suddenly watching the story of Thumbelina for some reason, and the ice cream bunny never did show up. Well, he did at the very end, so I heard later, but the boys and I couldn't make it that far.6 At 1:30am, it hurts your brain too much to try to follow that "plot."7
So all that was lots of fun.
Deeper Thoughts: This year, as we did last year, we spent a fair amount of time convincing ourselves that Perl is not dead. Mark Keating in particular did a fine job of boosting morale on Day One. I do think it worth noting that the mere fact that we need to try to convince ourselves that Perl is not dead is a sign of something, although I suppose I'll leave it to other, wiser heads to attempt to figure out exactly what.
But this year we also had another interesting meme which is a bit of an offshoot of that: is YAPC itself dead? or perhaps merely in danger of dying? On the one hand, it seems not, since conference attendance has just about doubled since 2009.8 But, on the other hand, genehack made an excellent point in his talk: I do remember how many people raised their hand in Austin when asked if that was their first YAPC, and, comparing to how many raised their hands in response to genehack's query if this was their second, it does seem hard to deny we may have a retention problem.
This year there was an entire track called "Awesome && !Perl", which featured talks on such topics as Scrum, devops, Selenium, cognitive linguistics, and brewing your own beer. You know: stuff that isn't technically Perl, but is probably of interest to a significant subset of Perl hackers. For next year, may I suggest talks on TDD, creating virts on AWS, Pathfinder for D&D grognards, and maybe get dha and DrForr to demonstrate how to sync up your RiffTrax with the original movie on Linux. What genehack pointed out is that such talks may help broaden our appeal, and possibly attract some new blood to YAPC. Maybe even attract some not-JAPHs.
And this was the tricky part of his talk. Which, as I say, was good (and definitely my second highest recommendation for a talk video to watch if you missed it) ... but that doesn't mean I agree with everything he said. Although I can't actually disagree either. I'm seriously torn on this one. Because, first of all: OSCON. A brief detour on my personal conference history: There are many reasons that I first learned Perl in 1996 but first attended YAPC in 2011, and most of them are pathetic. But probably the biggest (most non-pathetic) one is that it took me till 2009 to find an employer willing to spring for my tickets. So why wasn't I at the 2009 YAPC? Because I was at the 2009 OSCON instead. Silly me: I thought that was the best conference for Perl because it was the first: remember, YAPC is called "Yet Another Perl Conference" because the original name for OSCON was "The Perl Conference." But I found that I was bored much of the time ... I enjoyed talks by the Damian and Ingy and RJBS, but there just wasn't enough Perl there, and too much other crap I didn't much care for. Plus OSCON is stupidly expensive compared to YAPC ... sure, I wasn't actually paying for it, but I felt pretty damned guilty getting my employer to pony up that kind of cash for a conference I wasn't getting that much out of. In contrast, I found YAPC to be interesting, engaging, and pleasantly cost-efficient.
So, on the one hand, I'm leery of genehack's suggestions if they might eventually lead YAPC to fall down the rabbithole that OSCON has stumbled into. But, on the other hand, I can't refute the logic of his introspections: have some talks on topics that engage Perl-ers, but also may attract some non-Perl-ites, and suddenly we're cross-pollinating and bringing new blood into our community. Why the hell shouldn't we pursue that? It's like, win/win/win. Or something. And, I also have to admit: it makes sense to be thinking about this now, while our numbers are up, instead of waiting until we start to see a decline and then try to scramble to fix things.
In essence, I believe genehack wants Perl (and YAPC in particular) to pitch a bigger tent. And I guess I can't argue with that. As long as we tread carefully to insure that we keep our uniqueness. I don't want to try to emulate Python or Ruby. They, after all, started out emulating us. Let's try and keep it that way.
Closing Thoughts: I didn't speak this year, mainly because last year was quite the crappy year in terms of $work.9 This year is shaping up to be better, I feel, and I have a few good thoughts for topics for next year already. But, even though I was just another schlub, a couple of people made a point of coming up to me and complimenting my writing, including ribasushi, whom I'd never had the pleasure of meeting before. On top of that, David Farrell from Perl Tricks was kind enough to throw me up in a slide in his lightning talk, where I was called an "essayist." Check it out, and especially note the company I'm keeping up there on the board. I'm very flattered by that, and probably not entirely deserving, but I really want to thank all those folks who had kind words to say about my writing. Your encouragement is noted, and appreciated.
Likewise, I would like to echo the sentiments of esaym, who posted on the YAPC mailing list today, thanking the volunteers who organized this year's YAPC. I'm sure you don't get to hear it enough, so let me add my voice to the chorus: your efforts are so valued. For many of us in the Perl community, YAPC is the highlight of our year, and it couldn't be possible without a lot of blood, sweat, and tears on the part of a lot of people that most of us generally don't even recognize. Please accept my deepest gratitude.
As always, looking forward to next year!
My reflections on my first YAPC are available on my Other Blog. Reflections on my second YAPC are here on this one.
1 And you can bet even the children in this group are using wifi connections. I know mine were.
2 This essentially means we sat around and had extended bitchfests. But that can be cahtartic.
3 I suspect she may not have been the only Ann there. But she was the only one I met.
4 The latter half of his keynote was from Asheville in 2011.
5 I'm terribly sorry, but I forgot Alex's nom de femme.
6 My first YAPC, I brought no one. Last year, I brought one child. This year, two children. Next year I suppose I'll have to bring three, and, for the year after that, I suppose I better get busy having more children.
7 And I'm being very generous in calling it a "plot." In fact, some might say that I'm blatantly lying in doing so.
8 For a nifty graph, see TPF's prospectus; click on the "Conferences and Outreach" header. I think Mark Keating made use of that graph in his talk. If he didn't, he probably should have.
9 At least up until I found my current position. But, even then, a new job can suck up a lot of free time as you try to find your footing.
Anyways, maybe next time, if I have more time to prepare. For now, you'll have to settle for plain ol' me.]
At $work, I'm the guy who maintains the tool we use for branching, merging, pushing ... release management, I suppose you'd say. I don't actually do the pushes most of the time, but I'm usually the guy who takes the tickets when there's something wrong with the process.
At the end of the push, we build a new AMI. That's Amazon-speak for a master virt which we can clone to bring up new production machines quickly. By making a new one after every push, any new instances we bring up will automatically have the latest code on them. Recently, my coworker doing the push reported that the AMI build failed. Well, he clarified: it says it failed, although it actually looked like it succeeded. Probably a transient error, I assured him (with fingers crossed). But, no such luck: the next week, the same thing happened. This time my coworker figured out what underlying command the release management script was running and ran it by hand. Yep, it definitely succeeds, he reported. It just says it failed.]]> And so it went for the next three pushes. You know how it is ... it's irksome, but there's nothing actually broken, so why waste time fiddling with it? Finally someone decided to make a ticket for it. I spoke up in tech chat: you know guys, probably someone changed something with the underlying command so it doesn't exit cleanly. The poor release management script is stupid; all it does is run the command, then check the exit value. Non-zero, report failure. I bet anyone could fix this with a bit of time and some Git archaeology.
Two more pushes with people grumbling about it and I finally just grabbed the ticket myself. I found the exact date of the first push where it happened (and that's why you always log your tech chat, people), then started poking around the Git commit logs for any changes just before that date.
As it turns out, another coworker had decided to refactor the command entirely, moving the main logic out of the
bin/ script and into its own class. Well, nothing wrong with that, I thought ... sounds like a pretty good idea to me. The guts of the script was reduced to dealing with configuration, environment, and options, and then this code:
Well, nothing much to see there, I thought. Better look where all the action is and see what's really going on:
my $command = CE::Command::ec2_launch_instance->new(
# bunches o' parameters here
exit $command->success ? 0 : 1;
Hunh, I thought. I still don't see it.
sub execute {
my $self = shift;
# do some stuff
return unless $self->wait;
# more stuff, including creating $manager
if (not $manager->all_success) {
# a wee bit more stuff
Do you? I'm sure some of you out there spied it instantly and are even now nodding your heads in commiseration. If you didn't spot it though, go back and look. I promise you, I gave you enough of the code to figure it out ... nothing I left out is relevant to the problem. Go ahead and ponder it for a bit. I'll wait.
Sure you want to peek at the answer?
Really sure?
Okay, I'll tell you what it was.
I was all set for a marathon debugging session ... I figured I'd have to run the whole stupid script a bunch of times, adding more and more debugging. Or maybe (<shudder>) actually fire up the debugger.* Which wouldn't be so bad, except building an AMI takes a fair amount of time, so it's not the sort of thing you really want to be doing over and over again until you winkle out an obscure bug. Especially for a bug where nothing's actually broken.
But then I had a nagging hunch. I wish I could tell you what triggered it, but I honestly don't know. Probably something I read somewhere a long time ago ... and this is why you must read about Perl (or whatever your language-of-choice is). Read the manuals, read the books, read the blogs. Go download Modern Perl. Subscribe to Perl Weekly. The more you know, the better chance that, when you hit that weird bug, something's going to tickle the back of your brain, something's going to stick in the sewer grate at the bottom of your mind, something's going to blink a little light way off at the periphery of your vision, and the little light will say ...
Wait. Go back a bit. What was the middle thing?
Although my forebrain told me to concentrate on the
execute method—that's where all the action is, after all—something made me glance back at the script itself. Remember this part?
Even more specifically ... this part?
my $command = CE::Command::ec2_launch_instance->new(
# bunches o' parameters here
exit $command->success ? 0 : 1;
Something wrong here ... maybe
exit $command->success ? 0 : 1;
exit can't take an expression?
Well, so much for that theory. Still ... something ...
[absalom:~] perldoc -f exit
exit EXPR
exit Evaluates EXPR and exits immediately with that value. Example:
Could this be one of Perl's weird precedence things? I mean, I know ternaries are pretty low in the precedence list ... let's look:
What's that thing roughly in the middle there? "named unary operators"? What are the chances, do you suppose, that
[absalom:~] perldoc op
PERLOP(1) User Contributed Perl Documentation PERLOP(1)
perlop - Perl operators and precedence
Operator Precedence and Associativity
Operator precedence and associativity work in Perl more or less like they do in mathematics.
left terms and list operators (leftward)
left ->
nonassoc ++ --
right **
right ! ~ \ and unary + and -
left =~ !~
left * / % x
left + - .
left << >>
nonassoc named unary operators
nonassoc < > <= >= lt gt le ge
nonassoc == != <=> eq ne cmp ~~
left &
left | ^
left &&
left || //
nonassoc .. ...
right ?:
right = += -= *= etc.
left , =>
nonassoc list operators (rightward)
right not
left and
left or xor
exit is a "named unary operator"?
Pretty good, as it happens.
[absalom:~] perl -le 'exit 2' ; echo $?
[absalom:~] perl -le 'exit 1 ? 2 : 3' ; echo $?
You mean this
would fix it?
exit($command->success ? 0 : 1);
Yep. That fixes it.
Well, lesson learned and all that, and maybe this will help someone else out there, but I also wonder if maybe the man page could be clarified a bit. I can't help but think that this:
is a bit misleading. Sure,
[absalom:~] perldoc -f exit
exit EXPR
exit EXPR ... but only if
EXPR consists only of terms and operators higher in the precedence list than "named unary operators." That seems like a bit of a gotcha.
Anyways, mystery solved, and here's my commit message:
[P.S.: I hope to see many of you at YAPC in a week or so. If you see a short dumpy guy wandering the halls with a scruffy beard and no shoes, walk over and say "hi!"]
Author: Buddy Burden <firstname.lastname@example.com>
Date: Wed Jun 4 15:06:49 2014
oh look: named unary operators have a higher precedence than the ternary <sigh>
* You may recall from previous posts that I hate the debugger. Part of the reason is that I never reach for it unless I have a really weird problem, and really weird problems are not the debugger's forté. In the past several months, I've only tried to use it twice, and once it worked like a charm, and the other time it did something completely different (and even wronger) than I got when I just ran it straight. Actually, 50/50 is a better record than I usualy have with the stupid thing. But I digress.
This week we look at what comes next.
I suppose at the end of any long rumination on the past, it's only natural to think about the future. I've tended to avoid pontificating on Perl's prospects for a number of reasons, not the least of which is that I don't have any special knowledge to share. I'm not an author of Perl books, a contributor to the Perl core, or even a particularly prolific creator or maintainer of CPAN modules.1 I don't have any inside knowledge of Perl from hanging out with Larry, or the folks that run the Perl Foundation, or the current Pumpking (nor any past ones, for that matter). I'm just a regular Joe JAPH—admittedly one who has made his living off Perl for 17 years, well enough to support a nice house in southern California and a complement of five humans, three cats, one guinea pig, and a tank full of tropical fish, snails, and shrimp—but just a working schmoe nonetheless, whose opinion isn't any better or smarter or wiser or more likely to be true than yours. So, you should definitely not listen to me.
But, as time went on, my enthusiasm cooled a bit. This was partially because I didn't always agree with the decisions of Perl 6. For instance, I'll freely admit that I'm one of those people who could never quite get over throwing out a perfectly good concatenation operator just because all the other languages were jumping off a cliff. I'm also not sure that the change in the way sigils work is actually an improvement, although at least for that one I can see both sides. Then there's twigils. I've said it before, and I'll say it again: I hate twigils. But these are trivial complaints overall, and completely overwhelmed by things that I'm drooling over: declarative classes and function signatures and multi-methods and optional strong typing and junctions and pipe operators and lazy lists and alias binding and macros and custom operators. Oh, and not having to type
qw so much. Oh oh, and not having to type
(?: ) ever again. Oh oh oh, and ... I could go on, and on, and on. There are so many things I love about the design of Perl 6.3
But then there are the implementations of Perl 6, which are decidely less inspirational. The delays in getting The Butterfly into a usable state are interesting, from an anthropological perspective. There were the things which were completely unavoidable (such as Larry's health problems) and those which were completely avoidable (such as the Parrot debacle4). Some setbacks have been blamed on the fact that the project is being implemented by an all-volunteer team, but that's true of any open-source project, no? Decisions were made (such as encouraging multiple implementations, or not using the existing CPAN structure for Perl 6 modules) that we can still debate whether they were vital or detrimental. But, while such ponderings are academically intriguing, arguing about why it's not ready doesn't help much. Let's face it: if we had been told, as children, that it was "almost Christmas" for eight years, we'd all have slit our wrists and we wouldn't be here having this discussion right now.
The Rakudo folks have decided to start calling Star a "useful and usable distribution of Perl 6." This attempt to reframe the perception by rearranging the labels is understandable, because how can we answer the question "is it finished yet?" when there's no clear agreement on what "finished" entails? We could say it's finished when it passes all its tests, but then we'd have to know when all the tests are finished. Besides, I can easily imagine scenarios where I could be using Perl 6 without all the tests passing, and also scenarios where all the tests pass but it continues to be useless in a practical sense. See, it all comes down to CPAN ... you want to know when Perl 6 will be "finished" (or close enough for government work, as my grandmother used to say)? Easy: when we have the Perl 6 equivalent of DBI, and 10 or so drivers to connect to real databases with. I can live without large swaths of CPAN, but I can't live without that.5
So when people ask me if I believe Perl 6 is the future, I'm never quite sure what to say. I'd like to believe it is—or at least that it could be—but, as pretty as it is, as many features and improvements as it touts, as beautifully as it may embody context and linguistics and evolution ... if I can't Get Shit Done with it, it ain't Perl as far as I'm concerned.
But turning from The Butterfly to The Velociraptor doesn't leave me any less conflicted. On the one hand, I know intellectually that Perl is a 25-year-old language, and it only makes sense for it to be on the decline, and that my desperate attempt to pretend that it's not must be viewed from an outside perspective in the same way that I viewed the COBOL grognards of my youth. When trying to look at myself objectively, I can see that I'm obviously biased and my opinion shouldn't be trusted. And yet ... it's also true that Perl has already lasted far beyond what could be reasonably expected. Other scripting languages have been born, flared briefly, and died in less time than Perl has been alive and kicking. Even Python's popularity is declining according to some measurements,6 and you may recall that Python proclaimed victory over Perl because it was engineered rather than artistic. (Now that I ponder it: With a few exceptions, projects that are engineered inevitably become obsolete. Art, however, is forever.7) Against all odds, Perl continues to evolve, CPAN modules continue to be uploaded, and new sites continue to be created in Perl. There's an echo chamber argument to be made, there's a confirmation bias argument to be made, and probably all sorts of other reasons I should not put faith in these things. And, yet, I continue to do so.
What it comes down to for me, I suppose, is what I said back in Part 7: there just isn't anything that competes with Perl. I'll happily upgrade to a better language ... as soon as I find one. It has to be expressive, and it has to be creative, and it has to be legible, and it has to be multi-paradigm and flexible and above all it has to make it easy to Get Shit Done. Where is the language that does all that? Oh, wait: it's Perl. Yeah, suddenly I remember why I don't switch. I'm not stubbornly refusing to move on because I'm nostalgic, or because I'm too old or lazy to learn something new, or because I'm a cranky old man who believes these kids today are taking the world to hell in a handbasket8 ... I'm just honestly unaware of anything worth switching to. I'm not going to switch just to switch, and I'm not going to abandon the best language I've been able to find in 26 years of searching just because it might be tough to find a job in it one day.9
At least year's YAPC::NA, there was a running theme, if not exactly a concensus, that the best way forward for Perl was for the experimental offshoots (perlito, p2, et al.) to try out new features and let the evolutionary winners be backported to Perl. I wrote about this, and also about p5-mop, which could be considered another one.10 I'd like to tell you that I too believe this is the future of Perl, but I can't say that yet. Again, I want to believe it, but let's stack up a bit more evidence before I begin to state that with any confidence.
I suppose I'm not really sure what the future holds for Perl. Recently, I received a color brochure from the Perl Foundation (as I'm sure did many of my fellow YAPC attendees). My partner11 asked, in some confusion, "what did they send that to you for?" I told her I wasn't really sure. It didn't seem to be begging for money, as perhaps might a letter from the alumni association of your old school, although I think statements pointing out the generosity of TPF donors were a respectful and gentle reminder that they could always use a bit more generosity. After pondering it for a bit longer, I opined that it was just a way to declare "Perl's not dead!" Much like JoJo Who's "yopp," it will save us from the cauldron of irrelevance by allowing the rest of the world to hear our cries of "we are here!" At least it reassures us Perl programmers that someone is out there fighting that fight for us. I haven't yet decided whether I will allow myself to be reassured, or whether I will absorb the chilling fact that TPF considers such reassurance necessary.
In the end, I suppose there isn't a clear-cut answer. Perl cannot last forever: nothing does. Larry Wall knows this, and Perl 6 is his attempt to turn the velociraptor into an archaeopteryx—to evolve us past the inevitable dead end by creating a whole new creature who shares some of the same DNA but can adapt to a changing environment. If Perl 6 can't be the answer, perhaps some other new language will be. But, damn: somebody better get to inventing it, tootsweet. If all else fails, perhaps Perl 5 can evolve itself beyond its shortcomings. But, honestly, that's a long-shot. Sure, there are some species that have survived mostly unchanged since the Cretaceous (like crocodiles, or sharks), but they're definitely the exceptions. Which is not to say we shouldn't keep trying. But I think we all know that, in the long term, we can expect some other language to come along and displace ours.
Except ... how long of a "long term" are we talking here? 'Cause I still say it's difficult to get all disturbed by the thought of Perl being replaced when currently there's zero prospects of anything that can replace Perl. Whatever timeline we're talking about, it continues to look like the first phase—invention of radical new language which will supplant us—has yet to occur. In the meantime, CPAN modules continue to be created, Perl conferences continue to break attendance records, people around the world continue to make a living doing Perl training, and I continue to sleep well at night knowing that even if my current job disappeared tomorrow I have several decent backup prospects, and a couple of good recruiters that specialize in Perl placements.12 For now, that's good enough.
I want to thank you all for coming along this journy with me, and for all the thoughtful, intelligent comments I've received, and for all the private encouragement. I hope some of it has been useful, or at minimum entertaining. May your own personal journey with Perl be as eclectic and rewarding as mine has been for me.13
1 Although of course I have a few.
2 Which wasn't that long ago ... in geological terms.
3 Although Perl 6 design changes rapidly. For all I know, some of those things I listed are not even planned any more.
4 I don't claim that chromatic's description of that clusterfuck should be taken at face value, but it remains in my opinion the best account to try to understand it if you were not personally involved, in that it a) is complete, b) is internally consistent, c) makes sense of the observed outcome, and d) doesn't contradict any other account that I've read that I didn't find too self-serving to put much stock in.
5 As an alternative plan, I would accept a way to use Perl 5 modules in Perl 6. That would be even better, actually, as it would mean we don't have to give up any parts of CPAN.
6 Tiobe, for one, which shows a more-or-less steady decline since its peak in around 2011. Side note: it's extremely entertaining to watch the Python advocates who crowed gleefully when Tiobe proclaimed them awesome now backpedal furiously and claim that maybe Tiobe isn't really a reasonable measurement of popularity after all.
7 I allow some exceptions on general principle, out of distaste and distrust of absolute statements. The only one that springs to mind, though, is the Great Pyramids, and I think you could make a reasonable argument that the original purpose they were engineered for is now obsolete, and that they are now, in fact, art.
8 Although I suppose I won't deny being any of those things either. Well, except for that last one—I'd like to think I'm a bit more progressive than that. But, point is, even if I am any of those things, that's not why I refuse to let go of my Perl.
9 Although some folks have said that this is already happening: for instance, chromatic wrote about it recently. Personally, I recently changed jobs and didn't experience any difficulties finding enough job prospects to have a choice of which one to take. Perhaps, to quote chromatic, I "got very lucky." But I prefer to believe that it means that there are at least a few geographical areas where Perl jobs can be found without the need for superhuman effort.
10 Although you could also argue that it's not really an "offshoot," since it's specifically designed to be incorporated into core. But I still think of it as a bit of an experiment.
11 My parenting partner, that is, not a business partner. A.k.a. the mother of my children, a.k.a. simply "The Mother." If you're interested reading more about her, you could also check out my Other Blog.
12 If you have need of such a thing, my top recommendation is The Perl Hunter.
13 For those of you who worry that you might suffer some sort of withdrawal without this long-ass blog series to sustain you, I invite you to pop over to my Other Blog next week where I plan to reveal the sources of my individual post titles. Each title in this series is an allusion to, or paraphrase of, something. (Possibly some may be allusions to multiple things, but there was definitely one thing I had in mind for each.) No doubt you recognized some; others may have been unfamiliar. If you want to try to guess them before I reveal all next week, here are some hints:
1. The Road So Far: a Winchester recap
2. The Power of OOP: Johnny Colla would have done a mean sax solo
3. A Moose Once Bit My Sister: I apologize for the obscure references; those responsible have been sacked
4. A Worthy Program, Exceedingly Well Read: also, profited in strange concealments ...
5. Speaking with the Speech of Coders: a present from Vietnam
6. Perl is Engineering and Art: at least one had to be obvious ...
7. The Most Powerful Weapon Which You Can Use to Change the World: according to Tata, not Perl at all ...
8. Endless Forms Most Beautiful and Most Wonderful: there was grandeur in his view of life from the Beagle
9. That's Why I Failed Recess: it was funnier when Rudy said it to Fat Albert
10. What We Talk About When We Talk About DWIM: involving two couples and a bottle of gin
11. Please Mr. Perl, Will You DWIM?: a plea to m'colleague Hugh
12. The End of the Beginning: once described as "sounding more like the Primitives than the Primitives"
13. Here's to Future Days: why are they called "twins" if there's three of them?
This week we look at how to start putting it all together.
Last week I talked about the trade-offs of DWIM, and wondered if it might be time to try to make sense of all the various parts and perspectives. Thus, this is the end of the beginning.1 This is the convergence of all my reminiscences.
Within the past year, I started a new job, again. This is something that many of us programmers do on a semi-regular basis.2 When you first start a new job, you spend a few days or a few weeks or a few months figuring out what's going on. Part of this is learning new concepts, new vocabulary: even if you're working in an industry you've worked in before, every company accretes its own terminology over time, words and ideas that only make sense to the people who work there. But a big part of it is learning the codebase. This may have been growing for years—sometimes a decade or more—and it may have been touched by dozens of hands, but still there will be patterns, consistency even in the inconsistencies. Once you've been working there a while, you will know this codebase in a fundamental and almost inexplicable way—there will still be plenty of code in it that you've never seen before, but, when you finally do see it, it'll feel familiar. Codebases build up character. It's not always a good character, but you learn to recognize it, and live with it, and even feel fond of it, in a weird sort of love-hate way.
When I write code that is concise and idiomatic, sometimes I am criticized. I hear people say that I should keep it simple; otherwise, new employees will have difficulty understanding it. To which my response is: so what? From a business perspective, what's more cost-effective? Having a new employee struggle with my code for a few weeks? or having a seasoned employee easily maintain it for a few years?
This is actually something that Larry pointed out a long time ago, although he being who he is and I being who I am, he said it a lot more succinctly than I just did:
You learn a natural language once and use it many times. The lesson for a language designer is that a language should be optimized for expressive power rather than for ease of learning. It's easy to learn to drive a golf cart, but it's hard to express yourself in one.3
Of course, there's a crucial difference in what Larry said vs what I just said: Larry was talking about the language itself. I'm talking about code written in the language. But I contend that the principle can be extended. Some while ago in this series,4 I drew an analogy to the dense technical language found in PhD theses. Those papers are hard for people outside their particular fields to read. Because they're not optimized (to steal Larry's turn of phrase) for outsiders. They're optimized for people who already know what all those big words mean. And every field has its own language. Medicine, law, mortgage banking, law enforcement, stock market analysis, golfing, art criticism, web site design, statistics, warehouse management, or playing D&D: all those things have terms that mean specific things to insiders and fly right over the heads of everyone else. Why is that?
Because people invent words. This is what we do, to faciliate communication. We create a new term, that no one else but us will understand. Anthropologically, some people believe that we do this in order to bond those in the know, to differentiate Us from Them.5 I don't subscribe to this point of view. I think the point is, you may have to look up this term the first time you see it, but, forever after that, you're going to know exactly what is meant when you see that word, without anyone having to waste time explaining it. Technical langauge (jargon) thus increases efficiency in communication. The cost is a small window of ignorance. The payoff is a lifetime of concision and precision. That's a pretty decent ROI.6
So I'm all about inventing new syntax. New syntax is a little tough for people to grasp the first time they see it, but, once they look it up and understand it, it makes everything so much easier, both for me to express and for them to read. And even the initial shock can be softened somewhat if you make the syntax look familiar, in the same way that Perl has always borrowed syntax from other langauges, so it looks and feels a bit homey to C coders and shell coders and awk coders and C++ coders and Smalltalk coders, all at once. So that, when people look at your new syntax, they won't know exactly what it means, but it will still seem vaguely familiar, so they can make a decent guess at it. This is just like when you encounter a new word in your reading: you may need to consult a dictionary7 to understand the full denotation and connotation, but most likely you can make a pretty good guess based on the context. And we all know that Perl is all about context.
In this way, new syntax actually makes code easier to read, after an initial learning curve. Does it contribute to Perl's reputation for inconsistency and difficulty? Sure. Just witness the bashing the Python folks give us. But the point is, they're missing the point. The point is, consistency is overrated. If everyone talked the same—worse, if every piece of literature was written in the same style, with the same limited vocabulary (for fear of accidentally using a word that someone might not know)—it would be harder to get our points across, not easier. Inconsistency across the spectrum of possible programs you may write makes it possible for this codebase to be inconsistent with that one, even while it is entirely consistent with itself, and, since those two codebases are for entirely different audiences of entirely different coders working in entirely different industries, the fact that they're entirely different from each other is a great blessing.
When I think about all the modules I want to write for CPAN one day (my dream modules, I mean: the ones that I hope to get around to one day but don't know if I'll ever find the time or inspiration), they're all declarative syntax modules. "Declarative," meaning that I declare exactly what I mean instead of having to cobble together bits and pieces to try to imply what I mean. As I mentioned before, this:
package Foo;
use Moose;
is not declarative. This:
class Foo
is declarative. And, while I understand that there are people who dislike declarative syntax, I don't really understand why. It's difficult for me to comprehend that saying exactly what I mean can ever be bad. It's sort of like when people describe themselves politically as being anti-progressive. I sort of know what you're saying, but then what does that make you? pro-stagnation?8
Hopefully this makes me make better sense: why I feel as I do about smartmatching, why I'm comaintainer and evangelist of Method::Signatures, why I'm willing to overlook the warts of Devel::Declare, why I consider modules such as Future and Bread::Board::Declare to be the future of Perl.9 But also hopefully it strikes a chord in some of you, reading this. Perhaps some parts of Perl will now make more sense to you than they did before; perhaps some parts that you've always liked, you'll now better understand why you like them. Maybe I've given you new tools to help explain Perl to others and help them understand why they should like it as much as we do. If I've done any of these things, I'm happy; if I haven't, I apologize for wasting your time. But none of that was my primary goal when I started writing this (it was just a nice side benefit). It's been an exploration that forced me to write down my own thoughts on why I've chosen Perl and stuck with it so long, and, by forcing the thoughts into verbal form and then releasing them into the world, it's made them real in a way they weren't when they were just half-formed thoughtlets in my head. In that regard, it's been successful.
Next week, in the final installment, I'll go where I've feared to tread before: my thoughts on the future of Perl.
1 Wait a minute. Strike that. Reverse it.
2 I have theories on why that is, but it's a bit off-topic in this discussion. Perhaps one day I'll write a blog post series called "How Corporations Fuck Up Good Tech Ideas."
3 From "Natural Language Principles in Perl."
4 Specifically, in Part 5, if you can remember way back then.
5 There is an amusing anecdote here about a former co-worker, Dave T., who advanced this opinion in a discussion about the utility (or lack thereof) of "Netspeak" abbreviations, such as ANAICT or AFAIK. The argument had been going back and forth between "it doesn't really save you any time if you have to explain what they mean" and "it may not save time the first time I use it, but it saves time once everyone understands it," when Dave spoke up and opined that the real purpose of these abbreviations was that they were "exclusionary language" designed to make those who understood it feel superior. This prompted a few of us to make up a new acronym on the spot: DTTTIEL (Dave T. thinks this is exclusionary language), which thereafter radically increased our communication efficiency by enabling us to condense a large amount of snark into 7 letters.
6 Return on investment. (Note: DTTTIEL)
7 And here I show my age. I probably should have said "you may need to consult dictionary.com."
8 Hopefully we can consider this to be a rhetorical question.
9 Pun somewhat intended. Perhaps I should say: pun used advisedly.
In the meantime, if you're interested in a rerun, I was just talking to a co-worker about my (much shorter) series on agile development over on my Other Blog. Might be worth checking out, if you're into that sort of thing.
This week we look at trade-offs in expressivity.
Last week I told a story about a CS professor who was inexplicably afraid of recursion. Other than the obvious moral of "don't be afraid of things that use underlying complexity to make your external code simpler,"1 what exactly does this have to do with Perl?
We're programmers. Nearly every single decision we make involves a trade-off of some sort. This way will be more efficient, but harder to maintain. This algorithm will be faster, but use more memory. This piece of software is more widely used, and therefore easier to find people to work on it, but feature-wise it sucks ass and can't hold a candle to this other piece of software, which is dead brilliant but almost entirely undocumented and nobody's ever heard of it. If we upgrade, we risk introducing bugs and downtime from weird backcompat issues; if we don't, we risk becoming outdated, insecure, and left behind in the marketplace. There's always some trade-off, and it's our job to pick one. We don't always pick the right one. In fact, most of the time there isn't really a "right" one. What seems right may end up being wrong, and what's actually right today may end up becoming wrong tomorrow. But we do the best we can. And when the benefits outweigh the consequences, that doesn't make the consequences go away. It just means we have to learn to live with them.
$_ just because I've spent a few multi-hour sessions trying to figure out what in holy hell happened to my loop variable. Some people do, of course. I've met a few, even. "Oh, I never use
$_," they'll say. "Too confusing." And I always think the same thing: uhhh ... hunh. Well, more power to ya. As for me, I'm going to stick with the syntax that helps me Get Shit Done quickly and efficiently and with a minimum amount of unnecessary and redundant code.
And Perl gives you that. It does some clever shit. Sometimes it's so clever that it fools even you, and you wrote the damn code. Many coders will say they don't like it when code is too clever. Well, maybe so, but what's the alternative? Code that's too stupid? Was my 200 lines of Pascal2 likely to be clearer and easier to maintain than my original 25 lines? You know, I don't think it was. It's true, the shortest amount of code is not always the best. It's possible to make your code so concise and compact that it becomes dense enough that even the reader's brain cannot escape its specific gravity. But let's not forget that more code is not always better either. Who wants to have read and decipher 1,000 words when 10 would have done just as well? At the risk of going completely off-topic,3 I will quote the eminent Lewis Carroll:
"I quite agree with you," said the Duchess; "and the moral of that is—'Be what you would seem to be'—or, if you'd like it put more simply—'Never imagine yourself not to be otherwise than what it might appear to others that what you were or might have been was not otherwise than what you had been would have appeared to them to be otherwise.'"
So there's always a balance we strive for regarding verbosity, and I believe Perl strikes that balance for us perfectly. And that it does so by liberal application of DWIMmery. Taking the DWIM out of Perl would make it not-Perl, in some fundamental way, and that's what really bugs me about the whole smartmatching debate. "It's too magical," is often just another way of saying "I don't fully understand how this works and it scares me."4 Don't be scared. Let it do its thing. Will it eventually bite you in the butt? Of course it will. But you have to look at the trade-off. How much expressive, productive code are you going to write vs how many times are you going find that it goes off in the weeds and does something entirely wrong? My experience with
$_, and with curly braces, and with Moose, and my experience thus far with smartmatching, tells me that, in the end, I'm going to come out on top.
I mentioned that I continue to use Devel::Declare-based modules, even though they're currently out of fashion. In fact, I use three such modules (MooseX::Declare, Method::Signatures, and TryCatch) very heavily, and all three make very heavy use of DD. When I say "very heavily," I mean "nearly every day"—remember that I write programs for myself a lot. I have exactly one problem with these modules, although I freely admit it's a bitchy one: it throws off the line numbers. That means what when you get an error, and that error is after some code that DD fiddled with, there's a chance that Perl will report the error as being on the wrong line. How big of a chance? Well, that depends on a lot of factors. The chances aren't very high if you do everything just so. But they increase with the more such constructs that precede your error, and most especially with how many lines those constructs take up. If they never have any newlines in them at all, you're good. But I have a tendency to do things like stretch my method signatures and class declarations over multiple lines, sometimes even sticking comments in there. And I buck the typical Perl trend of putting opening curly braces on the same line as the thing that introduces them.5 So I probably hit this problem more than anyone else, and it really irks me. It even irked me enough to try figure out a patch for it, which involved deep diving into the guts of the code that you may recall mst has proclaimed to be "a big bag of crack" (even though he wrote it). At the end of the day, I think the problem probably relates to the fact that DD uses Perl core code to skip whitespace, and it's tricky to know how many newlines that skips, and/or to communicate that back up the many layers of modules to the ultimate client for them to be able to do something about it. In the end, I think I might have to make a small but fundamental change to DD itself, plus probaby patch the things that are actually using DD to figure out where the line numbers are getting off and fix them. Or else I'm completely off and I don't understand what's going on at all, which would probably be worse. So it's definitely what you'd call non-trivial. I may roll up my sleeves and wade in there and do it anyway, one day, if I get inspired.6
But, in the meantime, I just live with it. It doesn't come up all the time, and, even when it does come up, most of the time I can figure out where the error actually is just by looking at it. For those rare cases where it happens to come up, and it happens to be far enough off that the general neighborhood isn't sufficient to give me a clue, and it happens to be an obsure enough error that I can't just grep for it and find it even without the proper line number, I just made a macro for my editor to sprinkle some
#line directives around,7 and another to clean them up afterwards. It usually costs me about 4-6 extra keystrokes, once or twice a month. From my perspective, that's a pretty small price to pay for being able to write clear, concise code.
Of course, that's me. I already discussed at some length why code legibility is a pretty important thing to me. Others may decide that this is not a good trade-off for them, and that's okay too. Everybody's different, and that's what keeps the world an interesting place, I always say. I just think it behooves us to be aware of the particular trade-offs we're making.
For my part, I'll take the shorter, more concise, more legible code using
$_, and Moose, and smartmatching, and my various favorite DD-based modules, over being more verbose and convoluted in my code.8 Many of you will retort that they don't find code using, say, smartmacthing clearer. You may point out that, when you see the smartmatch operator (or the
when keyword), you now have no idea what it actually means because it's so complicated and DWIMmy. But I say that's because you've decided you don't like the concept of smartmatching, and you've become convinced of its complexity. The truth is, 90% of the time you see
~~—probably more—you know exactly what it does without even thinking about it. The remaining 10% of the time you can look at the table, and you won't need to read the whole thing: just the one relevant line.
Look, people are going to hate new things sometimes. I hate new things sometimes. When I do, I just don't use those things. Sure, I can't avoid them entirely, because other people will use them, and I will have to read or even maintain that code. This is part of life. People use words I don't like all the time too, and they take actions I don't like, and they pass laws I don't like, and they make hideously crappy reality TV shows that I absolutely detest. That's just the way the world works. I been living here on this planet for coming up on half a century now, and I'm used to this sort of shit by now. There: I just used a word that some of you won't like. You can stop reading my blog posts now if if bothers you.9 But please don't try to stop me from using that word. Don't take away my freedom to use whatever word I like.
I guess that's what bugs me about the whole smartmatching thing. I feel like something sorta nifty is being taken away. Perl is all about DWIM, and all about TMTOWTDI, and TMTOWTDI is just another way to spell "freedom." Maybe losing smartmatching isn't that big a deal, but it's the general attitude that vaguely unnerves me. The idea that some folks have decided that Perl is too ... whatever. In this case too DWIMmy. So Perl is too this or that and must therefore be reined in. For our own good. I don't dig that.
I have no idea where we'll go next week. Maybe I'll actually start looking for some conclusions after all.
1 Which is a pretty good moral all by itself, now I consider it.
2 From my college assignment that I discussed last week, I mean.
3 And, yes, I'm sure there are many of you clever-clogs out there who just yelled "too late!" Rest assured that I sufficiently appreciate your wit and humour without the need for adding comments to that effect.
4 Note my specific use of the word "often." You, obviously, are one of the exceptions to that rule.
5 I do this because I'm one of those eccentric people who likes their open and close curly braces to line up vertically. Maybe it has something to do with the fact that I was a C++ programmer before I was a Perl programmer. But I don't really want to debate matters of style; the point is, this is Perl, and TMTOWTDI, and I should be able to program in the style to which I have become accustomed.
6 Assuming DD isn't rendered completely obsolete by Perl's new keyword API first, which is actually far more likely to happen.
7 See "Plain Old Comments (Not!)" in
perlsyn if you don't know what
#line does.
8 And, yes, I'm aware of the irony that I like my code to be concise and elegant while my prose tends to be meandering and tangential. Another thing you needn't point out in the comments.
9 Although, realistically, if you were inclined to stop reading my blog posts because I used that word, you would have been gone long before now.
This week we look at DWIMmery.1
Last week we talked about the awesome productivity of Perl without really asking why it's so good at it. For one answer to that, we'll need to look at a new spectrum.
There are several different spectra in Perl. In fact, one of the coolest things about Larry is that he's not only given us a cool language to play with; he's also given us cool new ways to think about the languages we play with. Along one spectrum we have the tripartite concept of laziness, impatience, and hubris. This series has only touched on these indirectly: laziness drives the language be legible, impatience drives it to be productive, and hubris drives it to be closer to a natural language.2 Along another spectrum, we have the orthogonal concepts of manipulexity and whipuptitude, which we just discussed last week.
But there's another spectrum to look at, which includes the twin concepts of TMTOWTDI and DWIM.3 TMTOWTDI is closely linked to context, which we covered pretty thoroughly. But what about DWIM?
$_ are hard to learn. They're not. They're hard to teach. They're sometimes hard to understand completely, difficult to grok, we might say. But they're not hard to learn, because there's nothing to learn. They just work. They just DWIM.
Recently, a co-worker pointed me at an RJBS module that I'd never seen before: Config::Singleton. It's a very cool module; you should go check it out. Now, I've already talked about RJBS code (more than once, even). It's some of the most modern Perl around, and should be part of everyone's Getting Shit Done toolbox. Here's a line from the POD for Config::Singleton:
Config::Singleton finds a config file via a series of DWIM-my steps that are probably more complicated to explain than they are to understand.
This statement is very nearly redundant. DWIMmy things are always harder to explain than they are to understand. It's the nature of DWIM. Because I'm a human, and therefore what I meant to do is always complex, once you break it down. The human brain is naturally complex, and handles complexity at an almost invisible level. You don't even notice it the vast majority of the time. Until you have to translate a mental process into a programming language, that is.
Last week I ever so briefly touched on the topic of smartmatching: just lightly brushed it with the barest fingertip. It's a pretty broad topic, and it probably deserves a whole post to itself.4 But this seems like as good a place as any to address one comment I received.5 Here's the relevant bit for our discussion today:
... you pretty much had to open up the full table of "left-hand-side X right-hand-side" behavioral combinations in the manual whenever you wanted to use the operator, to make sure you didn't miss any edge cases for the kind of inputs you were dealing with.
I know the table of which he speaks. It's in the
perlsyn man page, and it surely is complex; I can't argue with that. But have I consulted it every time I've used the operator? Nah. I skimmed it once, and I've glanced it a couple times since then. But I have used smartmatching constantly since it became available. Mostly in the context of
when, which is where it really shines. I've also used
when without
given, which can be a pretty cool thing that just DWIMs. Also, when the Damian sent us the patch that implements
where and
when for Method::Signatures, he did it using smartmatching, and I use that all the time as well. I use smartmatching a lot, and nothing has ever broken, the world has never come to an end, and the only trouble I've ever had from it is what I mentioned last week: people not liking the fact that I used it because it's now labeled "experimental." Not because of any problem that it actually caused, in other words, but for fear of what it might do. I must defer to the words of m'colleague H. Granger on this point:
Fear of a name only increases fear of the thing itself.
So, as it happens, I've never hit a case where Perl's concept of "the right thing" failed to match mine in the use of smartmaching. But I may. In fact, I'm pretty damn sure that eventually I will. So does that mean I better forego the use of smartmatching now, to save myself the hassle? Because of the inevitable horror that awaits me?
Well, what about when
$_ doesn't act as I expect, due to its being occasionally global and occasionally
local?6 That one I've hit, more than once. And, let me tell you, it's a real horrorshow to debug. How about the fact that sometimes curly braces mean a code block and sometimes they mean a hashref, and so Perl has to guess which one I meant, and sometimes it guesses wrong? Yep, hit that one too—once just recently, in fact. That one's not quite as bad, but the error messages sure don't give you much of a clue, that's for sure. What about error messages from Moose? Have you ever tried to figure out just what you did wrong from that hideous stack trace with all the insane layers of MOPpiness going on in there and eventually just threw up your hands in frustration? Boy, I sure have.
So I should stop using all these things too ... right?
Let me tell you a story. It may seem unrelated,7 but bear with me.
In college, I was an erratic student. In some classes, I was completely absorbed, attending every class and listening avidly. In others, I barely showed up at all, fell alseep in class, and did as little work as possible. Some of the latter classes I came perilously close to flunking out of, but others I did quite well at in spite of my best efforts. Most of my computer science classes were in that boat. Remember, even during my first stint in college I'd already been programming as a hobby for 4 or 5 years; by the time I went back to finish up my degree, I'd actually been a professional coder for two years. So mostly I took CS classes for easy As. I had a never-fail strategy: I'd show up on the first day and collect the "syllabus" (if you don't know what a syllabus is, it's a handy one-page cheatsheet for people who want to come to class as seldom as possible). Now I knew all the relevant dates, so I could just show up on the day each program was assigned, get the specs, write the code, and show up again when the program was due. Make sure to attend for any tests, and bam: minimal effort, maximal grade.
My second Pascal course was taught by a nerdy sort of gentleman who talked a lot about fly fishing and wrote tons and tons of code on the board—essentially all you needed to do to get the homework programs correct was just copy down all the blackboard code, rearrange it a bit, and hand it back to him. Of course, that was way too boring for me. I didn't need to have code spoon-fed to me: I could write my own damn code. The first program went entirely according to plan and I got an A on it. With the second program, though, I made a crucial error: I screwed up my dates. I not only missed getting the assignment, I somehow managed to miss turning it in as well. Considering that I followed this plan for nearly a dozen different classes throughout my college career, I suppose it was inevitable that I was going to muck it up at least once. Frantically I calculated a GPA: even if I got a perfect 100 on every other program and test, a 0 here was going to tank me.8 Could I drop the course altogether? No, I'd missed the deadline. Except ... if you could convince your teacher to sign off on it, get some sort of special permission slip, then you could drop after the deadline. Fine then: I'd just go to the teacher, give him a sob story, and get him to sign the form.
I don't recall exactly what I told him—some litany of the ever-popular, ever-vague "personal problems," I suppose—but I'm pretty sure I didn't really lie ... just embellished a bit. Okay, a lot. And, in the end, he bought it. I could see it in his eyes. But then he surprised me. "Son," he said, "you don't want to drop this course. I'll give you a second chance on that program." Then he went on to tell me how he'd love to take me out fly fishing sometime. I nodded encouragingly; there was no way in hell I was going to go fly fishing with anyone, much less this near-total stranger, but, hey: he was going to let me turn in the program late. I could pretend to be interested.
So I got the assignment, looked it over back at home, and cranked out the code in about an hour. It was a simple recursive algorithm: too simple for a second-semester CS course, really, but who was I to complain? I turned in my two dozen or so lines of code9 and, while I was there, I just happened to mention that my solution involved recursion.
His face fell.
"Oh, no, son. You can't use recursion." Whyever not, I asked. "Recursion is complicated! It's hard for people to understand. You try to write something like that for a business and they're going to tell you turn right around and write it over." I refrained, at this juncture, from pointing out that I had already worked for businesses, and knew damn well that was extremely unlikely—he was doing me a favor, after all. I stared at the problem for a bit. This wasn't one of those moronic examples of how to do recursion, like factorials,10 where it would be just as simple—simper, even—to use a loop instead. This was something like a binary tree traversal, where you really needed the recursion. Hesitantly, politely, I asked him exactly how it should be done then. "You need to look at the code I put up on the board, son. It's all in there."
I don't recall exactly how I got said code. Found someone else in class and borrowed their notes, maybe. But I do recall my utter incomprehension upon first seeing it. I had to stare at it for what seemed like forever, but eventually I caught on. He had converted the recursion to a loop, of course, 'cause that's how you do recursion without doing recursion. But that was insufficient. He had to create a whole separate data structure to keep track of where he was: he basically recreated the program stack, only in highly verbose, super-inefficient Pascal code. In the end, I needed a couple hundred lines of code to replace the couple dozen, all to recreate what the compiler would have done for me automatically. And why? I can only surmise that, somewhere along the line, this teacher had been bitten by some overly clever recursive algorithm and had vowed: never again!
Some people feel the same way about Perl's DWIMminess. I don't. Next week we'll see if we can figure out exactly why not.
1 Yeah, yeah, I know: I originally said last week that I was going to start tying it all up. But I totally forgot a whole 'nother couple of corners I wanted to explore. So I've retroactively modified history so that I never said that. We'll speak no more about it.
2 Admittedly, I should probably expand on how I think those topics relate to those programmerly virtues, but this week's post is too damn long already.
3 And some would no doubt throw in DTRT, but I feel that's so close to DWIM that it doesn't really require separate treatment.
4 Which I may just do, once the current series is done. Or I may end up mining all the goodness out of it and not have anything much left to say. We'll see.
5 That comment was inevitable, and could have been posted by nearly anyone in the Perl community, so I hope its author won't feel singled out. That certainly isn't my intention here.
6 And, these days, occasionally lexical.
7 And meandering. I meander. It's what I do. Did I mention that I wanted to be Stephen King when I grew up?
8 Although, looking back at it, I'm not sure how ... you wouldn't think a single 0 could impact the overall grade that much. Maybe I'd blown off a quiz somewhere along the line as well.
9 And it was only two dozen because it was Pascal. In Perl, I could've probably done it in a one-liner.
10 I believe I've mentioned before how much I detest the use of factorials as an "example" of recursion.