planet.donarmstrong.com

You may have seen Ovid's recent post on his discussions with the Perl Steering Committee about moving forward with implementing an initial subset of the Corinna proposal in the Perl core.

One of the issues that came up during those discussions was the best way to provide private methods in Corinna. The current Corinna proposal is that this would be done (like almost everything else in Corinna) via an attribute:

method do_internal :private () {...}

Thereafter, the do_internal() method can only be called from within the current class, and is never overridden by derived-class methods when it is called within its original class.

In other words, the :private method effectively prepends the following code to the start of the method:

croak "Can't call method 'do_internal'"
    if caller ne __CLASS__;

...and, in addition, causes the compiler to treat any call to that particular method from within the current class as a fully qualified method call. That is, within any class C with a private do_internal() method, any call to $self->do_internal() is treated as a call to $self->C::do_internal().

All of which means that there is no way to call a private method from anywhere except within the same class. Which, of course, is the whole point of having private methods in the first place.

But the need to automagically convert any call to a private method into a fully qualified method call is (to put it mildly) a complication for the compiler. So the members of the Perl Steering Committee Cor design team suggested that rather than having private methods, Perl should have lexical methods instead. Specifically, they suggested that instead of:

method do_internal :private () {...}

method do_external () {
    $self->do_internal();   # Call private method
    ...
}

...Perl would provide anonymous methods, which you could place in lexical variables and then call using the existing call-via-reference method call syntax:

my $do_internal = method () {...};

method do_external () {
    $self->$do_internal();   # Call lexical method
    ...
}

That neatly avoids of the challenge of rewriting private methods to check their caller, or the much greater challenge of rewriting private method calls to be fully qualified. Instead, it cleverly enforces the “can be called only from the current class” requirement by making it impossible to refer to the method at all, except in the lexical scope of the $do_internal variable.

You could even consider the slightly uglier $self->$do_internal() call syntax as a win: because it means that private method calls are explicitly marked as such.

The only downsides are:

• The call-via-reference syntax is more obscure and perhaps a little more challenging for less-experienced Perl developers. That might discourage them from using this approach, even for methods that really should be private, thereby penalizing a good OO practice.

• Using anonymous methods to implement private methods is structural rather that declarative. That is: we can’t just say what we want to have happen, we have to say how to actually implement it. In practical terms, this means that private method definitions are no longer explicitly marked as being private, and hence are far less searchable.

And that second issue is the real problem here. Because they’re structural, not declarative, lexical methods specified like this are also much easier to “enbug”. For example, if the developer mistakenly wrote something like this:

method other () {
    ...
    $do_internal++;
    ...
}

Then any call to the $do_external() method will be fine so long as the other() method is never called. But after other() has been called, the next call to $do_external() will throw a weird exception:

   Can't locate object method "140362266577153" via class "C"

Of course, we can get around that potential bug by making $do_internal immutable:

use Const::Fast;
const my $do_internal => method () {...};

but that’s now an extra module load, and even more infrastructure to get right.

(BTW, wouldn’t it be cool if the existing :const subroutine attribute could also be applied to scalar variables, in which case we’d just need:

my $do_internal :const = method () {...};

Maybe one day!)

Yet another issue with using anonymous methods as private methods is that they make it easy to subvert the intended encapsulation:

our $do_internal = method () {...};

That’s no longer a private method, because you can now call it from literally anywhere in your code:

$obj->$C::do_internal();

And, even if it were still a my variable, there’s nothing to prevent a class from explicitly exporting it:

method circumvent_privacy () { return $do_internal }

As an OO purist, I find that possibility worrying. And as someone who has reviewed a vast amount of real-world Perl OO code over the past few decades, I think that kind of expedient kludge is...inevitable.

So what do we do?

The good folks of the Perl Steering Committee Cor team suggested an approach that does address some of these issues. They proposed that the syntax for private methods be:

method $do_internal () {...}

method do_external () {
    $self->$do_internal();   # Call private method
    ...
}

That is: when you want to declare a private method, you put a $ at the start of its name. This form of the method keyword would then create a (presumably immutable!) lexical $do_internal variable, initialized with a reference to an anonymous method. You could then use the reference in that variable to call the private method.

While it doesn’t solve the problem of evil developers explicitly exporting that variable, it does make the syntax declarative again, and it could solve the issue of accidentally changing the variable (assuming the variable were indeed to be created as immutable).

The only problem is that now we have the method keyword not just declaring methods,
but also declaring variables.

Which, from the perspective of a language designer, is...less than ideal.
And, from the perspective of someone who actually still teaches Perl,
is much less than ideal.

So, how do we solve that?

Well, Raku solves it by marking private methods not with a $, but with a !
and then calling them with the ! as well (rather than Raku’s usual . method call operator):

# In Raku...
method !do_internal () {...}  # Private method

method do_external () {
    self!do_internal();       # Private method call
    ...
}

That’s a neat solution, but it probably wouldn’t work for Perl.

Nevertheless, the ideal of prefix-marking private methods and private method calls in some way might work. For example, we could consider formalizing the “private methods start with an underscore” convention inside the new class blocks:

method _do_internal () {...}

method do_external () {
    $self->_do_internal();
    ...
}

In other words, we could specify that any method definition that starts with an underscore doesn’t get added to the method symbol table. Instead, it implicitly creates an immutable and invisible lexical scalar within the class. Then, within any class block, any call to a method whose name starts with an underscore actually calls the method through the associated invisible lexical scalar instead.

Or, to put it more simply: it’s precisely the Steering Committee Cor team’s proposal, but: s/\$/_/g
And with the added advantage that the lexical is not otherwise accessible within its scope,
and therefore not accidentally modifiable, or maliciously exportable.

That’s by far the safest alternative I can think of, but I concede that it might still be too magical.

In which case we’d instead need to address just the immediate problem: that we’re contemplating using the method declarator to declare something that isn’t actually a method. Specifically, we’re using method to declare variables.

And the easiest solution to that problem is simply to define another declarator:

private $do_internal () {...}   # Declare private method

method do_external () {
    $self->$do_internal();      # Call private method
    ...
}

Now method once again only declares named methods, and we can easily explain that private declares a lexically scoped variable whose immutable value is a reference to the anonymous method.

We get very clear declarative labelling of private methods and equally clear labelling of calls to those private methods.

We don’t solve the “naughty folk can export that variable to circumvent privacy” problem, but I suppose we could always just spin that as a feature, rather than a bug. /s

For the moment, however, unless the Steering Committee Cor team is happy with either the “underscore enforces private” approach or the addition of a private keyword, I expect what we’ll actually have to do is to remove private methods entirely from the first stage of the implementation, and simply keep on pondering how to get them exactly right.

In the meantime, don’t let anyone ever tell you that language (re-)design is easy. :-)