Mathematica Notation and syntax mods
Question: I am experimenting with syntax mods in Mathematica, using the Notation
package.

I am not interested in mathematical notation for a specific field, but general
purpose syntax modifications and extensions, especially notations that reduce
the verbosity of Mathematica's VeryLongFunctionNames, clean up unwieldy
constructs, or extend the language in a pleasing way.

An example modification is defining `Fold[f, x]` to evaluate as `Fold[f,
First@x, Rest@x]`  
This works well, and is quite convenient.

Another would be defining `*{1,2}` to evaluate as `Sequence @@ {1,2}` as
inspired by Python; this may or may not work in Mathematica.

Please provide information or links addressing:

  * Limits of notation and syntax modification

  * Tips and tricks for implementation

  * Existing packages, examples or experiments

  * Why this is a good or bad idea


Answer: Not a really constructive answer, just a couple of thoughts. First, a
disclaimer - I don't suggest any of the methods described below as good
practices (perhaps generally they are not), they are just some possibilities
which seem to address your specific question. Regarding the stated goal - I
support the idea very much, being able to reduce verbosity is great (for
personal needs of a solo developer, at least). As for the tools: I have very
little experience with Notation package, but, whether or not one uses it or
writes some custom box-manipulation preprocessor, my feeling is that the whole
fact that the input expression must be parsed into boxes by Mathematica parser
severely limits a number of things that can be done. Additionally, there will
likely be difficulties with using it in packages, as was mentioned in the
other reply already.

It would be easiest if there would be some hook like `$PreRead`, which would
allow the user to intercept the input string and process it into another
string before it is fed to the parser. That would allow one to write a custom
preprocessor which operates on the string level - or you can call it a
compiler if you wish - which will take a string of whatever syntax you design
and generate Mathematica code from it. I am not aware of such hook (it may be
my ignorance of course). Lacking that, one can use for example the _program_
style cells and perhaps program some buttons which read the string from those
cells and call such preprocessor to generate the Mathematica code and paste it
into the cell next to the one where the original code is.

Such preprocessor approach would work best if the language you want is some
simple language (in terms of its syntax and grammar, at least), so that it is
easy to lexically analyze and parse. If you want the Mathematica language
(with its full syntax modulo just a few elements that you want to change), in
this approach you are out of luck in the sense that, regardless of how few and
"lightweight" your changes are, you'd need to re-implement pretty much
completely the Mathematica parser, just to make those changes, if you want
them to work reliably. In other words, what I am saying is that IMO it is much
easier to write a preprocessor that would generate Mathematica code from some
Lisp-like language with little or no syntax, than try to implement a few
syntactic modifications to otherwise the standard mma.

Technically, one way to write such a preprocessor is to use standard tools
like Lex(Flex) and Yacc(Bison) to define your grammar and generate the parser
(say in C). Such parser can be plugged back to Mathematica either through
MathLink or LibraryLink (in the case of C). Its end result would be a string,
which, when parsed, would become a valid Mathematica expression. This
expression would represent the abstract syntax tree of your parsed code. For
example, code like this (new syntax for `Fold` is introduced here)

    
    
    "((1|+|{2,3,4,5}))"
    

could be parsed into something like

    
    
    "functionCall[fold,{plus,1,{2,3,4,5}}]"
    

The second component for such a preprocessor would be written in Mathematica,
perhaps in a rule-based style, to generate Mathematica code from the AST. The
resulting code must be somehow held unevaluated. For the above code, the
result might look like

    
    
    Hold[Fold[Plus,1,{2,3,4,5}]]
    

It would be best if analogs of tools like Lex(Flex)/Yacc(Bison) were available
within Mathematica ( I mean bindings, which would require one to only write
code in Mathematica, and generate say C parser from that automatically,
plugging it back to the kernel either through MathLink or LibraryLink). I may
only hope that they will become available in some future versions. Lacking
that, the approach I described would require a lot of low-level work (C, or
Java if your prefer). I think it is still doable however. If you can write C
(or Java), you may try to do some fairly simple (in terms of the syntax /
grammar) language - this may be an interesting project and will give an idea
of what it will be like for a more complex one. I'd start with a very basic
calculator example, and perhaps change the standard arithmetic operators there
to some more weird ones that Mathematica can not parse properly itself, to
make it more interesting. To avoid MathLink / LibraryLink complexity at first
and just test, you can call the resulting executable from Mathematica with
`Run`, passing the code as one of the command line arguments, and write the
result to a temporary file, that you will then import into Mathematica. For
the calculator example, the entire thing can be done in a few hours.

Of course, if you only want to abbreviate certain long function names, there
is a much simpler alternative - you can use `With` to do that. Here is a
practical example of that - my port of Peter Norvig's [spelling
corrector](http://norvig.com/spell-correct.html), where I cheated in this way
to reduce the line count:

    
    
    Clear[makeCorrector];
    makeCorrector[corrector_Symbol, trainingText_String] :=
    Module[{model, listOr, keys, words, edits1, train, max, known, knownEdits2},
    (* Proxies for some commands - just to play with syntax a bit*)
    With[{fn = Function, join = StringJoin, lower = ToLowerCase, 
     rev = Reverse, smatches = StringCases, seq = Sequence, chars = Characters, 
     inter = Intersection, dv = DownValues, len = Length, ins = Insert,
     flat = Flatten, clr = Clear, rep = ReplacePart, hp = HoldPattern},
    (* body *)
    listOr = fn[Null, Scan[If[# =!= {}, Return[#]] &, Hold[##]], HoldAll];
    keys[hash_] := keys[hash] = Union[Most[dv[hash][[All, 1, 1, 1]]]];
    words[text_] := lower[smatches[text, LetterCharacter ..]];
    With[{m = model}, 
     train[feats_] := (clr[m]; m[_] = 1; m[#]++ & /@ feats; m)];
     With[{nwords = train[words[trainingText]], 
      alphabet = CharacterRange["a", "z"]},
      edits1[word_] := With[{c = chars[word]}, join @@@ Join[
         Table[
          rep[c, c, #, rev[#]] &@{{i}, {i + 1}}, {i, len[c] - 1}], 
         Table[Delete[c, i], {i, len[c]}], 
         flat[Outer[#1[c, ##2] &, {ins[#1, #2, #3 + 1] &, rep}, 
           alphabet, Range[len[c]], 1], 2]]];
      max[set_] := Sort[Map[{nwords[#], #} &, set]][[-1, -1]];
      known[words_] := inter[words, keys[nwords]]]; 
     knownEdits2[word_] := known[flat[Nest[Map[edits1, #, {-1}] &, word, 2]]];
     corrector[word_] := max[listOr[known[{word}], known[edits1[word]],
       knownEdits2[word], {word}]];]];
    

You need some training text with a large number of words as a string to pass
as a second argument, and the first argument is the function name for a
corrector. Here is the one that Norvig used:

    
    
    text = Import["http://norvig.com/big.txt", "Text"];
    

You call it once, say

    
    
    In[7]:= makeCorrector[correct, text]
    

And then use it any number of times on some words

    
    
    In[8]:= correct["coputer"] // Timing
    
    Out[8]= {0.125, "computer"}
    

You can make your custom `With`-like control structure, where you hard-code
the short names for some long mma names that annoy you the most, and then wrap
that around your piece of code ( you'll lose the code highlighting however).
Note, that I don't generally advocate this method - I did it just for fun and
to reduce the line count a bit. But at least, this is universal in the sense
that it will work both interactively and in packages. Can not do infix
operators, can not change precedences, etc, etc, but almost zero work.

