3. Advanced topics: See "J as a Tool of Thought - Code Clarity".
4. Learning and teaching J: See "Coding Kids".
We reviewed a recent presentation on J that intended to address some of the more difficult aspects of the language. We also studied some examples of the strength of J in allowing for simplification of code by a process reminiscent of an algebraic proof, as well as the utility of J for clearly describing an algorithm.
We explored some different avenues for writing GUIs in J: either a desktop-based method using the new "wd" that runs under Qt, or an example of a browser-based interface as shown by an example in online APL.
Finally, we looked at some recent efforts to introduce programming to young people and some of the controversy surrounding this idea.
The talk for the "7 Languages in 7 Months" Meetup (now named "n Languages in n Months") is presented, with some explanation here.
An example of simplifying code by replacing certains forms of expressions with known equivalents of them.
So, you want to have 1 verb applying across the lists 2,3,5 and e2,e3,e5?
Should be possible. I'll step through my derivation, since that's the only way this'll be helpful.
In that form, it would be easier to assign to some e array, rather than the individual e2 e3 & e5 items. Also, why explicit p, rather than p=.^i.?
A fairly extensive exchange of e-mails illustrated the power of J for expressing algorithms very clearly.
We often say the APL family of languages allow us to use language as a tool of thought. How does this play out in practice? Do we approach reading J programs differently from those written in other languages? If so, how?
Which is a verb which takes the configuration text as input and produces a table of name-value pairs as output. My first thought was "wow, I was able to knock that together in literally less than a minute, through simple incremental iterations in the REPL: J is AWESOME".
But then, thinking about posting it, I realized "this is awful, no one's going to be able to read it like this, and it's going to take more work to make it readable than it took to make it actually work".
So that got me thinking about what exactly we mean by J as a notation. And I wondered: how could we use the language to express our thoughts more clearly, and how does that differ from how we write J when we just want to get something done? And is this a different or more difficult problem for J than other languages?
So, how would you write an configuration file parser in J, if clarity were an important concern? I'm interested in not only the actual program, but the reasoning behind the decisions you make.
I find that my experiences and insights from working with J transport to other languages (often with some frustrations, but life is full of frustrations) in a useful fashion.
It would take some time, but I imagine you could translate that code you wrote into another language (perhaps paraphrasing parts, depending on your thinking and the target language).
In a much smaller amount of time, I imagine you could take that code and rephrase it to be more suggestive and "digestible". As a start, I'd try thinking up names for every parenthesized phrase (possibly adjusting the boundaries of the parenthesis or the behavior of parts of the code, if that makes it easier to name or describe).
Put differently, it's often worthwhile to think of code as being a "draft document". J is a tool of thought, but that should not prevent you from using other tools nor from having other thoughts.
J can be incredibly productive but it's sometimes worth putting in a little extra effort with an eye towards helping other people understand your thoughts.
That said, you do not need to be immediately critical of your work. Sometimes it's better to just write it down and get onto other issues and trust that you will probably have time to come back later. I've often noticed myself being impatient with things I had written before, and I like to think that my subsequent edits have sometimes helped other people understand my code (especially at rosettacode).
Finally, I think that one of our big limitations, as technical people, is that we often lack audience for our work as individuals. But since everyone faces analogous difficulties (and team efforts get some emphasis for a variety of reasons), I'm not sure if that's so much a characteristic of the technicalities or just something inherent in people.
An important consideration is, "clarity for who"? I want the code to be clear to me when I come back to it later. For that reason I usually code explicitly, calling explicit helper verbs, although many of the lines are "tacit in spirit". Suggestive names help.
> programs differently from those written in other languages? If so, how?
I completely agree that it's very easy to write unreadable programs in J. Some people have pushed back that it's easy to write unreadable programs in any language; but I would actually counter that it's easier in APL derivatives.
But I would contend that this is actually a consequence of APL derivatives being developed as a tool of thought rather than a tool of communication (or a tool of command).
Thinking is hard. Communicating is also challenging, but it's not the same as thinking.
I would like to point to the general teaching that APL people give for reading APL code -- what they say (I don't have a link) is that you can best understand APL code by rewriting it, and then comparing what you write with what was originally written. In other words, you learn to THINK about what the author was thinking about, and then you try to understand the WAY the author was thinking.
This reminds me of Chuck Moore's approach to program design, which he called "Thoughtful Programming". He advocated using many prototypes to cull out decent and unacceptable designs.
My brain is refusing to give me the name of the guy who's developing a thinking system using an APL derivative.
I'll counter your suggestion that it's easier to write unreadable code in APL derivatives with an observation that looks to me like a social issue rather than anything intrinsic in the language.
I say this because with minimal training (one class in APL at a community college, and occasional use in other classes, like biology), I was able to debug and improve other people's APL code in a large codebase.
From my point of view the thing that has been holding back APL is that it is too valuable and too productive. Consider, for example, the impact of Arthur Whitney on wall street. Things like that tend to drive up the price of the implementations which makes business people nervous. And when a business tries to switch away, and fails? That makes them even more nervous.
The problem with APL is mostly a lack of source material, for people who might be interested in using the language. This leads to people being intimidated and also leads to a lack of implementations.
So that probably means that admirable books and examples would help.
I find it helpful to identify the audience when writing - code or non-code. I then try to write for the audience. If there won't be an audience other than a computer and I will never read it again, then I'm not so worried about clarity. Otherwise, it's a real concern. My audience typically includes a lot of imperative/procedural programmers. I don't know if that's completely due to experience or how brains are actually wired for some people. I just know that's what I learned growing up. It's stuck with me for 20 years. I think of myself as a logical, "step by step" thinker.
This J implementation feels more like code golf or a compressed string. How many tokens/operations are included in it? I won't count, but I am fairly sure it's more than the (pipe, in, while, echo, till, read, skip, set, or, line) 9 in the PicoLisp example.
When reading a long J string or an entry in the obfuscated C code contest, I try to recognize patterns or operations. Having used J for about 6 months, I can recognize probably about half the operations in that string without having to look them up. That's progress. It still feels like a "run on sentence" which is harder to read than short sentences.
> make it readable than it took to make it actually work".
That's normally true for any type of writing. The difference between a first draft and final version is a fair amount of work. Taking a stream of consciousness and turning it into something other people understand takes some effort.
I learned lisp back in highschool, and I've used drracket and emacs and other such lisp environments, but never learned picolisp. I tried to install picolisp but it would not build for me and I do not feel like debugging the source for picolisp just for this message.
In other words, read the file as lines, removing blank lines and comment lines.
If all you are doing is saving the unparsed lines then we should expect simpler code. But maybe I have missed a subtlety of picolisp?
I get that @ is a wild card, but I do not understand the mechanism well enough to say whether your implementation is correct, nor do I know whether (while (read) .. is stashing the read result somewhere or what. Nor do I know if your skip is assuming a rigid file structure or is allowing free-form comments in the config file.
> read, skip, set, or, line) 9 in the PicoLisp example.
I count 64 tokens in the J implementation and 54 tokens in your PicoLisp example. I'm not sure why you have implied that parentheses are not tokens but I do not think they qualify as whitespace?
We could get further into parsing and punctuation issues, but I'm not sure whether that would be relevant.
I also usually prefer shorter sentences. Not always, but I'd probably try to split Dan's code into two or three lines. Posting a fair bit of code to email has been an influence there.
I imagine I would also favor a shorter implementation than what Dan has done here. For example, in his code I see the phrase e.&' =' but I see no equals signs in the config file nor in the specification on the companion task (whose J entry should perhaps be simplified?).
> I think there's a fine balance between tacit expressions and clarity.
> trains, etc for a readable sentence?
It uses some uncommon tricks. It leverages the read function which is the same function used in the repl to read input characters. So the goal is to take the file and skip any comments and then pass it on to set the variable with the key and value.
My brain has been trained to think of parens as whitespace. It didn't start that way. I can see why you may consider them tokens. I was also counting unique function/operation tokens, not characters. The idea being if I only have 4 english words with 3 characters each on a line, that is easier for my brain to parse than 5 operations using 2 ascii symbols that I don't recognize the meaning.
However as my J vocabulary improves it becomes less of an issue. I can parse i. or e. As fast as a function called idx or el?
The basic issue, here, seems to be that PicoLisp is stream oriented and this is a stream oriented task. No one in the J community has cared enough to build a stream oriented library for J. J has enough to do stream oriented design for academic purposes, but ... Consider xml/sax as an example of how one might approach streams in J – call out to some standardized implementation and instead focus on what is unique to the application.
Meanwhile, for a J programmer, words like [| & @ [ and ]] occupy a role not too different from parenthesis for a Lisp programmer. Parenthesis might seem simple, but in fact there are a fair number of contextual rules that one must learn before really understanding their significance. Do the parenthesis delimit a lambda definition? An argument list? Do they denote a function call? Some other special form? That, I think, is the issue you were focusing on when counting tokens - how many special rules does a person have to understand to parse the code. J has 9 parsing rules, each roughly at the same complexity level as a lisp-like lambda. Explicit contexts add a few more, though that's mostly syntactic sugar.
Meanwhile, J is and is not fast. It can be fast, but only if you design your code using big, homogeneous data structures to represent large data sets.
I'm not sure if I am making sense, so I suppose I should write some code instead.
This uses a state machine to deal with all the low level character munging and then forms the result into a two column table where the first column is the name and the remainder is whatever followed that (with redundant whitespace removed).
Is it readable? Not if you are not familiar with the language. In fact, this took me about half an hour to write. And I would not bother doing something like this, normally, unless performance really mattered (which implies large file size). But, if performance does matter, this approach should behave reasonably well and (at least for J) should have much better performance than an implementation which loops or otherwise uses separate primitives for separate states.
That said, I should also note that the idea of using decimal fractions for state machine instructions was Ken Iverson's. I'll not go into why sequential machine was not implemented that way, because I feel guilty about it.
What I have found is that it is important, more than with many other languages, to document the code. With J it is even more important. This is a consequence of the power involved in a statement combined with what the hell did I mean when I wrote it..
Other languages are more concerned with the"computer science structure" than the problem that is being approached (hence APL and J are overall considered as "lesser",.
The downside is that one can become overly focused on tight tacit programs and it might be better to have a documented explicit program filed somewhere, even in the same script file, that spells out the reasoning behind the tacit version.
Reading J is different from other languages because it removes much of the overhead from the tiddly details that can be handled more efficiently in the background by the idiot box.
I find that it helps to describe the J or APL code as if you are writing a paper about it for the expert. For example, see the essays in the J wiki. If carried to the extreme, it becomes the Literate Programming<https://en.wikipedia.org/wiki/Literate_programming>of Knuth.
The problem of human vs computer readability resurfaced for me recently when planning a J paper for MagPi, the how-to journal for the Raspberry Pi.
which essentially covers the ground of this topic for the "educated layman".
It's always been a mystery to me why it is OK to spend several hours (or sometimes days) analyzing several pages of FORTRAN or C but when reading a few lines of APL or J which do the same thing I must grasp it in a few minutes or I start feeling overwhelmed. But I have written similar "run-ons". Why? Because I can set up test data and add a little at a time to a line or a few lines, executing it and looking at the results as I go.
I have to force myself to break that monster up into more readable chunks. I can't do that in other languages as I have to compile or whatever, So I tend to write all the code then start debugging.
It might be interesting to try it on a large file.
I think the performance problem you observed is because the first version started boxing too early. Here, I save boxing till the end, and create fewer boxes, both of which should reduce overhead.
I have also struggled with documentation, and not only in the context of apl and/or j.
I sometimes wonder, though, how important it really is. So much of the skill of computer programming comes through seeing the code through experimentation and seeing both the code and its variants in action.
Anyways, writing documentation is a mentally and socially intensive task, and I have the utmost respect for people that can do it well. And good documentation gives valuable perspectives and insight into the underlying code. But... it's a struggle for me.
The difficult part in my experience is describing the structure and contents of my inputs and outputs.
Usually, I provide sample datasets, and show what verbs to run on them. But I don't have too many readers of my code so I am not sure how practical that really is.
I was wondering whether naming the main data structures encountered would be useful?
This page was last edited on 8 September 2015, at 04:45.