Duckspeak vs. Smalltalk: Decline of the Xerox PARC Philosophy at Apple (2011) - <AUTHOR> http://dorophone.blogspot.com/2011/07/duckspeak-vs-smalltalk.html?view=classic <comment><USER2> Here's some stuff I wrote about a HyperCard-inspired system called HyperLook (nee HyperNeWS (nee GoodNeWS)) and some stuff I developed with it:

SimCity, Cellular Automata, and Happy Tool for HyperLook (nee HyperNeWS (nee GoodNeWS))

HyperLook was like HyperCard for NeWS, with PostScript graphics and scripting plus networking. Here are three unique and wacky examples that plug together to show what HyperNeWS was all about, and where we could go in the future!

[https://medium.com/@donhopkins/hyperlook-nee-hypernews- nee-g...](https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee- goodnews-99f411e58ce4)

Some highlights:

The Three Axis of AJAX, Which NeWS Also Has To Grind!!!

NeWS was architecturally similar to what is now called AJAX, except that NeWS coherently:

…(drum roll)…

1) Used PostScript CODE instead of JavaScript for PROGRAMMING.

2) Used PostScript GRAPHICS instead of DHTML and CSS for RENDERING.

3) Used PostScript DATA instead of XML and JSON for DATA REPRESENTATION.

The Axis of Eval: Code, Graphics and Data

We will return to these three important dimensions of Code, Graphics and Data as a recurring theme throughout this article. But which way to go from here?

Alan Kay on NeWS:

"I thought NeWS was 'the right way to go' (except it missed the live system underneath). It was also very early in commercial personal computing to be able to do a UI using Postscript, so it was impressive that the implementation worked at all." -Alan Kay

What's the Big Deal About HyperCard?

"I thought HyperCard was quite brilliant in the end-user problems it solved. (It would have been wonderfully better with a deep dynamic language underneath, but I think part of the success of the design is that they didn't have all the degrees of freedom to worry about, and were just able to concentrate on their end-user's direct needs."

"HyperCard is an especially good example of a system that was "finished and smoothed and documented" beautifully. It deserved to be successful. And Apple blew it by not making the design framework the basis of a web browser (as old Parc hands advised in the early 90s …)" -Alan Kay

<comment><USER3> Yikes, am I reading this right? That PostScript here is implied as a good thing? Here is xeyes implemented in PostScript. It was probably better than the X Windows implementation of the era, but compared to JavaScript... I shudder.

[https://groups.google.com/forum/#!original/comp.windows.news...](https://groups.google.com/forum/#!original/comp.windows.news/pIoof2p2eew/HSBV2prw3CoJ)

<comment><USER2> That's 93 lines of PostScript code, including comments. And how many lines of C code is XEyes? (Not including the Makefile.am, autogen.sh, configure.ac, and other supporting cruft, to be charitable. I'm counting xeyes.c (141), Eyes.c (642), Eyes.h (56), EyesP.h (56), transform.c (111), transform.h (32), eyemask.bit (22), eyes.bit (22) = 1082 lines of really ugly code -- not easy on the eyes!)

[http://cgit.freedesktop.org:80/xorg/app/xeyes/snapshot/xeyes...](http://cgit.freedesktop.org:80/xorg/app/xeyes/snapshot/xeyes-1.1.1.tar.gz)

By the way, XEyes was a later imitation of Jeremy Huxtable's original NeWS "Big Brother" eyes.

[https://web.archive.org/web/20140228084012/http://en.wikiped...](https://web.archive.org/web/20140228084012/http://en.wikipedia.org/wiki/Xeyes)

Here's a much more elaborate version of NeWS eyes, that lets you drag and drop eyes into each other (see "MoveStopSub"), nesting them to any depth (so child eyes recursively move around with their parent eye, and look really creepy), and split eyes in two with a menu (see "SplitEye"), in 375 lines of PostScript code:

[https://donhopkins.com/home/archive/NeWS/eyes.ps](https://donhopkins.com/home/archive/NeWS/eyes.ps)

Have you ever actually used the X11 SHAPE extension to make a round window, did you know all X11 windows actually have FOUR shapes (client bounding region, client clip region, effective bounding region, effective clip region), and do you understand all the nuances and implications of the ICCCM ("Ice Cubed") window management protocol?

[https://en.wikipedia.org/wiki/Shape_extension](https://en.wikipedia.org/wiki/Shape_extension)

[https://www.x.org/releases/current/doc/libXext/shapelib.html](https://www.x.org/releases/current/doc/libXext/shapelib.html)

[https://blog.dshr.org/2018/05/recreational- bugs.html](https://blog.dshr.org/2018/05/recreational-bugs.html)

"You will get a better Gorilla effect if you use as big a piece of paper as possible." -Kunihiko Kasahara, Creative Origami.

And what is the net speed velocity of an XConfigureWindow request?

[https://medium.com/@donhopkins/the-x-windows- disaster-128d39...](https://medium.com/@donhopkins/the-x-windows- disaster-128d398ebd47#5a77)

<comment><USER4> Apples and Oranges. Postscript itself includes the code for the primitives used in xeyes, whereas the C code implements them first. Take away that code and it is much more illuminating.

<comment><USER2> Are you saying that HTML Canvas should not have a method to draw circles because you can implement a circle drawing algorithm in JavaScript?

What is your point? That it's a liability that PostScript has a far superior, more powerful, scalable, device independent, future proof, and vastly more convenient and easier to use and understand imaging model than X11?

And by the way, NeWS eyes came first: "xeyes" was a lower quality knock-off, not the original. The first version of "xeyes" actually used a rectangular window, which kind of missed the whole point, until somebody added hundreds of lines of extremely complex code to implement the SHAPE extension. I linked to the source above: download and read it if you don't believe me.

NeWS lets you shape a round (or any shape) window the exact same way you draw the same shape. See how pizzatool shapes the spinning full or half circle pizza window floating inside a rectangular frame, so you can easily move and resize it:

      % Reshape the parent popup pizza window,
      % to reflect a change in the pizza's shape.
      % In all truth, the pizza canvas is rectangular!
      % It gets its round (or semi-circular) shape
      % from the shape of its parent canvas,
      % which is a hollow rectangular popup window frame
      % (just the window borders),
      % with a discontiguous pie floating in the center.
      %
      /ReshapeParent { % - => -
        gsave
          Parent setcanvas
          /bbox PreviewWindow send /reshape PreviewWindow send
        grestore
      } def

      % The popup pizza window path includes the window borders and the
      % round (or semi-circular) pie floating in the middle, but excludes
      % everything between.
      %
      /path { % x y w h => -
        matrix currentmatrix 5 1 roll		% mat x y w h
          /minsize self send xymax
          4 2 roll translate			% mat w h
          0 0 3 index 3 index rectpath
          WInset SInset translate			% mat w h
          EInset WInset add				% mat w h ewinsets
          NInset SInset add				% mat w h ewinsets nsinsets
          xysub					% mat insidew insideh
          0 0 3 index 3 index rectpath

          2 div exch 2 div exch			% mat centerx centery

          2 copy translate
          min dup neg scale				% mat

          { % send to Center client (the Pizza):
    	/radiusscale self send 0 moveto
    	0 0 /radiusscale self send
    	0 360 /fraction self send mul
    	arc closepath
          }						% mat {msg}
          /Center /client self send			%       ... client true | false
          /WhatPizza? assert			% mat {msg} client
          send					% mat
        setmatrix					%
      } def

      % Reshape the window canvas with the even/odd rule.
      %
      /reshape { % x y w h => -
        /invalidate self send
        gsave
          4 2 roll translate			% w h
          0 0 4 2 roll				% 0 0 w h
          /path self send				%
          self eoreshapecanvas
        grestore
      } def

So it looks like this:

[https://donhopkins.com/home/catalog/images/pizzatool.gif](https://donhopkins.com/home/catalog/images/pizzatool.gif)

But with X-Windows you have to bend over backwards manipulating pixmaps or lists of rectangles, so the window shaping code is completely different and much more complex than the drawing code.

Case in point: take a look at the "ShapePieMenu" function in the Tk pie menus I implemented for X11/TCL/Tk SimCity, for another example of how much of a pain in the ass it is to shape a window in X11. It has to first get the display and query to make sure the extension is supported, then if it is, make the window exist, get the window id, make a pixmap, make a graphics context, set the foreground color, erase the background, set the foreground color, fill a bunch of rectangles, free the graphics context, and call XShapeCombineMask. And that's only shaping the window to a simple list of few rectangles for all the labels, not a circle, which would have been MUCH harder:

[https://github.com/SimHacker/micropolis/blob/master/micropol...](https://github.com/SimHacker/micropolis/blob/master/micropolis- activity/src/sim/w_piem.c#L2305)

So funny that you'd compare drawing circles in X11 -vs- PostScript. Have you actually tried to draw a circle with either API yourself -- let alone used the X11 SHAPE extension to make a round window? Here's an excerpt from the X-Windows Disaster which addresses that very topic -- specifically "in the case of arcs":

[https://medium.com/@donhopkins/the-x-windows- disaster-128d39...](https://medium.com/@donhopkins/the-x-windows- disaster-128d398ebd47)

Myth: X is "Device Independent"

X is extremely device dependent because all X graphics are specified in pixel coordinates. graphics drawn on different resolution screens come out at different sizes, so you have to scale all the coordinates yourself if you want to draw at a certain size. Not all screens even have square pixels: unless you don't mind rectangular squares and oval circles, you also have to adjust all coordinates according to the pixel aspect ratio.

A task as simple as filing and stroking shapes is quite complicated because of X's bizarre pixel-oriented imaging rules. When you fill a 10x10 square with XFillRectangle, it fills the 100 pixels you expect. But you get extra "bonus pixels" when you pass the same arguments to XDrawRectangle, because it actually draws an 11x11 square, hanging out one pixel below and to the right!!! If you find this hard to believe, look it up in the X manual yourself: Volume 1, Section 6.1.4. The manual patronizingly explains how easy it is to add 1 to the x and y position of the filled rectangle, while subtracting 1 from the width and height to compensate, so it fits neatly inside the outline. Then it points out that "in the case of arcs, however, this is a much more difficult proposition (probably impossible in a portable fashion)." This means that portably filling and stroking an arbitrarily scaled arc without overlapping or leaving gaps is an intractable problem when using the X Window System. Think about that. You can't even draw a proper rectangle with a thick outline, since the line width is specified in unscaled pixel units, so if your display has rectangular pixels, the vertical and horizontal lines will have different thicknesses even though you scaled the rectangle corner coordinates to compensate for the aspect ratio.

<comment><USER4> I created a new language called zis, which has a function named "doXeyes()". I can do xeyes in literally one line of code. It destroys postscript with it's amazing simplicity. Sorry for the snarkiness, but that's my point.

<comment><USER2> Snarkyness? What snarkyness? I don't understand what you mean. So I'd appreciate it if you'd explain what your point is more clearly, by directly addressing the questions I asked.

Please be kind enough to link to the source code of the language "zis" that you wrote, just like I linked to the source code that I and others wrote to illustrate my point. I'd like to see the actual code you wrote, so I don't miss your point, and can see that you're arguing in good faith. Thank you, kind sir or ma'am!

So what else is your language "zis" good for? Is it fully general purpose, easily editable at runtime, graphically skinnable with a build-in drawing editor, and dynamically scriptable and extensible by normal users, like HyperCard and HyperLook and Smalltalk? Do you think Alan Kay would describe "zis" as the "right way to go" like he described NeWS, and what other end-user problems does it solve, in the sense that he thinks HyperCard is brilliant and that Smalltalk pioneered? Is "zis" as "finished and smoothed and documented beautifully" as Alan Kay describes HyperCard?

Have you developed and shipped any commercial products or free software with your language, like SVR4 or SimCity, that I can see and run for myself, so I can be confident you're not just making stuff up or wildly exaggerating, and that it works as you advertise? And have you written any books or documentation or articles or research papers about it that I can read to learn more about it? Or at least a screen snapshot, please? What are the URLs?

Have you finished reading the X-Windows xeyes source code that I referred you to? Did you simply copy that code somebody else wrote into "zis" verbatim, or did you actually write your own original code? How is it licensed, and where has it been distributed: is it open source or not, and where's the repo?

Now that you have presumably used the X-Windows SHAPES extension API first hand yourself, please share your experience and tell me what you think of it: do you think that it is well designed and easy to use or not? What changes or improvements would you suggest? Did you run into any of its limitations, or notice any unnecessary complexity, or ICCCM window manager incompatibilities? How well does your language "zis" support the SHAPES extension and ICCCM protocol, or improve upon them, and what else is it good for than xeyes?

Finally, does "zis" support dragging and dropping eyes into each other, nesting their windows into an arbitrarily deep hierarchy, like my NeWS eyes did 27 years ago, and what ICCCM window managers support that feature, or did you have to write your own window manager or X extension to support that feature?

Thanks for taking all this time and effort to design and write and share all that code, and to support and clarify your fascinating arguments. I'm looking forward to seeing your code and your answers to my questions.

<comment><USER4> So, yes I've made xeyes in both X11/C and another language - it was part of a computer graphics course taken years ago, oddly enough, and it _might_ have been PS, but I don't really remember. I did write one or two small programs in PS, I know that.

My point with the (imaginary) "zis" language is that comparing a domain- specific language like PS to C code is a strange comparison - not without merit, but PS is a higher level abstraction, which is great, but saying it is an amazing language because it is slightly easier than raw C is misleading. If I build up (or use) a nice library (and I'm not going to get pulled into defending X, I'm not as harsh as you on it, but it is ... painful ... to use - also, side note, I remember skimming through the Unix Haters Handbook when it came out, so I probably read your words back then, funny!) written in C, it is fair to compare that to PS.

I'd rather do it in OpenGL/C for instance, plus my version will be more performant. There's a reason that combination (or DirectX) is used in the vast majority of graphics software out there.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> Since this article was written, the iPad has gained a number of excellent programming tools from third party developers (Pythonista is my favorite).

And perhaps more importantly, Apple has released two: Swift Playgrounds and Shortcuts. Swift Playgrounds is explicitly designed so that kids (and adults) can learn to program. But it's also pretty full featured, with access to all of the APIs and frameworks on the system. Shortcuts is programming-in- disguise, much like HyperCard before it. It's an end user app that nearly requires you to "program", albeit visually for the most part.

I think the vision of every computer user also being a programmer hasn't happened because we really still haven't figured out how to make that work. But that doesn't mean no one is trying anymore.

As for me, I'm a professional programmer because when I bought a Mac 15 years ago, it came with Xcode in the box, and I was able to start building my own Mac apps without too high of a barrier to entry.

<comment><USER6> > Since this article was written, the iPad has gained a number of excellent > programming tools

That's true, and good, but misses the point. In the smalltalk world (and Lisp machine worlds of PARC and MIT) _everything_ running in the machine (above the microcode layer) was inspectable, breakpoint able, and modifiable.

While on the iPad everything is, as the article says, opaque and unmodifiable, unless its your own app.

note: I worked on Lispms at MIT (and other places) and on D-machine Interlisp at PARC (and other places)

<comment><USER7> > In the smalltalk world (and Lisp machine worlds of PARC and MIT) everything > running in the machine (above the microcode layer) was inspectable, > breakpoint able, and modifiable.

So Linux plus maybe some elegance features?

<comment><USER6> There were no kernel/user space distinctions in any of the systems I mentioned, and all of it could be dynamically modified at runtime (no recompilation or restarting) down to the function and variable level. Philosophically completely different from Unix et al.

But you raise an interesting point: one of the reasons I started Cygnus with John and Michael was that I had never used a machine to which I didn't have access to the source code and the idea of that horrified me.

</comment></comment></comment></comment><comment><USER8> I have spent 100's of hours building a simple simulator using Julia recently. I mention this because this is what this view of computing misses; despite having hugely productive tools (compared to the 70's, remember "compile time?") vast reuse (library after library filled with wonderful goodies) and amazing computers (I use an X1 and have access to servers and so on and if I really need it I can use the cloud any time), software is complex and time consuming. That's why programming / developing is a full time job. Everyone else needs to get on with all the other things they've got to do.

<comment><USER9> The view of computing you are criticizing in fact doesn't miss your point. It attacks it straight on. What does "programming" mean? Insofar as it involves people using Algol inspired languages -- with some features here and there -- typing up instructions in expensive systems that are (still!) teletype emulators, then what you say is always going to be true.

The point of the article is that once upon a time people began to think another way was possible and started going in that direction. The reasons that all of this has fallen out of the computing pop culture have little to do with viability, but rather the needs of the market.

</comment></comment><comment><USER10> Discussed in 2015, including comments from an HN user who worked at both PARC and Apple:

[https://news.ycombinator.com/item?id=8976872](https://news.ycombinator.com/item?id=8976872)

</comment><comment><USER11> Windows has a surprising number of ways to code out of the box and all require nothing more than opening notepad and saving it with the right file extension.

.js .vbs .bat .ps1 or .hta

The problem is they are not simple to use for the average user. Windows lacks a straightforward procedural automation system. Flow, Workflows and PowerApps address this in some ways but are really jank.

I want something like:

      GET COLUMN 'customer name' FROM accounts.xls
      REMOVE ROW 1
      SAVE customers.xls
      EMAIL foo.xls TO 'email@example.com'

</comment><comment><USER12> Smalltalk is the product of a very well funded research lab - Xerox PARC. It was not so much a successful product.

It's less interesting about think what Apple did with it - because they had other, more conventional, goals in the end of the day.

I think it's more interesting to think what happened at Xerox - why didn't they get it out to millions of people as the software a computer boots into or as the primary development environment. Or even as a system design philosophy?

Xerox sold expensive systems like a few Smalltalk machines, but much more they sold office systems on the same hardware NOT implemented in Smalltalk. IIRC they might have sold larger laser printers with Smalltalk-based control software - can't remember.

Xerox gave Smalltalk 80 to a few companies - like Apple and others. Apple ported it to their machines - this is also the origin of Squeak (a later open source Smalltalk). But it never really caught on. Even though Apple gave developers access to it.

Now the philosophy of Smalltalk was already in the Apple ][. It booted into a BASIC prompt. But later computers were more universal and current incarnations are only seen as appliances. Nobody needs to program to use an iOS device.

On a Lisp Machine one can force any application into a REPL (called listener) at any time. The keyboard even has a key for that. Live programmability is the main purpose of such a computer. Source is only a keystroke/mouse click away.

We don't have that anymore and there must be a reason for it. Probably it would also be a security nightmare in current networked surroundings...

<comment><USER13> The reason Smalltalk didn't take over the world was it was so resource intensive. A single workstation would have cost about $40,000. There's no way the early Macs could have run Smalltalk, they only became capable of that years later and it was slow as molasses. I suppose it should really have had a chance by the Java era, but C style syntax languages ruled the world and the state of the art had moved on by then.

<comment><USER14> But would it take over the world today? Our latest smartphone properly have 10x to 100x the processor power of the "$40,000" Workstation back then.

I am not entirely sure if Smalltalk was the right approach, most people, the 98% who don't read HN don't want to code. I think Hypercard is a much closer to that vision but it doesn't seems there are massive interest for something like that either, The world today is like 90% consumption, people consume news, media, video etc.

<comment><USER9> > it doesn't seems there are massive interest for something like that either

It's been a long enough time that most people don't even know that "Hypercard like things" are even possible, or what they mean, or how one could use them. The "appification" of personal computing has ensured that there are "programmers" and "users" (scribes and plebs) and that the relationship will be transactional and commercial.

> he world today is like 90% consumption, people consume news, media, video > etc.

The more the development community realizes this, the better off we will be -- because it will become clear that in a technical sense a completely different way (ie the direction of Smalltalk or Hypercard) is possible and that the truly limiting factors are cultural more than anything else.

</comment></comment></comment></comment><comment><USER15> _But while computers rapidly increased in power, the tools that programmers used to program them developed relatively conservatively. It is easy to imagine a world where those tools developed along with the computers, until programming itself became so easy that the average user would feel comfortable doing it._

It's nice to imagine. But is it so easy to imagine?

<comment><USER9> He has provided historical examples that trend in that direction. The point is: industry stopped looking into this (and an implicit point is also that investigating these things is no longer well funded).

</comment></comment><comment><USER16> Maybe I'm cynical but I just don't think the average person cares enough to want to program even if it's really easy. I mean it's easy to criticise the software for being locked down, and fair enough, but maybe .0001% of users would really want to modify things anyway.

<comment><USER17> The amount of half-baked Javascript bookmarklets, Excel scripts, and SQL stored procedures I have come across in my lifetime beg to differ.

People are already programming, and they'd do more of it if everyone had basic knowledge of how. It's like literacy. Before general literacy, people thought it was pointless to learn to write, now everyone jots down grocery lists and working notes because they're capable of doing it without a second thought.

<comment><USER18> > _The amount of half-baked Javascript bookmarklets, Excel scripts, and SQL > stored procedures I have come across in my lifetime beg to differ._

Even those are made by what? 1% of users?

</comment></comment></comment><comment><USER19> Creators love to create, thinkers love to think, programmers love to program.

Most people do NOT like creating(when they can use something already created by someone else without effort), programming or thinking(it is slow, takes effort and gratification is not instant) at all.

It is not a critic on most people, in fact I have seen people spending months of work for installing or compiling a custom Linux like gentoo just because they can. Thinking on problems that will probably be a dead end forever or programming things that would have been done cheaper, better and way faster manually.

That is the reason Steve Jobs succeeded, he made what people (most people)wanted. Instant gratification like music on iPods. Simple solutions to problems people have(simple install). Making sure creators could live from their professional work(simple payment system).

There is a talk somewhere when Steve Jobs talk about him realizing people really demand things like Trash TV.

When you sell products you see reality, Alan could only see the idealism of Academia.

It is not that people do not want to improve themselves. It is just that most people do not want to pay the price, in effort, in time or focus. Just look at the Ads on TV: Lose weight with no effort with product X, in no time, while you watch TV.

</comment><comment><USER7> > It is easy to imagine a world where those tools developed along with the > computers, until programming itself became so easy that the average user > would feel comfortable doing it.

I would argue it IS SO EASY. The thing is that programming itself is not a problem. The problem is he complex world the programming happens in. Programming itself even helps to reduce that complexity though by allowing you to traverse some amount of that complexity automatically.

The problem is not, that programming is too hard for people, but that people are not willing to take responsibility for themselves and hteir problems. Most people look to others for solutions to their problems. That's also why people who promise them solutions, like Jesus or Jobs, are praised as saviours, even though they are never able to fulfill all the promises, at least in the amount and dimensions that the receiver would hope for.

> It is interesting that at one point, Jobs (who could not be reached for > comment) described his vision of computers

A kind of statement we haven't seen for a long time. RIP, Steve.

PS: The article didn't really explain why he thinks there was a Xerox Parc philosophy inside of Apple and how/why it was in decline, right? It basically just says, there was Xerox Parc with that awesome philosophy and there was Apple who copied some of it, but altogether was a closed system developer.

</comment><comment><USER20> This has been something that I've been thinking quite a lot about for the past few years. The early Apple was heavily influenced by the work that came out of Xerox PARC. Even during Apple's low point in the 1990s, Apple maintained a research lab that was the logical successor to many of the ideas that came out of Xerox PARC. The Dylan programming language would have been an interesting environment for programming Newton applications, and OpenDoc would have brought the idea of larger applications and compound documents composed from smaller applications and documents, which I find quite similar to the Smalltalk vision in some ways. Unfortunately Apple's research lab would be shuttered in 1997, but the rationale was understandable; Apple was on the verge of bankruptcy back then and Apple desperately needed to focus on its core product strengths.

Apple used to be the champion of personal computing. Personal computing is about empowering individuals by giving them access to computation in a relatively accessible and affordable fashion. Apple's mission was to empower the user through usability, and they applied the research from Xerox PARC and other places to accomplish this. Even though the classic Macintosh operating system is not as powerful as the Smalltalk environment, and even though certain important proposed additions such as OpenDoc unfortunately were cancelled, Mac OS enabled people to be more productive and more creative, and it even helped create many industries such as the desktop publishing market and the early web design market during the 1990s. Apple's usability guidelines for Mac programs were well thought out and enforced a coherent vision of usability for the entire platform. Mac OS X in the 2000s was the pinnacle of Mac, combining the Mac's focus on usability and good design with a solid Unix foundation that provided features that the classic Mac did not have such as preemptive multitasking and protected memory. And, let's face it, once competitors like the Amiga and BeOS died, Apple seemed to be the only major player remaining in the computer industry to have a coherent view of personal computing for the masses. If the Mac at its peak was like In-n-Out Burger, then Microsoft Windows is McDonald's, and the desktop environments for Linux are like frozen microwaveable burgers from the grocery store.

Unfortunately once Apple started making piles of money from the iPhone and other parts of the iOS ecosystem, Apple started to neglect the Mac (especially on the desktop hardware side) and the overall vision of personal computing as a way of empowering the masses. It seems today that all Apple is concerned about is the iOS ecosystem, which is a locked-down walled garden instead of the freedom and empowerment that the Mac provides.

Right now personal computing needs a champion. Many hundreds of millions of people, if not billions of people, rely on personal computers in order to carry out their business and creative tasks. Unfortunately there are no companies that are passionate about personal computing. Apple has become the iPhone company, Microsoft is focused on upholding its monopoly and building its cloud business, Google is all about mining personal data, and the Linux desktop world is too fragmented in order to put up a united front. Even worse, many of the major players in personal computing bought into the notion in the late 2000s and early 2010s that personal computers will be replaced with smartphones and tablets. This led to Apple's neglect of the Mac, Microsoft's failed Windows 8 Metro interface, and some odd design decisions in the early days of GNOME 3 and for Ubuntu. While the industry has been backing away from the thought that tablets will supplant personal computers, personal computing still lacks a champion.

My dream is for either a company or a team of open source software developers to pick up from where Alan Kay, Don Norman, and other people left off and create an personal computing operating system that combines the best ideas of Smalltalk, Lisp machines, Hypercard, OpenDoc, and Apple's usability guidelines from the Mac OS 8-9 days. It will be an operating system that is focused on composable documents similar to OpenDoc but with Smalltalk-like levels of flexibility and control. It will also have a strong emphasis on usability with a "back-to-basics" viewpoint instead of the flat design promoted by contemporary desktop and mobile GUIs.

<comment><USER21> I am curious to know which GUI you think was better, Mac OS 8-9, one of the versions of OS X, or the original black and white Mac OS (System 6)

I am currently writing a window manager / desktop environment, and I'm kind of torn. At the moment, I'm using the look of the black and white Mac GUI, which I prefer over the colorized Mac OS 8-9 GUI, but at the same time I am fond of the early OS X GUI (10.1 Puma or 10.2 Jaguar).

You say that Mac OS X in the 2000s was the pinnacle of Mac, but then you say you want Apple's usability guidelines from the Mac OS 8-9 days. Which GUI do you think was better? What exactly was it that made OS X in the 2000s the pinnacle?

My goal is to build a "desktop programming GUI environment" using the Objective-C runtime as the base, but with the look and feel of the black and white System 6 GUI. I've written my own simplified Foundation-subset (only the stuff that I need) as well. I'd like for it to be as simple as possible without making too many sacrifices. After all, the original Mac used 128K of RAM, the original OS X could run with 128MB of RAM, and the original iPhone also came with 128MB of RAM. It's ridiculous that everything requires gigabytes of RAM nowadays.

<comment><USER20> I should've clarified in my original post my preferences regarding the Mac OS.

When it comes to overall systems, I believe that Mac OS X is the pinnacle of the Mac due to its combined usability and stability, and I also believe that the Mac as a platform peaked around the Snow Leopard era (2009-11). Mac OS X has the usability and consistency of the Mac platform with the stability of Unix. Even though Mac OS X hasn't gotten worse since Snow Leopard, unfortunately it hasn't dramatically improved since then; if it weren't for the necessity of modern browsers and security updates, I could still be productive on a computer running Snow Leopard. And the hardware situation since 2012 has been discussed on Hacker News numerous times.

However, when it comes to usability alone, I have a soft spot for the classic Mac OS. Personally I believe the classic Mac OS's interface is actually more user-friendly than Mac OS X's (Mac OS is really simple with its controls and its spatial Finder, while Mac OS X is more complex due to its NeXT roots), and I also believe that the Mac platform back then was more compliant with Apple's user interface guidelines than today (although unfortunately I don't know of a way I could measure this). I find Apple's A/UX and Rhapsody projects to be quite interesting attempts to bridge the classic Mac environment with Unix. I also like the conservative choices that Apple made at the time regarding color. While Mac OS X's Aqua is quite impressive, there's something about the subdued atmosphere of Mac OS, whether it's the original black-and-white interfaces of System 6, the light touches of color in System 7, or the Platinum theme from Mac OS 8 and 9, which is still quite conservative compared to Aqua.

</comment></comment></comment><comment><USER22> Funny how smalltalk was one of the first object oriented language, and Kay is among the inventors of object oriented programming, and yet the vision is completely opposed to the concept of encapsulation (which is one of today's core principle of OOP).

Aka: to provide a simple interface to manipulate an object, rather than expose its internals.

<comment><USER23> > …the vision is completely opposed to the concept of encapsulation…

Why do you think that?

p.286 Design Principles Behind Smalltalk [author Daniel H H ingalls]

[https://archive.org/details/byte- magazine-1981-08](https://archive.org/details/byte-magazine-1981-08)

<comment><USER22> I'm not talking about smalltalk's vision as a programming language but rather the operating system exposing the source of every component in read / write mode.

<comment><USER23> That is not what you wrote.

</comment></comment></comment></comment><comment><USER24> I'm very much in favor of constructionism as an educational philosophy, and thus of teaching programming as a basic skill to a wide audience.

However, I also think there is a need for professional software development, and the interfacing between professional software and casual end user programmable systems is problematic in my experience.

Try dealing with an organization where business critical processes are handled by a FileMaker database, a HyperCard stack, or an Excel macro written by a rando. For extra credit, try dealing with an organization where there are dozens of copies randomly copied to different users' machines, with different strains of customizations, none of them under version control (even if the authors were willing and knowledgeable to use VCS, casual programming environments often don't seem to play nicely with version control).

I have little personal experience with pervasively programmable systems, but I can't imagine that they interface well with professional software development either. How do you install a word processor on a system where it's anyone's guess how the addition operator behaves in a given week?

</comment><comment><USER9> Gets right to the heart of the matter. A clear and well-written article.

</comment><comment><USER25> If you want to run an Alto, the Living Computers Museum has an emulator that can be downloaded.

[https://livingcomputers.org/Discover/News/ContrAlto-A- Xerox-...](https://livingcomputers.org/Discover/News/ContrAlto-A-Xerox-Alto- Emulator.aspx)</comment>


Show HN: Humbot – Duolingo for science education - <AUTHOR> <comment><AUTHOR> Hi guys,

I am the co-founder of Humbot. Our goal is to make learning a habit for more people and increase the public understanding of science. You can get the app for iOS and Android here: [https://humbot.io](https://humbot.io)

I appreciate feedback on every level and am here for questions.

Enjoy!

<comment><USER2> This looks interesting, giving it a try now. Thanks!

 __edit __: As an aside, i long for this type of thing for politics. There is obviously a lot of complicated issues, but i feel woefully uneducated in politics and it 's made far worse in a climate where News is untrustworthy and politicians lie and slander without fear.

<comment><AUTHOR> Hey, thanks for giving it a spin! Looking forward to your feedback!

We've heard this from a number of people now regarding politics. I definitely think it's an important topic to educate people on, but I still believe that the foundation for most things lies in having a basic and solid understanding of natural sciences. I would, however, not rule out that we add politics in the future.

</comment></comment></comment><comment><USER3> I am bit confused here. I install the app on Android, the bot asked few preferences and after signing up, I see two science articles on my screen. I can't even bring back the bot to ask for more articles.

Am I missing something?

<comment><AUTHOR> Hi ganessh, Thanks for the feedback! The initial questions are not directly related to the articles you see, they help us to better understand what interests you.

Initially, everyone only sees two articles. We push new articles to your feed every couple of days. The next push is this afternoon.

I understand that it's confusing that one only sees two articles without any further explanation. I am working on a coach mark right now to clarify the issue for new users. It will be available with our next release next week.

Thanks for the feedback, this is really valuable to us and I appreciate it.

Does this clarify the situation for you?

</comment></comment><comment><USER4> FYI, the link you posted is for the HN post.

<comment><AUTHOR> Thanks peternicky for pointing it out. I only realized after submitting the post that I forgot the add the URL. It wasn't possible to add it in edit mode, and deleting and re-adding the post was also not possible. So, I figured the best is to add it in the comments.

Maybe if someone from HN sees this: Could you add the URL to the post? :-)</comment></comment>


Crypto-Judaism - <AUTHOR> https://en.wikipedia.org/wiki/Crypto-Judaism <comment><AUTHOR> Stumbled across this phenomenon after reading NPR. [https://text.npr.org/s.php?sId=789864201](https://text.npr.org/s.php?sId=789864201)

<comment><USER2> In case you want to read some other comments, a version of this story was also discussed here 11 days ago at [https://news.ycombinator.com/item?id=21779174](https://news.ycombinator.com/item?id=21779174) .</comment></comment>


List of Quantum Computing Simulators - <AUTHOR> https://www.quantiki.org/wiki/list-qc-simulators <comment><USER2> This list is great, but appears to be a little out of date. A new crop of programming tools has recently emerged as the field grows from academia into industry efforts:

Forest [http://forest.rigetti.com](http://forest.rigetti.com) is an open source Python toolkit out of the startup Rigetti Computing (full disclosure I'm a co-founder of this project). Here's a link to a recent demo and tutorial:

[https://www.youtube.com/watch?v=IpoASc18P5Q&feature=youtu.be](https://www.youtube.com/watch?v=IpoASc18P5Q&feature=youtu.be)

ProjectQ is another open source Python project out of ETH, and LIQUi|> is an F# simulator from Microsoft's quantum research group.

These would be the best place to start to see the cutting edge of developments in quantum programming.

<comment><AUTHOR> Thanks for sharing. This list just happened to be the first thing I found that looked really useful, when I went looking for a list of QC tools earlier. Like a lot of the people in this thread (apparently) I don't know much about QC right now and am looking to get started learning about it. But since actual quantum computers are a bit hard to come by at the moment, I figured some kind of simulator(s) existed.

</comment></comment><comment><USER3> Quantum Circuit Simulator: Quirk (interactive, JavaScript, [http://algorithmicassertions.com/2016/05/22/quirk.html](http://algorithmicassertions.com/2016/05/22/quirk.html)) is especially nice for playing with circuits (rather than academic work).

And for one-particle QM, I wrote [http://quantumgame.io/](http://quantumgame.io/), if anyone is interested (some context: [http://p.migdal.pl/2016/08/15/quantum-mechanics-for-high- sch...](http://p.migdal.pl/2016/08/15/quantum-mechanics-for-high-school- students.html)).

<comment><USER4> I have been doing VLSI design as a student and quantum mechanics, quirks speaks to me. It feels like having real time chronogram with a board on which you plug your gates except that it is not on 1/0 bit but on amplitude and phase in parallel.

It really looks like an fun thing. I cannot see how to make from my old knowledge anything interesting though :)

I was at best thinking of using something sensitive to the initial condition and unstable to generate a PRNG, but is it possible?

</comment></comment><comment><USER5> It's interesting that they list Perl and PHP as if they are the same language or language family. They list ML and Haskell with Lisp and Scheme but CaML separately from ML. It doesn't make the list much less useful, but it's a little surprising.

</comment><comment><USER6> For someone that has 0 background in Quantum Computing, what would be a good book to start learning more about the topic?

<comment><USER7> Nielsen and Chuang

<comment><USER8> Seconded. This is _the_ definitive guide.

</comment></comment></comment><comment><USER9> EE (comp. eng) here, what's the best resource to learn about quantum computing from this background? Only did 100 level Physics.

<comment><USER2> The textbook by Nielsen and Chuang is great. If you're looking for an abridged interactive one then you might be interested in the one in the documentation for pyQuil:

[http://pyquil.readthedocs.io/en/latest/intro_to_qc.html](http://pyquil.readthedocs.io/en/latest/intro_to_qc.html)

It goes through the basics of quantum computing using the Forest toolkit.

<comment><USER9> Any online courses similar to Coursera?</comment></comment></comment>


Facebook backs down, reverses on user information policy - <AUTHOR> http://www.cnn.com/2009/TECH/02/18/facebook.reversal/index.html

<comment><USER2> already on the homepage:

<http://news.ycombinator.com/item?id=485460>

oh, and you're duping yourself!

<http://news.ycombinator.com/item?id=485716></comment>


Ask HN: Why does it take "up to x days" to unsubscribe? - <AUTHOR>

A lot of websites have newsletters or mailing lists, a lot of them are managed by internal systems and many are external.
Sometimes, regardless of whether it seems to be internal or external, when I unsubscribe from a newsletter I'll be told that my request will take "between 2 and 5 days" or "up to 48 hours" to be processed.
Why is this? Surely it's just a matter of deleting an email address from a table. Is it a ploy to keep me subscribing, in the hope that those 48 hours will be enough time for me to realise the mistake that I've made? <comment><USER2> It may be a matter of delayed syncing for certain websites with multiple legacy systems that aren't tightly integrate.

What I think is more likely, though, is that they are using a fairly sophisticated email service which is capable of generating millions of emails and queuing them up in advanced of a scheduled delivery date.

Say you had 10 Million subscribers that you wanted to email a coupon or offer to that was only good for the next 24 hours (e.g. "Saturday's 1 Day Sale"). In order to get those into 10 Million inboxes within a 10 minute window you need to pre-process as much as possible. The email marketing person schedules emails to go out at a certain time, they are generated and queued, and then pushed out in bulk. Checking if you've unsubscribed is most efficiently done when generating the emails to be queued, not when going to send.

So when you go to unsubscribe, they might have an email or two in the queue ready to go to you. It's much easier to say it will take 48 hours for your unsubscribe request to go through than it is to explain that "you're unsubscribed, but we might have a few emails in our pending queue that would be a pain for us to delete or prevent from going out, so please just forgive us during this period."

(Side note: There's a surprising amount of interesting tech going on in some, but not all, email marketing programs.)

</comment><comment><USER3> I used to work at a "very large email advertising company". We would take another companies email list and advertise to it, splitting the profit with the company that owned the data.

When an unsubscribe came in- sure, we could remove it from our own list, but we'd still have to give the unsubs to the original data owner, who'd need to then send the unsubs to any other companies that were using their data. And that's at the simplest, it can run multiple levels deep (i.e, Company A sells data to Company B who sells data to Company C).

Another reason is that "targeting" emails on a user-by-user basis can take lots of processing time, too much to be done in real-time when the email is being sent out, so they are often queued up several days in advance.

</comment><comment><USER4> If everything runs on one system in one location then yes your right. They don't and in that changes are propergated periodicaly. By saying it will take upto X days covers them on this front and in that protects there liability.

Also note that they don't do them instantly to prevent forms of account denial of service attacks by having bots create and delete accounts instantly.

But if you run your own mail server then you can just instantly add those email senders to your bounce list and get a much better effect.

</comment><comment><USER5> >Surely it's just a matter of deleting an email address from a table.

Actually, they have to add you to an opt-out table. If they just delete you, there's the chance that you'll get re-added at a later date.</comment>


The Global Climate Change Consensus: Peter Norvig's Experiment - b-man http://www.norvig.com/oreskes.html

<comment><AUTHOR> I found Norwig's little experiment interesting. Even more interesting is the response it raised with our local HN deniers who seem more interested in discrediting global warming and climate change than in building our understanding of the current state and the climate. They are the moral equivalent, in my opinion, of the folks who dispute evolution.

The denier culture is sociologically interesting. Those interested in the process should a look at John Mashey's report

The current(185-page) PDF, has a much higher production quality and a lot more information on funding and activity patterns than earlier versions. The current version is at:

<http://www.desmogblog.com/crescendo-climategate-cacophony> John suggests that you read the first 4 pages, and, then, if you want to read more, do the following:

a) Print pages 2-4, the navigational aids

b) Download the full PDF, and for on-screen reading, open a second window (Acrobat: Window>new). Use one window to read the mainline narrative, and the second for rummaging the Appendices. Different people are familiar with different subsets, so I know of no way to linearize it that makes sense.

A few highlights:

p.167-168 on plagiarism:

Not just the tree-rings, but a big chunk of the "social network" part of the Wegman Report seems plagiarized from the Wasserman and Faust(1994) textbook. Deep Climate's 4-page side-by-side is:

[http://deepclimate.files.wordpress.com/2010/02/wegman- social...](http://deepclimate.files.wordpress.com/2010/02/wegman-social- networks.pdf)

That one is interesting because it seems unlikely to have originated with McIntyre and/or McKitrick.

DC is doing a further piece on that, but after that, I may look into letting those authors know.

See Figure 2.1, p.10, for the overall flow of anti-science memes and money- laundering "cloud"

See Fig A.2.2, p.46 ExxonMobil & Foundation Chronological Funding for Some Think Tanks

That identifies only _visible_ funding for (Annapolis Center, CEI, CFACT, GMI, and Heartland), which leaves ((84%, 78%, 53%, 36%, 87%) unidentified, but it is certainly enough to be interesting. Someone with subpoena power could find out more, as there are many potential funding routes.

For amusement, see Fig A.3.1 p.50 "What's in a Name?" to see how often names like "Institute" and "Science" pop up in entities that basically do PR and lobbying, despite mostly being tax-free 501(c)3s. I don't know the (murky) 501(c)3 law enough to understand whether the amount of lobbying they do is OK or not …

However, here's an interesting GoogleMap:

[http://maps.google.com/maps/ms?hl=en&ie=UTF8&msa=0&#...
Hint: zoom in and see how many of these are located within one block of a Washington's K Street. Not every organization located there does lobbying, but I doubt that it is a low-rent district.
See Table A.6.2, especially p.96-99 which lists active people versus visible anti-science activities, more or less chronologically from 1990 to now (Climategate), including continuing attacks A.Santer, A.Oreskes (on those folks), A.GATE (the current climategate etc, might be called A.Jones), and then shows organizational connections. I'd guess someone might do some more social network analysis.
At point, I was going to do PeopleXpeople … but in that group, no one was more than 1 hop away
The attacks on the hockey stick (which could also be called A.Mann) are now split:
A.Hockey is the 2002-current visible use of attacks on the hockey stick as a pillar of climate anti-science, basically used by almost everybody in Figure A.6.2(a).
A.HOCKX is my label for the 1998-2006 effort culminating in the Wegman Report, much of which was behind the scenes.
That's the part that may well be investigated for 18USC1001 (misleading Congress) and 18USC371 (conspiracy) as per A.14, Possible Legal issues, p.184.

</comment><comment><USER2> This was possibly interesting ... before the leak of the emails that made it crystal clear that the "consensus" group systematically kept the other side out of the peer-reviewed literature.

I'd also note that "consensus" is not a word you find in real science; one inconvenient fact can destroy the position of a consensus, although as Kuhn says ([http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Rev...](http://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions)), _grossly_ oversimplifying, you might have to wait until enough of the members of the consensus retire or die.

<comment><USER3> > The leak of emails that made it crystal clear that _a handful of members of > the consensus group, on a handful of occasions, talked about keeping_ the > other side out of the _process of peer-review_.

There, fixed that for you.

The consensus of biologists is that evolution is true. The consensus of astronomers is that the Big Bang happened. The consensus of archaeologists is that the KT extinction event happened and that it was due in significant part to the Chicxulub impact. The consensus of physicists is that quantum physics and relativity are accurate models of the universe at the appropriate scales.

The consensus of climate scientists is that the planet is warming and it is anthropogenic. Why is this fact irrelevant _here_ , but not in other fields?

<comment><USER2> The talking only confirmed their community's observed actions up to that point. Anyone paying attention knew what they were doing, the emails just made it "crystal clear".

And it's funny you should bring up Chicxulub and the K-T extinction event. When the Alvarez team proposed that hypothesis, it was not ... warmly welcomed. But they were allowed to argue their point and the community eventually agreed they were (likely) correct. If their work had been in the climate science domain I don't think we would have seen the same outcome. I _know_ we wouldn't have seen the same process.

With regards to the "consensus" that the "planet is warming", that "fact" has not been true for the last decade or so, so that "fact" is indeed potentially irrelevant (we need to wait a bit longer to be sure).

<comment><USER3> > If their work had been in the climate science domain I don't think we would > have seen the same outcome. I know we wouldn't have seen the same process.

But of course, that is entirely your own conjecture. You "know" this for a fact.

<comment><USER2> I "know" this as a "fact" of human behavior. Yes, it's an appeal to authority (myself :-), but I've been watching politicized science for about three and a half decades so I'm pretty confident about my judgment.

</comment></comment></comment></comment></comment><comment><USER4> It would be interesting if some topic in the field of software got as much public attention and debate by non-practioners as does climate change.

On one side (because there have to be two sides, and presented in a "fair and balanced" way, of course), you'd have the vast majority of working programmers all agree with the position that, say, writing software in a high level language is much more productive and useful than writing everything in machine language. And on the other side you'd have some non-programmers who are "skeptical" and coming up with all kinds of arguments why machine language is best. The skeptics would point out what they say are flaws in the reasoning, gaps in the data, etc.

This line of thought is not meant to argue that man-made global warming or catastrophic climate change are true and urgent, but rather to argue that perhaps that it doesn't make sense for non-scientists to take a stand on this issue -- especially taking a stand against the position of those who have vastly more knowledge and experience with the topic. Might they be wrong? Of course. But they are probably way more likely to be right than the average bloke on the street.

<comment><USER2> Non-scientists can take a legitimate stand on the methods, e.g. the statisticians from Canada who destroyed the hockey stick. We'd see even more of that if the data, code and methods were public (pity the original raw collected data was lost (much of it might be recoverable if someone were to go back to the original national weather service sources)).

I think you're also misrepresenting the nature of the debate; it would be a lot more like everyone arguing about managed (GCed) code vs. unmanaged (C/C++/Objective C etc. with manual memory management). The other side, and plenty of them _are_ scientists and even specialists in this area, are not arguing the equivalent of "machine code is best", an idea that died _very_ shortly after the advent of symbolic assemblers.

</comment></comment><comment><USER5> I don't have the time to double check this hypothesis, I'd feel fairly confident that after reviewing the titles of the journals that these articles were published in, that the bulk of the authors are not climatologists, but rather have expertise in other fields such as biology, meteorology, or agronomy. And that their work speculates on the impact of "climate change" on their respective fields, and that they rely heavily on the work of a few select climatologists.

I think it is also very important to remember that we're likely seeing a very large selection bias with respect to what kinds of papers get published. There are very few institutions that will give research money to a scientist who thinks that everything "will be ok". The people who are yelling that we're all gonna die and the earth will be destroyed (unless you give them money), are the types that get funding.

<comment><USER6> I'm not sure I understand your last point: Usually when you ask for research money, you don't know what the outcome of the study you want to do is.

Besides that, I'm not even convinced that you are right: I would think that a whole bunch of people would be interested in concluding that AGW is not a problem, given the political climate in the US especially. If you could convince everyone that the scientists got it completely wrong, I'm pretty sure even most of them would be relieved. It's just that you can't do that by attacking one argument here and one methodology there. You need to present a comprehensive and consistent framework that shows that AGW is not happening, and if someone did that I'm pretty sure it would be taken seriously. But noone does.

Instead of grasping at straws, why don't you just accept the maximum likelihood solution: most of the papers are correct and global warming is a problem.</comment></comment>


An exploration of why Python doesn't require a 'main' function - <AUTHOR> https://utcc.utoronto.ca/~cks/space/blog/python/WhyNoMainFunction <comment><USER2> One benefit of using a main function in Python is performance. If you're doing significant computation at the top-level, it'll probably run faster if moved into a function.

I believe this comes from the fact that local-variable lookup is faster than global-variable lookup - because the former is lookup-by-index and the latter is lookup-by-name.

The same is true in Julia: "Any code that is performance critical or being benchmarked should be inside a function." [0]

[0] [https://docs.julialang.org/en/v1/manual/performance- tips/ind...](https://docs.julialang.org/en/v1/manual/performance- tips/index.html)

<comment><USER3> Another benefit is re-usability. If you want your program to be callable as module or script from elsewhere without going through the tedious shell functions, "def main()" is the only way to go.

<comment><USER4> Would that be different than checking the __name__?

      if __name__ = "__main__":
          mainMethod()

<comment><USER3> The difference is that you've encapsulated your coding logic in mainMethod() instead of writing under that if condition itself. If latter was the case, you can't call that code from another module without resorting to os shell level features.

<comment><USER5> The code encapsulated into "__main__" assumed to be module-specific, this is the purpose of that pattern. If you want to make some code available for another module, you just refactor it into separate function.

</comment></comment></comment></comment></comment><comment><USER6> I always saw Python and similar languages (Perl, Ruby, PHP maybe) as attempts at replacements for complicated shell scripts, so it never seemed odd to me that there was no main function, since a shell script has no main either. It just starts executing at the top.

Edit: Just saw that ktpsns and I made pretty much the same comment. I'll add that when I learned C I found the idea of a main function so confusing at first. I wanted to know why it had to be called main and not whatever else I wanted it to be. On the plus side, it led to me exploring how compilers and linkers work, which made me understand hardware a lot better.

<comment><USER7> Interestingly C# is introducing top-level programs now.

[https://devblogs.microsoft.com/dotnet/welcome- to-c-9-0/#top-...](https://devblogs.microsoft.com/dotnet/welcome- to-c-9-0/#top-level-programs)

<comment><USER8> Looks pretty similar to csx scripts

</comment></comment></comment><comment><USER9> There's a big benefit to Python's convention

      if __name__ == "__main__":
          ...

especially in contrast with c and other languages that shoehorn you into defining a main function. Namely, you can put a "main" into a module, and import that into another module without causing trouble.

I use this a lot during a development cycle (not so much in prod). Say I'm trying to replace a function foo with a new implementation foo_new in a large package. I write some rudimentary tests and benchmarks in "main" which compare the two methods (including a late-import of the top-level package), and directly execute that file until I'm satisfied. At that point, I'll switch (foo, foo_new) -> (foo_old, foo) and start running unit tests. Once those unit tests pass, I remove "main" (usually, promoting some fragments of it to new unit tests) and foo_old.

<comment><USER10> > especially in contrast with c and other languages that shoehorn you into > defining a main function

i don't think that's a problem in languages with namespaces/modules -- a module might define a main(), it just wouldn't be used when it's imported. (unless you're in C, then you'd get a name clash)

<comment><USER9> That's a great point... with a small caveat that some people abuse namespaces badly and clashes may still occur.

</comment></comment></comment><comment><USER11> > Many languages start running your program by calling a function of yours > that must have a specific name. In C (and many C derived languages), this is > just called main(); ... Python famously doesn't require any such function...

Nor did BASIC.

I think the better question is, why does C (and C derived languages) require a main function?

<comment><USER12> Maybe it goes back to assembly language. If you look at a block of assembly or machine language code, it's not immediately obvious where the entry point is. You could guess that it's at address 0, but I don't think that's the case for all machines.

I think more important than having a main() function is to clearly delineate the entry point. And in Python, the entry point is the first line of the program.

Now I've noticed, at least within the Arduino environment, that it uses setup() and loop() as entry points, but code is executed before these functions are called, if for instance a global variable definition invokes a class constructor. I don't know if that's how "regular" C works, but it creates ambiguity as to what runs first.

There are other languages where it seems hard to guess where the actual entry point is located unless you know where to look.

<comment><USER13> You don't get quite this level of ambiguity with standard C, since it does not provide objects (in the object-oriented sense) with constructors, in contrast to C++ that is used in the Arduino environment (in C++, initialization order of global objects is indeed unspecified, inviting subtle bugs to occur).

However, many common C compilers provide methods to declare functions to run at initialization time, e.g. gcc's `constructor` attribute, opening this rabbit hole also for C programmers.

</comment></comment></comment><comment><USER14> And still I do write a main procedure in Python usually, which is called from that if __name__ == "__main__" thing. It is convenient to leave the top level asap and begin as quickly as possible to use procedures/function, getting out of the area, where one can accidentally change global variables, without noticing (in procedures you would have to write global as a keyword and would hopefully then notice you are doing something possibly dangerous).

<comment><USER15> Absolutely. But on the other hand, it's still very useful that Python does work like this: it's nice to be able to directly do things at the top level (e.g. setting up "constants" with a little computation, conditionally importing modules or importing fallbacks, etc.). As the article says, it's also an extremely clean way of defining what gets run in what order during startup.

</comment></comment><comment><USER16> Here's a thing: Coming from a world of Perl, PHP, Ruby, Bash, BASIC, batch etc. scripts, a main function is not neccessary at all. The procedural code just starts from the first line.

With this in mind, it is weird to even ask for a main function. :-)

<comment><USER17> My immediate reaction was "it does—but it's in the interpreter so of course you don't have to write it".

<comment><USER18> And you can perfectly well compile them and keep that same behavior.

C's entry point is in the runtime too, it's not your main function (but yeah, the optimizer will probably inline your main). At the end of the day, there's no real difference between compiled and interpreted languages, all the difference is on the toolset.

</comment></comment></comment><comment><USER19> Once I worked with a library, which when imported, started printing with a printer to paper. So yes, your Python program absolutely needs a main function and please no side effects at import time!

<comment><USER20> I don't see how these two are the same. No side effects at import time? Sure. But a program isn't meant to be imported, it's meant to be run, so there's no real expectation that a program should be importable without side effects.

<comment><USER21> they're not the same, but they're both symptoms of the same underlying python behaviour: when you run a python file or import a python file, the interpreter immediately starts executing statements from the top of the file down.

</comment></comment></comment><comment><USER22> The _main_ function, in C/CPP programs, satisfies an operating standard in Unix and Windows systems for a defined program instruction entry point. With CPython, _main_ is written into the _python_ executable's source. Python scripts are not C/CPP-source executables so they don't need a _main_ , they don't need this constraint, so what sense is there in imposing it?

<comment><USER23> No, this isn't so. And besides, the main function isn't even the first thing in your program to run. The standard library does some setting up first. For instance populating argc and argv.

You may enjoy reading this: [http://www.muppetlabs.com/~breadbox/software/tiny/teensy.htm...](http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html) It explains a lot of how linux loads programs. It's a bit dated though: everything is 32 bit.

<comment><USER24> There is definitely a main function in the CPython source code.

</comment></comment></comment><comment><USER25> This article really complicates a simple concept: EVERY program needs a main(). Period. Whether it is called "main()" or ".ORG", or it is just an address given to a linker map, or simply a hex address in a BIOS boot sequence. There is no avoiding it: one instruction must come first.

Python, Perl, Ruby, Bash and every other scripted interpreted language IMPLICITLY wraps "main" around your program based on the context, otherwise it wouldn't know where to start.

I find this article obfuscating a simple point.

EDIT: When I say .ORG, I'm referring to the standard assembly language designation for "execute this instruction first", and not ICANN. ;)

<comment> munk-a Yup - and even the most obfuscated language's main (PHP) has a single entry point - that entry point is just in apache[1] which figures out which of your bazillion PHP files to execute.

This article is bordering on nitpicking.

1\. Or whatever other webserver - and actually you can directly invoke PHP via CLI, but I'm ignoring those for the most common way people tend to use it.

</comment></comment><comment><USER26> Bit snarky, but all I got from this post is "because it doesn't." Many languages don't require a main function. In fact, the earliest languages like LISP (and even punchcards!) don't require it, let alone many other languages like BASIC, awk, etc. The presence of a main function in later languages is, in a way, the outlier.

<comment><USER20> FORTRAN does not require one either, as you can see in this example:

[http://www.99-bottles-of-beer.net/language- fortran-77-264.ht...](http://www.99-bottles-of-beer.net/language- fortran-77-264.html)

</comment></comment><comment><USER27> This does seem like one of those deeply bikeshedded philosophical arguments.

<comment><USER28> Yes the argument is trivial. But it illuminates design choices of each language which is interesting

</comment></comment><comment><USER29> Why do some languages require a main function? I started out with JavaScript decades ago, and of course JS doesn't require a main function. When I started learning Java I thought the requirement that there must be a main function was just bizarre. I still feel that way, it just feels wrong.

<comment><USER30> Because languages that compile to a platform binary must obey rules of a system linker and a corresponding stdlib. Javascript is an active host for an entire js program, but C compiler is not. C source files turn into object files (.o/.obj and then .so/.dll for libraries) that have a particular format and may be linked with object files and libraries from other languages. A system linker then decides where the entry point is by looking in every object file for a _start symbol, which is usually implemented by clib, and it basically does:

      void _start(void) {
        extern int main(...);
        init_clib();
        exit(main(...));
      }

The exact procedure heavily depends on a build target. Also, C compiler cli tool (e.g. gcc) is usually a frontend for a compiler and a linker, among other tools, which may add to the confusion.

Object files are collections of symbols, either external, or defined in that file. Every non-extern symbol is a region (addr, len) with either data or code. There is no unnamed/main symbols by design. In theory, C could abstract that and allow code at column zero, but there is no rationale for that.

</comment></comment><comment><USER31> Well, the python interpreter has a main function.

This whole topic is like comparing apples to oranges. The python interpreter(which IS compiled and has a main function) defines the rules of whats required in python. And it can reasonable assume that the entry point is the beginning of the .py file its running.

</comment><comment><USER32> In JavaScript only functions declared with function first will be hoisted, so I create my own main function and call it at the end of the file.

</comment><comment><USER33> Why not keep "main" as a reserved function word and just append the magic "if name == "__main__" behind the scenes if that function is seen? For a language focused on immediate usability, it's a stupid first hurdle to have to clear to greet the world.

<comment><USER34> 1) One of pythons design philosophies is "explicit is better than implicit".

2) This is not a hurdle to hello world.

        print("Hello World")

is a complete program that does exactly what you want. Needing to wrap it in a function would be an additional hurdle.

For complex programs; that is a hurdle well worth jumping. But for simple scripts (python is a scripting language), or teaching purposes, it is not always needed.

<comment><USER35> Ad 1,

the current idiom uses dunder name, which is implicitly defined and looks special.

Making main special would be about the same level of a solution, yet more conventional

<comment><USER36> Dunder methods don't "look" special, they are. Marking them as such has side- effects, and as soon as you see a dunder, you know you're going "lower".

If "main" has to be a special function, it must keep the dunder.

</comment></comment></comment></comment><comment><USER37> Better yet, why not explore why other languages insist on having a function, or a class with methods, before being able to print hello world?

C# literally got the ability to System.out.writeLine("hello world") (probably wrong, I'm not a C# programmer) in 2020.

I suspect the answer is "we care about existing programmers more than people who are learning to program".

<comment><USER38> What's the point of teachin newcomers something they need to give up later anyway though? (once their programs get big enough to be split over several files)

<comment><USER39> Nim compiles to C, can be split into multiple files/modules, and does not require a "main" function.</comment></comment></comment>


Tesla announces solar panel system for $50 a month - <AUTHOR> https://electrek.co/2019/08/16/tesla-solar-rental-service-solar-panel-system/ <comment><USER2> I don't see the rental options anywhere on this page.

Can someone help me figure out what I'm missing?</comment>


Facebook paid 4,327 corporation tax despite 35M staff bonuses in UK - <AUTHOR> http://www.theguardian.com/global/2015/oct/11/facebook-paid-4327-corporation-tax-despite-35-million-staff-bonuses <comment><USER2> Disclaimer: I haven't read the article in full.

Corporation tax in the UK is payable on 'profits subject to corporation tax'. In laymen's terms this is roughly equivalent to to accounting profit.

Employee salaries, bonuses, etc are generally deductible; if you pay me 50K, that's 50K out of your bottom line.

The employees in question likely pay a variety of taxes including but not limited to the standard income tax. In an extreme scenario (recent graduate + highest tax band) that could be 61% rate.

(For comparison, the 2015 rate for Corporation Tax is 20%; so as far as the Exchequer is concerned, a salary payment is preferable, at least in immediate terms).

In summary, I think there's a discussion to be had here about abuse of taxation frameworks but the headline here is bait; the bonuses are irrelevant, any oddities regarding transfer payments etc. are what should be focused on.

<comment><USER3> Slightly Off Topic - I would presume this is the same everywhere else. Tax only on Accounting Profit?

If so did Amazon ever pay any tax? ( Apart from Sales Tax where it is required) Since they haven't really been making any accounting profits at all.</comment></comment>


The unexpected Google wide domain check bypass - <AUTHOR> https://bugs.xdavidhu.me/google/2020/03/08/the-unexpected-google-wide-domain-check-bypass/ <comment><USER2> Many joyous moments of genuine positive hackery. Notable for me were:

\- At several points, unusual things were done "for some reason", following an intuition.

\- The one character that does the trick was discovered by brute-forcing the browser's URL parsing logic.

\- After finding that this little trick works, other related Google apps were explored (for "some reason") to see if the trick is generally applicable, which indeed it was.

\- Deep diving into Chromium source to confirm why the trick works.

\- How nicely the ends are tied up at the end, seeing the result of his work have an effect deep in a Closure library commonly used at Google.

\---

Loved this part (my emphasis):

> I was always "afraid" to dig this deep, and would never have thought about > looking into the source code of a browser, but after browsing around a bit, > you realise that _this code is also just code, and you can understand how it > works_. This is super and interesting and can be really helpful in many > situations.

</comment><comment><USER3> The impact of this bug is probably very wide including server-side - not just a Google or Closure because as Someone1234 said: this buggy RegX was from the [RFC3986] standard on URL parsing: [https://tools.ietf.org/html/rfc3986#appendix-B](https://tools.ietf.org/html/rfc3986#appendix-B)

Edit 3: tl;dr: [/?#\\] in browsers (including backslash) ends the host/authority but the RFC only specifies [/?#] as ending characters, so for example [https://xdavidhu.me\\.corp.google.com/blah](https://xdavidhu.me\\.corp.google.com/blah) is recognised as host `xdavidhu.me` by the browser, and host `xdavidhu.me\\.corp.google.com` by the RFC - so host name whitelisting or matching can be borked for security purposes.

Edit: maybe do a search of github for this Regexp, and see what significant finds there are. Certainly bad actors are already doing so...

Edit 2: presumably there will be a gold-rush to find other bug bounties for the same underlying issue in other software or services!

Also, Google's $6000 payout seems extremely disappointing: Google should payout on the total impact of the bug that they measure (not just the one place he happened to find). Why should the reporter have to investigate the total impact of the bug on Google in order to get a fair payout?

<comment><USER4> >Google should payout on the total impact of the bug that they measure

And what is that total impact?

<comment><USER3> As said: only Google has the resources and motivation to measure the potential impact. David already identified a few places the bug might affect:

"I found this regex in the Google Cloud Console's JS, Google Actions Console's JS, in YouTube Studio, in myaccount.google.com (!) and even in some Google Android Apps. A day later I even found this line in the Google Corp Login Page (login.corp.google.com)".

<comment><USER4> >only Google has the resources and motivation to measure the potential impact.

And apparently they have judged the impact less than folks who think $6000 is too small.

</comment></comment></comment></comment><comment><USER5> One of these days, I'm building an integration test suite of all the weird corner cases in HTTP and whatnot to throw at libraries.

God help the poor engineer here. Let's say he or she (reasonably!) looked up the grammar for a URI in RFC 3986[1], the standard that defines them. Well, according to that grammar, the browser is wrong: backslash isn't valid in that position, and the exploiting string isn't a valid production of the URI grammar.

But of course, the browser has its own standard — the WHATWG URL spec — which doesn't even define a grammar for a URL, but instead defines a program in pseudocode to parse them. Normally, \ isn't valid after an authority … unless the scheme is http or https or one of several other Chosen One schemes.[2]

Uuugh. We've wrought this upon ourselves.

Also included in this test suite would be the wild and wonderful things you can type into a browser's URL bar that result in malformed HTTP requests hitting the wire… AFAIK, WHATWG doesn't yet have their own copy of HTTP…

[1]: [https://tools.ietf.org/html/rfc3986#section-3](https://tools.ietf.org/html/rfc3986#section-3)

[2]: [https://url.spec.whatwg.org/#hostname- state](https://url.spec.whatwg.org/#hostname-state)

Maybe one should argue that regex should not have accepted the URI, since it's isn't valid according to the RFC.

<comment><USER6> As someone who has also worked on various bits of HTTP and URI parsing I have also wished for such a test suite - or even that these RFCs and specs would come bundled with such a suite.

There are a lot of weird cases to consider. For example, how many tools correctly handle character encoding in HTTP headers?

[https://tools.ietf.org/html/rfc8187#section-3.2.3](https://tools.ietf.org/html/rfc8187#section-3.2.3)

<comment><USER7> I attacked it from both directions: an arguably supervised dataset of urls (hundreds of GB from various internet survey projects), and unit testing the insane markov gibberish the parser makes when you cram random data where it doesn't belong and run it in reverse. The dataset was way less helpful for a strictly conforming design, but I imagine it would be useful for a mostly correct parser.

</comment></comment></comment><comment><USER8> Windows again. "Let's use a backslash instead of a slash in paths" has to be up there with "NUL-terminated strings" in terms of tiny bad ideas from long ago that are still haunting us.

Of course it's not as though Google had a choice at the outset, I'm sure some non-trivial number of web sites magically didn't work (and maybe still don't work) if you insist that [https://www.example.com\works\fine\on\my\pc](https://www.example.com\\works\\fine\\on\\my\\pc) doesn't do what the person writing it expects... I think we can blame some early Internet Explorer build for first having this work (I can't see Netscape allowing it) and then it becomes a choice whether to be a purist or sigh and move on once IE gets popular and real people do it.

<comment><USER9> NUL terminated strings are a great representation. We can take a suffix of a non-empty NUL-terminated string just by incrementing the pointer, which is great for recursion. A pointer to a NUL is itself a valid string.

E.g. tail recursive strchr:

      const char *strchr_rec(const char *str, int ch)
      {
         if (*str == 0)
           return NULL;
         if (*str == ch)
           return str;
         else return strchr_rec(str + 1, ch);
      }

strrchr (rightmost match):

      const char *strrchr_tail(const char *str,
                               int ch,
                               const char *seen)
      {
         if (*str == 0)
           return seen;
         if (*str == ch)
           return strchr_tail(str + 1, str, ch);
         return strchr_tail(str + 1, seen, ch);
      }

      const char *strchr_rec(const char *str, int ch)
      {
         return strchr_tail(str, ch, 0);
      }

That kind of thing makes me feel that null terminated strings are the shiznit compared to some clunky header + data or whatever.

The only problem is that NUL-terminated strings are so darn easy to program with, everyone who comes into contact with them wants their share of the action, instead of using someone else's well-tested code.

As we speak, someone is likely writing yet another loop that builds up a NUL- terminated string directly, and will forget to write the final NUL in some case, or go past the end of the buffer in another.

Want to break a string into tokens on delimiters? Just overwrite the delimiters with null bytes and you're done. (Oops, the caller didn't expect their string mutated. Worse, it was a literal ...).

If you make programmers maintain a length field followed by data, or some such thing, they will quickly go off scampering for library functions to everything.

What we should never do is compare unencapsulated NUL-terminated string manipulation with the manipulation of another representation using a robust, debugged library.

Compare open-coded to open-coded, wrapped to wrapped.

<comment><USER10> NUL-terminated strings are a poor representation, feature-poor and directly responsible for a great many entirely avoidable bugs.

I much prefer the approach used by languages like Rust, where &str and &[T] are (pointer, length) rather than just a pointer.

Usage can be identical to your classic NUL-terminated string, depending on the language. (It is in Rust, with CStr.)

There is only one potential disadvantage: pointers use twice as much memory; but in practice, this disadvantage is entirely theoretical.

But there are several serious advantages: it lets you represent zero values (U+0000 for &str, integer zero for &[u8], _& c._), take slices that aren't suffixes (reducing either code complexity or memory usage, sometimes drastically), calculate the string length in constant time (and that's regularly a big deal), and avoid a pitfall that has been the cause of many critical security vulnerabilities (buffer overrun).

Want to break a string into tokens on delimiters? Easy, regardless of the length of the delimiter (it could be zero, two, seventeen—while with NUL- terminated strings it had to be one), without needing to worry about whether it's safe to mutate the string.

<comment><USER11> > You speak as though programmers scampering off for library functions to do > everything is a bad thing. I consider it a wonderful thing.

No, that's not what they are saying.

They're just saying that to blame the representation for user mistakes and then present an alternative representation where user mistakes are prevented by library functions is comparing apples to oranges.

<comment><USER10> Ah yes, I did misread it there; thanks for the correction. Line deleted, the rest stands without it.

</comment></comment></comment></comment></comment><comment><USER12> This was a really enjoyable, thorough and readable write-up. Thanks, and nice work!

<comment><USER13> I'll add that I usually have trouble following these, but it was a breeze here. Glad I read it.

</comment></comment><comment><USER14> $6,000 seems low for such a severe bug across almost all Google web properties. I would expect an order of magnitude larger.

<comment><USER15> Why? Who else is going to outbid Google for this serverside URL parsing bug?

<comment><USER16> The "competition" that Google is bidding against is all the other ways that people capable of finding this sort of bug might choose to spend their free time.

<comment><USER15> If this were Android drive-by RCE, I'd agree with you. But they're bidding for the attention of the kinds of researchers who find URL whitelist validation bugs. $6k is a strong bounty for that kind of work.

(The work is important! But people who find web bugs like this tend to make their money on volume, not on a couple major hits a year, like the RCE researchers do.)

<comment><USER17> > But they're bidding for the attention of the kinds of researchers who find > URL whitelist validation bugs.

No they're not. Just because this happened to be a URL whitelist bug doesn't mean that this is the primary work that the researcher who found the bug does. Did a URL whitelist validation researcher find this, or did someone researching user auth access find this bug? If Google knew exactly who they wanted to find this bug, then Google would have already found it themselves.

<comment><USER15> Yes, it does. Vulnerability research is a big field and it is specialized. There are different kinds of vulnerability researchers and, as an imperfect but generally useful rule, the ones doing RCE work aren't looking for web bugs. And the people running the Google bounty program are generally well plugged into the vulnerability research community; they are way, way, way, way, way, way more credible than the median HN comment.

<comment><USER15> It also just doesn't make sense as an objection, because it's not like the bounty rates on reliable clientside RCEs in browsers and mobile devices is capped at $6k, and everyone doing this work knows it.

</comment></comment></comment></comment></comment></comment></comment><comment><USER18> Something something regex and two problems. \s

Seriously though, _nice find!_ It's often surprising how tiny little curiosities can lead down the rabbit hole to such interesting problems. In this case, it was a fairly logical flawed assumption, but the mess of regex made it just hidden enough to be tough to spot.

<comment><USER19> This is one area where bug bounty programs can really outperform an internal security function. There are others, generally, but this deep rabbit hole exploration is one of the best.

</comment></comment><comment><USER20> I feel like Google is not the only company parsing domains to check security origins...

If a backslash can end a security origin, I am going to guess there are a lot of other vulnerable bits of code...

One more reason joining and splitting strings with security impact is a bad idea.

<comment><USER21> > One more reason joining and splitting strings with security impact is a bad > idea.

Doing it in a standard/shared library is more secure than bespoke code on each page, since at the very least as we witness here it can all be updated together as needed. You can add additional abstraction, but someone eventually has to do this work.

Even web browsers themselves aren't immune from bugs in "joining and splitting strings with security impact" for URL comprehension. All you can really do is try not to re-write known secure code.

Plus this buggy RegX was from the standard on URL parsing:

[https://tools.ietf.org/html/rfc3986#appendix-B](https://tools.ietf.org/html/rfc3986#appendix-B)

</comment></comment><comment><USER22> > I put this regex in www.debuggex.com. This is a really cool website

What a great find.

<comment><USER23> There is also regex101.com. It is my favourite personal.

It highlights individual parts of the regex and provides human explanation, e.g. [https://regex101.com/r/xKG7pE/1](https://regex101.com/r/xKG7pE/1)

</comment></comment><comment><USER24> This seems pretty brittle. Did anyone look into Closure to see what the motivation for the regexp based checking is? There's a browser native URL splitting function available to JS after all.

But I guess Closure dates from the IE6 era where trusting browsers for correctness was probably not a good idea..

<comment><USER25> > There's a browser native URL splitting function available to JS after all.

It's the URL object ([https://developer.mozilla.org/en- US/docs/Web/API/URL](https://developer.mozilla.org/en-US/docs/Web/API/URL)), isn't it? We tried using it at work... and found out that, as always seems to be the case, it's available everywhere _except on Internet Explorer_.

</comment></comment><comment><USER26> Again, whitelist vs blacklist. Blacklist loses.

        blacklist: [^/?#]+
        whitelist: [a-z](?:[0-9a-z]|-(?!(?:$|\.)))*(?:\.[a-z](?:[0-9a-z]|-(?!(?:$|\.)))*)*

<comment><USER3> Your whitelist loses in this situation? It accepts local domains (e.g. 'a') but not fully qualified domain names (e.g. 'myhost.example.com.' nor IP addresses. Generally when checking origin, you also wish to check port (e.g. :8000) which the Google code works for; and @username:password may need thought for some mostly obsolete purposes. That said, the whitelist does avoid a bunch of other problems (control chars, Unicode, high bit, etc).

<comment><USER3> Also, your whitelist is vulnerable to redos DoS attack - according to [http://redos-checker.surge.sh/](http://redos-checker.surge.sh/)

I'm just trying to point out that securing via Regexps is never simple, even for the very skilled. If someone posts a comment with a Regexp, it is often broken and one can usually find problems with it.

Edit: and what are the $ in there for? They look redundant.

<comment><USER26> This is not meant as a complete solution for all the worlds ills, it's just an illustration of the concept and how proper matching would avoid the issue in the OP. Adding \ to the exclusion list just plasters over the pile of garbage that is the original "magic" regexp.

$ are there because you can't have '-' at the end of the label. You'd need more chars there if you want to use this as a part of full URL parser. Or just use browser's facilities for parsing URLs.

Also noone cares if you can DoS your own browser tab/iframe. You probably can't anyway, because browser has other mechanisms to prevent that. But yes, you'd need to limit this too in real code, to the max length of the label part of the domain name.

</comment></comment></comment></comment><comment> kyle-rb I love that the fix is simply a quadruple-backslash (aka an "actual backslash, for real this time" per [https://xkcd.com/1638/](https://xkcd.com/1638/))

<comment><USER27> More than a couple of times have I chosen a different solution to a problem (especially shell scripts) to avoid having to double backslashes more than once.

Because fuck trying to explain how that code actually works to anybody else, or myself a year from now.

</comment></comment><comment><USER28> URL parsing is a mess. The specification is poorly defined, and different libraries and/or servers handle them differently. This is one kind of attack; many SSRFs exist which leverage that weakness.

The regex approach, btw, is even worse. Parse your URLs before doing things with them.

<comment><USER29> IMO the spec is well defined and simple. What's broken are the browsers. Permitting a backslash as a delimiter rather _obviously_ breaks the syntax. No surprise that security issues are introduced when the browsers don't follow an otherwise simple standard, while everybody else does.

That's what makes this crazy. Generally I agree that people shouldn't abuse regular expressions for parsing security-critical data. But it just so happens that a standards-compliant URL can be trivially parsed using a naive regular expression that relies on the semantics of the gen-delimiters character set in the standard. In fact, it's really hard to break such a naive regular expression even when we move beyond the domain of standards compliance. One glaring exception is anything that introduces ambiguity between the userinfo authority section and the path, which the browsers have chosen to do. _sigh_

A new RFC should be issued that updates the syntax to include the semantics which browsers give to backslash. Among other things, this will help to publicize the issue.

<comment><USER30> The browsers are implementing the WhatWG URL spec, not the RFC, and it mentions \ is valid for "special" URLs (scheme is http/https, ftp, file, ws/wss): [https://url.spec.whatwg.org/#hostname- state](https://url.spec.whatwg.org/#hostname-state)

<comment><USER3> There's some comedy in that URL standard:

Calculating file origin: "Unfortunate as it is, this is left as an exercise to the reader."

3.7. Host equivalence: "To determine whether a host A equals B, return true if A is B, and false otherwise."

"The application/x-www-form-urlencoded format is in many ways an aberrant monstrosity"

</comment></comment></comment></comment><comment><USER31> Great find! True hacker news worthy.

</comment><comment><USER32> Such a relief some really smart and persistent people are not evil.

Though you have to wonder, if there wasn't a bounty...

</comment><comment><USER33> That regex explorer to map is pretty cool.

Good idea - I know this would work on some places. I never thought of it like this.

</comment><comment><USER34> Its my opinion that when a regex gets this complicated it should be hand-coded as a state machine parser.

<comment><USER35> Complicated != complex ... it's a straightforward expression.

My experience is regex seems like line noise to people. If you show them the multi line commented form they get it, and can then appreciate the one liners.

I've thought a neat IDE trick would be doing that expansion, into this visualization followed by multi line text you edit, and a column to paste sample text into and see matching / back-refs / captures happen. Stays one liner in code.

Standalone tools do it, slicker if inline.

</comment></comment><comment><USER36> Thanks for the write up. Really interesting and approachable for non-security people.

</comment><comment><USER37> And.. it's down..

<comment><USER38> Down for me too. Mirror. [https://web.archive.org/web/20200309211517/https://bugs.xdav...](https://web.archive.org/web/20200309211517/https://bugs.xdavidhu.me/google/2020/03/08/the- unexpected-google-wide-domain-check-bypass/)

</comment></comment><comment><USER39> It looks like regex originally came from Appendix B of RFC 3986 so this type of vulnerability may exist in other places. According to the RFC, the regex is "for breaking-down a well-formed URI reference into its components." In this case, the URL was invalid because the host component cannot contain a backslash unless percent encoded.

</comment><comment><USER40> Well done!</comment>


Show HN: Sleepless Mac – Status bar app that prevents your mac from sleeping - <AUTHOR> https://github.com/gsurma/sleepless_mac <comment><USER2> Looks like this tool executes 'caffeinate -i -d'

There are a couple of similar apps on the Mac App Store, but this is the only open source app I'm aware of. I also like the icon.

If you have a Mac, and don't know about caffeinate, go read the man page now! You can also use it with a command argument, eg. 'caffeinate make', to make sure your Mac doesn't sleep until the command is finished.

<comment><USER3> keepingyouawake (open source): [https://github.com/newmarcel/KeepingYouAwake](https://github.com/newmarcel/KeepingYouAwake)</comment></comment>


Computers that aren't computers - <AUTHOR> http://blog.printf.net/articles/2010/02/07/computers-that-arent-computers

<comment><USER2> There should always be guitars. Without them, you won't get new music.

Problem is, not everyone that listens to Abbey Road wants to learn to play the guitar.

</comment><comment><USER3> Relax, guys. The general purpose computer is never going away.

</comment><comment><USER4> NoPC (n): A personal computing device that's neither personal, nor a computer.</comment>


Paper Laptops, SoundBulbs and Designer Power Bars: Best of Yanko Design 2009 - <AUTHOR> http://www.yankodesign.com/2009/12/24/best-of-yanko-design-2009

<comment><USER2> From the disposable paper laptop article: _I kno eventually someone will figure out the tech bit, so let's see who will take the bait._

Well that's the real trick, isn't it? But it seems to me that if we could have games like this for $20, 15 years ago: [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=12049...](http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=120490613303&rvr_id=&crlp=1_263602_263622&UA=WVF%3F&GUID=b3302d791240a0b58f7226b2ffe3b8c2&itemid=120490613303&ff4=263602_263622) we ought to be able to get a computer capable of browsing the web for that price today.</comment>