Reimplementing TeX's Algorithms: Looking Back at Thirty Years of Programming - <AUTHOR> http://www.infoq.com/news/2015/01/implementing-tex-in-clojure <comment><USER2> "According to Glenn, it is very hard to figure out what TeX code is doing, mostly due to its terseness and extreme optimization"

This quote surprises me. The TeX code is extremely well documented compared to just about any other piece of source code I have seen.

The literate programming style never caught on but I think you see the beauty of it in the TeX source code. If its hard to read due to the mix of code and comment it can be turned into a book. Or into a compiled program.

I am also surprised that optimized Pascal from decades ago, that compiles on a wide variety of different hardware and software platforms outperforms a modern re implementation.

The point being that if it was optimized for a specific cpu on a specific OS, making it really fast makes sense. But when writing code for "some" or in reality any (ok that isnt quite true but TeX runs on a lot of different hardware) cpu, and any operating system optimizing code is harder.

It does get compiled by a C backend these days though.

<comment><USER3> For a better sense of what he means, check out the video go get more detail about some of the complexities.

As a counter-point, a codebase can be well documented and yet be hard to understand if the abstractions are not meaningful to the reader. I can say this without attaching any normative judgment to the quality of the code.

If I recall correctly, Glenn found it difficult to reason about the code because many parts were tangled in complex ways. For example, subsequent processing steps often triggered previous steps to repeat in surprising ways.

To speculate a little bit, these parts of the code may be well-documented down in the weeds, but they could still feel non-intuitive in the broader context if they didn't feel consistent or non-surprising.

</comment></comment><comment><USER4> I find it a sad reflection on the "thirty years of programming" that we are a long way from making anything as a) stable, b) bug free, c) fast, or d) readable as TeX.

The entire pedagogical point of this exercise seems to have completely missed its goal, but then a hand is waved and we all learned something valuable anyway. But what? That for an industry that hates "technological debt," we are highly indebted into those that wrote our platforms for all of the points listed above.

<comment><USER5> I see this claim a lot, but it's simply not true. TeX is a relic of another age, frozen in time.

a) TeX isn't stable, it's dead. It can't generate PDF, or PostScript, or HTML. It can't handle Unicode, it can't use OpenType, TrueType, or even Type 1 fonts. Without the need to conform to external standards, it's easy to declare a defect to be a feature. Everything which makes TeX usable in the modern world is a third party program, there's an entire "life support" ecosystem written by others, with stability issues, and bugs, and worts in abundance. TeX is by definition "stable" because it's maintainer does not wish to develop it further - and that's not good for TeX, which has now fragmented in to various competing successors.

b) Much of the TeX ecosystem is buggy, CTAN packages frequently conflict with each other, in ways which are very challenging to resolve. The minimal and low-level nature of TeX's macro language is the root cause of such bugs, because it's hard to write macros correctly, and even harder to debug them. By keeping the core of TeX unchanged and minimal, the responsibility for dealing with complexity is forced onto macro package authors.

c) As for the claim that it is fast, that is questionable. TeX is needlessly slow at resolving cross-references, requiring documents to be re-typeset up to three times in a row. It also spends a lot of time doing I/O on its many macro files - the end result is waiting 10sec or more to typeset a file which should have taken under half a second.

d) TeX is written in Knuth's own private language, WEB, which makes it very hard to read. WEB uses a complex system of macros which is highly unconventional by modern standards. The code is highly non-linear, with macros jumping all over the place. Everything is a global variable. No amount of explaining or literate programming can make this mess "readable". It requires the utmost effort and study to extract meaning from - as the OP explains so well.

If you want to praise any open source project of the past 30 years, I'd suggest looking at the Linux kernel. It's alive and well, for a start.

<comment><USER4> If we are just allowed to say things are true by adding the adjective simply, then I will counter with it is simply true. :) To the points, though.

a) I actually mostly agree with this concern, with the caveat that I have fewer problems producing pdfs of documents from TeX than I do any other tool I have ever had the joy of using for the same purpose. I will also note that even just treating TeX as a "core" for LaTeX, it is still a good example of stability. Specifically in the, "I can still typeset _any_ LaTeX document I have." This is far from true for opening pretty much any other file I have. Period. Hell, even just compiling old c programs is less guaranteed.

b) Applications have bugs. Pretty much period. While the overall ecosystem has unforeseen bugs, especially when you mix and match different contributions, I still know of pretty much no examples of a core as large as TeX that is essentially bug free. Care to provide any examples? And, to my point, I would wager there are more bugs in this clojure rewrite than there are in the original.

c) This is pretty much only claimed because of the failure of the clojure version to achieve similar speeds. I mean, if we are fine with setting contrived expectations ("should have taken under half a second"), then I don't understand why I have to wait at all for a fully typeset product to emerge. In the meantime, I have yet to see a product that competes in speed. Tons of claims for those that should be able to, but few actual examples.

d) Is it a language you are familiar with? Obviously not. Does it require effort and study to extract meaning? I'd actually agree that it does. I fail to see how that is a detriment, though. I have yet to see _any_ solution to _any_ non-trivial problem that does not require the same.

Specifically to the complaint of non-linearity of the code. Code is pretty heavily understood in a non-linear fashion. Same for many other topics, actually. One of the things I actually admire about WEB is that it is non- linear and allows the author to introduce a narrative into the process. Seriously, this is hugely beneficial once it is understood.

<comment><USER6> > Applications have bugs. Pretty much period. [...] I still know of pretty > much no examples of a core as large as TeX that is essentially bug free. > Care to provide any examples?

Linux. You can run still run a statically linked executable from 20 years ago. Of course, Linux today has many bugs, but if we use TeX's definition of stability we are only allowed to use the features we had 20 years ago.

qmail. Bernstein's bet is still open.

sh, ed, vi, grep, sed, and more of the default UNIX tools provided you restrict them to their POSIX functionality (no Unicode etc).

<comment><USER4> sh, ed, vi, grep, sed and such are all good examples. But, much smaller than TeX. Also, probably much less understood at the source level.

Linux is actually a good example. Of course, one of Torvalds' main drivers is "never break user space." Which is essentially the stability I am talking about here.

It is also a great example, in that it is also not written in a way that academia approves of. Is famous for this, actually. (Among many other points, of course.)

I'll have to look up Bernstein's bet.

And, you'd be surprised how many statically linked binaries from 20 years ago won't run on modern setups.

<comment><USER6> Bernstein's bet: [http://cr.yp.to/qmail/guarantee.html](http://cr.yp.to/qmail/guarantee.html)

Since IBM z-Series just hit the frontpage, you can add lots of COBOL and PL/I code which runs unchanged for the last 50 years. However, it is all proprietary and impossible to estimate.

<comment><USER4> Thanks for the link. That is a rather fun read. Is interesting to read someone that has "mostly given up on the standard C library." I think I understand and agree with the reasoning, to be honest, but is still far far from what is recommended in academia. Or industry, for that matter.

The COBOL and related code is an interesting data point. I would be interested to know just how bug free it all is, versus just used in ways that don't trigger bugs. Still, they definitely count as long running software, if not highly ported.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> I read basically the whole of the TeXbook (the spiral-bound book documenting the TeX language and implementation) many years ago, and found it fun and quirky, and very educational regarding how typesetting is done, and how typesetting practices can be transported into a computer implementation.

It was, for example, where I learned what an "em" is, the difference between a hyphen and an em-dash, what a kern is, what italic correction is, what leading is, and on and on.

But I found the language description not very clear. There are passages in the TeXbook that refer to different stages in processing as the stomach, mouth, gullet, etc. I later read an offhand comment that Knuth did not use a conventional lexer/parser setup in implementing TeX, and that decision made TeX more ad hoc from a language point of view. This, in turn, may have made the macro expansion setup more complex, with its expandable vs. not expandable tokens, restricted modes, etc.

I wonder if others who have more experience with language design had the same reaction?

<comment><USER8> I think that there aren't many fans of TeX's design. (Well, _I_ rather like it, but probably because I am not a language designer.) Seibel ([http://lambda-the-ultimate.org/node/3613#comment-51120](http://lambda-the- ultimate.org/node/3613#comment-51120)) quotes Knuth as saying (in "Coders at Work"—but the relevant page is not on Google books):

        Mostly I don't consider that I have great talent for language design.

<comment><USER7> Yes, I guess to put a finer point on it, lex and yacc were available in 1975, and the first version of TeX was started in 1977, with a 1982 rewrite.

(OT, I just learned that _that_ Eric Schmidt was a co-author of the 1975 version of lex.)

<comment><USER9> Some writers of real compilers (vs. school projects) even today would avoid lex and yacc. Stroustrup, for example, wrote that in hindsight he shouldn't have used yacc for his cfront, apparently it brought him more pain than benefit.

Yacc and lex (and even more modern versions of the parser generators) have good "educational" value, but are far from being the silver bullets in practice.

<comment><USER10> Yes. David Conroy who wrote the first couple of Mark Williams C compilers, said of yacc "It makes the hard part harder and the easy part easier".

<comment><USER9> To add my example: For my first and simple compiler used in the commercial application I used yacc. The start looked simple, then I hated it. It looked to me that I invested more energy in managing to do something "the yacc way" than to actually do something. For my more recent one, much more complex, I avoided yacc using a variant of recursive descent and never came to the point to miss anything, quite the opposite.

<comment><USER10> The one production compiler I worked on used YACC because we thought it was a good idea, but this was probably correct in that the PhD that did the parser part was a student of Hopcroft. She actually published a paper about error correction based on the work there.

Any compiler I write these days would do what you are saying--recursive descent. There is a nice technique called "chart" that helps with this.

<comment><USER9> Thanks. Do you mean this: [http://en.wikipedia.org/wiki/Chart_parser](http://en.wikipedia.org/wiki/Chart_parser) It seems it's something to be more used for the natural language parsing, less for the programming languages?

<comment><USER10> That turns out to not be a very good writeup. The published work is [http://dl.acm.org/citation.cfm?id=801348](http://dl.acm.org/citation.cfm?id=801348) and Dick Vile did a lot of work on it years after that, using it to describe an in-house production language.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> "Clojure implementation was of course far slower than TeX."

I think that sentence is the most important one in the article. I'd like to know how much, "far slower" sounds like some order of magnitude.

<comment><USER11> I found that bit slightly disappointing, and almost dismissive against the PL/compiler/optimization research done in the past 30 years or so. I don't believe that a modern clean rewrite would inherently need to be "far slower" than the original.

<comment><USER12> Compared to something extensively hand-optimized by Donald Knuth, it doesn't seem that surprising, especially considering Vanderburg went with better abstractions in preference to optimization.

<comment><USER4> I find it hard to consider the abstractions "better" when they are so far behind the performance of the ones that were used. Higher level? Certainly. Better?

<comment> dj-wonk Glenn's goals are clearly tilted towards education and intelligibility. So yes, the abstractions are better by his definition.

<comment><USER4> I still reject that they are better, and more that education should strive for better results. That is, if you define them as better with no argument allowed, then they are of course better. However, if you were to objectively compare them... I am unsure why they would be awarded the success. Again, higher level and/or easier to understand? Sure. Better?? By what metric?

This would be akin to declaiming that Newtonian physics is "better" than quantam, because it is easier to understand.

Also consider that TeX is still one of the more widely read source programs in existance. And there have been frighteningly few bugs. There are undoubtedly more bugs in this rewrite with "better" abstractions than there are in what is being rewritten. So it loses stability. It is slower. So it loses speed.

What, exactly, is it truly better at?

As indicated in another post, I am unsure that this really succeeded at the pedagogical goals that were intended. If anything, it almost shines as an example of how we have gone awry in teaching developers.

<comment> dj-wonk I found his emphasis on history to be unusual and refreshing, but we don't certainly don't need to agree on someone else's goals for their project.

That said, I personally don't see a lot of value in being too critical of projects with different goals. (That feels a little bit like back-seat driving where you have a different destination in mind than the person behind the wheel.) If someone has the motivation and creativity to embark on a project, good for them.

If you or someone else wants to port TeX with different goals, that's also fine.

In any case, until Glenn's source code is available, neither one of us can say very much about the code he has written.

<comment><USER4> Oh, I am not against the excercise. I would even go so far as to applaud the work. What I do not care for, is essentially sloppy science in how we declare successes.

I mean, this is essentially back-seat driving Knuth, of all people.

And... TeX has been widely ported as is. I am in the crowd that is somewhat unconvinced it needs a rewrite.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> Was there a link anywhere to the source code? Would be interesting to take a look. I'm currently learning Clojure.

It's not on his Github-account ([https://github.com/glv](https://github.com/glv)).

<comment> dj-wonk Good point. I haven't found the Cló source code. To make it educational for others, I hope Glenn V. shares it.

</comment></comment><comment><USER14> The problem with TeX is that it gets most of its current power from the large library of packages written on top of it (or on top of other packages, e.g. LaTeX). People expect those to keep running on any new implementation of TeX. Which means that you need to be almost 100% compatible; you cannot really mess with the TeX language in any way. So the best you could hope for is a clean implementation (or whatever we perceive as clean these days) of a processor of an ugly (again by today's perception) typesetting language. It's not clear that that is worth the quite substantial effort.

You can of course aim for a new typesetting language and just take the typesetting algorithms from TeX, but then it's a completely different product, incompatible with existing TeX packages.

<comment><USER15> The packages are secondary; TeX stripped of all its accouterments already solves a really really hard problem, and is extremely fast and free to boot. The rate of bug discovery has slowed to about one a decade so there is really no need to mess with the internals at this point. It "just works". I think TeX will be with us for a very long time yet.

<comment><USER16> Hopefully someone will get so fed up with it that they just make a powerful replacement from scratch. (Maybe with a TeX export utility.)

<comment><USER17> Alternately, you could rely exclusively on a TeX export utility and make a clean, less verbose language that compiles to TeX. Then existing packages could be either wrapped similarly or inserted as TeX blocks.

<comment><USER18> I think one reason most people wanting a next-gen-TeX project don't aim to compile to TeX is that some of the biggest pain points of TeX are baked into the core. So if you want to improve them, you need to change or replace at least some of the core layout algorithms, not just the front-end input language. For example a big wishlist item for many years has been some kind of improvement on TeX's quite frustrating figure placement, possibly with a more pluggable layout algorithm.

</comment></comment></comment></comment></comment><comment><USER19> I think it would be great if there was a concerted effort to do such a reimplementation (maybe in clojure, but maybe in another language, eg Python). This may be done in the style of NeoVim, I am sure a similar or even greater amount of funds could be fundraised if a project leader with sufficient credentials steps up to do it.

The challenges are of course formidable, as detailed in TFA. Any such project would have to maintain almost full backwards compatibility, unless the huge amount of work that has gone into CTAN is to be lost.

<comment><USER20> What's TFA?

<comment><USER21> "The Fine Article". Originated from Slashdot, I believe.

<comment><USER22> That's a joke, right? :) I hope that's a joke... I remember RTFM and RTFA from usenet before the web existed.

<comment><USER21> The phrase existed long before, but as far as I know with a somewhat different meaning, due to the Usenet connotation of "article". I was referring specifically to the usage in the comments of a news post, applied to people who have only read the other comments but not the actual article.

</comment></comment></comment></comment></comment><comment><USER23> There are many claims that I think are incorrect, misleading or unrelated.

> _According to Glenn, it is very hard to figure out what TeX code is doing, > mostly due to its terseness and extreme optimization, as outlined above_

The TeX source code is in literate programming, so it's not "terse" in the standard sense.

> _there was no IEEE standard for floating point arithmetics;_

I always suppose that the integer arithmetic was to ensure 100% portability. Every system handles integer equally (if you avoid undefined behavior), but there are lot of small incompatibilities with the floating point numbers (for example, single, double or extended precision).

> _portability meant supporting almost 40 different OSes, each of them with > different file system structures, different path syntax, different I /O and > allocation APIs, character sets;_

The original version of TeX was ASCII only, and using other character sets is always a problem. (For example, in LaTeX you must use a package like inputenc.)

<comment><USER4> I would be curious to know if the woven sources were used over the tangled ones. As probably the only program I own in printed form, the source is much more readable than I would have ever thought possible.

</comment></comment><comment><USER24> After reading the article, it sounds like a terrible idea to port that sort of program directly to Clojure (gotos, globals, and mutable state) without performing an intermediate port to a C-like high-level language first.

TeX may have fewer dependencies (given that it predates many libraries), so `Go` would probably a good target for an intermediate port - imperative/procedural, globals, goto support, static compilation.

Modernize this program in Go, then port it to Clojure.

<comment> dj-wonk Perhaps Glenn has different goals than you might expect? The video goes into this better than the InfoQ article.

In short, I think Glenn embraced the differences and challenges of adapting TeX in Clojure from an educational perspective. I don't think he intended to port it quickly or verbatim.

Glenn wanted to head-to-head comparison to promote education. To explain, in Glenn's talk around 11:36 ([https://www.youtube.com/watch?v=824yVKUPFjU](https://www.youtube.com/watch?v=824yVKUPFjU)) he says:

> While I'm glad that the source code of TeX is still available to study, I > sure wouldn't point a new programmer toward it because we should not be > doing programs today like we did then.

> But what might be really valuable is to have a modern reimplementation > written in a modern functional style to study alongside [the original TeX] > to see how much has changed and to appreciate what he have and the value of > improved tools and techniques and the tradeoffs that have been made.

His slide on his educational goals can be seen at 12:06:

> Illustrate what's changed

> Demonstrate values of expressive code

> Provide real examples of expressing procedural algorithms in functional > style

> Show how functional programs are easier to reason about

> Show different styles of optimization

</comment></comment><comment><USER25> It's written in an obsolete programming style, and I'd never write code like that today, but boy did I learn a lot from reading through the entire source code of TeX.</comment>


Ask HN: Modern OLAP Cube? - <AUTHOR>

What are startups using as alternatives to Oracle and Microsoft for olap data?
The only decent open source project I can find is Mondrian on top of Postgresql.
I'm investigating cubes because I believe it fits my current problem. We have a star schema that we currently just run tons of different reports on. We take advantage of Postgresql materialized views and abuse the crap out of functions.
One of the larger problems we have is dealing with non-additive metrics. E.g, we might want to see how many distinct ip addresses are in a time frame. And since "distinct" is not additive we currently keep large sets around in the rollups to make them additive. I've investigated HyperLogLog data structures to solve this problem but I couldn't get product management on board with probabilistic data structures.
I'm looking forward to hearing what other people are experimenting with. <comment><USER2> Unfortunately for you, my work causes me to focus on the Microsoft stack, so I can't give any specific product recommendations.

That being said, technology-wise I can suggest a columnstore database to address some of your needs, particularly the need to use the distinct operator. The storage format of a columnstore lends itself to aggregation and distinct operations, with multiple orders of magnitude improvements possible compared to SQL.

Additionally, traditional OLAP does not handle distinct operations all that wonderfully to begin with, so it likely wouldn't help you too much for that option.

Further, you've not provided much detail about your dimensional model or your use case. It's worth exploring different fact table structures. The Kimball Data Warehouse books are always worth reading.

</comment><comment><USER3> Apache Kylin, created by eBay, is a modern cubes on HBase solution. However, the overhead of HBase is likely not worth it unless you have a data size problem.

[http://www.kylin.io/](http://www.kylin.io/)

</comment><comment><USER4> I'm not sure what language you're targeting, but you might look at [http://cubes.databrewery.org/](http://cubes.databrewery.org/)</comment>


Serverless Raspberry Pi Cluster with Docker - <AUTHOR> https://blog.alexellis.io/your-serverless-raspberry-pi-cluster/ <comment><USER2> > 'Serverless is an architectural pattern resulting in: Functions as a > Service, or FaaS'

Then call it FaaS - it's not serverless and that term is misleading, marketing bunk. Even the title of the post describes the servers used in it's deployment, I'm quite sick of this term - I don't find it at all helpful when describing application architecture.

<comment><USER3> I disagree; I find it very helpful.

It's not terribly complicated either - if you are only concerned with code, then your deployment is 'serverless' in the sense that _you are not concerned with servers_.

Of course 'servers' are involved; the point is whether that's of concern to your deployment.

<comment><USER4> I'm not sure that distinction is clear. For example, in AWS Lambda, you still have issues like functions being "warm" or not, and people write code specifically to handle that. Meanwhile, nowadays you can acquire and configure even dedicated machines using just code.

<comment><USER5> If you have to configure a machine, even in code, then you aren't "serverless". To me, serverless means that you do not manage the machines or even have direct access to them.

<comment><USER4> What's a machine? :) A dedicated server sure is, but a VPS? An Heroku Dyno? What's the line?

<comment><USER6> > What's a machine?

I would say "machine" means the hardware and/or the operating system of the "server" (which is a fancy word for "computer").

The "and/or" part being very important!

</comment></comment></comment></comment></comment></comment><comment><USER7> This may be a stupid question, but is there an actual use case for building clusters of raspberry pi's?

<comment><USER8> Part of me wants to believe that a solar powered raspberry pi cluster has a legitimate use case. The things use like ~1W of power, so the board itself is surprisingly efficient compared to a "real" server.

But really, I think the primary use case is cost. Actually having access to that many physical machines to play with in a classroom or home learning environment is sort of new! The market hasn't really had such accessible linux computers at "Ehh, if it breaks I'll just buy a new one, no big deal" prices. It's educational, and the more stable the ARM support is, the better a student's skills will transfer over into the real world of systems administration.

<comment><USER9> > _The things use like ~1W of power, so the board itself is surprisingly > efficient compared to a "real" server._

Try 3.5 watts[1], not counting overhead of most USB power bricks being incredibly inefficient.

A current-gen 35W laptop CPU will be some 10 times faster[2] as a RasPi, have _much_ faster storage available (SATA3 or NVMe versus… USB2), _much_ faster I/O (GBit LAN and GBit Wifi versus… USB2), and a lot of other benefits. (Like an integrated screen and battery and keyboard and …) It also won't need external hardware to communicate with other cluster members – that 10-port ethernet switch will need power, too.

 _One_ RasPi is relatively energy efficient; RasPi clusters… not so much.

> _But really, I think the primary use case is cost._

Indeed.

[1] [http://raspi.tv/2016/how-much-power-does-raspberry- pi3b-use-...](http://raspi.tv/2016/how-much-power-does-raspberry-pi3b-use-how- fast-is-it-compared-to-pi2b) , see the numbers for "Multi-threaded CPU Tests", which is the most applicable for server workloads

[2] Running that script manages ~9 runs/second on an i7-6700HQ, vs. ~0.9 run/second on a RPi3.

<comment><USER10> 6700HQ is 45 W TDP?

<comment><USER11> There are 35W configurations

[http://ark.intel.com/products/88967/Intel-Core-i7-6700HQ- Pro...](http://ark.intel.com/products/88967/Intel-Core-i7-6700HQ- Processor-6M-Cache-up-to-3_50-GHz)

<comment><USER12> /Looks at price...

Gets you 11 Pi's ... Gets you only 1 Intel CPU, no memory, motherboard, heatsink, fans.

Reminds me of the Celeron® Processor J3455... 10W rating on Intel there page. On AVERAGE! Then when you see the real power usage under load for MB + CPU + 16GB memory, its actually doing 35W.

Where as the Pi's are doing 3.7W max per piece. So even with 4 pieces to match the performance, your still half the wattage.

If Intel really scaled that good in power vs performance, why are we not seeing x86 phones all the time?

</comment></comment></comment></comment></comment></comment><comment><USER13> If you still like going "serverful", but want an easy method of deploying to your Raspberry Pi using Docker, check out this open source Docker Hub for ARM alternative

[https://marina.io/](https://marina.io/)

(Full disclosure: I am one of the authors)

<comment><AUTHOR> Marina looks very cool and I'll take a look at it soon. Let's get in contact somehow.

<comment><USER13> Sure, that would be great. Yeah you have our email and other contacts on – [https://cloudfleet.io/](https://cloudfleet.io/) or mine directly on [https://metakermit.com/](https://metakermit.com/)

</comment></comment></comment><comment><USER14> Does anybody really use this FaaS thing? This is the weirdest of the New Things I've seen recently.

<comment><USER15> Like most new tech, it has it's applicability. Depending on your needs, and the characteristics of your application load, it can save you money.

<comment><USER16> Could you give an example of a scenario where it can actually save me money?

<comment><USER17> I suspect it makes economic sense only when your workload is relatively elastic, having a relatively low duty cycle. The ability to pay less when you aren't actually using any resources is likely of more economic benefit as it becomes more fine-grained. If you aren't in that position, other models of lease or ownership of computing resources probably merit consideration.

<comment><USER16> Well, to be honest I pay 40€/month for a baremetal server (i7 SkyLake, 64 GB RAM, 4TB HDD). It's a powerful machine running many services, including a few virtual machines. I consider whatever it does would qualify as low-duty. Now, I know that each month I'm paying 40€ for all this. The last time about I read about serverless was when someone directly discovered the money saving part is quite tricky: [https://news.ycombinator.com/item?id=14982220](https://news.ycombinator.com/item?id=14982220).

</comment></comment></comment></comment></comment><comment><USER18> This is super awesome and fun but personally I had to migrate my micro datacenter from pis to nucs.

The "armhf tax" is that you tend to have to build your own images for stuff :( Then you need your own build infra (or "heath robinson" qemu builds) because pis run out of memory building a lot of stuff... but mainly if C++ is involved so ymmv.

That said, I got a rack of 8 pis doing nothing right now, so...

(unrelated [http://www.bitscope.com/product/BB04/](http://www.bitscope.com/product/BB04/) is handy if you want to rack a lotta pis, not affliated...)

There is probably a micro business for someone running a slick docker build system for armhf handling the qemu emulation or toolchain dirtiness "under the hood" in the cloud somewhere, on x86-64 boxes with a lot more than 1GB of RAM.

<comment><USER19> You could try running a Raspberry but add a Network Block device from another machine and. (xNBD: [https://bitbucket.org/hirofuchi/xnbd/wiki/Home](https://bitbucket.org/hirofuchi/xnbd/wiki/Home) )

Then export a RAM disk from that machine and add the NBD disk as swap on the Raspberry. It would be slow, but builds would complete. Then you'd need only one low-to-moderate power machine (a PC presumably) in your Raspberry cluster, just with lots of RAM in the one PC.

<comment><USER20> Scaleway has a good array of ARM cloud offerings: [https://www.scaleway.com/armv8-cloud- servers/](https://www.scaleway.com/armv8-cloud-servers/)

Could this work for speeding up builds of ARM images and then deploying locally?

<comment><AUTHOR> Packet.net can go one further - take those 8 cores and upgrade to 96 cores and 120GB RAM.. it's ARMv8 which will be next for OpenFaaS when the Docker support catches up :-)

</comment></comment></comment></comment><comment><USER21> Serverless meme means that all I got is a chrooted directory with only systemd and /lib but without signals, similar to an apache vhost with a cgi-bin, so I could run my full-stack crap and it is supposed to be so damn cool because actual server maintenance is now someone else's problem?

</comment><comment><USER22> great article, been a huge fan of your stuff in the past. Helped me get some good ideas of things to use my Pi and Clusterhat with. [https://clusterhat.com](https://clusterhat.com)

</comment><comment><USER23> So this blog instructs to install commercial docker instead of moby. I wonder how the licensing goes if you were to use this.

<comment><USER24> I believe it actually installs the Docker community edition (CE) which is under the Apache 2.0 license. Docker (the product) is assembled using the Moby libraries and components.

<comment><AUTHOR> Correct, Moby is not a distribution of a container runtime.. it's not the "docker" you are looking for. Docker CE is and that's what's used in the guide.

</comment></comment></comment><comment><USER25> I'm surprised that the raspberry pi monoculture prevailed even until today.</comment>


Tell HN: How old were you when you realized you wanted to work with technology? - <AUTHOR>

I want to steer clear of directly commenting on what PG may or may not have meant in his most recent interview. It did get me wondering though about when people had that "a ha" moment that they wanted to have a career in technology. How old were you? What triggered the realization? I always thought I would work in Finance or Banking etc. However, I landed in the Bay Area in 1995 and never looked back. Working with, in and around technology has been my thing ever since (I'm not a developer). Technology companies were (are) doing the most exciting things to change the world via their products and services. Secondly, they tend to also do the most progressive things to change workplace culture. That got me hooked. <comment><USER2> 25.

Funny, that. Since I've been writing code on a daily basis since I was 7 years old.

But the thing is, before around 1996, "Computer Programmer" was a kinda lame job in quite low demand with no real career prospect that fat bald guys did in the basement of some Fortune 500 company making a sad corporate salary and basically not having anywhere near as much fun as, say, a Mechanical Engineer. So that's what my degree is in.

Of course, anybody who's Engineered his way out of a paper sack will know that even in the early 90s that took several thousand lines of code and (back then) quite a few hours queued up on the VAX waiting for your models to run.

Now imagine you're living large in your felt cube, several floors up from those poor programmers, then suddenly somebody shows up with a ten billion dollar sack of cash saying "Hey, anybody want to double your salary?" and that the only requirement to do so was to be able to cobble together a Hello World program that output a few angle brackets.

So yeah, hey, sign me up. I'll work with technology.

Pretty smart move, looking back on it nearly 20 years later.

</comment><comment><USER3> October, 1997. I was 19. I'd just arrived at university to study Broadcast Engineering with dreams of directing films, and I used the internet for the first time. I was hooked instantly, changed all my ambitions practically overnight, and never looked back. I was very fortunate that I could tailor my degree towards internet and multimedia "broadcasting" instead of video really. Good times.

</comment><comment><USER4> I think I was probably about 19, I had started my first company at 18, I wrote my first line of code at 12 and started to leverage my knowledge of it with my company about 6 months in but I'm from Louisiana so surrounded by lots of blue collar types and honestly had no idea I could make a living in software much less a good one!

Once the company started to become profitable I started to get excited about all the possibilities and then I realized that my love for spending more time solving hard problems once and then repeating my newfound solution over and over again with minor tweaks or improvements was perfect for having a career in software.

I sold that first company and I've spent very little time as an employee but I've started three companies now and have been fortunate enough to have sold my first company and been a part of two of the best accelerator networks with the other two.

</comment><comment><USER5> 30\. During my graduate program in Economics, I was obsessed with behavioral finance but couldn't do anything with it. I thought it was too late for me to learn programming. Finally I bit the bullet and learned to code my first trading algorithm in VBA(!).

It was so liberating, because I always thought I was destined to be a thinker and not a maker.

Fast forward 5 years, I am now building a tech company (big data on social graph) to implement my economic ideas.

</comment><comment><USER6> 97\. I was 14 at the time. I was installing Linux on old workstations on bread racks in my father's basement (where I also lived) and running distributed.net's distributed computing client to break 3DES and RC5. I've never looked back (31 now, VP of Ops).

</comment><comment><USER7> 12 or so. This was 1996 and websites were getting to be a big thing, so I taught myself basic HTML and started making websites. I didn't get into any real programming until I was 24, staying with web design until then.

</comment><comment><USER8> i think it was '97, I was 23 and I met internet for the first time. Never programmed before. But there and then, I fell in love with computers. Sarted making music on the computers. Started doing illustrations on the computer. Learned programming.

</comment><comment><USER9> I was in 9th grade.

</comment><comment><USER10> Almost 30.

</comment><comment><USER11> A poll would have been a good idea. 5 or so?

</comment><comment><USER12> About whenever we got our NES, so about 7 or 8. When we weren't playing games, my friends and I would spend time drawing (on paper) graphics and concepts...usually to Final Fantasy derivatives.</comment>


Try Dart - <AUTHOR> http://try.dartlang.org/

<comment><USER2> How Opera is not a modern browser? Come on, it's not hard testing ajax calls...</comment>


Java autoboxing/unboxing madness (2007) - <AUTHOR> http://arstechnica.com/civis/viewtopic.php?f=20&t=168180

<comment><USER2> JLS: [https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.htm...](https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.25)

Otherwise, if the second and third operands have types that are convertible (§5.1.8) to numeric types, then there are several cases:

... binary numeric promotion (§5.6.2) is applied to the operand types, and the type of the conditional expression is the promoted type of the second and third operands.

... If either operand is of type double, the other is converted to double.

<comment><USER3> Right. This is only tangentially to do with autoboxing, the same would happen for (true ? 1L : 0.0).

</comment></comment><comment><USER4> I personally never understood why java has built in types vs. reference types. Why should I care, instead of just letting the compiler handle that for me?

<comment><USER5> Keep in mind that Java came out about 5 years before a certain similar language that has reference types. The designers of that language made some decisions based on trying to learn from things that didn't work out so well in Java.

To some extent both languages were trying to deal with C++'s having three different kinds of types: Atomics, objects and structs. If I had to tell a made-up story, I'd guess that Java's designers looked at that and decided structs were in many ways just a more limited version of classes with some confusing semantic quirks, and therefore decided to nix them. That left atomics and objects, and the rest is history.

C#'s folks then reconsidered it and realized that, from a high level perspective, atomics are really just a special case of structs. (Side note: still telling that made-up story.) So they kept structs, unified them with atomics in the language,* and let the compiler deal with the rest. It works well and seems like an obviously better way to do it, but I'm not sure that insight would have been so obvious in the moment.

* Edit: Oh, and made structs a subtype of System.Object. Unlike Java, .NET's more-or-less fully object-oriented.

<comment><USER6> C++ doesn't really make a difference between builtin and user-defined data types. They all have value semantics, and a lot of effort has been spent to make user-defined types able to mimic builtin types as closely as possible. Classes and structs have no semantic differences besides default visibility of member names. For reference semantics, you use explicit references.

Java wanted to enforce reference semantics (and heap storage) for classes, but still wanted to retain value semantics (and stack storage) with fundamental types for performance reasons. The result is the mess we have now.

</comment></comment></comment><comment><USER7> I'm not a java guy but having to deal with type conversion in Go isn't auto- boxing a good thing ?

<comment><USER8> In general, it is good. But when combined with generics, weird stuff happens. Say you have some generic collection of Integer (e.g., Map<Integer, String>) and you retrieve some autoboxed value that was coerced into a Long and you try to use that as a key. You might get a null result where you would expect otherwise. It makes you want to pull your hair out and undermines the confidence that type safety gives you in general.

Autoboxing and generics are clearly bolted onto Java and it would be designed differently if they were there in the beginning. This makes me doubtful when Go adherents say that they can just add generics later.

</comment></comment><comment><USER9> Few years back when I worked in Java, this was the most asked question in technical interviews.

<comment><USER10> It still comes up quite often. Autoboxing and type erasure are two of the biggest mistakes that have been inflicted upon the Java language after its original release. Not knowing about them will lead to a world of hurt.

They should have killed the primitive/object distinction, turning the primitives into stack-allocated objects and rewritten the collections libraries. But no, syntactic sugar is better than actually fixing the problem for those that want a fix.

<comment><USER11> >They should have killed the primitive/object distinction, turning the primitives into stack-allocated objects and rewritten the collections libraries.

That would have been a mistake, because it would have made Java unreasonably inefficient for manipulating large data sets. You really need both, which is why they included both originally.

Autoboxing is a problem in that it allows you to pretend you don't have to check for nulls when you really should be checking for nulls. If I had my druthers I'd forbid autoboxing on my projects.</comment></comment></comment>


Robert Peel's Principles of Policing - <AUTHOR> https://en.wikipedia.org/wiki/Peelian_principles <comment><USER2> >"The Peelian principles summarise the ideas that Sir Robert Peel developed to define an ethical police force. The approach expressed in these principles is commonly known as _policing by consent_ in the United Kingdom and other countries including Canada, Australia and New Zealand.[1][2][3][4]"

Related:

Consent Of The Governed:

[https://en.wikipedia.org/wiki/Consent_of_the_governed](https://en.wikipedia.org/wiki/Consent_of_the_governed)

Two Treatises Of Government by John Locke:

[https://en.wikipedia.org/wiki/Two_Treatises_of_Government](https://en.wikipedia.org/wiki/Two_Treatises_of_Government)

>"The Second Treatise outlines a theory of civil society. Locke begins by describing the state of nature, a picture much more stable than Thomas Hobbes' state of "war of every man against every man," and argues that all men are created equal in the state of nature by God. From this, he goes on to explain the hypothetical rise of property and civilization, in the process explaining that _the only legitimate governments are those that have the consent of the people._ "

Locke's Second Treatise Of Government (Full Text):

[https://www.gutenberg.org/files/7370/7370-h/7370-h.htm](https://www.gutenberg.org/files/7370/7370-h/7370-h.htm)

Note that the word _" consent"_ is used 111 times, which is a lot for an essay of this size...

 _" Consent Of The Governed"_ \-- a very important root-level understanding when it comes to understanding U.S. Law...

</comment><comment><USER3> "Policing by consent" \- if you're doing it any other way, you're doing it wrong.

</comment><comment><USER4> As a citizen of a former British colony, I'm glad we inherited this mindset in our constabulary, although there's always some who persist in the us/them thinking.

After the mosque massacre, the Police ran a trial where some Police patrols were always armed, it did little for community engagement with the 5-0: [1]

It really didn't help that they were mainly used for traffic stops, as opposed to responding to actual firearms incidents.

Then we had the (now former) Leader of the Opposition proposing "Raptor Squads" modelled after New South Wale's "Strike Force Raptor" (lasers sold separately), and that went down like a lead balloon.[2]

[1]: [https://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&object...](https://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=12336140)

[2]: [https://www.newsroom.co.nz/2019/11/27/920773/strike-force- ra...](https://www.newsroom.co.nz/2019/11/27/920773/strike-force-raptor-wont- stop-organised-crime)</comment>


Show HN: The Most Explosive Modal on the Web - <AUTHOR> https://jessekorzan.github.io/expodal/ <comment><USER2> The opening of the modal is being pushed to Browser history. This confuses me as I don't expect modals to be in the history.

Is this caused the way it is achieved with CSS only or is this actually wanted?

<comment><AUTHOR> Just meant to be simple for prototype purposes... And I agree that typically you wouldn't want a UI event like a modal to be part of the history().

<comment><USER3> First of all, modals create a bad user experience. But if you're going to use them, users on Android expect to be able to use the back button to cancel out of things.

<comment><AUTHOR> Yes but... This modal explodes. Which, in most cases with design patterns, should only be used when appropriate. Right tool for the right job.

</comment></comment></comment></comment><comment><USER4> "100% CSS and one tiny line of JS." ... and a GIF of the explosion!

Still, I love it!

<comment><USER5> kaboom

</comment></comment><comment><USER6> This is hilarious. Any chance that the gif could be upgraded to an html5 video?

</comment><comment><AUTHOR> Hey hey - thanks for the feedback.

Can totally be done with a <video> but maybe not as much fun on mobile... And "back button" issue... could be solved with another line of JS.

</comment><comment><USER7> This is hilarious and I love it. Nice work.

</comment><comment><USER8> It does not work correctly when hitting back button :(

</comment><comment><USER9> Love it.</comment>


Ask HN: Purpose of the AT operator in PHP? - <AUTHOR>

I've been using PHP for well over a decade now and I always remembered the use of the @ operator being a last resort against "noisy" modules that would throw spurious warnings/errors that could be safely ignored.  Those days have long gone (or have they?) .. but @ remains as far as I can tell for backwards compatibility.
Anyway, after seeing it come up as part of many arguments against PHP and realising it is actually used by developers in everyday error handling! I've tried to find a reference to it's correct use in the docs, source and web but to no avail.
Does anyone else remember it as I do, or is this just a figment of my imagination. <comment> shane-armstrong Three years of debugging and writing code and I have never seen an @ used, I would consider getting rid of the key but I need the darned thing for logging in to my hotmail :(

</comment><comment><USER2> [http://us3.php.net/manual/en/language.operators.errorcontrol...](http://us3.php.net/manual/en/language.operators.errorcontrol.php)

<comment><AUTHOR> I'm looking for a reference to the @ operator being used to suppress spurious module errors. I'd not spent the last decade oblivious to the PHP manual.</comment></comment>


Disposable emails for safe spam free shopping - <AUTHOR> http://couponinbox.com <comment><AUTHOR> Instant emails with no signup or password, temporary and 100% anonymouse. Great to get that spammy offers and promotions instad of your regular email box. Also, it provides safebox to open received emails that protect from harmful scripts in messages. Even access log is off.</comment>


Free online study groups for people learning web development - <AUTHOR> http://codebuddies.org/?rc=hn

<comment><USER2> Is this based in NYC or can people do it anywhere?

<comment><AUTHOR> The study groups are google hangout-based, so people can participate from anywhere.

<comment><USER2> Very cool! Is it possible to start our own study group via this service?

<comment><AUTHOR> Right now, if your study group corresponds to either HTML/CSS, Javascript, Python, or Ruby on Rails, you can propose a new hangout in one of the above Google Groups and invite people to join the hangout for the topic/tutorial you're interested in reviewing with others.

In the near future, though, my hope is that a platform will replace the Google Groups and make the study group creation process around supplementary resources (besides the ones I picked to experiment on, that is) much easier. Can give you updates on when the beta platform is up if you join the mailing list or send me a PM.</comment></comment></comment></comment>