Early Linux filesystem reliability - <AUTHOR> http://minnie.tuhs.org/pipermail/tuhs/2017-May/009935.html <comment><USER2> I've actually used Ext2 in production, back in the '90s. That's all we had back then. A bit later I've also played a lot with XFS, both on Linux and Irix (I've worked at SGI for a while in the '00s).

Back in ye olden days, yanking the power cord basically guaranteed a corrupt Ext2 FS and a visit from fsck. However, in the vast majority of cases, fsck would actually do the job. You had to peruse lost+found and recover the lost souls out of it, but that was usually the extent of it. I did 'fsck -y' many, many times - in most cases with good results.

XFS on PC was worse. It tended to be stupendously fast when dealing with lots of I/O to/from very large files (video editing, running VMs), but a power failure on PC was a lot worse than with Ext2. There was a good chance you would lose the whole volume.

On Irix / SGI hardware it was a very different story. Those suckers were quite reliable. Heavy as hell, too.

\---

One habit from back then that's very hard to shake off is to run sync before reboot, often as "sync; reboot" \- you know, just in case something gets stuck on the way down and you have to hit the reset button. A more extreme example would be to manually stop all services except sshd, then do "sync; reboot".

It's completely unjustified today, and yet I still do "sync; reboot" occasionally. It's baked into the muscle memory of my fingers after sleepless nights caused by losing stuff due a flaky driver that froze the system on reboot.

<comment><USER3> If you're concerned about flushing writes consider `mount -oremount,ro` -- this guarantees writes are flushed and is the only truly important step of a system shutdown anyway WRT filesystem integrity. Once filesystems are mounted read-only you can safely power off the machine.

Filesystems can be remounted read-only via the serial console as well, with a break-u. Useful even when userland is otherwise inaccessible such as in the event of a fork bomb.

<comment><USER4> Nice to learn this - but how does one send a "break-u" e.g. from inside minicom, or in KVM virtual machines, via virsh console?

<comment><USER3> I usually use `cu`, but the docs for minicom tell me it's control-A followed by F to send a break signal. Then just hit "u." There are a number of single characters following a break with actions to them, the full doc on this kernel feature is here: [https://www.kernel.org/doc/Documentation/admin- guide/sysrq.r...](https://www.kernel.org/doc/Documentation/admin- guide/sysrq.rst)

I'm not terribly familiar with KVM but the VM console tools probably have some way to generate a break signal. Check the docs for how to "send a break."

Oh, and don't forget to enable this feature via sysctl -- per the above linux doc.

</comment></comment></comment></comment><comment><USER5> Hell the attitude in the article evokes strong memories to my last workplace.

Almost all development teams tried so hard to build smart software doing the right thing, recovering from everything, tried to fix so many things. They spent insane amounts of work trying to be smart in error situations. And all of that was useless or harmful - it either didn't work, or it it made things worse.

My team operated under simple principles. Crash early, crash hard, log well, trust the operator. But you know, Ops at that place loved our applications. It took time to get them going, but it was easy to understand what to do whenever they crashed. It was easy to understand why it stopped. It was easy to understand when to file a bug.

<comment><USER6> Getting good at error handling, queue sizing and writing relevant logs requires some repeated exposure to real world pear-shaped events.

Failing early and visible is always good, especially before a program have been taken into production. When things have stabilized for a while, recoverable errors can be ignored, and just logged.

It's also important to take the opportunity to improve the error handling and logging first, before fixing the actual problem. Errors are hard to fake "right" so getting a good reproducible error is an opportunity.

</comment></comment><comment><USER7> Quite a coincidence. I recently went back from XFS to EXT4 in my work laptop using Arch Linux, and before that I went from BTRFS to XFS.

I went from BTRFS to XFS because I got some really bad corner case of performance in BTRFS. My work consists of mostly backend development using Ruby web stack and PostgreSQL/Redis, and often my laptop would freeze completely and only get back after a dozen of seconds. This being in a SSD was unacceptable*. So I decided to go back to XFS and for some time everything went well, no more random freezes.

In this weekend I updated my system by running pacman, as usual. Chrome seemed to consume all memory and the system freeze during the update. Ok, not bad I thought, I rebooted and tried to run pacman again, hopping that only some files was corrupted, however sufficient files were corrupted that I needed to reinstall all packages in the system. Another freeze during the update and my system was essentially death after reboot. I tried to recover using chroot however multiple files were broken beyond repair.

So I decided to go back to EXT4, and reading this article does make me more confident that this shouldn't happen again.

<comment><USER8> Hard not to come off smug but as a ZFS user it is always jarring to read anecdotes like this in {{ current_year }}.

<comment><USER7> I am quite curious to test ZFS, however Arch Linux does not maintain official ZFS packages in repositories, so I would need to maintain my own kernel updates just to use ZFS. Not interesting to me, specially considering that this is my work laptop.

When (if?) Arch supports ZFS in main repos I will probably test it. That is, unless bcachefs comes first.

<comment><USER9> > maintain my own kernel updates

Not really: zfs is available in aur as a dkms package. Pacman automatically rebuilds dkms modules when it updates the kernel. All you'd need to do is modify mkinitcpio.conf to include "zfs" in the HOOKS at the appropriate stage.

So: more complicated that just using ext4, but not "maintain my own kernel" level of difficulty.

</comment></comment></comment></comment><comment><USER10> So last I checked, btrfs was the way of the future according to Ted, but every time I see it discussed, it's Here Be Dragons galore. Is there some timeframe where btrfs will take over? Or at least be stable enough for a Debian or Red Hat to switch to it as a default?

<comment><USER11> SuSE/OpenSuSE has been using BTRFS by default for a while and it seems to work well enough. There's a default schedule of running 'btrfs balance' every week, seemingly based on when the OS was installed (it relies on the timestamp of a file that gets updated with every run), that makes the system(1) virtually unusable for about 15 minutes.

(1) I've only seen this on one machine, so maybe it's a quirk of that machine's workload. But it sure does suck when it's in the middle of a workday.

<comment><USER9> On btrfs causing latency: I've got a few systems with btrfs as the rootfs (on top of lvm, on top of dm-crypt, on a SSD).

I recently started using `snapper` to create snapshots on a schedule on them. I enabled quota support in btrfs so I could see how much space snapshots were using.

I noticed that filesystem wide latency tended to spike when removing snapshots (several minutes of all fs access stalling).

Balancing with quotas enabled is even worse: my systems were hung for multiple days, until I forcibly restarted them and disabled quotas. Then the fs hangs were much smaller (a few seconds) and not to noticeable. Balancing finished in something on the order of an hour.

While I had quotas enabled, I was constantly having btrfs tell me the data was bad and needed rescanning (rescanning quotas would also induce fs wide latency).

The thing is, ZFS has snapshot space usage info, and doesn't have awful latency (it also doesn't have a "balance" operation, but I'm not sure how relevant that is).

Given my experience with both btrfs & ZFS, I'll likely consider using ZFS as my rootfs in the future.

<comment><USER11> I have no idea what the state of ZFS on Linux is, but I've been using it on FreeBSD for a while now and it's fantastic. Comparing FreeBSD to Linux is a bit of an apples/oranges thing though.

</comment></comment></comment></comment><comment><USER12> > There is yet another example of "Worse is Better" in how Linux had PCMCIA > support several years before FreeBSD/NetBSD. However, if you ejected a > PCMCiA card in a Linux system, there was a chance (in practice it worked out > to be about in 1 in 5 times for a WiFI card, in my experience) that the > system would crash. The *BSD's took a good 2-3 years longer to get PCMCIA > support, but when they did, it was rock solid. Of course, if you are a > laptop user, and are happy to keep your 802.11 PCMCIA card permanently > installed, guess which OS you were likely to prefer --- "sloppy but works, > mostly", or "it'll get there eventually, and will be rock solid when it > does, but zip, nada, right now"?

Time to market matters. Probably explains how codebases with low reputed quality still seems to win in the real world : MySQL vs Postgres ? MongoDB vs RethinkDB ?

</comment><comment><USER8> Ted's post is quite balanced and accurate vs the revisionist fawning in the quoted message from McVoy. But Warner's message elsewhere in the thread is also good [http://minnie.tuhs.org/pipermail/tuhs/2017-May/009880.html](http://minnie.tuhs.org/pipermail/tuhs/2017-May/009880.html).

The biggest Linux defect was not ext{2,3,4} but LVM and MD, which would throw away write barriers until something like kernel 2.6.31 (which was especially painful on Ubuntu LTS). Many distros used LVM by default, and many servers used mdraid somewhere in the stack. I saw many corrupt Linux systems in the 2000s through the first part of this decade.. it was especially egregious for DBs and hypervisors with file based disk images.

</comment><comment><USER13> It took me a few passes to figure out that "Things Just Worse" should be "Things Just Work." Autocorrect much?

</comment><comment><USER14> _PC class hardware tends not to have power fail interrupts, and when power drops, and the voltage levels on the power rails start drooping, DRAM tends to go insane and starts returning garbage long before the DMA engine and the hard drive stops functioning._

This is insane, I refuse to believe it. Even a junior EE knows how to design a PCB so that the RESET signal is asserted on all ICs as soon as the voltage drops below a safe operating level.

<comment><USER15> The PCB may be well designed, but the power supply may not be. I have witnessed what a cheap PC can do. Purchased from a Walmart some years ago. Windows 7 immediately replaced with Ubuntu. After a brownout, that machine was unbootable. Also un-fsck'able.

My thought was that the power supply should guarantee adequate power, or none at all. Not something in between. Also, no a rapidly alternating states of adequate power / no power.

Short term solution: rebuild that system and get it a UPS.

Longer term solution: get a much better box.

<comment><USER5> > My thought was that the power supply should guarantee adequate power, or > none at all. Not something in between. Also, no a rapidly alternating states > of adequate power / no power.

That is very hard to do, actually. Voltage does fluctuate to a certain degree, because physics. And you don't want to cut power to a system just due to a small power dip. And you can't distinguish a small power dip from 10 small power dips over some time ago. And your flapping detection only works if the power drops are close enough to each other.

Proper control theory is amazingly hard. As my prof on that said - no one fully understands PID controllers, but some blokes have a really lucky thumb.

<comment><USER14> The case Ted Ts'o describes is much simpler: DRAM returning garbage while the CPU is still executing instructions and sending commands on the SATA bus.

This can't possibly happen even if the power supply is crazy bad, because the reset logic on the main board will halt everything before the DRAM starts malfunctioning.

<comment><USER16> > RAM returning garbage while the CPU is still executing instructions and > sending commands on the SATA bus.

When I read it, I got the impression he was saying the DRAM was going crazy while a DMA transfer to the hard-drive was still going on. That doesn't require the CPU to be functional when the DRAM is corrupt, only the DMA controller. I can't personally say if that makes it any more likely though.

<comment><USER14> That's right, but all stateful ICs on the motherboard have a reset pin, including the north and south bridges when they were still separate packages. Even PCI cards will receive the reset signal simultaneously.

Not sure what the hard drive would do with a truncated ATA command though.

</comment></comment></comment></comment></comment></comment><comment><USER17> A more in-depth explanation of journaling vs soft-updates:

[https://www.freebsd.org/doc/handbook/configtuning- disk.html](https://www.freebsd.org/doc/handbook/configtuning-disk.html)

</comment><comment><USER18> This post brought up memories of trying Linux in the late 90s and giving up (or reinstalling the os) because ext2 crashed to a point unrecoverable by fsck. Anyone else have a similar experience back then?

</comment><comment><USER19> I stand corrected, sorry everybody.

<comment><USER20> If you can afford ECC then that is obviously preferable. However the horror stories regarding running ZFS on non-ECC are _greatly_ exaggerated.

<comment><USER21> Not merely exaggerated, nonsense. All filesystems using non ecc memory are more likely to experience damage to the data stored therein.

Zfs is no more likely to be damaged that anything else.

This whole confusion stems from zfs devs suggestion that ecc ram be used and people who don't know any better thinking oh that must be some special requirement for zfs.

These folks then spread this misinformation all over the Internet where its passed on by people who know nothing about the topic.

Pro tip: don't talk about things on the Internet that you only know about 3rd or 37th hand. If you know nothing about the topic you aren't improving the store of human knowledge by passing on noise.

<comment><USER20> > _Not merely exaggerated, nonsense. All filesystems using non ecc memory are > more likely to experience damage to the data stored therein._

If it were "nonsense" then one could flip the argument and say "ECC memory has no beneficial effect what-so-ever" \- which be both know is untrue. However I can see why people do say "nonsense" given the ECC-myth seems to only be discussed in relation to ZFS and the probability data errors due to non-ECC memory is small.

For what it's worth, I did originally include a comment about how the risk being the same as on any other file systems - but then deleted it fearing I might have overlooked something on another fs I'm less familiar with.

> _Pro tip: don 't talk about things on the Internet that you only know about > 3rd or 37th hand. If you know nothing about the topic you aren't improving > the store of human knowledge by passing on noise._

I don't think that's fair. The problem with 1st hand experience is that it's often just based on anecdote. Which can often be worse than 3rd hand advice. And in the case of failures: often some products are too widely used to be worth constantly badgering the development team for help; which means you end up having to rely on the advice of others. The problem is really more that some people are terrible at researching so take 3rd hand advice without bothering to fact checking it.

In any case, I don't know if your comment was aimed at me or not, but I do have nearly 10 years of experience (wow that's gone quick!) running ZFS across a variety of systems, some running ECC memory, others not. I've also been a keen study of the Sun/Oracle docs. So I do consider myself reasonably well informed from both credible sources and personal experience. Though I'm not arrogant enough to assume I'm an expert either - communities like HN can be deeply humbling places.

<comment><USER22> > > Not merely exaggerated, nonsense. All filesystems using non ecc memory are > more likely to experience damage to the data stored therein.

> If it were "nonsense" then one could flip the argument and say "ECC memory > has no beneficial effect what-so-ever" \- which be both know is untrue.

The original argument is like an implication ("no ECC => don't use ZFS"), which he refutes by saying (correctly), that file systems are generally affected the same way by memory corruption. Your argument seems to invert the original implication and drawing conclusions from that (fallacy of the converse, I believe). ∎

<comment><USER20> He was refuting my " _greatly_ exaggerated" remark by going on to make the same points I was alluding to albeit directly and in more detail. So I was in turn elaborating on my choice of language.

I feel between this post and you're previous one that you are basically just agreeing with me via the process of nitpicking the language I used.

</comment></comment></comment></comment></comment></comment><comment><USER23> Complex Things Fail In Complex Ways: An Object Lesson

</comment><comment><USER24> slightly off topic: How facebook uses btrfs:

[https://www.linux.com/news/learn/intro-to-linux/how- facebook...](https://www.linux.com/news/learn/intro-to-linux/how-facebook- uses-linux-and-btrfs-interview-chris-mason)

<comment><USER24> [http://masoncoding.com/presentation/ks-14/btrfs.html#/1](http://masoncoding.com/presentation/ks-14/btrfs.html#/1)

</comment></comment><comment><USER24> Ted is brilliant</comment>


How to be a Programmer: A Short, Comprehensive, and Personal Summary - <AUTHOR> http://samizdat.mines.edu/howto/HowToBeAProgrammer.html#src=hn <comment><USER2> Cheeky side note: it'd be nice if the author could _program_ their web server to send the correct character set so that the © symbol doesn't show up as �. It's being served with:

        Content-Type:text/html; charset=UTF-8

whereas the charset is actually in ISO-8859-1.

<comment><USER3> Unsure of the charset standards history, but article is from 2003 and hosted on an edu domain. So possibility since then site has been moved around?

<comment><USER2> Good call. I sent an email with a UTF-8 version of his article attached to the address listed at <http://samizdat.mines.edu/howto/>, but it bounced. Any ideas for how to contact him?

</comment></comment></comment><comment><USER4> As a Mines alum I can say I damn good first step to becoming a programmer is surviving a school as tough as Mines (if you're academically minded). I graduated with a chem degree but decided to switch to web development about a year and a half ago, and if there's one thing that school really taught me how to do was be a good learner which ended up being an awesome asset in trying to get hired.

</comment><comment><USER5> Shorter: learn, think, code, debug, solve, share, enjoy.

</comment><comment><USER6> I presume the author is female, according to her use of the feminine pronoun in referring to programmers generally. Nice to get a female perspective. I appreciate her emphasis on all the qualities beyond coding itself that contribute to a good programmer.

<comment><USER7> I've actually noticed only a weak correlation at best between gender of the writer/speaker and the generic pronoun e uses. It's pretty common for people of the progressive set to use the feminine personal pronoun for programmers generically, which is a Good Thing (tm).

<comment><USER8> As opposed to the gender-neutral 'they', which is the Best Thing (that few people seem to realise).

<comment><USER9> Many people still like to claim that 'they' can only be used as a plural.

<comment><USER10> I personally think it just sounds awkward. Spivak pronouns can pick up all of the benefits of a singular "they" without the drawbacks, though.

<comment><USER11> I would say the opposite, actually.

"They" as a gender neutral singular pronoun is, to me, perfectly natural. I use it dozens of times a day as such without noticing.

Spivak pronouns, on the other hand, are blatantly marked, and were I to come across them in writing or speech, I'd find them very distracting.

<comment><USER12> Thanks for alerting me, indirectly, to the Spival pronouns. I think they're a good, minimalist approach to a fix, but agree they will look distracting until or unless they become commonplace.

I guessed - correctly - it would be the same Spivak who wrote the excellent Calculus textbook.

</comment></comment></comment></comment></comment></comment></comment><comment><USER13> a short summary?! i could learn python in the time it takes to read this.

<comment><USER14> This is short by the standard of well-considered material of durable merit warranting serious reflection.

It's not short by the standard of contemporary consumable Internet content, which is to be expected, as most of that content is not designed to enlighten. It's designed to catch your attention long enough to tell you about a brand, whether corporate (advertising) or personal (blogging).

And at any rate, serious writing implicitly asks for an investment of time in contemplation above and beyond line scan time, so its 'reading time' is only loosely coupled to the apparent length of the material. Making that investment increases the payoff, it doesn't dilute it. "You get out of it what you put into it."

If content consumption time is directly linearly related to content length, that's a pretty good flag that you're probably not getting much real value out of it. I've found low value content to be a difficult honeypot to resist even on HN, which is comparatively a pretty high-quality aggregator.

This is one of the best HN submissions I've ever seen. Given its due respect, it could change lives.</comment></comment>


The Gulag Archipelago: A New Foreword by Jordan B. Peterson - <AUTHOR> https://quillette.com/2018/11/01/the-gulag-archipelago-a-new-foreword-by-jordan-b-peterson/ <comment><USER2> I wish they still pushed this book in schools. The Gulag Archipelago is a fantastic look into the absolute nightmare that was the Soviet Union.</comment>


I'm Upset: The "zero waste" people must be stopped - <AUTHOR> https://theoutline.com/post/7500/i-m-upset-the-zero-waste-people-must-be-stopped <comment><USER2> Seriously? They're upset that people are using the term "zero waste" when that's physically impossible thanks to the 2nd law of thermodynamics? How semantic can you get?

Instead of being upset about your "bratty" roommate reducing plastic waste but driving half a mile to work everyday why not use that as an opportunity to say hey, I know you care about a lot about the environment and you've been doing a great job reducing your plastic waste but there's even more you can easily do by walking to work - you'd get some exercise too!

See? There's constructive ways to have these conversations, and they're good to have because we all need to be concerned about reducing our impact on our home planet. Working together you might even be able to come up with new ways for you and your roommate to reduce your impact. That'll improve the quality of life for both you and your roommate.

</comment><comment><USER3> Author would rather live in the world of Wall-E than be bothered by guilt.</comment>


Ask HN: how much it cost to file tax? - <AUTHOR>

I have early stage tech startup in Seattle, USA with handful of transactions. How much would it typically cost to file tax? <comment><AUTHOR> Also, is that okay to handover all the information to the bookkeeper and let it take to his office?</comment>


Form data to js object to json - jQuery plugin - <AUTHOR> https://github.com/mazondo/formalizedata

<comment><AUTHOR> This should exist already, and probably does, but I couldn't find it. It's become my favorite plugin ever, would love some feedback!

</comment><comment><USER2> I need this</comment>


Not designers. Not coders. Just builders. - <AUTHOR> http://blog.studiofellow.com/2012/05/09/not-designers-not-coders-just-builders/

<comment><USER2> All of these "coders must learn design" articles ignore the fact that most programmers are not web developers and that sizable chinks of the industry simply never touch frontend code at all.

For a lot of developers there is little career incentive to develop design skills.

<comment><USER3> _All of these "coders must learn design" articles_

This isn't one of those. It's a "designers should learn to code" article. Which is interesting since I generally find that there are more developers interested in design, than designers interested in code.

In fact I asked HN about this earlier today! (<http://news.ycombinator.com/item?id=3949467>)

</comment></comment><comment><USER4> Coder trying to be a designer here. I've spent a lot of time reading books and experimenting, but what I am really missing are beginner tutorials on how to make the popular elements I see on websites in photoshop (or even principles of creating the different elements).

Maybe I am not looking in the right places, but I would love to be able to find an exploration of gradients (for example) in Photoshop and/or css and how different configurations create different visuals. When I open up Photoshop and try it myself, I feel overwhelmed by the seemingly infinite color and position choices I have in front of me - and every attempt comes out looking horrible.

<comment><AUTHOR> I think learning design can seem really difficult after a cursory glance at the design community.

Sorry to mention this, but it's the best way I know to help. I wrote an ebook about design specifically for coders and technical founders. It actually covers several of the issues you mentioned, like creating a gradient in Photoshop and how to use and pick colors. You can see a sample at <http://bootstrappingdesign.com/pdf/sample.pdf>

<comment><USER3> It's interesting that in my observations of designers and developers - it's the designers who have more catching up to do... See <http://news.ycombinator.com/item?id=3949467> for some of my ramblings on this earlier today.

<comment><AUTHOR> I hadn't seen that thread. Interesting that our experiences are practically opposite.

I left a comment on that other thread, but I think this part is relevant here:

 _Maybe we're all just pointing fingers at each other. It's obviously productive for everyone to gain basic competency in the opposite discipline._</comment></comment></comment></comment>


A Review of the Amazon Books Store - <AUTHOR> http://programmingzen.com/2016/02/10/a-review-of-the-amazon-books-store/ <comment><USER3> Looks like a great way to buy a bestseller from Amazon without having to wait for shipping. But it's not going to appeal much to the kind of people who are willing to spend hours wandering the aisles of a real bookstore. One of the things that makes me a very happy Portlander is that Powell's City of Books ([http://powells.com](http://powells.com)) is always _crowded_ with blissed- out bibliophiles perusing the million-plus new+used volumes.

But I could be all wrong. I don't understand the business of books at all, because I would think that a store like Powell's would flourish in other literate locations (e.g., Minneapolis / St Paul), and yet it remains sadly unique in the U.S.

<comment><USER4> There are other places like Powell's: Haslam's in St. Petersburg, FL - [http://www.haslams.com/](http://www.haslams.com/) and others.

Edit: Thanks for the location correction - I lived there for five years and should know better. ;)

<comment><USER3> Thanks so much for the tip. Are new and used volumes shelved together at Haslam's?

<comment><USER5> No, they are in different sections.

</comment></comment></comment></comment><comment><USER6> In the various reviews I've seen for this store, no one seems to have raised the question of, how would a store like this be profitable? It's selling books at the same prices as Amazon online, but with the added overhead of being in a very high-rent space in an upscale shopping center. There seem to be lots of employees for a book store. At the discounts that Amazon offers, their margin on books is thinner than most bookstores. Sales would need to be very high to make this pencil out, from what I can see.

This might be a play like the old Crown Books stores which sold only best sellers at a big discount with the idea of making it up on volume. But Crown did not last long. Amazon does have good data about what people buy, so perhaps they could maximize revenue per square foot by packing popular sellers in. On the other hand, one departure of this store from typical bookstores is that books are all facing out on the shelves, meaning fewer titles and probably less revenue per square foot.

I'm wondering how this approach to bookstores could pay off for Amazon, except by putting competitive pressure on any other bookstores in the area. But Barnes and Noble closed the store it had in this shopping center quite some time ago. Is it advertising, an attempt to upscale Amazon's brand? Amazon has the number one favorable brand among consumers from what I've read, and it's certainly where most people go these days for books -- last report I saw said over 80% of book buyers use Amazon. So what the business purpose of this store is, unclear to me. But it's Amazon, and surely they're just a few steps ahead of what I can see.

<comment><USER7> Those physical Microsoft stores are often practically vacant.

But what would the marketing budget for a really well placed billboard be per month? What if you could walk into the billboard? Host launch parties starring famous bands, getting you into more news stories. Fans can make a pilgrimage to something tangible.

Apple stores are clearly profitable, but I wonder if "close to break even" isn't a good enough goal for brand awareness stores like these.

<comment><USER8> Really? The one here in Cleveland always has people in it - not packed, but always more customers than employees.

<comment><USER9> Interesting. The Microsoft store here in Seattle (same shopping mall as the Amazon Store) generally has more employees than customers when I walk by in late evening. In contrast, the Apple store ~100m away, facing the Microsoft store, is reliably busy.

</comment></comment></comment></comment><comment><USER10> >The second advantage is actually a side effect of their limited selection approach. They are essentially offering a pre-selection service to the public.

this seems less like "a review of the amazon books store" and more like "an amazon user finds out what a book store is"

</comment><comment><USER11> Does anyone have a good online bookselling that isn't Amazon? I started using abebooks before realizing its Amazon owned. bookdepository.com is good, but not great. oreilly.com is good for their stuff. B&N is OK I guess.

<comment><USER12> I like to buy specialty books directly from publishers. Art books are often difficult to find elsewhere, and tech books tend to come with nice extras.

I used to prefer FNAC, but they only exist in France.

<comment><USER13> We have FNAC in Italy :)

<comment><USER14> And in Spain :P

<comment><USER15> "As of 2015, the company has stores in France, Belgium, Brazil, Monaco, Morocco, Portugal, Spain and Switzerland" (Wikipedia)

It seems they sold off their Italian stores in 2013.

</comment></comment></comment></comment></comment><comment><USER16> _" The central section however is dedicated to various gadgets, mostly created by Amazon itself, such as Echo, Fire TV, Kindle Fire, etc. In this the shop reminded me a little of the Apple Store, which clearly inspired the layout of this section."_

The focal point of the "amazon books" store is its tech, not books. They are using "book store" as a cover. The U-Village is an upscale shopping experience and easily allows for impulse buys on new $100+ tech products. It's their version of the Apple Store.

</comment><comment><USER17> I wonder if the book store is large enough to host a data center or if there's already one in there

<comment><USER12> I think Seattle City Light would be a bit too expensive.

</comment></comment><comment><USER18> What? No Jeff Bezos robot?! [http://www.geekculture.com/joyoftech/joyarchives/2247.html](http://www.geekculture.com/joyoftech/joyarchives/2247.html)

</comment><comment><USER19> Short summary: verdict - <USER2>

</comment><comment><USER20> Knowing the terrible privacy- and tracking-issue history as related to the Amazon app, I would never install it for any reason, much less just to check prices.

In fact, after reading this, it makes me wonder if the real purpose behind these stores isn't simply to force people to install the app.

<comment><USER21> > In fact, after reading this, it makes me wonder if the real purpose behind > these stores isn't simply to force people to install the app.

Oh yea, the Bezos long con where by he knows if he makes a store you will visit it and you WILL install his app... Or maybe he just wants a place for people to use and demo their physical devices?

<comment><USER20> According to the article, there are no prices posted in the store and the only way to get them is to install the app, so apparently yes you WILL install the app if you actually want to buy something.

And just in case you are not aware of the security issues with the Amazon app[1]

[1] [http://www.computerworld.com/article/2489480/security0/amazo...](http://www.computerworld.com/article/2489480/security0/amazon --at-t--snapchat-rated-among-the-least-trustworthy-with-data--eff-finds.html)

<comment><USER21> Actually, according to the article:

> For those with a dead battery or who are not inclined to take their > smartphone out of their pocket, there are a few scanners around the > bookstore, which you can use to scan books and discover their prices.

People who refuse to install the Amazon app are also very unlikely to visit the Amazon store. Just get your books elsewhere.

Also your link has nothing to do with security issues in the Amazon app and is just a ranking from the EFF for how companies respond to records requests.

> Amazon, Snapchat and AT&T rank among the least trustworthy technology > companies when it comes to how they handle government data requests, > according to a report from the Electronic Frontier Foundation.

<comment><USER20> > People who refuse to install the Amazon app are also very unlikely to visit > the Amazon store. Just get your books elsewhere.

Exactly...that's what my original post said I was going to do, so we agree.

I don't really understand what the big deal was.

<comment><USER21> Because you theorized that the stores were a plot to get people to install the Amazon app. Which is absurd.

> "it makes me wonder if the real purpose behind these stores isn't simply to > force people to install the app"</comment></comment></comment></comment></comment></comment>


Apple WWDC 2018 - <AUTHOR> https://developer.apple.com/wwdc/# <comment><USER2> Is it just me or is this announcement earlier than usual? Anyway now I have to think about whether or not I want to try to go...decisions decisions...

<comment><AUTHOR> The announcement is indeed earlier than usual. The usual announcement date for WWDC is April with the event happening in June.

Last year was an unusual exception where the announcement was made in mid- february.</comment></comment>


Ask HN: Anyone up for a Project Euler group study of C, Python, Haskell, and J? - <AUTHOR>

We'd write solutions for one Euler problem per week in each of the four languages and share and compare solutions on a private blog. <comment><USER2> I would be up for that, I've wanted to learn Haskell for a while now.</comment>


Giving Away Our Recommendation Engine - <AUTHOR> http://blog.mortardata.com/post/82195614895/giving-away-our-recommendation-engine-for-free <comment> mck- Fwiw, here are two light-weight feature-based recommendation engines I built for Node.js (for situations where you have the cold-start problem and therefore can't rely on user/item based collaborative filtering): Alike [1] and Look-Alike [2]

[1] [https://github.com/axiomzen/Alike](https://github.com/axiomzen/Alike)

[2] [https://github.com/axiomzen/Look-Alike](https://github.com/axiomzen/Look- Alike)

<comment><USER2> Thanks for sharing. What do you mean by the "cold-start" problem? Just want to know exactly when I can use your engines.

<comment><USER3> Just speculating: not having a recommendation when you first begin because you don't have any data.

<comment> mck- Exactly right. I borrowed that term from Chapter 2 on Collective Intelligence [1]

[1] [http://shop.oreilly.com/product/9780596529321.do](http://shop.oreilly.com/product/9780596529321.do)

</comment></comment></comment></comment><comment><USER4> So hang on, what exactly is a recommendation engine?

They give examples of LinkedIn ( _people you may know_ ) and Amazon (presumably _other people who bought this_ , _so-and-so 's list of such-a- subject books_).

That makes sense, though the segment of businesses that may actually benefit seems limited. Social stuff, sure. Most of us? What's the minimum recommendable-entity/category-or-user threshold that this makes sense for? Is success with these sorts of engines merely a reflector of poor UI design in your normal UX? (Of the above examples, the first seems very unidimensional - in that it's basically a simple graph distance - and the latter also rather rudimentary and often irrelevant).

So what exactly is this thing providing? Graph analysis? I think not. It reads more like some kind of raw timestamped user behavioural event data processing to infer relationships between users or products they interact with. Reading through the docs it seems this is a layer on top of Apache Pig ([https://pig.apache.org/](https://pig.apache.org/)) - _a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs_. I think clarity in explaining this thing could be improved, particularly selling clearly what a recommendation is and when its useful. Using phrases like "award winning" doesn't help.

PS. Why all the downvotes? Sheesh.

<comment><USER5> I suspect you're being downvoted for having a dismissive tone in the same breath as you admit to not understanding the problem space. My guess is that the marketing copy on the site isn't targeted towards you, so it shouldn't surprise you that you don't understand it, whereas their target customers know all sorts of things about, say, Pig. That's fine, but then your comment should read like, "Can someone explain to me what this is for?" Instead, your comment is dripping with condescending snark, lecturing someone or another about how this thing you don't understand probably isn't useful, and incredulous that you don't understand something on the internet.

Imagine opening an advanced textbook on a subject you don't understand, reading two paragraphs of it, and throwing up your hands in disgust because _what does this even mean?_

<comment><USER4> Apologies, condescending snark was not the intent (I can't even see where you see that, actually!). In response to your more salient points, it could be argued that a web+EC2 layer on top of existing software is hardly an _advanced textbook_. Likewise, their announcement's stated intent is to gain customers, so feedback on what's unclear should be well within an acceptable scope of discussion. Finally, I doubt any of us are excluded from their intended market as software people move frequently between problem domains.

<comment><USER6> For the record, this reply seems as snarky and dismissive as the first. Hopefully some constructive criticism.

<comment><USER4> Thanks.

</comment></comment></comment></comment></comment><comment><USER7> FWIW we've been using the mortar platform to run large pig jobs without a fuss at [http://datadog.com](http://datadog.com) and we've been very happy with it. Glad to see them contribute their recommender code too.

<comment><USER8> Can you please suggest why you need a recommendation engine for datadog?

<comment><USER7> We don't use the recommendation engine but the underlying platform, which makes it really simple to write and run pig jobs. Though the majority of our business deals with real-time data processing, the ability to crunch numbers in batch without dev or ops overhead is attractive and well worth the price to us.

<comment><USER9> Is this better or similar to Hue?

</comment></comment></comment></comment><comment><USER10> I'm curious what the business case was for open sourcing the code. Maybe to create an ecosystem?

<comment><USER11> From the "What you'll need" section of the first tutorial -

A Mortar account. You can sign up for a free Public account with Mortar here. If you want to keep your customized recommendation engine code private, you will need a Solo-level account ($99/month). Beyond that, you'll only pay for your actual usage of AWS cloud services (we never add an upcharge).

Kudos for the open source, but it looks like to actually use this for business you'll still need to pay. Unless i'm misreading it, "Open source but you'll still have to go through our platform" is pretty disingenuous.

<comment><USER12> It reads like "open source but not free to make proprietary." First, it's awesome just to see source as something to learn from. Second, it seems reasonable they don't want people forking, modifying then profiting from their work without contributing back to it - either by also releasing source or by paying.

I think it's a nice model actually.

<comment><USER13> It's called the GPL. If that's really the model they're trying to create, it would be nice if they just used the GPL.

</comment></comment></comment></comment><comment><USER14> Anyone know of any comparisons between this and Apache Mahout? I've used Mahout's Item-Item recommender in the past, and it's worked well, just wondering if there were advantages to this recommender.

</comment><comment><USER15> I'm sure plenty of good karma (even the non-HN kind) is headed your way - kudos.

</comment><comment><USER16> WOW, Awesome Documentation and Product!! Kudos and Greetings from Germany 😊

Those who know what Hadoop, Pig and the whole "Data Science Stack" is, will find this surely useful.</comment>


SF TV producer Gerry Anderson has died - <AUTHOR> http://www.jamieanderson.me.uk/gerry-anderson-has-died/

<comment><USER2> That man was a hero to me! Space:1999 was a brilliant show and had so much love in it — as a nine year old in 1975 it made very excited about the idea that I could live on a moonbase by the time I turned 33 years old. To me the internet seems like a less exciting second place prize by contrast...

</comment><comment><USER3> Thunderbirds were the totality of my existence as a boy. FAB!!!

</comment><comment><USER4> Gerry's work filled my imagination as a young child. He was a genius!

Thunderbirds are GO!</comment>


Show HN: Memeforth – a "Forth" based on text manipulation and text-based memes - <AUTHOR> https://szy.io/memeforth/ <comment><AUTHOR> So this is something I created, it's a forth in that it's a stack based language and shares some of the syntax, but there are many features that forth has that this doesn't, and it's implemented quite differently, but I still call it Memeforth.

Just figured I'd share, hopefully it isn't too broken or awful. I'm not a professional software dev or anything, just a hobbyist, and came up with this as a bit of a joke and it turned into something sorta real.

<comment><USER2> Love the look and setup, just can't get it to work though by just entering plain forth such as 3 4 + .

I get no result. Any trick to get this to work (using Chrome)

<comment><USER2> I think the problem is a css issue, I can't see the 2 buttons at the bottom :)

<comment><AUTHOR> Ah yeah, sorry. You can also press Ctrl+Enter to run your code!</comment></comment></comment></comment>


Ask HN: Bypass App Store Subscription Fees? - <AUTHOR> Anyone have any suggestions on how to get around Apple App Store subscription fees?  Netflix and Spotify do it, but also several smaller apps too. I can't seem to find the magic combo for Apple ignore it in an app I'm developing. <comment><USER2> In your app, don't put a link or any text that would lead or let user know they can make payment outside of App Store.

Set up an external website that registers/login user and process user payment there. Then in your app, require user login and then your app can communicate with the backend to check if the user is subscribed.

</comment><comment><USER3> One trick might be to just put a 10% higher price within the app, and lower off app. Consumers will think it's a mistake and spread the word. You can also send them an email telling them that they get a discount if they subscribe now.

On the other hand, some people prefer to pay within app. I don't trust many subscription services to be responsible with my credit card information, but Apple/Google have proven to have consumer protection. So you should probably do both.</comment>


Fotostat is now monitoring over 1,000,000 photos - <AUTHOR> http://blog.fotost.at/fotostat-is-now-monitoring-over-1000000-photo

<comment><USER2> Is it just me or did I see this thread 15 minutes ago, and now the link is dead?</comment>


The role of software in spacecraft accidents – the morning paper - <AUTHOR> https://blog.acolyer.org/2017/11/30/the-role-of-software-in-spacecraft-accidents/ <comment><USER2> For me, one of the key quotations and take-aways is this:

> _In almost all software-related aerospace accidents, it turns out that the > software is behaving as designed, but the designed behaviour was not safe > from a system viewpoint._

There is more that is of value. For people - like me - who have designed safety-critical systems, and for people who want to do stuff that potentially has deeper consequences when it goes wrong, this is an interesting article.

But it's not a tutorial. You need to study it to find the lessons, you need to read with attention and extract the value. As with learning to program, learning to dance, learning math, learning to juggle, _it 's not a spectator sport._

If you want to get into anything that has a safety aspect, this article isn't a bad place to start.

</comment><comment><USER2> See also: [https://news.ycombinator.com/item?id=15814712](https://news.ycombinator.com/item?id=15814712)</comment>


Show HN: Mince pie tracker - <AUTHOR> http://mpt.zencodeworks.com

<comment><USER2> This is completely awesome.

It's going to be a very regional competition - I wouldn't even know where to find a mince pie in Seattle. Which I imagine is going to be a pretty common problem for much of the world.

As a side note - I was curious if those countries the Southern Hemisphere, (and thus a warm sunny XMas), would be as into Mince Pies, but a quick google of turned up Yes!

<comment><USER3> Mince pie is a thing in America, they're just much larger than the British treat. From what I can tell, it's a similar filling, just, expanded to a full- size pie.

In my experience, you can find it in any supermarket around Thanksgiving and Christmas.

(Example: wal-mart [http://www.walmart.com/ip/Mrs.-Smith-s-Classic-Mince- Pie-27-...](http://www.walmart.com/ip/Mrs.-Smith-s-Classic-Mince- Pie-27-oz/10850121))

</comment></comment><comment><USER4> It's unfortunate that it apparently doesn't parse _how many_ you eat -- you need a separate tweet for each consumption.

<comment><AUTHOR> I did consider that, but it would be far too easy to just put #mpt #eaten 10000. So I left it as 1 per tweet.

</comment></comment><comment><USER5> You might be interested in something similar I've been running for a few years:

<http://mincepi.es>

</comment><comment><USER6> Ok, I'll bite (terrible pun intended), what's the point?

<comment><USER7> <http://isitchristmas.com/>

<comment><AUTHOR> I generally hate anything christmassy before December 1st, but after that point, it's fair game. That includes mince pies.</comment></comment></comment>


Book Review: A Philosophy of Software Design - <AUTHOR> http://www.pathsensitive.com/2018/10/book-review-philosophy-of-software.html <comment><USER2> This book has been my own personal technical revolution over the past few months. I've been programming professionally for 7 years now, so while I'm not the most experienced I'm certainly no one new either. Some takeaways from how it changed my flow:

1\. It gave me specific goals in terms of my own improvement, and the chapter titles did a good job of being a stand-in for challenging my own intuition.

2\. In mentoring other developers, those I shared some of the lessons in the book with saw huge improvement literally week over week. I saw, personally, one of my mentees go from producing code that was hard for me to step through without bringing up half a dozen issues, to producing code where my largest criticisms were ecosystem dependent.

3\. It created common language for discussing with teammates why certain code felt wrong.

4\. It significantly reduced time spent refactoring later. The subtleties in some of the points make a re-read valuable as well. One of the better examples of this, was "Make code just a little generic"\- often, we attempt to code things as generic as possible, and end up making what is effectively code soup that doesn't help accomplish the problem. Instead, building specifically for this problem, but trying to spot places it can be made just a little bit more generic, has altogether eliminated times I know I would've had to refactor again later if I hadn't been thinking of it.

5\. The book knows what it isn't. He even says right in the forward and on the related Google Talk[1], that this first version is probably not the most value it can produce, but that hopefully by version 3 it'll be there.

For my time and energy, it's been more influential on how I develop software than anything else I've read. Furthermore, despite coding in primarily functional and/or dynamic languages, the examples used never made that a problem despite always being based on C. I recommend it highly.

1\. [https://www.youtube.com/watch?v=bmSAYlu0NcY](https://www.youtube.com/watch?v=bmSAYlu0NcY)

<comment><USER3> > _4\. It significantly reduced time spent refactoring later. The subtleties > in some of the points make a re-read valuable as well. One of the better > examples of this, was "Make code just a little generic"\- often, we attempt > to code things as generic as possible, and end up making what is effectively > code soup that doesn't help accomplish the problem. Instead, building > specifically for this problem, but trying to spot places it can be made just > a little bit more generic, has altogether eliminated times I know I would've > had to refactor again later if I hadn't been thinking of it._

Wow, I've often felt this during my time coding. Unnecessary abstractions often just make things more convoluted and increases boilerplate when adding new things. Great to see this being addressed.

<comment><USER4> This principle is well known: 'YAGNI'.

<comment><USER5> You Ain't Gonna Need It is a great principle, but I think this is slightly different, as it implies it's useful to abstract it a little... So maybe JAIL?

Just Abstract It a Little

<comment><USER6> Agreed. JAIL is YAGNI/KISS with a little bit of sanity sprinkled on top of it. You generally want to keep code simple and straightforward, but that doesn't mean avoiding the most obvious and most probably useful generalizations.

</comment></comment></comment></comment></comment><comment><USER7> I always wanted to write a "Philosophy of Software Design", except I wouldn't dictate HOW to write software. Instead I'd focus on what questions to ask and what trade-offs to expect. It would be Yoda-ish: Yoda teaches you how to ask questions and how to find answers, not the answers themselves.

In the end: there is NO right way, only trade-offs. At least know what the tradeoffs are. My book would be a trade-off recognition training guide.

I would state my experience, though. Such as "In my experience, typical shops are more comfortable with pattern X over pattern Y. I have no idea why: it's a psychology question I don't have the funds to answer."

And there'd be a chapter on how to recognize and scrutinize fads.

<comment><USER8> Write it! Start with a blog post series if "a book" is too great a hill to climb!

<comment><USER9> Seconded. Start with a series of blog posts to gauge interest, level of effort etc.

</comment></comment></comment><comment><USER10> I haven't finished the book and I am not sure I will. I was quite disappointed because from the title and description I was expecting something deep, that presented more questions than answers, navigated constraints, attempted to discuss the nuances of defining the problem of designing itself. On the contrary, it is a collection of superfluous recommendations for Object Oriented programmers presented without much justification.

The writer uses a very narrow perspective and just "assumes" all this can be generalized. One of the most recurring topics in the book is "complexity", but it is just introduced with a narrow one sentence tautological definition without offering any interesting discussion. All kinds of assumptions are taken for granted and not discussed.

So overall, the book reads so far like a bland collection of "do this, don't do that blog-posts". Papers like "Out of the tarpit", speakers like Rich Hickey, etc. offer much more interesting discussions on the topic of design and complexity, and are definitely more argumentative and "philosophical".

I even feel bad for writing something in such a negative vibe about a book. But really its pretentious title lead to such disappointment...

<comment><USER11> I've read his book cover to cover and have enjoyed and learnt from a number of the concepts he discusses.

That being said, after recently reading "Out of the tarpit" and being a long time Rich Hickey fan, I completely agree that the model of complexity presented by the book feels simplistic.

I can't quite put my finger on why though. Maybe cause complexity in software has a lot more to do with understanding the problem domain rather than just the structure of the code. IMO it was worth the read, but I was slightly disappointed and it was without a doubt not a be all end all reference. (And it would be unfair to expect it to be.)

</comment></comment><comment><USER12> I feel like this review is making an important point, but the point it's making is talking past the point Ousterhout is making, and so this isn't much of a rebuttal at all.

The idea behind "deep modules" could be expressed as _learning an interface should save you work, not cost you work_. There's a couple of deep insights in that. One is that every new interface you create imposes a cost on the programmer who is reading the code, and so you should be sparing in them. New functions and new classes are not free; each time you add one, it makes it that much harder for the maintenance programmer or library user to grok the API as a whole.

The other big insight is in _embedding complexity in the code_. This was a perspective shift I learned from talking with Ben Gomes (my boss's boss at the time, and now head of Search at Google) while complaining about how messy some of Google's code was. His argument was that the code is complex because the problem is complex. By embedding the complexity into the code, you let the computer deal with it on every search request instead of letting the user deal with it on every search request. The number of times a programmer will look at the messy code probably numbers in the single digits, while the number of searches that code gets per day numbers in the billions, so by making things _a little_ better for the user at the cost of it being _much worse_ for the programmer, you are saving millions of hours in mental tax for humanity.

As programmers, we all have a drive toward simplicity, but it's worth remembering that we exist within an economic reality too. If you want people to use your product, your product has to _do something they don 't want to do themselves_, and if it's easy for them to whip up a quick tool that solves their problem, your product is doomed in the marketplace.

The point the review is making seems to much more about _protocols_ , and the ability for interfaces to serve as a firewall that lets different implementations interoperate. If you don't specify that interface precisely, you get abstraction leakage, where an interface that seems simple at first actually can become very complicated.

This is a valid and interesting point too, but it's worth remembering that the vast majority of interfaces have only a single implementation. It's not necessary to specify that interface in precise detail as long as the user of that code has a reasonably intuitive view of it. They can always go look up the details as necessary, when the strange behavior occurs, while the "deep module" saves them a significant amount of work as long as they stick to the happy path.

<comment><USER6> > _The number of times a programmer will look at the messy code probably > numbers in the single digits, while the number of searches that code gets > per day numbers in the billions, so by making things a little better for the > user at the cost of it being much worse for the programmer, you are saving > millions of hours in mental tax for humanity._

Thank you. I'm going to quote it, and keep quoting it, at the "developer time is more important than machine time" crowd - the one that then goes and externalizes all this "machine time", making it "user's time (and electricity)".

<comment><USER13> I think it's great but it shouldn't be quoted without the part about the complexity of the problem. Because often you have simple problems solved in a complex manner that also lead to slow/buggy code that angers the user.

</comment></comment></comment><comment><USER14> By far the most interesting part of this is the contention:

interfaces should be simpler than their implementations

Which the author then goes to some lengths to absolutely reject.

I find this very interesting, because 1) this is the view I've always held and 2) I'm having trouble accepting the authors proposition than an interface and a _formal specification_ are equivalent.

...if they are, certainly, the non-code 'level 3 artifacts' that define code behaviour but are not code, will be much more than the implementation in some cases.

However, I dispute more strongly that the principal is wrong.

Surely not every implementation is longer than its interface, but _in general_ striving to avoid bloated hideous interfaces is ideal.

Certainly it's a sliding scale... and the idea of 'too simple' an interface that is superficially simple seeming, but actually very complex (eg. the file I/O example) is food for thought.

...but I feel too much is made of rejecting the point that all interfaces should be simpler than their implementation _absolutely_.

Did the book really claim that?

Hm.

I want to actually read it myself now.

<comment><USER15> I agree with you that the review seems to have misunderstood the author on this point. I wrote a comment on the blog to explain more. I think the review mixes up the words abstraction and specification, especially once I dug a little bit into the correspondence on the draft he posted for the book author's feedback.

</comment></comment><comment><USER16> The reviewer seems to say that he has a higher level of programming mastery than the author of the book.

Thats where I could not listen to him anymore. I would entertain the idea that theoretically programming languages that have features like contracts or specifications could be more robust or effective in some ways or for some applications. And actually I am interested in seeing examples of feature rich applications written in languages with these types of features.

But I am not convinced of the practicality of these languages or the superiority of the reviewer. The proof is in the pudding. I would challenge the reviewer who has clearly placed their knowledge and skills above the creator of Tcl/Tk and RAMCloud to show the useful projects he has created that prove this superiority.

<comment><USER17> I like to be open minded but he does seem to have spent most of his time in academia, and not a lot of time spent on commercial software. So I wonder if its genius or hubris it can be hard to tell the difference.

</comment></comment><comment><USER18> JO (John Ousterhout) is so thoughtful and lucid.

My experience with him is firstly through Tcl, which I like more than he does now (time for a TIP: "-jo" switch to [unset] which removes complaint to unset non-existent variables)[0], and his book on it[1], and various papers. His work is at once both powerful and humble. Note in [0] how he without much emotion he suggests how Tcl came to be less prominent, and how even at Tcl's height, he explained without ego why it was there, and how it would leave[2]. His papers on threads[3] and comments on performance and testing[4] are also lovely reads. I encourage any developer or architect to read or re-read them.

[0] [https://www.youtube.com/watch?v=bmSAYlu0NcY](https://www.youtube.com/watch?v=bmSAYlu0NcY) (thx `quaunaut)

[1] [https://www.amazon.ca/Tcl-Tk-Toolkit-John- Ousterhout/dp/0201...](https://www.amazon.ca/Tcl-Tk-Toolkit-John- Ousterhout/dp/020163337X)

[2] [https://vanderburg.org/old_pages/Tcl/war/0009.html](https://vanderburg.org/old_pages/Tcl/war/0009.html) (This is The Law, and it is good.)

[3] [https://web.stanford.edu/~ouster/cgi- bin/papers/threads.pdf](https://web.stanford.edu/~ouster/cgi- bin/papers/threads.pdf)

[4] [https://web.stanford.edu/~ouster/cgi- bin/sayings.php](https://web.stanford.edu/~ouster/cgi-bin/sayings.php)

</comment><comment><USER19> I'm going to disagree witha lot of this review. Here's an example:

" _In other words, take a conditional out of the spec for the function. But in Section 10.5, he tells us that we should add a conditional to the spec of a function, namely in making Java's substring method defined for out-of-bounds indices. I'm not completely sure he's wrong (as I discuss in my Strange Loop talk, it comes down to: is there a clean way to describe this behavior?), but I find his claims that this makes the code "simpler" only slightly more credible than his claims about the Unix file API._ "

Ousterhout contrasts that with the similar operation in Python. Substring throws exceptions if the indices are out of range, Python returns the part of the string that is in-range. For example, calling it witha start, end pair where start > end results in an empty string, not an exception; it's much easier to use in my experience. And I doubt the formal specification is any larger, to address the author's point.

The formal does of Unix I/O is a horror show, sure, but what does the spec of it plus all of the filesystem that can back it look like?

<comment><USER7> This sounds like the "limp versus die" design decision: [http://wiki.c2.com/?LimpVersusDie](http://wiki.c2.com/?LimpVersusDie)

The "right" answer depends on the domain. For a web startup, you may want to live with occasional errors being ignored in order to get your product up quickly and grow market share. Early Amazon.com used to fudge up my orders, but such didn't end the company.

But a bank probably doesn't want the software to "guess" if there is a potential numerical problem. It would rather have the batch process "crash". Otherwise, it could generate thousands of bad transactions and get sued to Pluto.

People used to debate this with MySql's default "truncation" setting. MySql was popular with start-ups because you could get it up and change the tables quickly without dealing with persnickity details, living with stuff occasionally falling thru the cracks.

<comment><USER19> The difference is between total and partial functions. A total function can have well-defined results (thus, no guessing) and is much easier to use than a function that unnecessarily fails.

</comment></comment></comment><comment><USER20> The greatest thing about this book is that it develops an antidote to the work of the gang of three (Martin, Fowler, Beck), which would have everybody write only 3 line functions TDD style if they could.

This can be seen for example in the intro chapter to Fowler's 2nd edition of Refactoring, where he pulverizes some code into submission.

<comment><USER21> That has been exactly my sentiment after reading the book (although I'm only anti-Uncle Bob, not against Fowler).

</comment></comment><comment><USER22> I'm enjoying Ousterhaut's "A Philosophy of Software Design", but it is striking how much it is shaped and limited by the languages that are implicitly considered (mostly, imperative OO languages).

One oddity is the view of the class as the only real scope at which one can build abstractions. The idea that one can use little abstractions to build big abstractions, without cluttering the final abstraction to be presented to the user, is nowhere in sight.

Most shockingly to me, I've found no reference to types as a design tool. Sure, precise names help you avoid confusing physical block ids and logical ones. But wouldn't a type-based distinction be way better?

\- [https://twitter.com/yminsky/status/1005066579929911296](https://twitter.com/yminsky/status/1005066579929911296)

<comment><USER23> Good point about using types. I should have mentioned that as a way to eliminate the block number problem in Section 14.1; I'll fix that in a future revision of the book.

<comment><USER22> That'll be a great addition to the book.

Can I also recommend Yaron Minsky's talk on OCaml, specifically the part where he talks about "making invalid states impossible": [https://youtu.be/-J8YyfrSwTk?t=1079](https://youtu.be/-J8YyfrSwTk?t=1079)

I've found this to be the single most valuable programming technique I've learnt.

Richard Feldman dives deeper into the topic with a lot more examples in [https://youtu.be/IcgmSRJHu_8?t=73](https://youtu.be/IcgmSRJHu_8?t=73)

</comment></comment></comment><comment><USER24> Meta 1: I enjoyed his review. It got me interested in learning more about all these concepts I've never heard about: "Level 3 Software? What's that?". It is not clear to me if a lot of this terminology is of his creation or if it comes from literature.

Meta 2: What better way to establish yourself as an expert than disagreeing with a currently recognized one? Not saying he did not back up his claims. In fact, I want to read more of his materials now to check that up.

Meta 3: This reading also gets me excited about investigating model driven development more. In this post, he talks a bit about the value of strict types related to such modeling, but lately I've been reading and writing some Clojure code and I'm slightly swinging back to more dynamic typing as a more flexible way of writing modular systems.

<comment><USER25> Level 1, level 2 and level 3 are the authors conceptions. Another post of his details what he means... [http://www.pathsensitive.com/2018/01/the-three- levels-of-sof...](http://www.pathsensitive.com/2018/01/the-three-levels-of- software-why-code.html?m=1)

</comment></comment><comment><USER26> What are some of the best design books you people would recommend for a junior developer?

<comment><USER20> This one. I also read Code Complete (2nd ed) when I was a junior and found it very helpful, but I don't know how well it has aged.

Object Design by Wirfs-Brock is a practical OOP book.

I would avoid anything with clean code in its name, they're generally dogmatic and obsessive about TDD.

</comment></comment><comment><USER27> > Thanks John Ousterhout for his extensive comments on this review

That's a bit meta, eh? The author of the book being reviewed reviewed a review of his book.

</comment><comment><USER28> I'm a recent college grad from a good CS program. How do I get better at bugfixing and diagnosing code? To give a specific example: I was recently working with a Tensorflow code base for image recognition, and Tensorflow kept hanging (no error, just hanging) or giving me shape errors. I felt stuck. First, the lack of error made it hard to diagnose, and when I did get the shape error, I didn't understand why quickly. I find myself too dependent on stackoverflow for bugfixing and if someone hasn't solved it, I feel helpless. I've tried reading through the Tensorflow source but it's pretty verbose and hard to understand. Are there any books that give you concrete methods to really understand other people's code well and fix bugs?

<comment><USER29> In my understanding, to begin learning diagnosing issues in code you need to understand your stack very well. Try to invest time explicitly learning bits and pieces of how your platform works, in detail.

Once you learn few different platforms and spend some time diagnosing problems you will start recognizing recurring patterns in problems. Hopefully this will gradually become your ability to diagnose difficult issues on unfamiliar platforms.

</comment></comment><comment><USER30> I've been wanting to read this book but there's not a digital version yet. I wonder if it's to avoid pirate copies everywhere.</comment>


All You Did Was Weaken A Country Today - <AUTHOR> http://techcrunch.com/2010/09/10/all-you-did-was-weaken-a-country-today/

<comment><USER2> Arrington is completely inconsistent when it comes to the right to free speech. For instance, when it comes to another unsavory free-speech issue, Holocaust denial, he writes:

"Sure, we can't shut down the dark places on the Internet where people are free to hate Jews [...] But Facebook can take a stand and say it won't happen in their back yard. Holocaust denial is hate speech, and it cannot be given a place to take root." (from [http://techcrunch.com/2009/05/12/facebook-remains- stubbornly...](http://techcrunch.com/2009/05/12/facebook-remains-stubbornly- proud-of-position-on-holocaust-denial/))

He could just as easily swap out prostitution for Holocaust denial and Craigslist for Facebook, and make the exact same argument, no? After all, I'm pretty sure sex traffickers are doing more damage to society today than a handful of anti-semitic cranks.

<comment><USER3> That's backwards.

Both facebook and craigslist have rights to block what they want. In facebook's case they wanted to block. In craigslist's case they don't want to block. Free speech is about letting fb or cl decide what they want to block or allow in their site and it is not contradictory to support both: fb's right to block and cl's right to allow.

<comment><USER2> Read the article. FB did not want to block and Arrington was arguing that they must.

<comment><USER4> Did he say they should be forced to block by the law or was he making a strong moral argument that they should block? My impression is that both FB and CL were (possibly inconsistent) moral arguments and not so much about the right to free speech.

</comment></comment></comment></comment><comment><USER5> Why doesn't the EFF publish a voter's guide?

Screw republican v democrat, these are the kind of issues that matter to me but I'm too lazy to look up if my state's AG is one of the 17 that shookdown Craigslist. Even if I did I don't know when that election is, and if it's 2 or 4 years from now I'll probably forget this issue. And there are other issues and other candidates that I might care about but don't know about.

So I need an organization that I trust, like the EFF, to come out with a guide each election telling me who I need to vote for or against.

</comment><comment><USER6> Ironic he chose the quote from A Few Good Men since Nicholson in that movie was arguing in favor of the forces for security, not the forces for freedom. His character would have been on the opposite side of this debate.

<comment><USER7> Perfect example of how we've become a soundbite nation. Nothing is presented in context anymore. Post-modernism...isn't it great?

<comment><USER8> In the context of this particular thread of the meta-narrative w/r/t the impedence of the distribution of control throughout the legal/cultural/informational network by widely recognized but contextually barren pop-culture touchstones, one can understand Arrington's outburst as a performative reference aimed at the distracted postmodern crowd, with little relation to the original semiotics of the sampled media. </pomo>

</comment></comment></comment><comment><USER9> Arrington's post seemed unnecessarily condemning of craigslist. Why should a listings website be burned at the stake for removing adult services?

I can't make the connection between "that kind of emotion", or rather, craiglist's decision to remove a section from its listings and "weakening a country". I know that it is hyperbole, but is America really less of a nation because craigslist doesn't have an adults services section?</comment>


Io.js v3.0.0 - <AUTHOR> https://github.com/nodejs/io.js/blob/master/CHANGELOG.md <comment><USER3> I thought they would merge back into node? Why are they releasing a 3.0 version (new major version breaks even more compatibility?)?

<comment><USER4> They will _probably_ release a merged version in October which will be named Node.js 4.0 and will be supported for 2,5 years.

[https://medium.com/@nodesource/essential-steps-long-term- sup...](https://medium.com/@nodesource/essential-steps-long-term-support-for- node-js-8ecf7514dbd)

<comment><USER5> Ah, seems version skipping is all the rage nowadays. Having an LTS version will be welcome though. Having to go in there and upgrade node every few months is a mild inconvenience on my project.

<comment><USER6> It isn't skipping versions. io.js is strictly following semver and have introduced breaking changes (usually due to a V8 upgrade) in each major bump.

<comment><USER7> Ah, seems semantic versioning is all the rage nowadays.*

Fixed it!

<comment><USER8> Yeah, having your version numbers mean something? What'll those wacky JS hipsters come up with next?!

</comment></comment></comment></comment></comment></comment><comment><USER9> The faster url parser is still not merged [1] (was supposed to be added to 2.0.0). All in all, I think even io.js is still very conservative compared to how node was pre-0.10

[1]: [https://github.com/nodejs/io.js/issues/643](https://github.com/nodejs/io.js/issues/643)

<comment><USER10> Thanks for the reminder - I saw this floating around months ago. @petkaantonov does fantastic work and I'm doing what I can to help get the PR merged.

<comment><USER9> Haha, no problem. Its been a bit frustrating watching that PR, as requirements went back and forth from complete backward compatibility to whatwg compatibility a few times and petka did everything to accommodate.

I just want to see node near the top of the list of benchmarks again - <USER2> never good publicity to be so close to PHP :)

</comment></comment></comment><comment><USER11> Was hoping for the inclusion of ES6 fat arrows. V8 has a solid implementation now.

<comment><USER12> You'd have to check which version of v8 added fat-arrow support... I've been using babeljs for most of my stuff regardless of node/v8 version as I want async/await support which is a ways off still.

<comment><USER13> The next+1 branch has a fairly recent version of v8 4.5 that has fat arrows and other new language features shipped. My guess is that next+1 will become 4.0.0.

</comment></comment></comment><comment><USER14> I'd love for there to be some sort of indication of moving towards using a lot of ES6 features in the Node.js API. Having the file system module returning Promises would be amazing, especially when async functions are standardised. What'll also be awesome is if the streams and HTTP module become Observables (async generators, an ES7 proposal).

</comment><comment><USER15> > Status codes now all use the official IANA names as per RFC7231, e.g. > http.STATUS_CODES[414] now returns 'URI Too Long' rather than 'Request-URI > Too Large'

Nice. Using the names (from a reversed copy of http.STATUS_CODES) can make a code using non-common HTTP codes a lot more readable.

</comment><comment><USER16> PowerPC support is exciting. Can't wait to give this a whirl on some of my better hardware.

</comment><comment><USER17> This can't be stable at this rate.

<comment><USER18> Do you have any reason to sustain this claim?

<comment><USER19> Have you tried io.js v3.0?

It may be stable itself but breaks many modules which may be why major version got bumped.

<comment><USER20> Have you tried reinstalling these modules? Often, some modules have C++ bindings that need recompilation every few V8 version bumps.

In any case, running `rm -rf node_modules` and `npm install` helps

</comment></comment></comment></comment><comment><USER21> The future of yesterday, today.

</comment><comment><USER22> I'm hoping they will converge towards Go, with its fine-grained threading model (with shared state), as opposed to an asynchronous model which is effectively the cooperative-multitasking that we remember from Windows 3.1.

Asynchronous programming forgets that the CPU is also a resource that needs to be managed in a non-blocking way. Currently, in Nodejs, long-running computations (e.g., computing a large prime number), totally block the CPU.

EDIT: For anybody saying that shared-state is a bad thing, read up on functional datastructures with structural sharing.

<comment><USER23> Never say "shared state" and "threading" in the same sentence.

If anything, all languages need to move to an Erlang shared-nothing model. But, Erlang was doing all this back in the 80s, so people have to re-invent it worse in the 2010s to feel special. You can't just go around using things other people made (OPC—other people's code) that just _work_ , you have to make it your own and start your own conferences and books and webinars and vine instaslacks.

Erlang is also preemptive (bumping reductions) so your "long running computation spinning on a thread" problem goes away too. Though, we can't say "goes away"—sounds too much like Go—how about it "erlangs away" the problem?

Decent write up on erlang scheduling at [http://jlouisramblings.blogspot.com/2013/01/how-erlang- does-...](http://jlouisramblings.blogspot.com/2013/01/how-erlang-does- scheduling.html?view=classic)

Also see decent writeup about shared-nothing at [http://jlouisramblings.blogspot.com/2013/10/embrace- copying....](http://jlouisramblings.blogspot.com/2013/10/embrace- copying.html?view=classic)

Erlang also has per-process (green thread) heap/GC (i.e. no 'stop the world' required), full load balancing over your CPUs, unbelievably useful pattern matching, atoms/symbols, full serialization of all native data types, built with network distribution in mind from the ground up, plus a syntax that doesn't look like somebody woke up from the 70s in 2010 and decided to write a new language just like their old favorite language. Any knocks against Erlang syntax is because it stands on the shoulders of giants, not because somebody made it in 2010 and ignored all programming language and usability research from the past 40 years.

<comment><USER24> Any benchmarks comparing Erlang, Go, C, Node performance for network applications (eg. max # connections, max requests/s, request latency, etc) that you can point to?

<comment><USER23> Benchmarks are interesting these days. Software can generally be made "fast enough," but there's no benchmark for maintainability or ease of coding.

What good is a Java program being 2x as fast as a Python program if the Java program requires 3,000 lines and two weeks to write while the Python program requires 300 lines and two days?

The idea for Go is "become the new Java but without that startup time penalty." The goal of Java was to be a "dumb enough" language where individual talent or skill or knowledge doesn't matter. So, you get a language not targeted at expressiveness or ease of thought or ease of programming, but at ease of using people as interchangeable cogs in a multi-billion dollar software development machine. Not really exciting to be a part of.

We've come full circle back to Averages ([http://www.paulgraham.com/avg.html](http://www.paulgraham.com/avg.html)) — startups _not_ using Go will fare much better than encumbered startups just due to the brain rot you're required to experience while using Go itself. (I mean, TABS? In 2015? Non-versioned web-placed dependencies? What is wrong with these people?)

<comment><USER24> >What good is a Java program being 2x as fast as a Python program if the Java program requires 3,000 lines and two weeks to write while the Python program requires 300 lines and two days?

I will stop you right there.

If there is a 2x difference it very well might be worth it to me if that means I only have to run half the machines I would with the slower solution. Run- time costs can dominate all other concerns for some problems.

Specifically, I am looking for Erlang vs. other (ie. C, Go, Node) performance commentary.

<comment><USER25> > If there is a 2x difference .. I only have to run half the machines

No, it doesn't work that way. CPU is almost never an issue, it's all about startup time, network latency -- that kind of stuff. So, unless you're mining bitcoins, 2x difference is negligible.

<comment><USER24> I guess I don't understand where Erlang would fit in your (my) stack.

If you want to handle millions->billions of external client connections, you may want to be able to pack as many on to a machine as possible to reduce costs (the difference between paying for 5,000 machines vs 10,000 machines). If you want these to be secure connections (which I do), there is significant CPU overhead in encrypting and decrypting all network traffic. Even if you aren't doing encryption work, there is still significant CPU work in managing large numbers of sockets. Implementations may also vary significantly in the amount of memory required to support the connection, affecting costs and machine count.</comment></comment></comment></comment></comment></comment></comment>


Glenn Beck's the Blaze Conducts Mass Layoff - <AUTHOR> https://www.mediaite.com/online/breaking-the-blaze-conducts-mass-layoff/ <comment><AUTHOR> I don't think we do ourselves any favors by delving into and attacking the politics of Glenn Beck or The Blaze (tensions are high enough already), either for or against.

This headline is interesting to me because The Blaze has been a pioneer in many ways as an internet-based media company that is attempting to disrupt a well-entrenched and very powerful establishment.

<comment><AUTHOR> Relevant linK: [https://www.disruptorawards.com/2013-honoree- blog/2017/1/19/...](https://www.disruptorawards.com/2013-honoree- blog/2017/1/19/glenn-beck-the-blaze)</comment></comment>


Chinese schools are using 'smart uniforms' to track their students' locations - <AUTHOR> https://www.theverge.com/2018/12/28/18159042/chinese-schools-smart-uniforms-track-student-location <comment><USER2> In Australia we simply track every single students mobile phone location by law and store it for years, a far more efficient way towards Orwellian dystopia.

<comment><USER3> Source please?</comment></comment>


Benzene and worker cancers: 'An American tragedy' - <AUTHOR> http://www.publicintegrity.org/2014/12/04/16320/benzene-and-worker-cancers-american-tragedy

<comment><USER2> The scary thing is how this type of behaviour has been happening for decades (maybe centuries) and implies that the scientific literature on a whole range of subjects simply cannot be trusted 100%.

If there is or was an economic, political or idealogical motive for skewing or misreporting something then it is quite probably there distorting the historical record.

</comment><comment><USER3> My father was an organic chemist and worked at the DuPont Experimental Station from 1947 to 1977. He was diagnosed with acute myeloid leukemia in 1977 and died after several months of chemotherapy. Our family suspected that exposure to benzene in his research lab at DuPont contributed to his illness, but we had no proof. I remember hearing stories about how, in the early years, lab workers commonly used benzene as a general cleaning solvent.

</comment><comment><USER4> I witnessed the cavalier attitude towards benzene and its relatives when I was in grad school (chemistry). But had I learned already during my bachelors how carcinogenic they were and stayed as safe as I could. I gave up and became a programmer instead (for many reasons including when software blows up you don't usually die). This was a long time ago but I doubt much has changed.</comment>


Ask HN: How does this device claim to "speed up your internet"? - <AUTHOR> See this device, "Winston".
https://winstonprivacy.com/
One of the product's claims is to increase your connection speed. I'm curious about the way they are doing that.
My only thought would be some reliable UDP? Anyone know or care to elaborate? <comment><USER2> Best guess: the blocking of trackers/adverts at the hardware level should increase your browsing speed marginally but I doubt it's anything more than that.</comment>