71 TiB DIY NAS Based on ZFS on Linux - <AUTHOR> http://louwrentius.com/74tb-diy-nas-based-on-zfs-on-linux.html <comment><USER2> I know I'm going to suffer hard for this comment. However I'd suggest reading the ZFS optimisation guide.

I know the allure of raid 7 is strong, however if a disk were to fail on a LUM thats 24 disks wide, the chances of a rebuild is pretty low. (I know you've since changed that) the rebuild time is around 60-90 hours with no load.

(we use 14 disk raid6 LUNs for the balance of performance vs safety, however we've come pretty close to loosing it all.)

You'll get better performance if you have many smaller vdevs. You'll also compartmentalize the risk of multiple disk failure.

a raid 0+6 of four LUNs of 5 disks will have much greater performance, and will rebuild much faster, without risking too much. However you will loose more space

<comment><USER3> Nope, you're right on the money. My first thought was "this guy needs more VDEVs". Given the hardware and use case, 4x 6 disk RAIDZ2 VDEVs would make a ton more sense, and could also be cross-cabled to allow a controller failure.

<comment><USER4> Smaller VDEVs make it a bit easier to increase size if you're rolling your own system. Buying 6 more drives instead of 8 or 10 is a smaller cost.

</comment></comment></comment><comment><USER5> ZFS is pretty great, but it's not perfect. Occasionally you have to dump and re-create a whole pool to fix corruption:

        [email@example.com ~]$ zpool status -v
          pool: arc
         state: ONLINE
          scan: scrub repaired 0 in 0h43m with 0 errors on Wed Aug 13 20:14:53 2014
        config:

            NAME          STATE     READ WRITE CKSUM
            arc           ONLINE       0     0     0
              mirror-0    ONLINE       0     0     0
                gpt/arc0  ONLINE       0     0     0
                gpt/arc1  ONLINE       0     0     0

        errors: No known data errors

          pool: tank
         state: ONLINE
        status: One or more devices has experienced an error resulting in data
            corruption.  Applications may be affected.
        action: Restore the file in question if possible.  Otherwise restore the
            entire pool from backup.
           see: http://illumos.org/msg/ZFS-8000-8A
          scan: scrub repaired 0 in 5h3m with 0 errors on Tue Aug 26 00:19:05 2014
        config:

            NAME           STATE     READ WRITE CKSUM
            tank           ONLINE       0     0     0
              raidz2-0     ONLINE       0     0     0
                gpt/tank0  ONLINE       0     0     0
                gpt/tank1  ONLINE       0     0     0
                gpt/tank2  ONLINE       0     0     0
                gpt/tank3  ONLINE       0     0     0
            logs
              mirror-1     ONLINE       0     0     0
                aacd5p1    ONLINE       0     0     0
                aacd6p1    ONLINE       0     0     0

        errors: Permanent errors have been detected in the following files:

                /usr/nginx/var/log/nginx-access-appengine.log.0

I can't delete that file or recover it from backup, and don't really trust the pool or our disk controller anymore. So we're moving to a new server, again. Good times.

I imagine this would be disastrous for someone who's filled a 24 disk array with data, and has to purchase a duplicate to safely restore to.

<comment><USER6> This is one of the key strengths of ZFS. Your log file would be _silently corrupted_ without checksumming.

<comment><USER7> Regardless, it's still frustrating that so many failure modes for ZFS have an answer of "blow it all away and start again". And that's with Solaris/ZFS.

<comment><USER8> There are other ways to fix issues with ZFS pools without having to " _blow it all away and start again_ ". In the example that started this conversation fork, there's a simple parameter to pass to the zpool utility that would have saved the OP from dropping his pool:

        zpool clear tank

ZFS does actually come with a decent set of tools for fixing and clearing faults - which I'm starting to realise isn't as widely known (possibly because people rarely run into a situation where they need it?)

Anecdotally speaking; I've ran the same pool for years on consumer hardware and that pool has even survived a failing SATA controller that was randomly dropping disks during heavy IO and causing kernel panics. So I've lost count of the number times ZFS has saved my storage pool from complete data loss where other file systems would have just given up.

</comment></comment></comment></comment><comment><USER9> This guy would have more predictible performance and higher reliability if he had organized the array as two 12-drive raidz2 vdevs (instead of 18-drive and 6-drive). The reasons are because (1) the chances of triple drive failure though low is higher in a 18-drive vdev than in a 12-drive one, and (2) performance (IOPS) is variable between two asymmetrical vdevs.

There is very rarely a good reason for doing asymmetrical vdevs in a single ZFS pool.

I approve of his choice of HBA (based on the SAS2008 chip). For those looking for even bigger setups, I maintain a list of controllers at [http://blog.zorinaq.com/?e=10](http://blog.zorinaq.com/?e=10) where you can even find 16-, 24- and 32-port controllers. 8-port ones are the most cost effective as of today.

I have been a happy ZFS user for 9 years, both personally and professionally. Using snapshots, running periodical scrubs, having dealt with a dozen of drive failures over the years. Everything works beautifully - it's a pleasure to admin ZFS.

<comment><USER10> Any hints on building a ~16TB array? Drive size, RAM size?

I've been eying this - [http://www.supermicro.com/products/motherboard/Atom/X10/A1SA...](http://www.supermicro.com/products/motherboard/Atom/X10/A1SAi-2750F.cfm), but the project is firmly in the "one day when I have a spare $5k" basket.

</comment></comment><comment><USER11> Echoing one of the comments -- why Linux? I thought most open source ZFS configurations used FreeBSD. (I know there are the open source Solaris variants, but I would imagine most customers still pay for support.)

ZFS is production ready on Linux now?

<comment><USER12> There is indeed a great and active open source ecosystem of Illumos (ex open- solaris) distributions.

Illumos is the de-facto upstream of open-zfs and has many features not found in oracle solaris. [1]

So it is well worth checking out at least one of those:

\- SmartOS, which is a fantastic Cloud-Hypervisor [2]

\- OmniOS, a classic server distribution, probably best for what OP is building. [3]

\- NexentaStor, a storage appliance - but i have not tried it yet [4]

I can definitely recommend using ZFS on a Illumos system as it has much better integration with things like kstats and fault management architecture that I would miss on any other OS.

[1] [http://open-zfs.org/wiki/Features](http://open-zfs.org/wiki/Features)

[2] [http://smartos.org](http://smartos.org)

[3] [http://omnios.omniti.com](http://omnios.omniti.com)

[4] [http://www.nexentastor.org](http://www.nexentastor.org)

<comment><USER13> I notice you didn't mention OpenIndiana. Is OmniOS now recommended over OpenIndiana for the classic-server-install case?

<comment><USER14> snw's post might be reflecting the fact that OpenIndiana isn't seeing a lot of updates (except perhaps for those that come directly from Illumos.) For example, when the heartbleed bug came to light, it turned out that OpenIndiana wasn't vulnerable, because the version of ssh they had was so old it had never had the vulnerability introduced into it in the first place.

As yet I don't have any hands-on experience with OmniOS, but I'd tend to agree that it seems the best direction to go looking in, for a conventional OpenSolaris-derived OS.

<comment><USER12> this is indeed the biggest reason. They have a "hipster"-branch [1] that is getting some updates but I don't follow that closely. Another reason is that they try to target the "developer desktop" which is not that interesting to me personally and also probably a lot more work (driver wise) than they might have the manpower for...

OmniOS on the other hand is pretty solid and with pkgsrc (default package system on SmartOS and NetBSD) you get nearly 13k packages of open-source software.

My List is also missing a few other commercial players that use and contribute to Illumos:

\- Delphix, uses ZFS snapshots to do fancy things with big databases

\- Pluribus has "Netvisor OS" that is some SDN Router/Networking appliance [3]

And then there is the long tail of small hobby distributions:

\- Tribblix, "Retro with modern features" [4]

\- DilOS, which has a focus on Xen and Sparc [5]

and probably more that I forgot and don't know.

It is a very nice community and lots of interesting technology gets developed. Our company migrated nearly all servers from Linux to SmartOS in the last 2 years and we could not be happier.

[1] [http://wiki.openindiana.org/oi/Hipster](http://wiki.openindiana.org/oi/Hipster)

[2] [http://pkgsrc.joyent.com/installing.html#installing-on- illum...](http://pkgsrc.joyent.com/installing.html#installing-on-illumos)

[3] [http://www.pluribusnetworks.com/products/netvisor- os/](http://www.pluribusnetworks.com/products/netvisor-os/)

[4] [http://www.tribblix.org](http://www.tribblix.org)

[5] [http://www.dilos.org/about-dilos](http://www.dilos.org/about-dilos)

<comment><USER13> Thanks, that's helpful! I'm mainly experienced with Debian, but I've been exploring whether Debian-for-everything is still the best policy. In terms of actually trying anything else, I've only been doing some test setups of FreeBSD so far, but "something Illumos" has been on my radar as well and I've been trying to understand the landscape there, along with playing a little bit (not very seriously, I'll admit) on a SmartOS instance through Joyent's cloud.

Technically it seems quite impressive, and Illumos being more or less the ZFS upstream is appealing. The distribution situation has been more confusing, though. OpenIndiana seemed to have a community, which is why I was first looking into that option, but it does indeed not seem to be very active. That's one attraction of FreeBSD & Debian, that they're community supported, and I can with fairly high confidence expect they'll be around in 5 or 10 years supporting the distribution. It's less clear to me how to wade into Illumos in a way that mitigates risk of the upstream going away. I think Illumos itself fits that description, with a community that will be around, but does any individual distribution? There are clearly resources behind OmniOS and SmartOS, which gives some confidence, but they are also quite heavily concentrated resources (one company drives each, and it's not impossible that they could change focus and deemphasize development of the public distribution).

<comment><USER12> Thats a very valid concern. For SmartOS it would be a major blow if Joyent were to pull all ressources. They have some of the brightest engineers I know working on it and I have learned a lot from them by just using the system and hanging out on the irc channel in freenode. Without them the speed of development would definitly take a hit.

On the other hand they have been - and still are - very good at building a community around it. There are multiple people and organisations building their own SmartOS images/derivates ([1], [2]). Pull-requests on github come in from a diverse enough group that I think it would survive even without joyent.

For OmniOS I don't have enough insight to make that judgement. They have a very clear and detailed release plan and recently hired some good people just to work on OmniOS. These things definitly add some confidence but of course is not a guarantee forever. But if that would be needed one can always buy support from them.

[1] [http://www.dogeos.net](http://www.dogeos.net)

[2] [http://imgapi.uqcloud.net/builds](http://imgapi.uqcloud.net/builds)

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> We've been running ZFS on Linux in production at MixRank for 6 months. We push it hard and haven't had any issues yet. We're using arrays of 18x Intel 530 240GB SSDs in raidz2. It's performing very well: 697k read, 260k write IOPS; 3768 MB/s read, 1614 MB/s write throughput.

I recommend it, but only after you've done enough research to feel comfortable with all of ways to configure it and understand the internals from a high level. For a personal setup though, you can just install it and forget it.

</comment><comment><USER16> Nice build, I have something similar but with 3TB disks. My disks are grouped into 8 disks as the thought of trying to recover the whole array if that fails I'd probably cry a little.

Also I invested in 10GbE, at first was only server to server using some SFP+ NICs and DAC leads I picked up off eBay for between £10 and £80. But have since bought a couple of Dell 5524 switches and a PCIe enclosure for my Mac and having 650MB/s read/write to my server from the desktop certainly reduces the waiting around :)

</comment><comment><USER17> Used Dell C2100, 2xXeon E5620 (quad core) 96GB ram, Chelsio 10GbE

Two 2.5" 160GB internal SSDs for ZIL and l2arc.

12x4TB drives.

$2000 + Chelsio NIC ($230) plus 12 x 4TB WD Red ($2000)

So for $4250. I get 48TB and a better performing system in 2U.

His system costs 6012 EUR, or about 7900 USD at today's exchange rate. For $8500 I can fit 96TB in the same space, have 192GB of RAM, 16 cores @ 2.4GHz, 740GB of SSD for ZIL and l2arc.

And redundant power.

<comment><USER18> You're not comparing correctly. Your system would only have 96TB if you weren't using any form of RAID-like caching. The guy from the article is using ZFS caching to improve data reliability.

<comment><USER17> He has 24 4TB drives in 4U, but only 16GB of RAM, and a single E3-1230 (quad core) at 3.30GHz, and 4 x 1Gbps networking.

I'm not sure what a "RAID-like cache" is. I know what redundancy is, and I know how ZFS works.

16GB is likely adequate for his storage needs. He doesn't have that many clients accessing the storage.

and I could keep two complete copies of the data on two machines, each with redundant power, etc.

</comment></comment></comment><comment><USER19> Reading this just makes me think of this [http://www.smbitjournal.com/2014/05/the-cult-of- zfs/](http://www.smbitjournal.com/2014/05/the-cult-of-zfs/)

<comment><USER20> This article is quite ignorant.

ZFS uses integrated software RAID (in the zpool layer) for technical reasons, not merely to "shift administration into the zfs command set". Resilvering, checksumming, scrubs are all a unified concept that are _file_ aware, not merely _block_ aware as nearly every other RAID. The implications of this are massive and if you don't understand, please don't write an article on file systems.

For various reasons, the snapshots and volume management are more usable due to the integrated design and CoW, and also as pillars for ZFS send/receive.

The "write hole" piece is bullshit. ZFS is an atomic file system. It has no vulnerability to a write hole.

The "file system check" piece is bullshit. Again, ZFS is an atomic file system. The ZIL is played back on a crash to catch up sync writes. A scrub is not necessary after a hard crash.

Quite frankly, for any modern RAID you probably should be using ZFS unless you are a skilled systems engineer and are balancing some kind of trade off (stacking on higher level FS like gluster/ceph, fail in place, object storage, etc). You should even use ZFS on single drive systems for checksumming and CoW, and the new possibilities for system management with concepts like boot environments that let you roll back failed upgrades.

Hardware RAID controller quality isn't spectacular, and the author clearly has not looked at the drivers to dish out such bad advice. You want FOSS kernel programmers handling as much of your data integrity as possible, not corporate firmware and driver developers that cycle entirely every 2 years (LSI/Avago). And there's effectively one vendor left, LSI/Avago, that makes the RAID controllers used in the enterprise.

ZFS is production grade on Linux. Btrfs will be ready in 2 years, said everyone since it's inception and every 2 years thereafter. It's a pretty risky option right now, but when it's ready it delivers the same features the author tries bizarrely to dismiss in his article. ZFS is the best and safest route for RAID storage in 2014 and will remain such for at least "2 years".

</comment></comment><comment><USER21> > My zpool is now the appropriate number of disks (2^n + parity) in the VDEVs. > So I have one 18 disk RAIDZ2 VDEV (2^4+2) and one 6 disk RAIDZ2 VDEV (2^2+2) > for a total of twenty-four drives.

That's a catastrophe waiting to happen. EMC recommends no more than 8 disks in a parity array group (ZFS recommends no more than 9), specifically because larger arrays cause longer rebuilds, which are more likely to trigger a URE, in which case your SOL.

<comment><USER22> Can you expand the acronyms? I'm having the most trouble with URE.

<comment><USER23> unrecoverable read error.

</comment></comment></comment><comment><USER24> Has anyone tried HAMMER (a DragonFly BSD creation) and have any thoughts on it compared to ZFS?

<comment><USER13> HAMMER is currently in the middle of a major rewrite to HAMMER2. That's supposed to support multi-master cluster usage, in addition to ZFS/btrfs-like features, so is quite ambitious. The new version is the main focus of full- time development, but not yet considered production-ready. I think the old version is stable-ish but was considered a dead-end by the main developer, so it doesn't have much deployment or development. I would consider the project research-stage at this point, though it's quite interesting research.

</comment></comment><comment><USER25> How many Libraries of Congress is that?

Isn't there still a danger of typing something silly, an 'rsync --delete' type of command that bricks the whole thing? Rather than the one Borg Cube I think I would prefer to add disks/machines with disks on an as required basis, or keep the last system in the loft 'just in case'.

I am still amazed though at how much disk people 'need'. Even in uncompressed 4K resolution it would take a long time to watch 71TiB of video.

<comment><USER26> He does not mention it, bug ZFS has great support for snapshots (compared to e.g. LVM snapshots). You don't have to carve out disk space in advance where snapshots are stored and they are relatively high performance, so you can have an hourly, daily and weekly rotating set of snapshots (there are some scripts around for that retention). As the snapshots take up space, your available disk space just diminishes. And you can set it up so while in you are in dir "foo", your "foo/.snapshot/16:00/" has the contents of that directory when shapshotted hourly.

Another cool feature: you can do a send a diff between 2 snapshots as a file stream (over e.g. SSH) and replay it on another machine. So that's like using rsync except it's actually a stable snapshot (rsync can't fully handle files that changes while it's copying them) and you don't have to re-checksum every part of every changed file. So you can use that for DR.

Transparent compression is also great (as is deduping -- if you have tons of spare memory!)

BTRFS will be able to do the same thing... one day.

<comment><USER27> > BTRFS will be able to do the same thing... one day.

It already can. btrfs sub snap does snapshots (and they work the same way), btrfs send/receive do the diffs, supports lzo/zlib compression and deduplication

<comment><USER8> The last time I tried Btrfs (which was about a year ago on ArchLinux) I found Btrfs snapshots to work very differently to ZFS. At least superficially as most of the guides I could find advocated using _rsync_ to restore snapshots - which even now I find it hard to believe there isn't a cleaner solution.

Aside all of the positive things already mentioned about ZFS, one of the biggest selling points for me is just how easy it is to administrate. The _zfs_ and _zpool_ command line utilities are child's play to use where as _btrfs_ felt a little less intuitive (to me at least).

I really wanted to like Btrfs as it would have been nice to have a native Linux solution that would work on root partitions (my theory being that Btrfs snapshots could provide restore points before running system updates on ArchLinux) but sadly it proved to be more hassle than benefit for me.

</comment></comment></comment></comment><comment><USER4> I have a similar system at home (almost exactly the same except I don't have the quad network adapter) using 2T drives from Hitachi. I quite love being able to save everything, and the peace of mind that ZFS brings.

As a plug, I've been using CrashPlanPro for offsite backup, I've been very happy with their service.

<comment><USER28> Do you also use linux?

<comment><USER4> ZFS on Linux is a really good project with a wonderful community. I have happily been using Linux and ZFS together for a long time.

</comment></comment></comment><comment><USER29> I'd like something like this but not such a big box. Are there any open source drobo-like projects out there? I'd like something like a drobo but one that's hacker friendly. Anyone know of a chassis like that?

<comment><USER7> Closest I can think of is the SilverStone DS380, which is a NAS-sized case that gives you 8 hot-swap SATA bays, 4 2.5" internal mounts, and supports mini-ITX motherboards.

</comment></comment><comment><USER30> ... if you're going to use ZFS, you should really think about using FreeBSD. Open Solaris would be the obvious choice, but with solaris's lack of support, FreeBSD is second to none in ZFS support...

</comment><comment><USER31> I have a similar capacity solution running at home, though I have far more (fewer disks per) vdevs. I've been running it for a few years now, so some things aren't as optimized as they could be, but it still works well for my uses...

      # zpool list
      NAME      SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
      lanayru  73.3T  53.7T  19.6T    73%  1.00x  ONLINE  -

      # zpool status
        pool: lanayru
       state: ONLINE
      status: The pool is formatted using a legacy on-disk format.  The pool can
              still be used, but some features are unavailable.
      action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
              pool will no longer be accessible on software that does not support
              feature flags.
        scan: scrub repaired 1.58M in 201h47m with 0 errors on Fri Feb 21 13:37:24 2014
      config:

              NAME                                                  STATE     READ WRITE CKSUM
              lanayru                                               ONLINE       0     0     0
                raidz2-0                                            ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                raidz2-1                                            ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                  ata-SAMSUNG_HD204UI_______________-part1          ONLINE       0     0     0
                raidz2-2                                            ONLINE       0     0     0
                  ata-ST2000DL004_HD204UI_______________-part1      ONLINE       0     0     0
                  ata-ST2000DL004_HD204UI_______________-part1      ONLINE       0     0     0
                  ata-ST2000DL004_HD204UI_______________-part1      ONLINE       0     0     0
                  ata-ST2000DL004_HD204UI_______________-part1      ONLINE       0     0     0
                  ata-ST2000DL004_HD204UI_______________-part1      ONLINE       0     0     0
                raidz2-3                                            ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68AX9N0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68AX9N0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68AX9N0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68AX9N0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68AX9N0_WD-____________-part1    ONLINE       0     0     0
                raidz2-5                                            ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                raidz2-6                                            ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
                  ata-WDC_WD30EFRX-68EUZN0_WD-____________-part1    ONLINE       0     0     0
              logs
                mirror-4                                            ONLINE       0     0     0
                  ata-KINGSTON_SV300S37A60G_________________-part1  ONLINE       0     0     0
                  ata-KINGSTON_SV300S37A60G_________________-part1  ONLINE       0     0     0
              cache
                ata-KINGSTON_SV300S37A60G_________________-part2    ONLINE       0     0     0
                ata-KINGSTON_SV300S37A60G_________________-part2    ONLINE       0     0     0

      errors: No known data errors

If anyone has any questions, I'd be happy to try to answer.

<comment><USER32> Hey mate, I've got a few questions if you don't mind:

How much does it cost in terms of power to have something of that size running 24/7 (I'm assuming it is?)?

Now you've set it up, does it require much of your time to manage monthly?

What sorts of things are you using the space for?

Lastly, how much did it cost in parts/time to get it all up and running, does it work out cheaper rolling your own or would you have been cheaper using a cloud solution instead, or does that just not meet your use-case/needs?

<comment><USER31> It is running 24/7; I don't feel comfortable powering it down regularly; that is something I would worry about from the OP's setup, I wouldn't want to subject all those mechanical drives to so many power cycles over time. I don't have figures for only that machine, but my entire rack, which includes a router machine, two ISP modems, the ZFS-running machine and two SAS expanders, averages around 330 watts.

After setting it up, I wouldn't say that it requires any time to manage. Getting it all set up just right, with SMART alerts and capacity warnings and backups and snapshots, all of which I roll myself with various shell scripts, took a long time. Besides that initial investment, the only "management" I have to do is respond to any SMART alerts, add more vdevs as the pool fills up, and manage my files as I would on any other filesystem.

I use the space for just about everything. Lots of backups. I use the local storage on all of my workstations as sort of "scratch" space and anything that matters in the long term is stored on the server. The highest density stuff is, of course, media: I have high definition video and photographs from my digital SLRs, I have tons of DV video obtained as part of an analog video digitization workflow, media rips, and downloaded videos. (I even have a couple of terabytes used up by a youtube-dl script that downloads all of my channel subscriptions for offline viewing, and that's something I doubt anyone would do unless they had so many terabytes free.) I keep copies of large datasets that seem important (old Usenet archives, IMDB, full Wikipedia dumps). I keep copies of old software to go with my old hardware collection. I have almost every file I've ever created on any computer in my entire life, with the exception of a handful of Zip and floppy disks that rotted away before I could preserve them, but that is only a few hundred gigabytes. I scan every paper document that I can (my largest format scanner is 12x18 inches, so anything larger than that is waiting for new hardware to arrive someday), so almost all of my mail and legal documents are on there too.

(I had a dream the other night that someone got access to this machine and deleted everything. Worst. Nightmare. Ever.)

A cloud solution would not have met my use case, since one of the primary needs I have is to be self-sufficient in terms of storing my own data, and I also want immediate local access to a lot of the things on there. I do use various cloud solutions, but only for backup, never as primary storage.

Rolling it myself was definitely cheaper than any out-of-the-box hardware solution I've seen. The computer itself is a Supermicro board with some Xeon middle-of-the-range chip and a ton of RAM, and an LSI SAS card. Connected to the SAS card are two 24-bay SAS expander chassis, which contain the drives, which are all SATA.

I'd say that building something like this would cost you maybe about 4000USD, not counting the cost of the drives. The drives were all between $90 and $120 when I bought them, but of course capacity eventually started going up for the same price over time, so let's say another 3500USD for the drives.

<comment><USER33> Would you recommend building something like this for a much smaller system? 10TiB or so maybe, I do not need that much, or do you think buying a NAS of some kind would be better?

I kind of want to set something like this up while spending the least amount of money. I am comfortable enough with Debian/Linux to do most things, but I have never managed anything like this. In the end I want to end up with somewhere relatively safe to store data pretty much in the same way you are, I just do not need 70TiB, and I have no experience with ZFS/hardware stuff/storage.

<comment><USER31> By "something like this", do you mean ZFS? I am a HUGE fan of ZFS, and I do think that it's worth using in any situation where data integrity is a high priority.

As far as ZFS on Linux, it still has its wrinkles. I use it because, like you, I'm comfortable with Debian, and I didn't want to maintain a foreign system just for my data storage, and I still wanted to use the machine for other things too. (I actually started with zfs-fuse, before ZFS on Linux was an option.)

So, I don't know. If you just want a box to store stuff on, you might want to just look into FreeNAS, which is a FreeBSD distribution that makes it very easy to set up a storage appliance based on ZFS. FreeBSD's ZFS implementation is generally considered production-ready, so you avoid some ZFS on Linux wrinkles, too.

So, I'd recommend checking out the FreeNAS website, and maybe also [http://www.reddit.com/r/datahoarder/](http://www.reddit.com/r/datahoarder/) for ideas/other opinions. I do a lot of things in weird idiosyncratic ways, so I'm not sure I'd recommend anyone do it exactly how I have. :)

<comment><USER8> If you're comfortable with Debian then you shouldn't have too many issues with FreeBSD as there is a lot of transferable knowledge between the two (FreeBSD even supports a lot of GNU flags which most other UNIXes don't).

Plus FreeBSD has a lot of good documentation (and the forums have proven a good resource in the past too) - so you're never going it alone (and obviously you have the usual mailing groups and IRC channels on Freenode).

While I do run quite a few Debian (amongst other Linux) I honestly find my FreeBSD server to be the most enjoyable / least painful platform to administrate. Obviously that's just personal preference, but I would definitely recommend trying FreeBSD to anyone considering ZFS.

<comment><USER31> As far as I'm concerned, the most identifiable characteristic of Debian is the packaging system, dpkg/apt. I've used FreeBSD occasionally, and that's what I always end up missing about Debian. I did consider going with Nexenta or Debian GNU/kFreeBSD, but whatever, ZoL works well enough. :)

<comment><USER8> FreeBSD 10 has switched to a new package manager, so it might be worth giving it another look next time you're bored and fancy trying something new.

I can understand your preference though. I'm not a fan of apt much personally, but _pacman_ is one of the reasons I've stuck with ArchLinux over the years - despite it's faults :)

<comment><USER31> I'll keep that in mind; I do sometimes find myself with some time to play with things. :)

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER34> Can somebody give me some recommendations on how to do this with encryption? I am fine sshing into my server and putting in a password after reboot.

<comment><USER35> [http://serverfault.com/a/588056](http://serverfault.com/a/588056)

</comment></comment><comment><USER36> For people interested in a ZFS build, there is zfsbuild.com

</comment><comment><USER37> "The purpose of this machine is to store backups and media, primarily _video_ "

o_O

<comment><USER4> Took a 10 day vacation with a single DSLR shooting RAW... 60GB of photos/videos. I've not gone through and deleted anything yet... but I don't really have to either.

<comment><USER38> That's 1000 such vacations!

<comment><USER39> Better get busy.</comment></comment></comment></comment>


Ask HN: Form C Corp before or after product is fully built? - <AUTHOR> <comment><USER2> Neither. Form C Corp after investor writes 6+ figure check.

<comment><USER3> This. You should essentially be "triggered" into creating a C Corp. If you're raising money from reputable folks, they will direct you to the right people to do it.

<comment><AUTHOR> So are you saying just wing it and build the product up until the point where I have folks that want to invest? Are there any drawbacks to just incorporating prior to that?</comment></comment></comment>


A few words on Doug Engelbart - <AUTHOR> http://worrydream.com/Engelbart/ <comment><USER2> I found that the quip about how the document that they are sharing while having a conversation is not screensharing as we commonly understand it in modern parlance to be pretty illuminating. That modern screensharing is in several ways worse than a demo that came out decades ago is frankly amazing.

I've tried to recapture a bit of that early magic with [https://coderpad.io/](https://coderpad.io/) (multiple cursors, realtime editing/execution of code), but I can only imagine how jaw dropping that demo must have been when it came out.

<comment><USER3> It really is amazing.

I feel like there is a similar mismatch and almost backwards progress when you look at the concepts and tools behind smalltalk and the concepts most working programmers today reason with and are familiar with.

<comment><USER4> I think the huge focus on web development set CS back at least two decades in certain areas. The industry has been so intent on reinventing applications through the browser, it's lost ground in what can happen in areas like Engelbert's or Smalltalk or real hypertext.

<comment><USER5> If you think of technology progress as a global simulated annealing problem, it is ok to step back a bit to get out of the local maxima. Web Dev may have taken us back 20-30 years in certain areas but once caught up, it can take us much further than the previous methods could have ever expected us to. The previous methods in this case being, compiled software that require specific devices/OSes and require a very high level of competency in all aspects of development.

<comment><USER6> Web solutions also require specific devices and web browsers, really; I can't think of a single jaw-dropping JS demo around here that didn't meet with several complaints related to it simply not working under certain browsers (and a lot more complaints about performance). Aye, if you put enough time and hacks into it, you get it to run on pretty much any browser. Until the next update, that is.

> require a very high level of competency in all aspects of development

Crude tools, development methods and principles are a surrogate for simplicity, not an actual manifestation of it. Barring the fact that you sacrifice performance, code maintainability and, to a high extent, security, I also think the question of productivity remains open. I find it stunning that young web developers are ecstatic about how their tools allow you to get from idea to result _so quickly_ , when they are pretty much on-par with where Motif was in the mid-'90s. Not to mention the truckload of CSS hacks you need to make something that looks like a button (but without native looks) out of an anchor-that-really-shouldn't-be-a-button-but-it's-the-closest-thing-html- has-got. I don't think that can take us much further than compiled software could have taken us -- and slowly, but surely (with stuff like asm.js), it looks like a lot of people are rediscovering that.

<comment><USER7> [https://developer.mozilla.org/en- US/docs/Web/HTML/Element/bu...](https://developer.mozilla.org/en- US/docs/Web/HTML/Element/button)

<comment><USER6> I think we both know that's not what I was referring to. It's stuff like this one (PhoneGap, but really, it's like this almost everywhere): Back. Unless I am mistaken and you can build the whole UI out of form elements (perhaps you can, but I'm not sure I've actually seen that done anywhere), that's simply not sufficient.

<comment><USER7> I took your comment to mean that no button element exists, but I suppose that was not your intention. But I did realize you were referring to links being abused as buttons.

I think the issue is simply that buttons tend to have more default styling you need to override, which is more likely to vary across platforms since the default style is generally chosen to mimic the system's native buttons. Whereas default styling for links is quite simple across all platforms. Besides that, there is no particular problem with using real button elements instead of link elements as far as I'm aware, just a matter of habit and convenience.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I found this article by Bret enlightening. The expectation, though, that Engelbart's legacy could be communicated clearly by the New York Times in the obituary is too high.

The headline reads like this:

> Douglas C. Engelbart, Inventor of the Computer Mouse, Dies at 88

Imagine, then, if the headline read:

> Douglas C. Engelbart, Augmenter of Human Intellect, Dies at 88

The former is much clearer and understandable, especially taking into consideration the audience of the New York Times. Bret's article has the luxury of expounding and explaining to an audience that is sympathetic to his values; I myself was ignorant of Englebart's contributions as well as his personal ideology when it came to this life mission, and so greatly appreciate Bret's writing.

But, while the NYT article might seem untrue or ignorant to those who knew Engelbart and understood him, for most people, the appreciation for Engelbart comes out of the more 'mundane' things that were the side effects of his vision: the mouse, hypertext, etc. Most can appreciate the significance of these inventions, though the value of Engelbart's core works might escape them.

<comment><USER9> Douglas C. Engelbart, Computer Visionary, Dies at 88.

</comment></comment><comment><USER10> "Ignore today; just think about it in terms of his goals."

reminds me of

"Do not follow in the footsteps of the masters; seek what they sought."

</comment><comment><USER11> This book, "What the Dormouse Said: How the Sixties Counter culture Shaped the Personal Computer Industry" is by the same guy who wrote the NYT obit with "the most facile interpretation of Engelbart" headline. It goes into far more depth about Engelbart's goals of augmenting human intellect.

</comment><comment><USER12> I found this article to be very thought provoking with regards to Engelbart's work, but I can't help but read Bret's own worries in it. As if he himself fears being presented/remembered purely based on his inventions (Bret Victor, the inventor of the iPad interface) and not his goal (intends to invent the medium and representations in which the scientists, engineers, and artists of the next century will understand and create systems.)

As a side note: why can't we, the general public, appreciate people like Engelbart with the same fierceness before they are gone? I'm sure they'd like to know their life was incredibly meaningful.

</comment><comment><USER13> > Engelbart __hated __our present-day systems.

I'm not trying to say "[citation needed]" here, but I would be interested in seeing a source for this. Does anyone know of one?

<comment><USER14> The citation, in this case, is Doug Engelbart. Bret Victor is reporting what Doug Engelbart told him about present day systems.

<comment><USER15> [citation needed]

<comment><USER14> [http://worrydream.com/Engelbart/](http://worrydream.com/Engelbart/)

<comment><USER16> I take it you're being sarcastic, but the article doesn't contain any such information. Or are we missing something?

</comment></comment></comment></comment></comment><comment><USER17> This is one of those times when I can't be emphatic enough without crudity.

That was a fucking excellent eulogy.

</comment><comment><USER3> every time i read something brett writes, i learn something new and feel like my world view has expanded a little bit.

</comment><comment><USER18> This is a good article but it's important to understand that vision doesn't necessarily trump implementation. Steve Jobs also had a vision of augmenting human intellect (one at a time rather than collective), but he's remembered for the things he built in pursuit of the goal, not the goal. Ted Nelson's vision for hypertext is also more transcendent than what we got from the world-wide web, but on examination it turns out that hos vision is flawed and unrealizable (in practice there is never a canonical reference for any text) while what we've actually built actually works.

</comment><comment><USER16> The closest present-day analog to Engelbart's collaborative editing demo (edit: I mean specifically the document-editing portion, not the video etc.) is not screen-sharing, but modern collaborative editors (Google Docs, Etherpad), which do provide multiple cursors. Thus the screen-sharing analogy is a straw man, and the author hasn't really made his case against "drawing correspondences to our present-day systems" from Engelbart's visionary work. He may still be right, but I'd like to know why.

<comment><USER19> In Engelbart's original demo, the users can see each other, talk, collaboratively work on a document, and do together pretty much anything that the computer offers. The full OS, including the GUI, is intended to be multi- user.

Google Docs et al. only offer one aspect of that; all they aspire to do is solve the problem of "collaborative editing of a document". But say we need to work on something that involves listening to audio together or watch a video or use a third party program, we're SOL with Google Docs.

What Engelbart's vision aspired to do was to allow people to work together through computers, no matter what the work was. Document editing is a microscopic facet of that.

<comment><USER16> _Google Docs et al. only offer one aspect of that; all they aspire to do is solve the problem of "collaborative editing of a document"_

Yes, but it's just the aspect the OP criticizes screen-sharing (as a proxy for present-day technology) for missing. I take your point that these things could be better integrated, though.

</comment></comment></comment><comment><USER20> How did I not hear about this guy in detail before? His high-level goals sound exactly like mine. Where are the people working on this now?

<comment><USER21> Look for IA as an reversal of AI, "Intelligence Augmentation" or "Intelligence Amplification".

</comment></comment><comment><USER22> I remember watching his NerdTV interview several years ago and being inspired by how principled and focused his work was. He wanted to apply his capacity toward solving interesting problems, and he did something about it.

Original PBS page with now broken media links: [http://www.pbs.org/cringely/nerdtv/player/?ext=mp4&show=011](http://www.pbs.org/cringely/nerdtv/player/?ext=mp4&show=011)

The Archive has it: [http://archive.org/details/nerdtv011](http://archive.org/details/nerdtv011)

</comment><comment><USER23> Anyone knows who was Doug Engelbart's PhD advisor at Berkeley? We have his profile at phdtree:

[http://phdtree.org/scholar/engelbart-douglas- carl/](http://phdtree.org/scholar/engelbart-douglas-carl/)

but there is no where to find out info about his PhD advisor.

<comment><USER24> John Woodyard

[http://www-sul.stanford.edu/depts/hasrg/histsci/ssvoral/enge...](http://www- sul.stanford.edu/depts/hasrg/histsci/ssvoral/engelbart/main1-ntb.html)

More:

[http://www.ee.washington.edu/people/alumni/profiles/woodyard...](http://www.ee.washington.edu/people/alumni/profiles/woodyard_john_r.html)

[http://en.wikipedia.org/wiki/John_Robert_Woodyard](http://en.wikipedia.org/wiki/John_Robert_Woodyard)

Its also listed on his wiki page directly.

<comment><USER23> Thanks a lot, updated!

Another question: I figured out John_Robert_Woodyard's PhD advisor at Stanford was William Webster Hansen, [http://phdtree.org/scholar/woodyard-john- robert/](http://phdtree.org/scholar/woodyard-john-robert/)

but then I couldn't find out info about William W Hansen's PhD advisor, any help?

</comment></comment></comment><comment><USER25> This is a bit needlessly aggressive.

<comment><USER24> It's called passion.

</comment></comment><comment><USER26> As I began to read his thesis I was reminded _powerfully_ of Conor McBride's Epigram system [1]. It's a very different kind of system, but the parity between a user's intent and the computer's ability to interact seamlessly with that intent is a powerful feature of Epigram.

[1] I can't seem to find the source anymore, but there are many papers like cs.ru.nl/~freek/courses/tt-2010/tvftl/epigram-notes.pdf

</comment><comment><USER21> "'Douglas C. Engelbart, Inventor of the Computer Mouse, Dies at 88' This is as if you found the person who invented writing, and credited them for inventing the pencil."

That's what obituary headlines do: connect the dead person's achievements in a concrete way to readers' lives.

</comment><comment><USER27> Just curious if anyone else here has tried Screenhero? I've found it to be quite excellent in a handful of situations where I needed to use it.

It's screen sharing where each person has their own cursor. Works surprisingly well for a small company product.

<comment><USER28> No 2 text cursors? Meh. j/k :)

</comment></comment><comment><USER29> Real-time collaborative text editing based on operational transform, like Google Wave and Etherpad, is probably a better analogy to Engelbart's system than screen sharing.

</comment><comment><USER30> Beautiful eulogy. Compels me to read more about the person.</comment>


Twitter Faced SEC Questions on Changing User-Engagement Metrics - <AUTHOR> http://www.bloomberg.com/news/articles/2015-08-24/twitter-faced-sec-questions-on-changing-user-engagement-metrics <comment><USER2> It's been interesting to watch this story unfold. Twitter is a broadcast system and therefore has many (useful [and not useful]) bots. These aren't users, but not all of them are spam. It's a simple counter-example to show that simple user (and engagement) counts will almost certainly be misleading. Twitter is in a big-time catch 22 with reporting here. It'll take skill to report a qualitative aspect like engagement with a quantitative metric (like they're expected to).

</comment><comment><USER3> I've advertised on Twitter before and their engagement metrics are curious and a bit dishonest.

They measure any click on a tweet's body (not a hyperlink) as an engagement; where as an advertiser I define engagement as a favorite, retweet or reply.

<comment><USER4> Does advertising on Twitter work? I am curious if engagement matters too much provided Twitter is delivering the results? Of course if it is not cost effective then they have a problem.

<comment><USER5> It's not just "does it work?" but "what kind of customers are you getting?". Twitter seems more productive to me than any part of facebook, which seems to desire to ravenously consume one's life. It's only a slightly less consuming task than some online games (MMORPG).

You're purchasing time from people that are busy, and productive, and have a life that on the whole it not completely virtualized. Compare that with Facebook and MMPORGs of sorts, and there are some highly successful virtual people who live real lives in mobile homes and shitty apartment complexes. Not to say their overall quality of life is bad, but maybe their balance is off kilter some.

</comment></comment></comment><comment><USER6> Funny how bearish news gets decimated when markets are in turmoil.

</comment><comment><USER7> Twitter can avoid unwanted SEC attention by taking massive risks that will potentially blow up the global economy.</comment>


Notes of Kafka Core - <AUTHOR> https://rammy.in/posts/technical/kafkanotes1/ <comment><AUTHOR> quick brief entry for the kafka and quick read to understand kafka</comment>


Ask HN: Selling a Website & IP - <AUTHOR>

Holy cow.
I've received an offer for a developed website + a few related domains.
Are there any open source or public domain boiler plate contracts out there for a sale of this type?
The dollar value of this deal makes me want to cross all of my t's. <comment><USER3> Hire a lawyer. If the dollar value is that good that it makes you want to do it right. Then its probably worth the cost to do it right and get good advice.

<comment><AUTHOR> Any lawyer worth his IP-transfer-contracting-salt is outpricing my budget on this sale.

<comment><USER3> In that case, maybe try one of the online startup docs repositories. Maybe you can find something in one them that would work.

</comment></comment></comment><comment><USER4> Not too sure about this, but moniker.com might be worth a quick look - <USER2> just might have something you could use.

</comment><comment><USER5> try flippa.com

<comment><AUTHOR> I already have a tendered offer on the table. Does flippa.com provide contracts outside of their ecosystem?

Also...not too keen on paying commission to a third party for a deal that has already been negotiated.</comment></comment>


Fed Should Resist Market Pressure to Do More - <AUTHOR> https://www.bloomberg.com/opinion/articles/2020-05-31/fed-should-resist-market-pressure-to-do-more <comment><USER2> All over the world people on various forms of lockdown are suffering various levels of income loss. Civil servants still get their entire pay packer. All the others are in a mixed bag - some are fully laid off and some may have unemployment benefits, some have cash that has been trickling down from federal/state coffers as they sit at home. Look into the history of the great depression - the lack of purchasing power in the great mass of the population created a self perpetuating depression. It was only the "New Deal" in the USA that ended it. Google that and see, then WW2 came along = stuff made and paid for to be blown up to wage the war. Landlords and mortgage holders are salivating as months of rents/payments accumulate to the point where the coming depression enables a huge era of foreclosure on homes/apartments/shops/cars etc - all of which will drop into their hands a a year or so from now, they will be able to sell rent etc. So the Feds/states/cities must act to print the cash to give to each of the people at the very bottom (with income taxes witheld) so they are able to keep their homes/cars/apartments etc. If not the sustained recession will cause the treasury to lose even more due to the recession's decline. Look at 1929-1939 - do we want 10 years of that?

an extract from the web:- Did you know? Unemployment levels in some cities reached staggering levels during the Great Depression: By 1933, Toledo, Ohio's had reached 80 percent, and nearly 90 percent of Lowell, Massachusetts, was unemployed.

The next day, Roosevelt declared a four-day bank holiday to stop people from withdrawing their money from shaky banks. On March 9, Congress passed Roosevelt's Emergency Banking Act, which reorganized the banks and closed the ones that were insolvent.

In his first "fireside chat" three days later, the president urged Americans to put their savings back in the banks, and by the end of the month almost three quarters of them had reopened.

The First Hundred Days

Roosevelt's quest to end the Great Depression was just beginning, and would ramp up in what came to be known as "The First 100 Days." Roosevelt kicked things off by asking Congress to take the first step toward ending Prohibition – one of the more divisive issues of the 1920s – by making it legal once again for Americans to buy beer. (At the end of the year, Congress ratified the 21st Amendment and ended Prohibition for good.)

In May, he signed the Tennessee Valley Authority Act into law, creating the TVA and enabling the federal government to build dams along the Tennessee River that controlled flooding and generated inexpensive hydroelectric power for the people in the region.

<comment><USER3> Thank you for this thoughtful reply. I agree with you. I would also point out the history of depressions being a catalyst for tyranny, especially under right leaning fascist leaders, because the population is scared and hurting and look for stability through my a powerful controlling leader and are further persuaded by visions of a return to the good times before the depression and a future of a "restored" stronger country. We saw that in parts of the world after 2008 and this climate will likely strengthen the trend.

<comment><USER2> Thanks for the support. I fear the republicans will choose to enrich business interests at the expense of the common man = the top 5% will own more of the USA than before. We sorely need some form of redress - capital taxes? The democratic candidate is a total loser. Good time for a hird party = but it will not get enough traction with the electoral college system - that must have been thought up by the ghost of King George...;)

</comment></comment></comment><comment><USER3> Why. What's the risk? An over heated economy? Inflation and wage growth? Sounds what the world actually needs after years in a liquidity trap.

Actually the fed should ignore political pressure to do less.</comment>


A tiny language called Z - <AUTHOR> http://chrisdone.com/z/ <comment><USER2> It seems like everyone else is giving you flak so let me offer some words of encouragement:

This looks great! This really makes functional programming more readable. It forces the use of whitespace which is my favorite part about python. It lets me easily sort the arguments to each function.

Don't mind the other people telling you about smart editors, that's just a band-aid over the problem of parens.

You've come up with a better way of writing functional code, just like CoffeScript is a better way of writing JavaScript.

Now for some constructive criticism:

You might want to add a note that your implementation on Github requires Haskell.

You only have syntax highlighting for Emacs. Maybe give vim some love next?

If you develop it further you will want better documentation or people will walk away just because they can't get it started.

<comment><USER3> I also love it.

I had been designing a language in my head like this. I called it "spacy" as in space rich and a bit off the wall. (As others have noted there are similar specs/languages around)

I had been struggling with what to do about continuation lines and multiple arguments. I think you solved the second very nicely.

As for chosen indentation; let the programmer decide. The first use of space (number of) or tabs determines it for the rest of the file. Any other indentation is a syntax error.

</comment></comment><comment><USER4> I like fiddling around with weird languages... but...

 _You find it. Then you start walking forward, looking for a closing parenthesis._

This is the worst way to work with Lisp code and is unfortunately the most naive way. It happens to be the way most people immediately jump to when editing Lisp code and hence how we end up with the pernicious "parenthesis" meme.

The way most Lisp programmers handle parenthesis is by using a smart editor. The "parens" structure the code very closely to the AST of the actual program. Computers love trees and editors can therefore readily manipulate them with little effort. You rarely edit the text of a Lisp program this way and instead edit it's tree.

 _Note_ : See the 'paredit' package in emacs for an example.

<comment><USER5> My impression sometimes is that RPN (like in Forth) could be used as well and do with parenthesis altogether

But of course, Forth has a stack, Lisp does not need one

To write 5 * (2 + 3)

Lisp: ( * 5 (+ 3 2))

Forth (ignore the dot): 2 3 + 5 * .

<comment><USER6> Forth actually has _two_ stacks. Lisp has garbage-collected storage and environment structures, and Forth does not need them. :)

Logo is a Lisp-like language which uses prefix notation but does not require parentheses surrounding every procedure invocation. Your example could be written as:

      product 5 sum 3 2

Parens are allowed for adding clarity and are only required for a vararg-style invocation. The main downside of this approach is that it is necessary to know the arity of every procedure in an expression to unambiguously parse it- Logo implementations typically either defer some safety checks to runtime or compile in two passes. Forth doesn't have this problem because it doesn't parse expressions per se- it compiles and interprets as it goes along.

</comment></comment></comment><comment><USER7> Right-associative function application is an excellent choice. Perl has it, and I often wish for it in Haskell when writing long compositions:

        f . g . h $ i x

However, this kind of indentation rapidly gets out of hand, and requires some editor awareness to be remotely usable:

        defun map f xs
              if unit? xs
                 unit
                 cons f car xs
                      map f
                          cdr xs

Moreover, there doesn't seem to be any virtue in preventing multiple indentation styles. Readability often depends not on consistency, but on redundancy, and using the right style for the situation. I would much rather read and write this:

        defun
          map f xs
          if
            unit? xs
            unit
            cons
              f car xs
              map f, cdr xs

That is, indentation would _only_ introduce a block of arguments, and arguments on the same line could _only_ span that line. If you wanted multiple arguments on one line, you could use a separator—here, a comma.

<comment><USER8> To be perfectly honest your re-formatting is actually worse, to my eye. Which is surprising because Z is already ugly! You've made it longer, moved the condition away from the if (that is really mental!), and added a rather spurious looking comma (which has problems that I describe below). Actually, Z looks a lot like Common Lisp or Elisp, fully normalized. Let's say "fully normalized" means: indent everything that could be, aligned with the operand. E.g. here's Z:

      defun map f xs
            if unit? xs
               unit
               cons f car xs
                    map f
                        cdr xs

Here's Lisp:

      (defun map (f xs)
             (if (unit? xs)
                 unit
                 (cons (f (car xs))
                       (map f
                            (cdr xs)))))

I don't think any Lisper would really complain about that code if she came across it. That is actually how I would write it when coding in Elisp or Common Lisp, with two exceptions: Perhaps for the that "defun" and other def-* forms are often indented only two spaces after the parent's offset. Now, I thought about allowing that, and that's a good point, stylistically. But it seems not to change the overall z-expressions point much. To argue about needing editor support would also be to argue against Lisp or Python, in which case there are other people interested in arguing about that.

The Lisper might also complain, if she were being stringent, about the "map f …" being line-separated. And for that I'll say, ok. Too bad. Personally, I find myself line-separating arguments in Haskell and Lisp more often than not, so it's not a use-case I would feel the need to optimize.

Regardless, you can pretty much ignore all I said above, because it's just an aesthetics thing, kind of irrelevant to the core concept. The whole point of the indentation and right-associative application is for the Markdown-inspired macros: _everything after the name is up for grabs for a macro (or special operator)_. The idea of adding a comma pretty much negates that and the whole point of z-expressions. If I could have commas, if I were to make that concession, I'd just have parentheses, and use s-expressions and reader macros.

<comment><USER7> I see what you mean. A separator doesn't work with the concept at all. Apart from that, my reformatting still has two distinct advantages: it does not require a fixed-width typeface, and indentation is not affected by the lengths of identifiers. It's silly to format code in such a way that you need to re- indent just because you renamed a function.

<comment><USER8> That's true. I don't like _reading_ it, but I would prefer _writing_ it. It's pretty much why in Haskell I write:

      foo $ bar $ do
        hello
        world

rather than

       foo $ bar $ do hello
                      world

You could easily have:

      foo bar
        mu

be equivalent to

      foo bar
          mu

but if you had

      foo bar bob
            zot
        mu

And you rename foo, you're back to square one again (zot is no longer syntactical). Unless you just have a rule like:

      foobarmuzot bar bob
            zot
          mu

means

      foobarmuzot bar bob
                      zot
                   mu

But it seems comparatively difficult to read, and deceptive.

</comment></comment></comment></comment><comment><USER9> There is already a language called Z, it has been around for a while: <http://en.wikipedia.org/wiki/Z_notation>

</comment><comment><USER10> It is very beautiful because it seems to match quite closesly the diagrammatic interpretation of a symmetric monoidal closed category. For example because of the functoriality of the the monoidal tensor we have

(g ∘ h) ⊗ (i ∘ j) = (g ∘ i) ⊗ (h ∘ j)

so the two expressions should be indistinguishable. Indeed, in this Z syntax the expression

      k g h

        i j

could be parsed either as k ( g(h) ⊗ i(j) ) or as k( (g⊗i)(h⊗j) ) but this ambiguity does not matter since they are equivalent. I wonder if

      k g

        h

is a partial appilcation that would give you a function of the right type, i.e. the type of h ⊗ j.

Anyway, beautiful stuff.

PS. Not sure how to typeset this properly in this web site. PS. Thanks to commenter.

<comment><USER11> Funnily enough, code blocks are created by indenting with spaces. But since it is too late for you to edit your comment, here are your examples:

      k g h
          i k

      k g
          h

<comment><USER8>
      k g h
          i j

Presumably?

</comment></comment></comment><comment><USER12> So this is, if i'm not missing anything major, lisp with indentations instead of parentheses. Funny, I spent my new year's eve writing a script to turn any Photoshop layout into an atlas with JSON metadata sufficient to rebuild the layout in code.

No regrets :-)

</comment><comment><USER13> This is why I come to Hacker News. A cool technical idea at the top, then in the comments I discover a bunch of other cool stuff. If nothing else, I'll probably use this syntax in my hand-written notes.

</comment><comment><USER14> Would be nice to see if this could be converted to javascript to run in the browser, just like markdown. I don't know Haskell myself, but there are javascript ports of the Parsec parser library here: <https://github.com/weaver/ReParse> <http://code.google.com/p/jshaskell/>

As well as several Haskell to JavaScript compilers here (Fay, ghcjs, & jshaskell seem to be more popular): [https://github.com/jashkenas/coffee- script/wiki/List-of-lang...](https://github.com/jashkenas/coffee- script/wiki/List-of-languages-that-compile-to-JS)

<comment><USER15> FYI, chrisdone is the author of Fay.

</comment></comment><comment><USER16> See also the SRFI for I-Expressions, for Scheme: <http://srfi.schemers.org/srfi-49/srfi-49.html>

</comment><comment><USER17> That looks a lot like <http://srfi.schemers.org/srfi-49/srfi-49.html>, honestly.

</comment><comment><USER18> In 2006 someone did something very similar (although generalizing from Lisp and not starting from scratch or looking at niche applications); it was discussed on HN in 2009.

<http://news.ycombinator.com/item?id=823524>

</comment><comment><USER19> I like the syntax; it feels like one I've been groping towards myself. As another comment said, it reminds me of TCL.

I'm not convinced on the string-based macros. They enable really cool stuff, like scala's XML literals, but it seems like they'd make syntax-aware editors impossible.

I'd suggest moving away from the lispy names for things (cons, cond, defun, cdr). This is (presumably) a language that appeals to people who like the elegance of lisp's reflexivity but can't stand the syntax; names from a more familiar language would appeal to them, while anyone who's firmly attached to lisp names is probably also attached to brackets.

</comment><comment><USER20> Oooh, so it's a notation, called Z? Sounds familiar...

<comment><USER21> No, they are Z-expressions.

No-one has a problem with the S programming language co-existing with S-expressions.

<comment><USER22> Just to be sure: you're familiar with Z notation? Given that the Z from this article is special quite specifically because of its notation, choosing the name Z when there's already a "Z notation" feels a bit, ehm, odd.

Oppose this to S-expressions, which is a term not used a lot when not programming Lisp. You wrote that program in Lisp, not "with S-expressions".

<comment><USER23> Conjecture: For all characters _c_ taken from the set of 'A' through 'Z' (inclusive), there is a notation _N_c_ such that { <http://www.google.com/search?q=c+notation> } is nonempty.

</comment></comment></comment></comment><comment><USER24> Reminds me a little bit of Tcl.

</comment><comment><USER25> Macros seriously take in a string and output another string? That has to be formatted correctly with newlines and indentation? That is _bats_.

<comment><USER8> How would you implement the string macro (":") or the regex ("~") macro without that?

</comment></comment><comment><USER26> Pretty neat, I've long wanted a Lisp that used indentation rather than unreadable parentheses, but didn't get far in trying to set it up. Good work! Next step is to make it transcompile down to a standard Lisp so it can make use of all the other existing libraries made for Lisp.

</comment><comment><USER27> Is there an advantage to having the macros operate on strings, instead of the AST directly like lisp/scheme?

<comment><USER28> It took me a while to understand, but it's the whole point of the article: macros that can also do their own tokenizing and parsing. That enables syntax you can't have with lisp macros (including unbalanced parens!) It's still unclear if it enables anything _useful_ , but still a cool idea.

(I have an interest in this space: <http://github.com/akkartik/wart#readme>)

<comment><USER29> Common Lisp does have reader macros, which change how code is parsed into ASTs

</comment></comment></comment><comment><USER28> The comments are the coolest part of this idea. But there's nothing I care more about syntax-highlighting, so in practice I'd just say:

      -- lorem ipsum
      -- quidquid dior
      defun ...

</comment><comment><USER30> I kinda like the idea but...

As as been pointed out, it's pretty much a Lisp where parenthesis have been replaced by whitespaces.

The problem is: you need syntactically aware editors to work with such a thing. You probably need a _fully_ structural text editor (a semi-structural editor, not unlike Emacs' paredit, may not be enough)...

Because how do you cut&paste between different levels of indentation?

Worse: how do you cut & paste inside an incomplete AST ?

Sometimes it's the programmer who _knows_ what he wants to do, and here he better be helped with structural text editing or it looks like things could quickly go apeshit.

I understand parenthesitis and Lisp-hatred is high on HN but you need to know that a lot of people who do not hate parentheses are using things such a semi- structural text editing (which, amongst other, allows to automagically match parentheses).

So it's not by saying: "zomg, parenthesis, I hate these. Look here ma', no parentheses... But you'll have one heck of a time aligning code blocks" that you're going to convince Lisp users...

<comment><USER8> It's not a hatred of parentheses. I use Elisp and Common Lisp all the time, I'm _fine_ with parentheses.

If you read the article it's specifically about having liberated macros, something you can't have in the presence of parens. You can't write these macros in Lisp:

      -- Hello,World! =)
      print : Hello, World! :-)
      let x = 23/6²*
              z/+84

That's "--" and ":" and "=". Those are macros.

Cutting and pasting is easy:

If I copy

      foo bar mu
          zot bob   <-- these two
              bil   <-- lines

and then I got to here

      bar zot bob
          dod rob
              |    <-- with my cursor here and paste

what do you expect to happen?

Whereas here

      bar zot bob
          dod rob
          |    <-- with my cursor here and paste

what do you expect to happen?

This is trivial to do in Emacs.

</comment></comment><comment><USER31> so it's Python but functional?

<comment><USER32> It takes whitespace to the extreme, too.

I like everything but the "statements span multiple lines" instead of using comma seperated values. I think CSV syntax is clearer and more concise than having newlines act a seperators like that, and it is easier to read.

I like high information density languages with minimum glyphic overhead. That means python is close to the top, but I also consider C++ competitive because it has a lot of expressionism in its verbosity. I also rank YAML high because it has barely any glyphic overhead but still conveys high information in a frame.

It is why I use K&R braces, because I use indentation to deliminate scopes, and having extra semicolons on lines by themselves breaks my information density and drives me nuts, because I like having a consistent per-line flow (I don't mind blank lines though, because I just treat them as null, but one character lines require processing, just not enough to warrant them worth existing in my crazy head).</comment></comment>


I Hate Commander Chris Hadfield - <AUTHOR> http://www.amazon.com/review/R6L2EVLMPPEQS

<comment><USER2> I hope one day we can all stop doing the 'Post title that's the opposite of the popular opinion, but the content actually supports the popular opinion, it just couches it in an unpopular way, so as to gain clicks with a headline that makes it seem like the article was ACTUALLY going to support the unpopular opinion'.

This is, in essence, just another form of slimy clickbait (albeit, one that flies on HN).</comment>


Ask HN: What "things" would you like your children to learn? - <AUTHOR> At 37 with 2 young children, I'm starting to realize there are some facts (and non facts) about the ways the world and nature and people work that I would hope my children come to understand one day.
This led me to want to put together a list of such "things" I would like them to learn, but I want to avoid the natural bias to validate my own sense of self (the instinct to want my kids to be like me thus validating my own worth).
The question is phrased - "would like my children to learn", as opposed to - "want to teach my children", if you understand what I mean?
I realize that by choosing HN for this I'm already biasing the list...
...but I wanted the list to be about facts as in "Life the Universe and Everything", rather than "Psico-Socio-Economic" internet advice like - "Top 10 Things to teach your kids"
...so I think HN is a good fit.
What are some things you wish your children will learn as they grow up? <comment><USER2> How to swim: Growing up in Australia, a country that is surrounded by ocean naturally means a lot of time spent at the beach. Learning at any early age how to swim without any flotation devices, while fully clothed and how to get out of a rip enabled me to have a lot of fun at the beach/pools without causing my parents anxiety.

How ride a bike: I was fortunate enough to have my parents teach me how to ride (without training wheels) at the age of 2. From around the age of 8, I was able to explore the city with friends and organise our own activities without having to rely on parents to drive us to and from each other's houses.

A second (or third) language: I'm not fluent in any languages other than English, but I've seen how many opportunities/experiences can come from being able to communicate with someone with a different culture. Ideally I'd like for my children to learn a second language alongside English (There's a huge benefit to learning a second language at a young age compared to one that is 'translated' from one's mother tongue).

</comment><comment><AUTHOR> Learn to recognize common fallacies and misconceptions. Especially our own tendency to "embark" in them.

Anything in this list is a good place to start: [https://en.wikipedia.org/wiki/List_of_fallacies](https://en.wikipedia.org/wiki/List_of_fallacies)

</comment><comment><AUTHOR> Try to understand the scale of the Observable Universe.

Question what else is there beyond that? Realize that if the Universe is proven to be expanding, what is observable now is less that what was observable in the past and more than will be observable in the future.

</comment><comment><AUTHOR> That there are always multiple points of view on any subject and that you should understand them before you make up your mind about that subject (and then be open to changing your mind about it at any point in time).

</comment><comment><AUTHOR> That pure facts are extremely hard to prove, even through the scientific method. Understand the limits of science. Be a skeptic by default.

</comment><comment><USER3> How to think analytically, how to act compassionately, and how to balance the two.

Though that seems a lot to ask for in the first 18 years.</comment>


Dig This - Construction Equipment Theme Park - <AUTHOR> http://enr.construction.com/products/equipment/2011/0525-HeavyDutyPlaygroundOpensinLasVegas.asp

<comment><USER2> There was a similar concept in Germany somewhere I seem to recall.

Always thought it would be fun to do. :)</comment>


Adderall spam pages on my website? - <AUTHOR>

Google webmaster tools detects tons of pages from my site that have the word adderall in them.
Checking my site's files, these pages definitely do not exist anywhere, and they lead to 404 errors when you link to them. Any idea how to fix this?
I've pasted from the google search for adderall reco jeans, here:
# Smoke Adderall - Buy Cheap Adderall Online - Best Prices!!! IF YOU MISS A DOSE OF THIS Adderall Medication take it as. Possible for my Internet DOSE OF THIS Adderall for Schedule II Substances Schedule II substances ... www.recojeans.com/shop/adderall-52.html - <USER4> #
Adderall Overdose - Buy Cheap Adderall Online - Best Prices!!! Attention Deficit Disorder medication DOSE OF THIS Adderall Medication take ... www.recojeans.com/shop/adderall-65.html - <USER4> #
Adderall Prescription - Buy Cheap Adderall Online - Best Prices!!! Adderall XR webpage entitled DOSE OF THIS Adderall in adolescence was ... www.recojeans.com/shop/adderall-70.html - <USER4> <comment><USER5> Hey Kevin, normally this means you've been hacked by a spammer. Often the spammers do sneaky things like return a spam page if you have a google.com referrer (e.g. if you click on search result on Google). Meanwhile, they might return a 404 if you enter the url directly in the address bar.

The first place I'd check is your .htaccess file--that's a common place where the hackers hide stuff. Also check for new/recently-added files in your directory. Some WordPress hackers will go so far as to inject malicious code straight into the WP database.

One tool you can use is a feature from google.com/webmasters called "Fetch as Googlebot." Once you prove that you own the site, you can tell Google to fetch your page and you'll see exactly what we see. This is a great way to tell if you've really cleaned out the hacks.

Another tool is our automatic url removal tool, also at google.com/webmasters . You can block a url in robots.txt and then use that tool to remove that page from Google's index.

If you've gotten everything, then the pages should start to leave Google's index after a few days. The hackers can be quite smart though, and if they compromise your webhost then it's even more painful to scrape them off your site because the hacked pages can return even if your site is well-patched.

<comment><USER6> Matt, If it's a WordPress Hack which I'm sure it is. Here's the solution: [http://www.pearsonified.com/2010/04/wordpress-pharma- hack.ph...](http://www.pearsonified.com/2010/04/wordpress-pharma-hack.php)

Can we not ask them to remove their site from Google that seems like drastic errancy as it takes months and sometimes longer for reinclusion requests to go through.

Seems like a hasty and unneccesary methodology to the problem.

<comment><USER5> Hacked sites can serve up malware, viruses, trojan horses, etc. so we really do need to remove hacked sites from our index until we're confident that the site is clean.

</comment></comment></comment><comment><USER7> I have the same problem. I looked in the.htaccess file an this is what I found. Is this the redirect they are using? RewriteEngine on RewriteBase / RewriteCond %{HTTP_HOST} (^|www.)example.com RewriteCond %{REQUEST_FILENAME} ![^a-zA-Z0-9](css|js|jpe?g|gif|png|zip|swf|doc|xls|pdf|ico|tar|gz|bmp|rar|mp3|avi|mpeg|flv)(\?|$) RewriteCond %{REMOTE_ADDR} ^66\\.249\\.[6-9][0-9]\\.[0-9]+$ [OR] RewriteCond %{REMOTE_ADDR} ^74\\.125\\.[0-9]+\\.[0-9]+$ [OR] RewriteCond %{REMOTE_ADDR} ^64\\.233\\.1[6-9][0-9]\\.[0-9]+$ [OR] RewriteCond %{REMOTE_ADDR} ^65\\.5[2-5]\\.[0-9]+\\.[0-9]+$ [OR] RewriteCond %{HTTP_USER_AGENT} (google|msnbot) RewriteRule ^(.*)$ Images/Images/amex2.class.php [L] Options +FollowSymLinks

<comment><USER5> This looks quite suspicious, yes. :)

</comment></comment><comment><USER8> The Pharma hack Krug describes: doesn't it show up a weakness in Google's ability to detect spam, if hacked sites show up in search results at all? They exhibit classic cloaking techniques, feeding search engines certain content which it hides from human visitors.

Surely you'd expect sites like that to get penalized pronto? Isn't Google able to detect this algorithmically?

<comment><USER5> The hacked sites go to considerable lengths to evade detection. That said, we made recent changes in April/May reduced the level of hacked sites bearing malware in our index by 90%.

</comment></comment><comment><USER9> This happened to me a while back, I had to change the password to the server as every domain on it was vulnerable. How do you smoke adderall anyway?

</comment><comment><USER10> I am with krug sounds like an injection into the wordpress theme - I have had a few of thesae.

check you footer and header espesialy

</comment><comment><AUTHOR> Thanks everyone but it's not wordpress, it's magento. Otherwise I will follow your advice, thanks.</comment>


A UX Metric Called Bounce Rate - <AUTHOR> https://blog.bouncelytics.com/a-ux-metric-called-bounce-rate/ <comment><USER2> Great blog entry. Some of it I don't fully agree with but still some great points!</comment>


My Life In a Cube - <AUTHOR> http://www.mylifeinacube.com/post/34191073/after-that-it-just-starts-repeating-itself

<comment><USER2> Some of those are hilarious! Thanks...

</comment><comment><USER3> "As above, so below."</comment>


Congress wants the US military to challenge Russia with a new Arctic port - <AUTHOR> https://www.defensenews.com/naval/2019/06/21/congress-wants-the-us-military-to-challenge-russia-with-a-new-arctic-port/ <comment><USER2> Finally, freedom for polar bears, f..k yea!!</comment>


Show HN: Ray tracing 3D metaballs in real time - <AUTHOR> https://mapehe.github.io/shader_demo/ <comment><AUTHOR> Code: [https://github.com/mapehe/shader_demo](https://github.com/mapehe/shader_demo)</comment>


What is tail recursion? - <AUTHOR> http://stackoverflow.com/questions/33923/what-is-tail-recursion

<comment><USER2> tail recursion (n): how to refer to tail-call optimization when you want to troll your programming languages professor

</comment><comment><USER3> It's a stack-efficiency hack, a way to rewrite your (mutually) recursive functions such that the compiler knows it can reuse stack frames instead of growing the stack at every function call. This breaks some debugging methods, but the fact it allows a lot of code that would otherwise blow out the stack to run has made it worthwhile enough for the Scheme people to mandate it in the language standard.

You do this by making sure nothing is waiting for the result of the last function your function calls; its return value is your function's return value.</comment>


Breakthrough bionic hand restores amputee's sense of touch - <AUTHOR> http://ca.news.yahoo.com/blogs/geekquinox/breakthrough-bionic-hand-restores-amputee-sense-touch-211553303.html

<comment><USER2> The diversity of sensors in the skin is impressive--it's not all sensed by a generic nerve. There are specialized mechanisms for vibration, heat, pressure and even pain. Here's a great summary with diagrams:

[http://faculty.washington.edu/chudler/receptor.html](http://faculty.washington.edu/chudler/receptor.html)

</comment><comment><USER3> I really hope the surgeon said "I am your father!" after the procedure. :)

<comment><USER4> Nooooooooooo!

</comment></comment><comment><USER5> We're going to spend a century trying to hack our bodies until we realize we can just abandon them and stop worrying all the time.

<comment><USER6> I hope it doesn't take a century, really.

</comment></comment><comment><USER7> I believe we've had technology similar to this for a long time, is there anything new with this in particular?

I believe the biggest problem is the formation of scar tissue around the implanted electrodes. This reduces the strength of the electrical signals over time, and eventually the electrodes need to be adjusted.

<comment><USER8> There have been prosthetic arms that can be controlled by thought alone for a while. I think the new part is getting real-time feedback from the arms back to the brain.

</comment></comment><comment><USER9> EPFL is cool... one day I will move there too :)</comment>