Who Killed the Desktop Application? - <AUTHOR> http://www.codinghorror.com/blog/archives/000883.html <comment><USER2> It's 2007 and smart people stopped separating "desktop" and "online" applications a long time ago, because it is pointless: the best approach to solving any problem is always picking the proper balance of server/desktop horsepower/capabilities.

Desktops are very powerful. People forget (or never realize) how powerful their computers are. For a long time only games have been taking advantage of all those millions of transistors. I _really_ hope that next generation of browsers will start moving towards that direction.

For instance: if you have a flash movie occupying 100% of the page, it becomes simply unusable if a user maximizes the window, even on 1280x1024. Isn't that ridiculous? At our age and time, a simple 2D vector graphics plays at pathetic 5-7 frames per second... (even if he's got a $700 video card) And something tells me it's not entirely Macromedia/Adobe's fault.

It's changing, though. And Google/Microsoft are moving towards that direction. Good (early) example can be iTunes or apps built on GoogleGear platform.

</comment><comment><USER3> Good point.

And I think the number one killer of desktop applications is Microsoft, however controversial it may seem at first.

It was Microsoft who convinced developers that a commercial application along with its installation procedure must look and feel heavy, so that users feel comfortable with the money they paid for it. The heavier the better. It is so rare in the commercial desktop world that developers _strive_ for simplicity.

Without Google though, the Web world could have adopted the same complication syndrome. Look at the MSDN web site, for example.

<comment><USER4> I don't think Microsoft had any part in convincing anyone that apps need heavy installation procedures. It evolved as a generic way to do some basic things: moving the executable to a place the user wants, and setting up some registry values. Isn't that a reasonable solution for desktop applications?

There is nothing wrong with the MSDN site. When you're storing millions of pages of documentation, things may get a little bloated. If you need to look up anything about the Windows APIs, or any information about .NET languages, you'll quickly see that MSDN does the job well.

<comment><USER3> One more point: you can set your registry values when your application runs for the first time. It's a common practice especially on UNIX and MacOS to generate configuration silently, when needed instead of focusing user's attention on that during installation.

Other than that, installation is essentially copying. The proper place, file- type bindings, version checks etc can be done by the system automatically, again, without annoying the user (and the developer, too). And that's exactly what MacOS was doing since version 5 in 1980's.

(Besides, Windows Registry is one of the main sources of complication).

</comment></comment></comment><comment><USER2> And, just to disagree with author, I'd recommend comparing Google's Picasa to itself.

That would be a fair comparison, because it is virtually the same application, with same set of features, done with online/offline UI's... slowly merging together.

BTW: as far as PC "desktop" software is concerned, Picasa is probably my favorite application. Even though I don't use it that much, but as an engineer concerned with usability, I always try to point our "desktop people"'s attention at it.

<comment><AUTHOR> Picasa is great, the only reason I still have a windows PC.

<comment><USER5> What makes it so much better than, say, picturesync?</comment></comment></comment>


Amazon will pull the plug on dash buttons - <AUTHOR> https://www.marketwatch.com/story/amazon-will-pull-the-plug-on-dash-buttons-at-the-end-of-august-2019-08-01 <comment><USER2> I never really got the point of them since they were usually for things that stored really well and get used up at a relatively constant rate so it's usually not a surprise when you run out, so like why would I need to rush and blindly push the button to buy more detergent/toothpaste of a certain brand without knowing the price when I'd just eventually buy it in my normal shopping run/order.

<comment><USER3> The goal was to reduce friction in the purchase. You had to walk to a computer, bring up Amazon, look up the product, click the version of it you want, and push "one click order" (or maybe two or three more clicks) -- that's a lot of opportunity for you to just add it to your shopping list and just buy it at the grocery store.

Or from your point of view, that's a lot of effort for a product that you already know you want. If you're busy doing laundry, you want to keep doing that, rather than stop and order detergent, or risk forgetting it in the next five minutes (and then not having it the next time). The price is somewhat variable, but the variation is likely below your threshold of caring.

These aren't huge costs, but it's very much web-UX thinking: people have very short memories (and attention spans) and even a single extra click is an opportunity for people to change their minds or get distracted. If your washing machine came with an infinite supply of detergent, or your cat food canister were infinitely deep, your life would be a tiny bit better. If they knew the cost were comparable to other sources, many people would take that deal without worrying too much about whether the exact price.

That's Amazon's schtick. They make your life a tiny bit better and take a tiny profit on that. It works only if they can get that tiny profit on basically everything you buy. Everything everyone buys.

They've since discovered other ways. As you say, you know the rate at which these things get used up, so it's even better for them to just replace the button with a recurring order.

<comment><USER4> One problem with their approach IMO is there was no direct feedback on when you would actually receive the product both before AND after you press the button.

You don't know if the toilet paper is going to arrive the same day, next day, 2 days, or 2 business days. Many times if you know beforehand it's going to take 2 days to get toilet paper you'd just run to the grocery store instead.

Not just toilet paper -- even with shampoo, body wash, kitchen soap, laundry detergent, kitchen sponges, I'm usually unaware I'm running out until it actually runs out, and with most of these things I can't wait 2 days. I also live 2 blocks from a 24-hour Safeway so it's not actually an issue.

<comment><USER5> These things all keep well, so I just keep extras and order when I have to start using my last one, so it doesn't really matter how long shipping takes. For me, the worst part of dashes was that Amazon didn't keep the exact same thing in stock for the exact same price, and sometimes it'd be a very different price if it had to fall back to a 3rd party seller. So I usually ended up checking and then ordering on the computer anyway.

</comment></comment></comment></comment><comment><USER6> Amazon was no longer allowed to sell them in Germany earlier this year. Consumer protection agency said the button isn't a clear enough buying agreement (here payment forms need to make clear it costs money to click, exact price, what you buy). Amazon allowed itself to change the product, e.g. amount of washing liquid.

<comment><USER7> In the US, I looked at them with a high degree of skepticism. How do I know that I'm getting the same product, at the best price?

In the US, quite frequently the same product is sold in different packaging, and the best price often varies between the packaging. Sometimes the 20-pack of toilet paper is cheapest. Sometimes the 30-pack is cheapest.

Besides, on the mobile app, it doesn't take that long to figure out the best deal.

</comment></comment><comment><USER8> I had a few, but stopped using them because the pricing was erratic.

Push a button for pistachios, and a pound bag shows up for $8.99.

Push the same button two weeks later, and get charged $17.99.

I liked the idea, though. It would have been good if it had been incorporated into the whole Whole Foods thing.

<comment><USER9> A simple solution would have been an e-ink screen that listed the price per unit/press, but that probably would have increased button costs by an order of magnitude.

Also more users probably would become aware of price variance (as you have) and changed their shopping habits, which is not good for business.

<comment><USER10> The simpler solution, is to just not price-gouge customers and hope they don't notice, while trying to build a subscription-based business.

If Amazon isn't able to offer any level of price stability, then don't offer the subscription. Toilet paper is convenient to get on a subscription basis, but that convenience isn't worth paying $5/roll.

<comment><USER9> Definitely. It would be amazing if they offered subscription with price lock- in and a preset range lockout (i.e. if current price reaches 4x subscription lock price the subscription is suspended until price goes down).

They will lose some money but they'd also gain money from people afraid to drop subscriptions (e.g. gym subscription model).

</comment></comment></comment></comment><comment> dfeojm-zlib Amazon's :CueCat

[https://en.wikipedia.org/wiki/CueCat](https://en.wikipedia.org/wiki/CueCat)

[https://hackaday.com/tag/dash-button/](https://hackaday.com/tag/dash-button/)

<comment><USER11> Those CueCat scanners were a great way to scan used textbooks when I was selling them in college. They just magically spit out a text string followed by a <CR> when scanning a barcode. They were going for dirt cheap on eBay for a number of years.

In college, on finals week I'd get $100-200 cash and go to the bookstore. I'd hear they were buying back certain popular textbooks for next to nothing. I'd offer the seller $10 cash right there on the spot. "Sell to them or sell to me, it's the same price. Here's a $10 bill." After getting a stack of those (or when profs were cleaning out their bookshelves), I'd lug them home, scan with my CueCat and sell them for a substantial markup on half.com (RIP). One semester I found a stack of books outside faculty offices labeled "free" \- sold those for $500 that weekend.

</comment></comment><comment><USER12> For anyone interested in weird low cost embedded devices, I suggest googling for Amazon dash button teardowns. There's some interesting choices made in their components and spec.

[https://mpetroff.net/2016/07/new-amazon-dash-button- teardown...](https://mpetroff.net/2016/07/new-amazon-dash-button-teardown- jk29lp/)

[https://www.edn.com/design/consumer/4460866/Teardown-- Design...](https://www.edn.com/design/consumer/4460866/Teardown--Designing- the-second-generation-Amazon-Dash-Button-)

There is also this thing, the Amazon wand:

[https://www.edn.com/design/consumer/4460319/Teardown-- Amazon...](https://www.edn.com/design/consumer/4460319/Teardown--Amazon-Dash- Wand-with-Alexa)

</comment><comment><USER13> I've no real love for the intended use of the dash buttons (re-orders of items) but do hope that some other company comes up with a similarly easily hackable wireless button for interactions.

<comment><USER14> As of now, Amazon appears to still be selling their "IOT button" (which is basically a programmable dash button)

[https://aws.amazon.com/iotbutton/](https://aws.amazon.com/iotbutton/)

<comment><USER15> The $15 "1st Generation AWS IoT Button" is out of stock, but the $20 "AWS IoT Enterprise Button" is still in stock. Is there a difference besides the label?

<comment><USER16> AWS IoT Enterprise Button creates a JIRA ticket and sends approval requests as well. /s

<comment><USER17> This is the first thing I've ever upvoted on Hacker News

</comment></comment></comment></comment></comment><comment><USER18> I think I've had 3 physical buttons since they were introduced. Only used one of them one time. It was a novel idea but never really caught on.

I'm sure Amazon would rather you just ask Alexa to order things for you now anyway.

</comment><comment><USER19> These were the ":CueCat" of 2015.

[https://en.wikipedia.org/wiki/CueCat](https://en.wikipedia.org/wiki/CueCat)

I wonder what today's equivalent is?

<comment><USER20> Today's equivalent is scanning a QR code with your phone.

<comment><USER21> (following this tangent) My impression of QR's problem was:

1) High ergonomics barrier; the user needed to install a QR app from the appstore or find the icon for it and start it.

2) People who are not on HN don't know what it is anyway.

Now that it's built in to most Android/iPhone cameras, #1 is solved. Now that #1 is solved, #2 is self-solving.

QR gets a lot of jank but when I use it, it's very handy.

</comment></comment></comment><comment><USER22> It was always baffling to make non-trivial custom hardware in an age of universal smartphone ownership.

Does Amazon even have any good apps? Seems like easy subscription management and re-ordering is something they could do there.

<comment><USER23> Amazon already has their Subscribe and Save program.

I actually really liked the dash buttons for those items that I did not use as regularly, but the number of items like that are pretty low to warrant having more than 2 or 3 of these things.

Then when I rebuilt my wireless network I just did not care to move them over.

<comment><USER24> Really liking them and abandoning them anyway is a pretty solid indictment of the idea.

<comment><USER23> I will say this is an issue with iot devices in general right now.

When I rebuilt my wireless network, every smarthome device has to be modified and that was the worst part about it.

At least these were basically free anyways (you bought them for $5 and then got a $5 credit on your first purchase with them) so I did not feel a need to switch them over.

</comment></comment></comment></comment><comment><USER25> I'm probably alone but I'm pretty bummed about this. I regularly used them to order detergent and Clorox wipes when we were getting low. Haven't been fully out of either in a couple years.

How much work would it honestly be for them just to keep the existing ones going.

<comment><USER16> "Alexa, add X to my shopping list"

<comment><USER26> Well you can directly buy the same products with Alexa, which is what I'll probably do.

</comment></comment></comment><comment><USER27> I wonder if anyone has gotten the Wi-Fi pairing process to work outside of the Amazon app. Otherwise, you can't change your wireless password without bricking any buttons you might be misusing for other DIY uses after Amazon kills support.

</comment><comment><USER28> Relegated to the parts bin. Usefulness perhaps in a second life... [https://hackaday.com/tag/amazon-dash- button/](https://hackaday.com/tag/amazon-dash-button/)

</comment><comment><USER29> Biggest problem I had with them is the products you could order were hard coded to the device itself. If the product you want to order doesn't have its own dash button, the whole system was useless.

</comment><comment><USER30> Surely they will announce a Dash button recycling program. Right?

Edit: Read rest of article and have my answer. Sorry! Don't spank me.

<comment><USER31> it was announced in the article!

</comment></comment><comment><USER32> They are ahead of their time.

If I pressed one and <1h delivery would happen it would be great.

But we aren't there yet.

</comment><comment><USER33> Does anyone know of any programmable alternatives that are not tight to AWS?

</comment><comment><USER34> Makes sense with the ubiquity of Alexa/Echos in homes. Why quantize consumer selection to a finite number of items when one can be able to order anything with the same seamlessness.

</comment><comment><USER35> I don't understand the appeal behind ordering something without knowing the price.

The only thing I ever buy without knowing the cost is a drink in a pub, and that's some sort of strange anachronism.

And in that situation, if the bartender turns around and tells me 25 quid or whatever after pouring it, I'd just walk off.

The economy doesn't work properly if you just blindly pay whatever.

<comment><USER36> A lot of the products seem to be small household items, like razor blades, trash bags, water filters, and the like, that when you run out you are (1) probably going to want to replace with the same brand, (2) probably don't want to put off replacing to wait for a bargain, and (3) probably will still buy even if the price is a little more than the last time, but within the normal Amazon price variation.

Also, according to their documentation, pressing the button queues the order and sends you an email, which includes price information. You can cancel the order before it ships. I'd expect that the normal flow for most people is to press the button when they notice they are running out of something, and then check the order details when they next deal with email.

<comment><USER37> This seems incredibly non-optimal on a few levels.

I rarely buy small household items at full price, just wait until my brands go on sale or I have a coupon to buy several. They go on sale often enough that I almost always have items in the house. I probably have a five year supply of razors at my house, they take up basically no room. If I need something right away I'll put it on the grocery list and pick it up next time I go to the grocery store.

On top of that I don't really believe in shipping my household products to me one by one, seems very wasteful and inefficient. Plus you'd either need to pay extra for shipping every item you need or buy a prime subscription on top of that.

Amazon pricing is all over the place, not like the normal variation of other stores. You could easily be paying double or even triple since the last time you bought.

</comment></comment></comment><comment><USER38> They seemed to have little value.

Cool project though.

</comment><comment><USER39> I've added up all the shits that are given, and the stunning total is... zero.</comment>


Ask HN: Is Alpha-Go winning due to improvements in AI or raw computing power? - <AUTHOR> <comment><USER2> Does it have to be one or the other? I imagine AlphaGo is too complex to have ever been effectively on, say, a 486, so without advances in hardware and networking it wouldn't work. At the same time AlphaGo is solving the sort of problem that no throw-more-computers-at-the-problem would have solved things either.

We live in a world of subtly and nuance. It's rare that there's a simple 'black or white' answer to a question.

</comment><comment><USER3> It's both. Most of the Reinforcement and Deep Learning techniques used in AlphaGo have been around for many years and while there are a couple of novel algorithms many overestimate their impact. It's most a mix & match of techniques that hasn't really been done before. Plus good engineering.

IMO compute power (GPUs) played the bigger role. DeepMind themselves say that training AlphaGo wouldn't have been possible without access to Google's large- scale infrastructure. They've been training it on thousands of machines simultaneously. That's not to be confused with the hardware necessary to _play_ the game, which isn't much and can be done on a single machine. Only the training phase has these extreme hardware requirements.</comment>


Eaten by a Grue: podcast on Infocom games, text adventures and interactive fiction - <AUTHOR> http://monsterfeet.com/grue/ <comment><USER2> If you're into good old text adventures, you might be interested in an upcoming book that (as far as I'm aware) is fairly unique in that it's written as a text adventure. Not a choose your own adventure book, but as if the main character is literally in a text adventure, with prompts on what to do and everything: [https://www.goodreads.com/book/show/31374622-post-high- schoo...](https://www.goodreads.com/book/show/31374622-post-high-school- reality-quest) It's coming out June 2017.

Disclaimer: I'm married to the author, so I might be biased :P To be fair though, I read a very early copy, and thought it was solid. It's only gotten much, much better since what I read. The head of the small press publishing it has compared it to Microserfs.

<comment><USER3> sounds really interesting, if more akin to a litrpg than to microserfs. i'll definitely read this when it comes out. is there an announce mailing list?

<comment><USER2> She apparently has a mailing list on her website: [http://www.megedenbooks.com/](http://www.megedenbooks.com/) She just told me that she basically never uses it, so if you sign up for it, you won't be getting any spam. But she said she'll make sure she sends out something announcing the release of the book and/or a reminder some small period in advance.

You can also put it on your "want to read" list on Good Reads (linked in my OP) and should get information about release via that. The Amazon page (also available via the Good Reads page) also seems to have a way to get information about release.

Sorry if I seem like I'm dumping a bunch of information, just trying to convey what my wife told me about how to get what you're looking for/be helpful/give as many options as possible for people to get information depending on what they're comfortable with :)

<comment><USER3> thanks, i signed up for the mailing list :)

</comment></comment></comment></comment><comment><USER4> Alright, time to get this off my chest:

I've been around this culture for a long time. I was on Slashdot, Digg, Reddit, and now HN. I've ascended in NetHack more than 10 times (and beaten a bunch of other Roguelikes as well). I've heard about the Infocom games repeatedly over the years.

I've tried to play Zork several times and I just don't get it. Every single time I end up in the maze of twisty little passages and I give up. What am I supposed to do? I tried dropping various objects as breadcrumbs and mapping the damn thing on graph paper... well, it turns out to be non-Euclidean. WTF is that? Is this tedious, bizarre exercise supposed to be fun? Will somebody help me _get it_?

<comment><USER5> The fun used to be that you needed to figure out the object dropping thing and then the thing that you need to think about the rooms as an abstract graph instead of something that makes spatial sense. (Zork started off as a mainframe game whose audience was pretty much all computer scientists, so they would've been expected to know about graph data structures.)

Once you know the trick, the rest is annoying busywork, and adding a maze has been considered bad design in most text adventures made after the Zork era.

The generally annoying thing about text adventures is that if you hit a puzzle you don't get, your progress can grind into total halt. There's no alternative strategies or just getting lucky like there is in roguelikes. And there are plenty of badly designed games where after having the puzzle explained, you'll go "how was I supposed to figure out THAT?" (Zork 2 has a couple infamous puzzles like this.) If you want to get text adventures, you might want to just use hints when needed to get through a few so you'll get a sense of what the games consider appropriate hints and puzzles.

<comment><USER4> Yeah, I'm aware of graph data structures. I tried creating a map like that too. It ended up a big mess of nodes with all of these edges intersecting each other and looping back around. I might have made a mistake or two and that screwed up the graph even more! It drove me crazy with how tedious it was.

I guess I reached a point where I said "this is just _too complex_ , it can't be the right solution, there's got to be a smarter way to do this". So to find out that there isn't a smarter way just makes me think it's a terrible puzzle design.

<comment><USER6> I solved it with a pencil and paper when I was 12 years old or so, and I'm pretty sure lots of other people did too.

I agree it's a little tedious by modern standards but it's really not that complicated. If I recall correctly you can represent the entire thing with about 5-10 boxes connected by a couple lines between them plus a bunch of little loops where the player remains in the same place.

It's a succinct area of the map. Perhaps you should try again with a fresh perspective.

</comment></comment></comment></comment><comment><USER7> To understand Interactive Fiction, you have to travel back to the 70's when computers took up a large room, paper terminals ruled, and the primary programming language was COBOL.

In those days, there were no graphics. None. There was no animation. Apple and IBM had not created microcomputers yet. There was no Amiga or Commodore.

At this time, in the mid to late 70's, high school students were just getting access to paper terminals that "dialed in" over coupler modems. Taking a programming class was an elective and few did.

There also wasn't any D&D or role playing games. None of this existed.

So along comes these two free games on these school mainframes; Adventure and Dungeon (later split into three commercial Zork games).

They had the audacity of allowing you, the player, to type anything you wanted, and the computer would respond, at that time, very intelligently. By today's standards it may seem simple, but back then, it was the equivalent of playing Assassin's Creed today.

This seemingly "open world" gaming changed everything about computers. It inspired everything that came after, including video games and still is the foundation of much of the standard game play of today.

Now. Zork was old school "IF". It was hard, illogical at times, and on a fairness scale probably as high as it gets. But puzzles are puzzles. If you like easy puzzles or puzzles that give you a cheat sheet, then are you really being challenged or are you just looking for entertainment?

I prefer a challenge because that _is_ what entertains me.

So first realizing that the Thief in Zork will steal all your possessions (whether you drop them or not), then realizing you have to kill him, then realizing he's the only one that can open the egg, then realizing that he's really hard to kill in certain situations, then realizing you can use the DIAGNOSE command to warn you when you're about to die so you can run away and get better.

To me, that's not boring at all. That is essentially the same kind of challenging puzzle you'll find in any of today's video games.

It's just that the "graphics" are in your head. Not on a screen.

<comment><USER8> Adventure was first released in 1976. Dungeons and Dragons was first published in 1974. Adventure is clearly influenced by D&D.

<comment><USER7> If you believe the accounts of Crowther and Woods, they knew nothing of D&D and there were maybe a few thousand original poorly made copies of D&D at the time. Only "wargamers" knew about it until the 80's.

From a historical perspective, even though D&D was published in 1974, its influence didn't kick in until years later.

<comment><USER8> Source on that? This article claims Crowther did play D&D, based on personal email from a co-worker (M. F. Kraley): [http://www.digitalhumanities.org/dhq/vol/001/2/000009/000009...](http://www.digitalhumanities.org/dhq/vol/001/2/000009/000009.html#p20)

</comment></comment></comment></comment><comment><USER9> There are still a lot of MUDs out there, which I'd consider to be multi-player extensions of games like Zork. Some still draw 500+ paying accounts logged in at peak times, and have characters that are 25+ years old, like Gemstone IV (play.net/gs4).

I tend to explain it as "World of Warcraft without graphics", but "rapid-paced AD&D" is a fair approximation, too. The medium gives the people running the show a lot of flexibility in how they guide collaborative stories - no need to wait on art assets!

</comment><comment><USER10> Lot of good infocom stuff here: [https://archive.org/details/infocomcabinet](https://archive.org/details/infocomcabinet) and also interesting stuff on [http://www.filfre.net/sitemap/](http://www.filfre.net/sitemap/)

</comment><comment><USER11> I used to play a lot of text adventures when the iPod Touch first came out. It was a surprisingly nice platform for them with the iPod Touch on-screen keyboard. The app was named Frotz.

<comment><USER12> Yes, Frotz is the de facto standard for playing text adventures. It is available for almost every platform. I used a version of Frotz on the Nintendo DS.

</comment></comment><comment><USER13> The Infocom parser is still smarter than Siri.

An Infocom adventure would have known how to handle "Siri, continue playing podcast".

</comment><comment><USER14> I've played a lot of adventure games, the Zorks, Hitchhikers guide, Planetfall, Leather Goddesses of Phobos and also the magnetic scrolls games like The Pawn and Guild of Thieves.

but for some reason my favorite zork game isn't a text adventure, it's Zork Grand Inquisitor. Humor, actors, acceptable graphics. man I love that game.

</comment><comment><USER15> You can play Zork in Slack with Slork: [https://github.com/devshane/slork](https://github.com/devshane/slork)

It's a Zork wrapper written in Elixir.

</comment><comment><USER16> Here's a cool Zork map visualization project:

"Exploring the original Zork source code with Graphviz and an interactive d3 map, using JavaScript and an extension of Peter Norvig's Python Lisp parser to handle MDL."

Source: [https://github.com/bburns/Lantern](https://github.com/bburns/Lantern)

Demo: [http://owlsign.org/Lantern/](http://owlsign.org/Lantern/)

</comment><comment><USER17> Didn't see a link here, so I'll drop one for "Get Lamp", a documentary about text adventures.

[http://www.getlamp.com](http://www.getlamp.com)

</comment><comment><USER16> The original Zork source code in MDL which is available here: [http://retro.co.za/adventure/zork-mdl/](http://retro.co.za/adventure/zork- mdl/)

It's also here on github: [https://github.com/itafroma/zork- mdl](https://github.com/itafroma/zork-mdl)

It is fascinating to read, and really beautiful code, quite understandable even if you don't know MDL, and practically a form of literature.

I played the original Zork on MIT-DM and also the Infocom versions of course. Reading the source code is like seeing the behind-the-scenes underground rooms and passages at Disneyland!

While I was playing Zork, I found a bug. First some context: when you're battling the troll, you can give things to him, and he eats them! Sometimes he drops his axe, and you can pick it up and kill him with it. He blocks the exits until you kill him.

So I tried "give axe to troll," and he ate his own axe, then cowered in terror: "The troll, disarmed, cowers in terror, pleading for his life in the guttural tongue of the trolls." Not satisfied with that, I tried "give troll to troll", and he devoured himself: "The troll, who is remarkably coordinated, catches the troll and not having the most discriminating tastes, gleefully eats it."

...Except that I still could not get out of the exit, because every time I tried, it said "The troll fends you off with a menacing gesture."

I figured there must be a troll flag that wasn't getting cleared when the troll devoured itself. And sure enough, I found it in the code, and it's called "TROLL-FLAG!-FLAG"!

Here is an excerpt of the MDL troll code, where you can see the bug, where it should clear the troll flag when the troll devours itself, but doesn't (well that's how I would fix it!):

[https://github.com/itafroma/zork- mdl/blob/be079a4ed234071222...](https://github.com/itafroma/zork- mdl/blob/be079a4ed234071222e991d0da0f9e8c7f96d125/act1.mud#L1337)

                   <COND (<VERB? "THROW" "GIVE">
                          <COND (<VERB? "THROW">
                                 <TELL
        "The troll, who is remarkably coordinated, catches the " 1 <ODESC2 <PRSO>>>)
                                (<TELL
        "The troll, who is not overly proud, graciously accepts the gift">)>
                          <COND (<==? <PRSO> <SFIND-OBJ "KNIFE">>
                                 <TELL
        "and being for the moment sated, throws it back.  Fortunately, the
        troll has poor control, and the knife falls to the floor.  He does
        not look pleased." ,LONG-TELL1>
                                 <TRO .T ,FIGHTBIT>)
                                (<TELL
        "and not having the most discriminating tastes, gleefully eats it.">
                          <REMOVE-OBJECT <PRSO>>)>)
                         (<VERB? "TAKE" "MOVE">
                          <TELL
        "The troll spits in your face, saying \"Better luck next time.\"">)
                         (<VERB? "MUNG">
                          <TELL
        "The troll laughs at your puny gesture.">)>)
                  (<AND ,TROLL-FLAG!-FLAG
                        <VERB? "HELLO">>
                   <TELL "Unfortunately, the troll can't hear you.">)>>

The troll bugs are also in Zork I! Here's a newspaper article from January 8 1985 about "Infocom Glitches Bug Game Players"!

[http://articles.sun- sentinel.com/1985-01-18/features/8501020...](http://articles.sun- sentinel.com/1985-01-18/features/8501020995_1_typing-troll-adventure-games)

And here's another article from March 29, 1985: "Zork I Fuddles Mental Health With Choices":

[http://articles.sun- sentinel.com/1985-03-29/features/8501120...](http://articles.sun- sentinel.com/1985-03-29/features/8501120597_1_trap-door-computer-games-troll)</comment>


Software That Lasts 200 Years (2004) - <AUTHOR> http://www.bricklin.com/200yearsoftware.htm <comment><USER2> This doesn't make much sense to me. Bridges, roads, sewers, pipes, etc are still around because the problems they solve haven't changed, and in the thousands of years we've been building bridges, we're only seeing the ones that survived, and completely missing the ones that crumbled. It seems like survivorship bias.

Software is evolving because we have new problems to solve. I'd point out that Unix is 50+ years old and still going strong and used daily by millions of people.

I don't get where the assumption that these are timeless structures comes from - the New York City Subway system is falling apart and in need of continual maintenance. Roads constantly fall apart and need to be re-surfaced - it's not like we use incredibly elegant long term materials, we use blacktop because it's cheap _now_. Bridges and tunnels are being maintained all the time.

<comment><USER3> Indeed; the comparison of software engineering with architecture (or other classes of engineering), notably when taking into account history, are always misled.

For example, the assumption that old bridges still solve modern problems. Compare:

[https://www.youtube.com/watch?v=o4eM0qoUhaE](https://www.youtube.com/watch?v=o4eM0qoUhaE) [https://en.wikipedia.org/wiki/Puente_Romano_%28M%C3%A9rida%2...](https://en.wikipedia.org/wiki/Puente_Romano_%28M%C3%A9rida%29)

On the close to 2000 years old Merida's bridge, only peatons are allowed nowadays. If you tried to run a traffic of hundreds of 30tn trucks per hour on it, it would soon fall into pieces. Also, you have to take into account the maintainance that has been added to it during the ages (for example, the banks have grown closer and the extremum piles are now under ground).

</comment></comment><comment><USER4> Knuth had actually this "it should last" goal as he designed TeX:

[http://scicomp.stackexchange.com/questions/3462/increasing-t...](http://scicomp.stackexchange.com/questions/3462/increasing- the-archival-longevity-of-code)

"Ever since those beginnings in 1977, the TeX research project that I embarked on was driven by two major goals. The first goal was quality: we wanted to produce documents that were not just nice, but actually the best. (…) The second major goal was _archival: to create systems that would be independent of changes in printing technology as much as possible. When the next generation of printing devices came along, I wanted to be able to retain the same quality already achieved, instead of having to solve all the problems anew. I wanted to design something that would be still usable in 100 years._ " – Donald E. Knuth: Digital Typography, p. 559 (quoted from [http://de.wikipedia.org/wiki/TeX](http://de.wikipedia.org/wiki/TeX) )

[https://en.wikipedia.org/wiki/TeX](https://en.wikipedia.org/wiki/TeX)

"Even though Donald Knuth himself has suggested a few areas in which TeX could have been improved, he indicated that he firmly believes that having an unchanged system that will produce the same output now and in the future is more important than introducing new features. For this reason, he has stated that the "absolutely final change (to be made after my death)" will be to change the version number to π, at which point all remaining bugs will become features.[11] "

<comment><USER5> Having a system that produces the same visual output in 100 years is nice, but to achieve that it is sufficient to maintain the old TeX installation in usable form. We have release versions for this. Why does he think his software should be approaching some state of immutability is incomprehensible to me. Evolution of civilization will bring new requirements, perhaps better way to deal with colors, better vector image notation, better font handling, better web/HTML integration, ...

<comment><USER4> Knuth has answered your worries in 1990:

[http://www.ntg.nl/maps/05/34.pdf](http://www.ntg.nl/maps/05/34.pdf)

"I have put these systems into the public domain so that people everywhere can use the ideas freely if they wish. I have also spent thousands of hours trying to ensure that the systems produce essentially identical results on all computers. I strongly believe that an unchanging system has great value, even though it is axiomatic that any complex system can be improved."

"Of course I do not claim to have found the best solution to every problem. I simply claim that it is a great advantage to have a fixed point as a building block. Improved macro packages can be added on the input side; improved device drivers can be added on the output side. I welcome continued research that will lead to alternative systems that can typeset documents better than TEX is able to do. But the authors of such systems must think of another name.

That is all I ask, after devoting a substantial portion of my life to the creation of these systems and making them available to everybody in the world. I sincerely hope that the members of TUG will help me to enforce these wishes, by putting severe pressure on any person or group who produces any incompatible system and calls it TEX or METAFONT or Computer Modern — no matter how slight the incompatibility might seem."

</comment></comment></comment><comment><USER6> As long as underlying data is preserved, software is better done fungible. Data formats change, processing models and architectures change, underlying hardware change - allowing us to do more with less. In my career I have found there are no stable architectures - systems require refactoring every couple of years to stay manageable and full rewrites possibly once every technology cycle to make it in tune with the times. This is not a limitation, but an opportunity as software is constantly evolving and is more like a living,breathing organism than a permanent structure.

<comment><USER7> Sorry for the downvote, I meant to hit up and I can't seem to change it

</comment></comment><comment><USER8> Most of the software I write is based on business processes, which surely won't even last 20 years (more likely 2 to 5 years), and/or technology (I work at an ISP and data center provider, so we manage stuff like IPs, VLANs, Hardware, Racks) which likely also won't last 200 years.

The example from the article about accounting software is another good example: legal requirements change, which means that the software must also change; book keeping software from 30 years ago is utterly useless today, not because it wasn't done well, but because too much changed.

Another example from the article: parking tickets. There were no parking tickets 100 years ago (80 years ago according to [http://theexpiredmeter.com/2009/08/first-parking-ticket- issu...](http://theexpiredmeter.com/2009/08/first-parking-ticket-issued- in-1935/)), and I doubt that however traffic looks in another 100 years, it will have not much in common with our current structure.

My takeaway is that we shouldn't worry too much about longevity of the software; the more important aspect is the data formats. They should be open and well-documented, so that folks in the future have less trouble dealing with our data footprint. Maybe our data will still be of value in 200 years; at least some of it, and at least a bit. But I doubt our software will be, regardless of how we write it. I do agree with his points about the need for better review processes and oversight.

Update: Forgot one important point: Input method and user input devices change a lot. Who wants to use software today that assumed a max screen resolution of 600x400 or even lower? And no mouse, no touch screen, no audio or image or video input, even it would be highly desirable for the problem domain?

<comment><USER9> Interestingly, Nearly all airline reservations are made on software written in the 60's. Still.

</comment></comment><comment><USER10> This is a fantastic article and it confirmed what I've learned from the Rich Hickey talks: software isn't about software. It's a medium for information. Information is light and it deserves a place to live that isn't creating uncertainty. The world will be shaped by software, and the root of software that deserves to last is lasting respect of the people who use it...

</comment><comment><USER11> This argument is specious. Bridges only continue to be relevant a thousand years on because humans have barely evolved in that time frame, not because bridge builders put any effort into ensuring their bridges would still be relevant to future mankind.

Computer programs on the other hand are driven by fashion, and evolve rapidly. Digital documents created by programs will cease to be readable when the programs that could read those documents fall out of fashion, but this is not due to any short-sightedness by the developers of these programs, but rather by users that choose to prefer new (more fashionable) programs even when these are incapable of reading the documents they previously wrote.

<comment><USER12> You've explained partly why the phenomenon exists, you haven't explained why it's not a problem.

<comment><USER13> It's not the onus of commentators to provide a complete valid alternative to back up any criticism.

</comment></comment></comment><comment><USER14> Some folks are working on a standard curriculum for software engineering and a standard process to follow that incorporates both waterfall and agile methodologies. In fact it also includes having an engineering team think about what they are doing and evaluating the quality of their process using a number of standard dimensions and metrics.

Read more about SEMAT here [http://semat.org/](http://semat.org/)

Stuff like this is a prerequisite to creating software that is done, and can be used for 200 years without any substantial change.

Could Java ever get to that stage? What about some of the JVM ecosystem such as Tomcat or Camel?</comment>


The harm of "non-commercial" - <AUTHOR> http://blog.georgikobilarov.com/2010/01/the-harm-of-non-commercial/ <comment><USER2> Creative Commons have known this was a stupid licence since they introduced it, software licences had been through the same thing and learned from it. They really should withdraw it, which is what I thought the recent consultation was leading up to.

If you have to ask the users of your pro forma legal document "By the way, what do you think this actually allows and prohibits?" then you're clearly failing at your assigned task.

A link to the research paper:

<http://wiki.creativecommons.org/Defining_Noncommercial>

</comment><comment><USER3> Why would anyone think that "non-commercial" would _only_ prevent earning money on the licensed material? if the material plays any role in a commercial endeavour obviously you cannot use it if it is CC "non-commercial". That is why the license reads: "You may not use this work for commercial purposes." _not_ "You may not earn money of this work."

<comment><USER4> This isn't as "obvious" as you think.

First, you quoted the human-readable summary of the license, _not_ the license itself. The CC-BY-NC 3.0 license actually says,

 _"You may not exercise any of the rights granted to You in Section 3 above in any manner that is primarily intended for or directed toward commercial advantage or private monetary compensation."_

Note the word _primarily_ \- so if the use is only incidentally or secondarily commercial in nature, it may still be compatible with the "non-commercial" clauses. There are more details in the legalese of the license, but nevertheless this is a very vague and somewhat controversial license - so much that the Creative Commons organization did a survey of community members and published a lengthy report on the different ways of interpreting it:

<http://creativecommons.org/weblog/entry/17127>

One question is: Do commercial businesses ever engage in non-commercial activities? Is Hacker News a non-commercial community site that happens to be operated by Y Combinator, or is it _"primarily directed toward commercial advantage"_ merely by virtue of its affiliation?

<comment><USER3> I find the license even clearer than the summary I quoted. I am missing a actual usage scenario where one could be uncertain if the use was inside or outside the license.

<comment><USER4> Suppose I'm a tenured professor and I publish a blog with educational posts about computer science. I write blog entries during my working hours. The blog isn't a requirement of my job but it may help raise my academic profile. Am I free to use your CC-BY-SA 3.0 photo to illustrate my blog post?

What if, instead of a professor, I am a freelance programmer. There are no ads or other exchange of money directly related to the blog, but sometimes potential clients find me through my blog. But the blog is also a way for me to converse with a group of regular readers whom I already know. Is this blog _primarily_ directed toward commercial advantage?

In either of the above scenarious, suppose the blog has ads but that I don't turn a profit on it (i.e. the ads defray but do not cover the hosting cost). How about now?

Or what if the revenue from the ads is more than the hosting cost, but much less than I could make through my professional activities. If the amount of money is tiny relative to the time I spend on the blog, could I argue that the blog is not _primarily_ directed toward commercial advantage?

You may find each of these cases perfectly clear and obvious, but wherever _you_ draw an "obvious" line, the above study probably shows that many Creative Commons members will think it should be somewhere else. Even the organization that drafted the license admits that it doesn't provide very explicit guidance for common cases like web sites that include (but don't rely on) advertising, or sites that do not involve exchange of money but may provide less-tangible commercial advantages for their owners.

</comment></comment></comment></comment><comment><USER5> Non Commercial is basically useless. The only non-commercial site I can think of is some mum blogging about her dog. Even if I just have a personal blog it is kind of commercial because it is marketing for myself.

</comment><comment><USER6> I work for a registered non-profit charity financed partly by The Arts Council and National Lottery Funding. You'd certainly call our licensed bar and restaurant a commercial enterprise, as would the employees.

I also volunteer for a registered co-operative part-financed by said charity, the rest from ticket sales of our film screenings. We have no legal obligation to be non-profit or otherwise.

If I used your non-comm image on our website, how would you feel ?</comment>


A Web Start-Up Counting on Ad Sales? Good Luck - <AUTHOR> http://bits.blogs.nytimes.com/2008/12/08/are-free-ad-supported-web-sites-over/

<comment><USER2> A little ironic I think.

Newspapers Counting on Ad Sales? Good Luck!

They've been in trouble for quite a while thanks to Craigslist and the web in general. Plus didn't news get out today about the NYT mortgaging their HQ to alleviate the hit they've taken from falling ad sales themselves? If I recall, a few other papers filed for bankruptcy or went up for sale.

Regardless, I was surprised by that Economist article posted last week that pointed out that online ad sales were, in fact, still growing, but at a lower rate than previously projected.

</comment><comment><USER3> It is, sadly, a little content-free.

>> Web start-ups struggle with the question of whether to sacrifice revenue for several years, build a huge audience and then sell them ads, as YouTube did >>

Hmm, my recollection was that Youtube built up a huge audience and then got acquired by Google, which runs them as, essentially, public utility. (They spend more on costs than they recoup from ads but they get the cross-subsidy from AdWords like, well, every other Google property.)

For those folks who can't cross on perpetual cross-subsidization and don't want to play the "get acquired before we are destroyed by the economics of hosting terabytes of content that we don't own the copyrights to" game, could I suggest, say, charging money for value? Its a business model for all seasons.

</comment><comment><USER4> I thought this was a poorly researched article. Our CPMs are up year over are, as are most other AdSense supported sites that I'm aware of. The latest numbers that I've seen, including those recently referenced by the Economist, show online advertising growing again in 2009.

It's print advertising that's falling off the cliff.

Of course multiple revenue streams is preferred. But we'll certainly continue to see ad supported business models get funded.

</comment><comment><USER5> I think that the article is a bit naive. While the growth of ad revenues is undoubtedly decreasing and most likely spending will greatly dip as the "panic" continues, this doesn't mean that a web businesses cannot be successful on ad revenue.

Newspapers succeeded during the Great Depression as did radio and other ad- based businesses.

But this time, newspapers are going out of business because people are getting their news from the web (the classic Innovator's Dilemma pattern). Indeed, with the recession, people will have even more time to surf the web so web traffic and the importance of the web will continue to increase.

Where the eyeballs are, ads must go especially during a downturn. In the next 2-3 years, especially after we move away from "panic" and back to recovery, it seems to me, web ad spending will start growing again.

I should also mention that I currently work at an start up that depends on web-based ad sales. I've had opportunities to move somewhere else but my gut feeling is that a lean business with a high quality product is the place to be right now, even if it depends on ad sales for its profitability.

</comment><comment><USER6> Online ad spend is continuing to grow year-over-year and accountable channels like search and performance/affiliate marketing will be the largest drivers of that growth.

The brand dollars are the ones that get thrown out of the window in an economic downturn. But companies still have to acquire new customers when the economy is bad and they are very likely to double down by spending their ad dollars on accountable media channels to hit their numbers.

Startups that are building targetable/monetizable audiences will be just as likely to make an ad supported model work now as they would have been able to a year from now. Startups who believe that they will be able to just grow a really big general audience and magically monetize it with high price CPM ads from big brands are the ones that are in trouble.

</comment><comment><USER7> I don't know that anyone really counts on ad sales from the beginning. I think most startups just subscribe to the "get users now, figure out how to monetize later" lunacy, and that usually means advertising.

<comment><USER8> I don't think it's lunacy if your startup is cheap enough to run. If you can support a million users on $100/month, why not... It's worth that just to put it on your resume, or just for the knowledge learned from scaling. Then you just have to figure out how to make $0.0001 per user/month and you're into profit.

I do agree about the ones that take a ton of funding and have a ridiculous burn rate though.

<comment><USER7> If there's scaling involved, you're spending a lot more than $100 per month. The only websites I can think of that could get high traffic for low cost are blogs, since they can just run on TypePad or something, but I don't really consider them startups generally. TechCrunch and HuffPo maybe.

There's almost nothing that's profitable at a fraction of a cent per user per month. The net is cheap, but it isn't that cheap.

<comment><USER8> You're definitely spending more than $100/month in terms of time, but in terms of servers/bandwidth, you can support a million users pretty well on $100 if you don't use the standard web frameworks etc which are woefully inefficient.

<comment><USER7> Well, at $100 per month there's no real scaling. You're running everything on one inexpensive server. Scaling would be just getting a better one. (Can you even get a dedicated for that low?)

I guess maybe you and I define scaling differently. I'm thinking like adding in web servers, load balancing, replicating db. You may be just talking optimizing code and caching or something.

Curiously, can you serve even some simple dynamic content like a Wordpress blog to a million somewhat frequent users a month?

<comment><USER8> That's the issue though... you don't need a dedicated server to serve 1m users, and you don't need to add web servers, load balancing, replicating etc etc

VPS servers are so cheap and more than enough. The current web stacks are just ridiculously lousy.</comment></comment></comment></comment></comment></comment>


Ask HN: Chatbot for Litigation lawyers - <AUTHOR> Idea just popped into my head.  A chatbot for litigation lawyers to help them prepare for their argument in court, like an arguebot!.....maybe even useful in the courtroom telling you when to object?  like or dislike? <comment><USER2> "Like" or "dislike" isn't as big of a question as "possible" or "impossible". Understanding complex legal arguments, which would vary from city to city, county to county, and state to state sounds impossible with today's technology.

<comment><AUTHOR> I agree, but I did see this one that seems to be working for traffic tickets, 160,000 wins and counting. [https://venturebeat.com/2016/06/27/donotpay- traffic-lawyer-b...](https://venturebeat.com/2016/06/27/donotpay-traffic- lawyer-bot/)

<comment><USER2> That's a radically different idea. That chatbot isn't AI. It's probably barely using NLP. Users are basically just filling out a form.

Understanding and countering legal arguments is a vastly different problem. Current technology is just not even close.</comment></comment></comment>


Is Quantum Entanglement Real? - <AUTHOR> http://www.nytimes.com/2014/11/16/opinion/sunday/is-quantum-entanglement-real.html <comment><USER2> _All_ components of a quantum system are always entangled. It's not some mysterious property that arises under unusual circumstances. In fact, I would argue that quantum entanglement is _the_ defining feature of quantum mechanics that distinguishes it from classical mechanics.

However, the way that most articles (and even physicists) use the term "entanglement" is in a loose way. Within a good approximation, a multi- component quantum state can be represented as a tensor product of single component quantum states. When this isn't a good approximation anymore, you can say that the entanglement of the system is much more apparent.

To give an example of what I mean, there's a technique in quantum chemistry known as density functional theory (DFT), which is used to compute ground state energies of various molecules. For some molecules, it works pretty well. The benefit of DFT is that it is a fast calculation technique (well, as far as quantum chemistry goes), but it's speed comes at a price. Rather than using the Coulombic interaction of every pair of electrons to compute the energy of a molecule (or more technically, using every Coulombic interaction as terms in the Hamiltonian), a probability cloud of electron positions is computed instead, and this cloud is used as the energetic term.

This works really well for a lot of systems, but in some cases (superconductors, metals, solid-state physics, van der Waals interactions), this approximation falls apart because the effect of electron-electron correlation significantly affects the energy of the system. In this case, the full, non-separable wavefunction is required. The fact that this full wavefunction cannot easily be broken into an approximation of simpler, single electron wavefunctions means that the entanglement of the system is very apparent.

<comment><USER3> I agree completely. Entanglement is everywhere.

To calculate the correct energy of the electrons in a molecule it's necessary to use entanglement. But these molecular examples may be explained using some weird classical models with local hidden variables, and with electrons that interact with each other to conspire and provide the right result of the measurement. The quantum mechanical explanation is actually more simple, but not "intuitive".

Almost all the discussions about entanglement discuss the strange case of two entangled particles that are far away. The distance between the particles is only a trick to:

* be sure that one of them can't communicate to the other and tell the result of the measurement

* be sure that there is no a local hidden variable theory that explain the result

<comment><USER4> why one hidden variable, when you could have one for each atom between the entangled? :)

</comment></comment></comment><comment><USER5> I have never been comfortable with the metaphors that tech reporters use for describing quantum entanglement.

They always talk as if it's a spooky interaction but the metaphor falls down. Here's an example: I take too halves of an Oreo cookie and send them in opposite directions. Then I use a detector to measure how much filling is on one half and instantly know how much was on the other half. Was that spooky?

I understand that at the physics level it's not like this, but the metaphors are never satisfyingly 'spooky' to me as I can easily think up a classical explanation.

<comment><USER6> I feel I'll never get to understand entanglement (I hope it's like monads :P). If it's not "spooky" but just half an Oreo... then what is so special about it and why would physicist study it then?

<comment><USER7> Let's play an imaginary game. It will be you and a friend as a team against the house, represented by me and my two assistants. Here's how the game works.

1\. You and your friend can consult with each other before the game starts and agree on a strategy. You can build or obtain any equipment you think will help you play the game.

2\. You and your friend, and any equipment you bring, meet me at the place where the game starts. I have two spaceships waiting. You and one of my assistants will get into spaceship A and go off into space. Your friend and my other assistant will go into spaceship B and go off into space. You and your friend can bring along whatever equipment you want.

3\. My assistants will take you to separate locations in space. The locations will be at rest with respect to each other, and several light minutes apart. My assistants will establish synchronized clocks between the two locations, and then play starts. I'll only describe what goes on at your location. The same thing is going on at your friend's location.

4\. There will be 1000 rounds, and each round lasts one minute. In each round my assistant will use a random number generator to generate a single bit, 0 or 1, and will tell you the bit. You can assume that this random number is truly random. There is no known way to predict it, and there is no correlation between it and the random number generator at the other location. The rounds start on a strict schedule so that you and your friend start round N on your respective ships at the same time.

5\. After you are told the random number, you must generate a single bit, 0 or 1, by any means you wish. You can come up with it in your head. You can use any equipment you brought. You can read it from a list of predetermined bits you and your friend generated before the game started. You can try to communicate with your friend (or with anyone else) if you brought communications equipment, and use the results of that communication--but keep in mind you are several light minutes from your friend.

6\. After the 1000th round, my assistant will fly you back to Earth where I will be waiting. My assistants will each give me the list of random numbers and responses. I will then compute your team score by adding up your scores for each round. Scoring for each round is as follows:

• If you both received a 1 in the round from the random number generators, then your team scores 1 point if you and your friend gave DIFFERENT bit values for that round. I.e., if the random bits were both 1, then you score if you gave 0 and your friend gave 1, or if you gave 1 and your friend gave 0.

• Otherwise, your team scores 1 point if you and your friend gave the SAME bit value for that round.

Your win prizes if your score is above 800.

Question: what is your plan?

You can win 75% of the rounds with a very simple strategy: always pick 0. You'll win the rounds where the random bits were 00, 01, or 10, and lose when they were 11. That turns out to be the best you can with non-quantum methods.

If you use entanglement you can do better. You prepare 1000 pairs of qubits, numbered 1 to 1000. The two qubits in each pair are entangled in what is called a Bell state. It's an equal superposition of the two states "both qubits are 0" and "both qubits are 1". What this means is that if you take these two qubits and you measure one and your friend measures the other, AND YOU MEASURE IN THE SAME BASIS (I'll explain that in a moment), you will either both get 0 or both get 1.

What I mean by measure in the same basis is this. Suppose your qubits are implemented by, say, polarized photons. A photon can be polarized at any angle from 0 to 360 (I'm going to use degrees instead of radians because I think that will be more comfortable for most). To make qubits out of this, you might decide that 0 is represented by polarization at 0 degrees, and 1 by polarization at 90 degrees. To see if a qubit is a 0 or a 1, you could try to pass it through a polarizing filter set to 90 degrees. If it comes through, it was a 1, and if it is blocked it was a 0. If we are using 0/90 degrees for our qubits, we say we are using the 0 degree basis.

What happens if you pass that qubit through a polarizing filter at 45 degrees? Half the time it will pass, and half the time it will be blocked. So a qubit that has a definite value in the 0 basis can be any value with equal probability in the 45 degree basis. How about if we measure using a 22.5 degree basis? It can still be any value, but now there is about an 85% chance it will have the same value as in the 0 degree basis. In other words, if someone generates a qubit with a particular binary value in the 0 degree basis, and you measure it in the 22.5 degree basis, 85% of the time you'll get the same value that the person who made it set it to in their basis.

In general, if we have two qubits in the same state, and you measure one of them in one basis, and I measure the other in a basis that is T degrees different from yours, we will get the same measurement with probability cos(T)^2. E.g., if we use the same basis (T=0), then we get the same measurement all the time. If my basis is 90 degrees from yours, we'll get opposite measurements. If my basis was 45 degrees from yours, half the time my measurement would agree with yours and half the time it would not.

Here is how you use your prepared qubits to improve your game outcome. As I said, you take pairs of qubits and put that system of two qubits into an equal mix of the states "both qubits are 0" an "both qubits are 1". When you head off to space, you take one of the qubits from each pair, and your friend takes the other.

For round N, after you get the random bit from my assistant, you take your qubit for round N, and you measure it to see if it is a 0 or a 1. If my assistant gave you a 0 bit, you measure in the 0 degree basis. If my assistant gave you a 1 bit, you measure in the 45 degree basis. You report the result of your measurement as your bit.

Your friend measures in the 22.5 degree basis if he gets a 0 random bit, and he measures in the -22.5 degree basis if he gets a 1 random bit. He reports the result of his measurement as his bit. Here's a diagram showing the angles you each use depending on the random bit you receive:

          1---- friend -----0
          |                 |
       -22.5      0       22.5      45
                  |                  |
                  0----- you </comment></comment></comment><comment>--1

Note that if the random bits are both 1, then you two do your basis and his are 67.5 degrees apart. On the other three cases, they will only be 22.5 degrees apart.

So, in all cases except both random numbers being 1, which means that you want to pick the same bit as your friend, your measurements will agree with probability cos(22.5)^2 = 0.854.

In the case where both random numbers are 1, which means you and your friend want to NOT agree, you will agree with probability cos(67.5)^2 = 0.146, which means you will disagree with probability 0.854.

Net result: your probability of winning a given round is 0.854, which is quite a bit better than the 0.75 that is the best you can get with a non-quantum approach.

Try to duplicate this with a "half an Oreo" model of entanglement, and it won't work.

<comment><USER5> Thanks! This is a pretty good description of a scenario where quantum effects can do something that a classical model can't.

</comment></comment><comment><USER8> Even with experiments still ongoing, quantum entanglement is most likely real. I think David Kaiser wouldn't suggest otherwise.

There's a real nice section in wikipedia re: the testing of quantum entanglement (see [http://en.wikipedia.org/wiki/Quantum_entanglement](http://en.wikipedia.org/wiki/Quantum_entanglement))

"Systems which contain no entanglement are said to be separable. For 2-Qubit and Qubit-Qutrit systems (2 x 2 and 2 x 3 respectively) the simple Peres- Horodecki criterion provides both a necessary and a sufficient criterion for separability, and thus for detecting entanglement. However, for the general case, the criterion is merely a sufficient one for separability, as the problem becomes NP-hard.[59][60] A numerical approach to the problem is suggested by Jon Magne Leinaas, Jan Myrheim and Eirik Ovrum in their paper "Geometrical aspects of entanglement".[61] Leinaas et al. offer a numerical approach, iteratively refining an estimated separable state towards the target state to be tested, and checking if the target state can indeed be reached. An implementation of the algorithm (including a built in Peres-Horodecki criterion testing) is brought in the "StateSeparator" web-app."

</comment><comment><USER9> Good opportunity to again promote »The Theoretical Minimum« [1] by Leonard Susskind (born June 1940, director of the Stanford Institute for Theoretical Physics with research interests in string theory, quantum field theory, quantum statistical mechanics and quantum cosmology). It is pretty accessible and a key takeaway is that there is a lot of misinformation in the wild and especially on the internet when it comes to (modern) physics as he tell his class not only once when someone asks about something he read somewhere.

[1] [http://theoreticalminimum.com/](http://theoreticalminimum.com/)

<comment><USER3> The site looks interesting, but it's difficult to find the information that is related to the discussion. Does this site have a page that discusses specifically entanglement? EPR paradox? Bell inecuality? I'd like to see a link to the specific relevant page and not a link to the general site.

<comment><USER9> Here is the course on quantum entanglement [1] but depending on prior knowledge it may be wise to first go through the courses on classical [2] and quantum [3] mechanics. He tries to keep everything as self-contained as possible but does not go through the very basics things like Lagrangians and Hamiltonians every time.

[1] [http://theoreticalminimum.com/courses/quantum- entanglement/2...](http://theoreticalminimum.com/courses/quantum- entanglement/2006/fall)

[2] [http://theoreticalminimum.com/courses/classical- mechanics/20...](http://theoreticalminimum.com/courses/classical- mechanics/2011/fall)

[3] [http://theoreticalminimum.com/courses/quantum- mechanics/2012...](http://theoreticalminimum.com/courses/quantum- mechanics/2012/winter)

</comment></comment></comment><comment><USER10> Some questions:

\- Can we use this to send large amounts of data around the world quickly? Can it replace intercontinential data links?

\- Can we use this for interstellar communication channels of limited amounts of data?

\- Can we use this to pass messages between two locations in a way that is immune to easedropping?

\- What are other real-world applications of this?

<comment><USER11> It's a very common misconception that entanglement allows faster than light communication. It doesn't.

Describing quantum phenomena with real world metaphors always breaks down if we carry things far enough. I'm going to use coin flipping as a metaphor to make a specific point, but don't assume this metaphor will still work if you try to extend it. To even begin to understand entanglement you really need to look at the math. That being said, let's proceed.

 _The Setup:_ (this explains how the metaphor is being made, but you can skip to the payoff if you just want the what without any of the why).

We can think of a pair of entangled particles as a pair of coins. If we flip each coin, we have a fifty/fifty odds of gettings heads or tails. This is what we normally think of, in the macroscopic world, as a random outcome (It's not really, but let's not go there). An entangled pair of coins behaves very unusually. The outcome of flipping one coin is related to the outcome of the other coin. For example, if one coin comes up as heads, the other will _always_ come up as tails, or vice-versa. Let's say we separate the two coins by several light years and then flip them. The strange correlation will hold. For a non-Quantum particle there are several possible explanations for this behavior. We might think that, somehow, the measurement of one particle causes the outcome of the other article to change. However, time depends on reference frame. If the two coins are in relative motion to each other in the right manner, which particle was measured first is relative (There is no universally preferred reference frame). i.e. One observer might correctly say that the first coin was measured before the second, but another observer might see things reversed. So, which measurement is the cause and which is the effect? Another notion is that the outcome was preordained when the coins were made. I'd have to go into the math a bit to adequately explain this, but with entangled quantum particles you can create experiments that demonstrate this is impossible. Tests of Bell inequalities are especially notable for this. This is Einstein's "spooky action at a distance", a behavior exhibited by entangled particles.

 _The Payoff:_

So, if a measurement performed on two entangled particles is like flipping a pair of coins, how can we use that to communicate. Well, we can't. Not directly. You see, we can't force a coin to come up heads or tails. If we look at the outcomes of just one coin they will always appear random and uncontrollable. It's when we talk to the guy with the other coin that we'll say, "Hey, I knew what you were going to get because of what I got!". So, classical communication, limited by the speed of light, is necessary to figure out what these correlations are. So forget about FTL communication. However, what about communicating secrets? Well, that's a whole other kettle of fish! It turns out quantum correlations are _fantastic_ for communicating secret keys. Google quantum cryptography.

As for other applications, google quantum computing. Entanglement is absolutely intrinsic to this field. There are also applications in medical imaging and metrology (i.e. standards of measurements).

<comment><USER12> What about using those entangled states to coordinate actions? You could plan ahead and say if you use heads or tails as binary, flip enough to get digits to use as (hopefully hospitable) coordinates and a time to meet. When the two parties meet successfully that location was coordinated using communication faster than light, no?

Maybe you could that same mechanism for superfast decision making communication in automated equipment. Drone dispersal, late stage missile targeting. Think shooting a bunch of missiles into the sky with a map of possible locations and then entanglement used to have each missile pick its target, no missile would hit the same target and no one would know which missile was going where at launch. Stuff like that works?

You could use it for frequency hopping on radios making tracing communication pretty difficult.

<comment><USER13> You could just stick a random seed and tell them to meet at a place determined by that seed. What's the difference?

The closest use is probably quantum cryptography, somehow, I'd guess. But that's still not FTL communication.

<comment><USER12> I'm not sure how you use a random number generator to randomly generate values that generate knowledge about previous uses. Those missiles aren't picking their targets randomly, they know where the other missiles are going and where they have to go. No missile is going to hit the same target, with a random number generator they would.

</comment></comment></comment></comment></comment><comment><USER14> It seems the assertion is that there are 2 options, 1\. quantum entanglement exists 2\. "Some unnoticed causal mechanism in the past may have fixed the detectors' settings in advance, or nudged the likelihood that one setting would be chosen over another."

The human brain may or not be deterministic, but it is certainly a complex system. For the results to be so consistent for the multitude of experiments that have been performed, it seems highly unlikely that #2 is the case. I think in this case it's pretty fair to apply Occam's razor.

On a side note, I hate titles for popular physics articles. The next article titles: "5 super simple steps to splitting the atom" "20 crazy tips for growing high temperature superconductors" "Make free cash farming Higgs bosons"

<comment><USER15> Option 2 is similar to the Novikov self-consistency hypothesis of time travel, which states that if an action would cause a time travel paradox, its probability is zero.

Under this hypothesis, even if entanglement does allow changes to move faster than light--essentially travelling backward in time--any action that would expose this time travel would be forbidden. The scientist is physically prevented from making the "wrong" choice when setting the detector.

The problem with this hypothesis is that it seems essentially untestable.

Another way to think of option 2 is that the universe is 100% deterministic. Under this hypothesis, the speed of entanglement is only apparent, not real-- like planning with a friend to both turn on your flashlights on opposite hilltops 2 hours from now. Without knowing that it was planned in advance, an observer would wonder how you reacted to one another so fast.

This is also impossible to fully test; at best you could prove that the consistency extends as far back as we can see into the past. And that is exactly what these scientists will be testing.

<comment><USER14> The Novikov self-consistency hypothesis is pretty neat. I like the thought of the researcher travelling back in time to create the apparatus such that the result always confirms quantum entanglement. It's classical and not quantum so it's not clear to me how it relates. I'm not sure anyone (besides philosophers) takes the loopholes seriously.

</comment></comment></comment><comment><USER16> I assume that photons act as waves and not particles. Though they have quantized energy (frequency), there's no reason for me to think that they have locality. If so...

The most plausible explanation I can think of is a Monty Hall paradox at the source of the emitter. Thousands of photon pairs? How are they generated? Is their emission dependent on the destination via say, pilot wave resonance? Either resonance or waves that travel backwards in time. Experiments of entanglement speed should point towards one or the other.

</comment><comment><USER17> I see nothing counter-intuitive about quantum entanglement. Photons travel at the speed of light--time does not exist for them. Alter one, the entangled one is instantaneously connected.

<comment><USER18> "If you think you understand quantum mechanics, you don't understand quantum mechanics." \- Richard Feynman

<comment><USER19> I wish you comment were downvoted instead of karmakaze's one.

He is expressing an opinion and even if he is wrong, it is still an interesting idea, and shows some effort. A decent short explanation could show the mistake, and make a good comment->answer combo.

Instead you quoted a comment which is out of context ( Feynman's comment is intended for physicists of his level ), contributing nothing.

( Please don't take this as a personal attack ( it isn't), but more a thought what's wrong with HN in light of recent debate )

<comment><USER18> No, you're right - it was snide and I should have reconsidered it.</comment></comment></comment></comment>


Location Sharing is Coming to Facebook - <AUTHOR> http://www.readwriteweb.com/archives/location_sharing_is_coming_to_facebook_-_how_will_users_react.php

<comment><USER2> I'm excited to learn more about this. What's interesting is that they plan to feed off of Foursquare, Gowalla, etc. I'm happy that they decided not to kill everyone off. Working together is great =)</comment>


Zstandard – Real-time data compression algorithm - <AUTHOR> http://facebook.github.io/zstd/ <comment><USER2> > Zstandard is a real-time compression algorithm, providing high compression > ratios. It offers a very wide range of compression / speed trade-off

This would have been very handy about 17 years ago when I was doing a virtual CD-ROM driver, and needed to store compressed images, and the client wanted a wide range of speed/compression options for the user--much wider than zlib offered.

I ended up doing a hack that was either the most disgusting thing I have ever done, or was brilliant. I have not been able to decide which.

My hack: I gave them a compression/speed slider that went from 0 to 100. If the slider was set to N, I would apply zlib at maximum compression to N consecutive blocks, and then apply no compression to 100-N consecutive blocks. Repeat until the whole image is stored.

The client loved it.

<comment><USER3> Reminds me of how LEDs are dimmed using Pulse Width Modulation: "40% brightness" actually means "on at 100% intensity for 40% of the length of each refresh cycle, and then completely off for the rest."

<comment><USER4> Or playing PCM on the old PC speakers having only on/off beep function... It seemed like a miracle at the time :)

<comment><USER5> > Or playing PCM on the old PC speakers having only on/off beep function

That _is_ PWM, pulse width modulation.

[https://en.wikipedia.org/wiki/PC_speaker#Pulse- width_modulat...](https://en.wikipedia.org/wiki/PC_speaker#Pulse- width_modulation)

<comment><USER4> Ok then: playing PCM with PWM...

</comment></comment></comment></comment></comment><comment><USER6> zstd is incredible, but just in case the thought hasn't occurred to someone here that may benefit from it: if you're in control of the send and the receive, type-specific compression is hard to beat.

For example, if you know you're dealing with text, you can use snappy, if you know you're dealing with images, webp, videos x264 (or x265 if you only care about decode speed and encoded size), etc and then fall back to zstd only when you don't have a specific compressor for the chosen file type.

<comment><USER7> Agreed that if you have the control (and potentially the time depending on the algorithm), type specific compression is the way to go. Having said that, zstd beats Snappy handily for text ^_^

On enwik8 (100MB of Wikipedia XML encoded articles, mostly just text), zstd gets you to ~36MB, Snappy gets you to ~58MB, while gzip will get you to 36MB. If you turn up the compression dials on zstd, you can get down to 27MB - though instead of 2 seconds to compress it takes 52 seconds on my laptop. Decompression takes ~0.3 seconds for the low or high compression rate.

<comment><USER8> Why not use a XML/XSD specific compression like EXI for that ?

[https://www.w3.org/TR/exi-primer/](https://www.w3.org/TR/exi-primer/)

<comment><USER9> It really is mostly just text. It's not _quite_

<Article>... [30 kb of text] ...</Article>

but almost.

</comment></comment></comment></comment><comment><USER10> I used to object to using zstd based on Facebook's onerous license and patent policy; but now that zstd is plain BSD+GPLv2, I endorse it.

</comment><comment><USER11> Zstd recently added a long range mode that can find matches up to 2 GB in the past using a specialized algorithm [1]. It can be enabled on the command line with `zstd --long` for a 128 MB window, or `--long=windowLog` for a `2^windowLog` byte window.

[1] [https://github.com/facebook/zstd/releases/tag/v1.3.2](https://github.com/facebook/zstd/releases/tag/v1.3.2)

<comment><USER12> I'm not sure if you'll see this, but I have a question. I have a situation where I need to implement in-place compression with perfect rollback in case of failure. The first approach I thought of was feeding LZMA input until I had X amount of output, then finalizing and starting over, but then I started wondering if I could simply periodically serialize the LZMA state machine data every 30 seconds or so.

I haven't investigated either approach, but this is one of my next projects I'm working on, so I'll be figuring it out either way. I was wondering, does Zstd provide any ability to serialize/restore and continue where it left off?

(Context: Disk compression. Due to long-term factors beyond my control I've had severe disk space issues for over a decade (with frequently only tens of MBs free). Complex story ending in "cannot work". I recently realized that some of my disks contained data I didn't immediately need, and began wondering about ways I might be able to compress this data out "of the way" so I could carefully partition the remaining space. This would need to be done in-place as I would have nowhere with enough space to hold the compressed data, at least not to begin with.)

<comment><USER13> Lending/buying an empty (external) drive for starting seems to be much simpler and safer.

<comment><USER12> That would be a highly preferable option.

The last time someone gave me an old empty 320GB HDD they weren't using, one of my own disks started clicking about a week later and I was able to save everything on it. I still shake my head at that perfect timing.

Of course, this meant I lost all the additional free space, haha. One step forward, two steps back...

</comment></comment></comment></comment><comment><USER14> We added zstd to largely supersede gzip use at work. It performs better than oldschool gzip in both compression and speed.

That said, I'm not sure I'd call it a "real-time" compression algorithm. It's still an factor of 2x slower than lz4 compress and a factor of 4x slower decompress.

<comment><USER15> It's not clear exactly what "real-time" is supposed to mean, but I don't think it makes sense to define it in terms of other algorithms. If someone developed an algorithm faster than lz4, would lz4 cease to be real-time?

<comment><USER16> I thought that 'real-time' is not about how _fast_ a program is. Instead, it's about being able to guarantee that a program will always complete in under a fixed amount of time, under all circumstances.

I can't see how that applies to Zstandard either.

<comment><USER17> It's a definition not used much anymore but this relates to on demand (streaming) versus ahead of time data processing.

With only a small delay, can I compress and decompress at least one second of data per second, indefinitely.

Think live TV versus film.

<comment><USER18> That's usually called an "online" algorithm, in my experience. It does seem to agree with the way it's used in this context.

<comment><USER19> Online isn't quite the right distinction here imo, since that's really more about possibility than speed. An algorithm could be online (doesn't require all the data at once to run) but still not be fast enough to stream.

</comment></comment></comment></comment></comment></comment><comment><USER20> A curious fact I encountered: First, the zstdcat executable that ships with zstd (actually a symlink to zstd itself, roughly equivalent to "zstd -cqdf") can decompress .gz files. Second, zstd is _faster_ than gzip at decompressing .gz files, taking about 2/3 as long.

I'm not kidding. I couldn't believe it myself, but subsequent testing stubbornly bore it out—on one file that was 15 MB after compression, and on a mix of smaller files. I tried compiling gzip from source, using the same compiler I used for zstd, and the results were the same. strace seemed to show zstd read and wrote in chunks 2x the size, but the number of syscalls didn't seem to be nearly enough to explain the difference. It seems to have this "zlibWrapper" subdirectory; its README has some benchmark numbers that I don't fully understand, but some of them seem to match the 2/3 factor I got.

I'm wondering if this is a clever tactic to drive adoption—get people to use the zstd executable even when they're still using gzipped files. ;-)

Also, the fact that it has support for reading (and, apparently, writing) gzip, lz4, and xz on top of its own format really makes "z standard" an appropriate name.

<comment><USER21> A zlib feature is being patent free, that might very well affect speed. I do not know if that is the issue you are describing.

<comment><USER20> So, the zstd executable links to libz.so (says ldd), and I don't think it reimplements the core algorithmic part of gzip. I gather that it's simply more intelligent in how it uses the API... e.g. it says some things about reusing contexts, though that seems to apply only to handling multiple files.

</comment></comment></comment><comment><USER22> Supported on Btrfs since kernel 4.14. Like other compression options, it's a mount time option.

Edit: Also in squashfs. Here's the git pull request which includes some benchmarks. [https://lkml.org/lkml/2017/9/11/369](https://lkml.org/lkml/2017/9/11/369)

<comment><USER11> Upstream SquashFUSE also has zstd support.

[https://github.com/vasi/squashfuse](https://github.com/vasi/squashfuse)

</comment></comment><comment><USER23> I wonder how the RAD Game Tools compressors would fit into that benchmark list. In their benchmarks Oodle Selkie has roughly the same ratio as Zlib but is 3-4x faster at decompression than Zstd (not the same benchmark though). [http://www.radgametools.com/oodle.htm](http://www.radgametools.com/oodle.htm)

<comment><USER24> Asset compression has a specific optimization function (Charles Bloom of RAD Game Tools has a ton about this on his blog, cbloomrants): maximize output bandwidth given this specific CPU and input bandwidth.

Compression speed, for example, doesn't really matter. Sure, it should be something reasonable (e.g. not like PAQ), but if you can bump the above metric by 10 % while compression takes 50 % longer: probably worth it.

<comment><USER23> Aha, he actually addressed zstd in a post. tldr is zstd usually lower ratio and lower speed than mermaid but sometimes better ratio for text.

</comment></comment></comment><comment><USER25> We've been using zstd for a while in production, switching over from rar/gzip/7z for archiving huge XML files. It's speed to compression ratio is really impressive. Hats off to the development team.

</comment><comment><USER26> I use the C bindings for ZStandard, it performs very nicely as an algorithm. I wish the documentation was a little easier to use. I am still not sure if the provided examples really properly handle the degenerate cases of /dev/urandom and /dev/zero when using `ZSTD_DStream{In,Out}Size` and friends.

In any case - thanks for releasing this, it's been very helpful to me.

</comment><comment><USER27> Some good discussion in earlier postings too: [https://hn.algolia.com/?query=zstandard&sort=byPopularity&pr...](https://hn.algolia.com/?query=zstandard&sort=byPopularity&prefix&page=0&dateRange=all&type=story)

</comment><comment><USER28> Anyone using zstd along with an incremental rsync approach? Like gzip --rsyncable ?

Say I have 1000 files. I want to compress them and let the cron rsync do it's thing. Next day, if only one file had changed, rsync should pickup only the differential instead of the whole archive.

Or is there a better way of doing it?

</comment><comment><USER29> In the "Compression Speed vs Ratio" graph, the Y-axis doesn't start at zero. If it were changed to start at zero, it would be easier to evaluate the magnitude of the compression ratio change at a glance. IMO, that's probably worth the extra whitespace in the graph.

</comment><comment><USER30> From that benchmark list I'd prefer lz4..

<comment><USER11> lz4 and zstd are complementary, not competitive. lz4 targets applications where extremely fast decompression speed (and compression speed) are required, at the expense of compression ratio. However, many applications would prefer to spend a bit (or a lot) more CPU in exchange for better compression, while maintaining fast decompression speed.

</comment></comment><comment><USER31> Last time this was discussed Zstandard didn't have a splittable mode and by the looks of it[1] they still don't. That doesn't make it a bad algorithm it just means that it's not a good choice yet for Hadoop et al. As far as I know no container format has implemented Zstandard yet.

Does anyone know any better? It seems like we could use a better alternative to Snappy.

[1] [https://github.com/facebook/zstd/issues/395](https://github.com/facebook/zstd/issues/395)

<comment><USER32> > As far as I know no container format has implemented Zstandard yet.

ORC has a reserved future enum for ZSTD [1] (my guess is that FB already uses it).

The original headache was the patent licensing scheme[2] for the actual codebase, but now that hadoop has a ZStandard built into libhadoop.so, this is much more easy to ship to Apache.

My branch is so far behind now that I probably would implement it faster if I started over & used the libhadoop.so impl as the basis - [https://github.com/t3rmin4t0r/orc/tree/zstd](https://github.com/t3rmin4t0r/orc/tree/zstd)

[1] - [https://issues.apache.org/jira/browse/ORC-46](https://issues.apache.org/jira/browse/ORC-46)

[2] - [https://github.com/facebook/zstd/issues/775](https://github.com/facebook/zstd/issues/775)

</comment></comment><comment><USER33> In a lot of big data workflows, zstd is no brainer replacement of gzip. Always faster decompression, plus you can save tons of storage and data transfer.

</comment><comment><USER34> When one talks about general data compression, isn't that relevant that it generally applies to text compression, and not video or audio?

I guess general data compression works on audio and video, but most of the time you either choose to compress text, audio, video or you create a file format that indexes your data.

</comment><comment><USER35> We switched many of our Redshift compression encodings to zstd. Works fantastically on text and JSON.

</comment><comment><USER36> Not really totally this topic, but I deal with a bunch of 8 Gig images of device firmwares. Is there anything that can make compressing these and de- duplicating these images fast? I have used borg in the past, but archive operations are just seem slow.

<comment><USER37> You may want to store them uncompressed and rely on ZFS block level deduplication. It's similar to borg but may be faster.

Also, you may want to play with borg options and use lz4

</comment></comment><comment><USER38> So like 'middle out'

</comment><comment><USER39> see Compression Benchmark at ([https://github.com/powturbo/TurboBench](https://github.com/powturbo/TurboBench))

</comment><comment><USER40> That's the best Weissman score I have ever seen.

<comment><USER40> Come on people, take a joke

</comment></comment><comment><USER41> Why is this site an SPA?!?!?

<comment><USER42> It's... not? It's a static page with some JavaScript-driven charts.

<comment><USER43> It's a page with the majority of the content wrapped in an XMP element with style="display:none;". I'd consider that pretty hostile.

</comment></comment></comment><comment><USER44> TL;DR: Facebook acquires "Pied Piper" to gain exclusive rights on the Mean- Jerk-Time (MJT) compression patent; renames it to Zstandard for political correctness.

</comment><comment><USER45> That patent clause though. Stick to the fork from before Facebook bought the man and the algorithm.

<comment><USER11> It was relicensed in v1.3.1. It is now dual licensed under BSD without the patents clause [1] and GPLv2 [2]. GPLv2 was added for inclusion in the Linux kernel.

[1] [https://github.com/facebook/zstd/blob/dev/LICENSE](https://github.com/facebook/zstd/blob/dev/LICENSE)

[2] [https://github.com/facebook/zstd/blob/dev/COPYING](https://github.com/facebook/zstd/blob/dev/COPYING)</comment></comment>


Ask HN: Are there any startups targeting the credit reporting market? - <AUTHOR>

I know there are a few places like Credit Karma that work to help you get access to the existing big 3 credit agencies in the US.  But is anyone actually tackling the reporting and scoring itself?
I know some of the micro-lenders themselves score people based on their social media and other factors, but is anyone trying this on the larger scale to disrupt not so much the lending but the reporting side?
I am also thinking about those that move to another country and don't have established credit and sometimes struggle to get themselves established.  I think in some cases this is easier than others, but would love to understand it better. <comment><USER2> As you said there has been a few tries but usually tied to the lender. Lenders like 'Prosper' and others use other criteria to make the credit decision (social contacts, other bill payments, etc...) There are several FICO scores for many financial products (e.g. your car FICO score, credit card FICO score) and the main one. I guess one way to disrupt would be a way to collect the data not usually compiled (payment history for utilities bills, rent payments, micro loan payment, medical bills) However, in any case your customers are not the people, but the institutions who check for credit, so find a need where someone needs to make a credit decision but doesn't want to use existing credit reports and scores.

<comment><AUTHOR> Your last point is probably the biggest issue in some ways. The true customers of reporting and scoring are lending institutions and you need to provide them with accurate data. But at the same time you have legal and IMO moral obligations to make sure you are reporting accurate and complete (as possible) data to make sure the lender gets a true risk profile and consumers aren't left screwed.

</comment></comment><comment><USER3> Thats actually a very good idea. Here in Germany credit-reporting is done from one central institution called the Schufa and their methods of scoring as well as their practices are completely intransparent and can sometimes have quite a big impact on peoples lives (about even the most basic things like being able to have a cell phone contract or renting a flat) without any alternative for both creditors and debitors. This area could really benefit from some innovation.

<comment><AUTHOR> That is similar to how it is here in the states but there are 3 main agencies today. There is one primary scoring system called FICO that is not very transparent either. They give rough ideas as to the weight things have on your score but that is about it. There are some alternative scoring systems too but not as widely used.

In the US banks are the primary funding source for most credit reporting agencies, so it seems to me that CRA's interests are more in line with the bank then in being neutral with their reporting. Meaning they would rather report something that is negative and maybe not accurate then not report it because it protects their client and raises consumer rates. Many stories exist around these types of issues. Is the funding source for Schufa similar in Germany, primarily banks?

</comment></comment><comment><USER4> While not credit related, a startup here in Canada is trying to tackle identity verification at a global scale: [https://www.trulioo.com/](https://www.trulioo.com/)

Your last point about moving from another country to one where you don't have established credit is interesting. If there's a service where you can verify credit cross-borders, I think there might be something there.

Credit scores are great but the real problem is not the score itself. It's about knowing how likely the person is who they are claiming to be and how likely will they pay their bills on time - we use it as a proxy to score the quality of the customer. If there are alternative data points that can tell us the difference between a good customer and a bad customer, we would use it.

<comment><AUTHOR> That's a great point, and thank you for the information and the link. I will research it and see what comes back. I am thinking your point about identity verification is really the true problem. The credit scoring is what it is, the fact that errant records are linked because of poor identity discrimination is really a core issue. Cool! Thanks again.

<comment><USER4> You're most welcome. Happy to help. If you need more insight, feel free to send me an email (in my HN profile). Would love to hear what's to come of it.</comment></comment></comment>