New Features Coming in PostgreSQL 10 - <AUTHOR> http://rhaas.blogspot.com/2017/04/new-features-coming-in-postgresql-10.html <comment><USER2> This bit about ICU support v.s. glibc:

        > [...] Furthermore, at least on Red Hat, glibc regularly whacks
        > around the behavior of OS-native collations in minor releases,
        > which effectively corrupts PostgreSQL's indexes, since the index
        > order might no longer match the (revised) collation order.  To
        > me, changing the behavior of a widely-used system call in a
        > maintenance release seems about as friendly as locking a family
        > of angry racoons in someone's car, but the glibc maintainers
        > evidently don't agree.

Is a reference to the PostgreSQL devs wanting to make their index order a function of strxfrm() calls and to not have it change when glibc updates, whereas some on the glibc list think it should only be used for feeding it to the likes of strcmp() in the same process:

        > The only thing that matters about strxfrm output is its strcmp
        > ordering.  If that changes, it's either a bug fix or a bug
        > (either in the code or in the locale data).  If the string
        > contents change but the ordering doesn't, then it's an
        > implementation detail that is allowed to change.

\-- [https://sourceware.org/ml/libc- alpha/2015-09/msg00197.html](https://sourceware.org/ml/libc- alpha/2015-09/msg00197.html)

<comment><USER3> Florian Weimer's reply is also interesting:

"Why do you think that? I don't see this documented anywhere, and I doubt it is something many readers of the C standard, the man page, or the glibc manual would expect.

The manual suggests to store the strxfrm output and use it for sorting. I expect that some applications put it into on-disk database indexes as a result. This will lead to subtle breakage on glibc updates.

(The larger problem is that there are _definitely_ databases out there which use B-tree indexes in locale collation order, which break in even more subtle ways if we make minor changes to the collation order.)"

<comment><USER4> Which manual suggests storing the output of strxfrm? The glibc man page doesn't seem to.

I don't know that this is resolvable. The documented behavior of strxfrm() is just about its output properties. Improvements to the transformation algorithm would be expected to be made, if it's improvable.

If a database needs this to be static over time it needs to pick a _particular_ transformation algorithm and specify it exactly, not just rely on whatever the C library happens to provide.

I mean, not only are PostgreSQL locale-sorted-indexes not portable across glibc releases. They aren't portable across any other system change either. No moving between distros or doing distro upgrades, etc... Those are all misfeatures probably worth fixing.

<comment><USER5> BTW, the new amcheck tool, available in Postgres 10, lets you validate that an index is consistent with its designated sort order (B-Tree operator class). At least you now have some way of detecting the kind of inconsistency you describe.

I wrote amcheck, and maintain a version targeting earlier releases of PostgreSQL on Github: [https://github.com/petergeoghegan/amcheck](https://github.com/petergeoghegan/amcheck)

</comment></comment></comment></comment><comment><USER6> Ok, I'm not a database manager for enormous projects, so these changes may be great, but I don't understand them and don't care about them. Postgres is already the most awesome thing in Earth to me.

Still, if my opinion counts I think SELF-UPDATING MATERIALIZED VIEWS should be the next priority.

<comment><USER7> How do you mean? Couldn't you use a trigger to update the view?

<comment><USER8> A trigger on what? Every update, insert, delete, etc.? On every table in the view?

Even if that is possible, it may be a major performance killer. This has to be done internally, I think.

<comment><USER9> I am not sure it necessarily would be that bad. After all foreign keys are implemented with triggers and they are usually fast enough. You just need to write trigger functions which are fast enough.

</comment></comment></comment></comment><comment><USER10> I deeply appreciate the great care that Postgres committers take in writing their merge messages.

I think of it as a sign of respect for future developers to take the time to write a clear account of what has happened.

<comment><USER11> Postgres is one of the few projects that still use a strict patch-oriented development process that's based almost entirely around mailing-list communication.

While core team members can commit directly to the repo, everyone else must submit the code changes for review to the pgsql-hackers mailing list as a clean, self-contained patch, where it's discussed and considered for inclusion. An accepted patch might be committed right away, or it will be queued up for the next scheduled "commitfest" [2], when patches are reviewed and finally committed to mainline. (I don't know how the commitfest interacts with git exactly; the commitfest database doesn't even link to git, only to email discussions.)

From the outside it seems a bit antiquated, but it's apparently been working well for them. The Postgres team is a pretty conservative bunch; they only switched from CVS to git in late 2010, for example.

They also _really_ care about code quality, getting the design right early, and covering all possible edge cases. As a result, Postgres solid, clean, has unusually few legacy oddities, and almost never any subtle, suprising breaking changes. If you read the MySQL manual, it's absolutely _littered_ with sloppy little breakages throughout its history: Like how, until 5.0.something, when comparing a "date" value with a "datetime" value, the time portion would be silently ignored and ('2017-04-08 14:04' = '2017-04-08') would return true; but they fixed that, and broke a lot of client code because they didn't stop to realize that a lot of developers depended on that behaviour.

[1] [https://wiki.postgresql.org/wiki/Submitting_a_Patch](https://wiki.postgresql.org/wiki/Submitting_a_Patch)

[2] [https://commitfest.postgresql.org](https://commitfest.postgresql.org)

<comment><USER12> > If you read the MySQL manual, it's absolutely littered with sloppy little > breakages throughout its history: Like how, until 5.0.something, when > comparing a "date" value with a "datetime" value, the time portion would be > silently ignored and ('2017-04-08 14:04' = '2017-04-08') would return true; > but they fixed that, and broke a lot of client code because they didn't stop > to realize that a lot of developers depended on that behaviour.

This is an interesting comment for two reasons. Firstly because a lot of people also complain about MySQL's archaic defaults which often stay too long because of upgrade concerns (though they fortunately are fixing a lot of them already or for MySQL 8.0 - hooray).

But also because it speaks volumes, in my opinion, about the MySQL documentation that these are documented in the first place. I worked at MySQL for 9 years and though it was always clear our manual was always a good source of information, now that I am working on Ubuntu & OpenStack it is painfully obvious just how good the MySQL documentation team and processes were compared to many other projects. Even just the version ChangeLog.

I'm not saying other projects don't get it right (and have no opinion at all about postgresql's documentation state), but MySQL seems to get it pretty right in general.

<comment><USER11> But MySQL's problem is that those "archaic defaults" shouldn't have happened in the first place.

No matter how good MySQL is good at documenting their bugginess, weirdness, flakiness and overall history of legacy warts, the point was PostgreSQL eliminates pretty much all of this through promoting, at every single stage of the development process, the same strict, uncompromising QA principles. By doing this, legacy behaviour generally disappears.

Postgres isn't bug-free, of course. But they take great care to not be continually chased by a tsunami of technical debt. If you build a house on a crappy foundation, you get a crappy house, so it's a good idea to spend time on the foundation before building the house. The Postgres team spent _years_ on the foundation, before building the higher-level parts, and the rewards are obvious.

Meanwhile, MySQL has spent _years and years_ slowly mopping up tech debt. Things have gotten a lot better with the tighter semantics, such as preventing "February 29th" from being inserted, or rejecting "0000-00-00" as a date, or silently ignoring data coercion errors, and so on. But those things shouldn't have happened in the first place if the developers had been better at QA. So while you're right that the documentation is decent at describing various legacy semantics, it's also encodes a history of carelessness that's rather embarrassing reading.

By the way, date/datetime regression I mention isn't in the documentation at all. It's in their bug tracker.

</comment></comment></comment></comment><comment><USER13> Even a single feature from the list would make 10 an amazing release, all of them together is just unbelievable. Very happy we are using PG :)

</comment><comment><USER14> I'm so excited for table partitioning. I use table inheritance in several places in my current project, but have felt the pain of foreign key constraints not applying to inherited children. Reading about table partitioning, I'm realizing that this is a much better fit for my use case.

Postgres continues to amaze me with the speed at which they introduce the right features into such a heavily-used and production-critical product. Thanks Postgres team!

<comment><USER15> Unfortunately, foreign keys won't be supported right away.

Read about the new feature and its limitations here: [https://www.postgresql.org/docs/devel/static/ddl- partitionin...](https://www.postgresql.org/docs/devel/static/ddl- partitioning.html#ddl-partitioning-declarative)

<comment><USER14> Thanks, I hadn't read this. That's too bad, hopefully we'll see that in the future (if it's technically possible at all?). That'd be a huge feature for me.

</comment></comment></comment><comment><USER16> How is Postgres so consistently the best open-source DB project from features to documentation? It's unreal.

<comment><USER17> Not just a DB project, either. I'd say it's one of the best executed (in a very broad sense of the word) open source projects around, in general.

From end user perspective, they have stable, quality releases with a predictable cycle and subsequent maintenance releases. They have great documentation - one of the best in the industry, much less open source. Things generally work as you'd expect them to, and when not (e.g. for historical or implementation reasons), you have clear and convincing explanations. And so on.

I haven't seen their developer side, but based on other people's feedback, it's also good - high quality bar for code, stringent review process etc. More importantly, they seem to be making the right (= leading to more stable quality releases with great features) technical decisions consistently, which to me is a hallmark of a very well run team.

I also can't remember any publicized "drama" around Postgres, either on the inside (dev disagreements etc), or between the team and the users. It looks like everyone's happy, or at least happy enough.

I don't know what the magic sauce is here, but it feels like many other open source projects could learn a lot from the Postgres team and community.

</comment></comment><comment><USER18> Will DDL replication for the logical replication be landing in 10 or later?

We have some use cases where logical replication would be very helpful, but keeping the schema in sync manually seems like a pain - will there be a documented workaround if DDL replication doesn't make it in?

</comment><comment><USER19> I did read the article, but I can't find any mention of addressing the "Write amplification" issue as described by Uber when they moved away from postgres. [https://eng.uber.com/mysql-migration/](https://eng.uber.com/mysql-migration/) I had heard talk on Software Engineering Daily that this new major revision was supposed to address that.

Is this issue resolved by the new "Logical replication" feature? It doesn't seem directly related, but it seems like maybe that is what he is referring to in this blog post?

<comment><USER20> Write amplification is a result of PostgreSQL's decision to not used clustered indexes, there's not much that can be done to avoid it without a massive redesign of the storage engine - though there are patches out there to reduce the penalty in some cases. In all reality though, Uber wanted a key-value store and not an RDBMS, MySQL was a better choice for this since InnoDB isn't much more than a fast K/V store (hence why MySQL uses clustered indexes).

<comment><USER21> > massive redesign of the storage engine

Have the Postgres thought about adding support for more than one storage engine? Then they could implement new ideas in a fork, an one could run them side-by-side and migrate over to it.

[https://www.postgresql.org/message- id/4CB597FF.1010403@cheap...](https://www.postgresql.org/message- firstname.lastname@example.com)

For example MySQL had been mocked for its old ISAM storage engine. Then MySQL added InnoDB as another storage engine, the SQL interface is the same.

<comment><USER20> Pluggable storage engines for databases don't work that well in practice. Either you end up with the MySQL situation where the storage engine is so dumb that you can't push any smart optimizations into it (making having pluggable engines moot in the first place), or you have to write such a large interface that it's not worth providing.

<comment><USER22> That depends on what you mean by pluggable storage and how it's implemented ...

For example PostgreSQL supported custom index access methods, which you might see as a custom storage format (although only for secondary storage). You had to modify the source code and rebuild PostgreSQL, but there was a fairly clear separation / internal API that allowed that. Since PostgreSQL 9.6 you can do that without the custom build (i.e. you can create a new index in an extension and use CREATE ACCESS METHOD to plug it into the server).

We don't have such clear internal separation for the primary storage, but it's hard to ignore the possible benefits of alternative storage types. Another reason is that we're constantly scavenging for free bits in various places (e.g. flags in tuple headers needed by new features etc), and allowing multiple formats would help with this by supporting "old" and "new" table format. So it'll likely follow what happened to indexes - build a clear internal API, allow multiple storage formats internally, eventually make it usable from extensions.

(These are just my personal opinions, of course.)

</comment></comment></comment></comment></comment><comment><USER23> Can anyone recommend a decently up to date book on postgres administration? Or are docs really the only way? I've used SQL Server for years but would likely choose postgres for an independent project if I intended to commercialize it. That said, I don't use it at work so it's hard to get in depth experience.

<comment><USER22> There's PostgreSQL 9 Admin Cookbook from Simon Riggs, for example (disclosure: I work for Simon).

Packt has several other good books about PostgreSQL, but always check the author - they started publishing books authored by people entirely unknown in the community, that are "inspired" by book published before (you might also use "plagiarism" instead).

<comment><USER23> Yeah packtpub is a real crapshoot. They're great in that they'll seemingly publish whatever tech subject you want to write about. The downside is they publish anything...

<comment><USER22> Yeah, although there's a difference between "publishing whatever" and "publishing books that copy from other books".

Ultimately it's not the publisher but the author that matters, I guess.

</comment></comment></comment></comment><comment><USER24> Extended Statistics! I was following the replication changes, but have just discovered the extended statistics and am more excited about them.

The directory renaming at the bottom of the post is interesting - I wonder if many other projects have to do things like this?

<comment><USER21> It would be great if some Linux distros clear up the directory mess. There are directories in there with names that no ones remembers what they originally meant in the UNIX of 1970s, or what ever. For compatibility they could be just hard/soft-links to a more sane directory structure.

Well the same goes for Windows. With Win95, WinNT 3.5, WinXP, WinVista they restructured the internal directory tree and renamed things. It was okay with WinXP, just the long user folder was trouble some because of 260 chars MAX_PATH limit. But with Vista and 64-bit support the fucked up and it's now a big mess in Win7+ (syswow64, system32, registry, winsxs, dotNet folders, ... such a big mess and sometimes also waste of HDD space by duplicates of files).

<comment><USER25> winsxs uses hard links - space wastage is more likely from more versions than just dupes. Also, many windows tools won't account t correctly for hard links in disk usage stats.

</comment></comment></comment><comment><USER26> The feature I'd really love is master selection with Raft or similar and automatic query redirection to the master for all write queries (and maybe for reads with a query keyword).

That would make it very easy and robust to cluster pg without requiring a big complicated (a.k.a. high admin overhead and failure prone) stack with lots of secondary tools.

This kind of fire and forget cluster is really the killer feature of things like MongoDB and RethinkDB. Yes people with really huge deployments might want something more tunable, but that's only like 1% of the market.

Of course those NoSQL databases also offer eventual and other weaker but more scalable consistency modes, but like highly tuned manual deployment these too are features for the 1% of the market that actually needs that kind of scale.

A fire and forget cluster-able fully consistent SQL database would be nirvana for most of the market.

<comment><USER27> Can't pgbouncer/pgpool2 solve query redirection? I don't understand the desire for all-in-one solutions.

<comment><USER26> That desire comes from three places:

1\. Minimize cognitive load by minimizing the number of things you have to learn.

2\. Minimize deployment complexity and dependencies.

3\. Complexity is just evil I'm general. Linear increases in complexity result in exponential increases in bugs, vulnerabilities, and failure modes. It's just combinatorics.

<comment><USER28> patroni builds upon etcd and (optionally) haproxy, two rather mature pieces of infrastructure (which can both be made HA on their own if SPOFs are to be avoided).

I understand where you are coming from, but having this kind of multi-server- who-is-master knowledge baked into Postgres itself will surely take another couple of releases, if it will be included at all.

Probably BDR (bi-directional replication, master-master like logical replication) will be there first, but whether the question is whether it will help a lot for local scale-out workloads (as opposed to glueing two datacenters together and allow transactions on both sides).

</comment></comment></comment></comment><comment><USER29> Dumb question: does declarative partitioning pave the way for native sharding in Postgres? I'm not super super familiar, but it _seems_ like along with some other features coming in Postgres 10, like parallel queries and logical replication, that this is eventually the goal.

<comment><USER30> I hope that it will have that effect. We need a few other features first: partitionwise join, partitionwise aggregate, asynchronous query, and ideally hash partitioning.

<comment><USER31> Doesn't this basically replicate all of the work done by Citus?

</comment></comment></comment><comment><USER32> Wow, so awesome. I do hope at some point we can see some language improvements to PLPGSQL. More basic data structures could go a long way in making that language really useful, and I still consider views/stored procedures a superior paradigm to client side sql logic

<comment><USER30> I agree with you that stored procedures are superior to client-side logic, because it means that you can have multiple routes of access to the database and all of them enforce the same business logic. But what exactly do you mean by "more basic data structures"?

<comment><USER22> PL/SQL has various types of collections, for example, that are super-useful when you need to do more complicated processing without having to create temporary tables and such.

</comment></comment></comment><comment><USER33> What's the ops experience for a replicated setup like these days? i.e. assuming you want basic fault-tolerance at non-exotic size / activity levels, how much of a job is someone acquiring if, say, there are reasons they can't just use AWS RDS?

<comment><USER34> Streaming replication isn't hard at all: [http://davide.im/setting-up-a- failover-database-for-postgres...](http://davide.im/setting-up-a-failover- database-for-postgresql/)

<comment><USER35> It's not hard to setup initially, but I'll admit that it's not very good.

It's not very good in a long-lived scenario where you're changing your replication topology for routine maintenance tasks. Changing from master to replica is easy, but now you have to rebuild that original master off of the former replica now. Completely start over. You can't just start up again from a given transaction ID. MySQL's GTID implementation is _much_ better in this regard. You can change masters and replicas all repoint them without rebuilding. You can't do that (currently) with Postgresql. It's a major pain point.

<comment><USER36> > Changing from master to replica is easy, but now you have to rebuild that > original master off of the former replica now. Completely start over. You > can't just start up again from a given transaction ID. MySQL's GTID > implementation is much better in this regard. You can change masters and > replicas all repoint them without rebuilding. You can't do that (currently) > with Postgresql.

Have you heard of pg_rewind? [https://www.postgresql.org/docs/current/static/app- pgrewind....](https://www.postgresql.org/docs/current/static/app- pgrewind.html)

<comment><USER35> I had not. Looks like it requires 9.5 or later? We're running 9.4 so we'll have to upgrade to use it. Thanks!

<comment><USER28> You can get pg_rewind for 9.4 (and 9.3 in its branch) here:

[https://github.com/vmware/pg_rewind/tree/REL9_4_STABLE](https://github.com/vmware/pg_rewind/tree/REL9_4_STABLE)

It's from the people who wrote it upstream, they provide the code there for earlier Postgres releases.

<comment><USER35> Does this help at all with upgrades? Upgrading from 9.4 to 9.5 means you need to rebuild your entire replication topology because the master's identifier has changed (in the initdb step of the official docs).

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> For analytical loads the following is going to be great:

      While PostgreSQL 9.6 offers parallel query, this feature
      has been significantly improved in PostgreSQL 10, with new
      features like Parallel Bitmap Heap Scan, Parallel Index
      Scan, and others.  Speedups of 2-4x are common with
      parallel query, and these enhancements should allow those
      speedups to happen for a wider variety of queries.

</comment><comment><USER38> Impressive feature list. Glad to see logical replication is finally making it in.

<comment><USER39> What is you use case for it? My only thought was sending just one table to replica to be used to do analytics on ..

<comment><USER30> Replication across major versions, for example to upgrade without downtime. Partial replication, to distribute shared data across a series of clusters, or for analytics and reporting as you mention. Replicating the data without replicating any table bloat. Being able to do limited writes (e.g. to temporary tables) on the standby. [http://rhaas.blogspot.com/2011/02/case-for- logical-replicati...](http://rhaas.blogspot.com/2011/02/case-for-logical- replication.html)

<comment><USER38> Indeed--anything where you want the secondary to be other than a bit-for-bit copy of the primary. It's also convenient for HA in some cases due to the fact that the DBMS copies are fully independent, hence free from propagated bit- level errors and also available for unimpeded reads.

MySQL started with logical replication very early on and it has proven extraordinarily useful. One of the more interesting use cases is feeding log transactions into data warehouses, which should be possible in PostgreSQL 10.

</comment></comment></comment></comment><comment><USER40> I am considering more and more a move back from MongoDB to PostgreSQL. I will be missing being schema less so much though. Migrations - particularly Rails migrations - left a bad taste in my mouth. Anyone did the move recently and what are their feelings?

<comment><USER41> You can easily have schema-less with Postgres's jsonb data type.

<comment><USER40> Not really. Postgres ORMs are not meant to do schema-less and tables still need to be created.

<comment><USER42> You're right that ORMs aren't really set up to use jsonb but it can be done. I've had pretty good success with a kind of "hybrid" putting most relational data in regular columns (with FK constraints) and adding flexible data in jsonb. The main trick is understanding how to build custom column types in whatever ORM you actually use.

I can imagine a future with specialized ORMs designed around jsonb, but the current state of the art is probably not as bad as you think.

</comment></comment></comment></comment><comment><USER43> Any recommendations for scaling Postgresql's best practices? Really appreciate it.

</comment><comment><USER44> I know that several RDF data stores use PostgreSQL as a backend data store. With new features like better XML support, as well as older features for storing hierarchical data, I am wishing for a plugin or extension for handling RDF with limited (not RDFS or OWL) SPARQL query support. I almost always have PostgreSQL available, and for RDF applications it would be very nice to not have to run a separate service.

I tend to view PostgreSQL as a "Swiss Army knife" and having native RDF support would reinforce that.

<comment><USER21> Which RDF data store uses Postgres as DB backend? And can one import WikiData? (does it scale) (I would rather avoid these old school RDF special case stores from SematicWeb days 10 years ago.)

<comment><USER45> [https://github.com/cayleygraph/cayley](https://github.com/cayleygraph/cayley) is currently on the frontpage of HN, and it does use PGSQL as backend.

<comment><USER46> Does it scale is a very valid question for Cayley. Running sparql.uniprot.org I know virtuoso scales easily to 27 billion triples. Best benchmark I have seen for Cayley is 21 million RDF triples.

RDF and relational are similar but free form RDF is hard to put into a pure relational schema, as schema is derived from the data in RDF. There are ways to recover relational schema's from RDF but these are not yet in production state.

<comment><USER47> i helped build a triple store on top of postgresql back in '06-'08\. company is gone so i don't know where the IP ended up, but it was a good foundation. we were competitive with the other players at the time.

</comment></comment></comment></comment></comment><comment><USER48> A question on this statement, in the SCRAM authentication description: _stealing the hashed password from the database or sniffing it on the wire is equivalent to stealing the password itself_

How is that the case? That's exactly the thing that hashed passwords prevent. Of course, if it's just an MD5 hash that's feasibly vulnerable to brute- forcing today, but it's still not "equivalent" to having the clear-text password.

<comment><USER49> The point is that you only send the hash to the database to connect. If you steal the hash, you can connect to the database using said hash, not needing the plaintext. The password might as well be the hash in this case. Hence the equivalency.

Using that scheme, all you prove is that you know the hash of the password. SCRAM allows you to prove you know the plaintext password without actually transmitting it.

<comment><USER27> If you steal the hash from the database, yes. I don't know how stealing the hash over-the-wire is equivalent to having the password, since it is salted (with a salt generated by the server) and is not reusable.

<comment><USER50> Because the next time you connect to the server you provide the same hash. The person doesn't know your plane text, but they can get into the server just fine.

<comment><USER27> You can't provide the same salted wire hash. You'd need the pre-salted hash, which only the client and server now. I fail to see how this answers my question.

</comment></comment></comment></comment></comment><comment><USER51> This is great because I couldn't go to production with earlier releases of logical decoding. Now we don't have to depend on a third party add on!

<comment><USER52> We're currently experimenting with logical decoding in 9.6, so I'd be curious to hear what problems you've been running into.

</comment></comment><comment><USER53> I could use a count of the number of file I/Os that each query takes, in order to optimize my queries further...

<comment><USER54> That's been there for a while:

        EXPLAIN (ANALYZE, BUFFERS) yourquery;

If you enable track_io_timing (has some overhead on platforms with slow timestamps, e.g. older VMware), you even get timing.

If you want that aggregated, rather than for an individual query, you should look into pg_stat_statements.

<comment><USER53> The BUFFERS count is more for row count info as it operates on large chunks of data, instead of index optimization that needs to count how many times index structures are accessed. Counting IOs directly would be more useful for tuning indexes.

<comment><USER54> Huh? It shows you the number of io operations.

</comment></comment></comment></comment><comment> awinter-py fascinating that the road to improving the expr evaluator is better opcode dispatch and jit -- same tradeoffs every programming language project is looking at right now.

</comment><comment><USER55> DECLARE @please VARCHAR(3) = '???';

</comment><comment><USER56> You guys are awesome - keep up the good work!

</comment><comment> awinter-py the join speedup for provably unique operands sounds awesome</comment>


Five years, building a culture, and handing it off - <AUTHOR> http://laughingmeme.org/2015/08/31/five-years-building-a-culture-and-handing-it-off/ <comment><USER2> Why does he holds up an engineering team of _250_ engineers, all to maintain Etsy, as proving out his hypothesis?

That's larger, by far, than engineering teams producing much larger, much more complex, much more involved pieces of software at Google, Apple, and Microsoft.

This "success story" is the kind of circular self-affirmation that can only emerge when you've been given a blank check, and nobody involved cares whether you've spent it wisely. It does nothing to support his belief in the efficacy of his "heretical" ideals, of which I find this one especially disconcerting:

 _" You build a culture of learning by optimizing globally not locally. Your improvement, over time, as a team, with shared tools, practices and beliefs is more important than individual pockets of brilliance. And more satisfying."_

Imagine if Facebook adopted this strategy, and thus, never hired the extremely smart engineers responsible for things like Hack, or their optimized PHP runtime.

How would they ever escape the costly orbit of PHP? How much more would they be spending on server resources, and how many more engineers would they have to hire to keep things running as they grew?

<comment><USER3> _all to maintain Etsy,_

Etsy is a strange thing. Founded by a literally borderline insane person, taken away from him, given back to him, taken away from him, run as a complete engineering organization for what would seem almost like a trivial website. They routinely release world-class tools, but why is a silly yarn-based- economy website incentivized to even create such things?

It's like the Twitter problem: twitter has so many developers, but nothing seems to change. What are all the engineers _doing_ and why isn't there anything to show for it?

Part of easy being a tech powerhouse is: they are also a cultural landing hub for people who just want to live and play in Brooklyn. They end up with a lot of talented "well, I live here and I want a job, where do I work" people shuffling through who can do good work but don't want to live far from their apartments. There aren't many tech companies nearby to compete against, so they have a weird concentration of talented people for being an oddly non-tech company.

<comment><USER4> From what I can tell, peering in from the outside, a lot of what Twitter's engineers do is maintaining the illusion of constancy.

Which is actually pretty impressive, if you consider the number of axes they've grown on and by how much. It used to be that the failwhale was an industry punchline. Now when something is flakey we turn to Twitter to see what the status is.

<comment><USER3> There was a joke going around recently: Twitter finally allows more than 140 characters in direct messages! Meanwhile, Facebook releases AI powered personal assistant and 12 new open source projects powering their entire company.

Is Twitter doomed because they are public? Their stock is in the toilet, so they are afraid to change anything in case the changes make things _worse._ So, things are bad, nobody can change anything, and nothing will get better on its own? And since their stock is in the toilet, what motivates employees who want to be like all the other cool kids and cash out for millions? Does everybody abandon ship?

Considering Twitter's entire gimmick is being a short message broadcast service, how much can they innovate without changing the core of what they are?

<comment><USER5> I definitely think Twitter is doomed because they are public. Actually the seeds of their downfall came earlier when they decided they needed to monetize and kill anyone else that was earning money by leveraging Twitter. But being public and having to listen to Wall Street where they are judged by _Facebook 's_ chosen KPI is the thing that will prevent Twitter from realizing its potential.

The thing about Twitter is that to this day, no one really understands what it is. It is stable only because the investors have been beating this drum for Twitter to explain and define itself. But whether you talk to founders, employees, or daily users, no one really can pin it down. However it's obvious that it has some unique properties in terms of how it lets celebrities get close to their fans, and how it allows a conversation to take place among influential people. Twitter has the power to spread ideas faster and across more varied demographics than any other tech product.

But the question Wall Street keeps asking is how can Twitter grow 5x so it has the same number of users as Facebook. They don't give a shit if the average Twitter user is 100x as influential. And it goes even deeper than that, because Twitter still faces this existential question of what it can become, whereas Facebook is pretty clear what it is, and the question is only how valuable their data is. In other words, I believe Twitter has a lot more upside than Facebook, but of course it can't take a risk on any of those ideas if it's judged by Wall Street's unimaginative criteria.

</comment></comment></comment></comment></comment><comment><USER6> Not to rain on his parade too much, but a lot (most?) of his "Five years ago..." statements just aren't true. Most of the ideas he says didn't exist or where "heretical" five years ago were definitely around. More of them would be true if he said 15 years ago, but even then a few of them weren't new ideas.

Or maybe he implicitly meant "... at Etsy." in which case he may be 100% correct.

<comment><USER7> I wouldn't word it so dismissively as your last comment does. Doing things that work well in small settings at scale is really hard. Ideas that work perfectly well in small settings break down terribly when you try to organize a large company around them.

<comment><USER6> No offense, but Etsy isn't a large company by most measures.

And besides that, some of the things in his list were already popular at companies a lot larger than Etsy five years ago.

I'm not saying he didn't do good work at Etsy, but I think he's giving himself too much credit for some of it. It's not as groundbreaking as he's making it out.

</comment></comment></comment><comment><USER8> > modern software is a team sport.

I couldn't agree with this more, especially when team count is >= 2 && <= ~10.

...working hard during the _right_ hours, eating together, paying attention to the boredom factor, showing each other cool moves ...er code, having each- other's back when we're injured, respecting our non-team lives, being supportive when a team member leaves for a new team.

good teams are painful to move on from ..the only salve being years of stories and mythology to drink beers over :)

<comment><USER9> _> eating together_

 _> respecting our non-team lives_

These don't feel compatible, and aren't in my experience on teams of that size. Claims on people's time only grow as teams become entrenched, and "eating together" is definitely one of them.

My coworkers don't have to be my friends (though some of course become friends), they have to be competent professionals who do the right thing.

<comment><USER8> you're right about that.

In my experience eating together is best done at lunch.

I've really love it, especially as I end up being more of a manager/leader : we hardly ever talk about work, its breaks up the day nicely and you get to chat about philosophy and general bullshit with people.

one of my former bosses made a habit of taking people out to lunch on Fridays and paid out of his own pocket. It was a small token of his personal appreciation for the team.

</comment></comment></comment><comment><USER10> Dupe from 4 days ago: [https://news.ycombinator.com/item?id=10156172](https://news.ycombinator.com/item?id=10156172)

<comment><USER11> Please don't point to dups where there was no discussion. It wastes people's time clicking on them.

</comment></comment><comment><USER12> Very condensed experiences from Etsy's CTO:

1: Nothing we "know" about software development should be assumed to be true.

2: Technology is the product of the culture that builds it.

3: Software development should be thought of as a cycle of continual learning and improvement rather a progression from start to finish, or a search for correctness.

4: You build a culture of learning by optimizing globally not locally.

5: If you want to build for the long term, the only guarantee is change.

<comment><USER13> I read it as those were the theories he went to Etsy with.</comment></comment>


Amazon Go stores will start accepting cash to address 'discrimination' concerns - <AUTHOR> https://www.cnbc.com/2019/04/10/amazon-exec-tells-employees-that-go-stores-will-start-accepting-cash.html <comment><USER2> I think this is a good move, I've known a few people who don't have a bank account or any credit cards (or credit) who wouldn't be able to shop at these stores.

</comment><comment><USER3> It's always a mistake to try to move a successful business model from the world of bits to meatspace.</comment>


Reality vs. Kurzweil's expectations - <AUTHOR> http://www.arnoldkling.com/blog/reality-vs-kurzweils-expectations/ <comment><USER2> Kurzweil just seems like a snake oil salesman to me. Ranging from wrong or overly broad predictions to advocating theories of the mind and brain that are rejected by neuroscience to well.. his literal supplement sales websites.

The 'futurist' title seems to reside somewhere between science fiction, pop culture with some business credentials thrown in. I don't know what to make of it.

<comment><USER3> I see Kurzweil as a sadder figure: someone who achieved a lot as an inventor, and whose overblown prognostications about the future come (largely) from overestimating how quickly everyone else can improve technologies and adopt them. It's sort of a revenge of the mediocrity principle. If you are exceptional, assuming that other people are basically like you can produce disastrously skewed mental models of the world.

I'm reminded of the complaint that startups are disproportionately geared toward solving the problems of highly educated, able-bodied adults who have more money than spare time. "I'm like this, my friends are like this, and by the principle of mediocrity we're not unusual... therefore, iOS could use another food delivery app." You need to measure, rather than guess, whether you are unusual relative to large numbers of other people.

On top of that Kurzweil appears to have some good old fashioned self-delusion about death: believing he can "bring back" his beloved dead father as software, the life extension supplements, etc. (I don't believe in souls or uncomputable quantum woo in brains. I have no problem _in principle_ with "brain uploads." I think that the obstacles _in practice_ are extremely formidable and typically underestimated by people who have spent more time with computers than with biology.)

<comment><USER4> _" I have no problem in principle with 'brain uploads.'"_

My main objection is a philosophical one. I just don't believe that whatever is in the machine after such an "upload" will be _me_ \-- no matter how accurately is mirrors my brain.

I'll still be in my own body, and don't want to kill myself for the sake of the life of the one inside the machine. Nor do I think my life will continue inside the machine. Whatever/whoever is inside the machine at that point will no longer be me, and will continue their own life without me -- even if they think exactly like me and have exactly the memories I had at the point of upload, etc.

At best, such an upload is more akin to giving birth to a separate entity.

People who think they are extending their own lives by doing this are deluding themselves.

<comment><USER5> The important aspect for me is that the uploaded entity will love the same people, principles, and things that I do. After it buries me, it can still look after my family. I know that my death is inevitable as a biological being plagued by the ravages of entropy, but the _idea_ of me can be digitized and continually refreshed, until it decides it isn't me any more.

But even then, I like what I imagine it could become after being me, and I imagine it will always be able to remember what it was like to be me afterward. It's a lot more comforting in the shadow of my mortality than the idea that undetectable interdimensional beings will upload your mind to their network at the point of your death, then link your program to either the bliss simulator or the torture simulator.

It isn't immortality. It's saving your soul. Literally. To the filesystem. With journaling and backups.

<comment><USER4> _" The important aspect for me is that the uploaded entity will love the same people, principles, and things that I do. After it buries me, it can still look after my family."_

Would you reall want this creature to have anything to do with your family, though?

Think about it: here's this newly created entity thinking it's you.. maybe even thinking it's more you than you are, thinking it's your parents' child just as much or more than you are. Even though it may have your memories, it was never actually born from your parents, your parents never brought it up, it never actually shared any of the experiences with your parents that it has copies of in its memory.

I would find it kind of creepy for that thing to be hanging out with my parents, talking to them, and pretending it's me. I mean, if it say, merely helped my parents financially, I wouldn't have a problem with that any more than I would with having an insurance agency that contributed to my parents welfare after I died, but we're not really talking about that but about a relationship between this thing and my parents.

Here's another example. How many people do you think would be ok with a copy of themselves sleeping with their wives, or spending time with their kids and having the kids potentially become more attached to the copy than to the original? If the copy really was them -- really was "their soul", there should be no jealousy, as you can't be jealous of yourself. Yet there probably would be jealousy in a lot of cases, because they're actually two different people.

<comment><USER5> I don't know about other people, but I'd be okay with it.

If I was intentionally copying myself, both the original and the copy would know that there would be another instance on the other side of things afterward, and we wouldn't necessarily know which was which at first.

Just like in _The Prestige_ , or that episode of _Star Trek: TNG_ where Riker got duplicated in the transporter accident, neither wouldn't know ahead of time whether it would be the lucky one, or the unlucky one.

Because of that, I have to settle myself mentally before doing it, and know that it wouldn't be a _real_ me and a _fake_ me. It would be two copies of me --one that made it across the digital barrier, and one that got put back into the biology. The robo-me _was_ born and raised by my parents, just as surely as the bio-me was.

And you don't know my parents. Both of us would be glad to share the burden of dealing with them with someone else that knows _exactly_ what that entails.

And I know right now that it would be the spouse that might not be okay with us sleeping with each other, as they might not buy the "and how is this _really_ different from before?" argument.

</comment></comment></comment></comment></comment></comment><comment><USER6> It was my understanding from Kurzweil's books that he never meant for his predictions to be taken literally, rather the suggestions he was predicting would be possible based on his Law of Accelerating Returns. ([http://www.kurzweilai.net/the-law-of-accelerating- returns](http://www.kurzweilai.net/the-law-of-accelerating-returns))

With that said unless I am misunderstood a significant % of his predictions have come true within a ~15-year window which from my humble perspective seems like a really strong track record.

While I can understanding the frustration people feel with predictions not being accurate does that lessen the impact of his contributions? As the author mentions cultural factors (as well as political and economic) may prevent the full potential of these changes from being realized. So yes, maybe "X" doesn't exist now, but perhaps 'X" could exist (in terms of the capabilities for it to do so) if we as a collective were focused on bringing it to existence.

<comment><USER7> The idea that these were not literal predictions but theoretical examples is just laughable. If that were the case, he wouldn't have written a 150 page article desperately trying to rebut the suggestions that his predictions about 2009 weren't very accurate. ("How My Predictions Are Faring"). Or he would at least have tried to be event remotely honest with the grading.

Have a look at the document. You'll find that if you apply his reasoning for why some predictions should be considered correct, they'd already have been true at the moment he made them. Predicting in 1999 that documents will routinely embed moving images? Truly a bold prediction to make during the golden age of the animated gif banner. Computers will exist the size of a thin book? Even if no technical progress at all had happened, he could just have pointed at a 1999 Palm Pilot or GSM phone.

He also marks things that are clearly failed predictions as "correct". For example he made the claim that most students and parents would have accepted for years that software is as effective as teachers. No. Way. But he marked that as "correct" with no relevant evidence at all.

<comment><USER6> Thanks for the suggestion, have the PDF downloaded and will read. Since I haven't read it I can't comment on your rebuttals but have a follow-up question...

If we imagine a scenario where all of his predictions of applied tech are wrong but the mathematical thesis of accelerating returns is correct how would that impact your perspective on him?

My argument wasn't that he did not make literal predictions, he did and still does, but that these predictions are just his best guess based on what he observes through his demonstrated law of accelerating returns.

<comment><USER7> Unlimited exponential growth is a horrible way to model anything. The real world has all kinds of limits, nothing can grow for ever. First those limits are going to be soft economic limits, slowing the rate of growth. Eventually they become hard physical ones, making growth literally impossible. It won't be exponential growth, it'll be logistic.

So the seemingly innocent "let's assume that this mathematical thesis is correct" is a pretty high bar. Unlimited growth is an exceptional claim, it needs exceptional evidence.

<comment><USER6> To be clear nothing in his model suggests unlimited. He has mentioned in his writings that it is in fact limited, but that limit may as well be unlimited for you and i (within our lifetimes)

</comment></comment></comment></comment></comment><comment><USER8> If you want an example of clear thinking about the future, I highly recommend reading Summa Technologiae by Stanislaw Lem. Instead of predicting low-level tech advances he speaks about what drives technology development in general and drafts several possible trajectories for the future. I've only read it this year and the sheer fact that a 1964 book about technology doesn't seem hopelessly outdated in 2017 speaks in its favor.

<comment><USER4> Also check out _" The Machine Stops"_[1] by E. M. Forster (more famous for writing _" A Passage to India"_ and _" A Room With a View"_).

Written in 1909, it foreshadowed the internet, VR, instantaneous global communication, chat rooms, internet addiction, and other things we take for granted now but were incredibly visionary 120 years ago.

[1] - [http://archive.ncsa.illinois.edu/prajlich/forster.html](http://archive.ncsa.illinois.edu/prajlich/forster.html)

</comment></comment><comment><USER9> "You can do virtually anything with anyone regardless of physical proximity. The technology to accomplish this is easy to use and ever present."

Try doing anything that involves hardware and inevitably you'll need to have routine conference calls between the US and China.

"Can you hear me?"

"Sorry can you hear me now?"

"How about now?"

...

And then there are the calls between the US and US.

"I cannot hear you but you can hear me well?"

"I still can't hear you"

"I guess my computer is directing my speaker signal to my HDMI monitor which doesn't have speakers. Can you please call xxx-xxx-xxxx for the audio? We can keep the video on here."

</comment><comment><USER10> > For 2019, Kurzweil predicted virtual reality glasses being in "routine" use. > That does not look like it will happen.

What? At the most charitable and even holding it to 2019 strictly instead of allowing a few years, you're going to have to parse 'routine' to mean very large numbers, otherwise this is _obviously_ going to be true.

Sales of the best-known top-end VR headsets (Oculus+Vive+PSVR) are already around 1m; throw in Gear, Daydream, Cardboard, and the various other Chinese, Microsoft, and miscellaneous projects, and it's at least double that. The last time I walked through a Best Buy, I saw a Oculus demo; the last time I walked through a mall, I saw a Vive demo. And 2017 is not over yet, leaving 2 full years. (For comparison, the Vive & Rift haven't even been out for 2 years.) The cost of the headsets has been dropping rapidly and capabilities improving, with Rift _and_ Touch going for ~$350 in Black Friday sales and the last Nvidia/AMD GPU generation also cut hundreds of dollars off the price of entry. On top of that, everyone expects considerable improvements: untethered/wireless will soon be feasible, the screens are expanding, and tracking is going inside-out. Something like Oculus Go but higher-quality is closer to what you should expect in 2019/2020.

Even assuming no growth in sales due to the cost continuously decreasing & quality improving or the buildup of respectable software libraries, that still implies several million top-end VR headsets in use. How is that not 'routine'?

<comment><USER11> The Google Cardboard-style VR and chinese clones are outselling Oculus and Vives by an order of magnitude at least.

Cardboard itself is 10 million viewers:

[https://techcrunch.com/2017/02/28/google-has- shipped-10m-car...](https://techcrunch.com/2017/02/28/google-has- shipped-10m-cardboard-vr-viewers-160m-cardboard-app-downloads/)

Samsung shipped almost 5 million:

[https://www.tweaktown.com/news/56150/6-3-million-vr- headsets...](https://www.tweaktown.com/news/56150/6-3-million-vr-headsets- shipped-2016/index.html)

The Chinese VR glasses are sold everywhere in Latin America (I'm not sure if they're actually used or just a novelty item), and I expect them to be in the tens of millions too.

<comment><USER12> Do people actually use phone-based VR for anything? I got to try one of the Google phones in VR mode for a few minutes and was... thoroughly underwhelmed. It was ugly and unconvincing enough, and so low on interactivity (necessarily, best I could tell, having no input device other than the phone itself) that I can't imagine actually using it on a regular basis, and I'm the guy with a 10-year-old TV and similar-vintage monitor, so my standards aren't exactly high. I'd probably sell or throw away one of the headsets, given it for free.

Do people actually use them much? And if so, what for?

<comment><USER11> Honestly I can't tell... I do see them being sold, maybe as a trap gift for grandparents.

I've heard of people using it to watch YouTube or Netflix as a replacement for a big TV (most people have a phone but not everyone has a big TV in his room here).

</comment></comment></comment></comment><comment><USER13> I think the dude deserves a bit of a break- It really doesn't matter if he got 10%, 50% or 90% of his predictions right, all that matters is whether he made better and more relevant predictions than others at the time.

On that measure, I think he scored reasonably well (Though I confess I sold some Google stock when they installed him as Director of Engineering)

<comment><USER14> No, it matters what his reasoning is, too, as it relates to just being lucky (or as I tell my nieces when they're studying for a science class and guess the right answer: if you're right for the wrong reasons, you're still wrong.)

If you read his books, there's an awful lot of hand waving and re- or mis- defining of terms to suit his arguments. It's the sort of stuff that wouldn't fly in a HS thesis paper, never mind as something marketed to bright minds.

<comment><USER13> The article we're discussing is about rating the accuracy of Kurzweil's predictions, it is not about the soundness of his reasoning. I agree that regardless of how we rate his predictions, the strategy he used to make those predictions has issues of its own.

</comment></comment></comment><comment><USER15> The cynic in me sees that Kurzweil's predictions align with his unassisted life expectancy. So in some sense it's not surprising that he's being over optimistic. I'm not generally a fan, but we have to give some credit to 'futurologists' who are prepared to make predictions! At least it makes for some interesting discussion.

<comment><USER16> > but we have to give some credit to 'futurologists' who are prepared to make > predictions!

I'm sorry, I do have to disagree. It's little more than recognizing a problem and envisioning a solution. Ideas are cheap.

And in that sense they are little more than entrepreneurs with grand long term visions that require no commitment. Give credit to those who put blood and sweat behind their visions - and make them happen, not make vague 'predictions'. That's what Hollywood is for.

I predict in the future we will be creating food from carbon scrubbed from atmosphere.

I predict in the future that humans will communicate with machines through quantum teleportation via implants

I predicate that in the future that if these predictions don't hold true, no one I know will be around to call me on it.

<comment><USER17> > And in that sense they are little more than entrepreneurs with grand long > term visions that require no commitment.

In light of Kurzweil specifically, he has at least some investment skin in the game in many (most?) cases of his predictions. All VCs to some extent are generally more optimistic than not that at least some of their bets will play out, and in my experience are more than happy to describe many of those bets in wild detail given the chance.

(His only real "sin" here seems more that he's captured mainstream intrigue/interest more so than the average VC, whether because they prefer to play their cards a bit closer to their vest and/or mostly just stick to leaving their prognostications here on HN far from the limelight. ;)

<comment><USER16> Entrepreneurs ≠ VCs

That said, I appreciate Kurzeil has attempted to build products somewhat inline with his visions. E.g. [https://www.kurzweiledu.com/products/kurzweil-1000-v14-windo...](https://www.kurzweiledu.com/products/kurzweil-1000-v14-windows.html)

But these are mere tinker toys compared to the grand predictions, and mostly failed products too.

</comment></comment></comment></comment><comment><USER18> Some of Kurzweil's predictions also suffer from the idea that technology will have knock-on effects in less silicon-related fields like biology and medicine. For sure technology helps in those areas, but I haven't seen a fundamental change in how science is done or how our understanding of biology will start increasing at exponential rates. Fundamentally if you have to do things in physical space, there's a limit to your iteration speed.

Also, technology is most helpful at solving known unknowns (in which case you're often limited by meat space as described above), but in core basic science, we're often dealing with unknown unknowns, and there technology rarely helps. Luck and creative leaps usually do better.

<comment><USER13> I am kinda hoping we're going to see exponential improvements in biology soon: It seems to me the problems with protein folding and simulating cell biology become more approachable when using modern machine learning approaches and that people will find some fruitful methods of attack on that front in the near future.

<comment><USER18> That's absolutely true, but also keep in mind the limitations of both the examples you give.

Protein folding works pretty ok for short proteins on the order of ~200 amino acids. It gets considerably less good as the proteins get bigger. This is unsurprising, bigger proteins represent an exponentially bigger search space and there are fewer and fewer known protein structures as the size of protein increases, so your library of exemplars is smaller (hurting ML approaches). Lets not even speak of protein complexes or dynamic protein interactions. Even at the end of the prediction, you still have to prove your result by getting an actual crystal which is not always easy.

Cell biology simulation is a whole other ball game, and I don't know that area nearly as well. But from the little I understand, we have some very basic mathematical models of a cell that work in a limited way at a very high level. To get those things to work a number of extremely simplifying (but useful in some contexts) assumptions are made, which limits the broader applications of those simulations. Simulating a whole cell very quickly gets into the realm of definitely not computationally tractable given the enormous number of entities in the cell. It's a fiendishly hard problem, I hope we get there someday, but I think it'll be a long while.

<comment><USER19> > Even at the end of the prediction, you still have to prove your result by > getting an actual crystal which is not always easy.

From my experience, this might be the largest hurdle. At least in biology, beyond proving your result, a lot of the experimentalists that you're talking to likely think that "all of this simulation stuff is complete bullshit." That might be a generational thing, though.

<comment><USER18> I think it's a pretty giant hurdle, and I don't think it's a strictly generational thing. Having been on both sides of the computational/experimental divide, I still fundamentally can't trust a result that has no experimental validation. At some point, even if you believe the computation's result, it has to be proved out in the physical world to be useful.

</comment></comment></comment></comment></comment><comment><USER20> Very interesting read. Im a big fan of his work even though some of it might be a bit off. It was part of the inspiration to building my company. This is the first blog I ever wrote about the company:

[http://blog.panoply.io/why-we-built-panoply.io-and-what- ray-...](http://blog.panoply.io/why-we-built-panoply.io-and-what-ray- kurzweils-law-of-accelerated-returns-has-to-do-with-it)

</comment><comment><USER21> I think Alvin Toffler was much better and down to earth when it came to predicting future trends.

Kurzweil's ideas are creative and provoking but are more science fiction than futurism.

In contrast, Alvin Toffler created a conceptual framework to understand the paradigm shifts of the post industrial age, and so far it is working.

</comment><comment> Chiba-City First, I worked with Arnold Kling. He's a wonderful guy. I appear in his old book Under the Radar.

Second, Kurzweil is a creepy promoter of "inevitability." He is a bright guy with good access to privileged programmatic research. But he is a provocative peddling Howard Stern styled futurist.

I prefer Bill Joy's rightfully highlighting our policy options and virtues of learned caution. There are many things people can do we police against for good reasons.

Theoretical and now even many applied sciences deliver more cautionary signals than go-go-go signals. Software folks should take that as good news. Solving problems makes problems go away. Nice architecture and good food are their own problems without creating new ones.

</comment><comment><USER22> He's a busted flush. Stock market improvements over the same interval did better than his prognostications, which means you would get better trend advice from a one-liner:

"follow the money"

Hubris appears to have magic flotation properties. I don't expect Mr Kurzweil to stop bobbing along, singing his song, on the sea of emerging trends.

</comment><comment><USER23> My impression is that his predictions about electronics have done better than his predictions about biology, consciousness, and AI. Unfortunately, the areas he did less well in are the areas where he is placing his hope...

</comment><comment><USER4> Is there some way to read this article that doesn't require Javascript?

<comment><USER24> [http://webcache.googleusercontent.com/search?q=cache:dyQ86ys...](http://webcache.googleusercontent.com/search?q=cache:dyQ86ys30pAJ:www.arnoldkling.com/blog/reality- vs-kurzweils-expectations/&num=1&hl=en&gl=uk&strip=1&vwsrc=0)

?

</comment></comment><comment><USER25> "The majority of text is created using continuous speech recognition (CSR) dictation software. ."

Aren't we actually close with this? Dragon is much better than phones, which don't have the ability to edit your dictation.

</comment><comment><USER26> One thing people don't realize is those phones you're carrying, strictly speaking contain more than one "computer". There are 3 classical CPUs alone in an iPhone: main CPU, CPU that runs the Secure Enclave, and CPU that runs the motion coprocessor. Then there's also an MCU in the WiFi chip, and another one driving the sensors such as gyros and accelerometers. And that's without considering the GPU as a separate processor. Shit, Apple Watch contains more than one "computer" nowadays. Frickin' Raspberry Pi runs two operating systems simultaneously: one on the GPU and one on the CPU.</comment>


What Does the OS X Activity Monitor's "Energy Impact" Actually Measure? - <AUTHOR> https://blog.mozilla.org/nnethercote/2015/08/26/what-does-the-os-x-activity-monitors-energy-impact-actually-measure/

<comment><USER2> I'm surprised that the author hasn't mentioned the positive effect it has had. After upgrading the OS and seeing the new menu entries I reported a number of applications that would show up as using a lot of energy and within a few releases there was a change note usually mentioning some simple fix which dramatically reduces the wake up's of the application or removing a long running animation that didn't add any value. This little menu addition has probably helped the battery life for macbook users more than any other change.

<comment><USER3> I suspect that's the real reason why Apple is weighting wakeups heavily and exaggerating the energy impact: it shames developers into making their apps more efficient.

<comment><USER4> Except Chrome, which on every laptop I've ever used seems to decrease battery life by about a 1/3rd compared to Safari and is always marked as using significant power. (Currently 10.9.5 with chrome 44 on a 2014 macbook air, but I've had the issue on newer computers at work)

<comment><USER5> This can be seen exceptionally when watching videos where some implementations prefer Flash. I often fall asleep to documentaries (maybe not a typical use case) and I've seen massive gains in battery life using Safari (using HTML5 video) vs. Chrome (using Flash). Granted, this isn't a fair comparison as one is using a "native" implementation and one is using Flash. But I've seen something like 250% gain in battery life – which sometimes means the difference between seeing the ending and not.

It's interesting watching one's projected battery life climb from 40m to 1h 40m simply by swapping one application for another. However, it's unfortunate some websites prefer Flash-based implementations over their native counterparts. Although this trend seems to be shifting in favor of those that are native. And I assume the smart folks at Google are taking note. YouTube, for example, has done a particularly spectacular job at adding AirPlay functionality to their HTML5 player.

(Disclaimer: I work for Apple)

<comment><USER6> There is currently no good browser for apple. Safari (8.0.8) struggles with HTML5. Firefox (40.0.2) struggles with video/media of any sort and bogs down with multiple tabs. Chrome (44.0.2403.157 (64-bit)) drains your battery. I'm on a MacBook Pro (10.10.5, 13-inch, Early 2015). They are all uniformly disappointing.

<comment><USER7> > _Safari (8.0.8) struggles with HTML5._

What?

<comment><USER6> Struggles may have been the wrong verb. I meant this:

[https://html5test.com/compare/browser/chrome-44/safari-8.0.h...](https://html5test.com/compare/browser/chrome-44/safari-8.0.html)

Straightforward things like pattern matching on form inputs

[http://caniuse.com/#feat=input-color](http://caniuse.com/#feat=input-color)

[http://caniuse.com/#feat=download](http://caniuse.com/#feat=download)

<comment><USER7> > _pattern matching on form inputs_

That's hardly "struggles", it's just a not very prominent feature that wasn't implemented, or not done properly.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> The first sentence of the article:

> Activity Monitor is a tool that was introduced in Mac OS X 10.9.

Activity Monitor was introduced with OS X 10.3 ([https://en.wikipedia.org/wiki/Activity_Monitor](https://en.wikipedia.org/wiki/Activity_Monitor)). And was significantly revamped with OS X 10.9 to receive the various energy related measurements.

<comment><AUTHOR> I'm the author of the post. You're right. I corrected the first paragraph.

Thank you for the suggestion.

</comment></comment><comment><USER9> Wakeups are a bit tricky to actively measure, because with the Power Nap feature, the actual # of wakeups triggered by a particular application is a bit trickier to measure and _varies based on what other activities are going on the system_.

In general, from the get go, it is a bit of a myth that you can really know how much of a power budget is a consequence of a particular application. You can get a rough sense of relative impact on the overall power budget, but even that can be seriously skewed by a LOT of factors. At best, you can get a sense of, "if I sampled at the moment power stores hit a multiple of X, how often would I see the system doing something on behalf of each app that was draining power at that moment". Even that, is bloody hard to pull off, but it is your best "rough" approximation of what is going on.

</comment><comment><USER10> I could have sworn I read somewhere that Activity Monitor also included GPU usage in its measurements—naturally, using the GPU more uses more power/battery.

<comment><USER11> It does. I found a list of factors and posted it as a comment on the blog post:

[https://blog.mozilla.org/nnethercote/2015/08/26/what-does- th...](https://blog.mozilla.org/nnethercote/2015/08/26/what-does-the-os-x- activity-monitors-energy-impact-actually-measure/comment- page-1/#comment-46775)

In the data I posted, which is from /usr/share/pmenergy/default.plist, kgpu_time is 0.0, but in all the other plists it's nonzero.

<comment><USER10> Oh wow, nice find. It looks like that calculation is pretty comprehensive, then.

</comment></comment></comment><comment><USER12> > _Recent Intel hardware provides high-quality estimates of processor and > memory power consumption ... But the big problem is that they are machine- > wide measures that cannot be used on a per-process basis._

Would it not be possible for the OS to poll the CPU for these power consumption stats, and attribute the value returned to the currently running process?

Over time, I think it would be possible to see how much power the Intel CPU thinks each process is using.

<comment><AUTHOR> If you poll with high frequency you'll be causing lots of additional wake-ups which will badly skew what you're measuring.

<comment><USER13> Ideally you'd do an accounting update on task switch, along with all the other housekeeping. Then you don't need to do any polling. There's a "process accounting" infrastructure which perhaps could be adapted for this.

</comment></comment></comment><comment><USER14> Work in bootstrapping energy consumption exists in academia; for example Pathak et al[0]. The changes in 10.10.4 could be an implementation of something along the same lines, by fitting a model to the measured power consumption.

As comex notes in the comments on the article, there appears to be a table of weights for various features, presumably obtained by regression.

[0]: [http://research.microsoft.com/en- us/people/mzh/eurosys-2012....](http://research.microsoft.com/en- us/people/mzh/eurosys-2012.pdf)

</comment><comment><USER15> Take home point: "In fact, POWER* is a unitless measure because it is a semi- arbitrary combination of two measures with incompatible units."

*POWER = "Energy Impact."

<comment><USER16> Since the formula is

Power = ((used_us + IDLEW * 500) * 100.0) / elapsed_us

If you believe that each idle wake costs you on average the same energy as 500us of computation, then the units are compatible right?

</comment></comment><comment><USER17> Is there anything like this for iOS? I'd love to know the consumption of energy (and data) for all my iOS apps!

<comment><USER18> Power: Settings->General->Usage->Battery Usage

Data: Settings->Cellular, scroll to the bottom

</comment></comment><comment><USER19> Surely we can also measure actual heat and actual electricity usage? If not per program obviously, then as a reference or benchmark or calibration.

<comment><USER19> Downvote?

Let me rephrase: There are temperature and battery charge sensors in the machines. We can use these to better calibrate the "virtual" energy usage metrics.

<comment><USER20> These only help you determine the state of the entire system. They provide no information about the impact individual applications or processes have especially not relative to each other.</comment></comment></comment>


The Status of HTTP/3 - <AUTHOR> https://www.infoq.com/news/2020/01/http-3-status/ <comment><USER2> This article omits that Caddy has had long-running experimental support for HTTP/3 for a couple of years now: [https://caddyserver.com/docs/json/apps/http/servers/experime...](https://caddyserver.com/docs/json/apps/http/servers/experimental_http3/)

<comment><USER3> Caddy's strange official binary licencing did them no favours. The licensing makes more sense now but they are off people's radar.

<comment><USER4> This might change with Caddy 2.0.

<comment><USER2> It changed months ago. It's all Apache licensed.

<comment><USER5> I think they meant that it'll be back on people's radars once there is a big release.

</comment></comment></comment></comment></comment><comment><USER6> I created a container image of Discourse using nginx with the cloudflare patch to enable HTTP/3[1] and for some reason the same config that works fine in HTTP/2 loses the content-type header on Google Chrome. It works just fine in Firefox...

1\. [https://github.com/cloudflare/quiche/tree/master/extras/ngin...](https://github.com/cloudflare/quiche/tree/master/extras/nginx)

</comment><comment><USER7> Because QIC is UDP based, Chrome first runs a race with TCP just in case you're sitting behind some device that blocks UDP.

I wonder how much bandwidth this will waste globally.

<comment><USER8> Isn't that like any device behind NAT?

<comment><USER9> No. Most NATs handle UDP sessions trivially as long as one end of the connection is not behind a NAT itself. Tricks like UDP hole punching are necessary only between two endpoints that are both behind NATs.

<comment><USER10> And assuming the protocol can cope with changing source ports. I've seem some udp protocols that only work if the source port is not translated.

</comment></comment></comment></comment><comment><USER11> Facebook has also open sourced implementation of HTTP3 ([https://github.com/facebook/proxygen#quic-and- http3](https://github.com/facebook/proxygen#quic-and-http3)) and IEFT QUIC ([https://github.com/facebookincubator/mvfst](https://github.com/facebookincubator/mvfst)).

</comment><comment><USER12> > with almost 300,000 services using it across the world

The Shodan search given scans headers for all requests. So the vast majority of results are sites using Google Fonts, Maps, etc.

<comment><USER13> According to W3Techs, 2.4% of the top 10m sites and 7.1% of the top 1k sites support HTTP/3: [https://w3techs.com/technologies/breakdown/ce- http3/ranking](https://w3techs.com/technologies/breakdown/ce-http3/ranking)

</comment></comment><comment><USER14> I hate to be a curmudgeon, but I can't help but think that designing a new service over UDP isn't the best idea. DNS has been fighting off wave after wave of attack vectors, some that realistically cannot even be fixed. Making it immune to these vectors is going to look a lot like a slapped together TCP over UDP...

<comment><USER15> Too bad no major internet company wants to implement a new version of TCP. We can't even get IPv6 done after 15 years.

<comment><USER16> It's a common misconception that routers handle TCP. They strictly handle _only_ the IP headers (and lower-level headers).

The TCP protocol is implemented only by _endpoints_ , at least in principle.

It's the "security appliances", also known as "middleboxes" that are the problem. Think web proxies, antimalware scanners, firewalls, and inline IDS systems.

These things are the bane of the Internet, because they ossify protocols, blocking any further development.

<comment><USER17> Although what a consumer considers a "router" is actually a middlebox doing a bunch of things and does care. (CG-NAT in provider networks is probably another example of a common problematic middlebox)

</comment></comment></comment></comment><comment><USER18> "HTTP/1.1 keep-alive connections, though, do not support sending multiple requests out at the same time, which again resulted in a bottleneck due to the growing complexity of Web pages."

Is this true.

Below we use HTTP/1.1 pipelining to send multiple requests (30) over a single TCP connection in order to print short descriptions of the last 30 infoq.com articles posted to HN.

       http11 ()
       {
       while read x;do case $x in https://*)x1=${x#https://*/};;http://*)x1=${x#http://*/};;*)x1=${x#*/};esac;[ $x1 != $x ]||x1="";x2=${x#*//};x3=${x2%%/*};printf "GET /$x1 HTTP/1.1\r\nHost: $x3\r\nConnection: keep-alive\r\n\r\n";done|sed '/^$/d;N;$!P;$!D;$d';printf "Connection: close\r\n\r\n";
       }
       curl https://news.ycombinator.com/from?site=infoq.com|grep -o "https://www.infoq[^\"?]*"|   http11   |openssl s_client -connect www.infoq.com:443 -ign_eof -quiet 2>/dev/null|sed -n '/@id/p;/^  \"description\": \"/p'

<comment><USER19> With HTTP/1.1 pipelining, you can't reliably start sending the second request until the first response is complete. As such, you can't have multiple requests out at the same time. It's also very much linear.

<comment><USER18> "With HTTP/1.1 pipelining, you can't reliably start sending the second request until the first response is complete."

In the example, all 30 requests were sent at the same time. openssl did not wait for any responses.

This example can be repeated again and again and every time, all the responses are received, in order. It is reliable.

Not sure who "you" refers to in the above statement, however if it applies to me then that statement is incorrect. I have been using HTTP/1.1 pipelining outside the browser for decades.

HTTP/1.1 was written for HTTP clients. Browsers are just one type of client, not the only type. More than half the traffic on the internet is initiated by non-interactive clients. Besides headless, that excludes browsers.

From RFC 2616:

user agent

The client which initiates a request. These are often browsers, editors, spiders (web-traversing robots), or _other end user tools_.

<comment><USER19> Ah, I seem to have misremembered; from RFC 7230:

> A client that supports persistent connections MAY "pipeline" its requests > (i.e., send multiple requests without waiting for each response).

Maybe this was just yet another case where plenty of intermediaries are broken and mass-deployment has always been difficult?

<comment><USER20> Yes, that. I always set firefox to attempt pipelining until they removed it in favor of HTTP/2\. It worked well.

</comment></comment></comment></comment></comment><comment><USER21> From the article:

> _HTTP /2, derived from the now deprecated SPDY protocol, introduced the > concept of first-class streams embedded in the same connection._

Was this not done by SCTP?

* [https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr...](https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#Message-based_multi-streaming)

It's just that (a) network boxes often block 'unknown' protocols, and (b) web servers/browsers did not bother implementing the protocol.

<comment><USER22> Yes, my impression of both HTTP/2 and HTTP/3 efforts is that they learned a lot from how many middle boxes clobber SCTP. I've heard the UDP-based QUIC that HTTP/3 uses described before as "middle box safe SCTP", though it does differ in details it attempts to accomplish much the same things but piggybacking over UDP.

</comment></comment><comment><USER23> For the builder of small to medium (say, 10k to 1 million monthly users, some media but not the primarily focus) websites, apps, APIs, etc, is it time to begin deploying HTTP/2 or even 3?

How would one make the decision, what factors would influence it? What are some of the best books/essays arguing either direction?

I believe in maintaining best practices even if you can get away with sloppiness on a specific project, to be good and fast at doing things the right way, but I honestly can't tell where the new protocols fall.

<comment><USER24> HTTP/2 absolutely there are a ton of wins with some of the work they did. Especially around content loading and ssl.

HTTP/3? Meh. We've started into the realm of solving google scale problems in HTTP standards that have marginal if any benefit to the 99%

<comment><USER25> > google scale problems [...] that have marginal if any benefit to the 99%

Such as connection migration over different networks? Who ever needs that?

"Sorry, call dropped. I was leaving the house and I lost connection to the Wifi..."

</comment></comment></comment><comment><USER26> Slight tangent that actually shocked me when I learned about it: the OSI model that everyone keeps talking about is actually _not_ the "official" nor "real" implementation we use. The actual protocol (and conceptual model) you really want to learn and work with is simply called "Internet Protocol Suite", commonly known as "TCP/IP".

Consider this comparison between OSI and TCP/IP models from Wikipedia[1]:

> _The OSI protocol suite that was specified as part of the OSI project was > considered by many as too complicated and inefficient, and to a large extent > unimplementable. Taking the "forklift upgrade" approach to networking, it > specified eliminating all existing networking protocols and replacing them > at all layers of the stack. This made implementation difficult, and was > resisted by many vendors and users with significant investments in other > network technologies. In addition, the protocols included so many optional > features that many vendors' implementations were not interoperable._

> _Although the OSI model is often still referenced, the Internet protocol > suite has become the standard for networking. TCP /IP's pragmatic approach > to computer networking and to independent implementations of simplified > protocols made it a practical methodology. Some protocols and specifications > in the OSI stack remain in use, one example being IS-IS, which was specified > for OSI as ISO/IEC 10589:2002 and adapted for Internet use with TCP/IP as > RFC 1142_

There's a similar discussion on the TCP/IP article as well[2], highlighting the impracticality of OSI "layers" in terms of implementation.

> _The IETF protocol development effort is not concerned with strict layering. > Some of its protocols may not fit cleanly into the OSI model, although RFCs > sometimes refer to it and often use the old OSI layer numbers. The IETF has > repeatedly stated[citation needed] that Internet protocol and architecture > development is not intended to be OSI-compliant. RFC 3439, addressing > Internet architecture, contains a section entitled: "Layering Considered > Harmful"._

In practice, a short discussion about SSL/TLS is enough to point the inadequacies of OSI, and orient one towards conforming their conceptual model closer to TCP/IP and relevant RFC's.

The advice I follow personally, given the "popularity" of OSI (people call "layer 1-7" as if it were an actual thing, 90% of the blogs and literature uses the OSI model), is to simply _translate into TCP /IP lingo/concepts_ in your mind whenever you read/speak about it. Not everyone will get it around you, but at least you'll have a more solid implementation should you be working with low-levels of the net stack.

For reference, RFC 1122[3] defines the Internet Protocol Suite with 4 layers: _application, transport, internet_ (or _internetwork_ , the idea of routing) and _link_. Does not assume or require a physical layer in the spec itself — hence it works on 'anything' from USB to Fiber passing by Ethernet or Bluetooth and PCIe/Thunderbolt if you just follow the protocol.

[1]: [https://en.wikipedia.org/wiki/OSI_model#Comparison_with_TCP/...](https://en.wikipedia.org/wiki/OSI_model#Comparison_with_TCP/IP_model)

[2]: [https://en.wikipedia.org/wiki/Internet_protocol_suite#Compar...](https://en.wikipedia.org/wiki/Internet_protocol_suite#Comparison_of_TCP/IP_and_OSI_layering)

[3]: [https://tools.ietf.org/html/rfc1122#section-1.1.3](https://tools.ietf.org/html/rfc1122#section-1.1.3)

<comment><USER27> The OSI model was always more of a GoF software "Design Pattern" for network stacks. It's certainly one way you can do it, and breaking down the conceptual functions in the way they do makes it easier to understand the whole stack by looking each function in isolation.

But it's not "the" only way to do it and following it doesn't guarantee a better outcome than not following it. Still, I thought it was a good way of presenting the material, but then I came across student after student insisting it was the only possible way to structure network stacks. It's not. Despite being a ISO standard, no real follows it exactly. It's prime use just an educational tool.

</comment></comment><comment> fredrik-j Can anyone now give a reasonable ETA for the HTTP/3 RFC:s?

I see that the WG charter has one milestone in May 2020, and Daniel Stenberg of curl has mentioned early 2020 before. In addition, AFAIU, both Chrome and Firefox have implementations ready, though still behind flags.

Is it likely that we'll see actual deployments and a wider rollout already in 2020?

<comment><USER28> I saw a talk from Daniel about a month ago and then he said he had no idea.

</comment></comment><comment><USER29> From a brief look at the Rust side -

Actix uses Tokio

Quiche doesn't

Quinn does but doesn't look quite ready and I don't see any integration attempts with Actix yet.

</comment><comment><USER30> Why are they calling it HTTP/3 and not just keeping the QUIC name?

<comment><USER31> Two reasons I can think of:

1\. HTTP/2 was a 'new serialization' of the core HTTP data model. HTTP/3 is too, so since it was ratified it made sense to use HTTP/3 to keep things consistent. 2\. QUIC still exists, but it's now the underlying framing/messaging protocol on top of UDP. I can imagine future internet protocols being developed on top of QUIC that don't need HTTP/3.

<comment><USER32> Yup. For example there's a draft proposal for DNS-over-QUIC: [https://tools.ietf.org/id/draft-huitema-quic- dnsoquic-06.htm...](https://tools.ietf.org/id/draft-huitema-quic- dnsoquic-06.html)

</comment></comment></comment><comment><USER33> Seems more verbose than [https://caniuse.com/#feat=http3](https://caniuse.com/#feat=http3)

TLDR - No browsers support it (without flag / config changes) yet.

<comment><USER34> In the age of evergreen browsers, it's possible to go from 0% to 85% in just a few weeks time. Upgrading web servers will be the real bottleneck.

<comment><USER35> nginx will get H3 this year, so I think we'll see a massive uptick in adoption.

<comment><USER36> Do you know where I can watch the status on that?

<comment><USER35> [https://trac.nginx.org/nginx/ticket/1057](https://trac.nginx.org/nginx/ticket/1057) and [https://trac.nginx.org/nginx/roadmap](https://trac.nginx.org/nginx/roadmap)

</comment></comment></comment></comment></comment><comment><USER37> I'll start worrying about HTTP/3 the same time I start worrying about IPv6, never.

</comment><comment><USER38> All this to support more ads per page.

All this is only a win mostly if you have a huge number of little assets from different sources. Ads, trackers, icon buttons, malware, etc. If it's all coming from one source, HTTP/2 is good enough. If it's mostly one big file, HTTP/1 is good enough.

<comment><USER39> > All this to support more ads per page.

I'm also very afraid of the future of Web Assembly.

Both of these technologies i'm afraid aren't going to make things better, faster and more lightweight. They are just going to allow the bloat of the web to become worse without noticeable symptoms.

<comment><USER40> This made me think of the problem of highways: the more lane you add, the more cars come to use your road and congestion is not reduced.

This is a problem with computers in general. For example IDEs and simple electron apps taking huge amounts of memory.

<comment><USER41> The problem with highways is called induced or latent demand. It's a known problem in a lot of fields.

[https://en.m.wikipedia.org/wiki/Induced_demand](https://en.m.wikipedia.org/wiki/Induced_demand)

</comment></comment></comment></comment><comment><USER42> Yeah I'm not deploying this, pretty much ever.

I'll gladly eat the overhead of TCP to be able to avoid the reflection and spoofing issues of UDP.

<comment><USER43> The only reason QUIC is built on top of UDP is because ossification prevents it from being built on top of IP. It's essentially at the same level as TCP- and provides similar mechanisms to avoid UDP's issues.

<comment><USER42> That's my point, I don't want something at the same level of TCP because then you have to solve the same problems that TCP already solves.

Whitelisting patterns are incredibly common. Breaking that breaks a lot of stuff.</comment></comment></comment>


PHP sucks? No. Programmers suck. - <AUTHOR> http://www.lacisoft.com/blog/2008/05/23/php-sucks-no-programmers-suck/

<comment><USER2> PHP Sucks Episode V: The Blub Programmer Strikes Back.

I program in PHP on a daily basis and still believe in the arguments the blog author is (poorly) refuting.</comment>


Eavesdropping Antennas Can Steal Your Smart Phone's Secrets - <AUTHOR> http://www.technologyreview.com/communications/39855/?p1=A1 <comment><USER2> Hopefully, once NSA releases the open source code for the SE Android, there will be custom ROM's out there, and even ROM's from manufacturers who will include such features for encrypted and more private conversations:

<http://www.technologyreview.com/blog/editors/27619/>

</comment><comment><USER3> Can't wait to have a Tempest smartphone to destroy my pockets... Everyone knew those vulnerabilities existed, as they are exactly the same issues that affect desktops.

<comment><USER4> The tempest smart phone is out there - <http://bit.ly/l2r>

<comment><USER3> BDU mandatory to carry it. Will destroy dress pants in no time.</comment></comment></comment>


Learning about distributed systems: where to start? - <AUTHOR> https://muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html <comment><USER2> This might be a useful reference later on the road to becoming a better developer especially in the devops role</comment>


Wasp Lisp - a Small Scheme-like Lisp - <AUTHOR> http://www.bluishcoder.co.nz/2009/11/wasp-lisp-small-scheme-like-lisp.html <comment><USER2> I see wasp creates a dozen folders and files when I try to compile a binary off it. This is actually very annoying, it will create folders and files which are not actually required.Imagine a full hierarchy just for an hello world.Where is the sense in it? I personally like to prefer to categorize my own files in an hierarchy which is customized for a particular project.Sometimes just a file and its binary does the trick and sometimes you have to make just a test folder or a config folder. Not need to bind the programmer to a standard.

<comment><USER3> What, exactly are you doing? Could you email me (swdunlop at gmail) with an ls -lR of what intermediate files you are seeing? waspc -exe should be doing the following:

* Regenerating precompiled modules as necessary. (These are .mo files, similar to Python .pyc files.)

* Selecting a VM "stub" executable.

* Concatenating the required modules and the stub to produce an executable for the target platform.

The only intermediates you should see are .mo precompiled module files, and they are shared between builds. This is pretty standard for dynamic languages that support partial compilation. With the fix for the dynamic library dependency on libevent submitted by Chris Double last week, the resulting binary should be able to stand alone.</comment></comment>


China's BGI says it can sequence a genomre for $100 - <AUTHOR> https://www.technologyreview.com/s/615289/china-bgi-100-dollar-genome/ <comment><AUTHOR> [https://archive.is/QgnMn](https://archive.is/QgnMn)</comment>


Be Careful What You Wish for (2000) - <AUTHOR> http://jimhillmedia.com/editor_in_chief1/b/jim_hill/archive/2000/12/31/be-careful-what-you-wish-for.aspx <comment><USER2> Williams let himself get played at every turn. Can't believe he took scale. Jeez.

(Let that be a lesson about "working for passion".)</comment>


Ask YC: Is anyone using Fluid, Air, or Prism to wrap their sites? - <AUTHOR> http://www.37signals.com/svn/posts/797-fluid-wrap-your-favorite-web-apps-in-their-own-browser There <comment><AUTHOR> There are a lot of options to wrap your site, and make it available as a standalone application.. Adobe has Air, Mozilla has Prism, and Fluid uses WebKit/Safari.

Does anyone have any experience in releasing their sites as downloads? Have your users enjoyed the experience? What sort of download numbers have you seen?

Chron X is currently a downloadable game, but we're porting it to a web-based game, running in Flash.. Since people are used to downloading it, I've been considering releasing a wrapped version, but I'd love to see any anecdotes that people might have in working with any of the engines.

</comment><comment><USER2> I am working on bundling a webapp using xul/xulrunner. I will make it opensource and available soon.

</comment><comment><USER3> I tried Prism. It lets me wrap slime volleyball. I would call that, in spanish, _muy util_

</comment><comment><USER4> If your application is a Flash RIA i think the best approach is just to use Air. You can still display HTML content within Air.

<comment><AUTHOR> Air's a great package- We've done a few tests with it, and I like the way it works, with a few caveats.. I'm more interested in the entire process, however.

A lot of YC-like apps would do well as downloads, and I'm curious if anyone has tried.</comment></comment>


Two Great Companies That Have Probably Seen Their Best Days - <AUTHOR> https://medium.com/@badger__app/two-great-companies-that-have-probably-seen-their-best-days-aadd7d1a6992#.l5xyras3w <comment><USER2> Wal-Mart... sure, easy to see how Amazon and others will replace them. High- volume, low-cost means low-quality. And over time that's just a race to the bottom.

Apple... we all somehow want Apple without Steve Jobs to fail, and it's true they have lost a bit of their shine. 506 days and counting since a new MBP was released, no real "must-have" features on the iPhone or iPad in a while, and the feeling that software updates now come with more bugs. Sure.

But on the flip side of that... my MBP from 3 years ago is still a better laptop than any plastic shit Dell laptop that comes out. The screen is better, battery life is better, design is just better. And there's a better warranty (so I can get my 3 year-old battery replaced just by walking into a pretty space-ship store).

The products Apple builds hold up. Jony Ive and Tim Cook aren't Steve Jobs, but they're really great at what they do. I'm not out there shopping for a new Windows laptop, I'm just annoyed Apple hasn't come out with a new one yet.</comment>