How is a system of axioms different from a system of beliefs? - <AUTHOR> http://math.stackexchange.com/questions/182303/how-is-a-system-of-axioms-different-from-a-system-of-beliefs/182308#182308 <comment><USER2> This question gets asked a lot, in particular by math students, and I think that the answer that is usually given, which is the same as many answers in this page, is overly simplistic: we don't care about ontological truth of our axioms (and, by the way, of the deduction scheme), we just define a set of axioms and define true as _provable_. This is the formalist view of mathematics, which was the dominant view at the beginning of the 20th century.

This reduces math to a meaningless mind game, as acknowledged by Hilbert himself, one of the major proponents of this view: "Mathematics is a game played according to certain simple rules with meaningless marks on paper".

Unfortunately, this sweeps under the rug important questions, such as "if I prove the correctness of a an algorithm, how does this increase my confidence that the algorithm will actually work?", or "if I compute the hypotenuse of a triangle with Pythagoras theorem, then cut a piece of wood and measure it, will they match?"

This is why ultimately there is some belief involved. From Wikipedia [1]: "In practice, most mathematicians either do not work from axiomatic systems, or if they do, _do not doubt_ the consistency of ZFC, generally their preferred axiomatic system" (emphasis mine).

To work with with a set of axioms, we must _believe_ that that set is both logically consistent, and that in some way its interpretation is consistent with reality. The first question cannot be answered for the interesting sets of axioms, thanks to Godel, and the second is not even well-defined.

For this and many other reasons, philosophy of mathematics has mostly moved on from the logicist/formalist view [2], but as most philosophical problems, there probably will never be a conclusive answer.

[1] [http://en.wikipedia.org/wiki/Foundations_of_mathematics#Part...](http://en.wikipedia.org/wiki/Foundations_of_mathematics#Partial_resolution_of_the_crisis)

[2] [http://en.wikipedia.org/wiki/Philosophy_of_mathematics](http://en.wikipedia.org/wiki/Philosophy_of_mathematics)

<comment><USER3> Completeness is overrated. Why does it matter that a system can't prove every possible statement you can write in it? It's not going to give you a wrong answer, it's just going to fail. Why is that particular lack of capability more important than the millions of statements you can't formalize in the first place? Take consistency and build something with it.

P.S. Using a theory to prove its own consistency would be way more suspicious than using a more powerful theory to prove it. "The Bible is true because the Bible says so."

<comment><USER4> On the other hand, consistency alone isn't really useful!

Let suppose I design a new proof system to prove arithmetic propositions. Unfortunately, my system is such that it can't produce any proposition. That makes it consistent but useless.

<comment><USER3> That's why you look at completeness in terms of coverage, not as a binary.

</comment></comment></comment></comment><comment><USER5> Axioms are not beliefs. Axioms are a priori and you formally derive stuff from them using particular logic rules.

Generally speaking, nobody claims that the axioms of set theory are true in any ordinary sense of the word true. Does it makes sense in the real world to assert the existence of an infinite set, and have different sizes of infinities as a consequence? It is irrelevant, you assert the existence of an infinite set because it is practical from a pure mathematical point of view, because you want to model infinite sets like the natural numbers (an abstraction, do they exist? it doesn't matter to mathematicians generally speaking).

Mathematics no longer pretends to describe what is true, what holds in our reality. That idea was abandoned some time ago. A canonical example were non- euclidean geometries, that were studied in the XIXth century for the sake of it. They had applications later, but the motivation for their study and changing Euclid's axioms was formal.

<comment><USER6> > Generally speaking, nobody claims that the axioms of set theory are true in > any ordinary sense of the word true.

Mathematical platonism is a pretty widely held belief, at least amongst pure mathematicians. From Hardy's A Mathematician's Apology,

"I believe that mathematical reality lies outside us, that our function is to discover or observe it, and that the theorems which we prove, and which we describe grandiloquently as our 'creations', are simply the notes of our observations."

A lot of set theorists will speculate about the truth of axioms beyond the standard axioms of ZFC, e.g. large cardinal hypotheses or projective determinacy.

</comment></comment><comment><USER7> There's an infinite variety of axiomatic systems, most of them not much different from, say, a set of beliefs in Santa Claus and magical elves living at the North Pole.

However the axiomatic systems we happen to use most have one important property: they are _useful for acquiring new knowledge via scientific method_. I.e. they serve as a foundation for theories that make correct predictions as verified by experiment.

That's the main difference.

<comment><USER8> A story:

"There's two scientists on lab coats observing a spider. They shout JUMP! and the spider jumps.

They proceed to remove one of it's legs and repeat the procedure. They shout JUMP! and the spider jumps...

After some time of repeating the procedure the poor spider has no legs. They shout JUMP! and it doesn't do a thing.

The scientists then come to verify their useful hypothesis: ..spiders use their legs to listen..."

<comment><USER9> Cute story, but don't mistake it for a second for any sort of casual indictment of the scientific process. ...the story does not even feature the scientific process in the first place (which goes a long way to explain why we find it silly).

</comment></comment></comment><comment><USER10> This is pure Platonism. I reject Platonism in modern mathematics.

Mathematics should, first, be used to describe the real world. (That is what scientists like Newton were using it for.)

To do that, you don't need an axioms. Nor do you need beliefs. You only need observation of reality. It starts with the concept of a "unit," which can be added or subtracted from other "units." There is, of course, some prerequisite _philosophy_. ("Reality, you say?" "Why, yes, actually...").

Secondly, mathematics can deal with axioms and "games" and "universes" (to take words from the StackExchange post). But all of mathematics should not be subordinated to such a framework.

I realize my argument is not "obvious." Fully establishing it would take a lot of work, and even then it would be widely rejected by the status quo Platonic mathematics establishment. I'm not claiming to do that in this comment. I'm just putting this alternative perspective out there for anyone who is interested.

<comment><USER11> _Mathematics should, first, be used to describe the real world._

If by "real world" you mean the things and events we can directly witness, that seems to be deliberately crippling mathematics from the start and limiting it to things we already know about.

<comment><USER10> Depends on what you mean by "directly witness," but that's neither here nor there.

What I am proposing would not cripple mathematics, it would simply replace "axioms" with _facts_ (such as, for a simple one, that adding one unit to another results in having two units). You can then get arithematic, algebra, calculus, and on and on.

If people want to go off and define axioms for made-up worlds that do not necessarily correspond to reality and then go do math with them, that's fine. In fact, I would encourage it.

But we should not be approaching _all of mathematics_ in that way, and that is my point.

Math has the same basis in perceptual concretes (i.e. "real things") as physics, biology, chemistry, etc.

<comment><USER12> The ZFC axiom set [1] doesn't define a made-up world. Actually, once you understand the axioms (they're somewhat difficult in notation and definitions, but the ideas behind are understandable) they make sense. There are some axioms made to avoid paradoxes (notably ZFC3 [2]), but the majority describe how we understand the world. It'd be difficult to imagine a world where two sets aren't equal even when they have the same elements, or where you couldn't construct a set that contains all elements of two other sets.

How's this different to a set of _facts_ as you say, like one unit + one unit is two units? It's not. I'm not sure of this, but I'd say you could construct a set of axioms based on those _facts_ that is compatible with the ZFC axiom set.

So, why choose ZFC? Because it's simple to define (how'd you formally define what is a unit, and what does the + operation mean?) and because it's more or less consistent with our world. We can extrapolate the things we find while doing mathematics to the real world. And, sometimes, we can discover things of the real world based on findings on things that don't _exist_. Complex numbers, for example: they don't exist, but they're really useful to define and explain electrical experiments [3].

We approach _all of mathematics_ like this because its easier to study everything within the same system. And it's important that we keep a consistent underlying system so we can be sure that everything we discover based on the set of axioms and rules we've defined is _true_ in a mathematical sense; and then those discoveries could describe our world. Wouldn't we have a consistent system, we could end up with contradictions. If we extracted mathematical concepts from the real world, we wouldn't be sure if a _discovery_ we make from those concepts can be applied to the real world.

TL;DR: Our set of axioms describes properly the real world, is simple and consistent enough and that enables us to infer new ideas that can be applicable to the real world too.

[1] [http://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_th...](http://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory) [2] [http://en.wikipedia.org/wiki/Axiom_schema_of_specification](http://en.wikipedia.org/wiki/Axiom_schema_of_specification) [3] [http://en.wikipedia.org/wiki/Complex_number#Electromagnetism...](http://en.wikipedia.org/wiki/Complex_number#Electromagnetism_and_electrical_engineering)

<comment><USER11> _Complex numbers, for example: they don 't exist, but they're really useful to define and explain electrical experiments_

At risk of turning this into a discussion of what "exist" means, I reckon they do exist :p If they didn't exist, how could we possibly use them, and use them so effectively and consistently?

Of course, if a number "existing" means "I can see this many number of apples" then sure, it doesn't "exist". Yes, this is definitely becoming a discussion of what the word "exist" means applied to number. I'll stop. :)

<comment><USER12> Well, yes, I was talking about exist in the sense "I can't count _i_ apples", and like you say better stop here because this can get infinitely long :)

</comment></comment></comment></comment></comment></comment><comment><USER13> You don't blindly believe axioms, it is just that certain sets of axioms happen to allow us to make useful predictions about the world. While you could have philosophical discussions about weather certain axioms are true (whatever that means), and what their exact relationship with the world is, you don't need to do this to see that axioms are useful to arrive at mathematical conclusions, which then can be used to make predictions about the world.

Why do we believe that if we take 1111 apples and then add 2222 more we will get 3333 of them? Because we have seen from experience that a certain way of applying mathematics to the real world gives great results. The same for conclusions following known axioms, they seem to generally work well for describing our world.

</comment><comment><USER13> You don't blindly believe axioms, it is just that certain sets of axioms happen to allow us to make useful predictions about the world. While you could have philosophical discussions about weather certain axioms are true (whatever that means), and what their exact relationship with the world is, you don't need to do this to see that axioms are useful to arrive at mathematical conclusions, which then can be used to make predictions about the world.

Why do we believe that if we take 1111 apples and then add 2222 more we will get 3333? Because we have seen from experience that a certain way of applying mathematics to the real world gives great results. The same for conclusions following known axioms, they seem to generally work well for describing our world.

<comment><USER14> A good post although I suggest rephrasing "useful" to something like "experimentally falsifiable". With a pretty wide definition of "experimentally" and "world", especially for non-applied math.

If you have no way to do a falsifiable test, and can't just substitute in a X and do abstract analysis of X instead of worrying about X, then its just a belief.

I'd be careful categorizing stuff eternally as beliefs not axioms. Blindly only permitting geometry to be Euclidean meant some exciting non-euclidean results couldn't happen until it was "allowed" to be thought about somewhat recently (well, a century or two ago...). Also physicists have this amazing ability to turn pure math into applied math, at least over a long enough historical scale. And physicists have a pretty good ability at coming up with crazy experimental methods to test, look at the last 75 yrs or so of quantum physics... so let me get this straight, you do what with two geiger counters and a truly giant magnet and some radioactive atoms or what? Or shine a light beam or ion beam thru that weird magnet?

</comment></comment><comment><USER15> This sort of thing is also studied as part of [http://en.wikipedia.org/wiki/Epistemology](http://en.wikipedia.org/wiki/Epistemology) and it may be of interest that it is studied also as part of computer science for example as [http://en.wikipedia.org/wiki/Belief_revision](http://en.wikipedia.org/wiki/Belief_revision) which is in turn closely related to [http://en.wikipedia.org/wiki/Reason_maintenance](http://en.wikipedia.org/wiki/Reason_maintenance) which used to be the basis for general problem solvers. By the way, view maintenance in relational databases could be seen as a subset of reason maintenance.

Note that belief revision and various kinds of paraconsistent logics in a way allow to work around the classical requirements for consistency.

Another really interesting related area is [http://en.wikipedia.org/wiki/Modal_logic](http://en.wikipedia.org/wiki/Modal_logic) which allows you to reason about statements such as "I know that you know that I know that the sky is blue." and that can be used to analyze the coordinated attack problem ([http://en.wikipedia.org/wiki/Two_Generals'_Problem](http://en.wikipedia.org/wiki/Two_Generals'_Problem)) which is very much related to the [http://en.wikipedia.org/wiki/Byzantine_fault_tolerance](http://en.wikipedia.org/wiki/Byzantine_fault_tolerance) of distributed systems.

</comment><comment><USER16> There maybe some axioms that are analogues to rules of a game and so are neither true nor false.

But in practice, the bulk axioms are chosen and retained because the inventor thinks they or their consequences have some mapping to reality. This mapping is tested through applying the resultant math to some field and getting predictive success. In this regard the bulk of axioms are the same as any other beliefs except they are not arbitrary beliefs, and they are useful or believed to have the potential of usefulness.

And part of the problem is merely semantic. The word "belief" differs from the term "axiom" in other critical ways: "belief" regularly involves a call to action and includes concept that do not go though any tests for consistency let alone predictive success. If one asks "what is the difference between an assumption and an axiom" there is a lot less excitement.

Most interesting, I think, is the idea that basic axioms should be different than assumptions at all. That they can be definitively decided or the part of reality they map to is real and can be found.

I do not see why we would necessarily have evolved the mental equipment to work out every law of nature. No other animal has that, and while we clearly have better mental capacity I can't see why we would have evolved _every_ mental capacity.

</comment><comment><USER17> The major difference is that nothing requires that your set of beliefs be consistent: however, if a given set of axioms allows you to prove and disprove some assertion _P_ , you pretty much can't use that set of axioms anymore! Well, you can, but rather than the normal huge castles made of sand that follow from any reasonably strong set of axioms, you will end up with a small puddle of mud that's useless.

In this sense, a simple discovery (of an axiomatic system's inconsistency) would essentially cause that set of axioms never to be used again.

Can you imagine that a simple logical proof would immediately cause the Pope to abandon making certain statements together?

For example check out this post:

[http://www.religiousforums.com/forum/1993062-post3.html](http://www.religiousforums.com/forum/1993062-post3.html)

Suppose a religious scholar produces an argument that the universe described by,

"Jesus Christ, the only Son of God" \+ "who was born of the Virgin Mary" \+ "his kingdom will have no end" is an inconsistent Universe.

It won't suddenly be the case that nobody will mention these three things in the same axiomatic set again, just because they're (through some rigorous proof) inconsistent.

That's a huge, huge difference. It's like the difference between chess and Shakespearean criticism!

</comment><comment><USER18> A couple of important distinctions between axioms and beliefs:

1) axioms are assumed to be logically consistent with each other, at least when the word "axiom" is understood strictly; it's possible that someone may believe two things that are inconsistent with each other

2) belief is an involuntary mental acceptance that a particular claim is true. You can't _make_ yourself believe something that you know to be false

</comment><comment><USER19> Axioms are definitions you set before you start to derive the rest of the system. If you want to, you can test the axioms and the derivations against reality and you have science.

Beliefs are definitions you think somebody else set that you can't change before you start to derive the rest of the system. In most cases, you are not permitted to test beliefs against reality.

</comment><comment><USER20> I stopped reading when he implied that projective geometry is a wonderful game. My last party ended up with me defeated and crying.

</comment><comment><USER8> Axioms are building blocks. Beliefs are the idols you make with or without building blocks.

</comment><comment><USER21> Assumption (or axiom) + ego = belief

You don't get your feelings hurt when an assumption turns out to be wrong. But if you were to be betrayed by someone you believed (same as assumed they were telling you the truth or had your interests in mind), it would hurt your feelings because you had invested some of your ego with that assumption. Same for those with religious beliefs.

I'm using axiom and assumption interchangeably, as I don't see what the difference between the two is. However, assumption is a much simpler word and means pretty much the same thing to everyone, so it's easier to use. I guess axiom is something like a generally accepted assumption.

</comment><comment><USER22> I also religiously believe that a flush is better than a pair in a game of poker.

<comment><USER23> That isn't a belief with a quality of religiousness; it's exactly the same as saying you religiously believe 2 is greater than 1.

</comment></comment><comment><USER24> A reality, often unacknowledged, is that mathematicians proceed largely on the basis of elegance. As Achilles and the Tortoise shows us, basic logic has built-in points of infinite regress, and the only way past them is to agree that the logic is sound.

Mathematicians who must choose between two sets of axioms will pick the one they consider most elegant, every time. The fact that this is ultimately an aesthetic judgement is clear: Erdős Pál, arguably the finest mathematician of the latter half of the 20th century, described proofs that tickled his sensibilities as "Pages from the Book".

</comment><comment><USER25> The question is about comparing axioms with belief and religion; so my question is kind of off-topic, but I have to ask: How about philosophy? How does proving a philosophical theorem start? Is it by an axiom or something else? To put it another way, what is the philosophical equivalent of scientific method?

<comment><USER26> Descartes and Spinoza tried to apply scientific method to philosophy.

<comment><USER23> [http://plato.stanford.edu/entries/peirce/](http://plato.stanford.edu/entries/peirce/)

</comment></comment></comment><comment><USER27> In my opinion, if the set of beliefs is orthogonal and irrefutable, it is a set of axioms. Orthogonal: It is not possible to derive one belief from another or a combination of other beliefs. Irrefutable: Counterexamples have not been found or counterexamples can simply not ever be found. It is obvious, however, that orthogonal and irrefutable do not mean "true". On the contrary, "true" means that a statement can be derived from the axioms, while the axioms by definition themselves cannot. Axiomatic systems are simply a form of rigorous and highly systematic beliefs, but not necessarily internally consistent nor necessarily consistent with reality.

<comment><USER12> In the same sense that an axiom can't be proved, it can neither be refuted. I think you mean they're consistent: that you can't infer a paradox or a contradiction from your set of axioms.</comment></comment>


Building a Shop with Sub-Second Page Loads: Lessons Learned - <AUTHOR> https://medium.com/baqend-blog/building-a-shop-with-sub-second-page-loads-lessons-learned-4bb1be3ed07#.adk9fij92 <comment><USER3> I work in energy, and there's a saying: "The cheapest kwh is one you don't use."

Want to make your website faster? Don't send so much stuff. Removing unnecessary code or content from your site has always had the biggest impact on performance. Unfortunately, it seems that often the people in charge of increasing performance aren't the people who get to choose what has to be on the website.

The Website Obesity Crisis: [https://vimeo.com/147806338](https://vimeo.com/147806338)

        I want to share with you my simple two-step secret to improving the performance of any website.

        1. Make sure that the most important elements of the page download and render first.

        2. Stop.

<comment><USER4> Here's an easy way to do it:

Want to make your website faster? Use a 3G connection when developing or testing. Chrome Dev Tools make this easy and you can also adjust the latency as needed.

<comment><AUTHOR> That should be done a lot more than it's usual today. Part of "mobile first" should be testing mobile speed.

One problem is of course that mobile networks vary greatly in both latency and bandwidth. A great resource on performance considerations for mobile networks is Ilya Grigoriks open book [1].

[1] [https://hpbn.co/mobile-networks/](https://hpbn.co/mobile-networks/)

</comment></comment></comment><comment><USER5> Great work. When I loaded the page I even thought to myself, oh shit this is fast.

One huge problem, with most large platforms for e-commerce stores, they are terrible at speed optimization!

For example, I have a store on Shopify and it's basically impossible to do most of what you explained. With Shopify, you can customize HTML / SCSS files with dynamic content. All of which gets compiled on their back-end. You have no access to the final compiled HTML and markup. You're sadly limited in what you can do.

It seems you need to have a custom store to really be able to optimize this hard-core.

<comment><USER6> This is largely incorrect. All the front end customizations that this article describes are possible with Shopify because you have complete control over the HTML, JS, and CSS that the platform spits out. While true that you can't "access" the final compiled HTML to edit it, the templates that power your theme are very close to that output, and not hard to change if you're familiar with HTML/CSS. The Javascript optimizations are all completely possible as well but you either have to pick a theme that has been optimized or implement them yourself in your own theme. Shopify also supports AMP and responsive themes (because it's just HTML).

It is true that with Shopify or similar platforms you aren't able to make backend choices or implement caching like the article has described, and yes it's true that all the themes in the theme store aren't perfectly optimized for page load time -- but the question really is do you want to do any of this stuff yourself? Shopify has invested years in their platform to make your website as fast as possible for as little money as possible. The themes that Shopify publishes in the theme store undergo huge amounts of performance testing to ensure that you aren't losing money from latency. The platform comprises a world class CDN, and Shopify doesn't charge for bytes through it. The platform uses many layers of backend caching at the data access, logic, and rendering layers to make your shop as responsive as possible if anyone has been to it before. Shopify powers some of the biggest sales on the internet that truly dwarf Good Morning America. The platform is also geographically redundant such that an entire datacenter can fail with only a moment of visible impact to you and your customers. If what I'm saying is true and you don't actually sacrifice that much flexibility on the Shopify platform, why would you choose to have to implement all the above?

Another angle is operations: How many people do you think they have monitoring it and ready to respond to any incidents? Shopify serves an order of magnitude more traffic than Baquend saw at the very peak of this occurrence every single minute of the day with a better response time on average.

The Shopify platform took an army of engineers 10 years to build so it would seem unlikely that Baquend has all the same features and stability. How much do you think Baquend charged Thinks to build this? Is it more than $29 / month?

Source: work at Shopify and built many of these systems.

<comment><AUTHOR> > _It is true that with Shopify or similar platforms you aren 't able to make > backend choices or implement caching like the article has described, and yes > it's true that all the themes in the theme store aren't perfectly optimized > for page load time -- but the question really is do you want to do any of > this stuff yourself?_

You're right, that is surely a thing you do not want to do yourself. This is why Baqend does all the browser and CDN caching, TTL estimation, query invalidations and so on automatically.

In terms of the frontend if you do not want to rely on a stock theme - <USER2> many larger ecommerce site won't - then you will have to do these optimizations yourself. Fortunately with good tooling available as outlined in the article.

I think Shopify is in fact a great improvement over systems like Magento. But if Shopify is well-enginneered (which I'm convinced of), how do you handle sudden load spikes in the database? Do you reshard and migrate data in real- time? How does Shopify scale the application tier? How can tenants run scalable custom backend code? What happens if a database or server node crashes?

The advantage of Baqend is that its inner workings are published in research papers and blog posts, so people can judge the performance and scalability not only through benchmarks and real-world examples like Thinks but also by looking at the design.

In contrast to Shopify, Baqend is backend-as-a-service which means customers have full freedom on how they design their site or shop. They do not have to live with a server-generated HTML that they cannot change - they can for exmaple easily build a React/Angular single page applications, run server-side JavaScript code, do custom queries, build a native app around the same business logic and APIs, subscribe to websocket-based change streams etc.

While I respect that it took you 10 years to build Shopify, this is usually not an indicator for scalability and a low-latency design. Often such projects end up with solutions like MySQL sharded at application-level, intermingled technology stacks and legacy systems.

Disclaimer: I'm the Baqend CEO and its caching approach is my PhD thesis.

<comment><USER6> > then you will have to do these optimizations yourself

For the optimizations you listed they actually all still apply with a custom theme. The author doesn't have to do anything. Shopify caches all generated HTML, serves all assets from a CDN, and controls expiry without using TTLs because we know when anything changes. The stuff custom theme authors do have to do is things like bloat reduction / post-first-paint behaviour addition / css pruning etc etc. We're considering shipping mod_pagespeed for all customers too which would help automate a bunch of this stuff.

> how do you handle sudden load spikes in the database

Most of our read load is served by a datablob caching layer[1] so the DB is doing mostly stuff only it can do, writes, and we then have a carefully tuned schema that allows for massive throughput. We also buy really expensive databases -- we just took shipment of over a ton of database hardware in one datacenter in preparation for Cyber Monday. If we do hit our limit, we're able to throttle the number of people in the checkout while maintaining a good experience for customers.

> Do you reshard and migrate data in real-time

Yes

> How does Shopify scale the application tier

We shard by shop and build failure domains called pods that are self contained versions of each piece of Shopify that can failover independently between infrastructures.

> How can tenants run scalable custom backend code

Using Shopify Scripts. They're only available in the domain of carts and checkouts right now, but are rolling out to all our backend domain models in time. See [https://help.shopify.com/api/tutorials/shopify- scripts](https://help.shopify.com/api/tutorials/shopify-scripts)

> What happens if a database or server node crashes?

We have automated failover in place at almost every level of the system, so unless we've screwed up somewhere, no one notices!

> they can for example easily build a React/Angular single page applications, > run server-side JavaScript code, do custom queries, build a native app > around the same business logic and APIs, subscribe to websocket-based change > streams etc

Did the Thinks store end up using any of these features of the platform? I am sure lots of apps need these things, but for web storefronts a lot of this seems like overkill. Shopify also supercharges you when you want to move beyond just a web storefront: we have a mobile SDK for building native apps, connectors to let you sell on Amazon, Facebook, and anywhere you can inject HTML, a world class mobile administration app, a POS solution, a massive app store for other integrations, etc etc etc.

> this is usually not an indicator for scalability and a low-latency design

Again, sub 100ms response time and an order of magnitude more traffic every minute than the peak the Thinks sale saw... that's pretty good!

I'm trying to get at the fact that building a web store from scratch, be it with a handy backend as a service or not, is going to be really hard in comparison to using Shopify. You sacrifice some small amount of flexibility, but you gain so so so much more by buying into the ecosystem. I still think it is unwise to try to compete with other platforms who's core competency is commerce.

1: [https://github.com/shopify/identity_cache](https://github.com/shopify/identity_cache)

</comment></comment></comment></comment><comment><USER7> I checked out Baqend ("we" in the article) and the site prevents me from navigating back when I press "back" (I think by injecting itself as the previous page - I'm not really up to date on dark patterns).

This is the kind of behaviour that is an auto-no for me when buying B2B, not just by itself but because of what it signals about the company. I recommend you remove it.

On the article itself: you can do very fast pages off a relational DB. In one case, the round trip on a Postgres full text search for auto complete on millions of rows from multiple tables was so fast we had to add a 10ms delay - on a $5/month VM.

<comment><USER8> It looks like the dark pattern they used to get that "no back" behavior is: "open in a new tab".

Edit: I stand corrected, link from the article opens in a new tab, so the back issue is moot there. Definitely red flag behavior. Baqend folks, if you are listening, you should fix it, and if it is the default behavior of your framework you should _definitely_ fix it. No one likes hijacked back buttons.

<comment><USER7> How to replicate: google "Baqend", click on their page (should be top result), press back button, press "back" arrow on browser.

Screenshot: [http://imgur.com/a/mLXdc](http://imgur.com/a/mLXdc)

Compare with: [http://imgur.com/a/UnQOG](http://imgur.com/a/UnQOG)

<comment><AUTHOR> Thanks a lot, it's fixed now.

</comment></comment></comment></comment><comment><USER9> Great info on how to optimize.

However, in basket, I clicked "Zur Kasse" to checkout, and the response took 11.08 seconds to get to the checkout page per Chrome network tab. Repeated multiple times with same result.

Also, I don't see any product detail pages, categories, search result pages, or anything other than 7 products on the homepage that add directly to cart. Does this strategy scale to regular ecommerce site levels?

<comment><AUTHOR> You're pointing out an important point: third-party services have to be fast, too. The call that starts the checkout does a synchronous request to Paypal Plus, which has very ugly request latencies from time to time.

The shop is indeed quite minimal at this time. The optimizations scale very well to regular ecommerce sites, though. Detail pages and categories can be cached very well, as they only change by actions of human editors. A notable exception are recommendations. If they are item-based (e.g. a-priori algorithm) they can also be cached very well. If they are user-based (e.g. collaborative filtering) that part has to be fetched from the server-side recommendation model. The same is true for search results. The heavy-hitters, i.e. the most-searched-for keywords are cached, while the others are fetched from the database. That is the part where server-side scalability and latency optimization is of utmost importance. At Baqend we support both MongoDB full text indexes and automatic separate indexing in an ElasticSearch cluster with automatic CDN and browser caching for the most frequent search results.

<comment><USER9> Why do you need to call Paypal before the payment selection screen? Still happening for me, and there is no visible feedback when you click checkout. It just looks broken for 10 seconds.

The rest of it is so well done, but I have to wonder how many sales were lost because of this checkout issue.

<comment><AUTHOR> You're absolutely right, latency in the checkout process is a conversion killer. Somehow Paypal currently seems broken, maybe it's still related to the Dyn DDOS attacks which took Paypal down completely.

During our testing, Paypal had latencies of ~100-500ms but these new latency outiers of 10 seconds are a no-go. We will definitely change the code to make that API call asnychronous. Generally Paypal Plus is still quite buggy and not well documented.

</comment></comment></comment></comment><comment><USER10> I am amazed that it is 2016 and people talk about sub-second page loading times as if it is an accomplishment to be proud of.

That it is an accomplishment of a sort (in contrast to the depressing reality of bloated websites everywhere) shows just how badly wrong we have gone.

Maybe we should force web devs to go back to dialup; then we'd get sites that have the performance of [http://info.cern.ch/hypertext/WWW/TheProject.html](http://info.cern.ch/hypertext/WWW/TheProject.html) again.

</comment><comment><USER11> We also realised that tackling web performance takes a lot of effort..so have been building different tools which can automate front end optimization techniques. Our recent tool ([https://dexecure.com](https://dexecure.com)) tackles image optimization by generating different formats (WebP, JPEG XR), different sizes of images and compressed to different qualities depending on the device and browser the user is using. All of the images are served with a fast HTTP/2 connection.

We have seen people integrate with it in under 5 minutes and cut down their page size by 40%!

<comment><USER12> +1 Images are a great starting point for improving page load performance as they sometimes represent a big chunk of the total page weight.

</comment></comment><comment><USER4> Very good view to an often ignored part of web development.

And as usual, the Medium posts loads way slower than the actual website.

</comment><comment><USER13> > Furthermore, browser caching is highly effective and should always be used. > It fits in all three optimization categories _because cached resources do > not have to be loaded from the server in the first place_.

That is incorrect.

The browser issues requests for all cachable content with a last-modified-date [or similar special metadata], the webserver replies with a 304-not-modified + empty body, if the file was not updated.

There are still HTTP requests sent, there is still network latency, there is still server processing involved. [Worst case scenario: For tiny JS/css/icon files, the latency might dominate everything and nullify the benefits of caching].

It gets more complex: Since a few years ago, Firefox stopped issuing HTTP requests ENTIRELY for some cached content. It speeds the load time but it causes caching issues.

Last I checked, chrome and IE didn't adopt this behavior yet. That may or may not chance in the (near?) future.

P.S. I felt like being pedantic about caching tonight. Otherwise, a nice article =)

<comment><USER14> > There are still requests sent, there is still network latency. [In the case > of small JS/css/icon files, the latency can be as much as the transfer > time].

What? When the "Expires" header indicates the item is still valid the browser should use the cached object without making any requests.

Some servers (Rails for example) just serve assets with "Expires" set to some distant future time. When they want to expire these files, they simply change the name (hence the long numeric suffix appended to js/css/etc. files)

<comment><USER15> Yeah, I was wondering why I bother to set the expires header if the browser is going to ignore it and check every time anyway.

<comment><USER16> It doesnt ignore it. In chrome, if you keep the network tab in dev tools open, you will see "From disk/memory cache" in the size column with a status of 200 when it comes from local cache.

<comment><USER17> It ignores it on refresh before the page load event. It respects it otherwise.

</comment></comment></comment></comment></comment><comment><USER18> Great article Erik et al. Shows how engineering requires grounding in theory, clever implementation, and lots of trial and error.

Is there a particular target customer space that you are focused on?

<comment><AUTHOR> We built Baqend around the caching algorithms that we developed at the university. Our goal was to make the performance optimizations applicable to as many contexts a possible. We went for the "serverless" pardigm and backend- as-a-service to accelerate any application (website or app) that uses our APIs and hosting.

Currently most of our customers are e-commerce, web and app agencies, startups and software companies that look for web performance in combination with scalability.

We're currently getting lots of request from e-commerce, so we are building out our support for shop features (e.g. advanced search, payment, content management). We are developing a Magento plugin to make the transition to our platform and its performance benefits as easy as possible.

</comment></comment><comment><USER19> FWIW there are some clever Magento shops running pretty stout at sub second load times on AWS. Puts Magento back in the upper echelon for carts again, imo.

<comment><USER20> Can you please share some URLs of faster Magento stores?

I'm working on a similar project that will offer optimization as a service for e-commerce. I'm curious to see what others are doing.

<comment><USER21> Not a site, but what I've seen that looks really interesting is LiteSpeed's LiteMadge Cache: [https://www.litespeedtech.com/solutions/magento- acceleration](https://www.litespeedtech.com/solutions/magento-acceleration).

Not used, only seen and wanted to understand how it works.

</comment></comment></comment><comment><USER22> This must be really frustrating when your startup gets so much attention that your shop breaks down and you end up making no money out of that

<comment><AUTHOR> And it not only hurts because of revenue. Slow sites also let users preceive quality and credibility as lower (Fogg et al. 2001; Bouch, Kuchinsky, and Bhatti 2000). The startup/brand will end up being seen as less interesting and attractive (Ramsay, Barbesi, and Preece 1998; Skadberg and Kimmel 2004). So being slow or offline can even have a long-lasting negative impact on the coolness of your startup.

<comment><USER9> The site is taking 10+ seconds on the "createTransaction" call when trying to checkout right now.

<comment><AUTHOR> Unfortunately on the server that method executes a synchronous API call to Paypal Plus, which has very flaky response times.

</comment></comment></comment></comment><comment><USER23> Oh the irony. The Medium page this blogpost was hosted on is ~3.5MB in size and took ~5 seconds (on my 20Mbps cable internet connection with 11ms ping to cloudfront) to first readable render.

</comment><comment><USER24> It's interesting to me that the website "thinks.com" is entirely in German. I'm used to seeing sites on .com in English, and indeed I can't even think of another site right now that isn't.

Why didn't you use the .de ccTLD? Given that most German websites are not using .com domains, isn't this potentially confusing, and causing you to lose traffic? You don't even own thinks.de; it's blank.

<comment><USER25> You're mistaken. A lot of businesses in many countries use .com addresses even for their local shops.

For one othing, .com stands for "commercial". .us is for USA.

Regarding whether it's right to do so or not is another question. For starters, it depends on the regulations of each country, as in some of them you simply are not allowed to operate under a different TLD. But most countries are not this restrictive.

One of the many things that were supposed to "make sense" at the dawn of the web, and that were quickly abused and evolved into more than what anyone could possibly have thought of at first. (No, no, I'm not thinking of HTTP at all while writing this. Why would I?)

</comment></comment><comment><USER26> 5ms average load latency. But the max looked like it was more than 5 seconds.

<comment><AUTHOR> The JMeter load tests were conducted to test the throughput of the CDN and backend setup (browser caching was disabled). The load generating servers where hosted in Frankfurt (Germany) where the CDN (Fastly) also has an edge location. This lead to average response time of 5ms. We experienced some hickups within JMeter causing single requests to stall up to 5 seconds. For the 10 million requests we got an 99.9th percentile of below 10ms. We couldn't quite figure out why JMeter had these hiccups, maybe those were GC pauses, maybe something else, but it was consistent across different setups and servers.

<comment><USER27> One wonders about the irony of carefully avoiding bad stuff (like GC) when serving pages, and then using a test rig that has potential GC delays to test performance.

<comment><USER28> You have a point there. I found jMeter, however, really easy to use. I could simply let it monitor my browser (via a proxy) while i clicked through the website and the checkout process to record the requests of an average user. Then I configured the checkout process to only be executed in 20% of the cases to simulate the conversion rate. Even executing the test distributed over 20 servers wasn't that hard.

Which tools would you use to generate this amount of traffic?

<comment><USER29> wrk2: [https://github.com/giltene/wrk2](https://github.com/giltene/wrk2)

<comment><USER27> That looks pretty good -- not only does it have a "requests per second" generator, which I wanted, but it also presents 50/90/99/99.9 percentile results, which are a must. (Website latencies are not a normal distribution so it's inappropriate to compute standard deviation.)

</comment></comment></comment></comment></comment></comment><comment><USER30> I get pretty bad layout thrashing upon load.

</comment><comment><USER31> I Look at it only to criticise but it is really fast, congrats.

</comment><comment><USER32> I feel concerned by this new trend of using slow languages/stacks and the "we'll fix it later" mantra. I can't believe there are webpages such as reddit with such horribly awful generation times, do their tech guys sleep well at night?

<comment><USER33> This is not a new trend. This is a trend since the dawn of, well, as long as I've been programming, ~20 years now. Turns out that most of the time there are many more important things than your stack or language speed, and yes, often times "fix it later" is the right tradeoff.

<comment><USER28> On the other hand, there are things where "fix it later" just won't work. Especially when it comes to scalability, which has to be considered right from the start to get it right.

<comment><USER34> Yes and no. Facebook-scale technologies when you have 30 requests per minute is a horrible use of your time and money. The goal is to come up with the scale you need to fund your next scale growth. I'm working on a system that needs to handle a few thousand simultaneous users in bursts, and our growth rate looks like it might grow to a couple tens of thousand simultaneous users in bursts. We can horizontally scale for "hot" periods (like Black Friday weekend).

We're working on designing our next iteration so we can handle tens to hundreds of thousands of constant users, and a couple of million in bursts—with the capability to horizontally scale or partition or some other mechanism to buy us other time when we need to look at the next scaling level (which will probably be some time away). It would be irresponsible of me to design for 10MM constant users now.

<comment><USER28> Considering scalability from the start does not just mean optimizing for millions of concurrent users, but choosing your software stack or your platform with scalability in mind. I get that it's important to take the next step and that premature optimization can stand in your way but there are easy to use technologies (like NoSQL, caching) und (cloud-) platforms with low overhead that let you scale with your customers and work wether you're big or small. This can be far supirior to fixing throughput and performance iteration to iteration.

<comment><USER34> I chose my software stack with scalability in mind: team scalability and rapid iteration (we started with Rails, displacing a Node.js implementation that was poorly designed and messily implemented). Because of that previous proof-of- concept implementation we needed to replace, we were forced into a design (multiple services with SSO) that _complicated_ our development and deployment, but has given us the room to manoeuvre while we work out the next growth phase (which will be a combination of several technologies, including Elixir, Go, and more Rails).

One thing we didn't choose up front, because it's generally a false optimization (it looks like it will save you time, but in reality it hurts you unless you _really_ know you need it), is anything NoSQL as a primary data store. We use Redis heavily, but only for ephemeral or reconstructable data.

The reality is, though, you have to know and learn the scalability that _your_ system needs and you can only do that properly by growing it and not making the wrong assumptions up front, and not trying to take on more than you are ready for. (For me, my benchmark was a fairly small 100 req/sec, which was an order of magnitude or two larger than the Node.js system we replaced, and we doubled the benchmark on our first performance test. We also reimplemented everything we needed to reimplement in about six months, which was the other important thing. My team was _awesome_ , and most of them had never used Rails before but were pleased with how much more it gave them.)

<comment><USER35> I think the main argument for (distributed) NoSQL as a primary data store is availability, but there's other ways to achieve that too.</comment></comment></comment></comment></comment></comment></comment>


In which Erlang, Haskell, and Oz beat the crap out of C/C++ in speed on a benchmark - <AUTHOR> http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=threadring&lang=all

<comment><USER2> So...the benchmark is to rapidly switch between threads? Seems like more of a test of the system call bindings than the language speed.

<comment><USER3> It tests whether or not the language has cheap/easy concurrency. C only works at the system thread level, but the languages that beat it use built-in lightweight concurrency.

<comment><USER4> The thing is you can implement lightweight concurrency in C too and get the same/better results check out the Cheap Threads library

<http://freshmeat.net/projects/cheapthreads/>

<comment><AUTHOR> "At this writing Cheap Threads is beta software. I am releasing it without having yet used it in a real application, beyond the artificial drivers used for testing and development."

This is not a compelling alternative to what Erlang has to offer.

Supposing you do get cheap concurrency in place, you still have to implement the logic for your application, and you're going to do that in C. If you're writing something roughly as low-level as a kernel or a device driver, super. If it's a business application with rapidly changing requirements, good luck.

<comment><USER2> Well, now you're making a different argument. You've gone from _"Erlang beats the crap out of C/C++ on a benchmark"_ , to _"Erlang beats the crap out of C/C++ in this particular circumstance, unless, of course, you happen to make a fair comparison to C/C++, but that would be hard, therefore Erlang is better."_

To which I say: thppt.

<comment><AUTHOR> What? It's not at all clear that you can reproduce Erlang's concurrency ability in C. Someone gave a link to an untested, unknown library that hasn't been tested in real situations hardly at all and that's supposed to be equivalent to 2 MLOC codebases with 99.9999999% uptime?

I was suggesting it to be fair. It has not yet been demonstrated because I have not seen any benchmarks to the contrary or anything approaching real data, and certainly no instances of this occurring in real production software.

Thus, the opportunity to make a fair comparison never arose, so you have some work to do before you say "thppt".

The point was it's easier to get correctness in languages like Erlang, irrespective of what speed is being achieved. Of course, if you're happy with buffer overflows and core dumps, more power to you.

<comment><USER2> _"What? It's not at all clear that you can reproduce Erlang's concurrency ability in C."_

The Erlang VM is written in C:

[http://planeterlang.org/story.php?title=Inside_BEAM_the_Erla...](http://planeterlang.org/story.php?title=Inside_BEAM_the_Erlang_Virtual_Machine)

You might want to step back from the ledge a bit.

<comment><USER5> > The Erlang VM is written in C:

It has been written _once_ in C. That does not mean that re-implementing it in C again would be a good idea or would get you the same set of benefits as the existing Erlang VM and its associated libraries. Just because one particular team has accomplished a task in C does not mean that any random collection of C coders have the skill or experience to do the same.

(A pedantic point perhaps, but the quote you reference said _reproduce_..._

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> Haskell beats C on memory usage too, interestingly enough.

Poor old Ruby though, let's make it win: [http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?t...](http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=threadring&lang=all&sort=gz)</comment>


The Coming Software Apocalypse - <AUTHOR> https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+TheAtlantic+%28The+Atlantic+-+Master+Feed%29&single_page=true <comment><USER2> Great article. Nice to see Chris Granger mentioned here, but I didn't see Eve mentioned anywhere, only Light Table</comment>


Ask HN: What are some things you wish you had learned earlier in your careers? - <AUTHOR> Curious about this as I am currently going through my first job search out of a startup.<p>Please be as specific as you can, and add as much reasoning or context on how this effected your work and career. <comment><USER3> My two bits of advice are: 1) always try to be the dumbest person in the room (i.e., surround yourself with smart people from whom you can learn), and 2) remove the phrase "it works for me" from your vocabulary when dealing with others. The latter may not be something you utter often, and if that's the case, then feel free to ignore it. :)

The first should be pretty obvious, but the second is a little less obvious. But think about it this way: if it worked for me, I wouldn't complain about it. Agree?

So there are two possible choices: either I'm wrong or your code is wrong (assuming you're an engineer). As a member of a startup, it is generally in your best interest to assume that the complainant is right and your code is wrong.

Assume that your code is wrong. Investigate it. Ask for more information if you need it. Track down the problem. If it works for you, then think of all the reasons it might not work for someone who doesn't know how it works. Think of all the edge cases. Clear the cookies. Bust the cache. Make sure it's not Wednesday ([http://gyrovague.com/2015/07/29/crashes-only-on- wednesdays/](http://gyrovague.com/2015/07/29/crashes-only-on-wednesdays/)). Whatever it takes to figure out why it's not working for the user (which may or may not mean it works for you!).

Maybe the code is wrong -- if so, fix it. Maybe it's a feature -- if so, explain that. Maybe the user made a mistake -- if so, identify _why_ and fix that.

Only after identifying the problem and potential solution do you go back to your complainant and say, "Here's what happened and here's what we're doing about it." If the complainant made a mistake, you can explain "it works", but you should also follow up with some way that you're going to make sure this doesn't happen again.

This is basically the single best way I know to resolve complaints. People respect you significantly more when you can always present a solution -- even if it turns out they were wrong all along!

<comment><USER4> This is great advice! A common sub-case of this is not throwing proper error messages/not validating inputs. Without that, users will be stuck when they make mistakes. If you stick your stand that "it works for me", you have lost the opportunity to fix it for others. When you realize that, you would have lost many users.

</comment></comment><comment><USER5> For me it was switching over from doing what others wanted to doing what I wanted. More specifically I was working at Sun and had been "in the market" for a bit over 5 years and sought out career advice about how people got to be senior most engineers (Sun called them Distinguished Engineers). The advice took me by surprise which was, stop doing what others want and start doing what you are passionate about and go from there. I've always been passionate about systems, whether they are operating systems, systems of government, or economic systems. Anything where there were lots of interacting parts which created something that none of the individual parts could do on their own.

Then my path switched from asking "What are people looking for that might be interesting?" to "Who is working on something interesting to me and how can I become part of that?"

It changed no only how I approached looking for jobs, it also changed how I felt about the CRUD stuff that came up in the middle of some of them. Just like a largely ignored my general education classes in school, knowing that I needed to pass them to get my degree. Staying focused on what I was building let a lot of the annoying stuff about earning a living sort of roll off my back.

I don't know that my life would have been largely different but I would have felt better about my path.

</comment><comment><USER6> Negotiate hard for salary and do not apologize for it. Save money.

</comment><comment><USER7> I wish I had not assumed that management , and the culture they create, are representative of a wider trend in an industry. This assumption was based on what is known as the "False Consensus" effect.

I graduated from the university (undergrad) in 2003 with a bachelors degree in computer science. I joined a company that was prestigious at that time. Soon after joining, I felt very disappointed in my career choice as all around me were teams that were outsourcing major roles to TCS and InfoSys consultants. Adding insult to injury, in order to receive their full severance packages, the full time employees were required to stay on long enough to train their low-cost replacement. Seeing this, I felt "awful" about the IT career path.

If I could go back, I wish I knew that IT wasn't as easily commoditized as the managers around me assumed it was. Further, I wish I knew that this company was not representative of a broader trend across the entire American economy. Knowing this, I think I would have doubled down on acquiring more tech skills and spent less time diversifying my skillset into business domains requiring tech.

On the other hand, a positive outcome of this early career experience is that how I reacted to that culture made me who I am today. I make the most of having depth in multiple disciplines.

</comment><comment><USER8> This first one applies generally when you are working in established companies more than early startups. Anyway, stick out difficult projects longer than you think you should. I jumped shipped too early on a couple of projects early on and missed out on some amazing growth/learning opportunities that could have saved me significant learning curves and frustrations later on, plus it could have helped my career even faster. Not suggesting anyone stick out an abusive situation where the team is abusive or the team is being abused by management, but stick out projects that are in trouble to learn from that adversity.

The other recommendation, do not let yourself get stagnant at a company just because they are paying you and the job isn't bad. Keep your eyes on the industry and trends to make sure you are not being used or missing great opportunities. Sorry to say but there are many employers through ignorance or willful deceit that will try to take advantage of you and you need to control your own destiny. I tend to think more is out of ignorance and fear then willful deceit but none the less it happens.

</comment><comment><USER9> Never learned JavaScript when I was starting out in 1995-8. Always found a way to hack things together, but only by adjusting existing scripts. It's frequently a choke point for me now where I have to delegate to someone else to fix something that's beyond me. At 38 and being very busy, I don't have (or make) time to learn it now.

I have worked for myself for almost the entirety of my career, so never had to face a job search or technical interview or anything like that.

<comment><USER10> I think you should forgive yourself a bit more: Back in 1995-8 most programmers never took JavaScript seriously because the "real code" was be done in a language like Java or Perl. We tend to forget that in a pre-web 2.0 era it was sort of seen as a hack to do small things rather than giving an app a front end.

Besides if you really wanted to make a career mistake back then it would have been to master Lingo.

<comment><USER9> That's correct! CGI-BIN folders in every site.

You might get a laugh out of this after your Lingo comment. I should probably consider it a career mistake but I've got by for a long time despite it - <USER2> settled on ColdFusion when it was an Allaire product, and have stuck with it since. Now using Railo. CF is not the trendiest of setups to use. ;)

<comment><USER10> First thought: "Wait, there's NO way ColdFusion can still be around today!"

Looks online "OMG!"

Although I think Adobe is still supporting Director, but alas my paying CD-ROM gigs ended in the 90s...

</comment></comment></comment></comment><comment><USER11> The number one biggest lesson I have discovered the last few years, and the number one thing I have seen that holds people back when it is missing is : You always have a choice, and taking responsibility works.

Choosing to blaming other people for whatever is not working for you, instead of choosing to take responsibility to create improvements that may work for you, is not something that works for you. Fabricating a story where you are a fake victim, complaining about things that aren't working for you, pretending things that aren't working for you are not your responsibility, and pretending you are fully justified in trying to blame others for what you declined to improve yourself, is just creating the very powerlessness for yourself you are pretending others forced upon you.

As soon as you blame someone else, you give away your power, you reduce your agency, and you pretend that you don't have the power of choice, paradoxically creating the very circumstances about which you are complaining. As soon as you take responsibility you increase your power, agency and your choices, reducing the likelihood you have anything to complain about.

Blaming others is a compelling fake narrative to be seduced by, because substituting getting real results with the fake pay off of pretending others are wrong, and by implication pretending you are right, has the color of "safety" because because it's a pay off you invent in your mind, rather than a result in the real world. So you don't have to do anything to create this fake pay off ( except delude yourself ) and even if you fail, you can still blame others, whereas if you fail while trying to create a real result, then you take responsibility, reflect that it was your choices that created this failure and think on this opportunity for creating improvements.

This carries over into hiring. You want to hire people who aren't seduced by blaming others instead of taking responsibility to create improvements. These people are more productive and create more results than people who don't take responsibility.

The number 2 would be : following your heart, let your feelings be your guide, and at every decision point, choose to walk closer to the life you want rather than further from it.

The number 3 would be : be brave, live, do things that bring you alive, speak, let your light shine, give of your unique contribution.

</comment><comment><USER12> People who are smart and cunning are dangerous and there are plenty of them out there. Technological feat is not enough in software industry as personal interest and politics is prevalent. If you really despise these things, it's better to think ahead and come up with a plan or work for yourself.

<comment><USER4> I have seen the opposite - People who are smart become more generous as they realize they don't have to be cunning to succeed. On the other hand, dumb people think others succeed by being cunning and justify they becoming more cunning. Since, these mediocres tend to be high in any large organization, politics sets in.</comment></comment>


Google whistleblower: the medical data of millions of Americans is at risk - <AUTHOR> https://www.theguardian.com/commentisfree/2019/nov/14/im-the-google-whistleblower-the-medical-data-of-millions-of-americans-is-at-risk <comment><USER3> Why does an ad company need my medical data?

</comment><comment><USER4> I salute you, O Whistleblower!

</comment><comment><USER5> The are HIPAA regulations supposed to protect the data but I take it people don't trust that there's sufficient oversight.

</comment><comment><USER6> We can only rely on whistleblowers to tell us truly what's going on behind the scenes with these kinds of things.

Kudos to the whistleblower for coming out.

Look at how this is done vs. how Apple is doing their health program. There's tons of transparency, they're openly talking about doing studies with Medical Schools, and the medical parties are publishing results.

</comment><comment><USER7> A single HIPPA violation for a single record alone is costly. There was a Ask HN post a few weeks back where someone posted a medical record horror story. They later deleted the post after pointing out that even programmers are criminally liable.

</comment><comment><USER8> This strikes me as a whistleblower fail.

The story isn't, "I know that something bad happened", as whistleblowing is supposed to mean. Instead they complain that essentially, "I am not personally privy to the consent requirements and privacy protections required, nor do I know what has been implemented." Which... congratulations on not being important, I guess?

But in particular, this person gives an emotional plea that people must be given the explicit opportunity to "opt-out", not from the medical research in general, but from any subset of research conducted in collaboration with Google, specifically because it's Google, rather than another research contractor.

I donno. Should Internet users be given the opportunity to opt-out from having their IP packets transit fiber owned by AT&T? I mean, sure I agreed to a bunch of stuff when I signed up with my ISP, but my provider is Comcast. They never mentioned AT&T. And some of those packets might not me encrypted....

</comment><comment><USER9> The recent thread on this: [https://news.ycombinator.com/item?id=21514952](https://news.ycombinator.com/item?id=21514952)

</comment><comment><USER10> Disclaimer: Googler here, my opinions are my own.

I have no non-public knowledge of this topic besides these two Guardian reports, and the Google public blog post on the same subject[1].

I do, however, have nearly a decade in non-Google work experience working in clinical documentation technologies for a company who had BAAs with literally dozens of health companies.

I simply do not understand the objection this whistle-blower is raising. As far as I understand it, the controversy is simply because Google is involved.

> Above all: why was the information being handed over in a form that had not > been "de-identified"

DeID is typically used at the edges of an IT system, and is tailored to the rights of certain users accessing the system. If you have a system that says, "A ha! There's a patient with a 5mm AAA with no evidence of follow-up! They need a procedure STAT!", you obviously need to have the original documentation to know who to contact.

There are ways to keep PHI (identifying info) separated from documentation, but if Google is both the cloud storage provider _and_ doing R&D, both sides of that system would fall on the Google side of the fence.

The bulk transfer of documents was almost certainly done via HL7v2 messages which, IIRC, don't have any built-in mechanism for redacting PHI, and if it did usually health systems lack the expertise do this consistently between all BAAs they contract with.

> I was worried too about the security aspect of placing vast amounts of > medical data in the digital cloud.

I mean, yeah, this is an important set of data. In previous years the issues were people walking out of the health system with their laptop and it getting stolen with 1ks of records on it. Cloudification has certainly reduced the vectors to steal large quantities of data.

> data potentially being handed on to third parties;

Their BAA specifically prevents this.

> adverts one day being targeted at patients according to their medical > histories.

 _Again_ , the law specifically prevents this use. These are all hypothetical scenarios.

> Full HIPAA compliance must be enforced, and boundaries must be put in place > to prevent third parties gaining access to the data without public consent.

There simply is no evidence that full HIPAA compliance isn't being followed.

> Employees at big tech companies having access to personal information

 _Thousands_ of employees at Ascension have this level of access. I personally had access to millions of health care records. BUT! There were auditing systems in place. If you attempted to use that access outside of the scope of your job, you're fired, no second chance.

> To quote one of my role models, Luke Skywalker: "May the force be with you".

Are you kidding me? Is this satire?

> In short, patients and the public have a right to know what's happening to > their personal health information at every step along the way.

In short, the concerns here are all hypothetical. There's no basis of any wrong doing. There is no proposal to actually address these concerns in a practical way.

[1] [https://cloud.google.com/blog/topics/inside-google- cloud/our...](https://cloud.google.com/blog/topics/inside-google-cloud/our- partnership-with-ascension)

</comment><comment><USER11> The real story here might be about how many exceptions are allowed within HIPAA, and how often health data gets transferred.

My understanding is that patient consent is not necessary for things like research, which Google's AI efforts could be argued to fall under. This makes sense in that requiring researchers to get consent for every piece of patient data would quickly become cost-prohibitive for many types of studies or introduce sampling issues (e.g. selection bias) or some combination of the two. Patient health data is frequently handed over to researchers, but HIPAA probably did not anticipate the case in which a powerful consumer-facing entity that is notorious for using personal data also could do legitimate research on health data.

Additionally, given that Google adheres to the HIPAA business associate agreement, healthcare institutions are allowed to give non-anonymized patient information without patient consent. A typical example might be a medical group that outsources its billing procedures. My understanding is that the medical group does not need to get patient consent in order to hand over medical data to the medical billing coders.

Someone who is more knowledgeable about HIPAA might be able to add to the discussion, but from the few details that have been publicized, it could be that Google is following the law, but people are surprised by what is allowed by the law.

<comment><USER12> >My understanding is that patient consent is not necessary for things like research, which Google's AI efforts could be argued to fall under.

HIPAA allows for entities to share data with other entities which provide a service to the original entity. For example, Hospitals may share data to a billing provider or a claims analytics provider. It's almost certain that Google is following the letter of the law.

</comment></comment><comment><USER13> I created this anon account because I don't want to lose my job.

I work at a major hospital/university as a research engineer and, 100%, the whole system is completely broken. Using our hospital and the 10 or so other major hospitals we work with as my source, I cannot come to any other conclusion.

HIPAA is constantly touted as the reason to push more and more CYA hurdles on to the staff's day to day interactions. One of the hospital I work with has an email system where you receive a notification (via email) that you have a message from email@example.com, in the body is a link you have to click through to reach a "secure" portal, then you 2FA, just to read an email someone sent _you_ (with no PII).

While the system that actually handle all the PII are basically rubber stamped with no real security reviews. Prototypes I've built that should never pass a security review, regularly do and get let out into the wild. I've been screaming from the top of a mountain for years and all I ever hear back is that it would cost too much, or prolong the dev cycle, or <enter reason here>. If I counter with "I can't believe this will pass HIPAA muster," the answer is always the same: "It passed the review and that is what matters."

When that something eventually happens, we can say we went through the proper vetting with the review team and that's it, next to no liability for building crap infrastructure. Of course the security review team will say, they used an analysis software that cleared everything so it's not their fault. Finally the company that made that software will say it's not perfect, but it got certified by XYZ, and just like that the whole thing blows over.

The hospital itself is what facilitates this shifting of responsibility, they pays millions upon millions of dollars for any software claiming to securely retain and protect PII, when all to often that software isn't even in beta, let all vetted and hardened. But hey, they got some certification that the hospital can point to and say "It wasn't us." Every year there is some new thing we are supposed to do that feign interest in security, but the weakest links just keep getting weaker. No one cares about your personal information, hospitals only care that they aren't liable when your personal information getting leaked.

I literally lie to my doctor, not because I don't trust her, but because I know that it's just a matter of time before my information is out there.

<comment><USER14> Stuff like this is everywhere. I used to work for a company that had millions of credit profiles. We had the chief of IT spending enormous amounts of time to make things "secure," but in the meantime junior PMs (and pretty much everyone else too) had access to a web portal where they could decrypt and inspect anyones credit information, address, name, etc. Inspecting the personal financial details of people in our system was something employees would do on a lark. We had a horrible password manager that nobody used, so the passwords that were employed to validate logins to this portal were literally the names of the employees with one or two extra characters. No 2FA, no IP address restriction, nothing.

We would deploy enormous resources to protect something if IT believed that had a legal requirement to do so (and make a lot of noise about how "secure" we were), but we would leave treasure chests of information sitting around in the open if there wasn't a box they needed to check saying "don't leave unattended treasure chests of private data in the open."

If anything this convinced me that the regulations surrounding this sort of thing are a joke. We don't need rules about security or how to build X - <USER2> will just see a list of boxes, check them, and then ignore everything not specifically enumerated. We need a white hat law for certifying hacking teams that can legally try to crack corporations with sensitive data. If they succeed, the company has to pay enormous fines _to the team that hacked them_ and solve the problem or get their certifications/contracts revoked.

</comment></comment><comment><USER15> My housemate is not careful with their digital privacy so their devices are all unambiguously theirs and the profile assembled by advertisers is likely cohesive across platforms.

Two days ago they said they've been getting a lot more ads about prescription vaginal creams lately. This wasn't happening a week ago. There is reason to expect that someone with access to their medical history would segment them as a likely target for such ads. The topic hasn't come up in their life recently so it would have had to have been from medical records.

To be perfectly clear: this probably wouldn't be happening if Google was telling the truth about not using the data for ads.

It's time to stop believing ad networks when they say they're "not going to use this data for ads" and "are doing everything they can to keep the data safe".

Singular anecdatum, I know, but if we find enough examples of similar occurrences we may be able to build a strong case that they're simply lying about the security of our data in their hands.

<comment><USER16> As a man, I had ads about custom fit bras follow me around the internet for awhile. This had nothing to do with my medical records. I read an article about a related company.

Sorry, getting ads about medical products is not evidence that the advertising platform has your medical records.

<comment><USER17> Facebook has been giving me ads for a quit smoking drug for a month or two. (Chantrix? Shows how well the ad worked. I remember there's a picture of a cold turkey.) I don't smoke now, and I've never smoked regularly. Machine learning!

</comment></comment></comment><comment><USER18> > Two simple questions kept hounding me: did patients know about the transfer > of their data to the tech giant? Should they be informed and given a chance > to opt in or out?

That's literally what HIPAA was created to address. The user doesn't have to opt-in to every single solitary business that touches their data, because there is a chain of business contracts that explicitly dictate what they can do, that originates with the care provider.

This is like "blowing the whistle" because your dentist sent your dental impression to a company to create a crown for you. You didn't "opt-in" to that dental company getting your impressions, because your dentist had a contract with them to cover what they would do with it in the first place.

Jesus christ.

<comment><USER19> > This is like "blowing the whistle" because your dentist sent your dental > impression to a company to create a crown for you.

I see a huge difference here, though. the company making my crown doesn't get my complete medical records. They get what is necessary to make the crown.

Google is getting everything, to use for purposes beyond the patient's immediate medical needs.

> That's literally what HIPAA was created to address.

That HIPAA allows this sort of thing to happen is a great reason to pressure lawmakers to improve HIPAA.

<comment><USER18> > Google is getting everything, to use for purposes beyond the patient's > immediate medical needs.

That claim was never made anywhere in this whistleblower account. Furthermore, it's illegal, and anyone working with health care records knows that. You can't use the records for anything other than what was covered by the business associate contract.

It's clear from the letter that the whistleblower literally has no idea at all what's going on, so they got frightened and yelled fire, in hopes that somebody who knows more than them will come and look for a fire.

If they did about 30 minutes of research, they'd know that any person whose records are covered by this agreement has the legal right to request from Google all information about how their medical records are being used. All they'd need to do is ask 8 random people to request records from Google, and one of them might be covered. They could then find out the answer, or at least have _some kind of evidence_ of a lack of process or oversight. They have provided none.

Multiple times in the letter they mention that the public must consent to this. There is no law requiring this, so really the whistleblower is trying to assert their own opinions about public policy, and using the veiled threat of malfeasance to get press time.

I'd expect this anonymous source to out themselves soon so they can lead some kind of public demand for more visibility (hence more regulation). Which might be good, except the HHS (which actually regulates HIPAA claims) already doesn't enforce it much at all. So it's probably just going to be used as a political tool to gain votes without actually improving people's lives.

<comment><USER19> > That claim was never made anywhere in this whistleblower account.

I may have misread the various reports and press releases, of course. If so, please do correct me. But I believe that everyone, including Google and the medical group, has mentioned this data is to be used to train a ML engine.

> You can't use the records for anything other than what was covered by the > business associate contract.

My understanding is that the ML use is covered by the BAA. Of course, I haven't read it, so I don't know, but it seems likely.

I seriously doubt that anyone is actually overtly and intentionally breaking any laws here.

</comment></comment></comment></comment><comment><USER20> I work for a large US based corp that focuses on technology and data services for the healthcare field. We have massive amounts of PHI for the majority of people who have visited a provider within the country.

We apply all sorts of stuff to this data, ML, AI or whatever other buzzy tech you can think of.

Most of this work happens within our own data centers but there is significant work done within public clouds.

We have BAAs (business associate agreement) with every cloud vendor we work with. We also have gone to extreme lengths to be confident that our cloud deployments are as secure (or more) than our on premises stuff.

However, none of that is unique. We're no industry trail blazers with adopting public clouds. Just about every other major player is doing this in a way fairly similar to ours

So, what I _really_ don't understand with this story is, did Ascension just simply give up their data to google without boundaries? Their BAA should be very clear that Ascension intends to use google's cloud services but not giving rights to their data to google.

It would not be unusual to engage a vendor or form some other partnership with another firm to work on problems or generate new products. I assumed that this is what Ascension and google were doing but this whistleblower and other stories make it seem like google just has free and clear access to this data outside of their relationship with Ascension.

Is that true? If so, that's crazy! Otherwise, business as usual?

<comment><USER21> I've seen a ton of responses from people in the industry along the lines of "this is normal" or similar. People who work on this stuff are incredulous that there's even an issue, since everyone's health info is already being uploaded to AWS or something. This is business as usual, they say.

The uproar is taken by people actually working in the business as a sign that the public are ignorant and misinformed. Things are actually HIPPA compliant, they say. This isn't a big deal, they say.

But perhaps the education should be going in the other direction, and the people in the industry should realize they are the ignorant ones for realizing that this is totally not OK for a huge number of people.

Realize that we are absolutely horrified that this data is being shared, that a reasonable response is to say that if HIPPA is OK with this then we need stronger laws, that we don't want faceless algorithms studying our most intimate personal and medical issues at companies we never had a relationship with.

And, especially, we are absolutely fucking certain that we want literally none of it to be seen by employees of the sociopathic tech companies that are surveilling every aspect of our life in order to better manipulate politics, markets, and our society.

The thing to take away here is that people are shocked and horrified at what's apparently business as usual.

<comment><USER22> But if they're just using Google as cloud storage/compute it's not being shared. Only very few Google employees would have access, they'd have very careful limitations and access on who accessed what - that's not the same as giving to google for some big AI experiment.

<comment><USER23> > _Only very few Google employees would have access_

That's "very few" Google employees more than zero. I expect _zero_ Google employees to have access to my medical data. Any number above zero is absolutely not acceptable to me.

> _they 'd have very careful limitations and access on who accessed what_

Yeah, just like Equifax, right?

<comment><USER24> > That's "very few" Google employees more than zero. I expect zero Google > employees to have access to my medical data. Any number above zero is > absolutely not acceptable to me.

Heck, people owning my medical data who are not my doctor/GP and related medical professionals is a big no go in my opinion.

Medical data is rather private.

<comment><USER23> > _Heck, people owning my medical data who are not my doctor /GP and related > medical professionals is a big no go in my opinion._

Not even "owning", but having.

Having even "anonymized" data is not acceptable to me.

</comment></comment></comment></comment></comment></comment><comment><USER25> What is the risk of exposing our health data? To me it is not so obvious, other than maybe embarrassment? Is it like how in our culture we don't like to talk about how much money we make? Why are all these things supposed to be secret in the first place?

<comment><USER26> Because your medical information can be used against you.

Want a new job? Nope! We don't want someone with your condition on our team.

Want to buy a house? Nope! An AI bot says may not live long enough to pay the loan.

Want to get some ice cream? Can't haz. When you swipe your electronic payment method the database says you're at a risk for diabetes.

There are thousands of other scenarios.

<comment><USER25> The can't get ice cream one is a bit far fetched eh? I'm sure that no matter if they knew my health history they would still sell me some ice cream :)

<comment><USER26> Not if the Point of Sale system won't let them.</comment></comment></comment></comment>