Boeing 737 MAX crash and the rejection of ridiculous data - <AUTHOR> https://philip.greenspun.com/blog/2019/04/08/boeing-737-max-crash-and-the-rejection-of-ridiculous-data/ <comment><USER2> It should be noted in the doomed Air France 447 flight, the plane activated the stall warning because of a high angle of attack that was leading to stall. (thanks pdx for the corrected info)

At some point the system rejected the data and stopped the stall warning because the angle of attack was so severe that it considered the data erroneous. This is speculated to have caused the co-pilot to keep pulling back on the stick and to maintain the stall because everytime he let the nose of the plane come down the stall warning activated again as the AoA was decreasing into the range that the airplane considered a real signal. [https://www.vanityfair.com/news/business/2014/10/air- france-...](https://www.vanityfair.com/news/business/2014/10/air-france- flight-447-crash)

A plane's instruments, the actions software takes, and it's interactions with the humans that fly the plane isn't as simple as an if statement.

If anything, fewer and simpler controls or automated systems are easier to debug and work around than a plane that has an internal calculus of what is valid data.

Air France 447 crashed because of a stall caused by a severe AoA that MCAS might have prevented (if MCAS simple pushes the nose down at high AoA it might. I am unsure of the exact implementation). MCAS obviously had a different impact on the Lion and Ethiopian Air Flights.

Basically, it's complicated.

<comment><USER3> Your basic point is spot on, it isn't simple.

The challenge I think is to keep two things separated, one is the flight control laws that the system is implementing to keep the plane in the air (to the best of its ability), and the other is the situational awareness indicators for the pilots so that they can tell what what the plane is "thinking" about how it is flying (or not).

The closest analogy I can come up with is the SQL explain command. That command will generate the complete decision tree for how records are included or excluded from a SQL query. The air equivalent might be display that shows the flight status, and the state of the instruments that are being used to determine that status. And then it is up to the pilots (or DBA :-) to figure out if what is happening is what they think should be happening.

To use this particular example, it is, in my opinion, negligent on Boeing's part not to include an indicator for every change in the flight control laws. If MCAS activates to avoid a stall it should always indicate that it is, and why it is activated. It has been reported that this was an "extra price" option for the jet, and it is _that_ choice that makes it feel negligent to me.

Generally, there seems to be enough indications with backups in the cockpit so that the pilot can ascertain what is going on with the aircraft reliably (backups and such), but what was missing here was, again in my opinion, was the rationale the plane was using for the flight laws it was implementing being available as well.

<comment> chupa-chups This is not the problem. MCAS in itself sadly is the problem.

They had to solve the issue that the pitch-up moment shouldn't be accelerating on its own just depending on AoA, which was (aerodynamically) inevitable without automated adjustments due to the placement of the engines.

They had to resort to the worst possible cludge, since they weren't even allowed to add new electrical systems (which would have caused a recertification and/or a retraining), so they resorted to an already existing system (assisted trimming by autopilot).

The effect of MCAS is only "fixed" by manually adjusting the trim, which involves moving a jackscrew which holds the last section of the elevator at a fixed angle. Unfortunately the required force of moving this jackscrew increases with the airspeed. There is no easy way out. A bit more background can be found here: [https://www.satcom.guru/2018/11/stabilizer- trim.html](https://www.satcom.guru/2018/11/stabilizer-trim.html)

Btw, the _extra price_ item was an AoA disagree warning. This is related to MCAS in a way like an odometer failure to traction control. It wouldn't have helped, if pilots stuck to their memory items and checklist (which they have to: they recognize runaway trim and have to react accordingly).

To get a slight feeling what it means to have runaway trim (without assistance which they had to disable in concert with MCAS) please have a look at this video: [https://vimeo.com/329558134](https://vimeo.com/329558134)

<comment><USER4> The video is gone (censored?), does someone have a backup copy?

</comment></comment></comment></comment><comment><USER5> With all due respect, this sounds too much like the kind of armchair quarterbacking that routinely appears on HN when avionics/politics/astronomy is mentioned, where a lone programmer feels competent enough to criticize an industry for missing "something obvious".

I mean, this particular change might have saved the particular 737, but I'd rather hear it from someone who actually knows how 737s fly.

<comment><USER6> Philip Greenspun is a highly experienced pilot:

[https://philip.greenspun.com/flying/milestones](https://philip.greenspun.com/flying/milestones)

<comment><USER7> He doesn't seem to be an experienced owner of a home outdoor thermometer though.

Or if he is, I need to know where he found one that goes to 452 degrees and why he even needs that much range. ;)

<comment><USER8> The thermometer sensor and circuitry might be perfectly ok, still the 452 number might be a single digit gone bonkers in the panel, so that data being transmitted (maybe also recorded?) is ok while the readout is plain wrong. A human acting according data being shown there might react very differently from someone reading the temperatures log. Sometimes software problems can be awfully subtle.

</comment></comment></comment></comment><comment><USER9> Do we actually have the raw data from the sensor in the flight recorder, or do we have the flight computer's account of that sensor's data?

> IF AOA > 15 AND AOA < 25 THEN RUNAWAY_TRIM();

And if the AOA is frozen at 16 due to some fault? What is a loss of signal interpreted as? Does it use last-known value? 0? 100?

How often does the AOA get sampled? Is there any attempt to smooth the data? Was the data corrupted (bit errors) during transmission/reception? Perhaps there's a flaw in the hardware (or microcode) protocol implementation of the data bus (or whatever it would be in this case) doesn't disregard packets with parity errors? Or perhaps it only checks for 1 bit of parity and it needs to check for 3? Or perhaps it's sending int64 and the flight computer is expecting int32.

It's all so simple to make completely uneducated guesses.

When Boeing made such obvious mistakes such as "don't re-command full-nose down after reset" that should have been caught be any competent user acceptance / QA testing, I have to call the entire engineering effort into question. I cannot assume they did even 1 thing correct in this system, and it really needs to have an independent review of the entire system hardware and software implementation.

<comment><USER10> You seem to be arguing multiple sides here.

Philip Greenspun's speculation is a plausible concern, that is all it needs to be a valuable point, that type of data limit handling ought to be considered as the system is looked at. I read his post that he was talking a specific _simple_ example, all the other types of things you mention could also be looked at. How the AoA sensors failed, any potential issues with signal handling, and especially what happened with the Lion Air AoA sensor repaired in Florida need to be investigated, that sensor repair sure seems to be. (BTW data handling in aviation is pretty interesting,

Greenspun is a fairly unique combination of EE/CS/past MIT lecturer/geek and experienced pilot, including holding an ATP certificate and flying for regional airlines.

The system as implemented could not have a "re-command full-nose down after reset". To start with there is not really a "reset" for MCAS. The pilot's only way to fully disabling MCAS continuing to do bad things is via STAB TRIM cutout switches. The STAB TRIM cutout switches are required to handle lots of other problems. e.g. runaway full nose down trim. The stabilizer trim system itself is a dumb as a rock, and has no idea where the trim should be set to if the power is restored to it. I suspect the only likely sensible behavior of the trim system itself is that it makes no automatic change. If MCAS is driving the trim wrong and if it was possible to remove the MCAS input to the trim system then the pilot should be able to reset the trim they want (with then hopefully functioning electric trim switches), or allow the autopilot to do it (outside of MCAS the A/P is the other automatic system that manages the stabilizer trim). The problem is that was not anywhere in Boeing's plans here, there was no way to separate MCAS going nuts and commanding extreme trim changes, from the actual stabilizer trim system.

However I do agree with your sentiment about (all the other) bad mistakes. In my view when a seemingly largely self-regulated group goes off and designs something with so many glaring issues (single AoA sensor source, lack of documentation and training, not even having a standard AoA disagree alert, etc.) and other possible issues (rationale of extension of MCAS trim authority, trim wheel forces needed to crank mechanical trim at stabilizer trim limits, Boeing slowness in responding to issues etc.) then _everything_ needs to be looked at. My hope is there are very thorough investigations, of the actual systems, of all the proposed remedies (which separately, I am not convinced are enough), of Boeing, and of the failure of FAA oversight here. And I hope that is done as fast as possible, and as slow as really needed.

<comment><USER11> > not even having a standard AoA disagree alert

Sorry to nitpick on one thing here; but I don't know that having an AoA disagree alert would be that useful. There was already an airspeed disagree alert, and AoA disagree strongly implies airspeed disagree, because AoA is involved in the airspeed calculation. Furthermore, knowing an AoA sensor is broken wouldn't be super useful without a workable procedure to do something about it; in the Ethiopia crash, the procedure followed was insufficient to regain control.

<comment><USER10> AoA disagree does not imply airspeed disagree. Indicated airspeed is purely a pitot and static pressure driven measurement. AoA is there largely as part of the anti-stall warning system, e.g. driving the stick shaker/pusher. An important point of having AoA sensors is to provide a stall warning systems that is independent of IAS, you don't want these systems crossed with each other.

B737 airspeed disagree I believe just requires 5 knots disagreement for more than 5 seconds between left/right IAS. No AoA involved.

There may be some confusion happening due to the events in the Lion Air B737 Max. The maintenance crew tried to address what they believed were ADIRU/ADR issues in previous flights... including as a possible source of erratic airspeed indications and airspeed disagree warnings. So there are some discussions of airspeed disagree in relation to that aircraft. There may be a lot going on there we don't know about yet, with say faulty ADIRU/ADR causing multiple problems. The ADIRU has inputs from the AoA vane but does not use that in calculating airspeed, it is using that to drive stick shakers/pushers and optional display AoA, and unfortunately in these cases to also drive MCAS.

Having a AoA disagree indication (and/or full AoA display option) _might_ have allowed more prompt diagnosis of problems, but only if pilots were aware of and trained on MCAS. And even then, yes it's a minor point. I was not intending to claim that itself is a deep solution here, just not having a disagree warning as standard was a bad decision. The entire system, and overall approach taken by Boeing seems tragically deeply flawed.

<comment><USER12> I think the Ethiopian preliminary report says they were in IAS disagree for the whole flight too. I don't know what caused the IAS disagree. It played a role in the crash, since its checklist disallowed lowering flaps (which would have disabled MCAS) or throttling back (which would have avoided surpassing Vmo, which likely made manual retrim impossible).

<comment><USER10> Ah now my addled brain finally wakes up, well maybe. What may be happening here is the ADIRU is doing AoA based static port compensation. It will normally be a small correction, and I was trying to keep that out of the discussion because I thought there there have been reports of erratic airspeed in previous Lion Air flights and that was not jiving with small changes of airspeed I was expecting (and now I'm going to reread the Lion Air preliminary report to check what was reported). But to start with you only need 5 knots to trip the IAS disagree, so sure I'll expect it could trigger that (and also possibly an altitude disagree). But potentially more intersting is what does that usually small static compensation change do when the ADIRU thinks the AoA is really out of whack, does it keep applying the "correction" or does it ignore it? So that question circles back to be similar to what Greenspun is asking in the original article.

</comment></comment></comment></comment></comment></comment><comment><USER13> On the other hand, one of the contributing factors to the Three Mile Island nuclear disaster was the fact that the temperature sensors were showing their maximum programmed value of 280 degrees Centigrade. The actual core temperature was far far higher, but because the engineers designing the reactor never thought of the meltdown scenario, they programmed the temperature gauge to cut off at 280C, rejecting higher readings as obviously erroneous. This (along with a water gauge malfunction) led the reactor operators to misdiagnose the fault. The operators thought the reactor was overfull with coolant and was at risk of overpressure, when in reality the coolant was draining away.

I wouldn't be so quick to dismiss "obviously incorrect" sensor data.

<comment><USER7> While this is true, the point that it's worth thinking through the nature and consequences of "obviously erroneous" sensor data is still a valuable exercise holds.

If the temperature is past 280, generally speaking the same steps to diminish temperature can be taken (... I'm speaking broadly; this may not actually be true of nuclear reactors and if it's not, additional sensors with larger ranges were definitely warranted if there's discontinuity in the safety and disaster mitigation strategies at temps higher than 280).

There is no amount of automatic airplane wing trim that can arrest a 70-degree AOA. When the sensor's getting 70 degrees, the failsafe operation would have been to back out of the control calculation and defer to pilots.

<comment><USER13> I agree. In the case of the temperature sensor, my opinion is that the correct behavior should have been to show no reading, or an error value. At least that way the operators would have known that they didn't know the true temperature inside the reactor. As it stood, they saw high, but constant temperature, combined with rising pressure. That indicated that the water level was rising inside the reactor, so they opened drain valves to let water out. That was exactly the wrong thing to do, and it contributed directly to the severity of the meltdown.

</comment></comment></comment><comment><USER14> > Beyond 25 degrees, therefore, it is either sensor error or the plane is > stalling/spinning and something more than a slow trim is going to be > required.

I am not a pilot, and I'm going to take this pilot at his word. But my first question would be, if we add this additional rule into the system (that runaway trim turns off above 25° AOA), will any pilot ever need to know about this rule?

If the answer is "absolutely not, never" then that's all well and good. But if there's some way-out-there scenario where the plane is wavering between 24° and 26° AOA, and in that scenario the pilot needs to be aware that the computer's trim behavior is switching back and forth between two different laws, then I'd want to ask whether that's presenting pilots with too much complexity.

There's a rule of thumb in software design, when we're thinking about designing a complicated system to solve some messy problem. Will users have to deal with the system getting things wrong? If the answer is pretty much never, then the system can be very complicated if that means doing a good job. But if the answer is that the system won't always be right, and that users will have to step in some of the time, then making the system complicated inevitably means that users will have to learn all that complexity. I wonder if that applies here.

EDIT: Ah yeah, top comment right now is about Air France 447. That's a very good example.

<comment><USER15> Thanks for sharing your perspective.

</comment></comment><comment><USER16> There are several inexcusably egregious errors in the design of the MCAS system, and this "solution" addresses none of them.

\- Single point of failure: The system makes command decisions based on the readings from a single sensor. The fact that nobody asked (or was bothered by the answer to) the question "what happens when that sensor fails?" is negligence.

\- No re-training of pilots: Pilots were not aware of new ways in which the plane might take command away from them, and were left in the dark with only seconds to react to a deadly situation. The decision not to train was a cost and marketing motivated decision that sacrificed safety, to the tune of hundreds of lives lost.

Slapping on heuristics to condition unreliable data is not a good solution for life-critical systems. As another commentor pointed out, this is armchair quarterbacking, and it is not good armchair quarterbacking. This article should not be here.

<comment><USER17> >No re-training of pilots: Pilots were not aware of new ways in which the plane might take command away from them, and were left in the dark with only seconds to react to a deadly situation

Both the Lion Air and Ethiopian Air flights recovered from the initial MCAS. The mistake was assuming that pilots could reliably recover from a runaway trim situation. That might have been true when that was a more common failure and when pilots had more experience flying with manual trim.

<comment><USER12> Agree with you, though as a small correction it now appears (according to Ethiopian preliminary report) they made it to 7k altitude AGL, not 1k AGL. Still not so much to work with when you're above the plane's max airspeed and trimmed all the way down.

</comment></comment></comment><comment> chupa-chups This video from "Mentour pilot" has been deleted (supposedly upon request from boeing):

[https://www.youtube.com/watch?v=EzgBft-79U8](https://www.youtube.com/watch?v=EzgBft-79U8)

You can see it here (european version of youtube): [https://vimeo.com/329558134](https://vimeo.com/329558134)

Or more info here: [https://news.ycombinator.com/item?id=19627525](https://news.ycombinator.com/item?id=19627525)

<comment><USER18> Strongly suggest everyone check out the video. It's...kind of shocking.

<comment><USER19> One thing I don't understand is that if the trim is stuck in a position that pushes the plane down, shouldn't the high speed (and therefore strong air flow) help to put the trim back in a neutral position rather than making it more difficult?

<comment><USER20> Because of the nose up elevator..

Here's the explanation in one picture: [1]

Here's the article the picture came from: [2]

[1] [https://3.bp.blogspot.com/-6c8hfXS8WO4/XKd93voscrI/AAAAAAAAF...](https://3.bp.blogspot.com/-6c8hfXS8WO4/XKd93voscrI/AAAAAAAAFw0/AXPqRNGD3kYefHUUWM4RwxvVsTKuTZNIgCEwYBhgL/s1600/Screen%2BShot%2B2019-04-05%2Bat%2B9.05.25%2BAM.png)

[2] [https://www.satcom.guru/2019/04/stabilizer-trim-loads-and- ra...](https://www.satcom.guru/2019/04/stabilizer-trim-loads-and-range.html)

</comment></comment></comment></comment><comment><USER21> Now, imagine you have 5,000 such checks in millions of lines of flight control system code, many of them interdependent, and you have to fly the ship to test each one. You need to schedule time with the test pilots (who have lives of their own) and get the data dump from IT post-flight. It's aerospace so all this undergoes review, documentation, and signoff, and it all takes time. How do you prevent a single check from slipping through? It's not easy. These software engineers and others in the process screwed up, but it's a failure of processes and not just forgetting a conditional statement.

<comment><USER22> I'm sorry i'll have to mention that the software can be thoroughly tested in simulation flights. Funnily enough i was involved with some virtualisation software used to test booking systems for Airports. If you can virtualise a booking systems, trust me you can virtualise the on-board flights systems.

<comment><USER23> Is this really true though?

From what I have read it sounds like part of the problem is that manually adjusting the trim wheel requires more strength than at least some pilots possess due to the mechanical forces on the plane. I don't think it's reasonable to expect simulators to replicate those types of forces.

<comment><USER12> But they do! See the Vimeo link elsewhere in these comments.

</comment></comment></comment></comment><comment><USER24> I accidentally copypasted my one time login code into the amount of bill payment and my ebank didn't refuse the sixty seven million dollar payment (I caught it before sending it). Needless to say, I don't have and never had even one percent of that money.

My brother one time actually succeeded in wiring ten million euros instead of ten million forints. The exchange ratio is 1:320. Obviously the account didn't have 10M EUR on it.

I just entered 300 USD as a courier tip into the food order app I just tried for the first time because I expected it'll just do 3.00 for me. I caught it before sending it.

All of these are lacking this kind of defensive validation. I recognize the irony of not even knowing the terminus technicus for it despite I am a senior sw developer with decades of experience.

<comment><USER25> I sell stuff on Amazon European marketplaces. I sometimes change the price; you have to do that on each marketplace (there are 5: UK FR DE ES IT) in the interface for sellers.

The decimal separator for UK/English is the dot; in all other marketplaces it's the comma. If you put a dot where a comma is expected it's silently ignored (so for example EUR 9.99 becomes EUR 999)... but "slowly": the dot you type appears normally, and then after some periodic ajax validation, it's removed.

I now know this and am careful about it, but the first few times I had my items priced at 100x their normal price, until customers emailed me to ask if this was normal.

It would maybe make sense to issue a warning if the new price differed from the old price by two orders of magnitude. But there's no warning.

<comment><USER26> I inherited a system that let you put AAA in the product price field and would persist it.

Validation is hard to get right so they simply didn't try, not even common sense.

I've spent years thinking about the optimal way to structure validation and I've never come up with an approach I really like on an architectural level so I do the usual approach of simple sanity checks on the client and proper validation against business logic on the server but even that isn't as isolated as I'd like.

</comment></comment></comment><comment><USER27> When working on my PhD in mechanical engineering, I created fairly large data compilations and I would include some assertions like this. I'd make them assertions because I considered data that had these faults to be not worth adding, so if the data wasn't a typo then I'd just cut it.

It can be hard to think of good tests for data like this. Sometimes my assertions were wrong, e.g., I thought R^2 couldn't go below 0. That's actually false.

One I've found useful to spot typos was to check whether a column which is incrementing is in fact incrementing. E.g., if a table is ordered by the pressure tested at, then obviously there's a typo is the pressure is not increasing.

</comment><comment><USER28> According to Captain Chesley Sullenberger (the former air force pilot who saved US 1549, dubbed the Miracle on the Hudson) that the existing avionics are not designed to take our best advantages[1]

         Humans are much better suited for the doers role with
         technologies monitoring us, instead of what we are
         currently assigning the human component as exactly the
         opposite.

[1] [https://www.fastcompany.com/video/the-future-of-aviation- acc...](https://www.fastcompany.com/video/the-future-of-aviation-according-to- captain-sully-sullenberger/Zigmp07v)

</comment><comment><USER29> If the leaked information so far is at all reliable, it seems that the MCAS system was obviously inappropriate for FAA certification; a single AOA sensor cannot be used as a critical component of a DIL B system, full stop.

The suggested change makes the system not actively crash the airplane under ordinary situations, but does not account for the increased load on a pilot due to the need to manually trim down under the conditions that a properly operating MCAS would be needed. If the MCAS is necessary to prevent crashes due to pilot overload, then this 10 character change is insufficient.

Sure the suggested change makes the cure no longer be worse than the disease, but doesn't address the fact that the cure was needed in the first place.

Another way of putting it is that the crash could have also been avoided by using this pseudo code instead of what is suggested:

        IF FALSE THEN RUNAWAY_TRIM();

but the MCAS system wasn't added by Boeing because they wanted to spend more money on the aircraft, so it (like the fix suggested in the article) seems likely to be insufficient from the point of view of passenger safety.

</comment><comment><USER30> If you read about Air France Flight 447, at some point instruments data went so ridiculously far from expected values that the stall warning stopped, because the computer did exactly what is suggested here: rejecting ridiculous data. So trying to fight the stall would... trigger the stall warning, adding confusion and cognitive load to an already dire situation (IIRC it is very likely, though we will never know, that the pilots didn't quite understand until the very end that the plane had stalled).

The "solution" described in the article seems to add more complexity and surprises to a system (MCAS) that already behaves in surprising and unexpected ways. Engineering is hard, adding a band aid on top of an ill devised clutch without carefully thinking, writing, and peer reviewing all the possible consequences, is unlikely to help.

<comment><USER31> There is a difference between the AF 447 situation and what is recommended in the article.

What is suggested is that the aircraft refrain from performing risky manoeuvres based on such wildly out-or-range sensor outputs.

This does not mean that aircraft should suppress warnings of such sensor readings from the pilots.

</comment></comment><comment><USER32> Maybe someone can illuminate this.

This comment seems to say that Boeing has recently implemented a bunch of fixes to prevent MCAS from activating in various scenarios, which allegedly make MCAS safety a non-issue:

[https://philip.greenspun.com/blog/2019/04/08/boeing-737-max-...](https://philip.greenspun.com/blog/2019/04/08/boeing-737-max- crash-and-the-rejection-of-ridiculous-data/#comment-325105)

I'm not knowledgeable on planes, but some parts of this make me wonder if these fixes will cause the opposite problem.

For example if the two sensors disagree then MCAS now won't activate. How long until a crash happens because of that new behaviour? Either MCAS is necessary safety equipment, in which case this sounds dangerous, or it isn't, in which case why bother?

<comment><USER33> You are correct. The rather humorous aspect is that they really can't stick to the "no retraining" claim or really the claim the plane is airworthy because the plane is only certifiably airworthy when MCAS is operating.

So in short, yes. Their fix is introducing problems elsewhere. This characteristic is what has many thinking their was a series of grievous process, business, and regulatory errors layered on top of the known poor engineering decisions that came together to set the stage for these catastrophes.

</comment></comment><comment><USER34> The problem with this is the idea that this kind of software would ideally be written or specified with "if" statements.

I'm not sure about the argument that AOA values of more that 25° constitute an error, but a more plausible design would be a module that monitors the signal from the AOA sensor and classifies it into various advisory categories. That module could get complex internally but still have a clean interface, providing data of the form (AOA, Advice) where "Advice" would be an indication of the modules' conclusion or recommended action.

Rejecting ridiculous data is potentially a very subtle ML problem. To suggest that it should be taken care of in such a simplistic ad hoc way really doesn't do justice to the problem.

</comment><comment><USER35> I refuse to take any US domestic flights on 737 MAX until it is decomm and engines are replaced with the ones that were intended for its body.

Boeing is not going to fly its execs on domestic flights for certain number of months to regain consumers confidence.

</comment><comment><USER36> This needs more than an IF statement. This needs calculation of the rate of change of the variable, in addition to observing the value of the variable. Rate of change calculations are atleast an order of magnitude harder than just reading the variable. Doing it for infinite number of variables, in embedded, memory constrained, real-time systems is hard. So this wasn't going to be fixed via trivial 10 characters of code. But, yes, a more complex system would have avoided it.

</comment><comment><USER37> The sensor reading could have just as easily been 24 degrees rather than 70 and caused the same crash w/ the author's proposal.

Not saying or excusing the failures that led to this crash, but it seems like an oversimplification of the needed solution to suggest that ridiculous data be thrown out, with the benefit of hindsight being used to determine where the threshold of "ridiculous" is.

</comment><comment><USER38> Assuming that there are many software subsystems that need angle of attack input, it shouldn't be the responsibility of every one of those systems to try and determine if they are receiving bad input from the AOA indicator. Rather, there should be one angle of attack (AOA) sensing software system which feeds AOA data into all the other dependent systems. If the AOA sensing software system cannot determine a reliable value, then it should feed a value of "I don't know" to the downstream subsystems.

Then, all possible expertise about how to determine if the AOA input is valid (and I'm sure there are many, many, such factors, redundant physical sensors being just one) can be directed to that one AOA sensing system.

If MCAS gets an input of "I don't know" from the AOA sensing system, then clearly it is going to disable itself. So, a complex decision has been turned into a very simple one.

If you agree with me, then this is a really important example of why separation of concerns is so important.

</comment><comment><USER39> I have nothing to comment about this article, but this other one contains very detailed information: [https://www.skybrary.aero/index.php/B38M,_en- route_south_eas...](https://www.skybrary.aero/index.php/B38M,_en- route_south_east_of_Addis_Ababa_Ethiopia,_2019)

</comment><comment><USER40> Despite all we've read about this subject, are we really still implying that this is a software problem?!

Very basic rules of commercial aircraft design were violated during the development of 737MAX.

No point in listing all the issues here again. Let's just point out one: There should have been more than 2 AoA sensors feeding MCAS. No excuses.

<comment><USER9> > Despite all we've read about this subject, are we really still implying that > this is a software problem?!

We don't know it's _not_ a software problem (in addition to lack of hardware inputs). We don't know that anything in the system is fit for purpose whatsoever. That's what an obvious exclusion of basic failure testing tells us. That the system had NO critical oversight.

</comment></comment><comment><USER41> All the talk about how crappy Boeing engineering here was is bullshit and speculation and I am surprised PG participates in it. What we can discuss objectively here is incident response in which Boeing allowed the situation to continue after the first crash. How did they not run hundreds of hours of simulations, code reviews etc, etc on the system assumed to be at fault? How did they not immediately change the safety features associated with MCAS to be free and mandatory for everyone? Engineering mistakes happen and are hard to prevent. Business mistakes like this are a sign of terrible culture, lack of priorities and are an existential thread to the company.

<comment><USER33> It's not really speculation. The proof, as they say, is in the impact crater.

The only mystery left is, what is the nature of the paper trail that led to this catastrophe?

Was there malicious malfeasance? Overt and irresistible pressure to certify at all costs?

Was it all just a tragic mistake? We don't know. We only know the physical systems that contributed to the crashes, and some of the motivations that would have contributed. The technical implementation can be roughly inferred by any programmer, and it doesn't take a rocket scientist to figure out a ball was dropped somewhere for a plane development program to fall afoul of such a foreseeable failure state.

<comment><USER41> "The technical implementation can be roughly inferred by any programmer", "such a foreseeable failure state"... how many years of experience do you have?

<comment><USER33> How rude. Here I was thinking we were having a civil discourse over the Internet. More than born yesterday, less than since the Moon landing.

Regardless, my assessment is based on most juniors I've worked with. By their third year most seem to have already grasped the need to test for boundary conditions, and to ensure proper error handling for GIGO failure. Any 1 year+ with at least FizzBuzz levels of understanding can be handheld toward it with the right nudge, and in fact, the less experience they have the more eager and likely they tend to be to pick up on error handling since they haven't yet developed sufficient skills to be able to get their head around the "test you don't need to write" because the result can be inferred from a test at another level of the system (a frequent coping strategy that starts creeping it's way in with increased levels of familiarity with a complex system).

Any problem grokking the above points is usually solved with an impromptu exercise and lecture where I have the junior play the part of a computer until they realize just how much the computer "doesn't know", and has no capacity to derive from reasoning, unless it's actually coded/implemented to. I've not yet had a junior who failed to grasp this to some degree (though a recent one is giving me a run for my money), and become capable within a couple months of inferring two to three-function away error states to test for. Within the year, I can typically point them at an arbitrary code block and get back a reasonable testing surface.

Which brings me to my next observation, where I think you may be attempting to make a point:

 _If I run into juniors of 1-3 years experience who need coaching to fully understand what I explained above, then perhaps the average programmer is not capable of inferring what I claim._

To which all I can say is, my observation may be skewed, because I'm a bloody paranoid polyglot of a tester when it comes to safety critical systems. Even when I was pre-collegiate programming calculators, the more someone else actually depended on something, the greater the lengths I'd go through to test things before cutting them loose with anything I was producing for them. The THERAC-25 postmortem is bedtime reading for me, and I've pushed myself to understand computer science and software engineering as more than mere 'coding'.

If the argument then, is that I'm an atypical representative of my software composing brethren, then I'd like to know why in the $deity's name we're not triple-checking safety critical code at system integration time, seeing as we can assume this level of inattention to detail by the average programmer. Especially given as the languages these types of systems are implemented in are typically _not_ the most 'friendly' languages.

This suggests cultural issues, undue pressure to fast-track approval, disincentive to raise red flags that could impede delivery, or an "over-the- wall" hyper siloing of expertise/responsibility that lead to the least experienced in complex system implementation being blindly trusted by those who had the experience to realize something was horribly wrong.

If the above doesn't assuage any concerns relating to my experience, I'm afraid not much else will.

<comment><USER41> You think I was rude asking you about your experience. Now think how rude this unsubstantiated allegation of obvious simplicity of the code in question is to the person who wrote it -- with the weight of hundreds of lost lives on their shoulders. These control systems can get arbitrary complex. We don't know anything about the hardware this runs on and what it has to interface with. We don't know the constraints and age of the codebase. Nothing. To assume that this boils down to a simple if statement is something I would expect from a recent college graduate, or someone who has only worked at a web startup, not a person with 5+ years of real world experience building complex systems. I agree about all the points about testing and business processes. We have enough evidence to conclude that unforgivable mistakes were made there (and I point to that in my original comment).

<comment><USER33> >You think I was rude asking you about your experience.

You asked for a number. Who I am, doesn't factor in. The experiences and insights I can bring to the conversation do. Of which I provided more than enough to get you in the right ballpark experiencewise.

>We don't know anything about the hardware this runs on and what it has to interface with.

These are essentially networked embedded microcomputers, likely utilizing various potential protocol stacks such as

CAN, CAN FD AFDX® / ARINC 664 ARINC 429 ARINC 825 Ethernet CANopen for networking.

They are likely highly constrained, and must be compliant with DO-178B/C, which includes a need to verify the software down to the op codes spit out by the compiler.

The most popular languages for this purpose are known to be C, C++, FORTRAN, and Ada.

There's this wonderful place called the Internet where Engineers and other really dedicated people share information about what they use to do things.

>Arbitrary complexity Is a possibility, but tends to be bounded by the fact humans still need to be able to implement and verify the systems they make in a reasonable amount of time. Which coincidentally, seems to have missed a few layers or so given we're here talking about this.

The world has very little that can't be found with a little digging, and in the interests of saving time, we tend to reuse technologies when appropriate from things like, cars, in other things, like airplanes.

If you can gain a mastery of how to network and program computers in general, you gain insights into how other physical systems, even though they aren't Turing machines, interconnect and propagate information and forces.

If you can then understand engineering principles well enough to decompose complex things into a network of simpler basic parts, and understand how to employ mathematics to analyze and predict the behavior of those systems, you can quickly formulate broad guesses about contributory factors to a failure state given the even small amounts of information.

And if you say all that's impossible to appear in one person, I don't know what to tell you. I'm not asking you to have faith, I'm asking you to think, question, imagine, and connect the dots between what information is available out there.

But hey, what do I know? I'm just a guy who objects to having credibility pidgeonholed based on some number instead of the content of what is being communicated.

I apologize if I sound aggravated or hostile, but I do not appreciate it when something as tightly regulated as aircraft out of the blue starts killing people, and the reason looks to be a lack of scrutiny/verification, rushed implementation, intentionally sparse communication, and unethical sales practices for whatever reason.

There are ways to do things, and there are ways _not_ to do things. I expect a leader of an industry to at least show a level of effort such that I can entertain the benefit of a doubt that gross incompetence Or greed was not a factor. I have no such illusions left to me based upon what I've been able to work out. The cause is somewhere in their culture or business practices, and I want it ripped out into the light as an example to everyone, everywhere.

I don't care half as much about what happens to the people involved as long as it is enough to dissuade anyone thinking of doing the same thing from going down that path.

</comment></comment></comment></comment></comment></comment><comment><USER42> I have no practical experience with sensors but for those gained from a few university courses.

In one course, we were re-assembling systems on a weekly basis, and the touch- screens we used would all return somewhat quirky offsets, so I had to calibrate the input after each re-assembly because I could never be sure which module I was working with. I learned to never really trust a sensor.

And this was an undergraduate course. I would have assumed that in safety- critical applications, in a mature industry, performing every possible check on a sensor reading would be the obvious thing to do.

Was this particular case just out of the ordinary, or is it really that uncommon to do that?

</comment><comment><USER43> There must be all kinds of neat things one could do with access to the raw sensor data. Perhaps manufacturers should be forced to make it available, so owners can hook up a laptop and run independent analysis software in real- time.

I imagine a frozen Angle Of Attack sensor would stand out like a sore thumb to a neural net that has access to all the sensors. In fact, I imagine it would return a suspiciously constant value even considered on its own.

This doesn't seem all that difficult, really. Teams at engineering universities do more challenging stuff for their master's project.

</comment><comment><USER44> I am surprised that an aircraft control system would not have fault tolerance built in that will recognize a physically impossible condition.

</comment><comment><USER45> I mean of course Arizona does get hot enough to melt street signs: [https://www.accuweather.com/en/weather-news/its-so-hot-in- ar...](https://www.accuweather.com/en/weather-news/its-so-hot-in-arizona-that- street-signs-and-mailboxes-are-melting/70002032)

Maybe those were metal ones a 400+ degrees does happen? I mean survivor bias means we won't have met anyone from Arizona who went out in that heat :D

<comment><USER33> Completely off topic, but I hate Accuweather. The guy that runs it has lobbied left and right to keep the NWS and NOAA from being able to just deliver weather products because they'd cut into his revenue stream he's developed by charging people for access to public data.

Anyway... End rant.

<comment><USER45> I agree entirely, but we all use adblockers right? :D

</comment></comment></comment><comment><USER46> Why don't these planes use internal gyroscopes and compare the plane's orientation wrt gravity rather than external sensors?

</comment><comment><USER47> How does it feel to enter a world where software is killing people ?

Driven a new car lately ?

<comment><USER48> > How does it feel to enter a world where software is killing people ?

How is this new? Software has been killing people for a long time.

<comment><USER47> Smaller groups, yes it has. Software is in everything now so you have to actively avoid it.

</comment></comment></comment><comment><USER49> This 'article' is very low quality and is written by a layman. If his main source of information is one line of wikipedia, he shouldn't be telling experienced engineers how to write flight software.

Furthermore, it is incredibly unsafe to reject so-called "ridiculous" data completely.</comment>


AIG may get saved... by Credit Default Swaps - <AUTHOR> http://www.thedeal.com/dealscape/2009/10/aig_credit_default_swaps.php

<comment><USER2> Ah - scam de jour. How nice to see the World's Best People haven't lost the touch.</comment>


5M Bid request/s, 2ms max response time – The Road to Damascus - <AUTHOR> https://github.com/eloraiby/fs-pacer/blob/master/fs-pacer.md <comment><USER2> Interested to know what Rust was missing. I built an ad exchange last year and it has been great. I have been using nightly builds, mostly for access to async/await, and it has been very fast and stable.

I have had to submit a few pull requests to various projects along the way, but didn't find the ecosystem prohibitively lacking.

<comment><USER3> Would you mind sharing what libraries you are using?

<comment><USER2> actix, cdrs, rusoto, cadence, diesel, futures/tokio, serde, chashmap, slog, postgres, criterion, r2d2, chrono

That isn't all of them but those are the main ones.

<comment><USER3> Thanks, that helps!

</comment></comment></comment></comment><comment><USER4> TechEmpower's Plaintext scenario is currently limited at 7M RPS due to network limits, though it uses a 10Gb NIC. Knowing that the Plaintext scenario is a very simple HTTP request (standard headers) that returns "Hello World!", how close to network saturation are you with 5M in this case with only "2 Gigabit Ethernet cards"?

</comment><comment><USER5> Did you consider Vert.X? it's built on Netty and has it's own Linux epoll driver, async, and fiber support. It's impossible to know if it would be faster, but likely comparable and way less work than rolling your own.

In techempower benchmarks it exceeds 2 million http requests/second and it's a full REST framework.

And if you use the fiber support through Quasar you can pretend most things are normal blocking code.

Have to tried it or is this a case of NIH?

</comment><comment><USER6> I spent a few cycles in media buying and later in sell-side ad tech. Please do say what you will about advertising and its effects on the web, but I will say this: it is a world of fascinating tech. As a buyer I experienced janky pacing all of the time across various platforms, because this is a HARD problem. We had to manually adjust campaigns on a daily basis to ensure pacing worked properly. It was common to stop a campaign and overspend by hundreds of dollars while all of the caching spun down.

I'm fascinated to see they are running that all on a single node. Its a massive amount of state aggregated from billions of events that needs to be served at extremely low latency, but couldn't it be partitioned somehow??? Google Fi/Spanner and BigTable have certainly been developed to support these issues. I've been trying to dig up what infrastructure powers Google AdX, but I haven't found anything. AdWords seems to be tied to Spanner, but AdX is/was an entirely different beast. In any case I'm quite certain that it isn't running pacing on a single, gigantic node.

<comment><USER7> As an anecdotal data point, I once configured a test campaign on Doubleclick Bid Manager (now Google DV360) about two years ago that I needed some quick exposure on. So I set a budget cap of 100$ just for safety and didn't do any targeting, so I was effectively bidding on half the worlds' ad inventory. What I didn't check or notice was that pacing wasn't set to even, but to Flight ASAP.

Suffice to say, I spent 730$ within _seconds_, so fast actually Googles systems couldn't even switch off fast enough to prevent 7,3x overspend, and the only thing that prevented stupid me from a five digit spend was probably choosing an unusual ad size.

Fascinating stuff indeed :)

</comment></comment><comment><USER8> Glad to see this kind of stuff written with a strongly typed language instead of Python.

<comment><USER9> Why, they wasted months evaluating with an obsession for statically typed languages that have so far not produced anything more quickly or markedly better than that which others are producing with less pedantic languages.

<comment><USER10> Static typing should be viewed as a tool for long term prosperity/health of a codebase rather than competing in the short term with a Python/s (etc.) - Modern one's can, but static typing isn't always obviously useful until it is.

</comment></comment></comment><comment><USER11> Is this complete note? You didn't mention what you end up using.

Is it golang or pony or F$? CoreFX mention in the end confused me more.

<comment><USER12> I work with Wael. Development is still ongoing. One implementation uses Golang, the other uses F# with a library that wraps libuv for faster network performance. Pony was used to write the stress-testing client for both implementations.

<comment><USER13> Do you use libuv on go side as well?

<comment><USER14> You don't want to use C lib in Go for performance reason.

</comment></comment></comment></comment><comment><USER15> Curious if there are numbers for other languages in high performance applications.

I am learning Clojure so I would like to know if anyone knows of the highest performant applications written in it.

</comment><comment><USER16> > (scala? That's another story for another day)

This makes me curious - was it the language or the runtime characteristics?

</comment><comment><USER17> It's kind of sad that all this engineering effort was spent to essentially make the internet a worse place for everyone and waste users' time and attention.

Imagine if a crime syndicate would brag about their efforts to make their worldwide criminal activities more efficient.

<comment><USER18> Do you legitimately think the world would be a better place if gmail, youtube, flickr, reddit, EVERY search engine, and basically every web content site disappeared?

Because that's what happens if you don't have web advertising. Free things disappear without revenue.

Or maybe you'd prefer to go back to the days of randomly-targeted or "PUNCH THE MONKEY" ads. Because THAT'S what happens without ad auctions and targeting.

The reality is: advertisers and ad-supported sites WANT to show you a relevant ad that you're likely to click (modulo obvious bad actors). That's how they get paid. Anything else is, by definition, "[wasting] users' time and attention."

<comment><USER19> Everything you describe could exist as a paid service, and I suspect would be much higher quality that way.

<comment><USER20> A paid youtube? Not sure they would be worth it without the endless amounts of free content by free users.

Facebook? There are plenty of sites like facebook without users. Charge for an account.. reduce the userbase, the platform loses value without the free users.

<comment><USER17> A paid Facebook means it would be designed to actually serve you (instead of wasting your time) and people would use it to connect with friends instead of liking endless shitposts.

<comment><USER20> All your friends or anyone you want to contact would need to be paying customer.

If it is just friends why waste money on facebook just to talk. Most would rather spend that money on a more focused community/hobby like gaming and connect ingame.

</comment></comment></comment></comment></comment></comment><comment><USER21> The Best Minds of My Generation Are Thinking About How To Make People Click Ads (or serve them efficently)

<comment><USER22> Take a look at finance / trading. Things are even worse over there, but they don't blog about it.

<comment><USER21> Used to be a semi-professional poker player, which seems to have a high crossover with finance/trading types for obvious reasons. I enjoyed the debates, but sometimes it was a bit overwhelming how everyone had convinced themselves that they were magically providing a ton of value to the world and that's why they were filthy rich. Also the massive intelligence that seemed to only work when applied in one domain. Didn't mind taking their money, but never pretended I was providing society a useful function while doing so. :)</comment></comment></comment>


Soylent CEO's shipping container home is a 'middle finger' to LA, locals say - <AUTHOR> https://www.theguardian.com/us-news/2016/jul/27/soylent-ceo-rob-rhinehart-shipping-container-home-la <comment><USER2> Gabriel flouted zoning bylaws, plunked down a building with inadequate security and no permits, ran disruptive parties, left trash lying around which only encouraged vandals and lowlifes. Basically, he acted with the entitlement and arrogance of the nouveau riche. And now he's wondering why they hate him.

You don't live in a bubble. The people who live around you DO matter.

</comment><comment><USER3> I think the people in the neighborhood have a legitimate complaint considering it's "open to the public" which is not permitted. Just because you own the land doesn't mean you can do whatever you want with it. That's why we have zoning laws.

But I have to laugh at all of these people indicating they are entitled to this land as some sort of free park, which I assume, was likely private land, not public land before it was purchased by the Soylent guy:

 _" Residents have long lobbied to keep the rugged hilltop, which is popular with hikers and joggers, a de facto park free of development."_

If they want it to be a park, why don't they pool resources together and do that and then set it up as a park? Everybody wants to complain but nobody wants to backup their complaining with their own money.

<comment><USER4> I don't understand what "open to the public" means. I suspect that a complaint on those grounds is grasping at straws from a coalition of NIMBY neighbors and entitled hikers.

If I buy a property that people have been illegally trespassing on for years and put a house on it, is that open to the public?

I would think you would have to explicitly post permission to be on private land because I assume it's implicitly not permissible to trespass...

<comment><USER5> Probably, just not secured to prevent people from entering. The complaint is likely that the property is a nuisance and/or hazard due to failure to properly secure the building/property (it doesn't help that the building/container wasn't permitted).

If you own abandoned property you are usually required to board up windows/doors or otherwise prevent people from entering as well as place no trespassing signs on the property.

For permitted active residential construction you can usually get by with construction fencing and no trespassing signs.

It doesn't seem like he is actively developing or living in the property; more or less just using it as a occasional getaway.

</comment></comment></comment><comment><USER6> What's wrong with that if he owns the land? Would it make any difference if he painted the container green instead of red?

<comment><USER7> Because apparently having an "abandoned or vacant building open to the public" is illegal. You also need to have the correct permits.

Even for things on your own land.

As to the color, no, it won't make a difference. It's only one of many complaint or problems given.</comment></comment>


Why does Siri seem so dumb? - <AUTHOR> http://www.recode.net/2016/10/12/13251618/mossberg-apple-siri-digital-assistant-dumb <comment><USER2> >>>But I suspect that people don't ask those questions because, after trying a time or two and getting no answers or wrong answers, they just give up on Siri. <<<

Absolutely this. I mean, you try a few of this these "long tail" queries and you eventually say "Fuck it!" I attempt these "long tail" queries (which to me, that term sounds like some shitty play-it-down excuse from Apple) weekly just to see if Apple is finally getting their act together in regards to Siri usefulness. I am consistently disappointed and never surprised and delighted.

I try to screenshot "Siri fails". Here are the last few in my screenshot album:

\</comment><comment></comment><comment></comment><comment>-

"Open last screenshot" → "You don't seem to have an app named 'last screenshot' We could see if the App Store has it [App Store]"

"Share this with my wife" (Photo was open in Photos.app) → "I'm sorry Joe, I'm afraid I can't do that"

"When is the last time I exercised" → "Interesting Question, Joe"

"What year was this song recorded" (In Music app with song playing) → "Interesting question, Joe"

\</comment><comment></comment><comment></comment><comment>-

We're living in a world where we can do more and more without using the screen as an interface. It's happening. I'm worried that Siri is a low priority product for Apple right now and they will soon be scrambling to play catch up if they aren't already.

EDIT: I'm hoping that Apple's recent buddying up with IBM allows for some of Watson's intellect to seep into Siri. Who knows, maybe in a future keynote we'll hear Tim say, "Ladies and gentlemen, Watson is coming to your Mac and iPhone!"

<comment><USER3> FWIW none of those work with the all mighty google's personal assistant either on my nexus 5x.

<comment><USER4> Out of curiosity, is this the the actual "Google Assistant" you're referring to? Or simply Google Now?

I believe there is a difference, but that's purely speculation on my park. I'll be able to test it more in ~2 weeks when my Pixel XL arrives.

<comment><USER3> Ah good point - this might be google now. It's the voice search feature on the search bar that persists on all home screens.

Didn't realize they might be different products.

<comment><USER4> I _believe_ they have added ML in the Assistant, along with some new features like context recognition. Granted, it's of course difficult to say how much is shared between Now and Assistant.

</comment></comment></comment></comment></comment><comment><USER5> _" It also can't distinguish between the question of "who is" a person and a request for that person's contact card. For instance, I have a contact card for Apple CEO Tim Cook. When I ask, "Who is Tim Cook?" Siri shows me the contact card, not his bio."_

Tech reporter humblebrag of the year.

<comment><USER6> I don't see it as humblebragging (any of us could create a Cook contact / ReCode readers likely know Walt Mossberg's relationship to Apple) but, worse, a faulty premise.

Odds are, if you have them in your contact database, you already know them; you're not going to want Siri to give you their Wikipedia bio.

Siri has myriad faults, and thankfully someone of Mossberg's stature might push Apple to address them, but this is not one of them.

<comment><USER7> If you are asking specifically for _who_ somebody is, it should tell you who that person is - not give the contact card, because that card does not tell you who somebody is. At least, that's what I would expect if I asked who somebody is.

<comment><USER6> I'm presupposing the contact card has their job position/title, and therefore still answers the question – also presupposing the original (odd) premise of wondering who someone already saved in your phone is.

The Venn overlap between the set of "contacts in your phone" and "people with Wikipedia bios" is likely rather small. Hence why I think it's a faulty premise to complain about Siri defaulting to contact card when these two sets do intersect.

<comment><USER8> That's not the point either. The point is that if I asked you, a human, "Who is Tim Cook" and you replied "123-555-1212" or "firstname.lastname@example.com" I would be a little dumbfounded.

If I'm asking the question - maybe a friend is nearby who doesn't know them and I'm too lazy to explain - then I want to know _who_ they are, not _what_ their contact info is.

<comment><USER9> I could imagine some scenarios when I ask a human assistant "who is Joe Bloggs" and it would be quite reasonable for them to answer "oh you've met him, he even gave you his business card".

<comment><USER10> Sure, but only after telling you who Joe Bloggs is, because that's what you asked, you didn't ask 'Have I met Joe Bloggs?', an assistant (human or virtual) that doesn't actually help isn't going to keep their job for long.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> My wife is exactly the customer that's given up on asking Siri questions and uses it for timers. I can agree that Google does a better job at answering questions, but I don't like that Google requires access to just about everything in my life to be at all worthwhile, which is generally why I avoid google services. I realize I'm a minority but I don't like their all or nothing approach to slurping my data.

<comment><USER12> Does Android really answer questions? The only one I know to work is "How high is Mount Everest?". For everything else, it just reads the first sentence of the Wikipedia article, or presents a search. I'm using a German phone, so an English one might work better. But mine doesn't even know "What color is the sky?"

<comment><USER13> The thing that always impresses me about Google Now and their voice parsing in general isn't so much about speaking answers back to spoken queries but just how good it can be at guessing what you mean.

My favorite example was a couple of years ago when I was at a bar. There was a song on the jukebox ("Coffee Pot" by Cajmere) which, for reference, is a house music song with a repeated vocal sample that says "It's time for the perculator [sic]". I assume they mean "percolator" but that's how the guy pronounces it in the sample.

Anyway, we were joking around and even though the bar was quite loud with people talking loudly all around and music playing, I pulled out my phone and asked "Ok, Google: is it time for the perculator?" I pronounced it incorrectly like the guy in the song and within seconds, I had results. The top one was a link to the video for "Coffee Pot" by Cajmere on Youtube and the rest were links to other stuff regarding the song.

Now, this was at least 2 years ago and possibly more. The response to my silly joke of a question wasn't so witty as to give me a spoken "it's time for the perculator" or anything like that....but it could parse my words despite all the noise and competing speech around me, it understood a mispronounced word, and it knew that this lyric sample referred to a song with a different title (Coffee Pot) and linked me to the video.

That was seriously the first time I was really impressed by their voice assistant even if it was a ridiculous request. No idea what the result would be on today's Google Now or tomorrow's Assistant (guess I'll find out when my Pixel gets delivered). Still, even if just from an engineering and software angle, that made me grin a bigger geeky grin than I might have imagined.

<comment><USER14> I find all of the voice responses from both to be tediously slow. I'm a fast speaker, and I like it when people speak at a fast pace with me.

I like that Google seems to just display information for may of the responses vs. trying to cram in some overly long witty reply or verbatim reading of the text that takes too long to speak back.

Having some sort of speed/succinctness setting would be helpful in that regard.

</comment></comment></comment></comment><comment><USER15> > When I asked, "What is the weather on Crete?" Siri gave me the weather for > Crete, Illinois, a small village which — while I'm sure it's great — isn't > what most people mean

Most of the narrative about Apple's maps products has simply been "Apple bad, Google good" and hasn't looked much deeper than this (with the exception of Justin O'Beirne's cartographic commentary, which is remarkably detailed though much of it is just a matter of taste). But Mossberg really hits on something here:

Apple's geocoding is way below par. The cartography is superb IMO. The routing is very good[1]. The source data isn't bad at all. But the geocoding is really error-prone. I've asked it for directions to Milton Keynes, a town with a population of 250,000 just 40 miles from here, and it won't give me anything other than the nearest branch office (in a completely different town) of a company whose HQ happens to be in MK. I've asked it for directions to the village of Brize Norton and it flat-out refuses, sending me instead to the RAF base of the same name, even if I ask for "Brize Norton village", "Brize Norton School" and so on.

Yes, geocoding is famously hard. But as an OpenStreetMap volunteer I see that for anything other than granular street-level addresses (where we don't have the data), our grassroots geocoder is more reliable than that built by the most valuable company on earth. And as feedback to OSM shows, bad geocoding is the easiest way to make people think your entire map product is no good.

[1] except there's no bike routing... but hey Apple, if you want top-notch bike routing, give me a call ;)

<comment><USER16> Route information with Siri doesn't seem to be useful even if they know the location. I asked Siri (in central London) how to get to heathrow the next day. It just opened apple maps and showed me how to walk there. I guess I used maps for in pedestrian mode before, but that answer was absolutely useless.

</comment></comment><comment><USER17> While I ask Android for occasional content (while watching tv, "How tall is blah blah actor" etc) I usually use Androids ability to get my usual directives. "Directions to ..., Weather in..., Call <business> in <location>..., Set timer for...". It does both very well.

I stay signed out and opted out of all Google Now features and tracking so this is just it does by default. Voice is my primary ui for my phone. It is very useful and I find it parses my input correctly nearly everytime, it's very rare for it to error.

Meanwhile, my wifes iphone 6 gets basic commands wrong frequently and she spends minutes typing all the time. Siri isn't losing, it has lost.

I don't think this actually matters to the iOS population however. It's well known they don't buy iphones for features.

edit: A sample question from the article "when is the presidential debate" is handled fine as an example

<comment><USER18> On the occasions when I have witnessed it, _every_ attempt my spouse makes at using Siri in an ordinary, reasonable way _always_ ends in cursing and shouting.

It reminds me in many ways of the handwriting recognition on the Newton--which is to say it appears to be only a hairsbreadth from absolutely useless.

<comment><USER17> The most frustrating part of it for my wife is that she has great enunciation and I'm a mumbler.

<comment><USER19> Also, accents. Google sells phones all around the world and has worked very very hard to get training data from everywhere. For example, it has no trouble with my Indian accent at all. Siri... She's very American.

</comment></comment></comment></comment><comment><USER20> The main gripe I have with Siri is that is still can't understand bilingual (german/english) conversation. This makes this service unusable for me. How do you ask Siri for a song, a movie, a person name which happens to be english, when the device is set to german?

<comment><USER11> Honest question, do other assistants offer that? Just curious.

<comment><USER21> Yes, I'm a native Spanish speaker and Google Now works great with bilingual queries for me, that has been improving in recent years.

I always select English as my default language in my phones, some years ago if I wanted to call someone with a Spanish name saying something like "Call Ramon Hernandez" or asking for directions "Take me to Periférico de la Juventud" I used to had to fake an American accent. Today I don't have to do that anymore, I can speak natural Spanish and then it can understand my Mexican accent when I speak English. So it's not just improving in bilingual queries, they have focused in accents and mispronunciations.

</comment></comment></comment><comment><USER22> Siri seems so dumb because she is dumb, incredibly annoyingly dumb. Not compared to humans, but compared to what Apple could have achieved, what others achieved some time ago. Examples see other comments and: ... "Stop searching the web" "OK" … 1 minute later: "open my email" "I found this on the web for 'open my email'" (... fixed by now? ...)

"here is what I found on the web for 'what is Yen's email address'"

O. M. G. ...

</comment><comment><USER23> I wish Siri wouldn't ever show me web results; If I'm talking to my phone I don't want to click around or often even look at it. Everytime Siri offers to search the web for something, it's a fail.

Siri is great for setting alarms and timers -- much faster than using the UI.

</comment><comment><USER24> I recently switched back to iOS after using Android for a couple years and one thing I found shocking is how little Siri had improved. I only use her for creating timers these days. I'll try to schedule events but I find her success rate at adding things to my calendar accurately is < 50%. Try correcting a time to a just-added event. She can't do it.

I asked her for the nearest gas station the other day while driving and I think she responded with a list of google search results? Which is almost comical. I'm not going to grab my phone and sift through Google search results while behind the wheel.

<comment><USER25> nearest gas station works for me. i use it all the time. perhaps it's the phrasing. either way, i agree with the general sentiment. only use it for timers and directions while driving

<comment><USER24> I'm in Canada which perhaps has something to do with it.

</comment></comment></comment><comment><USER26> >It puts much less emphasis on what it calls "long tail" questions, like the ones I've cited above, which in some cases, Apple says, number in only the hundreds each day.

And every one of those "hundreds per day" is another person one step closer to disabling Siri. Which is what I did last month. Siri got things wrong so often, it was worse than useless.

<comment><USER27> It strikes me that they are still struggling to solve the questions that get asked thousands of times a day. So, yeh, it'd be nice to have the long tail work too. But, clearly this is still very much a work in progress from all vendors.

</comment></comment><comment><USER28> I'd like AIs to have settings so they start engaging me, rather me always having to engage them. e.g ask me what my tasks are today, or other types of interactions initiated by them at a propitious time.

</comment><comment><USER29> Because voice recognition from all the major providers goes at it completely backwards, being ready to answer any of the world's questions in every situation.

In the real world nothing goes like this. We start with context and work outwards from there. And I don't mean know-everything-about-your-personal-life context, there's no need for that either.

I'd say more but that is what I am solving at Optik. We're a bit over a year into development and things are really starting to pick up. Using Cortana in the Hololens is making it painfully obvious just how close but how far off the mark remains most voice command software.

</comment><comment><USER12> What I don't understand is why we don't try to solve this using a huge fact database derived from natural language parsing of the web. Basically the same approach as deep learning - something that was unfeasible 20 years ago, but due to massive improvements in processing power now works.

What I'm thinking of is basically SHRDLU [1] on steroids. Parse a ton of web pages. There are great natural language parsers that can parse most well- formed English sentences. Start with simple sentences like "Golden delicious is an apple" or "Barack Obama is the President". Then you store this in a Subject-Verb-Object database (I just learned that this is called a Triplestore [2]).

Every statement gets a plausibility value. Deal with ambiguity by adding multiple interpretations of a sentence (with different plausibilites if available). Assign an origin (e.g. website, author, quoted person ...) to each statement. Then, you could query this by asking "What do mice like?"... and it would make "Subject: Mice, Verb: like (enjoy), Object: ???" and return a list of solutions, ordered by plausibility.

Does anybody have any insight into why this isn't done or wouldn't work? It seems wierd that I can't ask my phone simple facts about the world, other than those whose form have been hardcoded.

(Now that I think about it, the opposite would also work. Hardcode a ton more commands. Hire 100 people, let them sift through the most common queries. Watch a few dozen testers, add add all queries they try to use. Instead of throwing computing resources at the problem, this would throw cheap labor at it.)

Anyway, it boggles the mind that I can't shout "OK Google, play 'itsy bity spider' on youtube" when my toddler demands it but won't release my phone :-). It opens a search and shows what I want as the first result (probably customized from my history), but I have to go the last mile myself.

[1] [https://de.wikipedia.org/wiki/SHRDLU](https://de.wikipedia.org/wiki/SHRDLU) [2] [https://en.wikipedia.org/wiki/Triplestore](https://en.wikipedia.org/wiki/Triplestore)

<comment><USER30> Obviously there's still a long way to go, but yes, Google is working on everything you suggest and more.

This is actually Google's bread and butter, and why Mossberg found so many cases where Google Now does better than Siri. It's not clear how Apple can catch up, either, given Google's massive advantage in training data.

</comment></comment><comment><USER31> Apple and Google are both shooting themselves in the foot by insisting on keeping their voice products within walled gardens.

Amazon is doing an amazing job of allowing the community to extend the voice functionality of their Amazon Echo. You can build an app that does just about anything you want and release it on their Amazon Echo store. There are already hundreds on there, and it's only a matter of time before someone releases a truly conversational A.I. integration for it.

My family uses our Amazon Echoes (all 6 of them, scattered strategically around the house) for hundreds of little things throughout the week. I control all of my lights, my thermostat, my entire home theater (via custom voice activated scripts). I even use it to make my tempurpedic bed vibrate ("Alexa, turn on bed vibration"). I use Echo for timers, and my morning alarm to wake up (which also triggers my bed vibration). I use it to query wikipedia subjects, perform quick math calculations, order new paper towels...

If you gave me a couple of hours, I could even whip up an Alexa integration that would let me open my garage door and remote start my car in the morning ("Alexa, turn on my car's air conditioner")! It would just be an Alexa command that triggered a custom script which would log into the web interface for my car link and would click the remote start button. Easy. I love building this sort of thing.

The possibilities are endless when you have an open, extensible platform. I don't understand why Apple and Google are being so dumb and close-minded, and still don't understand this!

</comment><comment><USER32> Siri seems dumb because we are shown demos that suggest the tech is far more advanced than it actually is.

</comment><comment><USER33> Due to my (very slight) Eurotrash English accent, Siri doesn't even get over the speech recognition hurdle four times out of five.

When it does, it replies 'Here's what I found on the web...'

I try tricking Google's app with unlikely stuff but it gets it right most of the time, and gives a reasonable answer too.

<comment><USER34> This always cracks me up: [https://www.youtube.com/watch?v=NmWRhhvf60Y](https://www.youtube.com/watch?v=NmWRhhvf60Y)

</comment></comment><comment><USER35> I have mixed feelings about Siri. The voice recognition has definitely gotten much better. The domains that it knows about isn't increasing nearly enough. That's not even a matter of AI it's just doing the work. For instance knowing what's on tv.

It does come in handy when I'm driving and for reminders. It seems to do pretty well with directions, playing songs, playing podcasts, and messages.

I much rather tell Siri "remind me not to forget my lunch when I get out of the car" than try to do that with the reminders app.

</comment><comment><USER36> When I updated to Sierra, I was excited to try out Siri (I only ever user Android's assistant).

I quickly gave up on all the features but using it to control music.

Then, I figured out that it's useless at that too unless I set specific playlists.

"Play Underoath" > "Finding results for 'under oath'"

... It's even worst for artists with strange names.

How am I to say "Play 30h!3".

It is even worst when you try to have it figure out music in other languages.

...

I turned off Siri.

</comment><comment><USER37> I don't use Siri much, but I tried to ask "How many hours until 7 am", and it gave me the wrong number.

</comment><comment><USER19> The difference between Siri and Google Now is the real downfall of iPhone (at least for me). I have enough comfort level with Google Now today that I feel comfortable just flinging any question at it when I'm in hands-free mode (say, when driving) and expect a useful answer of some sort. Trying Siri on friend's or colleagues phones, it's always a frustrating experience.

And this doesn't even include the automatic actions taken by Google Now. Alerting me about travel times, flight delays, sports scores etc. etc. without even being asked. Ultimately, in Google's vision, the phones are rapidly becoming just a tiny window (no Microsoft.. not you), into all the knowledge and power that actually resides on the cloud. Not so much for Apple, whose core competency is, and has always been, in hardware design and supply chain management.

</comment><comment><USER38> I also think Siri fails with Text messaging. I should be able to have an IM discussion using just my voice. But the Siri interface is simply not that involved. I'm constantine having to say "Read Latest Text Message".

</comment><comment><USER39> Best use for Siri: it can turn off all alarms on its own. Helpful for when I set 5 or more occasionally for something, I don't need to keep scrolling the list and disabling them all one by one.

</comment><comment><USER40> Add even a faint hint of an ESL speaker accent, and Siri becomes outright useless even for the mundane tasks... like setting a timer or sending a message.

</comment><comment><USER22> Serious question: Is it possible that Apple keeps Siri artificially annoyingly dumb because they still have to pay Nuance billions for all this 'useless chatter' on a usage bases?

</comment><comment><USER41> I don't know about the new Google Assistant, but I found Siri to work much better than Google Now.

I've been an Android user for about 10 years and just switched to iOS within the last week.

Siri is really the only feature on iPhone (so far) that I have found I prefer to Android.

While it's far from perfect, so far I have been quite pleased with Siri. It is able to let me do my most common tasks via voice commands. 9 times out of 10 Google Now couldn't even tell what I was saying. I attributed it to my somewhat strange accent, but Siri has been nearly perfect on this front. Good thing though as I loved the Swype keyboard on Android and hate typing on the iPhone; I fat finger everything on that tiny virtual keyboard.

</comment><comment><USER42> Siri on Sierra is noticeably dumber than Siri on iOS, and has a much worse sounding voice, I wonder why that is.

</comment><comment><USER43> > Me and a friend carrying furniture down the stairs. Im sorry i didnt catch > that..

>At the Chinese drive in & Then.. (ProTip: Cooking recipes line by line for a win)</comment>


Justin.tv - Paul Graham at AngelConf - <AUTHOR> http://www.justin.tv/clip/15489851960ba5d5 <comment><USER2> This was really awesome to watch, for a few reasons.

I've never seen PG in person or on video. Based on his essays and comments I was expecting someone, for some reason, with a beard.. and maybe a medium- paced speaker with nothing in particular too exciting about him. I don't know why, but I just wasn't expecting someone all that exciting. Turns out he's an overgrown kid who is genuinely enthusiastic and into what he does. I cannot wait to get involved with people like him, and I'll read anything he posts with that enthusiasm in mind. (And I vow never to try to guess what somebody is like based on online text... it's just impossible)

Another reason I enjoyed it was that I'd heard of AngelConf, and I know what Angel investors are, have worked with a couple, and I've read about it, etc, etc. But up until watching this video, I realize I only knew the world in a surreal sense. There are _actually_ conferences with people devoting their time and energy into investing angel-sized amounts of money, and it seems like an awesome niche to be involved in. I cannot wait until I can afford to place bets on businesses. Up until now, my only outlet for this has been my time, not my money.

As a side remark... this "surreal problem" happens to me a lot, particularly with social things. I think it's because if you show me a chunk of code, I can look at it and eventually figure "yeah, ok, I see what's going on here." And if you explain to me how some social system works, and it makes sense, I'll think "yeah, ok, I see." But what I've not factored in is all the information you send/receive when being involved in that social system, the subconscious decisions you learn to make, things like that. I guess that experiencing the "Blink" effect first hand, or in this case as a derivative of watching video, effectively opens the doors in my mind and shows me there's _a lot_ more to understand and explore. I really love it when that happens.

The third major reason I enjoyed this was the content. I never really read anything from the Angel's perspective or watched much videos, so it was really cool to see the types of advice they have to juggle around, and what things are typical in the progress (just by how they're talked about). How much to invest? How much percent to take? How to valuate? It's great to know that in the world of seeding businesses, it really _is_ as much an organic process as it seems like... something I'll recall when I'm not sure if I'm doing things right, crossing into new territory, etc.

Thanks, PG. Ever thought of doing videos like these to supplement essays? It really helps points get through a lot better.

edit: this is also good: <http://www.justin.tv/clip/1c8fd85375b>

<comment><USER3> _Based on his essays and comments I was expecting someone, for some reason, with a beard_

This is probably because I often go for a while without shaving, but I feel obliged to shave for talks.

<comment><USER2> That explains it. I know beard-authored text when I see it.

</comment></comment></comment><comment><USER4> As a new yorker, my favorite quote is:

 _You have to be good. Really. That's probably why angel investing hasn't taken root in New York - you can't screw people"._

</comment><comment><USER5> Just curious: what is YC's motivation for hosting AngelConf?

<comment><USER6> more angels = more money to invest into YC companies after graduation. i.e. in the talk they pretty much covered that VCs mostly want to invest in the million dollar range.

In this day and age a lot of startups simply don't need that much money

<comment><USER7> This is tied to the the root motivation in YC: do good for society by enabling more startups to exist. That's is a side effect not directly tied to investments in YC companies, as new angels will certainly also invest elsewhere.

This isn't to say you're wrong. More investors in YC companies is a direct motivation for the conference.

</comment></comment></comment><comment><USER8> It seems like a lot of people there were full time investors or independently wealthy people who dabbled in it. My conception of the common angel investor outside of SV, due mainly to watching my father go through this with his own personal investments, is that of a doctor or lawyer or other high income bracket person looking to invest some spare cash but who doesn't really know much about investing per se, but that didn't really seem to be the focus of this conference.

</comment><comment><USER9> Informative, fast-paced talk. From an entrepreneur's perspective it's an interesting behind-the-scenes perspective when dealing with angel investors.

<comment><USER7> The crowd there is interesting. They want to become angels. This is a ridiculously awesome way to intro them to the biz. PG telling them to "be good", for example, starts everyone out of the same page: we all want to get rich together.

Good for YC's portfolio, good for the new angels, and good for society.

</comment></comment><comment><USER10> Highly recommend Naval Ravikant's talk as well. Those two were by far the most informative.

</comment><comment><USER11> Here is a transcript and notable quotes I wrote up from all the speakers from the justin.tv video check it out <http://tinyurl.com/d8tpzd>

</comment><comment><USER12> Will we eventually get an online transcript of this talk?

<comment><USER13> Yes, pg said (about 3:30 into the video) this talk will soon be online as an essay.

</comment></comment><comment><USER14> Related Cips > Ron Paul, Ron Paul, Ron Paul, Ron Paul...

Is PG the next Ron Paul?! PG 2012 (oh, I get it now...)

\</comment><comment>

On a related note, I like the section on decisiveness.

I read that when Warren Buffet considers investing in a company, he prefers them to have a number ready immediately on the spot. Indecisive business owners are to be avoided.

</comment><comment><USER15> really enjoyed that. loved the bits about being decisive, why angel investing has not taken off in NY, and during a recession startups will take money at any valuation, which is good for all of you because your already rich.</comment>


Asus: Nexus 7 sales climb toward 1 million a month - <AUTHOR> http://news.cnet.com/8301-1035_3-57542796-94/asus-nexus-7-sales-climb-toward-1-million-a-month/

<comment><USER2> It's at a brilliant price point and I think that's why the sales are doing pretty well. I wonder if the Nexus 7 is also sold under cost similar to the Kindle Fire?</comment>