Are we shooting ourselves in the foot with stack overflows? - <AUTHOR> http://embeddedgurus.com/state-space/2014/02/are-we-shooting-ourselves-in-the-foot-with-stack-overflow/ <comment><USER2> If I'm reading the testimony correctly, there is actually no evidence that a stack overflow caused unintended acceleration. The idea is that Toyota used 94% of the stack, and they had recursive functions. If (big if) the recursive functions used enough stack to cause an overflow, memory corruption could happen. If (big if) that memory corruption happened in exactly the right way, it could corrupt variables controlling acceleration. And then, maybe, unintended acceleration could occur.

But that's a far cry from the stack overflow actually causing any cases of unintended acceleration.

<comment><USER3> Isn't 96% of stack usage a bit high? When you are dealing with recursions?

You pay a lot for those cars, can't they at least put better electronic hardware. They probably have less than my phone from 5 years before

<comment><USER4> > When you are dealing with recursions?

They _shouldn 't_ be dealing with recursions. If stack corruption is what caused their failures, inappropriate testing played an important role IMHO.

> You pay a lot for those cars, can't they at least put better electronic > hardware.

This isn't how it works for cost-sensitive designs. You don't hear people boasting about how they have a quad-core car computer and how the touchscreens from their motor control are perfect for Facebook interactions.

The way people think about this is, if half your RAM memory never gets used, then you used twice more than you need and your module is more expensive than it should be. CPU use never increases past 20%? It's about 80% more powerful than it needs to be. And so on.

"Better electronic hardware" (in the sense of "more powerful" or "faster") also introduces additional complexity. This means more difficult constraints in testing, longer and more expensive verification processes, additional non- deterministic behaviour and so on.

Not that their system wasn't at fault. It was, but throwing more hardware at it wouldn't have made it better.

<comment><USER5> But having some additional capacity available gives them the ability to do field upgrades. The firm I worked for a while ago had to undertake a very expensive hardware refresh because there just wasn't any way to get any additional bug fixes into the field -- they were down to 20 bytes free. In something like a car, that you know people are going to drive for 10+ years, you need that extra space not only to make bug fixes, but also to comply with new legislation (such as brake override), and also to offer a few new features to your customers.

<comment><USER6> Another valuable engineering lesson; Always Have Headroom. If one designs or operates to the limit there is no margin for error. Resilient systems can get pushed beyond their acceptable limits and recover.

<comment><USER7> 1+.

IMO, You need several sets of limits: standard limits posted to the consumer, engineering limits posted to the techie/maintenance guy/developer/etc, and actual limits.... each of those is comfortably beyond the others. Know the actual limit, but design well under that if at all possible, because the system _will_ be misused.

</comment></comment></comment></comment></comment></comment><comment><USER8> Even if you work in a gc language with a vm and all memory errors are checked, here is the major, MAAJOR, wisdom you should take with you:

 _The crucial aspect in the failure scenario described by Michael is that the stack overflow did not cause an immediate system failure. In fact, an immediate system failure followed by a reset would have saved lives, because Michael explains that even at 60 Mph, a complete CPU reset would have occurred within just 11 feet of vehicle's travel._

We have seen this scenario played out a million times. Some system designers believe it is acceptable to keep the system running after (unexpected) errors occur. "Brush it under the rug, keeping going and hope for the best." Never ever do that. Fail fast, fail early. If something unexpected happens the system must immediately stop.

<comment><USER9> When you're talking about a control system "Fail fast, fail early" isn't necessarily an option. In this particular example, while it's arguable that temporary loss of control might be better than the actual outcome, it's still a pretty unacceptable thing to happen.

At the point where you are trading off one failure state for another, you need to think very carefully about which one is truly worse, and sometimes "Fail fast, fail early" is a _much much_ worse choice than the alternative.

<comment><USER8> Experience (which always should trump theories about how things should work) proves that you are wrong. What would have happened in this case if fail fast was used and the hardware had immediately resetted upon the first byte being written outside the stack area? Likely the bug would have been caught and fixed during testing because debugging a randomly resetting cpu is comparatively easy. It's a hundred times better than people being _killed_ because small unexpected errors are allowed to accumulate.

<comment><USER9> >Likely the bug would have been caught and fixed during testing

Sure, for testing, you should absolutely fail early and loudly, because there are no consequences for doing so. But in the real world, and especially in control systems, "failing" can cause more damage than persisting in a corrupted state.

> small unexpected errors are allowed to accumulate.

I'm not arguing that you should just ignore errors completely, just that the correct response to a particular error must be very carefully considered, and your blanket suggestion to reset immediately is often a terrible idea.

I think a far more appropriate response to a single byte being written outside of the expected area, is to shut down all _unnecessary_ processes, and tell the driver to pull over. This would be _far_ safer than just randomly disabling the controls, however temporarily.

<comment><USER10> Sending uncontrollable accelerations for hundreds or thousands of feet is safer than a maximum of 11 ft of coasting?

Yes, there are always trade offs to be made. But a corrupted system is utterly unpredictable - if a control port _can_ do something, it eventually will, at the speed of electronics. On the other hand, a shutdown is entirely predictable. We design for it, because every system _does_ eventually fail and shut down. It's a 'normal' mode of operation in this sort of system design. That tends to weight the design heavily towards very fast resets and redundancy in mission critical (people die if you mess up) systems. I can design a system to handle a shut down control board (limit the robots arm's travel, etc). I often can't do anything if the arms are waving about randomly, under power.

I'm not opining randomly; I've worked in flight control, robotics, UAVs, and factory machinery. Until my current job, I've always had to worry about killing somebody. You really can't let software that controls dangerous equipment continue to run in a damaged state unless the rest of the system has a supervisor mode that can override and limit the system's behavior. Even then, I'm having trouble thinking of a scenario where I would prefer to leave that process running vs shutting it down.

<comment><USER11> What does it even mean to shut down the control systems on a moving car? All stop? Coast? What's the physical behaviour that isn't likely to kill someone?

<comment><USER12> At most, it means you're coasting with no power assist for steering or brakes -- you'll need some extra braking distance, but that's what the shoulder is for, and it'll take a bit more than the usual effort to turn the wheel, which means that getting _onto_ the shoulder, especially if you're in a middle lane when all this goes wrong, is going to need to be smartly done.

Taken all together, the impression I get is of a problem in driving skill that's roughly as difficult as a blown tire at highway speeds -- possibly a bit easier, actually, considering the deleterious effect a blown tire has on steering control. An alert and competent driver should be able to handle either situation without posing a deadly danger to herself or anyone else.

The same, I think, cannot reasonably be said of uncommanded acceleration. With a blown tire or a failed ECU, all you have to do is use your ordinary driving controls to get off the freeway and bring the vehicle to a safe stop. With a throttle stuck wide open, you are suddenly a race car driver, only you're neither in a race car, nor in a race. You _can 't_ bring the vehicle to a stop with the ordinary driving controls, at least two of which -- the accelerator and the brake -- are no longer responding properly or at all, which is itself frightening and disorienting to the driver. In order to bring the vehicle to a stop, the driver must shut off the engine with the key, at which point the problem reduces to our failed-ECU worst case above, just with some more speed to burn off.

But there is no circumstance, in either the normal driving regime or even any other _ab_ normal one, where turning off a moving car is a proper or safe response to any situation, which is why almost no driver has ever given the slightest thought to doing so -- and when your car's speeding up past ninety all of a sudden, and you're not telling it to, is maybe not the best time to be thinking up new ways to interact with your car that you've never thought of before. At a guess, I'd say some people whose Toyotas ran away on them were able to come up with the idea in time, and they mostly survived. Others ran out of time before they thought of it, and those unlucky souls mostly died.

<comment><USER13> I agree with most of what you've said. But point out that you didn't go with the worst case example, loss of steering / brakes on ice.

Unwanted acceleration on ice would be worse though. I can't imagine, in rush hour conditions, being unable to control my acceleration. At best, you'd rear- end someone, and their car would stop you.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> First I was annoyed at yet another upvote fishing blog post on stack overflow. Then I read it, while I was annoyed at getting caught by the catchy headline that I conciously despised. Then I saw that it was not at all about some forum on the web and now I cannot stop smiling.

</comment><comment><USER15> Isn't recursion—even if it's indirect—disallowed completely when doing embedded C programming for safety-critical devices?

UPDATE: Yup, #70 on the MISRA C rules: [http://home.sogang.ac.kr/sites/gsinfotech/study/study021/Lis...](http://home.sogang.ac.kr/sites/gsinfotech/study/study021/Lists/b7/Attachments/91/Chap%207.%20MISRA-C%20rules.pdf)

</comment><comment><USER16> So what's the catch? We have been developing memory architectures, and embedded systems, and OS's for decades now. So if the solution is as simple as this post says, why hasn't it ever been implemented before?

I am hoping there are experts here that can shed some light on this

<comment><USER17> A mix of technological and cultural path dependence, cowboy attitude to performance/optimization, and inability to quantify risks.

(Somewhat interrelated, eg cowboy attitude -> C, path dependency in C usage -> hard to reason about programs -> hard to quantify risks)

<comment><USER18> This. I've done consulting in embedded for years and years. The discipline of modern desktop/web development has not reached the embedded world, except in a few high-margin areas (such as military and transportation). There are 1,001 reasons for this, ranging from people who didn't learn source control, open source and unit-testing doing much of the coding (iow people from other disciplines besides software engineering and people who don't keep their skillset current), to the fact that cross-compiling is much harder/slower to debug than workstation debugging to the fact that resources are more limited in embedded, so code is more terse, with fewer convenient debugging functions.

Other problems are with management- they see embedded code as a "write it and forget about it" schedule item, not a continuous improvement one. Much of this is due to the fact that embedded code is not as portable as workstation code.

All of these shortcomings sufficed when the number of time pressure of embedded projects was much smaller 5-10 years ago. Now that embedded has exploded in ubiquity, its requirements are increasing and it's getting less time to be perfected in most markets.

So yeah, I guess it's safe to say the embedded world is in a bit of crisis at the moment.

<comment><USER19> Having work in a similar environment, I know at least one company where most managers have a hardware/electronics background, because their product was mostly HW when they started to work 20 years ago. But now, the split is more akin to 80%SW/ 20%HW and they are managing SW developers without having much idea of what SW development consists of. I know at least of one example of a top engineering manager not understanding what version control was : "Please merge all the features but not the bugs from branch X". (Which is a noble goal in itself, but might be hard to attain !)

</comment></comment></comment></comment><comment><USER20> Talking about how to catch stack overflows and protect your data against them isn't useless, but it misses the point. There are rules/guidelines, like MISRA[0] (which the testimony mentions 54 times!) for the automotive industry that prohibit recursion, and tools that will check for conformance.

Toyota should not have been using recursion in the first place, and it seems they were too cheap to invest analysis tools like Coverity.

[0] [http://en.wikipedia.org/wiki/MISRA_C](http://en.wikipedia.org/wiki/MISRA_C)

<comment><USER21> To me banning recursion in order to prevent stack overflows is like banning arrays to prevent buffer overflows. It misses the problem, doesn't it?

<comment><USER20> Sort of, but not really...

If you statically allocate an array, the compiler will ensure that you get the amount of space that you asked for, or raise a compile-time error. If you dynamically allocate an array (which you probably shouldn't be doing in this case anyway) then you'll either get a pointer to an array, or NULL. Either way, you'll know when it's safe to use the array. With a little bit of discipline, it's not difficult to avoid buffer overflows.

Recursive functions don't have a guarantee of safely running. Yes, there are ways to show that certain kinds of recursion will always terminate, and it might even work when you're calling the function at the top frame, but what happens if it's called further down the stack? What happens if the data structure guiding the recursion changes and now it takes a deeper stack than before?

<comment><USER21> Most recursions can be written as tail-recursions. If the compiler can optimize tail-recursions, in which case it will behave like a loop and throw warnings when a recursion is not a tail-recursion, then the point is moot. With algebraic data-types and pattern matching, often used for indicating the stage of a recursion which includes the exit condition, the compiler can even warn you that you missed a branch. In fact I find it easier to come up with complex loops expressed as tail-recursions, because in time and with practice, it gets easier to reason about all the possible branches and invariants.

The real problem is that we need a higher-level systems programming language.

> _Recursive functions don 't have a guarantee of safely running._

Neither do loops for that matter. A loop doesn't have any guarantee that it will ever terminate. Most stack overflows that happen are due to recursions with bad or missing exit conditions, but you can have the same problem with plain loops too.

> _With a little bit of discipline, it 's not difficult to avoid buffer > overflows._

Buffer overflows is amongst the biggest, most expensive problems in this industry and the primary reason for all the invulnerabilities you're seeing in the wild.

</comment></comment></comment></comment><comment><USER22> Completely unrelated to yesterday's "I No Longer Need StackOverflow" [https://news.ycombinator.com/item?id=7251169](https://news.ycombinator.com/item?id=7251169)

I was all excited to defend StackOverflow.com.

</comment><comment><USER23> My first thought was: How could stackoverflow.com be responsible for car crashes?

<comment><USER24> I suggest we _sing one tune to the song of another_ and re-purpose this thread to be about stackoverflow.com, as if nobody read the article!

<comment><USER25> I give it at least a 30% chance that this will happen automatically with nobody doing it on purpose.

</comment></comment></comment><comment><USER26> I had an "unintended acceleration case" in my old Austin Morris 1300; the cable connecting the pedal to the throttle snapped, jamming it at a fixed (fairly high revs) level, requiring me to control the speed using the brake.

The solution was to pop open the bonnet and swap in a replacement cable, which probably cost a couple of quid.

This recollection combined with the Toyota story merely convinces me that automobile automation has got completely out of control.

<comment><USER27> I think this issue is more to do with complexity than anything else; my car has an ECU, it's based on an 8-bit CPU from the 70s, never had any problems with it.

<comment><USER6> So does mine, and it is fuel injected. A cable controls a plate that controls the amount of air that gets into the engine. The only way for that design to fail WFO is for the return spring that keeps the plate closed to snap. At that point one would press in the clutch and turn off the ignition.

This kind of user corrective action is not possible on modern cars which I consider a huge engineering flaw.

<comment><USER12> I'm not tremendously knowledgeable about automobiles, but in a car with an automatic transaxle, shouldn't dropping it in neutral and switching off the ignition do essentially likewise?

I ask because that's how I practice responding to uncommanded acceleration, which I've done on occasion since I first heard of the failure mode. I've done this a few times a year in each of several cars, and as far as I can tell it's had no ill effect, but if "as far as I can tell" isn't far enough then I'd like to know it.

<comment><USER6> I know of no specific danger in shifting an AT into neutral. You need to be _EXTREMELY_ careful when switching off the ignition that you don't lock the steering column. It would be better to

* switch to neutral

* safely exit the road and stop

* turn off engine

I am glad you have practiced it.

My concern is over the code running tiptronic transmissions, they are computer controlled manual transmissions where you no longer have an effective physical connection.

<comment><USER12> My experience has been that switching the ignition from "run" to "accessory" doesn't risk locking the column, but now that I think about it, dropping the transmission into neutral should suffice -- the only concern I'd have there, with a runaway throttle, would be that it'd wreck the engine to have it running flat out with no load.

<comment><USER6> The engine will only be run at max for 15-30 seconds until you pull off, it will be fine. Better that than a potentially fatal accident. Only hardware after all.

</comment></comment></comment></comment></comment></comment></comment><comment><USER28> Example 7 on page 18 of "UNIQUE ETHICAL PROBLEMS IN INFORMATION TECHNOLOGY" by Walter Maner seems quite appropriate here:

Quote:

"A program is, as a mechanism, totally different from all the familiar analogue devices we grew up with. Like all digitally encoded information, it has, unavoidably, the uncomfortable property that the smallest possible perturbations -- i.e., changes of a single bit -- can have the most drastic consequences."

[http://faculty.usfsp.edu/gkearns/articles_fraud/computer_eth...](http://faculty.usfsp.edu/gkearns/articles_fraud/computer_ethics.pdf)

</comment><comment><USER29> Stack overflows hate the elderly:

[http://www.forbes.com/2010/03/26/toyota-acceleration- elderly...](http://www.forbes.com/2010/03/26/toyota-acceleration-elderly- opinions-contributors-michael-fumento.html) (forbes.com)

<comment><USER30> Yes, I thought the whole issue was myth.

Which does lead into, why are we trying to learn about stack overflows and critical system issues about imaginary subjects.

And even if it's not myth, as above there's no proof that a stackover flow was the supposed issue.

</comment></comment><comment><USER31> The least specialized in SW a company is, the worse the software is.

What we are accustomed to in discussing in HN for example _does not exist_ in these worlds. Continuous integration? Unit test? Even complexity analysis.

And very very old code that's patched over and over and shipped "when it works"

It's usually people who have had an academic contact with programming languages and embedded development and _don 't know anything_ about code quality. But you can bet their bosses incentive CMMI and other BS like that. (Yes, complete and utter BS)

Not to mention ClearCase which seems to be a constant, the worse the company the more they love this completety useless piece of crap

<comment><USER10> _does not exist_

I've worked in that world for a long time, and I assure you we did continuous integration, unit tests, and complexity analysis." Way back in the early 90's, long before it made it into the general population, so to speak.

I agree that there are terrible groups out there, but in general there is a far greater emphasis on safety, quality, and correctness than in the non- mission critical world.

<comment><USER32> Yeah, the software methodology at car companies makes a lot of the seat-of- the-pants just-ship-it stuff that HNers are used to look like kindergarten.

The car companies know how to do this. Maybe they messed up in this case (I'm skeptical of the article), but it's not because they don't know software.

<comment><USER7> Yes and no.

The transcript is very enlightening. It was _extremely_ clear that on this particular project, the software development process was a total trainwreck. _No one_ who was familiar with the SW dev literature had technical leadership and authority over the codebase. As a matter of fact, the transcript is so shocking it could be used as a manual of antipatterns for SW development both in embedded and out of embedded. A friend and I (we used to both work at an embedded systems company) spent an evening going over the transcript and mocking the errors. :-) By and large, the errors were of the design form. E.g., too much work on the critical threads. Not separating brake and acceleration threads. Four _thousand_ globals. I think the cyclomatic complexity was something like > 1000 for the control path function. Etc.

One of the remarks is actually that Toyota had taken some lessons learned from the time the codebase was developed and had been working on improving since then. So that's good.

</comment></comment></comment></comment><comment><USER33> It's sad that recursion is considered dangerous. Tail calls have been known about for a very long time, and the duality between stack and heap for just about as long.

<comment><USER34> Tail calls are only helpful if optimized into jumps instead of function calls.

<comment><USER7> And this only happens if your compiler will recognize it. Not all compilers are smart enough. And embedded compilers often don't have the love that mainstream compilers get.

In the absence of specific & concrete evidence that your compiler does this optimization and that you have and will tested this (including checking the emitted assembly code), it is correct to assume that TCO does not happen and to perfom stack depth analyses based upon that.

<comment><USER35> If you also built the computer, wouldn't you already have that specific and concrete evidence?

<comment><USER7> Beg pardon? I don't know what you mean. Most embedded companies purchase a chip... sometimes the compilers come from the same company that made the chip, sometimes not.

Regardless, it's the engineer's responsibity to not assume about such a critical part of the design.

</comment></comment></comment></comment></comment><comment><USER36> Would it be considerably expensive to check in runtime that SP is in an expected range every time it gets moved? This'd work with multiple stacks, too.

<comment><USER37> Well assuming you run in virtual mode you could always leave the page/segment directly below the stack unmapped, this way if the stack overflows it'll trigger an exception by accessing invalid addresses.

<comment><USER38> There is an enormous class of embedded devices not running on chips capable of address virtualization.

<comment><USER39> You don't need virtual memory, just memory protection. Many microprocessors and microcontrollers offer just an MPU, without the full fat of address translation machinery needed for an MMU.

<comment><USER6> [http://www.freertos.org/FreeRTOS-MPU-memory-protection- unit....](http://www.freertos.org/FreeRTOS-MPU-memory-protection-unit.html)

</comment></comment></comment></comment></comment><comment><USER40> Another example of C's impact into our daily life.

</comment><comment><USER41> I haven't waded into all this, and it's been years -- and years -- since my education touched upon systems that physically separate operating instructions from data memory.

But... sooner or later, it seems, we are going to go (back) there.

Instructions will become truly privileged, physically-controlled access. Data may go screwy -- or be screwed with -- but this will not directly affect the operating instructions.

Inconvenient? As development becomes more mature, instructions will become more debugged and "proven in the field". Stability and safety will outweigh ease and frequency of updates.

My 30+ year old microwave chugs along just fine. It doesn't have a turntable nor 1000 W, but I know exactly what it will do, how long to run it for various tasks, and how to rotate the food halfway through to provide even heating.

My 34 year old, pilot-light ignited furnace worked like a champ, aside from yet another blower motor going bad. I listened to the service tech when he strongly suggested replacing it before facing a more severe, "winter crisis" problem.

The new, micro-processor based model is better in theory (multi-stage speeds, and longer run times for more even heating). In practice, it's been a misery. The first, from-the-factory blower motor was defective. When that was replaced, the unit started making loud air-flow noises periodically.

Seeing the blower assembly removed, its constructed of _sheet metal_. The old furnace, by contrast, had a substantial metal construction that was not going to hum and vibrate if not positioned absolutely perfectly and with brand new, optimized duct work.

Past a point, _reliability_ starts to -- _far_ \-- outweigh some other optimizations.

This is going to become true in our field, as well.

</comment><comment><USER42> Are there any downsides to having the memory set up the "safe" way that they describe? It seems like a win-win situation.

[edit] I guess I was thrown off by the shoot-yourself-in-the-foot scenario, where the stack grows toward fixed data structures. If the heap and stack grow towards each other, you have quite a bit of flexibility (though with some danger of collision). If you have the stack grow towards fixed data structures, its size is fixed _and_ it can cause a dangerous overflow. The only disadvantage of the safe example is less flexibility, but for a critical embedded system, that is fine.

</comment><comment><USER43> Although I use managed languages, I wouldn't want my code audited by the NASA.

When 180+ IQ brains analyze your work they're bound to find "horrible defects" that no "competent" programmer would ever make.

<comment><USER44> I just refactored some code yesterday, I feel pretty confident, I'd love to know what they think :P Though my app is just some silly ruby code, not a realtime human life critical C embedded software.

<comment><USER34> Awesome, let's walk through every library, system, and machine call exercised by your ruby code. What happens if you gc while hitting a chunk of ram that was just hit with a corruption causing cosmic ray while your OS paged out your app because your intrusion detection/prevention service went defcon two on a leap millisecond that was announced since your last OS patch and your locale was set to C?

Please support your answer with tracing analysis over a set of one billion Monte Carlo simulations and present an accurate and up-to-date IDEF6 model of the application system.

<comment> mkr-hn Parking an SUV on Mars is a lot more complicated than parking one at the grocery store.

<comment><USER34> Plus, if the one on Mars kills anyone, we _all_ know how that movie goes. One does not simply walk away from killing an extraterrestrial.

I'm going to lick someone with a very bad cold now, just in case.

</comment></comment></comment></comment></comment><comment><USER45> I'm always skeptical about non-trival recursive calls and generally pass a "depth" variable in as the first param, increasing it each time I do another call, with some sane cut-off point where it just returns.

</comment><comment><USER46> I think cars should just come with slots where we could put in our phones and bam!, powerful computing that you could carry along.

</comment><comment><USER47> tl;dr: make the stack bigger! (then is it really a stack overflow?, oh by the way, this won't work in most systems due to virtualized stacks on top of the physical memory making concepts such as order of memory meaningless... but nevermind that)

The obvious solution to stack overflows is to make the stack bigger. The obvious problem with this solution is that it just kicks the can down the road.

</comment><comment><USER48> Before you wonder about stack overflows, as yourself why the occupants never applied the brakes.</comment>


The Modern Life of Origami, an Art as Old as Paper - <AUTHOR> https://www.nytimes.com/2019/11/22/arts/design/modern-origami-art.html <comment><USER2> [https://web.archive.org/web/20191123000145/https://www.nytim...](https://web.archive.org/web/20191123000145/https://www.nytimes.com/2019/11/22/arts/design/modern- origami-art.html)

</comment><comment><USER3> "You will get a better Gorilla effect if you use as big a piece of paper as possible." -Kunihiko Kasahara, Creative Origami.

</comment><comment><USER4> It's a shame that the origami association folded.

<comment><USER5> hobbies like origami have become inCREASingly antiquated</comment></comment>


Ask HN: Do you ask to use cookies or check IP? - <AUTHOR> This whole EU thing...
Until today I had no problem throwing Google Analytics on a site, same with Facebook for a share button (not sure if this uses a cookie).
Recently when I figured out how to use cookies and now use it for tracking/faster lookup for site usage... I came across the thing of "asking for permission to use cookies" ahh... this is like the noscript/accessibility/modifying social media buttons thing...
Is this something to be concerned about? For me it's not a credentials thing, though it is an identifying thing, just for easier database lookup to update the user's interaction with a site ie. pages visited, scrolls, stuff clicked on... for data driven dev.
I do intend to put the question but it seems like an annoying thing for them to deal with. When I see it I'm like "Ahh..." and don't click them myself. But I've seen very few sites ask.
Edit: I think the most useful thing I've found for cookies is the persistence for example using guides to use your website to help people get used to how it works. Then they only see it once until the cookie expires.
I realize that's probably the definition/reason for cookies is persistence. <comment><USER2> Check out: [http://ec.europa.eu/ipg/basics/legal/cookies/index_en.htm](http://ec.europa.eu/ipg/basics/legal/cookies/index_en.htm)

First party session cookies (e.g. login cookies) and several other reasonable uses are exempt.

The law is most strongly targeted at google analytics / facebook / omniture / etc cookies, which are third-party tracking cookies that follow the user around the Internet.

In your case, if you are setting a first-party non-persistent cookie which does not "identify" the user (except to determine usage patterns on the site) then it would be pretty reasonable to consider it exempt from notification.

However, you should throw up a cookie warning if you are setting a persistent cookie or using third-party tracking scripts (which will go ahead and use third-party tracking cookies).

<comment><AUTHOR> Oh yeah

>...follow the user around the internet

that is interesting, I'm not even sure how that would work once they leave your site (I am not aware at this time).

<comment><USER3> uhh… your site embeds google analytics, which sets cookies on google's domains. other site embeds google analytics, it reads the google cookies… and so on.

<comment><AUTHOR> right I thought of something else. this thought did cross my mind but can't prove it. caught in the web (of Google)

</comment></comment></comment></comment><comment><USER4> As a European who had to make sure the provisions contained in this law were implemented on a few high-traffic sites targeting consumers, I can definitively say that the law is at least a little bit "controversial" and unclear. We had to request the assistance of a law firm to make sure we complied.

Originally it looked like you had to give the users an option to decide if they allow the use of tracking cookies or not, potentially having to deal with NOT pushing such cookies in case the user did not consent. In reality, most of the implementations I've seen are just informing the customers that tracking cookies are used and let them know that usage of the site represents consent of such usage, adding that they can modify the browser preferences if they want to modify their cookie preferences. Additionally, you have to link to a cookie policy that you publish on the site.

As a clarification, this law does not apply to cookies in general: certain cookies, sometime referred as technical cookies (e.g. session) are exempted as long as they're not used for tracking purposes.

<comment><USER5> > As a clarification, this law does not apply to cookies in general: certain > cookies, sometime referred as technical cookies (e.g. session) are exempted > as long as they're not used for tracking purposes.

That's good to know. Any chance you know/can link to the specific section that makes that exemption?

<comment><USER4> Here's an excerpt taken from the EU Internet Handbook [0]:

Cookies clearly exempt from consent according to the EU advisory body on data protection- WP29 [1] include:

\- user‑input cookies (session-id) such as first‑party cookies to keep track of the user's input when filling online forms, shopping carts, etc., for the duration of a session or persistent cookies limited to a few hours in some cases

\- authentication cookies, to identify the user once he has logged in, for the duration of a session

\- user‑centric security cookies, used to detect authentication abuses, for a limited persistent duration

\- multimedia content player cookies, used to store technical data to play back video or audio content, for the duration of a session

\- load‑balancing cookies, for the duration of session

\- user‑interface customisation cookies such as language or font preferences, for the duration of a session (or slightly longer)

\- third‑party social plug‑in content‑sharing cookies, for logged‑in members of a social network.

[0] [http://ec.europa.eu/ipg/basics/legal/cookies/index_en.htm](http://ec.europa.eu/ipg/basics/legal/cookies/index_en.htm)

[1] [http://ec.europa.eu/justice/data- protection/article-29/docum...](http://ec.europa.eu/justice/data- protection/article-29/documentation/opinion- recommendation/files/2012/wp194_en.pdf)

<comment><USER5> Hmm, interesting to note that it _doesn 't_ allow for long-lived 'remember me' style cookies without the notice.

</comment></comment></comment></comment><comment><USER6> Please, somebody, make a service such that I can centrally "agree to cookies" for all sites that use this service. It would make my life easier as a user and also as a potential web site owner.

Side note: I personally hate this kind of thing where good meaning people force me to consent to or decline something. I liked the grey area where I didn't consent nor forbid tracking, and could be morally outraged and use a website at the same time. I think there is little use in displaying terms of service that nobody reads or understands, and especially terms that you have to accept if you want to use a service (which you most of the time not strictly have to use to live, but realistically, to take part in out society... yes, I have to access most of the services I do).

<comment><USER7> What we should've done is made browser vendors add EU cookie consent instead of every. single. website.

Then I can configure my browser for Always/Never/Ask or whatever.

That would've been too simple, of course.

<comment><USER8> Well, if you look in the options you'll find Firefox actually does have an option whether to accept all cookies, only first party, no cookies or always ask. Chrome also has an option to deny all cookies, third party cookies or accept all (no ask here).

<comment><USER9> Is it still only Safari that has the "from websites I visit" option? That's the most sensible one to me.

<comment><USER7> Does that mean same-domain? That basically meets the EU requirement for most tracking systems I think (where they're third party that is).

"Accept the session cookie but not the Google Analytics one."

<comment><USER9> It means it will accept cookies for other domains if you have visited the other domain explicitly before. This makes things like embedded widgets work.

<comment><USER7> Ah see, that's the opposite of what I'd want, personally at least.

The whole point imo of blocking cookies for privacy reasons is so that things like that can't track me around the web - explicitly visited or not.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> There may be same changes to the cookie law soon: [http://www.lexology.com/library/detail.aspx?g=859d2614-cf11-...](http://www.lexology.com/library/detail.aspx?g=859d2614-cf11-4f9a-a0e3-f0f8ed15600a)

<comment><AUTHOR> > from scanning the contents of their users' email for the purposes of > delivering targeted advertising

that's how you make money... if only I could read people's minds as they dreamt... cue the exploding egg (Futurama)

</comment></comment><comment><USER11> Has anyone ever been prosecuted or fined under this law, in any country?

Although many high profile sites now show the cookie banner, it still seems like 99% of sites operating out of the EU don't show anything, even when they use Google Analytics or use ad networks that use cookies.

<comment><USER12> I think the EU is dropping this bs law. They figured out how the internet works, finally.

<comment><USER13> Are you sure? Seems like it's still Neuland.

</comment></comment></comment><comment><USER14> This law is nonsense. You don't have to respect it if you are not in the EU.

<comment><USER15> It sure is nonsense since, most of the time, the site may already have dumped a boat load of cookies before the opt-in appears on the screen.

"You don't have to respect it if you are not in the EU."

Like most of these silly laws, it depends how big you are, Google couldn't get away with it but 99.9% of smaller websites could.

<comment><USER16> Google couldn't get away with it because Google does business in the EU and has subsidiaries in the EU so it needs to comply with EU law.

It's no different than physical products. If you produce and ship your product locally to the local market that is not part of the EU and someone exports your goods to the EU, it's their problem if they need to comply with EU law, not yours. If you want to distribute your product in the EU directly and maybe even create a separate company within the EU to do so, you have to comply with EU law or face consequences.

<comment><USER15> Yes, absolutely, thanks for the clarification.

</comment></comment></comment></comment><comment><USER17> >For me it's not a credentials thing, though it is an identifying thing, just for easier database lookup to update the user's interaction with a site ie. pages visited, scrolls, stuff clicked on... for data driven dev.

This is exactly what the law is targeting, using cookies to track users.

Realistically, as long as you don't and have no plans to do business in the UK (Edit: EU, not just the UK) you should be fine. But it can't hurt to put small notice to be safe.

<comment><USER18> My understanding is the rule comes from some form of EU legislation, in which case the issue is far more extensive than just the UK, ~500 million people.

<comment><USER19> That's correct.

[https://www.cookielaw.org/the-cookie-law/](https://www.cookielaw.org/the- cookie-law/)

 _> It started as an EU Directive that was adopted by all EU countries in May 2011. The Directive gave individuals rights to refuse the use of cookies that reduce their online privacy. Each country then updated its own laws to comply. In the UK this meant an update to the Privacy and Electronic Communications Regulations._

...

 _> All websites owned in the EU or targeted towards EU citizens, are now expected to comply with the law._

<comment><AUTHOR> Not tempting fate here, I will update the site(I'm pretty sure I will to be safe, at the risk of annoying users with a popup, possibly losing functionality on the site) with this in mind, what would they do? Fine me? Air lift me into a European prison? Haha that would be interesting.

<comment><USER20> If you have no business in EU this law is not for you and you can ignore it. If you're in the EU you may get a fine if you're unlucky.

<comment><AUTHOR> The whole part of business is "you made money through exploiting their otherwise personal information" ?

Interesting... it's like when you sign up for those discount cards at stores, do I see some law somewhere that says "By the way we collect and sell your data for money which can potentially be used against you in the future to deny you health care..." I don't know... out of my field here... but. Interesting, probably worrying about nothing as usual.

Like worrying about modifying the icons of big social media sites... what blog/website doesn't create their own version/theme of those buttons...

Yeah I'm not clear on that definition of business, if your site has ads, it's meant for the Philippines, and a person in the EU visited your site, and you made money from those ads... is it my fault? Doesn't Google Adsense have to know the content of my site and the visitor to serve ads... Ahhhh

</comment></comment></comment></comment></comment></comment><comment><USER21> When I write my terms of service, which I try to keep in simple language, I usually make known what I use cookies for or that I do log IP addresses, which is only used to prevent abuse of the application.

<comment><AUTHOR> Is that necessary ie. if you log an ip, you need ToS? Also a ToS you just right yourself, no "legitimate" way? Like those "Copyright 2017" at the bottom of a page.

The ips are normally dynamic/I wouldn't think you could reliably use them to target them for whatever.

Something to address though as we log IP's though servers in general know IP's by connecting right? How is that different? "Not logged"

<comment><USER21> I feel if you are taking any personal information, including an email address, you should have some kind of terms of service. If you set cookies on a personal computer, that could warrant a terms of service.

You can find Terms of Service generators online and you just modify it to be even more simple terms. If I were a large corporation, like Apple, I'd probably have lawyers writing it for me, but I do my best not to store too much personal data, while informing my users of the data collected from them. And if I do store it, its encrypted. I can't read it or recover it.

It is good practice to spend a few hours, at least once before you develop a good program, to check out the terms of services on a bunch of websites. It helps to understand the types of language to use and what to mention or not mention, especially those websites closely related to whatever you are doing.

My personal opinion is that the shorter your privacy policy and terms of service, the better. No reason to put up content that you can't even understand, so you may as well write something that you and your visitors can easily understand.

Terms of Service: information about the usage of your product.

Privacy: information about your considerations when collecting and storing data from your visitors.

<comment><AUTHOR> Thanks a lot that is really helpful.

Interesting point about the encryption. Do you think IP's need to be encrypted? I sometimes get IPV6's but mostly IPV4. We don't collect/ask for emails yet at this point... pretty much just using IP (not even sure why we collect it) and generate a 12-char unique string set to cookie for analytics upload/lookup.

I will have to take a look at some generators.

edit: wait that part about encryption, if I can't read it, what use is it? You're talking same thing with passwords like don't store it as plaint text, store the hash? If that's the case yeah... but then the cost server-side to decrypt/run batch processing... I guess part of the job...

<comment><USER21> It is just common practice for me to encrypt most data that goes into the database. First and foremost, for security. Privacy is important too. And also, encrypting data actually saves more bytes, as the encryption method I use outputs a pattern of random characters. While it can be long at times, it tends to eliminate special characters, periods, spaces, etc. So I'd think the database is more optimized by practicing data encryption.

Also, if you think about all the major companies that have gotten hacked in the past, with sensitive data leaks, you have to wonder how that data was able to get breached and then deciphered. At least if that happens to you, you can at least make it harder for the hacker. And people know about these data breaches.. yet it doesn't stop them from using those products, but that is because those companies are huge. A small business cannot afford to have a data breach and expect its reputation to remain stable.

Edit: The thing about data encryption is that for some methods, you can encrypt two strings so they end up being the exact same thing when encrypted. Then you compare that data for a match or an unmatch. It is just a more secure way of doing things. It is up to you on how you want to store your data. Databases do come with some protection themselves, but encrypting data is just one additional step for security.

<comment><AUTHOR> What about performing operations on that encrypted data though? Is it still "indexable"

Not disagreeing with you. Encryption is something I have to learn/get into I think I was looking at AES256, I mean I use the standard bcrypt with openssl for password/login.

</comment></comment></comment></comment></comment></comment><comment><USER22> Well, none of the pages I saw asked me to allow them for storing cookies. They just inform me about the cookies and the ways I can use to remove them.</comment>


Top 10 Brands on Digg - <AUTHOR> http://www.pronetadvertising.com/articles/10-most-popular-brands-on-digg23546.html <comment><USER2> I wonder how Digg is the tenth among those. I wonder how it made the list actually.</comment>


The Cars of 1960 Explain Why Your Grandparents Drive Differently - <AUTHOR> http://www.roadandtrack.com/car-culture/classic-cars/a26254/regular-car-reviews-ford-falcon/

<comment><USER2> That's an entertaining video that really demonstrates what it's like to drive a car from 1960.</comment>


Content Moderator Sues Facebook, Says Job Gave Her PTSD - <AUTHOR> https://motherboard.vice.com/en_us/article/zm5mw5/facebook-content-moderation-lawsuit-ptsd <comment><USER2> Did she not sign a waiver that explained the types of content she may be exposed to? Can someone post an example of the documents FB would have you sign to be a moderator?

<comment><USER3> _Can someone post an example of the documents FB would have you sign to be a moderator?_

I feel like this is something that gets signed right before you sign another thing saying you wont disclose or talk about the first thing you signed for.

(But if anyone posts it I'd read it in a heartbeat)

Also: This seems like the type of job role where Facebook should really (if they don't already) have immediate/on-call professionals for. I know people who have worked in 911 call centers and listening to the horror stories they've lived through or watched others go through that involved frequent mental breakdowns of colleagues and coworkers-while on the clock even-seems that with how much money Facebook has to throw around they don't have an excuse for not going above and beyond to ensure the mental stability of this group of workers, if any.

If anyone has any details on _that_ , I'd be just as eager to read them.</comment></comment>


Elon Musk to get no salary unless Tesla hits milestones - <AUTHOR> http://www.cbc.ca/beta/news/business/tesla-elon-musk-salary-1.4499831 <comment><USER2> As if it matters to a billionaire... (not to mention the money he'll be getting anyway for other expenses).</comment>


Google Wallet: Person To Person Money Transfers, Paypal-Style Money Storage - <AUTHOR> http://www.androidpolice.com/2012/10/08/google-wallets-next-big-features-person-to-person-money-transfers-and-paypal-style-money-storage/ <comment><USER2> P2P money transfers could be a huge step towards a cashless society. It's something that's just not really possible with other cashless systems (at least not in as low friction a process as this).</comment>


Ask HN: What do you scrape the web for? - <AUTHOR> Hey folks,
It's not rare for me to see myself scraping the web for both side projects and personal needs - checking whether the prices of the groceries are good or not based on the local supermarket's website; gathering some emails to cold-email about ideas ...
It seems to me that just by having the ability to quickly write some code to gather information from a government web page or anything that shows stuff on the web is a huge power.
What about you? Any great web scraping stories? Have you ever got in trouble by doing so? Made an entire company of it? <comment><USER3> I just wrote a crawler for a client who needs to check vendor licenses (plumber, electrician, etc). Given a license number, state, and trade, it looks up the appropriate licensing agency in that state and pulls the license info (issue/expiration dates, biographical information, infractions, etc). Luckily most of these are old school sites so they're easy to crawl, but a few have captcha (easily solved with OCR) or are single page apps.

<comment><AUTHOR> yeah, that's been my experience too: old (or in my case, gov websites) are much much easier but at least here I never needed to solve captchas - <USER2> it's just that scraping is not a big thing where I live (Brazil).

</comment></comment><comment><USER4> I think some of the most real-world use cases are in the realestate/hotels/coworking-spaces sector. The second space, email gathering (guess what, all the time people have to use <blabla> at <something> dot com to try to not have their emails scraped).

<comment><AUTHOR> totally agree! I've never been exposed to realestate stuff but it definitely seems a place where that makes a lot of sense.

Did you have some experiences with it yourself?

</comment></comment><comment><USER5> I made the State of Delaware change its search form so that you can only look up a company if you know both the name and the number.</comment>


Payza dropping payment to and from US - <AUTHOR> https://blog.payza.com/payza-updates/announcements/important-update-u-s-members/ As the title says, if you are accepting/offering payments through Payza you might want to disable that as they stopped accepting payment to/from US customers and businesses.<p>And yes, the funds are frozen. <comment><USER2> Any ideas on what the issue is/was?

<comment><AUTHOR> No idea, they don't seem to have made that public. But seeing as more and more outside US processors stop catering to US businesses and customers it's not hard to see why.

Edit: Funny thing is that they didn't even bother sending an email, found about it from one of the forums i follow...

<comment><USER3> The way they put it on their weblog, their partners in the US stopped. It does seem to have anything to do with "[...] more outside US processors stop catering to US businesses and customers [...]".

Never heard of Payza until now. That much will affect me.

<comment><AUTHOR> The Payza name is a bit new, it's the former AlertPay.

</comment></comment></comment></comment><comment><USER4> ahhh, the crushing of creativity by the US Government.. America will end up in the dark ages at this pace.. gotta love government stealing everything they can get their hands on.

<comment><USER5> This article didn't contain any mention of the US government.</comment></comment>


The Virtual Bay - <AUTHOR> http://thepiratebay.se/blog/236

<comment><USER2> Early April fools joke?

<comment><AUTHOR> most likely, one of the comments hinted it's from the movie brainscan</comment></comment>


Talentpad is shutting down - <AUTHOR> http://talentpad.com/blog/?p=453

<comment><AUTHOR> They didn't keep their costs low (too many employee perks/parties and stuff), the operation was wholly dependent on investors, and to start with, the idea wasn't all that scalable anyway (1-on-1 hiring is tricky, to say that you'll find the best talent for a company is an over-statement.)</comment>


Receiving emails in your web application easily [rails] - <AUTHOR> http://isotope11.com/blog/receiving-emails-in-your-web-application-easily

<comment><AUTHOR> This is I believe my first submission to Hacker News. ("Long time listener, first time caller.") This article just outlined a quick and dirty way I got over the problem of me not posting to the blog as frequently as I should. I think this method would benefit others as well

NOTE: I did not outline handling authentication / avoiding exploits. Trying to get Cloudmailin to outline suggestions on that front in their docs.</comment>


Live Now: Curiosity Rover Update - <AUTHOR> http://www.ustream.tv/nasajpl

<comment><AUTHOR> Curiosity detected organic compounds! Still trying to make sure they are not from Earthly contamination!</comment>


Paul Grahm: [Do What you love] (2006) - <AUTHOR> http://www.paulgraham.com/love.html

<comment><AUTHOR> Whenever I have to choose between two paths. I read this. Many hackernews readers may already know it. But for the newcomers it may be useful to bookmark.</comment>


Are you still displaying text testimonials on your website? - <AUTHOR> https://www.vocalreferences.com <comment><AUTHOR> We are looking for websites which want to do more with their testimonials than just display them. If you are interested in video testimonials which are mobile ready, a Review SER for your website and mobile friendly testimonial capture tools then check us out. Send us an email to info@vocalreferences, with the title "Saw you on Hacker News) and we will send you a coupon.</comment>


Ask HN: What book(s) would you recommend to future parents? - <AUTHOR> <comment><USER2> It's one of the greatest books I've ever read in general and it's one of those I'd recommend to anyone. Especially parents.

[https://www.goodreads.com/book/show/769016.How_to_Talk_So_Ki...](https://www.goodreads.com/book/show/769016.How_to_Talk_So_Kids_Will_Listen_Listen_So_Kids_Will_Talk)

</comment><comment><USER3> I'd recommend a good pediatrician (with a night nurse line if possible) since you will see them often the first six month. The books and classes are kinda a waste of time. But whatever makes you comfortable and eases you're stress etc is gunna be good.</comment>


Ask HN: What do you use to keep distinct strong passwords? - <AUTHOR> <comment><USER2> Password Gorilla: [https://github.com/zdia/gorilla/wiki](https://github.com/zdia/gorilla/wiki)

</comment><comment><USER3> I created my own password generator in Python. When it comes to applications there is PWGen which is really good.</comment>


Elections are bad for democracy - <AUTHOR> http://www.theguardian.com/politics/2016/jun/29/why-elections-are-bad-for-democracy <comment><USER2> You know, there is so much "Blah blah blah" about how bad democracy is, implicitly in context because "I think it came to the wrong conclusion" (which I , that I really expected the article to segue into something that basicaly amounted to "elites should just be given the power". I expected all the blah blah was trying to obfuscate this. But that's not where it went.

If you are interested in reading the whole article, please be my guest, but if you'd like to jump to the payload, use your browser search for "sortition".

(Though I can't help but point out the article undercuts its own premise accidentally. The EU, to the extent that it is even a democracy, is certainly subject to the exact same flaws described in the first half of the article! Therefore, logically, the decision to leave an organization where democracy is failing must not be so bad, even if it was made by a process that was itself democratically flawed. This does not "destroy the argument", in the parlance of the day, but it weakens this particular vote being used as the hook for the article. A vote on any other topic would arguably work better.)

<comment><USER3> Could it be argued that an understanding of how x works is irrelevant, only the outcome matters, in that individuals who don't understand the workings of x are still able to tell if the outcome is positive or negative for them, and vote on that.

<comment><USER4> Not in this case. The full outcome of the Brexit vote will not become clear for another ten years, probably. Both sides have tried to win over voters by scheming with "irrefutable facts" that clearly showed the other side's facts were wrong.

<comment><USER5> Not only that - ten years from now people will probably assign positive and negative effects of Brexit to other, more recent causes. The idea of people voting based on the perceived outcome for themselves is limited by their memory / attention span. A lot of important decisions, like those related to infrastructure, have immediate costs but bring benefits only years later.

</comment></comment></comment></comment><comment><USER6> Now, where is the "wisdom of the crowds" when it comes to decisions that we, "the elites", don't like?

This annoys me. There is no doubt the result would have been "very fair" if it had been remain.

What we are seeing now is IMO hypocrisy and contempt of ordinary people at a scary level.

Now, did I want Brexit? No (I had no opinion). Do I think they should have requested more than 50% of the votes? As a conservative biased person I guess so, -don't take a risk unless there is clear demand, simple as a discriminator in electronics: you want to protect against bounces and wobbling...

\- but this should have been said up front, not after losing.

Edit: upvoted this for the discussion, not for the original article.

<comment><USER7> Its hypocrisy and contempt coming from "elites" via the very institutions that are critical to a democracy, the media. When the 4th estate works in a neutral, non-biased, fact based way democracy can work. However, once you have major media bought and paid for by those "elites" to push some billionaire's agenda you sow distrust in media. Once the voters don't have a place to go for trusted information all bets are off...

<comment><USER5> > _institutions that are critical to a democracy, the media. When the 4th > estate works in a neutral, non-biased, fact based way democracy can work._

I don't believe mainstream media _ever_ worked in a neutral, non-biased, fact based way. They definitely have not been for the past few decades I've been on this planet. _Maybe_ in the past they were more under control by various "elites", government or otherwise. Now, in the west, they are "free" \- which means they actually serve Moloch[0], by writing what sells. That is, the most outrage-inducing, biased misrepresentations (and often outright lies) they can get away with.

Not sure what is worse[1], but it doesn't matter much. The point is, media isn't "a place to go for trusted information", it never has been, and if this is an institution "critical to democracy", then we have a big problem right here.

[0] - [http://slatestarcodex.com/2014/07/30/meditations-on- moloch/](http://slatestarcodex.com/2014/07/30/meditations-on-moloch/)

[1] - "elites" at least care about something identifiably human; metaphorical Moloch does not, so the outcome may be very well something _nobody_ wanted, including any of the "elites"

<comment><USER8> > I don't believe mainstream media ever worked in a neutral, non-biased, fact > based way. ... they are "free" \- which means they actually serve Moloch[0], > by writing what sells. That is, the most outrage-inducing, biased > misrepresentations (and often outright lies) they can get away with. ... > media isn't "a place to go for trusted information" ...

I think painting all news media with the same brush is significantly misconstrues the situation.

As a comparison, no software developer is perfect and can be absolutely trusted, and some devs are toxic. But I would be wrong to say: Therefore all software developers are toxic.

Similarly, there is better and worse news media. Can we absolutely trust any? Of course not; these are human institutions. But there's a long way from The Sun and The National Inquirer to The Financial Times and NY Times.

<comment><USER5> I don't think it misconstrues the situation much.

To follow your comparison - sure, we don't say software developers are all toxic[0]. But we like to complain that almost all software is shit. There are gems of almost perfect beauty and usability, and we love them, but there are so few of those. Here on HN we sometimes like to discuss dynamics of what makes most of software shit - and we often find similar sort of economic pressures to those mass media faces.

The thing is, we put extra care in the software that directly affects lives of people. For anything else, we rarely bother. So if the institution of media is so important to democracy, maybe we should put some extra care here too.

(Though I don't know how to do that.)

[0] - also note, that I didn't say "all journalists are toxic"; however, the products of the institutions they form are a different story.

<comment><USER8> > we like to complain that almost all software is shit

People like to use hyperbole, but IME I've never met someone who thought all software was bad or equally bad. There is a very wide range of quality.

</comment></comment></comment></comment></comment></comment><comment><USER9> Sortition is just too corruptible and disagreeable I would think. You can sample a society and "educate" them but then that process will forever be under fire too.

I believe that in a democracy things should be hard to change. Changing things should take a super majority. This will lead to more compromise and possibly better results for all. If there was a 2/3 requirement to change things then there would be little room for controversial changes to occur that cause an uproar among the citizenship. There would be more cooperation because to change something you'd need to compromise with your opponent for them to get a change they want instead of trying to get a slim majority in the government by election, etc.

If this principal was applied in the first place the UK never would have joined the EU and Brexit wouldn't have ever even been an issue. In fact, many of the things a lot of people find disagreeable wouldn't have happened.

We are divided because we let our form of democracy divide us.

<comment><USER10> If you impose too high a barrier to change, then laws cannot stay in sync with society, and that will lead to severe unrest.

<comment><USER11> Yes. I think this is a common misunderstanding about democracy. Democracy is not good because it provides the best government, or even competent government; much of the time it manifestly doesn't. Democracy is good because it provides for stable succession; if a discontented group is strong enough to stage a successful coup or revolution, then in a democracy it's probably strong enough to achieve its ends at far lower cost and risk by waiting a couple of years until the next election.

The EU was founded in large part to prevent yet another European war; that's a noble goal and for all its flaws I think the EU can claim a lot of credit for 70 years of peace. Warnings against complacency that "it could never happen again these days" are well-made and well-taken.

But we haven't had a violent revolution or civil war in western Europe for a fair while either - anything since Franco? - and I think that's largely thanks to functional democracy. Messing with that is not to be taken lightly either. You don't have to go back very far to find everyone collectively soiling their breeches every time a monarch started getting on a bit without a solid heir in sight.

</comment></comment></comment><comment><USER12> This sounds like a variation of Plato's Philosopher Kings, which uses the 'random sampling' argument to get around the obvious criticism of the elites being given the power.

> The Ship of State is a famous and oft-cited metaphor put forth by Plato in > Book VI of the Republic (488e–489d). It likens the governance of a city- > state to the command of a naval vessel and ultimately argues that the only > men fit to be captain of this ship (Greek: ναῦς) are philosopher kings, > benevolent men with absolute power who have access to the Form of the Good. > The origins of the metaphor can be traced back to the lyric poet Alcaeus > (frs. 6, 208, 249), and it is found in Aeschylus' Seven Against Thebes > before Plato.

[https://en.wikipedia.org/wiki/Ship_of_State](https://en.wikipedia.org/wiki/Ship_of_State)

Unfortunately, many of the same criticisms apply to sortition as apply to Plato. The problem is that no-one really has access to the "Form of the Good", and no-one can accurately predict what the overall outcome of a decision is going to be over the very long run, however much they study it. In fact, as Popper argues, you're quite likely to draw the wrong conclusion and end up going horribly wrong.

On balance, I'd rather put my faith in a large number of people selected from all social backgrounds than a small number of people selected from a potentially narrow social background. There's also a sense of shared agency about the decision making. If it turns out the wrong decision has been made, what are we to say to the person who is not only on the receiving end of said bad decision, but wasn't even allowed to vote on it?

It also seems to me that one should be wary about making sweeping statements about a system that has endured for thousands of years in the wake of a decision that they themselves disagreed with.

<comment><USER13> Did you read the whole article? His most significant proposal is that modern democracies adjust their procedures to use something called "sortition" that dates back to ancient Greece.

In sortition, rather than polling the entire populace about complicated issues on which they are likely not well informed, we select a balanced random sample of people from the populace , provide them information, and then allow them to engage each other on the issue. Then we have those people vote.

This has several advantages:

1\. It actually places "faith in a large number of people selected from all social backgrounds" depending on how large your sample, as opposed to letting elected politicians decide, which is the definition of "a small number of people selected from a potentially narrow social background".

2\. It solves the problem where a huge number of people end up voting because of gut instinct or misinformation. Consider the passage of Proposition 13 in California which prohibits the raising of property taxes. If you poll the entire population, that referendum is going to win 100 times out of 100, because for most people the gut check decision is "no I don't want my taxes to go up". However, the correct way to consider that proposition is to get an understanding of how property taxes are used, how they are collected, what the procedures are for changing them in the legislature, etc. and then come to a determination. That determination might be "Yes", it might be "No", or it might be "we should not limit increases in property tax rates but instead limit increases to the real value of collected taxes, by accounting for changes in property values". [1]

[1] I'm not proposing that is the right answer, I'm just suggesting that you might arrive at that conclusion if you actually had time to study the issue. Almost by definition, the vast majority of people do not have that time.

<comment><USER6> Sortition is almost what we have elected politicians for isn't it? The default case?

Referendums are for very important cases were we suspect politicians are out of touch with ordinary people.

This has been a nagging issue in UK for years and finally they figured it was better to ask the people.

<comment><USER4> Sortition is based on chance, instead of agreeableness.

Referendums are a great idea, but I think that the British yes/no vote was way too simple to game. Using hindsight to save the day, given that there appears to be no gameplan for where to go now, maybe the different options should have been explored before putting it to a vote:

* remain in the EU

* leave the EU, remain in the single market

* leave the single market

* leave the single market, destroy the chunnel

At least with a referendum like that, you force the people to have a content- filled opinion, instead of relying just on an empty "no". That also reduces the options for tea-leaf reading by the losing side.

<comment><USER12> I wonder if a referendum constructed in that way would actually make decision making worse.

As I see it, the public have voted to Leave, but some of the options for Leave are actually quite appealing, and would even appeal to some pragmatic Remainers (Norway-style EFTA agreement, join Schengen, negotiate trade agreements with the EU from the outside on a common basis with other EFTA countries, allow Eurozone to integrate closer).

By allowing the public to decide not only the decision but the details of the implementation (although I'm sure you weren't 100% serious with the "destroy chunnel" option) you increase the scope for voters to choose the worst one out of sheer spite.

<comment><USER4> I'm not sure what you mean with "make decision making worse". Are you suggesting that having the EFTA option explicitly listed might have made even less people choose "remain in EU"? I'd say that in that case, the government has been given a clear mandate on what route to pursue. Right now, I don't think the Leave side has any idea if the public wants the EFTA option, the "own island first" option, or the chunnel option.

Thinking about that last option some more, maybe it would make sense to put a nuclear option (like "destroy chunnel") on the ballot, to weed out the spiteful votes from the constructive ones. Even if you explain beforehand that the chunnel will never be closed -- some people just need to vent.

<comment><USER12> I'm saying that if we're agreed that EFTA is the moderate Brexit option, having nuclear options (destroy chunnel) on the ballot means that you effectively give a democratic mandate for the nuclear implementation (which let's face it, will always be appealing to a substantial minority, see Corbyn) and sideline moderate voices.

Saying that the public are in control of the direction of travel but leaving the experts in charge of the implementation (as long as it doesn't go against the wishes of the public, i.e. politicians can't just choose not to invoke Article 50, but saying they're free to explore constructive EFTA options) seems preferable.

<comment><USER4> Given his stance, I find it ironic that you'd equate Corbyn with the nuclear option...

As I see it, it doesn't really matter if we agree that EFTA is the moderate brexit option: the primary focus of the Leave campaign was on freedom of movement, and it was clear (even before the referendum, given the situation with Norway and Switzerland) that freedom of movement is an integral part of the single market treaties -- a point reiterated by Merkel and Tusk over the past few days.

So regardless that you and I may prefer for the Leave side to explore the EFTA route, it isn't in their mandate -- they won on a campaign of curbing free movement. Had the "moderate Brexit" been a separate choice on the ballot, they might have had a mandate to go for that option. But right now, nobody really knows how many of the Leave voters voted against free movement, and how many voted "just" against the EU.

<comment><USER12> Referendums aren't the same thing as elections - they don't come with a set of manifesto promises or a promised programme of government afterwards beyond carrying out the policy suggested in the Referendum.

The politicians making the promises may not even be in the government (it tended to be Leave.EU and Farage that focused on immigration, Vote Leave was wary of doing the same and focused on the economy). All politicians can do in a referendum campaign is talk about implied benefits one way or the other.

It's now up to the politicians to work out the best deal for our country. It's going to be a much smoother ride (and much better for the country as a whole) if free movement of people is allowed and we join EFTA than if the government tries to hold firm on that one, for sure.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Bit long winded, but boils down to recommending sortition, which I had not heard of. Interesting:

"With sortition, you do not ask everyone to vote on an issue few people really understand, but you draft a random sample of the population and make sure they come to the grips with the subject matter in order to take a sensible decision."

<comment><USER15> Who decides when someone has "come to grips" with a topic? I mean, anyone who disagrees with me obviously hasn't come to grips with the issue, right?

<comment><USER2> It's very similar to jury duty. And you'd try to use similar mechanisms to keep the system fair and balanced. And it would have similar failure cases, but "system without failure cases" isn't really on the table here.

<comment><USER16> A jury is a very restricted case. There is a supervising judge and two (or more) competing advocates. The process for choosing those advocates is very formalized (defendant selection and public official).

Every one of these other participants breaks the analogy enough to make it useless.

<comment><USER2> I see I may have skipped showing my work on too many steps here for my point to have made sense initially. Sorry, my fault (no sarcasm). So, _when you try to concretely manifest_ the sortition idea, you find yourself stepping in an awful lot of the same problems you get with juries. With a twist, but very similar. I'd intend to have advocates on hand for the various positions, and if you think the bureaucracy won't be stepping in here with procedures and rules, probably enforced by a process moderator that looks an awful lot like a "judge", along with formalized rules for the other parts of the process, you don't know bureaucracies. Surely we don't expect the participants to all be auto-didacts? And yet, we can't just have one person teaching the relevant issues, or that one person has too much of the power.

And I don't say this often, but the bureaucrats would be right here. They would be necessary.

There's no way sortition is going to be "randomly choose 100 people, stick them in a room with nothing but a problem statement, accept their decision". It shouldn't be, either; it needs more structure than that. Right down to things like shielding the chosen participants from undue influences, and a lot of really interesting questions around transparency... again, quite like a jury.

<comment><USER17> > I'd intend to have advocates on hand for the various positions, and if you > think the bureaucracy won't be stepping in here with procedures and rules, > probably enforced by a process moderator that looks an awful lot like a > "judge", along with formalized rules for the other parts of the process, you > don't know bureaucracies.

In a US-like system, if you replace election of leaders with sortition, there is no superior bureaucracy that can impose a judge-like supervisor on the sortition-based bodies.

And if you have an bureaucracy that can impose supervision on the elected bodies you are considering replacing, then you don't have a democracy to replace with sortition to start with, you have bureaucratic aristocracy, and talking about problems of elections and democracy is kind of irrelevant to dealing with your system of government.

> There's no way sortition is going to be "randomly choose 100 people, stick > them in a room with nothing but a problem statement, accept their decision".

There's basically a couple of possible models of sortition; there's a legislature-like model (where sortition is how you choose legislators for a term, and then they are replaced with a new body chosen by the same method.)

And there's a case-by-case model where there is some method of _proposing_ an issue, and then there is a "jury" chosen for an adversarial "trial" over the issue. (One way you could do this would be to replace one house of a bicameral legislature with case-by-case sortation, so that the remaining house would propose legislation and provide advocates for [and against, usually] that legislation, who would argue in front of a sortition-chosen panel.)

In the latter, case-by-case, approach, there would probably be some bureaucracy supervising the process, and the sortition-based body would be very much like a trial jury in relation to that bureaucracy. But then you aren't replacing the elected body with sortition, you are replacing it with a bureaucracy with sortition serving some function within that bureaucracy.)

In the legislature-like approach, though, a supervising bureaucracy is fundamentally inconsistent with the concept.

> Surely we don't expect the participants to all be auto-didacts?

I would expect members of a legislature chosen by sortition to educate themselves about issues the same way as any other legislature. This probably involves a _subordinate_ bureaucratic infrastructure (e.g., staff employed by each member, and bodies answering to the body as a whole, like the GAO), but not a _supervising_ infrastructure.

</comment></comment></comment></comment></comment></comment><comment><USER18> Democracy has always been about the people being able to remove their leaders from power when they become egregiously bad. Kind of like evaporative cooling. The means by which they choose them is a second order consideration.

<comment><USER19> I always understood election as a kind of checking power over whoever is already at the power, the ultimate authority even above supreme court and president. Only problem is its not immediate but eventual. Can mass voting system replace such authority ? [ A robotic president, which will post questions for which decision needs to be taken for polling via internet, tons of people choose from the option. ]

</comment></comment><comment><USER20> "No one pretends that democracy is perfect or all-wise. Indeed, it has been said that democracy is the worst form of government except all those other forms that have been tried from time to time."

      - Winston Churchill, House of Commons, November 11, 1947

<comment><USER21> The same could have been said of the horse until someone tried a train for the first time.

We've never tried sortition on a large scale, and on smaller scales it's already been shown effective.

A perfect opportunity to experiment with it would be the Canadian senate, IMO.

Also note: sortition _is_ a form of democracy. In fact I would argue it's the most democratic process proposed which does not run a risk of mob rule. You still have an "elite" making decisions... only that elite is constantly changing, is made of a representative sample of the general population, and is at less risk of being easily corrupted by special interest groups.

It's actually quite beautiful and elegant: so simple, yet it addresses so many of the criticisms of every other system proposed. While at the same time it's incredibly flexible in implementation: it can be perfectly fitted to a parliamentary government structure or any other variation of already existing governments. Any country or state would be able to experiment with it without needing a major overhaul of how its legislative/executive branches operate (only how its members are appointed).

<comment><USER17> > sortition is a form of democracy.

No, its not. Democracy is either direct (in which each citizen has a direct vote on proposals) or indirect (in which each citizen has a vote on representatives, who either directly vote on proposals or select other officials who do.) The _common_ element of democracy is that there is a chain of choice from the citizenry at large to the decision, what differs among forms of democracy is whether there are intermediate links on that chain, and how many, what those particular intermediate links look like.

Sortition is does not share the unifying characteristic of democracy, it is, instead, one of many forms of rule by an unelected, not-democratically- accountable elite.

> and is at less risk of being easily corrupted by special interest groups.

People that aren't trying to get _reelected_ , and which are rotated fast enough to be "constantly changing" (which also means that public attention on each of them is fleeting), is, if anything, _more_ subject to corruption.

They may be less _valuable_ to corrupt (since you are more likely to be renting them for a single decision than buying them for a whole extended career), but I don't think that the combined effect makes them less risk of being corrupted. At a minimum, I think that claim requires substantial support that hasn't been offered.

<comment><USER21> On corruptibility, true, it's not been tested so one can't say one way or the other. But at least in principle it rules out many of the common paths to corruption.

Let's take four major forms corruption takes: (1) a big bag of money dropped on your table. (2) I rub your back, you rub mine, reciprocal relationship. (3) Blackmail. (4) Personal: the desire for power.

(1) is of course still possible, but you're gonna need a lot more bags, and you're gonna have to be very careful with how you approach. Usually, the path to bribing someone requires a "romancing" phase; you have to make sure the other side is open to your proposal and, most importantly, won't give you up. This is a lengthy process that is much easier when the pool of candidates is small and doesn't change very often. How valuable is that bag of money when that person won't have any influence in 4 years' time? You're gonna need a lot more bags, and you're gonna have to be a lot more careful about who to approach.

(2) This one is out of the question. These sort of reciprocal relationships take years to cultivate, and we've immediately removed one of the prime motivating factors which is campaign funding.

(3) Blackmail. This one will still work, but again: you're gonna need _a lot_ more detective work to dig up brand new dirt, every 4 years, on every member of a senate. This becomes very impractical. Also I'm gonna go out on a limb and say the sort of secrets an average sample of the population can be blackmailed on is not the same as that of a small group of career politicians.

(4) Again, mostly out of question. There is a much higher incentive to pass laws that benefit people "like you", but not much you can do to benefit yourself directly (excluding the unlikely chance that the person who drew the lottery ticket is, say, CEO of some large corporation), and still, this only works if by some chance a large proportion of other lottery winners are like you. Otherwise, strategically, the only personal benefit you can get out of a position of power is to pass laws that benefit the country as a whole.

On the definition of Democratic. Yes, that's a fair point. Or perhaps sortition lives in a sort of quantum state being both democratic and dictatorial.

I hope someone manages to test this out. To me it seems obvious that the biggest threat to society is not climate change, or antibiotic resistance, or bad economies or poverty: all of these have a single root cause: corruption.

There's no lack of scientific innovation to solve the other problems; it's corrupt institutions that stand in their way.

Our number one focus as a species right now should be to figure out how eliminate corruption from our institutions and lessen the impact of bad actors.

<comment><USER17> Notably, you leave a widely-recognized big source of corrupting influence: a moneyed special interest establishing a reputation for providing rewards (often in the form of high-paying jobs) to people in government who serve their interest when they leave government, without an explicit quid pro quo.

This, of course, remains eminently workable with sortition, especially a rapid turnover form, where decision-makers will be leaving government soon after decisions are made and are no longer prefiltered to tend to include people who have fairly excellent job prospects _even without_ corrupt rewards.

</comment></comment></comment></comment></comment><comment><USER22> Democracy is not failing, but elites do. You need to educate people, rather than labeling everyone who disagrees as a racist.

<comment><USER23> As someone who is actively anti-racist, I weirdly find myself partially agreeing with you.

Racism is hard to overcome and widespread. People are not either 'racist' or 'not racist'. Certainly people can be pro-racism or anti-racist in their intentions, but that is separate from how we actually act, since a lot of racism is unconscious and invisible.

Labeling actions and activities and even groups as racist is necessary, because we need to become conscious of these forces if we are ever to overcome them.

However, whether labeled as racist or not, everyone's voice needs to be heard in the political process. So where I agree with you is in using the label of racism as a tool to silence or ignore people.

</comment></comment><comment><USER16> This article can be summarized as "Oh no! The 'wrong sorts of people' are winning! Lets ditch popular elections."

<comment><USER24> Or, specifically, the people voted the wrong way on the referendum. Let's stop letting people vote on referendums.

You see the same thing with the EU Constitution - the voters rejected it in two countries. That can't be allowed. So we'll do the same thing via the Lisbon Treaty, but we won't let anybody vote on that, because they might vote wrong.

<comment><USER4> Yes, the same thing happened with the NL referendum on the EU constitution. The reason people voted "no" was because they had never been asked before, so this was their first chance to have their voice heard.

That's about the only part of the article I agree with: if you're going to do referendums, then do them regularly. Referenda are only a useful tool if the public feel they really are involved in the process. Otherwise you'll have to answer for every decision that went around them.

</comment></comment></comment><comment><USER25> Ah the Guardian. It's gone into full on hypocrisy mode after the referendum fell to the 'exit' vote.

It styles itself as the voice for the downtrodden and disenfranchised, but the moment they make the 'wrong' decision it posts article after article dedicated to denigrating these same people, advocating their voice be ignored. Quite sad really.

</comment><comment><USER26> There are a ton of great alternatives. My favourite is the Demand-Revealing Referendum:

[http://stumblingandmumbling.typepad.com/stumbling_and_mumbli...](http://stumblingandmumbling.typepad.com/stumbling_and_mumbling/2007/03/trident_how_to_.html)

</comment><comment><USER27> As usual, people confuse democracy, vote, and freedom. In many coutries, we have vote. In some countries, we have a lot of freedom.

But to this day, we don't have democracy in a single country.

Democracy supposes that the people has the power. It's not the case in what we call democracies today.

You have a small group of people getting in the election pool because of where they come from and how much money they can manipulate. Then, this small group takes most of the decisions, which is what we call an oligarchy.

Voting then, is just a way to swap people from this group. In no way it puts power in the hand of the people.

Quite often people opposes dictatures with our systems to justify that if they are in that state, then we are in a democracy. The fact we have a better, and confortable system doesn't make it magically fit the definition.

For the people to have power, you would need them to want it in the first place. But most don't read political programs, most don't think hard about what their country should do, and most certainly don't accept any responsability for anything happening today in their country. They won't choose what they buy or whatch on TV according to the impact on the social life.

Hence, elections are not bad for democracy, it's just we never manage to use them to setup a democracy in the first place.

In fact, to have a democracy we would need to educate people to want it first. To make everyday's decisions by thinking about it. And to think about what society can become on a regular basis.

Right now, as most of us think and act mostly for ourself, we can't achieve democracy. Democracy cannot be brought only by using a fair political system (that we don't even have), you also need citizens.

</comment><comment><USER28> >Our voting system worked well for decades, but now it is broken.

Now that the elites don't get the desired result, it's broken.

The beatings will continue until morale improves!

</comment><comment><USER8> (Excerpting from another post:)

Most nations are republican democracies, not democracies. People choose the best of them (or the best person running) to deal with these issues. Direct democracy, where the people decide issues directly, is actually not part of the system. The UK has had 3 nationwide referendums and the US none, AFAIK.

There's a good reason for that: The average person has no idea what they're talking about on most issues. They have other jobs and interests, or no interest. Do I want the wisdom of the crowds deciding how the military should be funded and structured, for example, or do I want elected officials who have hearings, a staff of aids, who study the issue and talk to the generals ... egads, I pay a lot of attention to public affairs, but I wouldn't want that responsibility.

Note also that while elected officials are very human and imperfect, their credentials and experience are far above average.

</comment><comment><USER29> I propose a simple fix for major elections such as referendums and presidential elections. I call it the "really?" approach:

An election needs to get the same results twice back to back.

So for instance, you could replace the primaries and November presidential elections with that system. Vote once: whoever is first needs to win the next round to become president. Candidates may decide to drop out, or not. The public knows: if on round two, they still vote for the candidate that won the first round, that's the winner.

For the UK referendum, a second vote would have eliminated the protest vote. You see the results of the first vote, and you really ask yourself: really? Do I want this to happen? And vote one more time to confirm.

Simple system, with one huge caveat: no guaranteed outcome. But it would eliminate the need for polling, and hopefully result in better democratic decisions.

<comment><USER30> The result would be a very low turnout on the first vote, with a meaningless result that may or may not be representative of the entire population (because everyone knows the first vote is meaningless so why bother voting), followed by a large turnout on the second vote... and then followed by howling and wailing by which ever side loses and the demanding of a third referendum.

Interesting New Zealand recently had a double/two part referendum that was reported on Hacker News (due the the complexity of changing the National flag)

<comment><AUTHOR> Unless you make voting in the first vote a pre-condition to vote on the second vote.

<comment><USER31> to howls of "VOTER SUPRESSION!"

</comment></comment></comment></comment><comment><USER32> The goal of democracy is not to be good but to be fair. Democracy gives people what they deserve and in the long run they will try to make what is good for them. Making mistakes is the only way to learn (individually or collectively) so we shouldn't care too much for each error.

</comment><comment><USER33> Qualitative Democracy: [https://www.youtube.com/watch?v=X5cCyAqCkIw](https://www.youtube.com/watch?v=X5cCyAqCkIw)

</comment><comment><USER34> I, for one, am exceedingly happy that the media establishment has so loudly proclaimed their disdain for democracy in the last week. Open your eyes people, they think they can run your lives better than you and they're quite upset that you've dared fight back.</comment>