The Bytemark Guide to Dodging Spies - <AUTHOR> http://blog.bytemark.co.uk/2014/06/19/know-your-cables-an-easy-way-to-dodge-the-uk-spying-outrage

<comment><USER2> I have known the Bytemark guys personally and professionally for not far off 20 years, and can highly vouch for their professionalism, ethical stance, and skill at what they do. If you need a UK hosting provider, you should strongly consider them.

</comment><comment><USER3> If you're in the UK, it's bytemark on the server and Andrews and Arnold for your connection. They're the only two respectable companies I've found.

Glad to hear some sense being spoken in the blog post. Everyone has their head in the sand here in the UK.

One more point to add: dump your smart phone. All smartphone vendors are external to the UK as well so loading malicious firmware OTA is permitted under these statements.

Probably sound like a tinfoil hat nutter here but there are a lot of people popping up on the amateur bands suddenly (I listen with my scanner).

</comment><comment><USER4> Happy to be a customer of Bytemark's BigV. Like others on this thread, I too have known Bytemark people personally for about a decade or so.

</comment><comment><USER5> Even if the server and both endpoints are entirely within the UK, lots of packets are going to traverse UK's borders, due to the nature of the internet and routing.

It's a good start, but ...

<comment><USER6> In your described situation, GCHQ would term it internal and wouldn't _legally_ be able to intercept it. As I understand it one of the server or end points have to be outside UK under their current legal interpretation to be declared external. Of course instead they'll just ask the US for the data, as it's external to them.

<comment><USER7> And this is why all this "cooperation" is a real problem. It is basically an end run around the legal authority they've been given in regards to domestic v. foreign surveillance capabilities.

Realistically, the only way to secure things is to run your own privately encrypted VPN in which all data remains or use PGP. At least until the NSA and GCHQ can crack them.</comment></comment></comment>


The FCC Wants to Eliminate Net Neutrality Protections. We Can't Let That Happen - <AUTHOR> https://www.eff.org/deeplinks/2017/04/fcc-wants-eliminate-net-neutrality-protections-we-cant-let-happen <comment><USER2> Some of us actually worked in telcom when this problem started. The telcos (aka last-mile owners) really DID want to charge the Netflixes of the world for "backhaul." And they wanted to use their networks as leverage for their own products. More than one VP mentioned such desires in their talks with employees.

This situation was more drastic than Microsoft's favoring of Internet Explorer back in the mid-90s, for two reasons - 1) It was much less transparent; 2) Most homes have only one broadband provider in their area, or at best, one relatively fast provider and one not so fast. The main barrier to entry seems to be the cost of deploying plant - a figure I heard a VP give was 1500 USD per home passed - not installed. Witness Google Fiber's retreat from the market.

So the argument of consumer choice, even assuming perfect information of a provider's offerings, rings a bit farcial at present.

The scenario mentioned earlier -- shaking down popular services under threat of throttling or blocking -- might be countered by acquisition -- say, Google (600B USD) buying AT&T (250B USD) -- but this assumes the telcos wouldn't successfully deploy anti-acquisition tactics, both financial and political.

Before you trash this as anti-telco propaganda, I understand that businesses, especially public companies, have a fudiciary duty to their shareholders, which one can reasonably interpret as doing everything in their power, in compliance with the law, to make money. That includes legal political maneuvers such as lobbying. It's up to consumers to make our voices heard, blah blah blah.

Another interesting twist in this problem is common carrier status. Does the current legal climate impose risks for moving to a different regime?

</comment><comment><USER3> The greatest impact I see isn't really the price, its probably priced today pretty close to the what full bundle pricing would be without NN.

The real danger is creating an impenetrable barrier to entry. Most people wont want or need full bundle, and will settle for just Facebook/Snap and Wikipedia tiers, making it impossible for startups to reach these customers.

The startup costs to entering one of these tiers would be similar to creating your own TV network. Allowing monopolies to push their tax onto the market effectively kills the market.

</comment><comment><USER4> I do not like the gutting of fish. I do not like the gutting of net neutrality. I will not stay quiet, said Sam I am.

</comment><comment><USER5> Watch out for FUDsters.

</comment><comment><USER6> The US net neutrality system is a fast track to government regulation of the Internet. If you want to keep the Internet free, oppose it.

<comment><USER7> Oppose net neutrality to ensure an open internet? Are you sure you understand what net neutrality is?

<comment><USER6> Yes, are you sure you understand what the legal situation for the Internet is under the current US system? I'm all for net neutrality, but _NOT_ the way it'd being done now.

<comment><USER8> How is it being done now?

</comment></comment></comment></comment><comment><USER9> Net Neutrality is a joke. It's a scam and 100% certain to result in major existing players using NN rules to prevent competition. Anyone with a modicum of understanding of regulated markets can see this.

<comment><USER8> You said a lot without providing even simple arguments for anything you said.

Net neutrality is the idea that no matter the content of a message sent over the internet it should be treated the same as any other message.

The equivalent for power/water utility service would be that you can use power/water for anything in your house and you will pay the same price per unit you consume.

Imagine if power/water companies were allowed to charge different prices for drinking water compared to dish washer water. Or your power company charges you a different rate for A/C than your refrigerator. That's power/water companies without neutrality.

Do you want an Internet provider charging you more for using your preferred streaming service over using their video streaming service? In a world without net neutrality that's what you will get. You will end up paying for the internet like we currently pay for cable channels. Websites will be packaged into bundles and if you want something from another bundle... To bad you have to buy the entire bundle.

<comment><USER10> >Do you want an Internet provider charging you more for using your preferred streaming service over using their video streaming service? In a world without net neutrality that's what you will get. You will end up paying for the internet like we currently pay for cable channels. Websites will be packaged into bundles and if you want something from another bundle... To bad you have to buy the entire bundle.

Can't they already do this now? Imagine an ISP provides a browser or even a browser plugin as a part of their service and certain websites will not work properly unless you are using that particular browser.

Imagine they give discounts to customers that use preferred sites over other sites.

I don't think they can do much more than this with net neutrality and they, should they wish, already have the power to do so now. It seems like the reason they don't do this is because there isn't much motivation for them to do so, artificially restricting their services would cause them to lose money to providers that don't artificially restrict services.

<comment><USER7> No, they cannot do that now _because_ of net neutrality.

<comment><USER10> But what rule or law exactly, I don't see anything in the internet freedom act of 2009 or anything else that would prevent them from taking these steps. Is this whole net neutrality debate a case of legislators not understanding technology and technologists not understanding legislation.</comment></comment></comment></comment></comment>


E.gg, from Facebook - <AUTHOR> https://e.gg/ <comment><AUTHOR> It's only the About page that looks like Myspace or Geocities, in spite of comments elsewhere: [https://e.gg/about](https://e.gg/about)

</comment><comment><USER2> Its a direct ripoff of the original Byte (by Dom Hofmann of Vine fame) and I just realized it looks like its made by the Giphy team that facebook acquired.</comment>


Camille Paglia: The Modern Campus Has Declared War on Free Speech - <AUTHOR> http://heatst.com/culture-wars/camille-paglia-free-speech-modern-campus-protest/ <comment><USER2> Kind of weird to read this sentence in a piece attacking political correctness on campus:

"In my Anglo-Saxon class one day, the otherwise very affable young WASPy professor did a crass sexist stunt, also involving an ethnic slur against working-class Italian-Americans, that still shocks and disgusts me after all these years."

<comment><USER3> I think the Big Think video with John Cleese tackles the distinction between what she's addressing and that one line.

"... political correctness has gone from a good idea, which is let's not be mean, particularly to people who are not able to look after themselves. That's a good idea. To the point where any kind of criticism of any individual or group can be labeled cruel ..."

[https://www.youtube.com/watch?v=QAK0KXEpF8U](https://www.youtube.com/watch?v=QAK0KXEpF8U)

We made progressed when political correctness was used to combat nasty falsehoods, but digressed when it "over corrected" and suppressed valid criticism of observable phenomenon.

I would check out some of the ideas from Slavoj Žižek as well.

[https://en.wikipedia.org/wiki/Slavoj_%C5%BDi%C5%BEek](https://en.wikipedia.org/wiki/Slavoj_%C5%BDi%C5%BEek)

<comment><USER2> Well, it's tricky drawing that line isn't it?

19th century science proved that native americans, the irish, ... well basically everyone but the scientists themselves were inferior beings. The people who held those views didn't secretly believe that everyone was equal and they were making up lies for political advantage. They geniunely thought they were documenting objective facts about the inferior races according to science even though we can see from our perspective that it was all self- justifying nonsense.

But if you'd asked them they'd say, yes, fine don't be mean, but this is valid criticism based on observable phenomenon.

</comment></comment></comment><comment><USER4> Heatst is a conservative garbage rag.

</comment><comment> norea-armozel Not to be mean, but isn't this the same Camille Paglia that likes to call being trans a fad? I can't say I really trust her opinion considering her biases.

<comment><USER5> Well, I really don't feel like working at the moment so I'll bite!

The article, of course, doesn't just give an opinion. It tells an historical story---partly from her own experience as a social justice activist---and makes an argument that a wrong turn was taken. You could well disagree with her conclusion, some premises, or think she left out key facts, but I think her grounds are a bit less flimsy than whatever biases she happened to inherit.

FWIW, I didn't know who this person was, but I googled out of curiosity and she appears to identify as trans herself? (but, sure, this doesn't matter inside this dialectic since she could still have "cis privilege").

One of her recent criticisms seemed to be parents subjecting their children to sex reassignment surgery before they are old enough to give informed consent. And that the explanation for this---obviously, right?---involves the prominence of gender and sexuality in our cultural zeitgeist and as technologies for self-identification. She also points out that sex change is now being misleadingly presented as a solution to deeper problems that would really remain unaddressed.

A Twitter-inflaming interview had some interesting remarks on a connection between sexuality and cultural decadence---which she has apparently written a book on the history of---but she seems to be actually well read and was working with a much more complex structure than bloggers picked up on ("decadence" is used as a technical term, the link isn't causal, a richer concept of self-identity is assumed, the objectification of bodies is placed in a more complex network, etc).

But again, this is just my impression from procrastinating for a bit!

<comment><USER6> > she appears to identify as trans herself

no, she doesn't. she's a cis-gendered lesbian. she was born female and still identifies as female nor has she ever done otherwise as far as I know.

I'm surprised its your first time hearing about Camille Paglia though. She's moderately famous (in my circles anyway). Has been stirring the pot in the academia/inteligentisa scene for decades now.

She's a very iconoclastic and contrarian thinker and has made enemies on both the left and the right. Her ideas are fairly deep and well thought out though and a lot of the controversy she finds herself in is basically the media trolling their audience with inflammatory out-of-context quotations.

Not to say Paglia isn't sometimes deliberately inflammatory. She can light it up with the best of them.

<comment><USER3> Not saying you're wrong, but please substantiate your statements with citations/facts.</comment></comment></comment></comment>


PayPal long standing problem when you try to add a US bank - <AUTHOR> http://www.aspkin.com/forums/paypal-talk/60544-paypal-you-cannot-add-vba-time.html

<comment><AUTHOR> This affect only users outside the US</comment>


So you want to write a fast Python? (2011) - <AUTHOR> http://alexgaynor.net/2011/jul/10/so-you-want-write-fast-python/

<comment><USER2> The main point I believe is that the standard library is a drag.</comment>


Implementing State Machines in PostgreSQL - <AUTHOR> http://felixge.de/2017/07/27/implementing-state-machines-in-postgresql.html <comment><USER2> Nice interesting post, we've got a similar one that's a bit less on the Postgres side though Postgres is our backend for it on how we use this model to manage and run our database as a service - [https://www.citusdata.com/blog/2016/08/12/state-machines- to-...](https://www.citusdata.com/blog/2016/08/12/state-machines-to-run- databases/)</comment>


SoftBank's WeWork Bailout Draws Investor Concern - <AUTHOR> https://www.wsj.com/articles/softbanks-wework-bailout-draws-investor-concern-11572086091?mod=rsswn <comment><AUTHOR> > _SoftBank had told investors and credit-rating companies that it planned to > make big investments principally from the Vision Fund, not its own balance > sheet. It said that except for the capital that the group had contributed, > it wouldn't be on the hook for any losses in the fund.

The WeWork bailout throws that fundamental premise into question and makes it harder to calculate SoftBank's liabilities and obligations, said a person who watches the credit markets closely._

This looks bad both ways. From a SoftBank investor's perspective, Masa is bailing out the Saudis. From a Vision Fund investor's perspective, SoftBank is buying for $8 billion what the former were sold at $47.

<comment><USER3> > Masa is bailing out the Saudis.

Can you elaborate? genuinely interested to understand, isn't he investing more Saudi money in potentially doomed unicorn?

<comment><USER4> The Saudi empire is built on oil, which won't last forever. They are desperately trying to diversify by throwing money at bad ideas in a way that makes VCs look like toddlers playing with Monopoly money.

Eventually, a lot of those fail and the Saudis want their money back.

<comment><USER5> > Eventually, a lot of those fail and the Saudis want their money back

I'd wager that in 20 years the Saudi regime doesn't exist anymore, Softbank/Vision Fund or not. They're going to crash _hard_ , the youth worldwide does not want dictatorships and theocracies any more and the old guards are literally dying off.

Besides, the Saudis (and other Gulf states) have lost more money to way stupider efforts than wework, for their scale wework is chump change.

</comment></comment></comment></comment><comment><USER6> This article talks a lot about how the failure of WeWork's IPO might cause serious trouble for SoftBank. Not just the Venture Fund; all of SoftBank. Wow, talk about contagion.

<comment><USER7> Softbank has been basically existing off the one bet that they made back in 2000. They still own 1/3 of Alibaba

</comment></comment><comment><USER8> I'm no expert in real estate, residential or commercial, so I don't really see the big appeal of WeWork. Are there that many independent contractors and very small companies that _need_ office space, currently don't have it, and would otherwise work at home?

I can see renting out some space for an afternoon, a full day, or even up to a week if you're needing to meet with clients, but WeWork seems to be banking on very small teams that have no where else to work. Their spaces look nice but again unless you're really meeting with clients it seems like office space like they provide is largely a waste of money.

Obviously there's something I'm missing because Softbank feels different.

<comment><USER9> In short, yes. Some companies also feel it's a good idea to house their tech workers in a cooler trendy wework unlike the rest of their employees. But the price is pretty key. At reasonable prices for what they're offering, quantity demanded would probably go down

<comment><USER10> I don't think cooler/trendier is the reason major companies with established long-term offices use WeWork. I spent a few months in a WeWork office - <USER2> was more uncomfortable than it was cool or trendy - as an employee of such a company and the reason was mundane: needing temporary space for a subset of the workforce during an office remodel. I'd wager for most big name clients the draw is the short-term/low-commitment nature of the offering rather than cooler trendiness of Class C office space that's been tarted up with a thin layer of Silicon Valley chic.

<comment><USER11> Well, other companies provide short term rentals. It's probably more the investor subsidized pricing.

</comment></comment></comment></comment><comment><USER12> [http://archive.is/ibv4f](http://archive.is/ibv4f)

</comment><comment><USER13> is it a bailout? I thought it was a thinly vieled takeover.

<comment><USER14> Kind of both. The takeover was necessary to preserve their investment and any possibility of future gain: improved governance and pivot to profitability instead of growth are the cornerstones of a better IPO valuation in a year's time. In order to make it that year, they have to pump more money in. If they can convince IPO investors in a year that it's a $20billion company by doing so, they might approach break even territory.

<comment><USER11> $20B? That seems like at least one too many zeros.

My impression is their largest competitor is IWG plc, formerly Regus, which has a market cap of around $4.5B or 1.3x sales. Applying that to WeWork I get around $2-3B.

<comment><USER14> $20B is optimistic, but not outside the realm of possibility if they can show revenue growth by filling the inventory they've been building, at the same time they drastically curtail the cash burn needed to obtain still more inventory. They're on track for a 50% increase in 2019 revenue over 2018 revenue, and if that growth continues into 2020, then an IPO in a year wouldn't be completely unreasonable at 5x revenue on the basis of continued revenue growth. Still a bit high for a traditional real estate company, but they may still have some tech-adjacent mojo to prop it all up. It's a longshot, but not impossible.

<comment><USER11> It seems unlikely to stop selling space at a loss while maintaining revenue growth. After all, if they _can_ do that, why didn't they already?

</comment></comment></comment></comment></comment><comment><USER15> Theory as to why this front page HN article has no comments so far: it is behind a paywall.

I am not one to complain about paywalls myself but here is a similar article that is not behind a paywall:

[https://mybroadband.co.za/news/business/324915-investors- con...](https://mybroadband.co.za/news/business/324915-investors-concerned- about-softbanks-debt.html)

<comment><USER16> I sometimes wonder how many new subscribers NYT and WSJ got from HN. There are quite a few posts linking to them. They're quite expensive for semi-casual readers unfortunately.

<comment><USER17> But how come they are usually upvoted so much to end up in the top 10?

<comment><USER18> Becaues NYT and WSJ are the still the top names in news, and most people who want to be informed have some form of subscription to them.</comment></comment></comment></comment>


The wonderful CSS calc() function  - <AUTHOR> http://webdesignernotebook.com/css/the-wonderful-calc-function/

<comment><USER2> There's a min()/max() polyfill at <https://github.com/indyarmy/jquery.cssMinMax.plugin> right now. Someone with more patience might be able to extend it to support calc() as well.</comment>


United Nuclear Scientific Equipment and Supplies - <AUTHOR> https://unitednuclear.com/ <comment><USER2> I bought some of their neodymium magnets a few years ago

[https://unitednuclear.com/index.php?main_page=index&cPath=70...](https://unitednuclear.com/index.php?main_page=index&cPath=70_71)

They're pretty amazing to play with. For example, you can make a small one hop around in your hand by turning over another one relatively far away. Much more dramatic experiences than you get playing around with refrigerator magnets.

The safety warnings there are serious -- depending on the size, these magnets will be pulled toward each other with tens to hundreds of pounds of force, and they don't care what's in the way. Even the medium-sized ones that I got drew blood when they pinched me.

(As with other magnets, the ones that are small enough to be swallowed would also be a severe hazard if a child swallowed them, although most on that page are much too big to swallow. On the other hand, if you let some of the larger disc-shaped ones loose from far enough apart, they'll fly together with enough force to shatter!)

<comment><USER3> I managed to build a 0.9T Hall effect measurement system with two 1.5 inch diameter N52 magnets.

The scariest part was successfully assembling the magnets into the yoke. If we made a mistake, could easily use a finger, or, if they touch, we'd have to scrap everything and buy more to try again because you couldn't pull them apart.

</comment></comment><comment><USER4> It genuinely makes me happy that this exists. When I was a teenager this site felt so mysterious and cool. I wish there was more stuff like this in the world.

Side note: if you live in Phoenix: take a trip down to Apache reclamation sometime. They have a lot of chemistry gear, old lab equipment etc. It gives me a similar feeling to wander around there.

Bummer they're sold out of tshirts. Might have to bootleg my own!

<comment><USER5> There used to be a lot of surplus lab equipment and light industrial surplus shops in San Jose, CA and up the peninsula into Silicon Valley when I went to school in the 90s. Triangle comes to mind.

I suppose rents and eBay put an end to that.

</comment></comment><comment><USER6> This is Bob Lazar's company ([https://en.wikipedia.org/wiki/Bob_Lazar](https://en.wikipedia.org/wiki/Bob_Lazar)). What do people think of the plausibility of his UFO claims?

My assessment is this: if the probability that he's telling the truth is fat tailed enough (say roughly above 5%), then that's enough to justify being extremely interested in what he has to say. But the #1 thing that makes me doubt his claims meet this threshold is that he claims the craft he worked on violated the known laws of thermodynamics, which sends this probability to a mere epsilon over 0%.

(Since how often are known laws of physics "changed", as opposed to being merely "extended")?

<comment><USER7> From what the wikipedia article described, it sounds like the publicly-known relevant technology simply isn't far enough along to comment intelligently. e.g., the propulsion system was allegedly based on a stable isotope of element 115, but the best we can say is that of the isotopes we've found (and publicly described), we haven't yet found one that's stable.

If (big if) there were aliens on nearby star systems, it's fairly likely they'd follow something similar to the Star Trek 'Prime Directive', which "prohibits Starfleet personnel and spacecraft from interfering in the normal development of any society..." [1]

If that were the case, their actions would match fairly closely the supposed alien sightings that have been reported over the years. To me, that possibility is enough to consider the general nature of his claims not outside the realm of what our species' encounters with aliens would look like. Tech that's a little to far beyond ours for us to analyze intelligently, encounters between our governments and their emmissaries, occasional slip-ups and public sightings, etc.

[1] [https://en.wikipedia.org/wiki/Prime_Directive](https://en.wikipedia.org/wiki/Prime_Directive)

<comment><USER8> ...and apparently, going by a surprisingly sizable volume of the literature of abduction narratives, a whole lot of at best dubiously consensual butt stuff? I mean, forget the whole "aliens would definitely be a lot like Star Trek humans" thing, I want to hear how you explain the butt stuff.

Yes, I'm poking fun, but only a bit - any general exegesis of modern alien visitation on Earth, that attempts to incorporate claims such as Lazar's, really does need to also have a position on the butt stuff. So where do _you_ stand on The Alien Butt Stuff?

<comment><USER7> I mean, it's a fair point. My stance on the Alien Butt Stuff? Fetishes. Alien butt fetishes. We have bigger butts than they do, and they want to have fun on their spare time.

Or, we could construct an alternate theory based on the Alien movie and facehuggers. Butthuggers?

<comment><USER8> I can't believe your comment about alien butt stuff got flagkilled and _neither_ of mine got so much as a _single downvote_.

I vouched yours, of course, but I'm still insulted. I mean, what am I to the HN horny cops? Chopped liver?

<comment><USER7> I dunno, man. The Anti-Butt Brigade must not be the most consistent bunch.

Re: a _modus viviendi_ , though - I'd rather not see Earth become a sex tourism location for the local star systems. After a few generations of selective breeding, we'd all look like Kim Kardashian.

<comment><USER8> Eh. I see no reason to imagine that extraterrestrials so similar to humans that they'd find _any_ of us sexy, would also be so _different_ from humans that their tastes in such matters never varied. Or, at the very least, that they wouldn't be interested in the _range_ , the sheer butt-buffet our species has to offer, rather than uniformly focused on some Platonic ideal of the dumptruck.

That's what I'd hope, at least. But you're still right that there's a real risk here. Maybe that explains the weirdly targeted flagkill - not that I don't probably know one or two people who'd be here for that in theory, but from where I sit, it _is_ a frightful prospect.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> A gem of the internet I remember from 15+ years ago when using AOL via telephone modem. When I was a teenager I bought some of the trinitite and uranium ore samples. Mailed them cash since that's all most teens had back then.

Site still looks very much the same as it did other than a mobile friendly version available. Great source for various odds and ends. Especially for home science projects.

<comment><USER10> Any example ideas of interesting projects?

<comment><USER11> He used to have a guide on how to make uranium hexafluoride, UF6, aka "yellow cake". I assume he was "encouraged" to take it down

<comment><USER12> I think you must have meant on making UF6 from yellow cake. Yellow cake and UF6 are not at all the same thing.

</comment></comment></comment></comment><comment><USER13> Anyone know of a source of bulk (e.g. 20kg) uranium ore like United Nuclear used to sell? I always wanted to refine some when I was a teenager but by the time I actually had a way to spend money online they were sold out for good.</comment>


Engineering Faster Sorters for Small Sets of Items - <AUTHOR> https://arxiv.org/abs/2002.05599 <comment><USER2> In the similar topic, I'm trying to find maps optimised for a small maximum number of items (8 or 16) indexed by integers. Curiously I failed to find discussions about this topic.. But std::map seems very badly fit for this kind of map.

<comment><AUTHOR> Perhaps it's worth considering succinct data structures ([https://en.wikipedia.org/wiki/Succinct_data_structure](https://en.wikipedia.org/wiki/Succinct_data_structure)), e.g.:

\- BitMagic: [http://bitmagic.io/](http://bitmagic.io/), [https://github.com/tlk00/BitMagic](https://github.com/tlk00/BitMagic)

\- Succinct Data Structure Library 2.0: [https://github.com/simongog/sdsl- lite](https://github.com/simongog/sdsl-lite), [https://github.com/simongog/sdsl- lite/wiki/Literature](https://github.com/simongog/sdsl-lite/wiki/Literature)

<comment><USER2> Thanks but these structures are size optimised, I'm looking for CPU usage optimised structures.

</comment></comment></comment><comment><USER3> The most interesting thing to me is "the compiler reduces its optimizations with increasing compilation effort, when compiling only a single source file." Is this GCC-specific, or does it also hold for clang? And can it be tuned away?

<comment><USER4> Clang will not "reduce compilation effort" and I do not believe GCC does that. There are heuristics that look at things like function size but I don't think that is what they mean. Generally speaking, translation units are independent if you do not run link-time-optimizations (LTO) and optimizations follow a (basically) predetermined order which might be influence by the input but not by the amount of work done.

<comment><USER3> I've never heard of or experienced gcc using some kind of cost model for compilation time, either. With all these huge protobuf-generated files you'd think I would have noticed if it existed.

</comment></comment></comment><comment><USER5> Love papers like this, hyper-focused on a fun problem and then full of hacks.</comment>


How the Resignation of Italy's Prime Minister Threatens the EU and the Euro - <AUTHOR> https://www.cassandracapital.net/post/how-the-resignation-of-italy-s-prime-minister-threatens-the-european-union-and-the-global-economy <comment><USER2> The constant allusion to invariably vague doomsday scenarios that fills articles on the global economy is quite grating. What would really happen if Italy defaulted on its debt and started issuing its own currency? I suspect that the answer is no more and no less than, "Bond holders would lose money." But so what? What is Germany going to do, invade Italy? Somewhat surprisingly, I can think of few things less politically feasible in 2019 than Germany invading another country. I think that the political impossibility of direct military conflict between developed nations is actually the cause of the hysteria that permeates the media whenever one threatens to default. All that the creditors can do is scream bloody murder at the top of their lungs and pray that the debtors don't call their bluff.

<comment><AUTHOR> Respectfully, I think the risks from Italy leaving the euro are extremely underrated.

Italy defaulting on it's debt would be substantially worse than just "bond holders losing money." Italy has an extremely large amount of debt - if that debt dropped in value (like mortgage debt did in 2008)it would be catastrophic for every country in Europe. Who would bailout the entire country of Italy? That's the problem with sovereign debt, it's too large and there is no higher entity responsible for bailing it out.

<comment><USER2> What specific, concrete events would occur, and why would the occurrence of these events constitute a catastrophe? Why would anyone need to "bail Italy out"? If they decide not to pay back their debt and start using their own currency, the problem is solved from their perspective. Sure, they'll have to pay a much higher interest rate if they want to issue further bonds, but so what?

Again, please avoid abstract terms when describing this supposed "catastrophe" that would follow a default.

<comment><AUTHOR> If Italy was to leave the Euro, and pay back their debts in a devalued currency:

First it was be a massive deflationary shock to have that debt fall in value (very similar to how mortgage debt falling in value was a shock). Banks in France and Germany would see massive losses. Because of the leveraged nature of sovereign debt in the European banking system it would be extremely deflationary.

Secondly, the debt of every other nation in Southern Europe would also be put into question. If Italy is going to devalue, how long before Spain, Portugal, Greece, etc. do as well? And the scale of all this debt is much larger than subprime mortgage debt was.

Beyond that - imagine the monetary chaos if nations start to leave the euro and issue their own national currencies again? Will people be hoarding Euros? Selling them?

If you look back at the reporting from back in 2010 when the European Sovereign debt crisis was at its peak - you'll see that just about every serious economist agreed that a sovereign debt crisis and the idea of defaults had the potential to be very bad.

<comment><USER2> The only concrete effect you've suggested is that French and German banks would lose money. Well, yeah, obviously. So what?

<comment><USER2> I can't reply to your last reply for some reason, so I'm replying here. I'll read the article you linked to, but I don't expect anything other than more econobabble and fear-mongering.

If a country's entire economic system is based on a stupid assumption—in this case, that a sovereign nation will never default on its debt—then it ought to collapse. Still, I don't believe that collapse would actually occur. I think that a lot of people would lose a significant amount of money and be quite upset. And then, the next day, the sun would rise, and people wouldn't be starving in the streets, and life would generally go on.

<comment><AUTHOR> Hopefully you're right.

But I think if you look at how the people of Greece have already suffered since almost having a sovereign debt crisis you'll agree the potential is there for it to be quite bad in Italy.

[https://www.nytimes.com/interactive/2015/07/09/business/inte...](https://www.nytimes.com/interactive/2015/07/09/business/international/is- greece-worse-off-than-the-us-during-the-great- depression.html?mtrref=www.google.com&gwh=4DDBD9724BFDA10B3F4DABE72365B654&gwt=pay&assetType=REGIWALL)

<comment><USER2> I'll read both of those articles later; they're behind a paywall. I'm not an economist, and I'm open to being shown that I'm wrong, but I still think that if Italy were to default on its debt, there would only be two inevitable consequences: French and German banks would lose a lot of money, and Italy would have serious trouble borrowing at a reasonable rate in the future. I don't see how anyone could call either of those outcomes "catastrophic" with a straight face. But I can easily see why those who stand to lose a lot of money would want the public to _believe_ that a default would be catastrophic.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> Short Version:

The Resignation of the Italian prime minister last Tuesday will likely lead to the dissolvement of parliament and elections as early as this fall.

With the right-wing League party soaring in the polls there is a very real chance that they will be able to form a majority government without partners .

This is a risk to the EU and the euro because the League has been vocal about their plan to issue a parallel currency to the euro called the miniBOT. Having such a currency will enable the Italian populists to achieve where their Greek counterparts failed - because the EU will not be able to ensure compliance by threatening to freeze the banking system and paralyze the economy.</comment>


Tests that sometimes fail - <AUTHOR> https://samsaffron.com/archive/2019/05/15/tests-that-sometimes-fail <comment><USER3> We've had a couple of cases of flaky tests failing builds over the last two years at my company. Most often it's browser / end-to-end type tests (e.g. selenium-style tests) that are the most flaky. Many of them only fail in 1-3% of cases, but if you have enough of them the chances of a failing build is significant.

If you have entire builds that are flaky, you end up training developers to just click "rebuild" the first one or two times a build fails, which can drastically increase the time before realizing the build is actually broken.

An important realization is that unit testing is not a good tool for testing flakyness of your main code - it is simply not a reliable indicator of failing code. Most of the time it's the test itself that is flaky, and it's not worth your time making every single test 100% reliable.

Some things we've implemented that helps a lot:

1\. Have a system to reproduce the random failures. It took about a day to build tooling that can run say 100 instances of any test suite in parallel in CircleCI, and record the failure rate of individual tests.

2\. If a test has a failure rate of > 10%, it indicates an issue in that test that should be fixed. By fixing these tests, we've found a couple of techniques to increase overall robustness of our tests.

3\. If a test has a failure rate of < 3%, it is likely not worth your time fixing it. For these, we retry each failing test up to three times. Not all test frameworks support retying out of the box, but you can usually find a workaround. The retries can be restricted to specific tests or classes of tests if needed (e.g. only retry browser-based tests).

<comment><USER4> > If a test has a failure rate of < 3%, it is likely not worth your time > fixing it.

How do you know? What you say is plausible, but it's also plausible that these rarely-failing tests also rarely-fail in production, and occasionally break things badly and cause outages or make customers think of your software as flaky.

Since you say this, I presume you've spent the time to actually track down the root causes of several tests that fail < 3% of the time? If so, what did you find? Some sort of issues with the test framework, or issues with your own code that you're confident would only ever be exposed by testing, or something else? I'm very curious.

<comment><USER5> It's possible, but after fixing lots of these, my experience says usually talking about stuff like clicking a button before a modal animates out of the way.

It's sort if a "bug" in that yes, clicking here and then here 1ms later doesn't do do the best thing, but it's basically irrelevant.

Testing is inherently a probabilistic endeavor.

"What can I do that is most likely to prevent the largest amount of bugginess?"

Fixing tests that rarely fail is -- in my experience -- a poor answer to such a question.

<comment><USER6> > Testing is inherently a probabilistic endeavor.

That's a pretty powerful insight!

I think that a lot of developers who are firmly in the test-driven camp don't realize this, but instead think that if you have 100% test coverage, your code will work 100% of the time. Fixing bugs, to them, is "just" an inevitable result of increasing your test coverage, so that's what they focus on.

</comment></comment></comment></comment><comment><USER7> Every company I've founded or worked for has struggled with flaky tests.

Twitter had a comprehensive browser and system test suite that took about an hour to run (and they had a large CI worker cluster). Flaky tests could and did scuttle deploys. It was a never-ending struggle to keep CI green, but most engineers saw de-flaking (not just deleting the test) as a critical task.

PhotoStructure has an 8-job GitLab CI pipeline that runs on macOS, Windows, and Linux. Keeping the ~3,000 (and growing) tests passing reliably has proven to be a non-trivial task, and researching why a given task is flaky on one OS versus another has almost invariably led to discovery and hardening of edge and corner conditions.

It seems that TFA only touched on set ordering, incomplete db resets and time issues. There are _many_ other spectres to fight as soon as you deal with multi-process systems on multiple OSes, including file system case sensitivity, incomplete file system resets, fork behavior and child process management, and network and stream management.

There are several aspects I added to stabilize CI, including robust shutdown and child process management systems. I can't say I would have prioritized those things if I didn't have tests, but now that I have it, I'm glad they're there.

<comment><USER8> Sorry for the OT, what is "TFA"?

<comment><USER7> Sorry. The Fine Article. I didn't mean it in the disparaging connotation.

It's a reference to RTFM, Read The Fine Manual.

TIL: RTFM was a phrase from the 40s : "Read the field manual."

<comment><USER9> I never seen the F in RTFM mean "fine" before. I've always seen it used as the more vulgar "read the f __*ing manual".

<comment><USER10> I believe that's the joke.

</comment></comment></comment></comment></comment><comment><USER11> In an old job, we had a frustrating test that passed well over 99 times in 100. It was shrugged off for a very long time until a developer eventually tracked it down to code that was generating a random SSL key pair. If the first byte of the key was 0, faulty code elsewhere would mishandle the key and the test failed.

Keeping the randomness in the test was the key factor in tracking down this obscure bug. If the test had been made completely deterministic, the test harness would never have discovered the problem. So although repeatable tests are in most cases a good thing, non-determinism can unearth problems. The trick is how to do this without sucking up huge amounts of bug-tracking time...

(Much effort was spent in making the test repeatable during debugging, but of course the crypto code elsewhere was deliberately trying to get as much randomness as it could source...)

<comment><USER12> It doesn't seem to be a strong argument to have non-deterministic tests.

There was the logic that generates the SSL key pair, and there is the faulty logic that consumes it. Based on the description, it seems it's an indication of missing test coverage around the faulty code. If, when the faulty code was written, more time were spent on understanding the assumptions the code has made, then maybe the test wouldn't appear in the first place.

This anecdote, however, does bring up a good point: Don't shrug off intermittently failed tests - Dig in and understand the root cause of it.

<comment><USER13> The only other solution is exhaustive property testing. And even that is not workable when concurrency is in play.

<comment><USER14> Good luck exhaustively testing something with a cryptographic key as input. Non-exhaustive property testing is also pretty cool, though.

</comment></comment></comment></comment><comment><USER15> What I found to be the major reasons for flaky tests:

* Non-determinism in the code - e.g. select without an order by, random number generators, hashmaps turned into lists, etc. - Fixed by turning non-deterministic code into deterministic code, testing for properties rather than outcomes or isolating and mocking the non-deterministic code.

* Lack of control over the environment - e.g. calling a third party service that goes down occasionally, use of a locally run database that gets periodically upgraded by the package manager - fixed by gradually bringing everything required to run your software under control (e.g. installing specific versions without package manager, mocking 3rd party services, intercepting syscalls that get time and replacing them with consistent values).

* Race conditions - in this case the test should really repeat the same actions so that it consistently catches the flakiness.

<comment><USER16> > e.g. calling a third party service that goes down occasionally

I thought tests weren't meant to have external dependencies (or at least, ones outside the control of the test harness)?

<comment><USER17> In this context, yes, tests shouldn't require external dependencies. By "tests" we're really talking about tests like, "is this particular build consistent with its spec?"

There could be other types of test where a remote call would make sense, for example, "was the deployment successful?" tests might try to verify that the deployed version of the software can communicate with external dependencies correctly.

<comment><USER18> There are also cases that are less justified that you might have, especially once you start going down the road of "my dev environment should be a clone of production"

If you have an Employee model and it returns certain attributes of an employee like Salary, you might have tests that depend on the structure of an employee. You might have, say, Job and Position models which define an employee-job and the base definition of the particular job. Say Position has a salary range associated, and Job has validation rules which check that the salary is in range.

You could define factories for all those things, or you could use real examples that are served by a live Employee API.

The canonical way to address this is with factories and mocks, if you have time do that! (It will probably save you in the long-run, when that complexity has grown a bit.)

If you just grab the example person whose salary is out of the range for their position and quickly test that the behavior in nearby modules matches your expectations, well, those are still tests, and you could be forgiven for writing them this way.

I think they call these the "London" and "Detroit" styles of mocking, but the short version IMHO is that a mistake was making dev as a clone of production, and any errors in judgement that came after that were merely coping mechanisms.

If you want your tests to tell you when something has changed that requires your attention, you need a test that hits this Employee API and will fail if the structure of the employees returned is no longer conforming to your expectations, even though it's external. The design of such a thing is something I won't profess to know how to do well.

(It's better to version your API and write a changelog that tells what you need to know if the old version has been replaced by a new version, but if you're writing these microservices all for yourself it can seem pedantic to explicitly version your API, too. There are also coping mechanisms you'll need to embrace once you get to "we're not incrementing the API version" and surprise, many of them are the same ones...)

</comment></comment></comment></comment><comment><USER19> There was one weird bug reported to me in an microcontroller based project I was recently working on which shut off half the LCD screen. I wrote a test which blasted the LCD screen with random characters and commands and did not see the same error for awhile... but it finally happened during a test! I was able to then see that when I was checking the LCD state between commands I only would toggle the chip select for the first half of the LCD (there were 2 driver chips built into the screen and you had to read each chip individually). There would be no way I could have recreated the bug without automated tests.

I have had to deal with non-deterministic tests with my embedded systems and robotic test suites and have found a few solutions to deal with them:

\- Do a full power reset between tests if possible, or do it between test suites when you can combine tests together in suites that don't require a complete clean slate

\- Reset all settings and parameters between tests. A lot of embedded systems have settings saved in Flash or EEPROM which can affect all sorts of behaviors, so make sure it always starts at the default setting.

\- Have test commands for all system inputs and initialize all inputs to known values.

\- Have test modes for all system outputs such as motors. If there is a motor which has a speed encoder you can make the test mode for the speed encoder input to match the commanded motor value, or also be able to trigger error inputs such as a stalled motor.

\- Use a user input/dialog option to have user feedback as part of the test (for things like the LCD bug).

Robot Framework is a great tool which can do all these things with a custom Python library! I think testing embedded systems is generally much harder so people rarely do it, but I think it is a great tool which can oftentimes uncover these flaky errors.

</comment><comment><USER20> Related stories: "unit tests fail when run in Australia" [1] and "the case of the 500-mile email" [2]. There is a whole GitHub repository dedicated to some very interesting debugging stories [3].

[1] [https://github.com/angular/angular.js/issues/5017](https://github.com/angular/angular.js/issues/5017)

[2] [http://www.ibiblio.org/harris/500milemail.html](http://www.ibiblio.org/harris/500milemail.html)

[3] [https://github.com/danluu/debugging- stories](https://github.com/danluu/debugging-stories)

</comment><comment><USER21> We call them Flip Floppers.

We do a lot of integration testing, more so than unit testing, and those tests, which randomly fail, are a real headache.

One thing I learned is that setting up tests correctly, independent of each other, is hard. It is even harder if databases, local and remote services are involved or if your software communicates with other software. You need to start those dependencies and take care of resetting their state, but there's always something: Services sometimes take longer to start, file handles not closing on time, code or applications which keeps running when another test fails... etc, etc...

There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this? Fixing things correctly can even take more time and usually does not guarantee that some new change in the future disregards your correct code...

<comment><USER15> >There are obvious solutions: Mocking everything, removing global state, writing more robust test setup code... But who has time for this?

I find that doing all of this tends to actually save time overall it's just that the up front investment is high and the payoff is realized over a long time.

Most software teams seem to prefer higher ongoing costs if it comes with quick wins to up front investment.

<comment><USER22> Those are the age-old arguments against TDD. Every team will have to analyze the value proposition in their context to see if the return is worth the investment.

</comment></comment></comment><comment><USER23> I have learned to love non-deterministic tests.

The world is non-deterministic. A test suite that can represent non- determinism is much more powerful than one that cannot. To paraphrase Dijkstra, "Determinism is just a special case of non-determinism, and not a very interesting one at that."

If a test is non-deterministic then a test framework needs to characterize the distribution of results for that test. For example "Branch A fails 11% (+/\- 2%) of the time and Branch B fails 64% (+/\- 2%) of the time." Once you are able to measure non-determinism then you can also effectively optimize it away, and you start looking for ways to introduce more of it into your test suites e.g. to run each test on a random CPU/distro/kernel.

<comment><USER24> But you pay the cost of retrying the failing tests and lack of clear signal. And if the application code is flaky, users get to experience the breakage too.

<comment><USER25> > And if the application code is flaky

This is the only relevant factor. Forget the rest. Users don't experience your flaky tests just like they don't experience your messy Jira boards or your bad office coffee.

<comment><USER13> How do you know which is failing without exhaustive analysis?

See, once you know why the test fails and it's not the tested application, which is exceedingly rare in practice, you can just disable it or fix it. But only if you're actually sure, not before.

<comment><USER24> In my experience it is usually the test.

<comment><USER23> In my experience that's because tests are usually written in a timid style that tries _not_ to provoke flaky behaviour from applications.

If your test suite can handle non-determinism then you can approach test- writing in a completely different - braver - way.

</comment></comment></comment></comment></comment></comment><comment><USER26> Call it a pet peeve, but if we call it "chaos engineering" it costs a ton and gets people conference talks when a sporadic system integration issue is found. But if you have the same thing happen in a plain old CI half the time it will be ignored or flagged flaky.

<comment><USER14> IIUC, Chaos Engineering is about moving things out of "eh, it won't happen in production, I'll ignore it" into "it will happen in production, I'd better handle it", and making sure mitigation and recovery code is actually exercised in a realistic setting. "Periodic errors in CI that go unmitigated and produce test failures" seems very meaningfully distinct from chaos engineering.

<comment><USER26> If I use spinnaker and chaos monkey in a prod scale (or even in a prod experiment) to create a circumstance where I can't perform a write of a resource because I couldn't achieve quorum in a replica set and that let to an inconsistency between two data stores... is that meaningfully different than observing the same issue but caused by incorrect vpc routing or insufficienty resourced test instances leading/slow node startup times/race conditions in a CI test environment?

I think there is overlap and that it does not have to be a choice between either approach.

<comment><USER14> The meaningful question isn't in observing the issue, but in what happens after. Chaos engineering is about making sure you still have enough of a chance of _success_ in the face of failures. For CI, success means an error report that correctly captures whether the PR in question is breaking anything (... at least, anything we're testing). If your process means you can be sloppy about isolation and still get that, then I'd be okay with calling that an example of "chaos engineering". If being sloppy about isolation means you have failing tests in many CI runs that have nothing to do with the changes under consideration, that's not "chaos engineering" \- it's just bad CI.

</comment></comment></comment></comment><comment><USER27> As soon as I saw that whole section on database-related flakiness my mind went from "flaky unit tests" to "tests called unit tests that are actually integration tests". I worked on a team where we labored under that misconception for a long, long time. By the time we finally realized that many of the tests in our suite were integration tests and not unit tests it was too late to change (due to budget and timeline pressure).

I really like the different approaches to dealing with these flaky tests, that is a good list.

<comment><USER7> I think it's important that engineers can distinguish between testing code in isolation versus "integration" or "system" testing, but I've seen a sophomoric stigma around integration tests that lead to mocking hell, and a hatred towards testing in general.

Unit tests are great. You want them. Craft your interfaces to enable them.

Integration and system tests are important too. Again, crafting higher level interfaces that allow for testing will, in general, lead to a more ergonomic API.

Analogously: unit tests ensure each of your LEGO blocks are individually well- formed. Integration tests ensure that the build instructions actually result in something reasonable.

</comment></comment><comment><USER28> Hi - I'm Jon, creator of "Flaptastic" ([https://www.flaptastic.com/](https://www.flaptastic.com/)) and passionate advocate for unit test health.

Having coded at both Lyft and at DoorDash, I noticed both companies had the exact same unit test health problems and I was forced to manually come up with ways to make the CI/CD reliable in both settings.

In my experience, most people want a turnkey solution to get them to a healthier place with their unit testing. "Flaptastic" is a flaky unit tests recognition engine written in a way that anybody can use it to clean up their flaky unit tests no matter what CI/CD or test suite you're already using.

Flaptastic is a test suite plugin that works with a SAAS backend that is able to differentiate between a unit test that failed due to broken application code _versus_ tests that are failing with no merit and only because the tests are not written well. Our killer feature is that you get a "kill switch" to instantly disable any unit test that you know is unhealthy with an option to unkill it later when you've fixed the problem. The reason is this is so powerful is that when you kill an unhealthy test, you are able to immediately unblock the whole team.

We're now working on a way to accept the junit.xml file from your test suite. We can run it through the flap recognition engine allowing you to make decisions on what you will do next if you know all of the tests that failed did fail due to known flaky test patterns.

If Flaptastic seems interesting, contact us on our chat widget we'll let you use it for free indefinitely (for trial purposes) to decide if this makes your life easier.

</comment><comment><USER29> At Undo we develop a "software flight recorder technology" \- basically think of `rr` reversible debugger, it is our open source competitor.

One particular usecase for Undo (besides obviously recording software bugs per se) is recording execution of tests. Huge time saver. We do this ourselves - when a test fails in CI, engineers can download a recording file of a failing test and investigate it with our reversible debugger.

<comment><USER30> Yeah, this is huge. rr also has "chaos mode" to randomize things to make test failures easier to reproduce. (I understand Undo has something similar.)

I think that's one message that is completely lost in the article and in the rest of the comments here: it _is_ possible to improve technology so that flaky tests are more debuggable.

With enough investment (hardware and OS support for low-impact always-on recording) we could make _every_ flaky test debuggable.

</comment></comment><comment><USER31> At our place, we call them "peuteterli" (losely translated: "could-be-ish" constructed from the French "peut être" and slapped on the local German diminutive -li.

For the ID issue I have a monkey patch for Activerecord:

          if ["test", "cucumber"].include? Rails.env
            class ActiveRecord::Base
              before_create :set_id

              def set_id
                self.id ||= SecureRandom.random_number(999_999_999)
              end
            end
          end

Unique IDs are also helpful when scanning for specific objects during test development. When all objects of different classes start with 1, it is hard to following the connections.

</comment><comment><USER32> I deal with this issue a lot in my current job, and did in my last job too. IMX timing issues are by far the most common culprit. Usually it's because a test has to guess how long a background repair or garbage-collection activity will take, when in fact that duration can be highly variable. Shorter timeouts mean tests are unreliable. Longer timeouts mean greater reliability but tests that sometimes take forever. Speeding up the background processes can create CPU contention if tests are being run in parallel, making _other_ tests seem flaky. Various kinds of race conditions in tests are also a problem, but not one I personally encounter that often. Probably has to do with the type of software I work on (storage) and the type of developers I consequently work with.

No matter what, developers complain and try to avoid running the tests at all. I'd love to force their hand by making a successful test run an absolute requirement for committing code, but the very fact that tests have been slow and flaky since long before I got here means that would bring development to a standstill for weeks and I lack the authority (real or moral) for something that drastic. Failing that, I lean toward re-running tests a few times for those that are merely flaky (especially because of timing issues), and quarantine for those that are fully broken. Then there's still a challenge getting people to fix their broken tests, but life is full of tradeoffs like that.

</comment><comment><USER33> We're usually calling them "blinker tests" in our integration test suite. Reasons for blinker tests vary, but most are in line with what others here have already stated: concurrency, especially correct synchronization of test execution with stuff happening in asynchronous parts of the (distributed) system under test, is by far the biggest cause for problematic tests. This one is often exagerrated by the difference in concurrent execution on developer machines with maybe 4-6 cores and the CI server with 50-80, which often leads to "blinking" behavior that never happens locally, but every few builds on the CI server.

Second biggest is database transaction management and incorrect assumptions over when database changes become visible to other processes (which are in some way also concurrency problems, so it basically comes down to that). Third biggest is unintentional nondeterminism in the software, like people assuming that a certain collection implementation has deterministic order, but actually it doesn't, someone was just lucky to get the same order all the time while testing on the dev machine.

</comment><comment><USER34> "Making bad assumptions about DB ordering" That's caught me out before. Postgres is just weird, I had to run the same test in a loop for an hour before it'd randomly change the order.

<comment><USER35> There's several reasons for potential ordering changes:

\- the order of items on the page is different, due to the way tuples have been inserted (different external scheduling, different postgres internal scheduling) \- concurrent sequential scans can coordinate relation scans, which is quite helpful for relations that are larger than the cache \- different query plans, e.g. sequential vs index scans

Unless you specify the ORDER BY, there really isn't any guarantee by postgres. We could make it consistent, but that'd add overhead for everyone.

</comment></comment><comment><USER36> If anyone is looking for ideas for how to build tooling that fights flaky tests, I consolidated a number of lessons into a tool I open sourced a while ago.

[https://github.com/ajbouh/qa](https://github.com/ajbouh/qa)

It will do things like separate out different kinds of test failures (by error message and stacktrace) and then measure their individual rates of incidence.

You can also ask it to reproduce a specific failure in a tight loop and once it succeeds it will drop you into a debugger session so you can explore what's going on.

There are demo videos in the project highlighting these techniques. Here's one: [https://asciinema.org/a/dhdetw07drgyz78yr66bm57va](https://asciinema.org/a/dhdetw07drgyz78yr66bm57va)

</comment><comment><USER37> The two big problems seem to be concurrency (always a problem) and state, which immediately suggest that making things as functional as possible would help a lot.

Ideally all state that's used in a test would be reset to a known value at or before the start of the test, but this is quite hard for external non-mocked databases, clocks and so on.

For integration tests, do you run in a controllable "safe" environment and risk false-passes, or an environment as close as possible to production and risk intermittent failure?

A variant I've seen is "compiled languages may re-order floating point calculations between builds resulting in different answers", which is extremely annoying to deal with especially when you can't just epsilon it away.

<comment><USER13> Why not both? Test suite too slow? Live test too dangerous or inconsistent?

</comment></comment><comment><USER38> Both this article and this comment thread include a number of different ideas regarding controlling (or randomizing) environmental factors: test ordering, system time, etc.

But why do all of this piecemeal? Our philosophy is to create a controlled test sandbox environment that makes all these aspects (including concurrency) reproducible:

[https://www.cloudseal.io/blog/2018-04-06-intro-to-fixing- fla...](https://www.cloudseal.io/blog/2018-04-06-intro-to-fixing-flaky-tests)

The idea is to guarantee that any flake is easy to reproduce. If people have objections to that approach, we'd love to hear them. Conversely, if you would be willing to test out our early prototype, get in touch.

</comment><comment><USER39> I used to work at a company with over 10,000 tests where we weren't able to get more than an 80% pass rate due to flaky tests. This article is great and covers a lot of the options for handling flaky tests. I founded Appsurify to make it easy for companies to handle flaky tests, with minimal effort.

First, don't delete them, flaky tests are still valuable and can still find bugs. We also had the challenge where a lot of the 'flakiness' was not the test or the application's fault but was caused by 3rd party providers. Even at Google "Almost 16% of our tests have some level of flakiness associated with them!" \- John Micco, so just writing tests that aren't flaky isn't always possible.

Appsurify automatically raises defects when tests fail, and if the failure reason looks to be 'flakiness' (based on failure type, when the failure occurred, the change being made, previous known flaky failures) then we raise the defect as a "flaky" defect. Teams can then have the build fail based only on new defects and prevent it from failing when there are flaky test results.

We also prioritize the tests, which causes fewer tests to be run which are more likely to fail due to a real defect, which also reduces the number of flaky test results.

</comment><comment> pure-awesome > A few months back we introduced a game.

> We created a topic on our development Discourse instance. Each time the test > suite failed due to a flaky test we would assign the topic to the developer > who originally wrote the test. Once fixed the developer who sorted it out > would post a quick post morterm.

What's the game here? It just seems like a process. Useful, sure, but not particularly fun...

</comment><comment><USER40> I'm the primary developer for a heuristic, nondeterministic algorithm. It's both production software, and also a neverending research project. Specifically, I can't guarantee that a particular random seed will always produce identical results because that hobbles my ability to make future improvements to the heuristic. I've got reasonable coverage of my base classes and subroutines, but minor changes to the heuristic can have significant impact on the "power" of the heuristic.

My solution was to add a calibrated set of benchmarks. For each problem in the test suite, I measure the probability of failure. From that probability, I can compute the probability of n repeated failures. Small regressions are ignored, but large regressions (p < .001) splat on CI. It's fast enough, accurate enough, and brings peace of mind.

I understand that, and why, engineers hate this. But it's greatly superior to nothing.

</comment><comment> tom-jh We run in-browser end to end tests for our browser extension. There were several reasons for flakiness:

* Puppeteer (browser automation) bugs or improper use. Certain sequence of events could deadlock it, causing timeouts relatively rarely. The fix was sometimes upgrading puppeteer, sometimes debugging and working around the issue.

* Vendor API, particularly their oauth screen. When they smell automation, they will want to block the requests on security grounds. We have routed all requests through one IP address and reuse browser cookies to minimize this.

* Vendor API again, this time hitting limits on rare situations. We could have less parallel tests, but then you waste more time waiting.

Eventually, we will have to mock up this (fairly complex) API to progress. It's got to a point where I don't feel like adding more tests because they may cause further flakiness - not good.

</comment><comment><USER41> Flaky tests are indeed a big issue, the main concern being loss of confidence in the results.

The otherwise good advice for randomization has its drawbacks-

\- it complicates issue reproduction, especially if the test flow itself is randomized and not just the data

\- the same way it catches more issues, it might as well skip some

Something else that was mentioned but not stressed enough is the importance of clean environment as the basis for the test infrastructure.

A cleanup function is nice but using a virtual environment, Docker or a clean VM will save you a lot of debugging time finding environmental issues. The same goes for mocked or simplified elements if they contribute to the reproducibility of the system- a simpler in-memory database can help re creating a clean database for each test instead of reverting for example

<comment><USER13> Sometimes it's the code that is flaky and not the test.

In case of concurrent execution there are a only a few reasonably working tricks like Relacy and other exhaustive ordering checkers as well as formal proofs. Neither is cheap to use, so you will always get flaky tests there - <USER2> rather tests that do not always fall.

<comment><USER41> if the code is flaky then I have earned my pay honestly, this is a problem that should be solved.

Subtle concurrency issues are indeed very difficult to be found debugged and reproduced and randomization could help with that simply by covering more space.

</comment></comment></comment><comment><USER32> Here's a Google testing blog post about the same thing in 2016.

[https://testing.googleblog.com/2016/05/flaky-tests-at- google...](https://testing.googleblog.com/2016/05/flaky-tests-at-google-and- how-we.html)

<comment><USER42> If any Googlers are reading this and have the knowledge, I'm curious whether things have improved since that article. The numbers are sobering.

<comment><USER43> Not a googler, but they posted an update in 2017 with some more information: [https://testing.googleblog.com/2017/04/where-do-our-flaky- te...](https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come- from.html)

</comment></comment></comment><comment><USER44> Personally I've always called them flaky tests. I agree with the article that flaky tests shouldn't be ignored completely. But the issue is they take much more effort than usual test failures to debug. So it comes down to a balancing act of how much effort you're willing to spend debugging these vs the chance that it's an actual issue.

In my few years of automation experience, I've only seen 2 actual instances where the flaky tests were an actual issues and one of them should've been found by performance testing. Almost all of the rest were environment related issues. It's tough testing across all of the different platforms without running into some environment instability.

</comment><comment><USER45> Tests are part of the system too, and if you accept lower standards for your test suite than you think you hold the product to, you have actually lowered your standards for the product to those you accept for the tests.

</comment><comment><USER46> For annoying flaky features tests, I use rspec-retry gem to repeat the test a few times before marking it as failed. It helped for integration tests with external sandbox API.

I noticed discourse had a lot of flaky tests while using their repo to test my knapsack_pro ruby gem to run test suite with CI parallelisation. A few articles with CI examples of parallelisation can be found here [https://docs.knapsackpro.com](https://docs.knapsackpro.com)

I need to try the latest version of discourse code, maybe now it will be more stable to run tests in parallel.

</comment><comment><USER47> One recent test that was sometimes failing was ordering a list. It was due to how I made a sequence of my fixtures using numbers as a affix to a string so it was ordering correctly unless e.g. "string 8, string 9, string 10".

I fixed it for me by creating a random selection from /usr/share/dict/words to make a large array of sorted words to choose from. This made the fixtures have better and amusing names such as "string trapezoidal, string understudy"

</comment><comment><USER48> These sort of tests are perfect examples for me to add to [https://boyter.org/posts/expert-excuses-for-not-writing- unit...](https://boyter.org/posts/expert-excuses-for-not-writing-unit-tests/) Tongue in cheek it is but I'm always on the lookout for additional examples to flesh it out.

</comment><comment><USER49> Flaky tests are one of the factors that led me to leave a previous job. Test coverage was already so bad (and honestly, so was the code) that it was difficult to do anything with confidence - add to this that tests _sometimes_ worked meant that writing code was basically a dice-roll. I got tired of the stress.

</comment><comment><USER50> "Non-deterministic tests have two problems, firstly they are useless, secondly they are a virulent infection that can completely ruin your entire test suite."

"To this I would like to add that flaky tests are an incredible cost to businesses."

I think that the misconception here is that "tests should not fail", because they are "cost", "has to be analyzed and fixed", etc.

An integration or functional test that is guaranteed to never fail is kind of useless for me. Good test with a lot of assertions will fail occasionally since things are happening - unexpected data are provided, someone manually played with the database, ntp service was accidentally stopped and date in not accurate and filtering by date might be failing, someone plugged in some additional system that alters/locks data.

In case of unit tests, well, if everything is mocked and isolated then yes, such test probably should never fail, but unit tests are mostly useful only if there is some complicated logic involved.

<comment><USER51> You clearly have not worked on a codebase with thousands of tests. At my previous job the build system had an option to run a test N times concurrently in the cloud. I used this whenever I wanted to commit to some other project but some of their tests were garbage (to prove that test is flaky, and therefore to be ignored). You could even binary search (running 1000 times on each pivot point) to see who introduced the flakiness. Expensive but gets the job done.

In my projects I either fix the nondeterminism or delete such tests.

<comment><USER13> Pseudorandom deterministic tests have their value, presuming you store faulty input and/or seed.

These are not exactly nondeterministic but sometimes people end up with that instead of pseudorandom ones.

</comment></comment></comment><comment><USER52> _" You won't have code like this obviously contrived example, but you might have code which is equivalent."_

Ha, yes! The problem sounds super dumb and obvious once you explain it, but can be a PITA to track down or recognise in the code.

</comment><comment><USER53> To me, unit tests only make sense for pure code.

For impure code, it made no sense to make a unit test.

Ability to separate pure vs impure code determines your test suites, where should be put in unit test, where should be put in integration test.

<comment><USER54> It looks like you are coupling your unit tests with your integration tests. At integration level, we test if the integration paths work under various conditions, that is, only the code that deals whether our unit has been called correctly, with the right parameters, etc. At unit level, we mock all of the dependencies and test the branches of the effective code under various conditions. And at the acceptance level we should be testing our business logic requirements, to make sure all of our features are working the way they should, especially during refactoring (where integration and unit tests are subject to change).

</comment></comment><comment><USER55> This is a great article. Grounded in experience, detailed, actionable. Nicely done.</comment>


Technical debt in Firefox and Chromium (2013) - <AUTHOR> https://almossawi.com/firefox/prose-part-two/ <comment><AUTHOR> Excerpt:

Why is technical debt important?

"Some things tend to degrade with time. Within the context of software, degrading means that a system deviates in two specific ways: that it no longer functions as well and that it no longer satisfies its quality attributes. Quality attributes are those cross-cutting requirements such as maintainability and reliability, which one may argue are more important for a system's success than functional requirements.

The buildup of harmful elements that lead to degradation may be said to be inevitable in software for a number of reasons, primary of which is the inherent complexity of software.

To paraphrase Fred Brooks, this inherent complexity is due to the fact that no two components in software are the same, which is in contrast to the physical disciplines where there is such a thing as matter and where components have physical attributes that are unequivocally apparent and constraining. Other factors that contribute to the buildup of these harmful elements include changes in team dynamics, business pressures that lead to tight deadlines, industry shifts and so on.

A term that is sometimes used to describe a subset of these harmful elements is _technical debt_.

It is a fitting metaphor, as it indicates that a software engineering malpractice here or a concession with respect to coding standards there can incur a debt that one must then pay off in the form of time, effort or defects."</comment>


Show HN: piCards- Computer vision based audience polling solution for classrooms - <AUTHOR> We have created a computer vision and smartphone based audience polling solution that promises all the benefits of clicker devices at minimal cost and that too without the students requiring any electronic devices.
Use of continuous feedback systems like clicker devices is proven to improve learning outcomes in classrooms and such devices are widely used in Universities. BUT such devices are costly and hence cannot be used in mid-low income setups. Also, carrying devices in bulk is a problem.
In our solution, we replace the clicker devices with normal sheets of paper with QR Code like patterns printed on them(We call the sheet — piCard). The four orientations of the sheet translate to options A, B, C and D. Each student gets one such sheet. The teacher asks a question and students rotate and lift up their sheets and show it to the teacher who has our scanning app running in his/her smartphone. The app scans all the sheets at once. Yes, you heard it right — at once! Whatever is visible in the single camera frame, gets scanned at once. The teacher gets to see the responses instantly with counts of correct and incorrect responses. These responses are synced to our analytics platform where we perform question, class, quiz, subject and student level analysis to track each individual student's performance over time and clearly find out their strong and weak areas.
Here is a blog post related to it - https://medium.com/22bate7/picards-hello-world-3a2da6f7b885 <comment><USER2> _These responses are synced to our analytics platform where we perform question, class, quiz, subject and student level analysis to track each individual student's performance over time and clearly find out their strong and weak areas._

Another startup that builds dossiers on kids.

</comment><comment><USER3> Very cool! I work in Education (Higher Ed) but I've seen K-12 use this app before: [https://www.plickers.com/](https://www.plickers.com/) which kinda offers the same functionality.

I LOVE seeing these kinds of apps come from low tech solutions combined with small bits of high tech!

Great job, and I hope you run with it.

<comment><AUTHOR> Thanks :) Yes, Plickers also works the same way and is really awesome but for now lacks anonymous polling which I'm sure they must be working on.

<comment><USER4> Can you expand on this? I looked at the blog post and as far as I can tell piCards operate the same way as plickers. In my use of Plickers, I can either choose to give each student a specific card for personalized polling, but I can also just not identify the student with each card which allows for anonymous polling.

What have I missed here?

<comment><AUTHOR> In plickers, you still get to see responses on the cards and since you know the students in most cases, you can identify and if you want to single out a student (which I'm sure most teachers don't do), you can do it. "piCards: Anonymous Polling App" is a truly anonymous feedback system where you cannot know the students' response unless you read their cards. This will also give more confidence to students to freely answer without worrying about getting it right.

</comment></comment></comment></comment><comment><USER5> Here is it in action: [https://www.youtube.com/watch?v=DZ5Cu7rEs_Q](https://www.youtube.com/watch?v=DZ5Cu7rEs_Q)

</comment><comment><USER6> Just curious, was this based on Gwerns tweet?

<comment><USER5> I am unaware of any such tweet. Can you please direct me to that?

<comment><USER7> A tweet from 2016 or so referencing a longer writeup I made: [https://www.gwern.net/Statistical-notes#proposal-hand- counti...](https://www.gwern.net/Statistical-notes#proposal-hand-counting- mobile-app-for-more-fluid-group-discussions) My idea was that CNNs were good enough that you didn't have to use hacks like QR-codes or specially-designed papers; I think that someone has already done something equivalent, unless I'm thinking of you guys! (Even if you did go to CNNs for pure hand-counting, it's an obvious enough idea I doubt you got it from me, and in any case, it's implementation that counts.)

</comment></comment></comment><comment><USER8> Good job. Is the code available?

<comment><AUTHOR> Nope. We'll open source parts of it but that'll take time.

</comment></comment><comment><USER9> Awesome Idea Great job

</comment><comment><USER10> great!

</comment><comment><USER11> Nice</comment>


Show HN: Imperimetric.com – Convert whole texts between systems of measurement - <AUTHOR> http://www.imperimetric.com/ <comment><AUTHOR> I made this because I (a Swede) was tired of converting every measurement in american recipes to metric. The code is over here: [https://github.com/Dexterminator/imperimetric](https://github.com/Dexterminator/imperimetric). More information about supported units and other things is on the about page: [http://www.imperimetric.com/#/about](http://www.imperimetric.com/#/about).</comment>


Pins now optional in Signal v3.13.1 - <AUTHOR> https://www.reddit.com/r/signal/comments/hrlmoe/pins_now_optional_in_signal/ <comment> java-man Once shared, the data remains in the signal servers, accessible to anyone with subpoena. Even if there will be an option to turn it off, the damage is already done.

Or may be I am missing something?

<comment> 0-_-0 >accessible to anyone with subpoena

I assume it's encrypted.

<comment><USER2> Well, it is encrypted with a pin. Unless you make a long pin or switch to optional alphanumeric, the contact list is just protected by SGX and a short pin.

The SGX works like the iPhone auto-wipe. But it is in an online service with lots of juicy data in one place, so the target is bigger. A vulnerability could leak your contact list.</comment></comment></comment>


Ask HN: "medium.com" Posts - <AUTHOR>

It seems like all of a sudden there's always one or more posts from medium.com on the front page of Hacker News.<p>I had never heard of it before a couple weeks ago. Any particular reason for the upswing? Just curious. <comment><USER2> Very like svtble in that it's invariably a woolly Gladwell-style "thought piece" based on some flimsy anecdotal "data" and puffed through a lens of pure ego.

Also the space bar doesn't scroll down in my Firefox.

<comment><USER3> "a woolly Gladwell-style "thought piece" based on some flimsy anecdotal "data" and puffed through a lens of pure ego."

I love this description. You've put into words something I've long felt about a particular, and pretty large, set of HN submissions.

</comment></comment><comment><USER4> Given the frequency of the submissions it is not hard to believe that one or two of their stories makes the front page:

[https://www.hnsearch.com/search#request/submissions&q=me...](https://www.hnsearch.com/search#request/submissions&q=medium.com&sortby=create_ts+desc&start=0)

</comment><comment><USER5> You asked about the same issue I was about to ask about. I just looked around on the site to see a company introduction page

<https://medium.com/about/9e53ca408c48>

and the more detailed page "How Medium Works"

<https://medium.com/about/f13f5eb058c6>

I too am wondering what other HN participants think about Medium.com and how it compares as a source of story submissions for HN with other sites.

I see Medium is hiring

<https://medium.com/jobs>

and I wonder what tells in-the-know people about the company.

</comment><comment><USER6> i believe its because they have extremely good content. Only few people are allowed to publish and most of them are pretty good tech writers or founders

<comment><USER7> I came here to say the opposite. I make a quick domain check if the title isn't something I'm dying to read about, and won't bother if it is medium.com. I should note that I'm here primarily for the interesting technical articles, not the culture pieces, though I wind up reading too many of them anyway. I've read too many content-less pieces on medium.com that it is starting to feel like writing-for-the-sake-of-hearing-yourself and I want to avoid spending my time on that.

<comment><USER8> I came here to ride the fence. medium's articles are typically thoughtful and well-written, but generally are not topics that appeal to me very much when I'm looking for tech news or other more mainstream HN topics. I usually skip medium.com links when posted here, but I read them from other sources when I have time to read something a little more general.

</comment></comment></comment><comment><USER9> twitter guy trying to reinvent the wheel.</comment>


Fruits and Vegetables Are Trying to Kill You - <AUTHOR> http://m.nautil.us/issue/15/turbulence/fruits-and-vegetables-are-trying-to-kill-you?utm_source=nextdraft&utm_medium=email

<comment><AUTHOR> Some of the conclusions are a stretch, but the science behind the triggering molecules is pretty awesome.</comment>