Antivirus Companies Shouldn't Have Hidden What They Knew About Regin - <AUTHOR> http://www.technologyreview.com/view/533136/antivirus-companies-should-be-more-open-about-their-government-malware-discoveries/?utm_campaign=socialsync&utm_medium=social-post&utm_source=facebook <comment><USER2> It's easy to say it should be publicised, but I'm not sure how someone can do that if the infected organisation specifically asks the antivirus company not to publicly discuss the malware, like what appears to have been the case with F-Secure. ([https://mobile.twitter.com/mikko/status/536959936476221440](https://mobile.twitter.com/mikko/status/536959936476221440))

Sure, they could just go ahead and disclose it against the customers express wishes, but how many would hire their services in the future if there's no expectation of customer confidentiality?

Edit: There likely is an NDA in place even before externals are allowed in, so breaching it might invite some additional problems than just image issues.

<comment><USER3> It would be possible to discuss the find publicly without disclosing the source. I'm pretty sure that's how all national security journalism functions today.

<comment><USER4> Even that may be against the contract. I've worked with companies where their NDA said their security troubles can't be disclosed even if the information was completely sanitized. Even saying "One company in the US was impacted by Malware X" would be inviting the lawyers.

But I don't know this particular contract, so that may not be in play.

</comment></comment></comment><comment><USER5> Cleaning up a modern malware infection is quite a big task. You can't immediately wipe it off a single computer because you will be playing whack-a- mole forever.

You have to carefully monitor and learn everything you can about the malicious actor, and discover all the infections. Then produce a plan to remove it and prevent further infection. This is all implemented at a single instant.

However, it doesn't end there, you then have to monitor very carefully to see if it comes back. If this can all be done in secret it is much easier, especially if the malicious actor doesn't know you know they are there.

If you immediately reported everything you knew it would greatly assist the malicious actor - keeping it secret is part of trying to stay ahead in the game. Even after the first incident keeping it secret helps with future incidents.

<comment><USER6> > Cleaning up a modern malware infection is quite a big task.

And the only way to clean up a compromised computer is a full reinstall. You can't possibly know what has happened on the compromised computer during the compromise. This is what the desktop support jockeys and most companies get wrong - obviously it's probably because of the cost associated with a full reinstall, but it doesn't make it any less valid. If it costs too much, companies should then focus on preventing machines getting compromised in the first place.

Think about it, if I ask you to hand over your laptop for say, an hour, during which I have completely free reign over it, can you tell me everything I've done during that hour and all the backdoors installed, if any?

And to nitpick, obviously these days not even a full reinstall might do it when there's BIOS viruses and even hard drive firmwares can be compromised etc. of course.

<comment><USER7> "Think about it, if I ask you to hand over your laptop for say, an hour, during which I have completely free reign over it, can you tell me everything I've done during that hour and all the backdoors installed, if any?"

Yes, I can. Read up on modern digital forensics. Everything you do on a machine leaves a trace and there are ways to recover those traces and put together exactly what you did. That is exactly what Incident Response/Digital Forensics Firms do. An IR firm would never tell you to just reimage a machine when you're dealing with an advanced attacker. They'd want to go through, use the tools they have to identify exactly what happened on the machine and what other machines were compromised before they even started talking remediation. Wiping the one machine that you got an alert on would do absolutely nothing to solve the problem.

<comment><USER8> "Read up on modern digital forensics. Everything you do on a machine leaves a trace and there are ways to recover those traces and put together exactly what you did."

As a followup, both you and GP should read up on digital forensics from someplace OTHER than their marketing material...

</comment></comment></comment></comment><comment><USER9> Non-ironic question: do many people on HN use anti-virus of some form or another? Generally i'd probably trust people's judgement here as far as that goes. I naively use Linux with no externally accessible services running and no active anti-virus -- am i an idiot, should i clean-install everything ASAP? Or do yous reckon this is reasonable?

edit: clarification

<comment><USER10> No.

<comment><USER9> I'll take the bait: what kind of system would _you_ run? Linux? No evil web browser plugins? A personal computer which you never use to browse the web (RMS style)? Anything more exotic?

<comment><USER10> I have a Macbook, like most of us. I turn the firewall on, and I use Chrome. That's about it.

</comment></comment></comment></comment><comment><USER11> "My guess is that none of the companies wanted to go public with an incomplete picture." \-- or, see Assange's censorship pyramid: [http://wikileaks.org/Transcript-Meeting-Assange- Schmidt#279](http://wikileaks.org/Transcript-Meeting-Assange-Schmidt#279)

</comment><comment><USER12> Here we see the other side of the "responsible disclosure" coin -- if the ethical white-hat security researcher is required to withhold publication of a critical vulnerability for a set amount of time, is there a corresponding deadline for timely publication as well? And if that deadline is not met by meaningful attempts at remediation or disclosure, is the researcher not compelled to publish the findings independently?

Obviously these companies failed miserably to meet any reasonable person's timeline of disclosure. One question is whether the extra time researching this malware reasonably would have produced additional worthwhile intelligence about its function and targets. If so, then the delay was "worthwhile". Another question is whether it's not better to simply release an incomplete picture to the security community (perhaps selectively) and let the larger hive mind go to work on finding and corroborating additional clues.

It seems like the firms chose the former; many HN readers would advocate the latter. So finally, the question remains whether such a forced disclosure would be perceived as an irresponsible "leak" based only upon the disagreement in methodology and interpretation of "responsible"? Would its withholding be considered likewise irresponsible? Can a single firm, a collection of firms, or the security research community at large meaningfully stay ahead of a dedicated state-funded attacker? (Probably, Probably, Probably not).

If a nation-state is producing malware, it logically will also be monitoring the channels of disclosure for evidence of its release and detection in the wild. But that's no reason to limit the resources being dedicated to protecting the public; it's egotism at best and collusion at worst.

</comment><comment><USER13> I think the issue is... Windows is the biggest problem because most of the 'malware' kids come from hacking communities... Most communities use VB.net, C# when i coded in both of these languages things seemed easy; and there is really only FUD packagers on Windows; to make the executable undetectable. It's easy to code a RAT program or anything in vb.net or c#

</comment><comment><USER14> I didn't see a compelling argument for why they should be publicized in the article.

<comment><USER15> You mean other than the obvious "trust" benefit? If "anti-virus" companies don't want to protect you against "certain" viruses for political reasons, why bother trusting them and buying their product?

<comment><USER14> What would your alternative be? Not buying their products?

<comment><USER16> if the free alternatives protect you equally fine, yes.

<comment><USER17> > if the free alternatives protect you equally fine, yes.

Well, that's a Catch-22 and a half if there ever was one. Absence of detection doesn't mean absence of malware, it just means you haven't found out how badly you've been infected. The paid-for guys are halfways in the pockets of people who've paid more and the free guys are lagging behind.

</comment></comment></comment></comment></comment><comment><USER18> I don't care if they publicize it or not, what I care is that they protect me against the malware.

I'm very disappointed by Kaspersky : I chose them specifically because they were Russians, I though they would not be susceptible to NSA pressures.

<comment><USER19> From the article it appears that they did protect against it.

At least, that's what I understand from "all the companies had added signatures for Regin to their detection database".</comment></comment>


Breakthrough in understanding the chills and thrills of musical rapture (2016) - <AUTHOR> https://www.theguardian.com/science/2016/jun/17/breakthrough-in-understanding-the-chills-and-thrills-of-musical-rapture <comment><USER2> For those who haven't heard the term (which isn't in the article or paper), it's also known as Frisson:

[https://en.wikipedia.org/wiki/Frisson](https://en.wikipedia.org/wiki/Frisson)

</comment><comment><USER3> It links to this article: [https://www.theguardian.com/science/2016/jun/17/breakthrough...](https://www.theguardian.com/science/2016/jun/17/breakthrough- in-understanding-the-chills-and-thrills-of-musical-rapture)

Which has this explanation from a neuroscientist referring to the study in question:

> "The two pieces of evidence together indicate that this interaction, between > auditory and emotion systems, is the basis for musical pleasure, and that > people who get more of a direct "thrill" from music have a stronger > connection," he said.

<comment><USER4> Ok, we switched to that from [https://www.smithsonianmag.com/smart- news/researchers-look-w...](https://www.smithsonianmag.com/smart- news/researchers-look-what-happens-brain-when-music-causes-chills-180959481/). Thanks!

</comment></comment><comment><USER5> I'm looking at a dataset now of people listening to 12 music tracks with 1000hz sampling rate 128 channel EEG. We are trying to predict which tracks the 25 people are listening to, based on the EEG.

It's a good ML problem. We have accuracy up to 85% when predicting person&track together, falling to 30% when just predicting track.

Since we also had people rate enjoyment and familiarity, the main theoretical question is whether there is an direct neural resonance with the music during highly enjoyed tracks. I.e., that more of the brain is coupled to the music when you feel it deeply.

<comment><USER6> Is this a public dataset?

<comment><USER5> Will be when BIDS formatting is complete.

<comment><USER6> Awesome, is there a website I can keep checking for this dataset?

</comment></comment></comment></comment><comment><USER7> warning: the article fools around but doesn't give an answer...

<comment><USER8> A man can only take so much.

<comment><USER4> Could you please stop posting unsubstantive comments to Hacker News? You've been doing it a lot, and we're trying for better than that here.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment></comment><comment><USER9> The subtitle on this article is ridiculous:

> The brains of people who get chills when the right song comes on are wired > differently than others

The entirety of your personality and conscious thought are controlled by how your brain is "wired". The structure of your brain and the way your neurons etc interact together is literally "you". This is one of the biggest "duh" pop-sci articles I've seen.

<comment><USER8> Breaking: People are wired differently than others.

</comment></comment><comment><USER10> Happens to me all the time. And I probably have better orgasms, too. :)</comment>


Red Hat got $34B and you got $0.  why - <AUTHOR> https://blog.tidelift.com/open-source-creators-red-hat-got-34-billion-and-you-got-0.-heres-why <comment><USER2> The point here of course isn't that it's bad to make a profit with OSS; it's that maintainers can learn from how this was done and participate in it themselves to sustain their work.

</comment><comment><USER3> Red Hat doesn't "get" $34B, its shareholder do.</comment>


Help Me? Web Developers Unite - <AUTHOR> I am the newest addition to our web development team. I have been assigned the task of figuring out how to make a link from AWS that will result in a file downloading. EX. Click the link and a photo will automatically download. We are using Wagtail and Django, please help. <comment><USER2> If you are working with Wagtail, I recommend you the tutorial [https://learnwagtail.com/wagtail-for- beginners/](https://learnwagtail.com/wagtail-for-beginners/). You will see how to create good links in Django/Wagtail. Also you can see from the associated github repo [0] that they create links but you will not really understand well how this stuff work unless you go trough the whole tutorial.

[0]: [https://github.com/CodingForEverybody/rocketman/blob/master/...](https://github.com/CodingForEverybody/rocketman/blob/master/menus/models.py)

</comment><comment><USER3> The <a> tag allows you to set a "download" attribute which will prompt the user to save the URL/file rather than navigating to it.

[1] [https://developer.mozilla.org/en- US/docs/Web/HTML/Element/a](https://developer.mozilla.org/en- US/docs/Web/HTML/Element/a)</comment>


CDC reviewing 'stunning' universal testing results from Boston homeless shelter - <AUTHOR> https://www.boston25news.com/news/cdc-reviewing-stunning-universal-testing-results-boston-homeless-shelter/Z253TFBO6RG4HCUAARBO4YWO64/ <comment><USER2> [https://news.ycombinator.com/item?id=22880335](https://news.ycombinator.com/item?id=22880335)

</comment><comment><USER3> Some possibilities: the number of asymptomatic carriers is higher than expected, these people were mostly infected at the same time and just haven't shown symptoms yet, these people were infected by a less severe variant of SARS-CoV2, the tests are simply incorrect, or the inhabitants of that homeless shelter are in some way non-representative of the population in general (e.g. they're all in their mid-20's).

They imply that those asymptomatic cases didn't all stay asymptomatic and that one was hospitalized, so I'm guessing the most likely explanation is just that most of them were infected around the same time, and the test just caught them before they started to show symptoms.

<comment><USER4> I would love to see a follow up on their symptoms in 2-3 weeks. Not sure we'll ever see that info though, follow ups don't seem to sell eyeballs these days..

<comment><USER5> italy did this in a small town, testing weekly, by week three 80+% showed symptoms, and concluded they would all eventually show symptoms.

</comment></comment></comment><comment><USER6> The CNN article has a picture of what a Boston homeless shelter looks like:

[https://cdn.cnn.com/cnnnext/dam/assets/200417103746-01-pine-...](https://cdn.cnn.com/cnnnext/dam/assets/200417103746-01-pine- street-homeless-shelter-boston-2018-exlarge-169.jpg)

So yeah, it's a bunch of tightly packed beds. One or two asymptomatic carriers can probably infect hundreds in just a night.

</comment><comment><USER7> Those appear to be just PCR tests that tell you if the person has an active infection or not. Those that are negative should get a serology test to see if they were previously infected. The combination of both of those tests would be a true indication of how far this has spread.

Of course, given how infectious this virus is, it's no wonder that half have been infected in the last 1-2 weeks at the shelter. There's probably little to no safety measures in place to help isolate homeless people at that shelter, so it would spread like wildfire.

<comment><USER8> Yeah, it seems like it spreads to everyone in every nursing home it hits, so it'd be shocking to see it fail to spread in a homeless shelter.

<comment><USER9> In a nursing home I would've expected nurses to be transmitting COVID-19 between patients.

But merely having a lot of people in a room be enough for widespread transmission is a bit of a wake up call. I can't understand how given that you could ever open up bars, restaurants etc.

<comment><USER8> Well, a lot of people in a room for at least 8 hours, and unless this is a one-day shelter probably iterated over quite a while. I don't think this suggests a story where you sit in a restaurant for 2 hours and then everyone's infected; there's only been one report of a case like that, and multiple contact tracing reports from other countries where a carrier only infected a few other people in their bus or restaurant.

</comment></comment></comment></comment><comment><USER10> Transmission via asymptotic carriers isn't a new thing. It's been known for months (e.g. the analysis of the first cluster in Germany) and is one of the main reasons this virus is so problematic.

</comment><comment><USER9> I am actually surprised people aren't more concerned about the long term effects of COVID-19.

We have seen patients who got through it but were left with persistent neurological effects i.e. loss of smell, heart damage, weakened immune systems etc.

If people are asymptotic but have reduced life expectancy then that is a really big problem. Especially if we are talking about just letting COVID-19 basically run wild through healthy people.

<comment><USER11> > We have seen patients who got through it but were left with persistent > neurological effects i.e. loss of smell, heart damage, weakened immune > systems etc.

Not a significant percentage of patients by any means.

<comment><USER12> Lasting lungs damage is expected in ~5% of hospitalized cases (not counting ICU/reanimation protocol cases) in eastern France. I'd says it is a significant percentage. We will need to wait a couple month to be sure it is irreversible (pulmonary fibrosis are irreversible for the moment, as we can't chirurgically extract damaged lung tissue yet) and also the extent of this damage.

A young lady got asthma for the first time in her life last sunday, asthma that did not left her at least until wednesday (when my cousin told me about it, no news since). As nurses and doctors there now can afford to take vacation now (two-day vacations are vacations when you're working everyday since early march), they will probably have some time to start evaluating the non-deadly damages sometime soon.

<comment><USER11> > A young lady got asthma for the first time in her life last sunday, asthma > that did not left her at least until wednesday (when my cousin told me about > it, no news since).

Anecdotal. While you say lasting lung damage is expected, you also say that a couple of months are needed to be able to prove the expectation.

</comment></comment></comment></comment><comment><USER13> > "It tells you, you don't know who's at risk. You don't know what you need to > do to contain the virus if you don't actually have the details or facts," > said Marty Martinez, Boston's chief of Health and Human Services.

Plans to reopen states are conditioned on testing, but how long --and will we ever-- have enough testing data (among the general population, not just those showing symptoms) to make the informed decision?

If we can't get the cotton swabs, the reagents, with the ability to test nationwide at scale, health policy experts will not be positioned to green light re-opening.

<comment><USER14> Well to do folks can work from home, or not work from vacation homes, have all their needs delivered, and not catch this thing. All the "essential" people will catch it and some will die, and the well off will consider this very cheap, good, proper, and mandatory for economic reasons.

It's hardly worse than the exploitation for profit of the tobacco industry. Known for decades it would maim and kill, but millions invested in, were shareholders, were owners. And they've never been sanctioned. This will be no different. The lower class will take one for the team, again, for the greater good of society so that the better classes can live with minimal inconvenience.

By all rights, essential workers should quit. But they have a boot on their head: landlords and debts. It's economic coercion.

<comment><USER15> That's a very cynical view. Doctors are by no means lower class. Most if not all of them can quit, and are not subject to "economic coercion". Yet almost of them choose to work and overwork.

I am not an "essential" and am working from home, but I donate and try to help my local community, and if I am called to help in other ways, I will.

And no, I don't actively invest in tobacco.

<comment><USER16> Many doctors have a substantial amount of debt from med school that they took on with the assumption of having a doctor's income.

</comment></comment></comment></comment><comment><USER17> I wonder what the age breakdown was of those tested. It seems like there have been a couple of (very preliminary) indications that prevalence may be higher (and thus case severity lower) than we thought. It's probably too early to be optimistic, but I'm looking forward to getting more and more cases with better population samples than "people who think they have it", to get a better sense of what the rate of symptoms is.

</comment><comment><USER18> These articles really need to state what kind of testing was done: RNA or serological.

Very different conclusions if 50% test positive for one vs. the other.

</comment><comment><USER19> Probably better off on the street in the open air, than a 'shelter' full of asymptomatic COVID cases...

</comment><comment><USER20> The most stunning testing result is the recent Stanford blood study in Santa Clara showing that potentially 50 times as many people have been infected with Covid as previously thought.

We might already have 50 million Americans that have had Covid before another couple weeks is up.

Of course the media won't cover this story widely, as it won't sell properly, there isn't enough sensationalism. It lowers panic (it would mean the Covid mortality rate is radically lower than previously feared, even versus common low estimates), rather than raising panic.

It's crazy that the US isn't putting a lot of manufacturing and money behind doing widespread blood testing to see how common Covid actually is. This should have been done a month plus ago. Based on deaths, 10-15% of NYC probably has already had it at this point and if so, we need to know that asap. I've been talking about this for months now, it's not a premise requiring any kind of specialized knowledge to grasp (obvious question: how widespread is the virus), and I've yet to see any media interest in it despite how blatantly obvious it is.

It's almost like there is some interest in ensuring we don't find out how far Covid has already spread. The first thing they should have figured out, is the last thing they're apparently going to figure out.

<comment><USER21> The Santa Clara study has gotten a fair bit of press and attention but it's not clear whether the results will hold up. The antibody test is new and they're assuming very few false positives, and it's a non-random study (recruitment via Facebook).

Here's some more criticism: [https://medium.com/@balajis/peer-review-of- covid-19-antibody...](https://medium.com/@balajis/peer-review-of- covid-19-antibody-seroprevalence-in-santa-clara-county- california-1f6382258c25)

<comment><USER20> It's why we should be doing massively increased blood testing and test manufacturing to resolve that question. For the last few months it has been the single most important question and they're putting very few resources behind trying to answer it.

We can't test blood properly? Of course we could. They are choosing not to, is the actual context. It's a large mistake.

How many infections have you already had? How widespread is it really? No answer, no attempt to answer at scale. At its best, it's immense incompetence in action by the authorities in charge of dealing with this mess.

<comment><USER22> You seem to conflate antibody testing with viral rna testing. The former is somewhat difficult because you can't make the reagents necessary overnight, and the reagents are unique for the virus. Essentially, you have to clone the gene for an immunogenic portion of the virus (the spike protein most likely), express it in a recombinant host (which may require extensive optimization), purify it, and then validate that it properly detects blood of people who were confirmed positive for viral RNA and then recovered. That all takes time, and then you have to scale up manufacturing. In contrast, the the viral rna (RT_PCR) test takes about a day to design and the reagents are synthesized overnight and then you can run the test in the morning of the second day. Scaling that up to 1 million viral RNA tests a day should add at most a month, but our country already runs this number of tests a day if you count every academic, research, hospital, and CLIA lab, so the manufacturing capacity already exists.</comment></comment></comment></comment>


6 Reasons Windows Phone 7 is Dead in the Water - <AUTHOR> http://www.i4u.com/46167/6-reasons-windows-phone-7-dead-water

<comment><USER2> Let me start by saying that I absolutely love my WP7 phone (I've had both an LG Quantum and now a Samsung Focus). I can't imagine having to go to Android, and I'm not a fan of iOS either.

This author misses one big thing in all of his reasons. The average consumers don't care about these technical issues.

Now that WP7 phones have 'cut and paste' and most of the other features of comparable phones (except a front-facing camera) so consumers aren't going to be concerned about update schedules, Microsoft not standing up to partners (why would you 'stand-up' to partners, you want to work together), and most of the other reasons in this article.

A consumer goes into a store, speaks to a staff member (hopefully knowledgable) plays with the device and picks the one which they like best, most of which is probably based on appearances and ease of use and what everybody else has/recommends.

Microsoft has done an amazing job with the phone UI. They are showing lots of promise and commitment to the platform. Android was able to take significant market share, and there is no reason to believe that WP7 won't be able to do the same.

Early numbers aren't realistic as the more people using the phone, the more it gets accepted in the marketplace. There have been quite a few posts on engadget recently where TV shows are overlaying Windows UI onto the non- windows phones. This breeds familiarity and branding. Windows phone UI looks so different from all the other OS's that this is a very strong move.

Plus, it's really good (if you've tried it). I'm not saying that WP7 will absolutely be a success, but if it fails, I doubt it will be for the reasons in this article.

</comment><comment><USER3> Too bad it wasn't '7' reasons, that would have been much more convincing...</comment>


Ask HN: Does there exist an actually useful ebook reader? - <AUTHOR>

I'm on the move a lot, unable to take my library(ebooks + pdfs) with me. I tried Amazon Kindle, but its pdf rendition is pretty much useless. Also, as much as I'm tempted, LCDs like that of ipad/nexus don't work for me. Any advice/suggestions? <comment><USER3> I've got an ebook reader specifically for A4 PDF's

It's Onyx BOOX M96 - 9.7" screen, 1600x1200, running on android. The dpi could be better, there is no backlight and only weird stylus-only touchscreen. Build quality is decent, but nothing amazing, the touchscreen is acceptable if you have the damn stylus around (you can actually take notes on that thing though) It is more or less stock android, with special launcher for ebook part - <USER2> can change it though, and it does come with google play. This means you can replace shoddy inbuilt apps with good ones from the store (therefore no issues with supported formats -dvju, epub, fbr, mobi,personalised dictionary,editing doc, excel files (bluetooth keyboard) whatever you want) - it pretty much is an android tablet with eink. However the android comes with no tweaking regarding that - some apps come as black on black - unusable The sheer screen size lets you browse most technical books without zooming. I routinely use it to study for uni. (Thats mainly why i got it)

It is quirky and not that good looking, but it is very usable - the android part works good (if you kill connectivity it will power down after inactivity so battery life is not an issue)

Bottom line is: I'm glad i got it. Play store means i can supplant broken pre installed stuff. Missio critical objective - reading B5/A4 pdfs - works better than i hoped. The isn't pretty or sleek or polished, but it gets the job done and it does it good

PS. Previous version got an community ubuntu flavour, dont know about this one.

<comment><AUTHOR> bull's eye!

</comment></comment><comment><USER4> What issues in particular did you have with PDFs on the Kindle? If it was the low resolution for images and such, the Paperwhite and Voyage have higher DPI displays (212 and 300, respectively, versus the original Kindle's 167).

I've had some issues with PDF rendering on my Paperwhite also, but I've usually been able to convert them into a more readable form with Calibre.

<comment><USER5> I'm pretty happy with PDFs on my new Kindle Voyage, too.

</comment></comment><comment><USER6> errata: The Onyx BOOX M96 - 9.7" has just 1200x825 pixels as written on the box (actually in Polish it has 1280x825). That's indeed VERY low in DPI compared my various Kindles, and yet this is a honestly engineered PDF reader - beating all what Amazon has in the PDF department for 2-column academic papers. Despite its low DPI, which is its major flaw...

I love sending HTML pages to my kindles for proper reading, and regret that even with Calibre most tech PDFs are just too small, greyish low contrast for reading.

The Sony DPS has 2x the resolution (still insufficient by 10x to compete with paper) and 2.5x the price, while unable to cope with anything else than PDF.

Kindles and Kobos HD have nearly 300dpi, though too small and restricted to their own format and a poor WWW experience.

It's the 1st time that i praise a reader that's neither high DPI (blame the eInk monopoly for stagnation in this field), nor of highly polished (pun intended) quality, nor cheap (at $350 is 3-4x more than sundry Kindles/Fire), nor easy to use for the common iCrowd.

Yet the M96 proves that a generic Android with decent PDF rendering and supporting all the ePub, mobi/azw, MSO files is doable... even in monochrome and lacking video. I'm glad i bought it against all odds and specs, as support of honest engineering and risk taking companies.

Its basics are right, it's a device for geeks, students, nerds and pros willing to accept all the compromises enumerated above and in emp_zealoth's post (+1) in return for sparing their eyes from glowing/glaring LCDs and OLEDs.

With about 10x the current resolution for a 13" panel with perhaps 3-4 colors, this could become in a few years the ultimate ereader and paper replacement.

</comment><comment><USER7> Afaik duokan for kindle is the best alternative. Also , the place to ask is mobileread.com

<comment><AUTHOR> Intriguing, but does it address the pdf reading problem?

<comment><USER7> Haven't yet used it.

</comment></comment></comment><comment><USER8> > LCDs like that of ipad/nexus don't work for me

Why don't they work? Also, do you mean that you are looking for an e-paper solution?

<comment><AUTHOR> Yes, I'm looking for an epaper solution since my eyes get readily fatigued while staring at an LCD, which I have to do a lot.

<comment><USER9> Hmmm... that sounds like you might need glasses (or updating your prescription if you already have ones).

Several of my coworkers were prescribed glasses for the same problem. Go visit your ophtalmologist.

Edit: emph_zealot's suggestion looks like exactly what you need, but I'd still have my eyesight checked if I were you.

<comment><AUTHOR> thanks for your suggestion, but let me clarify- I read books to wane off the fatigue caused by looking at the computer screen for long intervals; this purpose would be defeated if I were to use another lcd screen!

Also, my eyes are in perfect health as verified by a recent checkup, but I appreciate the advice. Thanks again.

<comment><USER5> If you are reading on an LCD screen, please also give flux / twilight / redshift a try. They adapt the colour temperature at night.

</comment></comment></comment></comment></comment><comment><USER10> I use an iPad mini + Google Drive for Oreilly/Manning books + Goodreader (app).

The iPad was a gift and I just started collecting pdf books. I also use the Amazon app whenever I purchase ebooks from their marketplace. This allows me to stay versatile and automatically sync all my notes, etc. remotely.

<comment><USER11> +1 for Goodreader. Such a usable app. You'd think a pdf reader is a pdf reader, but there are many small features and UI niceties I would miss if I switched from Goodreader. I wish there were something as good for Android.

On Android (and jailbroken Kindles) Koreader (open source on github) does a decent job of reflowing even 2-column PDFs.

</comment></comment><comment><USER12> What worked for me was converting the PDFs to Kindle format, using Calibre. Most of the time it works fairly well.

<comment><AUTHOR> > Most of the time it works fairly well.

Did not for me!

<comment><USER12> What type of books are you converting? I've found text only works perfectly, but anytime there's images it's either a hit or a miss.

<comment><AUTHOR> correctly guessed, I read books that have a lot of images/illustrations!</comment></comment></comment></comment>


The boiling, erupting Sun - <AUTHOR> http://blogs.discovermagazine.com/badastronomy/2010/10/28/the-boiling-erupting-sun/

<comment><USER2> These kinds of astronomical pictures are why I got into astronomy. Simply breathtaking.</comment>


Ask HN: How do you get things done in hourlong intervals? - <AUTHOR> I'm talking about code.
When I was alone, I would go to the coffee shop and do work for 6 hours, and tune out the rest of the world. But now I'm visiting a friend, and keep getting interrupted by them in order to do various things, and it seems like my intervals to get things done are altogether too small to really dive into my code. I've made minimal progress, even though there are so many things to do, and I have so much time(on aggregate) to do them.
Possible solution: every time I wrap up a coding session, write a paragraph summarizing what needs to be done next, why, and where? <comment><USER2> When I know I am wrapping up a coding session, I'll often leave a few trivial things unfinished - so that when I sit back down again for the next session, I'll have an easy place to jump right back in, and it will force me to (consciously or otherwise) get back to somewhere near the mental state I was in when I half-finished the thing.

In this particular situation, pick what you want to do: get your coding done or visit your friend. If the priority is the former, tell them "I've got some work to do for a few hours, so I'm going to go find a Starbucks and set up station. Want to meet for a drink around <five hours from now>?" If the priority is the latter, then embrace the distraction, spend some quality time/activities with the person you've traveled to visit, and if you must code, keep it to small discrete chunks like bugfixes or writing unit tests.

</comment><comment><USER3> Work in 48 minute increments...

[http://successbeginstoday.org/wordpress/2006/09/the-power- of...](http://successbeginstoday.org/wordpress/2006/09/the-power- of-48-minutes/)

I've been doing this for years (depending, of course, on what I'm working on). It can be very effective.

Get a digital timer. Set it for 48:00. Start it. You may not stop working until it beeps. 12 minutes for stretch, email, bathroom, snack, etc., is plenty. Then back to work. And no alt-tab to hacker news during the 48 minutes. You're only cheating yourself.

<comment><AUTHOR> That doesn't answer my question, though. My point is, how do you know WHAT to work on for 48 minutes? How do you facilitate things so that you can load the problem into your working memory quickly?

<comment><USER4> I've been reading about the American inventor Elmer Gates.

<http://www.elmergates.com/egamu/egamu_link_page.htm>

I think his three tips would be:

(1) quiescence (2) periodicity (3) dirigation

Which in modern language means (1) remove or zone out all distractions, (2) work at the same time each day, (3) pay deliberate attention to each aspect of the problem in turn in your imagination. (4) Repeat.

Step (3) helps to build the relevant mental structures. Visiting them habitually will strengthen them.

'Each aspect' will include mental images (visual, aural, kinaesthetic), concepts, ideas (including all the relevant constraints and conditions that must be met). Generating mental images is important as the mind works best with things that are translated into sensory terms.

Eventually a conjectured solution will pop into your conscious mind, possibly at a time outside the working hour. You may know it is coming before it arrives.

He might also add that thinking draws from one's personal store of vitality. So it may be necessary to adjust sleep, diet, exercise and any addictions you have in order to conserve this.

Being unstressed is vital. Partly this means only working on important problems and working with the intention to do good. It also means switching to a different problem if you get bored of the first one. (You can go back to it another day).

</comment></comment></comment><comment><USER5> <http://www.pomodorotechnique.com/>

And for the best tool to use to apply the technique: [http://www.apple.com/downloads/macosx/development_tools/pomo...](http://www.apple.com/downloads/macosx/development_tools/pomodoro.html)

This has really changed my work habits. The tool is just perfect. One easy click and you can see what you've worked on the whole day for how much time. Brilliant.

</comment><comment><USER6> End of day, I usually list out the things I want to get done the next day (just point format a small list) on my notepad (a physical paper and pencil). Then I stick to it.

I don't think writing paragraphs helps, keep it real simple. "Fix bug #12345, Test blahblah, Tune XXX, etc"

<comment><USER7> Totally agree with this. I also find if I separate the "research" aspect of something from the implementation things go more smoothly as well.

For example at the end of each day I'll gather documentation, code samples, etc... for the things I want to get done the next day.

</comment></comment><comment><USER8> I have and am experiencing what you experience atm.

The first thing which is helping me very, very much is focus training. After like a month of training to focus fast and hard, I am able to dive into a problem and ignore everything around me within minutes, allowing me to work more efficiently faster, resulting in more getting done in less time.

The second thing, closely related to the first (and an item on my list of mental improvements) is improving focus stability, but I know several people with a very, very stable focus. Interrupting them is pretty much like being an IO-Interrupt in a modern processor. It takes a few seconds, but then everything they need right now is swapped out and they can answer simple things or do something easy for a few minutes until the interruption is over and they go back to their focus and continue doing what they were doing.

Going a bit further, this is a certain experiment me and several people are performing on a fun base: Is it more efficient to sleep less and thus get more done due to having more time, or is it more efficient to improve ones focus and get done more in the same time, by being better? :)

The third thing which works very well for me is dynamic, hierarichal task structuring. Whenever I have a task and I consider if I am very confident if I can finish this particular task in the time I usually have (I don't start working seriously if I have less then half an hour to an hour). If I can, I finish it, if I cannot, I split it into more subtasks (or sometimes decide that this is not possible and somehow mark this task). Doing this, I always have a set of tasks which fit nicely into a short time and tasks which need to be refined further.

<comment><USER9> A reference/link to focus training would be much appreciated.

As for your experiments with getting less sleep vs better focus, I think that stuffing more hours in the day is only more effective for tedious work, where you don't have to think much to get it done. In my experience, when the quality of your ideas and code matters more than just getting it done, you definitely should be well rested.

In most cases, the self-inflicted pain from getting less sleep is just a psychological substitute for real productivity

</comment></comment><comment><USER10> <http://www.paulgraham.com/makersschedule.html>

</comment><comment><USER11> Extremely detailed microtasking. Like this: <http://taskulus.com/taskulus17.png> or <http://taskulus.com/taskulus16.png>

Each of these is a very small tasks that 1-20 minutes but it's important that I can devote my complete concentration and focus to it, without the need to load everything else in my brain.

That of course requires a lot of what SICP calls "deferred decisions" and so units of my code are forced to be much more independent.

</comment><comment><USER12> Much of what I am doing these days takes a couple of hours to spin up, so I don't have any ideas for you here. Maybe find a way talking with your friend about your need for long stretches of uninterrupted time. Already mentioned here is Paul's very relevant essay <http://www.paulgraham.com/makersschedule.html>.

<comment><USER13> Yes, this sounds like an occasion for social optimization rather than technical. See whether you and your friends can agree to a schedule that leaves you with a good block of uninterrupted time. Perhaps they would prefer to do things in the afternoon, leaving you the mornings free. Or they may be earlybirds, leaving you the evenings. This latter might be hard, though, as your work time would come at the end of the day and you might be tapped out.

Another recommendation might be meditation, rather than additional overt effort. It would take practice, but 5 to 10 minutes of quieting the mind might allow the context you want regain to emerge more quickly. I'm not too experienced in this personally, however, although I do find lost context returning to me once my physical environment quietens.

</comment></comment><comment><USER14> <http://smacklet.com/>

</comment><comment><USER15> Don't let them interrupt you..?</comment>


What is C in practice? - <AUTHOR> http://www.cl.cam.ac.uk/~pes20/cerberus/notes50-2015-05-24-survey-discussion.html <comment><USER2> _It remains unclear what behaviour compilers currently provide (or should provide) for this._

It might be nice if future surveys explicitly asked a followup question "Regardless of the standard or behavior of existing compilers, is there one of these answers that is the 'obviously correct' manner in which compilers should behave? Which one?"

If practically all users believe that the same answer is 'obviously correct', compiler writes might want to take this into account when deciding which behavior to implement.

      For MSVC, one respondent said:

      "I am aware of a significant divergence between the LLVM
      community and MSVC here; in general LLVM uses "undefined
      behaviour" to mean "we can miscompile the program and get
      better benchmarks", whereas MSVC regards "undefined
      behaviour" as "we might have a security vulnerability so
      this is a compile error / build break". First, there is
      reading an uninitialized variable (i.e. something which
      does not necessarily have a memory location); that should
      always be a compile error. Period. Second, there is reading
      a partially initialised struct (i.e. reading some memory
      whose contents are only partly defined). That should give a
      compile error/warning or static analysis warning if
      detectable. If not detectable it should give the actual
      contents of the memory (be stable). I am strongly with the
      MSVC folks on this one - if the compiler can tell at
      compile time that anything is undefined then it should
      error out. Security problems are a real problem for the
      whole industry and should not be included deliberately by
      compilers."

I'm much less familiar with MSVC than the alternatives, but this is a refreshing approach. Yes, give me a mode that refuses to silently rewrite undefined behavior. Is MSVC possibly able to take this approach because it isn't trying to be compliant to modern C standards? Does it actually reduce the ability to apply useful optimizations? Or just a difference in philosophy?

<comment><USER3> _" First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period."_

You cannot implement that efficiently without rejecting valid programs. Consider code like this:

      int x;
      if( f()) x = 1;
      if( g()) h(x);

For sufficiently complex functions f and g, there's no reasonable way to decide at compile time whether x will be set whenever g() returns true. For example, f() might always return false because Fermat's last theorem is true.

And that 'reasonable' likely isn't a necessary part of that statement.

<comment><USER4> Yes, this is exactly the reason for undefined behavior. It is often overlooked in the "well, the compiler should just throw an error instead of invoking undefined behavior" debate: the compiler doesn't just "choose to invoke" undefined behavior: it relies on it to do program analysis.

For example, your example could legally be rewritten to:

        int x;
        f();
        x = 1;
        if (g()) { h(x); }

The only difference is if f() were false, and someone would then access x and see 1; but that's undefined behavior, so you can ignore it. In fact, assuming x is not accessible outside this block:

        f();
        if (g()) { h(1); }

These optimizations happen all over the place, it's not the compiler invoking or causing undefined behavior, but assuming that it won't ever happen.

EDIT: Note the further optimization that looms: if f() can be proven pure (no side effects), then it can be removed. This makes little sense for a function with no arguments (in which case it would just be a constant). If, however, f(y, ...) is some expensive but pure function, it can just be removed completely.

<comment><USER5> _" Note the further optimization that looms: if f() can be proven pure (no side effects), then it can be removed. This makes little sense for a function with no arguments (in which case it would just be a constant)."_

It could apply if f computes a value based on global state but doesn't change anything, which is slightly weaker than "pure".

</comment></comment></comment></comment><comment><USER6> There is a relevant article [http://blog.regehr.org/archives/1180](http://blog.regehr.org/archives/1180) and discussion [https://news.ycombinator.com/item?id=8233484](https://news.ycombinator.com/item?id=8233484) about what programmers really think C should be like (i.e. the "portable assembler" it was designed to be originally); it incidentally also shows what they think a sane machine architecture should be like.

</comment><comment><USER7> Question 2 is :

Is reading an uninitialised variable or struct member (with a current mainstream compiler):

(This might either be due to a bug or be intentional, e.g. when copying a partially initialised struct, or to output, hash, or set some bits of a value that may have been partially initialised.)

a) undefined behaviour (meaning that the compiler is free to arbitrarily miscompile the program, with or without a warning) : 128 (43%)

b) ( * ) going to make the result of any expression involving that value unpredictable : 41 (13%)

c) ( * ) going to give an arbitrary and unstable value (maybe with a different value if you read again) : 20 ( 6%)

d) ( * ) going to give an arbitrary but stable value (with the same value if you read again) : 102 (34%)

e) don't know : 3 ( 1%)

f) I don't know what the question is asking : 2 ( 0%)

\</comment><comment></comment><comment></comment><comment>--

I know of one datastructure (a sparse set of integers from 1-n) which relies on this behavior: [http://research.swtch.com/sparse](http://research.swtch.com/sparse) . I always thought it was a neat trick. However, from the article is seems that may NOT give stable values to uninitialized members. Which may make that data structure behave strangly or cause the program to miscompile.

<comment><USER8> That's a question for which multiple answers are correct: a, b, _and_ c.

<comment><USER9> I disagree regarding b. I think you'll find the outcome of this expression quite predictable:

int b; int c = b * 0;

<comment><USER8> Sure, if the expression doesn't depend on the value at all, it _probably_ won't have unpredictable results. (Though as with any kind of undefined behavior, don't count on it, as compilers can be "clever" sometimes.)

</comment></comment></comment></comment><comment><USER10> A lot of C programmers make valid assumptions based on their system architecture and compiler. Is there any point trying to unify their obviously different practices, while at the same time ignore the Standard? No.

<comment><USER11> To me, the whole point of C is to enable one to take advantage of one's system architecture. Thats not really possible in higher level languages.

<comment><USER12> Well, one of the interesting takeaways from the article is that C as specified often does _not_ let you take advantage of your system architecture. The specification has things like GPUs and segmented memory architectures in mind when it forbids you from doing seemingly reasonable things like taking the difference between the addresses of two separately-allocated objects, even though chances are very good what you're trying to do works just fine on all architectures you care about.

<comment><USER13> Yes and no. The standard specification defines a C that is very portable. It is quite reasonable that you cannot _portably_ take advantage of your system architecture, which means you cannot do it using C as defined by the standard.

But you still can write stuff that looks just like C, that a C compiler will accept, and that lets you take advantage of your system architecture. You just need to understand that a new version of your compiler can break all of your nifty tricks.

<comment><USER10> Finally someone that understands.

<comment><USER14> Not really because there are two kinds of c programer out there.

1\. Those that communicate with the outside world exclusively via library calls to an operating system.

2\. Those that do not.

People that live in world #1, usually don't get people living in world #2. Example.

      // wait for write ready
      while((spi.S & SPI_S_SPTEF_MASK) == 0)
        ;

      spi.D;
      spi.D = data;

Guess what,

No spi.D; is actually important. No just because the program never writes to spi.S does not mean you can delete the while loop. No you cannot reorder any of this and have it work.

<comment><USER10> Sorry, I have no idea what you are trying to say. Maybe stick to the relevant issue and be less abstract?

<comment><USER15> His example is anything but abstract. Embedded programming is a world unto itself. You sometimes need to read an address before you can write, etc.

<comment><USER5> I think the parent meant "implicit" rather than "abstract". The code is not abstract; but the pragmatic intent of the comment is not made very clear. I believe I follow but I'm not confident enough to risk confusing things by guessing.

(I believe I follow the intended conversational import; I certainly follow the code.)

<comment><USER14> Sort of the point is, in pure languages only the symbolic operations are important. And the side effects (memory operations) are unimportant and beyond the control of the programmer. So you can do all sorts of transformation on the code without effecting the end result.

C is definitely impure. There is a large set usage cases where the memory layouts are important. Where the orders of operation and memory accesses are important.

What I worry about is when the optimizer is allowed to make assumptions about undefined behavior, that may be actually important on certain targets. Consider referencing a null pointer.

Winows7, Linux, in user land if you do that and you'll get a seg fault. And if the default handler runs your program dies.

The ARM cortex processor I've been slopping code for, reading a null pointer returns the initial stack pointer. Writing generates a bus fault. Since I actually trap that, and error message gets written into non-initialize memory and then the processor gets forcibly reset.

On an AVR reading a null pointer gives you the programs entry point. Writing to that location typically does nothing. Though you can write to it if running out of a special 'boot sector' and you've done some magic incantations.

So that's the problem I have with the idea that 'undefined means the optimizer can make hash of your program' instead of trusting the back end of the compiler will do something target appropriate.

<comment><USER13> Yes, embedded is very side-effect-ful. When you have to make that kind of thing work, you usually have to know both your hardware architecture and your compiler.

      *(unsigned long*)0xEF010014 = 0x1C;

presumes memory mapped I/O, a 32-bit hardware register at 0xEF010014, and what 0x1C will mean to that register. It also presumes that the compiler will generate a 32-bit access for an unsigned long.

Going back to Gibbon1's code sample, you have to know what your compiler is going to do at what optimization level. You either have to declare spi.S to be volatile, or you have to compile at -O0, or some such. And you may well need to review the generated assembly code a few times in order to _really_ understand what your compiler is doing with your code.

If you're writing the Linux kernel and you want to be portable across hardware architectures, it gets harder. You can't just be processor- or hardware- specific. You probably need the volatile keyword. I don't know what the kernel compiles in terms of optimizations, but I bet it's not aggressively optimized.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> As noted in the article, a summary of this document (about one-third the length) from the same authors is available at [http://www.cl.cam.ac.uk/~pes20/cerberus/notes51-2015-06-21-s...](http://www.cl.cam.ac.uk/~pes20/cerberus/notes51-2015-06-21-survey- short.html)

</comment><comment><USER17> It's scaring to notice that all this is undefined behavior. Hell, glibc network functions rely on undefined behavior!

</comment><comment><USER18> Note that the C99 (I'm not up on C11 yet) standard specifically allows type- punning through the use of unions (and disallows it in essentially all other cases except when one type is a character type).

Also, there seems to be some confusion about storing and loading pointers, when the standard speaks to this as well; roughly: a pointer which is converted to a "large enough" integer type will point to the same object when converted back. It is permissible for an implementation to not provide a large enough integer type, but excepting that, the behavior is well defined.

<comment><USER19> C89 prohibited union type-punning. C99 allowed it, but mistakenly left the prohibition in the (non-normative) list of undefined behaviors. C11 did rectify the list.

<comment><USER18> One of the TCs for C99 fixed the list I believe.

</comment></comment></comment><comment><USER20> As far as it's possible compilers should check as much as possible when false assumptions are made.

Is it possible to build a language, which would reduce the number of false assumptions?

</comment><comment><USER21> Interestingly git used bit fields in it's code.

</comment><comment><USER22> This is why we need to implement everything in JavaScript

 _hides_

<comment><USER23> Apparently, to some people, irony is an undefined behavior...

(Just to be clear, I am referring to the people that downvoted you.)

<comment><USER22> Good one. Too bad humour is not tolerated here :)

<comment><USER24> As I've said before: the thing is, the HN crowd is a really tough audience -- they will only be amused by jokes if they're _actually funny_.

If your idea of humour is "make some pop-culture reference and it will automatically be funny" or "be generically cynical about everything and it will automatically be funny" then, yeah, you're going to have a difficult time on HN.

Something like 1% or so of my HN comments are jokes. They usually do pretty well for karma. (Slightly to my surprise, the most recent one to flop completely was one that was also making a slightly serious point, and one that I don't think many HN readers would disagree with. Ah well, can't win 'em all.)</comment></comment></comment></comment>


EU deleted post blaming critics of copyright directive of russian influence - chupa-chups https://web.archive.org/web/20190215114522/https://medium.com/@EuropeanCommission/the-copyright-directive-how-the-mob-was-told-to-save-the-dragon-and-slay-the-knight-b35876008f16 <comment> chupa-chups As you can see, gone now:

[https://medium.com/@EuropeanCommission/the-copyright- directi...](https://medium.com/@EuropeanCommission/the-copyright-directive- how-the-mob-was-told-to-save-the-dragon-and-slay-the-knight-b35876008f16)</comment>


Google and Microsoft Cheat on Slow-Start. Should You? - <AUTHOR> http://blog.benstrong.com/2010/11/google-and-microsoft-cheat-on-slow.html <comment><USER2> This isn't much of a secret. As it says in the article Google are lobbying to change the initial window size in the RFC. A lot of people here at Yahoo! want to see that too, and personally I think we should be more aggressive with our initial window, RFC be damned.

This topic was covered really well by Amazon's John Rauser at Velocity Conf: [http://velocityconf.com/velocity2010/public/schedule/detail/...](http://velocityconf.com/velocity2010/public/schedule/detail/11792)

To address the points in the conclusion:

1\. Fast is good. Fast is also profit.

2\. The net-neutrality argument here is totally bogus, anyone that knows how can up their slow-start window today if they choose to. There doesn't really have anything to do with traffic shaping.

3\. Google have been using their usual data driven approach to support their proposal for IETF. We need a lot more of that. It's great. The only way we can really find out how the Internet in general will react to changes like this is to test them in some real world environment.

4\. I agree, slow-start is a good algorithm with a very valid purpose. The real problem here is that the magic numbers powering it aren't being kept inline with changes to connectivity technology and increases in consumer/commercial bandwidth.

</comment><comment><USER3> There's all sorts of latency problems caused by the congestion window size (and how it gets reset), because of how the algorithm works unless you're sending a continuous stream of data (which allows the congestion window to grow) than the window gets reset to it's initial size which can mean waiting for an ack round-trip before you get the whole message.

While it's not that big a deal if your users are local to you, if they're on a different continent each extra roundtrip can easily add 100ms.

I used to do TCP/IP tuning for low latency trading applications (sometimes you need to use a third party data protocol so can't just use UDP), this sort of stuff used to bite us all the time.

If latency is important it is worth sitting down with tcpdump and seeing how your website loads (i.e how many packets, how many acks, etc.) as often there are ways of tweaking connection setting (either via socket options or kernel settings) that can result in higher performance.

(Try using tcp_slow_start_after_idle if you're using a recent linux kernel; this won't give you a bigger initial window, but it means once your window size has grown it won't get reset straight away if you have a gap between data sends)

</comment><comment><USER4> This is interesting, but the article and I differ greatly at this point: "Being non-standards-compliant in a way that privileges their flows relative to others seems more than a little hypocritical from a company that's making such a fuss about network neutrality."

No, no it's not. This has nothing to do with network neutrality; it's a purely server-side change/fix. Not only that, they're benefiting users without requiring anyone else to change while they wait for standards bodies to catch up. This is a similar scenario to HTML5 video, and distinctly more clear-cut than e.g. '802.11n draft' wireless routers in my opinion.

<comment><USER5> "Benefiting _their_ users," yes. But they're not benefiting whoever else is sharing the smallest-capacity link -- what he's arguing is that Google is crowding them out.

Net neutrality is not the only way to privilege your flows on the Internet: There's nothing to stop me from writing a crude application-layer protocol atop UDP that implements reliability but not congestion control. (You maybe had to implement something like this for your networking class in school; otherwise you could start with a protocol like DCCP.) If I were to use that to send data as fast as I could to some remote computer, I could be sending more data than the smallest-capacity link could handle. Other TCP/IP connections sharing that link would detect data loss and thus reduce the amount of data they put in transit, but my protocol wouldn't have to. I can monopolize that link.

So assuming your physical layer is tin cans and string, what he's arguing is that if you have a link with a capacity of 12 segments, then data from Google will use 10 of them and a client will never expand its outstanding data beyond 2 segments. If both used vanilla TCP/IP, they should share the link evenly.

Of course, speed is a critical factor for Google. Android by default uses TCP Westwood+.

It's been six years since I tinkered with TCP/IP and really focused on networking, so someone please correct me if I'm wrong >_<

<comment><USER6> > So assuming your physical layer is tin cans and string, what he's arguing is > that if you have a link with a capacity of 12 segments, then data from > Google will use 10 of them and a client will never expand its outstanding > data beyond 2 segments. If both used vanilla TCP/IP, they should share the > link evenly.

He's not showing any evidence that Google wouldn't back off to a smaller window in the face of packet loss; he's just saying their _initial_ window is 9 segments. Once one of those 10 segments in your tin-can router falls out of its receive buffer, Google will be down to 9 and the other guys get up to 3, and packet loss will random-walk toward fairness.

Right? I've _never_ tinkered with this stuff, so please correct me if I'm wrong.

<comment><USER5> I thought that the minimum value cwnd could assume is IW, but looking at RFC 2581 that isn't true: "Upon a timeout cwnd MUST be set to no more than the loss window, LW, which equals 1 full-sized segment (regardless of the value of IW)." They even explicitly call out what I erroneously believed, so you are right -- I apologize!

If you really have a tin cans and string physical layer, Google's larger IW could be more disruptive to other connections on the link: A vanilla TCP connection would ramp up cwnd from 1 segment until congestion is observed (in the slow-start phase) and then grow cwnd conservatively (after ssthresh is first set). If congestion would be observed at a cwnd value less than 10 segments, then starting with IW at 10 segments could be very disruptive to others sharing the connection.

Mind you, this argument feels very... academic. As you pointed out, Google's connection would converge toward fairness anyway. (Unless so little data is actually transmitted, then the connection probably isn't open long enough for that to happen.) And most shared links don't saturate at 12 segments. I'd guess that a high-capacity link would only be at risk if it has a lot of connections (so that every connection has a cwnd not much greater than 10) and there are always many (albeit, short-lived) connections to Google always being created (which could appear as fewer long-lived connections with a constant cwnd of 10, more than would be fair).

</comment></comment></comment></comment><comment><USER7> Google is proposing this should be allowed as a modification to rfc-3390. Their draft is <http://tools.ietf.org/html/draft-hkchu-tcpm-initcwnd-01>. Active discussion of the issue may be found at [http://www.ietf.org/mail- archive/web/tcpm/current/maillist.h...](http://www.ietf.org/mail- archive/web/tcpm/current/maillist.html)

<comment><USER8> Minor correction: the current draft is <http://tools.ietf.org/html/draft-ietf- tcpm-initcwnd-00>

</comment></comment><comment><USER9> Really interesting research, but man, if you really, _really_ have to worry about premature optimization for your web app, I'd start with the usual bottlenecks first - i.e. anything that involves disk IO and/or processor work, such as databases and mathematical calculations.

Unless you are serving static content only (in which case you are hardly creating an "app"), the milliseconds you might save with TCP-level optimizations are _peanuts_ in comparison to the multiple seconds your database and computations will be requiring.

<comment><USER6> This is exactly backwards. My network latency to North America is >200ms (RTT). Three round-trip times is about 750ms. You can do 75 disk accesses and three _billion_ mathematical calculations in that time.

If your database and computations are requiring _multiple seconds_ on a normal web page, you have _serious_ user experience problems. When you're under 140ms, it feels like the response is happening at the same time as the request (Dabrowski and Munson weren't able to reproduce the old 50- or 100-millisecond rule of thumb in what sounds to me like a poorly-controlled experiment; [http://books.google.com/books?id=aU0MR-MA- BMC&pg=PA292&#...](http://books.google.com/books?id=aU0MR-MA- BMC&pg=PA292&lpg=PA292&dq=200+millisecond+user+interface&source=bl&ots=Pxl6LkkTpQ&sig=5hOX- eKJwi95Ete- PsdgL7CczbI&hl=en&ei=YSPwTJPONoT48AbOjNHzCw&sa=X&oi=book_result&ct=result&resnum=3&ved=0CB0Q6AEwAg#v=onepage&q=200%20millisecond%20user%20interface&f=false)). Increasing Google search page render time from 400ms to 900ms dropped traffic by 20%, according to Marissa Mayer ([http://glinden.blogspot.com/2006/11/marissa-mayer-at- web-20....](http://glinden.blogspot.com/2006/11/marissa-mayer-at- web-20.html)). Traditional OLTP systems tried to keep response times under one second; beyond a second, people start to get frustrated and wonder if something is broken.

So, for a normal application, the milliseconds you might save by optimizing your database and computations are _peanuts_ in comparison to the second or more that TCP-level optimizations could save you.

<comment><USER10> > If your database and computations are requiring multiple seconds on a normal > web page, you have serious user experience problems.

This is almost always the case when I think "this website is slow", though. When HN is slow, it's not because of some added network latency, but because something is making HN take 3 seconds to serve up my "threads" page, or 2 seconds to successfully post my comment. Same with "reddit is slow", or "this Wordpress blog is taking forever to load" or "Twitter spins the 'working' icon for 2 seconds when I click on that 'retweets' thing in the sidebar before returning anything". Those things are _really_ common in my experience, and those rather than network round-trip times are by far the biggest and most annoying slowdowns, at least in my browsing.

<comment><USER6> Yes, of course, because network latency is probably almost the same to almost all sites to you. So the vast majority of pages take 400ms or whatever to load, and you don't think anything of it. You only notice when you hit a site that's out of the ordinary — whether it's Google loading in 85ms by using IW9, or Twitter taking multiple seconds to query the snail farm or frozen molasses or whatever it uses for its database now that it's dumped Cassandra.

If IW9 or IW10 gets widely adopted, you'll find yourself thinking "this website is slow" when you visit the rare backwater that doesn't use it.

</comment></comment></comment></comment><comment><USER11> There was a large discussion earlier about the subject. I posted detailed comments in that thread so I won't repost but just link. <http://news.ycombinator.com/item?id=1143317>

</comment><comment><USER12> Can anyone else repeat his experiment?

I tried repeating the experiment. I'm in Sweden, so, annoyingly, a request to google.com redirects to google.se. If I send my request directly to google.se, I get 9k response in 130ms and the initial window looks like 4 to me, i.e. I can't see anything unexpected happening.

I then tried repeating on Amazon EC2. I can't see anything unexpected there either, but the RTT from EC2 to google is only about 3ms, which means I can't assume that the ACKS don't get there.

(The original article author looks at how long the initial 3-way handshake takes and then assumes that all packets take that long, or, probably, half as long, i.e. he assumes that ACKS sent up to one RTT before a packet from google can't have arrived at google in time to affect that packet)

Can anyone else reproduce the experiment?

Other ideas: repeat from Sweden, but send a cookie so that I really get google.com. Repeat from EC2, but make sure I never send any ACKs after the three-way handshake. I'm not curious enough to do the latter, it's a fair bit of work.

</comment><comment><USER13> It seems Linux does not have option to skip slow start and just use receiver's advertised window. Does anybody know where in net/ipv4/tcp.c this should be set?

<comment><USER14> Note that we're not talking about skipping slow start completely; we're talking about changing slow start parameters.

</comment></comment><comment><USER15> Does anyone know what you would do to easily tune this for FreeBSD?

<comment><USER16> My guess is setting the sysctl "net.inet.tcp.slowstart_flightsize" from the default value of "1" to something else.

<comment><USER17> Any idea if that would work on Linux?

<comment><USER16> No idea. Type "sysctl -a | grep slow" and see what knobs the kernel offers.

</comment></comment></comment></comment><comment><USER18> I believe the current limit for slow-start are not adapted to the current Internet anymore.

According to my own observations, the first 30Ko of my pages seem to be transfered faster then the next 30ko. It is not until much more is sent that the average throughput eventually get up to what it was during the first 30ko.

This is definitely weird.

Note: I am using Ubuntu on EC2 hosted VMs.

As a result, for as much as I can, I try to keep the size of my content below 30ko, using multiple concurrent HTTP requests.

I believe this is related to "slow-start" being pessimistic.

Unfortunately, "slow-start" is not configurable on Linux and I don't feel confident enough to go with some kernel level patch...

Any clue?

<comment><USER19> You can't use custom kernels on Amazon EC2 anyway, so kernel patches aren't really an option (unless you had some kind of kernel module you can load that would change the value in memory, which seems dangerous).

<comment><USER20> You can use custom kernels on EC2 now.

[http://ec2-downloads.s3.amazonaws.com/user_specified_kernels...](http://ec2-downloads.s3.amazonaws.com/user_specified_kernels.pdf)

</comment></comment></comment><comment><USER21> "It is better to ask for forgiveness than permission" - Rear Admiral Grace Hopper

</comment><comment><USER22> We have been measuring google's "reachability" performance and it is quite amazing. The results of their tuning is that they can achieve downloading of their initial HTML in under ~250 milliseconds and many locations under 100 ms. The other thing the data shows is the standard deviation on the download times are very small making the site consistently load fast.

[http://www.yottaa.com/url/4be004065df8ca5a730001fb/reachabil...](http://www.yottaa.com/url/4be004065df8ca5a730001fb/reachability)

</comment><comment><USER23> _"They actually managed to deliver the whole response in just 70ms, 30ms of which was spent generating the response"_

Isn't part of that just the network latency? Based on the timestamps for the SYN and SYN-ACK it looks like a RTT of about 16ms.

EDIT: Nevermind.

Request was sent by the client at 00.017437

Request ACK was received by the client at 00.037139

RTT of about 20ms, so the request was received by the server around 00.027

First packet of the response was received by the client at 00.067151

67-27=40. Assuming a latency of 10ms it took 30ms to generate the request.

</comment><comment><USER24> One should also note that when IE is talking to IIS, the request will be sent in the first packet and the initial response will be sent in the first ACK. You can actually complete a request and response (if small enough) in 3 packets. Also, when tearing down the connection, it's left half-open.

[http://osdir.com/ml/mozilla.devel.netlib/2003-01/msg00018.ht...](http://osdir.com/ml/mozilla.devel.netlib/2003-01/msg00018.html)

</comment><comment><USER25> Mike Belshe - An Argument For Changing TCP Slow Start (Jan 11, 2010):

[http://sites.google.com/a/chromium.org/dev/spdy/An_Argument_...](http://sites.google.com/a/chromium.org/dev/spdy/An_Argument_For_Changing_TCP_Slow_Start.pdf)

</comment><comment><USER26> On Ubuntu 8.04 (at least), you can set this per route via something like:

ip route change default via x.x.x.x dev eth0 initcwnd 6

but please test thoroughly if trying this.

</comment><comment><USER27> Do app engine apps also serve like this?

<comment><USER28> I don't think so. I'm pulling a 27k URL over a 100ms latency and I'm seeing roughly 2, 4, 8, 8... for the send bursts.

</comment></comment><comment><USER29> Very interesting. Is such a thing configurable in Apache or nginx? It seems to be a rather rude behavior, but I'm curious how accessible it is.

<comment><USER30> something like: ip route change default via 0.0.0.0 dev eth0 initcwnd 10

<comment><USER6> That doesn't seem to work for me:

        RTNETLINK answers: No such file or directory

Apparently I need to patch my kernel?

<comment><USER31> slight offtopic: that's one of my favorite error messages that is actually relatively common.

</comment></comment></comment></comment><comment><USER32> very interesting post.

</comment><comment><USER33> Interesting. But it really annoys me when people use "begs the question" incorrectly. Look it up!

</comment><comment><USER34> Interesting, but there are so much more important things to consider before worrying about the load time. (i.e. 0 user experiencing 30 ms is far worst..)

<comment><USER35> Half agree: this level of optimization is less useful when you're not starting from Google's performance baseline. That said, can't agree with point generally applied to load times: optimizing them made a difference even at BCC scales back in 2008ish. Implementing half of the YSlow recommendations takes under an hour in modern web frameworks.</comment></comment>


AirBnB Payments and Plaid - <AUTHOR> https://fin.plaid.com/articles/finterview-airbnb-jonathan-golden <comment><USER2> Woah. Cuba story is super interesting. Shocking how difficult it is to make payments there. Is this a company that does this? Or does AirBnB hire the bike messengers directly?</comment>


How getaccept achieved 100% uptime - <AUTHOR> https://blog.ycombinator.com/how-getaccept-achieved-100-uptime/ <comment><AUTHOR> I'm curious about how much time and effort was involved in building this environment.

Though I'd love to do it for my app, and I'm hoping and assuming this sort of environment will be available as a simple service in the near future, what was the decision which made it valuable enough to do this sort of engineering task as a start-up?

Netflix runs a similar environment, though they've also got the Simian Army regularly tearing around the joint.

If you're building a document signing service, and you are likely working between Europe and North America, you've got a perfectly acceptable update window, assuming you are able to roll an update out within a few minutes of downtime. You've spent X (hundreds?) of hours to save you those minutes, but will anybody notice?

Pushing a fix while on the phone with a customer may provide a wow factor, but would an update within a few hours not be equally wowable?

Possibly some insight into how many documents are signed per minute made this all necessary.

It's impressive, I'm admit, it just seems to fly in the face of all being lean as a start-up.</comment>