New Snowden Docs Indicate Scope of NSA Preparations for Cyber Battle - <AUTHOR> http://www.spiegel.de/international/world/new-snowden-docs-indicate-scope-of-nsa-preparations-for-cyber-battle-a-1013409.html <comment><USER3> Here's a story for you.

I'm not a party to any of this. I've done nothing wrong, I've never been suspected of doing anything wrong, and I don't know anyone who has done anything wrong. I don't even mean that in the sense of "I pissed off the wrong people but technically haven't been charged." I mean that I am a vanilla, average, 9-5 working man of no interest to anybody. My geographical location is an accident of my birth. Even still, I wasn't accidentally born in a high- conflict area, and my government is not at war. I'm a sysadmin at a legitimate ISP and my job is to keep the internet up and running smoothly.

This agency has stalked me in my personal life, undermined my ability to trust my friends attempting to connect with me on LinkedIn, and infected my family's computer. They did this because they wanted to bypass legal channels and spy on a customer who pays for services from my employer. Wait, no, they wanted _the ability_ to _potentially_ spy on _future_ customers. Actually, that is still not accurate - they wanted to spy on _everybody_ in case there was a _potentially bad person_ interacting with a customer.

After seeing their complete disregard for anybody else, their immense resources, and their extremely sophisticated exploits and backdoors - <USER2> they will stop at nothing, and knowing that I was personally targeted - I'll be damned if I can ever trust any electronic device I own ever again.

You all rationalize this by telling me that it "isn't surprising", and that I don't live in the [USA,UK] and therefore I have no rights.

I just have one question.

Are you people even human?

[1][https://firstlook.org/theintercept/2014/09/14/nsa- stellar/](https://firstlook.org/theintercept/2014/09/14/nsa-stellar/)

[2][https://firstlook.org/theintercept/2014/12/13/belgacom- hack-...](https://firstlook.org/theintercept/2014/12/13/belgacom-hack-gchq- inside-story/)

<comment><USER4> This agency gathered information on you, attempted to friend you on linkedin and infected your family's computer.

I don't think any of those actions warrant accusations of inhumanity. I can understand that it's disturbing to feel that you're being stalked, or even just monitored. But it doesn't sound like anyone took action to intervene in your private affairs or intentionally lead you to feel threatened. It's legal and at least arguably moral to hire a private investigator to follow a private citizen around, so long as they follow certain rules. Attempting to friend someone on linkedin under false pretenses is also legal and arguably moral. Infecting a computer with a virus is not legal as far as I know, but I think the real concern there is the nature of the software and not it's existence. As long as they did not harm, the morality of even infecting a computer is questionable.

These people are spies, their job is to spy. To them at least, it isn't a question of the rights granted to you by your citizenship status. It's a tradeoff between the emotional discomfort that you may feel and their job requirement to develop resources that will allow them to stop bad people from doing bad things. I think they made the right decision. If influencing an employee at an ISP can give them more insight into the capability and intent of people they suspect to be up to no good, and the main cost is the distress that said employee or employees will feel as a result of being spied upon, then do it. The needs of the many outweigh the needs of the few.

I understand that we disagree on that, and there is ample room for both sides of the disagreement to be right or one of them to be utterly wrong. But I don't think that either decision is inhumane in any way.

<comment><USER5> >Infecting a computer with a virus... As long as they did not harm, the morality of even infecting a computer is questionable.

This is bullshit. Even if a virus doesn't intentionally harm, it does anyway harm.

Connections will become very slow, because high priority is given to uploading stuff from one's computer to nsa. User will get inexpicable blue screens or exotic error messages that nobody can help to fix: asking for help in forums will just bring annoying answers of "experts" suggesting to upgrade Windows/Flash/HW/programs versions. Installation or execution of some programs will be prevented, making computer usage a pain.

Infecting a computer is legally and morally a crime.

</comment></comment></comment><comment><USER6> Looking at the comments in support of the NSA here makes me suspect an astroturfing campaign is happening.

Edit: I should add that my suspicion came from noticing that the vast majority of the comments when this was first posted seemed aligned in favor of the NSA's mission.

It wasn't the presence of pro-NSA comments that was interesting but rather that these opinions were the overwhelming majority. This is, of course, how astroturfing becomes effective, it is not the rhetoric that is important but the cognitive bias imparted by the facade of so many people falling to one side of an issue.

This is of course, only a suspicion, but it seemed worth noting.

<comment><USER7> There is absolutely no reason for the NSA to give a flying fuck about what you or I think.

There just isn't, and it'd be a waste of resources to even attempt something like this.

People do exist (I am one of them) who think the NSA is doing a nasty job that isn't very appealing, but is absolutely necessary for me to be able to sleep in my bed at night safely. They may not have the best guidance from the government, but there are people who do believe they're doing the best they can.

Why must there be a conspiratorial astroturfing campaign taking place? Why can't there just be people who actually agree with some/most of what the NSA has done, based on the laws that govern it?

<comment><USER8> > Why must there be a conspiratorial astroturfing campaign taking place?

Regardless of how you feel about it, there _is_ a conspiratorial astroturfing campaign taking place - it's a well documented NSA activity. Whether it accounts for specific comments is impossible to say, but it does exist. Why then is it so absurd to think that it may be in play?

<comment><USER7> Why is it absurd? Because there's no evidence of it taking place. It's the same reason we don't believe in unicorns.

<comment><USER9> [https://firstlook.org/theintercept/2014/02/24/jtrig- manipula...](https://firstlook.org/theintercept/2014/02/24/jtrig- manipulation/)

EDIT: Added firstlook url per reply, originally pointed to [https://www.techdirt.com/articles/20140224/17054826340/new-s...](https://www.techdirt.com/articles/20140224/17054826340/new- snowden-doc-reveals-how-gchqnsa-use-internet-to-manipulate-deceive-destroy- reputations.shtml)

...not really sure I see a substantial difference between the 2 articles, but there you go.

<comment><USER7> Why link to the blogspam and not the firstlook.org article it's sourced from?

Oh right, because it doesn't support what you're saying. Got it.

<comment><USER10> Are you saying that GCHQ don't do it?

Or that GCHQ does do it, but because they're GCHQ and not NSA that it's irrelevant?

Because now you have evidence that GCHQ does it you should allow the possibility that NSA does it. The reason you don't have evidence that NSA does it might be because NSA is a secret organistion.

<comment><USER7> So Snowden decided to leak the GCHQ presentation, but chose not to show any of the evidence he had that the NSA does it?

<comment><USER10> Snowden did not gather everything. Maybe he just missed it? Or maybe GCHQ does it but NSA doesn't? We know that other bits of the US government have different levels of online presence so I'm not sure why you're so hostile to the idea that NSA has people that disrupt online conversation about NSA.

<comment><USER7> I for one try to only accuse folks of doing things I actually have evidence of them doing, and I know this sounds crazy, but there's currently no evidence the NSA is on HN astroturfing comments, so maybe we shouldn't pretend like we know things we don't.

<comment><USER11> _I for one try to only accuse folks of doing things I actually have evidence of them doing_

That's not true.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> There was a long interview with Snowden posted recently, which didn't make it to the frontpage. I guess because of Snowden penalty on HN and Snowden fatigue. Anyway, he kept repeating a point which is quite easy to understand for the public I think.

[https://news.ycombinator.com/item?id=8859606](https://news.ycombinator.com/item?id=8859606)

 _And the reality is when it comes to cyber conflicts [...], we have more to lose.

We spend more on research and development than these other countries, so we shouldn't be making the internet a more hostile, a more aggressive territory. We should be cooling down the tensions, making it a more trusted environment, making it a more secure environment, making it a more reliable environment, because that's the foundation of our economy and our future.

[...]

The concept there is that there's not much value to us attacking Chinese systems. We might take a few computers offline. We might take a factory offline. We might steal secrets from a university research programs, and even something high-tech. But how much more does the United States spend on research and development than China does? Defending ourselves from internet- based attacks, internet-originated attacks, is much, much more important than our ability to launch attacks against similar targets in foreign countries [...].

[...]

When you look at the problem of the U.S. prioritizing offense over defense, imagine you have two bank vaults, the United States bank vault and the Bank of China. But the U.S. bank vault is completely full. It goes all the way up to the sky. And the Chinese bank vault or the Russian bank vault of the African bank vault or whoever the adversary of the day is, theirs is only half full or a quarter full or a tenth full.

But the U.S. wants to get into their bank vault. So what they do is they build backdoors into every bank vault in the world. But the problem is their vault, the U.S. bank vault, has the same backdoor. So while we're sneaking over to China and taking things out of their vault, they're also sneaking over to the United States and taking things out of our vault. And the problem is, because our vault is full, we have so much more to lose. So in relative terms, we gain much less from breaking into the vaults of others than we do from having others break into our vaults._

<comment><USER13> There's no reason to pick Offense vs. Defense any more than there is Social Security vs. Education. It's just a question of budgeting. Why wouldn't we invest in both?

<comment><USER14> In this situation, you can't pick both. The offensive measures we are discussing exist by breaking the defensive measures. The problem is breaking the defensive measures breaks not only _their_ defensive measures, but _ours_ as well. Hence Snowden's analogy to making a secret entrance to _all_ bank vaults, even ours.

In this case, the idea is that the NSA will keep knowledge the secret door to the bank vault to themselves, and we'll all be secure. The problem with this is that other countries can inspect the bank vault and discover the secret door for themselves, or steal said knowledge from the NSA via espionage. Now our bank vault is just as exposed as theirs.

</comment></comment></comment><comment><USER15> To be fair here, the NSA should very well be doing these things, for the purpose of attacking other states. The reason is very clear as the Russian attacks on Estonia ( [https://en.wikipedia.org/wiki/2007_cyberattacks_on_Estonia](https://en.wikipedia.org/wiki/2007_cyberattacks_on_Estonia) ) demonstrate a clear need for defensive capability in this area, and where you have defence you end up needing offence.

This persistent confusion between legitimate NSA operations such as preparing to intercept communications of foreign governments and illegitimate such as mass slurping of everyone's email merely serves to discredit the entire privacy defending position, and in the long run will just play into the hands of those that want to read everyone's email for nefarious purposes.

<comment><USER16> Agreed. I think the Snowden revelations have harmed the NSA's reputation to the point where most people on HN just assume NSA is a bond-level villain. The NSA serves a purpose, they are meant to be prepared offensively and defensively for the USA's security, which includes the internet. I have no problems with the NSA's purpose and would prefer they continue to exist and prepare against threats to the country.

I do believe the NSA overstepped their mandate in domestic surveillance, and should be held accountable, but let's not forget that this world does have bad guy's - plenty of nations would lose no sleep at night if they hurt America.

<comment><USER17> precisely, and this is why I'm starting to think that snowden has - at least - caused as much harm as good by his revelations. if he had stopped at revealing the domestic surveillance he would be an undisputed hero/patriot. but with all of these unnecessary leaks that does nothing but harm to national interest I can understand how others might view him otherwise.

<comment><USER18> Others have said as much. Here is a quote from William Binney, another NSA whistleblower, on advice that he has for Snowden:

> I would tell him to steer away from anything that isn't a public service — > like talking about the ability of the U.S. government to hack into other > countries or other people is not a public service. So that's kind of > compromising capabilities and sources and methods, basically. That's getting > away from the public service that he did initially. And those would be the > acts that people would charge him with as clearly treason.

The whole interview is a great read: [http://www.usatoday.com/story/news/politics/2013/06/16/snowd...](http://www.usatoday.com/story/news/politics/2013/06/16/snowden- whistleblower-nsa-officials-roundtable/2428809/)

<comment><USER19> The US public has a legitimate interest in knowing about NSA's hacking abilities, because vulnerabilities used by the NSA abroad can just as easily be used domestically by foreign or criminal attackers.

<comment><USER18> Against a well-prepared adversary, a vulnerability can only be exploited once -- a zero day. After disclosure, or evidence of an exploit in circulation, such vulnerabilities become useless towards the agency's mandate, whether or not you agree with that mandate. Realpolitik? Definitely.

</comment></comment></comment></comment></comment></comment><comment><USER20> Everything at the lowest levels needs to be tightened up now.

Buffer overflows in trusted code have to go. This means getting rid of the languages with buffer overflow problems. Mostly C and C++. Fortunately we have Go and Rust, plus all the semi-interpreted languages, now, and can do it.

We need something that runs Docker-like containers and, all the way down the bare metal, has no unsafe code. We need dumber server boards, with BIOS and NIC code that's simpler and well-understood. The big cloud companies, Amazon, Facebook, and Google are already doing their own server boards.

Companies which put in "backdoors" should face felony criminal prosecution. That doesn't happen by accident.

Latest CERT advisory: "Vulnerability Note VU#936356 Ceragon FiberAir IP-10 Microwave Bridge contains a hard-coded root password ... Ceragon FiberAir IP-10 Microwave Bridges contain an _undocumented default root password_. The root account can be accessed through ssh, telnet, command line interface, or via HTTP. ... CERT/CC has attempted to contact the vendor prior to publication without success."

All Ceragon customers should demand their money back, and their products should be seized at US customs as supporting terrorism.

<comment><USER21> One problem with most "interpreted languages" you cite as preferable is that they rely on C/C++ run-times for now. This means these languages are only as safe as the underlying C/C++ run-times and the core libraries they rely upon (glibc and the like.)

We need to start to think of replacing underlying runtimes and core libraries with Rust and Go-based alternatives (or similar) to make them safer. Ultra- large goal and probably impossible in the near term, but it should be done.

<comment><USER20> It may not be that big a task. How much code do you really need to run a container instance? If you could get an airtight Xen-like system, you might not need much of an OS inside each container. Xen already does memory allocation, CPU dispatching, I/O handling, timer handling, and message passing, which is all an OS really needs.

Rust programs running on "libnative" do not, I think, use "libc" any more.

</comment></comment></comment><comment><AUTHOR> There are a number of objectionable elements to the NSA foreign operations.

\- Mass surveillance of all humans is objectionable on human rights terms.

\- Attacks on civilian infrastructure. The NSA is executing military operations against civilian infrastructure even in NATO countries.

It isn't conventional foreign policy or warfare for a military agency to be actively and continuously attack the civilian cultural and economic infrastructure in preparation for war.

<comment><USER22> Attacking the economic infrastructure in preparation of War has been the US' m.o. since at least the 50's. Economic Hitmen, and what-not.

<comment><USER23> Of _all_ countries? I doubt that. Many people don't seem to realize the _scope_ is the main problem here, and they are equating spying on _one person_ in targetted surveillance, with spying on 4 billion people and saying "NSA has already been doing that!". No they haven't. So stop confusing the message.

Another big problem is that NSA has started undermining crypto. Before 2001 you could maybe actually trust NSA that they're releasing crypto that is safe because you thought they actually have an interest in "security". Not anymore. Anything and everything NSA has touched now has to be considered corrupt.

</comment></comment></comment><comment><USER24> Judging by the scope of the attack on Belgacom in 2011 that battle is already underway, the surprise I guess should be that it is the allies attacking each other. If China or North Korea would have made an attack like this it would be trumpeted as an act of war, but because it is the UK with NSA assistance it's downplayed as much as possible.

</comment><comment><USER25> The premise that we need "beneolvient power" to "protect us" from "evil doers" is the oldest trick in the book. If there is no threat, one will be generated. Almost organically, it does not even take overt orginization. The players know cui bono.

<comment><USER26> _She has abstained from interference in the concerns of others, even when the conflict has been for principles to which she clings, as to the last vital drop that visits the heart. She has seen that probably for centuries to come, all the contests of that Aceldama, the European World, will be contests between inveterate power, and emerging right.

Wherever the standard of freedom and independence has been or shall be unfurled, there will her heart, her benedictions and her prayers be. But she goes not abroad in search of monsters to destroy. She is the well-wisher to the freedom and independence of all. She is the champion and vindicator only of her own. She will recommend the general cause, by the countenance of her voice, and the benignant sympathy of her example._ \- John Quincy Adams

It was until recently (the early 1900s) that the policy of the United States to stay entirely out of any foreign affairs other than her own. It was acknowledged that America could spread her virtues and values by being the paragon example of them as inspiration to others to follow suit.

Today she holds the opposite view - that it is her unquestioned duty and obligation to interfere everywhere with the justification and rationalization that she seeks justice - and that discarding some of these principles for her own people is necessary for this noble mission.

<comment><USER27> It's kinda odd that our politicians in the past used to be, for lack of better words, poets. I have often wondered about this and wondered what has happened.

<comment><USER28> This is sort of off topic, but it is amusing.

I would guess that it is a combination of:

\- A deemphasis of poetry and literary studies in the concept of being educated and cultured

\- The rise of writing staff and PR professionals in the practice of engaging with the public

\- The relative lack of importance writing has today compared to newer picture and video delivery (media is message, etc)

\- Inherited nostalgia for forms associated with 'classic' art styles

</comment></comment></comment></comment><comment><USER29> Speaking as a citizen, the problem with the US's newest brand of digital weapons, is that they can be used on US population under the radar and w/o killing anyone. This may justify legally their extended use for surveillance unfortunately. The development of the atom bomb and chemical weapons had no "convenient" use on the USs own citizens, and they clearly couldn't get away with it. However, these weapons do, and they are being developed with all the same force, purpose, and financial backing as the a-bomb and chemical weapons were ~100 years ago.

<comment><USER30> Not only could use. These weapons are built using either weaknesses introduced into civilian domestic systems, or found in domestic civilian systems and not fixed. This means that the basic premise for this weapons created to protect US citizens is to weaken their security.

</comment></comment><comment><USER31> I won't pretend to know what's going on here or its implications. So far humanity has lucked out considering our capacity for building some pretty nasty weaponry. We'll probably go through a series of cyberwars before we come to our senses.

I don't blame the NSA for trying to be ready to fight a cyberwar. Other nations probably wouldn't stop their programs even if the US did. Our culture isn't the only one infected with a sense of Manifest Destiny.

Where we might consider drawing the line begins with necessity. Deciding which actions are necessary and which are gratuitous might prove difficult, assuming we even know, which is why I find it hard to fault Snowden for leaking this information.

As fearsome as the NSA sounds, certainly they have some limits. For instance, why don't they just clean out everyone's bank accounts? Might pay their bills for a few days anyway. But why haven't they gone after certain criminals? Many shady operations keep their money in jurisdictions that probably can't compete with our cyberwar capabilities. Maybe these operations enjoy the protection of a powerful entity but probably not all of them do. And probably many operations still use cash and couriers but the US and others seem to have gotten better at tracking movements of people so it's doubtful such tactics will remain viable forever.

Maybe in the end we have to somehow conquer the notion of distrust. Not sure how it can be done except through telepathy and even then the transition to a telepathic society will probably be full of misery.

</comment><comment><USER32> How is this whistleblowing? What benefit do we as the public gain from this knowledge?

<comment><USER33> I would ask the other way round. Why should this knowledge be hidden from the public?

<comment><USER34> The obvious answer is that it becomes much harder for the NSA (and related entities) to do their job when their tactics and capabilities are known to the world. And yes, this _is_ their job, and there's nothing wrong with it. Domestic spying is something that needs to be reigned in and tightly controlled, but to think that everyone should know everything is just naive and shows little understanding of how the real world actually operates.

<comment><USER33> _The obvious answer is that it becomes much harder for the NSA (and related entities) to do their job when their tactics and capabilities are known to the world._

Harder as in how?

That handwavy excuse gets thrown around a lot, I have yet to hear someone back it up with anything tangible.

 _Domestic spying is something that needs to be reigned in and tightly controlled_

Well, we tried with blind faith for a while. Turns out that didn't work all too well.

So, how do you propose we control something that we are not allowed to know anything about?

<comment><USER34> "Well, we tried with blind faith for a while. Turns out that didn't work all too well.

So, how do you propose we control something that we are not allowed to know anything about?"

Agreed, with the first sentence at least. Let's be real though; no incarnation of oversight is going to have you and I involved directly, nor should it. You cannot expect the populous to be sufficiently educated on the intricacies of foreign policy and global threats.

So my proposal would be for the lawmakers to propose a better form of oversight which includes strict provisions for surveillance, especially the domestic variety. These provisions also cannot hamstring the NSA or the like; there has to be balance.

The parent comes off as a proponent of opening up the floodgates. That's ridiculous and it will never happen (nor should it). You cannot have a system which A) allows for full disclosure, and B) does not weaken our ability to defend ourselves.

<comment><USER35> "You cannot expect the populous to be sufficiently educated on the intricacies of foreign policy and global threats. So my proposal would be for the lawmakers to propose a better form of oversight which includes strict provisions for surveillance, especially the domestic variety. These provisions also cannot hamstring the NSA or the like; there has to be balance."

but then, if you and I (the "populous") aren't allowed to know what they are trying to achieve, or how they are doing it, how on earth are we supposed to have any faith in the oversight that "lawmakers" provide?

I cannot overstate how strongly I abhor the "you cannot expect the populous..." type arguments when used to continue keeping them in the dark about matters for which THEY HAVE ULTIMATE RESPONSIBIILITY.

if "the populous" is expected to select lawmakers based on the quality of the job they do, then "the populous" sure as HELL needs to have the information on the intricacies of foreign policy and global threats made available to them, and the response that their lawmakers support.

</comment></comment></comment></comment></comment></comment><comment><USER36> The NSA should just leak documents showing all of the other governments doing this stuff so we can move on from the Anti US/UK circle jerk. Seriously if you think your government hasn't invested serious time and money in digital subterfuge you are living in a dream world and need to wake up.

<comment><USER37> Classic straw man. Many of the documents leaked have shown that our allies are doing the same, usually with the NSA's help. Of non U.S. allies most countries are too small to have serious programs, or are plainly doing much worse. I suppose this is some sort of perverse defense of the NSA, but the fact is that luckily most people would like to hold their government to a higher standard, instead of saying "the other guys are worse". If more people had had your attitude a few hundred years ago, we wouldn't even have democracy, as there would have been plenty more brutal and totalitarian regimes to point to.

</comment></comment><comment><USER38> What I find really frightening is where they write:

 _From a military perspective, surveillance of the Internet is merely "Phase 0" in the US digital war strategy._

[...]

 _This enables them to "control/destroy critical systems & networks at will through pre-positioned accesses (laid in Phase 0)." Critical infrastructure is considered by the agency to be anything that is important in keeping a society running: energy, communications and transportation. The internal documents state that the ultimate goal is "real time controlled escalation"._

This isn't about fighting terrorism. It's also not about the usual warfare it's more like the infrastructure or a set of tools to control nearly every other country or the planet or at least make sure that the US will always be able to keep them from disagreeing.

</comment><comment><USER39> I can understand the support of Snowden for blowing the whistle on domestic spying, but how is this defensible?

<comment><USER40> It does not. This stuff gives argument that Snowden went over to FSB long time ago and that the whole thing is a Russian op a lot more weight.

<comment><USER41> No, it doesn't. Snowden isn't the one who's been releasing these documents; the journalists are. For your claim to make any sense, you'd have to believe Der Spiegel is secretly working for the Russians.

<comment><USER40> They can disclose it because they were given it. Do you think he did not know this will get published sooner or later?

Also depth of Russian penetration of security services and media in Europe.. in many places it is quite complete. It would not actually be surprising at all if Spiegel had someone working for FSB/GRU.

Having said that it is not 100% certainty. If he was not turned before the disclosures then US whistleblower witchhunt that drove him into arms of FSB is even worse because he really had no choice once US started hunting him.

</comment></comment></comment></comment><comment><USER42> > the only law that applies is the survival of the fittest.

Is this such a problem? In a world where exploits are used to break infrastructure, isn't the best solution simply to build increasingly more secure code? If that won't solve the problem I don't know if legislation will. Right now a determined hacker can harm a company via the internet (e.g. Sony). Are laws really going to stop that from happening?

If not, please correct me. I know little about cyber warfare and would love to know more.

<comment><USER43> Legislature can't prevent bad people from doing bad things. But it can prevent good people from doing bad or ignorant things, and it provides a framework to punish those who cause harm.

Every law can and does get broken, but that doesn't mean laws are useless.

<comment><USER44> There are other ways to have an impact with laws than punish the aggressor. Airplanes aren't safe because the pilot or mechanic gets punished when something happens, it's because (very simplified) the airlines and manufacturers do.

</comment></comment></comment><comment><USER45> The last 2 articles I read were this one and the earlier piece on Google and neural nets [1]. It's easy to connect the two, add the pervasive integration of technology in our lives, mix in a healthy dose of paranoia --> see a SkyNet future.

[1][https://medium.com/backchannel/google-search-will-be-your- ne...](https://medium.com/backchannel/google-search-will-be-your-next- brain-5207c26e4523)

</comment><comment><USER46> I'm pretty late to the party here but there are some fascinating parallels between the USG's actions in the physical world, as in the digital one.

First to USG has made a concerted and successful attempts to place secret digital strongholds and black sites across the globe, including some 'behind the enemy lines' so to speak. In the physical world these are the equivalent to CIA black sites and safe houses, from which you can attack and spy on the enemy, feed in extra weaponry to partisans and rebels (similar to the CIA Benghazi compound, sorry 'consulate', sorry 'embassy'.

The NSA has all these smart dangerous and arguably immoral minds employed to defend the digital borders of the US. But in truth these minds are busier establishing secret pathways through the digital trenchlines in order to have a definive and effective advantage when the cyberwar comes (which of course they are actively encouraging to validate their position, historical actions and future funding).

At the same time they are making a concerted effort to make sure that the security protocols everyone uses are undermined and backdoored. In effect they are making sure that the digital nuclear weapons held by their enemies aren't going to get in the air when the time comes.

Through strong encryption we could make sure that we have the digital equivalent of mutually assured security, but as ever the US isn't interested in this, because the reality is that the military industrial compound aims to make billions of dollars from the industry.

In a world where all communications and hardware devices were secure, they wouldn't make any money. A secure, stable and safe world just isn't profitable.

</comment><comment><USER47> At best this seems like an arms race but at worst, there are actually battles being fought (of a kind). I wonder what the digital equivalent of a nuke would be such that govts decide that diplomacy is better. Some kind of Digitally Affected Mutual Destruction (DAMD).

</comment><comment><USER48> Domestic surveillance was controversial and surprising. Is a spy agency preparing ways to attack and cripple foreign infrastructure that unexpected or contentious?

<comment><USER49> > Is a spy agency preparing ways to attack and cripple foreign infrastructure > that unexpected or contentious?

In light of current revelations, it may or may not be all that surprising. Surprising or not though, it is most definitely contentious!

</comment></comment><comment><USER50> How should NSA spying figure into the public's view of Obama's efforts to help municipalities build out competitively priced broadband networks?

<comment><USER51> Add some legal pot and it will all be forgiven and forgotten.

</comment></comment><comment><USER52> One thing to be said about the offence vs defence side of this story is that the adage the best defence is a good offence definitively applies here. It's so much easier to attack another system than defend your own. It's inherent to the way systems are set up:

A computer system has many services, programs and tasks running on it. Only one of these needs to contain a flaw for a system to be vulnerable, obviously this means that on a secure system everything must be perfect, for 1 flaw compromises everything.

Therefore I don't find it strange that the NSA allocates the resources it does to research and expand it's offensive capabilities, since trying to defend the systems of the US is probably a lost cause. The question remains if this is ethical and/or legitimate. Being a non us-citizen I'm certainly opposed to the practises of weakening standards and harvesting/exploiting services on the internet.

I also find the double-speak of the US government deplorable, on the one hand we have the government declaring that in many ways a cyber-attack will, and can, be reacted upon as if it were a conventional attack. And on the other hand we have the US government attacking and targeting civilians (Belgacom Sysadmins). I fail to see how attacks against the US can be labelled as a conventional attack, but that attacks from the US against civilians are apparently OK.

</comment><comment><USER53> seems to me a lot of people flaged the story (443 points, 1 day ago, on page 3 with a couple of stories with 200 points also 1 day ago) ... wondering why _conspiracy theories_ ;)

</comment><comment><USER54> There is also this very American reassuring belief that quantity will somehow take you to quality.

</comment><comment><USER55> For me the only bunch of people who wants and loves to start war is USA government, note I am not saying americans, I am saying US government. Why should you prepare if you dont have intention of war? "defense", from whom? from people where you started war? directly with Afghanistan, Iraq and some others, indirectly with Syria, even with Russia (started economically) and many others where they got governments with bribes. Now preparing for D war. Every war started with flag of "Demo hypocracy", defense and some other pseudo defensive words, reality is USA starts war and government fuckingly loves when people die. Probably you are going to downvote, thats because you didnt lost any of your brother, sister or even relatives and friends in such wars which just started because they wanted more oil and more money.</comment>


Saturday Night Fever: Layer 7 attacks against CloudFlare sites - <AUTHOR> http://blog.cloudflare.com/saturday-night-fever-layer-7-attacks-against

<comment><USER3> _The trend across the year shows some intriguing, and dramatic, dips in layer 7 DoS activity. The dips in the chart are around the following dates: January 30, February 21 (Mardi Gras), March 20 (attackers recovering from St. Patrick's Day?), April 22 (did attackers take Earth Day off, or did people switch off their home machines making botnets smaller for a day?), May 29 (Memorial Day weekend), June 28 (just before July 4)._

We should tell people to turn their PCs off when they're not using them - <USER2> to save power, but to reduce the total number of bots in the botspace available to attackers.

But then what to do about iZombies and botdroids?

<comment><AUTHOR> I don't, unfortunately, have statistics on the type of devices involved in these attacks because we currently don't keep them. I'll ask if we can start keeping track.

</comment></comment><comment><USER4> Interestingly, visiting cloudflare from Germany get's me redirected to de.cloudflare.com, which, being down (according to the site itself) seems quite funny to me.

Wanted to take a look around, encountered site down and tested with chrome (adblock plus and ghostery activated) and firefox 15 (clean).

hope nothing serious. I really love reading these accounts, as they show, what can be done to protect sites from malicious requests.

<comment><AUTHOR> Yes, sorry about that. Kernel panic occurred on the www server and it's being brought back up now.

<comment><USER5> Yikes. Panics are never good; usually indicators of buggy hw, buggy drivers or overheating. Hope you have a cold spare handy :o

<comment><USER6> I've had these only a few times over the years mainly back in the day for some of the reasons you mentioned. At this point I can't even remember the last time I encountered one.

For reference, for anyone interested (based on your comment), I just pulled this up relative to Mac OSX:

<http://thexlab.com/faqs/kernelpanics.html>

</comment></comment></comment></comment><comment><USER7> So, layer 4 in terms of the model people actually implement?

<http://tools.ietf.org/html/rfc1122>

Or does Cloudflare run X.400 email servers or something? Maybe a nice X.500 directory server?

<comment><USER8> Very few protocols fully or properly implement the entire 7 layer OSI stack. Most times you see layers 3-5 lumped together. The way that they are looking at it is that the application (a browser, something else?) is being used to generate HTTP requests. So while it's technical accurate to say that because a browser or other application is acting as an HTTP client, the attack itself it not at layer 7 because they are receiving the attachs on layers 3 and 4 on their side.

Where the article says "But layer 7 attacks, where the attacker actually connects to our hardware using TCP and makes apparently valid HTTP requests are another matter"

Those would be layers 3 and 4.

Mis-communication and outright wrong communication about layer 7 in networking has been rampant for years.

<comment><USER9> Someone stole my copy of Comer, so I'll have to go from memory, but HTTP would best correspond to layers 5-7 IIRC.

On the other hand it's stupid to use OSI layers when talking about the internet since the internet has its own, well defined, terminology for layers. In that case HTTP is clearly at the Application layer.</comment></comment></comment>


Apple acknowledges Mac Defender malware, promises update - <AUTHOR> http://arstechnica.com/apple/news/2011/05/apple-acknowledges-mac-defender-malware-promises-software-update.ars

<comment><USER2> Malware is so rare on the Mac that Apple set up an entire web page to deal with the _one_ trojan currently known to be impacting Mac users.

Macs aren't more secure than Windows machines, but they're still undeniably safer (at least for normal people).

<comment><USER3> "Malware is so rare on the Mac that Apple set up an entire web page to deal with the one trojan currently known to be impacting Mac users."

I read that and I thought Apple had set up a top-level page. I then saw it is an article in Apple's knowledge base.

As far as I know, this is the same thing Microsoft does for distinguished malware: articles in the knowlegde base. E.g., here is an old one for Blaster:

<http://support.microsoft.com/kb/826955>

<comment><USER2> Blaster wasn't a trojan; it was a worm that actively compromised Windows machines, took out entire huge enterprises (including major military networks), hit the front page of CNN, and (IIRC) prompted a Congressional inquiry about regulating software security.

Not exactly apples/apples.

</comment></comment></comment><comment><USER4> <http://support.apple.com/kb/HT4650>

</comment><comment><USER5> As an IT professional, I'm honestly not sure how to feel about this.

On the one hand, it's great that all the variants of MacDefender currently out there will be neutralized.

On the other hand, we're surely going to see new flavors that go undetected by the update. Apple is getting into the anti-virus game, and potentially starting an arms race. But, what else could Apple do in this situation?

<comment><USER6> I think the key here is that this is a trojan installed by the user. Education is the greatest weapon against this type of attack -- not trying to keep up with users reinfecting themselves with each variant.

</comment></comment><comment><USER7> Wow, check out the removal steps in the kb article. Quit the app using Activity Monitor then delete from /Applications. Clearly we have a long way to go...

</comment><comment><USER8> An interesting question is whether this is merely a blip or part of a trend. Apple's user base must be an enormously tempting target for malware creators.

<comment><USER9> To answer this question, we'd need to know whether various things will change in the future:

 _Will criminals stop wanting to scam people?_

 _Will non-technical users somehow get more savvy?_

 _Will the Apple userbase get smaller and thus less lucrative?_

</comment></comment><comment><USER10> It is inevitable that as mac becomes more popular, we'll see more viruses and malware.

</comment><comment><USER11> heh, given that Apple was previously directing staffers not to acknowledge it, when I read this headline all I could think of was Jobs telling a press conference "OSX now has malware. This is a new, must-have feature that will revolutionise the industry!"...

<comment><USER12> Apple is pretty damn careful about not making press releases until they have a clear, concise and informative statement to make. Staffers were likely instructed not to address it because the official response was forthcoming. Yes, it's a slow response, but not worth attributing to malice.

<comment><USER13> As Gruber says: Measure Twice, Cut Once is the Apple approach</comment></comment></comment>


Autism epidemic? - <AUTHOR> http://arstechnica.com/science/news/2011/05/autism-epidemic-more-likely-were-just-better-at-diagnosis.ars <comment><USER2> Hint: the definition of "autism" has broadened considerably (especially if you take into account the legal definition in the US, which is even broader than the medical one), and there is a great deal of financial incentive to diagnose a child -- any child -- with autism.

<http://politicalilliteracy.us/node/51>

</comment><comment><USER3> The best take on it I have seen so far, [http://pipeline.corante.com/archives/2011/05/04/what_autism_...](http://pipeline.corante.com/archives/2011/05/04/what_autism_epidemic.php)</comment>


Wherever oil goes, the stock market goes - <AUTHOR> http://www.bloomberg.com/news/articles/2016-02-11/oil-is-the-cheap-date-from-hell <comment><USER2> Can anyone explain how exactly cheap oil makes the stock market crash? Gas companies do not make up most of the stock market. Cheap energy means cheap everything else. Sense when does cheap energy cause everything else to crash?

<comment><USER3> Alan S Blinder recently pointed out that cheap oil is mostly good for the economy (except in North Dakota and Iran). He also pointed out that markets can stay irrational for a long time.

I don't think oil is the cause of the market drop: stocks had been overbought for a while (if you look at historic P/E ratios), so it is natural for it to return to normal.

Also, the increasing interest rate is going to put downward pressure on the stock market, even if the economy does well. Right now that is what I consider to be the main cause of the drop. What I mean is:

In the market, there are day-traders, and long-term traders. The day traders are mostly responsible for the day-to-day gyrations of the market, and they've been looking at the price of oil as something to follow. Underneath that, though, the long-term traders have been removing money from the market (leaving it in cash, or switching to bonds), so even after the day traders get done, there is still a net negative.

Of course, that's a simplification for explanation purposes because I don't want to write a book here.

<comment><USER4> > ... (except North Dakota and Iran) ...

And Norway, and Russia, and probably others.

The economy in Norway looks pretty bad right now. And Russia is having very bad times right now I believe.

At least in Norway it's because the oil price is way to low for them -- I heard that they break-even at $60 or $70 per barrel. That's pretty bad if practically the whole economy is based on oil.

<comment><USER5> Not whole economy is based on oil. It makes 30% of revenue and 50% of exports.

[https://www.regjeringen.no/no/aktuelt/the-norwegian- experien...](https://www.regjeringen.no/no/aktuelt/the-norwegian-experience- in-the-oil-and-/id726975/)

<comment><USER6> 50% of exports is pretty bad in a world where you need/want things like iPhones and don't make them yourself. We're all very connected at this point.

</comment></comment></comment></comment></comment><comment><USER7> This article is entirely wrong.

Wherever the dollar goes, stocks and oil go (in the opposite direction). They're both priced in dollars fundamentally.

The Fed ending QE mostly put a stop to the stock market moving higher, dramatically strengthened the dollar, and decimated the commodities market, devaluing the price of nearly all commodities including oil.

If the dollar goes up a lot, it becomes extremely difficult for the stock market to do anything but fall. Earnings growth has to outrun the gain in the dollar. During especially fast climbs in the dollar that becomes essentially impossible. To make things even worse, the S&P 500 companies are highly dependent on exports, they get to show off artificially high earnings courtesy of a too-cheap dollar; when that condition turns, the artificially high earnings go with it, which is exactly what we're witnessing now. Then doubling down on that effect, the stronger dollar is pulling trillions in capital back out of emerging markets, which has thrown most of them into recessions or worse, leaving the export heavy companies in an even worse bind.

</comment><comment><USER8> Cheap gas + cheap stocks. Sounds like this is a good thing unless you're either a) a full-time investor, or b) retired or planning to retire very soon.

If you're working a job and putting money into an index fund for retirement, then news like this should make you smile and maybe put a little extra into your IRA/401k/etc. next paycheck.

<comment><USER9> Ideally, if you're getting ready to retire, most of you assets should not be in the volatile stock market, so it should not affect you terribly.

<comment><USER10> If you intend to live for 30 years in retirement, shouldn't you still have a good chunk of your assets in stocks?

<comment><USER11> Many people use much of their retirement savings at the moment of retirement to buy a special retirement-income product. This can have tax benefits over normal investments. So you try to maximize the probability of having a large fund at that moment.

</comment></comment></comment></comment><comment><USER12> A regrettable consequence of cheap oil is that clean energy technologies become harder to deploy because they're competing against a lower cost alternative. The best thing EVs and solar had going for them was $100+ dollar per barrel oil in prior years.

</comment><comment><USER13> I'm a complete layman in this field - but looking at the oil price charts it seems that its never been lower. Even from production/export cost from various countries - it doesn't seem like this price can go on for prolonged periods. I.e. it seems to me that it has to go up...

Any experts here - would this be a sound assessment?

Also (again as a layman) - what would be a good way to bet on this? I.e. can I buy crude oil options virtually, hold them for 1-3 years and sell them as easily as I could stock online? What would be the best way to do this + any lowdowns (except that my bet might be wrong)?

<comment><USER14> Many people believe that the prices will stay the same for at least the next 2 to 3 years. Some of those people think they will go down to about $20 for a while.

</comment></comment><comment><USER15> Oil and stock market correlate? Nope ...

 _" It is popular to correlate changes in major factor prices, such as oil, and the performance of major stock market indexes ... Andrea Pescatori, an economist at the International Monetary Fund (IMF), attempted to test this theory ... his variables only occasionally moved in the same direction at the same time, but even then, the relationship was weak. His sample revealed that no correlation exists with a confidence level of 95%."_

[http://www.investopedia.com/ask/answers/030415/how-does- pric...](http://www.investopedia.com/ask/answers/030415/how-does-price-oil- affect-stock-market.asp)

</comment><comment><USER16> This is simply a badly written article. In 2008 oil was very high, the market was not. Now in 2016, oil is very low, and the market is heading down. There's little correlation, and the market conditions today could be argued to be a result of bad regulation and poor government interference in the private sector (conservative view anyway), and now people want even more government influence when it's in $19 trillion in debt and the Federal Reserve mark to market is insolvent.

<comment><USER17> > In 2008 oil was very high, the market was not.

In 2008 oil and the market was very high and they both crashed toward the end of they year and into 2009.

[https://en.wikipedia.org/wiki/World_oil_market_chronology_fr...](https://en.wikipedia.org/wiki/World_oil_market_chronology_from_2003)

</comment></comment><comment><USER18> There is pressure on both sides of the equation. The supply is high because of shale oil and Iran. On the other hand the growth projections for economies, demand side, has not materialized. So yes. Oil price is low AND the economy is tanking. The simple narrative of wherever oil goes etc does not really work any ways, mostly because oil industry and associated industries is a big part of the economy.

<comment><USER19> Not to mention the fact that its production is, at various times:

a) The result of cartel manipulation (OPEC)

b) A tool of economic warfare (ISIS), and punishment (Russia)

Unfortunately both of those are happing at the same time as well.</comment></comment>


Chess master Lev Alburt teaches strategy, patience, and prognostication - <AUTHOR> https://www.bloomberg.com/news/features/2017-09-21/wall-street-s-best-kept-secret-is-russian-chess-master-lev-alburt <comment><USER2> Is a chess coach like Lev Alburt worth the money to someone who does play chess at a professional level? Does a world class coach make a big difference when we're talking about amateur chess players?

Obviously, there are other reasons to go to a guy like Lev for lessons (the conversation, social cache etc.), I'm just curious as an amateur chess player if seeking someone out like Lev would make a notable difference in my game as compared to a more average chess coach.

<comment><USER3> Assuming your goal is to be a better chess player (reaching expert/master level or up), the answer is no. Grandmasters typically charge exorbitant price (at least $50/hour, I recall that Lev can charge $300/hour), but the main problem is that unless they have so much experience teaching chess, they won't understand why amateurs struggle at certain positions (since they get such idea effortlessly). The rule of thumb for choosing a coach is picking someone about 400 points higher than you (either USCF or FIDE works), who has gone through similar struggles and practices to get to that level. Honestly, getting GM coach only worths the money when you are 2200 and up.

Disclaimer: never have a personal coach before. Did play for a while in a chess club where the team leader is about 2200 FIDE.

<comment><USER4> Sorry slightly off topic, how does one get their chess level measured? Thanks

<comment><USER5> You get an initial rating by playing a number of tournament games against rated opponents.

The results of every rated game (played at tournaments etc) are reported to a rating body. Back when I played (low level) competitively, rating lists were published every couple of months in paper form; nowadays it's all online, of course.

<comment><USER6> >The results of every rated game (played at tournaments etc) are reported to a rating body

Sounds like another instance of centralization that must be disrupted by the blockchain

<comment><USER7> You can chess go on lichess.com and get a rating by playing games

</comment></comment></comment></comment></comment></comment><comment><USER8> OK, nice article. I'm a chess player myself and so yes I did make some connections.

However, when I opened the article, I was looking for what those lessons were. At least glimpse of them.

<comment><USER9> Quotes from OP suggest that the lessons might be about chess problems and ways for solving them, but might also go wider. A diversion for the kind of people for whom $150+ for an hour out of the office is not a major expense.

 _" Ushering me inside to the small one-bedroom apartment, he offers wine, vodka, or water, in that order."_

...

 _" Many of his clients from the finance world, he says, "have this idea that it's a good idea to distract themselves from their work.""_

</comment></comment><comment><USER10> A question I was thinking about this week: Why do some GM's drop in peak performance? It's common to see players have a golden age of performance where they are near unbeatable. But unlike physical sports, GM's don't exactly suffer from physical deterioration do they?

<comment><USER11> > But unlike physical sports, GM's don't exactly suffer from physical > deterioration do they?

The mind is just as physical as anything else.

It's a deeply complex question, but _generally speaking_ most forms of cognition -- including processing speed, working memory, and fluid intelligence -- decline with age beginning in your twenties:

[https://www.quora.com/Intelligence-and-Genetics-Can-does- IQ-...](https://www.quora.com/Intelligence-and-Genetics-Can-does-IQ-change- over-time)

</comment></comment><comment><USER12> I can see why people with analytical minds can play chess well and create good financial models.

I don't think that learning to play chess better helps one create better financial models.

<comment><USER13> I agree that this claim is false: "If you study chess, then you will be a better trader."

On the other hand, I do think it's beneficial to exercise the muscle of strategic thought and quick decision making. Maybe it's poker instead of chess. But I think the general class of hobby is beneficial.

<comment><USER12> That's the explanation I often hear that I find suspicious. It's not the same muscle.

Chess is a game with complete information, trading isn't. This is an important distinction, heuristics are very different. Maybe poker is better, but then there is opportunity cost. You can get better results by studying applied statistics or another subject matter area closer to trading.

I was playing chess competitively when I was young and am doing financial models for a living now. I don't think any skill or learning transferred.

<comment><USER13> Sounds like we have similar backgrounds and careers.

I can't think of any way of conducting a study to test a hypothesis.

I suppose I would say that, strictly speaking, chess is nothing more than a game of complete information, that has a big search space, filled with interesting corners.

On the other hand, when most non-experts play OTB, much of the thought is psychological — the classic leveling of "what is my opponent trying to do, and thinking I'm trying to do, and therefore trying to do..."</comment></comment></comment></comment>


AMD processors from 2011 to 2019 vulnerable to two new attacks - <AUTHOR> https://www.zdnet.com/article/amd-processors-from-2011-to-2019-vulnerable-to-two-new-attacks/ <comment><USER2> [https://news.ycombinator.com/item?id=22513906](https://news.ycombinator.com/item?id=22513906)</comment>


Ask HN: Switching career path at 36. What is the right way - <AUTHOR> I am a software developer of 36 years. I have been a java/PL_SQL dev for most of my career and been working on Angular for the past couple of years. However I don't see my career going forward and I am not interested in learning Spring Boot or any additional Java Skills as I am afraid it is not going to help in long run. Also Angular is also becoming irrelevant thanks to Vue/React etc. So I am thinking of switching role and transitioning into a Data Scientist or AI/ML engineer. I can put in the hours to study and do self projects, but I am worried if I am too old for a transition now, given that I am already 15 years old in the industry. Should I stick with what I know already and improve on it or can I transition into a Data Scientist role? Thoughts? <comment><USER3> It's your life. Follow your dreams, as they say. 15 years is a good long time. I am well over a decade older than you and I'm planning my next career after I retire in 10-ish years.

I'll throw another tip in here: stay in excellent physical shape. It'll help fight any perceived ageism that comes along. I by no means look young: I am wrinkled and my head is very salt-and-pepper and my beard is pretty much white at this point. But I am in excellent shape - better perhaps all of my peers. And so the ageism thing doesn't seem to be an issue. As a bonus, you'll enjoy life more and probably live longer. STAY FIT. In a few years, it'll get much harder to do so.

</comment><comment><USER4> At age 56 (last year), I switched from a 20+ year career in information security to organizing an AI/ML startup. It's something I always wanted to do. I did what everyone else wanted me to do for so long. I decided to do something I wanted for the last 10 years of my career.

36? You're barely hitting your stride!

<comment><USER5> When you were an infosec pro did you know how to code at a pretty high level? I'm a security guy specializing in forensics and incident response that's contemplating the same switch, however, my math background pretty weak, I don't have a CS degree (CIS Major), and I could probably only land a jr. position as a programmer. I can't abandon 50% of my salary to start over with a wife and a kid just because I'm bored so I feel kind of stuck.

Also, what was your path to learning ML like? Got any advice?

And congrats, as a guy in his late 30s I'm starting to be afraid of my long term employability prospects as someone without the social skills for management. Its inspiring to see people successfully dive into a new field after a long and successful career in something else.

</comment></comment><comment><USER6> It is never too old to transition and 36 is not old. Have a growth mindset. Don't listen to anyone who says it is too old to switch. You have one life to live, do what makes you happy.

Here is one advice, check that you want to move to AI/ML for the right reasons, not because it is the next cool thing.

One strategy: Take some of the courses in ML, for example Andrew Ng's course on ML and Deep learning. If you enjoy the course, next find a real project to do and see if you truly enjoy it. If you do enjoy it then switch, otherwise Java programming is always there. I have found real ML is a lot different from Andrew Ng's course for example, pretty much like doing algorithms in class is different from real world programming.

</comment><comment><USER7> It's not late to switch at 36. My suggestions would be

* do not throw your past experience and start from zero, cash on your experience: learn some Data Science / ML / Data Engineering and transition to the role where you can work closer with them: building data services, productionising ML, etc.

* try before committing. I personally thought that due to my experience in software and passion for mathematics ML would be a natural chemistry for me, but I found it quite boring (comparing to both doing pure mathematics and building software).

* the area is still hyped and there are lots of people who wants to get there - it raises the hiring bar.

</comment><comment><USER8> So I would switch but not to AI/ML. Most of the work in AI/ML is not training models , but the infra around (getting the data, scrubbing the data, securing the data, etc). So most of the work is regular software engineering.

Also, most of the simple DS tasks, will be automated by AutoML.

If I were you I would learn golang or Rust. You can join a data engineering group or any company that is moving to cloud native architecture.

</comment><comment><USER9> You aren't that old. I've been doing Oracle PL/SQL development with reporting/BI for over 20 years.

I started doing Java dev and found out that it's really a full time job to have a decent awareness of all the Java and spring framework or EE stuff so I never really progressed it. I've also done the same with .Net and web dev (including various frameworks) and mobile.

This has given me a good overall technical knowledge and I'm now working as a technical architect for various types of systems which include databases, microservices, rest, queuing, cloud etc. I still get to do some technical low level work as well as high level design and discussion/mentoring with devs and business. It seems to fit well with my skills, is well paid and currently in demand. It does require a bit of a change of approach compared to low level work.

I'd avoid ML/AI or anything "cool" unless you have a particular interest or aptitude for it. Don't spend too long on any particular framework as they will change soon enough.

Good Luck!

<comment><USER10> "I'd avoid ML/AI or anything 'cool' unless you have a particular interest or aptitude for it."

Ditto. I wanted to get into ML/AI so I posted for some roles internal to my company. It was not great. The hours would have been much higher with no salary increase. The company also wouldn't provide any real training (generic stuff on Pluralsight). Also, I would still just be coding like in other software since they only give the interesting parts of the work to people with PhDs - I'd be scrubbing data and testing models, not doing research and creating them.

</comment></comment><comment><USER11> I'd say the majority of your demographic (mid 30s Java jockey) feel this way. Knuth said always do the opposite of what's hot.

</comment><comment><USER12> > _So I am thinking of switching role and transitioning into a Data Scientist > or AI /ML engineer. I can put in the hours to study and do self projects, > but I am worried if I am too old for a transition now, given that I am > already 15 years old in the industry._

There's a lot of the heavy lifting done in Spark (written in Scala), and many parts of the "ML" ecosystem are on the JVM, either with Scala or Java, so you can leverage your experience with that. Kafka is also used and written in Java. Often times "data scientists'" output will be a Python notebook, and in many organizations someone "translates" that to either Scala or Java if they don't want to use pySpark.

My point being that you can have a smoother transition than what you imagine. A _huge_ part of the work in "AI/ML" is not model building, and that could be your entrypoint.

</comment><comment><USER13> Plenty of useful software is written in Java. It's not going away. But if you want to move away from Java, then I'd suggest staying in programming, but learning Go. Plenty of enterprise software is now beginning to be written in Go, so having a background in it can generate initial interest from Hiring Managers. However, it is just that: initial interest. You still need to demonstrate general coding ability and capabilities that any good candidate would be expected to demonstrate, such as good communication, being personable, showing enthusiasm for the work, paying attention to detail etc in order to get ahead (it would also include a fair bit of moving on from dead- end jobs). All of that will not change simply by moving to Go programming or Machine Learning.

</comment><comment><USER14> I would advise you to question what you don't like about Spring Boot. I also don't like Spring Boot and React but I write them because money. My personal projects are not in spring boot and react and those keep me plenty happy on the side when I need to detach. Good luck!

</comment><comment><USER15> You're kind of conflating two different types of career change - <USER2> tech stacks vs switching the domain in which you work. You could switch to a different web framework if you think Angular is becoming irrelevant (same domain, different tech stack). Or you could look for companies that run their ML on Java (familar tech stack, different domain). Yes, those companies do exist.

With either of those your past experience carries over to some extent. But if you decide that you want to be a data scientist and use Python, then you're putting yourself at massive disadvantage. First, because your past experience is far less relevant. Second, because there's a huge oversupply of beginner data science/ML people who use Python.

</comment><comment><USER10> I see a lot of comments about not being too old to switch. I generally agree with that but just want to point out that it is different for everyone. So it's also ok to stick to what you know if you think that is best.

I'm a little younger than that and I'm already starting to struggle with constant stack switching and the much larger scope of the stacks (AWS DevSecOps using multiple services and languages). I didn't really have a choice since I was in Neoxam and FileNet. I think Angular is much more popular than those, so you should be ok for at least a couple years if you wanted to stick with it (my company uses Angular and has no plans to switch any time soon).

</comment><comment><USER16> At 34 I transitioned from a product manager to a deep learning researcher. No previous coding experience, and most of the math I learned in college (as a mechanical engineer) was long forgotten. I went the phd route, so it took me ~6 years, but with your background it would probably be faster. I really enjoyed my grad school experience, and I'm enjoying my current role.

</comment><comment><USER17> Data Scientists use heavy math. You might need a PHD for that. On the other hand, I think you are suited for big data engineering which may use scala, python, cloud based computing and most importantly your exp relates.

<comment><USER18> Not really. That is the whole point of machine learning. The computer tries millions of combinations to help tune a model. You aren't trying to find the closed form of an equation. There is a lot of math in the various tuning algorithm that helps the computer find the optimum solution. You need to be able understand things like convolution, gradients etc., but that is like high school math.

</comment></comment><comment><USER19> Yeah thing is people now routinely have their first kid at 36. Just everything has shifted forward and you've no frame of reference for it because you're living it. Your generation is going to do everything older.

</comment><comment><USER20> It's never too late. Invent yourself then reinvent yourself.

</comment><comment><USER21> If you believe you're too old you're. If you believe you're not, you're not.</comment>


Ask HN: Is There Any Good Open Source Voice-Activated Virtual Assistant? - <AUTHOR> <comment><USER2> OpenJarvis: [https://github.com/alexylem/jarvis](https://github.com/alexylem/jarvis)

I'm also on that with some folks on irc (check my profile for contact).

</comment><comment><USER3> [https://snips.ai/](https://snips.ai/)

This was posted on HN the other day. I can't vouch for whether it's any good or not, but at quick glance it looks interesting.

</comment><comment><USER4> Check out [https://mycroft.ai](https://mycroft.ai)</comment>


New rsync-based dropbox alternative for Windows - <AUTHOR> http://www.acrosync.net/acrosync_win32.html

<comment><AUTHOR> I'm the lead developer and we're currently recruiting beta testers. If you're interested, write to the email address in my profile.</comment>


Wireguard VPN: Typical Setup (2017) - <AUTHOR> https://www.ckn.io/blog/2017/11/14/wireguard-vpn-typical-setup/ <comment><USER2> "It's just as easy as SSH!" and "Ok, set up your iptables rules...." do not go together. Wireguard might be easier than OpenVPN, but it's still prohibitively difficult to set up, and especially to use in a client computing scenario (i.e. I'm on a desktop machine, and want to connect/disconnect this VPN frequently)

</comment><comment><USER3> I tried streisand / wireguard setup awhile back (MacOS to DO server). It worked great until I tried to disconnect (it would save the old WG settings in my connection tables and then just keep me completely disconnected from the internet). Anyone know if this is improved? Is there any easy way to get wireguard setup on macos and connect to a DO server?

<comment><USER4> I used the WireGuard package from homebrew to setup a vpn back to a Raspberry Pi at home and it worked pretty well.

<comment><USER5> The WireGuard client for macOS behaves differently than the Linux client when setting DNS settings.

<comment><USER6> We do need to port wg-quick to use scutil instead of networksetup.

</comment></comment></comment></comment><comment><USER7> having played with wireguard I can definitely say that it takes something that used to be complicated and makes it trivial (ie setting up a vpn server).

it almost feels too easy compared to the openvpn monstrosity.

now that they have an iphone app (in beta) I switched to using it everywhere and not looking back.

<comment><USER8> I picked up WireGuard yesterday, and within an hour, I had my server and 4 clients setup. Freaking gravy.

And this morning, I connected to the bus WiFi, I turned on WireGuard, and kept a constant connection throughout my commute to work. It just works so well.

I could never do that with my OpenVPN connections.

<comment><USER9> Just curious; but what kind of device are you using here? I can't imagine using a laptop on the bus.

<comment><USER8> Today was my iPad. But I've ran my laptop before many times. This is an express bus, with only a couple of stops and runs mostly on the interstate for 25 miles.

<comment><USER9> Yes but how do you put Wireguard on an iPad?

<comment><USER6> [https://lists.zx2c4.com/pipermail/wireguard/2018-November/00...](https://lists.zx2c4.com/pipermail/wireguard/2018-November/003526.html)

</comment></comment></comment></comment></comment></comment><comment><USER10> It says you get about a Gbit of throughput. That isn't bad but I wonder if that is simply because the processing isn't or can't be efficiently paralleled.

You might be able to make several of these connections between 2 machines and trunk/bond them on each side into a combined interface for even more throughput. 10Gbe and 40Gb fiber cards are getting pretty cheap.

<comment><USER11> > It says you get about a Gbit of throughput. That isn't bad but I wonder if > that is simply because the processing isn't or can't be efficiently > paralleled.

The encryption is parallelised, Jason Donenfield talks about how the parallelism is designed in most of his talks about WireGuard.

Effectively the problem is that network processing is done using "flows" and generally you want to pin a single flow to a single CPU -- but that would result in bad performance (especially if related packets were encrypted on separate CPUs). So instead, WireGuard pretends to support hardware packet offloading -- so the rest of the net infrastructure figures out what packets are related and gives WireGuard a coalesced packet which can then be split and encrypted in one go.

<comment><USER10> Cool, I just realized they also used a 1GbE card in the perf test so they maxed it out. That would be pretty neat if it scaled right on up.

</comment></comment></comment><comment><USER12> I guess 20 years as a sysadmin and systems programmer make me a little jaded but you wouldn't convince me to pick up a new VPN based on it being OpenSSH- alike. I'd prefer a pure TLS/SSL or IPSEC solution that fits either paradigm.

If I want ssh-alike I'll harden my ssh and PAM configuration and use ssh (+ tunneling/forwarding and socks5).

</comment><comment><USER13> This article is a year old. Certainly Wireguard setup / config has improved this past year, yes?

<comment><USER14> Could you clarify "improved"?

It's already absolutely fantastic compared to any other solution I can think of (_especially_ OpenVPN, which requires generating a bunch of x.509 certs).

This article has a ton of steps which aren't even WireGuard specific like server firewall rules.

The actual configuration of WG is basically 'generate a private and public key on server and client, stick it in a config file, run wg-quick up, done'.

<comment><USER15> I don't know, I really like the idea of wireguard but I had a pretty tough time setting it up (and had to give up after an hour)... more like I didn't have the time.

I think I wasn't doing something right, though, since I was getting very weird errors that I wasn't able to google. Also I think my issue was like more with the macOS client rather than the server.

I think Wireguard is still at the same level that git was about 10 years ago: powerful, elegant, but absolutely not user-friendly. But nobody is claiming that it is (same with git).

I suspect in the next couple of years, you'll see an explosion of really nice tools to make this setup even easier than it is now.

Until then, I'll keep fiddling and try to get wireguard running on my macOS client/ubuntu server setup.

<comment><USER16> Try putting something like this on your Mac in /usr/local/etc/wireguard/foo.conf:

        [Interface]
        PrivateKey = redacted
        Address = 10.100.0.3/24

        [Peer]
        PublicKey = redacted
        AllowedIPs = 10.100.0.0/24
        Endpoint = 18.xxx.xxx.xxx:51820
        PersistentKeepalive = 25

and then doing

        sudo wg-quick up foo

You should see something like

        INFO: (utun2) 2018/12/16 18:43:24 Starting wireguard-go version 0.0.20181018
        [+] Interface for foo is utun2
        [#] wg setconf utun2 /dev/fd/63
        [#] ifconfig utun2 inet 10.100.0.3/24 10.100.0.3 alias
        [#] ifconfig utun2 up
        [#] route -q -n add -inet 10.100.0.0/24 -interface utun2
        [+] Backgrounding route monitor

That's assuming you have a matching server config set up on your Ubuntu...

</comment></comment></comment></comment><comment><USER17> Hmm. There still doesn't seem to be a Windows client.

<comment><USER6> It's actually under fairly consistent and active development right now. Hopefully we'll have something to share soon.

<comment><USER18> Hello Jason could you please give an estimated availability date for the Windows client and a valid link to the section of your repo/website where someone can follow the progress? I have searched extensively for any info on the schedule. road map, estimated availability for this and have had no luck finding any info about this very important client for Wireguard. I have been patiently waiting for this and holding off on other Windows VPNs but need something very soon. Thanks for your help.

</comment></comment></comment><comment><USER19> Does wireguard work with docker?

<comment><USER20> [https://www.wireguard.com/#ready-for- containers](https://www.wireguard.com/#ready-for-containers)

</comment></comment><comment><USER21> I've been trying out Wireguard with IVPN ([https://ivpn.net/](https://ivpn.net/)) and so far I'm impressed. Very quick tunnel setup (especially important on mobile iOS devices) and great performance afterwards.

<comment><USER22> For anyone looking at retail VPN services, Mullvad and AzireVPN also offer it.

<comment><USER23> So does Astrill, they just updated there client.

</comment></comment></comment><comment><USER24> What exactly does IP forwarding do on the server?

<comment><USER25> The author is setting up NAT w/ the iptables rule in the POSTROUTING chain that does `-j MASQUERADE`. With that rule in place, combined with `allowed-ips 0.0.0.0/0` on the wireguard client, the client can tunnel all their IP traffic through the server as opposed to just having a private subnet setup between the two endpoints.

</comment></comment><comment><USER26> Any recommend iOS client?

<comment><AUTHOR> WireGuard for iOS (beta via Testflight App): [https://lists.zx2c4.com/pipermail/wireguard/2018-November/00...](https://lists.zx2c4.com/pipermail/wireguard/2018-November/003526.html)

<comment><USER22> I've been using it for a while and I've found it quite usable and reliable. Blows OpenVPN's iOS client out of the water actually.

</comment></comment></comment><comment><USER27> Wireguard is UDP only. The reliance on UDP means wireguard is unusable in many networking environments.

<comment><USER22> For those in this unfortunately position, you can run a TCP <-> UDP conversion without much difficulty: [https://lists.zx2c4.com/pipermail/wireguard/2018-March/00250...](https://lists.zx2c4.com/pipermail/wireguard/2018-March/002508.html)

WireGuard works just fine like this.</comment></comment>


Ask HN: Bill Gates or Steve Jobs? Who do you hire first for your start-up? - <AUTHOR>

<comment><USER2> Bill Gates. Seems like a much easier guy to get along with, and eventhough Steve Jobs's vision and ideas are great, it doesn't really make sense to hire someone for the vision: you should have that yourself.

</comment><comment><USER3> <http://img576.imageshack.us/img576/430/microsoftg.jpg>

</comment><comment><USER4> Gates. Because if you take Jobbs, then whatever you make will be painted white, made shiny, and the price doubled.

</comment><comment><USER5> This should have been a poll. I prefer Jobs style.</comment>


Zip Files: History, Explanation and Implementation - <AUTHOR> https://www.hanshq.net/zip.html <comment><USER2> In college we had a project to write a compression/decompression utility.

We had to implement all the data structures from scratch and implement several compression schemes.

I got the simplest one, run length encoding, working but couldn't get the fancier LZ77 stuff to work correctly. Mostly because I had trouble implementing some of the data structures required in C++, from scratch.

The professor had a few of the better students present their programs and one of them demonstrated that not only could they make smaller files than WinZip, they did it _faster_ too.

It absolutely blew my mind and that's probably the day I realized I needed to really step up my game and put in the work if I wanted to be successful in this field.

<comment><USER3> If you're not too worried about speed nor memory consumption, a simple brute- force LZ77 compressor is, to a first approximation, two nested loops (one to find a match, and one to extend it.) This will also compress better than the hash-table-based approach discussed in the article, since it always finds the _best_ match, although at the expense of speed. In comparison, RLE is one nested loop (to count repeated bytes.)

<comment><USER4> Do you mean following the hash chain to the end of the window?

If you want an exhaustive LZ matchfinder, a trie, binary tree such as LZMA's bt4 or suffix array are more efficient algorithms, especially with window sizes larger than 32KB, and there are other better matchfinders past 64MB such as Rabin Karp for longer matches.

Zip is inefficient as a format on modern out of order CPUs. Modern replacements such as zstd have blocks of literals, lengths and decisions to decode an entropy coded table at once rather than conditionally branch.

Also, beating zip compression is quite easy: use a larger window size and a relatively quick (small number of tested slots) hash chain (or multiple e.g. 3, 4 and 7 bytes) can find better matches.

If decompression time and speed is not an issue, and you aren't limited to LZ based approaches, a bytewise model such as ppm or bitwise such as paq will provide much better compression without too much thinking.

Edit: forgot to mention, the parsing strategy is also important. Finding an approximately optimal lowest cost path through a block can often save 10%+ ratio with LZ + Huffman.

</comment></comment></comment><comment><USER5> As a side note, one of the "tightest" ZIP compatible programs around AFAIK, Ken Silverman's KZIP:

[http://advsys.net/ken/utils.htm](http://advsys.net/ken/utils.htm)

Worth of note is also ZIPMIX that in a few cases is very useful.

</comment><comment><USER6> As a recovering alcoholic, I've always looked at the story of Phil Katz as a cautionary tale. He made enough money from people registering PKZip for discs and manuals that he was able to succumb to his tendency to socially isolate and drink to excess. He drank himself to death.

<comment><USER7> I remember the name Phil Katz from "PKZIP" back in the BBS days. I had no idea he had such an unhappy story!

The wikipedia page covers it briefly: [https://en.wikipedia.org/wiki/Phil_Katz](https://en.wikipedia.org/wiki/Phil_Katz)

<comment><USER8> I can recommend that you watch "BBS: The Documentary", they cover the arc wars as well, there's also this document: [http://www.bbsdocumentary.com/library/CONTROVERSY/LAWSUITS/S...](http://www.bbsdocumentary.com/library/CONTROVERSY/LAWSUITS/SEA/katzbio.txt)</comment></comment></comment>


Bootstrap 4 ditches webfonts - <AUTHOR> https://github.com/twbs/bootstrap/pull/19098 <comment><USER2> One of the comments really highlights a good point here: For default bootstrap, this will mean less to load, and more readable fonts. For sites that want consistency across all devices, it seems extremely unlikely they'd also want the default font from a css framework anyway.

</comment><comment><AUTHOR> After Github and Medium, it looks like the industry is backpedaling on webfonts. Probably Terabytes of bandwith saved by this move.</comment>


Legos in the lab – condensed matter physicist uses Legos as research tool - <AUTHOR> http://www.pnas.org/content/112/42/12901.full <comment><USER2> You know you have a real hit when people use your product in ways that you never ever envisioned when you designed it.</comment>


How to use feature flags without technical debt - <AUTHOR> http://blog.launchdarkly.com/how-to-use-feature-flags-without-technical-debt/ <comment><USER2> Hmm. This is a different kind of feature flag than I've used, to solve a different kind of problem.

If the feature you're writing takes several man years of effort, you can't have a feature branch living for several months; continuously keeping it up to date with the trunk is expensive and easy to procrastinate.

Migrations are expensive and you want to front load them to make turning the feature on less stressful. And you may want to let customers use the feature on a beta basis for a few months before committing to it, and then it may take a year or more before all customers have moved.

For a big feature that cuts across large segments of a big app, I don't think there's an alternative to if statements.

Different apps, different business models, etc.

</comment><comment><USER3> This misses the problem -- often, the new feature is buggy in ways that are not seen in the initial testing. In critical systems, it's often desirable to keep the ability to flip back to the old code around for a couple of release cycles. In a previous life, that has saved my team's bacon.

This comes at a cost, and somehow saying "just delete the flags promptly" is a facile solution; If it was easy to just delete them quickly, it would be even easier to just land the change without the flag, and use rollbacks as your 'undo a bad feature' hammer.

<comment><USER4> > This misses the problem -- often, the new feature is buggy in ways that are > not seen in the initial testing.

Indeed. The only good way to enable features gradually is to use a tool like GitHub Scientist [0] that exercises the new code path and records its effects but then uses the effects of the old code path in production. This allows weird edge cases to be found and dealt with before enabling the new feature.

[0] [http://githubengineering.com/scientist/](http://githubengineering.com/scientist/) Previous discussion: [https://news.ycombinator.com/item?id=11027581](https://news.ycombinator.com/item?id=11027581)

<comment><USER5> Yeah, I actually wrote about something similar recently (in the context of a database migration): [http://blog.launchdarkly.com/feature-flagging-to- mitigate-ri...](http://blog.launchdarkly.com/feature-flagging-to-mitigate- risk-in-database-migration/)

</comment></comment></comment><comment><USER6> To clarify...

This is to roll out a feature to a small amount of customers for inital live beta testing before rolling out to all customers?

If so i think this is good. It is documenting the real issue of complexity of comming back to old code and old problems (old being weeks), even if you don't merge it cause of merge hell. At least you have a document of what to do.

And you are culling dead/dangerous code.

<comment><USER5> Yes, this clarification is accurate. I was thinking of 'canary launch'-style releases, where you release the new thing to a small group, then a larger group, etc, until everyone is getting it, and you don't need the flag any more.

</comment></comment><comment><USER7> In my experience with large codebases and multiple teams, another developer might copy that flag into another part of the code to get some desired side- effect.

Yes, this is horrible, but in the real world...

I find you have to grep through the code and think about all the changes that impact your feature flag before systematically removing it. You're cleanup branch isn't being maintained and is could provide a false sense of safety.

<comment><USER5> You are right that the real world is always more complicated. However, I think the idea holds. If another dev needs to use the flag in another area of code, the flag cleanup branch should be maintained with this change.

The point is not to make flag cleanup automatic. It is to front-load the work of cleaning it up when the complexities involved are fresh in your mind. That way, when it comes time to clean it up, it is much easier to be more confident that you found all of the edge cases.

</comment></comment><comment><USER8> "You will need to merge master back into your cleanup branch periodically, but that is usually easier than it would be to recall all of the context relating to the original change."

Won't there be merge conflicts when you do this the first time as the clean feature branch code be different from the flag based feature on the master? Of course, all the subsequent merges should be conflict-free.

<comment><USER5> There shouldn't necessarily be any merge conflicts if you branch the cleanup branch off of your feature branch. So, it might look like this:

      -master</comment></comment><comment></comment><comment>---*</comment><comment></comment><comment></comment><comment>--*--
        \-feature-branch---/                    /
                          \-cleanup-branch-----/

</comment><comment><USER9> On a tangent, how long-lived are feature toggles usually?

I have very limited experience, and it points to a wide range from a few days to few months. When I stumble about a 1y+ old flag, I tend to delete it (and the dead code path that it comes with).

What's your experience?

<comment><USER10> Feature toggles can last decades.

For instance GCC has a feature flag called -ansi which gives you C90 compatibility.

C90 was superseded in 1999 by C99, and so that's 17 years of compatibility, and counting.

<comment><USER11> I don't think this is a feature flag in the same way the rest of the discussion is using the phrase.

<comment><USER10> How so? C99+ support/conformance is a compiler feature. That feature breaks/conflicts with some aspects of C90 support, an existing, older feature. So you need a feature flag. Inside comiler there are various places where you have the equivalent of "if C90 do this, else do that".

<comment><USER11> "Feature flags" tend to be for behavior that is being developed and tested. The -ansi flag is more like configuration. It's pretty valuable to continue to support C90.

</comment></comment></comment></comment></comment><comment><USER12> I've had good success in the past adding @deprecated annotations to the old code when adding new code behind a feature flag.

It makes it much easier to come along later and know which functions can be deleted when the decision is made to kill the old code.

</comment><comment><USER13> Sounds dangerous. Later commits on master might actually add more if(feature- flag) statements, so if you then just mindlessly merge the cleanup branch, you'll miss the added ifs.

I'd prefer to create a cleanup branch like any other feature branch only when actually going to clean up, and spend the extra cost getting back in context, studying all the if(feature-flags) from master. Otherwise you might miss some, or you might forget some interaction that you learned after feature deploy.

<comment><USER5> Yeah, I think if you mindlessly do anything, you're gonna have a bad time.

Think of the cleanup branch as a running list of changes that you know you will need to make to remove the flag. Any future references to the flag should keep this cleanup list in mind. Code reviewers should keep these cleanup lists in mind.

This list of cleanup tasks happens to be expressed as a branch in your VCS (this is a pretty good way to express changes that need to be applied to a codebase). You will still need to be careful when you execute that list, but it will be helpful to have the running tally of things that need to be done.

</comment></comment><comment><USER14> Haven't tried it myself, but why not use authorization libraries instead of specialized 'toggle' libraries?

After all, both are concered with whether user X is allowed to do Y.

Using just one approach might be a clean, maintainable approach.

The original code `if can?(:use_feature_x, user)` is written just once, and then never needed to be removed. The only thing that changes, gradually and cleanly, are the business rules in :use_feature_x (e.g. update the method in your ability.rb, using Ruby CanCan terminology)

<comment><USER5> In canary launches, you might want to roll a new feature out to 10% of your users, then 20%, etc. Once it is released to 100% of your users, you might want to remove the check, since it is a no-op.

I'm not aware of any authorization libraries that let you grant access to a percentage of your users, but maybe they are out there? It is a strange use case from an 'authorization' standpoint.

<comment><USER14> no-op point is true (except for logged out users - then the check is still useful)

Anyway, how do you consistently decide to which 10% you show the new feature?

That piece of data is better stores in your Users table, as I see it. Plays well with authorization libs.

<comment><USER5> The way LaunchDarkly does it is to hash the user key, along with the feature key. This way the same users aren't always included in the '10% set' for all features, but they are consistently in the 10% set for a single feature.

This also allows the decision to be made in memory, without an additional round-trip to the DB.

</comment></comment></comment></comment><comment><USER10> In the TXR language interpreter, I have a -C option which takes a numeric argument: it means, simulate the old behaviors of version N. If you don't specify -C, you get the latest behavior.

Throughout the code, old behaviors are emulated, subject to tests which look similar to this:

       if (opt_compat && opt_compat < 130) {
         /* simulate 130 and older behavior */
       } else {
         /* just the new behavior please: -C was not specified,
            or is at 130 or more. */
       }

I think that tying specific old behavior to a proliferation of specific options is a bad idea. It does provide more flexibility (give me some old behavior in one specific regard, but everything new otherwise), but that flexibility is not all that useful, given its level of "debt".

The purpose of compatibility is to help out the users who are impacted by an incompatible change; it gives them a quick and dirty workaround to be up and running in spite of the upgrade to the newest. They can enjoy some security fix or whatever, without having to rewrite their code _now_.

However, they should put in a plan to fix their code and then stop relying on -C.

If users are given individual options, that then encourages a behavior whereby they use new features with emerging releases, yet are perpetually relying on some compatibility behaviors. This leads to ironies: like being on version 150, and starting to a feature that was introduced in 145 and changed incompatibly in 147 and 148---yet at the same time relying on a version 70 behavior emulation of some other feature. Hey we don't care that this new thing was broken recently twice before being settled down; we never used it before! But we forever want this other thing to work like it did in version 70, because we did use it in version 70. It's like using C++14 move semantics and lambdas, but crying that GCC took away your writable string literals and -fpcc-struct-return (static buffer for structure passing).

It's very easy to hunt down the opt_compat uses in the source code just by looking for that identifier, and the version numbers are right there. If I decide that no emulation older than 120 will be supported in new releases going forward, I just grep out all the compat switch sites, and remove anything that provided 119 or older compatibility. The debt is quite minimal, and provides quite a bit of value.

<comment><USER15> Is there any way you could explain that again? I don't quite get what you are doing.

<comment><USER16> the user can optionally specify which version of behaviour they want. This is named the `opt_compat` value in the code. All through the code there are checks against the `opt_compat` value to decide which version of which old/current behaviour to use.

<comment><USER10> And the opt_compat has C integer/boolean semantics in this case, so the test

      if (opt_compat) ...

tests whether the option has a nonzero value (has been specified).

And so

      if (opt_compat && opt_compat <= 130)

means "user has requested compatibility, with a value of 130 or less".

By the way -C 0, which would look as a Booealn false, as if -C were not specified, is not allowed. If the user specifies -C N such that N is lower than the oldest version that we emulate, the implementation terminates with an error message like "sorry, compatibility with versions less than 70 is not supported by version 140".

</comment></comment></comment></comment><comment><USER17> If you do feature flags by inserting if blocks throughout your code you will create tech debt anyways. The goal is to have one if block and hide the changed behavior behind interfaces (or polymorphic functions if you are using functional languages). Dependency injection is your friend.

If you don't do this, you won't scale beyond a hand full of feature flags. Chrome has hundreds, for example.

<comment><USER18> Can you expand on this? I'm interested to see how this works in a real code base.

I'm thinking something like an initial (maybe massive) if block in the setup of the application that sets all of the behavior/features by declaring which implementations get set to which interfaces? After this if block, all of the DI stuff is set?

This of course means you need to use a DI framework of some sort.

Using feature flags is something I'm investigating because our current model is a git branch for every feature, and I wonder/fear it only works because we're a small team that has worked together for a while and in the future when we grow this will break down.

<comment><USER19> I was about to say the same thing as your parent. I recently had a nice feature toggle experience using a strategy pattern with DI.

Basically there was one 'if' statement in the DI container configuration code that looked up a config. Basically

if (newPricingStrategy) bind IPricingStrategy to NewPricingStrategyImplementation else bind IPricingStrategy to OldPricingStrategyImplementation

</comment></comment></comment><comment><USER20> This obsession about avoiding technical debt is quite strange. It's a tool to use when it makes sense, like loans in the bank...

<comment><USER21> Debt, both technical and financial, is _always_ best avoided. And in both cases, once you have it, you have to pay it down sooner or later, and the later you do the more expensive it gets.

It is a tool, but having it is always a negative that is offsetting a bigger negative. By all means, take the loan when you need a boost that you can't otherwise afford. But take the smallest loan you need, and pay it back as fast as you can.

<comment><USER19> no, often you don't have to pay for technical debt. the code may never be refactored, and / or the whole system / module replaced before lots of code is touched.</comment></comment></comment>