NSA's top talent is leaving because of low pay, flagging morale, unpopular reorg - <AUTHOR> https://www.washingtonpost.com/world/national-security/the-nsas-top-talent-is-leaving-because-of-low-pay-and-battered-morale/2018/01/02/ff19f0c6-ec04-11e7-9f92-10a2203f6c8d_story.html?utm_term=.0388eaf826c3 <comment><USER2> It seems like it's pretty hard for the government to compete with private industry in compensation. Federal pay grades are capped by Congress, and the top grade is around ~$150k a year [1]. And note that since it's the top of the scale, you have to start people off lower so that there is salary progression.

Pretty sure a top cryptographer or security expert should be worth several times that, especially if they are also dealing with classified information and safeguarding all the data collection (e.g. if the NSA is going to spy on us, wouldn't you like that to be as secure as possible?).

This is another reason why we're going to enter an era of corporate feudalism. There has been a cycle of:

1\. perception of gov't incompetence

2\. normal people ask 'why are we paying people so much for doing a bad job?'

3\. gov't pay scales fall further behind private industry

4\. rise in relative mediocrity

5\. repeat

I think voters largely don't realize how much competence actually costs in the market now and will vent about their justified perceptions of inequality by preventing equalization of pay between the gov't and private sectors, but that will paradoxically just make things worse in the long run as the private sector snaps up all the talent and we end up with a barely-functioning federal bureaucracy.

[1] [http://work.chron.com/nsa-pay- scale-16399.html](http://work.chron.com/nsa-pay-scale-16399.html)

<comment><USER3> The problem is that we can (and should!) raise government pay scales, but we must _also_ make it easier to fire people (or manage them downwards) for incompetence. Government work is for two types of people (and there's some overlap on the Euler diagram): patriots, and folks who can't hack it in civilian life.

It's not just a 'perception of gov't incompetence': it's a widespread phenomenon. Government employees tend to excel at working organisational politics (i.e., in serving their organisational customers) rather than in serving their taxpaying customers. Government work is a kind of kabuki theatre, in which everyone agrees to praise the emperor's new clothes and ignore the boy who points out that he's naked.

The unfortunate thing is that government work is really important: it _requires_ the best and the brightest, not the lazy and the uninspired. Patriotism should of course be a _sine qua non_ for the civil service, but so should excellence. We should pay market rates, but we should pay them for market quality.

Raising rates without working to improve quality would mean getting mutton when we pay for lamb.

<comment><USER4> It's popular to say things like some government employees are "folks who can't hack it in civilian life," but there's basically no evidence this is true.

First of all, let's make sure we look at the full scope of civilian life with a sober eye. Anyone working the counter at your local Post Office could easily work the register at any grocery store in your town. And any medium to large company has plenty of internal politics. Don't make the mistake of comparing the worst case of government to some idealized notion of private enterprise.

Second, take a look at what happens to people who do leave the government. Very often, their salaries go up in the private sector. This is especially true for specialists like the folks who work at the NSA. Seems like they can hack it.

Folks who have not worked for the government, or very close to it, have a hard time comprehending the fundamental differences between government and private enterprise. For one thing, when you work for the government, many rules of your job carry the force of law. Being too flexible might not just get you fired, it might get you prosecuted.

And there are _so many rules_. I'll give you one example scenario: a person leaves their job, and a subordinate steps up, takes on their workload, and does great.

In a private company, you'd just promote the subordinate. You already know they can do the work, you retain institutional knowledge, and it's easier to hire junior positions. Efficient and effective.

But in most federal agencies that would be illegal, as it might permit some form of corruption. The subordinate would have to keep doing the extra workload, while the supervisor requisitions a new position to replace the old one, and then runs a (highly regulated) open public application process. The subordinate can apply too, of course.

This is not because the supervisor is some terrible inflexible or dumb person. _The rules are just stricter and more burdensome in the government than in private enterprise._

Why? Because everyone loves to believe the worst about the government, so it's easy to sell voters on the necessity of a shitload of burdensome rules.

So: if you want people to act smarter in the government, you have to give them permission to act smarter. Firing has little to do with it. You have to give people the opportunity to try things and make mistakes in good faith.

<comment><USER5> > _It 's popular to say things like some government employees are "folks who > can't hack it in civilian life," but there's basically no evidence this is > true._

Have you ever worked for government? I'm a civilian contractor in a software architecture role to a public safety agency in a state government. Half the "senior application developers" haven't written code in 15 years and don't know what a unit test is. There are maybe 2-3 individual contributors for each manager. Not team lead, manager, and is in a $105-110k+/yr position that does annual reviews, on call scheduling, etc.

The floor I'm on has 25-35 developers, DBAs, and BAs, and I can count on one hand the number of people who could get hired into a junior technical position at my employer. There are people who are unironically counting the days they have left until they can retire and draw their pension when they have more than 5 years left.

> _Anyone working the counter at your local Post Office could easily work the > register at any grocery store in your town._

And they'd take a 60% pay cut to do so, wouldn't they? Anyway, this is about folks in technical roles. Running a register and handing out stamps isn't particularly technical.

> _Second, take a look at what happens to people who do leave the government. > Very often, their salaries go up in the private sector. This is especially > true for specialists like the folks who work at the NSA. Seems like they can > hack it._

Well that's sort of tautological, isn't it? "The people who get a civilian job are capable of getting a civilian job." These folks are in the "Patriot, not inept" section of the diagram.

> _Firing has little to do with it._

Firing has a lot to do. I've spoken candidly about my colleagues here with higher ups in the agency. They've openly bemoaned their inability to fire _anyone_ , especially the people who they know for a fact do nothing all day. Those people are soaking up hundreds of thousands, even millions a year, in taxpayer dollars, doing little work of substance or value, and preventing their positions from being filled by someone capable.

<comment><USER6> Payroll is the number 1 expense for government agencies. Easier firing would make a world of difference. I think the root cause is pensions. When people are tied to pensions for a retirement, firing people becomes a much bigger deal. Just pay a match into a retirement account and then when people can go, they go. Private sector pensions are for the most part long gone. Why should public sector keep them.

<comment><USER5> Pensions just don't make sense mathematically. If they're solvent based on the state's investments, then the employee is getting a raw deal compared to having just invested the money themselves. If they're not, then I as a taxpayer am getting a raw deal by helping to pay for someone's retirement because they worked for a state agency decades ago.

My state starts doling out pensions at 5 years. A friend of mine worked for a state senator _in the 90s_ for 6 years. He's got something like $1200/mo guaranteed during his retirement for 6 years of borderline political work ~25 years ago.

I know it's a politically charged subject but I don't see why from an economic/mathematical perspective it's any better than moving the same investments to the market. Or at least if you're going to keep the pensions, only give them out for full service (20 years) like the military, _and_ make it easier to fire a government worker for poor performance.

I shouldn't be able to sit on my thumbs for a few years and have my mortgage covered by the taxpayers during my retirement.

<comment><USER7> Pensions make a lot of sense mathematically. If I personally fund my retirement I have to save enough for the "worst" case scenario of living many years longer than average. A pension on the other hand only has to invest enough for the average lifespan because the person who dies a week before retirement and collects nothing helps fund the person who lives to be 105.

State pensions are in trouble because govt pensions are allowed to assume overly optimistic rates of return.

<comment><USER8> If you're worried about living too long you can buy an annuity. It makes no sense for employers or the government to make that choice for you.

<comment><USER9> Mandatory safety net/insurance policies make sense because what we know from experience is that if you don't have a mandated program, some people will blow it off. And as long as we don't have the political will to let them die on the street alone in poverty, it'll be very expensive to try pick up the pieces of the mess once they hit the safety net. Especially if that's something like hitting the ER needing tens of thousands of treatment to stabilize, before just getting kicked back out to the street. Or wasting people's time in the ER without medical need because the ER has heating and a bed and the cold winter street doesn't.

Not everybody has family to fall back on in retirement, but we still don't want to let them simply die. So the money has to be given somewhere.

<comment><USER10> You've gone a bit off the rail here because most people in the US don't get pensions and we aren't forcing tons of people to die in the street.

I think you are confusing pensions with social security. Social security already provides that bare minimum mandatory safety net.

<comment><USER5> Social Security is a pension (defined _benefit_ after a certain point as opposed to defined contribution).

<comment><USER10> Ok, but nobody in the US calls it a pension so when you refer to a pension everyone is assuming the one you got from an employer.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> There's a lot of blaming of external factors being injected into this discussion. Namely, government pay scales, and the Trump factor.

But let's not forget the NSA's own self-inflicted wounds as well. Let's see, here's what I remember:

* mission creep

* overreach

* LOVEINT

* massive collection of bulk intelligence with dubious justifications

* withholding information about 0 days it wanted to stockpile for its own use

* misleading technical committees so as to weaken encryption and security related standards

* introducing flawed security protocols

* paying money to security companies (RSA) to introduce weaknesses

* (thus with the above four items) actively sabotaging its own mission of helping to secure our nation's systems

* allowing its documents to be leaked en masse

* allowing possibly its most powerful and dangerous (though we don't know) sets of tools to be leaked as well.

Did I forget anything?

<comment><USER12> Yes, you forgot one thing:

* We need a powerful spy organization to compete on the world stage.

This is an unpopular truth that people in our circles tend to forget. Rome was an influence solely due to the success of their military. What do you think will happen when ours becomes ineffective?

And the NSA is a key part of our military. Intelligence is everything.

I think it's good the overreach was exposed, but I was worried about exactly this outcome.

<comment><USER13> > We need a powerful spy organization to compete on the world stage

I disagree or at least don't agree to the extent I might have in a more embattled time (i.e. any time in the past). I do agree intelligence serves the military well, I just don't agree with military intelligence scope growth at a time when military use is shrinking. I'm not sure history repeats itself here as much as perpetuates the justifications being used.

<comment><USER12> If you're interested in the subject, [https://www.amazon.com/100-Decisive- Battles-Ancient-Present/...](https://www.amazon.com/100-Decisive-Battles- Ancient-Present/dp/0195143663) is pretty fantastic.

"Anyone who clings to the historically untrue -- and thoroughly immoral -- doctrine that violence never settles anything I would advise to conjure up the ghosts of Napoleon Bonaparte and the Duke of Wellington and let them debate it. The ghost of Hitler would referee. Violence, naked force, has settled more issues in history than has any other factor, and the contrary opinion is wishful thinking at its worst. Breeds that forgot this basic truth have always paid for it with their lives and their freedoms."

\- Robert Heinlein

This quote has always stuck with me, and I posted it here a couple years ago. The reaction was almost universally negative. But after wising up a bit, I think offensive ideas need to be faced squarely if we're to evolve as a society.

I don't like it either. But I'd like it even less if it were a truth we conceal, rather than a falsehood we suppress.

It's hard to tell. But one thing is certain: If you read 100 Decisive Battles, you'll see the supremacy of intelligence in warfare. It matters more than any other factor. And the NSA would be our key way to get it.

<comment><USER14> Personally I'd think that in more modern times, as close as we've been to wars (e.g cold war times, which I'll admit predates me), there are probably countless times where frantic phone calls and backroom diplomacy (importantly, that you do not see) have saved everybody's skin from war (or nuclear war).

I'll admit, this is speculation, but I'd be seriously blown away if not true.

But yes, sometimes nations just have to battle it out. I'm just not one of those people.

<comment><USER15> Both of the instances you mention would almost certainly rely or result on, and from, what intelligence might be shared or derived from having that frantic phone call or backroom diplomacy.

It's not just who you know, but what intel you can share to strengthen an alliance that matters the most.

What troubles me about this article is the after effect on those who might be employed by nation states who seek to better understand our intelligence mechanism, and while I realize that every actor in the schema is a cog in a very complex machine, it stands to reason that if tons of people are leaving public service for the private sector, there is a vast amount of leaky intel out there for the having.

I don't think there is anything wrong with consolidation, necessarily, but we need to have stopgaps in place to quell the unwanted outflow of personnel from what is essentially a defense-centered brain trust because of how much we as a public have invested in their training and relied on them to help us stay out of harm's way. I'm especially worried about the younger set (not to foment discrimination, per se, but maturity does shape one's thinking) who may more easily be duped by some seemingly friendly industry that is a shell company for nefarious activity.

</comment></comment></comment></comment></comment></comment><comment><USER16> Not surprising. They've gone from having the perception of being the most elite organization for hackers to more of what is described in Good Will Hunting [1].

Perception is everything. I say this as someone who dreamt of working at the NSA for most of my high school years, but would now never even consider it.

Edit: although I bet a lot of it is the 4x-5x salary jump you can get in cyber security jobs by having the NSA on your resume. :-)

[1] [https://m.youtube.com/watch?v=mJHvSp9AKYg](https://m.youtube.com/watch?v=mJHvSp9AKYg)

<comment><USER17> I think it's become pretty clear recently -- looking at what Snowden leaked and at the trouble the Iranians had with their centrifuges or at the way Comey knew that Sessions was going to recuse himself -- that the NSA _is_ the most elite organization for hackers/crackers. It's hard to be certain from the outside, of course, but when I look at what Henninger et al have written about what they're doing... well, it's a breath-taking technical achievement

As for perception: These technologies exist, and it's up to us to make sure that they're used wisely. Right now, the institutions of the US and allied governments are the caretakers and safeguards. They're not perfect, but I'll take them over the competition every day of the week.

<comment><USER18> The NSA leaks we're at once impressive in their technical scope, but largely just things we knew we're possible if you threw engineers at the problem.

It's likely we didn't get the really juicy bits, but the vast majority of the stuff we learnt about was just operationalizing things we already knew.to be possible.

</comment></comment></comment><comment><USER19> Speaking from experience, I applied to the NSA a few years ago and its surreal how unrealistic the expectations were.

a major problem with NSA employment is the mandatory 5 year contractor like status period with half-pay. the NSA expects seasoned professionals from the private sector to join its ranks with a 33k salary? get real. The agency also prides itself on an almost obscene reliance on Polygraphs, a technology no more accurate or scientifically sound than a babushka shuffling chicken bones and reading tea leaves.

I also wasnt exactly blown away by the interview questions. my technical screening for a Linux position came from former members of the military, who grilled me on 2 unix questions before asking if they could switch to windows, confessing they 'didnt know much linux.'

<comment><USER20> > The agency also prides itself on an almost obscene reliance on Polygraphs, a > technology no more accurate or scientifically sound than a babushka > shuffling chicken bones and reading tea leaves.

I can't find out why I see smart people repeating this all the time. It's not true. Polygraphs are very flawed, and they are especially vulnerable to preparatory countermeasures, but they have non-trivial predictive power.

When I ask people to justify their dismissal of polygraphs, they generally point to the fact that they are not allowed in court, but this is a terrible reason. Hearsay isn't allowed in court either but that doesn't mean it isn't useful.

<comment><USER21> If it doesn't stand up to "beyond a reasonable doubt" how can you have such faith in it? Hearsay is a fine example, as I wouldn't want my innocence predicated upon someone's flawed memory or whim.

<comment><USER20> Because "beyond a reasonable doubt" is a much higher level of certainty than we have for making many decisions in life? The NSA isn't jailing people based on a lie-detector, they are just using it as a necessary but not sufficient criteria for granting security access. It's OK to deny such access based on weak or circumstantial evidence.

</comment></comment></comment></comment><comment><USER22> When I was just about to graduate from University, I had an offer from Amazon on the way (yay internships) but not really in-hand yet, so I kept interviewing. One of the options was CSEC, Canada's NSA.

The stark reality was that at Amazon, in 2 or 3 years I would be making double what CSEC was offering. Early career pay in one was end-career pay in the other. They seemed nice, and I suppose what they were doing felt _important_ , but the money was simply too good to turn down. (No regrets either- I worry if I'd gone with them, I might have wound up living in Russia with a roommate named Ed.)

<comment><USER23> It worked better for you and I'm glad to hear that. For some people, the stability of a Govt. job, the regularity of promotions, the lack of long term uncertainty and perhaps the absence of as high expectations makes it worthwhile to pursue a job that pays less.

<comment><USER22> Actually, this stability and security was another big reason I chose _not_ to join Government. I had done a few internships in the public service, and what I saw horrified me.

When _you_ have job security and low expectations placed on you, it's wonderful. When _everyone_ does, it's horrible. There is relatively little room for the lazy and useless in the private sector.

<comment><USER23> Again, I ask that you refrain from characterizing everyone as lazy and useless. I know a lot of women friends who choose to join Government service because of its long term stability which is important for them as they raise a family. They are some of the hardest working and smartest people I know.

Sure bad apples are harder to eliminate, but that doesn't mean everyone is bad.

<comment><USER22> I agree that my choice of words implies everyone is- and that is not true.

What I saw in government was a lot of dead weight adding little value and taking home paychecks, being carried by a small group of people working their asses off to keep everything afloat. The dead weight couldn't be fired short of them doing something crazy, like sneaking into the office over the weekend and drinking half the wine set aside for the Christmas party (so the story went of the only guy who got fired from that office). The heavy lifters felt unable to even take a vacation because they know nothing would get done if they left- example, a boss I had who wasn't sure she could go to her sister's wedding in Jamaica for two weeks because the office wouldn't manage without her.

My own wife is an engineer in municipal government. She's a superhero as far as I can tell. The dead weight on her team can't be fired, and she does many times the work they do for a smaller salary- they've been here longer, you see, so they make more money.

Perhaps my perspective on public sector jobs is skewed, but I've yet to hear a compelling reason I would want to have one.

</comment></comment></comment></comment></comment><comment><USER24> This is a problem across all of government. As a mediocre senior software engineer with no reports, I was paid more last year than the goddamn President of the United States (and no, it's not because I got lucky with stock). This is completely stupid and unacceptable.

Of course our choices are going to be Hillary vs Trump if we pay the top leader in the country less than a mediocre software engineer. Of course Hillary will give secret speeches to Wall St firms - how else is she going to pay her bills? Of course the government is going to be incompetent and corrupt - we can't hire good people at the salaries we're paying unless they use their influence to obtain side gigs in the private sector (often after their service, but that means they won't do anything to annoy their future bosses).

Why does Sundar Pichai make 500x more than the President? Do we, as a society, value running Google well 500x more than running our government well?

Government salaries need to be raised significantly (2-500x) across the board, or the government will continue being the mess it is now.

<comment><USER25> Presidential pay is not outrageous because those seeking the Presidency should be motivated by public service, not remuneration.

Of course, the President doesn't really have bills. He or she doesn't exactly pay to rent the White House.

<comment><USER26> It's true that public service is a major part, but another way of thinking about it is if government officials are highly paid, they are less likely to be bribed or act in their own financial self interest and more motivated by what is good for the country. Plus it attracts talent.

Singapore is an extreme example of this (perhaps too extreme) - the Prime Minister earns over $1.5M a year.

<comment><USER27> This Singapore comment needs to die before someone else ever repeats it again. Lee Hsien Loong (2004-now) is the son of Lee Kuan Yew (1965-1990) -- compensation has nothing to do with it in this case. Past a certain compensation range you are selecting for greed and nothing else -- if you earn in the top 10% and can't make do I seriously doubt you are 'intelligent' or your children must eat a lot and you have my sympathies.

</comment></comment></comment></comment><comment><USER28> > The brain drain has been so pronounced that at one gathering in 2016 of the > agency's elite hacking division, one individual raised the concern with > Rogers directly. According to several people familiar with the exchange, > Rogers disputed that there was any increase in attrition and told his > employees that they should stop complaining and get back to work.

Remember this. As someone who started their professional life in 2000, lived through 2009, and went through a multinational's "reorg" (mass-firing by geography, several 10s of thousands of people), let me tell you :

As soon as you notice there is general discontent, and people no longer care enough about their jobs to ask questions like this to the top brass, it is time to make sure you have another job lined up. You can get better jobs this way, and the hammer will come down.

Above all, know that this Rogers guy is fairly honest. And yet, he directly contradicts the Washington Post. He's an exec, and he has ONE job : do exactly what his superiors say without bother them.

He (once for me it was a she) will NOT give you any piece of information you do not already have. They are there to get people to work together.

By "motivating".

By lying.

By cheating.

This means that when it comes to the health of the company and your future job prospects, a sober chat with the cleaning staff will yield more useful information than an hour long "fireside chat" with a highly placed manager. Not because they know a lot, but simply because they don't lie, and they know a little. An executive is not talking to you, except in the way a vet is talking to stray dogs at the impound. They don't respect you, they can't do anything for you, they don't like you ... and they're very, very good at hiding this (just look at financial presentations on youtube, say of the IBM CEO, knowing that she has for years, every 2 months or so, fired 1000+ people, with the constant online reports of them getting cheated out of severance. Listen to her, then think about the kind of person who does that. You half expect to get free candy and a hug every 2 minutes or so when listening to her. Or listen to Clinton talking, knowing that he's the sort of man that threatens women to have sex with him. Think about what those people he talks to really mean to him)

<comment><USER29> I feel like many organizations, of which NSA is one example, make the mistake of putting technical program leads in charge of people. Leading people and motivating them to do their best work together requires a different skill set, and Rogers' response to the vocalized attrition concern suggests that he isn't capable of keeping people in the organization. I appreciate that he took time to talk with staff and take questions but he needs to become a better listener, or in other words ... work on his signals intelligence.

<comment><USER28> You misunderstood my post. No he doesn't need to listen. In fact, listening is the quickest way for him to lose his job. The fact that you even say this means you partially believe him, and that's a mistake on your part. The only signals intelligence this guy needs or wants is listening to his superiors, and his job is one-way data transmission from his boss to the rank and file. STRICTLY one-way (and trust me, it doesn't matter if his boss is Trump or Obama).

Of course he "talks" with staff and takes questions. Why ? Because despite his wages, he is not all that good at correctly transmitting information, so he is doing this so he might do his job better by having his team interpret his ideas, and try to clarify them to him by asking questions. This is done mostly to avoid the worst of screwups. Only questions that follow this pattern get answered, or even seriously considered (check those financial videos again, trust me you'll see this happen).

He does not have an opinion (I mean he does, but none that you'll ever hear). Remember that. You cannot ask him for it, you will never get an answer. And when you do, because there are amounts of alcohol that will make him divulge opinions, please don't start destroying careers (mostly your own, but you may drag others down) by telling other people what those opinions are.

He won't care about attrition until the ability of the organisation to carry out his boss's orders is in danger, or when his boss cares. Nothing other than that can make him care. But he's paid a LOT of money to make you think he cares.

<comment><USER29> Well, I agree with you about the one-way transmission. My point about listening is that I expect that a lot of their technical capability is initially developed through individual/team interest and not top-down direction so in the interest of advancing the organization's operational capabilities management should support two-way communication.

</comment></comment></comment></comment><comment><USER30> This combines a long running problem in government – not being allowed to offer competitive salaries for high-demand skills — with the new presidential administration's chaos, and the specific feeling that the agency is not a force for good. I feel bad for everyone trying to do their jobs but struggling under the weight of so much baggage.

<comment><USER31> > the specific feeling that the agency is not a force for good.

There are a mixture of feelings about the agency. The same can be said of the CIA and FBI. And this is nothing new.

<comment><USER30> Not new but accentuated: I think a lot of people used to know there was some gray area but thought we were generally on the right side (not unlike the U.S. attitude about torture until the Bush-era reversal), before the pattern of relentless disregard for the law and American interests really went public.

A lot more people are comfortable with, say, targeted foreign attacks than broad domestic dragnets or attacking American companies or standards.

<comment><USER31> > relentless disregard for the law and American interests really went public

Are you talking about metadata collection?

</comment></comment></comment></comment><comment><USER32> Disclaimer: I work for the federal government, but my thoughts are my own.

From what I have seen, there are a mix of issues that cause this: \- It takes a long time to hire someone (usually 2-3 months), and if they need a clearance, the process is even longer (some clearances can take easily over a year now, and that person cannot do any work until they have them). \- A lot of people want to go into these organizations to do cool things, but then find out they have a lot of additional duties they do not wish to do. \- Government salaries are capped, and it is very difficult to fire a government civilian.

The issue of taking a long time to hire someone means that if someone leaves, their job does not go away. So their duties have to go somewhere, usually on someone else, and that will take a minimum of 3 months if you hire someone off the street. If the job requires a clearance, you now are waiting over a year if the person did not have the right clearance (while the person sits on that spot, so you can't hire anyone else if they come up).

From what I have seen, most people hired off of the streets are from college/grad school, and they are excited to do technical work. However, since the government caps civilian salaries, they will frequently contract out technical work (so they can hire the very expensive technical personnel). This means most of the technical work is done by contractors. Civilians more often than not oversee the contractors doing the technical work. If you have a technically inclined government civilian, then those contractors frequently snatch them up.

So now you have a compounding problem of it is difficult to hire/keep technical people, and the work they do is more and more non-technical, as most of the work to oversee contractors is what is called an "inherently governmental function", so those have to be done by a government person. This is a negative feedback loop that really hurts keeping these types of technical people.

</comment><comment><USER33> NSA is in the middle of a bozo explosion. It's hard to keep good people when they have to deal with so many bozos.

To give you an example, there was a reorg a few years ago to outsource NSA's IT. (Makes a lot of sense that an agency that pretty much only does IT would outsource IT, right?)

In the reorg, 500+ people were identified as having no discernible skill that was relevant to the mission. They were segregated into their own org and given busy work. When the outsourcing occurred, the contractor was incentivized to employ and retain them for 2 years, including the 500+. To entice them, they paid them above their government salary.

The contractor could not figure out what to do with them either. It treated them as "casual employees". They were given a salary, but did not have to report to work and had no responsibilities. For 2 years.

Once the incentive ran out, these folks went back to NSA and got rehired as government employees. According to the hiring rules, they went right to the top of the stack.

</comment><comment><USER34> > The people who have left were responsible for collecting and analyzing the > intelligence that goes into the president's daily briefing

Cool, because the demand is gone too.

<comment><USER35> Exactly, right? Maybe they are not even angry about losing the analysts.

</comment></comment><comment><USER36> The State of California recently initiated a reclassification project for IT positions, which includes evaluating salaries. They last reclassified positions in 1976...back when Elvis was alive (note: salaries have risen modestly since that time, thankfully).

</comment><comment><USER37> Might it also be because of the cost of overcoming their conscience? I understand that there are multiple groups in the NSA that do things other than illegal surveillance and hacking, but I would have an even higher price than other companies' offers I would require in exchange for my moral values if the NSA ever offered me any position at all...

<comment><USER29> still less than the cost of someone overcoming their conscience for working at Facebook

<comment><USER38> Only on HN can you find people who think the NSA is worse than Facebook.

</comment></comment></comment><comment><USER39> Government work also suffers from over regulation. A friend of mine who works for a major defense contractor as an electrical engineer is literally not allowed to use a screwdriver driver. He must requisition a union worker for jobs involving a screwdriver. This takes tons of time and the simple job may ultimately never get done. In practice, he and his colleagues wait until after 5pm, when the union guys have gone, to use screwdrivers.

I told this story to my Uncle who worked for 30 years at a national lab. He said they had a similar problem and solution for using a broom to sweep up a mess in their work area. These stories sound unbelievable but I believe they are common and that situations like this regularly cripple and demoralize government work.

<comment><USER40> And this is why government projects are essentially doomed to fail versus capitalism, in the long-view.

</comment></comment><comment><USER41> _Rogers disputed that there was any increase in attrition and told his employees that they should stop complaining and get back to work._

Can't imagine why anybody would want to leave.

</comment><comment><USER42> By way of comparison, GCHQ personnel seem to be leaving for 4-5x salary from big tech companies[1]. So do the math and figure out what the GCHQ folks are being paid.

Also, GCHQ has hired 494 contractors for 71m GBP[2].

[1]: [https://sites.google.com/a/independent.gov.uk/isc/files/2016...](https://sites.google.com/a/independent.gov.uk/isc/files/2016-2017_ISC_AR.pdf?attredirects=1), page 40

[2]: Ibid, page 70

<comment><USER43> I live in Cheltenham and pre-Snowden may have entertained the idea for working there. I tracked a job they were advertising for nearly a year - they were offering ~32k for a Linux expert with years of experience. Good luck on that one. I think they eventually converted it to a contract role, but even then I wouldn't entertain it.

Another friend worked at the big James Bond building in London, was regularly taken around the world at short notice to do security things, and was on less than 40k. He loved that job more than anything in the world but literally couldn't afford to live there so went contracting for 5x the money. Apparently in the government a large part of salary reviews is based on (age + seniority), so you're waiting a long time for a decent uplift that may never arrive.

<comment><USER42> Just to be clear, your amounts are in GBP right?

<comment><USER43> Yup.

</comment></comment></comment></comment><comment><USER44> Question: how does this compare to other states?

I'm particularly interested in China's equivalent(s?) of the NSA. Do they manage to retain talent? Is it because the PRC's elite believes more in its mission? Is it higher status than the NSA? Do they offer higher relative salaries?

<comment><USER45> I think the best we can do is speculate on this, as China's cyber capabilities are still relatively unknown by the public [0]. However, based on their investment in academics and a generally more positive view of government jobs in general (possibly due to historic sociocultural contexts [1]), I believe it's likely that their equivalent groups are probably treated quite well.

For example, for successful US-trained Chinese researchers, the Chinese government offers generous funding, publication bonuses [2], and high pay (comparable to, or better than, US institutions) for those who are willing to return to China and run research labs there.

Additionally, due to the relative lack of commercial opportunities for talent, creative, and ambitious home-grown talent (vs the opportunities in the US available to similar US-trained talent), the bar the government must exceed is commensurately lower.

0\. [https://en.wikipedia.org/wiki/PLA_Unit_61398](https://en.wikipedia.org/wiki/PLA_Unit_61398)

1\. [https://en.wikipedia.org/wiki/Imperial_examination](https://en.wikipedia.org/wiki/Imperial_examination)

2\. [http://www.sciencemag.org/news/2017/08/cash-bonuses-peer- rev...](http://www.sciencemag.org/news/2017/08/cash-bonuses-peer-reviewed- papers-go-global)

</comment></comment><comment><USER46> Wouldn't worry too much. If they're heading to SV, their talents may still benefit the NSA.

<comment><USER47> Those fine people who have shown such a good jugement in choosing their previous job will be perfectly fine managing my google history and the cookies in my browser, I'm really looking forwards to it.

I'm really curious if they'll have the power to get more people killed at google or at the NSA.

</comment></comment><comment><USER48> I'll never forget talking to an NSA recruiter (cybersecurity expert doing a recruiting assignment, not a full-time recruiter) at a university job fair. I asked if he felt like he was really spending most of his time protecting people or if a lot of it was political agenda and budget protection, etc. His answer was, "well, it's not _all_ politics..." Thank you, recruiter, for making that career choice a supremely easy one.

</comment><comment><USER49> Might expect this to ultimately lead to privatization of these services. Which is black mirror territory.

</comment><comment><USER50> The top talent might be leaving, but the databases and the lovely toys are staying put.

To be used by not-top-talent.

Interesting times.

</comment><comment><USER51> Honest question: What's the skillset of a hacker at NSA? Not that I'm targeting to become one, just curious as to how someone would become a hacker.

</comment><comment><USER52> Why work for the NSA when you can overcharge Being a contractor for an equally inept organization

</comment><comment><USER53> The NSA needs educated smart people. The president has denigrated education and smart people. NSA employees work for the president. Reap what you sow.

<comment><USER35> With the speed of development in government organisation one could wonder if the current fallout is Trump's fault or Bush's faulth, though.

<comment><USER54> I think that you're missing someone in between Mr. Bush & President Trump …

<comment><USER35> I think I leave him out on purpose.

</comment></comment></comment></comment><comment><USER55> Or maybe anyone who understands cryptography enough to work at the NSA, understands bitcoin and blockchains enough to realize they can make a lot more money for a lot less effort in cryptocurrencies. Same with machine learning. The crypto bubble goes beyond money, it goes to creativity and brain drain from the rest of the economy.

<comment><USER56> Plus security. It's always been a skill in relatively high demand, but in the last ten years or so companies have been willing to pay whatever it takes to avoid getting hacked. My company brought in all sorts of consultants just after, coincidentally I'm sure, the Target CEO lost his job because of a hack.

</comment></comment><comment><USER57> Actually I remember reading at some point that the people who have the best careers in tech move between government and private at the right times, when there is going to be a downturn in the economy they move into the security of government work and work on the rules and regulations that will govern the future, as the economy turns back up they move into consulting regarding those particular rules and regulations.

This actually points to one of my major career mistakes in that when I left my government job I was burned out on the standard I had just spent some years seeing through to law, and instead of working consulting on it I took a totally unrelated job (based on the enticements of a friend who promised we would get to work on something I really wanted to work on)

I left millions on the table just by that one stupid decision. but then again I was really burned out on it.

on edit: improved formatting

</comment><comment><USER58> This is not a good thing at all. This will lead to a shittier NSA and with their power, this is a lousy combination,

</comment><comment><USER59> This isn't just NSA, but tech positions GOV wide. Lots of locations are requiring Contractors be put on an equivalent pay scale as the GOV employees, causing them both to look elsewhere... Why take a position with less stability and benefits, for the same, and in some cases less, pay.

</comment><comment><USER60> I wonder if there is also some people morale in this. I mean, if I was a hacker and knew what the NSA had been up to in regards of mass-surveilance, my integrity and morale would be like "heck no, thies org' fucked up and I want nothing to do with them, at all!".

</comment><comment><USER61> A lot of EU security experts (UK included) take time off to make extra money through companies like ours - senseiclub.com - to train corporate in anything from ethical hacking to certified GDPR compliance - they can't be making much at their regular jobs.

Some even register their own company - which makes sense - as freelancing full time, end of the day, is possible. Make your own schedule, decline to go to work, take a month off - it's good fun. We got some people booked for over 110 days out of 250 for 2018 already.

As for pensions and savings - folks go to wealth managers.

Hit me up if you want some really good training courses for your company or your buddy is looking for extra gigs (GCHQ folks are welcome).

Think productschool.com for cyber security, compliance, cloud and data.

Cheerio

</comment><comment><USER62> Seems like we should redirect some of the funding the traditional military branches get to the NSA. The work they do is more relevant and important than ever before, and it doesn't look like we're going to have a hot war any time soon.

</comment><comment><USER63> I'd suggest separating these anonymous complaints into those which at least have boolean choices and those which have unary choices.

 _Reorganization of dept. has created red tape._ Ok, that seems like one of many opinions a source at the NSA could have.

 _Snowden hurt our agency and made our jobs more difficult._ I'm not sure that an NSA source could have any other opinions on this matter, especially given that a) I've never read anyone who currently works at the NSA call Snowden anything but a traitor and b) the NSA certainly knows the identity of the sources for this story. I think we have to reject such complaints as facile.

</comment><comment><USER64> Another view: [http://nymag.com/daily/intelligencer/2017/12/who-is- reality-...](http://nymag.com/daily/intelligencer/2017/12/who-is-reality- winner.html)

That was circulated with the comment "the risk is not so much people running to the Russians as people running to civil society - the NSA is not as popular as they think."

The NSA, like much of the US system, demands absolute unconditional loyalty. Internal dissent is not possible. Taking dissent public will get you sent to prison where you're not allowed to speak to journalists.

<comment><USER35> replace prison with getting fired and you have the same in the industry.

<comment><USER23> Clearly you've never been to prison.

<comment><USER35> Clearly the point is not the comparison between the impact of both experiences on the receiving end.

</comment></comment></comment></comment><comment><USER65> I read somewhere that 70% of US intelligence spending these days was on private contracting companies anyway. I guess if you're good you'll end up at one of them?

</comment><comment><USER66> Finally some good news from NSA.

</comment><comment><USER28> > the nation's 17 spy agencies.

17.

WTF.

<comment><USER42> From Wikipedia[1]: CIA, NSA/CSS, NRO, DIA, NGA, intelligence elements of the five armed forces (Air Force, Army, Navy, Marines, and Coast Guard), and intelligence elements of the State Department, Department of Energy, DHS, Treasury, DEA, FBI, and finally overseeing them all, ODNI.

Each specializes in collecting a different kind of intelligence or in a different focus for analysis.

[1]: [https://en.wikipedia.org/wiki/United_States_Intelligence_Com...](https://en.wikipedia.org/wiki/United_States_Intelligence_Community)

<comment><USER67> Why do the DoE and Treasury need intelligence elements?

<comment><USER68> DoE handles the nuclear arsenal. Treasury contains the Secret Service.

</comment></comment></comment></comment><comment><USER69> Related: [https://www.recode.net/2017/5/8/15557846/stripe-peiter- mudge...](https://www.recode.net/2017/5/8/15557846/stripe-peiter-mudge-zatko- jon-kaltwasser-security-hire)

</comment><comment><USER70> I remember predicting this back when the Snowden first leaked, and someone popular on HN saying it wouldn't happen. I suspect the reorg and general politics outside NSA have had a bigger impact than "oh, spying is wrong" in popular culture, though.

</comment><comment><USER71> In this case, an inevitable outcome is likely the privatization of more of these industries. Let companies like Palantir take over the job and ensure we have some solid regulations to ensure Thiel can't read our emails.

</comment><comment><USER72> I've got an idea, stop giving Congress free health care and stop giving Congress a $100k a year pension after they've resigned. Bam, there's your competitive salary to offer NSA engineers.

<comment><USER73> wow thank you so much for your brilliant solution. considering that both of those require Congress to accomplish, do you have another brilliant idea for getting them to do it?

</comment></comment><comment><USER74> Now they really won't be able to analyze all that data...

</comment><comment><USER75> They end up going to private corporations like Booz Allen, then the government ends up contracting these companies in the private sector lol.

</comment><comment><USER76> Consider it has such bad reputation and public image, and subpar salary this is not surprising at all.

</comment><comment><USER77> The salary cap sounds like it will lead to more outsourcing, which would increase expenses overall.

</comment><comment><USER78> They probably hoped, if they work long enough for the man, they'll become the man.

</comment><comment><USER79> I would love to work for them, creative interesting projects with funding, pick me!

</comment><comment><USER80> What's wrong in leaving when somebody is paying better?

</comment><comment><USER81> >If the NSA is going to spy on us, wouldn't you like that to be as secure as possible?

I'd rather those perverts not peep on us; this isn't an unreasonable request.

</comment><comment><USER82> Don't hire these people!

</comment><comment><USER83> Good?

</comment><comment><USER84> with luck, mentioning NSA on your resume results in an immediate trip to the discard pile during applications at most tech companies.

if you're going to invade privacy on a massive scale, you better be ready for the consequences.

<comment><USER85> > if you're going to invade privacy on a massive scale, you better be ready > for the consequences.

Like a job at Google/Facebook, where you'll continue invading privacy on a massive scale?

<comment><USER86> I like Google and Facebook. They provide a lot of value to a lot of people, invest in R&D, and contribute to the open source community.

I've seen an uptick of negativity on HN lately, especially targeted at the big SV companies. Not intelligent criticism mind you, just basic trolling. For example your account appears to be 99% dedicated to shitting on SV companies.

<comment><USER87> > Google and Facebook .. provide a lot of value to a lot of people, invest in > R&D, and contribute to the open source community.

VS

> Facebook is attempting to kill all open communication protocols and force > everyone to communicate through their platform with powerful network > effects.

There's no contradiction here. Both can be true at the same time. Large organisations (government & commercial alike) have even less of a fixed & permanent nature than do individuals, and can carry out huge numbers of activities in parallel, some harmful, some beneficial.

You can try some sort of rough consequentialist calculus on an organisation's overall benefits / harms to society, which is complex and subjective to an extent that gives almost completely free play to prejudice.

Alternatively you can look at the systems within which organisations grow and assess the functioning of the whole thing. I think this is the more powerful approach, having more potential for sidelining empty moralising. Personally I believe that for fundamental reasons, hypergrowth-oriented technological society inevitably leads to the destruction of everything of value (in truth a subject for a book, not a forum comment). Facebook & Google are natural & unwitting agents of these fundamental forces. At the same time, decent & capable humans work for them in large numbers, so inevitably make positive countervailing contributions.

</comment></comment></comment></comment><comment><USER88> I'm hiring ;)

<comment><USER35> And who are you? Working for a hospital or something? [https://www.youtube.com/watch?v=-8gaoEQqoA8](https://www.youtube.com/watch?v=-8gaoEQqoA8)

Your website can't be reached btw.

<comment><USER88> I work on and advise several projects involving protection and cyber-security, specifically within the blockchain space.

</comment></comment></comment><comment><USER89> This is happening across the US government as a whole. The US State department is dead for instance.

When you have talent and options why stay with a creaky ship that's sailing straight into a hurricane with a mad captain at the helm?

</comment><comment><USER90> My favorite is that the more the FBI/NSA spends, the more government debt get printed. You may know that the government shutdown is because of a debt ceiling.

</comment><comment><USER91> LOL.

</comment><comment><USER92> good.

</comment><comment><USER93> Tis article reeks of Executive Branch propaganda from anonymous sources "according to current and former U.S. officials" etc etc

</comment><comment><USER94> Been the story since George bush. I wonder if they would have acted narrowly within the law if they would have had better org

</comment><comment><USER95> > The people who have left were responsible for collecting

> and analyzing the intelligence that goes into the

> president's daily briefing.

I seem to recall that the president's daily briefing is now a single page with pictures. How many people does it take to produce that?

<comment><USER96> Why is this getting down voted?

</comment></comment><comment><USER97> the CIA funded Washington Post sullying the good name of those noble domestic spying NSA'ers?

Faker News

<comment><USER98> Would you please stop posting unsubstantive comments here?

<comment><USER99> The only real problem I see is a lack of elaboration on some of the important little-known politics at play, here.</comment></comment></comment>


Nothing like this will be built again - <AUTHOR> http://www.antipope.org/charlie/blog-static/rants/nothing-like-this-will-be-buil.html

<comment><AUTHOR> A fascinating account of a special access tour of an operational Advanced Gas Cooled nuclear reactor (AGR).

[http://en.wikipedia.org/wiki/Advanced_gas- cooled_reactor](http://en.wikipedia.org/wiki/Advanced_gas-cooled_reactor)

[http://en.wikipedia.org/wiki/Torness_nuclear_power_station](http://en.wikipedia.org/wiki/Torness_nuclear_power_station)</comment>


The OPM data breach details you haven't seen yet - <AUTHOR> http://fcw.com/articles/2015/08/21/opm-breach-timeline.aspx <comment><AUTHOR> Government timeline shows the two data breaches were the result of one sustained attack effort.</comment>


Registration now open for Heroku's 2013 Waza Developer Event - <AUTHOR> https://waza.heroku.com/

<comment><USER2> is there some kind of animation of the purple tag at the top of the page that i'm missing? the svg image is only half visible, with a "string" above - <https://waza.heroku.com/images/layout/logo-waza.svg>

<comment><USER3> when you register it shows the string</comment></comment>


Revenue from the Google Play store will overtake Apple's app store in 2018 - <AUTHOR> http://qz.com/235992/revenue-from-the-google-play-store-will-overtake-apples-app-store-in-2018/

<comment><AUTHOR> If Clay Christensen is right, the lower income, underserved market that can't afford Apple products is open for Google to disrupt. IDC has forecasted an average smartphone selling price of $335 this year, which is nearly half the price of an iPhone 5s. As the iPhone price premium grows relative to Android, the differences between what Apple and Google offers to consumers and developers has narrowed significantly.</comment>


The Fellowship: Literary Lives of the Inklings (Tolkien,Lewis,Barfield,Williams) - <AUTHOR> https://audioboom.com/posts/4420047-the-fellowship-the-literary-lives-of-the-inklings-j-r-r-tolkien-c-s-lewis-philip-carol_zaleski-1-of-3 <comment><AUTHOR> These audio pages comprise a 3-part radio interview of authors Philip Zaleski and Carol Zaleski spanning a little over two hours. They talk about the background of these famous authors who formed their own literary/review circle that met once or twice a week:

J.R.R. Tolkien (The Hobbit, Lord of the Rings)

C.S. Lewis (the Narnia series, the space trilogy beginning with Out of the Silent Planet, and numerous Christian writings),

Owen Barfield,

and Charles Williams.

Parts 2 and 3 can be found under "Top Posts" to the right of the page, or direct links can be copied from here:

[https://audioboom.com/posts/4425071-the-fellowship-the- liter...](https://audioboom.com/posts/4425071-the-fellowship-the-literary- lives-of-the-inklings-j-r-r-tolkien-c-s-lewis-owen-barfield-charles-williams- by-philip-zaleski-and-carol-zaleski-part-2-of-3)

[https://audioboom.com/posts/4429994-the-fellowship-the- liter...](https://audioboom.com/posts/4429994-the-fellowship-the-literary- lives-of-the-inklings-j-r-r-tolkien-c-s-lewis-owen-barfield-charles-williams- by-philip-zaleski-and-carol-zaleski-part-3-of-3?playlist_direction=forward)</comment>


Migrating 23TB from S3 to B2 in 7 hours - <AUTHOR> https://nodecraft.com/blog/development/migrating-23tb-from-s3-to-b2-in-just-7-hours <comment><USER4> @ USD$0.05 cents per gigabyte out from S3, that's about USD $1150 to transfer 23TB is that right?

[https://aws.amazon.com/s3/pricing/](https://aws.amazon.com/s3/pricing/)

I'd be pretty nervous about hitting the delete button after the data transfer.

<comment><USER5> Would this crazy-ass strategy work?

1\. Spin up 50 AWS Lightsail instances[0] for parallelism

2\. In each instance download from S3 and upload to B2.

S3 to any AWS service in the same region is free[1]. $5 Lightsail instances come with 2TB of data transfers each, so 50 of them can easily handle 23TB. The whole transfer can be done within a few hours so the total computing cost is less than $10 ($5 / 30 * 50 = $8.3). Total data retrieval cost for S3 is ($0.0007 per GB) * 23,000GB = $16.1.

[0] [https://aws.amazon.com/lightsail](https://aws.amazon.com/lightsail)

[1] "Transfers between S3 buckets or from Amazon S3 to any service(s) within the same AWS Region are free." according to [https://aws.amazon.com/s3/pricing/](https://aws.amazon.com/s3/pricing/)

<comment><USER6> LightSail has (or at least had) a hard limit of 20 instances. They also have a soft limit of 2 instances, after which you must request an upgrade to a higher limit. I had to submit a support request explaining my intended usage. It took a week to get approved.

The stated reason for these limits is to avoid unexpectedly large bills. But I suspect that it's also to prevent crazy-ass strategies for getting around bandwidth costs.

</comment></comment></comment><comment><USER7> Great writeup on data migrations. I was wondering whether you did a comparison for this method vs using AWS snowball[1] to export S3 Data and B2 Fireball[2] to ingest it.

[1] - [https://docs.aws.amazon.com/snowball/latest/ug/create- export...](https://docs.aws.amazon.com/snowball/latest/ug/create-export-job- steps.html)

[2] - [https://help.backblaze.com/hc/en- us/articles/360001918654](https://help.backblaze.com/hc/en- us/articles/360001918654)

<comment><AUTHOR> We looked briefly at the snowball and fireball, but wanted to do this as quickly as possible, whilst keeping the process entirely transparent to our users. It was also an excuse for our team to get intimately familiar with the B2 API, since it's not compatible with S3.

If we were to consider another large migration like this, physical media would probably be the way to go.

<comment><USER8> I evaluated moving 2PB with snowball v. putting in 10g/100g links. The issue with snowball ( i started a company that did what snowball did and shut it down (failed )) and other fedex/RAID solutions is that you have 3 transfers. You think the LAN transfer will be quick - but you're generally rate-limited by the systems more than you are by the bandwidth-delay product. If you're in a high traffic DC area, it's pretty easy to get temp bw or install circuits to carry that. 10g for 2pB is 18 days of transfer - which sounds like a lot - <USER2> that's 5 days of transfer on each site, 1 day of setup, and 1 week of shipping. Those numbers aren't real but they're close.

So, snowball works in a lot of area, but like so many AWS products, it works if you adapt to it.

pigz/scp/zstd works extremely fast in line.

In your case you're pulling from S3 to another object store.

I moved ~1PB from one S3 region to another. "Why not use replication," they asked. That only works if it's turned on when you upload the object - <USER3> fine-print 'gotcha' in the easy AWS service. Then you get into rate-limits. In 2010 I asked AWS if I could spin up 1000 servers to test something - nope - elastiticy at that level is for the big boys.

Now I work for a large cloud company and we still run into elasticity.

To move the 1PB from one S3 region to another we spun up hundreds of spot instances (oh, we were compressing and glacierizing it too) and built a perl/mysql batch job "s3 get | zstd | s3 put" process and parallelized it. One thing nice about S3 is it pulls the md5 hash - unless multipart, in which case it's the hash of the hash, oh yeah.... So you should split it in advance if you want to verify the hash (more fine print).

Worked great. Good for you for sharing this project, very cool.

<comment> semi-extrinsic As a physicist I've always found the name "elastic scaling" funny. If it's elastic in the physical sense, it means that the energy required to grow to some size is quadratic (or higher) in the size. The marketing meaning is "easy scaling", but the physical meaning is "really hard scaling".

E.g. compare a soap bubble versus a bubble gum bubble. It's a lot easier to scale up the soap bubble, which is _not_ elastic.

<comment><USER9> It's a very good observation, and I think it's more than just a funny aside. The word 'elastic' connotes increasing resistance as the cluster grows, but this is a false intuition. From AWS's POV 'resistance' to adding a node is generally small, fixed, and, in general, independent of cluster size. I suspect this is what makes cloud computers in general, and EC2 in particular, such a cash-cow.

Moreover it turns out that elasticity is a very valuable quality of a cluster for most workloads; we _want_ this intuition to be true, that our cluster meets resistance as it grows, in the sense that it will shrink when the workload decreases. This matches our economic intuition, too. We want this so much we have to build another software layer to make this happen - e.g., k8s.

</comment></comment></comment></comment></comment><comment><USER10> I assume I'm not the only one who has never heard of B2 before now, so I'll summarize everything I've found out:

[https://www.backblaze.com/b2/cloud-storage- pricing.html](https://www.backblaze.com/b2/cloud-storage-pricing.html)

It appears to be an S3 clone. Competing via lower price and less micro charges. Highlights compared to Digital Ocean and Wasabi:

DO/Wasabi: minimum $5/month, but great deals compared to AWS/GCE otherwise.

B2: First 10GB storage free (probably not including bandwidth).

For a side project or startup looking for its first storage option, B2 seems compelling. But something important: is it a drop in replacement? Is the api accessible on your platform?

[https://help.backblaze.com/hc/en-us/articles/218513487-Is- th...](https://help.backblaze.com/hc/en-us/articles/218513487-Is-the-B2-Cloud- Storage-API-Compatible-with-Amazon-S3-?mobile_site=true)

I don't have a clear answer right now, but when I get closer to deploying my current side project, it's something I'll take a deeper look into.

<comment><USER11> Their API is _not_ S3 compatible. There are a bunch of client libraries on various platforms for it though.

They have 1GB free of outgoing bandwidth per day. If you put them behind Cloudflare then you effectively get your bandwidth for free. So half a cent per GB per month is all you way. Plus your time building out the integration instead of S3.

</comment></comment><comment><USER12> I'm guessing the payback time on this is fairly long by the time you factor in the cost of moving all the backups. S3 One Zone is pretty competitive against Backblaze.

Cloudflare is betting on most of their customers serving HTML not large uncachable blobs, that "Bandwidth Alliance" will disappear pretty quickly at any sort of scale.

<comment><AUTHOR> I can't speak for Cloudflare, but we've talked to them pretty extensively about the whole project, and even did a case study about our use-case with the bandwidth alliance, and as we switched cloud providers. Things may change in the future of course, but they very much encouraged what we were doing. [https://www.cloudflare.com/case-studies/nodecraft- bandwidth-...](https://www.cloudflare.com/case-studies/nodecraft-bandwidth- alliance/)

</comment></comment><comment><USER13> Would you consider open sourcing the micro service you wrote to perform the migration? I could see it being helpful to others interested in migrating from S3 to B2.

<comment><USER13> This request can be disregarded. I'm going to explore extending s3proxy for the same purpose (migration and backfill of disparate object storage systems through an abstraction layer).

</comment></comment><comment><USER14> Did you compare it against Wasabi?

<comment><AUTHOR> We did not. Wasabi looks interesting - thanks!

<comment><USER15> We have about 125TB on Wasabi. Costs about 650/mo. Their performance for large files is great.

<comment><USER16> Haven't they had stability issues? I know it was troublesome for some Mastodon instances.

<comment><USER15> Once so far in last 8mo. It wasn't a big issue in our use case, especially since there was no data loss.

</comment></comment></comment></comment></comment><comment><USER17> If this is purely backup data, wouldn't glacier be a better fit than either S3 or B2?

Glacier is already cheaper than B2 and has the advantage of storing data redundantly across datacenters. And glacier deep archive is 4 times cheaper than _that_.

Full disclosure: AWS employee

<comment><USER18> No - the pricing on glacier makes it a steaming pile of shit

<comment><USER17> It's built for off-site backup type workloads, where you write once and read hopefully never.

You are probably using it wrong.

</comment></comment></comment><comment><USER19> Wouldn't it be simpler if you gradually moved? [https://martinfowler.com/bliki/ParallelChange.html](https://martinfowler.com/bliki/ParallelChange.html)

I understand your main concern for moving was pricing. Developer hours also cost money. It seems like you had to invest much more developer hours than you would have if you gradually moved over the course of one month or so. (probably a week?)

</comment><comment><USER20> Are they still one data center only?

<comment><USER21> It seems that they have two DCs now, not sure if the second is used for redundancy though.

</comment></comment><comment><USER22> I mean, the best way to migrate this huge amount of data was still has had to be using a physical migration service like AWS Snowmobile[0], right?

[0]: [https://aws.amazon.com/snowmobile/](https://aws.amazon.com/snowmobile/)

</comment><comment><USER23> Can I ask why ZIP? Isn't it quite heavy on CPU at a not-that-good compression rate? I'm thinking of LZ4 or Zstandard instead.

<comment><AUTHOR> We had to pick something that was "good enough" for compression time/size, as well as easy for our customers to download/view if they wish, on any OS. Zip being supported in every popular operating system, and the average user using Windows being able to right click -> unzip, was the primary reason for choice.

There are of course significantly faster and more efficient compression formats like LZ4 which would be ideal if we were solely using the data internally in managed environments, but we offer these backups as downloads to our users, some of which aren't very technically inclined and still need to be able to access the files easily.

</comment></comment><comment><USER24> The stated reason for moving to B2 was pricing. No breakdown of cost was given however. I'm not sure how they arrived at that conclusion.

</comment><comment><USER8> What's the monthly storage cost on that?

<comment><USER25> $115.

</comment></comment><comment><USER26> I've heard Backblaze has only one region. Is that true?

<comment><USER27> They said they use Cloudflare as well...

<comment><AUTHOR> We do use Cloudflare, but a lot of the instance backups we store are multiple GBs - Cloudflare doesn't cache those. Not to mention uploads from regions like Singapore can be very slow all the way to the US.

It's not a deal-breaker for us, but we're very much looking forward to when they can support more regions.</comment></comment></comment>


Show HN:Bitcoin parody site - we use carrots. - <AUTHOR> http://weusecarrots.com

<comment><USER2> Carrots also have intrinsic value because they are directly useful: should I get hungry, I can eat them.

<comment><AUTHOR> You can also feed your alpacas with them.

</comment></comment><comment><USER3> As it's not immediately obvious, the original site being parodied: <http://www.weusecoins.com/>

<comment><AUTHOR> that is the one. We added the link for you.

</comment></comment><comment><USER4> I have to say, I appreciate a 'financial' community with a sense of humor.</comment>


Ask HN: How Would You Teach Kids Computer Science and Engineering Full Time? - <AUTHOR>

I would like to do this for a living. Kids get lessons in piano, sports, etc... and I think a lot of parents would pay to have someone teach their kids how to use computers for more than checking Facebook. I also think back when I was 6 years old I would have LOVED to have someone who could have taught me how to do more with PCs than play games on them. I feel like even teaching them the very basics (simple Python, HTML) would give kids such a huge advantage when it comes to starting a career and finding good paying work in the future. You could sell it to the parents as teaching a vital skill, and to kids in any number of ways. Make an Arduino part of your lesson fee and let the kids keep a robot they built, etc... So how do you think I could make this pipe dream a reality and quit my day job? <comment><USER2> The only advice I can give is to share my story.

I started my son at five with MIT's Scratch [1] followed up by Lego Mindstorms [2].

I then created a text adventure engine [3] that could run adventures written in a very simple Lua based DSL so he could make a simple text game [4]. Mom helped with spelling and grammar and I suggested a puzzle but he wrote the adventure himself.

Then I transitioned him to Lua on the iPad with Codea [5] and he (with a little help from me) made a game called StarFighter [6].

I choose Lua because it is a great intro language. Very simple with minimalistic syntax, few concepts, few primitives, few keywords, dynamically- typed / garbage collected, variable arity, no real gotchas, good tool support, great speed, the ability to access a key in a hash either using bracket or dotted notation, and one of the best programming books for any language (PiL).

But it also scales well; closures, first-class / true anonymous functions, metatables / metamethods, nice simple API for talking between script and C, tail-call optimization, coroutines, short circuiting operators.

And, the path from Lua to JavaScript is very straightforward. In fact I'd say that mastery of Lua would make you a mid-level JS programmer right out of the box...

The next step was JavaScript and ImpactJS [7]. I got him two books [8][9] and he loved them.

He is now doing HTML/CSS/JS and is in the middle of a fantastic book called Pro Javascript [10]. Once he is done with that book I am going to consider him on his own...

But just to give you an idea; he is 11 now and I am learning Scala for work. He's been watching videos with me and I paused it and ask questions and then I asked him where he thinks the presenter is going to go. He ran to his Ubuntu desktop and apt-get'ed Scala and cranked up a REPL session and showed me; he was correct!

\----

[1]: [http://scratch.mit.edu/](http://scratch.mit.edu/)

[2]: [http://mindstorms.lego.com/en- us/default.aspx](http://mindstorms.lego.com/en-us/default.aspx)

[3]: [https://github.com/shawndumas/adventure.lua](https://github.com/shawndumas/adventure.lua)

[4]: [https://github.com/shawndumas/adventure.lua/blob/master/theT...](https://github.com/shawndumas/adventure.lua/blob/master/theTenthFreighter.lua)

[5]: [http://twolivesleft.com/Codea/](http://twolivesleft.com/Codea/)

[6]: [https://gist.github.com/shawndumas/2762088](https://gist.github.com/shawndumas/2762088)

[7]: [http://impactjs.com/](http://impactjs.com/)

[8]: [http://www.amazon.com/HTML5-Game-Development-ImpactJS- Cielen...](http://www.amazon.com/HTML5-Game-Development-ImpactJS- Cielen/dp/1849694567/ref=sr_1_1?ie=UTF8&qid=1374590779&sr=8-1&keywords=impactjs)

[9]: [http://www.amazon.com/Building-HTML5-Games-ImpactJS- Introduc...](http://www.amazon.com/Building-HTML5-Games-ImpactJS- Introduction/dp/1449315178/ref=sr_1_2?ie=UTF8&qid=1374590779&sr=8-2&keywords=impactjs)

[10]: [http://www.amazon.com/Professional-JavaScript-Developers- Wro...](http://www.amazon.com/Professional-JavaScript-Developers-Wrox- Guides/dp/0764579088)

<comment><USER3> This is awesome. My girlfriend and I were just talking yesterday about our 2-year-old and how I want to write games with him once he's old enough.

I begged my parents for LEGO Mindstorms when I was a kid but we couldn't afford it. I'm hoping that he'll be as interested in learning to code as I was when I was young.

</comment></comment><comment><USER4> You've rubbed me the wrong way. I'm sorry if what follows hurts your feelings.

Kids should learn how to program because they're interested, not because you want to quit your day job.

Marketing a camp for kids as a "job skill" is completely disgusting. It's like you're giving baseball lessons to kids while promising the parents that it makes kids more likely to play in the MLB later on in life. It's crass and insincere.

Kids like to play, have fun and be happy, not sit in a room and learn HTML. A real "benefit" you could provide is a safe haven for gifted kids who don't have the benefit of rich parents or even a caring school district, centered around programming, that markets itself as a "special place" for smart, sensitive kids who need that environment. Fill it with lots of cool science toys, challenging books and make it a happy place. Some gifted kids go home to a scary, dark place, and you could provide a haven for them, instead of setting unrealistic expectations for overbearing soccer moms.

"So how do you think I could make this pipe dream a reality and quit my day job?"

If it's such a pipe dream, remove your head from your ass and figure it out yourself. This is advice from the bottom of my heart to you. Finding your own "path" is a struggle, especially if you want to blow the system and do your own thing. Asking this question is a bad first step.

It's not a "pipe dream" though - you just don't want to work 8 hours a day and will market to pushy parents who want their kids to become startup millionaires to do it.

Working with kids takes a special kind of love, patience and understanding. They are very impressionable, and gifted kids are extraordinarily sensitive, speaking from the experience of being one. If you want to work with them just to make money, do the world a favor and find another scheme (there are lots of ideas to make money and carve your way in the world). If you'd like to talk about how to nurture bright kids who are interested in programming, you'll find a different type of commenter coming out of the woodwork, and I'll be happy to give you my perspective.

<comment><AUTHOR> Heh, no hurt feelings here and I don't think you really needed to make a throwaway account just to play devil's advocate. I guess "how do I quit my day job?" sounds a bit off when you put it like that. I recognize that most good teachers work more than 40 hours a week and I wouldn't mind doing that. I didn't bring this up yet, but my wife is a social worker and I admire her for what she does every day, helping people who really need it. I just feel like if I could teach kids advanced computer topics it would be a better (subjective) use of my talent than working directly or indirectly for the military. If I just wanted money the path of least resistance is right in front of me. I could leverage this internship for any entry level software engineering job and at a lot of companies / locations be making plenty to provide for my family. I just wouldn't be happy.

<comment><USER4> By the way, if I were a highschool student, I would totally take a programming class in this vein, so don't take me the wrong way.

May I suggest age-segmented lessons? Pretty pictures for a little kids who want to play with it, and a focused curriculum for older kids?

<comment><AUTHOR> Of course, you can't teach 6 year olds the same way you teach 12 year olds, their brains aren't even fully formed yet. That was already a given ;)

</comment></comment></comment></comment><comment><USER5> I lived in NYC in the mid-late 90's, and people who were good at teaching math were getting ~$80/hour tutoring kids, and helping them prep for the SAT. If you live in a place that could support it, I imagine you could get a decent rate tutoring kids interested in programming. It's an interesting thought, with parallels to developing a freelancing/ consulting business.

I was never keen on the idea, though. The thought of only helping kids whose parents are rich enough to pay $80/hr felt like I was just widening the already-deep achievement gap. If you have any of this sentiment, you might try to put together a cohort of four kids, and charge each $20/hr.

<comment><AUTHOR> I actually am about to send an email to the local Youth Council of a city near mine for exactly this reason. I am going to ask them if they'd be interested in sponsoring me so that any kid who is interested could learn.

</comment></comment><comment><USER6> I think since I'm in high school, I can shed some light on the kind of demand there is for what you're describing. Generally, parents would be happy to sign their kids up for an intro programming class. When I was in elementary school I certainly did at least a few, ranging from a robotics focused one, to one specifically with UDK for games.

I'm not sure how the classes are marketed for parents, but to kids, the thought of building your own robot, or creating your own game is very exciting. Obviously, the reality of the class is that instructors end up spending the majority of their time working out technical issues and helping the kids whose parents signed them up for something they have no interest in. It happens to be much like a typical classroom. I didn't end up learning nearly as much as I wanted from any of the classes, and spent most of the time working ahead with little to no supervision.

I think the main thing you have to decide if you really want to do this, is whether you want to teach the kids who have the interest, and the basic level of experience, or the ones whose parents are looking for a good way to spend a few weeks. It sounds like you, like me, fall into the former group. If you're really passionate about what you're teaching, it will be a much better experience for everybody involved if you can get those kids. I can't offer much more advice than that, only being in high school, but I'd be happy to answer any other questions (or try to answer for my parents).

As somewhat of a disclaimer, I'm not sure where you live, but where I am (Northern Virginia) _many_ of these kinds of classes/camps offered after school, or during the summer. Of course, this has to do with the region, so take that with a grain of salt.

<comment><AUTHOR> I can't say I'm surprised by your experience, which is why I figured at most I would only teach a group of 5-6 kids at a time. Then they would each get individual attention while still being able to work together to solve problems, and I wouldn't have 10-20 Arduinos to debug at once.

</comment></comment><comment><USER7> Teaching is a catering gig. You must set out an assortment of foods and hope that the majority of the plate appeals to a palette that isn't your own. Children are finicky eaters. They lack the vocabulary and experience to tell you when a flavor isn't working for them. You have to puzzle out the situation, which oft times looks suspiciously like boredom or defiance, and seamlessly redirect to another dish. In many ways, adults are easier students. Cooking for someone excited about eating is a joy.

Start teaching by teaching. Approach your local [Parks and Rec Department, Adult Ed program, homeschool group, SBA/SCORE office, science museum] about offering a short intro class. Be modest in your proposal but be prepared to exceed the lessons outlined. You just don't know your student base yet. Different programs will bring in different kinds of people. Experience will allow you to dial in your favorite student type as well as your methods and materials before going independent. You may even keep these gigs as a steady source of new private students and cash. Try it out.

(As an aside: Six year olds are still learning basic grammar structures and logic. Many still need physical objects to perform basic math equations. Even those who appear to grasp abstract concepts are often "looking" at objects or example situations in their mind's eye. Programming is a stretch at this age. Isla succeeds because offers immediate and "tangible" feedback and works using a basic adjective-noun structure young children understand. Scratch is a steep step up from Isla. Probably age eight would be the soonest a typical child would enjoy playing with the language without an adult sitting next to them helping.)

</comment><comment><USER8> I think a computer-themed after-school care program plus summer camps is a good idea. As another poster mentioned, however, you would end up with the problem of teaching a bunch of kids that are only there because their parents work and need day care. My daughter is going through this right now. She is in an electronics camp for the week. On the plus side the counselors love her because she is one of the only kids in the class that pay attention. On the downside, she is one of the only people in the class that pay attention.

Alternatively, I've heard of franchises that market to existing daycares to come in once a week to do computer classes, charging the parents a nominal fee per session.

I'm very interested in hearing other people's ideas, this is something I've looked into and would be interested in doing as well.

<comment><AUTHOR> I actually stopped my own daycare's director today and she brought this exact approach up. She is going to try to get me in contact with a group of other instructors that do this at her location.

</comment></comment><comment><USER9> Well summer is almost over now, but I would think starting off doing a summer day camp would be best. So your biggest problems there are that you need a location and legal protection. If the local University doesn't offer something like it, you could take to someone in the CS department and pitch your idea. They'd also have some of the resources you'd need like a computer lab, and then they would also do a good deal of the administrivia (fliers, insurance, etc.)If it has good results you might be able to convince the uni to take you on as some sort of employee. Best of luck!

</comment><comment><USER10> I think that Lego Mindstorms are a great intro, and that's something parents can wrap their heads around and "sell" to their children. We have a franchise company down the street from me that does summer camps built around Mindstorms - [http://www.bricks4kidz.com/](http://www.bricks4kidz.com/). It seems to be a fairly large franchise also. My 9yo son is taking their "Introduction to Game Programming" weekly camp/class in a few weeks. Might be worth looking at their curriculum/marketing for ideas.

<comment><USER11> Mindstorms used to be great, but compared to Arduinos and the whole ecosystem that sprang up around them, it is just too expensive, too proprietary and too limited. Here is one example: [http://www.seeedstudio.com/depot/shield- bot-p-1380.html](http://www.seeedstudio.com/depot/shield-bot-p-1380.html) a pre-assembled $69 robot where you plug an ordinary Arduino Uno or compatible board, with a chassis full of possibilities for expansion using off-the-shelf components. For block-oriented programming there is now Scratch which is free software and works with Picoboards (also open hardware) and Arduinos.

<comment><AUTHOR> Yeah, there are just so many possibilities now. There is something wrong with your link, though.

</comment></comment></comment><comment><USER12> My son (6yo) is very interested in learning programming and I don't have the skills to help him advance beyond Scratch. If you would be interested in teaching him remotely, pls email me at gthomps at gmail. Thanks!

<comment><AUTHOR> I am flattered by your interest, but I really am more interested in teaching kids in person, and with teaching aids like robotics kits that I can inspect for problems when the kids need help. I think kids that young need something they can hold in their hands to make it all seem more real, though I do not have a degree in childhood education to back up that opinion ;)

</comment></comment><comment><USER13> I really love this idea. I had always wanted to go back to my middle school / high school and try to get the school to allow me to run a programming course on the weekends.</comment>


A Superb Weather Site – Free and Donation Supported - <AUTHOR> http://windy.com <comment><USER2> Is it possible to simulate or go back in earlier days, maybe last 2 weeks? Best site ever, i personaly have seen for my special interests in weatherview. Very good visualised for a good perception. Easy to use. I love the differnt possibilities to view at and the option to personalize it. Great work, let me say thank you.</comment>


Announcing NodeConf London 2016 - <AUTHOR> Please join us on May 11th, 2016 at the Barbican Center in the heart of London for all things Node.js.  We have worked hard to put together the best lineup we can, to have a great day and would love some support<p>Visit http://london.nodeconf.com <comment><USER2> Clickable -> [http://london.nodeconf.com/](http://london.nodeconf.com/)

<comment><AUTHOR> hey - that's a good idea :-)

</comment></comment><comment><USER3> Love to be able to attend, unfortunately kind of hard to fit in the schedule. Side question: Is wednesday a good day for a conference?

<comment><AUTHOR> Hey leichtgewicht,

Its a fair question. It's unknown right now, we thought long and hard about the day so lets see if its a success.

Ian

</comment></comment><comment><USER4> I know that a lot of effort has been put into creating the line-up of talks on offer - should be a great conf!

<comment><AUTHOR> Thanks funjs7, really looking forward to watch the flow of the day.

</comment></comment><comment><USER5> Thanks ian crowther and nodeconf london team for the hard work, you are really taking it to the next level.

<comment><AUTHOR> Thanks ismnoiet, that's lovely feedback!

</comment></comment><comment><USER6> Awesome!</comment>


.NET Foundation Elections - <AUTHOR> https://election.dotnetfoundation.org/ <comment><AUTHOR> The .NET Foundation is holding elections for is Board of Directors from 3/21-3/28.

While only .NET Foundation members can vote in this election, anyone who has a question for the candidates about the .NET Foundation, its mission, or their proposed plans to submit a question at: [https://election.dotnetfoundation.org/questions](https://election.dotnetfoundation.org/questions).

Candidate statements can be found at: [https://election.dotnetfoundation.org/candidates](https://election.dotnetfoundation.org/candidates)

I hope folks will join the .NET Foundation in widening its membership / leadership and looking for new ideas to build a more vibrant community the .NET ecosystem.

(Disclosure: I am one of the candidates running in this election)</comment>


Firefox: HTTPS and response code 407 - <AUTHOR> http://lcamtuf.blogspot.com/2013/02/firefox-https-and-response-code-407.html <comment><USER2> For those that were wondering, HTTP 407 is "Proxy Authentication Required".

</comment><comment><USER3> I don't quite understand. If you already went to the trouble to set up a rouge SSL proxy on a network, can't you just silently inject content into the original response that would then run in the same origin context?

Surely I'm misunderstanding the meaning of SSL proxy. Can someone explain how such a thing works.

<comment><USER3> After a bit more reading I realize where the misunderstanding is. The proxy is just a regular HTTP proxy. You coerce the client into using it using some sort of forced auto-discovery protocol which would be transparent to the user. Then the 407 trick makes sense to get JavaScript to execute in the "secure" page's context, since an HTTP proxy wouldn't have access to the response content of a request made over HTTPS.

</comment></comment><comment><USER4> I have to ask, since I like playing with HTTP 418

HTTP/1.0 407 Boink

What is up with the Boink here?

<comment><USER5> The text after the code doesn't actually matter. It's just there for human consumption.

<comment><USER4> Ah thank you. Not particularly surprising since I've never seen anything beyond the status code checked.</comment></comment></comment>


Google: Material Components for the Web - <AUTHOR> https://material.io/components/web <comment><USER2> Looks pretty awesome! Will for sure need to read the documentation. Anyone know how stable it is?

</comment><comment><USER3> The link doesn't seem to be opening. Or is it?</comment>


ABCL – Common Lisp on the JVM - <AUTHOR> https://common-lisp.net/project/armedbear/ <comment><USER3> I poked around a bit to figure out why this was even posted and there was a new release 1.6.0 on November 22nd which adds support for jdk 11, among other minor things. ABCL provides a java class which can load/eval lisp code, and the ability to call java classes from your lisp program, stuff like that.

[https://abcl.org/svn/tags/1.6.0/CHANGES](https://abcl.org/svn/tags/1.6.0/CHANGES)

As an "old" I learned LISP in college but have never had the opportunity to use it in my career. Still on my todo list!

<comment><USER4> Don't do it, you'll hate every other language after that (I do).

<comment><USER5> I like LISPs conceptually. However, I have a strong preference for static type systems, so I have probably written 10x more lines of code of LISP implementations than LISP code itself.

<comment><USER6> Racket has Typed Racket, you might want to try that out!

<comment><USER7> Have you tried it yourself though? That thing is so slow it's barely usable.

<comment><USER6> Is it the type checking or the runtime that's slow?

</comment></comment></comment></comment></comment></comment><comment><USER8> A well-established implementation, but there haven't been many HN threads.

Just this from 2009: [https://news.ycombinator.com/item?id=930514](https://news.ycombinator.com/item?id=930514)

and a bit from 2017: [https://news.ycombinator.com/item?id=14605334](https://news.ycombinator.com/item?id=14605334)

<comment><USER9> It's also mostly a full implementation of the Common Lisp standard with extensions, which is quite an achievement.

</comment></comment><comment><USER10> I'm amazed it literally has a class

        public final class Cons extends LispObject
        {
          public LispObject car;
          public LispObject cdr;
        }

I would have thought it would provide that abstraction to the programmer but use another representation internally. Chasing pointers like this on modern hardware is surely crazy?

<comment><USER11> A lot of things in Lisp force the implementation of lists as cons cells. It's probably possible to do something different, but no one has, and the result won't be a Common Lisp.

(Consider that the detail of improper lists is enshrined in the language.)

<comment><USER12> Common Lisp does have proper arrays as well, which can be used if the situation calls for it.

As it turns out, arrays aren't actually used that much (except for strings of course) in CL code. Most lists are just a few elements long, and regular lists are perfect for that.

<comment><USER11> The structure of a list is still exposed, and that decision is baked into the essence of Lisp. If you have a list, you cannot substitute a skip list or an array. If you have a skip list, it is (I think) not a language-level sequence and so things like rest and nth won't work on it.

Arrays, struts, all other things are possible, but the things that make Lisp Lisp mean that you may get a list where repeatedly calling cdr may result in something that is not listp (cons or nil). And that's weird. It enables some things that are wonderful, but I think the Lisps are practically unique in conflating pairs and list elements.

<comment><USER11> I bet CL allows generalizing nth and cdr. My point is still that the list structure in Lisp is very baked into the language.

<comment><USER9> Common Lisp has a standard abstract data structure called 'sequence' and generic operations for it. Sequences are either vectors or lists. Vectors are bit vectors, strings, general vectors, ... Sequence operations are then supported for all subtypes of sequences.

[http://www.lispworks.com/documentation/HyperSpec/Body/c_sequ...](http://www.lispworks.com/documentation/HyperSpec/Body/c_sequen.htm)

There are some problems with this design. The main motivations for it were:

* basic operations in Lisp are type specific, one wanted to have some generic operations and a generic sequence abstraction over lists and vectors

* the low-level data structures like conses, arrays, tables are not object-oriented and generic

* generic operations in a dynamically typed language are slower, since there is always a runtime dispatch necessary

* to remove runtime dispatch one can use type declarations (which are optional in Common Lisp) or a JIT compiler (which is not used in most implementations, they do AOT compilation)

* the language at the type of this design had no agreed OOP extension - CLOS came some years later -> sequences are non-extensible and don't support adding new sequence types

The design of this was later improved in languages like Dylan: see the Collections of Dylan

[http://lispm.de/docs/prefix- dylan/book.annotated/ch12.html](http://lispm.de/docs/prefix- dylan/book.annotated/ch12.html)

Recently some CL implementations experiment with extensible sequences.

[http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-...](http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf)

</comment></comment></comment></comment></comment></comment><comment><USER13> Besides clojure, might be worth noting;

kawak - scheme for the JVM [https://www.gnu.org/software/kawa/](https://www.gnu.org/software/kawa/)

bigloo is a scheme can compile to C. I think it can also make java class files with bindings [https://www-sop.inria.fr/mimosa/fp/Bigloo/](https://www- sop.inria.fr/mimosa/fp/Bigloo/)

<comment><USER14> Just "Kawa" \- not "kawak".

(There is actually a very-incomplete Common Lisp implementation as well in Kawa, but the Kawa Scheme dialect is the main Kawa language.)

(Hoping to make an official Kawa 3.1 release by the end of the year - but no promises. Until then, use git HEAD.)

</comment></comment><comment><USER15> The title of this post should be changed to "Armed Bear Common Lisp - <USER2> Lisp on the JVM." It's an iconic name, and then people would actually know what the post title was referring to.

<comment><USER16> I guessed it was Aluminium Bank Common Lisp, some kind of lightweight version of Steel Bank Common Lisp.

<comment><USER17> That's actually a nice pun with a good explanation, while Armed Bear only raises questions for me. What kind of weapon would you arm a bear with? Why? Aren't bears unassailable enough as it is?

<comment><USER16> Bears are most commonly armed with 87.6 mm artillery:

[https://en.wikipedia.org/wiki/Wojtek_(bear)](https://en.wikipedia.org/wiki/Wojtek_\(bear\))

It seems it's the author's brand name for software, and is a pun on the US constitution's second amendment:

[http://armedbear-j.sourceforge.net/](http://armedbear-j.sourceforge.net/)

We may thus infer that you would arm bears because of the threat of armed bears. After all, to stop a bad bear with a gun, it takes a good bear with a gun.

</comment></comment></comment></comment><comment><USER18> A bit off topic, but is Common Lisp still a good choice?

I spent a lot of time learning and coding CL during the last decade. I love lots of things about the language. Overall, it's great it's true multiparadigm. Few languages achieve that.

However, the implementations are too fragmented which makes the standard quite stagnant. And libraries are falling a bit behind.

I love Clasp is putting CL on top of LLVM. I wish we could have something with the best ideas of Racket, Clojure and CL on the LLVM.

<comment><USER19> >A bit off topic, but is Common Lisp still a good choice?

As always, it depends on your application. For instance I decided recently that I would work my way through the cryptopals challenges, and opted to avoid Lisp since bit-twiddling is such a pain in that language.

<comment><USER20> Since bits are addressable directly in Common Lisp, I found the language to be very very nice for bit twiddling. Maybe you're mixing it up with Clojure? (bit twiddling in clojure without extra libs is a PITA).

</comment></comment></comment><comment><USER21> What would be the best way to get (free) IDE support?

<comment><USER22> Emacs with SLIME is probably your best bet, as with most other Common Lisp implementations.</comment></comment>


Show HN: Degrees of Separation - <AUTHOR>

Link: https://apps.facebook.com/degreesapp/
We built this for fun to see how people are connected to one another and to help people connect/network.  We're starting it as a Facebook App and will be adding more search features soon, ie by Work, Education, as well as the ability to import email/phone contacts as a Web/Phone App, but wanted to see what the feedback was for the basic search.  We do have the chicken and the egg problem in that the more users we have, the shorter the paths will be, but we have to start somewhere right?
There used to be something like this on FB, but it seems to have shutdown.
Thanks in advance for looking and the feedback. <comment><AUTHOR> Clickable: <https://apps.facebook.com/degreesapp/></comment>