Pile of shit (shit on top of shit) - <AUTHOR> http://www.firatatagun.com/blog/2016/04/12/pile-of-shit/ <comment><USER2> Good title! :-)

I know how it feels. My longest function which I had to work with was around 1000. And on top of that the author and all the others who had to work on it left useless comments how bad the code is etc...

Well no shit Sherlock! Then why didn't you fixed it instead of leaving shit messages for future use?

Welcome to the REAL world of software development!

When will it change? Never?!

</comment><comment><USER3> Please provide more descriptive title next time.</comment>


The Technical Interview Rift - <AUTHOR> https://blog.techmasters.chat/the-technical-interview-rift-cf5d4cf6d2c#.yi5qfn705 <comment><USER2> Some useful advice, but something about the overall tone seems a bit off:

 _You are a manager and it's time to hire a new developer to join your impressive team of A+ players._

That's where things started to go of course. You're _not_ an "A+ player", and most of your team aren't "A+ players" either. You're just human beings doing the best you can and (hopefully) trying to improve a bit each day -- like anybody else.

No one wants to work with losers. But any (serious) talk of of "We're all A+ players here!" or "I know how to spot A+ players!" is just motivational kool- aid, and ultimately a distraction from the real work you have to do -- including the task of finding the best people you can hire, and who are willing to throw their lot with your cause.

Especially when it's quite often the people hired for their seeming "A+" qualities (which they are able to exude in spades) who turn out to be the most toxic, morale-killing members of your (once) impressive team.

<comment><USER3> _> , but something about the overall tone seems a bit off: [...] You're not an "A+ player", and most of your team aren't "A+ players" either._

I believe you misinterpreted the author's backhanded "compliment" about teams' _self-proclaimed_ _" impressive A+ players"_. His tone is sarcasm if you combine it with the repeated fixation on "Fibonacci" puzzles in the rest of the essay:

\- _quote: , what possible insight does questions like "solve a Fibonacci sequence" + whiteboard + no internet give you to know about a person fit for a development role?_

\- _Go Beyond Fibonacci Pen /Paper Tests to Assess Candidates_

\- _If you get Fibonacci'ed in your next job interview, perhaps you should look elsewhere? If you are the one doing the Fibonacci'ing, you are doing it wrong._

(In other words, if your team bombards candidates with Fibonacci, you of course will think your company consists of A+ players!)

His "tone" might have tripped the obvious sarcasm detector more readily if he wrote it to say: _" it's time to hire a new developer to join your impressive Project Euler hackathon champions. blah blah blah"_

<comment><USER4> But he also says "After all, the primary purpose of doing technical interviews is to ensure A+ players are identified and are persuaded to join your team. If your process is preventing this, then you are doing it wrong."

That doesn't sound like sarcasm to me...

<comment><USER3> It may not be _in that particular sentence_ since he's directing that "A+" label towards the _candidates_. (He's being genuinely charitable towards the job seekers.) However, when he's directing the "A+" label towards the _interviewing team_ , it comes across as a backhanded swipe.

Perhaps calling it "sarcasm" is too strong since his sarcasm is _very subtle_. If you integrate his _entire_ essay about _" Technical Interview Rift"_, he essentially wants to lecture the "too cool for school" dev teams that pride themselves on Fibonacci puzzles, etc. However, he can't go into full-frontal sarcasm mode like Stephen Colbert / John Stewart because he doesn't want to make the very audience he's trying to reach _tune out_ from his message.

Therefore, he has to dance around his intended message in such a way as to not insult his target. There's probably a better word to describe what he's trying to accomplish but I couldn't think of it so I called his passive-aggressive tone: "sarcasm".

</comment></comment></comment></comment><comment><USER5> I think Triplebyte has the right idea: essentially a take-home test conducted over a period of a couple of days. It allows a candidate to show off what they _can_ do under realistic conditions, instead of allowing an interviewer to poke at what they can't do under highly unrealistic high-pressure conditions. A typical tech interview is more like a spelling bee than a realistic test of a candidate's abilities: if you happen to get a word/question you know, you look awesome. If you don't, you don't. (Not long ago I screwed up a tech interview because I couldn't remember/figure-out-on-the-spot the iteration condition for estimating square roots by the Newton-Raphson method, and I was not willing to cheat by looking it up on Wikipedia while I was on the phone. Their loss.)

I just wish Triplebyte would expand their outreach beyond YC companies.

<comment><USER6> I have tried using take-home tests before. The level of plagiarism was astonishing. And that's just the plagiarism I could detect.

Even with a problem that's unique to my organization, I don't know how I could trust that the actual candidate themselves was the one who completed it, and that they completed it without unfair assistance (e.g. from other people). Similarly, I have a habit of looking over someone's resume and picking a few random technologies they mentioned to discuss. I'm surprised how often candidates claim experience with a technology and can't really describe what they've done with it or discuss it intelligently. The level of dishonesty is high.

For all of the downsides of interview-based hiring, at least I know what I'm getting (modulo error bars on assessment efficacy).

> I couldn't remember/figure-out-on-the-spot the iteration condition for > estimating square roots by the Newton-Raphson method, and I was not willing > to cheat by looking it up on Wikipedia while I was on the phone

I don't ask the kind of questions that require obscure knowledge. I just ask questions that you can problem-solve with regular rational thinking, with questions that usually admit a "naive" brute-force style solution that can be improved upon. If you did want to look up some algorithm that would be fine with me, but I am more impressed by a quick and confident reasoning through the problem followed by a fluid implementation of the naive solution, than I am of sophisticated solutions with better algorithmic performance or accuracy.

<comment><USER7> We give take home problems and have no issue with plagiarism. In fact we actively say use the internet, books, etc..., and we still get solutions that are wrong or do not even compile.

The ones that do come back and look good, give us a starting point for the interview. If the person did just copy it from somewhere else they have will have a hard time talking about the solution and various pros/cons.

<comment><USER8> This is how we do things as well. When I walk around our office I always see people looking at SO, Google and others. Why would I hold a candidate to a different standard?

</comment></comment></comment></comment><comment><USER9> I view technical interviews that require algorithm solving and coding on a whiteboard as ridiculous. Primarily, because I am _deathly afraid_ of them.

And here is why. I own and operate two online businesses in the Radio Communications space that are the de-facto standards for our industry. I've coded both of them from the ground up in PHP/MySQL and manage all the day to day administration of these sites. Our infrastructure is deployed on 20+ servers on AWS, Google Cloud, and some bare metal deployments, for which I manage solely by myself. We have 100's of TB of audio archive storage that is all developed and managed by myself. I've personally tackled the entire stack from the ground up, end to end. I've hired an outside consultant, _once_ , to do graphic design work. I've taught myself titanium accelerator and released a highly successful mobile app for my business for Android and iOS. I've even deployed much of our API using NodeJS! Gasp! I do all security, SEO, sysadmin, scheduling, upgrades, marketing etc.

But I'm not a computer scientist. If you asked me to outline a best-case sorting algorithm for x use case my response would be "uhhh... " If you asked me to write out a for loop in PHP on a whiteboard I'd say to myself "uh... where do the semicolons go again?" But I can piece together building blocks from AWS, Stack Overflow, open source projects, multiple SAAS providers. I can also write contracts, executive license agreements with third-parties, develop highly successful and consumable APIs, and manage the financials for a multi- million dollar business. I've also deployed multiple APIs in SOAP, XML, and JSON which form the most successful parts of my business

But get me up in front of a technical interview team where the startup is looking for a computer scientist and I'll have a ton of "yea, but..." and probably wouldn't last too long.

So when I see these examples of technical interviews in organizations where I _know_ I could add value, but realize their process for evaluating that value could certainly eliminate me very early, that scares the crap out of me. Fortunately, my business has been successful enough that this will never be a scenario I have to face. It still bothers me though.

<comment><USER10> You're argument is exactly what I call the difference between a "trade programmer" and a "computer scientist". You are describing the former, while all the Google-copycat interviews with algorithms and data structures are designed to hire the latter.

As you said, trade programmers typically glue together existing components and frameworks. Sometimes, this is really all that's necessary, as you have proven with your businesses. Computer scientists are qualified to create those underlying systems in the first place, properly weighing and selecting the algorithms and data structures to meet constraints.

I don't mean any of this to cast a negative light on trade programmers. They are 80-90% what a business needs, and I think there should be better opportunities for people to become such without the 4-year degree. [0] However, the limitations also need to be understood, and in my opinion a business is best served with a smaller set of computer scientists to provide technical guidance. Otherwise, you will see a lot of history-repeating-itself type of bad decisions.

[0] This is basically the hole that bootcamps have attempted to fill. They could also be filled with associates degrees or apprenticeships.

<comment><USER8> I think your percentages are too low. IMO it's more like 95-97% of businesses are 99-100% served by what you describe as a trade programmer.

The problem is that it seems that 90+% of shops think they're 0% served by trade programmers.

<comment><USER10> Sure; 87.3% of all statistics are made up on the spot anyway. [0] It will also vary greatly depending on your business scale. Google needs computer scientists; at their scale any efficiency slip is very costly. For the local small business looking to turn their Excel spreadsheet sitting on a shared network drive into a CRUD app, a trade programmer or two is probably entirely sufficient.

[0] Yes, you heard me right! 91.4% of all statistics are made up at the very moment!

</comment></comment></comment></comment><comment><USER11> > These are ways to gauge if they know the langue without making them write > code. You should never ask a developer to write anything from scratch, or > write anything.

But.. _why_? You are hiring someone to write code, what better way to gauge their ability to do so than a work sample?

Obviously you don't drill someone on sorting algorithms, unless that's what you are hiring them for, but make them solve a simple task that is on the level of abstraction and in the domain your company deals with.

Having sat through interviews where candidates who had made it through screenings and talked the technical talk turned out to be unable to write actual code to solve rudimentary domain problems, there's no way I'm hiring a developer without programming with them first.

<comment><USER12> The argument I usually hear is that many people don't do well under pressure or with others looking on. So by doing coding tests or whiteboard tests or whatever, you're selecting for people who do well in high-pressure situations, rather than people who are talented coders.

But I agree with you. I've interviewed many people who have an impressive resume and can talk the talk, yet can't even do Fizzbuzz.

<comment><USER13> > I've interviewed many people who have an impressive resume and can talk the > talk, yet can't even do Fizzbuzz.

And I've interviewed many people who can do fizzbuzz because they studied for it (CTCI) and wind up being terrible programmers.

The end result? Technical interviews are, at best, a completely random hiring signal. You'd be better off flipping a coin. It's better to be lucky than good, so why not use luck as a selection criteria?

<comment><USER11> But, if you aren't hiring a developer specifically to implement fizzbuzz for you, why are you giving them that as a problem to solve?

Hiring someone to work on a Flask REST API? Make them add a REST endpoint to a little flask app. Hiring someone to do CSS? Make them style a form. Hiring someone to build a database? Make them safely write to disk.

This is far from random - it's been almost twenty years since Schmidt and Hunter showed that a work sample is one of the best predictors of future performance, 24%, vs 3% for unstructured interviewing.

[http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...](http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf)

<comment><USER13> I couldn't agree more. The problem with most work sample tests is the length and the format. Most are too long and they have you start with nothing. Adding a rest endpoint to a little flask app is perfect.

It's a shame nobody actually does this kind of work sample test. And thanks for that study. It'll make for some nice ammunition. :)

</comment></comment></comment></comment></comment><comment><USER14> Here is my take on hiring: an active Git repository with a decent amount of non-trivial code/projects generally gets applicants I have to evaluate to skip the whiteboard interview and instead grab lunch or coffee with me. We then discuss technology/software engineering. I find this approach to have yielded much more conclusive results, though it probably would not "scale" well. You can quickly tell whether someone is passionate and resourceful by talking with them, occasionally erring on research-ish stuff but always focusing on the implementation rather than the theory.

Plus I sometimes stumble on the occasional gem that is _very_ knowledgeable and from whom I can learn.

Whiteboard interviews, coding challenges, take-home projects. All of those things are so time-consuming for both the company, the interviewer and, of course, the applicant. And all of that just in the hope that the company will accept them... maybe? Of course when nothing else is available then it is the usual drill: phone, skype, on-site 1, on-site 2. But if other signals are available then to the trash it goes.

<comment><USER15> If you start with github repositories for your candidates, aren't you filtering out people who do things other than work? Most of us work 40 to 60 hours a week, and sign paperwork that says that any code we write belongs to the company. But now we need to work Saturday and Sunday on open source projects so we're worthy of sharing a coffee with a hiring manager?

<comment><USER16> > says that any code we write belongs to the company.

I have turned down several jobs when they tried to hit me with that clause. I have negotiated a better clause that gave them the work I made for them, which just happens to line up with state law here

> But now we need to work Saturday and Sunday on open source projects so we're > worthy of sharing a coffee with a hiring manager?

I wouldn't want to hire someone who didn't enjoy coding enough that they didn't have something on publicly visible repo.

<comment><USER13> > I wouldn't want to hire someone who didn't enjoy coding enough that they > didn't have something on publicly visible repo.

Why? Those people sound like terrible employees. Those are the people who come into work to punch the clock and get paid while saving their energy and creativity for their side projects, which is what they'd really rather be working on.

<comment><USER8> Most of my coding output for my job ends up on a public github repo, which is the best of both worlds.

<comment><USER13> That would be nice. Are you a library or framework developer?

<comment><USER8> I work for a nonprofit. The downside is the pay isn't as great as it could be elsewhere. The upside is that the work is meaningful and I get to write OSS as part of my day job.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> This is what I don't understand. During technical interviews, why don't we just give the potential employee a laptop and ask them to solve the problem as they would during a regular work day? Why can't they search Stack Overflow, and perhaps reach out to a friend or even a random person on IRC for advice? Isn't this how the real life works? They may not know anything about min heaps, but perhaps a 15 minute research will help them solve the problem better than anyone who actually memorized Cracking the Software Interview by heart? Isn't it better to measure discipline and ability to learn and solve problems instead of measuring existing knowledge?

</comment><comment><USER18> From a Silicon Valley perspective, I didn't even know that this was something that companies considered not doing. 'Round here, a technical interview is guaranteed, it's more a question of if/when there will be anything else involved in the interview. Also, will it be a mundane technical interview, or more like 6+ hours doing Top-Coder style questions on a whiteboard? It used to be only Microsoft and Google were known for that but now it's practically everyone as far as I can tell.

<comment><USER19> Re: your last sentence, it's been common in my experience for at least the last couple of decades: that is, interviews for most of a working day, including at least one each focused on coding and design. Maybe my experience is unusual, but there were both big companies and startups.

</comment></comment><comment><USER20> > All in all, would do again, except not more than 1 a month because flying > across a continent and back for 1.5 days is gruelling.

This is part of the problem. Aside from college graduates, who has time to fly around the country burning vacation time to _maybe_ get an offer?

<comment><USER21> On the other hand, would you really accept an offer without having at least met the people you'd have to work with? Especially if you have to move across the country?

<comment><USER22> Video chat is good enough for me. I don't need to know about their posture or how long their legs are.

<comment><USER21> This might just be me, but video chat doesn't really do it. You don't get an idea of the building, bathrooms, cafeteria, and the general idea of how often your coworkers bathe.

All of these can be pretty important to my general mental health.

<comment><USER23> Yes, and they could easily hide a noisy, chaotic work environment in a video chat as well.

</comment></comment></comment></comment></comment><comment><USER24> I am glad to see managers that give interviewing the level attention it deserves.

Many interviewers just assume they know how to interview people to find the so-called "A player" (yeah, we're up to A+ now too). People love the idea of magic gotcha questions the answer of which determines whether a person is or is-not skilled some specific area. But the reality is most orgs are barely sophisticated enough to manage FIZZBUZZ-level screening, let alone screening for "A players." Proven interview techniques that take practice, training and coordination like the Behavioral Interview are often skipped in favor of ad- hoc, unprepared interviews that end up with a go/no-go vote.

</comment><comment><USER25> When I worked as a technical recruiter, I found that a great way to screen candidates was to explain the "box" I was trying to put them in. I described the work I'd done to research the position, why I was screening for certain criteria, and that ultimately I wanted to get their approval on the summary I had written about them. Being non-technical, this worked better for positions I was experienced recruiting for of course.

For example, when recruiting a web developer, I'd start by asking a hiring manager what they wanted.

"Get me a full stack Rails developer who knows OO Javascript."

Then I'd just drill into why they asked for that until I didn' have manh questions. I'd be able to describe roughly what projects needed attention, how the tech stack helped solve that problem, and ask the candidate to help me sell them as someone who can do that. It wasn't perfect, but it worked better than most recruiting attempts I've experienced so far.

Now as a developer with a few years of experience, I feel I have no idea what I am doing, and am winging it at all times :)

</comment><comment><USER26> OP: "There is a slight proclivity to conduct technical interviews within the companies that comprise the community."

The companies _constitute_ the community. The community _comprises_ the companies.

<comment><USER27> "In addition to its original senses, dating from the 15th century, "to include" and "to consist of " ( The United States of America comprises 50 states), comprise has had since the late 18th century the meaning "to form or constitute" ( Fifty states comprise the United States of America)." [http://www.dictionary.com/browse/comprise](http://www.dictionary.com/browse/comprise)

<comment><USER26> This is like legitimatizing incorrect usage of "literally" or "unique" because some people don't know their definitions. It is ambiguous and sloppy to misuse a word as if it means its antonym.</comment></comment></comment>


A Taxonomy of Technical Debt - <AUTHOR> https://engineering.riotgames.com/news/taxonomy-tech-debt <comment><USER3> This is a fantastic article.

Contagion is a really great term. I've seen my poor abstractions be replicated by others on my team, to my horror -- "don't they see why I did that in this particular case, and not in this other case?" Of course, that's entirely, 100% my fault. I picked a poor abstraction, I put it in the code, I didn't document it well enough, and of COURSE other programmers are going to look to it when solving similar problems. They should!

That said... Sometimes I spend a bunch of time finding the right abstraction for a feature that we end up not expanding. And then it feels bad that I spent all this extra time coming up with the "right" solution, instead of just hacking out something that works. Hmm...

<comment><USER4> One team I was part of kept a separate backlog of technical debt and experiments. It was nice to have a place to say, "in 30 days, look at this hacky thing and see if it's worth making better". Or, "I noticed this is a mess, here's how I might clean it up." We'd occasionally talk over the backlog and prioritize it, which helped communicate both the general make-things- better spirit and specific issues like you mention. I really liked it.

One thing that made it work is that we worked on it in small slices all the time, without involving the product manager. It was still visible, so there'd be the occasional question, but as long as we kept delivering user value, nobody worried to much about our mysterious code concerns.

<comment><USER5> Funny enough, most companies I worked for, I had to follow "You can refactor if the PM doesn't catch you spending those precious minutes for this".

There was only one time, where we had every Friday, time to improve the codebase. 2 months later it became every 2nd Friday, though.

I'm really pissed that technical debt is considered as "Hey the dev guys are complaining again".

<comment><USER6> > I'm really pissed that technical debt is considered as "Hey the dev guys are > complaining again".

That's because it's very untransparent to anyone other than the engineers working on a project.

I've had a limited amount of success by making this more transparent. Signaling every time a feature will take longer because of a piece of technical debt the team wants to fix caused the fix to get priority before implementing the 4th and 5th feature affected.

<comment><USER7> How is "technical debt" handled in meatspace?

Don't the bean counters at Ford Motor Company (for example) nark on the assembly line workers and industrial engineers and QA/QC folks have work pile up, broken machines lying around, uncleaned trash?

<comment><USER8> It's risk/reward to the people who want to decide how their money is spent, isn't it?

In your example, the worst-case scenario is that someone could die, and that tends to spur on investors to discover the probity within themselves to spend some money avoiding an expensive lawsuit.

But when the devs are complaining about the old code being terrible and making their lives hard, it never seems to hinder them that much to management. They keep banging out new features and fixing bugs, and nothing bad seems to happen. But the drip-drip-drip of bugs keeps increasing, and the new features take a little longer each time, and nobody dies at least, but the thing becomes a haunted moneypit that nobody wants to touch, and you're stuck with it now unless you rewrite it all at huge expense, etc., etc.

Maybe everyone should just treat a piece of software as they would a life. I bet we've all seen some codebases where if it were a friend, you probably would have staged an intervention by now. Your software baby needs absolute care from the get-go until the very end, or it will get sick and probably die, and most likely in a very prolonged and painful way.

<comment><USER9> The place I used to work in has been hiring (junior) people like crazy. Part of the reason they need so many is the crushing foundational technical debt at the core. When they hired someone to capable of improving that they were unable to merge the changes due to fear, and the management couldn't see the business value of doing so. They've had a few nasty outages recently too. I believe the insides of the Atlassian kit are similarly riddled with technical debt.

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> This reminds me of the following, from the book Team Geek[1], chapter "Offensive" Versus "Defensive" Work:

 _[...] After this bad experience, Ben began to categorize all work as either "offensive" or "defensive." Offensive work is typically effort toward new user-visible features—shiny things that are easy to show outsiders and get them excited about, or things that noticeably advance the sexiness of a product (e.g., improved UI, speed, or interoperability). Defensive work is effort aimed at the long-term health of a product (e.g., code refactoring, feature rewrites, schema changes, data migra- tion, or improved emergency monitoring). Defensive activities make the product more maintainable, stable, and reliable. And yet, despite the fact that they're absolutely critical, you get no political credit for doing them. If you spend all your time on them, people perceive your product as holding still. And to make wordplay on an old maxim: "Perception is nine-tenths of the law."_

 _We now have a handy rule we live by: a team should never spend more than one-third to one-half of its time and energy on defensive work, no matter how much technical debt there is. Any more time spent is a recipe for political suicide._

[1] [http://shop.oreilly.com/product/0636920018025.do](http://shop.oreilly.com/product/0636920018025.do)

<comment><USER11> The XP guys had it right. Amortize all defensive work across EVERY piece of offensive work.

In the tech debt parlance most people are paying interest only payments instead of paying against the principle. Every check you write should do both (extra payments are good but they aren't good enough).

</comment></comment><comment><USER12> It's a great article, but I do have one quibble.

> A hilariously stupid piece of real world foundational debt is the > measurement system referred to as United States Customary Units. Having > grown up in the US, my brain is filled with useless conversions, like that > 5,280 feet are in a mile, and 2 pints are in a quart, while 4 quarts are in > a gallon. The US government has considered switching to metric multiple > times, but we remain one of seven countries that haven't adopted Système > International as the official measurement system. This debt is baked into > road signs, recipes, elementary schools, and human minds.

A not-so-hilariously stupid mistake is to think that the traditional measurement system is stupid. His picture illustrates one of its virtues: the entire liquid-measurement system is based on doubling & halving, which are easy to perform with liquids. The French Revolutionary system, OTOH, requires multiplying & dividing by 10, which is easy to do on paper or with graduated containers, but extremely difficult to do with concrete quantities (proof: with one full litre container and two empty containers, none graduates, attempt to divide the litre into decilitres).

The _real_ foundational debt is that we use a base-10 system for counting, due to the number of fingers & thumbs on our hands, rather than something better- suited to the task. If we fixed _that_ problem, then suddenly all sorts of numeric troubles would vanish. There's actually a lot to be said about the Babylonian base-60 system, to be honest.

<comment><USER13> That's an... interesting point I haven't seen brought up before. Makes me appreciate the "traditional" system more.

Still, I guess we aren't going to drop base-10 any time soon, so I believe the US should just accept the "traditional" measurement system as something that used to be very practical, but no longer is due to progress of technology, and switch to SI.

<comment><USER14> Agreed; this is a really interesting perspective. It points to how different applications yield different optimizations. Base 60 is fucking cool. I really like musing on how we arrived at the duration of a second.

I stand by the assertion that being one of 7 countries that only sometimes uses SI has very real costs. [https://www.jpl.nasa.gov/missions/mars-climate- orbiter/](https://www.jpl.nasa.gov/missions/mars-climate-orbiter/)

<comment><USER12> > Base 60 is fucking cool.

It really is! The number of digits might be a bit much for normal use, so perhaps base-12 is more realistic. If we're going to upend tradition, might as well do it for good, well-founded reasons …

> I stand by the assertion that being one of 7 countries that only sometimes > uses SI has very real costs. [https://www.jpl.nasa.gov/missions/mars- > climate-orbiter/](https://www.jpl.nasa.gov/missions/mars-climate-orbiter/)

Of course, that would have been equally a problem had one team been using kilogramme-metre-seconds and the other gramme-metre-seconds, and could have been avoided by standardising on customary _or_ on French Revolutionary units!

</comment></comment></comment></comment><comment><USER15> Great article, loved how the examples were presented.

In my time as an engineer, I've found that thinking of tech debt as financial debt also helps. There is the initial convenience (borrowed money) of using the debt-ed approach. Then there is fix cost as Bill Clark name it, i.e. how much to pay back the debt if it were money. The impact is akin to the amortization schedule, i.e. what is the cost every time. For normal money, amortization schedule is over time, but for tech debt it is over usage. The amortization schedule of tech debt is discounted over time, as with money, _now_ is more important that _later_.

Contagion is a great concept, and I think it is a better name than interest rate, as the debt will spread through the system, and not just linearly with time.

Tech debt is also multi-dimensional and not fungible like money, which makes it a harder thing to reason about.

But the good news is, in my opinion, that sometimes it is perfectly fine to default on some tech debt, and never pay it back, delete the code. Then taking that tech debt was a win, if the convenience was more than the amortized payments.

<comment><USER16> I think the main difference is that technical debt is not fungible, i.e. you can't necessarily easily choose to pay off the highest-interest technical debts first like you would for your personal financial debt.

<comment><USER17> put another way: you can have one item that is 5 days of work but really critical and another that's 2 days of work but way less critical. If you have 2 days to work on tech debt, you basically are forced to do the 2 day one. especially since you are evaluated on what you finish, not how much you worked towards some long goal.

</comment></comment></comment><comment><USER18> In data warehousing and BI, it's MacGyver and data technical debt all the way down. MacGyver because of all the "urgent" reports whipped up for CEO, duplicate copies of data and the reports done by consultants who barely understand industry. Data dept because of all the bugs and changes passed down as data from source system.

<comment><USER19> Does any programming paradigms protect better against data debt? The only way that I can imagine to significantly protect against this would be if there was some way to generate data migrations based on type changes.

<comment><USER20> I don't think there is any technical obstacle or pattern which can prevent dumbasses from shitting things up, humans are just too creative. As soon as you allow any extensibility, someone is going to start shoving integers in as strings, "Y/N" strings as booleans, etc.

It would help a lot if there was a well-formed, unambiguous specification for both sides to hold to. Something like the IETF terminology, in terms of MAY/SHALL, specifying things like "true/false" vs "Y/N", etc. Providing sample responses with decent coverage of the possible options is good as well.

Then you at least have the leverage to say "aha but the spec says it should be like this, why are you doing it wrong".

<comment><USER19> > It would help a lot if there was a well-formed, unambiguous specification > for both sides to hold to.

It does sound like you're describing the schema language part of GraphQL. I think that GraphQL is a great tool for making sure that the right stuff goes in and out. Although it's far from solving all input validation problems. Hmm, perhaps you're describing a different problem.

After having worked with GraphQL user input validation at least seems like a manageable problem. There still seems there should be even better methods for handling contagion problems in the data of historical mistakes though.

</comment></comment></comment></comment><comment><USER21> What about "fear"?

The most pernicious thing about technical debt, in my opinion, is that it creates fear in the sense of "I don't want to touch that module".

Even if you try to be objective and use hard facts to overcome the fear, it doesn't matter, because fear destroys creativity, so you've already lost.

<comment><USER22> Your tests should reduce that.

</comment></comment><comment><USER23> I might have missed it, but missing from the taxonomy: "Pay In Full" Debt.

In this debt, you pay the entire cost until the last use of it is cleaned up.

This kind of debt is especially insidious because there is no incremental benefit to cleaning it up.

<comment><USER24> I'd be curious to hear an example of that (I don't believe I have personally seen one that fits that pattern in the wild yet).

<comment><USER17> removing a library? you can remove 100 usages of it but not until every single one is gone can you remove it.

<comment><USER25> porting to a backwards incompatible a language version? you can't use most of Python 3's new features while some part of your codebase is in Python 2

</comment></comment></comment></comment><comment><USER26> Reminds me of risk analysis: Impact times Probability equals Risk.

Contagion seems like a probability factor. Impact is the cost of leaving things unchanged. Fix cost is the cost of fixing the problem.

Risk management in this context then means comparing Impact cost to Fix cost in terms of impact for the business.

<comment><USER27> The one difference is that contagion is multiplicative over time (potentially logarithmically, linearly, or exponentially—probably a reasonable definition for 1/5, 3/5, and 5/5 respectively).

</comment></comment><comment><USER28> Somewhat aside, but the brain having to "flip" visual information because it's "upside down" seems suspect to me. Turn it sideways while maintaining all the connections it has to the rest of the body, and what changes? Is it getting visual information sideways that it has to rotate now? Probably not.

<comment><USER29> Moreover the idea that the collection of neurons that your retina connects to has any concept of "orientation" is nonsense to begin with IMO. It's not that "there's an upside-down image that your brain has to fix", it's just that your brain interprets signals from your retina as a picture in your mind, full stop.

Rods/cones in the top of your retina connect to your brain through neurons, so do the ones at the bottom. But to say that "this 'top' retinal cone should really connect to a 'top' neuron in your brain", doesn't even make sense to me. Since when do the locations of the neurons interpreting the input even matter?

It would be the same with hearing too... you have a left and right ear, but if for some reason those were swapped and your left fed things to the right half of your brain and vice-versa, your brain wouldn't be "flipping it back", because how could the absolute location of the neurons interpreting the sounds even matter?

<comment><USER30> This is the right way to look at it. In fact, your brain is plastic enough that if you wear glasses that flip your vision upside down for several days it will eventually relearn the mapping of retinal cells to neurons so that you see things normally while wearing them. This was studied in the 1890s by a guy called George Stratton.

</comment></comment></comment><comment><USER31> There are writers who just ooze technical depth of understanding - i thinks it's something to do with trying to explain something at a laypersons level, but leaving many assumptions just there for the reader to follow. It's almost the opposite of baffling with bullshit.

Good read and a really useful concept

</comment><comment><USER32> I am a senior product manager for a large financial technology company.

Over the years I have learnt to become comfortable with allowing my engineering teams to refactor code whilst delivering new functionality.

This has been a process and largely one of trust between me and the engineering leads.

It has also helped that I have seen payback from the investment made from reducing down the debt in terms of us delivering new functionality quicker and less error prone code. Although, this payback can take a while to see (6months + which is a long time for a product person operating in a competitive space!)

Most of my managers don't get this or if they do they are too blinded by immediate kpi's from further above they can't justify it so in most cases I just tell the engineering guys to add a spread to their estimates to cover the paydown of the debt.

Over they years this has definitely helped me build tighter relationships with engineers which as any product manager knows can have huge benefits.

</comment><comment><USER33> I find it's always worth asking "will this get better over time, or worse" for everything, ever. Folks just fail to see past the next few months, having at least one person in the room asking this question makes them at least ignore it intentionally instead of complacency.

</comment><comment><USER34> "I've rarely encountered discussions of contagion."

This surprised me: contagion is a good metaphor because it is a compounding measure of the growth of the problem. Just like an interest rate (a compounding measure of the growth of debt).

Most senior developers I've met have considered the interest rate of the debt, which seems like it has been renamed here as contagion. Maybe I've been lucky to just know smart people!

From the point of view of explaining these concepts, I'd suggest keeping the metaphors consistent. Tech debt should have an amount owed and an interest rate, tech infection (?) should have a potency and a contagion level.

</comment><comment><USER35> At pretty much every game studio there is an epic internal battle of standard libs vs custom. std::string and [some custom string class] here it is AString is usually the spark. A constant of internal game development is that they think they can always build better strings, lists, dictionaries, collections etc than the standard lib, basically thinking the standard lib is as it was in the 90s and all the work that went into them is bunk. In some cases if you are really pushing memory and not writing custom allocators or using something like boost then yes, but in most cases the technical debt of custom classes written by an ancient from generations ago internally is more technical debt.

> _One of the best examples of MacGyver debt in the LoL codebase is the use of > C++'s std::string vs. our custom AString class. Both are ways to store, > modify, and pass around strings of characters. In general, we've found that > std::string leads to lots of "hidden" memory allocations and performance > costs, and makes it easy to write code that does bad things. AString is > specifically designed with thoughtful memory management in mind. Our > strategy for replacing std::string with AString was to allow both to exist > in the codebase and provide conversions between the two (via .c_str() and > .Get() respectively). We gave AString a number of ease-of-use improvements > that make it easier to work with and encouraged engineers to replace > std::string at their leisure as they change code. Thus, we're slowly phasing > std::string out and the "duct tape" interface between the two systems slowly > shrinks as we tidy up more of our code._

So now there are two string classes, that is technical debt... and one should be consolidated on and the arguments against std::string are sometimes valid but you can also do custom memory allocators or use better standard lib iterations.

EA even rewrote the whole standard lib EASTL [1] to adjust for some of these issues i.e. fragmented memory. Some games require it, some it is pure ego in game development teams. Game development teams have the highest ego driven development (EDD) I have ever seen and lots of _tricks_ that take _five minutes_ (but add 2-3 months to testing due to five minute solutions) but are more spaghetti than templates that write templates.

The one problem that comes about with your own standard lib or thinking you are better than boost or similar, is that the learning curve on the internal lib replacements add technical debt and start up costs, and the original guy that wrote them is long gone usually. Also, in the end portability suffers as there is invariably 3-4 versions of the internal libs.

Developers have to weigh the technical debt of your own custom classes outside standard libs and see if that outweighs the memory issues that may arise. Today most machines are not as affected by memory fragmentation issues and there is more cpu/memory to go around, and where they are you can write custom allocators for std/stl or use something like boost.

I do love Riot Games and all game development teams just I have never worked in one or with one that doesn't have the standard lib vs custom battle and wastes lots of time when one isn't standardized on or when not necessary. Some games and game engines require it, where they do you should fully commit one way or the other. Though going custom leads to slowdowns in coding for new devs and invariably there will be multiple versions of those internal libs over time that add up in the debt department.

[1] [https://github.com/electronicarts/EASTL](https://github.com/electronicarts/EASTL)

<comment><USER36> One of the biggest problems of this is the tribal knowledge that develops around it. I worked at a studio that had something very similar to EASTL, but had joined the studio after an exodus of senior people.

It meant I had no idea how to use the custom libs. No documentation, no one left in the office to tell me how its used, no Stack Overflow to answer even trivial questions.

I left after less than a year. The studio closed down 2 months after I left.

<comment><USER37> EASTL mostly doesn't have this problem because it as far as possible is a compliant implementation of the STL with a few specific extensions, mostly around memory management. It's not a library that provides STL-like functionality with a different API. Much of the custom allocator stuff has finally now been superseded with polymorphic allocators in C++17.

Source: former maintainer of EASTL (not the original author).

<comment><USER36> I would imagine that being a larger organization, there are dedicated resources for at least some documentation/POC's for questions/changes/etc.

The smaller the company, the less resources you have to maintain, the more issues you're going to run into.

</comment></comment></comment></comment><comment><USER38> In my experience often the term 'technical debt' will be hijacked by product- oriented folks resulting in feature debt being presented as tech debt.

</comment><comment><USER39> This seems far too focused on dev tech debt, which has a very narrow scope. I like the article, so I'm not knocking it, just offering a little perspective. As a senior sysadmin in the past my primary issues have been technical debt across the entire board, number one being too few hires for too much workload due to cheap or nearsighted execs, but I would definitely agree that contagion is a great term for how techdebt grows faster the longer it's left alone.

It's worth remembering the CTO and senior sysadmin and a few others are dealing with all the tech debt of the entire company and IT department of which dev is only a subset (of course this depends on the company, but on HN sometimes I see convos like this where it feels devs are just talking at each other and not receiving much outside feedback.)

<comment><USER14> I'm not surprised at all to hear that I have blind spots outside of "dev". I've been working on shipping games for a decade, so I'm very fixated on the types of stuff I run into day-to-day in that dev process.

<comment><USER39> Nothing wrong with that at all. Has that all been on LoL? You guys did a lot right with it from the start. Unfortunately I burnt myself out on mobas due to HoN.

</comment></comment></comment><comment><USER40> I've been binge listening to Software Engineering Radio for the past few months. I am currently listening to an episode where they are talking about technical debt.

He has the opinion that clean code is not as important as shipping code - <USER2> the code first and then refactor as needed after you get customers.

[http://www.se-radio.net/2015/04/episode-224-sven-johann- and-...](http://www.se-radio.net/2015/04/episode-224-sven-johann-and-eberhard- wolff-on-technical-debt/)

<comment><USER41> Definitely. I build a lot of MVPs. So I focus on "make it work, then make it work well".

Shipped code is so much more valuable than unshipped code :-)

</comment></comment><comment><USER42> "We gave AString a number of ease-of-use improvements that make it easier to work with and encouraged engineers to replace std::string"

Are you absolutely sure this itself won't become Foundational technical debt? You seem overly confident, given the metrics, that replacing std::string is a good decision.

<comment><USER14> We certainly can't know for certain. But we've had a significant, measurable reduction in CPU cost due to "hidden" memory allocations from things like passing a char* into a function that takes a std::string and stuff like that. (I may be being mildly inaccurate, as I wasn't the guy doing the perf captures etc. I just talked to him about it).

I'm particularly impressed by AStackString, which is a subclass that has initial memory allocated on the stack, but automatically converts to dynamic allocation if you exceed that space. So we get quick stack allocation by default, but it will safely handle when it needs to expand.

Most of the quality of life stuff is around having in-built support for printf style formatting, string searching (including case-insensitive).

</comment></comment><comment><USER43> I love this article. Quickly breaks down the types of debt.

</comment><comment><USER44> (MacGyver's name is Angus!?)

</comment><comment><USER45> I would love the idea of a technical credit score. For example, if you're the kind of dev that racks up technical debt and never pays it down, you should have a shitty technical credit score, and be considered a poor hire. Whereas someone with great credit, would be a great asset to bring onto the team.

<comment><USER46> Tracking "credit" score sounds like good idea, but I would not go as far and assuming that persons with bad credit scores are poor hires.

Maybe person who creates tech debt is really great at prototyping, fixing urgent issues with unconventional methods (aka MacGyver) or do other tasks you find boring. While credit score of this person will be low, such people are also great assets in the team.

In general, this metric could be useful as tracking number of pull requests, lines of code, and so on: to spot anomalies and investigate: maybe that person is suddenly blocked by something, overwhelmed and need help, or just works differently, or on different tasks and the anomalous metric is ok.

<comment><USER47> A metric like that would also discourage people from actually documenting the debt.

"When a metric becomes a target it ceases to be a good metric."

</comment></comment></comment><comment><USER48> Cool write up and classification system. A category that affects us is VendorDebt. Things that are inflicted on us by external vendors. Classifying then in a similar manner might help us decide which vendors to dump.</comment>


Why Snabb? - <AUTHOR> https://github.com/lukego/blog/issues/7 <comment><USER2> I've been following the progress of Snabbswitch for a while now, and I think it is absolutely fascinating technology. While its geared for network i/o, I think that the methods of reaching the performance gains attained by Snabb can be applied to other forms of software projects, too .. from my point of view, the architecture of Snabb would be very applicable to other applications in the embedded Linux space - particularly, for pro Audio/DSP applications. I think kernel bypass has a lot of potential to open up ways of developing apps for this market segment which I hope to continue to explore, and maybe .. one day .. commercialize.

I applaud Luke and the Snabb team - you've opened doors that I hope others will walk through, and see that the technology you've pioneered has value far beyond the networking realm.

</comment><comment><USER3> I used to work in networking. One of the things that always amazed me is people's willingness, or excitement to be pinned to one proprietary networking system. On the proposal of a network that utilized primarily open standards, they were very put off. I hope Snabb succeeds where I failed.

</comment><comment><USER4> I think Luke's work on Snabb is impressive and it's cool to see efforts to make a marketplace for Snabb based solutions. I don't know if it will work out since companies can be fickle when choosing solutions, but I'm rooting for Snabb nonetheless. It's great to see people building non-trivial network things with kernel bypass.

</comment><comment><USER5> Snabb Switch – A toolkit for solving novel problems in networking ( _kernel bypass_ )

[https://news.ycombinator.com/item?id=8008963](https://news.ycombinator.com/item?id=8008963)

</comment><comment><USER6> Oh, someone using GitHub issues as a blog. What a special snowflake!</comment>


Ask HN: Are startup hiring practices asking too much? - <AUTHOR> I recently did a trial day at a local startup for a position as a microservices developer. During the day, the conversation leaned more toward DevOps than pure development. This is right in line with my interests, so I was happy  when they came back the next day and asked if I would be interested in more of a 50% microservices, 50% platform position.
Then they asked me to come in for another trial day focused on that position. Having just spent 7+ hours there, I politely declined with words to the effect of "7 hours should be enough to make a decision on pretty much anything." And that's not even to mention the code sample, open source project links and extensive resume I'd already sent them. But in reality, I totally lost interest in them. I thought it was incredibly presumptuous to ask for 2 days of my time in-office, and not even offer to buy lunch in return.
Then I see job notices on HN like "Drchrono (YC W11) Looking for Python/Django Engineers" (https://www.hackerrank.com/tests/527036edb11ff/0ca5532381fbebad9fffaa28dc90475f) where they want you to 1) do a hacker challenge 2) phone screen 3) phone screen again 4a) come in for a 3 DAY hackathon (where presumably they keep whatever you produce) after 4b) "learning Django ~ 4-6 hours a week."
This all seems far beyond what's required to make a good hire. Are there really now so many developers competing for the same startup jobs that employers can make such excessive requests? Or does every startup now think that it's a baby Google that can ask for the world, just because? <comment><USER2> I don't think its a baby Google thingy. The costs associated with hiring the wrong person are huge. But its not only the money - it really is a pain in the ass and most people trusted with hiring other people at a startup have made many painful experiences in the past. Which, over time, makes them quite risk averse. I think this is the main reason why many startups implement intensive screening processes.

But having an intensive hiring processes doesn't guard you from hiring the wrong person. So they fail again and again, while making the hiring process even harder every time. Its a death spiral. You can easily spot a startup stuck in this death spiral if someone trusted with hiring other people tells you they get a lot of applications but none of them are any good.

I'm sorry that doesn't help you much in your situation, but I hope it explains why you see a lot of defunct hiring processes these days.

<comment><USER3> It irks me tremendously when we take this attitude. I think the problem is that developers have no spine or backbone and are used to bending over too easily - largely because of the personalities we tend to be "nice people".

As developers we have a number of failsafes and checks and balances in our processes: code reviews, TDD, unit through to integration testing, audit trails on every byte of code changed etc. To suggest that a wrong hire is "too costly" is ridiculous.

Compare this with a management consultant who is often on client site alone and whose work is heavily based on personality - "the airport test" \- any wrong move could end up risking the client relationship with the business.

<comment><USER2> I totally agree: Developers should show more backbone in situations like these.

The "too costly" mantra really is ridiculous, exactly. Maybe we also need to make sure the people trusted with hiring other people learn that their job is not risk assessment or risk management, but finding great talent.

</comment></comment></comment><comment><USER4> I'm inclined to agree. This is where I tend to prefer BigCo practices. Interviewed with a Big4 software company. 30 minute phone screen, flew me out as soon as convenient for me, great per diem, 6-7 interviews and they gave a hire decision within 24 hours.

Contrast this with some startups I've applied for:

-one did a phone screen where I explicitly said I wouldn't work for less than X dollars. Gave me two weeks to build an email client in a poorly documented platform. Then made me come in for interviews. A month from the application date, they extend an offer for X/3, citing that I didn't implement one of the three bonus features in the email "test". I would have made more flipping burgers.

-another one made me develop another sample app, then had phone screens, then proposed a full day of interviews.

-one startup proposed I quit my job and work on contract paying zilch for a month.

-and of course there always those lesser startups/angel investors that suggest you work for free to get a "feel" for the startup world.

Only one startup that I have interviewed so far has done it well. Made me come in for a day of work (virtually) after the first stage of the interview, but paid me 500USD after I wasn't accepted. I would have been happy with less frankly.

<comment><AUTHOR> That last interview is genius. That's exactly what employers should be doing. Ask for enough from the interviewee to confirm that they're competent, but also show respect for them by compensating them for their time.

It sounds like the startup world is such an echo chamber that those running startups have convinced themselves it's a privilege to work in it. In fact, the employee is taking a big risk given startup failure rates. At this point, so many BigCos have adopted enough of the startup culture in their development departments that I'm not really sure what makes working at a startup compelling, other than Cool Points and maybe (almost certainly worthless) stock options.

</comment></comment><comment> dimitri-gnidash It sucks the way you have been treated, and I am surprised that they did not cave in and offer you the position after you held your line on not coming in for the second day.

Having made hiring mistakes myself, I am afraid of the getting the wrong person on board and the only way to truly know is to test a relationship by working together. Contract-to-hire is a good way to go, but good developers have full-time jobs, and it is hard for them to jump ship without concrete promises from the employers.

I doubt they are thinking of these requirements as "we can ask for anything because we are awesome". They are adding more and more check and requirements after unsuccessful relationships trying to prevent another failed hire. "Last guy was a drug user, let's add a urine test to our offer process."

As far as consolation, I have seen DrChrono positions for months on HN so can't be easy to hire with all those requirements.</comment>


Free will, faciticity and their (not-so) surprising consequences - <AUTHOR> http://paraschopra.com/blog/personal/free-will-faciticity-consequences.htm

<comment><USER2> The terminology, premise, and references are sound, but there is an unjustifiably large leap to the conclusion. This entirely ignores the analytical portion of human thought. You can't stretch an anecdote and a few specific research articles to a metaphysical theory, certainly not in a blog post.

Negation of this universal quantifier only needs existential evidence, so I'll attempt to provide one. Say I'm unemployed and watching TV on a Monday afternoon. The thought occurs in my mind that I need to look for a job, but the facticity is setup such that I'm very lazy and unwilling to look for jobs, and the impulse for me is to continue doing what I was doing. A few minutes into it, my brain has analyze the consequences of my action, and I am now at a point where I'm making a choice. The research quote all agrees above that my unconscious brain made the choice initially, and yet there's room for "free will".

I think the problem is the extension of impulse as the basis of all human behavior, leaving no room for analytical thought. You're probably looking for a dialogue here, but unfortunately I've used up all my free time in trying to write this up :) There's no reason to respond, I only want to plant the seed that "lack of free will" theory doesn't hold up. I am sure you can stretch your metaphysics to accomodate my anecdote above, but I'm sure someone else can provide another. I urge you to step outside your thought process and really quantify free will as a result of analytical thought, come up with a different theory, and then reconcile it with the one you've come up with above.

<comment><USER3> True. What I meant was that free will as we typically consider it -- our _conscious self_ making a decision out of free will is an illusion. As you say:

>my brain has analyze the consequences of my action, and I am now at a point where I'm making a choice

If you differentiate between brain analyzing the consequences, and then _you_ making a decision, that is wrong. Research shows _you_ seem to be making a decision, but actually your brain has already made one. Of course, if you include unconscious neural activity as a conception of self, free will does exist but if only consider self that is conscious of itself, free will doesn't seem to exist.

</comment></comment><comment><USER4> >(At atomic level, I don't know if universe is deterministic or not, but at human level it definitely is not).

That level of certainty demands an explanation. The author seems to come to the conclusion that the complex interactions do not leave room for determinism. I'm not entirely sure why.

The usual idea is that if one were to know the position and velocity of every particle in the physical universe, one could theoretically predict how the universe would change at any given time. That should work as well for four particles as it should for four billion. I don't think complexity in a system can be used to show evidence of true randomness.

That being said, I don't believe the world is deterministic at either the atomic or the macroscopic level. But I wholly dislike the reasons presented here.

<comment><USER3> My point here is that universe is definitely not deterministic on a macro, human level. It may not be random at most fundamental level, but the number of variables there are, on a macro level it the universe does not seem to be deterministic.

<comment><USER4> >My point here is that universe is definitely not deterministic on a macro, human level. It may not be random at most fundamental level, but the number of variables there are, on a macro level it the universe does not seem to be deterministic.

Why does an increasing number of variables mean the system is not deterministic? Simply because interactions between people are complex, does not mean they are not deterministic. The number of variables has no impact on whether or not a system is deterministic.

<comment><USER3> I did not say it is not deterministic, however it _appears_ to be non deterministic on a macro level.

</comment></comment></comment></comment><comment><USER5> There's an important factor usually missing from "free will" discussions: that a deterministic consciousness engine must include the individual's beliefs about free will. Someone who believes they have free will behave differently than someone who doesn't. Moreover, a society that believes in free will exert different behavioral pressures than one who doesn't.

While I don't think "free will" exists objectively (every choice is traceable to causes under any model), it seems to me to be a necessary illusion for people to accept responsibility over their own behavior, internally and externally. However, it is at best a fuzzy abstraction, and there is absolutely benefit to continue exploring and unpacking the free will story.

<comment><USER3> I agree with you that it may be beneficial to maintain the illusion of free will in the society, however the truth that emerges from science and philosophical discussions must be called out and their consequences must be carefully analyzed (no matter how bitter they may be). It's like the good old debate between good v/s truth where two may not be compatible.

</comment></comment><comment><USER6> It's _fact_ icity, as in 'fact' of the matter. I don't think you've quite grasped the concept or the necessary, productive role it plays. You might also look into _Befindlichkeit_ and the section of B&T that deals with the hermeneutic circle (I don't have a copy at the office, but if you search ' _circulus vitiosus_ ' you'll find it).

<comment><USER3> Thanks for correcting it. Don't know why I missed it entirely! Thanks for pointing other references as well, will definitely check them out -- this subject interests me a lot.

</comment></comment><comment><USER7> No point trying to decide what to do about the consequences of a lack of free will. If you can decide whether or not to punish someone, they can decide whether or not to offend. If they have no choice in their action, neither do you.

</comment><comment><USER8> Daniel Dennett has a nice critique of the Libet experiment where he argues that there's a temporal mismatch between making the decision and recording that the decision was made. Basically that experiment can't actually measure the decision moving from unconsciousness to consciousness, only the shifting to making that decision to consciousness being aware that you made the decision and now need to focus on shifting attention. It's important to note too that people who have more attention report faster response times.

</comment><comment><USER9> > _we may either regard children who do well at studies as good kids, or we > may instead idolize cool kids who bunk classes and have fun all the time. So > early in our life, what makes us fall into one group and not the other?_

Another conundrum: Some people like to create false dichotomies, and others (who fell into both of the above groups) see them for what they are.

How do you explain that?

<comment><USER3> I'm not sure I got your point. Can you elaborate?

</comment></comment><comment><USER10> Being influenced by your environment doesn't mean you had no say in the choices you made.

And if there truly is no free will, then we have no choice but to feel pride in our actions, if that's how we are built.

So you can't say not to feel pride. Doesn't really make sense.</comment>


All images of the Facebook ads Russians purchased during the election - <AUTHOR> https://www.documentingreality.com/forum/russian-ads/ <comment><USER2> This is really a pretty fascinating example of psyops in the wild. Propaganda for both "tribes" assembled in opposition over various issues in order to sow more division between them. Imagery clearly designed to influence with emotion, twisting the dials of fear, anger and pride in the audience.

The power to disseminate information (or disinformation) is not one that should be taken lightly. It seems that most of the human race was deployed to production with a pretty critical RCE vulnerability, and the patches available (education in its various forms) do not appear to be sufficient to protect the whole system from corruption.

<comment><USER3> how are these any different than the standard appeal to emotion arguments that groups make in good faith? e.g., why is an ad trying to raise awareness about black lives matter from russia any different than an ad trying to raise awareness about black lives matter from black lives matter?

> The power to disseminate information (or disinformation) is not one that > should be taken lightly.

curious to know if you believe the power to disseminate info should be regulated by the govt.

<comment><USER4> Some of the ads I was reading were would take extreme stances that make the other side get a distorted view of a group. I saw an LGBT ad from Russia that suggested we make it illegal for heterosexuals to adopt. This would do damage to said movements in that people will ignore their legitimate messages as well when them deem them extremists.

<comment><USER5> Could you link that one? I can't seem to find it

<comment><USER4> This was the image I was talking about: [https://www.washingtonpost.com/resizer/buDyxBhuwEgTWpgvuiE_w...](https://www.washingtonpost.com/resizer/buDyxBhuwEgTWpgvuiE_wMVI6D4=/1484x0/arc- anglerfish-washpost-prod- washpost.s3.amazonaws.com/public/JGYIJPRTN47INP4WM3QAM7EY64.png)

in this washington post article ([https://www.washingtonpost.com/news/politics/wp/2018/05/10/t...](https://www.washingtonpost.com/news/politics/wp/2018/05/10/these- are-the-most-popular-russian-facebook-ads-from-each- month/?noredirect=on&utm_term=.70c7820d2adf))

</comment></comment></comment></comment></comment><comment><USER6> Does anybody have a source that isn't a gore site?

With that said: Do not visit the root domain if you don't like seeing lots of mutilated dead people.

<comment><USER7> [https://democrats-intelligence.house.gov/facebook- ads/social...](https://democrats-intelligence.house.gov/facebook-ads/social- media-advertisements.htm)

<comment><USER8> I found this imgur album easier to read than individual PDFs:

[https://imgur.com/a/ZjQ3PLS](https://imgur.com/a/ZjQ3PLS)

The WaPo also has them sorted by popularity:

[https://www.washingtonpost.com/news/politics/wp/2018/05/10/t...](https://www.washingtonpost.com/news/politics/wp/2018/05/10/these- are-the-most-popular-russian-facebook-ads-from-each-month/)

</comment></comment></comment><comment><USER9> Having seen my fair share of vitriolic content during the past election cycle, most of these images honestly seem quiet tame.

<comment><USER10> I think the point is that they pander to both sides of any argument to increase the divide and lessen the chances of society making a sane compromise. Much more effective than just spewing insanity and hyperbole.

Extreme progressives would destabilize the country by setting up structural racism and sexism in order to achieve "equity", dissolving borders, gutting the police, and over taxing companies.

Extreme rightists would destabilize the country by repealing too much of the social safety net, closing down the country to immigrants, militarizing the police, and letting companies behave anti-socially with no repercussions.

Our enemies are smart to try to push us into embracing extreme policies. Either way they win. Because when our institutions start to fail we won't be able to project power anymore. It will draw us off of the world stage to some extent.

<comment><USER11> Nothing could ever make compromise more unlikely than a winner take all voting system that has then predictably created exactly two parties whose followers again following the incentives rarely have an original though beyond "the enemy of my enemy is my friend".

<comment><USER12> > winner take all voting system

It was pretty entertaining watching people complain about the electoral college system in the 2016 election.

"Why should someone in Wyoming have more of a vote than someone in California!?!"

No mention of winner takes all, though: that would enable a certain basket of deplorables in CA to vote (and have it mean something.)

<comment><USER13> You're entitled to your own opinions, but not your own facts.

The State of California has __passed a law __declaring its opposition to "winner take all".

[https://en.wikipedia.org/wiki/National_Popular_Vote_Intersta...](https://en.wikipedia.org/wiki/National_Popular_Vote_Interstate_Compact)

<comment><USER14> I'm fascinated to know how you think that's in opposition to "winner take all."

</comment></comment></comment></comment></comment></comment><comment><USER15> Any ideas how prohibiting these types of ads next election cycle will pass constitutional muster? Most were several steps removed from references to any election. And non-citizens residents are generally held to have rights under the 1st Amendment ([https://www.google.com/search?q=non+citizens+1st+amendment](https://www.google.com/search?q=non+citizens+1st+amendment))

Or will it just be Facebook blocking them by way of policy?

<comment><USER2> > Or will it just be Facebook blocking them by way of policy?

How do you even formulate the policy? Sure, you can ban/annotate the ads that are specifically about Clinton or Trump, but where do you draw the line?

The fascinating thing about these ads is that most of them plucked on the political strings of the audience indirectly.

<comment><USER5> You require people to identify themselves when purchasing ads / displaying ads.

The ads pretending to be radical American extreme views are less effective when you learn it's Sergei in St. Petersburg posting...

<comment><USER2> I'm sure a foreign intelligence service will be stopped by identification requirements...

<comment><USER5> Your sarcasm isn't convincing or appreciated.

</comment></comment></comment></comment></comment><comment><USER16> I see a bunch of the pictures they advertised organized events. How did that work if these ads and supposedly pages were run by Russians? Were those events real? Who showed up? Who organized the protest on the ground?

<comment><USER17> Yes.. People showed up and got fighty.

And this is a tiny blip compared to hacking into the DNC's email servers. I see a lot of these ads are stupid and people totally think that was all there was. The ads were the part of the iceberg you can see.

<comment><USER6> Yeah, the ads were just the content they paid for. This doesn't take into account all the sockpuppet accounts on Facebook, Twitter and Reddit that were designed to swing conversations in the direction they wanted them to go.

Also, link aggregator sites (like our very own HN) are susceptible to vote manipulation, which can promote the content the attackers want, and demote the content they don't want.

These ads were just to rope people in. You can see that "Black Matters" has 200k+ members. Those are people who are now getting information directly from the Kremlin on their feeds now.

<comment><USER18> i cannot believe we still havent seen a major news story on reddit. the night and day takeover of the_donald was very obvious to me. i wish someone would write an in depth piece on the nature, scale and timeline of the operation there.

<comment><USER19> Not a major news source but online sources already broke it and reddit responded to those reports a year after the election.

[https://www.thedailybeast.com/reddit-check-out-our- russian-t...](https://www.thedailybeast.com/reddit-check-out-our-russian- trolls-accounts-2)

[https://www.inquisitr.com/4790689/reddits-the_donald-was- one...](https://www.inquisitr.com/4790689/reddits-the_donald-was-one-of-the- biggest-havens-for-russian-propaganda-during-2016-election-analysis-finds/)

</comment></comment></comment></comment></comment><comment><USER20> All Americans on both sides should be outraged at Russia. Their goal is to further divide us via social media and thus news which they have been extremely successful at.

This idea of misinformation, mallace, trolling, psychological warfare is the single most issue about technology that makes me uneasy and nervous.

<comment><USER21> Can we be outraged at ourselves for falling for this bullshit?

<comment><USER20> No, even some of the "quote" smartest people I know fall for it. It's a seperate part of the brain. That's why I always try facts > emotion politically.

</comment></comment></comment><comment><USER22> The wording on many of these ads is odd. In hindsight they are clearly written by a non-native english writer, but perhaps the Facebook audience is used to awkward writing by their peers?

<comment><USER23> This is an indicator of a large scale operation. Scaling an operation entails a degree of tolerance to the degradation in the quality of the product, especially when the marginal benefit from quantity significantly exceeds that of quality.

Cyber space is a high leverage domain which rules out the possibility of a shortage in resources from the opponent. It is also a recent evolving space which makes it more difficult to assess the opponents capabilities, reach and sophistication.

By its nature this operation seems to be part of a bigger destabilisation strategy, maybe a stress test to validate different tactics.

I would be really worried once the ads become well written which would increase the complexity of the situation since quality measures can be manipulated.

In other words once they can fake "fake news", things might get really weird.

</comment></comment><comment><USER24> That was anti-climatic. How exactly did these ads sway the election? In fact, many of them are pro-gay and have liberal idea.

<comment><USER25> I don't think their goal was actually to sway the election. They know well enough that their work is pocket change compared to what both parties' campaigns and ordinary American activists on both sides already do. What they want to do is increase the divisions in our society. The more we fight each other, the less political "bandwidth" we have to project power internationally.

If both parties are unified in pursuit of one goal, then we may act vigorously on it over the long term through the course of multiple changes in which party controls what. If both parties can't stand each other enough to cooperate on anything, then foreign policy and behavior will be half-hearted pushes in constantly changing directions.

Russia can't and doesn't want to control us entirely. They do want us to not interfere effectively in their attempts to control the neighboring Eastern European states.

</comment></comment><comment><USER26> Would be great if FB released the audience targeting data associated with these images. Context and audience matters when it comes to advertising.

<comment><USER27> See the WaPo post: [https://www.washingtonpost.com/news/politics/wp/2018/05/10/t...](https://www.washingtonpost.com/news/politics/wp/2018/05/10/these- are-the-most-popular-russian-facebook-ads-from-each- month/?noredirect=on&utm_term=.f3193924c613)

Each image includes some targeting info.

</comment></comment><comment><USER28> Is it just me or does Q3 2017 have the same ad 5 out of 8 times?

Also is it just me or does the domain "documentingreality.com" sound like it's probably going to document something else.

<comment><USER29> It sounds like that, however, if you check the root of that domain you will see that it's a forum for gore photos and videos from real accidents, deaths or medical conditions, so the domain name is somewhat proper :P

</comment></comment><comment><USER30> What's interesting is who they targeted - they targeted right wing patriotic movements as well as left wing gay rights groups and minorities who may feel disenfranchised like Muslims and blacks. It's as if they looked at America's very polarized and heated political environment and saw it as an opportunity to create division.

</comment><comment><USER5> So there are few distinct elements to this, that we should think about independently.

A - Is this type of advertising scary? [Really all this post is about]

B - Did Russia try to change the election?

C - Did they succeed?

My thoughts:

A - It is to me.

B - As far as I understand, this is conclusively yes, based on the fact that multiple politicians have plead guilty to collaborating. [1]

C - I don't know. For me it's not about that.

[1] [https://en.wikipedia.org/wiki/Special_Counsel_investigation_...](https://en.wikipedia.org/wiki/Special_Counsel_investigation_\(2017%E2%80%93present\)#Criminal_charges)

<comment><USER31> No politician/Russian has pled guilt to collaborating (which is not a real crime) the closest thing they have charged is conspiracy to defraud and I expect they will have to drop those in the long run, you should read the charges more carefully.

<comment><USER5> Rick Gates plead guilty of conspiracy to defraud the United States, however it appears that's not Trump related.

I think it may be correct to pick on some of the details, but as far as I can tell there's overwhelming evidence that Russian agents tried to influence the election, and anybody who denies that I assume does so for partisan reasons.

</comment></comment></comment><comment><USER32> Russia was allegedly buying pro-LGBT ads? Surprising.

Also there are several posts with russian text on the picture like this one [1].

But this doesn't look like a state-level operation. A single person could easily make all those pictures.

[1] [https://www.documentingreality.com/forum/russian- ads/2015-Q2...](https://www.documentingreality.com/forum/russian- ads/2015-Q2/P\(1\)0001844-00000002.jpg)

</comment><comment><USER33> Fun fact: this site is currently blocked in Russia by Roskomnadzor.

</comment><comment><USER34> Why were Russians doing ads to support democratic cause? I thought it was strongly believed that they wanted Trump to win. Doing some divisive thing to get some sort of twisted outcome feels like far out conspiracy. That bring me to question: how much confidence do we have these are Russian state sponsored ad?

<comment><USER35> I was wondering how far down I'd have to scroll to find the first sensible comment. Congrats.

The internal logic of these claims is non-existent. During the actual campaign Hillary and her supporters were convinced Russia (the generic bogeyman of the western establishment everywhere) was shilling for Trump. After all Trump was a lot more friendly and Hillary was stating that she'd shoot Russian planes out of the sky over Syria, so such a motive would have at least made sense.

Now that whole narrative fell apart entirely. We have the head of famously anti-gay and conservative Russia posting LGBT memes and supporting Hillary, despite the risk of war between Russia and America should she have won.

Yet, the desire to _believe_ is so strong, that like all conspiracy theories it's simply morphed into whatever the smallest step to fitting with new information is, that preserves the core. Now it's all a genius plan to "sow division" by posting pictures of fox statues made of shotgun cartridges. Quite why Russia specifically benefits from generalised "division" or why US culture needed help being divided is left unexplained.

This is exactly the behaviour you'd expect to see if people with strong motivations were looking at noise. Leading us to your last question:

 _how much confidence do we have these are Russian state sponsored ad?_

We only appear to have Facebook's word for it. But you have to watch out. One of the subtle ways this conspiracy theory tends to morph and warp is the distinction between the Russian government and Russians. Even in the headline of this thread, note that it's "Russians" and not "the Russian government".

There are liberal Russians. There are conservative Russians. There are 144 million Russians, a little under half the population of the USA itself. If even 0.1% of them decided they cared about the globally-famous US election, and if Facebook had simply selected "ads paid for by Russians", then this is exactly what you would see - a mishmash of stuff with no unifying theme or agenda.

Or Facebook could have just made a mistake. The government asked them to find evidence of Russian interference, and let's face it, going back with "there's nothing there you are all delusional" is not a good political strategy. I'm sure they felt they had to find _something_.

</comment></comment><comment><USER36> I'm sincerely at a loss for how most/many of these could have influenced the election. Part of me think that Congress is making mistakes and Putin is scrolling through these, laughing hysterically that we actually think he'd pay for this nonsense.

<comment><USER6> The ads were mostly for Facebook groups. Once those groups had a substantial subscribership, the Russians were able to push out far more vitriolic propaganda without having to pay for the audience.

The ads are just the tip of the iceberg.

<comment> c-smile Repeating "just the tip of the iceberg" three times in this discussion already looks like programming of public opinion kind of thing. Without proof of any sort.

That trick is as old as the Church that invented it for the obvious purpose.

That's like "election meddling" repeated on CNN 10 times per day just to convince people "it was something but we will not tell you what".

UPDATE: here is the story of the same trick used while ago: [https://paleofuture.gizmodo.com/meet-guy-sims-fitch-a- fake-w...](https://paleofuture.gizmodo.com/meet-guy-sims-fitch-a-fake-writer- invented-by-the-us-g-1787060769)

</comment></comment></comment><comment><USER37> Why are they in this format? They look like my grandma took a screenshot, pasted the image in a Word document and then printed it

<comment><USER38> Because the Democrats on the congressional committee that released them released them as 300+ MB PDFs. This site just printed them all as images and posted as-is.

</comment></comment><comment><USER39> All of these ads shared one thing in common... They appealed to ignorance and the uneducated across every demographic, and it worked.

</comment><comment><USER40> i wonder if this release is going to be the turning point for the current facebook anti-hype cycle? the ads don't paint such a horrible picture as it was supposed to be.

<comment><USER41> Yep, a picture of SpongeBob made me vote for Bernie instead of Hillary. Had nothing to do with her record or anything.

<comment><USER6> You're trying to make a psychological argument against a sociological effect.

No, a single Spongebob meme didn't sway your vote. This is obvious.

But hundreds of thousands little manipulations of social media did have ripple effects throughout existing communities, and created a more divisive political atmosphere that absolutely could sway peoples votes.

For example: Hillary doesn't really have all that bad a political history. She wasn't a great candidate, but she also wasn't the "Killary" character right- wing media portrays her as. You, however, think that "her record" is self- explanatory, because you keep hearing people talk about her as if she were the antichrist.

I would be curious to hear about "her record" from you, but I suspect you're just going to bring up Benghazi, Uranium One, her e-mails, or maybe even Pizzagate to explain why she was unelectable. This, in the end, just illustrates the point that repeating misinformation constantly will convince people that it is the truth.

At the end of the day, propaganda divided the DNC into two camps and turn those camps against each other, destroying any sense of unity during the election. You chose a side, it ended up being the losing side, and the whole debacle likely cost the Democrats the White House, yet you're still unable to see that you were played.

<comment><USER42> How do you know so much about this person, are you aquaintances in real life?

<comment><USER42> -2? What is happening to this website?

"This, in the end, just illustrates the point that repeating misinformation constantly will convince people that it is the truth."

"You chose a side, it ended up being the losing side, and the whole debacle likely cost the Democrats the White House, yet you're still unable to see that you were played."

These "observations" are in fact 100% imagined.

<comment><USER43> _What is happening to this website?_

Lots of brigading, of both comments and votes, on this thread. Just look at all the phrases that are constantly repeated. I thought this set of images would somehow inspire a different set of responses, especially since it came from "documentingreality.com", but in future I will just flag and move on.

<comment><USER42> I wonder how many people are now living in some sort of a self-made imaginary world.

<comment><USER43> I'd be happy if these were "self-made" imaginary worlds. That would at least be interesting, and the average of a bunch of random steps from reality is probably something pretty close to reality. Most of these individuals seem to be living in the same obviously-false war-media-created imaginary world, so in aggregate they're very far from reality.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER44> Is there any indication of any sort that this content isn't just random ads? Do I have to just take your word for it?

<comment><USER45> slipperymate posted the direct link to the official release: [https://news.ycombinator.com/item?id=17051425](https://news.ycombinator.com/item?id=17051425)

</comment></comment><comment><USER46> The complete and utter _intentional_ lack of understanding of the current Russian influence campaign, the ongoing congressional investigations, and the special council investigation in this thread is appalling.

Once upon a time I thought HN was:

A) Well-informed, and

B) Not a place for political discussion.

Honestly if you can't find fact from fiction in news articles, just read the damn source documents. You'd probably find it fascinating, instead of just something to dismiss as a bunch of politically-motivated time-wasting.

I know this comment is against the guidelines, but so is nearly every single comment on this story.

<comment><USER47> Your comment may have been cathartic but it basically leaves me wondering what you are talking about? Which source documents? What "understanding?" So far the press's reporting on this has been appalling, and in general the public has been kept in the dark.

<comment><USER48> [https://www.2016activemeasures.org/](https://www.2016activemeasures.org/)

</comment></comment></comment><comment><USER49> > These are images of the Facebook ads the Russians purchased to try and alter > the 2016 U.S. Election. These were released from Congress

And what does congress base this on? Looking through a bunch, they don't seem to favor either Hillary or Donald... is it supposed to prove that the elections weren't tampered with? Or am I missing something?

Edit: managed to find one! Against Donald: [https://www.documentingreality.com/forum/russian- ads/2016-Q4...](https://www.documentingreality.com/forum/russian- ads/2016-Q4/P\(1\)0004791-00000002.jpg)

Edit 2: Oh and 2017-Q3 has a lot in favor of Donald's wall and poses Hillary as satan (for no apparent reason). Perhaps that's the proof? By far the smallest album of them all? I am not saying I don't believe the Russians influenced the elections, but I do not see anything obvious for or against in this dataset, either.

<comment><USER50> The goal was to sow division, disrupt the entire process, and in the chaos their preferred candidate could thrive.

<comment><USER51> I think more division, disruption, and chaos was caused by the people freaking out over Russian influence than by the actual Russian influence.

<comment><USER52> Good thing that the media and their multiple layers of fact-checkers and editors didn't fall for the Russian's evil plan. They have learned a lot since the 30's and the Cold War.

<comment> the-pigeon > multiple layers of fact-checkers

Not sure if you are being sarcastic or not. One of the worst things to come from modern media is that editors do not fact check because they want to break the story before anyone else and fact checking takes time.

This has lead to tons of quiet retractions.

<comment><USER52> I was being sarcastic. The media, IMO, has learned little. They are useful idiots.</comment></comment></comment></comment></comment></comment>


Who Is More Dangerous: Mark Zuckerberg or Ted Cruz? - <AUTHOR> https://www.libertarianism.org/columns/who-is-more-dangerous-mark-zuckerberg-or-ted-cruz <comment><USER2> Cruz, obviously? He didn't understand a thing about Facebook at Zuckerberg's hearing, and overall is incredibly dangerous for both tech and modern democracy in the United States.

Zuckerberg donates actively to charity, doesn't sell user data, and overall is pretty cool - even if his company isn't the best.</comment>


FCC says Big Cable can now encrypt TV signals to halt "service theft" - <AUTHOR> http://arstechnica.com/tech-policy/2012/10/fcc-says-big-cable-can-now-encrypt-tv-signals-to-halt-service-theft/

<comment><USER2> Why couldn't they do that before?</comment>


George Orwell on Writing and the 4 Questions Great Writers Must Ask Themselves - <AUTHOR> https://www.brainpickings.org/2014/09/18/george-orwell-writing-politics-and-the-english-language/ <comment><USER2> Years ago I saved this brilliant little excerpt from "Politics and the English Language", the essay discussed in the link:

 _" I am going to translate a passage of good English into modern English of the worst sort. Here is a well-known verse from Ecclesiastes: 'I returned, and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all'. Here it is in modern English: 'Objective consideration of contemporary phenomena compels the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account'. This is a parody, but not a very gross one."_</comment>


Linode hacked, customer database taken - <AUTHOR> http://www.marco.org/2013/04/16/linode-hacked <comment><USER2> It was about a year ago when their customer service portal was breached that I mentally moved Linode off my list of VPS with a "good" reputation ([http://status.linode.com/2012/03/manager-security- incident.h...](http://status.linode.com/2012/03/manager-security- incident.html)).

Still, people seem surprised that they've been breached again. There are other VPS providers, why would one choose this one? How can you rely on their security self-assessment?

I think the only credible response to their situation is to hire an external IT security firm for an extensive audit and then publish the results. Like something on the scale of the Fox-IT report on the breach of DigiNotar: [http://www.rijksoverheid.nl/ministeries/bzk/documenten-en- pu...](http://www.rijksoverheid.nl/ministeries/bzk/documenten-en- publicaties/rapporten/2012/08/13/black-tulip-update.html) .

EDIT: this was also discussed 3 days ago here: <https://news.ycombinator.com/item?id=5541915>

</comment><comment><USER3> With the respect to the password, there was an additional log from the hacker stating that the key password was held in memory and was accessed by him via Cold Fusion.

He claims to have been able to decrypt the credit card data at that point. He did state that he only decrypted a couple of them and that he has also destroyed the data since Linode's blog post went live.

-

The above info came from line 75 of the following pastebin: <http://pastebin.com/7WXRDyAg>

07:52 < HTP> the CCrypter class of the linode application context was accessable from outside the wwwroot using undocumented ColdFusion methods. i was fully able to decrypt the ccs using the in-memory privkey that they supplied the password for.

-

I have no idea of the validity of any of that information. To be safe, I have modified my keys/passwords and have deactivated the card number associated with the account.

I figure playing on the safe side is better and that changing everything will result in fewer "surprises."

</comment><comment><USER4> > I've never used Lish, but if you have, consider those passwords permanently > compromised.

Even if you _haven't_ used Lish, consider it compromised, because according to Linode support all Linodes are assigned a random Lish password by default. The same is also true for API tokens (maybe - Linode support doesn't seem to know). See this thread for more details: <https://news.ycombinator.com/item?id=5553694>

</comment><comment><USER5> > It's up to you whether you're confident enough in the passphrase's > complexity to continue trusting any credit-card numbers you've used with > Linode.

I trust Linode's security and their password complexity is almost certainly sufficient to keep the key secure; I cycled my credit card number regardless. It's a minor inconvenience to reduce the risk of a bigger inconvenience.

Any time there's a compromise of this nature, I assume that they got the keys to the castle and any data involved I consider to be in the hands of attackers. The cost of that paranoia is fairly low in the grand scheme of things, even if it its a bit overboard.

</comment><comment><USER6> <https://news.ycombinator.com/item?id=5556846></comment>