Gray Hat Python: Python Programming for Hackers and Reverse Engineers Review - <AUTHOR> http://hamsterswheel.com/techblog/?p=167

<comment><USER2>
        The entire book is windows specific.

That's pretty disappointing to hear since I've already ordered this book and the only instance of Windows I have is a VM that I use for browser debugging. Given the chance I'd probably still buy it, but now I'm not as excited about it getting here.

<comment><USER3> Unfortunately, the debugging interfaces on Linux and OSX are very different from Win32, and very different from each other.

<comment><USER4> Windows has a debugging interface? toolhelp, drwatson and the debugging "api" are a joke. Everyone does debugging by circumventing the OS, hooking system calls and patching "protected" memory. Windows debugging is a messy witch- craft, so much so that everyone stood in awe, helpless and hapless when Rustock.C struck; a fine piece of malicious engineering that becomes _part_ of the OS, a rootkit's rootkit that's gotten rid off only by disk formatting and a fresh install :-)

<comment><USER3> A comment like this makes me think you've never used the Win32 debugging interface _or_ ptrace().

Win32 has one of the nicer debugging interfaces I've worked with. It's a well- documented syscall-level interface to sign up for process exceptions, to enumerate threads, modules, and functions in arbitrary processes, to pause and restart threads, and to read and write from process memory.

What debugger do you know of that hooks system calls and patches "protected" memory? Are you saying that writing CCh to a function prologue constitutes "messy witch-craft"? You know that's how Unix software breakpoints work too, right?

And what does kernel integrity have to do with debugging? Rootkits have been kernel resident since 1996 --- starting with SunOS and amodload.

<comment><USER4> I have used both, and the win32 API, debugging or otherwise, have often left me baffled and short-changed. Our argument will ultimately break down to source code and its availability. There is also the difference in uses of debugging on win32 vs unix: one is often adversarial while the other is cooperative. It's easy to debug a willing process on Unix and get all the information you want, while in Windows it's easy to debug _any_ process and get as much information as you could.

I know ptrace is horribly broken, and so is the entire GNU debugging chain, down to libopcodes. I haven't touched it since '02 when I went full Lisp, but I had many a nightmare debugging a process that forked other children. The mess got worse with LinuxThreads.

As for rootkits on Unix, you're the security guy, I will let you pretend removing a Unix rootkit is the same as removing a win32 one :-)

And finally, remember that it wasn't long ago that any good Windows debugger had to load _under_ the OS. And by a _good_ debugger I don't mean Immunity or Olly :-)

<comment><USER3> So basically your whole argument just unraveled.

Immunity is Olly.

No Unix debugger as powerful as SoftICE exists that doesn't run "under" the OS; that's the point of SoftICE.

Normal people don't debug with SoftICE. Even kernel developers use kd to debug driver code. You done much kernel debugging with gdb? Oh, here's a fun one: debug a running OpenBSD 2.x kernel. Hope you enjoy /dev/kmem.

You're the whatever-you-are. I'll let you pretend that disinfecting a compromised running kernel is feasible anywhere.

Sorry. I'm sure you're a great guy and you've had lots of experience doing interesting things, but I think you're just dead wrong here, and --- all due respect --- now you're just blowing smoke to cover it.</comment></comment></comment></comment></comment></comment>


Zimmerman not guilty - <AUTHOR> http://news.yahoo.com/george-zimmerman-murder-trial--live-video-and-chat-222843188.html

<comment><USER2> Read the Guidelines, please:

"Off-Topic: Most stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably off- topic."

Flagged.

<comment><USER3> The outcome of this trial teaches us that in American society, if you plan to stand up for yourself on the streets against someone who is harassing you, it is essential that you arm yourself to do so.

It teaches black American parents that they must tell their children either to carry legal guns, or that they must always back off from harassment. Otherwise those kids just might not come home from the shops.

That is an interesting development in US society that is certainly worthy of HN. Compare it to e.g. the pages and pages written about a Korean pilot getting a landing wrong. Also much more significant than a couple of immigrant kids placing backpack bombs at a public event.

<comment><USER4> It teaches none of that, black children are far more likely to die at the hands of another black American than any other race.

A show trial is not worthy of HN, this was an attempt to achieve social justice of legal justice.

The real crime is all those who pushed for this trial are doing so to redirect attention from the fact that blacks are far more likely to die at the hands of another.

The Korean pilot story was more appropriate because as geeks we like tech and tech could eventually negate pilot error if pilots exist in the future.

<comment><USER3> > blacks are far more likely to die at the hands of another.

Poor people are far more likely to die at the hands of another poor person.

FTFY. If you ever go to the USA, you'll discover that in the urban areas socio-economic class and race are just the tiniest little bit correlated.

<comment><USER5> Similar point raised by Jamelle Bouie: [http://www.thedailybeast.com/articles/2013/07/15/the- trayvon...](http://www.thedailybeast.com/articles/2013/07/15/the-trayvon- martin-killing-and-the-myth-of-black-on-black-crime.html)</comment></comment></comment></comment></comment>


Studying the world's oldest undeciphered writing - <AUTHOR> http://www.bbc.co.uk/news/business-19964786#sa-ns_mchannel%3Drss%26ns_source%3DPublicRSS20-sa

<comment><USER2> "Why they should make the intellectual leap to embrace writing and then at the same time re-invent it in a different local form remains a puzzle."

There is an example from a much more recent period of history that is well known to scholars of world scripts. Cherokee writing as developed by Sequoyah was based on the example of English writing known to Sequoyah from European settlement in America, but the letter forms

<http://en.wikipedia.org/wiki/Cherokee_syllabary>

are not entirely identical, and indeed Sequoyah's writing system is a syllabary rather than an alphabet.

The work on Proto-Elamite described in the submitted article is quite interesting.

<comment><USER3> Sequoyah is an interesting example. He had access to English printing and knew that it encoded the English language, but he didn't know exactly how, since he couldn't read or write English. He couldn't read or write _any_ language in _any_ script, so in the process of creating a script for Cherokee, he had to independently invent the concept of the syllabary. The proto-Elamite script might also have been invented by someone who know how others used the Mesopotamian script but wasn't literate in it himself. If you learn to read and write one language you'd want to re-use as much of that knowledge as possible when writing a second language (e.g., using romaji to write Japanese) but there's no loss in creating an entirely new script if you aren't literate in any existing writing system.

</comment></comment><comment><USER4> The "DIY black dome" object mentioned briefly in the article is used to create "polynomial texture map" images. If you're unfamiliar with it, it's a very cool technique. <http://www.hpl.hp.com/research/ptm/> has details, but in brief: an object is photographed multiple times from one camera position but multiple lighting positions. Software can then programmatically interpolate to how the image would look from all lighting positions. It's a way to improve on basic texture-mapping for games, of course, but it's mostly proving useful in archaeology, allowing more people to interact virtually with precious objects. PTM images can also be provide "impossible" views of objects -- e.g. giving every pixel a different lighting angle so as to maximize contrast.

</comment><comment><USER5> I assumed at first they were talking about Linear A[1]. Ah well.

[1]: <http://en.wikipedia.org/wiki/Linear_A>

<comment><USER6> I was hoping for the same thing. The Minoans are fascinating, I started reading about them sometime this summer just before my summer vacation to the Cyclades. Particularly I think that their art has a certain feeling or touch that has not been reproduced since then.

</comment></comment><comment><USER7> Why was the headline changed? The original headline [1] used the headline from the article, "Breakthrough in world's oldest undeciphered writing". For some reason it was changed either by the author or by a moderator to a meaning that is completely different.

[1] <http://i.imgur.com/agIfl.png>

<comment><USER2> _Why was the headline changed?_

That is a puzzler. The OP submitted the article with the original article title (that's how I saw the submission when it was a new submission). The guideline here

<http://ycombinator.com/newsguidelines.html>

is

"If the original title includes the name of the site, please take it out, because the site name will be displayed after the link anyway.

"If the original title begins with a number or number + gratuitous adjective, we'd appreciate it if you'd crop it. E.g. translate '10 Ways To Do X' to 'How To Do X,' and '14 Amazing Ys' to 'Ys.' Exception: when the number is meaningful, e.g. 'The 5 Platonic Solids.'

"Otherwise please use the original title, unless it is misleading or linkbait."

The last paragraph quoted above appears to be the newest revision to the Hacker News guidelines, just a few months old, and perhaps that guideline revision is so new that some curators here have habits that go back to an older version of the guidelines, which suggested more rationales for retitling articles on the part of submitters or curators. I really like original article titles whenever possible, myself,

<http://news.ycombinator.com/item?id=4624933>

and the BBC is a very reliable source with reasonable titles.

<comment><USER8> Perhaps it's because it's not actually a breakthrough? I'm kinda surprised the BBC called it that.</comment></comment></comment>


How Kickstarter Became a Lab for Daring Prototypes and Ingenious Products - <AUTHOR> http://www.wired.com/magazine/2011/03/ff_kickstarter/all/1 <comment><USER2> I'm really curious to know why did Kickstarter get traction when there were other similar sites doing the same thing before them? If memory serves me right there was at least one other company one with the exact model, but that one fizzled out.

...

I found the name of the company. It was called Fundable. Here's where the founder of the company states how he laid the groundwork down for Kickstarter to succeed. To say he's bitter is putting it mildly...

"I cannot tell you how painful it is to watch 5 assholes take your idea and run with it and not even give you credit. I hate all 5 of them for that. If I see them, I may punch each one of them in the face."

Here's the link: [http://groups.google.com/group/barcampla/msg/8c636525d0415fb...](http://groups.google.com/group/barcampla/msg/8c636525d0415fb0?pli=1)

<comment><USER3> They started in New York City instead of Silicon Valley. :-)

Isn't this a classic case of someone else executing better. Google, Intel, Amazon, and Facebook are all followers, for example. Microsoft was a big follower. The didn't invent spreadsheets, word processors, or OS's with a GUI. Diapers.com went for a fortune. I'm sure they weren't the first guys with that idea.</comment></comment>


This is Your Brain on Food - <AUTHOR> http://seedmagazine.com/content/article/this_is_your_brain_on_food/ <comment><USER2> "When you eat an unripe banana, its serotonin is free to act upon the serotonin neurons within your digestive tract. The consequence is...usually experienced as diarrhea."

This made me curious, so I googled it. The very first result is a scientific study demonstrating that unripe bananas are a great _remedy_ for diarrhea: [http://www.worldchiropracticalliance.org/tcj/2000/aug/aug200...](http://www.worldchiropracticalliance.org/tcj/2000/aug/aug2000loomis.htm)

I stopped reading the article after this, fearing that the author did not do his homework.

<comment><USER3> It seems strange to disregard an entire article written by a Professor of Psychology & Neuroscience because a quick google session found a link suggesting a contrary view to a specific point.

<comment> b-e-p Indeed; one could say that he himself did not do his homework by reading the entirety of a text.

</comment></comment></comment><comment><USER4> I get skeptical every time someone brings out a list of "chemicals" that are supposedly all around us— without mentioning their concentrations, of course, or the threshold dose for observable effects in humans.

But maybe I shouldn't be expecting better from pop science?

<comment><USER5> > _I get skeptical every time someone brings out a list of "chemicals" that > are supposedly all around us_

What is there to possibly be skeptical about in this article? Why did you put quotes around chemicals? His point was everything that exists is a chemical and is processed by the body as a chemical. I'm more skeptical as to whether or not you read the article than anything presented in it.

<comment><USER4> Where are you reading that "everything that exists is a chemical"? I'm not sure we _are_ talking about the same article, the one I read definitely doesn't say that.

I'm putting quotes around the word "chemicals" to refer to the use of the broad term "chemicals" to mean "chemicals with long names that are psychoactive, not like water and stuff." Every single use of "chemicals" in that article could be replaced with that, so I'm pretty sure the author's talking about "chemicals" and not chemicals.

As for what I'm skeptical of... I thought I made that pretty clear? That chocolate has some amount of phenethylamine and anandamide in it is meaningless without knowing a) how much of those chemicals is present in the food and b) how much of a psychoactive effect that amount of chemical would have on a human. Just from glancing at Wikipedia from those two chemicals, I'm guessing it's not as much as that article wants you to think.

<comment><USER5> > _I'm pretty sure the author's talking about "chemicals" and not chemicals._

No, you are completely missing the point. He was attacking the mythical duality you are perpetuating. There is no binary distinction between psychoactive and non psychoactive chemicals. Water IS psychoactive. Protein in chicken IS psychoactive. Sugar IS psychoactive.

Let's take chicken for example. We can turn that into a chemical with a long name quite easily! Assuming you are consuming low fat chicken, the majority is protein. Protein is a bunch of chemicals called amino acids. One of the most common amino acids by weight in chicken is (2S)-2-amino-3-(1H-indol-3-yl)propanoic acid, Tryptophan. Tryptophan is THE building block of serotonin and all other tryptamines your body produces.

Are you aware 2-(1H-indol-3-yl)-N,N-dimethylethanamine (DMT) is responsible for dreaming and near death experiences is synthesized in your body directly from that chicken and could not have been synthesized without it?

Sugar->Glucose-> 6-(hydroxymethyl)oxane-2,3,4,5-tetrol. Sugar has clear and obvious excitatory psychoactive effects on the brain. Some researchers even think cognitive decline\Alzheimer's may be related to insulin resistance in the CNS similar to diabetes.

The author tells you exact dosages in the article for the amount of Nutmeg you need to consume to hallucinate. He gives more than enough numbers for an opinion piece.

Regarding chocolate you can test your wrong hypothesis quite easily. Buy pure, NON-alkalized cocoa powder in the store today and consume 3 tablespoons.

<comment><USER4> I... guess we disagree? I don't see what you're reading into the article, at all. As I read it he's _clearly_ referring to long-named, psychoactive substances with every single use of "chemical". Anyway, not much point in arguing about that.

Your bringing up tryptophan is the _classic_ example of what I'm talking about. Someone notices that turkey has tryptophan in it, tryptophan is psychoactive on paper, and suddenly the reason eating a big dinner makes you sleepy is because it's _drugged_. (Never mind that turkey doesn't have significantly more tryptophan than any other meat.)

The point is, as you note, there isn't a binary distinction. The difference is in dosage— is the amount of this thing that you actually encounter in your life enough to cause noticeable effects? If not, it may as well be inert. And nutmeg is a good example: the one chemical that the article _does_ provide dosing on, and it turns out that it's basically impossible to get an actual psychoactive effect from it. I wonder what information about the actual concentrations of these "psychoactive chemicals" would reveal?</comment></comment></comment></comment></comment>


Prominent D.C. media firm implicated in fake FCC comments - <AUTHOR> https://gizmodo.com/how-an-investigation-of-fake-fcc-comments-snared-a-prom-1832788658 <comment><USER2> Sickening how easy it was for the telcoms to push their agenda through in spite of overwhelming public opposition.

I'm pretty sure most individuals following the story of net neutrality knew the fcc's ballot box was compromised.

It would be nice if someone could face consequences for the fraud put on by the FCC. I won't hold my breath though.

<comment><USER3> The thing is, "the ballot box" you refer to was just a way for random people on the internet to submit comments. It isn't intended, I think, as a way to guide policy. I expect that they switched to web-based commentary because, historically, they've accepted letters and the web-forms are just an "upgrade". Who's going to hire staff to open, read and respond to letters these days?

These fake "grassroots" operatives merely abused the web-commentary system in order to keep the FCC leadership (and Republicans) from being completely humiliated by the sheer number of people in favor of net neutrality. Turns out, they got sloppy and went a bit overboard.

What is really disturbing, however, is that the company named, "CQ" is in the business of automating "campaigns" with software and data analysis. To them, it doesn't matter WHAT the campaign actually is as long as they get a paying customer-- smears and disinformation is just another day of work for them.

There's so many layers of slime here, it's hard to know where to start.

<comment><USER4> > It isn't intended, I think, as a way to guide policy.

Public comment is explicitly intended to guide policy. It's not a vote but the entire point of the process is to get feedback before a policy is finalized and most agencies attempt to address common points.

It's reasonable to question the inherent assumption of good faith with the current administration but we're also in an unusual time where the most cynical view of previous administrations are now the smart money view of how things work.

<comment><USER5> Public comment is meant to surface arguments to be taken into account to guide policy. As such, having 1 or 10 million people make the exact same argument in the same words doesn't change anything.

<comment><USER4> This is only true in the most simple cases. If there's any portion of a proposed policy which is balancing different interests - which is almost always the case - the ratio of comments is a useful metric because it would surface problems such as e.g. a policy which big companies are comfortable with but hit many small ones with substantial compliance overhead.

In this case, having tons of AstroTurf comments allows them to say that there's public support outside of large ISPs. Given how anticompetitive the decision was, they needed that fig leaf.

<comment><USER6> No agency would be on firm footing pointing to a large number of identical form comments received for one side as a factor in a balancing of interests. It would not survive the "reasoned decision making" requirement. To the extent you're talking about balancing the needs of small and large companies, for example, you'd have to point to concrete examples and data. That could come in comments (small businesses talking about real challenges they've faced), but relying on a large number of identical form comments wouldn't pass muster.

<comment><USER4> If you assume good faith, yes. In the environment they were in where they knew that oversight was in friendly hands, however, I think they were just looking for enough to say the process was followed knowing that it'd be done before it'd get any critical examination.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> With so many rotten apples, maybe the tree should go. This is a gigantic, criminal subversion of Democracy that the FCC remains complicit in, something that only jail time can fix.

<comment><USER8> Why hasn't Trump launched a high profile investigation into this? This guy was appointed during Obama's presidency and he could score some easy points.

<comment><USER9> I don't know if you're just misunderstanding things or wilfully spreading misinformation, but just in case:

1\. Ajit Pai started working at the FCC under George W. Bush.

2\. Ajit Pai was nominated under Obama _for a republican seat at the recommendation of Mitch McConnell_ : no more than 3 FCC commissioners can be of the same party so the split is generally[0] 3 of the president's party and 2 of the opposition, Ajit Pai was a GOP pick.

3\. Ajit Pai was made chairman of the FCC by Donald Trump.

Ajit Pai is a thoroughly Republican operative. Getting rid of Pai would not score any points with the republican party (he's doing exactly what he's supposed to do there), nor would it score any points with the democratic party (way too little, way too late, no chance of the damage done being repaired, and what happened was exactly what'd been predicted when he was made chairman).

[0] when that comes up / is possible: commissioners terms are 5 years (though they may serve up to 6½ if their replacement is not yet appointed) and the seats are staggered so one commissioner's term is up every year

</comment></comment></comment><comment><USER10> Who are prominent clients of CQ Roll Call?

</comment><comment><USER11> Faking an identity online for political manipulation should be a felony, full stop.

<comment><USER12> That could end up outlawing Pseudonyms and fictional characters. The real problem is that the government would accept public comment through an unauthenticated web form.

</comment></comment><comment><USER13> Money quote:

 _an FCC spokesperson said comments from the general public are "generally not substantive, so thus have no impact on a rulemaking."_

There you have it. So the number of comments is useful only when it helps make whatever point they're trying to make.

<comment><USER14> Isn't the FCC supposed to be a consumer protection agency? Seems silly to disregard consumer concerns.

<comment><USER6> The FCC was not intended to be what we consider a "consumer protection agency" (an agency that one-sidedly advocates for consumer interests, like say the CFPB).

The Communications Act of 1934 states that the FCC is created to regulate communications with the purpose of making available "rapid, efficient, nationwide, and worldwide wire and radio communication service with adequate facilities at reasonable charges, for the purpose of the national defense, and for the purpose of securing a more effective execution of this policy by centralizing authority theretofore granted by law to several agencies and by granting additional authority with respect to interstate and foreign commerce in wire and radio communication."

If you look at the historical context of the FCC, including the "several agencies" that the FCC replaces, the focus was on coordinating resource allocation. Granting spectrum licenses, granting permits to build infrastructure, etc.

The AT&T monopoly exemplifies the sort of system the FCC was mandated to help create. It was a world class, gold-plated telephone network, available almost everywhere in the US. It wasn't consumer friendly in many ways (it was expensive and restrictive), but it was certainly "rapid and efficient."

</comment></comment></comment><comment><USER15> Paid for by.... Mr. Pai? Follow the money...

</comment><comment><USER16> This is treason and I don't see why it would be punished differently.

<comment><USER17> But...this isn't treason. Fraud, yes. Definitely fraud. But unless there is some other evidence that this fraud was committed with the specific intent of aiding a foreign power rather than just line the pockets of the telecoms, then no, not treason.

<comment><USER16> "the crime of betraying one's country, especially by attempting to kill the sovereign or overthrow the government."

Maybe I'm being a little hyperbolic, it isn't the government being overthrown but one branch of it and certainly what the public wanted was intentionally silenced.

<comment><USER18> You are being more than a little bit hyperbolic, and it's bad for the conversation when people do that.</comment></comment></comment></comment>


Who Controls the Internet? - <AUTHOR> https://themargins.substack.com/p/who-controls-the-internet <comment><USER2> A good question to ask when pondering "who controls the internet" isn't so much "who can cut off themselves from the internet", it's "who can cut off others from the internet".</comment>


Boardgames: a healthy snack for developers & designers - <AUTHOR> http://blog.rainbird.me/post/5419756951/boardgames-a-healthy-snack-for-developers-designers

<comment><USER2> I am also a software developer and an avid board gamer with a special interest in strategy games. From what I've seen, there is no connection between being a developer and enjoying strategy games. Like you, most of the people we game with are in completely unrelated fields, and amongst developers I know, the number who are really into board games is probably 1 in 50. Whatever the actual number is, I'd bet that a poll of the general public would result in the same numbers.

The key to good gaming is finding the right game, the right group, and the right location. Everyone will play for their own reasons and are unlikely to be the same as others.

Besides the casual game of Chess at lunch though, I wouldn't call it a break -- it's just something we love to do! Possibly another way to look at it is that the skills in logic and planning gained in a software development career translate easily to boardgames where there's a lot of if/then/else situations as well as cases where making decisions where more than one are good come in handy.

</comment><comment><USER3> I really hoped for a better reason than (paraphrased) 'I really like both of this, so they go well together for other developers.'

I happen to like both, also, but I don't see them as a good break from programming. When I step back from programming, I need some non-thinking time instead. I actually find that if I play too many thought-provoking games in my free time, it can impact my development time negatively. (That doesn't usually stop me, though... And it's usually got to be for weeks, not days. Certain video game releases are murder on my productivity and I actually take vacations to enjoy them better.)

<comment><AUTHOR> Thanks, the article was intended to be more of a "I don't understand why developers don't appear to be as interested as me". I was really just explaining my reasons why I like boardgames, not trying to say that it would work for everyone.

You have helped have answered my question though - since I'm not a full-time developer I usually have the brainspace ready to tackle a boardgame. I could see how you guys would rack your brains all day and prefer to take your mind off things - such as watching a movie etc.</comment></comment>


Infrastructure Debt - <AUTHOR> http://www.littlehart.net/atthekeyboard/2011/11/03/infrastructure-debt/

<comment><USER2> You have to walk a fine line. If you neglect your infrastructure for too long you get in a situation where you get bogged down by all the circumstantial complexity in your infrastructure. If you are too zealous about debt you automate things that don't need automating and you end up with an over- engineered infrastructure that is completely automated but never works quite right.

As with the design of software projects the design of the infrastructure takes time. People often go through phases where they take it either far too seriously (after they got burned) and through phases where they cowboy-code their way through.

The problem here is really mundane. It's difficult to set up an architecture right and you don't get there by throwing buzzwords to the wall (and see which ones stick). Even if you use multiplexed dictionary stores, Bazoop Clusters, BunnyHQ, and tricircular backups you still may end up with with a mess of an architecture if you don't know exactly which problems you're trying to fix. And you can easily waste a few weeks evaluating different deployment tools and automation software and still end up with something that works only barely. The opportunity cost is immense.

<comment><USER3> It's not a static trade off either. If you see two world class developers debating some tradeoff it's about as relevant to the average coder as what two Olympic level swimmers discussing their diet has to do with someone trying to lose some weight. AKA avoid holes to big for your team to get themselves out of.

</comment></comment><comment><USER4> I like the article, but am thinking about a bike shed. For each thing, like manual deployments or failure to use source control, I ask myself: Is it "debt?" Or "friction?"

Debt and friction both accumulate, but debt must be "repaid" in a lump sum. If "technical debt" or "infrastructure debt" means your velocity is falling over time, it's friction, not debt.

Infrastructure debt would be something that is eventually going to cause everything to halt while you sort it out, and the longer you wait, the worse it will be. Not using source control is debt, because with near certainty you are going to have at least one major SNAFU requiring spelunking through backups to recover a lost file or to restore some prior release.

Developing without deploying at all is definitely debt, sooner or later you are gong to have to deploy.

OTOH, lot of "technical debt," isn't. It's just friction, and it takes some experience to know when it's a bad tradeoff.

<comment><USER5> (Blog post author here)

Upon reflection your use of the term "friction" is a good one. Although I tend to look at "debt" as something you pay back as quickly as possible but sometimes you can only pay it back in installments.

I guess you could say having too much infrastructure friction eventually leads to one humungous infrastructure bonfire. :)

<comment><USER4> Well, "friction" is on ongoing cost but doesn't result in some catastrophic event down the road, whereas debt may or may not have an ongoing cost but carries with it some non-trivial probability of disaster. Like real debt, the longer you wait to eliminate it, the more expensive it is to fix and the worse the catastrophe if you on't fix it.

So... I tend to think of source control problems as being infrastructure debt, because you are definitely going to crash and burn eventually, and the longer you wait, the worse the problem will be. I am open to rethinking this, but I would classify automated deployment as being friction. If you can deploy by hand, and everybody knows how to deploy by hand... It seems that deploying by hand is probably friction while you are in development and then debt once the product is in "actual" production with end users. In development, you might make a mistake, forget a library, and fixing it is work but not catastrophic. But once you have actual users, making a deployment mistake could produce irrevocable disaster.

Anyways, just to be clear, I'm only bringing up the distinction for the sake of discussion. I like the post just the way it is.

<comment><USER6> Keep in mind, though, that even "friction" will eventually cause a team's productivity to drop down to 0. The amount of friction that a given team can tolerate depends on the team members, of course, but especially on the size of the team (see: Mythical Man Month).

There comes a point that even a tiny bit of friction, spread out over enough developers, eventually causes the marginal productivity boost of adding new devs to reach 0. That can happen even without friction, but the friction exacerbates the trend.

</comment></comment></comment></comment><comment><USER7> I get the benefits of having identical environments as production (and that's what I aim for, at least, with our staging server) but to be honest, I like having a small amount of diversity across our development servers. It's the small differences that have helped us to work out that bizarre bug (turns out, it was actually a problem specific to a particular version of PHP) or to find those small assumptions about the environment (eg. assumed file paths and so on).

The latter aren't _that_ much of an issue, but I find that being able to deal path changes, for example, makes the code that little bit more flexible, making it a little easier to make changes to the production environments going forward with less issue.

<comment><USER8> I just changed job and got from a very unified setup (Linux-only, everyone working on one server) to a much more disperse one (all OSes, working on local). I agree that some diversity helps to keep flexibility and have a code base, data model that is more forgiving for little discrepancies.

But I guess there must be some checkpoints, or rules, were you don't allow any flexibility. You build a much more flexible project on rocks. For instance, no deployment if one test don't pass, and tests should include "X"-lint checks.

</comment></comment><comment><USER9> I think the author has a different interpretation of "technical debt" than how it was originally intended.

Technical debt is not accidental and it doesn't consist of small mistakes, but it's due to a deliberate decision to cut corners when trying to get product or feature launched. Just like financial debt is not accidental, but a deliberate decision. (although you could argue that credit cards etc can create accidental debt too).

Idea is that it's okay and usually a good business decision to accumulate some technical debt to speed up your product development, but you need to keep eye on it and reduce it regularly, because if it grows too large, it can totally halt your product development.

<comment><USER5> My experience is that it is a very rare and enlightened management team that allows developers to go back and clean up the technical debt that was created during the mad dash to get your application out the door.

Most of the time you are stuck with technical debt because there is no room in your timeline to go back and fix stuff that you know is broken.

YMMV, but as the blog post author I'm keenly aware of this situation.

<comment><USER10> Sometimes it's up to engineers to say "I'm fixing this," and for QA people to say "I'm not signing off on this until it's fixed."

Maybe I've just been lucky, but all the places I've worked had developers and QA people that were willing to force quality into a product.

</comment></comment></comment><comment><USER10> 1\. I'm confused about who the audience of this article is supposed to be. If you're a software development team that is not using version control in a fairly deep way, you're not "in technical debt" so much as "wasting someone's time and money with your monumental incompetence." I suppose it's still good to note that you should be using it, but you're not at the point of worrying about subtleties.

2\. What the author calls "infrastructure debt" is just technical debt outside the application source code.

3\. Technical debt doesn't (just) happen because people are lazy, or take shortcuts, or "plan to do it the right way later.". It happens because you generally don't start an engineering effort knowing everything relevant up front. Indeed, it's "engineerIng" precisely because you're learning important things and uncovering subtlety as you progress. Technical debt is the inevitable outcome of the fixed past rubbing up against the newly-discovered present or anticipated future.

Update: So I guess I'm in the 'friction' analogy camp. :)

</comment><comment><USER11> What about database design debt? Or is this still considered technical debt?

In our company, this is 90% responsible for the paralysis that is keeping us from migrating from a 10 year old enterprise architecture. It is designed in such a way that makes it hard for us to extend and scale, but the whole foundation of our business rests on this outdated model. It's to the point that the only way we can move forward is to start over.

I've read about technical debt, but it's usually in reference to code design, lack of proper testing, and tightly coupled dependencies.

And we definitely suffer from "infrastructure debt" as this article describes, but I feel this is the least of our problems.

To me, possibly the most expensive kind of debt to be in is database design debt, as everything rests on this. At least this is the case where I work.

<comment><USER5> (I'm the blog post author)

I would place database design debt firmly in the technical debt side of the ledger (as it were). To me, infrastructure debt deals more with consistency across environments and consistency in moving code from one environment to another.

I also agree with you that database design debt is very expensive and, as you pointed out, leads to paralysis over fixing problems with an established application.

</comment></comment><comment><USER12> "Version control is a 20 year old, well-understood concept."

Closer to a 40 year old concept:

<http://code.google.com/p/pysync/wiki/VCSHistory>

The first, at least according to that article, being SCCS in 1972:

<http://en.wikipedia.org/wiki/Source_Code_Control_System>

<comment><USER5> Blog post author here. Yeah, I figured it has been around a lot longer than 20 years but I thought the 20 year thing would get the point across.

<comment><USER12> I was nitpicking, but it got me wondering about the history of revision control systems as I was pretty sure I had used RCS in the 1980s and I had to check to see if my memory was playing up. :-)

</comment></comment></comment><comment><USER13> Best quote to conclude article:

"Don't be scared of change, be scared of the debt growing in your code base and in in your infrastructure. It won't go away and there is no government bailout on the way to fix it."

</comment><comment><USER14> Some good points, but very narrow focus on web development. Not everyone can work in a VM for performance reasons, and many projects target multiple different environments making it useful to have your devs on different versions to get more eyes on problems that might arise there.

In the video game industry, we usually split our developers up between each game console, so there are a half dozen different dev environments in use and it works better that way.

I like the concept of infrastructure debt, and it's clear he's talking from his own personal experience. I'm sure many examples could be made for other software industries that are also valid.

The hard part about solving these kinds of problems isn't entirely a technical problem, because many of them are caused by bad habits and stubborn programmers. To fix these issues you need to change people's daily working practices, not an easy task.

</comment><comment><USER15> I work with 40,000+ unit tests. I deploy across a live cluster with a single command. My co-workers and I deploy to production dozens of times a day. Every new hire gets a VM image sandbox to develop in. We still have infrastructure debt. The truth is, debt leverages the future into the present, and as long as your net productivity keeps increasing, you're winning. You have to pay off enough debts along the way to not grind to a halt, but you will always be in technology debt. If you're not, you paid too much for what you have.

</comment><comment><USER16> I must advocate reading James Hamilton's paper: On Designing and Deploying Internet-Scale Services.

[http://www.usenix.org/event/lisa07/tech/full_papers/hamilton...](http://www.usenix.org/event/lisa07/tech/full_papers/hamilton/hamilton_html/)

<comment><USER5> Just took a look, that is an awesome paper.

<comment><USER16> It's invaluable. I can't believe it's not more well-known.

</comment></comment></comment><comment><USER17> I liked the article because it made me think, again, about the debt we have at work. However, the few tacit solutions are not going to get me anywhere. This article needs a part 2 for debt that cannot be fixed with trivial solutions.

</comment><comment><USER8> To the author: would a bad choice of language, framework, persistence layer count as infrastructure debt? If so, then it should be noted as the most dangerous of all.

<comment><USER5> Author here:

With the understanding that I do most of my work in the most-bullied, down- trodden programming language here on Hacker News (PHP) I think that your choice of language, framework and persistence layer leads to _technical_ debt or friction as raganwald commented above.

Like I said before, I view infrastructure debt as the cost of consistency in your environments and the cost of moving your code from one environment to another. Very rarely is the choice of language going to be a problem, unless you are trying to use a language in a way it's not intended to be uses (to throw out a completely random idea, like trying to use PHP as a functional language ).

Frameworks are a sore point for people, but mainly because they choose to fight them instead of trying to do everything the framework's way. Picking the wrong framework is a technical debt situation, not a problem of moving code from dev to production.

Persistence layer stuff is also a technical debt / friction issue. Chances are that you could use that particular data store without the persistence layer you chose. For example, I struggled to learn Doctrine1/2 but once I learned how DQL worked it became a lot easier to break out of the object-only contraints and create custom queries.

Hope that makes sense and answers your question.</comment></comment>


Post-Pandemic Advertising Guide - <AUTHOR> https://epom.com/post-coronapocalypse-guide <comment><AUTHOR> This e-book helps you navigate the post-coronapocalypse world with grace and wisdom. We gathered an awful lot of data on COVID-related losses, the experience of past recessions, best-performing channels, and more to help you break through the hindrances.</comment>


State Dept Announces Accomplishments for 2015: Bringing Peace to Syria - <AUTHOR> https://twitter.com/ianbremmer/status/681666202842886145 <comment><USER2> Not only do they only blame Assad for the violence in Syria, they spell his name wrong. This has got to be an image some troll put together.</comment>


Ask HN: Open source version for key-value store of life? - <AUTHOR> I want a key-value store of my life (e.g. When is the next holiday? Thanksgiving. What is my mom's favorite color? Purple. Where is my passport? In the third drawer.)
I found wonder-bot, but texting my super important info to a random company is sketch.
Is there an open-source version of this? <comment><USER2> this one,

[https://camlistore.org/](https://camlistore.org/)

another one:

[https://news.ycombinator.com/item?id=12211754](https://news.ycombinator.com/item?id=12211754)</comment>


Lies and deceit in the Blockchain and AI industries - <AUTHOR> https://medium.com/@alasaarela/everybody-lies-in-the-blockchain-and-ai-industries-da0a4aabc3e9 <comment><USER2> I've been working with the author of this article for the last five years on building Inbot taking care of the backend and technical strategy. For most of this year we've been planning to do an ICO and a few weeks ago we decided to not proceed with that for essentially the reasons outlined in the article.

We are actually keeping the utility token but are no longer planning to do an ICO with that. The plan is for this to be a proper utility token that our customers can buy (using fiat or crypto) to buy services from our user base. This makes it a proper utility token where the value is coupled to economic activity in our platform. Simply put, it's a function of how much our customers are spending on these tokens.

We are launching the token on the Stellar platform. This is after having prototyped our token on Ethereum as an ERC 20 token. Smart contracts have a lot of potential but the current state of affairs in Ethereum makes that a very impractical platform for doing business. There are challenges related to regulation/kyc, UX, performance/throughput, and getting your token listed on an exchange. Simply put, launching a token on ethereum is currently extremely expensive and using the tokens is comparatively hard.

It also requires you to do business with a lot of unsavory types in the industry. This is a big reason for us to move away from Ethereum.

Stellar is so far looking pretty good for us. It has a distributed exchange and compliance protocols. It is proof of work based, which solves the scaling issues with Ethereum as well as enabling microtransactions. Unlike ethereum, we can actually use it to build out our usecases and expect to have reasonable performance and cost. It also has a federation mechanism that allows us to shield our user base from the complexity of dealing with crypto currencies.

</comment><comment><USER3> Hello Mikko, I do believe the good guys will win one day. However, I am a firm believer that our economic platforms are what drives greed and are what needs to be changed. Currently, in a Capitalistic, Socialistic, or Sharing Economy companies compete to see who can create the cheapest product while selling it at the highest price. However, an interesting process begins after products are made free. Because if we're both distributing free products the next logical step is to offer more value than free. The moment that happens we are now trying to do more for our customer, society, and the environment. I'm sure this sounds crazy, but companies can earn greater profits offering goods and services for free while earning their revenue from other sources. My partners and I will be doing just that as we start with free water that pays you to drink it. But, the water will be more than free because half of our profits will be shared with our members, advertisers and charity. But,the most beautiful part comes next because our future competition will have to offer free water plus 51% profit-sharing. So, if you are concerned about corruption lead by example and begin to create profitable products that you can distribute to your customers for free! Best Regards, Josh Cliffords

<comment><USER4> Josh, well said. We are doing exactly that. Our startup Inbot helps businesses grow with trust, and our community of 64,000 members are essentially shareholders in the success they jointly create. I truly hope these new, benevolent models of capitalism will grow and flourish!</comment></comment>


The Raspberry Pi as a Stratum-1 NTP Server - <AUTHOR> https://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html <comment> stephen-mw I just built one of these. It's fun to have a little computer that can synchronize highly accurate time without an internet connection by listening to things in outer space!

I built an auto-install script that anyone here is welcome to use or reference[1]. If you know what you're doing, just change the GPIO pin to your PPS input and fire away.

Unlike the author I used Chrony instead of NTP. It's lighter-weight and much easier to configure with a PPS/GPS reference clock.

The GPS chip I used was an ATGM336H, which can be had for less than $8 on ebay. I'm able to lock consistently 11 satellites from my desk[2]

Keep in mind the Pi has no hw RTC. If you're doing this for fun spend a few more bucks and buy an rtc chip from ebay so you don't drift considerably between being powered off.

1\. [https://github.com/stephen- mw/raspberrypi/blob/master/roles/...](https://github.com/stephen- mw/raspberrypi/blob/master/roles/stratum_1_timeserver/install)

2\. [https://www.ebay.com/sch/i.html?_from=R40&_nkw=ATGM336H&_sac...](https://www.ebay.com/sch/i.html?_from=R40&_nkw=ATGM336H&_sacat=0&LH_TitleDesc=0&LH_BIN=1&_sop=15)

</comment><comment><USER2> I had one of these running with a 10 meter cable to the window so the receiver could get enough satellites. After 24/7 for about a year it just died and I haven't had time to figure out what's wrong. Lights blink but it's not reachable via ping, ssh or scp. HDMI output shows nothing. I had to move on to other projects, so I gave up and assumed I either got hacked or I hit the cheap-SD-card problem of corruption that seems to plague other users running their Pi continuously for longer periods.

I've started buying SanDisk's "High Endurance" microSD cards hoping it will mitigate any such issues in the future, but I haven't had time to get the NTP set up again... I really should have cloned the image.

I think the hardest part was finding a case with space for the GPS hat[0] with an open end for the coax connector. The hat sits a bit crooked because of something bulbous on top (condensator?).

[0]: [https://store.uputronics.com/index.php?route=product/product...](https://store.uputronics.com/index.php?route=product/product&product_id=81&search=GPS+Expansion+Board+)

<comment><USER3> I wonder how much the rpi foundation would need to sell a pi if it included 32gb of storage. I would gladly pay not to feel like I'm playing roulette with my microsd.

<comment><USER4> They just need to make booting from a USB drive easy. Plug in a $20 SSD and stop worrying about reliability.

<comment><USER5> They're working on it: [https://www.raspberrypi.org/blog/pi-3-booting-part-i- usb-mas...](https://www.raspberrypi.org/blog/pi-3-booting-part-i-usb-mass- storage-boot/)

</comment></comment></comment></comment><comment><USER6> Maybe add [2012] to the title, dear mods?

</comment><comment><USER7> Off topic: I love this couples website. If you click through to the homepage they've got lots of stuff to browse.

Yes, it maybe old-style, but it's these sort of sites that the internet was originally built on. It's a real shame that they are becoming a dying breed.

</comment><comment><USER8> > Configuring Wi-Fi

That alone will negate _any_ benefit of running a stratum 1. You lose the precision and accuracy over wifi. Jitter on intranet with the pi as a timesource would be magnitudes greater than the GPS link.

<comment><USER9> Not exactly. Sometimes it's more helpful to have a local time source that you know is accurate but has a little jitter than to have a remote time source with intermittent Internet.

</comment></comment><comment><USER10> this is pretty cool. i was looking at your case, could you tell me the exact model case you were using? i couldn't find the exact model on modmypi.

also, here's a writeup i did on my experiences creating something with an rpi 3 model b+:

[https://github.com/stephenhu/written/blob/master/raspberrypi...](https://github.com/stephenhu/written/blob/master/raspberrypi.md)

</comment><comment><USER11> What is the use case for microsecond-accurate timekeeping?

<comment><USER8> I've done it; and my primary argument to family and friends is time attack vectors.

The real reason though, is because I can, and I like saying my computers are more accurate and precise than their phones.

</comment></comment><comment><USER12> I built a nixie tube clock like that with a gps receiver, it works amazingly

</comment><comment><USER13> They still haven't separated the USB/Ethernet Bus :/

<comment><USER14> This is also possible with the Orange Pi/NanoPi boards which have actual Ethernet that doesn't go over USB, though I don't think there's such a good writeup of how to do that.

<comment><USER15> This would be a perfect case for a NanoPi Neo. [1] It's designed to be ultra- tiny and run headless. It's also $13. I find FriendlyElec's board better designed and supported than Orange by a bit and miles above the Banana folks.

1: [https://www.friendlyarm.com/index.php?route=product/product&...](https://www.friendlyarm.com/index.php?route=product/product&path=69&product_id=132)

</comment></comment></comment><comment><USER16> Are there performance differences regarding Pi 1/2/3?

<comment><USER17> Yes, very much so.

<comment><USER16> Do you have some numbers?

<comment><USER18> The biggest factor is that each generation of the Pi has gotten a better processor (700Mhz single-core in the original Pi to a 1.4Ghz quad-core in the 3B+). There are also a number of detailed benchmarks out there if you bother to Google it.

<comment><USER16> Especially in the context of a stratum 1?</comment></comment></comment></comment></comment>