Readability vs Instapaper - <AUTHOR> http://checkarun.blogspot.in/2012/04/readability-vs-instapaper.html Comparision of ReadLaterList apps <comment><USER2> This was originally brought up on another blog: "Ever wonder if any publishers have actually collected any of your contribution?".

Where's the accountability for Readability? How do you know if the blogs you read are hooked in with Readabilty? If they aren't, is the money kept in some sort of interest bearing account as an incentive to signup? Shouldn't that already be disclosed somewhere?

You don't have these problems with Flattr. Both Instapaper and Readability could simply scan for the Flattr link during the same time they're stripping out the other elements. The Flattr link could then be dynamically added to either program if it exists.

I don't understand why Readabilty, Instapaper, and blogs don't integrate with micropayment systems like Flattr (<http://flattr.com/>). Readability's 30% cut seems like a lot. Why reinvent the micropayment wheel and limit it to just blogs? Other than the payment system there isn't much functional difference from Instapaper. (disclosure: I've only sampled Readability whereas I've used Instapaper for a year+)

<comment><AUTHOR> Yeah this should be the one you are talking about <http://agblog.com/entry/3932> , that sounds interesting idea for readability to use flattr indirectly.

Moreover all articles a person reads may not be useful. So flattr's idea of "Click if useful" would provide money distribution to worthy.

</comment></comment><comment><USER3> Kindle support should be pointed out and while both say they support it, I found Readability's to be much more reliable. As much as I liked to support Marco with Instapaper I had to stop donating and switch. Sending to my kindle wasn't working any longer and Marco was never returning any emails I sent off inquiring about the issues I was having. So far Readability's Kindle support has been working perfectly for me.

<comment><AUTHOR> Thanks updated :)

</comment></comment><comment><USER4> possibly the worst article I have ever read

<comment><AUTHOR> Hi Thought of sharing my experience. Sorry If the content quality is very bad. Will improve :)

<comment><USER5> You may try to get your facts right. Instapaper is not a dollar a day. Also, if you are giving cons for Instapaper, you should provide cons for Readability as well. Otherwise the articles loses objectivity.

<comment><AUTHOR> Thanks that's typo :( and Cons added ! Hope I am having some good learning today.</comment></comment></comment></comment>


It's easy to make something incredible. - <AUTHOR> http://unalone.tumblr.com/post/88825593/its-easy-to-make-something-incredible-all-you-do

<comment><USER2> This reminds me of a passage from "Zen And The Art Of Motorcycle Maintenance" that I love to quote:

 _You want to know how to paint a perfect painting? It's easy. Just make yourself perfect and then paint naturally. That's the way all the experts do it. The making of a painting or the fixing of a motorcycle isn't separate from the rest of your existence. If you're a sloppy thinker the six days of the week you aren't working on your machine, what trap avoidances, what gimmicks, can make you all of a sudden sharp on the seventh? It all goes together.

The real cycle you're working on is a cycle called yourself. The machine that appears to be "out there" and the person that appears to "in here" are not two separate things. They grow toward Quality or fall away from Quality together._

<comment><USER3> That was a great quote. I guess no matter how much you know about the latest and greatest paradigms involved in the art of software. You can never truly become "perfect" if it does not become a part of yourself. And to extend that you can never truly become "perfect" if your description of software is not art, but work. When I think about it, it only shows how far I have to go to reach that kind of Zen like state.

</comment></comment><comment><USER4> This brings to mind an old saying:

      You can't polish a turd

The underlying assumption to the post is that somewhere deep within the turd being polished there is a nugget of gold.

Not every turd is hiding a nugget of gold. And not every polisher of turds has the capacity or understanding to find the nugget.

The other saying that comes to mind is:

      There is an art, it says, or rather, a knack to flying.
      The knack lies in learning how to throw yourself at the
      ground and miss. Pick a nice day, [The Hitchhiker's
      Guide to the Galaxy] suggests, and try it.

<comment><USER5> _The underlying assumption to the post is that somewhere deep within the turd being polished there is a nugget of gold._

It's not that so much as that in the process of polishing the turd, it's possible for you to realize you're working on a turd and start over again. That's part of the revision process.

One case that stands out to me was that in the writing of my novel, I had a 70-page part that I thought was great and slaved away on, then realized in one blinding moment that it was awful. I threw it away, rewrote the entire section over a day, and now it's the best part of the book. (I'll say in advance that the book wasn't exactly a nugget of gold, but at least I was aware of that when I published it. If I'd spent more time in it it could have become much better.)

</comment></comment><comment><USER6> I don't doubt that making something awesome is about removing the unawesome parts. But what most people are missing is the sense of what is unawesome and what is awesome. This trait of having excellent taste is what most are missing... at least from my experience of interacting with others.

The people who are really good at something have an evolved sense of what they are willing to accept. For example, I knew of a guy named Rod Weckworth who was/is really good with relationships. And I think he was really successful because he had a sense of what level of a connection he was willing to accept with people. He wasn't okay with having an okay relationship with people. He makes sure to write thank you notes, call on birthdays, etc.

So more than just not letting shit stay, you have to have a nose for detecting shit.

<comment><USER5> Yeah! On IRC somebody read this and disagreed with me: something to the effect of "I've seen amateurs revise and they still churn out shit."

But you can't just sit down and revise something. You've got to develop the ability to be self-critical, and that means learning what to be self-critical _of._ So in my case of an amateur writing a symphony, he'd have to be willing to learn just what makes symphonies suck, and if he's not willing to do that, chances are he won't be so successful after all.

</comment></comment><comment><USER7> "In the same way I fancied that those nations which, starting from a semi- barbarous state and advancing to civilization by slow degrees, have had their laws successively determined, and, as it were, forced upon them simply by experience of the hurtfulness of particular crimes and disputes, would by this process come to be possessed of less perfect institutions"

\- Rene Descartes, Discourse on Method

I think that Descartes' thrust here is that building anything up slowly will introduce artifacts that have nothing to do with the optimal solution.

Sometimes the only way to truly succeed is to come up with a truly amazing starting point. If you try to build off a bad idea, you will often get nowhere.

From the article: "It baffles me that people think making...worldshocking pieces of work is a particular challenge. It's mainly a battle of endurance."

I don't disagree with you here, but I think you are vastly underestimating the backbreaking effort in that single word: _endurance_. Try telling a marathoner that just hit the wall at mile 22 that all they need is a little more endurance.

"Genius is one percent inspiration and 99 percent perspiration."

\- Thomas Edison

<comment><USER8> I was struck, on reading your post, the similarities between your premise and the premise of "Intelligent Design" advocates. To paraphrase both you and they, "How can something so perfect have occurred iteratively?"

Not judging, just noting the similarity.

<comment><USER7> I admit there is some similarity there, and it might be because I didn't fully explain myself, but I would argue that it's neither a perfect conception nor iterative development that creates true greatness, but repetitive rebirth. When you simply build on the past, you aren't considering its true intentions. How many laws are there in the US that are simply reactions to British rule (i.e. safety from the quartering of troops)? Should those laws be held in as high esteem as those laws that are truly based on human rights? Descartes continues in _Discourse on Method_ :

        As for the opinions which up to that time I had embraced, I thought that I could not do better than
        resolve at once to sweep them wholly away, that I might afterwards be in a position to admit either other
        more correct, or even perhaps the same when they had undergone the scrutiny of reason. I firmly believed
        that in this way I should much better succeed in the conduct of my life, than if I built only upon old
        foundations, and leaned upon principles which, in my youth, I had taken upon trust.

The entirety of _Discourse on Method_ is a very interesting read. You can read the whole document at project Gutenberg (<http://www.gutenberg.org/etext/59>). Wikipedia also has a decent write-up (<http://en.wikipedia.org/wiki/Discourse_on_method>).

</comment></comment></comment><comment><USER9> "The longer you try and deshit something, the less shit it is."

Not true for the 20%, but true for the 80%. Software is a good example : all 1.0 versions are crap. The trick is to get to the next n revisions and still be interested in the project. You may end up with none of the original code, but it's the act of revision and correction that makes it better.

All those who say you can't polish a turd, I say look at Windows. From version 1 to version 7, there's a gloss in the side of that nugget that you would not have believed possible 20 years ago.

</comment><comment><USER10> Someone said it best, I can't remember who it was: _"It's like knowing a fabulous sculpture is hidden inside a block of marble, and all you have to do is remove the marble that isn't part of it. It's an encouraging thought, because it reminds you there is an answer, but it's not much use in practice because the search space is too big."_

</comment><comment><USER11> If it's easy to make something incredible, this post should be, and it's not. It's ok, but it's not incredible.

(Except in the original, literal sense, because of the fact that it's not in the colloquial sense.)

<comment><USER5> I'll explain something about this blog, because it's not something that I wrote specifically on the page and so it's not seen by a lot of the people that haven't been reading this daily.

I don't use this blog like other people use blogs. I essentially write down every thought that comes to me, without planning it out or revising it. So there's a ramble about Valve on the front page right now, and not a particularly interesting one, and I'm aware of the fact that it's not interesting. I'm fine with that, because my blog isn't a marketing tool. It's simply a place where I store my thoughts.

So this post was conceived of pretty much instantaneously, written in 5 minutes, and posted without a double check. I didn't even bother to go back and look at the spelling.

It's all about intent. My blog is not a masterpiece in any sense other than the fact that I attempt not to filter myself in any way. But at the same time, this post was interesting enough to provoke a nice little discussion, and it was interesting enough for somebody to think it merited posting.</comment></comment>


FBI 'Grappling' With Hiring Policy Concerning Marijuana - <AUTHOR> http://blogs.wsj.com/law/2014/05/20/director-comey-fbi-grappling-with-hiring-policy-concerning-marijuana/

<comment><USER2> A lot of things haven't really changed, some things have gotten worse, but it's been quite amazing to see the relatively rapid movement in the US on marijuana and gay rights in the past few years. I'm not quite sure how it happened.

I used to idly browse pollingreport.com a lot, looking for issues where a majority of American opinion didn't line up with the general political climate. Environmental issues are perhaps another area where we could see a major shift in the near future.

<comment><USER3> I don't have a lot of hope for environmental issues. Cable news has managed to turn climate change into a debate where everyone's opinion matters. You even see people who should really know better, on sites like hackernews, saying things like, "there's no scientific consensus" and "we think global warming is happening, but we don't really have an idea of how much of it is the fault of humans".

</comment></comment><comment><USER4> I mean...there are plenty of really good people who don't smoke marijuana too, and they do an exceptional job recruiting them from schools in the DMV area.

At the University I went to, anyone who expressed interest in security and had good grades could get a ridiculously generous scholarship (tuition + $20k stipend + books + $1000 for conferences, and a guaranteed summer internship). A catch to this is you must be able to get a clearance, and if your clearance is denied you lose the scholarship and have to pay back the money.

Basically, all the students who were not into that kind of stuff went straight for it and had jobs at 3 letter agencies on graduation.

Yeah a lot of security people like marijuana but a lot of them don't.

The way I looked at it was "If I'm going to choose to smoke marijuana I have to be twice as good as everyone else in order to ensure I get a job that does not require a security clearance" (which is odd because I later chose not to smoke marijuana).

But...there are plenty out there who already have no problem getting jobs at these agencies after being clean for 1year or just not being interested in those things.

</comment><comment><USER5> I would like to say that this is a remarkable change in rhetoric from an organization that has been lousy with dogmatic principles since its founding.

yes, there are non-marijuana users that are fit for those jobs. yes, the fbi could pay better to attract talent.

this is a fed agency (the fed agency) saying they are thinking about letting crazy wingnut pot smokers help them defend america.

this is a meaninful change in organizational behavior that is worthy of note.

</comment><comment><USER6> The FBI has a much bigger problem: their pay scales are crap. A non-field agent tops out at the end of his or her career as GS14, $110K. You won't get hired into that, of course.

<comment><USER7> Is that anything new though? I don't know the historical pay rates for FBI jobs but I wouldn't assume that they've undergone significant budget cuts.</comment></comment>


Rands in Repose: Managing Nerds - <AUTHOR> http://www.randsinrepose.com/archives/2011/01/17/managing_nerds.html <comment><USER2> Its interesting how Rands and Joel agree on the basic quiet/dark/cave/hoodie/headphones "zone" construct, and yet nearly every startup I've visited or seen office pictures of on their website insists on a faux-egalitarian setup where everyone sits in one giant noisy room.

<comment><USER3> It's the classic demo vs. reality problem (or hollywood vs. reality problem). What looks cool for short periods of time often doesn't match what is actually useful for 2,000 hours per year.

An IDE that displays code on the sides of rotating icosahedrons looks really cool for about 3 seconds, but is hugely impractical in reality. Similarly, giant noisy rooms seem more interesting in pictures and in planning but in practice are less useful than individual walled offices.

One of the great things about individual offices in terms of facilitating collaboration is that it actually makes it far easier to have impromptu meetings between a small number of people. In cubicles people tend to avoid such things since the noise is disruptive to others and there's not much room for people, with offices you can close the door and have 3 or 4 people in a single office without much problem or much impact on others. There are many other examples.

<comment><USER4> If cubes for software engineers are such a bad idea why does Google (so I've heard) swear by them?

<comment><USER5> Well for starters, the alternative of giving everyone a private office is really really expensive. Especially if you're also (like me) a believer in the idea that offices should have windows. I had my own windowless office for a year or so, and it was a really depressing environment in which to work -- I much prefer my current two-person office with a view.

The other point is that while an individual programmer might be more productive when he's alone in a room, a _group_ might be more productive if they're all in the same room together. No point in having everyone write twice as much code per day if the code no longer fits together and nobody has any idea what everybody else is doing.

How much of a programmer's time is actually spent sitting down in full-on focus mode belting out lines of code?

<comment><USER6> Of course a cheaper option is to allow remote working to those who want it, at least for most days of the week. Use the office for client meetings and the occasional get-together. That way I get to have my private office with a window, coffee machine, private snack bar, XBox console and anything else I want, and it doesn't cost you a penny.

Besides, groups of developers in large open source projects, distributed around the world, seem to manage fine.

<comment><USER7> On the other hand, being in the same room all day all week can be incredibly depressing. I've done it in the past and prefer any kind of office to working at home for extended periods.

</comment></comment></comment></comment></comment></comment><comment><USER8> As I sit in my cube, mandated business attire donned, bright fluorescent lights flickering above me, and listening to the constant drone that is the heating system, I read this article and utter a small prayer. That somehow, somewhere, someone who actually manages developers reads this blog, and hires me.

<comment><USER9> If anyone is in Atlanta or willing to relocate, we're always hiring Ruby engineers or smart hackers who will learn Ruby and Rails. No dress code and we're getting rid of most of the fluorescent lights. We have fun problems and work with a respectable amount of traffic and data. Shoot me an email; the info is in my profile.

<comment><USER10> I looked at your profile and I didn't find any information. You will need to put your email address in the about field for it to show up.

<comment><USER9> Fixed. I swear I did that right before that last post though.

<comment><USER11> Happens to the best of us mate. I have double-take moments like that with my work all the time.

Makes me paran^H^H^H^H^Hcautious.

</comment></comment></comment></comment></comment><comment><USER12> I'm often poor with my communication (in terms of: clarity, consideration, and courtesy), but when a new piece from Rands comes out I consistently find myself forwarding it or printing it for a loved one and saying "This is what I try to mean when I say ___________"

His Nerd Handbook ([http://randsinrepose.com/archives/2007/11/11/the_nerd_handbo...](http://randsinrepose.com/archives/2007/11/11/the_nerd_handbook.html)) quite likely saved my relationship with a highly non-technical increasingly significant other. I owe the man a bunch.

</comment><comment><USER13> Wow, he hit the nail on the head with the "hoodie" concept. I almost feel like it gives me a power boost.

<comment><USER14> No kidding. I like coding with a hoodie over my head, literally. Never thought about why, but now I know.

<comment><USER15> My big headphones are my hoodie. I could be completely alone, with no ambient noise, and I still put them on. Sometimes they're not even plugged in. My coworker does the same thing.

</comment></comment></comment><comment><USER16> I normally find articles with titles like that painfully condescending, but this was quite well-done and worth a read.

<comment><USER17> The thing to remember is, he came up through the ranks to became a manager, and seems to be one that decided to put the energy he used to into learning new tech/coding techniques/etc into becoming a good manager.

If you've not read anything from his blog before, it's worth at least skimming back through the archives to see what tickles your fancy. He's got some VERY good articles.

<comment><USER18> It's his current knot he's untying...

<comment><USER17> Funny, I failed to tie it back into his post, but that is the perfect way to look at it.

It does make me wonder how many different knots are common for one to be attacking at any given time (not moment, just in general).

</comment></comment></comment></comment><comment><USER19> this article is the "Boyfriend Operational Manual" my girlfriend has been looking for for months!

EDIT: i'm wearing my hoodie, yo!</comment>


Chapel 1.15.0 Released - <AUTHOR> http://chapel.cray.com/download.html <comment><AUTHOR> Here's the change log linked from the downloads page:

[https://raw.githubusercontent.com/chapel- lang/chapel/release...](https://raw.githubusercontent.com/chapel- lang/chapel/release/1.15/CHANGES.md)</comment>


How do you track a plane? - <AUTHOR> http://www.bbc.co.uk/news/world-asia-pacific-26544554 <comment><USER2> The more this story unfolds, the more I realize just how wrong I was about how I thought things like FlightAware and everything involved in tracking an airplane works.

What exactly is the barrier between just having two way satellite communications on every plane for the purposes of tracking and reporting its location?

<comment><USER3> Cost mostly if I got it right? The value in knowing the position doesn't justify equipping all airplanes with some kind of satellite position update system (the two-way part that is), and there is almost no value in that information outside freak incidents like this.

<comment><USER4> I would have to assume that at some point the cost of two dozen countries looking in between every nook and cranny of 1/3 of the planet exceeds the cost of a two-way position updating system, not to mention the value of being able to avoid even one disaster like this and whatever potential related disasters we have yet to see.

<comment><USER5> Yes, now that a plane has disappeared (instead of just being flown to a different airport and hostages kept; or being flown into a building) means that there is a reason to implement costly tracking devices that previously had little use.

I used to build devices for aerospace. These were groundside, never ever airside. The systems (and thus costs) for building these devices were extensive and complex. Every item, every component, every screw, every washer, all the chemicals (paint, conformal coating, etc) could be traced from a finished unit (via its serial number) back through the manufacturing chain. When my company moved from a paperworks systems (suprisingly good) to computers (unsurprisingly flawed) they kept the paperwork system for these products because it was so important to never fail audit.

It's easy to look at consumer grade off the shelf systems and think that the cost would be maybe four or five times higher, but I think the cost would be significantly greater.

Add to that the difficulty of getting any component on a circuit diagram changed and the need to continue producing these units for every aircraft for many years and there's additional complication there.

</comment></comment></comment></comment><comment><USER6> So I worked at Trimble Nav in the radio group, so this by no means perfect || expert on the first go:

Rough specs of something that might be respectable:

      - wind & solar powered
      - externally-mounted, self-contained, near zero maintenance
      - compatible with deicing chemicals & equipment
      - multiple redundant location sources: GPS, GLONASS, LORAN, cell-tower
      - jam-resistant multi antennnae / recvr config
      - highly compressed satellite telemetry with cellular, pager and HF backups
        - location & alt delta every 3 minutes (12 bits +- 100m)
        - absolute location & alt every 8 hours (46 bits +- 100m)
      - physically hardened against several hours blunt-force phsyical damage
        - (eg enough to slow down a die-grinder w/ a diamond cutoff wheel)
      - pretty app similar to google earth where carrier operators can see their fleet live or in the past

Unit cost of ~$100k USD at first, getting to $35k at scale

Shoe-string budget dev costs: $1.2 mil for a bump to fit the most popular model of jet first, then expand to others and airbus if successful.

Assumes engrs that can bust it out quickly and hustlers that can finagle enterprise & get distribution (Eg make it an FAA mandatory device for classes of airframes).

</comment><comment><USER7> They also omit the fact that the 4 digit code in the transponder is entered _by the pilot._ When I've flown with friends typically the transponder is set to 1200 until ATC tells them what number they want them to emit, and then the pilot sets that number and from then on they know who they are.

<comment><USER8> It is worth mentioning that transport aircraft carry Mode S [0] transponders which emit a unique ICAO 24 bit address assigned by aircraft registration. This data is available regardless the pilot-entered 4 octal digit Mode A transponder code. The transponder can still be turned off, however.

This contrasts to small aircraft which typically have Mode A/C only (no mode S).

[0] [http://en.wikipedia.org/wiki/Aviation_transponder_interrogat...](http://en.wikipedia.org/wiki/Aviation_transponder_interrogation_modes)

<comment><USER9> So.... why?

All airplanes should have a MAC address. No plane should have a user- servicable-power-switch to the transponder.

Can you tell why the opposite woud be needed?

<comment><USER10> In case an electrical fault is detected.

Protecting the passengers from the pilot is anyway a false goal. You can't really do it without removing the pilot.

<comment><USER9> This and: ___(electrical faults, interference with airport radars, safety of maintenance technicians)._ __

Sound like utter BS.

Show me a case where an IN-FLIGHT transponder shutdown is a needed issue, or where this does not need a better engineered solution.

Also - Don't tell me why this is not done (based on past issues) - tell me why this is impossible to fix.

I challenge you to give me any reason in the universe which is acceptable for not tracking, in real time - uninterrupted streams, flying packages of hundreds of humans.

Give me any reasonable response to why this is not something that should be fixed.

<comment><USER11> >Show me a case where an IN-FLIGHT transponder shutdown is a needed issue

How would the system determine it is in-flight and thus prevent a maintenance shut down? You would need another device to determine in-flight from ground operations, and thus another device that can malfunction and thus can be defeated by an attacker.

>Give me any reasonable response to why this is not something that should be fixed.

Give me a reasonable response why spending a bunch of money to fix this rare issue is worth it over other safety improvements? Old aircraft, poor maintenance, and overworked pilots kill many times more people than hijackers.

P.S. Why don't we track buses? Why don't we track cars? Why don't we track everybody?

<comment><USER9> >P.S. Why don't we track buses? Why don't we track cars? Why don't we track everybody?

Buses are at alt==0 and never over an ocean - you're an idiot for comparing the two.

Why don't we track everybody? Do you understand what the NSA has been doing?

<comment><AUTHOR> I'm sure you have valid points to make, but I would ask that you reconsider how you make them. From the guidelines[0]:

        In Comments

        Be civil. ...

        When disagreeing, please reply to the argument
        instead of calling names. E.g. "That is an
        idiotic thing to say; 1 + 1 is 2, not 3" can be
        shortened to "1 + 1 is 2, not 3."

[0] [http://ycombinator.com/newsguidelines.html](http://ycombinator.com/newsguidelines.html)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> What is the first thing you do when you land? I don't know about you, but I check my phone. If even one phone on that plane was not off or in airplane mode it would immediately connect with the nearest cell tower no matter where they landed, if there was a cell tower in range that is. I know for a fact that I've simply forgotten to turn my phone off... on occasion... and sure enough, my phone starts ringing as soon as I land.

Now, who here would say that the NSA isn't backdoored into every cell tower on that planet. I wouldn't be surprised if they are. How could we not know as soon as one of the cells on that plane popped up on a tower anywhere in the world? So, exclude all cell tower coverage areas in projected flight area and search there to rule out potential hijack/landing.

<comment><USER13> You're assuming that the passengers are still alive (the plane climbed to 45,000 feet after disabling its transponder), and if they are, that they haven't been intimidated into giving up their phones.

<comment><USER14> What does getting the last known cell phone tower contact have anything to do with whether anyone is alive? Like the OP says, all that is a _single_ phone making contact with the nearest tower.

<comment><USER13> Most people still turn their phones off before a flight to save power or because they think it will keep them safe. If they are dead, they can't turn the phones back on and the hijackers can go around searching for phones if they really want to. So it's pretty unlikely that phones will attempt to make contact with cell towers.

But assuming that a phone does make contact, there would be massive cooperation required to detect it. The airline and victims' families would have to give a list of phone numbers that could be on the flight, and all the carriers in the region would search through their towers' logs to see if any of those IMEIs (assuming they keep those logs for at least a week).

<comment><USER12> All it takes is one phone to be on, regardless of human intervention. I would wager someone forgot to turn their phone off. Nevertheless, if the flight was hijacked and the hijackers were pros, as they clearly must be, they might have had a cell jammer. Regarding IMEIs, cell numbers of passengers, and tower logs, well that's what we have all those 3 letter acronym agencies for. I'm sure they haven't just been sitting around all this time. I don't think it's just the navy and air force sending search and rescue.

</comment></comment></comment></comment></comment><comment><USER15> Does anyone know the exact information that the Rolls Royce engines would have transmitted if the airline had paid for the "basic plan"?

</comment><comment><USER16> Forgive my ignorance, but why the plane could not fly straight rather than north or south that is put forth as the only options? Straight is as suicidal as going south, still a vast ocean ahead.

<comment><USER16> Somebody down voted a genuine question, and left no comment at all for why did it? There's no corner of the world short of mean people.</comment></comment>


Early Stage Investing in New York Got Supercharged Mid-2009 - <AUTHOR> http://www.betabeat.com/2011/08/02/early-stage-investing-in-new-york-got-supercharged-mid-2009/

<comment><USER2> It will be good to see what are core reasons for this. Any correlation between recession & increased startup activity for example?

<comment><USER3> Wall Street layoffs.</comment></comment>


ICANN urges adopting DNSSEC now - <AUTHOR> https://www.networkworld.com/article/3343185/icann-urges-adopting-dnssec-now.html <comment><USER2> DNSSEC is an awful standard that doesn't even solve half the problems that DNS has.

Edit: To offer more constructive criticism: I'd rather see DNSCurve [http://dnscurve.org/](http://dnscurve.org/) see wide adoption.

<comment><USER3> It's a pity that you link to a page that compares DNSCurve to plain DNS.

DNSSEC has the feature that it works even in the presence of caching (recursive) resolvers.

DNSSEC only solves half of the problems. That's why you use DNS-over-TLS (or DNS-over-HTTPS) for the other half.

<comment><USER4> DNS-over-TLS is only intended for stub resolver - recursive resolver traffic (check the introduction sections of the relevant RFCs). We're still lacking a protocol to provide confidentiality between the recursive resolver and authoritative servers. DNSCurve would offer that.

<comment><USER3> Which makes sense, because there is (in general) not much point in protecting traffic between recursive and auth. servers if you don't protect the stub resolver.

At the same time traffic between stub resolver and recursive resolver is much more privacy sensitive.

So it makes sense to first get TLS between stub and recursive up and running. And that proves already hard enough.

<comment><USER5> Technically stub resolvers and recursive resolvers are the same thing (one is an inherent part of the other). So all is needed is recursive resolvers to communicate securely with authoritative servers. Tunnels to third party recursive resolvers do nothing but leak private data to them.

</comment></comment></comment></comment></comment><comment><USER6> I don't have the DHS "emergency directive" (referenced in the article) handy at the moment (and it's been a couple months since I read it) but I don't recall _ANY_ recommendation (or "directive") by DHS for the government agencies to implement DNSSEC.

In fact, if memory serves, DNSSEC was not mentioned _AT ALL_ in the DHS document.

That's somewhat telling, IMO.

I'm not sure what ICANN's motivations are in pushing DNSSEC so hard -- all of these "hijackings" were due to compromised accounts (at DNS providers, registrars, etc.).

Instead of requiring government agencies to implement DNSSEEC, DHS instead directed them to rotate/secure their accounts/passwords at the DNS providers, registrars, etc.

That makes much more sense to me.

<comment><USER7> OMB memorandum M-08-23 released in 2008 already requires US federal government agencies to use DNSSEC.

<comment><USER8> Since repealed, right?

 _Later_

Yep. It was rescinded in 2018. There is no longer a mandate for DNSSEC in federal government IT that I'm aware of.

</comment></comment></comment><comment><USER9> I don't understand what's the point of DNSSEC. Literally the only attack scenario that DNSSEC protects from is when someone is MITMing your ISP, which, I believe, is something that practically never happens.

It doesn't do anything about the most common attack scenarios, such as MITM between the client and the last mile DNS server or hackers stealing your domain registrar credentials.

To be honest, it seems to me that the _real_ purpose of this protocol is to make people/enterprises feel safer without actually doing anything useful.

<comment><USER10> > Literally the only attack scenario that DNSSEC protects from is when someone > is MITMing your ISP, which, I believe, is something that practically never > happens.

It also helps when your ISP is MITMing your attempts to get accurate DNS information. That happens _all the time_ , though most of it is just sending you to advertisements when you're supposed to get NXDOMAIN.

<comment><USER9> No, it does not protect from that. Stub resolvers (i.e. DNS clients) fully trust recursive resolvers by design and don't validate DNSSEC signatures.

In theory the client _can_ validate them but, for various reasons, pretty much none of the implementations do that.

</comment></comment></comment><comment><USER11> I don't expect a lot of movement, given things like:

 _" If you want to configure DNSSEC for a domain that is registered with Route 53, you must use another DNS service provider"_

<comment><USER12> Yeah. It's extremely disappointing that AWS hasn't implemented DNSSEC in Route 53. Even more so, since they have actually been a target of an attack on Route 53, that would have been ineffective if DNSSEC had been enabled.

<comment><USER13> If you are buying a domain through Route 53, there is a good chance the purchase is actually being handled by Gandi[0]. Gandi supports DNSSEC[1] (it's literally a check box in the options, very simple) so you may as well skip buying the domain through Route 53 and get it directly from Gandi.

No affiliation with either company, just a happy customer of both.

      [0]https://aws.amazon.com/route53/domain-registration-agreement/
      [1]https://docs.gandi.net/en/domain_names/advanced_users/dnssec.html

</comment></comment></comment><comment><USER14> Waiting for tptacek...

<comment><USER8> I feel like HN is getting to the point where, if I'm 24 hours late to a DNSSEC story, other people will have gotten the main points I'd have made first, and those points will be voted high on the page. You're all grown up, now, HN.

</comment></comment><comment><USER15> > "Some of the attacks target the DNS, in which unauthorized changes to the > delegation structure of domain names are made, replacing the addresses of > intended servers with addresses of machines controlled by the attackers. > This particular type of attack, which targets the DNS, only works when > DNSSEC is not in use," ICANN stated.

Isn't that a bald-faced lie? The attacks in question are "someone stole my credentials and told my DNS provider to change the records to point elsewhere", as I understand them. DNSSEC merely protects against DNS spoofing on the internet backbone (not everywhere, because you're not supposed to turn out in when querying your ISP's DNS server, so the last mile isn't protected).

<comment><USER3> The argument seems to be that if your attacker is stupid enough to only change your NS records and doesn't know about DS records, then DNSSEC will provide some protection.

It is not clear with ICANN mentions this, because somebody hacking your registry account (or somebody hacking your registry or the registrar) are attacks DNSSEC cannot protect against.

<comment><USER16> A threat model where the attacker is assumed to be stupid is a very weak threat model. Better to assume that the attacker is at least as knowledgeable as you are.

</comment></comment></comment><comment><USER17> > "However, most implementations are incompatible with modern DNS > requirements, including redundant DNS setups or dynamic responses from DNS- > based traffic-management features," Beevers said. "Legacy DNSSEC > implementations break even basic functions, such as geo-routing, and is hard > to implement across multiple vendors, which means poor performance and > reduced availability for end users."

Does anyone have more information related to the above quote? I'd like to understand better what they're referring to.

<comment><USER18> I can field this one. DNSSEC was conceived with support for a traditional zone transfer process. When you edit the zone, you sign it once (potentially offline) and send it via zone transfer to all your authoritative servers. In that context, the authoritative servers can't do things like serve the best A or whatever record given the recursive resolver contacting them and global state. Instead, you generally end up with your keys online on all the authoritative servers. Which means if your DNS account is hacked, chances are, the hackers are in a position to change your records. If your registrar account is hacked, the hackers are certainly able to remove or replace the DS records at the registry, so DNSSEC doesn't help there either (good auth at the registry and registrar and maybe registry lock help there).

DNSSEC might help with MITM, if you only communicate with recursive resolvers you trust to properly implement it and not sneak in any garbage, but it doesn't help against account compromise.

<comment><USER19> > serve the best A

Well, you have to serve them all and let the client decide what to use, like with MX or SRV. Too bad HTTP is stuck in time and can't get any proper functionality.

And yes, I mean it is literally too bad. Why can HTTP hold 3 current standards at the same time, but still fail to work with basic internet infrastructure?

<comment><USER8> Are we at the point of desperation in the death gasps of DNSSEC where its advocates are now blaming other protocols for not rearchitecting the way they use the DNS to make it DNSSEC more viable? Because: nobody was ever going to do that.</comment></comment></comment></comment>


Ask HN: Your thoughts on recreating a Digg.com like Website? - <AUTHOR> <comment><USER2> Why recreate it, if it still exists?

<comment><AUTHOR> With v1 features</comment></comment>


Distribution packages considered insecure - <AUTHOR> https://statuscode.ch/2016/02/distribution-packages-considered-insecure/ <comment><USER4> I've always considered the Debian model of support (freeze the world and try to support everything) to be wrong and swimming against the tide. I believe that the only ones capable of supporting a package of a given complexity are the authors themselves, and the distros should just handle packaging (and minimal patching on top if necessary). If the author drops support, then you should either be _extremely_ confident you have an adequately strong team to fork and maintain the package...or just don't.

In my humble opinion, the FreeBSD ports model is better in that regard. That's also why I try to use pkgsrc for various packages when maintaining systems running LTS Linux distros.

<comment><USER5> Reason why authors can't support packages is that their views may not represent one of Debian and this is why there external maintainers. E.g we have open source game engine and have launcher to download mods and Debian privacy policies don't allow it to check mod updates by default.

Also there is authors that don't care to support ancient library versions that present in all distributions or simply don't care about anything except static linking. Some people also don't use Linux at all even if their project does support Linux.

<comment><USER6> As an example, "ancient" for many projects means two years.

That's when enterprises might start considering moving onto that version, so the incentives aren't really aligned here.

<comment><USER7> > That's when enterprises might start considering moving onto that version, so > the incentives aren't really aligned here.

Easier said than done in many environments. RHEL 6, released in 2012, is still supported until 2017!

</comment></comment></comment></comment><comment><USER8> This is a real problem. It is a total joke to suggest that people move important systems to Archlinux though. Basically as an administrator you need to be aware of security advisories against the OS you run. This is already true because you need to know when to upgrade packages. If you are running Debian you should sign up for debian-security-announce:

[https://lists.debian.org/debian-security- announce/](https://lists.debian.org/debian-security-announce/)

There are equivalent mailing lists for CentOS. It would probably be helpful to have better tooling to warn about/mark packages with known vulnerabilities associated.

I'm not an expert on Debian's internal process but sometimes it seems that Debian add packages to their distribution without a clear plan for how to fix security issues in them. Sometimes upstream maintainers seem hellbent on making it impossible to offer long term support for software. Elasticsearch is a case in point:

[https://www.debian.org/security/2015/dsa-3389](https://www.debian.org/security/2015/dsa-3389)

<comment><USER9> Lazyweb question. Is there a way for me to easily keep track of important upstream security announcements for some given set of software—say, ElasticSearch, Rails, nginx, and Linux? Like I could check those check boxes and get an RSS feed, or something?

<comment><USER10> Related to open source, you probably want to track OSS Security mailing list [1]; or if you're part of an organization relying on open source software, you should have someone looking at that list.

I think it is a good complement to the security advisories of your Linux distributor (Debian, CentOS, whatever), but is isn't free as you'll need time to keep an eye on it.

1: [http://oss-security.openwall.org/](http://oss-security.openwall.org/)

<comment><USER9> Thanks. Regarding cost... someone should sell curated security feeds to startups.

There must be many dozens of engineers who monitor mailing lists for issues in (let's say) ElasticSearch. They could be incentivized to share their findings.

Then if one source tags a certain CVE as significant, I get a ping. If several sources tag it, the ping gets more urgent. Eventually I feel scared enough that I upgrade.

<comment><USER11> There was something made to ease exactly that pain... sadly it's no longer updated: [http://www.websecuritywatch.com/](http://www.websecuritywatch.com/)

</comment></comment></comment></comment></comment><comment><USER12> > But can you really be sure that people that do this stuff as an hobby can > deliver this in the quality that you expect and require? Let's be honest > here, probably not.

This must be the most short sighted description of Debian developers I have to see, cause 1) most do packaging as part of their paid job 2) apparently millions of people in the world believe that, yes, these people do a fine job at maintaining those packages. Not to mention those who develop the software they package or those who do academic research about software packaging and dependency resolution. But insulting Debian has become commonplace nowadays.

<comment><USER7> The author also seriously recommends Arch Linux and Tumbleweed as alternatives. Anyone who ever maintained a server in an enterprise production environment knows how valuable the kind of stability that the author criticisms is. Not everyone lives in a DevOps environment where you can "move fast and break things".

He does have a point about unmaintained packages, though. Debian probably shouldn't package things if they cannot support it over the lifetime of the release. Why would I ever want to use a two-years-old Wordpress version? I never understood why those kinds of applications are even packaged at all.

Most sysadmins deal with this by including a third party repository for applications and Elasticsearch (any many others), often by the authors themselves, which takes care of security and version updates. This eliminates most of the concerns. You get a rock-solid operating system and an up to date application on top of it. Web applications like Wordpress or Owncloud aren't usually deployed using distro packages at all.

<comment><USER13> Countering one of your points, and yet bolstering your statement here. Even in a DevOps environment, packaging is important. Same as a system image or random artifact, a distribution package can provide consistency and repeatability.

If I had to wait for a rebuild of the entire operating system to test deployment of a single application, how much confidence could I possibly gain in that application's deployment process?

Also... it seems the author is suggesting that newer software always has fewer bugs. Hmm... .

</comment></comment></comment><comment><USER14> Well, you have to chose something. You can't have security, automated updates to latest version and stability in the same place. You're either in debian/RHEL world with old kernels, old libraries and old userspace tools which miss a lot of fresh features or in npm/pip/curl|bash world, where you have latest version of everything all the time.

In former case you can do `apt-get upgrade`/`yum update` and be almost sure that everything will continue working, but - no - you can't have PHP 7.

In latter case you either use npm shrinkwrap-like tools to install the exact same version of everything every single time, or play Russian roulette with the new dependency versions. And - just in case if you didn't notice - <USER2> you pin some package to a specific version you no longer receive security upgrades for it. And let's be honest - you have a lots of those "^1.0.1", "~0.10.29", "^0.3.1" things in you package.json/Berksfile/... And for almost any package "^0.3.1" is the same as "0.3.1", cause the next version will obviously be "1.0.0" and 0.3.X won't be receiving any more updates.

It's obvious that no single distribution will be able to package the insanely large amount of packages from all the different sources, let alone backporting patches. So you either limit yourself to only the stuff available in your distribution, or you're on your own with updates (including security ones).

As for the packages updating themselves, sometimes it's a good thing, sometimes it isn't. I bet a wordpress installation which can't overwrite itself (because is owned by root), and doesn't allow executing user-uploaded .php files will be much more secure than one which has full access to itself.

P.S. no amount of tooling can solve this problem. If you're using version X of package A, then you find out that there is a security vulnerability in version X which is fixed in version Y and version Y is not fully compatible with version X (changed an API, config file, anything else in a backwards- incompatible way), you're semi-screwed. You will have to handle that situation manually.

<comment><USER15> I personally find that Debian stable for most packages, and then vendor/Packager repos for specific software (eg percona, varnish, dotdeb, nodesource, etc) works very well.

<comment><USER7> This is exactly what any company using Debian in production does.

</comment></comment></comment><comment><USER16> This seems to less a complaint about packages and more a debate over Rolling vs Non-Rolling Distro's

The Author seems to take the opinion that Rolling is always better for Security because he selected a few "web" packages and found vulnerabilities in the shipping versions.

Ofcourse for the people running mission critical enterprise applications they probally are not running phpmyadmin on that server so they could care less if the repo for their stable version of debian contains that older software.

I agree it is problem, but the solution to that problem can not simply be rolling all the time.

I use Arch on my Desktop system, but I would never run my employers mission critical database on Arch... I need stability and security, not simply security.

Ultimately I think we need a better way to define the Core OS, vs "User Applications"

Things like glibc, the kernel, etc are clearly coreOS, things like phpmyadmin are User Applications. Where it get grey is databases, webserver etc. Do you lable them a Core product or a User App, if I was running a mission critical applications I would not want my mariaDB system just upgraded to the lastest version with new features and possible breaking (removing) older features that my app may still need.

Enterprises move slow, I still have enterprise applications that require Java 6, I have some things that still have software that only run on Windows 2000. This idea that Rolling to the latest version of software all the time is a workable plan highlights the authors ignorance of how enterprises actually work.

<comment><USER17> The FreeBSD folks have made a pretty good separation between "base" (what you call Core OS) and "ports" (what you call user applications). It works wonderfully.

This isn't a new observation, and I'm kind of surprised it hasn't made it into mainline linux packaging philosophy yet.

<comment><USER16> My knowlege of FreeBSD is limited, but from what I understand this still does not resolve the issue.

While they have seperated the base OS, they have not resolve the issues of applications that can be mission critical, thus warrant to be frozen like the base os is. Things like apache, mysql, etc etc etc.

If you treat them like user apps then they will be updated when maybe I did not want them to be, but if you treat them like base os then you have the problem the author outlines.

Fruther my understanding is the ports system only supports the lastest version of FreeBSD, meaning if you want to use ports you always have to update to the latest version when released.

I have CentOS and RHEL Servers running version 5, if it were FreeBSD I would not be able to use this 2 release old version of FreeBSD and ports, I would have to upgrade.

<comment><USER17> >> Things like apache, mysql, etc etc etc.

That's well beyond the remit of the OS developers who are only responsible for FreeBSD, but the ports team has happily provided us with `pkg lock` to prevent edits to ports once they are installed.

>> my understanding is the ports system only supports the lastest version of FreeBSD

I believe the the ports tree will be compiled for a given branch until the underlying branch is EOL. FreeBSD 8, for example, had binary packages built from 2009(?) until the branch was EOL'd last year. That's six years of support for 3rd party software, after which you can maintain/build your own pkg repository.

I understand the desire for longer environment lifecycles in FreeBSD (I hate the continuous upgrade cycle myself), but without a company stepping up and providing long term commercial support, I'm not seeing how a volunteer project as small as FreeBSD can do more for longer.

<comment><USER16> >>That's well beyond the remit of the OS developers who are only responsible for FreeBSD,

That is exactly the point the author is making by declaring that "packages are not secure". they are not maintained by the OS, but are distributed as "official" thus giving some users the illusion they are supported or "in the remit" of the OS Developers

</comment></comment></comment></comment></comment><comment><USER18> > Did you know that for example Wordpress comes with automatic updates

I grant that Debian is not great at webapps. OTOH giving an application write access to itself is inherently risky. It's too bad shared webhosting and its limitations has so warped the PHP application community.

An alternative model: web app has r-x on the app files, and an app specific admin user has rwx to run the check and update script on a regular basis.

<comment><USER19> WP has the ability to use (S)FTP to update the files rather than direct file access.

That said, it's intended for the long-tail of sites. If you know how to run an update script, you're already more advanced than a fairly big chunk of the WP userbase, so you probably should be doing updates that way. Many professionally developed sites will be using source control, so autoupdates are disabled anyway.

<comment><USER7> > WP has the ability to use (S)FTP to update the files rather than direct file > access.

Well that doesn't help at all with security.

</comment></comment></comment><comment><USER20> I think that labeling point release distribution package managers as insecure because they are sub-optimal for a certain use case (updating web applications) seems a tad too excessive.

Having software go through a decent distribution's packaging process not only provides stability guarantees, but also off-loads many tasks that you would otherwise have to preform on a per-project/per-developer basis.

I perfectly satisfied with the trade offs.

</comment><comment><USER21> The author makes some important points, but there is a cruel irony: He's a main developer of owncloud, which in terms of security updates is a huge problem.

Owncloud has its own update mechanism, which unfortunately usually doesn't work in the real world (it breaks if you have any reasonable timeout for your PHP applications, which every normal webhoster has). There are likely countless owncloud installations with known security issues that their users tried to update, but couldn't. (The alternative is a manual update on the command line, but given the target audience of owncloud it's safe to assume that many of its users aren't capable of doing that.)

</comment><comment><USER22> It is true that in the stable branch there are dated version numbers, probably most of the time for a good reason (e.g. long term support).

On the other side I adhere to Slackware's vanilla philosophy and with slackbuilds you can have always fresh and up to date software.

Well, if you do this you should have dev, test, prod chain for you servers, otherwise you update at your own risk.

Last upgrade I went for Ubuntu12/Node0.10 to Ubuntu14/Node4 but also nginx changed and even logrotate from 3.7.8 to 3.8.7 introduced few modifications that broke my configuration files.

Upgrade is not that easy.

I would like to share a project of mine that brings vanilla philosophy on every distri. You have a script to build your software and installing it locally, so for example you have version z.y.x installed oj your system and you want to install z.y.(x+1) released yesterday.

Normally you download the tarball, bla bla bla and launch make install, most of the time you follow really similar steps, you can put in a script and launch

.software_install Foo

if the version is the same as in the README.md or even

.software_install Foo 1.2.3

to install a specific version. It is really was to add new software to the list. You can also package your software to avoid compile time on other hosts (test and prod). Give it a try, I think it can be useful to many system administrators and developers:

[http://g14n.info/dotsoftware](http://g14n.info/dotsoftware)

<comment><USER7> Your project reminds me of 0install.

</comment></comment><comment><USER23> Yup, the guy still doesn't have a clue what he's talking about.

[https://news.ycombinator.com/item?id=11095783](https://news.ycombinator.com/item?id=11095783)

<comment><USER24> The author brought evidence -- CVEs that aren't getting fixed in Debian repos. What's your counter-evidence? Letting software keep up with upstream is as insecure as upstream, no more or less. What's the evidence that pinning upstream to some older version and backporting some (not all) security patches and bug fixes is more secure?

<comment><USER25> _The author brought evidence -- CVEs that aren 't getting fixed in Debian repos. What's your counter-evidence?_

Just poking around for two minutes finds counter-evidence. Let's take the two critical Wordpress vulnerabilities. He uses an archive.org page of the Wordpress package from February 7. If he actually looked in the Changelog of the Wordpress package for e.g. Jessie, he would have seen that these issues were fixed the day before in stable and two days before in sid:

[http://metadata.ftp- master.debian.org/changelogs/main/w/word...](http://metadata.ftp- master.debian.org/changelogs/main/w/wordpress/wordpress_4.1+dfsg-1+deb8u8_changelog)

According to Red Hat, the CVEs were announced on February 4:

[https://bugzilla.redhat.com/show_bug.cgi?id=1305471](https://bugzilla.redhat.com/show_bug.cgi?id=1305471)

So, I guess that it depends on your definition of "aren't getting fixed". But the way the author writes it, it seems that issues are lingering from weeks/months, which does not seem to be true.

<comment><USER26> The archive.org links were used at the creation of the blog post. (which was not necessarily the release date ;-) - needed to find some time to write it)

So Wordpress is an interesting example. Because the CVE assignment date has nothing to do with the release date of the patches. Wordpress doesn't request the CVE on their own.

So we're still at a 4-5 day delay ([https://wordpress.org/news/2016/02/wordpress-4-4-2-security-...](https://wordpress.org/news/2016/02/wordpress-4-4-2-security- and-maintenance-release/)) for security fixes for a web-facing software. This is still far worse than just enabling automatic updates in Wordpress. I have not much problems if that would be a locally exploitable vuln, but web software usually is exploitable via web.

When it comes to web software I believe it's unacceptable to add any additional delay. (sure those bugs were not that severe, but as other examples in the blog show the problem with delayed or never updated packages is inherent)

<comment><USER27> > This is still far worse than just enabling automatic updates in Wordpress.

A webapp updating itself, having write access to its files (I see zero privilege separation in [1]) and getting updates from a hopefully not compromised source (see Linux Mint lately), that's just asking for trouble. I trust the Debian mirror infrastructure with signed packages that are updated by a privileged system user way more.

[1]: [https://codex.wordpress.org/Configuring_Automatic_Background...](https://codex.wordpress.org/Configuring_Automatic_Background_Updates)

</comment></comment></comment></comment></comment><comment><USER28> I think Debian's long release cycles don't make much sense in this day and age. To me, a rolling release model makes much more sense, especially in this world where security updates are being done constantly, and are generally focused on the more modern branches.

For enterprises where software is part of the core business, keeping up with updates on a regular basis is probably better than doing large upgrades every once in a while (for one thing, tracking down regressions is a lot easier when you don't have to search the entire haystack).

<comment><USER29> By using relase designation instead of toy codenames ("testing" instead of "Stretch") in apt config files I get pretty close to rolling realease. Using that and apt-pinning I am totally happy running stable/testing/unstable/experimental system with majority of packages from testing on my desktop. I am rather conservative and would avoid anything bleeding-edge on production systems so using stable with security updates (and stable-backports if needed) would be my choice. YMMV though.

<comment><USER30> Just want to emphasize (and this is not directed specifically at you): you almost certainly shouldn't run Debian testing on anything that is public- facing. Packages get migrated to testing after some days in unstable if no high-priority bugs are filed against them during the days after their upload.

If a security patch is uploaded to unstable today, you won't get it in testing for a few days, and possibly many more if the migration gets blocked.

[https://www.debian.org/devel/testing](https://www.debian.org/devel/testing)

<comment><AUTHOR> You can install security updates from unstable using the output of debsecan:

apt-get -t unstable install $(debsecan --suite sid --format packages --only- fixed)

<comment><USER30> Just keep in mind that unstable is not guaranteed to get security fixes promptly, either. The Debian Security Team only handles supported releases.

The Security Team FAQ is a good read: [https://www.debian.org/security/faq#unstable](https://www.debian.org/security/faq#unstable)

It's quite explicit in saying that if security is important to you, then you should run a supported release.

</comment></comment></comment></comment></comment><comment><USER31> The conclusion I take from this is that distros need to be a lot more selective in what they package. If packagers can't reliably backport security fixes for the several years that a distro release is supported, they shouldn't create that expectation by putting the package in.

<comment><USER32> OpenBSD purged many webapps from its ports tree one or two releaeses ago for this reason (too many security problems to keep up with). Most such packages didn't do much else than extracting .php files to somewhere in /var/www anyway. Users of such apps are now expected to do this themselves and keep track of updates.

</comment></comment><comment><USER33> A bit meta but articles that use the passive voice such as "Blah considered insecure" annoy me. Considered insecure by whom? The author? The why not just say "Distribution packages _are_ insecure". Sure it makes it sound like there is some consensus here but that does not actually seem to be the case.

<comment><USER34> "x considered y" is such a common template for titles of articles in computer science that it in fact has its own Wikipedia page:

[https://en.wikipedia.org/wiki/Considered_harmful](https://en.wikipedia.org/wiki/Considered_harmful)

<comment><USER33> Ha, from the Wiki, looks I'm not the only one rubbed up the wrong way by this phrase

[http://meyerweb.com/eric/comment/chech.html](http://meyerweb.com/eric/comment/chech.html)

</comment></comment></comment><comment><USER35> Well, I can agree with the concerns. But what are the alternatives? What GNU/Linux distributions do (and Debian in particular) looks to me the least of all possible evils.

</comment><comment><USER36> How does RHEL rate in regard to this issue? Isn't that one of Red Hat's selling points?

<comment><USER25> Red Hat seems to be very swift when it comes to security updates. They provide reports on this web page:

[https://www.redhat.com/security/data/metrics/](https://www.redhat.com/security/data/metrics/)

Nearly all the vulnerabilities were apparently fixed within one day. This is the report of RHEL 7:

[https://www.redhat.com/security/data/metrics/summary- rhel7-c...](https://www.redhat.com/security/data/metrics/summary- rhel7-critical.html)

One thing to note is that RHEL comes with far fewer packages than Debian (e.g. Wordpress and phpmyadmin don't seem to be in the main repository), so you may have to rely on slower third-party repositories.

<comment><USER37> When you actually have a subscription for RHEL, you can also get email notifications whenever security fixes, bug fixes, and/or feature enhancements are released for packages that are installed on your servers. There is some configuration you can do to pick what events you get notified for (like bug fixes only).

And if you find yourself managing an enterprise environment, Red Hat Satellite (basically on-premise RHN + promotion/environment/configuration management) tracks what errata are applied to or required by each system and can report on what the gaps are.

I work for Red Hat, and making it easy to support Linux is definitely our bread and butter.

<comment><USER38> And if you're like me and too cheap to pay for a Red Hat subscription, the CentOS-announce mailing list also announces all the Red Hat errata - <USER3> should absolutely be subscribed to this list if you run CentOS in production.

</comment></comment></comment></comment><comment><USER39> I thought about this issue some time ago:

[http://security.stackexchange.com/questions/109026/security-...](http://security.stackexchange.com/questions/109026/security- updates-foss-upstream-policies-how-are-they-chosen)

I agree that there's a distinction to be made between core/base and user- applications/ports (as mentioned elsewhere in this thread)...

Ultimately it's all softare, and the only distinction is fuzzy: e.g. the kernel won't easily break backwards compatibility, while databases, interpreters, etc. will... but it's not something you can easily measure without being vigilant for every change.

I think that an important distinction (at least for Ubuntu) are the main and universe repositories: I'd expect these problems to happen in universe and to be mostly absent in main.

From this point of view, a good choice would be to completely rely on main, and to weights pro and cons when deciding if using the repositories to manage your user applications/libraries/dependencies for your actual service. (I'd probably define all of them with Nix, but that's not a panacea)

The problem is that even main is guaranteed to keep up with all the security updates: In some cases updates aren't prepared and shipped because the default configuration is not vulnerable (but obviously a sysadmin could change that) and it's not worth the effort.. or just like in the Python2.7.9 case: A security update most often can be applied as a standalone patch, but if the changes are overarching and not easily distilled into a patch, the update will be too expensive/risky and won't be done.

</comment><comment><USER40> Oh yes, this is a big problem. I still run into things like installing ownCloud client on Ubuntu and then wondering why it doesn't work (it's years old). Or recently I found out that Python3-Pandas on the Raspberry pi is version 14.something which has annoyances that have long since been solved. If you install Drush (Drupal Shell) from Ubuntu 14.04, you get version 5.10! (They are at 8.1). Arch Linux is already much better but one breaks things occasionally and they were kicked off of Digital Ocean, sadly.

I really hope Ubuntu Snappy packages, which is essentially the same as the Arch User Repository but more secure If I understand correctly, will solve this messy misery.

<comment><AUTHOR> Based on what you have said it sounds like you want Ubuntu's rolling release, which is upgraded daily.

Ubuntu Snappy packages are based on the normal Ubuntu packages so they will be at the same versions.

<comment><USER40> My understanding was that snappy packages can be descriptions that point to i.e. Github to get latest versions. Hmm, never heard of Ubuntu rolling and it is not at vpn providers I know. But I will check it out!

</comment></comment></comment><comment><USER41> The fact that we haven't solved this yet is puzzling. It seems that we have actually gone backwards in some ways over the last decade. With languages implementing several broken distribution packaging systems trying to compete against each other. Not even to mention the horrible practice of piping random _sh1t_ from the web with curl/wget into a shell.

It would be more or less trivial to implement a wget wrapper that downloads the .sig and validates it. Hardly anyone these days understands or cares which seems to be the bigger issue (not a technical problem but a people's problem as _Gerald M. Weinberg_ would say)

</comment><comment><USER42> Regarding the issue of verifiable builds, everyone should read this:

[http://0pointer.net/blog/projects/stateless.html](http://0pointer.net/blog/projects/stateless.html)

</comment><comment><USER43> The job of the distributions would be much easier if more software projects would a) consequently use semantic versioning (MAJOR.MINOR.PATCH, see [http://semver.org](http://semver.org) for details) and b) explicitly and officially designate that they no longer support the version MAJOR.MINOR branch.

The latter should be a signal for the distribution to upgrade to a newer and supported upstream version instead of (halfheartedly) trying to support the software themselves.

</comment><comment><USER7> Nah, that's why packaging _web apps_ with a stable Linux distribution is a notoriously bad idea and should be discouraged.

Debian (and other distros) do a perfectly fine job updating the actual operating system.

</comment><comment><USER44> This. I agree with this.

Freezing the work does not work particularly with certain packages. For example, I still see lots of usage of old versions of openssl (0.9.8) and openssh.

If we even just make sure to cover these two packages and the related dependencies/affected applications alone, that would go a long way in covering attack vectors (obviously not comprehensive)

</comment><comment><USER45> Light grey, thin, sans-serif body text basically means you HATE your readers.

</comment><comment><USER46> That's why we have `pkg audit` on BSD.

</comment><comment><USER47> Considered harmful is considered harmful</comment>


Greplin (YC W10) open sources 10-15x faster protocol buffers for Python - <AUTHOR> https://github.com/Greplin/fast-python-pb

<comment><USER2> For a long time (much longer than I expected it would take) I've been working on a protobuf implementation in C that does _not_ use Google's C++ implementation at all. I've been through about three rewrites and I finally have the interface right. I'm hoping it will be usable with Python soon (weeks).

<https://github.com/haberman/upb/wiki>

(if anyone's looking at the code, I'm working on the src-refactoring branch at the moment)

The benefits of my approach are:

* you can avoid depending on a 1MB C++ library. upb is more like 30k compiled.

* you can avoid doing any code generation. instead you just load the .proto schema at runtime, so you don't have to get a C++ compiler involved.

* Google's protobuf library does have a dynamic/reflection option that avoids my previous point, but it is ~10x slower than generating C++ code. My library, last time I benchmarked it, was 70-90% of the speed of generated C++.

<comment><USER3> Can you clarify what you mean by "70-90% of the speed of generated C++"?

Suppose that the generated C++ takes 1.0 seconds. Does your implementation take 0.7-0.9s or 1.7-1.9s or something else?

<comment><USER2> If the generated C++ can parse 1MB/s, I can parse at 700-900kB/s. 70-90% of the speed, not the time. So 1.1-1.4 seconds, in your example.

</comment></comment></comment><comment><USER4> I too have a speedy Protocol Buffer implementation in Python:

<https://github.com/acg/lwpb>

It clocks in at 11x faster than json, the same speedup reported by fast-pb. Only with lwpb:

* There's no codegen step -- which is a disgusting thing in a dynamic language, if you ask me.

* You're not forced into object oriented programming, with lwpb you can decode and encode dicts.

Most of haberman's remarks apply to lwpb as well, ie it's fast, small, and doesn't pull huge dependencies. The lwpb C code was originally written by Simon Kallweit and is similar in intent to upb.

<comment><USER5> fast and small footprint, as components should be

</comment></comment><comment><USER6> We (<http://connex.io/>) use Protocol Buffers quite heavily, and Python implementation was the performance bottleneck in many places.

I was working on same thing, CyPB, which is 17 times faster than Google's Python implementation. <https://github.com/connexio/cypb>

This one seems more complete though at the moment. I might just mark the ticket in our tracker as closed and switch to fastpb :-/

</comment><comment><USER7> Nifty. I've passed it along to the appropriate folks.

Google uses SWIG-wrapped C++ proto bindings in Python pretty extensively, so I'm not sure how much this gets over that approach. I checked out the source; it's basically using Jinja templates to autogen Python/C API calls. Basically like SWIG, but not using SWIG.

<comment><USER8> When I was at Google I worked with very large structured protocol buffers in Python at one point. A single piece of data could be hundreds of MB in total, consisting of millions of smaller protocol buffers. I was doing a pass over the whole structure so needed to access each smaller PB from Python.

One day I decided my program was too slow so I profiled it and saw that the hot spots were in the Python protocol buffer implementation. "Easy", I thought, "I'll used SWIGed c++ PBs instead." Made some changes and ran the program again. Almost the exact same run time as before! I profiled again and found that this time the hot spots were in the SWIG layer. I was making so many calls through SWIG to c++ (because I was walking millions of objects), that using SWIGed PBs v. native Python PBs made no difference to my run time. Maybe I could have done some more custom SWIG work to lower the call overhead, but I remember being convinced at the time that SWIG wasn't going to do the trick.

So I ended up writing a 30 line Python extension that processed the protocol buffers in c++ and put the data into Python data structures. Run time was reduced by a factor of 10, hooray!

</comment></comment><comment><USER9> It doesn't appear to actually be open source:

> # Copyright 2010 Greplin, Inc. All Rights Reserved.

Where's the license?

I think the term you want is "publishes", and not "open sources".

<comment><AUTHOR> Good catch - updating now. It'll be under Apache 2.0. (edit: done)

<comment><USER9> Excellent! I mean, Apache's a bit of an overly complex license implementation for what it does, but what it does is pretty good.

Thanks.

</comment></comment></comment><comment><USER10> I hope to see these changes incorporated within Google's official implementation.

As of right now, deserialization of json and xml are way faster in Python: [http://stackoverflow.com/questions/499593/whats-the-best- ser...](http://stackoverflow.com/questions/499593/whats-the-best- serialization-method-for-objects-in-memcached)

</comment><comment><USER11> This is very welcome, but I hope Google fixes this problem in the official protobuf distribution.

It looks like protobuf 2.4.0 has experimental support for backing Python protocol buffers with C++ via the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable:

<http://protobuf.googlecode.com/svn/trunk/CHANGES.txt>

</comment><comment><USER12> Is this really a better approach than using Cython to wrap a c++ or c implementation?

</comment><comment><USER4> You should add cPickle to the benchmark as well -- I bet fast-pb still comes out ahead, and that may be an eye opener for many Python devs.

</comment><comment><USER13> MessagePack is up to 4x faster* than protobuf, and easier to work with btw IMHO.

<http://msgpack.org/>

I used it as the native format for DripDrop (<https://github.com/andrewvc/dripdrop>)

* In Some tests

<comment><USER2> > * In Some tests

In that test protobuf is forced to _copy_ the 512 byte string (200,000) times, while it appears that MessagePack is _referencing_ it.

Granted it's a bummer that protobuf can't do this easily (my protobuf library upb can -- see above post), but I think it's dishonest not to mention that a large portion of the difference (if not all of it) is just memcpy() that protobuf is doing but MessagePack is not.

It reminds me of when I worked at Amazon and we had a developer conference with several speakers. One speaker was plugging Erlang and showed a graph comparing C++ processes with Erlang processes, and the graph showed C++ being much slower or bigger. Scott Meyers was in the audience and raised his hand to ask "what are the Erlang processes not doing, to explain the difference?" The guy couldn't answer that question directly.

After a bit of digging, you realize that an Erlang "process" is a lightweight, interpreter-level abstraction that is implemented inside a regular OS process. So naturally it doesn't have any of the overhead that is associated with an OS process, and you don't have to make a system call to perform IPC.

So when you're posting benchmark comparisons, I think it's only right to mention any inherent differences in how much work you're doing.

</comment></comment><comment><USER4> Has anyone managed to run the fast-pb tests in benchmark.py? I'm not sure where this switch is coming from:

      protoc --fastpython_out

<comment><AUTHOR> Have you installed both protocol buffers and the fast-python-pb module? Feel free to email me: robbyw@(the-company-mentioned-in-the-title).com

<comment><USER4> Thanks Robby, got the benchmark working, was trying to do a homedir install of fast-python-pb earlier.

I added a couple more tests to the benchmark, here are the results:

      JSON
      3.57209396362

      Protocol Buffer (fast-python-pb)
      0.325706005096

      Protocol Buffer (native)
      4.83730196953

      Protocol Buffer (lwpb)
      0.32919216156

      cPickle
      0.837985038757

As you can see, lwpb and fast-python-pb are neck and neck. And I should point out that lwpb isn't using C++ codegen at all, just the compiled schema in the .proto file. Of course, if completeness of implementation was the critical thing, you'd probably want to stay closer to google's official implementation. There's a lot of the google implementation that I never use though, like the RPC stuff.

Also notable that both lwpb and fast-python-pb outperform cPickle by almost 3x. It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.

Here's a fork with the patched benchmark code: <https://github.com/acg/fast- python-pb>

<comment><USER14> > It would be interesting to know why a portable, cross-language serialization > format beats out the language-specific one.

Because protobuf parses messages with a fixed schema in a very structured format. Pickle, OTOH, is an interpreted bytecode microlanguage used to describe arbitrary python objects (for instance, pickle can call python functions: <http://nadiana.com/python-pickle-insecure>)

Also, pickle supports references (so if an object is referenced two times in the same serialization stream, it is serialized only once) and this has a cost at serialization time (need to keep the set of seen references)

<comment><USER4> Perl's Storable can also serialize code references which get eval'ed during deserialization, and can also serialize multiple references once, though you have to be more explicit about that. And Storable is still 2x-3x faster than the already quite fast JSON::XS.

It would seem the bytecode interpreter architecture in Pickle is the limiting factor. If anybody has some good profiling data on Pickle though, I'd love to see it.</comment></comment></comment></comment></comment>


Stanford MOOC on Government Surveillance - <AUTHOR> https://www.coursera.org/course/surveillance

<comment><USER2> Glad there are people like Jonathan that take the time to do things like dump federal/state caselaw and related materials:

[http://webpolicy.org/2013/12/29/advancing-empirical-legal- sc...](http://webpolicy.org/2013/12/29/advancing-empirical-legal-scholarship- state-materials/)

[http://webpolicy.org/2013/05/03/advancing-empirical-legal- sc...](http://webpolicy.org/2013/05/03/advancing-empirical-legal-scholarship- federal-appellate-opinions-and-rules/)

[http://webpolicy.org/2013/08/09/advancing-empirical-legal- sc...](http://webpolicy.org/2013/08/09/advancing-empirical-legal-scholarship- federal-trial-opinions-and-rules/)

Also, he had an interesting start to Coursera's security:

[http://webpolicy.org/2014/09/04/a-funny-thing-happened-on- th...](http://webpolicy.org/2014/09/04/a-funny-thing-happened-on-the-way-to- coursera/)</comment>