Show HN: I made a simple PDF text editor - <AUTHOR> https://simpdf.com/ <comment><USER2> So it claims to

> Edit pdf like a word doc while preserving structure and format.

While I'm sure there are cases where this works pretty well and can be very useful, it may be worth noting that -- just like every other tool in this space -- there will be many PDFs where it simply can't work. It'll be hugely dependent on exactly how the PDF-generating application/tool went about things.

One simple example: suppose your PDF uses a specific font (not one of the standards like Times or Helvetica), so the PDF-generating tool embedded the font. (This is common.) Further, suppose the generating tool embedded a re- encoded subset of the font, including only the glyphs that were actually required. (This is also common.)

Now, suppose the edit you wish to make involves adding a character that was not present in the original document -- let's say you want to change the date from "May" to "June". But the original document contained no occurrences of capital J (in this particular font/style), and so the capital J glyph is not present in the embedded font. No "PDF text editor" can get around this; the best you can hope for is a "J" in some fallback (such as Times) that may look terrible alongside the intended custom font.

And as for edits that would require reflowing multiple lines of text, maybe inserting a new paragraph in the middle of a page, etc.... not much chance of this working out well.

Yes, a tool like this can (in many cases) make it possible to make _minor_ changes (perhaps fixing a typo or updating a word here and there). To suggest that it can "edit pdf like a word doc" seems patently false to me.

<comment> saint-loup I learned a lot from this article (discovered on HN I think):

What's so hard about PDF text extraction? [https://www.filingdb.com/pdf-text- extraction](https://www.filingdb.com/pdf-text-extraction)

In a nutshell, PDF is fundamentally different than MS Word: it's a standard for visual layout, without notions of paragraphs or even words.

As OP said, it doesn't mean the tool is useless. It could quite often come handy for myself.

<comment><USER3> One of the weirdest things I've seen is a PDF where the text is complete gibberish if you copy paste, but perfect if you export it to HTML or Word in Acrobat. Never figured out how or why that might happen.

<comment><USER4> I read somewhere (likely here) that this oddness comes from the idea that PDF is a way of structuring documents for print first, and presentation in a user interface is secondary.

That the rendering of the document on screen is paramount as opposed to ability to manipulate the text itself. "These characters should be displayed at this position in the document precisely"

It would make sense that exporting the document as HTML or Word would make this easier - as these document formats have different goals.

</comment></comment></comment></comment><comment><USER5> Congratulations for the launch. Looks nice. For those that are cautious uploading sensitive PDFs, you can always just open them with Inkscape, and start editing pretty much anything in a document.

PS: I was once forwarded an application for someone who was supposed to replace me, and I had to interview them. The expected salary was hidden by placing a gray rectangle over it. I removed it using inkscape and saw the expected salary was 30% higher than what I made.

Inkscape Link: [https://inkscape.org/](https://inkscape.org/)

<comment><USER6> Inkscape to Illustrator is what GIMP is to Photoshop.

Glade someone brought it. For most common tasks is there really a point even using Photoshop and Illustrator now days? Especially with the cloud direction they are moving towards.

<comment><USER7> because GIMP sucks, maybe, and also because most people are required to interoperate with the rest of the profession. i am very disappointed that photoshop et al cannot be run under WINE :(

<comment><USER8> I have done/still do complex composition and editing in both GIMP and Photoshop, and there is really no missing features in GIMP. If you go into using GIMP expecting it to be PShop, you will always be disappointed, but in reality that is only a good thing.

</comment></comment></comment></comment><comment><USER9> You should provide a sample pdf file so we don't have to hunt around for a (small) pdf file just to see how it works.

</comment><comment><USER10> Wait wrapping a lib you didn't write and that hasn't been updated in 5 years is now called "I made a text extractor"?

</comment><comment><USER11> I tried using this with an invoice that I'd created using invoice- generator.com, in the hopes that it would be an easier way to make new invoices. When I tried to replace the To party's name, the text came back partly bold and partly not. There was also a weird overlay on an email address on the bottom that said something about email address protected.

Would love to have a tool like this that worked for making new invoices, among other things!

<comment><USER12> In EU it is illegal to edit an invoice outside a program that keep track of all changes, bill number etc.

Not the same in US?

<comment><USER13> at least for Germany: digital invoices shall be processed in a way that manipulation can be ruled out (GoBD). Hence companies use special scanner and document management systems which document the Revisionssicherheit.

<comment><USER14> And that is the new, more relaxed situation. At first, digital invoices were required to be signed with a qualified signature, i.e. spend a few hundred quid a year on certificates by a few select CAs (only the usual german "suspects" could qualify due to intentionally onerous requirements).

</comment></comment></comment></comment><comment><USER15> A few notes. It doesn't seem to work with documents that have multiple columns. Pushing text over just overwrites the other column. It doesn't seem to reflow text where the source document obviously had margins possibly because that information is looooong gone. Hitting enter to move text to the next line didn't move other text it just again seemed to overwrite it.

</comment><comment><AUTHOR> Hi, I bootstrapped this simple website. Let me know what you think :)

<comment><USER16> I tried it with a bit complex pdf structure and it worked like charm +1. Would love to learn more about the underlying techniques/tech.

<comment><USER17> [https://github.com/shashanoid/Simpdf/blob/1557bf838a8debeee1...](https://github.com/shashanoid/Simpdf/blob/1557bf838a8debeee1a6f21c12ee3e91c420b855/backend/run.py#L31)

Btw, arbitrary code execution vuln here, OP.

<comment><USER18> Yeah. Switch array & args disable the shell. I hope they're not running that locally as the download script suggests. But then you still have a bunch of other security issues. Shrug.

</comment></comment></comment></comment><comment><USER19> Unfortunate name: [https://www.urbandictionary.com/define.php?term=Simp](https://www.urbandictionary.com/define.php?term=Simp)

<comment><USER20> Fortunately, I don't think many people will be offended by the name...

...or not as much as The Gimp, at least.

</comment></comment><comment><USER21> I tried it with a relatively complex pdf and it blew my mind.

</comment><comment><USER22> I really wish the PDF layout was easier to parse. No matter which library you use, you always run into edge cases which make text selection and extraction an issue on certain files. I was recently extracting financial data from a bank which provides only PDFs and every time they changed the format just a little bit I had to change large parts of my code to extract the transactions I wanted.

<comment><USER2> PDF is designed to present a human-readable document, not to serve as a data interchange format.

</comment></comment><comment><USER23> @op line 105 on [https://github.com/shashanoid/shashwatsingh.github.io/blob/m...](https://github.com/shashanoid/shashwatsingh.github.io/blob/master/index.html) is missing an 'l' and will cause you to not get your email from your personal web link

</comment><comment><USER24> Now I just wish there were a version that could run locally. All the Linux PDF viewers suck -- they can't even save a fill-in PDF form or insert a signature.

<comment><USER25> I spent an afternoon trying a bunch of them some years ago. I settled on the freeware Master PDF Editor version 4 (version 5 inserts a watermark unless you buy a license). [https://code-industry.net/masterpdfeditor/](https://code- industry.net/masterpdfeditor/)

It is super lightweight and opens any complex PDF. Can insert signatures, can edit anything in the PDF (without changing the font even if the font is embedded in the PDF—as long as all the glyphs you are typing are present in the font), etc. My only complaint is that it won't edit an encrypted PDF, but a one-liner Ghostscript command can remove the encryption automatically: [https://gist.github.com/compleatang/6046249](https://gist.github.com/compleatang/6046249)

</comment></comment><comment><USER26> hackernews traffic crashed the host :( Cant wait to try tho!

</comment><comment><USER27> I love it. How do I save it?

<comment><USER28> ^ this

<comment><USER24> there's a "Save and Download" option if you mouseover to the left.

<comment><USER29> I found this too, didn't seem too obvious at first though.

</comment></comment></comment></comment><comment><USER30> I'll be honest, I thought the website was called simp df.. lol

</comment><comment><USER31> The website isn't loading anything for me right now.

<comment><USER32> I am getting a Cloudflare 522.

</comment></comment><comment><USER33> A little glitchy with complex PDFs but WOW, amazing work!

</comment><comment><USER34> Neat, would be useful for students' homework.

</comment><comment><USER35> Works, it changes the font type after editing</comment>


Ask HN: What to do with an idle server? - <AUTHOR>

I recently upgraded hosting and changed providers in the process. Unfortunately, I prepaid 5 years for a massive discount and have 2 years remaining on the older server. Due to the terms of service, a refund isn't an option - the cost is sunk. I'd considered leaving it up as a mail/svn server, but the benefit would be negligible. Are there any interesting projects (e.g. wave, diaspora) out there that would be worth setting up?<p>Edit- fixed grammar. <comment><USER2> Run ntp and join www.pool.ntp.org. NTP is one of those critical infrastructure components that everyone always forgets about. A stable longterm stratum II server is always a welcomed addition. Pool.ntp.org has really made it easier to keep your machines clock accurate. Debian and Ubuntu's ntp packages point to pool servers by default now. I think a lot of other distros do the same.

Run a tor middleman node. You won't have any of the legal worries you have about exit nodes. Im not a lawyer but its really unclear what the legal ramifications are for running an exit node anyway:)

</comment><comment><USER3> Ideas:

1\. Bitcoin mining. 2\. Sub-sell disk space. 3\. Host a mirror of your site (if you use your old site for downloads of any kind)

<comment><USER4> Mining is questionable. The cost is sunk, sure, but as a server (i.e. no gfx card) it will probably make less than $2 per week.

</comment></comment><comment><USER5> Lots of open source projects would be happy with an additional mirror. See <http://www.openbsd.org/ftp.html#layout> for one project's policy. I'm sure that many other projects have similar pages.

</comment><comment><USER6> 1\. Let it do some protein folding - do the world some good. folding@home I think the stanford initiative is called.

2\. Set up a 'private dropbox'

</comment><comment><USER7> You could run a Tor exit node: [https://blog.torproject.org/blog/tips-running- exit-node-mini...](https://blog.torproject.org/blog/tips-running-exit-node- minimal-harassment)

<comment><AUTHOR> Thanks for the idea. Tor sounds a bit challenging (legally) -but it would be isolated. Will give it some thought..

</comment></comment><comment><USER8>
      Install nginx and put some porn right on there
      Install nginx and put some porn right on there
      Install nginx and put some porn right on there
      Earl-aye in the morning

      Wipe its disk run a big TOR server
      Wipe its disk run a big TOR server
      Wipe its disk run a big TOR server
      Earl-aye in the morning

      That's what I'd do with an idle server
      That's what I'd do with an idle server
      That's what I'd do with an idle server
      Earl-aye in the morning

<comment><USER9> For anyone that has no idea what's going on here, this is a reference to a sea shanty called "Drunken Sailor" (<http://en.wikipedia.org/wiki/Drunken_Sailor>)

<comment><USER8> <http://www.youtube.com/watch?v=qGyPuey-1Jw></comment></comment></comment>


C is a Wasteland - <AUTHOR> http://www.rs.io/c-is-a-wasteland/ <comment><USER2> No. Learn your history and your computer science.

* At the time of C's creation, there was (among others): Fortrans, Cobol, BCPL, and a set of Algols. A cursory check of a programming languages book or Wikipedia will verify the error.

* There are a ton of C libraries. You know, if you want to go and look for them instead of expecting them to be handed to you.

* Concurrency at the C layer is provided by OS/HW primitives. pthreads ain't bad, for all the whinging.

* Manual memory management is very needed in systems programming (embedded programming).

* A need for array bounds checking reflects slower code and lazier coder. Look at Pascal strings for the grief that can cause.

* Better support for splitting- what? I can not perceive the confusion of mind that could generate this statement

* "It's got curly braces, wah". Of all the complaints I have heard, this asinine complaint about (curly braces/parens) is perhaps the most empty.

\---

If you want a powerful systems language that isn't C or C++, try Ada. I haven't found a need for it personally, but has one zillion features.

Other languages that might play in this space are Rust and D. Possibly some of the functional languages that compile out to binaries. (OCaml perhaps?)

C is designed for building operating systems with, on fairly marginal resources. It is so excellent at working _at that layer_ that no one has come up with a truly viable surviving competitor in thirty-five years. And, like any other tool, it doesn't really fit in places it wasn't designed to fit.

</comment><comment><USER3> Articles like this make me grind my teeth. I know I should have more self restraint, and I can forgive/concede a few points made in this article. Generally though it's just a major rehash of old news. I was not surprised to find that the author is a relatively inexperienced college student.

Glib eliminates a good chunk of the complaints outlined here. Some of it is simply coder responsibility. (Array bounds checking, memory management - should be problems handled with good code review and maybe the help of some 3rd party tools.)

But the last line of the article was what just pushed me over the edge. "Ruby/Python inspired syntax."

So really, the author would just prefer to code in Ruby or Python.

<comment><USER4> Suggesting that a cleaner/simpler syntax should be used doesn't mean he wants to code in Ruby/Python. They have plenty of their own flaws. He just admires their syntax since it is very readable.

</comment></comment><comment><USER5> >When C was created, it's main competitor was assembly language

C was created at a time of great effervescence in programming language design. FORTRAN and COBOL were well-entrenched, and it also had to compete against the likes of Algol, Lisp and Pascal. (Also, the correct spelling, in this context, is "its".)

>C, however, has it's flaws, foremost among them: [...] a woefully inadequate standard library

Sure, you can't do that much with the standard library alone, but C probably has more libraries overall than any other language in existence. Some of them have rather clunky APIs, but on the other hand they're usually very fast and very well tested.

</comment><comment><USER6> The manual memory management and lack of standard libraries make C programming language the only useful language when it comes to safety critical domain. (Exception is Ada).

Java or Python are super cool, there is no doubt about it. But when you have to program a metro rail or Eurofighter jet controls, one has to give hard deadline to the tasks. Unfortunately no high level language can be used in such cases. Garbage collection adds undeterminism to the temporal properties of the executing tasks and hence can not be user in real time safety critical domains.

<comment><USER7> There are new techniques wrt to garbage collected languages and hard deadlines (eg, every period has a dedicated amount of time to garbage collection, whether it needs it or not).

<comment><USER8> So it's possible my airplane might fall out of the sky because there is not enough memory because not enough time was devoted to garbage collection? That doesn't sound like a great solution. Not to say that C is necessarily the ideal solution. The main reason the author hasn't seen competitors to C is because the author didn't look for one. Vala is decent and there are many others, they're just not sufficiently better than C to result in people switching.

</comment></comment></comment><comment><USER9> _The most puzzling aspect of C's dominance, to me, though, is that C has had no competition in its problem domain. Nobody has really said "I'm going to create a better C."_

Pretty sure Bjarne Stroustrup said just that.

</comment><comment><USER10> Sounds like it was written by a young kid who learned to program with Java or Ruby.(Checks about page) Yup, 19 year old college student who probably thinks that computer science started with HTML and JavaScript.

</comment><comment><USER11> Not well informed. Go is a better C, albeit with garbage collection.

C++ was designed to be a better C.

D was designed to be a better C.

</comment><comment><USER12> I'm not super familiar with Go, but the author's ideal C replacement sounds pretty much like my understanding of Go.

<comment><USER13> It's close. The only thing the author lists that it doesn't provide is manual memory management. It's technically possible, but the language/libraries don't provide it. Type safety and lack of generics make it difficult to provide a reasonable interface for it.

</comment></comment><comment><USER14> Could Go switch between manual memory management and garbage collection after some modification? Or are some choices in the language preventing this?

<comment><USER13> It wouldn't be difficult to add new built-in functions for allocating and freeing memory manually. If you wanted to remove garbage collection, it would require a more significant language change. There are several constructs in the language that implicitly allocate memory.

If Go adds support for generics, manual memory management could be provided by a third party package.</comment></comment>


Why Russia and China's joint military exercises should worry the West - <AUTHOR> https://www.economist.com/leaders/2018/09/06/why-russia-and-chinas-joint-military-exercises-should-worry-the-west <comment></comment>


When selling software internationally, how do you collect taxes for each country? - <AUTHOR>

This is in reference to selling software via the internet.<p>Which countries require you to collect taxes?  What is the process involved? <comment><USER2> IANAL, but your obligations for corporate tax payment depend on where you incorporate as a business. If you are planning to do a significant amount of international business you may want to register your company in a locale like Hong Kong which only requires taxes paid on local sales.

The downside to registering a company internationally is that it is a massive pain in the ass to set up payment processor systems, create corporate accounts and handle paperwork. Especially if you have to deal with US banks and payment processors. Many will just refuse to support non-American businesses. Also bear in mind if you own a majority share in your business you may need to pay personal taxes on any company profits even if the company is registered abroad.

</comment><comment><USER3> For the small utilities that I sell, I go through a payment-processor: Regsoft. Regsoft adds the proper VAT to the shopping-cart total when the prospective buyer is in the process of paying for a license or licenses for the given program.

<comment><USER4> How much does Regsoft charge you?</comment></comment>


40% better single-threaded performance in MariaDB with PGO - <AUTHOR> http://kristiannielsen.livejournal.com/17676.html <comment><USER2> And a followup getting more specific [http://kristiannielsen.livejournal.com/18168.html](http://kristiannielsen.livejournal.com/18168.html)

Edit: This is all from January, has there been any followup?</comment>


Ask HN: Should I get an MBA? - <AUTHOR> I'm graduating this year with a B. Tech in Computer Science. I have obtained a job at a leading tech company. Will getting an MBA benefit me if I wish to stay in the Tech industry?(I'm in india) <comment><USER2> Getting an MBA to advance in the business world is like going to culinary school. You don't really need to do it, it'll cost you an arm and a leg, and some experienced cooks will look down on you for doing it instead of learning the ropes in an actual kitchen. However, you will benefit from a structured curriculum, a few letters after your name, and hopefully a bit of a network.

There is not a right or wrong decision. However, the credentials are probably more useful if you want to join a big catering company and work your way up the ladder than if you just want to open your own restaurant (where nobody cares what letters you have after your name, just how good your food is, and you'll have to figure everything out yourself anyway).

</comment><comment><USER3> Which job market are you targeting? Indian Tech/IT Job market is vastly different from Western World. Despite the chest thumping by startups and cool kids in khakis telling you that talent matters, it doesn't. Indian job market is about numbers and back-office work. A tech degree is not a pre-requisite, but an MBA, even from a third-tier college will help you manage people. Indian IT management is different than other management mostly because, again, it is a numbers game. The managers make up metrics like hours on desk or hours clocked.

A tech-degree, even from a first-tier college, is not worth in the Indian market, an MBA on the other hand will provide you with numerous opportunities and fantastic compensation.

</comment><comment><USER4> No. It is not worth the money or time to do more formal schooling than you've already done. You already have the paper that proves you've done the whole college dance.

Go read "The Personal MBA" by Josh Kaufman, and read the books he recommends therein. That will get you the same education, and you won't have a tuition bill when you're done.

Also, you'll have work experience that you'd miss out on while taking MBA classes. Focus on strengthening your tech skills, and stay working. That'll be far more beneficial to you than another degree.

Seriously, it's not worth it like it used to be.

</comment><comment><USER5> For the price of an MBA you can start a small business, and actually earn money.

Some people gets an MBA for the contacts: Don't do it. Not all the people you'll know will end in "high places". It is much better to call a high-level executive with a sound business proposal.

</comment><comment><USER6> The piece of paper may or may not be worth much. But, especially coming from an Indian tech environment, an MBA provides important skills to analyse and understand the entire business environment -- marketing, finance, organisational behaviour, etc. This is important both as a developer to understand why decisions that might not make sense from a technical perspective are made and will also help you in a startup or other entrepreneurial environment where you need a wide approach.

Unfortunately MBAs are becoming too common and qualities vary widely so only some MBAs from top tier schools offer the salary boost that they used to provide.

</comment><comment><USER7> You don't do an MBA for the education, you do it for the network and the branding. If you are able to get into a global Top50 program it is worth it. Or if you are relocating and trying to built a network in the city of the University. Otherwise it is a waste of money.

<comment><USER8> I would say more like Top 10.

</comment></comment><comment><USER9> Do you think getting an MBA would be a fun, enriching experience? This is the more important question, imho. Life is not all about your career.

</comment><comment><USER10> Only get an MBA if you lack creativity and enjoy busy work.

</comment><comment><USER11> I think the most beneficial case of MBA is firstly you are accustomed with macOS, secondly you have enough computing resource else where and thirdly you have stable network connection in case you are asking about MacBook Air

<comment><USER12> Not sure of this is a sarcastic response, but OP asked about the Master of Business Administration degree, not the Macbook Air.</comment></comment>


World's Fastest Fiber Line Can Support an Internet's Worth of Data - <AUTHOR> http://gizmodo.com/worlds-fastest-fiber-line-holds-an-internets-worth-of-d-1651327178

<comment><AUTHOR> Not sure how long this would take to be commercially available. But when and if it does it would change the way we use internet.</comment>


Promise Free Async/Await - <AUTHOR> https://www.npmjs.com/package/casync <comment><USER2> How's this different from co?

<comment><AUTHOR> I added a 'Why' section to the [readme]([https://github.com/bessiambre/casync](https://github.com/bessiambre/casync)). Maybe this helps understand the reasoning better?</comment></comment>


FCC admits it was never actually hacked - <AUTHOR> https://techcrunch.com/2018/08/06/fcc-admits-it-was-never-actually-hacked/ <comment><USER2> "I am deeply disappointed that the FCC's former [CIO], who was hired by the prior Administration and is no longer with the Commission, provided inaccurate information about this incident to me, my office, Congress, and the American people. "

The first envelope is open. Next up: reorganisation.

<comment><USER3> No admission of responsibility of course, just scapegoating. I find it highly likely that Pai and his like-minded cronies were in on this the whole time.

<comment><USER4> Why would he have posted such trollish pictures of himself if he weren't?

He got real serious suddenly when that nutter threatened his family.

<comment><USER5> A real-life consequence for abhorrent behavior? I'm all for that. Maybe not quite that extreme, but I think public shaming of these people and their friends/family is acceptable. They don't seem to respond to anything else these days.

<comment><USER6> A lesson the political and billionaire class have not learned is that when inequality gets as bad as what we have today the mob gets actual pitchforks and assassins start throwing bombs in your carriages and shooting at your motorcade.

Repression follows, more assassinations happen, and then before you know it Serbian meddling in the Austro-Hungarian succession starts WWI.

This is not limited to the countries ending in -ia in Central/Eastern Europe [0] [1].

[0] [https://en.wikipedia.org/wiki/1919_United_States_anarchist_b...](https://en.wikipedia.org/wiki/1919_United_States_anarchist_bombings)

[1] [https://en.wikipedia.org/wiki/Wall_Street_bombing](https://en.wikipedia.org/wiki/Wall_Street_bombing)

<comment><USER7> It's not inequality. It's almost never pure inequality. It's poverty going below a specific threshold, usually hunger or homelessness.

And when was there a movement like this in the US post-prosperity (WW2)? Your links are from 100 years ago.

<comment><USER8> Things got pretty bad during the Vietnam war. There is a reason why the draft was abolished.

<comment><USER7> Wasn't that because of the draft, though? That counts as "major" since you're being shipped off to be killed. But how likely is a new Vietnam? US involvement in wars doesn't require a draft anymore...

My point, inequality itself is rarely the trigger. It needs to pass a limit, usually threatening livelihood. And it needs to be super widespread (think 20%+ of the population).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> "Because it wasn't a hack, it seems that the comment-filing system, though recently revamped, needs yet another fresh coat of paint to handle the kind of volume it saw during the net neutrality repeal."

Or as an alternative, maybe don't shove through legislation that's so wildly unpopular with the american citizens that the entire internet comes crashing down on you in protest. Just a thought.

<comment><USER10> Nit: it's not legislation, it's regulation in the FCCs purview. The reason this matters is because it is likely going to take Congressional legislation to get net neutrality now, which means electing (mostly) Democrats.

<comment><USER11> If only it were as simple as voting for 1 of 2 parties.

[https://www.opensecrets.org/industries/totals.php?cycle=2018...](https://www.opensecrets.org/industries/totals.php?cycle=2018&ind=B09)

Neither party appears to be turning down money from the opponents of net neutrality. It feels more like everybody takes the money and the rest is just politics and propaganda.

<comment><USER12> Yep, you have the corporatist Democrats and the corporatist Republicans, both of which are happy to collude if it makes the companies that own them happy. I wish we would just make our politicians wear their owner's logos.

<comment><USER13> Sounds like the outfits worn in Idiocracy.

</comment></comment></comment></comment></comment><comment><USER14> I'm confused, where did they say they were "hacked"? When I follow the links I get here [1] where they said they were DDoS'd, not hacked. Even TechCrunch's old article [2] doesn't say the FCC was hacked.

[1] [https://www.fcc.gov/document/statement-fcc-cio-denial- servic...](https://www.fcc.gov/document/statement-fcc-cio-denial-service- attack-fcc-comment-system)

[2] [https://techcrunch.com/2017/05/08/the-fccs-comment-system- ta...](https://techcrunch.com/2017/05/08/the-fccs-comment-system-targeted-by- ddos-attacks-during-filing-period-for-net-neutrality/)

<comment><USER15> Here you go: [https://apps.fcc.gov/edocs_public/attachmatch/DOC-344764A1.p...](https://apps.fcc.gov/edocs_public/attachmatch/DOC-344764A1.pdf)

It's linked from that page you posted.

I guess you can be pedantic about the term "hacked," but they claimed their site was deliberately rendered unavailable to commenters by multiple attackers. In the common parlance, deliberately breaking computer shit is called hacking.

It may save unneeded stress to make peace with the fact that words can mean one thing to a specialist and something else to all the other native speakers of the language.

<comment><USER14> > Here you go. It's linked from that page you posted.

Uhm, yes, I have eyes, thank you. I think you can assume HNers are generally not obtuse enough to link to a nearly-blank page saying "Statement by FCC" and miss 3 glaringly obvious links referring to the actual statement. I was, indeed, referring to that statement. The very fact that their news release seemed to be missing "hack" or anything equivalent to it is exactly what I was referring to.

The difference between "attacked" vs. "hacked" isn't "pedantry" any more than the difference between "arrested" and "convicted" is pedantry. Being "hacked" means your system's security was breached, not that you were merely "attacked". Most of the press cares about being responsible with their choice of words, and you really should too. Nobody would say Google or HN or any other site was "hacked" the moment it was DDoS'd. Being hacked requires a security breach, and nowhere that I can see did the FCC claim there was such a thing. So unless they said this elsewhere, this is a recklessly irresponsible choice of words.

<comment><USER15> > Being "hacked" means your system's security was breached,

Nah. They claimed the site was rendered unfit for purpose by malicious attackers. If you don't think denial of service is a security event, you'd better start writing lots of angry letters to MITRE, because we need to revoke about 90% of the CVEs that have been issued over the years.

I'm not sure how to phrase this politely, but you are not the arbiter of language, TechCrunch is not beholden to any higher a linguistic standard than anyone else, and the word hack has been associated with DDoS attacks for years and years (a small sampling of references follow).

I don't particularly care about the FCC or this event, but this seems like a really strange hill to die on. Please note that I am not suggesting that you actually intend to expire on a topological elevation gradient; this is merely a figure of speech that means something else to some people (cf [https://en.wiktionary.org/wiki/hill_to_die_on](https://en.wiktionary.org/wiki/hill_to_die_on)).

"This class of hack, known as a distributed denial of service (DDoS) attack, has been around for a while.": [http://fortune.com/2016/10/22/ddos-attack- hacker-profit/](http://fortune.com/2016/10/22/ddos-attack-hacker-profit/)

"What is a DDoS Hack and How Do You Avoid Them?" (video): [https://www.wired.com/2016/01/hacker-lexicon-what-are-dos- an...](https://www.wired.com/2016/01/hacker-lexicon-what-are-dos-and-ddos- attacks/)

"DDoS hack attack targets Church of Scientology": [https://www.scmagazine.com/ddos-hack-attack-targets- church-o...](https://www.scmagazine.com/ddos-hack-attack-targets-church-of- scientology/article/554145/)

"Hack Attack Gums Up Authorize.Net": [https://www.wired.com/2004/09/hack- attack-gums-up-authorize-...](https://www.wired.com/2004/09/hack-attack-gums- up-authorize-net/)

<comment><USER16> You highlighted an issue in all of mainstream media's use of the term "hacking". They are extremely liberal with their use.

It's very tongue-in-cheek to claim an organization has been hacked because someone(s) launched a DDoS.

It's like someone claiming they perform their own auto repairs, but in actuality all they do is fill the windshield fluid and replace their air filter.

</comment></comment></comment></comment></comment><comment><USER17> Are there going to be any consequences for this? If no, then it is such a betrayal of trust.

<comment><USER18> If you read about Steve Bannon, his plan for years is really to have people lose faith in government agencies and have them thought of as garbage, and then ultimately shut down. I'm not sure this is some great conspiracy or just a bunch of scumbags at work here, but like I say there are major well financed efforts to actually make the FCC, EPA, etc look stupid, and appear to be run by idiots. Appointing incompetent people to them, is the plan by some of these folks like Bannon.

<comment><USER19> Invoking Steve Bannon is unnecessary: "losing faith in government" has been the Republican Party goal since Nixon.

<comment><USER20> "Government is not the solution to our problem; government is the problem."

President Ronald Reagan, in his first inaugural address.

<comment><USER21> Arguably a misquotation, probably misleading absent the context, and definitely an incomplete quotation.

Speaking narrowly and specifically about the economic crisis and stagflation of the late 70s:

 _In this present crisis, government is not the solution to our problem; government is the problem. From time to time we 've been tempted to believe that society has become too complex to be managed by self-rule, that government by an elite group is superior to government for, by, and of the people._

<comment><USER22> The surrounding context doesn't really change/clarify that core message very much.

<comment><USER21> It does, it is saying the government had been the cause of the economic crisis of the time, not that it was generally the problem of all things.

</comment></comment></comment></comment></comment></comment></comment><comment><USER23> So does this have anything to do with the fake comments (left in my name): [http://fortune.com/2017/11/29/fcc-and-net-neutrality- check-t...](http://fortune.com/2017/11/29/fcc-and-net-neutrality-check-to-see- if-your-name-was-used-for-fake-comments/)

<comment><USER24> Yeah, I always assumed they were unintentionally DDoS'd by the bots spamming fake comments. Which anyone non-technical could honestly but inaccurately describe as "being hacked".

<comment><USER25> Anyone non-technical who was not directly involved with the deed, perhaps. It is not at all honest to say "we were hacked" if the truth is actually "we hacked ourselves."

Honestly (and of course, I don't have anything concrete to back this up with) I'm quite sure the truth is "we hacked ourselves, maintaining just enough degrees of separation so as to preserve plausible deniability..."

<comment><USER24> Huh? There were plenty of interests who would have been willing to pay someone to generate a lot of fake anti-net-neutrality "opinions".

Why assume that the FCC stuffed their own mailbox?

<comment><USER25> Why else would they have made up this ridiculous story about being hacked? You're much too charitable.

<comment><USER24> They only claimed they were DDoS'd, it's the media that used the word "hacked". Maybe we can agree that they might not have been able to tell the difference between bot comments and DDoS, or did not want to acknowledge that difference.

<comment><USER25> Sure. Somebody made up their mind what they wanted to do, they were required by law to seek public comments.

That 98% of unique comments and an overwhelming majority of public discussion were on one side of the issue, and they were then overwhelmed by obviously fake comments on the other side of the issue very conveniently allowed Pai to say "the results of the comment period are invalid, so we're going to disregard the comments and do what we had already planned to do anyway"

Given the track record of this administration so far, I feel pretty comfortable extrapolating from limited data and drawing conclusions that would not hold up in a court of law. Unless these folks are suddenly becoming much smarter than they look, chances are this will either be proven soon, or more likely swept under the next larger scandal.

Hopefully we can also agree that they were not going to listen to the public and knew before the comment period had even opened what they would do, and wouldn't be stopped. But in the realm of things we can prove, they definitely did not want to acknowledge the difference between what happened and "what happened." It played strongly to their advantage and successfully muddied the waters in furtherance of their stated goals.

</comment></comment></comment></comment></comment></comment></comment><comment><USER26> Was there ever any doubt? What a joke ( and waste of time and resources ). At this point I'm surprised Pai bothered to release a statement.

<comment><USER9> Just face-saving. Trying to get out in front of the report and frame the issue rather than let others frame it.

</comment></comment><comment><USER27> I have been around a long time, and have observed many different presidential administrations. Some I liked and some I disliked. But I have never seen one that lied one tenth as often as the present one.

</comment><comment><USER28> Of course it wasn't but the damage has been done.

</comment><comment><USER29> Why don't people trust institutions anymore?

<comment><USER30> Because the institutions were corrupted by people who don't want instutions telling them not to be corrupt.

</comment></comment><comment><USER31> The FCC should be disbanded and replaced with a new regulatory body that is elected by the people (either directly or by elected representation).

Even though they could do good, and may have in the past, the lack of actual accountability for their actions means that it's too easy to do bad. They're not directly elected, so even the tiny shred of accountability that elected officials generally have is not there. If they can flat out lie to the public and get away with it, why do they exist and why should they continue to exist?

The justification for the existence of a government organization should _not_ be based on the possibilities of good they _might_ do, but it should be based on the unchecked bad they could (and will eventually) do.

<comment><USER32> Don't fall into their trap of "look how terrible government is (when officials are actively trying to run it into the ground)".

Fix the issues with officials lying, rather than throwing the baby out with the bathwater.

<comment><USER33> FCC does nothing that citizens need or want done. For it to be shuttered would harm only those who have invested billions in the fake "assets" that FCC invented and must constantly police.

It isn't the 1930s anymore. Modern radio technology could enable a much more complete utilization of spectrum. FCC's primary mission at this time is to delay that happy day as long as possible. "Happy" because with true competition, the consumer would forget that net neutrality was ever anything a government agency needed to enforce. Don't like how your WISP is handling your traffic? Just switch to one of the other 25 WISPs in your area.

It's true that if FCC were eliminated to the general benefit of the nation, curious citizens would wonder what else could be eliminated. Should we feel sorry for any organization unable to justify its own purpose and actions?

<comment><USER34> > FCC does nothing that citizens need or want done.

I wanted net neutrality the first time it was implemented. I remember being excited about a few other regulations that the Wheeler FCC implemented.

<comment><USER33> They "implemented" it a year before that president left office, knowing the next administration would roll it back. It didn't even actually get out of the courts, so the implementation never affected anything. A fairly cheap signal of virtue...

<comment><USER32> That's a mischaracterization of what happened.

Traditional telephone telecoms were always common carriers under Title II, and that applied to internet over dialup during the 90s. When broadband meant that internet connectivity was now being delivered via other medium, the FCC enforced what we now call net neutrality via a lighter regulatory scheme. Verizon ironically sued to get the courts to say that the FCC couldn't enforce the rules in a lighter scheme and therefore had to declare the new internet connectivity infrastructure to be full Title II common carriers to apply any of those rules to them. That court case ended in 2014, and so the FCC followed the court recommendation to apply the rules the were already enforcing.

Long story short: there was never really a time with both consumer internet connectivity and no net neutrality, despite the right wing talking points that this is some new thing.

<comment><USER33> Thread parent specifically credited Wheeler. I assumed that was for what Wheeler did in 2015? In my opinion (of course everyone is welcome to her own!) that action was calculated for reversal. In fact the whole "our backs are to the wall since VZN sued us" kabuki performance was planned years in advance. Big telcos are plugged into the parts of USA government that don't face reelection. They can bring extraordinary pressure to bear on judges and bureaucrats. Nobody in that system wants thousands of wildcat WISPs dotting the countryside.

I agree that the AM talk goofballs are full of shit on most things they say about the internet.</comment></comment></comment></comment></comment></comment></comment>


Scaling Linux Services: Before accepting connections - <AUTHOR> https://theojulienne.io/2020/07/03/scaling-linux-services-before-accepting-connections.html <comment><USER2> Nice, although if you want to explore networking with ad hoc tracing tools, please try bpftrace[0]. Only use BCC once you need argparse and other python libraries.

Here's my bpftrace SYN backlog tool from BPF Performance Tools (2019 book, tools are online[1]):

      # tcpsynbl.bt
      Attaching 4 probes...
      Tracing SYN backlog size. Ctrl-C to end.
      ^C
      @backlog[backlog limit]: histogram of backlog size

      @backlog[128]:
      [0]                    2 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|

      @backlog[500]:
      [0]                 2783 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
      [1]                    9 |                                                    |
      [2, 4)                 4 |                                                    |
      [4, 8)                 1 |                                                    |

The source:

      #!/usr/local/bin/bpftrace

      #include <net/sock.h>

      BEGIN
      {
              printf("Tracing SYN backlog size. Ctrl-C to end.\n");
      }

      kprobe:tcp_v4_syn_recv_sock,
      kprobe:tcp_v6_syn_recv_sock
      {
              $sock = (struct sock *)arg0;
              @backlog[$sock->sk_max_ack_backlog & 0xffffffff] =
                  hist($sock->sk_ack_backlog);
              if ($sock->sk_ack_backlog > $sock->sk_max_ack_backlog) {
                      time("%H:%M:%S dropping a SYN.\n");
              }
      }

      END
      {
              printf("\n@backlog[backlog limit]: histogram of backlog size\n");
      }

This bpftrace tool is only 24 lines. The BCC tools in this post are >200 lines (and complex: needing to worry about bpf_probe_read() etc). The bpftrace version can also be easily modified to include extra details. I'm summarizing backlog length as a histogram since our prod hosts can accept thousands of connections per second.

[0] [https://github.com/iovisor/bpftrace](https://github.com/iovisor/bpftrace) [1] [https://github.com/brendangregg/bpf-perf-tools- book](https://github.com/brendangregg/bpf-perf-tools-book)

<comment><AUTHOR> Thanks for the suggestion! I did come across the `tcpsynbl.bt` script as I was writing up this post, but wanted to add the additional information around namespaces and report additional information, which didn't seem as trivial in `bpftrace` as it was in Python, but that might be my lack of familiarity with the DSL :)

<comment><USER2> If it's a common use case it's trivial, and if it's not yet trivial we'll make it trivial. :) Niche functionality that doesn't fit well can be deferred to BCC.

</comment></comment></comment><comment><USER3> This is well written. I never gave much thought to the resource usage during the period between SYN and accept. This article explained it very nicely. Also, now I'm curious why don't these Linux limits don't scale with the amount of RAM available? Like, yes on a low resource machine you wouldn't want more than the default 128 for the backlog. But if I have 512GB of RAM then why not give me a backlog of a few thousand?

<comment><USER4> In general, Linux does favor automatic defaults over fixed static settings, if there's a reasonable heuristic to produce those defaults. But suppose, for instance, that you can't actually handle that many connections? There are two possibilities here: one is that you are processing connections fast enough to keep up, and the other is that you're not keeping up at all. In the former case, scaling the backlog up may help you keep up, though you may already have unacceptable latency. In the latter case, no amount of backlog will help you, and the backlog may make an attacker's job easier.

That said, there might well be a case for automatic backlog scaling. Or, for that matter, for increasing the default.

</comment></comment><comment><USER5> Is there a particular reason the Linux kernel favors names like `somaxconn` instead of `socket_max_connections`? It seems like a rather straightforward improvement for readability; so, why are shorter, compressed names preferred?

<comment><USER6> This particular name originates from BSD 4.2 [1], which was released in 1983. (For some context, GCC 1.0 is from 1987, pcc was used to build BSD 4.2. The first Linux release was 1991).

1: [https://github.com/dspinellis/unix-history- repo/blob/0f4556f...](https://github.com/dspinellis/unix-history- repo/blob/0f4556f12c8f75078501c9d1338ae7648a97f975/usr/src/sys/h/socket.h#L91)

</comment></comment><comment><USER7> What are some very rough estimates on when it makes sense to look at these low-level network settings when scaling an application? I assume the default settings are good enough for moderate loads, but at which point does this stuff become a bottleneck?

Are the default setting here reasonable for most cases, or is it more like something that you should tune even if you're not really pushing any limits?

<comment><USER8> My NGINX webserver configuration on AWS behind an ALB is:

/etc/sysctl.conf:

        net.core.wmem_max = 12582912
        net.core.rmem_max = 12582912
        net.ipv4.tcp_rmem = 10240 87380 12582912
        net.ipv4.tcp_wmem = 10240 87380 12582912
        fs.file-max = 1000000
        net.ipv4.ip_local_port_range = 1024 65535
        net.ipv4.tcp_tw_recycle = 1
        net.ipv4.tcp_tw_reuse = 1
        net.ipv4.tcp_max_syn_backlog = 262144
        net.ipv4.tcp_syncookies = 0
        net.ipv4.tcp_fin_timeout = 3
        net.ipv4.tcp_syn_retries = 2
        net.ipv4.tcp_synack_retries = 2
        net.ipv4.tcp_no_metrics_save = 1
        net.ipv4.tcp_max_orphans = 262144
        net.core.somaxconn = 1000000

nginx.conf (just the relevant directives):

        worker_rlimit_nofile 102400;

        events {
          worker_connections 102400;
          multi_accept on;
        }

        http {
            server {
              listen 80 default_server reuseport backlog=102400;
              ...
            }
        }

As you can see, the socket and backlog-related values have been cranked way up. I've never had any problems with this configuration. Because these servers are behind and ALB I don't know how relevant they are since the SYN and SYN- ACK relation to RTT is between the server and the load balancer, not the remote clients. But I could be wrong. Maybe there's something I'm missing. But I've never had a problem, and I've never had any performance problems related to TCP connections in the kernel or NGINX.

<comment><USER9> I think for ALB you'll see pooled connections (http or http2) so I would expect the number of TCP connections to stay pretty low. In http2 it could theoretically be as low as one.

</comment></comment></comment><comment><USER10> Why do these values default so low? Seems like you could have a few hundred or 1k instead of 128 with relatively little memory overhead.

<comment><USER11> I've wondered about that me too. What if it's because in the 1990s there wasn't that much memory

</comment></comment><comment><USER12> This was a great read and introduction with great practical examples, great graphics and bpf tracing. I loved it.

</comment><comment><USER13> I wish saw this when i was preparing one of the apps to huge traffic spikes. Did almost all of the changes described. But took some time to search and understand each of them. End result was pretty good.

</comment><comment><USER14> Yeah, just more reasons for microkernels and keeping those things in userspace, where applications can come with values that suit their usage.

</comment><comment><USER15> I wonder how io_uring changes this equation. The ability to have multiple concurrent accepts queued up could push the limits around a bit.

</comment><comment><USER16> A good advertisement for userspace networking.

<comment><USER17> Why? Because this stuff is in the kernel and thus harder to see? I don't expect moving it to userspace to reduce overall complexity —- just move it elsewhere.

<comment><USER16> Because it's in the kernel you aren't exposed to all the knobs. There are hundreds of parameters controlling Linux tcp behavior. Even experts overlook some aspects. Hoisting this up into your application makes it visible. Why should there be a system parameter that limits the accept backlog of your server, silently? It makes no sense.

<comment><USER18> Because the people who wrote it think it's better being a bit more inaccessible in the kernel so that regular users don't shoot themselves in the foot thinking they know better than the designers what the values should be. The people who know what they're doing will be able to set the parameters regardless of where they're hiding.

<comment><USER16> This condescending attitude is itself a strong argument against whatever it is you work on.</comment></comment></comment></comment></comment>


YouTube "Feather" Beta - <AUTHOR> http://www.youtube.com/feather_beta

<comment><USER2> It's existed for a very, very long time

</comment><comment><USER3> This is great! Hope it sticks.</comment>


Debian 9.4 released - <AUTHOR> https://www.debian.org/News/2018/20180310 <comment><USER2> The fact that the vast majority of these bugfixes and CVEs are minor is a testament to the continuing stability and quality of Debian and its ecosystem. I've run Debian on several machines for years and have upgraded confidently without a hitch. A big thank you to the maintainers and developers for making this possible.

<comment><USER3> Debian is like Arch but the opposite

<comment><USER4> Arch is very stable when using mainline packages, which are the default packages...

This is "Arch is unstable" thing is very much a myth pushed forward by people who only dabbled with Arch once or twice then had it break, then completely dismissed it as unstable. Without first learning how to use it even at an intermediate level - not even expert.

I too had Arch break when I was a newbie but I learned quickly how not to and it's been incredibly stable ever since. Not worse than Debian and more so than Fedora in my experience.

In retrospect my actions which had broke it originally had little to do with ArchLinux itself but a general inexperience with Linux and over-eagerness when using beta software (which you have to opt in to, it's not default with Arch).

The end result is that I've come out of it a far better Linux user with skills highly applicable to Debian and other distros. It's a far easy place to learn Linux/Unix properly and become familiarized with the OS/dir structure/config than any other distro.

<comment><USER5> > This is "Arch is unstable" thing is very much a myth pushed forward by > people who only dabbled with Arch once or twice then had it break, then > completely dismissed it as unstable.

This accurately describes my experience. I stopped using Arch around the time git broke when I updated. I'm not clear what mistake I made. What level of Arch intermediate or expert level Arch knowledge is required to keep git working?

<comment><USER4> Well, how did git "break" exactly?

<comment><USER3> Edit: I've been told this meme is pretty accurate [https://i.redd.it/jo3ylah5yti01.png](https://i.redd.it/jo3ylah5yti01.png)

</comment></comment></comment></comment></comment></comment><comment><USER6> Please note: Starting with Debian 7, the minor number is not part of the Debian release number, and numbers with a minor component like 8.7 or 9.4 now indicate a _point_ release. Basically, only security updates and major bug fixes, with new updated installation media images. This, 9.4, is _not_ a new major release of Debian.

</comment><comment><USER7> Thumbs up for Debian, the workhorse of the entire Linux ecosystem.

</comment><comment><USER8> Debian for server, definitely.

Ubuntu for desktop, which is similar to Debian.

Openwrt for devices that has limited RAM.

FreeRTOS for MCUs with real-time needs.

That covers the universe for me, enjoy them all.

<comment><USER9> I started installing Debian on desktop. It is far more stable than Ubuntu. And if you install the non-free Debian, there are usually no issues with wifi or graphic drivers (you need the non-free version though).

There was time where Ubuntu on desktop made sense, but now Debian is just better and more stable.

<comment><USER10> Has things changed ? Debian stable was too old and testing used to break something every time I updated. I moved to ArchLinux since then

<comment><USER11> From my perspective the mindset with Debian is that distribution packages are never going to be the super latest, but are going to usually work well.

If I need something bleeding edge it's straightforward to just download the source from wherever and install it myself with stow in /usr/local, which makes it easy to remove/upgrade if needed. I anyways do pretty much everything in VMs, so package availability for my "base" system (which I mostly use only to run VMs) is not that big of a deal. If I need bleeding edge I can always spin up an arch linux VM...

</comment></comment></comment></comment><comment><USER12> Can someone explain this to me? I've had an issue with Postgres on Debian stable because Postgres had to address a CVE which changed its dump format in a backwards-incompatible way. This meant that my local Debian on my laptop couldn't load a dump I created on a patched Ubuntu server. Postgres 9.6.8 is the one that fixes this CVE in the 9.6 series.

This says the stable package is still vulnerable:

[https://security-tracker.debian.org/tracker/CVE-2018-1058](https://security- tracker.debian.org/tracker/CVE-2018-1058)

But Postgres doesn't show up here:

[https://security- tracker.debian.org/tracker/status/release/s...](https://security- tracker.debian.org/tracker/status/release/stable)

The 9.6.8 version indeed isn't on stable yet. So is the latter link just buggy? Or is Postgres getting filtered out?

Edit: Never mind, after messing with the filter (guess there's no Debian Security Advisory for this CVE), I was able to get Postgres in the second link:

[https://security- tracker.debian.org/tracker/status/release/s...](https://security- tracker.debian.org/tracker/status/release/stable?filter=1&filter=unassigned_urgency&filter=nodsa)

<comment><USER13> On that later page, if you check "include issues tagged <no-dsa>" that Postgres issue shows up.

<comment><USER12> So this has taught me something I didn't know about all Debian: not all CVEs have the same priority or have to be fixed in point releases. Many CVEs don't even get acknowledged by a Debian Security Advisory at all. I guess if I want security to be the utmost priority, I should be using OpenBSD.

<comment><USER14> In this case you probably should just be using postgres' apt repository, which'll carry new releases very promptly.

<comment><USER12> Kind of the point for me of using Debian is to not have to go shopping around all of the web for packages and because I want to have a uniform packaging standard: Debian Policy. That's why I don't trust anyone but Debian to produce Debian packages. It's a lot of hard work to make sure all packages adhere to a common standard, a standard that upstream distributors will not follow.

In this case, however, the Postgres packager is also the Debian packager, so I decided to trust his direct upstream Debian packages and installed them. This is a rare exception, and I don't want to be doing this all the time.

</comment></comment></comment></comment></comment><comment><USER15> How is debian nowdays for setting up things like containers? I've tried lxc in the past few years ago and got it working but it was not a pleasant experience, templates were broken.

I would love them to gain back more server marketshare.

<comment><USER16> I use for nearly all my docker containers the distro Alpine Linux.

It is much lighter. I commonly find that Alpine images use at least 300MB less disk than Debian ones.

<comment><USER17> For containers I'd suggest using Debian slim.

<comment><USER16> Interesting, I didn't know about Debian slim.

For comparison, I just found this blog post [1], claiming the following:

> According to the docker images command, the debian:jessie-slim container > clocks in at 88MB, compared to the full-fat debian:jessie container at > 123MB. For reference, the ubuntu:xenial and centos:7 base containers are > 130MB and 193MB respectively, where as the alpine base container is only > 4MB, as mentioned in my post about the alpine base container.

[1] [https://elegantinfrastructure.com/containers/cotw-debian- jes...](https://elegantinfrastructure.com/containers/cotw-debian-jessie-slim/)

<comment><USER18> This is always a dilemma for me... Alpine allows for really small packages, but I never got used to its packaging tools (and I don't feel confident writing Dockerfiles with it). A matter of practice I guess. Besides, Docker caches layers so if you're using some common base layer (like Ubuntu or Debian) it's probably already on the machine. So I usually write Dockerfiles as `FROM debian` and then later adapt them to Alpine if they're meant to be distributed.

<comment><USER16> Yes, I did like you before. But once I got used to Alpine, apk (its package manager), and ash (its shell), I feel myself at home. Furthermore, in most occasions I find newer updated software on Alpine vs. Debian.

Also besides disk space, I really have no idea how docker manages memory, but my assumption (naive?) is that smaller images will consume less RAM memory. I didn't benchmark this though.

<comment><USER18> Sorry for late reply. Depends on what is in the package. Docker is just a thin separation of processes, so the situation is very similar to size of packages on host Linux OS. What matters is the size of binaries and loaded data, which I guess is not that different between distributions.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Depending on your use case, Debian Stable can be a great thing. If it worked great on a given computer (i.e. everything works), and I was just writing or using programs that were themselves fairly "stable," then it will work.

If you're a developer and don't mind older packages, or if you install and manage your dev tools from outside the distribution, it can also work for you. This is a lot of people.

But if you rely on your distribution to provide reasonably up-to-date packages (and for better or worse, that's me right now), then Debian Stable might not be the right choice. When I do development in Linux, I tend to be more comfortable in Fedora, which is much more aggressive when it comes to updating packages during the life of a release. I'm not crazy about major upgrades every 6 months, though you can do it once a year if you wish.

For better or worse, I think that Ubuntu + PPAs is the path of least resistance -- if you trust the maintainers of your PPAs, that is.

</comment><comment><USER20> Frankly, I do not understand the fawning over Debian. It became the same kind of "try to be everything and do nothing well" distribution Slackware was.

Up until 8 it was still good for servers but 8 brought systemd, which _maybe_ has a place on a Desktop, but is definitely not anything useful for servers. Ok, so lets pretend it is a desktop. Oh wait, it is a desktop that does not contain the lastest video drivers? It is a desktop that does not contain seamless switch between free nvidia and proprietary nvidia?

</comment><comment><USER21> I'm a bit surprised that the Docker image is not released as part of the same release process.

They seem to be released by a separate project / org called "debuerreotype"[1].

[1] [https://github.com/debuerreotype/debuerreotype](https://github.com/debuerreotype/debuerreotype)

</comment><comment><USER22> Debian servers and mirrors still showing 9.3 (UK, 17:45 Saturday 10th March)

I just downloaded v9.3 DVD1/2 to make an offline installation.

Anyone know where the 9.4 upgrade iso is?

</comment><comment><USER23> Obligatory reference to the list of platforms Debian has been ported to and the "blends" it can be customized to. [https://www.debian.org/ports/index.en.html](https://www.debian.org/ports/index.en.html) [https://www.debian.org/blends/](https://www.debian.org/blends/)

BTW, many moons ago I used Red Hat on Alphas, so I know it's not the only one. Still it's impressive.

</comment><comment><USER24> I realize the decision is said and done but can anyone share in easy to understand terms what the big deal is with systemd?

<comment><USER25> Systemd is an init system that tries to do everything: system init, hardware monitoring to load drivers on demand, configure networking, rerun failed jobs, service management, and it comes with a binary-format logger.

The claimed advantages of systemd are mostly relevant, IMHO, for mobile devices that change configuration often. It brings no particular advantage to servers or desktops, unless you tend to swap out lots of peripherals on your desktop.

The disadvantages are partially political -- systemd tries to take over every job it can; the originator has made it clear that his vision is for every system function to be handled by systemd -- and partially philosophical: it used to be the case that most subsystems were independent of each other, so that they could be replaced more or less easily if the sysadmin had different needs.

The current stable Debian still makes it possible to run a different init system with just a line of package installation and a reboot.

<comment><USER26> > The claimed advantages of systemd are mostly relevant, IMHO, for mobile > devices that change configuration often. It brings no particular advantage > to servers or desktops, unless you tend to swap out lots of peripherals on > your desktop.

Systemd provides huge advantages for administering desktops and servers as well. I'd actually go so far as to say those are the main reasons that systemd won (it's the default init system for all of the top ten Linux distributions by marketshare).

<comment><USER27> >Systemd provides huge advantages for administering desktops and servers as well

Can you list some of them?

<comment><USER28> Creating unit files is trivial and you're sure that you're getting high quality service supervision, unlike with home grown scripts.

Seeing the logs of every service is also trivial since it's standardized. No more spelunking through /var/log or some random location.

There's also other parts I'm probably forgetting right now, but in general having a standard interface for service interaction is really, really useful.

Sometimes I get the impression that Linux folks underestimate the benefits of standardization. For example there's still no good, standard way to check the exact name and version of a distribution. If they can't agree about something as banal as that... (I think there is a convergence towards /etc/os-release, but when was the first distro launched? 1992? :) )

<comment><USER29> > Creating unit files is trivial [...]

Unless you expect anything but most trivial use cases, e.g. some user-owned directory under /var/run (/run). Then you need to wild-guess at what combination of options needs to be set. Good thing that those options are at least described in docs.

> Seeing the logs of every service is also trivial since it's standardized. No > more spelunking through /var/log or some random location.

syslog _already was_ standard. We gained nothing really from journald.

<comment><USER30> Many services don't log to syslog. For example, on Debian derivatives, /etc/init.d/networking was logging to the console only. Good luck to get the output once the system has booted. With systemd/journald, you get the info.

<comment><USER29> > For example, on Debian derivatives, /etc/init.d/networking was logging to > the console only. Good luck to get the output once the system has booted.

Well, this is my luck: /var/log/boot, written by bootlogd, which is installed from distribution packages on Debian. So? What of it?</comment></comment></comment></comment></comment></comment></comment></comment>


What You Need and Want With Windows Azure Part II - <AUTHOR> http://compositecode.com/2010/10/08/windows-azure-part-ii/ One of the most useful tools to use in Windows Azure Development is the Windows Azure MMC.  The Microsoft Management Console, or MMC, is the management console that many of the Windows Server Management interfaces can plug into.  The Windows Azure ... <comment><USER2> MMC is a good start, but it isn't where it needs to be.

We have a love/hate relationship with Windows Azure at my company. It does a few amazing things but at the cost of cutting some simple but needed features. Specifically in the area of data storage, queries and node management.

I feel like I'm constantly shouting at the team on the forums, twitter and facebook to fix issues or release some simple functionality. Hopefully we'll see some big changes in the Q4 release.</comment>


Letter to hackers frm a non-hacker: Quit your "wow" app and build us something useful. - <AUTHOR>

First of I am not a hacker, so I can clearly see things you guys can't (vis versa). Stop building cool things and  build something useful for the 95% of us who don't know how to. Your web (yes the one you help build) is unstructured and there is so much data that I am sure we will never be able to organize it. But what we can organize is our individual data. BUILD  (us) MY OWN API. I don't want to go to 20 different sites a day to do the 20 things I want. Facebook's, Digg's and all this other APIs don't really matter, what matters is mine. Let me rephrase this, APIs are great, but you guys have been  going the wrong way about this. It should be the other way around. Services shouldn't have an API (ok they should) but I should be able to gather all my data into one place and let services I want plug into it and do the work. I am tired of supplying similar infos to one million different startups. I want to simply click "ADD" (the Facebook way) and give the service an access level. That is it. I dont even want to know your URL. And when I click delete,  give me back my data collected. I want a place where it all comes together. And please do not tell me (us) about your webOS project. I (we) want something helpful not a copy of the desktop. That's it. Sorry if I offended you guys, don't take it personally.
<comment><USER2> "First of I am not a hacker, so I can clearly see things you guys can't (vis versa). Stop building cool things and build something useful for the 95% of us who don't know how to."

I am a hacker.

I am also an adult and therefore I actively choose what to do with my time and what to build, and whether to build something cool, useful, both or neither.

What I really don't need is inarticulate people who haven't demonstrated any particular expertise in anything giving me advice on what to build and how I should spend <b>my</b> time and resources.

Now go away and do something useful.

As you said, "Sorry if I offended you .. , don't take it personally."

<comment><AUTHOR> Listen "adult" the way I see it is very simple. If you are offended you are probably one of those hackers I am talking about. If I made a similar statement and said that NBA players need to play better, I am sure Steve Nash wouldn't feel offended, the ones who will (like you) are probably the other 80% who quite frankly are just above mediocre. Now, Prove me wrong and show us what you have that is sooooo useful.

<comment><USER2> I am not offended. I just think you are just wasting everybody's time and have a misplaced sense of entitlement.

Why should I spend time on "proving you wrong" or "showing you what I have"?

Who are you to judge what is useful or not, how other people should spend their time and resources? If you are not a hacker that is fine. Just "prove to us" you are good at something (anything!) before making such outlandish requests asking other people to change how they work to satisfy _your_ notions of what is proper.

If a hacker wants to work on a webOS (to use your example) it is no one's business but his own. If you have some inchoate desires on how you'd like the internet to function, and you can't do anything about it and are not willing to pay anyone to do it, its just too bad.

"hey Wall Street Investment Bankers, I don't think you provide any value to me by working on exotic derivatives. I don't understand how they work or why you spend time on them. Please stop what you are doing and work on balancing my bank account and pay my credit card bills for me and persuade my bank to give me more money"

Bozo alert! Other people don't exist to satisfy your desires or work towards your vision of what the world should be. You might be able to get other people to work towards what you want _if_ you offer something in exchange - money, skills etc. In the absence of having anything to exchange, you are just babbling into the wind.

Demanding that someone work in a style that seems perfect to _you_ (when you admittedly know nothing about what they are working on or what the issues are) seems to be .... suboptimal.

<comment><AUTHOR> Edited: PP. I could ask people to all follow a new religion named "Zigajaga", but they probably won't. I am not demanding you. I don't know you, and honestly after today, your username won't remind me much, but all I was doing was asking you guys to help. Now you can look at the first layer and say "Oh just another ignorant one", or you can say "What can I learn from this post"? The answer will be "Am I doing something useful?" or "Do I really need to write a script that reads a post and knows to send it to me or trash it or block the username forvever?" and that would be building what I ask you to? Not Demand, Not Forced, But asked. At the end, seriously, I got love for all hackers. I trully do, but if that means I can't be honest and straight with you guys then it certainly means I was wrong in the first place.

</comment></comment></comment></comment><comment><USER3> I see what you're getting at - the user owns the data and access permissions (stored on the web), and the services work from that. With S3, this is a much more realistic idea. The problem is that many services will not allow your information out, such as facebook. The question is, is this something alot of people want? I know what I would like is to be able to always export my data out into a common format (like xml) so I can use it somewhere else.

If there were big demand for this kind of service, a startup could launch that became MyWebApps.com - a place where your data is stored and which has interfaces to the key other apps that you use and allow data to come out and possibly synchronizes etc. Your portal to the web where you always go first. This is a pretty ambitious idea with countless lines of code involved for getting all the different systems to speak the same language, but if it were pulled off, it could be big.

</comment><comment><USER4> No one cares what YOU want, we care about creating something useful, be it for ourselves or for a larger group of people. Further, utility can, in fact, be derived from "wow" apps. You wouldn't understand because, as you've said, you're not a hacker. You should try it though, because building something yourself is a lot of fun and very rewarding.

Regarding the "personal API," I think it's a good idea if properly implemented (read: incredibly easy to use). Obvioulsy you haven't done much research because there are people working on this already. Someone below mentioned OAuth, and OpenID is another effort in this direction.

You'd get a lot farther if you simply just asked the community if someone was working on this and/or pointed out that it's a good, useful idea.

</comment><comment><USER5> Your post is as disoriented and unfocused as the kinds of work you're railing against. If you can't even discuss the idea in a succinct and compelling way how can you condemn others for not creating it?

Fear not though, the problem your discussing is probably the trendiest of all trendy ideas anyone is working on right now. It will be solved, by people who can articulate it in terms users can understand _and_ create it.

<comment><AUTHOR> staunch. Blaming the way I express myself is not gonna get us anywhere today. The fact of the matter is you understood it, now let's do something useful plz.

<comment><USER6> Blaming entrepreneurs and hackers for how they express themselves through what they build isn't going to help either.

</comment></comment></comment><comment><USER7> I'm sorry to say that but what you're looking for is called the desktop (not that it's a bad idea). Unless you really like ads, privacy issues, slow applications and inconsistent user interfaces.

<comment><AUTHOR> I am sorry to say that but I want mobility. I dont want to drag a laptop or PC around.

</comment></comment><comment><USER8> I have no idea what you're talking about so I don't even understand what itch you want to have scratched.

</comment><comment><USER9> I like it. It sounds like you want to center the data on individuals, not on the applications they use.

</comment><comment><USER10> You make great points. I think good hackers who are interested in doing a startup really should pay attention - what we think is interesting is useful is often not anything people want. Great if you're just hacking, but not so much if you want to build a business.

</comment><comment><USER11> I think a problem is that either all the APIs have to agree, so new sites can be added without manual coding, or every startup has to agree on something, which could get hard. Also, the sites without APIs will not be included in this app?

</comment><comment><USER12> <http://oauth.net/>

<comment><AUTHOR> Thank you. Now we are getting somewhere.

</comment></comment><comment><USER13> You don't get it, Cool is Useful.</comment>


A Perk of Our Evolution: Pleasure in Pain of Chilies - <AUTHOR> http://www.nytimes.com/2010/09/21/science/21peppers.html

<comment><USER2> While the article is a good read, the best part is the ad on the first page that links to MySpiceSage.com offering Ghost Chilies! Yum--- or not, your mileage may vary along with your taste buds.

<comment><AUTHOR> My favorite line was "Or as Paul Bloom, a Yale psychologist, puts it, 'Philosophers have often looked for the defining feature of humans--language, rationality, culture and so on. I'd stick with this: Man is the only animal that likes Tabasco sauce.'"

</comment></comment><comment><USER3> If you put a bunch of monkeys in an ecodome with barely enough bananas to sustain but plenty of jalepenos, how many generations do you have to wait until they pick the lock and break out?</comment>


US-government issued phones run 'Chinese malware' - <AUTHOR> https://www.bbc.com/news/technology-51054901 <comment><USER2> > It had bought one to verify the claims and informed the company selling > them, Assurance Wireless, of its findings but had not received a response, > it said.

Knowingly distributing malware - doesn't that carry jail time under the CFAA? I sure know _I_ would be facing jail if I tried that.</comment>


Introducing the Canary Release Channel - <AUTHOR> http://blog.aerofs.com/introducing-the-canary-release-channel/

<comment><USER2> IMHO, a more explicit naming scheme like "Stable/Unstable" would be better to not confuse the user.

<comment><USER3> Maybe they are trying to subtly tell us that they have had an NSL or have been forced to put in a backdoor by the NSA. Warrant canary?

Thpugh more than likely it was just a poor choice of name!

<comment><AUTHOR> We're actually just borrowing from the phrase "canary in the coal mine" \-- [http://en.wikipedia.org/wiki/Animal_sentinel](http://en.wikipedia.org/wiki/Animal_sentinel)

Google Chrome uses a similar convention ([http://www.google.com/intl/en/chrome/browser/canary.html](http://www.google.com/intl/en/chrome/browser/canary.html)), as do many other companies when talking about unstable/nightly/etc. builds (simply google 'canary build' to verify :)

<comment><USER4> Ember also uses the 'canary' nomenclature.</comment></comment></comment></comment>


Cheat sheets for front end web developers - <AUTHOR> http://sixrevisions.com/resources/cheat_sheets_web_developer/

<comment><USER2> Thanks, bookmarked this for later.</comment>