Chinese scientist who edited twin babies' genes jailed for 3 years - <AUTHOR> https://lite.cnn.io/en/article/h_a23f39996d9c239544ae6130989a63f1 <comment><USER2> What is up with these comments? State sponsored behavior modifications? Eugenics? Public propaganda while hiding some sinister truth? China isn't some kind of dystopian conspiracy-laden techno-thriller enemy state. The team simply wanted to sneak this experiment past medical practice laws and scientific ethics -- they probably would have been fined and punished in a similar fashion have they done it in the U.S or Europe.

<comment><USER3> People really like sino bashing for some reason.

<comment><USER4> Complete mystery. No human rights abuses in China...

Quick reply with a false equivalency!!

</comment></comment></comment><comment><USER5> It will be interesting to see if the babies grow up with any side affects from the editing. DNA is hugely complex and changing some elements of it could easily effect other parts (I imagine).

<comment><USER6> Very unlikely. For a conception happening from people around 30 years, they will be passing 20 to 30 thousand DNA mutations to their offspring, and around 100-150 _genetic_ mutations.

Statistically cas9 is barely worse than just be born to a late parent.

<comment><USER7> >Statistically cas9 is barely worse than just be born to a late parent.

Depends on what's being mutated. It seems likely that 1. random mutations with catastrophic outcome would be negatively selected for evolutionary, 2. Humans try to meddle with genes that may be disproportionately important.

So I'd put the risk of gene editing quite a bit higher.

</comment></comment></comment><comment><USER8> Interesting. I still roll to disbelieve on China shunning human enhancement. The gains are just too great, the fruit too low hanging. Several standard deviation increases in polygenic traits are on the table, including the most important economic trait, intellectual ability: [https://www.gwern.net/Embryo- selection](https://www.gwern.net/Embryo-selection)

Given China's almost unique state capacity to think long term, history of eugenic policies in sport, an ironically less Lysenkoist population than America, and a cultural focus on educational achievement and competition, this seems overdetermined.

In fact, animal breeders are getting very keen on iterated embryo selection. So the technology will get investment even in environments where it is illegal for humans.

We live in interesting times. Even a 2 standard deviation increase would allow a small population to produce the vast majority of genius-level intellects. 2 standard deviations is on the way low end of what iterated embryo selection offers. Should we find the causal variants, direct editing offers vastly more.

A world-wide ban on this technology is not on the table. Plan accordingly.

<comment><USER9> Well, if human gene editing really would work, than china (and others) would not only select for intellect, but also for obedience. Which is scary.

Apart from that I do believe, that in the future human gene editing makes sense. Once it is actually understood. (until then volunteers are allright with me, but experimenting with babys ... is a bit off)

Because we are successfully eliminating natural selection in various ways. But evolving works naturally through selection. Degeneration would be the long term effect, unless we find other ways. And improving our genes as we see fit ... seems to be a solution.

<comment><USER10> There was recently a story about a "Paean to Alpha Centauri", so I'll leave you with an appropriate quote:

"My gift to industry is the genetically engineered worker, or Genejack. Specially designed for labor, the Genejack's muscles and nerves are ideal for his task, and the cerebral cortex has been atrophied so that he can desire nothing except to perform his duties. Tyranny, you say? How can you tyrannize someone who cannot feel pain?"

\- Chairman Sheng-ji Yang, "Essays on Mind and Matter"

I have no idea how you'd manufacture both a loyal and genius breed - what/whom are they loyal to?

<comment><USER11> The whole first chapter of Brave New World comes to mind as well...

"Bokanovsky's Process"... x-ray, and alcohol to clones of the "worker class" to make them slow and stupid to do your bidding.

[https://www.huxley.net/bnw/one.html](https://www.huxley.net/bnw/one.html)

</comment></comment></comment></comment><comment><USER6> I do feel this follows the same genre as what was going under Xi's hounding campaigns.

Not content with simple fraud charge, those guys always go for complete demonisation of the persona accused and self-denunciation.

</comment><comment><USER12> Surprise surprise, he has 3 million yuans to be fined.

Had he given more of those to the government officials and/or jurisdiction, he wouldn't even have to face the charges in the first place.

<comment><USER13> 1 million yuan, apparently:

> Zhang was sentenced to two years in prison and fined 1 million yuan > ($143,000), while Qin was given a suspended sentence of one year and six > months in prison and fined 500,000 yuan ($71,600).

</comment></comment><comment><USER14> Whoa, lite.cnn.io is back?

(context: [https://news.ycombinator.com/item?id=21804160](https://news.ycombinator.com/item?id=21804160))

<comment><AUTHOR> It comes and goes. I have no idea who maintains it. I check it every day and it is up at least part of the week. Some weeks it stays up.

</comment></comment><comment><USER15> Surely I'm not the only one who thinks this sort of research is necessary. The sooner we are able to genetically engineer improved humans - first eliminating the incidence of genetic disease and second making overall healthier, happier and smarter people - the better the world will become.

<comment><USER16> Shall they try your children first? Oh so sorry - they died young and in agony, from unintended side effects. But Science!

<comment><USER15> Of course we wouldn't try on humans first. Animal trials precede human trials.

<comment><USER17> animals are not humans so something that works on a mouse or a pig could have unknown bad side effects for humans.

<comment><USER18> That is a fully general counterargument against animal testing to show the safeness of any drug or treatment. Clearly humans and other animals have enough in common such that successful animal trials are evidence of safety in humans.

</comment></comment></comment></comment></comment><comment><USER19> We are gonna look back on this just how we look back on Giordano Bruno. Future generations would find it ridiculous that he was jailed for technological progress and science.

<comment><USER16> He was completely irresponsible, did no do what he advertised he would do, and endangered the health of many people.

Don't get seduced by the dream he was selling. He was in fact a fraud, not a researcher.

<comment><USER19> You could say that for any invention ever. Even the internet, our greatest invention so far. However, the net benefit is a non-negative value.

<comment><USER16> You can also say that for any fraudulent enterprise ever. He added no benefit whatsoever, having done wrong what his betters could have done right.</comment></comment></comment></comment>


20/10/2010, World Statistics Day - <AUTHOR> http://unstats.un.org/unsd/wsd/Default.aspx

<comment><USER2> I love the quote from Ban Ki-Moon. It's generic to the extreme:

"On this first World Statistics Day I encourage the international community to work with the United Nations to enable all countries to meet their statistical needs."

Could also be:

"On this first International Talk Like a Pirate Day I encourage the international community to work with the United Nations to enable all countries to meet their linguistical needs."</comment>


Migrating Github Projects to HTTPS - <AUTHOR> http://blog.brycekerley.net/post/1609476959/migrating-github-projects-to-https

<comment><USER2> What is the advantage of using https over ssh?

All I can see is being on a foreign computer. When on a foreign computer I generally don't even have git so why bother.

<comment><USER3> The big advantage comes when the URL is public somehow - like with git submodules. The http/https URL works for both read-only and read-write users unlike git:// or git@ URL's.

The disadvantage is that http/https can only be pushed to by users who have git 1.7.

<comment><USER4> technically, it's 1.6.6 and above, just to be pedantic, but quite a number of people are at that now, it's nearly a year old now.

</comment></comment></comment><comment><USER5> _1: Github crew, can we get a way to use our API token or some other revokable token as a password for HTTPS repo operations? I appreciate that anything involving "revokable tokens" is really hard to make a decent UI for, but it just feels bad keeping that in my .netrc file._

Isn't this exactly how their regular SSH access works? If you don't want an SSH key to work anymore, just remove it from GitHub.

<comment><AUTHOR> It is; however:

* sometimes ssh is blocked

* HTTPS can be used to pull from public projects anonymously, or without configuration; the same URL can then be used later to push.

* HTTPS also works just fine without any configuration or password storage, it just nags you for a username and password if it needs it

</comment></comment><comment><USER6> > git pull origin master

I think in general, as well as this particular case, it would be better to do

      git fetch origin

One doesn't always want to merge from the remote branch.

</comment><comment><USER7> Apparently, javascript is necessary to view the code snippets.

While I'm ok with that, it'd be nice to have a <noscript> section saying so, rather than just having large chunks of your content missing.</comment>


How to be memorable at any conference or event - <AUTHOR> http://www.succesure.com/2009/how-to-become-memorable-at-any-conference-or-event/#more-291 In the last few months, I've been to Angelconf, Demo Day, Codecon, and now off to Adtech.  I liked this post because I used to just go to conferences and "see what happened."  But, when I go with an idea of what I want to get out of it (whether it's meeting a particular person, etc.), the results are far more powerful. <comment><AUTHOR> In the last few months, I've been to Angelconf, Demo Day, Codecon, and now off to Adtech. I liked this post because I used to just go to conferences and "see what happened." But, when I go with an idea of what I want to get out of it (whether it's meeting a particular person, etc.), the results are far more powerful.</comment>


Ninja out. - <AUTHOR> http://ninja-out.f0o.com/

<comment><AUTHOR> It's a webpage that's supposed to evoke feelings of 'what is this', draw attention, and also to provide a slight grin feeling -- for ninja appreciators.

</comment><comment><USER2> The audio is AWEFUL and LOUD. Thanks for making me crap my pants.

</comment><comment><USER3> What?</comment>


Bad news: 'Unblockable' web trackers emerge. Firefox with uBlock can stop it - <AUTHOR> https://www.theregister.co.uk/2019/11/21/ublock_origin_firefox_unblockable_tracker/ <comment><USER2> "uBlock Origin" shouldn't be changed to "uBlock" in the HN title since they're two different things.

(As of this comment, the title is "Bad news: 'Unblockable' web trackers emerge. Firefox with uBlock can stop it.")

</comment><comment><USER3> The actual title at the site is:

Bad news: 'Unblockable' web trackers emerge. Good news: Firefox with uBlock Origin can stop it. Chrome, not so much</comment>


Do hackathons promote spaghetti code? - <AUTHOR>

Discuss. <comment><USER2> Hackathons promote the fact that your coding style should change depending upon the constraints of the programmer.

Having spent most of my career in R&D, I am constantly amazed by the fact that most folks keep the very same methodology no matter the budget, deadline, risk or importance of a project. Such people are one trick ponies that can only survive in a narrow set of environments.

Hackathons promote adaptation of approach. Spaghetti code is simply one possible result.

</comment><comment><USER3> Hackathon winners are dominated by polished design and clever presentations. Working code is often an afterthought let alone readable, maintainable, scalable code with proper test coverage.

I highly doubt you'd find Linus Torvalds, DHH, Guido van Rossum or any other highly productive programmers competing in a hackathon. Maybe they have good reasons.

Also this is worth reading: [http://www.thoughtworks.com/insights/blog/why- hackathons-suc...](http://www.thoughtworks.com/insights/blog/why-hackathons- suck)

<comment><AUTHOR> A nice read.

</comment></comment><comment><USER4> If you launch your hackathon project and never touch it again, then maybe...

But I think a better way to frame it is that hackathons promote streamlined MVPs which _must_ be iterated-on and refactored at a later date to be sustainable projects. Some would argue that this is a fine business model, inside or outside of a hackathon setting.

</comment><comment><USER5> Anything produced by multiple people in a restrictive time period will be spaghetti code. Hackathon's aren't about producing quality code.

<comment><AUTHOR> My observation is, the younger generation is being treated to believe that spaghetti code is all it takes to make something work, and this means the true art of software development such as design patterns, architecture etc. might be forgotten. A reason I personally don't attend hackathons is because they push me towards 'writing a lot of crap' code. I am wondering if other people have had the same thoughts about hackathons and the hackathon movement in all.

<comment><USER5> Again, I think you miss the point of a hackathon. Hackathons are suppose to be for fun. Rarely do you see anything professionally ship from a hackathon, but occasionally a few good ideas actually get started at hackathons.

If you're personally offended by bad code, that's fine, but hackathons are not shops on professional software development practices.

<comment><USER6> Exactly. A hackathon is for prototyping and MAYBE going "hey look, you should hire us if you want our idea so we can do it the right way".

</comment></comment></comment></comment><comment><USER7> HACKathon</comment>


Minute by Minute Twitter Sentiment Timeline of the VP debate using R - <AUTHOR> http://stathack.wordpress.com/2012/10/12/minute-by-minute-twitter-sentiment-timeline-of-the-vp-debate/

<comment><USER2> This would make for a great election stats website displaying live sentiment analysis.

EDIT: Even better if you could even annotate each point with the most tweeted quote.</comment>


Hiring: The First Librarian of Congress for the Internet Age - <AUTHOR> http://www.theatlantic.com/technology/archive/2015/06/hiring-the-first-librarian-of-congress-for-the-internet-age/396038/?single_page=true <comment><USER2> I can only think Jason Scott[1] would be the obvious HN sponsored candidate for this position. Imagine the good he would do with the resources.

[1] - [http://archiveteam.org/index.php?title=User:Jscott](http://archiveteam.org/index.php?title=User:Jscott)

<comment><USER3> I cannot think of someone better suited for the job, except perhaps Brewster Kahle [+]. Him and Jason Scott working together, with US government backing, would be (please forgive me for this) _killing it_.

"Brewster Kahle is an American computer engineer, Internet entrepreneur, internet activist, advocate of universal access to all knowledge, and digital librarian. He is the founder of the Internet Archive, the Internet Credit Union, Alexa and Thinking Machines and a member of the Internet Hall of Fame."

[+] [https://en.wikipedia.org/wiki/Brewster_Kahle](https://en.wikipedia.org/wiki/Brewster_Kahle)

<comment><USER4> You want someone who really "gets" that information needs to be carefully preserved - not just "available" (in the manner of: [http://hitchhikersguidequotes.tumblr.com/post/14333727462/mr...](http://hitchhikersguidequotes.tumblr.com/post/14333727462/mr- prosser-but-mr-dent-the-plans-have-been) ). I think Brewster is that kind of person.

</comment></comment></comment><comment><USER5> Change the Federal E-Rate guidelines to allow non-library-science master's degree holders to "be librarians", & for the districts that employ "nontraditional" librarians to receive full funding. Through regulatory capture, the American Library Association subsidizes a program of study that has an astonishingly narrow scope, to the exclusion of anything more technical than MS Word & General Reference Center Gold.

</comment><comment><USER6> Some ideas: train volunteers to do the digitization work; aggressively invest then rent out the digitization infrastructure to industry once backlog is cleared; sponsor competitions inviting open source solutions for LoC IT problems.

</comment><comment><USER7> Let's see what'll happen to the LOC-supported BIBFRAME.

And whether it or something IFLA-supported FRBR/FRAD/FRSAD/FRBRoo/PRESSoo will win out.

We might even have the answer to that by the time the next Librarian of Congress comes around.

</comment><comment><USER8> The OP mentions this midway, but this leadershift change is largely happening after a year-long GAO investigation found that the Library of Congress was poorly managed by its leader of 61 years:

[http://www.washingtonpost.com/entertainment/museums/americas...](http://www.washingtonpost.com/entertainment/museums/americas- national-library-is-behind-the-digital-curve-a-new-report- finds/2015/03/31/fad54c3a-d3fd-11e4-a62f-ee745911a4ff_story.html)

The report was damaging, but at the time (earlier this spring), Billington had refused to retire. Some of the findings from the report:

> _The GAO report is the most damning. Investigators noted that the library > could not confidently estimate how much its spends annually on IT costs, > although the GAO offered a guess of $120 million, about one-fifth of the > library's total budget._

 _"They don't know what they spend, they don't know what they have," said Joel C. Willemssen, the report's author. "The fact they have had five acting CIOs in less than three years, that's a big concern."_

 _GAO investigators found the library's master list of personal computers numbered 18,000 when it actually had fewer than 6,500. The number of systems used by the various units was listed at 30 but another was tallied at 46. "After we raised the discrepancy . . . [the library] provided us with a revised list of 70 systems," the GAO said._

Part of this will be spun as an "old-people-don't-get-technology" thing...that Billington being 86+ years old meant that he was unable to manage the library's transition to the digital age. To put that in perspective, he was nearly 50-years-old by the time the Apple ][ came out. But I don't think that's the right lesson to take...Older people _can_ learn computers just fine as an intellectual pursuit, it's just that many of them don't have much reason to, especially if they are near retirement or have a job that's not computer- related. It's a matter of incentive; if their job required computers, and they value having a salary, anyone can become adequately acquainted with the computer.

But it just sounds like Billington, among other things, was just a bad/aloof manager, something that can get exacerbated over time if you're at the same job for 10/20/30+ years. If he was also willfully ignorant/uncurious about the digital age, e.g. "Books, radios, computers; they're all the same thing"...then it would seem that the LoC's technological progress would happen in spite of, rather than because of its leader's vision.

But if an 85-year old leader who is still interested in the work even 20 years after he could've retired to a nice pension, and engaged enough to humble himself to learn new technology. Sure, I'll take that any day over a 30-something startup-millionaire. I think institutional memory is very important, and being liked and respected by everyone who controls your budget is not an insignificant asset, especially when it comes to libraries.</comment>


Digital subscription costs are bleeding consumers dry - <AUTHOR> http://gizmodo.com/5448321/the-subscription-war-youre-bleeding-to-death

<comment><USER3> There's a ton of overlap here.

3G (or equivalent) on four devices. (And if you have 3G on four devices, there's absolutely no reason to be paying for mobile WiFi hotspots).

And then there are five different services for streaming video.

And then a land-line phone.

If you're paying anywhere close to the $738 they estimate as the maximum it's only because you're not paying any attention to what you're paying for.

</comment><comment><USER4> > _You've got your smartphone bill, your cable bill, your home broadband bill. > Those are unavoidable expenses—there's not much you can do about them._

I don't have a smartphone or cable. Calling them 'unavoidable' is a dubious claim.

<comment><USER5> Sentences make more sense in context. In this case, the context is under the tongue-in-cheek reference to the "well-equipped greek" which is someone who, presumably, wants to have all the "must-have" gadgets. For someone who considers that list of gadgets "must-have", I'd argue that both cable and smart-phones are certainly at the very top of the list.

Your point, then, is that you aren't one of those "well-equipped geeks" who must subscribe to everything. Neither am I (is anyone?). Neither of those bits of information undermine the point he is trying to make.

<comment><USER6> Not to mention that paying for a 3G data plan with almost any carrier in the US also entitles you to a whole network of WiFi hotspots with no additional charge.

Oh, and I still don't get why cable television is required these days - <USER2> show I follow is available online at substantially lower total cost than paying for 200 channels I'll never watch.

</comment></comment></comment><comment><USER7> Utter pablum. Slow news day? Ethically dubious journalist?

What consumer buys so many duplicative services?

Here's a piece of advice. Don't buy services that overlap in functionality. Don't buy items you can't afford to use. Consider the total cost of ownership.

The entire tone of this article is one of yellow "jurinalistic bias" and does not reflect accurate market research on consumer behavior let alone consumer need.

The author comes across as little more than a petty financially irresponsible, and technically ignorant, whiner.

</comment><comment><USER8> This is such a dumb article. The author makes it seem like our hands are tied and we are forced to pay such high fees. No, I'm sorry, those are the fees _you_ have to pay for all those services since _you_ chose all of them in the first place. Consumers have choice and they will pay for the products they desire as they fit in the realm of their--atleast theorectical--budget.

</comment><comment><USER9> When I saw the low-end of the range was $208 for all that stuff (80% of which I don't have) my first thought was "how cheap!".

I pay more than $208/month for just 2 of those services.

Who pays for a navigation app?

</comment><comment><USER10> They finally lost me when they said that $738 - the highest calculated possibility - was nearly a thousand.

Shit costs money. Oh no.

<comment> brown9-2 I can't imagine someone subscribing to each and every one of these:

* Unlimited voice, text and data on a smartphone

* Netbook data plan

* A "slate" (no clue what this is) 3G data plan

* Home internet

* Mobile internet for a laptop

* Wi-fi hotspot access

Oh, and a home landline.

Sounds more like someone thought up a headline they wanted to write and then came up with the data to back it.

<comment><USER6> A "slate" is the new class of touchscreen tablet devices expected to emerge Apple's announcement next week.

The journalist here doesn't seem to understand that a "slate" owner is unlikely to own a netbook at the same time. Nor does he apparently understand that unlimited data on a smartphone entitles you to ubiquitous free WiFi (in the US anyways), so your netbook/slate data plan is entirely unnecessary, especially when considering tethering capabilities of the phone you're _already_ using.

<comment> brown9-2 I think that they do understand it, but they're ignoring logic for purpose of the article's premise.

</comment></comment></comment></comment><comment><USER11> I wouldn't subscribe to this many things, but it seems pretty clear: if I'm _that_ into all this stuff, then it'll be worth the money and I'll find a way to pay it. If it's _not_ worth it to me, then I'll cancel a bunch of them.

Nothing to see here, move along. (BTW: who the heck pays money to watch shows on Hulu?)

<comment><USER12> >>who the heck pays money to watch shows on Hulu?

I would. I'm outside the US and don't like waiting for e.g. Big Bang Theory (if that is on Hulu, I can't even get that far.)

<comment><USER10> Isn't part of the point of watching shows online separating yourself from a schedule? Hell, a lot of times I'll wait until a season of a show is over to watch it, so I doni't have to wait for upcoming episodes.

<comment><USER4> Some shows are good enough that you want to watch the episode when they come out. Watching it online still separates you from the schedule though. If the show airs at 8pm, you don't have to watch it at 8pm, you can watch it when you have time. But there's a difference between a couple of hours/days and a couple of months (maybe waiting for a dvd release to buy it, or for a local station to rebroadcast for your country).</comment></comment></comment></comment>


Forking Java, not a good idea at the moment - <AUTHOR> http://weblogs.java.net/blog/fabriziogiudici/archive/2010/10/05/forking-java-not-good-idea-moment

<comment><USER2> He goes on and on about how to do a successful fork, but doesn't explain why this doesn't apply to Java, or why 'the moment' isn't a good time for a fork.

I suspect he is trying to say that there isn't a giant, perfect council ready to lead Java into freedom.

Well, there never will be, unless someone creates it. There's no reason why it couldn't be created right now.

</comment><comment><USER3> Anyone is free to create an independent Java fork... well at least as long as Oracle doesn't sue. For this reason I find Java to be a highly unreliable technology in legal terms.</comment>


Building blocks of a scalable webcrawler. - <AUTHOR> http://blog.marc-seeger.de/2010/12/09/my-thesis-building-blocks-of-a-scalable-webcrawler

<comment><USER2> IIRC, sriramk from around here (<http://news.ycombinator.com/user?id=sriramk>) had also 'rolled his own' web-crawler as a project in college about 5-6 (?) years back. He blogged about it fairly actively back then, and I really enjoyed following his journey (esp. when after months of dev and testing, he finally 'slipped it into the wild'). Tried to dredge up those posts, but he seems to have taken them down :( A shame really - they were quite a fascinating look at the early-stage evolution of a programmer!

Sriram, you around? ;)

<comment><USER3> Thanks but mine was definitely a toy. I think I got it to around 100K pages or so but that's about it (seemed like a big deal back then).

You can see some of those posts here ([http://web.archive.org/web/20041206230457/www.dotnetjunkies....](http://web.archive.org/web/20041206230457/www.dotnetjunkies.com/weblog/sriram/)). Quite embarrassing to see the quality of my output from back then

Basically, I did the following

\- Pull down dmoz.org's datasets (not sure whether I crawled it or whether they had a dump - I think the latter) \- Spin up crawlers (implemented in C# at the time) on various machines, writing to a central repo. The actual design of the crawler was based on Mercator (check out the paper on citeseer) \- Use Lucene to construct TF.IDF indices on top of the repository \- Throw up a nice UI (with the search engine name spelled out in a Google-like font). The funny part is that this probably impressed the people evaluating the project more than anything else.

I did do some cool hacks around showing a better snippet than Google did at the time but I just didn't have the networking bandwidth to do anything serious. Fun for a college project.

The funny thing is a startup which is involved in search contacted me a few weeks ago precisely because of this project. I had to tell that person how much of a toy it was :)

<comment><USER4> Do you remember how fast the "toy" was? (pages/second, domains/s, ...) :)

<comment><USER3> Not really but given the terrible hardware/network connectivity , wouldnt have made much sense now.

Because of this thread, I looked through my old backups and I actually still have the code. Should get it working again sometime

<comment><USER5> are you gonna put up your code ?

It would be interesting to see how to think through building a crawler (as opposed to downloading Nutch and trying to grok it)

</comment></comment></comment></comment></comment><comment><USER4> Uh, look what the cat dragged in: my thesis :)

Hope some of you enjoy the read, I'm open for comments and criticism

<comment><USER6> Very timely and interesting. I am currently looking for a crawler that tightly integrated with Drupal and that can be easily managed through Drupal nodes. Any suggestions on a solution for a small site that only needs to handle thousands of pages/urls?

<comment><USER7> scrapy (<http://scrapy.org/>) is a well-documented and open source python scraping framework that I've used in a couple of projects.

<comment><USER4> Indeed, seems like a great framework.

Considering the timespan of the project, I had to rely on something I'm pretty ok at (Ruby), but I remember hitting a lot of posts about scrapy on the way

</comment></comment></comment></comment><comment><USER8> I like Ted Dziuba solution:

<http://teddziuba.com/2010/10/taco-bell-programming.html>

Full-stack programmer at work!

<comment><USER4> I loved his approach too, but if you want to end up with something that you can search freely for properties it gets a little tiresome with just bash and *nix tools :)

</comment></comment><comment><USER9> A good read and very timely from my perspective. We created a crawler in Python a couple of years ago for RSS feeds, but we ran into a number of issues with it, so put it on hold as we concentrated on work that made money :) We started to look at the project last week and we've been looking at rolling our own versus looking at frameworks like Scrapy. The main thing for us is being able to scale. Anyone who has knowledge of creating a distributed crawler in Python I'd welcome some advice.

Thanks again. Really good post

<comment><USER4> After having written the thesis and thought about that stuff for another few weeks, my résumé would be:

\- Use asynchronous I/O to maximize single-node speed (twisted should be a good choice for python). It might be strange in the beginning, but it usually makes up for it, especially with languages that aren't good at threading (ruby, python, ...).

\- Redis is awesome! Fast, functional, beautiful :)

\- Riak seems to be a great distributed datastore if you really have to scale over multiple nodes.

\- Solr or Sphinx are just better optimized than most datastores when it comes to fulltext-search

\- Take a day to look at graph databases (I'm still not 100% sure if I could have used one for my use cases)

<comment><USER9> Thanks for the tips! I really appreciate it. I'll check these out. All getting very exciting for my Christmas project!

</comment></comment></comment><comment><USER10> I'm having good luck using node.js's httpClient and vertex.js for crawl state / persistence.

<comment><USER4> Oh, node.js is definitely a great direction to go!

One of my problems was that a lot of the "usual" libraries are written in a synchronous/blocking manner behind the scenes. This is something that the node.js ecosystem would probably solve right from the start.

The downside of a relatively new library like httpClient is, that it is missing things like automatically following redirects. While this can be implemented in the crawler code, it complicates things.

How big are the datasets that vertex.js/tokyo cabinet is able to handle for you?

Node.js is on the list of things I'd like to play with a bit more (just like Scala, Erlang, graph databases, mirah, ...). Is your crawler's source code available by any chance?

<comment><USER10> My dataset is still small, but you can scale a single TC db to nearly arbitrary size (8EB). It can also write millions of kv pairs / second.

Vertex.js can't quite keep up with TC as its written in javascript. However, it does let you batch writes into logical transactions, which you can use to get fairly high throughput.

The source isn't open as its fairly specific to my app, <http://luciebot.com/>. I'd be happy to chat about the details without releasing the source. email@example.com / richcollins on freenode.

<comment><USER4> Be sure to check out this post: [http://stackoverflow.com/questions/1051847/why-does-tokyo- ty...](http://stackoverflow.com/questions/1051847/why-does-tokyo-tyrant-slow- down-exponentially-even-after-adjusting-bnum)

I did some experimentation with tokyo* and experienced that slowdown myself. I just didn't want to disable journaling in the end...

<comment><USER10> Thanks -- I've seen that. I'm just going to make frequent backups and hope that lack of journaling doesn't bite me in the ass o_O

</comment></comment></comment></comment></comment><comment><USER11> Can someone please explain what FPGA-aware garbage collection is?

<comment><USER4> Straight from the man himself: <http://buytaert.net/files/fpl05-paper.pdf>

Abstract:

—During codesign of a system, one still runs into the impedance mismatch between the software and hardware worlds. This paper identies the different levels of abstraction of hardware and software as a major culprit of this mismatch. For example, when programming in high-level object-oriented languages like Java, one has disposal of objects, methods, memory management, that facilitates development but these have to be largely . . . abandoned when moving the same functionality into hardware. As a solution, this paper presents a virtual machine, based on the Jikes Research Virtual Machine, that is able to bridge the gap by providing the same capabilities to hardware components as to software components. This seamless integration is achieved by introducing an architecture and protocol that allow recongurable hardware and software to communicate with each other in a transparent manner i.e. no component of the design needs to be aware whether other components are implemented in hardware or in software. Further, in this paper we present a novel technique that allows recongurable hardware to manage dynamically allocated memory. This is achieved by allowing the hardware to hold references to objects and by modifying the garbage collector of the virtual machine to be aware of these references in hardware. We present benchmark results that show, for four different, well- known garbage collectors and for a wide range of applications, that a hardware-aware garbage collector results in a marginal overhead and is therefore a worthwhile addition to the developer's toolbox.</comment></comment>


Startup Investing: The New Trend in Alternative Assets - <AUTHOR> http://blog.onevest.com/blog/2015/4/28/startup-investing-the-new-trend-in-alternative-assets

<comment><USER2> People have talked about alternative assets for a while. Realize that most such investments, at the end of the day, depend on the underlying economy. That's why they haven't fared considerably better than usual index investment.</comment>


Why you need not fear cellphone and wifi radiation - <AUTHOR> http://ap.io/blog/radiofrequency/ <comment><USER2> This seems to imply that the only possible danger would be from ionized atoms damaging DNA. Cellphones and wifi signals are modulated frequencies, therefore they could conceivably come in harmonic resonance with a wide variety of chemical bonds in your body, or have effects on ionized particles within cell walls or organelles. Any HAM radio operator knows that micro and radio waves can cook your body parts.

</comment><comment><USER3> This bullshit needs to stop. There is plenty of proof out there that microwaves interfere with the human body in a bad way on several levels. But I guess the hundreds of thousands of people who aren't yet completely numb to the effects are just making shit up? As did the Russian scientists who got microwave ovens banned there from the start.

Suggesting that people rub Aluminum and other poisonous chemicals into their skin to protect from life giving and healing sun light is just the icing on the cake. The sun heals cancer, sun-screen causes cancer; there's plenty of statistics out there to prove this and plenty of people who are old enough to remember a time before sun screens when cancer was a non-issue.

Stupid, ignorant, techno-babbling sheeple. I honestly don't know why I bother any more, saving this kind of stupidity from itself borders on interfering with evolution.</comment>


Mailytics - <AUTHOR> https://www.mailytics.io/ <comment><AUTHOR> Monitor & Analyze competitor Email campaigns

Hey there,

My name is Ivan and I'm a part of Mailytics.

Our story begins several years ago when we had an idea for a service that would help companies track their competitors email campaigns along with their own. Now this thought has become a reality and I am very happy to see that our project is now live.

We provide our users with a platform for tracking, analyzing and storing competitor email newsletter campaigns. You can easily see industry trends and find inspiration for your own email campaigns or just monitor your competition to assist you when making important decisions.

We pride ourselves on being very user-friendly and we honestly believe that Mailytics will be a valuable tool in your marketing arsenal.

Please let us know what you think, I'd be more than happy to answer any of your questions, comments or concerns. Thanks!

Ivan</comment>


Automatizing rewards for solving hard problems with Bitcoin - <AUTHOR> http://blogchain.fr/programmarket-en/

<comment><USER2> As for "M of N oracles running a public code" mentioned in the article - Orisi.org offers a framework for that.

As for using blockchain directly for contract - there are a few addresses that have funds locked for the first person to find collisions in sha1, sha256 and others: [https://bitcointalk.org/index.php?topic=293382.0](https://bitcointalk.org/index.php?topic=293382.0)

Finally, there's (or "there will be") Ethereum which seems best for preparing contracts that require no external inputs. Solving hard problems usually requires no such inputs.

<comment><AUTHOR> Thank you very much for the references!

As for Ethereum, I know that it is the goal, certainly the fact that it is Turing-complete argues in favor of this usecase. However, how practical and efficient is it, as in what kind of slowdown in the execution would you get on the blockchain?

<comment><USER3> Even if it's orders of magnitude slower, that should be okay, because you're not going to be using it for the hard job of finding a solution, only for the easy job of verifying the solution once found. (I don't know much about Ethereum, so I don't know whether it has any arbitrary limits that would be a problem, but it _should_ be okay.)

<comment><AUTHOR> Well, suppose what you want to check is a proof of the Riemann conjecture in Coq, this might take some substantial time. So whether it is fast could be relevant.</comment></comment></comment></comment>


US rejects global Covid-19 vaccine effort - <AUTHOR> https://arstechnica.com/science/2020/09/trump-admin-shuns-global-covid-19-vaccine-program-citing-beef-with-who/ <comment><USER2> This isn't a rejection of vaccines, just a rejection of using the WHO for them. It isn't clear what that means, but there is a lot of speculation.</comment>


Lawrence Lessig: What's really wrong with Goldman Sachs - <AUTHOR> http://www.cnn.com/2012/03/15/opinion/lessig-goldman-sachs/index.html <comment><USER2> A specific culture is not only the "Secret Sauce" of Goldman Sachs, but of our Western society at large, especially of the Northern European society (and its derivatives, all those formed by the ideas of the Enlightenment and with our Generalized Reciprocity, as R. Putnam [Book: "Bowling Alone"] calls it). Our current economic system, starting in the late senventies, is rapidly destroying that very basis of our societies' Secret Sauce. Sucks.

<comment><USER3> Paul Goodman published "Growing Up Absurd" in 1960. I wonder how much of our secret sauce is based on people trying to escape the absurdity of their environments --- would we have a Silicon Valley if people felt comfortable and happy remaining in their childhood communities?

</comment></comment><comment><USER4> This is a fairly content-less piece. As near as he comes to naming a specific problem or proposing a specific solution is this quote:

> Robert Reich, for example, has long argued that "professional companies > should not be permitted to become publicly held corporations." As he puts > it, "Such a step puts them into high-stakes competition for investors, > pushing them to maximize profits over their responsibilities to the public."

And even that's pretty hand-wavy. What makes a "professional company" and why would such a thing have a "responsibility to the public" above and beyond any other company?

Goldman may be a bogeyman to a lot of folks, but at least they had the sense to get _out_ of the crazy CDO world as fast as they could. If you're going to spend pages explaining how "toxic" the environment is there, some specific, verifiable examples would be nice. Neither Smith nor Lessig provide any. Goldman's clients aren't morons, and if they thought they could do better elsewhere, they were and are free to stop doing business with GS. Until you can really nail Goldman with black and white evidence of fraud, it's hard to take these generalized allegations seriously.

<comment><USER5> Assuming this comment is a response to what the article said, your reasoning is circular. Lessig blames both the change in laws/regulations and Goldman going public for the issues.

We can't nail Goldman with fraud under the existing laws. That's his point: laws and regulations should be reverted/changed to criminalize some of their current behavior.

Professional Companies are companies that provide professional services. These are companies that are hired because they are licensed to do something. They need to be licensed because their field is sufficiently complex or opaque that their clients cannot evaluate their competence or trustworthiness.

He suggests these company should not be able to go public because this class of companies cannot be fairly evaluated by the market without creating perverse incentives.

<comment><USER6> > These are companies that are hired because they are licensed to do > something.

What fraction of GS biz actually involves licensed activities?

Yes, stock brokers are licensed, but bankers aren't.

Most types of financial advisors aren't licensed.

Are any traders licensed?

<comment><USER7> This is not quite correct. Senior bankers (and certainly senior institutional salespeople, of which Greg Smith was one) are typically required by their firms to be licensed by FINRA. Certainly not all investment bankers or traders are required to be licensed (and I don't think they should be), but a significant fraction are.

Also, fwiw, mortgage brokers and lawyers are required to be licensed in almost every state. Licensing--or lack thereof--has little to do with ethical behavior.

<comment><USER8> Licensing almost always includes a legal commitment to specific ethical behaviors, fiduciary duty and the like. That is one of the main differences between licensed professionals and flea market vendors. Doctors, lawyers, and engineers all make specific ethical commitments in order to win authorization to practice.

<comment><USER7> That's true, almost every license for any profession (including everything from accountants to sports agents to, in some states, interior decorators and hair braiders) has an ethics component.

My point is that it's trivial to find examples of unethical behavior by folks with licenses, so clearly the license alone isn't sufficient. Many bankers are already licensed, and licensing won't change the behavior of the ones inclined to take huge risks. Maybe it helps at the margin, but it's not a cure all by any means.

</comment></comment></comment></comment></comment></comment><comment><USER9> This is one of the few things I've read on CNN in a long time I like. Lessig doesn't go into details about regulations, but I would think most people are familiar with the repeal of Glass-Steagal and the de-regulation that began under Reagan in the 80's.

GS has a conflict of interest, because it trades on both ends. It sells products to a client, but is often on the opposing side of those trades. That's why GS is so profitable, at least in the short term. The funny thing is all the people trying to defend GS, as though having undisclosed interest in the deals they do with clients is just fine. Looking back on the wreckage, bank bailouts and other things that have nearly brought the economy to a halt the last few years, I don't really understand how average people who don't have a vested interest in wall street banking can support any of those firms.

<comment><USER10> Actually, no, most people are _NOT_ familiar with the repeal of Glass-Steagall (not Glass-Steagal, incidentally). Including you; if you were you'd realize that there's no coherent argument for how the so-called "repeal" of Glass- Steagall[1] actually led to the financial crisis or to increased profits for GS. Lessig didn't name an actual regulation that might have caused this, and neither have you. There's a reason.

Further, you fundamentally misunderstand how markets work, what GS is doing in these cases, and even what a conflict of interest is. What GS is doing is called "being a broker"[2]. If you don't understand what a broker is, then you _may_ not be well placed to pontificate on financial markets.

[1]: Everyone and their dog likes to trot out the "repeal of Glass-Steagall" and feel clever. If pressed, a few of them will even stammer out something about it being a law that seperated investment and commercial banking. In actuality, the main function of Glass-Steagall was setting up the FDIC, and it's never been repealed. It did contain a lot of other rule changes and regulations, most of which have been repealed decades ago - and good riddance. Do you think it should be _illegal_ to offer interest on a checking account? No? Great, you too are a supporter of the "repeal" of Glass-Steagall. As for the restriction on retail and investment banking...god only knows how that's supposed to have prevented any problems. Not only did it not do what it _claimed_ to do (Citibank merged with Saloman Smith Barney while the rules were nominally still in effect), nobody can explain how the rules intention would have done anything worthwhile. None of the competing theories of "what went wrong" and "how to stop it" have anything to do with seperating commercial and investment banking (and none of the large merged banks failed while several large banks with only commercial or only investment banking operations did fail). So...

[2]: Of course, maybe you want to argue that being a broker is illegal? Or should be illegal? Pull the other one, it's got bells on.

<comment><USER9> Ok, you got me, more concisely, Sections 20 and 32 of the Glass-Steagall act of 1933 were repealed in 1999 after at least two decades of hard lobbying.

"Then, in 1998, in an act of corporate civil disobedience, Citicorp and Travelers Group announced they were merging. Such a combination of banking and insurance companies was illegal under the Bank Holding Company Act, but was excused due to a loophole that provided a two-year review period of proposed mergers" - <http://www.commondreams.org/view/2009/11/12-8>

So, without the repeal, Citibank would have probably been forced to release Travelers, at the least. Volcker and the Fed were opposed to slackening of regulations without new regulations in 1982 when the FDIC ruled in favor of banks being able to take on subsidiaries to underwrite and deal in securities.

The reason it's important to separate the banking activities, is risk. As we have recently seen with MF Global, trusting a company to follow rules about accounts not being used to cover trades are not well followed.

I make no claims of expertise on brokers. GS, however, was both a partner to the trades it made and the broker to clients. Basically betting against the people it was selling securities to. <http://www.sec.gov/news/press/2010/2010-59.htm> Technically I suppose GS wasn't on the other side of the trade, but since Paulson & Co. were paying GS to offer trades without full disclosure, I'd say it's pretty close.

I make no money (currently) from financial firms outside of my 401(k) holdings, I only make comments on random message boards, but I don't think my viewpoint is as foolish as you would have it be.

<comment><USER10> Heh. First, Commondreams is not _necessarily_ the best source for a citation. Let's turn to Wikipedia, which explains the Citibank/Travelers merger is fairly decent detail[1]. In short, while there was a time limit of five years (the two years is only without Fed approval, which in this case they would have received), that only applied to Travelers, _not_ to Citibank owning the investment bank Salomon Smith Barney. But that's really a minor quibble. Let's step back and think about the overall purpose of those restrictions.

The standard "Glass-Steagall repeal caused the crisis!" meme focuses on the idea that we don't want banks wagering FDIC-insured retail deposits on the financial markets and going bust, taking our savings accounts with them. And maybe we don't - but _this did not actually happen_. No retail bank went bust due to their investment banking arms getting overextended. Instead we saw retail banks go bust due to their retail banking operations (specifically, mortgages), and we saw investment banks go bust due to their risky bets on markets. Both of those were always legal under Glass-Steagall.

If the standard "Glass-Steagall repeal is evil" meme has any validity at all, it would seem to be in relation to AIG; an insurer who went bust after making risky bets on the financial markets. Surely Glass-Steagall repeal allowed THAT, right? Nope! The one form of intermingling that actually caused problems during the crisis is the one that _wasn't_ banned by Glass-Steagall. It's no wonder that no serious analysts thinks Gramm-Leach-Bliley had any real impact on the crisis.

So yes, as you say, without repeal Citibank would have been forced - eventually - to sell Travelers. And this would have done...precisely nothing, because as it turns out the purchase of Travelers by Citibank was one of the biggest duds of all time. Nobody actually wants to buy insurance at their bank, and giving access to Citibank (who already had a huge pool of retail deposits) access to the huge pool of premiums Travelers had...did, as near as we can tell, nothing whatsoever. And again, other than AIG (who had no retail banking operations), no major insurance company went under during the crisis, nor did any major bank which went under have an insurance arm. So once again, we ask: Did Glass-Steagall actually prevent anything meaningful?

As for MF Global...yes, they've the villain du jour, and very bad people. But they were not a retail bank, and their operations would have been allowed (or, if you prefer, would have been just as illegal) under Glass-Steagall. Again, what purpose do you think the restrictions in Glass-Steagall served? The only answer is "not letting banks gamble with insured deposits on the financial markets", and MF Global did not do that, and so Glass-Steagall repeal did not impact MF Global. (More generally, what MF Global did is illegal, and so any attempt to argue that MF Global proves we need more regulation is inherently flawed.)

As for the comments about brokers... we're talking past each other. However, since you mention it: There's a lot less than meets the eye to the ABACUS deal. At core, GS's wrongdoing was misrepresenting who picked the CDOs. That's illegal and serious, but they weren't on _either_ end of the trade, much less both. They were more like a sporting goods store selling both AP bullets and body armor to both sides of a gang war. They profited on both ends of the deal, but they couldn't care less which side won. Mind you, GS has often managed to find themselves on both ends of a deal. Check out the deal where GS "helped" El Paso Corp sell itself - suspiciously cheaply - to Kinder Morgan, which GS had a big stake in[2]. Dodgy as _fuck_. And the deal where GS "helped" Burlington Northern sell itself - suspiciously cheaply - to Warren Buffet (a very big investor in GS) wasn't much better... (Mind you, neither had anything to do with Glass-Steagall.)

[1]: [http://en.wikipedia.org/wiki/Glass%E2%80%93Steagall_Act#Fail...](http://en.wikipedia.org/wiki/Glass%E2%80%93Steagall_Act#Failed_1995_Leach_bill.3B_expansion_of_Section_20_affiliate_activities.3B_merger_of_Travelers_and_Citicorp)

[2]: [http://dealbook.nytimes.com/2012/03/05/advising-deal- goldman...](http://dealbook.nytimes.com/2012/03/05/advising-deal-goldman- sachs-had-all-angles-for-a-payday/)

<comment><USER9> I just cited the commondreams article because it was where I read the snippet (linked from Wikipedia) about Travelers and that's what Glass-Steagall would have clearly had an effect on (regardless of whether it mattered in the long run). As I said, the FDIC ruling in 1982 seemed to open the door to commercial banking being able to hold subsidiaries that deal in securities. Perhaps by the time the sections were repealed in 1999, it had little effect, but there were two decades of financial shenanigans leading up to that point.

If the repeals in 1999 had no effect, why did they need to be repealed?

MF Global is just an example of a firm not following rules about trading accounts.

I think the rules in glass-steagall at the least had a chilling effect, which once removed made the banks suddenly see high returns from high risk. The run into CDO's and new derivatives in 2000-2008 timeframe would still have happened, but I highly doubt the list of endangered banks would be quite as high as it is today. I check the list of newly closed banks every Friday at calculatedriskblog.com.

Greenspan's devotion to market efficiencies also made the Fed unwilling or unable to enforce or implement new regulations, or push congress for power to regulate new financial instruments.

I don't do this stuff for a living, so I'm not going to go find all the regulations and ruling over the last 30 years, but I occasionally listen to guests on The Daily Show that seem to know quite a bit about how financial markets work and read here and there, and my conclusion is that de-regulation up to and even beyond Glass-Steagall repeals in 1999 at the very least amplified the recent financial bombs in the US and around the world. I don't think there is a single "Aha!" moment, it has been a growing problem for 30 years. Quite possibly one of the worst items was a regulation...of de- regulation [http://en.wikipedia.org/wiki/Commodity_Futures_Modernization...](http://en.wikipedia.org/wiki/Commodity_Futures_Modernization_Act_of_2000)

Your points on GS seem to line up with the original point about GS being jerks to their clients..."They profited on both ends of the deal, but they couldn't care less which side won". That's the point of the guy that quit, right? GS is screwing their clients, they don't care if the client wins or loses as long as GS wins.

Maybe I'll be less enthusiastic about the rules of Glass-Steagall, but my overall opinion that banking fraud is more rampant after de-regulation and that if I came into a sum of millions the last place I'd trust with my investments is GS, is not changed.

<comment><USER10> My point is fairly narrow: People love to blame Glass-Steagall, and yet consider the roll call of shame during the GFC: Northern Rock, Bear Stearns, Countrywide, Fannie Mae, Freddie Mac, Merryl Lynch, AIG, Lehman Brothers, HBOS, Fannie Mae, HBOS, Lloyds TSB. Every one of those is on the list for two reasons:

1) Being very large ("too big to fail").

2) Having done stupid things which would have been legal under the restrictions of Glass-Steagall. The details of what each did differ, but in every case the stupid actions did not cross a line between retail and investment arms, or retail and insurance arms.

I think that, if you review the list, it's screamingly obvious that _size_ matters hugely, and any regulation reasonably expected to stop the "too big to fail" problem should be looked at very favourably. (Note: No such regulation has been passed, or seriously proposed.) And to a lesser degree, there are some good arguments to be made for bringing regulation of the "shadow banking" sector into line with the rest of the industry, extending deposit insurance to money market accounts[1], and possibly for reducing government involvement in the mortgage industry[2].

What's _not_ obvious is why a regulation that banned something _none_ of the entries on that list were doing would have helped. You say that it had a "chilling effect", but I'm sceptical. If it wasn't for Glass-Steagall, AIG wouldn't have decided to bet the farm on house price stability? Can you articulate any mechanism for how this might have occured?

(If only Citigroup or Wachovia had failed, there'd be an argument that Glass- Steagall repeal helped create "too big to fail" companies by allowing large specialized firms to merge into behomoth diversified firms - but of course, none of those diversified firms failed. If anything, there's a better argument for how Glass-Steagall repeal helped reduce the damage from the crisis.)

[1]: These factors were significantly involved in several of the largest bank failures.

[2]: From the point of view of the taxpayer, the most expensive failures all involved mortgages, most of all Fanny Mae. Abolishing the GSEs and walking back the bi-partisan multi-decade obsession with boosting home ownership rates seems sensible to me. Even today, the idea is highly controversial though.

</comment></comment></comment></comment></comment></comment><comment><USER11> The most recent Econ Talk (<http://www.econtalk.org/>) is an interview with a former Goldman quant who said that Goldman's culture changed "markedly" for the worse when it went public. Before that, Goldman's partners had a huge incentive to make sure "psychopaths" weren't handling the money, and access to capital was quite limited. Capitalization was a constant problem, and they fought for it. After going public, that problem went away, and so did the fear of the psychopaths.

<comment><USER12> permalink: [http://www.econtalk.org/archives/2012/03/derman_on_theor.htm...](http://www.econtalk.org/archives/2012/03/derman_on_theor.html)

</comment></comment><comment><USER13> It is a world of buy and selling securities. There is always someone else on the other end. GS was one of the first to realize that the subprime mortgage market was about to blow up. To cover their asses they began selling everything they owned and at the same time bought CDS's.

So if a German bank that GS does business with still thinks its a good idea to buy some CDO packaged with "AAA" bonds without doing their diligence then GS should have every right to sell them some CDO's, and also short those same securities. They realized there was a problem before everyone else, and they exploited those people to profit from them.

Isn't that how the business world works? Especially with start-ups? You look for inefficiencies and try to capitlaize on them. You use the information that others dont have to create value.

<comment><USER14> _Isn't that how the business world works? Especially with start-ups? You look for inefficiencies and try to capitlaize on them. You use the information that others dont have to create value._

I don't know about other startups, but speaking for myself, I don't consider the hoarding of information a virtue. I use _capabilities_ that others don't have to create value. Taking advantage of information asymmetry is, IMO, exploitative rather than creative.

<comment><USER13> I can agree with that.

</comment></comment></comment><comment><USER15> While I am /in no way/ a fan of GS (or the entire sector), I find the sudden blitz on GS to be quite smelly. First NYTimes publishes a relatively random VP's resignation slam-bye. Matt T. from Rollingstones was also cheerleading. And of course, NYTimes had a front page story followup. And now, this.

What is going on? Have they found their scapegoat for the generally sociopathic (and well hated) sector?

<comment><USER16> What 'blitz'?

There was one bridge-burning editorial by an outgoing VP. Then you have Taibbi doing what he's done day-in, day-out for quite some time now. And after that is just the 24-hour news groups doing what they do, which is latching onto and flogging the stories of the day that are generating interest.

And when you mix Rage-At-Wall-Street with what amounts to a Gossip column, the reader response is pretty predictable.

<comment><USER17> GS has thousands and thousands of VPs, many with as few as one direct report. This particular former VP apparently had zero direct reports. Part of what makes this story smelly is the prominence being given to this one voice; very little of the reporting on the story puts his title (normally a fairly senior one) into perspective.

According to the Dealbook post mentioned downthread, almost 1/3rd of all of Goldman's employees have the same title this person does.

<comment><USER16> And I'm pretty sure that if _any_ of those other VPs were willing to pen a bridge-burning gossip piece, plenty of editors would be _competing_ to give them space.

Similarly, the tilt in coverage can't be a new concept for anyone here. We've known for quite some time that you can more-reliably predict news coverage based on the outlet's target market than based on the facts.

The idea that some news agencies are playing this up and others are playing it down can't possibly be a surprise.

<comment><USER15> Then how come NYTimes is mum on Dr. Pham's spill the beans press release to zerohedge.com of yesterday? (Google: Pham CGO, and see if NYTimes.com shows up ..)

<comment><USER16> It's a good question, but I would suggest it's just that robosigning as a topic never much caught on with the wider public. (tragically)

And if we're still talking about possible evidence of concerted press collusion, it's worth noting that Taibbi has been pounding on the robosigning drum far louder and longer than any 'traditional' news organization. So surely we can't consider his coverage of Goldman to be evidence of a concerted 'blitz' and yet discount his coverage of robosigning in making an argument of coverup.

</comment></comment></comment></comment></comment></comment><comment><USER18> Lessig does not make the case for why Goldman, in particular, was so harmed by the act of going public. The incentives he cites exist for all public companies, and many public companies have existed, and done good work, for decades.

Speaking of which, before we talk about the incentives that led to Goldman's cultural decline, how about some objective confirmation that their culture has in fact declined? The article is in response to a single executive's published opinion.

<comment><USER19> This doesn't directly answer your question, but it's related: <http://news.ycombinator.com/item?id=3711278>

</comment></comment><comment><USER20> _Moral fiber is no doubt important. So, too, is culture. But it is a mistake to understand culture without also understanding incentives. The culture that Smith praises was the product of a firm with a particular legal form, in a particular financial environment. At just about the same time, both particularities changed. And given these changes, it is no wonder that the culture Mr. Smith celebrates soon disappeared._

</comment><comment><USER21> This is wonderful article written by a Harvard Law professor. Normally I don't agree with lawyers, but in this case Lawrence Lessig just tells it like it is without manipulating the facts in a well-respected journal.

</comment><comment><USER22> Sachs pronounced sucks.</comment>


C++ in Coders at Work - <AUTHOR> http://gigamonkeys.com/blog/2009/10/16/coders-c++.html <comment><USER2> Part of how C++ is successful is that it retains C compatibility, and C accurately models how the computer _actually_ works (at least computers of the 70s and 80s, which is all we know how to program well). Lovely languages like lisp, python, haskell may be nicer to work with, but they do not model the underlying machine properly, and for so many problem domains that is just not acceptable. It's not just a performance question.

And then C++ implements several programming paradigms (object oriented, generic, etc) without compromising that machine model.

Plus Bjarne was truly correct when he said: "there are languages people complain about, and there are languages nobody uses."

<comment> loup-vaillant I totally agree. C++ main strength lies in its "portable assembler" nature. However, C++ also is [quite a behemoth][1], while Lisp, Python and Haskell aren't so.

[1]: <http://yosefk.com/c++fqa/>

The answer then is obvious : we should change the hardware, so it is not C/C++ optimized, but Lisp/Python/Haskell optimized. Then, these languages are easier _and_ more practical.

<comment><USER3> In general, hardware optimized for high-level languages have been a failure, usually the extra baggage for handling things as machine-level interpreter slow things down nearly as much as a software interpreter. Nothing has been a real commercial success and it isnt for want of trying. Off the top of my head I can think of the Burroughs B5500 which had hardware support for typed data, the Lisp Machine, and the Intel 432. Sun had a Java machine specification, but I dont think anyone ever bit.

<comment><USER4> IBM has plug-in hardware JVMs for its mainframe big iron. They are ungodly expensive.

<comment><USER5> I can't find anything about that, it's quite interesting do you have a link ?

edit: found it:

<http://news.zdnet.com/2100-9584_22-135319.html>

</comment></comment></comment></comment></comment><comment><USER6> This is a disappointingly flamey article: roughly 90% of the comments are from people who got a bad impression of C++ during the pre-standardization days (which were admittedly horrible, but long since past). Most of the rest are from people who just skipped straight to Java, or who are so young that they never had to learn C++ at all.

The comments that really drop my jaw are the people who seem to hold up Java as a "better" C++. Java is okay, but from its sketchy real-world performance, to its crappy generic programming support and poor ability to enforce such type-safety basics as const-correctness, I've never been impressed. Take the time to learn C++ for what it is, instead of "C with objects" and you'll find that it's far more elegant than the backlash would suggest.

I think that what most people really hate about C++ stems from the widespread use of legacy compilers that incorrectly implement the spec, or implement older versions of the spec. This problem exists for every programming language in the universe ( _e.g._ Ruby 1.8.6 vs. Ruby 1.8.7 vs. Ruby 1.9), but since C++ is widely implemented by _multiple vendors_ , the problem appears worse. If Python or Java were implemented by Microsoft and GNU and four other smaller companies, you'd see the same horrible compatibility problems with those languages, too.

<comment><USER7> The one point that I see throughout the article is that every programming shop uses it's own subset of C++. Based on coding conventions, library usage, and business domain I have seen how the C++ code used in different companies can look like a different programming language altogether.

My professional experience outside C++ is limited, but I had always assumed this was the same for other languages as well. Is C++ really that much of a different animal from other languages?

<comment><USER8> In my experience with Java, the core language is small enough that every company pretty much uses the whole thing. Almost everyone has finally upgraded to using at least the Java 5.0 syntax by now. Occasionally you'll run across silly rules that prohibit things like the ternary operator or multiple returns, but those are minor differences.

The big differences come into play with frameworks. Depending on whether you're using Struts/JSF/Hibernate/Spring/etc your code may look completely different.

<comment><USER9> _Occasionally you'll run across silly rules that prohibit things like the ternary operator or multiple returns_

In avionics software, the ternary operator and multiple returns are often avoided to help ensure better code coverage during verification.

<comment><USER8> I don't think anyone is writing avionics software in Java? The automated code coverage tools I have seen handle those cases just fine.

</comment></comment></comment></comment></comment><comment><USER10> "Yet C++ is also frequently reviled both by those who never use and by those who use it all the time."

I was once listening to RMS speak at a conference. One guy got up and said that this was one of the n great talks he had been to.He mentioned that Stroustrup as another speaker who had impressed him upon which RMS said "You need to get your head checked".

Sometimes a great hacker community can help overcome the shortcomings of a language. Trevor Blackwell makes that point here: <http://www.tlb.org/faq.html>.

"Besides their intrinsic characteristics, languages define commmunities of programmers. You want to choose one that lets you communicate with good programmers, because you'll learn from them. They tend to prefer powerful languages like Python, Lisp, and C++. So for example, although Visual Basic is actually a powerful and complete language, few good programmers use it. C++, on the other hand, is a rather poorly designed language, but for historical reasons a lot of smart people use it so at least you'll be in good company."

</comment><comment><USER5> > Stroustrup campaigned for years and years and years, way beyond any sort of > technical contributions he made to the language, to get it adopted and used. > And he sort of ran all the standards committees with a whip and a chair.

> And he said "no" to no one.

And that is the core of the problem.

<comment><USER11> _And he said "no" to no one._

I don't really see this. Most of C++'s complexity emerges naturally from three things:

1\. Lack of GC.

2\. Direct support for user-defined types on the stack.

3\. The desire to implement every feature as efficiently as the corresponding C idiom.

There actually aren't very many controversial features in C++: multiple inheritance, operator overloading, templates, and exception handling are the ones that come to mind. With the exception of multiple inheritance, none of these is particularly heinous. Implementing them under the three restrictions above is where things get complicated.

<comment><USER5> Operator overloading all by itself is a source of plenty of headaches, in combination with multiple inheritance you can spend a good bit of time trying to find out what something _should_ do before you can begin to figure out what it _actually_ does.

In most languages my preferred view in the debugger is the source code of the language, in C++ I almost always just looked at the assembly. It seemed the more 'clear' language :)

Lack of GC never bothered me much by the way, that's no different in C, and that's where I really learned how to code, so keeping an eye on memory leaks and the opposite, double frees is somehow a built-in feature (of the programmer, not the language).

<comment><USER11> Is operator overloading in C++ really more complex than it should be, or did you just pay the inevitable price for dabbling in the black art of multiple inheritance? Anytime you say "in combination with multiple inheritance" you can't expect to get much sympathy ;-)

C++ is definitely a language where you can get screwed by "clever" programmers who would rather be reading TC++PL than actually coding, but it's not so bad if the people you work with exercise good judgment. I've never had to deal with multiple inheritance in real production code, for instance.

<comment><USER5> There was a time when I made most of my income debugging other peoples programs, call it 'troubleshooter' or something like that.

It gives you an excellent overview of the various ways in which things can go wrong, and C++ figured quite prominently in the 'gotcha' department.

C has it's share of issues, double frees, failure to initialize (but most compilers catch that one nowadays), and stale pointers. With a good discipline you can work around those.

C++ can obscure the bugs in such a way that it takes you a long long time before you can 'nail' them. The biggest problem I have with the language is that the code tends to obscure what is going on at the machine level.

I guess that's a 'feature' too, but I prefer to have a more direct correspondence between program code and what goes on below the surface. That's a problem with OO in general by the way, and I think that this is part of what makes software so terribly inefficient these days.

Gigabytes of ram are barely enough to accomodate a single user os, it's really pretty weird.

<comment><USER12> "The biggest problem I have with the language is that the code tends to obscure what is going on at the machine level."

Which totally defeats the purpose of being backward compatible with C.

<comment><USER5> I think that basing it off C was a quick way to get widespread acceptance though. It succeeded in that respect.

<comment><USER13> Yep. If you had a C program you were automatically already using C++. You could just suddenly start to use the extra features where you wanted them. For that it was great.

If it had been D instead, an actual new version of C designed to incorporate new features, instead of C++, additional features on top of C, it probably would be less reviled but there would have been a lot more resistance to adoption.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> There are parts of all languages that I find confusing, but I'm a little puzzled by the general criticism of C++ (e.g., the __getitem__ feature of Python seems to be similar in spirit to operator overloading. Do people criticize Python too?).

IMHO, C++ is an amazing extension to C which managed to increase productivity for those tasks for which C would have been the right choice (modulo non- standard compilers).

If I were writing a performance-critical system-level program today, using C++ would be no brainer. You can fake a lot of what C++ offers with macros and function pointers in C, but lose type-safety and/or performance. I am very thankful to all the people who have brought C++ to the state it is in today.

</comment><comment><USER15> Its really fascinating to me. We had Lisp and then the road for many lead to C/C++, Java, and related and then we needed a way to express data in more human-readable form so some genius came up with XML. XML was to cumbersome (close tags, how attributes are expressed, etc.) and then JSON came into the picture. So all that to just come back to where we were in the first place.

Now I clearly understand what some of the veterans in "Coders at Work" really meant by "Very little progress, if any, has been made in the Programming space".

<comment><USER16> JSON is not a programming language.

<comment><USER15> You are absolutely right. My point exactly.

Now you need "two hammers" to get a single job done. Your current programming language of choice and a data expression language.

For example: In C, C++, Java, or related, you "first" have to build you structure to represent a Person with first_name and last_name and then you have to write it to JSON:

Java:

      class Person { String firstName; String lastName; }

JSON with JavaScript evaluation:

      var names = [{"firstName": "John", "lastName": "Smith"}, {"firstName": "Bob", "lastName": "Jones"}]

      eval("(" + names + ")");

Lisp handles both naturally. You don't need intermediary human-readable data expression language. It comes natural to the language itself.

Lisp ver:

      (defvar names '((:firstName "John" :lastName "Smith") (:firstName "Bob" :lastName "Jones")))

      (eval names)

<comment><USER16> > or example: In C, C++, Java, or related, you "first" have to build you > structure to represent a Person with first_name and last_name

Not if you're using Protocol Buffers, which is essentially a version of JSON that has "batteries included."

> Lisp ver:

Lisp is not the only programming language around. How am I going to consume that data from another language? Embed an entire Lisp interpreter?

</comment></comment></comment></comment><comment><USER17> C++ is pretty awful to develop in, especially compared to many other languages. Some of these issues have to do with poorly designed features in the core language (templates), static variable instantiation, no good way for declaring complex data structures in the language (a'la JSON), multiple inheritance, etc.

But this isn't just a problem with the core language itself, but with the compilation model and linkage model - and the compilers and linkers. You need include guards on the top of your header files to make sure the code doesn't get re-included. You have to aggressively forward declare to work around declaration dependencies. And you have to trim the files that you include to keep compile times under control, because here's the fun thing about C++: For every .cpp file in your project, you will end up reading in and parsing the same damn header files again and again. In a large project, that might mean recompiling the same header files 1000s of times. Precompiled headers aren't a good solution, either - because when you have to touch the header, you pay a huge cost in recompiling the PCH.

In every other language I've used, I haven't had to worry about carefully restructuring code (sometimes at a cost in performance) to reduce compile times - because the core include and dependency mechanism sucks so badly.

Finally, C++ as a language is very poorly extensible. By this, I mean in compared to languages like LISP (with macros), or Ruby / Scala, which both make writing DSLs very easy. This is especially unfortunate, given that is so heavily used in a lot of very specific domains. And don't get me started on templates - I couldn't dream up a more backwards and awkward way of doing generics if I tried.

</comment><comment><USER18> I noticed this as well when I read the book: [http://www.cloudknow.com/2009/08/review-coders-at-work-by- pe...](http://www.cloudknow.com/2009/08/review-coders-at-work-by-peter- seibel/)

</comment><comment><USER19> I think it's a very good article detailing the various flaws of C++; I particularly have problems with the complexity of the languages making it very difficult for compiler writers to generate usable error messages (not only true for gcc as mentioned in the article, but Microsofts C++ compiler generates many unusable error messages, too! [At least it did in 2004, never touched it again...])

</comment><comment><USER20> the point about everyone making their own sub-language is exactly right. In fact, I've heard programmers say that's exactly what they love about it.

Problem is, every shop is inventing their own subculture. When programmers who haven't worked together look at each other's C++, they're basically learning a new language.

I do think this has changed a bit in the last few years. The whole 'design pattern' meme exists primarily to create nexuses of style that programmers can gravitate to. It just seems like such a slow way home, especially in comparison to a language like Python, which goes out of its way to restrict the number of ways you can skin the cat.

Has any serious programmer ever looked at a Python file and said "wtf?" for more than a minute or two? And, has any experienced programmer _not_ had the experience of looking at a new batch of C++ (say at a new job) and thought to him/herself, jeez, this is going to be a lot of work just to understand the basics of what they're doing?

</comment><comment><USER21> I'm liking these Seibel digest posts more than I'm liking the actual book, which, even in the Norvig chapter, drags a bit.

I think the book might have worked better if it was organized topically, instead of by person.

</comment><comment><USER22> Of course it's going to be more convoluted then newer languages like Java: Java doesn't have to maintain backwards compatibility with a 35+ year old programming language. That's just stating the obvious.

Yet for some reason people still use it. The question is 'why'. For many cases (not all though), I think the reason is largely just institutional inertia - you stick with the devil you know rather than the devil you don't know.

</comment><comment><USER23> Have there been experiments with creating a pre-compiler that checked that you were using a subset of C++? What downsides would this have (other than longer compile time)?

<comment><USER5> But which subset ?

When I program in C++ (and sometimes you have to) there are features that I'll avoid like the plague, once bitten, twice shy.

In fact, my subset of C++ was usually try to stay as close as you can to C and use C++ when you have to. That seemed to be a pretty safe route.

Most of my C++ stuff was using Borland C++ Builder or Microsoft visual C++, I'm happy to say I no longer have to support software for the windows platform, so no more C++ for me.

<comment><USER24> I try to stay as close to C as possible. But C++ has many temptations. If you start using boost, for example, you will be in no time using features of C++ that are not even in the standard yet :-))

</comment></comment></comment><comment><USER25> It's the only language I really feel comfortable programming in, and everything else seems 1) slow or 2) alien in comparison.

I'm digging scala's design, but have yet to really compare it for number crunching, and big library building.

By the way, the article didn't really dig into any specifics besides saying there was feature bloat in C++. Which languages don't have feature bloat? How much use can I get out of them?

</comment><comment><USER26> How could people complain on c++ when so many daily useful tools were written in it? Even their beloved JVM is a mere C++ program. Consider very successful and really cross-platform Webkit, V8, Mozilla's Geeko and so on. And there are thousands successful proprietary projects like Informix and even this popular MySQL.

If Google, Mozilla and many others can productively use C++ why other people can't? They even published guidelines how to use stable and portable subsets of language.

</comment><comment><USER27> Isn't C++ Compiler written in C?

</comment><comment><USER28> C++ sucks, why do all those idiots at Google use it? /sarcasm

</comment><comment><USER29> Argh. Not a very useful article. Start with an observation: "Everyone hates C++, and yet it is really widely used. That's odd." Next, 10 paragraphs of people hating on C++. End with: actually wait, no ending.

<comment><USER5> Well, the 'everyone' he is talking about are some of the most respected people in the computer world, that should count for something.

Also, they make fairly specific criticisms, and they have a track record of being right about such things.

Java seems to be the C++ replacement of the future, with C# pulling the other way.

<comment><USER30> _Well, the 'everyone' he is talking about are some of the most respected people in the computer world, that should count for something._

And while they're busy bitching and hating and pontificating, humble unknowns are getting things done and making the world a better place. C++ is to systems programming what PHP is to web programming: the loyal packhorse that the A-list wouldn't be seen dead on.

<comment><USER5> That is very true.

PHP, Python and perl are rarely seen at corporations that existed before the web came around. Their whole IT department is set up around a different kind of environment.

I think this is in part why the web is so disruptive, because it enables all these upstarts using very light and nimble stuff to challenge the big and established companies directly.

You can pretty much tell what is enterprise stuff and what is 'quick & dirty but does the job' by comparing hourly rates for programmers.

It also has everything to do with the length of time before someone can be productive in a new environment. If that length of time is very short then there is not a very high barrier, which tends to create a lot of 'wannabe' programmers in that language.

By raising the bar you only end up with people that are willing to invest a large amount of time in a platform, that tends to favour people that get paid for their work, which in turn is found mostly in enterprise locations.

<comment><USER31> _PHP, Python and perl are rarely seen at corporations that existed before the web came around. Their whole IT department is set up around a different kind of environment._

Not sure if this is true in general. I work at a bank. We use a lot of Perl. Banks existed long before "the web", and our Perl stuff is not a web application either. All the internal web apps are Java.

It is kind of weird, actually -- Java for stuff like the HR apps, and Perl for the stuff that makes us money. (If it were my decision, I would use Perl for the web apps and Haskell for the algorithmic stuff... but it's not.)

<comment><USER5> > We use a lot of Perl.

That's interesting! That's the last thing I would expect from a bank actually, wonder how common that is.

But then again, is there even an 'enterprise scripting language' ?

> All the internal web apps are Java.

That's what I would expect. Sun really did some good marketing in that sphere.

<comment><USER24> The area of a bank that generate money employs the smartest people, usually coming from academia. So, it doesn't surprise me.

<comment><USER5> Good point.

Yours and gcheong's comment just above makes me wonder if someone missed the boat in getting an enterprise level scripting language out the door. It looks like there never was a 'natural' successor to stuff like JCL.

<comment><USER24> Sun and MS are trying to bring scripting languages to their managed environments. .NET already supports Python reasonably well (I was surprised to see that SharpDevelop <http://www.icsharpcode.net/OpenSource/SD/> can easily convert most of my C++ into Python).</comment></comment></comment></comment></comment></comment></comment></comment></comment>


Google removes FriendFeed from Feedburner stats - <AUTHOR> http://thenextweb.com/2009/09/08/snap-magic-finger-google-removes-friendfeed-feedburner-stats/

<comment><USER2> Or perhaps FriendFeed removed stats from their user agent?

<comment><AUTHOR> Interesting possibility. How could we check that?

<comment><USER2> If you have a site that is being read by FriendFeed just look at the latest request. It should be in the user agent string (stored in your server logs). I've never been a FriendFeed user and can't see how to add an RSS feed or I'd do it myself.

</comment></comment></comment><comment><USER3> I know some people are on the fence about whether friendfeed subscribers should be counted in feedburner stats, and as a blogger my stance is that I believe they should be included. Many people are using services like FriendFeed as an alternative to whatever aggregator.. so why shouldn't their preferred method of reading feeds be counted?

That being said I went from ~10,400 to ~8,600 subscribers today.

</comment><comment><USER4> Could someone please explain what's goin' on here? I don't know the difference between feedburner and friendfeed. I don't use either.

Does this mean that some people aren't able to see the content they want? What exactly did Google pull the plug on?

Or is this simply the case where Feedburner isn't telling people that people using Friendfeed are reading their stuff?

<comment><USER5> Google/Feedburner (an RSS filtering service) would track FriendFeed (a Twitter-like service that lets you subscribe ot others' content) subscribers as a part of your total subscriber count. Now it appears they don't do this any more. This guy had his displayed subscriber count drop by over half as a result.

<comment><USER3> "This guy had his displayed subscriber count drop by over half as a result."

I think there is other stuff going on as well. he only has ~3400 friendfeed subscribers.. that doesn't explain going from 30,000 subscribers as reported by FeedBurner, to 11,000 overnight.</comment></comment></comment>


Telstra launches cloud service in Australia - <AUTHOR> https://cloud.telstra.com/

<comment><AUTHOR> Also see article at [http://www.theregister.co.uk/2011/11/08/telstra_opens_sme_cl...](http://www.theregister.co.uk/2011/11/08/telstra_opens_sme_cloud/)</comment>


Faster Smarter JavaScript Debugging in Firefox DevTools - <AUTHOR> https://hacks.mozilla.org/2019/05/faster-smarter-javascript-debugging-in-firefox/ <comment><USER2> I realize it's an insanely complicated feature, but is there any progress at all being made on live script modifications?

For years, it's been the feature I miss most from Chrome -- the ability to throw a breakpoint, edit the code that I'm looking at in the debugger, and then resume running with the new code. I'm never going to be able to switch 100% to Firefox dev tools as long as Chrome is the only browser with that feature.

Am I the only person who relies on this? I rarely see it brought up anymore, so it makes me wonder if with jsx and precompilers, people just don't use live editing as much as they used to. There must be some reason why more people aren't asking for it.

I know that at one point a couple years ago Mozilla was working on this, but since then I've heard nobody talking about it or giving progress updates of any kind.

</comment><comment><USER3> Cool to see logpoints. In Chrome dev tools you can write console.log messages inside of a conditional breakpoint to get quick logging. You can also do any other scripting in there. I've sometimes used conditional breakpoints to assign values to variables to probe past null references and test things out...</comment>


Why we aren't tempted to use ACLs on our Unix machines - <AUTHOR> https://utcc.utoronto.ca/~cks/space/blog/sysadmin/NoACLTemptation <comment><USER2> I am tempted and have started to use ACLs. Once you've got a ZFS filesystem (in my case, on FreeBSD like the author), you have full NFSv4 ACL support. If you export that over NFSv4, those ACLs are visible to all client systems. And if you export it over Samba, they are visible to Windows clients as well.

I don't think the support here is too awful: given that it's what NFSv4 supports, it should be available to all clients. It doesn't help that the tools are crap on the Linux side--the POSIX.1e DRAFT ACLs are all they support last time I checked, and this dead non-standard should really just be replaced by the NFSv4 model at this point. Not sure what ZFSOnLinux does to work around this shortcoming.

<comment><AUTHOR> I believe the author's current setup is running ZFS on Illumos (specifically OmniOS), with volumes exported over NFSv3 and Samba. It's not mentioned in this post, but some of the older ones talk about it [1]. The comment about FreeBSD was worrying that if they started relying on ACLs now, and later switched their fileservers to NFS-on-ZFS-on-FreeBSD instead, they might end up with incompatibilities in the ACL models.

[1] Here's one on the network filesystem setup: [https://utcc.utoronto.ca/~cks/space/blog/unix/NFSFutureDesir...](https://utcc.utoronto.ca/~cks/space/blog/unix/NFSFutureDesire)

</comment></comment><comment><USER3> Ha, I've literally just finished debugging an SSH connection failure which turned-out to be caused by ACLs!

An inappropriately-recursive setfacl added additional read / exec rights to the key files in a user's .ssh directory, which caused SSH to bail-out without establishing a connection. Subtle to find, it was only the "+" suffixed on the ls -al output that gave the clue...

ACLs are very powerful for solving specific problems but as the blog-post points-out they can have all sorts of unexpected side-effects.

<comment><USER4> The reason their side-effects are unexpected is not that ACLs are intrinsically weird, it's that very few people have familiarity with day-to- day use of ACLs on Unix. The reason that very few people have familiarity with day-to-day use of ACLs on Unix is that very few people use them. The reason that very people use them is because very few people are familiar with them. It's a bit of a Catch-22.

For other ways to completely baffle your users, have a play with file attributes:

[http://linux.die.net/man/1/chattr](http://linux.die.net/man/1/chattr)

<comment><USER5> ACLs are, for the most part, unneeded.

The old (other, group, user) * (read, write, execute) unix model is much easier to use, and sufficient for the vast majority of uses.

Occasionally, there is a need for something that can be expressed by ACLs, but not by Unix permissions. In most deployments, it is so rare that people who are not aware with ACLs just find some other way to do it.

ACLs are not the end-all-be-all permission system either. I would really like a permission system that says "in this directory, everyone can create a file, but no one can create an .mp4 or .mp3 or executable file". ACLs can't do that.

Some 20 years ago, when I occasionally had to audit file system permissions, Unix ones were usually properly set up (every user has write access to their own stuff, read access to shared stuff, and no access to other's users stuff, with groups used to do exceptions), and NT systems were always a mess, with lots of unexpected permissions and wrongly set up ACLs because complexity.

The best setups were always Novell Netware with their "Trustee" security model, which makes it much easier to get what you want than either ACLs or Unix perms

<comment><USER6> "The old (other, group, user) * (read, write, execute) unix model is [...] sufficient for the vast majority of uses."

It's really not, in an even moderately-complex environment. What I've seen most often is that the "user" and "group" permissions get eaten up by other factors (e.g. the "user" is a service account), so the only permission that can meaningfully be altered is "other".

The result is that either all of the people who support whatever the app is doing have to be given sudo permissions as the service account (drastically reducing the value of any audit/logging), or access has to be controlled at the OS level in terms of who can log onto the system in the first place. In other words, band-aids to work around the limitations of the traditional Unix filesystem permission model.

<comment><USER5> Could you describe what the "group" gets eaten by? In most cases I've seen, the group works well as a permission set - e.g. "dialout" is for all users allowed to use the dialer, "lp" for all those allowed to use the printer, "audio" for those allowed to use the speaker, etc.

Indeed, some pieces of software insist that the service user actually do everything, which is still supported with this model if the binaries are setuid with +x on owner and group but not other -- although, as you pointed out, auditing tends to suffer in this case.

Speaking of which -- what's your favorite auditing setup? I need to set one up soon, I've been using auditd but it feels cumbersome.

<comment><USER6> Filter this response based on my being a pen tester, not a Unix/Linux admin - I'm much better at finding vulnerabilities and shortcomings than I am at coming up with a really good solution to them :). That's also why I can't give you a worthwhile answer about an auditing setup - I could tell you which ones I'm not super fond of, but I've never run the auditing side of things on Unix/Linux systems.

The challenge I've run into with the group permission is that because there's only the owning group for the file/directory, there are basically two options:

1 - Leave the content owned by the primary group of the service account which also owns the file, and make support staff members of that group. This has the side effect of giving those support personnel access to just about everything else that group can get to, even if they don't need it.

2 - Create a new group that's just for delegating permissions for whatever content needs to be secured, and add both the service account and the users. This is analogous to using the new group as a quasi-ACL, IMO. It works, but quickly leads to group sprawl, and most of the Unix/Linux admins I've worked with are loathe to do it for this reason.

A secondary challenge with #2 is that larger organizations will usually have group membership controlled by some sort of administrative team. So getting the group membership changed is a much bigger headache than the filesystem object owner modifying the ACL on their own content.

</comment></comment></comment></comment></comment></comment><comment><USER7> ACL = Access Control Lists

NFS = Network File System

since sysadmins never bother defining anything in their articles.

<comment><USER8> why on earth would you expect a sysadmin to break those out? both acronyms have a pretty high usage saturation level. if you don't know these acronyms, a sysadmin isn't doing you a favor by telling the reader in a blog post about security features.

<comment><USER9> In several style guides I have seen, both in academia and journalism you always Introduce An Acronynm (IAA) before using it.

You don't get to pick your audience on the internet and IAA strikes me as a sensible thing to do.

<comment><USER10> Particularly since it's not dead print, you can have hovertext or a link to the expanded text without it taking up more space.

</comment></comment></comment></comment><comment><USER11> maybe i am using older tools here, but i found that getting a ls -l like readout of all the acls of a directory was a right pain.

<comment><USER12> No, that's a pretty legit concern. There is basically no way to take a list of files with varying ACLs and display their permissions in a manner as condensed as ls -l. There is potentially a lot more information to put up onto the screen.

<comment><USER11> Abused the syntax some more, and found what i wanted if i used:

getfacl --tabular ./*

For some reason simple * would not work.</comment></comment></comment>