How to be an open source gardener (2014) - grey-area http://words.steveklabnik.com/how-to-be-an-open-source-gardener

<comment><AUTHOR> Money quote:

" _See, people think working on open source is glamorous, but it's actually not. Working on open source is reading 800 issues over the course of a weekend._ "

Sign that the author does not suffer from the "Cascade of Attention-Deficit Teenagers" software development model ([http://www.jwz.org/doc/cadt.html](http://www.jwz.org/doc/cadt.html)) ^_^; after properly closing out feature and help requests issues:

" _3\. Was this issue for an older version of Rails than is currently supported? If so, copy /paste an answer I wrote that asks if anyone knows if this affects a supported version of Rails._"

Further signs follow, like following up!

Lots of good stuff here, like " _The only thing better than reproduction instructions are when those instructions say git clone._ "

</comment><comment><USER2> I have contributed to a number of open source projects over the years, and the number one stumbling block that I have noticed, time and again, is simply this: lack of leadership.

It seems to be quite an accepted practice to just push your code out there, wrap it in some nice license, and forget about the project - this can be the difference between whether the project goes forward or sits and stagnates - and stagnate it will, even if you've got an avid following of users. Contributors to the project may well come up with great patches and new features and so on, but it will go nowhere if someone is not sitting at the top of the hierarchy, accepting the patches, making a mainstream branch, and releasing often new versions of the software.

I'm going through this right now with an open source project that is very, very powerful (I won't name it, but you can guess if you look through my comment history) and very, very useful to a lot of people. The project is technically awesome - but the leadership, administratively, leaves a lot to be desired and for those of us struggling to contribute to the project, its extremely frustrating not to have someone holding the gavel and pushing things out there on a regular release schedule.

So its not enough to be a gardener. You also have to have a farmer, a leader type, who will make sure your produce gets to market in a timely manner. If you're going to garden, make sure there's a foreman/farmer type who will take the best of your work and put it on the truck to get to market. If there isn't such a position, overt and available, then the very first thing you should do on the open source project, if you want it to persist and do well, is insist that there be such a role holding the reigns. It really is very important to have leadership in the open source realm.

<comment><USER3> Yup. And part of leadership is handing over the reins cleanly once you're no longer engaged in a project. By "cleanly" I mean removing your own admin access, no longer merging patches once you're not really checking them thoroughly anymore, not commenting on tickets based on half-hearted scanning, etc.

I mishandled that one time with a pretty popular open source gizmo ([https://sf.net/projects/pmd](https://sf.net/projects/pmd)) and caused some unnecessary static. Live and learn.

</comment></comment><comment><USER4> [https://news.ycombinator.com/item?id=7593242](https://news.ycombinator.com/item?id=7593242)

<comment> grey-area I thought this might have been posted a while ago, but also thought it was interesting enough to post again after a year. Are there any rules on dupes?

<comment><USER4> Why yes there are! [https://news.ycombinator.com/newsfaq.html](https://news.ycombinator.com/newsfaq.html).

<comment> grey-area _in the last year or so_

OK, it's a fair cop ;) Will try to check manually for dupes in future.

<comment><USER5> It actually won't let you resubmit if there has been a recent dupe, as far as I know.</comment></comment></comment></comment></comment>


Funny loading statements to keep users amused - <AUTHOR> http://stackoverflow.com/questions/182112/funny-loading-statements-to-keep-users-amused

<comment><USER2> That's creative.</comment>


Show HN: Save postal addresses from your browser to your phone - <AUTHOR> http://www.placesteal.com/

<comment><USER2> This is such a simple yet useful app. I signed up for the private beta a few days ago and used it just this morning when I was heading out to meet someone.

</comment><comment><AUTHOR> We've worked our asses off on this. Feedback, please!

</comment><comment><USER3> Great app! Loved it!</comment>


Python API - <AUTHOR> http://www.pythonapi.com Provides API documentation and Python wrappers to different web services. It's based on Flask / Nginx and Gunicorn. <comment><USER2> This definitely could use a form to submit API wrappers. I used the contact form linked at the top to submit Stack.PY, a Python module for the Stack Exchange 2.1 API, which is available on Launchpad ([https://launchpad.net/stackpy](https://launchpad.net/stackpy)) and PyPI ([https://pypi.python.org/pypi/stackpy](https://pypi.python.org/pypi/stackpy)). The SE API documentation is at [https://api.stackexchange.com/docs](https://api.stackexchange.com/docs)

<comment><USER3> I think all the "You are missing X" comments suggest that the OP is on to something. Great idea!

<comment><USER2> Yeah, those are definitely the kind of "complaints" that I like to see about my own apps. :)

</comment></comment></comment><comment><USER4> The Urban Dictionary API wrapper[1] isn't in Python:

      installation

      $ npm install -g urban # bin
      $ npm install urban # lib

[1]: [https://github.com/mvrilo/urban](https://github.com/mvrilo/urban)

Edit: Though looking at the site it again it only lists it as "command-line tool and API" instead of "Python wrapper" like all of the rest. Seems odd on a page dedicated to Python API wrappers.

<comment><USER5> Updated the URL. If you can find a better wrapper, please let me know :)

</comment></comment><comment><USER6> It's interesting to see how differently these are all implemented. I guess it depends on what's at the other side of the api and how much work the developer wants to put in to data conversion.

When you flick through the first few examples you see that the result from querying a lib could be a nested dict, a bunch of objects or an lxml.etree. Every time you approach a new api, even with the help of a library so you don't have to deal with the transport, you still need to learn a unique set of data structures.

<comment><USER7> At Cosmic([http://www.cosmic-api.com/](http://www.cosmic-api.com/)), we are working toward making web APIs more consistent and easier to build/consume with higher level framework. We take care of transport work, as well as data structure work. The data structure work([https://github.com/cosmic- api/teleport.py](https://github.com/cosmic-api/teleport.py)) is similar to Thrift, Protobuf, Avro, etc but with more emphasis on 'information on web' rather than dealing with raw bytes.

Cosmic is trying to achieve similar goal as Docker, as we are trying to build a framework/tools/services which solves APIs X languages matrix(Slide 14 of [http://www.slideshare.net/dotCloud/why- docker](http://www.slideshare.net/dotCloud/why-docker)).

FYI, the project is still in very early design stage, any feedback is very appreciated.

</comment></comment><comment><USER8> Yeah, it'd be nice to be able to submit an API.

Like for Trello: [https://bitbucket.org/btubbs/trollop](https://bitbucket.org/btubbs/trollop)

<comment><AUTHOR> Thanks for the URL. I will look into that.

</comment></comment><comment><USER9> I always wanted a matrix of code in each language for accessing an API, the way that Rosetta Code shows examples for performing a calculation. I was shocked that there's no REST api category on rosettacode.org (although in fairness there's no http either):

[http://rosettacode.org/wiki/Rest](http://rosettacode.org/wiki/Rest)

[http://rosettacode.org/wiki/Rest_api](http://rosettacode.org/wiki/Rest_api)

[http://www.google.com/search?q=REST%20api%20site:rosettacode...](http://www.google.com/search?q=REST%20api%20site:rosettacode.org)

I would very much like a language-agnostic way to tie all of these APIs together!

</comment><comment><USER10> This is really neat!

You are missing the UPS API! (disclosure: I wrote this python library for it :)

[https://github.com/classicspecs/ClassicUPS](https://github.com/classicspecs/ClassicUPS)

<comment><USER5> Added!

</comment></comment><comment><USER11> Seems like a shame that there is only one option listed for the various services. There are several good libraries for twitter, and at least ptt ([http://mike.verdone.ca/twitter/](http://mike.verdone.ca/twitter/)) and tweepy ([https://github.com/tweepy/tweepy](https://github.com/tweepy/tweepy)) are better, imo, then the python-twitter wrapper linked here.

</comment><comment><USER12> It could use an entry for Echonest, which has all sorts of great APIs for working with music audio and metadata

[1] [http://developer.echonest.com/](http://developer.echonest.com/)

[2] [http://echonest.github.io/pyechonest/](http://echonest.github.io/pyechonest/)

<comment><USER13> great suggestion, I was looking for something like this

</comment></comment><comment><USER14> How can we add to the list?

<comment><AUTHOR> I will work on that! For now, just use the contact form or Twitter ([https://twitter.com/pyapis](https://twitter.com/pyapis))

<comment><USER15> Thanks but no thanks. I suggest that you come up with a github repo and ask people for pull requests.

<comment><USER16> agreed. setup a repo.

also, it would be nice if you would allow more than one wrapper per api.

</comment></comment></comment></comment><comment> old-gregg Nice! Rackspace Cloud SDK is missing though: [http://docs.rackspace.com/sdks/guide/content/python.html](http://docs.rackspace.com/sdks/guide/content/python.html)

</comment><comment><USER17> Google APIs client library [1]

Supports [2] various Google services (AdSense, Analytics, Calendar, YouTube etc etc)

[1]: [https://developers.google.com/api-client- library/python/](https://developers.google.com/api-client-library/python/)

[2]: [https://developers.google.com/api-client- library/python/refe...](https://developers.google.com/api-client- library/python/reference/supported_apis)

</comment><comment><USER18> "Python wrapper for Netflix" links to [https://github.com/ozgur/python- linkedin](https://github.com/ozgur/python-linkedin)

<comment><USER19> It appears to have been fixed. It now points to [https://github.com/jcopenha/pyflix](https://github.com/jcopenha/pyflix)

</comment></comment><comment><USER20> Super nifty, bookmarked.

Although it's missing API's for Hacker News[0] and Fuck Off As A Service[1].

[0] [https://github.com/dmpayton/python- ihackernews](https://github.com/dmpayton/python-ihackernews)

[1] [https://github.com/dmpayton/foaas- python](https://github.com/dmpayton/foaas-python)

</comment><comment><USER21> Nice idea.

Missing Evernote API: [http://dev.evernote.com/doc/](http://dev.evernote.com/doc/)

<comment><AUTHOR> Thanks. I will add the Evernote API later tonight

<comment><USER22> Shameless plug: Kindly add my startup's API, the Semantics3 Product API* ([https://semantics3.com/](https://semantics3.com/)) too :)

Documentation: [https://semantics3.com/docs](https://semantics3.com/docs) Our Python wrapper library: [https://github.com/Semantics3/semantics3-python](https://github.com/Semantics3/semantics3-python)

*We are a replacement for the soon-to-be-shutdown Google Shopping API

</comment></comment></comment><comment><USER23> For the Tumblr API, the link to the client is for our v1 API which has been deprecated in favor of our v2 API.

We have an official python client that I work on when I get time here: [https://github.com/tumblr/pytumblr](https://github.com/tumblr/pytumblr), Though I know there are tons of other awesome wrappers for v2 out there.

</comment><comment><USER24> Nice job. Another addition: Stripe Python API [https://github.com/stripe/stripe-python](https://github.com/stripe/stripe- python) . The documentation is on [https://stripe.com/docs/api/python](https://stripe.com/docs/api/python).

<comment><USER5> Stripe API just added.

</comment></comment><comment><USER25> A Lot of these libraries, are old and aren't maintained. Nice idea horrible curation.

</comment><comment><USER26> It really bothers me that API isn't capitalized in "Api Documentation" :)

<comment><AUTHOR> LOL. That's a good point. I will fix that tonight :)

</comment></comment><comment><USER27> Disqus Python, [https://github.com/disqus/disqus- python](https://github.com/disqus/disqus-python)

Nice comprehensive list, Thanks!

</comment><comment><USER5> Just updated the site with more API's. Check it out.

</comment><comment><USER28> also windows azure python apis: [https://github.com/WindowsAzure/azure-sdk- for-python](https://github.com/WindowsAzure/azure-sdk-for-python)

</comment><comment><USER29> Anyone know of a list like this for Node.js, Ruby?

</comment><comment><USER30> Very nice!! That ESPN one has some real potential.

Alas, you are missing Titshare

</comment><comment><USER31> Amazing! Thanks!

<comment><USER5> Thanks!

</comment></comment><comment><USER32> very handy! Thanks a lot

<comment><AUTHOR> Cheers.

</comment></comment><comment><USER33> interesting and handy stuffs! +1!

<comment><USER5> Cheers</comment></comment>


Improving Cithhash Performance by Go Profiling - <AUTHOR> http://zhen.org/blog/improving-cithhash-performance-by-go-profiling/

<comment><USER2> Good article, but the typo of Cithhash for Cityhash will reduce interest. Perhaps correct and resubmit?

<comment><AUTHOR> Thanks for the suggestion. I resubmitted here: [https://news.ycombinator.com/item?id=6710115](https://news.ycombinator.com/item?id=6710115)</comment></comment>


Ask HN: What if karma was worth real money? - <AUTHOR>

Would that do anything useful? I thought it might scale the influence of opinions to the esteem of the community member, since people with lots of karma would be able to upvote more. Also, I like the idea of giving someone more than just an intangible gesture of gratitude. <comment><USER2> As a blogger, I can tell you it already is. When one of my links gets karma, it gets attention, and attention can be transmuted into revenue through the magic of advertisements.

But for me, blogging has nowt to do with making points nor money—I've only made a total of like $50 this year from the many hours I've spent writing. If I cared about money, I wouldn't be wasting my time on my blog and vlog. What I do care about is having fun, making my ideas heard, and talking with new and interesting people.

Upvotes tell me how relevant my ideas are to the real world and other people's interests, which in turn helps me become a better writer and a better thinker.

</comment><comment><USER3> Sounds great, feel free to send your spare cash any time :-)

</comment><comment><USER4> If karma was real money I doubt anyone would ever post anything controversial in that you could lose your shirt.

<comment><USER5> How? I dont see how karma being money could work with downvotes. It only seems to work with up votes.

The idea being that you give karma to another person by taking it from your own stash. Up votes would cost you money. But how do downvotes work? Do you take another person's points away? That doesn't work because humans are greedy.

</comment></comment><comment><USER6> evincarofautumn mentioned this from one angle, but I too think that high karma levels on HN are already valuable. Just look at the false acclaim that gets heaped on every time patio11 posts something. It's not that he never posts anything worthwhile, but you can definitely see some wild responses that are directly attributable to Patrick's place high on the HN total karma list (like that thing on lionhearted's blog a couple of years ago, on how patio11 should charge people A LOT of money because he's like the smartest guy ever).

I suspect that many offers for consultation come in to HN's top commenters and many of the things they say get noticed above the sayings of the common pleb. If you have a high amount of karma, I think it's definitely worth some money.

</comment><comment><USER5> Hopefully you're not suggesting that it actually be worth money. That's ridiculous. If you mean what if it worked like money that might be interesting but I doubt it would be helpful.

What problem would that solve? How would that even be realistic the way you describe? This doesn't make any sense at all and here's why:

If karma is worth money then where does the initial money come from? Does PG give everyone 1500 to start like it's Monopoly? Do you put up the money yourself?

Assuming you put up your own money it just isn't fair. The richest of us get more influence and the poor get none. It'll actually work a lot like the real US economy.

Then, if karma is worth money then wouldnt it make sense to be able to withdraw it and use it in places other than HN? So what if you get people hoarding karma then they withdraw it all and end up with one. Imagine some of the very well respected people here suddenly having no karma at all because they withdrew it and spend it on a new computer.

It's just not realistic and it defeats the whole purpose of karma. If it's only good on HN then it isn't worth money. The way it works now is probably the fairest way possible. If karma is one measure of the quality of a user's contributions and how much respect they have earned then making karma work like money would totally undermine its value. Right now, karma is earned. In a money scenario there are more chances to game the system and skew it both positively and negatively for a user.

If you want to give someone money then check out their profile and see if you can either donate to one of their projects through PayPal or Flattr or something or you can just send an email and ask.</comment>


Ask HN: SaaS chatbot for free vs. charging users – what should I do? - <AUTHOR> Hi, I am founder of chatbot startup - it's fast growing, we are getting a ton of users signups, and almost 0 churn rate!
As a founder, I am confused between 2 roads: a- make it fremium/free for the sake of increasing the number of users, and then gradually increase it year by year through the next 5 yrs.
b- Make free very limited, and start charging users from day one, just like other chatbots. But this means I will get a small number of users.
What should I do? I am really having hard time making that decision! <comment><USER2> As a business, you must invest in chatbot technology, so that you don't have to see your customers departing to competition. Check out [http://s.engati.com/142](http://s.engati.com/142) and see how you can have an edge over others

</comment><comment><USER3> Split test it ;)

two options... split your traffic out OR create a different brand to test the theories

</comment><comment><USER4> I think it depends on what your goals for the startup are. If you want to bootstrap it and keep the company smaller, charge. It will keep your growth rate more manageable.

If you want to get VC money, go for the users.

</comment><comment><USER5> Is there some benefit to keeping it free and increasing users, like a network effect? Seems unlikely, so you should try to monetize from day 1.

<comment><AUTHOR> Yes there is a network effect, if you mean by that, that a user would bring more users in.... like Facebook or so.</comment></comment>


The Years of Experience Myth - <AUTHOR> http://www.codinghorror.com/blog/archives/001054.html <comment><USER3> Having two years of experience in an application or standard is a bad practice propagated by recruitment specialists. It takes a good programmer to spot a lesser programmer. (There are plenty of cases where lesser programmers look equally good or better to management.)

This forum is accessed by people who are above average programmers. Therefore, we'd be above average at spotting good programmers. Therefore, we'd be better at doing recruitment for programmers. So, what stops you from making a very good income from recruiting programmers in a existing company? Well, I researched this problem and I discovered the answer: you don't have two years experience in recruitment. Furthermore, during this investigation, I discovered that recruitment companies specialise to the extent that they outsource the recruitment of recruitment specialists.

Yep, that's right. Recruitment companies find it beneficial to outsource the recruitment of their own staff. And what stops you becoming a recruiter specialising in recruitment? Well, d'oh, you don't have two years experience. So, anyone could be a recruiter's recruiter - if they have two years experience. Based on this profound knowledge they can recruit or become technical recruiters. So, if you ever had the suspicion that recruiters just match buzzwords then you're right. If you ever had the suspicion that you could do the job better then you're right.

It is from this situation that recruiters demand two years of experience in an application or standard which has just been released. It is from this situation that human resources departments do likewise, even when they should know better.

The ultimate solution is to replace the industry with a small shell script.

</comment><comment><USER4> There are clear reasons why this is true:

Here are my theses:

\- People with lots of years in one field don't have the courage/curiosity to learn something new.

\- without curiosity you don't learn a lot in life.

\- People with tons of experience do a job for many years without being bored? Does that mean singlemindedness?

\- Our worst programmer has 20 years of experience in her field. She has a tenure track position and cannot be fired. She HAS this position because she was the only one, who did not leave in the last 20 years. Since nobody else wants her she will stay forever.

\- On the other hand: smart people are bored quickly. If you don't give them new challenges, THEY will leave (with few years of experience in every field they have ever touched).

My best guess is this: The HR departments are full of mediocre people who are not smart enough themselves to find smart people. Look around: the average rules.

If we find vacancy announcements that require lots of experience or a washlist of buzzwords, this tells us something about the company already: About who writes the announcements, about the influence of the smart people in that company and about how complex they are organized.

<comment><USER5> It's good for people to have a job, even if they are not the best one for that job. It helps keep the framework of society together. Ideally, people could exercise their true talent and get paid but it's not always easy to get paid for what you like to do. Like Jim Rohn says - if all you get from your job is a paycheck, you are underpaid. It's a series of checks and balances - <USER2> perfect job is out there - it's up to you to make the best of it.

<comment><USER6> > It helps keep the framework of society together.

A joke, yes?

<comment><USER7> I'm pretty sure he was serious. Large masses of unemployed people aren't good news for anyone. While having a dream job beats having a lousy job, having a lousy job is a heck of a lot better than the hopelessness and helplessness that comes from not being able to get a job.

<comment><USER6> I'll still argue that people having jobs isn't, and wasn't, what holds "the framework of society" together.

</comment></comment></comment></comment></comment><comment><USER8> This is very true, assuming you have someone willing to learn. I did a summer internship with Motorola once and did not know a thing about programming for phones or their environment. I was fortunate to have a professor who believed in me and was able to help me get the job. By the end of the summer I had put together demos that were front line booths at LinuxWorld and CTIA that year.

I encourage students that come to our lab to take on projects that involve skills outside of their comfort zone. I think it is important to broaden your skill set but it is more important to be comfortable with not knowing exactly how to do something and figuring it out anyway.

</comment><comment> daniel-cussen It all comes back to the pointy-haired boss paradox; the PHBs have no knowledge of tech, but still have strong opinions about it. It's like having a basketball coach who doesn't know the rules of the game:

"Why don't you stop dribbling, pick the ball up and just carry it to the other side?"

It's more or less hopeless, and this "years of experience" thing is just one more symptom of the PHB phenomenon.

</comment><comment><USER9> Essentially: "smart and gets things done."

<comment><USER10> And by "gets things done" -- it means STAYING ON TASK. Many "smart" new programmers have a tendency to stray off task...taking unnecessary initiatives and whatnot.

</comment></comment><comment><USER11> Jeff is getting boring. Stating obvious isn't new: he wants to blog almost every day, instead he should pause and come up with actual stuff to write about.</comment>


Tell HN: New twitter HN feed with links to comments @YCHackerNews - <AUTHOR> http://twitter.com/ychackernews

<comment><AUTHOR> I know there are other HN twitter accounts, but I couldn't find one that also links to the comments (which I usually enjoy more than the articles). So, I created @YCHackerNews to link to both the article and the comments.

</comment><comment><USER2> Great! Thanks for this, I was too lazy/busy to code it myself ;)</comment>


Bup: Git for filesystems - <AUTHOR> https://github.com/bup/bup <comment><AUTHOR> has a fuse mount: [https://github.com/bup/bup/blob/master/Documentation/bup- fus...](https://github.com/bup/bup/blob/master/Documentation/bup-fuse.md)</comment>


React Podcast: React Europe Recap and CSS Debate - <AUTHOR> http://reactpodcast.com/2015/08/4-conference-recap-css-vs-inline-styles/

<comment><USER2> I've been enjoying this podcast, keep up the great work!

<comment><USER3> Thanks! that's really great to hear.

</comment></comment><comment><USER4> I'm pretty interested in the discussion around React and CSS but can't bear tech over audio or video. Anyone got a good link to something readable?

<comment><USER5> If it's any help, the I've uploaded the Notes from the talk (mentioned in the podcasts), complete with speaker notes, here: [https://speakerdeck.com/chantastic/inline-styles-react- europ...](https://speakerdeck.com/chantastic/inline-styles-react-europe)</comment></comment>


Ask YC: How hard is it to keep a service like twitter online - <AUTHOR>

There have been so many posts about twitter downtime lately that it is must be pretty embarrassing for them, and do some serious damage to their reputation. The latest one is on the frontpage now: http://news.ycombinator.com/item?id=476910
Not having worked with high-traffic websites I've been wondering why they're having such problems. I would imagine that if you have money (which they do) you could just throw more servers and a good SQL consultant after the problem. Is there a problem specific to their architecture that is hard to scale? If so what is the constraint?
I'm just puzzled that they haven't solved it, and hope they do. <comment><USER2> Not all high-traffic websites are equal. Youtube has way more visitors than Twitter, but is a far simpler site to scale.

Twitter is a particularly difficult problem, because unlike most high-traffic sites the number of writes is pretty close to the number of reads. The latency expected by users is also very low - people are carrying on conversations over it after all, and these two factors combined mean that there's very little that can be done in the way of caching. Add this the complex relationships between all the different users and you've got a difficult site to scale to the sort of volumes they're experiencing.

If the guys at Twitter want to offer me a job, I've got a few ideas though ;)

<comment><USER3> _Twitter is a particularly difficult problem_

Excuse me? Twitter, at its core, is a large-scale pub/sub system with a bunch of frontends. Such systems are well understood and, frankly, quite trivial as far as the non-existant constraints of the twitter-app are concerned. After all no ordering guarantees or fancy routing are needed and obviously they don't give a damn about fault tolerance or latency either.

Their constant failure to keep that thing online just screams incompetency very loudly...

There are plenty of mature building blocks available in the OSS world to implement such a system. RabbitMQ and the spread toolkit come to mind.

Or, if you have the dough, you could buy a shrinkwrapped solution from TIBCO, Solace or the like. The latter will even come with fancy functionality and availability guarantees in writing because those are normally deployed in "mission critical" industrial systems, such as logistics, banks, stock exchanges.

Now, twitter is not SWIFT and doesn't need five nines of uptime. But saying that this is a "particularly difficult problem" is, sorry, ridiculous.

<comment><USER4> There is an old saying, _those who were asleep in CS 101 are doomed to be mocked on the Internet forever_ :-)

</comment></comment></comment><comment><USER5> Here is more about Twitter's architecture, I guess it has changed a lot right now. Also check out the presentation by Blaine Cook, Architect of twitter.

[http://highscalability.com/scaling-twitter-making- twitter-10...](http://highscalability.com/scaling-twitter-making- twitter-10000-percent-faster)

<http://www.slideshare.net/Blaine/scaling-twitter>

</comment><comment><USER6> They essentially deal with millions of real-time messages, each of which have a fairly complicated relationship tree. These issues aren't so much a database problem as a message passing and queuing problem.

<comment><AUTHOR> Why can't you just pass this to X number of servers?

<comment><USER4> The simple answer is: In the beginning they thought Twitter would more like Livejournal and architected accordingly (not that LJ is the pinnacle of good architecture or anything but even so). Instead it turned out to be more like IM and rather than starting again they decided to hack it into working.

<comment><USER7> Although the downtime must be frustrating for users and the temptation to build the shiny-new-system-that-will-solve-everything nearly irresistible for developers, I think they probably made the right decision not to do this. They have a runaway hit on their hands. That makes the risks associated with rewrite way higher (and they're already way high). One thing that people often do is start a "next-gen" project with an "architecture" team to figure out the new thing while a maintenance team (often lower-status within the organization) works on the existing product. This is so consistently bad an idea, it's pretty much a red flag all by itself. If I were ever going to rewrite a successful system I'd insist that it be the original team doing it.

I also wonder whether Twitter's downtime doesn't psychologically reinforce the sense that everybody's using it so it must be valuable. Scarcity and all that.

One thing about Twitter does make me pause. Haven't they hired tons of programmers? What are they all doing?

</comment></comment></comment></comment><comment><USER8> Hm.. isn't friendfeed similar to twitter? They don't seem to get (any) downtime. Also, there's some Indian twitter-like service which apparently has higher traffic volumes and no downtime either.

If all this is correct, then it's probably not "that hard" to keep something like twitter with an "acceptable" uptime.

</comment><comment><USER9> I once launched a Facebook IM application using AjaxIM. Everything was beautiful for about 10 days. I went from 90 users to 18K users in 21 days and the whole application came to a HALT. IMs were taking 5 seconds to get to the other end. I, for sure, did not have the technical ability to make it work and I decided to not invest money into it. We were dumping the DB of IMs every 3 hours, but with IM you are dealing with hundreds of thousands of real-time messages. This is extremely hard. Add to this the fact that each Twitter account has a different graph and your messages go to several people.</comment>


Progress has been limited on Germany's shift from nuclear to renewables - <AUTHOR> https://www.spiegel.de/international/germany/german-failure-on-the-road-to-a-renewable-future-a-1266586.html <comment><USER2> It's very popular to criticize the Energiewende as a failure because it apparently failed to reduce CO2 emmissions of the whole country. And of course, it's true that Germany could have had greater reductions if nuclear was kept alive instead of coal. But the Energiewende actually _changed_ Germany. Suddenly everywhere you go you have solar panel farms and many houses have them on their roofs and there are wind turbines. It was a cool time to live in germany to see these projects realized everywhere. They were done in an era where the price of renewables was very high compared to other technologies. The big contribution that Germany has done to the world with the Energiewende is:

* demonstrating that an industrialized nation actually can start a gradual transition towards renewables and most importantly

* by buying renewables at high prices, helping the manufacturers to get the price of renewables down. I think it's partially thanks to Germany that renewables have become the cheapest form of elecricity generation this fast, with impact observable around the globe!

From that second point alone I think the Energiewende should be considered a success. Could it have been better? Sure. Is it perfect? No. Do I like the high energy prices now? No. But I'm definitely glad we had it.

Also consider that we here Germany in fact do have a nuclear catastrophy right below our feet in the form of Asse 2.

<comment><USER3> > But the Energiewende actually changed Germany. Suddenly everywhere you go > you have solar panel farms and many houses have them on their roofs and > there are wind turbines. It was a cool time to live in germany to see these > projects realized everywhere

Does it really matter to see all those solar panel farms if the end result is they were not much helpful? The money which went into this is absolutely massive and could have been used in a better way to fight climate change.

<comment><USER4> I'm pretty sure if the survival of human race is at stake, money is no object. If someone in a foreign country invented a machine that everyone can buy to make themselves carbon-positive, the logical move for any government would be subsidize the purchase, even if that means great debt: but as the planet continues surviving, the country will go on collecting tax revenue from productive citizens/businesses and will be able to pay its debt back.

Being afraid of government debt if you're a stable country is illogical: it's also why Germany can borrow money at negative interest rates, meaning people would actually rather lend money to Germany and get less back in a few years, rather than park it at a bank. Unlike a business or an individual, the lender has very little risk that the stable country will go bust/get hit by a car and can't earn money to repay the bank.

<comment><USER5> The point is that while you can spend a lot of money you can't spend infinite money. There's a finite pile of euros. Every euro you spend on something stupid is one less euro you can spend on something smart.

<comment><USER6> Money is a tool of social organization that lacks instrinsic value. Any time we as a society refuse to do something worthwhile for the sole reason that it would cost too much, that's a capitulation to human stupidity and/or greed.

<comment><USER7> It's not so much about money, the implication is that making solar panels costs some amount of resources that according to the parent, could possibly be spent in better ways. The money just represents, indirectly, the amount of resources.

<comment><USER6> The question is, what's the limiting factor: Availability of manpower, natural resources, etc, or 'merely' the money to pay for them? The latter is an artificial constraint.

Take nuclear fusion as an example: Supposedly, this is a case where the projected investments necessary to go from theory to practice just haven't been made. With return of investment only anticipated in the far-off future, it was decided that the money could be spent in better ways.

<comment><USER7> Obviously money again is only representing some value or resource. You have to compensate workers or else they won't do any work (unless you plan on enslaving them, or have the government provide for all their needs).

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> Germany and California could already have near-zero greenhouse gas emissions from power production: [https://www.forbes.com/sites/michaelshellenberger/2018/09/11...](https://www.forbes.com/sites/michaelshellenberger/2018/09/11/had- they-bet-on-nuclear-not-renewables-germany-california-would-already- have-100-clean-power). The technology already exists. The problems are chiefly political.

<comment><USER9> Still waiting for an answer what to do with the waste. Beside

A) dumping in the ground and hope for the best

B) some magic tech will pop which will solve the issue.

<comment><USER10> It's pretty benign as waste goes. It just sits there and gets less dangerous as time goes on.

You could just keep it in a bunker and pay a crew to dust it off and repackage it when needed for the next 1000 years. But no, for some reason it needs to be taken out of the environment, like in the famous Australian sketch...

<comment><USER11> And, if technology keeps improving, at some point there will be tech to easily recycle all that "waste". Because it still contains energy.

<comment><USER12> They are doing that at Sellafield in the UK. Which is in itself pretty ugly business.

EDIT: The business is ugly, not the fact that it is in the UK.

</comment></comment></comment></comment></comment><comment><USER13> It's crazy to move from Nuclear to coal for the sake of the environment. Renewables are great when the sun is shining and the wind is blowing, but the rest of the time Nuclear is the better choice

<comment><USER14> There was a radioactive cloud over most of Europe after Chernobyl. It is much harder politically to defend nuclear energy once you lived through that disaster.

<comment><USER15> There's a cloud of particulants from burning coal and oil over most of Europe--- and it has killed orders of magnitude more people than contamination from Chernobyl.

<comment><USER16> Even if radiation is your only concern, coal emissions disperse a lot of radiation into the environment! (Still not very much radiation in absolute terms - but more radiation than the nuclear industry emits.)

<comment><USER17> > more radiation than the nuclear industry emits

buried the lede there

</comment></comment></comment></comment></comment><comment><USER18> For those of you that want to see the data behind this story: [https://www.energy-charts.de/energy.htm?source=all- sources&p...](https://www.energy-charts.de/energy.htm?source=all- sources&period=annual&year=all)

I think the history is one of a failed economic subsidy program that accidentally accelerated the PV-industry by a few years.

There were several growing PV manufacturers in germany up to around 2011/2012\. In that timeframe the manufacturers were all busy building new factories, but the government noticed that the subsidy program became large enough to be expensive and suddenly severely cut the subsidies. None of the manufacturers survived that, the remains were sold to the chinese.

<comment><USER6> Some relevant numbers (TWh) from that graphic:

                    2002    2018
        Total        503     545
        Coal + Oil   253     205
        Gas           40      44
        Nuclear      156      72

<comment><USER19> So, they've successfully got rid of about 50% of nuclear and 20% of coal by adding a very small amount of gas generation. Presumably the rest is coming from renewables. Sounds like a pretty good progress report.

<comment><USER6> _Presumably the rest is coming from renewables._

No reason not to complete the picture, I guess:

                        2002    2018

        Total            503     545

        Coal+Oil         253     205
        Nuclear          156      72

        Gas               40      44
        Biomass            4      45

        Wind+Solar+Hydro  39     177

        Other             10       3

</comment></comment></comment></comment><comment><USER20> The Energiewende is actually one of the reasons I still live in Germany: I want to see how it happens. It is stuff that our grandchildren will be proud of. And by the way, private energy consumption is heavily taxed (for a good reason), yet energy hungry industries are exempt from it. And they feel the effect: because of the growing effect of the renewable energies, they now pay one of the lowest electricity prices in the world.

There are actually good books that explain how all this works, why there are also short term "unsuccessful" years and how about everything will benefit in the end. And, most importantly: why the Energiewende actually enables a comfortable, good life.

<comment><USER21> > It is stuff that our grandchildren will be proud of.

The French can be proud today, consuming electricity with 1/6 of the carbon footprint per kWh, compared to electricity in Germany. And paying half the price, too, compared to what electricity costs in Germany.

[https://www.sciencedirect.com/science/article/pii/S136192091...](https://www.sciencedirect.com/science/article/pii/S1361920916307933)

<comment><USER20> CO2 is not the only thing that counts. One of those "clean" nuclear plants blows up and we can essentially close our countries or all die of cancer.

<comment><USER22> Nuclear plants do not "blow up", at least not the ones that are built after Chernobyl. Please stop spreading FUD.

<comment><USER19> Why do you keep posting this lie?

Chernobyl's unit 4 was destroyed by an explosion.

Fukushima had multiple explosions that destroyed the upper levels of reactor buildings, and ruptured the containment of unit 2.

During the three mile island accident there were several days where engineers were sweating bullets because there was a hydrogen gas bubble at the top of the pressure vessel, and they were afraid it would explode. It was only after about a week of effort that they were able to eliminate the H2 bubble.

Every nuclear reactor has the possibility of immense damage due to explosion - steam explosions, explosive gas, and secondary systems such as diesel and electrical system. It is one of the primary hazards that control systems and emergency cooling and venting are there to deal with. Lying to everyone that it's not possible isn't helpful to the nuclear community.

<comment><USER23> They do not blow up in the sense of a nuclear explosion. It's a normal explosion. With normal consequences.

<comment><USER24> They blow up in the sense of a dirty bomb. That's a lot worse. It would be great if NPPs just blew up in a nuclear explosion! We could safely use them by making the blast radius a forbidden zone.

<comment><USER23> > They blow up in the sense of a dirty bomb. That's a lot worse. It would be > great if NPPs just blew up in a nuclear explosion! We could safely use them > by making the blast radius a forbidden zone.

I do not know how to respond politely to such a ridiculous statement.

<comment><USER24> Then please explain impolitely where you think I'm wrong.

To clarify: "great" is no absolute, an NPP that generates electricity is obviously better than one that explodes. It also depends on the size of the explosion. A Hiroshima-class explosion would be acceptable, the one of a tsar bomb might not be. Also an NPP that fails like that would still kill its operators and perhaps some unlucky farmers. But compared to the actual worst case, that's negligible.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER25> I believe nuclear power is inevitable. Human power consumption has been growing exponentially, at some point renewable energy will not be able to catch up without causing extensive damage to our ecological system. There are finite amount of renewable energy per sqkm on earth, at some point the area required will begin to erode local ecology. Right now nuclear power appears to be the densest form of energy generation. I agree in the short term renewable energy is a great alternative to coal/natural gas.

<comment><USER24> > Human power consumption has been growing exponentially

That has to stop. Power generation cannot grow exponentially forever

</comment></comment><comment><USER26> I've always thought it would be more efficient to use a market system with something like a carbon tax and charges for other externalities rather than ad hoc policies for this and that.

</comment><comment><USER27> Nuclear or nothing, because with nuclear you need nothing else.

<comment><USER24> Except for a few spare cities for when the ones you have are made uninhabitable by a nuclear accident.</comment></comment>


Benchmark results of Kubernetes network plugins over 10Gb/s network - <AUTHOR> https://itnext.io/benchmark-results-of-kubernetes-network-plugins-cni-over-10gbit-s-network-36475925a560 <comment><USER2> [Disclaimer: I'm one of the Cilium authors]

We have been trying to reproduce the performance results ever since the article was published as this is not in line at all with what we measure daily in our CI. We can easily do a multiple of these numbers.

There are some obvious flaws in the benchmarking scripts [0] such as using the "used" column of `free` without taking into account cached file buffers.

However, it does not explain why HTTP and FTP are worse compared to the TCP benchmark which is doing wire speed at ease. None of the Cilium datapath is HTTP or FTP specific unless HTTP specific security policies are in place in which case HTTP traffic is actually parsed.

We have requested more information on the scripts used by the author and continue to investigate. We will publish results as soon as we can reproduce this.

As stated by other commenters as well, most of these benchmarks are measuring the same Linux kernel code except for Weave (OVS) and Cilium (BPF). However, at the specified MTU of 9000, the bottleneck for all plugins will not be the forwarding datapath but the actual client and server code copying the data in and out of the kernel as there are very few packets actually being created and forwarded.

[0] [https://gist.github.com/AlexisDucastel/ebb884831aeec5827e4df...](https://gist.github.com/AlexisDucastel/ebb884831aeec5827e4df82975e5c4eb)

<comment><USER3> > most of these benchmarks are measuring the same Linux kernel code

This, 1000x this. I'm afraid too many people treat their CNI plugin as 'magic' whilst many of them really aren't. 'Host' versus Calico is basically benchmarking the impact of a Linux bridge device, and maybe some more iptables rules than the host has (depending on whether the host benchmark has iptables enabled at all, whether there are K8s network security policies in place and enforced by Calico,...).

Also, configuration details are lacking. E.g. in the Calico benchmarks, was ipip enabled or not?

<comment><USER4> Yes most solutions are using Linux kernel, so what's being measured is indeed the impact of the way the kernel is being configured to achieve container networking. But that doesn't make those design choices, and the tests, meaningful. Calico, for example, contrary to your assumption, uses neither a Linux bridge device nor iptables for packet forwarding. (It does use iptables for policy enforcement, but that's not being tested here.)

<comment><USER3> I'm aware it doesn't use iptables, except to implement network policies, hence the reference. Good call about the bridge usage, my bad, makes sense that's not being used given Calico is L3...

</comment></comment></comment></comment><comment><USER5> I wonder what the latency stats are for each. Bandwidth per second is useful but in my experience latency is the more important statistic for the applications I manage.

<comment><USER6> Packets per second aswell, speed testers generally give you the absolute best case full MTU sized packets for 100% of the duration.

Real world traffic has lots of nasty small packets.

</comment></comment><comment><USER7> Any plans to run some packets per second benchmarks with latency? Everything today can run at line rate, especially with jumbo frames. These existing benchmarks don't show much. Networking people want to see how many PPS you can push, at what latency, and at what packet sizes can you achieve line rate?

<comment><USER8> Some IMIX benchmarks would be a good start. [https://en.wikipedia.org/wiki/Internet_Mix](https://en.wikipedia.org/wiki/Internet_Mix)

</comment></comment><comment><USER9> I find it slightly weird to test different protocols like that, especially the SCP test, where the author states:

"With SCP, we transfer the 10GB random file over scp using OpenSSH server and client. We can clearly see that even bare-metal performance are much lower than previously."

SCP is just notoriously slow, and it should be expected to not be able to max out the connection with it.

<comment><USER10> No one is saying you should use SCP for anything or that it's an alternative to any of the others - none of the tested protocols are practical alternatives to each other. It was included to provide a comparison point for the encrypted version of WeaveNet.

Encrypted WeaveNet handled plain TCP traffic at 1,330 Mbit/s. Without the SCP comparison there's no way to tell whether that's really really good or really really bad. Seeing that the unencrypted version of WeaveNet handled SCP at 1,594 Mbit/s shows that it's not horrible, but could probably be better. It's good for WeaveNet folks to know they have some room for improvement and it's good for you because you might want to consider alternatives if you require encrypted traffic and are very bandwidth sensitive.

<comment><USER11> [Note I work on Weave Net]

When we ran a similar test we got 3.75Gbit/sec; the figure in this article is closer to what we got in our "slow mode" which is used as a fall-back when it can't get the in-kernel path to work.

[https://www.weave.works/blog/weave-net-performance-fast- data...](https://www.weave.works/blog/weave-net-performance-fast-datapath/)

</comment></comment></comment><comment><USER12> For those interested in a feature comparison I did one here:

[https://kubedex.com/kubernetes-network- plugins/](https://kubedex.com/kubernetes-network-plugins/)

<comment><USER13> This is pretty neat. Would be interested to see aws-cni-vpc in there, too!

<comment><USER12> Added as the final column in this sheet.

[https://docs.google.com/spreadsheets/d/1qCOlor16Wp5mHd6MQxB5...](https://docs.google.com/spreadsheets/d/1qCOlor16Wp5mHd6MQxB5gUEQILnijyDLIExEpqmee2k/edit?usp=sharing)

</comment></comment></comment><comment><USER3> As mentioned here, the article doesn't specify whether the Calico setup uses ipip or not which could have a measurable impact.

In the Calico-without-ipip case, you're basically comparing host networking (let's assume with some iptables enabled) with host+iptables+bridge networking (yes, there may be more iptables rules involved). Let's assume the impact of iptables is the same between both, then it'd be interesting to measure the impact of the Linux bridge being used. As a colleague of mine mentioned, this may be barely noticeable on a 10Gb interface, but could be on faster networks. How about running these tests on a 20Gb or 40Gb network? These are quite common in datacenter networks, which is where you'll be using these CNIs (unlike GKE, EKS,... where you can integrate with the 'native' SDN).

Finally, I'd be interested to get some results when using the macvlan CNI plugin (though then one loses network policy support, sadly enough :( )

<comment><USER14> I don't work in the networking space, is ipip synonomous with IP-in-IP tunneling? (I can't find a definitive answer in my cursory attempt).

<comment> X-Istence Yes.

[https://docs.projectcalico.org/v3.2/usage/configuration/ip-i...](https://docs.projectcalico.org/v3.2/usage/configuration/ip- in-ip)

This is used when there is source/dest filtering happening on the network (such as in certain cloud providers where you can't send packets from 10.0.0.1/32 to 192.168.1.2/32 because 10.0.0.1/32 is not valid, so you pack it into another IP packet that makes the source seem like 192.168.1.3/32).

</comment></comment></comment><comment><USER15> Hm, running iperf3 stream is pretty synthetic benchmark, what about real-life workloads? iptables should be pretty damn slow the more rules are added as they are walked linearly, I doubt this has been properly evaluated in the blog post above.

</comment><comment><USER16> For those looking to go above 10Gbps, Intel makes a dpdk plug-in, and Cisco makes the VPP virtual switch with dpdk as a back-end.

</comment><comment><USER17> Flannel has encryption. It specifically has a documented IPSec mode. I've actually run it in production.

<comment><USER18> it also has a very very simple wireguard plugin that "just works" in my experience. You can read the entire thing in a minute.

[https://github.com/coreos/flannel/blob/master/dist/extension...](https://github.com/coreos/flannel/blob/master/dist/extension- wireguard)

</comment></comment><comment><USER19> Things like CNI take away from the development and understanding of general networking technology to scripts and json/yaml wrappers that not only add another layer of complexity but also prevent users from understanding what they are actually using.

For instance in this post itself we learn about all these CNI wrappers but learn nothing about Vxlan, BGP and Linux networking tools that these are using underneath. And those are actually easy to use directly. This does a disservice to both developers and users.

For instance true game changers in this space is tech like Wireguard that can deliver encryption at near line speed and would kill these benchmarks. Everyone can benefit from Wireguard today. Initiatives like CNI encourage wrappers and those then wrapping tools like Wireguard end up getting the recognition and success while the underlying tools like Wireguard fade into the background. This does not seem like a healthy or sustainable development model.

<comment><USER20> K8s is all about encapsulating devops practices into code, so it's kind of philosophically totally opposite to learning all the details of virtual networking and configuring it from scratch.

If Wireguard doesn't write the wrapper themselves they have no one to blame. People need to understand that the hard/cool part and the valuable part aren't the same and if you are too cool to create value then you can't complain about other people taking it.

<comment><USER19> This is a nice idea about abstraction but without knowledge you cannot run anything professionally with any degree of reliability or confidence. What happens when things break? Google, Facebook and any company dependent on delivery have full fledged networking and ops teams.

All these wrappers still require time investment in knowledge, setup and configuration. The cost for users knowing the wrappers but not the tech is being locked in a platform, if they know the tech even if Kubernetes crashes and burns their knowledge and time investment remains valuable.

Before Wireguard it was extremely difficult to get near line speed with encryption, now accessible to all. If this is not 'value' what is? Given this post is about networking benchmarks this is a huge deal. If people stop creating these kind of networking tools and technologies then there is going to be nothing to wrap.

</comment></comment></comment><comment><USER21> In many cases these containers are running on the same physical machine. Could the network stack be optimized so much so that writes from one program inside a container can write directly into pre-allocated memory of another container using our existing networking interfaces?

<comment><USER2> [https://www.slideshare.net/ThomasGraf5/accelerating-envoy- an...](https://www.slideshare.net/ThomasGraf5/accelerating-envoy-and-istio- with-cilium-and-the-linux-kernel)

Slide 21-25

</comment></comment><comment><USER22> Was calico ipip encapsulation enabled or disabled?

</comment><comment><USER23> I'm impressed that all non-encrypted methods are so close to bare-metal. Wonder how much CPU was impacted at those rates though?

<comment><USER24> CPU usage was the final test, near the end of the article.

<comment><USER23> Ah missed that, ta!

</comment></comment></comment><comment><USER25> I have to admin I've never quite understood the need for the overlay network, esp with such a large performance impact. Security has been moving higher up into the app layer anyway with Beyond Corp style approaches. What are the uses cases driving it?

<comment><USER26> In our case, service mesh and network isolation (including for potentially untrusted code). Being able to ensure that pods can't talk to each other by default and managing whitelists for service to service communication. Managing that without an overlay is a pain once you're past a handful of nodes.

</comment></comment><comment><USER7> Also, is Romana still actively developed? It seems like they had a lot of momentum, then suddenly things went radio silent on their Slack and git repo.

</comment><comment><USER27> Has not [https://news.ycombinator.com/item?id=18564340](https://news.ycombinator.com/item?id=18564340) achieved the same result?

</comment><comment><USER28> I wish there was a similar comparison of proxies.

</comment><comment><USER29> All these 'plugins' are simply scripts and tools which configure the linux kernel iptables and network forwarding rules right?

None of them actually handle packets, connections, routing etc directly I assume?

Given that, it seems disingenuous to benchmark them... You're simply benchmarking the Linux Kernel vs the Linux Kernel.

<comment><USER30> At the end of the day you're benchmarking a strategy. It doesn't matter that Linux is the underlying thing executing that strategy. All of these tools have branding and marketing around them... It's important to see which ones live up to claims.</comment></comment>


Android, Tipping Points and Western Electronics Entrepreneurship - <AUTHOR> http://open.neurostechnology.com/content/future-hardware

<comment><USER2> Could be. One of my pie-in-the-sky ideas is a heads-up display in glasses using eye tracking or hand gestures for input. If I ever actually tried to build that, using Android as a base would be the obvious choice.

</comment><comment><USER3> Why so few upvotes on this story? I think it's a pretty interesting analysis.

<comment><USER4> Maybe people haven't bothered to read it all the way through...</comment></comment>


Why You Have to Buy Saas - <AUTHOR> https://blog.sideci.com/why-you-have-to-buy-saas-9f814ceac48e <comment><USER2> Great post. Undervaluing time is a common mistake.</comment>


Employee 1: Yahoo - <AUTHOR> http://themacro.com/articles/2016/06/tim-brady-interview/ <comment><USER2> We're really excited about this new series on The Macro about the first employees at startups. First employees are often unsung heroes at companies, but often incredible crucial to getting them off the ground and setting the culture of the startup.

I'm always remind of Derek Sivers video on First Followers when I think of them. [https://www.youtube.com/watch?v=fW8amMCVAJQ](https://www.youtube.com/watch?v=fW8amMCVAJQ)</comment>


Show HN: Free social/portfolio site for aspiring artists - <AUTHOR> http://earlycanvas.com

<comment><AUTHOR> EarlyCanvas is your 'evolving portfolio'. Share your art work (whether it's in progress or complete), get feedback, follow other artists, pin inspiring art, even sell your art. All the while we'll create a beautiful, customizeable, professional portfolio site for you.

All for free. No limits, no restrictions, no fee. And you ALWAYS retain the copyright.</comment>


Ask PG: Do comments from new users get auto-killed too easily? - <AUTHOR>

Some comments that definitely aren't spam earn the status of 'dead.' For example:
http://news.ycombinator.com/item?id=1567347
http://news.ycombinator.com/item?id=1566744
Those posts seem to have two things in common:
* they are both from new users with low karma
* they both contain links.
Is there an auto spam-killing function on a rampage? :-) <comment><USER2> It looks like both those are users that got killed, their comments after that point may be 100% on-topic and legit but if they don't do something to ask for re-instatement of their accounts anything they do will be automatically killed as well.</comment>


Coronavirus survivors banned from joining the military - <AUTHOR> https://www.militarytimes.com/news/your-military/2020/05/06/coronavirus-survivors-banned-from-joining-the-military/ <comment><USER2> The way I read this, they're making it a disqualifying condition, but many, if not most, disqualifying conditions can be waived -- they simply need medical records and then a medical review to ensure the condition is no longer continuing and the person is fit for military service.

The way the instruction is written is confusing and I'm not sure it's written exactly as they intended it, but that's how I interpret it anyway.

</comment><comment><USER3> Given that most people will acquire this virus, could this be a tool designed as a loophole to deny entry to people that would otherwise be protected by anti-discrimination laws?</comment>


Newsweek ending print edition, job cuts expected - <AUTHOR> http://news.blogs.cnn.com/2012/10/18/newsweek-ending-print-going-all-digital/

<comment><USER2> Yeah, I mean we have an election going on, China's housing bubble, numerous other big things and this is the cover story they choose?: [http://www.theweek.co.uk/media/49493/newsweek-%E2%80%98hits-...](http://www.theweek.co.uk/media/49493/newsweek-%E2%80%98hits- new-low-%E2%80%98heaven-real-cover)

It's not a problem because it's heaven, it's because it's a self contained story.

<comment><USER3> it seems from that article that they've decided that their market is the religious right and are targeting that.

so are the religious right big consumers of online media? i don't know much about the usa, but my prejudices suggest they may be less connected than most?

and i guess their european sales never amounted to much?

(edit: maybe they plan to merge with readers digest ;o)

<comment><USER4> They've been going after the religious right for a long time now. I used to be a subscriber, and they started skewing more and more overtly religious with each issue. Eventually I got tired of the science-bashing, the Bible study, the Tea Party tubthumping, etc.

I wish them the best of luck in their online venture, though I do share your skepticism that their target audience is digitally connected.

A big issue with their target demo is that it's really hurting, financially. Those folks might be cutting back on subscriptions, or even ad hoc purchases, in general. That doesn't look to change anytime soon. That being the case, maybe Newsweek feels it can earn better ROI by selling digital ads against free eyeballs than it can by trying to sell print copies to people unable or unwilling to pay for them. If this is the case, the strategy might be somewhat reasonable.

</comment></comment></comment><comment><USER5> Having worked there, I can guarantee they'll handle layoffs in the least diplomatic way possible. I'm guessing an email saying, "Thanks for your 30 years of service, see ya!"</comment>


Show HN: CSV Explorer (YC F1) - Explore CSVs with Millions of Rows - <AUTHOR> https://www.csvexplorer.com/ <comment><USER2> I usually use csvkit ([https://csvkit.readthedocs.io/en/1.0.1/index.html](https://csvkit.readthedocs.io/en/1.0.1/index.html)). There are commands to list the columns, filter, browse the data in a somewhat formatted way with less. Typing the commands is a huge pain though, and I would be very interested in a tool that could _instantly_ pop open and let me peruse. Excel can take minutes to load and eagerly does a lot of unhelpful formatting on things like dates and decimals.

That being said I would never want to (and often legally cannot) etl my data to some third party. That would be terribly slow. But I would happily pay for a nice desktop tool to do it for me; command line or GUI.

<comment><USER3> At a former job, our embedded device logs decoded to csv. Some of them were too large for Excel.

Pandas handled them without a burp. Pandas in Jupyter (Ipython Notebook) was a godsend.

There's a minimal amount of variable setup, but once you've done that once or twice it's easy.

Of course, any analysis or manipulation takes a bit of python code, but I see that as a feature, not the least because you can read it right there in the open instead of having to hunt for formulas in cells.

<comment><USER4> Same here. I load the data using pandas or parsing the rows by hand in go. It's interesting to watch your RAM getting filled up while the data is loaded.

Looks like this tool is for non-programmers, it's interesting to see that there seems to be a market here.

</comment></comment></comment><comment><AUTHOR> Hi HN!

A few months ago, a college friend reached out because his consulting company had just received a 60 Gb spreadsheet, and they didn't know what to do with it! They actually tried opening it in Excel.

I'm excited to Show HN CSV Explorer - a simple web tool for opening really big CSVs! Try it out, and let me know how it goes!

<comment><USER5> Open it with a programming language like R, Julia or Python that supports the data frames. No reason to upload 60 gigs to the web or Excel.

<comment><AUTHOR> This project came out of some of my contracting work for consultants, journalists, lawyers - people who don't know what a command line is.

</comment></comment></comment><comment><USER6> Cool idea - Having to work with sensitive data, it'd be great to have this functionality without importing/uploading to the internet.

<comment><AUTHOR> Thanks - having worked with health data for a few years I can relate. Unfortunately, I don't have plans for a desktop app right now.

<comment><USER7> Think not of a desktop app, but of doing everything client-side in JS. That way, it's still a web app, but you're not schlepping [sensitive|large] data between front and back-ends. Also, by offloading the work onto clients, it scales much better - you could host the app on a CDN and have no real back- end.

<comment><AUTHOR> I originally wrote it in entirely client side JS, but it didn't scale nicely past a few hundred thousand rows. For the really big datasets, CSV Explorer loads them into Redshift - queries takes a few seconds!

</comment></comment></comment></comment><comment><USER8> This is really awesome. What sparked this idea?

Not sure if you have this feature, but I think it would be really cool if you could open APIs to your own datasets. This could be really useful for enterprise applications that do a lot of flat file imports/exports to push/pull data.

<comment><AUTHOR> Some consultant friends asked for help when they got a 60Gb CSV file from their client! Since then, I've also found that lots of tech companies share CSV internally. I've also worked with journalists attempting to look at large public government datasets.

<comment><USER7> Plenty of Big Data things also work with CSV. Like >1TB datasets big. It's pretty insane, but it works.

</comment></comment></comment><comment><USER9> This tool would be great were it not for me (rightfully) getting fired and possibly sued by any of my clients if I uploaded even a single file. In fact, what kind of business with millions-of-rows kind of files would entrust said datasets to a service that to me seems strangely vague [1] on how the data is secured, or where it is actually going to be physically stored.

[1] [https://www.csvexplorer.com/legal/privacy/](https://www.csvexplorer.com/legal/privacy/)

</comment><comment><AUTHOR> Update: A one minute demo video - [https://youtu.be/RBiDL5neWDc](https://youtu.be/RBiDL5neWDc)

</comment><comment><USER10> Is there any reason this tool couldn't be 100% client-side?

<comment><AUTHOR> Perhaps - I tried a client-side implementation, but I had issues scaling past a few hundred thousand rows. I understand hesitations of uploading data to the cloud, but I now have users looking at hundreds of millions of rows in seconds thanks to Postgres!

<comment><USER11> You can run PostGres natively as well and do the same thing no?

<comment><USER10> Not really an all-in-browser solution then.

</comment></comment></comment></comment><comment><USER12> My solution to this problem, so far, has been ipython and pandas. (and maybe jupyter notebook for visualization and sharing).

What's the value-add here?

<comment><AUTHOR> It's like a Jupyter notebook for non-engineers, people who don't know what python is, eg. consultants.

</comment></comment><comment><USER13> What's "YC F1"?

<comment><AUTHOR> I was in the first batch of YCombinator's Fellowship.

</comment></comment><comment><USER14> Does this do stuff lnav can't?

</comment><comment><USER15> you should integrate with commatocolumn.com! ;)</comment>


Interview with GoCardless CEO Hiroki Takeuchi after paralyzing accident - <AUTHOR> https://techcrunch.com/2017/09/18/hiroki-interview/ <comment> andy-wu This is really incredible. I'm a quadriplegic student and you don't really see people in my position in tech, and definitely not in executive roles so this means a lot in terms of proving it's possible.

<comment><USER3> Somewhat tangentially, the current governor of Texas is also paraplegic. I suppose if you place his politics aside since they are quite disagreeable to the typical HN reader you could probably find some inspiration in his success as a paraplegic in the political spectrum as well.

</comment></comment><comment><USER4> I think we need to start designing our cities so that people walk and cycle on a level above the street, sort of like the Highline in NYC. And cars and trucks operate below that without pedestrians.

That would make cycling a lot safer. Expensive to do for existing cities, but robotic construction may make it cheap enough to do in a few decades.

<comment><USER5> Why not just simply ban cars from cities in general? Most of the cities in Europe at least have a great public transport setup. Just imagine how much space used by parked cars would be available to public again!

<comment><USER6> Even without any cars, delivery, trade, construction and transit vehicles add up to a significant fraction of current traffic in big European cities. You can't remove this traffic without functionally harming city life.

London, where I live, has both congestion charging and emissions charging (ULEZ) but still has loads of traffic. I'd estimate about 75% of it is not private vehicles outside rush hour, and probably no more than 50% private during rush hour, in central areas. My estimates come from riding through it by motorcycle daily.

<comment><USER7> London has already proven, during the month of restrictions around the Olympic Games, that it is possible to massively reduce motorised vehicle traffic within zone 1 and zone 2 during the workday without causing harm to businesses and the functioning of the city. Most people felt that the quality of life in London dramatically improved during the games, and one of the major reasons cited is reduced traffic, cleaner air, pedestrianised zones, increase in cycling, smoother traffic flow for public transport, out of hour deliveries, etc.

It's already been successfully done, there is just a lack of political will to make such measures permanent.

<comment><USER8> > cleaner air

Ride 100 provides an excellent demonstration of this even just by closing -some- roads for -one- day.

cf this graph of pollution on Putney High Street:

[https://twitter.com/jwoLondon/status/892004529545379840](https://twitter.com/jwoLondon/status/892004529545379840)

<comment><USER6> People simply do business on a different day to cycle takeovers.

<comment><USER8> I can't really fathom what your point is here? No cars means no pollution -> reducing the amount of cars reduces the pollution. Is that disputed?

<comment><USER9> Their point is that the closing a road once doesn't just make all the pollution not happen, it time-shifts some of it to other days when people are driving again.

I expect there are some people who _do_ make other arrangements and take a bus, so it's a net gain, but there are also those who take their car on Monday since they couldn't on Sunday.

EDIT: I looked up how RideLondon works, map at the end of this PDF shows road closures [https://d1ffaecguugkl4.cloudfront.net/ridelondon/live/upload...](https://d1ffaecguugkl4.cloudfront.net/ridelondon/live/uploads/cms_page_media/19/Central- Road-Closures-2017.pdf)

So it's not even time-shifting necessarily. People can drive around that and just move the pollution from one street to the next. But yes, the pollution measurements on one of the closed roads would look better that day.

<comment><USER8> > closing a road once doesn't just make all the pollution not happen

Yes, I know, it was an illustration of how much pollution is caused by traffic in London and how much could be gained by reducing those traffic levels overall. I am baffled that this is causing confusion to people.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> > Delivered without sugar-coating and designed to manage expectations, the > prognosis from doctors wasn't good. The aim of surgery was to straighten > Takeuchi's spine and fix his posture, not to enable him to walk again. If > during the procedure the doctors saw additional work that could be done, > they would do it, but he shouldn't allow himself to think that he would wake > up and be walking.

Wow. We are a community of people who enjoy the ability to dream and ask, "what if?" It's very difficult to turn that off and accept that the impossible is, in fact, impossible. That kind of realization can be, and almost would certainly be to me, defeating.

Not for Hiroki - this guy is just wired differently.

I wish him the best.

</comment><comment><USER11> I did an internship at GoCardless last summer - Hiroki is an excellent leader and I'm not surprised to see him bounce back in such an inspiring way

</comment><comment><USER12> Interesting how work can bea meaningful distraction after a crisis. Something meaningful to think about other than adjusting to life in a wheelchair, in this case.

</comment><comment><USER13> About 5 years ago I was CTO'ing for a firm upstairs in the same building as GoCardless - both firms moved out of that abandoned shell in Finsbury Square around 3-4 months after I left, I believe. At the time they were focused entirely on validating direct debit payments were something people wanted to do online (they do), and we were figuring out how to move private jets around Europe (long story).

I only met Hiroki briefly, my main interactions were with others there. They seemed like a committed and focused group, determined to build something of value.

I have some empathy for some of this story: I couldn't walk in April of this year and had to have surgery to remove an abscess that caused some quite nasty neuropathy. It happened quite quickly - over a couple of weeks, not a car crash speed, at least. And I was treated by Imperial down in Hammersmith, the same trust who I believe run St Mary's.

Whilst my prognosis was not quite as dramatic - I was told that walking again was "a nice side effect we see in 70%-80% of people in this sort of case" \- I was prepared to be in a wheelchair the rest of my life. I told my GF that if she couldn't handle that, I'd understand. The silly cow didn't take her chance... :-)

That moment when you wake up in recovery, try and move your toes and realise nothing has really changed: pretty devastating.

I was lucky. Physiotherapy and time have contributed to me walking pretty well with a stick, and that might go soon. My spinal injury was no more than a compression, in which the nerves will over time regrow mostly - <USER2> Hiroki's which sounds more like a shear.

I can also empathise with the desire to get through physiotherapy quickly: whilst I still receive weekly physiotherapy sessions (I have one in a couple of hours, actually), I wanted to get off the inpatient unit ASAP: the normal stay was 3-4 months for my sort of case. I got done in 5 weeks. I've heard stories of people being there for years.

Also, being pushed around is stupid. I agree with him on that. My girlfriend, bless her, she did well. But never again.

And yes, it does all make you rethink everything you are and do and value in life, and in some ways accidents like this can be blessings in disguise. I suspect he will now be more focused and determined to do the big things in life that you dream about but never act on: I know I have become so since leaving hospital. No more sitting around watching Netflix every night for me...

I might be lucky in that I can walk again - and 6 months from now you'll never know I went through anything like I have - but I'll always have a certain appreciation for life that only people who have been through something that traumatic can share.

Good luck to him, and to the rest of the GC guys - they're all going to be fine.

</comment><comment><USER14> I don't think this is the first interview [1] but it's interesting. DSM is worth a listen if you've not heard it before.

[1] - [http://www.wnyc.org/story/death-sex-money-rachel- hiroki/](http://www.wnyc.org/story/death-sex-money-rachel-hiroki/)

<comment><USER15> "the GoCardless CEO's first with the technology or business press"

</comment></comment><comment><USER16> The 2012 Paralympic Games London were a huge triumph.

However London doesn't seem innately supportive to people in wheelchair users, I hardly ever see them in central London, and not at the many Meetups I attend; which I believe is a loss for us all.

The new Crossrail Line is being built with wheelchair access in mind, so hopefully it just won't be the tough as nuts, extrovert wheel chair users who can surmount getting into London.

Smaller towns may be easier to get into, but they probably don't have the critical mass, and events.

Which is the best large city for wheel chair users?</comment>


Using AI to extract Building Footprints - <AUTHOR> https://medium.com/@zeeshan_80154/building-footprints-and-ai-eeae3271ed89 <comment><AUTHOR> This article describes how building footprint polygons can be extracted from satellite, drone and aerial imagery through the use of AI.

</comment><comment><USER2> Basic "building" blocks of our civilisation. Pun intended

<comment><AUTHOR> Very true.

</comment></comment><comment><USER3> Building footprints are essential in so many industries.

<comment><AUTHOR> They are. In fact everyday, more industries are understanding the numerous ways in which they can use it.

</comment></comment><comment><USER4> One point solution for almost everything.

<comment><AUTHOR> yes, a simple yet far-reaching solution.</comment></comment>


Official Google+ Share Button - <AUTHOR> https://developers.google.com/+/plugins/share/

<comment><USER2> here is the direct link: <https://developers.google.com/+/plugins/share/></comment>


ThinkPad X230: good old thinkpad except the keyboard - <AUTHOR> http://www.lenovo.com/products/us/laptop/thinkpad/x-series/x230

<comment><USER2> IPS panels are great, especially on laptops (wide viewing angle when sharing a screen with someone). I wish more laptops had these.</comment>


Clojure's loop and recur in JavaScript - <AUTHOR> https://github.com/mjdesa/loop-recur

<comment><USER2> Neat, I did a similar thing in Ruby using continuations. [http://www.sitepoint.com/clojure-loops- ruby/](http://www.sitepoint.com/clojure-loops-ruby/)</comment>


Algorithms as Objects - <AUTHOR> https://gieseanw.wordpress.com/2019/05/10/algorithms-as-objects <comment><USER3> I'm honestly impressed. After more than 20 years programming and reading about programs, this is the first time that I find such a thing: a concise and well- written description of the _exact opposite_ of everything that I believe in. It's like the author was reading my mind and applied a NOT filter before writing it back.

<comment><USER4> I'm interested in learning more about your thoughts. Please elaborate?

<comment><USER5> Code smell #1/#2: If you have too long function/method break it apart. How does this relate code being in function VS method

Code smell #3/#4: If you don't want to put helpers to global scope then just don't. I don't see why closures are so bad. Top level function can act pretty much same as class scope so what's the difference? I don't know about python but if you really don't want to use closure another languages have other encapsulation methods too.

Code smell #5: If you use closure you can have shared state. Passing state can be noise but on the other hand if you only read/write "semi global" variables it's messy too. If you need to pass lot amount of state it's usually hint that you have some other problem in your code.

1\. Like said you don't need objects

2\. Function have single entry without ambiguity like @Someone shows.

3\. You should test behavior, not specific implementation. You don't need mocking for pure functions.

4\. Is X optional? What about Y? Does order matter when calling 'withs'? And that builder thing is directly stolen from all the Java jokes.

5\. I don't know what is Pythonish way to do delayed callability but I claim that it should be independent thing rather than directly bounded to specific function/object.

If it looks like a function, swims like a function, and quacks like a function, then it probably is a function.

<comment><USER6> 5\. I do not know if it's pythonish, but I use and abuse functools.partial

</comment></comment></comment></comment><comment><USER7> I don't think I agree with using objects as the public interface to an algorithm, especially if its primary purpose is to apply some transformation over the input (i.e. a natural fit for a function). The examples given have a bit of a two-phase initialization smell: initialize some internal state, then run the algorithm: steps that are never performed separately.

Objects as an algorithm internal state might be a better approach: the caller doesn't have to worry about the algorithm's internal state, they only need to call the external function entry point (and you'll destroy or otherwise forget the internal state at the end of the function, rather than putting the burden on the caller).

<comment><USER8> A good example is hashing

        hasher = Hasher(iv)
        hasher.hash(data)
        hasher.hash(data2)
        ...
        hash = hasher.finish()

Compression/decompression is similar.

</comment></comment><comment><USER9> _"It's impossible for a client to call the wrong function; there's only one way of doing things.

      my_algorithm = MyAlgorithmClass()
      results = my_algorithm.run()

"_

Only one? That API opens the door for

      my_algorithm = MyAlgorithmClass()
      results = my_algorithm.run()
      more_results = algorithm.run()

, which, hopefully, resets exactly the right amount of state to work just fine.

There's also the 'fun' of sharing an 'algorithm' between threads. That will teach you that this 'algorithm' is more a set combining function arguments with function state, and a method that uses them to compute a result. Next, if you're smart, you'll separate the arguments from the state.

⇒ if you do something like this, use that builder pattern.

<comment><USER10> Worse, it opens the door for stuff like:

my_algorithm = MyAlgorithmClass() results = [0..100].parallelMap(lambda x: my_algorithm.run())

<comment><USER11> Mandatory Rust evangelism: both of these examples would be forbidden at compile time in a language with affine (or linear) types ;)

</comment></comment></comment><comment><USER12> i really don't see how

        AlgorithmClass().WithXValue(10.0)
                        .WithYValue(20.0)
                        .WithEtc(...)
                        .Run();

is any better than

        Algorithm(x=10.0, y=20.0, etc=...)

the argument that text books teach us algorithms in form of single function with inputs and outputs is quite weak, there's a very simple solution - <USER2> up the large function into smaller functions.

i'll always prefer a set of isolated functions with explicit argument and return declarations over a class implementation with methods because:

\- argument list tells me immediately what is required and what are possible additional options as opposed to having to scan the method list for methods matching `WithFooBar` names

\- methods become coupled to internal state of the algorithm object because let's face it - code rots, maintainers take shortcuts, state proliferates, everything that is allowed by a compiler/interpreter will eventually make it into the codebase; it's much more preferable to avoid/diminish most of these problems by writing isolated functions

\- explicit arg/return declarations help with default static analysis/checks, much harder to do that with classes and internal state

<comment><USER13> Because it is not. It's oop distorsion field at its finest.

You can even find talks about why not do that (pyconf iirc).

Basically it's useless réification of functions as you state. But when you love oop (and probably didnt use other things) well it's the ultimate itch to scratch.

<comment><USER14> That really is a straw man. There are plenty of OO enthusiasts who would immediately call out fluent APIs as being very bad style.

Ideological demonization is pointless for advocates of either side.

<comment><USER13> I'm sorry, I've been burned too badly by the OO world (not helped by the fact that I went in college at peak J2EE).

And to a certain extent.. I think this crowd still has issues. I spent time on #java recently and I felt this acid confidence in OOP that stimulate their mind that I really cannot fathom. I used to be like that, until I left this field.

Now this is just personally, but of all the things I've read about, it was the less useful for my brain. SQL did teach me some, pre OO Ada did teach me some, ML taught me loads, Lisp worlds, forth etc etc yet OO falls flat to my eyes 99% of the time.

To be honest, when I read 'algorithms as objects' I upvoted, thinking it would be distributed logic as object ala smalltalk. But again, fluent lipid layers.. And I think, the paradigm is to blame.

Now IIRC, you have a lot more knowledge about PL than most, so maybe you see OO in a different light, a more rounded way.

<comment><USER14> I've been burned by FP ideologues as well, I really don't see a difference between junior developers cargo culting X vs cargo culting Y.

</comment></comment></comment></comment></comment><comment><USER15> I stopped reading here:

It's fine for the algorithm to have separate logical parts, but in a function this is a violation of the single responsibility principle.

If it's contained inside the algorithm, and never done elsewhere, then it's still following the SRP to have the algorithm as a single function. This is just someone who learned a new set of principles and over applies them.

<comment><USER4> The SRP is a bit fuzzy when it comes to algorithms, though. What counts as "one thing"? Bob Martin covers this in Clean Code. "If a function does only those steps which are one level of abstraction below the stated name of the function, then the function is doing one thing." SRP falls out of that, as noted by Jeff Atwood([https://blog.codinghorror.com/curlys-law-do-one- thing/](https://blog.codinghorror.com/curlys-law-do-one-thing/)).

The author is still overly vague here; it's not that the algorithm has multiple logical parts, it's that it has those plus the kitchen sink.

</comment></comment><comment><USER16> I sometimes wonder if people like Bob Martin aka "Uncle Bob" and Martin Fowler have done more damage than good with their arbitrary rules. Or maybe people are too blindly following their rules in every aspect of software design.

<comment><USER17> Fowler mostly just writes anthropologically about design patterns he encounters, explaining how to implement them _given that_ you've decided to implement them. He says a lot of "you've got to do X", but in the context of the article it means "evidence suggests that you've got to do X if you're doing Y; otherwise, what you're doing won't be recognized as Y and people will get confused when you call it Y." Fowler's work is to software design-patterns as [https://www.grammarphobia.com/blog](https://www.grammarphobia.com/blog) is to English idioms: someone who provides analyses of what usages are accepted and why, and what usages will get your text a bemused eyebrow-raise.

Robert C. Martin, meanwhile, is certainly paternalistic, but there's a bit of lost context to his writings: they're targeted at the army of "Java school" devs going directly from school to working at the big Java shops of the 90s. His advice actually works quite well if your goal is to bootstrap yourself from an education of "programming" but no "software engineering", directly to being able to leave a large pre-existing Java codebase "better than you found it." (His advice is no substitute for reading the code, absorbing the architecture, and then extending it in natural and intuitive ways—but 90s Java shops rarely gave their devs time to do that before moving them to the next project.)

<comment><USER16> Do not forget Fowlers twitter influences [https://twitter.com/martinfowler/status/893100444507144192?l...](https://twitter.com/martinfowler/status/893100444507144192?lang=en)

</comment></comment></comment><comment><USER18> The traversal example is a bit weird. There's a simple solution to what the author wanted. Replace `print(root.value)` with `yield root.value`. Now you've got a simple algorithm where the caller can decide what to do with the values. And it nicely resembles every kind of iterating things in python. Instead, the object idea makes the algorithm specific to one single use case.

Same with the second example: You can write this as a filter on a function which reads the data. You get a generic reader and a generic filter which can be tested separately.

Twisting the code to fit it into the strategy pattern and preserving some hidden state just to make that possible seems silly to me. Is the class really worth writing instead of composable / reusable / testable / streamable:

        def data_reader(fin):
            fin.readline()
            for line in fin:
                yield [int(val) for val in line.split()]

        def fill_zero(source):
            last_day = 0
            for next_day, value in source:
                while last_day < next_day - 1:
                    yield 0
                    last_day += 1
                yield value
                last_day = next_day

<comment><USER8> As far as the general principle goes, an iterator object is a great example of "algorithms as objects" as opposed to "algorithms as monolithic functions".

<comment><USER18> From the code simplicity POV they're very different though. Sure, an iterator is an object here internally. But continue that way and you'll get to the "objects are closures / closures are objects" discussion.

<comment><USER8> I see the fact that Python has a nice syntax for writing generators as incidental to the main point of "algorithms as objects". In Rust you'd (currently) explicitly write a struct and its next() method, but its still the exact same idea.

</comment></comment></comment></comment><comment><USER19> The start of the article is a very compelling description of why you should split your code into smaller components instead of just having a single structured procedure. Levels of abstraction, testability, etc. - Pretty basic stuff I think.

I don't know where he took the leap to consider objects those components though. You can perfectly expose a function that is internally split without making it public in the public interface. And you certainly don't want to deal with the extra problems of having shared state...

Object-oriented is not an antonym of monolithic. In fact, I'd say they tend to end up being related terms, rather than opposed.

<comment><USER11> The justification for shared state that the author presents is that threading a lot of (mutable) state as parameters through a tree of SRP-obeying internal functions gets unwieldy. Factoring that state into the fields of a class and making those functions methods is, of course, the "obvious" solution when you're thinking inside OOP box.

It is also completely isomorphic to the obvious imperative solution of factoring the state into a struct/record and passing that around. Both approaches have the same problem of being based on mutable state.

The obvious solution to a _functional_ programmer is to make the state immutable and make the functions _return_ the new state instead of modifying it in place. Furthermore, each function should _only_ take the state it actually needs. Of course, this doesn't really help with the unwieldiness, especially in a language without helpers such as the Clojure `->` threading macro, or a reasonable lens library (and lenses themselves are not the most obvious abstraction ever), so I guess YMMV. But getting rid of mutable state is often a win anyway.

<comment><USER20> If state is immutable, then it's constant data, not state.

<comment><USER11> Au contraire! Languages with no mutation at all, like Haskell, still have a well-defined concept of state. If anything, it's _more_ well-defined than in imperative languages based on side effects. I refer you to Rich Hickey's "Values and Change: Clojure's approach to Identity and State" [1].

[1] [https://clojure.org/about/state](https://clojure.org/about/state)

<comment><USER20> Those are both just hiding mutation and copying.

</comment></comment></comment></comment></comment><comment><USER21> Pendulum is slowly turning back to "Everything's an Object". Time to decorate Jabba sections in that CV and remember to drop in an obvious reference to GoF/Fowler/UncleBob writeups.

</comment><comment><USER22> This is a reasonable pattern commonly used to implement recursive-descent parsers, where you have a parsing function for each language construct but they all need to access common state (such as the lexer and the error output). Passing this common state as separate parameters becomes tedious, so it makes sense to group them into a struct.

It's not all that well-motivated in the article, though, since helper functions aren't in themselves a code smell. You can just make them private. It's also not strictly necessary to convert the helper functions into methods on the object holding the state, though this can be convenient.

Also, the state object is an implementation detail and should usually be private. The public API of of the parser can be a pure function.

</comment><comment><USER23> [https://en.wikipedia.org/wiki/Strategy_pattern](https://en.wikipedia.org/wiki/Strategy_pattern)

</comment><comment><USER4> Although the examples are predominantly Python, the approach is useful in any language where functions aren't forced to be in a class.</comment>


The billion-dollar CRISPR patent battle: A case of big money shaping science - <AUTHOR> http://www.latimes.com/business/hiltzik/la-fi-hiltzik-20160206-column.html <comment><USER2> I hope that scientist find an alternative system to CRISPR, perhaps by continuing the research that lead to the discovery of CRISPR --- how bacteria fight off phages. It would reduce the value of the patents on CRISPR.

<comment><USER3> Yes, because god forbid a biologist get rich contributing to humanity.

<comment><USER4> That very much depends on their institution.

</comment></comment></comment><comment><USER5> How long before the Nobel gets awarded?</comment>


What makes a community work? - <AUTHOR>

What makes different online communities like Hacker News, Reddit, Stack Overflow, and others thrive instead of going unnoticed like many others? Not only that, but what makes people want to be a long standing part of the community? <comment><USER2> I would say interaction and a point-system.

For more social oriented places like Reddit, people don't have to stick to one topic but can find a subreddit they like and talk there. They can communicate and share (or endlessly argue) over ideas.

An even bigger reason would be the point system. For all of the sites you mentioned, people earn points and reputation for commenting intelligently or with humor. This is an incentive to stick with a site and rise in the ranks.</comment>