Illiberalism Isn't to Blame for the Death of Good-Faith Debate - <AUTHOR> https://slate.com/news-and-politics/2020/07/illiberalism-cancel-culture-free-speech-internet-ugh.html <comment><USER2> What I notice about the "cancel-culture" debate is that it predominantly seems to define culture as being the sphere of the people who can be cancelled to begin with.

When I look at American discourse from the outside, it seems like liberalism, democracy, and so on are synonymous with the ability of a few high-profile pundits or 'public intellectuals' to express themselves.

I think the root of the problem isn't cancel culture but the limitation of the political to a bunch of celebrities.

I noticed this when in the US the debate about college culture wars started. They exist here as well, but they're a curiousity. Nobody equates tenured academics or a bunch of woke college kids going for each others throat with democracy or liberalism.

In tech people often argue that decentralisation is a good tool against tyranny. This simply is what democratic discourse is supposed to be. If political discourse takes place in every home at the dinner table, in the pub, in political parties and part of ordinary every day life between ordinary people and not boosted on twitter then there cannot be cancel culture because nobody individually is relevant or public enough to be cancelled.

If however political discourse is the job of the professional commentariat and the rest of society is some sort of passive mob waiting to be activated, sure then you get cancel culture. If however politics consists of people staring at TV debates for months or years and you treat politics like a season of survivor with the citizen being reduced to the role of some consumer, then why is anyone surprised by the result.

<comment><USER3> The more pundits change, the more they stay the same. For all the benefits of the internet, and perhaps a rise of alt-voices in the twittersphere, the talking heads on the tv all seem to be the same old people.

It should be clear from history these people aren't there for their accuracy but because they're part of an established narrative and they're "safe" for their markets.

Even basic contrarians really aren't heard from within a market anymore. An example for me would be Chomsky. In the 80's or 90's, you could rarely see him on the "liberal" networks, but he got cancelled long ago by them. We are "free" to see him on foreign media or RT though, but no American would bother.

It is interesting to see foreign networks like the BBC or NHK covering US news - the talking heads they use aren't really high-profile in the US so I wonder how they're chosen.

</comment></comment><comment><USER4> A good sign of how intellectually bankrupt the pro-"cancel" argument is that almost every single defense I've read grossly misunderstands the complaints.

> Have you met the internet? Chilled speech isn't new. Members of marginalized > groups online have from the start dealt with threats, insults, and > harassment campaigns for the crime of articulating their ideas in public. > But free speech defenders didn't sound the alarm about the marketplace of > ideas then. I'm not sure what's changed.

Free speech defenders don't complain about "non-marginalized" groups being hassled by a bunch of random accounts either. And nothing about "cancel culture" complaints preclude tamping down on abuse and harassment; they're barely even related. In fact, a common criticism of those complaining about cancel culture is that they _overemphasize_ politeness and civility, while allowing bad (and "bad") ideas to be expressed (the "pro cancel culture" habit, OTOH, seems to be about sticking to the a narrow range of approved views while acting as vile as possible).

What free speech defenders _absolutely_ did do is defend the rights marginalized groups to say their piece; the rights revolutions of the last half century+ have all grown out of the foundation of a society where people fight for the right of unpopular speech to be heard.

The complaint is about extending the consequences of political speech to every possible unrelated sphere of life, to letting those with the most cultural power codify their biases as the only acceptable speech, and in some cases to cutting off access to the modern public squares: "go build your own social network" does carry some weight as an argument, but by the time you get to "go build your own payment processor", you're making a mockery of the concept of a society tolerant of pluralism. The fact that right now the views protected by liberal values are rightwing is incidental: the purpose of these rights is to defend the cultural underdog, and diminishing them just hurts the marginalized in the longterm. This is something the left once knew: the ACLU's historical defense of the KKK/WBC et al has been rooted in a defense of these _rights_, not a defense of the groups.

There is probably a well-reasoned argument out there for cancel culture, and I don't consider myself to be informed on a topic until I've heard a compelling argument from both sides. But this article, like most of what I've heard in defense of cancel culture so far, is pretty garbage.

<comment><USER5> > Free speech defenders don't complain about "non-marginalized" groups being > hassled by a bunch of random accounts either. And nothing about "cancel > culture" complaints preclude tamping down on abuse and harassment; they're > barely even related. In fact, a common criticism of those complaining about > cancel culture is that they _overemphasize_ politeness and civility, while > allowing bad (and "bad") ideas to be expressed (the "pro cancel culture" > habit, OTOH, seems to be about sticking to the a narrow range of approved > views while acting as vile as possible).

While I tend towards the side of free speech myself, I think TFA is treating Reddit shutting down certain subreddits as equivalent to marginalized groups being threatened in terms of its chilling effect on speech.

This has been a common argument of the "new left" (is there a better term?); an environment of ostensibly free speech is not actually free if some groups are made too afraid to speak by the speech of others.

Through that lens, I think the rough thrust of the article is "Free speech on the internet has failed; your choice now is whether allowable speech should be dictated by the moderators or the trolls?"

Context, therefore, matters. If someone was lynched by KKK members last week, then a peaceful KKK march can be threatening by context without any harrasing or threatening language.

I'm not totally convinced by these arguments, but I think that's the general thrust.

As an aside, the fact that the "public squares" of the internet are privately owned certainly complicate matters; the ACLU certainly wasn't going to force private businesses to hang up KKK banners.</comment></comment>


Predictions for the world of January 1, 2020 (2010) - <AUTHOR> http://paulbuchheit.blogspot.com/2010/01/10-predictions-for-world-of-january-1.html <comment><USER2> Great predictions with a very thorough explanation and high degree of accuracy. All except for 4 and 10 are correct for a prediction that was from 2010.

I'd like to get hold of this guy's forecast on 2030 given that it looks like this guy has a functioning crystal ball.

</comment><comment><USER3> > There will be an even bigger economic crash. The system becomes increasingly > unstable as we try to paper-over the damage from one bubble by creating an > even bigger bubble elsewhere. Nevertheless, things continue semi-working for > some reason.

The "things continue semi-working for some reason" is the part that got me, since that seems to be exactly what's happening. And the answer seems to generally be "for some reason".

> The drug war will be over, mostly.

Unfortunately this hasn't happened, but I hope we're starting to trend in that direction.

> Political outsiders will begin winning elections. Steven Colbert will win > the 2016 election

Only half right. :'(

</comment><comment><AUTHOR> HN thread from 2010, when this was first posted: [https://news.ycombinator.com/item?id=1027916](https://news.ycombinator.com/item?id=1027916)

</comment><comment><USER4> Prescient! Does he have 2030 predictions?

<comment><USER3> [https://twitter.com/paultoo/status/1263941612876394496](https://twitter.com/paultoo/status/1263941612876394496)

> I actually spent some time thinking about it last year and then decided > against. This decade is crazier than the previous. Creating the future is > more important than predicting it, especially in times like this.</comment></comment>


Calling function from one language to other - <AUTHOR> http://stackoverflow.com/q/18364818/71910

<comment><USER2> First you should spend some time to understand how code runs on a computer. I'm talking about the process and thread model.

If you're in python, for example, python the executable is typically compiled code, running as a process. It reads your code as input and based on that, operates the system. It's able to call already compiled/linked C code. Many of python's data structures are implemented in C. It's not possible to dynamically run javascript code using the python executable itself.

You need to have some process running some program which can interpret/compile/run your other code. That could be the same process from your native code, or a different one. If it's a different one, you can use some form of interprocess communication to call/retrieve the output of the execution of the other code.</comment>


Most Americans Are Scared of Self-Driving Cars - <AUTHOR> http://www.popularmechanics.com/cars/technology/a25637/most-americans-are-too-afraid-to-ride-in-self-driving-cars/ <comment><USER2> Most Americans are also scared of planes. Do we have to ban planes until they feel safe?</comment>


Angels Get Carry for Helping a Startup Raise Money With AngelList Syndicates - <AUTHOR> http://techcrunch.com/2013/07/27/angellist-syndicates-lets-angels-get-carry-for-helping-a-startup-raise-money/ <comment><USER2> This only works for liquidity events.

Think about it.

No management fees are being charged; only performance fees.

I'm not in the VC business, but there are a lot of investors who are going to be very, very hurt in the next couple years once consumer technology valuations crash. On the other hand, this move may let the party last a bit longer.

Too many people are chasing companies believing the investment will lead to a 10x or 100x return. But, investing in unprofitable businesses is quite different than publicly traded investments and private equity investments.

<comment><USER3> In that sense, it puts the syndicate organizers in the same boat as the investors. As you point out, VCs can get quite comfortable off management fees without making investors a dime.

<comment><USER2> Agreed

I am in the hedge fund industry, so I see this first hand.

The media loves the "2&20" phrase, but fund managers now proffer lower and lower management fees in order to close funds.

Most of the top hedge funds charge 1.50% today, and newer funds bear lower management fees.

<comment><USER4> Agree. I've even seen some top funds with lower management fees, but it is very very tough to get in as an LP.

</comment></comment></comment></comment><comment><USER5> The short version-

A lead angel investor gets a commercial bonus for connecting other investors to an early-stage startup fund.

My reaction-

Not all exchanges of value have a direct commercial value associated to them. After we've torn all of the middle-men down with technology, we're now slowly bringing them back into the mix.

</comment><comment><USER6> This is disruptively (subversively?) awesome, not just for its initial incarnation but all the ways it could iteratively evolve.

Right now it's like a per-person "0 [fees] and 20 [carry]". Will AngelList try to enforce a 20% carry standard or let it vary? [UPDATE: I see a tweet by @naval confirming angel chooses carry rate.] How about the 0%?

Does the answer change if and when syndicates-of-syndicates arrive? Is that powerful or too much like an MLM-downstream?

Will loose affiliations of syndicate-leads cross-syndicate (simulating a mini- VC-firm)? What kinds of support services will syndicate-leads need (and perhaps share)?

</comment><comment><USER7> Although HN reaction to AngelList syndicates has so far varied from negative to blah, I think this could be big for startups outside of Silicon Valley. For early stage startups in e.g. Europe, it's quite significant investment of time and money to raise an international seed round and having an easier way to organize international syndictate rounds can be a game changer.</comment>


Just Mercy is $0 to rent on Amazon, iTunes, Google Play during the month of June - <AUTHOR> Amazon - https://www.amazon.com/Just-Mercy-Michael-B-Jordan/dp/B082YJ8THX
iTunes - https://itunes.apple.com/us/movie/just-mercy/id1490719646
Google Play - https://play.google.com/store/movies/details/Just_Mercy?id=4ONIbgWYcT8.P&hl=en_US <comment><USER2> Thanks! I will check it out</comment>


Alacritty Terminal Version 0.3.0 Release - <AUTHOR> <comment><AUTHOR> I'm one of the Alacritty maintainers and Version 0.3.0 has just been released, since this is a slightly bigger release I've decided to write up what has changed for people who haven't been following the project.

If you have any questions, please let me know. I'm happy to answer all of them.

The source of the project can be found here: [https://github.com/jwilm/alacritty](https://github.com/jwilm/alacritty)

<comment><USER2> I've used Alacritty quite often, and I particularly like font and color rendering. It does respect my Xdefaults, whereas XTerm or URxvt don't. It simply looks much better.

Now, playing a bit of devils advocate, why shall I abandon either of those in favour of Alacritty? XTerm, in particular, has much lower memory usage and significantly lower latency.

<comment><AUTHOR> If you want the lowest possible memory usage, Alacritty will likely never be for you. Initializing the OpenGL drivers alone ramps up the memory usage to more than what some other terminal emulators operate at, making it impossible for Alacritty to match that.

However, there are other advantages that I personally am really fond of (after all I myself joined the project long after it was started).

For many the incredible throughput is probably a big plus. And I have to admit it's just fun to play around with. Using ascii to play videos at almost HD quality.

But the reason why I use it myself is much more subtle, it's mainly about the little things. So it's hard to put a finger on 'the one thing' why I enjoy it.

It has sane defaults and is much simpler to configure than something like URxvt. It is lighter than VTE and doesn't pull in as many dependencies. It has all the right features that I value which is often much more a preference than something easily objectifyable. It's also easy for me to hack on because even before I joined upstream was very cooperative.

In general I think it really depends on what you value so my personal opinion might be of little value to you or anyone else. So really the only way I see to tell if you are a fan of it or not is to just give it a shot.

<comment><USER2> Thanks for the detailed writeup! You are certainly right with regards to memory usage and ease of configuration. Keep up the good work.

</comment></comment></comment></comment><comment><USER3> I really do love what Alacritty is doing. I've been using it as my principal TTY for a few weeks now, and it has been a joy to use.

Thank you for all the hard work. It might go unsaid sometimes, but it is not unnoticed!

<comment><AUTHOR> > Thank you for all the hard work. It might go unsaid sometimes, but it is not > unnoticed!

Thanks for going out of your way to say that! It's incredibly encouraging to see people enjoy Alacritty and hopefully that will drive many others to join and make it even better than it is now!

</comment></comment><comment><USER4> I've been keeping an eye on alacritty and sort of expect to switch at some point, especially if I begin using Wayland. I've got urxvt configured the way I like it but it can be annoying that default packaged builds are often configured for 88 rather than 256 colours. At the moment, I'd miss the search plugin for searching the scrollback buffer (though I'd prefer if it'd just let me use less) and the single process daemon mode of urxvt.

<comment><AUTHOR> A search feature is definitely planned and I feel like that would be a great addition to Alacritty.

Daemon mode has also been proposed, however there's only limited advantages Alacritty would gain from that. However it does seem like it's necessary to get proper support for macOS for example, so it will likely come too.</comment></comment>


FlexiColorPicker - No flash, images, external libraries, CSS or 1px divs. - <AUTHOR> http://www.daviddurman.com/flexi-color-picker/

<comment><USER2> To become fully usable, it needs to support "dragging" (i.e.: listening on the mousedown + mousemove events) and not simply click.

<comment><USER3> Was about to post the same thing. I'll be needing a colour picker for a Web app soon, but I can't use this one; a good UX pretty much demands that the controls (whether it's the thumb on the dial, or the dot in the field) immediately respond to user input, not after mouse/touch up.

<comment><USER4> It's been worked on. Next version will have those features included.

</comment></comment></comment><comment><USER5> It seems to break for me when using firefox 5.0 on linux. If you scroll down the page, the sliders and picker don't work correctly and display further down the page.

</comment><comment><USER6> Here's one I made, slightly different concept though. <http://jsdo.it/minikomi/RGBslider/fullscreen> Should work with iphone/ipad too.. no guarantee on android!

</comment><comment><USER7> Minor nitpick, there is visible banding. It would look better with dithering applied. <http://en.wikipedia.org/wiki/Colour_banding>

<comment><USER8> It appears it's using SVG gradients, so the banding depends on how your browser is rendering it.

</comment></comment><comment><USER9> This is the point where the browser should take provide the operating system color picker and the os vendor is responsible for adding the right functionality.</comment>


What the iPad might have looked like 21 years ago - <AUTHOR> http://www.fastcompany.com/1598501/smart-designs-ipad-circa-1989

<comment><USER2> How could they have failed to mention the Go Tablet?

<http://www.digibarn.com/collections/systems/go/index.html>

<http://en.wikipedia.org/wiki/GO_Corp>.

</comment><comment><USER3> That FastCompany wattermark on the pictures sure was annoying.

Were they afraid that pictures of 20 year old tech might be stolen ? Really ?</comment>


Container Networking Security Issue (CVE-2020-8558) - <AUTHOR> https://aws.amazon.com/security/security-bulletins/AWS-2020-002/ <comment><USER2> If you use ECS/EKS:

 _Customer action is required, and steps for immediate mitigation are available at_

[https://github.com/aws/containers- roadmap/issues/976](https://github.com/aws/containers-roadmap/issues/976)

 _All Amazon ECS and Amazon EKS customers should update to the latest AMI._

<comment> yjftsjthsd-h However:

> This issue presents no risk of cross-account data access. Processes within a > container on one host may be able to gain unintended network access to other > containers on that same host or on other hosts within the same VPC and > subnet.

It doesn't sound too bad, especially if you're using subnets to separate things already.</comment></comment>


Show HN: Kweb – a Kotlin web framework that eliminates the server-browser divide - <AUTHOR> http://kweb.io/ <comment><USER3> So, how does this work?

Is this just a DOM implementation that writes out the serialised form, or does it write our JavaScript to render whatever you've set, client side?

<comment><AUTHOR> It's the latter. It establishes a WebSocket connection and issues JavaScript commands to execute from the server. This connection remains persistently to allow bi-directional communication between client and server.

This occurs "behind the scenes", the developer will rarely need to think about it.

</comment></comment><comment><AUTHOR> This is my project, would love feedback, not just on the idea itself - <USER2> also how I go about explaining it on the site.</comment>


Coder? Programmer? Software engineer? Choosing a job title - <AUTHOR> https://medium.com/@isaaclyman/choosing-a-job-title-for-people-who-code-316a7673a83c <comment><USER2> This was mighty helpful as I prepare my resume for the job search I'm about to commence. I've always been categorized as a Programmer/Analyst and Senior Programmer/Analyst. This fits pretty well since I do everything from the business and requirements analysis all the way up to design and implementation of projects within both companies I've worked for. This article doesn't really address this scenario too much and it looks like maybe I should pick a different title to fit this? Resume polishing here I come.

<comment><AUTHOR> Author here. I don't actually know what "analyst" means in the context of software. Is anyone "just" an analyst? Is it a well-defined role? I didn't come across the term in my research, so I wonder how many people are familiar with it.

<comment><USER2> Well I think it is meant to imply Business Analyst since that is pretty much what I have done in both positions so far. I am analyzing the key KPI's that need to be measured, or the importance in Time/Money savings a particular project will have and then do the entirety of implementing that solution. There are times when I suggest a project and can sell it to management based on some savings analysis that I have that is not being done otherwise by them. I'm like part consultant and part developer. I like the business aspects a whole lot, so it works for me.</comment></comment></comment>


Calculus using Racket in 10 minutes - <AUTHOR> http://markfayngersh.com/post/1062137870/calculus-using-racket-in-10-minutes

<comment><USER2> It seems like writing a symbolic differentiation routine is the new FizzBuzz, which is JUST FINE BY ME.</comment>


How to implement logic gates using crabs - <AUTHOR> http://arxiv.org/abs/1204.1749 <comment><USER2> There's an overview of the paper in the item I submitted a couple of hours ago:

<http://news.ycombinator.com/item?id=3831987></comment>


Fleksy Keyboard SDK for iOS now free for all devs - <AUTHOR> http://techcrunch.com/2014/02/20/fleksy-opens-its-ios-sdk-for-all-developers-requires-zero-lines-of-code-to-implement/

<comment><AUTHOR> Hey guys, Kosta from Fleksy here.

We have just released our new SDK and wanted the feedback of the iOS developer community. We have managed to make this a ZERO (literally) lines of code implementation, as well as making the user experience of alternative keyboards similar to other platforms.

As developers, how do you think we can make this even easier and eliminate friction that is within our control?

<comment><USER2> Please remove the reliance of the external app, or make it optional.</comment></comment>


Flow, a typeface built for wireframing - <AUTHOR> http://danross.co/flow/ <comment><AUTHOR> Hey everyone, I'm Dan.

I'm a designer from Australia, now living in Canada. Design tools are my passion, and I wanted to share my latest project with you.

Flow is a typeface built for wireframing. The font comes in three weights – circular, rounded and block. It's a little project I've been working on, and I'd love for you to take a look.

Designing with real content is important, but sometimes we need something more abstract. Flow aims to provide an efficient and flexible way to create abstracted/placeholder content.</comment>


Amazon takes down a five-star fraud in the UK - <AUTHOR> https://www.theverge.com/2020/9/4/21423429/amazon-top-reviewers-uk-fraud <comment><USER2> And this is why I always look for the bad reviews, or at least, those reviews that expose some flaw. Nothing's perfect.</comment>


Pale Moon Official Branding Violation – Issue #86 – Jasperla/openbsd-wip - <AUTHOR> https://github.com/jasperla/openbsd-wip/issues/86 <comment><USER2> How bizarre. I can't say I've ever used Pale Moon or particularly care about it, but the behavior in that thread would have had me eject and find something different pretty much immediately.

Starting off with such forceful language and without a hint of wanting to deal with the issue in an amicable manner is nuts. The first guy comes across as an asshole, and no one wants to deal with assholes - it's perfectly reasonable to ask to talk with the rights holder and hope they're more reasonable.

Unfortunately, it turns out the rights holder isn't any more reasonable. An immediate 'I will not be as educational next time' is a clear threat, even though he tries to backpedal away from it later.

And then we find out that the person working on the port approached the Palemoon people days ago asking about the proper way to do things, etc.

So, in the end, we have a WIP repo that does not in any way show up in the official ports tree for OpenBSD, with someone trying to do things the Right Way, getting pushed around by two people being assholes.

Now, for the technical merit: Forcing people to use special bundled libraries instead of system libraries is dumb in the vast majority of cases, especially in something that will be running untrusted code as a basic function. Like, say, a web browser. It significantly increases the amount of work required to make sure you are running a system protected against known vulnerabilities. The reasoning seems to be that whatever they're doing in the browser seems to not play nice with the upstream libs, so they've instead started doing god know's what with libs they now bundle with it. Why don't the fix the issues in their own project? No idea. If they're actually flaws with the libs, why not submit upstream patches? Not only do you have to worry about them making sure they are quickly backporting any security related fixes, you also have to worry that whatever modifications they've made might open up new security bugs, or potentially nullify fixes from upstream.

The whole MPL style branding nonsense is also bizarre and something I think is incredibly counter to the spirit of the open source software movement. It was silly with Firefox and Iceweasel, it's silly now.

Questionable technical decisions, questionable behavior, questionable commitment to the spirit of open source software.

I'll pass.

<comment><USER3> "but the behavior in that thread would have had me eject and find something different pretty much immediately."

Ditto. And I speak as a PaleMoon user as it's the browser of choice on all my machines. They could be right on principle, but the OP had no right to act as an ass that way.

(edit: posted by mistake on top thread)

<comment><USER4> So, the question is, have you followed through on that feeling and dumped PaleMoon as a result? The only way they feel the pain of their behavior is if users leave due to that behavior.

<comment><USER3> After reading the whole threads both on github and Pale Moon forums, I believe the best option is to migrate everything back to Firefox and dump it. Unless they either sincerely and publicly apologize for being so rude or kick that guy off the project. But I'm not holding my breath for that.

<comment><USER5> Unless I'm mistaken, that guy is the head of the project.

I dumped Pale Moon a while ago because it was generally terrible, and have had better luck with Waterfox.

<comment><USER3> Never heard of Waterfox, from the features looks interesting, thanks for mentioning it.

</comment></comment></comment></comment></comment></comment><comment><USER6> It's now also reviewed for removal from FreeBSD:

[https://lists.freebsd.org/pipermail/freebsd- ports/2018-Febru...](https://lists.freebsd.org/pipermail/freebsd- ports/2018-February/112455.html)

<comment><AUTHOR> This message is a pretty good summary of why the shared library issue is important to the BSDs and might make one pause about using Pale Moon. [https://lists.freebsd.org/pipermail/freebsd- ports/2018-Febru...](https://lists.freebsd.org/pipermail/freebsd- ports/2018-February/112459.html)

</comment></comment><comment><USER7> Well, that escalated quickly. However, this was handled quite poorly.

Moreover, there are not that many choices of decent browsers. Mozilla has their problems, Chrome/Chromium is a google project. It even gets worse when you think of how few browser engines there are.

</comment><comment><USER8> This was a great read. Just came here to say that.

</comment><comment><USER9> Does anyone have an IPFS mirror yet?

<comment><USER8> Hosted where?

</comment></comment><comment><USER10> I wonder what the official Mozilla Firefox upstream does.

</comment><comment><USER11> In discussions about having more gender parity in programming, I sometimes hear people ask what benefits, specifically, it would provide to the projects themselves. Here's one possible example: if we had a more equitable gender ratio in open-source, we might have have fewer dick-waving contests like that thread. I don't even know what it's about, and I don't care: all the involved participants are just peacocking Internet Tough Guys. What a waste of time.

<comment><USER12> I'm not sure what gender has to do with that thread

From my experience. Being an asshole is not a gendered trait

<comment><USER11> > Being an asshole is not a gendered trait

When considering a sufficiently zoomed-out view of all the different ways to be an asshole, I think you're right. But the particular form of asshole- ishness in this thread– swaggering in and immediately starting with demands ("You will revise your mozconfig..."), responding with passive-aggressive logical handwaving about how the person will never be listened to again ("you are dismissed from all further conversation on this matter...")– _is_ strongly gendered, in my experience, and it's all too common in the open-source world.

The Atwood/Gruber Markdown spat, the Debian/XScreenSaver version warning, and the Bootstrap/JSMin semicolon debacle are all good previous examples of the same dynamic here. In all cases, there were no or very minor actual technical disagreements; all the sound and fury was just men digging in and refusing to give an inch of ground to another person who had very good points, while being as aggressive as possible to guarantee a real resolution couldn't be reached. We could use a lot less of that.

<comment><USER12> Again. I think you're drawing conclusions where there are none.

For the sake of better fitting your examples. I'll say ego isn't a gendered trait. Versus describing someone as an asshole

Having a massive ego is just a thing that some people have. It's human, and humans are flawed individuals.

Yes. There's a lot more men in computing. So there's a lot more examples of ego clashing from "men". But I'd suspect the dynamic not to change much if it was a true 50/50 split. Or even a complete reversal of the current gender bias (something like 90% men I'm told? it's hard to get accurate statistics)

For a specific citation against this theory (though I try to avoid mailing list fights). I'd point to the Libreboot project. Where the lead (and possibly only?) developer went on a personal crusade against the Free Software Foundation. Using the project as a spearhead to personally demean them for a perceived attack against someone else</comment></comment></comment></comment>


Why do so few people major in computer science? (2017) - <AUTHOR> https://danwang.co/why-so-few-computer-science-majors/ <comment><USER3> I don't think CS is a high social status field _at all_. You're deluding yourselves. Outside of our peers also in the industry nobody cares that you're a staff engineer at Google. Absolutely nobody. They'll assume you're doing IT work like the characters in the IT Crowd if they even bother to think about it at all and haven't already walked away.

Here's a concrete example to make it really obvious.

How many computer scientists are there in the Lords in the UK? I'm not sure there's any. There are nearly 800 lawyers, doctors, religious ministers, biologists, physicist, mathematicians, philosophers, business people, politicians, authors, composers. A computer scientist who defines the field for half a century is lucky to get knight bachelor.

Look at similar establishment institutions elsewhere. Are there any computer scientists in the Senate in the US? Are computer scientists often invited to lead major public bodies? How many computer scientists become deans of universities compared to other fields?

The social status of computer scientists is _zero_.

<comment><USER4> > I don't think CS is a high social status field at all.

I totally agree with you. I believe this is one of the factors why women hesitate to enter the field. They are more socially aware than we are.

Also, even in IT areas, technical capable people are relegated. We get good salaries but we don't get fancy titles. We are the only profession that allows to be managed by outsiders. Financial firms(and finance departments) are always headed by a experts in the field, same with lawyers, architects and finance. But we are managed by people that are not experts.

<comment><USER5> Please don't assume everyone on HN is male.

<comment><USER4> > They are more socially aware than we are

Sorry for the confusion. We as in "We men" not as in "We HN readers". I was just speaking as a man.

</comment></comment></comment></comment><comment><USER6> The problem is CS is just a miserable field to be in unless you really, really enjoy the grind of it.

To me it comes down to some CS-specific version of Mike Tyson's (perhaps apocryphal) quote "everyone has a plan until they get punched in the mouth" \- something along the lines of "Everyone has fun programming until they have to start debugging".

My city has several years of programming courses at the high school. A lot of kids sign up for the first year but the numbers rapidly fall off. They start the kids off with the basics and doing simple programming exercises. Generally they have fun while they are being successful getting boxes and lines to appear on the screen, but as soon as things get the slightest bit complicated and the students get into the weeds of debugging the enthusiasm dissipates pretty quickly.

It's not that most of these kids don't have the intelligence to tackle it if they wanted to; they personally just don't find it fun at all.

<comment><USER7> This

I've been working has a web developer for two years after completing a bootcamp. Peoples often ask me questions because they think about doing the same. I always say to them that this is not for everbody, you are banging your head against the wall most of the time. Off course nothing beat that sensation when you finally found the problem or even when you get closer.

But that has to be your thing.

<comment><USER8> > I always say to them that this is not for everbody, you are banging your > head against the wall most of the time

This is what I tell people who want to get into bench science. It's funny, because I find programming to be an escape from the 'head-banging' of science. At least when I get stuck debugging a programming issue, I know that there _is_ a fix, and I'll eventually find it. Compare that to science, where you might bang your head against the wall for weeks, months, or years, only to later find out that the problem simply isn't tractable, or that the problem had nothing to do with you at all.

<comment><USER9> Not all CS algorithmic or systems design problems, you know there _is_ a fix.

</comment></comment></comment></comment><comment><USER10> Can we talk about how much more CS people are expected to work than people in other fields ?

For most people here, CS is their hobby. So a lot of learning and knowledge acquisition happens during 'leisure' time. This is rarely the case in other professions.

If we count all the time we spend resolving environments, getting setups right, learning new tools and reading papers as work (as we rightly should), then most people in tech would be working far more than 55 hours/week.

This already ignores a lot of the quiet time people spend pondering over things and letting them stew semi-consciously. CS work is flexible, but is not easy or less-time consuming by any means.

If we try to force people into CS, then we will get what India has. A massive glut of incompetent tech 'talent' with BS in CS, but need to be retrained all over-again to even do the most mundane sweat shop coding that companies like Infosys and TCS need their employees to do. (I say this as an Indian). Millions who hate their job, earn low pay and have a very small set of nontransferable skills, because what they actually know is 'sweat shop tools' and not CS. IMO, if there is any demographic that AI/ML is most poised for eradicating, it is likely this one.

If CS in the US wants to increase participation without compromising on its present identity, then the only solution might be to foster interest and love for science/math/coding early in life and hope you can cast a wider net to capture everyone who would come to consider this profession a hobby.

Alternatively, it can go the way of every other mature profession and turn into a 9-5 boring thing you hate, but still continue doing because you need to put food on the table.

<comment><USER11> I know plenty of contractors who improve their own homes in their spare time, artists who paint in their spare time, writers with side projects, teachers who do planning/look for lessons, etc. etc.

That said, I think we'd all be better off if engineers demanded collectively not to work more than 40 hours. (Tech unions/guilds could be formed and focused just on stuff like limiting hours/paying for overtime and oncall, btw.)

I've done it individually, and it has never slowed my career progression -- even when I worked at Amazon, I was adamant that my family and personal time came first, but I still was promoted regularly.

I still code in my spare time, but it's totally unrelated to work.

<comment><USER12> In general, the less hours everyone works, the greater the advantage you have in working more. By just working 10 extra hours, I can be 25% more productive (there's a lot of simplying assumptions there, of course). This is why the 4 day work week never took off: I can just work one more day and realize a 25% boost in output, a major boon to my career.

Developers don't have much in common with the workers who originally created unions. Some of the differences are that work can be done at home, the potential for advancement and greater salaries is very high, and that the non- physical nature of the work allows you to work very long hours. All of these factors conspire to undermine any 40 hour max set by a developer union.

As average hours worked go up, the benefit of each extra hour goes down. I'm not sure where exactly the equilibrium is, but personally, I find that over the long term, over 60 hours per week leaves me feeling a little shitty and depressed, especially if I have a commute. I think most over can boost up to 80 and be fine for a couple months though.

If there's one profession where people are expected to work more than devs it's finance. New analysts in finance are pulling 100-120 For at least their first. I've done those kind of hours for maybe one week, but it kills you, both physically and mentally.

<comment><USER10> > most over can boost up to 80 and be fine for a couple months though

Staying alone and being a well put together human with relationships becomes completely impossible the second I go past 55-ish hours.

> people are expected to work more than devs it's finance

Finance is toxic at a whole another level. The rampant smoking, cocaine use, work hard/play hard etc. is in some sense mirrored in similarly intensive silicon valley startups where aderall abuse is just as rampant.

I am also not sure if the finance analytics work is as creatively taxing as coding (at least the design stage), which can be a major distinction.

Either ways, I hope CS never becomes like finance, ever.

</comment></comment></comment></comment><comment><USER13> I've always seen the same pattern in College decision: self-projection. That is, a combination of _what they know_ and _what they think they can do /like_.

Most people choosing Uni choose from what they know. Majority of young people know little about _anything_ CS or nerdy; They often know the end product (computers, phones, apps) but very little about the people working on it.

Compare that to doctors, managers, Wallstreet types, scientist, lawyers; they got incomparable exposure through movies, tv-shows and media in general. Even Scientist have more exposure.

Difference is that in those profession, you can see the people working in it (although often not an accurate representation) so it is easy for a young person to imagine "I can see myself wearing their suits, doing what they do". But just as no one applying for college imagine themselves in a janitors boots, there must be a observable social status in the profession too. "you will be socially respectable after 4 years here!" sells better than a salary figure. I really don't think CS majors come close to the (perceived) social status for the young, compared to the above mentioned professions.

The closet to high social status an CS gets to is the "IT/tech entrepreneur", and you don't need a CS major to get there.

<comment><USER14> CS types in the movies are basically magicians. Something needs to happen, they open a laptop, they type away furiously, there's a cutaway which shows them frowning, then a red dialog box on the screen turns green, and the thing happens.

It's hard to get from that to the reality developing CRUD apps.

There may be some drama, but arguments about which framework to use are not usually very cinematic.

<comment><USER15> I think part of this is the limitations of interest that come from a film. You can write a whole movie about an electrical engineer building robots and visually show the struggles along the way. If you compare that to a software engineer searching for another fucking memory leak or something then it's just not going to be as visually interesting.

An interesting thing about this is that I think it's easier to show in print. I can read a book about a maverick developer and have it describe the errors and the trials and tribulations and it isn't as slow by comparison. Hollywood then feels the need to emphasize what they show with wild CGI, which is a whole other issue when it comes to entertaining portrayals. I think the only realistic portrayal I've seen onscreen is Mr. Robot, and even that goes into fiction when it's most needed for the plot.

<comment><USER16> Valgeind and vim are both pretty neat looking IMO. Showing the real tools occasionally is something I've enjoyed seeing in some recent Sci-Fi movies. Tron Legacy and Chappie are both examples and even though the end result was unrealistic I thought they did a neat job showing the engineer working for years tediously on the project.

<comment><USER15> Tron did do a decent job, it was cool to see that. At least having a basis in reality goes a long way to making the the 'fi' part believable, if only for those that recognize the basis in current or previous technology.

</comment></comment></comment></comment></comment><comment><USER17> A few observations I have seen(US Based):

\- In college there were basically no women in any of my classes, like 1 or 2 in a class of 30-40. Not having an entire gender being interested in a field(generally speaking) makes numbers overall very low.

\- In a Java class I took in college, it was standing room only for the first two sessions, after 3 weeks the class was 70% reduced, by the end of the class it was about 25% of its original size. CS is hard, lets be honest(for most people), to excel you have to love it or be mathematically inclined.

\- People, when looking at a career don't see CS as long term choice, they hear of burnout due to extreme overwork, and blatant ageism when you hit 40's. I asked a friend who went into the medical field about CS and they said they want to work into their 60's-70's and didn't want to be forced out of work due to age bias.

\- Most engineers don't have much 'clout' in a organization that isn't a pure tech company, I have been solicited by GS and other hedge funds but always pass as I know I will always be 2nd fiddle to finance folks/business majors.

<comment><USER3> > In a Java class I took in college, it was standing room only for the first > two sessions, after 3 weeks the class was 70% reduced, by the end of the > class it was about 25% of its original size. CS is hard, lets be honest(for > most people), to excel you have to love it or be mathematically inclined.

I don't understand how a college survives 75% of their students dropping out like that?

And do they not interview and assess people going in for aptitude?

<comment><USER18> At my school, generally you would enroll in ~3-5 classes per semester. One or two of those classes would be your "main focus" for the semester, and then you might try to fill a breadth or take something that interests you. You would have a week or two at the beginning before your schedule is fixed to move around classes. So you're not dropping out - you're swapping an Intro to Programming class for something you think you'd get more value out of. I've dropped classes ~45 minutes in to the first lecture because I just... didn't need to take it and wasn't enjoying the professor speak.

</comment></comment></comment><comment><USER19> There's one other possible, additional reason.

I recently asked a 17-year-old high school senior who is heading to college what she's planning to study, and she said it would be mathematics, biomedical engineering, or some other kind of engineering. She's self-motivated -- says she will be studying multi-variate calculus, PDEs, and abstract algebra on her own this summer. She maxed out her high school math curriculum, which included linear algebra as an elective.

Naturally, I asked her about computer science, and she said something like this (paraphrasing):

 _" The kids who love computers at my high school seem to be able to spend their entire day focusing on a computer screen, even on weekends. I cannot do that. And those kids are mostly boys whose social behavior is a little bit on the spectrum."_

While I don't fully agree with her perspective, it makes me wonder how many other talented people shun the field for similar reasons.

<comment><USER20> That pretty much lays out one of the more standard explanations for why more women don't go into CS.

Things arguably changed in the early 80s with the rise of PCs and gaming. While programming certainly existed before that, it wasn't something that most kids were exposed to in high school and CS curricula in general were heavily theory oriented and often in the Math department.

Today, if you want to be a mechanical engineering major there's no expectation that you lived in a machine shop all through high school or otherwise spent all your spare time in hobbies related to your college major.

Whereas, a lot of CS curricula today at least implicitly assume that you're comfortable with computer command lines, basic programming, etc. And, if not, you'd better get up to speed in a big hurry. Because most people in your class are up to speed and program for fun. I went through a MOOC from MIT a while back (6.001) and I can't _begin_ to imagine working through that from a baseline of maybe one high school course.

ADDED: And, of course, you periodically see a similar attitude reflected on forums like this one when people argue that they won't hire someone without a GitHub repo who programs for fun nights and weekends.

<comment> radford-neal Could this actually be good, or at least quite understandable? (I'm not actually sure whether you intend your comment to just be descriptive, or also judgemental.)

Does anyone expect college music programs to be easily accessible to people who didn't play an instrument, or sing in a choir, during high school? Similarly for dance, and drama, and art. And if you're right that mechanical engineering schools no longer expect students to have been tinkering around in their spare time previously, is that actually good?

At one time, there was good reason to not assume that incoming students had any programming experience, since many of them wouldn't have had access to computers, and so would not have had the opportunity to develop computer skills, or even to know whether they were interested in computers. But today in western countries, for any but seriously impoverished kids (or I suppose, those with mis-guided parents), that is no longer true.

There remains the possibility that a toxic culture in high school has prevented some students from realizing that they are interested in computers at that point in their lives, and therefore some remedial action in college is necessary. But fixing the toxic high school environment would be preferable.

<comment><USER21> > "There remains the possibility that a toxic culture in high school has > prevented some students from realizing that they are interested in > computers"

I take some issue if the characterization is "the nerds" are the toxic ones here. It's been a while, but the high school culture was overall toxic if you were not attractive and popular, and computer geek types were frequently targets for their interests. Young women were as bad as the males in the shunning.

<comment> radford-neal I didn't mean to imply that the "nerds", or any of the students, were "toxic". It's the culture. Probably starting with the whole concept of segregating teenagers from the rest of society.

<comment><USER22> Interestingly, there was almost no bullying in my high school in Poland in the nineties. I think school bullying may be a product of Anglo-Saxxon culture, which is very individualistic, success oriented (i.e. if you're a "loser" you're barely a human being) and also kind of heartless. Currently, as Poland converges to the standard of global culture emanating from the US, there are reports of some cases of bullying, but still I don't think it's the everyday reality of each school.

<comment><USER23> There was plenty of bullying in my middle school in Russia in the nineties. Hell, there's a 1983 Soviet movie about bullying.

Any kid that was weird and had no "redeeming" qualities (strength or wealth) was likely to be bullied. Don't tell me your class was one big happy family and that you didn't have that one kid that no one cared about.

<comment><USER22> Yep, we had one guy like that, that was kind of weird and thus was unpopular and made fun of by a couple of guys. It was far from the scale of "industrial bullying" that I imagine the US/UK schools are - with regular beatings, putting heads into the toilet etc.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER24> The reason is because it is a terrible job. Unlikely 90% of other jobs, you have to mentally burn your brain out each day, constantly learn new things, and focus intensely, supposedly for 8 hours. The reward is nothing, not even good pay outside of a couple of tech giants in the USA. If you compare it to a trade, there is a night and day difference to your mental health.

The wage data in this article is meaningless because it ignores the fact that the few places with the 100k+ wages are all in ridiculously expensive cities. Those wages bring up the "average", making it seem high.

No sane person dreams of sitting at a computer all day doing harder things than everybody else sitting at the computer and making average pay. Not only that, but most employers would rather pay a person $5 an hour to make their app from somewhere in yemin then they would hire a local person.

You also never get to be part of any kind of inner circle within the company. You will be paid less than some random useless person who "knows somebody high up" and is in charge of "managing" you.

<comment><USER25> >No sane person dreams of sitting at a computer all day doing harder things than everybody else sitting at the computer and making average pay. Not only that, but most employers would rather pay a person $5 an hour to make their app from somewhere in yemin then they would hire a local person.

A bit arrogant to assume that your job is harder than everybody else's - <USER2> people feel that way. And pretty much any office job can be at risk of being outsourced, not just programming.

<comment><USER26> Let me know when a company outsources their Finance department to India...

</comment></comment></comment><comment><USER27> I never ended up finishing my degree because it became trivial and pointless by the time that I got a job and started gaining actual real experience and learning that the college/university teaches you idealistic and fantasy concepts of how IT should operate. Reality I've experienced is that it's absolutely nothing like that, places are build upon decades of mixed generation technology, only upgrading a little part here and there. I've moved on from the dig-in hands-on of IT and now do a strange combination of electrical engineering and network engineering. I was working in IT at an amusement park which I cannot compare to any other business to how it functions because the best comparable description would be to describe it as supporting IT for an entire town and all its businesses. What I ultimately learned and took away from that is a fresh BS graduate is basically worthless in comparison to someone who has at least one single year of real experience because it boils down to the requirement that schools do not teach. That skill is troubleshooting, to be able to swiftly and intelligently resolve a problem. It's not something my schooling ever put any major focus on and so as I gained more and more experience at a vastly faster rate compared to going to school, I realized how worthless the degree really was. It was just a meaningless statement. The second thing is all the certifications, and that's what, I believe, is valued higher than having a degree, but about equal to experience. Search engines have provided me with more knowledge and answers than 8 years in school could ever come close to. That's just my experience and view.

</comment><comment><USER28> FWIW I've been coding since my early teens and enjoyed it a lot, but when it came to university (in 2009) I had very little interest in doing a CS degree and instead opted for Maths. The two main reasons were: a) I knew that doing Maths instead of CS wasn't really going to hinder my job prospects in any way, and b) CS sounded a lot drier and had fewer options and choices than the Maths degree (in the UK you apply for the course and you generally don't do anything from other courses, so your choice matters _a lot_ ).

The first point I think has born out fairly well, even if it was probably a bit arrogant. Certainly when I'm interviewing grads I'm not actually that interested in if they did a CS degree (though that might be more because I didn't do one...). We don't really do early stage training so we're looking for evidence that the grads can actually code, whether its by pair programming, via questions or looking at personal projects, etc, its the people who have done it as a hobby that tend to shine there.

The second point is highly subjective and obviously quite personal, but equally if people know they can get into software engineering without a CS degree then I think they're more likely to do a course that really interests them. After all, if you it doesn't effect your job prospects that much then why wouldn't you? There is a fair argument to be said that the industry should be better at hiring CS and code camp graduates and doing on the job training, but that's not where we're at currently, alas.

If anything I tend to view CS as the academic arm, and software engineering as the practical/vocational arm. In the same way e.g. law works (at least in the UK), where actually most lawyers haven't done law as their first degree and do a conversion course after instead (often getting a contract before doing the conversion cause and then having the firm fund it). Really, its the classic argument about how much university degrees should be academic vs vocational.

<comment><USER29> I think this is the crux of it. What we're doing as software developers is not a scientific discipline, it's engineering. So if you want to be a software developer, computer science is the WRONG field for you.

Computer Science is largely concerned about things like algorithms and complexity, theory of automata, and stuff like that. They're doing research.

Software developers care about those algorithms, but we decidedly do not want to be implementing them. I've got libraries, where somebody already took care of coding the hashing or binary-tree-rebalancing algorithms, or databases with the same. There's really no reason I need to be able to explain how quicksort differs from bubblesort.

But what we DO care about is how to gather requirements, how to perform proper modeling and design, and stuff like that. Yet those are classes in the engineering school, and not required of CompSci majors (at least not back when I was in college).

The result is that folks with a CompSci degree are ill-prepared for a career in software development. They never use at least half of what they were taught, while on the other hand, at least half of what they do wind up needing was never taught to them.

<comment><USER30> I disagree pretty strongly. I wrote lots and lots of practical software as part of my undergrad. And the software I wrote was pretty diverse, including, file system drivers, image recognition, data mining / text classification, exploit utilities, etc. Most of the complex theory has been offloaded to graduate school to make way for "practical applications." Plus, I feel like my education set me up to be a little ahead of the job market, as data science was a track of my CS ten years ago, and you'd have left university with your own little scikit-learn library.

</comment></comment></comment><comment><USER31> I personally think CS is generally taught extremely poorly, specifically any portion where a professor attempts to teach you to code. It generally just comes down to opinion and memorization, its not really fun. I had the experience of having more project based programming classes (like a game or operating system) and that was totally fun, but classes where you have to implement tiny examples of OOP or something are so unbelievably boring (this is what I imagine the cs101 classes are at many schools). If that was my intro to it, I don't think I would have stuck with CS. I also think the theoretical stuff is taught poorly too, it can just be so dry. I think a big issue is you are taught that a concept exists, you don't necessarily go through and try to solve it yourself. I remember my intro to sorting was "here are all the algorithms you should know", not "try to come up with a strat for putting numbers in order." The latter is significantly more interesting and engaging.

</comment><comment><USER32> The author notices that the nr of bachelor degrees went down every year from 2005 until 2014, at which year it went up again. It takes 4 to 5 years from the moment of choosing your study until earning your degree. That means relatively many people chose a CS study in 2000 and also in 2009.

I guess the explanation for 2000 is that the dotcom bubble was still alive and well, the explanation for the popularity of 2009 must have been the credit crisis that made lots of people pursue the job certainty of a CS degree.

<comment><USER33> Their choice of 2005 is definitely the culprit (4 years after the dot-com crash), and it's only gotten more obvious in the years since this post. There's been 60% growth in the number of "Computer and information sciences" (CIS) BAs since then, compared to 20% for Bio-science, 15% for Physical- science, 25% for math, and 40% for Engineering.

[https://nces.ed.gov/programs/digest/d19/tables/dt19_322.10.a...](https://nces.ed.gov/programs/digest/d19/tables/dt19_322.10.asp)

</comment></comment><comment><USER34> Easy. The educational aspect is pretty awful. And after you get through that, the big companies treat you like a leper unless you went through a few chosen schools.

I went to a big state school in the 90s. The first two years were a hazing, with a brutal curve designed to drop class size from 1000+ (CSI 200) to about 80 graduates.

It was sink or swim and awarded grit. The only major worse was biology where organic chemistry weeded out the frat boys from premed.

<comment> non-entity > the big companies treat you like a leper unless you went through a few > chosen schools.

I was under the impression that they didnt care about your background, but you had to slave away on leetcode to have a chance to pass the interviews.

<comment><USER35> I agree with the GP. I had a similar experience at SUNY Buffalo in around the same time.

I terms of working for a FAANG, in order to pass the interview, you have to first get the interview. If you attend Stanford, Berkeley, MIT, etc, FAANG's will have career fair booths, and lots of other opportunity for internships and networking. You will be at the top of the pile and you're almost guaranteed a look.

Contrast that to attending a school like SUNY Buffalo. FAANGs don't generally recruit from schools like that. When I worked for Google, I offered to help with recruiting from SUNY Buffalo (which they called a "long tail" school). Their response was, rather than send people to the school, they set up a giant hangouts/meet video chat with auditoriums of kids from 4 different schools. The especially irritating thing to me was that my alma mater was not even included in the session that I led.

I think this sucks because I went to SUNY Buffalo because they offered me a much, much better deal financially than the "elite" schools did. I could have attended one of the favored few, but decided that I'd rather graduate without massive debt. Some of the smartest people I know I met there. They definitely weren't the rule, but a lot went on to get PhD's from places like Stanford and Caltech. So its not like big state schools have no qualified candidates. Its just hard to get the FAANGs to see that.

<comment><USER36> > Some of the smartest people I know I met there. They definitely weren't the > rule, but a lot went on to get PhD's from places like Stanford and Caltech. > So its not like big state schools have no qualified candidates. Its just > hard to get the FAANGs to see that.

If you evaluate candidates yourself, you don't need to be interested in their credentials.

That's not the point of an interview. The problem isn't that SUNY has no qualified candidates, it's that the hiring process doesn't distinguish the qualified ones from the unqualified ones.

</comment></comment></comment></comment><comment><USER37> Prof at a SLAC here, in the Math Dept but degree in CS theory. FWIW, I'll throw in that my advisees often tell me that they _hated_ the first CS course (which Math requires).

Our CS Dept is good. They are not slave drivers, or doing crazy stuff. They run a standard Java class doing very reasonable things and give a perfectly typical range of grades. And the students who say these things to me are not just the poor ones.

Is it inherient in the material? Is it the way the intro is typically approached in standard texts? I don't know. And I admit that I only have anecdotes. But the article didn't mention it, so I'm mentioning it.

</comment><comment><USER38> CS is a bifurcated world. Yes there are very well paid people in FANG companies and in Bay Area start ups. The reality is that 90% of IT people are working in a cubical making bug fixes to TPS reports in nondescript corporates where they're just another cost center - until they hit their forties when it gets tough to find a job.

Most people see that reality.

<comment> non-entity Goddamn, why didnt I choose a different field.

Funny enough when I was a teenager I wanted anything but to become the latter guy, I even had my own unrealistic goals to avoid it. Unfortunately reality would kick in not too much longer later and I was forced to resign to that life.

</comment></comment><comment><USER39> When I learned coding, CS wasn't really a "thing." I came up from the hardware end of things (EE).

Today, the only downside of this seems to be that I never learned all the binary tree stuff (BTrees haven't played _any_ role in the type of coding I've been doing for the last three decades), so I don't fare well in employment exams.

Otherwise, I'm actually thrilled at the way my software development education has gone. My hardware background has stood in good stead, in developing things like drivers and async stuff.

<comment><USER40> Thankfully I've never worked or applied to a company doing that kinda thing; at best it was a fizzbuzz. They mostly do practical in-your-own-time assessments - prove that you have the hands-on know-how to build a feature representative of the work you will be doing in the company.

Few companies will rebuild B-trees in the day to day job. Interview for the job your candidates will be expected to do, instead of giving them a college/uni exam question. Have them validate a form field and yeet it through a REST API into a database, like 99% of their job will be [1].

\--

[1] this may just be representative of my job. Somehow yeeting a form field into a database hasn't gotten much easier during my career. Why is input type number still broken?

<comment><USER39> I have an _enormous_ portfolio of code. I'm talking dozens of repos, hundreds of thousands of lines of code, shipping apps (with full source code), and at least a decade of checkin history.

Also, I have many, many articles on Medium, and on my own sites. These tend to explicitly and unambiguously cover my development methodologies, design ethos, and personal philosophies.

There should be no question _at all_ about my technical ability, methodology, creativity, or team integration.

But it is my experience that the portfolio is immediately ignored by hiring managers.

After a few instances of that, I just gave up looking to work for others, and started doing my own thing.

</comment></comment></comment><comment><USER41> My theory is that a lot of people would hate working in computer science and programming. For most people this kind of work is very tedious and boring. You have to be cut out for it.

</comment><comment><USER42> CS is seen as leading up to a blue collar job executed behind a desk that requires a particular kind of mental aptitude and dedication.

It comes with low intrinsic social status and even stigma attached.

The skill set and personal characteristics that it thrives on are nearly opposite of those that do well in corporate or societal careering.

</comment><comment><USER43> I was in college from 2010-2014 which is a period of high growth as shown in the article.

It may not have been an all time high but it felt like _everyone_ was taking CS. The 101 classes were incredibly oversubscribed. Maybe not many people graduated with the major but a ton of people started down that road and many people pursued the minor.

<comment><USER44> I was in college during the same period at the largest school in my state and we graduated a class of about 60 or so undergrads in 2014.

</comment></comment><comment><USER20> Some of the decline in 80s can probably be attributed to a reduced percentage of women getting CS degrees [1]. I tend to agree with the conventional wisdom that gaming and home computers led to CS becoming a less welcoming degree to someone who had never touched a computer before. Contra almost every degree outside of the arts where some interest and the usual high school curricula are all you need. (And it became a somewhat self-reinforcing cycle.)

That theory doesn't speak to the 2005 peak and subsequent decline however. [ADDED: Which is probably more related to the dot-com bubble bursting.]

[1] [https://www.aei.org/carpe-diem/chart-of-the-day-the- declinin...](https://www.aei.org/carpe-diem/chart-of-the-day-the-declining- female-share-of-computer-science-degrees-from-28-to-18/)

</comment><comment><USER45> What do people here consider to be CS? I couldn't really tell from the article. As I recall, CS was in the math faculty and dealt with theory and core principles or developments, stuff you typically read in whitepapers.

Regular programming/software programmes were in the engineering faculty, as well as some software development being covered to some extent in all the eng programmes. I also recall CS + computer eng made up a large fraction, about as much as all the other math depts and more than physics + biology combined.

The title suggests that there aught to be more and I'd agree that grads going into computer science fields should be better prepared but I don't think that has to be via a CS degree.

</comment><comment><USER46> Why would you major in computer science if you want to work as a software engineer?

It's like getting a physics degree to become a civil engineer.

<comment><USER47> Using your analogy, probably because there aren't many well established civil engineering degree programs.

I don't know what "practitioner program" could get you close to a guaranteed spot at a top software firm. I've been out of school for a few years and figure this stuff has been changing fast though, so that perception may be out of date.

Never even heard of a software engineering degree. Only computer engineering or computer science, either of which may include some software engineering courses.

<comment><USER48> This might be due to different naming conventions in different countries. Software engineering is a common degree in Australia. Example: [https://study.unimelb.edu.au/find/courses/graduate/master- of...](https://study.unimelb.edu.au/find/courses/graduate/master-of- engineering-software/)

Though I doubt the content is that different to courses in other countries

<comment><USER49> I did a BSc(Comp Sci) in Australia (USyd) starting in 2010. A lot of my friends were doing the Software Engineering degree. We did the same courses and went to the same lectures, the only real differences were a) what faculty we were officially part of, and b) the words on the piece of paper we got at the end.

(I'm stretching the truth a little here because I also did a Maths major in addition to Comp Sci, but that was not part of my Comp Sci requirements so I think it's still valid to the point I'm trying to make, that the two degrees are functionally identical with different names)

</comment></comment></comment></comment><comment><USER50> Great article. I majored in Electrical and Computer Engineering roughly 20 years ago at a top US university. Even then, my classes were probably 2/3 international students. I fear that the recent rhetoric from the White House (shutting down H-1B visas) and general impact of Covid on globalism is going to make this problem in many countries like the US. Just as the legal and medical fields got saturated in the 80s and 90s and the preferred career track for high pay and job security, I predict that the same will happen with computer science in the near future. I think most future innovation will come from fields like biology.

</comment><comment><USER51> Personal opinion: after four of the required five years of my CS major education, I felt like I had gotten about as much out of it as I ever would. I knew I would advance my own skills and knowledge more by spending that last year as part of an experienced team in the industry instead, so I did.

Other than it being annoying to explain the situation if someone asks, this has not ever set me back in any way I know of.

If, in the future, I feel like it would improve my abilities/position to write a Master's thesis and get a diploma, I might look into taking the last year. But so far, I have gotten way more leverage out of the additional industry experience.

<comment><USER52> Computer science has little to say about software engineering, I've found. There is not much high quality research on the subject, because it's more of a social science and it's inaccessible as a field of study to normal social scientists.

</comment></comment><comment><USER53> A CS degree program isn't something you can get into by knocking on the door and saying, "Hi, I'd like to study CS because there are well-paid jobs in Silicon Valley".

Moreover, university departments tend to only tighten their entrance requirements with increased interest in a field.

Article also conflates "earned degree" with "majored".

</comment><comment><USER54> I thought about adding CS as a second major. Then I saw how much the CS majors had to work. The CS program at my alma mater has/had a reputation for having some extremely time intensive courses — iirc, some courses had weekly homework assignments that took 20-40 hours per week (per course). Maybe it was just a few courses that were weeder courses, but I didn't think that was the best use of my time in college. It seemed excessive and perhaps abusive in a hazing kind of way. Maybe they thought having ENIAC on display would motivate people through the grind.

I didn't mind working hard in school, but I definitely wanted to explore things outside of coursework.

I also already knew how to program, and I didn't need a CS major to get programming work. The benefits of a CS degree just didn't seem to be worth that much at the time (this was decades ago, so it may be different now).

</comment><comment><USER55> I myself have Bachelors and Masters in Computer Science degree with 11+ years of full-time industry experience (Canada) and here is my opinion about popularity of computer science degree:

\- What is the point of getting a Computer Science degree when the people sitting besides you are also Tech Team Leads or Sr. Developers or Architects without a computer science degree?

\- Why bother spending so many years going through theoretical computer science courses when you could have gotten through some bootcamps, youtube vids, Leetcode, and spent time creating github portfolio? Ultimately its the frameworks (and sometimes certs.) which gets you the money. There is a reason why even Google dropped the 'degree' requirements for their Software Engineering positions.

\- When it comes to corporate IT: Among the most glorified of CS/IT career is that of a Software Developer (coding). And if I was to be biased, I would say those who didn't get CS degree and instead went to Community College for programming diplomas tend to better at picking up frameworks and churning out code. It seems that their minds are less cluttered with useless technical details which nobody in Corporate IT cares about.

\- Frankly speaking, Programming and even other CS fields (including niche ones such as Data Science / AI / Machine Learning) suffer from absence of licensing requirements. I am not advocating for one but from what I have seen, just about any office job outside of IT has set of licensing requirements (PEng, LLB, CFA, CPA, Securities course, etc.) and atleast in Canada, they want the "Canadian version" of CPA, CFA, etc. -- this ensures that new immigrants can't just walk into such Canadian jobs and safeguards them against offshoring to an extent.

Despite all this, I am still glad that I have my CS degrees because that is just me, it's fits my thought process and personality. I really didn't do it for the money. I just feel I couldn't have done any other degree besides a CS degree - it's not about harder or easier.

</comment><comment> dylan-m I got about 8 years into my four year degree program in between interesting part time and then full time jobs on the side, and recently came to a set of realizations:

1\. I'm never going to finish this damn thing because I keep failing or almost failing math courses (and it happens the three required courses I have left are gated by math courses). All it does at this stage is make me feel inferior. (I do suck at thinking mathemetically, but I guess I've sucked at it for long enough I can deal with it in practice. Alas, they aren't buying it :b).

2\. The only required course that remains which I'm really interested in doing is about writing a compiler, but I could do this myself. I'd miss having the excellent prof who teaches it, but that isn't worth the tuition until then. I already found my way into the others. There were some good ones which I'm really glad I did.

3\. Upon graduating, my job prospects wouldn't change. Things are going fine. I might regret not having the piece of paper if I want to switch fields, but…

4\. Outside of two or three excellent CS courses, the courses I have most enjoyed and grown from have been humanities and archaeology electives. They have a more interesting, diverse range of students; heaps of opportunity for cross-discipline thinking; and content I genuinely know nothing about, would like to learn about, and have no idea where to start. If I decide at some point I'd like to finish my degree, I'd probably take more of those courses and try to find an applicable degree than suffering through more CS.

The trouble with CS is (as a job training program, as opposed to a program about science and mathemtics), if you're the type of person who's doing the degree program because you're already doing this stuff, there's a decent chance you could find a relevant job while you're still at school. And that job _also_ provides educational opportunities, because that's how our field works. Unless you are in a place where that four year full time degree thing works for you, it makes itself irrelevant.

<comment><USER56> One of my professors I liked told me she "Will not start teaching a class until she can do 100 practice problems in the subject she teaches". We had an 84 year old lady pass the class with us.

I don't know what you're struggling with but maybe retake a previous Calc 1 class to pass calc 2. Audit calc 3/statistics so it's just practice.

Obtain a different book and do the entire thing on your own, not only the even or odd problems.

I had to hate math and the embrace the suck to start enjoying it. Wolfram Alpha is a good friend in math.

<comment> dylan-m Oh, I agree, I really would like to get better at math. I find usually this stuff makes sense for me after it's percolated for a while and I eventually find an application for it that I like, and _then_ it gets exciting and easy to sort through. (Which is perfectly normal, but I suppose I just don't have the patience for the first stage there, where the subject is more abstract). Indeed, Wolfram Alpha is awesome for working through problems. Perhaps having a book on the side for when I cross paths with discrete math stuff would help to connect the dots faster. Was there one that worked particularly well for you?

</comment></comment></comment><comment><USER57> I studied CS up to a German Diploma (comparable to a Major-Degree) and it was hard. Now, ten years later, with all that advances in AI I finally know why I had to learn all that theoretical stuff: for the future to come. Do your major degree, it makes the future more fun!

</comment><comment><USER58> This isn't a full explanation, but might be worth thinking about. CS bachelor's education track is unique enough it is hard to switch into. If you are electrical/mechanical/aerospace engineering you're taking a lot of the same classes until your 3 year or so. You can change your mind and switch majors within the hard engineering fields. It is similar if you want to switch from biology to chemistry etc. At least in my experience a CS degree starts and ends there. If you want to switch into CS you almost have to start your courses over unless you were computer engineering. So out of the average % of students switching majors, some switch out of CS, but very few switch into it.

<comment><USER20> Electrical is fairly different from mechanical. Mechanical engineers do take some electrical engineering (and there are analogs between the two fields in areas like system dynamics although the examples are different) but there's quite a bit of fairly early-on coursework about fluids, material behavior, mechanical design, etc. Mechanical is closely related to aero and ocean but not so much electrical. (At least beyond basic core physics, calculus, etc.)

</comment></comment><comment><USER59> Personally I think something like Electrical Engineering is more valuable. At my company EE folks are often competent in both hardware and software, but the CS folks are completely incompetent at hardware and don't even understand basic GPIO usage.

<comment><USER60> A lot of EE folks I've met _think_ they are competent in software, but I'd beg to differ. Similarly, I've met plenty of CS majors who know their way round hardware.

<comment><USER59> Depends on definition of "software competency". Lofty functional programming in Haskell is quite different from low level driver programming in C. I doubt you could take a world class Haskell programmer and throw them in linux usb driver land and expect them to perform well.

</comment></comment></comment><comment><USER61> Bootcampers mostly solidify that software is not well regarded. If you go through bootcamper after bootcamper LinkedIn profile, you can easily see software was there second or third choice. They pivot to it as a backup.

</comment><comment><USER62> Honestly, here's the truth of the matter as someone who has gone through Bachelor's and Master's.

Annectodal (sample size 1):

I went to a top university in Florida for Computer Science. The classes were either simple, useful or nintendo hard. Data Structures and Discrete Math were the courses most likely to have transfers out to IT or Business degrees.

Two good friends ended up dropping because they just found a job and said "Screw it" and just went into the industry. A couple of others went into IT because the coursework was "too hard and I can get the same job with an IT degree"

</comment><comment><USER63> The vast majority of people working on IT are not really doing software development but system/network administration.. and for the vast majority of the jobs you don't need a major in CS. I remember when I finished the university in the end of 2006, our knowledge of networking after the university, wouldn't top any Cisco Network Administrator certified professional.. Yeah I could implement a Double linked list and do a lot of Math, but I had no idea of how to work with BGP, spanning tree and etc..

</comment><comment><USER64> Eh. I grew up in a rural area. Took two basic coding classes in high school but I didnt realize being a developer was a real option for anyone other than very smart math lovers. My college didnt have an actual computer science program. I don't think I would have been an amazing developer but I think I could have done the job, and would have, if I knew it was a realist option at an earlier age.

</comment><comment><USER65> I love STEM, but I went towards biology because my high school had a fantastic biology program that showed me how fascinating life is. It wasn't until I needed to analyze data on biology experiments that I coded. There was literally zero coding or Linux or explanation of computers at middle school or high school, in Providence RI, FWIW, around 2000-2010.

</comment><comment><USER66> It's more to do with existentialist crisis of computer science education, not status or pay for computer science grads.

Computing has become a part of every major industry and computer science by itself is closer to applied math and such fields, valuable but by itself not very useful.

Most professionals in most industries need to know quite a bit of computing skills to be effective.

<comment><USER67> > existentialist crisis of computer science education

Exactly what I've been facing for the past few months. I graduated as a CS Major last year, and I've already started doubting it.

No clue how to deal with it

</comment></comment><comment><USER63> "Anti-women culture. Tech companies and CS departments have the reputation of being unfriendly to women. "

No way CS Department is worse than Laws Department.. It's hard to pinpoint the Women in IT issue, but last years there are a huge movement trying to make everything more gender neutral in IT... Which other branch is investing more on that that ours?

</comment><comment><USER68> As the article states, many developers have degrees other than CS. I have a BS/MS in Mathematics and have worked as a developer for the past 20 years. Interestingly, back in the early 2000s, it seemed that having a degree in Math was actually an advantage for me. I think I got a lot of interviews because of that.

</comment><comment><USER69> CS is a specialty. It is in high demand, but it's also centralizing. If you are in the periphery, it's easy to miss out on a lot of that demand.

It makes sense to reach out to build diverse CS talent, but I'm not sure that it makes sense to reach out for greater numbers.

</comment><comment><USER70> And yet, it feels like it's increasingly competitive to get into a prestigious CS program?

</comment><comment><USER71> Because there are such perverse incentives in Academia that what they do and teach have no relevance to anybody.

eg. There was a major linux conference at the local university.... Full of cutting edge real world OS / HW stuff.

The CS department didn't bother to attend.

</comment><comment><USER72> The perception of computer science is slowly changing as it seeps into popular culture through TV and Film. As that changes more status conscious people will take CS, they will then be depicted in TV and film and you get a snowball effect.

</comment><comment><USER73> This article is from 2017, and the author refers a previous discussion here at HN: [https://news.ycombinator.com/item?id=14440507](https://news.ycombinator.com/item?id=14440507)

</comment><comment><USER74> Could it simply be that the degrees are now being called "computer engineering" instead of "computer science" degrees? That would be a simple explanation of this observation.

</comment><comment><USER75> Even though this is anecdotal, I know a lot of people who have majored in Computer Science, but they're all employed as programmers doing mainly web development, in particualr, CRUD apps.

</comment><comment><USER76> Weed out courses, lack of real world requirements analysis/editing/testing skills. Think of Journalism where they don't actually teach you to gather info, edit , and collaborate.

</comment><comment><USER77> Maybe becuase industry pays developers so well,it's hard for Academia to compete and attract quality teachers for computer science - ie is supply constrained rather than demand constrained.

</comment><comment><USER78> It's a tough degree compared to many, heavy on the math and very tedious at times too. I know when I was in school the class sizes started large and ended small.

</comment><comment><USER79> > Given high wages for developers and the cultural centrality of Silicon > Valley, shouldn't we expect far more people to have majored in computer > science?

Are wages that high though? I don't know much about the UK, but I feel it's lagging behind Switzerland, for example, and that only London is the high paying spot because of inflated cost of living.

In Amsterdam, where I live, a junior developer isn't paid that much more compared to a junior business analyst or a junior marketeer. What is _way more_ noticeable is that a lot more developer vacancies seem to exist compared to the other two, even for junior developers.

> I consider this a puzzle because I think that people who go to college > decide on what to major in significantly based on two factors: earning > potential and whether a field is seen as high-status. Now let's establish > whether majoring in CS delivers either.

In The Netherlands it's either whether something is interesting. Or it is whether something is interesting and it's good for one's career. It's almost never only for career.

> Are wages high? The answer is yes. The Bureau of Labor Statistics has data > on software developers. The latest data we have is from May 2016, in which > the median annual pay for software developers is $106,000; pretty good, > considering that the median annual pay for all occupations is $37,000.

The comparison should be made for different categories from university. I wonder how business studies is doing as that major was a lot easier, in my experience. But I'm pretty sure that the pay isn't lagging behind (disclaimer: I don't have data, I can be wrong).

I've never seen someone join a field of study for social status and I asked the "why do you study xyz" question a lot. I have met a few people who started studying <insert_female_dominated_program> for finding a girlfriend, which usually worked. In most cases, they were also at least kind of interested in the content.

\---

Here is my partial guess. I have two factors.

I believe that there are more computer science people that are interested in math and physics than the other way around. So there is a crowding out effect.

In The Netherlands, CS suffers from a negative stereotype among Dutch women, which is truly evidenced at my uni by the amount of international female students versus Dutch female students (a good amount versus 0 in some years). Heck, I even met very traditionally feminine women in my computer science classes (caring about fashion, aesthetics/beauty and caring for people), but they were all non-Dutch. Of course, there were also non-Dutch women with other gender identities (I've seen integrated as well). The gender identity of Dutch women was either "not male and not female" (do they call this non-binary?), male, or integrated. So this might play a role in England as well?

Of course, my observations are limited, they're simply based on 8 years of walking around at the two universities of Amsterdam and talking to quite a few students.

</comment><comment><USER80> I wonder if this is a US or global phenomenon, I would love to see numbers from the rest of the world.

</comment><comment><USER81> I'll give my personal take as a ex-CS major who dropped out to major in something else (math), tried to escape a career in software development, and ultimately wound up back in it when I saw how much more work and stress the other alternatives were (for me at least).

\- Computer science to me sounded way more fun than it actually was. I loved building things and programming, but that's not what CS is. CS is theory. I was bored to tears learning about sorting algorithms and binary search trees.

Same applies to machine learning and AI. The reality of studying it was way less fun than the idea. Even my AI professor acknowledged that.

\- As someone who'd always taught myself programming on my own, I didn't see the point in dedicating my 4 years of study to something I felt like I could teach myself. Since there was a good chance I'd probably end up working as a software engineer anyways, I thought it'd be smarter to study something I probably wouldn't otherwise ever teach myself on my own.

\- This is rude and shallow, but I didn't like my CS classmates. There was such a disproportionately high amount of weirdos, and I didn't want to be surrounded with those people all day and god forbid become one of them. I vividly remember studying in the CS lounge and having to stop myself from face-palming. If I was a sociable kid it probably wouldn't matter, but as a socially awkward kid it would be too easy to only be surrounded like similar people the rest of my life and never evolve.

\- Software engineering seemed boring, just being stuck in a cubicle all day. I did an internship as a programmer, and although it was relatively easy, moderately interesting, and stress-fee, I was terrified of the thought of spending the rest of my life in that cubicle.

\- I wanted more money and status. At the time (almost 10 years ago), it seemed that finance was the highest paying and most prestigious field to go into. Finance sounded more exciting, and I liked the idea of their being no ceiling on compensation, whereas software engineering seemed capped at $200k/yr. Of course now things have changed, and tech comp at big corporations tops out at more like $500k/yr (or more if you get equity and win the startup lotto), and tech is way more respected than before. High finance still pays the most, but those jobs are basically limited to Ivy League graduates, the hours are insane, authoritarian work cultures, no remote work, and IMO the work is extremely boring and utterly meaningless, even moreso than software engineering where at least you're actually creating something.

Of course my views nearly a decade since graduating have evolved. I gave in and took on a career in software engineering, which I've attempted to leave at times but always ended up returning (though once I'm financially independent you better believe I'll be gone for good). But at least at the time those were some reasons why I dropped my CS major despite being convinced since high school that that was my calling.

</comment><comment><USER82> ???????????

Computer Science is by far the most popular stem major all around the world!

If somebody have numbers showing opposite, I would like to see them.

<comment><USER83> The featured article has these "numbers showing opposite".</comment></comment>


Onlive... The Future of Everything? - <AUTHOR> http://www.techvibes.com/blog/onlive-the-future-ofeverything

<comment><USER2> sign-up by July 15th and you might get a free chance to try it for a year as well <http://www.onlive.com/signup>

I've been interested in this idea after I watched the demos on youtube: <http://www.youtube.com/watch?v=2FtJzct8UK0> They explain their business model in this somewhat long presentation at Columbia

for me personally, we'll see how the lag is

I'm not too offset by the inability to play the games if I ever quit, or even so much by the inability to play if my net goes down, because with my costs of building a new higher end computer once every 3-4 years still exceeds a $15 per month cost for such a service</comment>


Pro scraping with Node.JS - <AUTHOR> https://github.com/chriso/node.io

<comment><USER2> I was intrigued to see what CSS selector engine it was using...

<https://github.com/chriso/node.io> uses <https://github.com/harryf/node- soupselect>

<https://github.com/harryf/node-soupselect> is a port of my <https://github.com/simonw/soupselect> library for Python

<https://github.com/simonw/soupselect> is a port of my getElementsBySelector function for JavaScript: <http://simonwillison.net/2003/Mar/25/getElementsBySelector/>

I'm always surprised to see that code still being used - it's the least complete selector library out there by a long way.

<comment><AUTHOR> Hi Simon, great libs - thanks! there's been many improvements added to node- soupselect and node.io though - the API is here: [https://github.com/chriso/node.io/wiki/API---CSS- Selectors-a...](https://github.com/chriso/node.io/wiki/API---CSS-Selectors- and-Traversal-methods)

<comment><USER2> Oh nice - the .rawtext and .striptags methods are particularly useful.

</comment></comment></comment><comment><USER3> <http://mojolicio.us> is way better for this kind of stuff. Here's the synopsis example redone using Mojo:

        $ perl -Mojo -e'g("reddit.com")->dom("a.title")->each(sub { warn shift->text })'

<comment><AUTHOR> The one liner is cool, but I guarantee that node.js's non-blocking IO will outperform perl any day of the week. Try scraping thousands of pages at once using perl..

<comment><USER3> mojolicious is using a non-blocking async runloop as well =)

<comment><USER4> The problem you'd have with anything that represents a page as some kind of graph is you have to construct the whole tree before you can start doing anything with it. The API largely precludes streams. Callbacks would be possible but some of the conditional CSS selectors need a complete knowledge of the page before they can be resolved.

So while GET-ting pages to scrape can benefit from async IO, you're effectively "blocked" while scraping pieces out of the page itself.

</comment></comment></comment></comment><comment><USER5> Really interesting, thanks! This will probably the first thing I will use for real projects in node.js.

Does anyone knows how it compares to say Nokogiri or Hpricot, both in terms of speed and in terms of ability to handle crappy html ?

</comment><comment><AUTHOR> This is in response to all the node/jsdom/jquery scraping posts that are popular lately. JSDom is hopeless for scraping - try parsing some slightly malformed HTML..

<comment><USER6> Hey Chris, I was just trying to share a few things I'd learned. I know I haven't done as much scraping as others (like yourself and richcollins). Glad I've helped get some discussion going :)

<comment><AUTHOR> Hey David, it wasn't a stab at your blog - your post was great - anything that builds some more interest in node.js is positive :) I just hate reading about people having trouble with JSDom and putting it down to the node platform. JSDom is an excellent parser, it just fails miserably when you feed it malformed HTML, and as we know, a majority of the internet falls in to this category. I needed a framework that could scrape anything on the web so I built it myself

<comment><USER7> just to be clear, jsdom is not a parser. By default it uses node-htmlparser which is not very lenient.

Have you tried using Aria's html5 parser? I hear it works better with malformed markup.</comment></comment></comment></comment>