C++20 Is Feature Complete; Here's What Changes Are Coming - <AUTHOR> https://hackaday.com/2019/07/30/c20-is-feature-complete-heres-what-changes-are-coming/ <comment><USER2> This article might seem confusing.

That's because it is. There is hardly a paragraph in it that is wholly correct, where they actually mean anything.

C++ itself is much easier to understand than the article.

The real purpose for all the constexpr-related apparatus, as well as much of Concepts, is to make it much rarer that anybody needs to do "template metaprogramming". Instead of writing Template Metaprograms to run at compile time (a very strange functional pattern-matching language practically nothing like C++) you can usually write ordinary C++ code.

Coroutines are a new flow control primitive showing up in lots of languages lately. The C++ version is the same as the others.

The main benefit you will see from Concepts is overwhelmingly better error messages when you misuse a library component, whether Standard or your own.

Modules is a big step toward enabling a modern build infrastructure, as already seen in lots of newer languages that got benefit of hindsight.

All the old stuff is still there, because billions of lines of old code still use it. You don't have to use it in new code, but it all still works.

The biggest difference between old and new C++ code is that good new code uses value types more, and reference types less, making everything safer.

Coding modern C++ is overwhelmingly more fun than coding C++98 was, but it is just as fast as ever, and better than ever for writing fast, powerful, nice- to-use, no-compromise libraries.

<comment><USER3> I got had to fight with template metaprogramming lately, and I can't be happier than reading about this. Do you have any sources when one can learn in a non-confusing way about this new stuff, especially Concepts?

<comment><USER4> If you're looking for a high-level example, the Wikipedia article one where a function is constrained to only accept an argument which implements operators "==" and "!=".

[https://en.wikipedia.org/wiki/Concepts_(C%2B%2B)](https://en.wikipedia.org/wiki/Concepts_\(C%2B%2B\))

</comment></comment></comment><comment><USER5> I'm not a C++ dev. I took one class of it in college and even then it was 90% C and only hit pointers and classes on the final unit.

But holy smokes this is not your grandmother's C++, is it just me or is the language trying to be all things to all people? It's getting too big, no? I mean how long do you have to be writing C++ code to fully master the language? Or can you be a high level dev without ever touching the new features? I'm assuming production codebases are likely still stuck on older versions of the language and might never move on.

I hope golang doesn't grow this large over time.

<comment><USER6> One of the worst interview questions I have had was "Rate your C++ knowledge from 1 - 10".

I had just finished my second year of college, and although I felt fairly confident in my C++ abilities I had no idea how to answer. I thought it would be best to be honest and said "4" based on the subset of the huge language I knew (honestly this rating was probably too high at the time!).

One of the interviewers was obviously unhappy with the answer, and was unpleasant the rest of the interview. I have always wondered what they expected from such an vague question.

<comment><USER7> >One of the worst interview questions I have had was "Rate your C++ knowledge from 1 - 10".

>One of the interviewers was obviously unhappy with the answer, and was unpleasant the rest of the interview. I have always wondered what they expected from such an vague question.

I knew enough of C++ to know my knowledge on the subject is approximately close to Zero.

Basically no one on earth knows C++, they mostly know a subset of it.

*C++ here referring to Modern C++, Pre 2000 era C++ was still manageable.

Edit: On that thought it seems to me every PL in 90s were all manageable. Somewhere along the line things got ugly.

<comment><USER8> I _love_ asking this question! I almost always get "7" as an answer! Except one notable time: the interviewee said, at first, "seven"... then he stops, slowly makes eye-contact and says "no: 10." Brilliant! Who does that!?

I've interviewed hundreds of candidates; HR has relayed to me that many candidates requested that HR tell me that they loved being interviewed by me, even though the interview was very, very hard; they say they were surprised to learn things, and also happy I was so interested in them as s person.

This always made me happy; I had to sacrifice a teaching career to program.

<comment><USER9> Curious as to why you love asking this question? I wonder what data you are trying to glean from this question. It's a very subjective question and wouldn't surprise me at all if the number of years experience a candidate has in that language is inversely proportional to the answer you're given.

I often get this question from recruiters hiring front end developers: From 1 to 10, how well do you know Javascript? I've given 8 for as long as I remember but often I think to myself if the people who wrote the V8 engine are 10 by default, there is no way in hell I'm even close to an 8.

<comment><USER10> A good self-rating strategy in an interview would be to try and percentile yourself among the people based on your knowledge and try to justify it with humility. C++ is huge, I expect I know more than 80% of the people due to (years of experience, different kinds of projects, yada yada), but I feel i have only been able to explore like 50% of the language so far (concretely learn that is) and there's more to learn like (1, 2, 3..) and probably more that I might have missed completely.

<comment><USER8> It's almost always 7. The actual question I ask is "rate yourself on a scale from one to Bjarne". Bjarne Stroustrup was the principle in the group I did my PhD, and _he_ rates himself a 7, so when interviewees give '7' as the answer, it makes me laugh.

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> I am C++ developer by career and I am pretty much bashing the language all the time. To preserve my sanity I like to use C in after-hours. However looking up C++20 and the direction whole language is going I see there are things I would really like to see in C, while preserving C simplicity. Standardized attributes like [[fallthrough]] and others for functions seem nice. Anonymous functions are nice. They look a bit ugly, but function pointers already look even worse. It would not hurt to have modules and consteval functions in C, but then one still needs templates to really ditch the preprocessor and that's a tough one to solve.

Having said all that C++ is really in a need of a clean cut between legacy features and the blessed way. Backwards compatibility is a noble cause, but there is also the whole weight it pulls around. Maybe something like [[deprecated]] attribute could be extended to handle more cases. Maybe C++ needs it's own "unsafe" block, something like

      [[unsafe]] {
        *((void*)0) = 0xDEAD;
      }

Essentially having an ability to clearly mark all those different styles of different versions of C++ really. So that compiler could help you to control it and in case of a green field project totally guide you to use only the most modern idioms with an escape hatch. Of course the problem is that to help with the complexity more complexity would have to be added.

I am a bit on the crossroads. From one side I start to see, that there is some hope for C++. But on the other I feel like running away, especially since I'm looking for a job. In no particular order: C, Rust, Java, Kotlin, Scheme, Julia, Go, OCaml, Free Pascal, Swift, TypeScript, Lua/Terra, Zig.

I had a funny thing with C++. One project I worked on was used in the debug version all the time, to be able to drop to debugger when needed. At that point I started to wonder why not just use Java already, instead of trying to do a manual Java and fail at it.

</comment><comment><USER12> Despite the slight differences they have from the C99 version of them, I'm pretty happy to see designated initializers in C++20

</comment><comment><USER13> Sounds amazing. Anyone have code examples that integrate a lot of the newer stuff?

Dumb question. Why isn't omitting 'return' always an error in gcc rather than just a warning?

<comment><USER14> int foo() { bar(); } is a perfectly well-formed function as long as bar() never returns. Allowing this without requiring bar() to be annotated as noreturn is generally considered a mistake, but correcting it would break existing valid programs.

<comment><USER15> C++17 already made backward compatibility a non-goal (within reason, obviously) and broke previously valid programs.

<comment><USER15> I don't care that my previous comment post has been downvoted, but I do want to clarify that I wasn't being facetious; C++17 removes a number of old features and constructs that had been previously deprecated and there exist programs that compiled under C++14 that need non-trivial changes to compile under C++17.

</comment></comment></comment></comment><comment><USER16> Looks pretty good. But I don't understand why they don't do string interpolation like C# and others do.

"I= {I}"

Is much better and less error prone than

"I= {}", I

Or am I missing something?

<comment><USER14> The format library is a pre-existing library which is being standardized. String interpolation cannot be implemented in a library, and requires language extensions. Perhaps they should have done that instead, but that's a significantly more complicated task than merely adopting something that's already well tested and proven (and even that was far from trivial).

<comment><USER11> I am not necessarily opposing string interpolation as a language feature, but I have an idea how could it work as a library. Implement string suffix and overload comma operator or overload reminder operator. It would look like this:

      "Hello {} {}!"i, 2, "worlds"

Or with reminder operator as in Python:

      "Hello {} {}!"istr % istr(2, "worlds")

But yeah, enough of this.

</comment></comment></comment><comment><USER17> I've had C++98 at school, but how would I learn C++17? Scott Meyers wrote C++14 but he retired from C++ development so what are the best alternatives?

<comment><USER18> Changes from c++14 to c++17 are nowhere near as dramatic as c++98 to c++11, so you can go with Scott Meyers and fill in the gaps from cppreference.com, for example.</comment></comment>


Cracked It, a social enterprise that repairs phones - <AUTHOR> https://www.crackedit.org/ <comment><USER2> > A society where no young person feels that crime is their only option. This > crime can become violent. Knife crime offences resulting in injury have > increased by 22% in the last year, and 85% of violent offenders are aged > between 16 and 29

I think they're saying that their workforce will include such violent offenders. Good for them. Probably most of these people need another chance and are worthy of it.

But I can't say I'd be comfortable calling them to my home. My father once hired a youth offender to do yardwork. He worked for him for several months. About a year later he came to the house with a girlfriend, robbed the occupants, and murdered two of them. They were the family I sold the house to after my father died.

Maybe my bias against violent offenders is bigotry. Surely this was a rare case. It isn't very fair to generalize. Yet I do fear them, and if I used this service I'd make an effort not to let them know where I live. It seems likely that many other people feel this way. That gives Cracked It a particularly tough hill to climb for a startup. I wish them well.

<comment><USER3> I don't think I'd be comfortable even giving them my phone unless I can wipe it first.

<comment><USER4> Turning off a modern phone should make it virtually impregnable to non-state actors. Do you have specific evidence that points otherwise?

<comment><USER3> It's not clear why I have to provide evidence that my phone has any vulnerabilities to justify being my being uncomfortable handing it to a stranger with a known criminal past. Security comes in layers and one of those layers is physical security.

</comment></comment></comment></comment><comment><USER5> In Toronto; we have 3+ of these for bicycles. Transportation is democratizing.

I love seeing the same principles applied to communications.

</comment><comment><USER6> The domain looks like "crack edit"

<comment><USER7> Another victim of the expertsexchange problem. It's unfortunate because hyphens really should be avoided in URLs. For the last couple of domain names I've registered, I made sure to put the potential names into the "Detect language" box in Google Translate to make sure they weren't inappropriate words in another language, in addition to reading them upside down and backwards for other English interpretations. Naming things is hard.

<comment> ezequiel-garzon _It 's unfortunate because hyphens really should be avoided in URLs._

Why? Due to SEO penalization?

<comment><USER8> The old reason was because in radio ads, saying the domain with the dash was confusing.

A friend of mine just registered a domain name with a made-up spelling of regular words kind of like "leet-speak". The two of us are the only ones who can type it from memory, which is not good.

<comment><USER9> Aitch tee tee pee colon slash slash slashdot dot com.

</comment></comment></comment></comment></comment><comment><USER10> These guys also do popups - eg they come to your office once a month and set up in the canteen. The feedback I've heard is really good.

A similar business locally working with ex-offenders is redemption roasters (coffee).

Another really good example of how this can be successful helping people is Timpson's.

</comment><comment><USER11> Looks like we've got ourselves another Experts Exchange (see the domain name).

<comment><USER12> Crack edit? Is this a term I should be familiar with? To be clear: I know 'crack' is a drug, I am asking if 'crack' in combination with 'edit' is something funny.

<comment><USER13> Read aloud, "crack edit" unfortunately sounds a lot like "crack addict".

</comment></comment></comment><comment><USER14> This looks really neat, but it seems like a really bad time to be in "workplace" repair when so many are working from home due to a pandemic. I just found out I'll be stuck at home until at least 2021.

</comment><comment><USER15> Wow an even dumber name for a copycat of icracked. Another dumb name.

<comment><USER10> Despite knowing about them for a year or so, you made me think, and I realise that their name is really quite clever: you've cracked it - they can fix it. But 'cracked it' also means they have also solved the problem!</comment></comment>


Peter Thiel's Rise to Wealth and Libertarian Futurism - <AUTHOR> http://www.newyorker.com/reporting/2011/11/28/111128fa_fact_packer

<comment><USER2> I get the feeling kids will be studying Peter Thiel in ~150 years. I don't think that's the case with most other people in business/politics today. That's both because most people are relatively less accomplished, but that the accomplished people are also usually a lot simpler and internally consistent.</comment>


Google Walks Away from America's Security - <AUTHOR> https://www.bloomberg.com/view/articles/2018-06-06/google-s-decision-to-ditch-project-maven-is-a-grave-error <comment><USER2> Divesting from the military industrial complex is patriotic. Waging peace is patriotic. Not pursuing militaristic and inhumane technology development is patriotic.

It is a confused idea for patriotism to be conflated with militarism, isolationism, nationalism, and racism.

<comment><USER3> is tracking and selling user data patriotic?

<comment><USER2> I would argue that it is unethical and becoming illegal to a certain extent in various jurisdictions (such as EU). It is taking a similar process to the use of unsolicited email mass-marketing we now consider SPAM. In the early days, it wasn't obvious how invasive SPAM would be, but in hindsight it is very clear and now there are laws to discourage the practice.

</comment></comment></comment><comment><USER4> This feels like a very bizarre propaganda / pro-military opinion piece. Google isn't a defense contractor, its employees aren't any more or less patriotic if they don't want to build software that goes in killbots.

<comment><USER5> But Google _wants_ to be a defense contractor and the employees are emotionally blackmailing it from not being one.

If those employees had a fucking conscience, they wouldn't slice and dice every online interaction of a user and serve ads. Stop pretending to be on a moral high-ground by refusing to serve the very entity tasked with securing YOU and the nation

<comment><USER6> You think it's irreconcilable for an individual to be okay with farming data, but not with indirectly killing people?

Everybody has a slightly different moral compass. Every "good" person does "bad" things.

I highly doubt, for instance, that everyone who commits check fraud would be okay with murder; I also doubt every murderer would be okay with committing check fraud.

<comment><USER5> >>> "but not with indirectly killing people?"

I'm absolutely not OK killing (innocent) people, and clearly, the pentagon isn't too. The aim of the program is to increase accuracy to prevent and minimize collateral damage.

If you truly cared about protecting the lives of innocent, you'd support this program.

</comment></comment></comment></comment><comment><USER7> Bloomberg calls Google's decision _a defeat for U.S. national security, patriotism ..._

This raises a couple of questions:

What does it even _mean_ for a US company to be "patriotic"?

In today's globalized economy, how "patriotic" does an "American" company have to be?

<comment><USER5> That's a hypocritical question. Google and co. go running to big daddy if hostile foreign adversaries cause it business or IP problems. Case in point, Google sought the support of the Obama administration when China attempted to steal it's source code from within Google-China offices.

Just as recent as a month ago Tesla Inc. was a big beneficiary of the current administration's attempt to level the tariffs placed on automakers. Company CEOs routinely accompany senators and diplomats on foreign trips to woo foreign governments into favorable business climates for said companies.

And, here's a non-exhaustive list of benefits that America has bestowed on Google -

1\. Stanford univ (and other world class univs), along with the economic ecosystem around them enabled 2 grad students to even think of forming a company.

2 . The VC ecosystem at that point was the most mature in America/Silicon Valley - they took a risk on two pimpled faced fucks, albeit smart.

3\. America provided an economically just, politically stable and equitable environment for a fledgling company to take on established industry giants. In Russia or China, you can be coerced by violence to hand over your work, just ask the founder of Telegram/VKontakt about it.

4\. The Islamic world is opposed to VC funding, interest-based banking, women employment, ... due to religious reasons. Google could not have been born there

....

After all the ways Google has benefited, to chicken out of a fair deal the gives back to the country it leeched on is stupid. Further - being allowed the freedom to walk away from a government deal without consequence is another luxury Google would not have enjoyed in MOST other countries.

Wake up!

<comment><USER8> > fair deal the gives back to the country

do they already give back by paying taxes? I guess the dispute is that some people don't trust government enough to give them even more advanced killbots, and goverment is not necessary the same thing as country.

<comment><USER5> I'd argue non-elected entities such as the Pentagon are driven by national security and patriotism. If you suggested not pandering to any elected administration, I'd agree, as politicians are driven by the singular need to get re-elected. But the other parts of government mostly serve the country for pride and security.

</comment></comment></comment></comment><comment><USER9> Google is arguably one of the company that treats their employees the best, and yet those employees seems to constantly be sabotaging their company.

"Give them an inch and they'll take a mile" I guess.</comment>


Show HN: Sound in front of YouTube for mp3 - <AUTHOR> http://www.soundyoutube.com/Audio?file=Tgokaw2ZOoA.mp3 <comment><USER2> This directly violates the Youtube API terms of service. [https://developers.google.com/youtube/terms](https://developers.google.com/youtube/terms)

Under Prohibitions: "8\. separate, isolate, or modify the audio or video components of any YouTube audiovisual content made available through the YouTube API;

9\. promote separately the audio or video components of any YouTube audiovisual content made available through the YouTube API;

10\. access any portion of any YouTube audiovisual content by any means other than use of a YouTube player or other video player expressly authorized by YouTube;

11\. store copies of YouTube audiovisual content;"

<comment><AUTHOR> I'm not using their API

</comment></comment><comment><USER3> Nice idea!

I'm not sure about the legal issue here though. If I see it correctly, you are downloading the songs to your server and serve the file from there. A lot of content that is uploaded is often times illegal, so you might be charged for distributing illegal content (illegal as in copyright).

What I'm also wondering. How come the parsing takes so long? I know other services that are a lot faster. Is it just the speed limit of the server itself?

</comment><comment><AUTHOR> First of all I'm sorry it takes so long. I actually used this for extracting the mp3 [https://github.com/flagbug/YoutubeExtractor](https://github.com/flagbug/YoutubeExtractor) As for the legal issue, I'll take it down if they ask me to. I've seen other YouTube video extracting things which makes me think YouTube doesn't mind so much

</comment><comment><USER4> WOW, thank you so much. There are lot of underground DJs on YouTube who upload their remixes and you can't find them anywhere else on the internet. Downloading the YouTube video and extracting the sound is the only way to preserve their work and listen to it offline.

This makes things so much easier. Thank you.

<comment><USER5> You may want to look into youtube-dl ([http://rg3.github.io/youtube- dl/](http://rg3.github.io/youtube-dl/)). This tool seems to limit the bitrate to 64kpbs, while youtube-dl lets you configure the audio quality.

<comment><USER6> I wrote a wrapper service around youtube-dl: [https://github.com/bertique/node-yt-to-mp3](https://github.com/bertique/node- yt-to-mp3)

</comment></comment></comment><comment><USER7> YouTube is a registered trademark of Google Inc.

I'm afraid, you could? run into legal issues by using the word "YouTube" in your domain name.

[http://www.google.com/permissions/trademark/rules.html](http://www.google.com/permissions/trademark/rules.html)

<comment><USER8> I'm sure I've read something in the ICANN rules that says domains containing a trademark can be petitioned against (although that may only be for exact wording). If they can do it, Google aren't exercising the right to it as there are other domains that have been around longer.

</comment></comment><comment><USER9> Meta: The headline makes no sense to me at all. Perhaps "Shown HN: Replace youtube.com with soundyoutube.com for any video to get just the sound."

<comment><AUTHOR> That does seem like it makes more sense. I was just trying to make it short. I don't know if I can change it now though.

</comment></comment><comment><AUTHOR> Thanks, I saw the gifyoutube one and made this. I was reluctant to share though because I was afraid they'd shut it down.

</comment><comment><USER10> Awesome! This is a great service; I've often wondered why YouTube doesn't offer this on its own?

<comment><USER11> Really, you have? How about because hours of music is illegally uploaded to their site every day?

</comment></comment><comment><USER12> Or just use youtube-dl.

</comment><comment><USER13> Isnt it against TOC of Google?

</comment><comment><USER14> www.youtube-mp3.org</comment>


PyFlag - <AUTHOR> https://github.com/anarchivist/pyflag

<comment><USER2> The link [http://pyflag.net/](http://pyflag.net/) is not exactly what I was expecting.

<comment><USER3> It's not anything any of us has expected. :D</comment></comment>


Say Goodbye to Alexa and Hello to Gadgets Listening to Voice Inside Your Head - <AUTHOR> https://medium.com/mit-technology-review/say-goodbye-to-alexa-and-hello-to-gadgets-listening-to-the-voice-inside-your-head-3405ef93835b <comment><USER2> Video demo:

[https://www.youtube.com/watch?v=uUa3np4CKC4](https://www.youtube.com/watch?v=uUa3np4CKC4)

<comment><AUTHOR> Thanks :)</comment></comment>


Most Korean workers victims of 'gapjil' on job - <AUTHOR> https://www.koreatimes.co.kr/www/news/nation/2016/04/116_203010.html <comment><AUTHOR> > Gapjil, roughly translated as bossing around or "playing gap," comes from > the contract term "gap" (party A, who leads) and "eul" (party B, who is > led). So bosses are gap and subordinates are eul, men are gap and women are > eul, and large companies are gap while their subcontractors are eul.

<comment><USER2> If it's 9 out of 10, then the people doing it must also have suffered it</comment></comment>


An Illustrated History of Objc_msgSend - <AUTHOR> http://sealiesoftware.com/msg/?hn <comment><USER2> It's amazing that such a performance-critical part of Objective C's runtime is using a per-class cache for message sends - the same level of technology that powered Ruby 1.8 if you can relate to that!

I wonder if you could do some very limited dynamic code generation to create inline caches, or just inline msgSend and allocate a cache line somewhere writable for each inlined instance.

Or is this method already a slow-path, fallback case?

<comment><USER3> There are a number (16?) of vtable lookup-based message-sends for some of the common selectors, but in general, this is the fast path.

<comment><USER4> The vtable-based dispatch (described in detail at [http://www.sealiesoftware.com/blog/archive/2011/06/17/objc_e...](http://www.sealiesoftware.com/blog/archive/2011/06/17/objc_explain_objc_msgSend_vtable.html)) is no longer in use.

<comment><USER5> You know, when I tell people that Obj-C's message dispatch makes it a bit trickier to do a lot of the optimizations you find in C++ compilers, they always look at me like I'm crazy, but this seems like very clear evidence of this...

<comment><USER6> Why would they think you're crazy? It should be quite obvious that an Objective-C compiler simply cannot optimize across the message send boundary in any way, while a C++ compiler often can optimize across method calls.

<comment><USER5> Because despite pointing out exactly why I thought it was true and explaining Obj-C's various dispatch optimization tricks and how they still left an optimizing compiler/runtime with a far greater challenge, the response was generally, "you have no idea what you are talking about".

Thanks to Dunning-Kruger, it's quite difficult to be sure if they just can't be bothered to explain what I don't understand (maybe performance differences arise elsewhere... maybe there were far more sophisticated tricks in the modern message dispatch code than I'd come to understand... though it seemed unlikely they'd be so significant) or its the other way around.

Empirical evidence would seem to point strongly in my favour, but you'd think that'd give people some pause before making such statements.

<comment><USER6> Unfortunately, the nature of Objective-C as an Apple-centric language means a lot of people like it because Apple says they're supposed to, and don't take criticism of it well.

objc_msgSend is pretty amazingly fast, and is not usually a bottleneck in code, but the inability to do things like inline method calls when the target is known at compile time is a downer.

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> Neat. I'd like to see the same thing for iOS (ARM).</comment>


Cutting Edge Deep Learning for Coders, Part 2 - <AUTHOR> http://course.fast.ai/part2.html <comment><USER2> Jeremy from fast.ai here. I've posted a summary of the course materials for anyone who's interested : [http://www.fast.ai/2018/05/07/part2-launch/](http://www.fast.ai/2018/05/07/part2-launch/)

Let me know if you have any questions about the material or approach. There's also a discussion for the course here: [http://forums.fast.ai/c/part2-v2](http://forums.fast.ai/c/part2-v2)

<comment><USER3> Hey Jeremy, thanks for making these courses! I was curious if you could offer any points of comparison between the fast.ai courses and the Udacity ML nanodegree course (or any other MOOCs you have opinions about).

<comment><USER4> I am not Jeremy but I have taken part in the current course and also took some lessons by Siraj (who also does lessons at udacity), Andrew NG, and Karpathy. I think all of them are awesome teachers. Like Siraj's lessons, the fastai course is full of practical examples, whereas Ng/Karpathy's courses are more theoretical. What makes the fast.ai course unique compared to Siraj (who covers a lot of topics very briefly) is that Jeremy delves into depth, taking a long time explaining things, so you have a lot of opportunities to understand what's going on. In the 2018 course there is actually not that much different content compared to the 2017 course, rather Jeremy took the time and tried to make everything easier for everyone to understand and built a better python library. Also another unique point about Jeremy is that he and Rachel want not only to teach people but rather inspire a community of people working together in teams. The fast.ai library will not be just a project for the course, it will be something that a lot of people will use and contribute to to make pytorch better.

<comment><USER5> Is Rachel the lady that always keeps interrupting Jeremy so that he goes out of flow all the time, making lectures less enjoyable?

<comment><USER2> Dr Rachel Thomas, the co-founder of fast.ai, is the person who takes the highest voted questions from our 600+ in-person students and passes them on to me, as I asked her to do. At least 4 different people must have wanted the question asked before she asks it.

This approach helps ensure that if I haven't explained something clearly enough that I get another chance to do so, and, more importantly, keeps me fresh and energized throughout each lesson (I get stale and boring without some interaction).

<comment><USER5> I am a firm believer in candid feedback; I talked to multiple people taking fast.ai and all of them took it as disturbing the flow of lectures. I really like fast.ai, it's an excellent hands-on course, so please don't get offended.

<comment><USER6> Personally, I find the questions helpful, and for everyone who's watching on video, you can skip whatever portion you like, can't you?

<comment><USER5> The thing is you can visibly see how it interrupts Jeremy; he is in the middle of explaining something, then has the face of a surprised person, loses context for a few seconds etc. Better IMO would be if he just finished a small section in its entirety then did Q&A, instead of allowing himself being interrupted all the time. And often those questions are missing the point, which is to be expected with newbies, so the lecture loses efficiency. If I didn't know most of the basics already, I'd have problem following them. I found just doing the exercises better for learning.

<comment><USER7> I also find the questions very helpful. I understand for many people who already have a good understanding, these questions might be interrupting the flow. Many a times the questions his students raised were things that I had not thought of, sometimes they even stumped Jeremy. Fortunately Jeremy in most cases gives answers which vary in length, according to whether it's a concept which he will clear later, or whether it is an important concept that needs to clarified then itself.

So I think this more effective way of learning/teaching even though there is a loss in efficiency.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER8> I can't say enough good things about the fast.ai courses. They may not be for everyone but they are definitely for me. The teaching style is a perfect match for my learning style. The style is to get some code working and then dive deeper and deeper to understand it better and better.

Note this link is for part 2 of the course which is significantly more advanced than part 1. Start with part 1 if if you are new.

</comment><comment><USER9> Jeremy - First of all, I just want to say thanks to you and Rachel for the course(s). I worked through part 1 (2017 version) last year and it was fantastic.

I've got part 2 on my to-do list for this year. Looking at the 2018 part 1 course, quite a lot has changed (from Keras to Pytorch?) from a code perspective in comparison to the 2017 part 1 course. If I want to start part 2, does it make sense to go-ahead with the 2018 version? Are there any 2018 part 1 lessons you recommend to bridge any gaps from the 2017 part 1 before I start 2018 part 2?

Thanks!

<comment><USER10> Not Jeremy (obviously) but having taken both versions of the course I would definitely recommend jumping into the 2018 version. The core content is the same (image recognition, recommender systems, NLP) but the 2018 version uses more cutting edge tools and has a tighter feel (as a v2 of anything usually does).

<comment><USER11> The field is changing so quickly, but fast.ai is always ahead of the curve teaching modern techniques! Hats off to them for being able to keep up, AND share that with others.

</comment></comment></comment><comment><USER12> I watched all of the 2018 Part 1 videos and it was not my style.

One thing I liked is the positive attitudes of Jeremy and Rachel, but there was not much theory and a lot of time was spent on questions and answers for the participants of the lecture that I don't think necessary.

I wanted to see definitions and "why", but the course spends too much time on "how". Sometimes I see "why" in the lecture, but many times it is just an answer for the question from a random participant and the answers seemed not well prepared, so it makes it hard to deeply understand the content.

I think fast.ai (at least Part 1) may be good after taking other deep learning courses and before participating in the Kaggle competitions that when you are already familiar with the theories.

<comment><USER2> Top-down vs bottom-up depends to some extent on what you enjoy, vs what you need to be patient about. With our top-down approach you _do_ get to all of the 'why', but only after understanding 'how'. So it's good for people who want to get started doing stuff right away, and don't mind waiting a bit to understand the details. It means you can start experimenting and building a good intuition for training models, which I believe is the most important skill for a practitioner.

On the other hand, with the bottom-up approach of Andrew Ng in deeplearning.ai, you start with a lot of 'why', and later on get to 'how' (although in less detail and fewer best practices than we show). So it's good for people who want to understand the theory right away, and don't mind waiting a bit to understand how to use it.

A lot of our students did Andrew's course after ours, and many did it in the reverse order. All have reported finding the combination more helpful than either on their own. When we describe 'why' it's mainly with code, whereas with Andrew it's mainly with math - so which you prefer will also depend on which notation and framework you're more comfortable with.

(But I promise - you do get all the 'how' with us, particularly in part 2! Our students have gone on to OpenAI, Google Brain, and senior AI leadership positions at well known startups, as well as writing and implementing new papers. Here's an example of a student who just implemented a paper that was released within the last month: [https://sgugger.github.io/deep-painterly- harmonization.html#...](https://sgugger.github.io/deep-painterly- harmonization.html#deep-painterly-harmonization) )

</comment></comment><comment><USER13> I have taken the classes and also did Udacity Nano Degree and Deeplearning.ai. All of them help you in your journey of mastering DL. But Fast.AI part-2 really teaches you to cutting edge of DL through a number of practical examples. Don't wait and join other fellow learners at forums.fast.ai

</comment><comment><USER14> >> Welcome to the new 2018 edition of fast.ai's second 7 week course, Cutting Edge Deep Learning For Coders, Part 2, where you'll learn the latest developments in deep learning, how to read and implement new academic papers, and how to solve challenging end-to-end problems such as natural language translation.

I would really like to know how to solve natural language translation. I think everyone would. Many people have been trying to solve this devilishly hard problem for several decades and failed. So I'm really curious how fast.ai has finally manged to do it.

<comment><USER15> Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks.

Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results.

I wouldn't call this model a solution for natural language translation, nor would anyone else. But I think fast.ai meant that they're going to explain and go through this model, and how it's helped bring a new generation of models with good performance in this particular space.

<comment><USER2> Yup it's multi-layer bidir seq2seq with attention, and a few tricks like teacher forcing. Same as Google Translate. Their version takes a long time to train on a lot of GPUs, so we simplify it by using less layers, and a smaller, simplified corpus (it only contains questions, and limits them to 30 words long).

By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end - I didn't mean to imply that the final model would be human-equivalent or perfect or anything like that.

<comment><USER15> Oh, nice!

Yeah, I understood the intention behind that statement. Great work with the course!

While you're here, what do you think about using temporal convolution for sequence tasks? I've read a few articles, this particular one by my professor comes to mind now [0], which say CNNs could work extremely well for the tasks traditionally done with RNNs. A recent paper by the people at Google Brain [1] mentioned that their CNN with attention network beats traditional RNN approaches. More surprising is that the network is 130+ layers deep, and yet trains faster than RNNs. Do you think we can potentially switch most machine translation tasks to CNNs?

[0]: [https://towardsdatascience.com/the-fall-of-rnn- lstm-2d1594c7...](https://towardsdatascience.com/the-fall-of-rnn- lstm-2d1594c74ce0) [1]: [https://twitter.com/lmthang/status/989261575482560513](https://twitter.com/lmthang/status/989261575482560513)

<comment><USER2> Yup potentially. The jury is still out on which will win, or whether both have their place!

</comment></comment></comment></comment></comment><comment><USER16> I have nothing to say other than that I loved DL1 and I'll dive into DL2 right away. I really like the overall philosophy ("see DL is approachable and doable, just do it") and I love that they encourage you to read papers even if they seem hard and to translate math to code etc. etc. Fantastic job fast.ai team.

It's pretty damn amazing that they build a library (and also the new fastai.text) that goes one level of abstraction beyond PyTorch with the goal of implementing interesting/helpful papers ASAP and because they wanted to make their teaching even more efficient.

</comment><comment><USER17> Hi Jeremy, Thanks again to you and Dr Rachel for these amazing courses and the support to the community. My question is the following: are you thinking of an organic fast.ai connected hub or foundry acting as a distillator, that is disseminating & implementing in / with / through the fast.ai library the most interesting research papers coming out day in day out from arxiv.org and events? It is nearly impossible to follow the progress without clear state-of- the-art perspective these days, which you have in abundance indeed.

<comment><USER2> We're certainly looking for ways to help develop the community. Obviously [http://forums.fast.ai](http://forums.fast.ai) is a good starting point, and is a very active and supportive community nowadays. But I'm interested in finding ways to support face-to-face communities too. Some students are already doing that - for instance AI Saturdays ([https://nurture.ai/ai- saturdays](https://nurture.ai/ai-saturdays)) has study groups in 50 cities now, and I'm told ~5000 people have been involved in them.

I'm always open for ideas in other ways we can help people get involved and stay up to date. And I'm very interested in supporting students who are trying to build these things in their local community.

</comment></comment><comment><USER18> Question: What is the job market like for folks who spend two months learning deep learning, either through a class like fast.ai or a bootcamp like Insight AI? What sorts of companies hire these people and how much do they pay? I know it varies, but I'm just trying to get a sense to calibrate my expectations of the current market. Thanks for any perspective anyone can offer.

<comment><USER17> As an outsider and / or away from the SV, where such an environment thrives and opportuinities are a few dozens, your mileage may vary a lot. Personal network is always better to refer to than shots in the dark though. In general, you might need showcasing your knowledge through marketable case studies in a sub-domain you are interested in, so that interviews will come from a shared context.

<comment><USER2> Exactly right - it's all about your portfolio. We spend a lot of time throughout the course talking about ways to build that portfolio, and folks on the forums support each other with feedback and help on these projects.

Completing a MOOC isn't going to help you land an interview for a job, generally speaking (although if it's any good, it should certainly help you once you do get an interview!) However if you use your study time to, for example, create useful projects that you make available on github, web apps that you host on heroku, deep technical blog posts you post on medium, etc then you should definitely get plenty of interview opportunities.

I've seen many many students go through this process, so I know it works. That includes students that hadn't been getting interviews or job offers - until I convinced them to spend time building a portfolio, and then they got multiple offers from top companies.

(There are some companies - not too many thankfully - that strictly require a PhD for DL applicants. I've noticed that such companies, that focus on credentials over actual work output, seem to overlap a lot with companies that turn out to have toxic cultures. So I'm not sure you should worry too much about them...)

</comment></comment></comment><comment><USER19> Wow great share! I was unaware of these courses! Great work @jph00

</comment><comment><USER20> That's a really good upgrade.</comment>


Drones, Reverse Engineering and the GPL - <AUTHOR> https://github.com/rs2k/raceflight/issues/225 <comment><AUTHOR> Github comments from my email (since they were all deleted by progod): [http://pastebin.com/raw/yLGpfgtc](http://pastebin.com/raw/yLGpfgtc) ooh, and i just found a github window I still had open! [https://www.dropbox.com/s/00zytxtzqmfgx3y/License%20violatio...](https://www.dropbox.com/s/00zytxtzqmfgx3y/License%20violation%20on%20binary%20release%20%C2%B7%20Issue%20%23225%20%C2%B7%20rs2k_raceflight.pdf?dl=0)

<comment><USER2> Everyone should have email turned on in their github notifications in case of disputes and deletions.</comment></comment>


Crime Visualization of San Francisco - <AUTHOR> http://www.padmapper.com/crimeTest/

<comment><USER3> This is very interesting. A friend of mine who lived on Frederick St, just a few blocks up the hill from Haight, said that crime was much lower on her block. Her theory was that bad guys didn't like to climb hills. And in fact there seems to be a correlation between altitude and crime rate.

There are probably other reasons too. E.g. that the hills are fancier neighborhoods.

But her explanation makes sense. If you're walking around looking for someone to rob, why walk up steep hills?

<comment><USER4> I lived on Beulah, one of the flat streets between Cole and Haight. There were countless bums from GGP & haight street milling about my block, which was considered "Cole Valley", a relatively yuppie neighborhood. My apartment was burglarized and my TV was stolen. I moved up to Stanyan and Rivoli, considered to be the same neighborhood, but straight up a very steep hill, and never saw another human soul again.

<comment><AUTHOR> Interesting. Maybe crime actually does flow downhill to some extent? It would be interesting to see if there's a correlation between crime rate and relative elevation.

<comment><USER5> Completely different city but with lots of hills (seven to be precise): Lisbon. The most dangerous areas are near the top of the hills in places like Bairro Alto, Sé, Alfama, while the main bottom of the hill, Baixa, is much better.

I believe Lisbon has a very low crime rate (for european standards) and my theory for more crime happening on the higher streets is pretty simple: on the bottom the streets are really big and there's lot of space while the streets on the top of the hills are really small so easier to be seen by the police when you're doing something evil.

I like thinking about these kind of relations though!

<comment><USER6> In Brazil (and probably elsewhere in South America), real estate values go down (and thus crime goes up) the higher you go on the hills. Basically, this is because walking up and down the hills is arduous and sucks, and thus people try to avoid living on the hills if they possibly can. Thus you end up with favelas.

If everyone had cars, or public transit, you would probably see better property values on hills, like you do in SF.

</comment></comment></comment></comment></comment><comment><AUTHOR> This is a new visualization layer on PadMapper (an apartment hunting site) which shows the relative frequency of crime in different areas of San Francisco. It's not differentiated between nonviolent crime and violent crime currently (should be at some point). Red is bad, green is good, and nothing means very little crime reported for that area (even better than green within the bounds of SF, simply no data outside).

<comment><USER7> It would be nice if you could filter by the type of crime, not even necessarily violent crime. I might care if someone's selling heroin outside my door, or cars are being stolen, but probably wouldn't care if prostitutes are running about or dogs are unleashed.

<comment><AUTHOR> Good point. It currently needs to be precomputed as images, so it's hard to make it customizable, but it probably makes sense to make a category of unsavory crimes. I would assume that most people dislike living near the same types of crime.

<comment><USER8> How are you precomputing the images? I thought GMaps provided a way to input data and get a heatmap, but apparently I'm mistaken.

<comment><AUTHOR> I couldn't find anything to that effect, so I wrote custom scripts. Basically, it creates an image pixel by pixel, with the pixel value corresponding to the crime density of that chunk of the map, blows it up, smooths it with a Gaussian kernel, and then uses some GIS tools to warp it and chop it up into tiles for different zoom levels.

</comment></comment></comment></comment></comment><comment><USER9> Unrelatedly, PadMapper is excellent (better than other Craigslist mappers I could find): it has a great filtering system, seems to fetch fresh results more often, and can alert you about new places. I found my current place using it.

<comment><USER10> I concur. Padmapper gives me a quality of life advantage over my peers since I can find exactly what I want and monitor for deals while they're getting the luck of the draw off CList.

</comment></comment><comment><USER11> Wow, this is excellent. I'm looking at moving to San Francisco later this year and this is such important information. I knew that places like the Tenderloin were dangerous, but there's an unexpected hotspot in the middle of the Mission that I know to avoid now. Thanks.

<comment><AUTHOR> Cool. It's available on PadMapper itself as an overlay, so you can see where the apartment listings sit on the crime map. It's a bit buried - have to maximize Filters and then Super Secret Advanced Features.

</comment></comment><comment><USER12> that is excellent. There's a 20 block area that is the reddest zone in the city and my guess is that's the Tenderloin.

I wonder what impact an intense cleaning would have on the crime rate there. And enforcement of simple things like public urination laws. And maybe some decent bathrooms.

I recall the kind of things New York did to restore a sense of order. Malcolm Gladwell talked about them in the Tipping Point. Surely similar measures would improve the crime situation in SF.

<comment><USER13> Most definitely, but you'll never get a Rudy Giuliani figure elected as mayor of San Francisco, at least not in the foreseeable future.

My prediction: sometime in the next 30 years the tech industry will have drifted elsewhere, and San Franscisco will be the new Detroit. Right now it's a mixture of the sublime and the awful, but the sublime can always pack up and move to Portland/Austin/Seattle.

<comment><USER12> Most of what is sublime about San Francisco can't move.

But you're right. People in San Francisco don't seem to value public order enough to take the actions required to establish it.

</comment></comment></comment><comment><USER14> About 10 years ago I had to go to SF for a week at short notice and left the booking of the hotel to our office manager - big mistake.

It was on Geary, and a bit of a dive. First thing that surprised me was when I got into the taxi at the airport and the driver asked "Do you really want to go there?".

Next sign that things were not too good was when the chap at reception said - "Do not walk along Geary to get to Union square, walk two blocks up the hill, along and back down to Union square.

Of course, late one night I decided to walk directly back, after all how can it be (feeling reasonably safe going out in Glasgow - which is practically a war zone late at night). I only did this once - getting a taxi back after that!

Turns out the hotel I was staying in is right in the orange zone there.

<comment><USER15> I stayed in a Youth Hostel right in Tenderloin. The crack dealers I met were pretty much only interested in you if you were either interested in buying crack, or were obviously scared of them (at which point they'd make fun of you - hey I would too).

The big surprise for me was how dirty downtown San Francisco was compared to LA. I'd never seen someone actually take a dump in the middle of a city sidewalk, or piss a level down into a train station before SF, and I haven't seen it since. My Full House illusions were well and truly shattered.

<comment><USER14> I asked the taxi driver on the way to the airport if things were always like that and he gave an explanation about adjacent police districts taking it in turns to move these folks from one district to another - apparently this had just happened so things were a bit more lively than normal.

I learned the lesson and booked my own hotel the nice time I visited - <USER2> somewhere _very_ nice right next to Union Square. :-)

</comment></comment></comment><comment><USER16> I can't be the only one who took one look at this and thought: Hey, that's SimCity!

<http://imgur.com/V2Bvo.png>

<comment><USER17> Looks like the City of San Francisco and I employ similar strategies for fighting crime:

[http://maps.google.com/maps?f=q&source=s_q&hl=en&...](http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=CW1yGmTpak6DFVJmQAIdKAe0-CkhAGkAbZqFgDH_rXbwZxNQSg&q=jones+%26+eddy+San+Francisco,+CA&sll=37.77965,-122.414228&sspn=0.016332,0.033023&ie=UTF8&hq=&hnear=Jones+St+%26+Eddy+St,+San+Francisco,+California+94102&ll=37.783901,-122.412887&spn=0.008166,0.016512&t=h&z=17&layer=c&cbll=37.783883,-122.412997&panoid=rO4uBWOPetgycW1rNGUFUA&cbp=12,122.62,,0,7.51)

</comment></comment><comment><USER18> I made CrimeDeskSF (<http://itunes.apple.com/us/app/crimedesksf/id339706696?mt=8>) to visualize some of this data on the iPhone, including a mostly-gimmicky-but-kinda-fun augmented reality view (drug bust 100ft away!). I'm using the datasf.org datasets for mine (not sure how they compare to the crimespotting ones, or if they're the same).

</comment><comment><USER19> Is this normalized by the population density or foot traffic in an area? If not, it's going to be hard to use to estimate personal risk.

<comment><AUTHOR> It isn't. My theory is that crime density by land area alone probably gives a decent estimate of how the area feels to live in. IE: if there's an assault or robbery every week in a one block area where you live, it's going to feel like a seedy place to live regardless of your personal risk of being involved in a crime.

I doubt there's an objectively right answer as to how to best normalize this - I think it just matters that it be understood by the user what it means. There are a number of ways to normalize it depending on what you care about.

</comment></comment><comment><USER20> Very interesting. I would love to see someone build something similar for other cities, perhaps overlaid with Points of Interest and mass transit lines. I think it could be made to be useful for tourists and people who are new to an area.

However even if it can't be made into a usable product it'd be very interesting to see.

<comment><AUTHOR> I used to have markers representing high-rated yelp locations on PadMapper, but they weren't too happy about having their data cached (which is necessary in order to make it smooth), so now it has to be pulled manually by the user.

Transit lines are something that should go on PadMapper at some point. I hope Google opens up their transit layer sooner rather than later.

</comment></comment><comment><USER21> The LAPD has a much more informative interface

<http://www.lapdcrimemaps.org/>

which separates out crime by types. You can clearly see patterns such as GTA's (car theft) along major streets in Hollywood.

</comment><comment><USER22> All I knew about SF crime was that the Tenderloin felt quite dodgy when I visited. So I opened the map ready to be suprised. Instead it turns out my feelings were right. Looks like the crime rate in SF is an inverse function of the distance from the Tenderloin.

</comment><comment><USER23> If you want more details, sanfrancisco.crimespotting.org has them, or you can download the raw data from datasf.org. The hot spot at 16th Street BART is drug dealers and in practice there is no reason not to go there.

<comment><USER11> I'm going to have to disagree. Its accessibility via the muni and the bart, combined with the proximity to the hard-drug areas of town means that there are a number of quick and dirty smash-n-grab robberies of transient ne'er-do- wells. Admittedly I'm biased because I experienced a single one of those datapoints first-hand.

<comment><USER23> OK, I shouldn't have dismissed it so quickly... there are some real issues there too. But I have spent a lot of time there without worrying about my safety.

</comment></comment></comment><comment><USER24> [http://www.floatingsheep.org/2010/06/mapping-crime-in- city-b...](http://www.floatingsheep.org/2010/06/mapping-crime-in-city-by- bay.html)

//related

</comment><comment><USER25> So most of Hunter's Point is crime free? Is this data correct?

<comment><AUTHOR> This is why I'd like to do violent/nonviolent versions. I've heard it described that the Tenderloin has a lot of non violent crime, and that Hunter's Point has much more violent crime. Ultimately, I'd rather be in the Tenderloin if that's the case.

I believe the data is correct, but I'm not sure what the SF police's data policies are. There could be some withholding, etc. The data comes from <http://sanfrancisco.crimespotting.org/>.

<comment><USER26> The San Francisco Police link to <http://www.crimemapping.com/map/ca/sanfrancisco> from their page at <http://sf-police.org/index.aspx?page=1618> which has some discussion of their data policies.

</comment></comment></comment><comment><USER27> No luck getting the presidio data?

<comment><USER28> The presidio is actually a government property, so any crime within the presidio is actually a federal crime and is reported differently. Beware of committing any crime there, the consequences are far more drastic.

<comment><AUTHOR> Thanks for clearing that up.</comment></comment></comment>


How Sun's need to control the code cost them the company - <AUTHOR> http://blogs.zdnet.com/BTL/?p=31418&tag=bitly

<comment><USER3> As far as I could tell perhaps the #1 Sun died was fantastically inept management, most especially in sales:

Sun just wouldn't talk to you directly if you weren't buying mainframe levels ($$$) of hardware. At some point they established a site where you could punch in your credit card, but if you needed more you were at the mercy of their "VARs" and resellers, who in far too many cases also wouldn't give you the time of day (note this was CPU agnostic, it was true for their x86 hardware according to the stories below and for SPARC hardware in my personal experience).

A little quality time with a search engine will find stories from all sorts of startups upset that they were forced to buy Dell since Dell actually wanted to sell stuff to them in any quantity (at the same time HP's sales function was also messed up in different ways).

So Sun missed out on all the startups that became really big; by the time these companies were buying in mainframe quantities Sun was just a bad memory.

One thing that I noticed happening later was companies dropping Sun because their hardware kept changing. In one case they dropped a great discrete Intel Ethernet chip in favor of the ones on the nVidia chipset, in another, they kept changing the management interface. (A bit like one of the things that hurt Compaq, you could buy many copies of the same model, yet adjacent serial numbers might have totally different motherboards.)

It's no accident that Sun's sales organization seems to be where Oracle is making the biggest quickest changes (hiring thousands of salesmen as I recall).

ADDED THOUGHT: I don't think the thesis in the linked article flies. As I see it, it posits that if Sun had done a (much) better job with open source, the business gained from that could have covered for their failures elsewhere.

Better, I say, to have fixed both sets of failures.

<comment><USER4> What you're describing is a classic case of channel conflict. The same problem largely killed off several CAD software vendors back in the day. It was also one of the key reasons why Novell's NetWare business failed after being highly profitable for years.

<comment><USER3> Isn't this more like "channel incompetence" than channel conflict? My understanding of channel conflict is that it happens when manufacturers disintermediate their channel partners by selling directly to customers (which is exactly what it seems Oracle will be doing now).

What Sun did was to fail to police their channel partners and confirm that they would actually sell intermediate quantities of equipment; I know that throughout the '90s it was extremely difficult for me to buy single Sun SPARC systems (workstations and servers), and as noted in my original message too many startups found it _impossible_ to buy six figures worth of systems.

I assume their partners understood that Sun handled the really big purchases (maybe they got a piece of the action???) and that the only conflict was when Sun set up their web storefront for the small purchases a customer could put on a credit card.

</comment></comment></comment><comment><USER5> _The difference between Sun systems and everything else out there in the late 1980's (mostly Novell Netware or Microsoft LAN Manager networks) is that Sun networks just worked, all the time._

Ugh, not quite. I installed and configured an early version of DECNet-PC (ultimately renamed PathWorks) on a 4-node Vax 8650 cluster in late 1986 that networked three Epson PC's over ethernet to a common virtual DOS drive mapped on an RA81. It was solid as a rock, behaved itself well, and easily shared files for WordPerfect, Lotus 1-2-3 and dBase.

An early Netware (2.11?) network had been setup three floors above us to show the power of 'client-server' computing. It crashed at least 5 times daily when it finally would run and its unreliability got two of my co-workers fired because they couldn't make Novell's empty promises work.

Lan Manager? Late 1980's? Are you kidding?

Sun started by unseating DEC in the scientific desktop market with their "workstations," not their network.

Just look at the nodes on maps of the early ARPANet if you don't believe me. PDP's, VAX's and mainframes.

<http://en.wikipedia.org/wiki/File:Arpnet-map-march-1977.png>.

When Scott McNealey announced that the "network was the system" he was taking direct aim at DEC whose installed systems already made up the world's largest collection of networked computers and pc's, were clustered to 16 nodes and more, offered full load balancing, were tied to gigs and gigs of storage using high-speed teflon interconnects and routinely ran 200 days and more without a restart.

Don't get me wrong, Unix/NFS was nice, but it needed a lot more work to compete with DEC, VAX/VMS and Ultrix. What prevented DEC from eating Sun's lunch was it's insistence on centralized computing using terminals. Had they fully understood what they had in DECNet-PC back in 1986 there never would have been a need for Novell, or Banyan Vines, or WFW 3.11 or any of the other wannabe's and Sun's "networked desktop workstation" would have been a much more difficult sell.

Here's the spec for a VMS filename circa 1985: NODE "accountname password"::device:[directory.subdirectory]filename.type;ver

Here's a typical EDT (text editor) command: $ edt orion"system somepwd"::dua0:[sysfiles.configs]mylog.txt;4

Meaning: Use the text editor "edt" to open version "4" of the file named "mylog.txt" that's located in the "configs" directory under the root directory named "sysfiles" on drive "0" attached to disk unit "a" on the node named "orion". Log me in using the "system" account with password "somepwd."

From anywhere in the world as long as it's on the network.

Everytime I configure Samba I think about those Epson PC's.

</comment><comment><USER6> It is hilarious to me that the Linux community continues to retell the "Have you ever kissed a girl?" chestnut after 14 years, and still fails to get the joke.

If you go back and read the original Usenet (not email) drama (<http://www.cryptnet.net/mirrors/texts/kissedagirl.html>), it is Miller who sounds like an adolescent. From describing himself as "chief architect of the SparcLinux port," to the laughable wishful thinking of "Linux is light weight, Solaris is a pig," to the non-sequitur cheapshot of "Sun Quentin" (what is that supposed to mean? that Sun engineers are held against their will? that they're felons? what?), the entire post drips of the naive self-regard that, sadly, marked Linux's kernel community at the time.

I was regularly using both systems in 1996. Solaris worked. Linux did not. Miller was asking to be taken down several notches, and Cantrill did so in the fewest possible syllables.

<comment><USER7> Man reading that reminded me of why I disliked Linux and all of the "hackers" in that space at that time. The adolescent smugness was so thick back then. That email really took me back. OS zealotry seems to have died down since then, that or I have just gotten too old to care or notice.

<comment><USER8> It's still a religious debate. I just talked about it with a professor in the field and he refuses to have a conversation about OS with other similar folks due to the lack of rationality in the debate.

</comment></comment></comment><comment><USER9> I think a confusing hardware lineup, confusing messages and overall confusing strategy have a whole lot more to do with its merger than any compulsion to control code.

Their tech is great. Their management not so.

And I don't believe in untenable situations.

</comment><comment><USER10> Sun was primarily a hardware company and that's where "The Innovator's Dilemma" is applicable.

The PC killed them.

<comment><USER11> Yet Apple survives and thrives. Sun wasn't really a hardware company or a software company. That was the problem, it seemed they really didn't know what they wanted to be, so they tried to be everything and failed.

<comment><USER12> Apple counterpoint is right on the "money". Apple was having quite a lot of problems before wunderkind returned.

This is obviously hindsight, but imagine:

It is early 90s, you are the company that goes by the motto "the network is the computer", and you have this incredibly hot tech called Java that can run on any tier. A bit later and your JME is on _billions_ of mobile devices all over the world.

A Steve Jobs type of leader would have been able to leverage that amazing potential. And the transformation would have been as decisive as Apple's.

<http://en.wikipedia.org/wiki/IMac> to <http://en.wikipedia.org/wiki/IPhone>

The gods must be laughing as key failure here was the abysmal execution of a tech called the "applet".

<comment><USER10> Yes, Sun's inability to execute in consumer markets appears in the failure of applets against flash. And it would also prevent them from creating something like an iPhone.

It seems that just about all current success stories in computing are in consumer markets: search, social websites, smartphones.

It's true that Oracle is doing well and they aren't in consumer markets - <USER2> their market is not showing rapid growth; on the contrary, they are actively consolidating it.

<comment><USER12> And I guess the business lesson here is to learn to spot dissonance in your business strategy.

Sun engaged the consumer market with propaganda. That was the extent of their engagement. (Remember all those Sun ads during the dotcom bubble?)

But in reality, Sun was in reality only engaging geeks. Sun has never failed as a company catering to geeks.

Somebody in that company should have noted that their desire (Sun for the masses) and their actions (Sun for techies) were not congruent.

Proof is in the fact that in the case where the general public has little input - the backend -- Sun was (and remains) spectacularly successful. The mistakes made here are far less specific to Sun, and very much related to the dissonance between being in business to make money and being enlightened hippies who are making substantial contributions to the industry, for free.</comment></comment></comment></comment></comment>


The Density Divide: Urbanization, Polarization, and Populist Backlash - <AUTHOR> https://niskanencenter.org/blog/the-density-divide-urbanization-polarization-and-populist-backlash/ <comment><USER2> This is a great summary of how urbanization intensifies the red vs. blue tribalism we've been seeing over time. Unfortunately, I don't know what the solution to this issue is (if there is one). And at least in my personal experience, it seems like a significant chunk of the left is more inclined to simply write off rural residents entirely rather than look for common ground and compromises.</comment>


Adventures in Curling - <AUTHOR> http://www.chrisfenton.com/2019/07/01/lightly-documented-projects-part-1-adventures-in-curling/ <comment><USER2> The curling broom was definitely one of the hardest projects I've ever worked on. The electronics are pretty trivial, but my fabrication skills just weren't up to the challenge - serious curlers can apply a crazy amount of force to the head of a broom when they're playing.

</comment><comment><USER3> My first curling hack was to attach a speedometer to a tiny RC monster truck wheel and mount both on the end of the broom. When playing as a sweeper, I would slide along and touch the wheel to the ice as the thrower released the rock to get an accurate speed reading of the rock. Especially during draw weight shots where you're trying to hit the button, this aided the call as to whether to sweep or not. The idea may have been good, but the nuances of curling and the fact the ice is constantly changing speed throughout the game made the data not very useful. In the end, a well trained 'eye' was just as accurate and much quicker to make the call.

</comment><comment><USER4> If you produce a good one, you can sell them for $3000 (probably Canadian Dollars) a pop: [https://www.nytimes.com/2018/02/06/sports/olympics/curling-s...](https://www.nytimes.com/2018/02/06/sports/olympics/curling- smart-broom.html)

</comment><comment><USER5> _> This has the interesting effect of decreasing the friction between the stone and the ice, causing it to both decelerate and rotate more slowly_

Decreasing friction decreases speed ? What is this sorcery ?

<comment><USER6> Read it as "both decelerate more slowly and rotate more slowly" and it's an accurate description of what happens when you sweep in front of a curling stone.

<comment><USER5> I see now, what an odd way to say it.

</comment></comment></comment><comment><USER7> Was hoping for more usage of `curl`.

<comment> alex-ant same here</comment></comment>


Cherry Picking: When People Ignore Evidence That They Dislike - <AUTHOR> https://effectiviology.com/cherry-picking/ <comment><USER2> Of course they use the example of focusing on climate change studies that are opposed to consensus, but what about when consensus is actually wrong? For example, high fat diet advocates were accused of cherry picking many years ago, but now we're swimming in cherries and there is no real consensus on high fat diets or the cholesterol theory of heart disease, we're in a transitional period. I think soon there will be a consensus that the prior consensus was morbidly incorrect. So was that really cherry picking?

<comment><USER3> > Of course they use the example of focusing on climate change studies that > are opposed to consensus, but what about when consensus is actually wrong?

Even funnier is that their claim is flat out wrong, based on the study linked in the article.

Claim: For example, a notable study on climate change examined thousands of scientific papers on the topic, and _found that approximately 97% of them support the consensus position that humans are causing climate change_.

Relevant text from the study: We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics 'global climate change' or 'global warming'. _We find that 66.4% of abstracts expressed no position on AGW_ , 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming. Among abstracts _expressing a position on AGW_ , 97.1% endorsed the consensus position that humans are causing global warming.

Fact: ~32.6% of the 11,944 peer-reviewed climate abstracts from 1991–2011 matching the topics 'global climate change' or 'global warming'peer-reviewed scientific literature endorsed the consensus position that humans are causing global warming.

This is one of my favorite memes.

And yes, I realize if we were to contact all of these folks who didn't realize their studies would become fodder for propagandists, _many_ of them would _likely_ change their stance in support of the "consensus" (but not actually, _based on this study_ ), but that's no excuse for literally lying. And people genuinely wonder why there is a distrust of media and science.

<comment><USER4> "found that approximately 97% of them support the consensus position"

versus the more correct

"found that among abstracts expressing a position, approximately 97% of them support the consensus position"

Doesn't strike me as particularly 'wrong' or 'literally lying'. They are looking at where the consensus position lies on a particular idea that doesn't need to be mentioned in every abstract in the field. A paper to do with global warming can analyze reality without stating a cause for global warming in the abstract; that doesn't mean that the paper should count against one consensus position or another.

<comment><USER3> Those two statements are not equivalent.

And the "fact" that is _usually_ printed in the news is 97% of _scientists_.

The distrust, of the media at least, is well earned.

</comment></comment></comment></comment><comment><USER5> At the time, the consensus opinion of Einstien's special relativity theory was that it was wrong. The Nobel committee gave him the prize for Brownian Motion, not relativity.

Galileo also went against the "consensus opinion" when he supported Copernicus's heliocentric theory.

I knew this was a AGW piece when I read the headline.

<comment><USER6> "The Nobel committee gave him the prize for Brownian Motion" This is not true.

The Nobel Prize in Physics 1921 was awarded to Albert Einstein "for his services to Theoretical Physics, and especially for his discovery of the law of the photoelectric effect."

Source: [https://www.nobelprize.org/prizes/physics/1921/summary/](https://www.nobelprize.org/prizes/physics/1921/summary/)

Einstein explanation of Brownian motion served as convincing evidence that atoms and molecules exist and was further verified experimentally by Jean Perrin in 1908. Perrin was awarded the Nobel Prize in Physics in 1926 "for his work on the discontinuous structure of matter".

Source: [https://www.nobelprize.org/prizes/physics/1926/perrin/lectur...](https://www.nobelprize.org/prizes/physics/1926/perrin/lecture/)

</comment></comment><comment><USER7> * or when picking commits they do like

</comment><comment><USER8> For those wanting to discover more cognitive biases:

[https://en.m.wikipedia.org/wiki/List_of_cognitive_biases](https://en.m.wikipedia.org/wiki/List_of_cognitive_biases)

<comment><USER9> And for a decent introduction to critical thinking in general, I can highly recommend Carl Sagan's book, "The Demon-Haunted World".

Among many other good things in this book, there is a chapter called "The Fine Art of Baloney Detection," which is the best primer in detecting bad thinking I've seen.

<comment><USER3> The Fine Art of Baloney Detection

[http://www.inf.fu-berlin.de/lehre/pmo/eng/Sagan- Baloney.pdf](http://www.inf.fu-berlin.de/lehre/pmo/eng/Sagan-Baloney.pdf)

[https://rationalwiki.org/wiki/The_Fine_Art_of_Baloney_Detect...](https://rationalwiki.org/wiki/The_Fine_Art_of_Baloney_Detection)

<comment><USER10> It's interesting to note that some time after writing this, Sagan became aware of the work of Jim Tucker, at the University of Virginia, attempting to validate the "past life" reports of children who've offered verifiable details in their accounts. Many children make such claims, but, as Sagan observed, "Young children sometimes report details of a previous life, which upon checking turn out to be accurate and which they could not have known about in any other way than reincarnation."

I won't assert the truth of it, and I can think of a couple plausible vectors that don't involve consciousness surviving death and into another body (however unlikely they might be), but if it's compelling enough for the author of _The Demon Haunted World_ to suggest it deserves "serious study", I'm the last person in the world to gainsay him.

<comment><USER11> The complete paragraph is in [https://www.reddit.com/r/carlsagan/comments/5q2hub/is_this_a...](https://www.reddit.com/r/carlsagan/comments/5q2hub/is_this_a_quote_by_carl_sagan/)

I think he is not asking a "serious study" because he think it's real, but because at some point we must make the experiments to verify or falsify the theories. (I think that the problem is that there is finite time and finite money for the experiments, so we must select only some of them.)

<comment><USER10> I think we're in agreement. This is him saying, "I think it's interesting enough to expend that effort." No one else has to agree, but that a mind like his thinks so, and based on what I've read of these accounts, I'm inclined to agree.

I have no idea how to go about falsifying that, though. You'd at a minimum have to strictly monitor the information flow into a child's environment, probably from birth, to control for any other vectors for verifiable details. All in the hopes that some of the study population's children will mention something you can confirm they didn't overhear, or whatever, which turns out to comport with the life of someone who's passed. That's perhaps a tough sell to the ethics committee.</comment></comment></comment></comment></comment></comment>


Understanding the code inside Tornado, the asynchronous web server - <AUTHOR> http://golubenco.org/?p=16 <comment><USER2> > Lets say you have 20 threads. You improved performance 20 times, so the rate > is now 4 request per second. Still, way too small. You can keep throwing > threads at the problem, but threads are expensive in terms of memory usage > and scheduling. I doubt you'll ever reach hundreds of requests per second > this way.

Let's say I'm using Java instead of Python. Lets say I use a lot more threads than 20. I will reach _thousands_ of requests per second this way. There are real problems with that approach, and event-driven approaches have genuine advantages, but can people please stop straight-up _lying_ and saying that you can't just throw a few thousand threads at a problem, because you can, and people do.

<comment><USER3> Even with Python (CGI backed by Apache), you can scale with threads. The issue of scaling with threads vs. events is a pretty hot debate, and I think the author sets up this kind of criticism by addressing it poorly. Personally, I fall into the event-driven camp, because it involves the operating system as little as possible (only file descriptors). The C10K link has a great overview of the threaded approach (<http://www.kegel.com/c10k.html#threaded>). Read through the rest of the page for a discussion on the pros/cons of other approaches.

<comment><USER4> Another issue with evented frameworks in python is that you have to ensure _nothing_ is blocking anywhere in your code, which becomes harder the more complex your application becomes. People will often mention IO, database, etc... forgetting that it is also an issue if your request handler takes CPU for N ms (e.g. encoding a relatively large payload in json, etc...). _everything_ needs to be written with async in mind.

Languages/frameworks where async IO is an implementation detail (like some web frameworks in haskell) don't have this issue.

<comment><USER3> I don't think of this as a problem. Tornado is just a small and lightweight non-blocking HTTP server with a couple of convenient libraries. There are some pretty awesome, elegant solutions to the problem you point out, and I like to think of these as extensions to the small Tornado framework.

One really cool one is <http://thomas.pelletier.im/2010/08/websocket-tornado- redis/>, which demonstrates how to use threading to support Redis pub/sub.

For heavy computational tasks that aren't time-critical, you could have an accessory worker thread that chugs queued computations (yay first-class functions) in computational downtime.

</comment></comment></comment></comment><comment><USER5> Some of this is a bit dated. It looks like it was written in late 2009, judging by the first comment. For example, IO loop timers are now stored using a heapq[1] instead of a sorted list, and many of the other issues have been similarly addressed over the last year and a half.

Still, this is a good general overview of Tornado's internals.

[1] [https://github.com/facebook/tornado/commit/b6c4d6d20196fa4fe...](https://github.com/facebook/tornado/commit/b6c4d6d20196fa4fec936ddc6fb252698c1933b9)

<comment><USER6> The heapq change was just last week and has not yet been in any release, so that's a perfectly reasonable omission. :) But yes, this is about a pre-1.0 release, so I would encourage anyone interested in exploring tornado internals to check out a more recent release.

</comment></comment><comment><USER7> There was a very good talk about this at #nodeconf this week by Tom from Joyent. The basic summary was that threaded network servers, like Apache, say, approach the speed problem by "pre-allocating" resources, a chunk for each thread/instance. The issue is that there are a finite number of resources on each machine (RAM mostly), and blocked threads are eating up a slice of those resources even when doing _no work_. This sets the upper bound of number of simultaneous connections that can be handled.

In an event based system the overhead for each connection is at least three orders of magnitude lower, sometimes four or five (hope I'm remembering this right). This translates into _considerable_ increase in number of connections that can be handled simultaneously, not quite the equivalent number of orders of magnitude due to other aspects of the system becoming more of the bottle- necks, but still dramatic.

This was a data-driven observation, not conjecture or assertion. I listened in during a break while the nodejs guys argued about approaches to getting TLS working better - they were worrying about the 1MB overhead for a TLS connection because that's a significant percentage of a connection handler. Think about that for a minute in the context of an apache threaded instance. 1MB matters? wow!

I'm new to this area and this was very interesting stuff and seemed related to the various discussions below about a threaded system can do what an event based system can do.

</comment><comment><USER8> > I also feel obliged to talk a bit on nonblocking IO or asynchronous IO (AIO)

Wait, that is not correct. His IO is blocking. He gets a notification when data is ready, from the select/poll/epoll (the asynchronous part), but when IO is actually performed (read, write, recv, etc), the operation it is still happening in the main thread in user space, and it blocks it.

Currently only the file system (I think) has truly asynchronous and non- blocking IO. It is provided by the aio_* set of system calls and is has been sort of an exotic beast (it is not that popular).

Here is a good chart of the possible IO types and their combinations:

<http://www.ibm.com/developerworks/linux/library/l-async/>

And of course:

<http://www.kegel.com/c10k.html>

<comment><USER9> No the network sockets are set to non block and will return E_AGAIN once you have emptied the kernel buffer. Thats still non blocking.

I dont think he is using aio_* which Linux does not really implement usefully.

<comment><USER8> No. They would return E_AGAIN after the data has been read or written in blocking mode. If select comes back and says "read is ready on socket 4" then user code does read on socket 4 and gets (say) 4K of data followed by E_AGAIN. Getting that 4K of data happens in a blocking mode. To make it non-blocking you would provide a pointer to your user space buffer to the kernel and it would determine when IO is ready _and_ proceed to put that data in the buffer.

<comment><USER10> The way you are using the term "blocking" is non-standard and not that useful. You are calling the syscall "blocking" just because it performs its work inline (in this case, copying a kernel buffer into a user-space buffer). By this definition, every syscall is blocking, even aio_read() because even aio_read() performs _some_ work inline (namely enqueing a read request).

In common usage, an I/O operation is considered "blocking" if the system call does not return until data is available. If on the other hand you set O_NONBLOCK on a fd, you will get EAGAIN when no data is available, so clearly that is the accepted meaning of "nonblocking."

<comment><USER8> > You are calling the syscall "blocking" just because it performs its work > inline.

Yes because there is _another_ mode of IO where it doesn't have to do that. So the reason that is called "blocking" is because there is another mode of operation (albeit it is exotic) where the process does not block while performing IO -- the kernel reads/writes the data on behalf of the process. If this second mode of performing IO did not ever exist you could just interchange arbitrarily asynchronous and non-blocking as synonyms.

Since you didn't bother reading the link I posted here is the basic matrix of IO operations:

                        Synchronous            Asynchronous

        Blocking         read/write           select/[e]poll

        Non-Blocking    O_NONBLOCK+E_AGAIN      aio_*

There has been talk and various attempt at implementing aio_* style IO for network sockets on Linux, but nothing good so far.

> Even aio_read() performs some work inline (namely enqueing a read request).

Enqueuing the read request is not the same as actually copying gigabytes or terabytes of data from disk where the actual work is performed.

<comment><USER11> I think the DeveloperWorks article you've cited is simply wrong, and that's sent you into a tailspin. Select and poll aren't "ways to implement asynchronous blocking I/O".

I think you'll find these links more helpful than the misleading one you've been using.

[http://www.circlemud.org/~jelson/software/fusd/docs/node36.h...](http://www.circlemud.org/~jelson/software/fusd/docs/node36.html)

<http://people.freebsd.org/~hmp/stuff/docs/freebsd_kse.pdf>

I could always be wrong about this stuff; maybe I'm the one with the broken semantics. But I'm pretty sure I'm not.

</comment></comment></comment></comment></comment></comment><comment><USER12> Reminds me of this: [http://teddziuba.com/2009/09/twisted-vs-tornado-youre- both.h...](http://teddziuba.com/2009/09/twisted-vs-tornado-youre-both.html)

</comment><comment><USER13> Personally, I would be more interested in understanding the code inside Tornado, a column of air. :-)</comment>


Show HN: Star Crazy for Android - <AUTHOR> https://play.google.com/store/apps/details?id=com.Miimax.StarCrazy

<comment><AUTHOR> Hi HN! This is my first Android game and I am very keen to get some feedback from you guys. It is an early release, but I have found that early releases are great for gathering feedback and progressing the product for the better.

I appreciate this game will not be for everyone, but constructive criticism is highly welcomed.

Thanks!</comment>