How many lines of code from Linux 1.0 are still in the modern Linux kernel? - <AUTHOR> http://www.quora.com/Linux/How-many-lines-of-code-if-any-from-Linux-1-0-are-still-in-the-modern-Linux-kernel-and-what-is-it <comment><USER2> Fuck you quora.</comment>


List of important publications in computer science - <AUTHOR> http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science

<comment><USER2> Allowed myself to humbly add Bjarne Stroustrups "The C++ Programming Language" to this otherwise very interesting list.</comment>


The Electro-Optic Camera – The World's First DSLR - <AUTHOR> http://eocamera.jemcgarvey.com/ <comment><USER3> Somewhat OT: I don't know much about photography but had always known DSLRs as the gold standard for digital photography. I was surprised to learn that there's actually a bit of a shift going on away from DSLRs towards "mirrorless cameras". If you haven't heard about them, the differences are pretty interesting. Basically, DSLRs are still like the one in this article - <USER2> sensors crammed into an SLR. Mirrorless cameras take more liberties, as if they were imagining a digital camera from the ground up rather than starting with an SLR. Eg: [http://photographylife.com/mirrorless-vs- dslr](http://photographylife.com/mirrorless-vs-dslr)

<comment><USER4> Photography is a hobby of mine. I currently use 2 camera bodies, one is a DSLR (Canon 70D) and the other is mirrorless (Sony A7).

The Sony A7 has a full-frame sensor, which is pretty crazy considering how relatively small it is (much smaller than the 70D body which holds a smaller 1.6 crop APS-C sensor) and under ideal conditions it produces better image quality than the Canon 70D (slightly higher megapixels, but really the physical size of the sensor is what puts it ahead with less noise, plus the Sony sensor technology has better dynamic range).

The Sony A7 is the first camera I've used where the electronic viewfinder is bright enough and fast enough that I can stand using it over an optical viewfinder. And having reached that point it really opens up some great abilities that you can't get with purely optical, like focus peaking (real- time contrast/edge detection to show you which parts of the frame have the sharpest focus) and high zoom in the viewfinder (both of which are really useful especially when using older manual focus lenses which really shine on this camera). Also, just plain seeing the image exactly like the sensor does in terms of depth of field is a real game changer... and yes, you could previously do these with liveview cameras on a bigger LCD attached to the back, but the high-res, bright viewfinder is a really big deal for me as I hate holding the camera away from me while looking at an LCD since it is much harder not to cause sharpness-killing movement that way than when using your face and arms as a sort of bio-tripod.

All that aside, DSLRs still currently win for fast autofocus. Pure phase detect autofocus is just so much faster than the contrast autofocus used by mirrorless cameras. For a lot of photos this doesn't matter, but for some it does. See, eg:

[http://gmcbay.com/slideshow?postId=d80f062f-46e1-44c0-81e7-9...](http://gmcbay.com/slideshow?postId=d80f062f-46e1-44c0-81e7-9d8a6042ca0d&imgId=b3cea71c-0bf8-4c45-5142-e6df0852a6c2)

That is a photo I took last weekend with my Canon 70D. I often carry both bodies (70D and Sony A7) with me when hiking, but there's no way I would have gotten those hummingbird photos with the A7 (regardless of which lens I was using on it) unless I was using manual focus and got really lucky with my timing.

</comment></comment><comment><USER5> Interesting! I always wondered why digital "conversion kits" for 35mm SLR bodies never became more common -- I would have loved to convert my old Nikon FE to digital.

<comment><USER6> The problem isn't that it's impossible - the article demonstrates otherwise - but that it's very difficult to pull off aesthetically well. Consider that what you'd want to add, to be comparable to a conventional DSLR, would be the sensor, LCD, image processor, additional controls, card cage(s), battery - it'd all take up extra space, and all on the back of the body, rather than integrated into the overall space of the body.

So, a fairly substantial engineering task, and that's only for a back which would be designed for either a specific model of body, or a small number of identically sized bodies.

Then consider the numbers you'd be likely to sell, versus competing with the units rolling out of Nikon, Canon, et al.

Which is not to deny the geeky appeal of doing it anyway. =:)

</comment></comment><comment><USER7> Wow, this brings back some memories! I worked with Jim at Kodak on the later DCS cameras; I was part of the team that wrote the Photoshop plugin, Camera Manager and Photo Desk.

Most fun job I've ever had.

</comment><comment><USER8> I'm kind of shocked by the quality. I've seen far worse from all manner of cameras over the last decade or so.

<comment><USER9> My company actually built one of the first fully digital aerial mapping systems (commercially at least) using the slightly newer and publicly available Kodak DCS's (~94). They were very nice cameras, especially for the time. I cant even imagine doing this kind of work with film, it would be so tedious. The speed to process the data was incredible when going all digital. This made up for the loss in detail compared to film for most jobs, especially since our costs were quite a bit lower.

Also interesting, is if i recall correctly Kodak actually made about 100 colour infrared versions too which was nice... until we couldn't reliably source them out! We ended up having to burn the filter off of the CCD of the standard RGB cameras systems with some sort of acid solution turning them back into their natural CIR mode.

</comment></comment><comment><USER10> One thing struck me as being kind of strange. It seems the customer wanted to hide the fact the camera was digital. To what end?

</comment><comment><USER11> 1.3MP (or 1.4 if you're marketing...) - pretty good for the late 80s, but unfortunately it's not colour!

<comment><USER12> I might be wrong, but I think adding color information would not be _that_ hard. Basically you could filter each sensor cell for one of the RGB colors and just interpolate the results for each color channel. See [https://en.wikipedia.org/wiki/Demosaicing](https://en.wikipedia.org/wiki/Demosaicing)

<comment><USER13> That's exactly how the majority of digial cameras work today: [http://en.wikipedia.org/wiki/Bayer_filter](http://en.wikipedia.org/wiki/Bayer_filter)</comment></comment></comment>


Algorithm Reveals Secrets of Leaf Shape - <AUTHOR> http://www.technologyreview.com/blog/arxiv/25117/ <comment><USER2> How is this even news, I have a book on my shelf here about fractals with _much_ better leaf patterns from simple equations that is over 15 years old.

</comment><comment><USER3> Last week: <http://news.ycombinator.com/item?id=1307294> (URL had '?ref=rss' tail)</comment>


Confessions of an ID Theft Kingpin - <AUTHOR> https://krebsonsecurity.com/2020/08/confessions-of-an-id-theft-kingpin-part-i/ <comment><AUTHOR> Part 2: [https://krebsonsecurity.com/2020/08/confessions-of-an-id- the...](https://krebsonsecurity.com/2020/08/confessions-of-an-id-theft- kingpin-part-ii/)</comment>


Ask HN: Critique this hiring procedure - <AUTHOR> I've been thinking about hiring processes in General and I think some of the problems that they face are a result collapsing a complex question very hastily into a binary decision.
Therefore I propose a "multi-layered" decision process, where the final step is made automatically. (This is obvious and I know it's probably not new, if your company tried this in the past I will be happy to hear about it)
First essential rule: "if the candidate has the highest amount of points and is willing to accept the offer, s/he gets hired"
Now, unlike other systems which try to prescribe, this system attempts to describe.
We define "a reason" as any true, legal (as in non-discriminatory) statement about the candidate that is precise enough.
Each reason can be assigned a number of points (negative or positive). The interviewer has complete freedom to assign as much points as he wants to any reason but is advised to respect previous jurisprudence and to justify himself in the cases where he goes against it.
Valid reasons can range (just examples) form: "S/he got Y GPA in University X" to "s/he didn't say hello at the start of the interview" Invalid reasons are for example "unprofessional attire" this is invalid because it is vague. Reasons need be blunt and precise. "S/He wore X and Y" should be valid.
Claim A: this makes the hiring process more traceable and open by providing a paper trail
Claim B: this fights part of the unconscious bias. By requiring interviewers to think and write down reasons. They can examine more critically their decisions.
Claim C: this helps control for overreaction. Assigning helps insure that the effect of each reason is reasonably proportional. I think many would be ashamed to write (-50) for just not saying hello but I worked with people petty enough to reject for less.
char limit, 1/2 to be continued at:
https://news.ycombinator.com/item?id=16105130 <comment><USER2> > some of the problems that they face are a result collapsing a complex > question very hastily into a binary decision.

I don't see anything particularly attractive or helpful here from a candidate experience perspective. The objective of hiring to find people with the right combination of skills/experience/attitude. Slow, bureaucratic processes (interviews by committee) tend to turn-off good people.

Relative to reducing bias in the process, HBR offers interesting advice > [https://hbr.org/2017/06/7-practical-ways-to-reduce-bias- in-y...](https://hbr.org/2017/06/7-practical-ways-to-reduce-bias-in-your- hiring-process)

</comment><comment><USER3> Honestly, its just very difficult to find talented people right now. 99% of your effort will be spent on recruitment. Not on the hiring design process. Which, you are probably right. Its biased, unfair and skewed against inexperienced candidates. But ultimately, it is an instinctual decision. "Do I wish to work with this person?"

Of course, to attract the top talent in the first place. You have to build something they would be proud to be a part of ;)

Good luck and keep iterating!

Also, check out CNBC's The Job Interview. I am sure it will hit close to home:

[https://www.cnbc.com/the-job-interview/](https://www.cnbc.com/the-job- interview/)

<comment><AUTHOR> The idea is that instinct has in itself an unconscious bias. By making the reasoning part conscious we hope to eliminate some of it.

</comment></comment><comment><USER4> Quick caveat before I get stuck into your post: I've been working in the hiring space for the past ten years with companies of all shapes and sizes. Your thought process throughout this post is absolutely on point but the big tl;dr here is that your proposed solution already exists in well established hiring platforms.

Feedback:

The primary difference between the process you describe and the process that most companies already follow is assigning points to influencing factors. In fact, some Applicant Tracking Systems like Greenhouse[1] already do exactly that.

 _check if the point values are reasonable_ \- Who decides this? If it's a person who has met the candidate, they are immediately skewed by their own bias. If they had a positive experience with the candidate, they are going to struggle to see how negative scores could be considered reasonable.

 _Claim E: this process if well documented should be a valid and easy defense against allegations._ \- I'm afraid I have to argue the opposite here. In the EU for example, new GDPR regulations mean that this level of documentation must be made available to the candidate on request and categorically makes you more liable to legal challenges.

[1][http://www.greenhouse.io/](http://www.greenhouse.io/)

[2][https://ico.org.uk/for-organisations/guide-to-the-general- da...](https://ico.org.uk/for-organisations/guide-to-the-general-data- protection-regulation-gdpr/)

<comment><AUTHOR> by check if the point values are reasonable

I meant that the amount of points is not a major outlier, and if it is the interviewer should have written an argument why it's different from the old cases if the other members find the argument convincing it should be fine.

>some Applicant Tracking Systems like Greenhouse

that's exaclty the type of feedback I wanted thank you very much

> Claim E

if we are innocent, why wouldn't this be in our favour.

</comment></comment><comment><AUTHOR> 2/2

Step 2: peer-review The highest candidate is selected, interviewers exchange notes and are asked to:

* check and discuss if their colleagues reasons' are valid and reasonable (for example "the hello" reason might be seen as too petty by some, and will be discarded)

 _check if the point values are reasonable (Checking old similar data to flag outliers is a possibility)

_ check if some of the reasons presented by other colleagues apply to their own candidates. For example, if the "hello" reason was not discarded and your candidate didn't say hello you must apply it to him too.

*Interviewers are asked to bring more focus on the current best candidate's reasons.

If after a peer-review round, the best candidate changes, another round is made or shorten the list and re-interview.

If it does not change, the best candidate is offered the position.

Claim D: this helps smooth out harsh or lenient interviewer bias.

Claim E: this process if well documented should be a valid and easy defense against allegations.

Data Analysis on hiring decisions becomes way more interesting. I'm sure there are tons of trends you can seek out.

Claim F: this allows an earlier detection of discrimination. For example if you find that an interviewer or a committee always removes points from a certain group for a subjective reason more than others in the company. It might be an early warning sign that a problem need be addressed.

Claim G: this allows for a higher quality debate on controversial issues like sexism and racism in hiring.

Machine learning can be used to give suggestions to interviewers about the amount of points to give for reasons. (They should be suggestions no decisions to avoid the pitfalls the lost nuance that classic machine based decision might suffer from)

Example: a fairly easy one is that after a high enough number of universities and GPAs is received one can aggregate the point values into scores that take into account the difference of grading and quality between universities. This might help decide if GPA x in Y is better or worse than GPA z in C, a very hard question to answer fairly if we didn't have the data.

I think this method offers a more traceable, open and perhaps fair way of hiring without suffering from the lack of nuance that traditional automatic hiring suffers from.

Problems:

\- Money: this process might require more man power.

\- Tooling: to be efficient this process requires tooling and automation

I would be happy to hear your ideas, improvements and experiences with similar systems. I think the idea of this system is very similar to that of a neural network.</comment>


Jupyter Receives the ACM Software System Award - <AUTHOR> https://blog.jupyter.org/jupyter-receives-the-acm-software-system-award-d433b0dfe3a2 <comment><USER2> The Jupyter team deserves every accolade they get and more. The console, notebook, and now JupyterLab are some of the key reasons why Python's data ecosystem thrives.

I think Jupyter notebooks are quite useful as "rich display" shells. I often use them to set up simple interactive demos or tutorials to show folks or keep notes or scratch for myself.

That being said, I do think the "reproducibility" aspect of the notebook is overblown for the reasons other comments cite. Notebooks are hard to version control and diff, and are easy to "corrupt." I often see Jupyter notebooks described as "literate programs," and I really don't think that's an apt description. The notebook is basically the IPython shell exposed to the browser where you can display rich output.

This is where I think the R ecosystem's approach to the problem is better (a bit like org-mode & org-babel). For them, there is a literate program in plain text. Code blocks can be executed interactively and results displayed inline by a "viewer" on the document (like that provided by RStudio), but executing code doesn't change the source code of the program, and diffs/versions are only created by editing the source. At any point, the file can be "compiled" or processed into a static output document like HTML or PDF.

This is essentially literate programming but with an intermediate "interactive" feature facilitated by an external program. RMarkdown source doesn't know its being interacted with or executed, and you can edit it like any other literate program.

Interaction, reproducibility, and publication have fundamental tensions with each other. Jupyter notebooks are trying to do all three in the same software/format, and my sense is that they're starting to strain against those tensions.

<comment> your-nanny I agree, 120%.

I like the r approach so much more.

<comment><USER2> I mean, as a medium for interactive exploration where you might want graphs and widgets or other rich/dynamic output, I still think the notebook is superior. But as a medium for developing complete, share-able, reproducible data analyses, I do think R has the upper hand.

<comment><USER3> Graphs, widgets and other rich/dynamic output is also possible with the R approach.

[https://rmarkdown.rstudio.com/](https://rmarkdown.rstudio.com/)

Additionally, Rstudio is an incredibly powerful IDE for data analysis.

EDIT: Interestingly, however, I still use ESS [https://ess.r-project.org/](https://ess.r-project.org/) but that's because I love Emacs too much :D

<comment><USER2> I understand. I believe I pointed that all out my comment above. I wasn't saying that I find the notebooks superior because they allow for rich & dynamic output, but that I find it superior to RStudio when __all you want __is a quick exploratory REPL capable of rich /dynamic output. I simply find it easier to fire up a notebook and start noodling around than writing an RMarkdown notebook. That really only holds if I'm not overly concerned with keeping or sharing the notebook. Otherwise, I believe RMarkdown is the better option.

I also tend gravitate towards ESS, and probably split my R development time between emacs and RStudio. I've even written a very kludgy Rmd notebook mode that uses overlays to show evaluation results from code chunks. But RStudio is very well-designed and ESS just doesn't compare feature-wise, sadly.

</comment></comment></comment></comment></comment><comment><USER4> So well deserved. Jupyter is critical infrastructure, helping the scientific community address the reproducibility crisis. It's so great that it's free and open source, and can be shared, used and contributed to by scientists all around the world.

<comment><USER5> Not only that, it's great for tutorial material as well!

</comment></comment><comment><USER6> Trivia: Ju-py-ter = Julia Python Terminal

Although it's gone far beyond just Julia and Python now.

Edit: Ahurmazda is right.

...the core programming languages supported by Jupyter are Julia, Python and R. While the name Jupyter is not a direct acronym for these languages, it nods its head in those directions. In particular, the "y" in the middle of Jupyter was chosen to honor our Python heritage.

[https://github.com/jupyter/design/wiki/Jupyter- Logo](https://github.com/jupyter/design/wiki/Jupyter-Logo)

<comment><USER7> Is it pronounced jupeeter or jupieter?

<comment><USER8> And is NumPy "Numb Pie" or does it rhyme with "lumpy"? (My apologies to everyone who will picture something lumpy next time they're doing a bunch of math).

<comment><USER9> It think it's pronounced numpy...

That said, I've always rhymed it with lumpy, haha

<comment><USER10> In conference talks, most people (including NumPy's creators) pronounce it "numb pie".

</comment></comment></comment></comment></comment><comment><USER11> Jupyter is awesome, well deserved award. Our CTO is working to integrate JupyterLab and JupyterHub directly into GitLab.

<comment><USER12> Getting a sensible git-diff would be great!

<comment><USER13> nbime can help ! NoteBook DIff and MErge. [https://github.com/jupyter/nbdime](https://github.com/jupyter/nbdime)

</comment></comment></comment><comment><USER14> I started using Jupyter + Python recently, can't say enough good things about the project.

Sometimes you want to present data, graphics and have a bit of interactivity. The notebooks make it easy to share your code/data/graphics. And it beats a PowerPoint any day (for this use case anyway).

Thanks Jupyter Team!

</comment><comment><USER15> Am I the only person in the universe who doesn't like Jupyter? I much prefer tools like Rmarkdown and Sweave.

<comment><USER16> We would love to hear what you like about Rmarkdown and Sweave. Jupyter tooling is always improving, and we are very interested in engaging with users about their needs, and helping grow the ecosystem to be able to address those.

<comment><USER17> My biggest frustrations with Jupyter are (see #4 for comments on Sweave etc):

1\. The default front-end is a weak platform for getting work done.

It's a JavaScript code editor. It will never be as good as my personal text editor configuration. It will never be as good as an IDE like RStudio, Spyder, or Pycharm. It's good that there are keyboard shortcuts for doing things like adding cells, and extensions for things like folding cells and adding a table of contents. But it still isn't terribly comfortable to use all day. Also I personally hate doing everything in a browser. Apart from some useful notebook extensions, there are no viable alternative front ends yet.

2\. Running a remote kernel is a pain in the ass (cat a config file then manually tunnel 4 ports over SSH), and I can't seem to get it to work on Windows at all.

This is an issue at my company because we do a lot of work on remote servers that can be accessed only through SSH or JupyterHub. Individual users do not have control over the latter, so we are stuck with the inadequate default experience I just described above.

3\. No kernel other than Ipython is mature.

IRKernel is getting there. Everything else is at best a beta-quality product.

4\. Notebooks are not a plain text file format.

Hand editing a notebook is messy. They do not play well with version control systems and diff tools. RMarkdown and Knitr/Sweave are just preprocessors for established plain text formats (Markdown and Latex with some extra syntax). With those formats you can take advantage of a wealth of existing tooling, as well as having the freedom to edit the file in a normal text editor without having to rely on a special front end. Ironically having everything formatted as JSON should make it easier to write those special front ends, but I have not seen any good ones yet.

<comment><USER18> 1\. Jupyter Lab (note: NOT Jupyter Notebook) is an attempt to make the interface more IDE-like. It's still not Rstudio due to the Jupyter's notebook nature, but it's close enough for me.

I do prefer Rstudio's REPL approach of being able to run code by line or by blocks (likely inspired by MATLAB's IDE), rather than Jupyter's approach of executing code by cell (which was inspired by Mathematica). They both let you try stuff out easily while maintaining state, but the former is far easier to productionize.

2\. Remote kernels over SSH aren't that hard -- I do this all the time via SSH tunnels. I start Jupyter Lab in an SSH console (usually on a cloud-based VM), and create a tunnel to port 8888 (the default) using my Windows SSH app (Bitvise). 1 port. That's it.

3\. No comment - I only use the Python kernel.

4\. Correct. Notebooks do present challenges for version control.

<comment><USER17> _Remote kernels over SSH aren 't that hard -- I do this all the time via SSH tunnels. I start Jupyter Lab in an SSH console (usually on a cloud-based VM), and create a tunnel to port 8888 (the default) using my Windows SSH app (Bitvise). 1 port. That's it._

I want the opposite. I want to use a remote kernel with a local client.

<comment><USER18> Umm, yes, in my case, the kernel is running remotely on a cloud VM. My client is a local browser (Chrome) which connects to localhost:8888, which is a tunnel set up to connect to the remote machine on port 8888.

This lets me run computationally heavy Jupyter calculations on a beefy remote backend in the cloud. My local browser merely talks to that backend via a tunnel.

Here's something on the web that describes this [1] -- except with Bitvise on Windows, you don't have to enter any SSH commands. The tunnel setup etc. is all done via a GUI. This is a pretty standard SSH tunnel technique. You can use this for more than just Jupyter.

[1] [http://www.vickyfu.com/2017/04/using-jupyter-notebook- remote...](http://www.vickyfu.com/2017/04/using-jupyter-notebook-remotely-in- azure-vm/)

<comment><USER17> Again, that's not what I mean. I want to run Jupyter (or some other front-end) on my laptop and have it talk to a kernel running on a server. You're describing running both Jupyter and the kernel on the server.

<comment><USER18> Oh I see now. You want to run the raw kernel with no front-end on the remote machine and communicate with it via the 0MQ/JSON transport layer. I'm curious, what is the advantage of doing this vs. simply running an instance of Jupyter on a remote machine?

<comment><USER17> I don't necessarily want to use Jupyter as the front end. This way lets me use e.g. Pycharm with the kernel running in a console.

BTW I managed to get it to work. I think I had missed a port the first time I tried.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER19> IPython's interactive shell is way more convenient than using the default python command line when trying things out. But I just don't get why one would need the whole Jupiter thing on top of that, except for maybe making presentations.

One thing that I think is missing from IPython though is the ability to save a given state of the interpreter, with all the variables in it. So that one could preform a time consuming data loading/parsing once, and restart from that point if some variables get messed up. Jupiter cann't do that either, afaik.

<comment><USER10> Anything graphical or interactive: maths, science, data, web, machine learning, etc. Interactive includes interactive widgets, but also almost any kind of exploratory programming.

Even stuff that's technically plain text is easier when you can display tables and other formatted text. E.g. I have a tiny little notebook that generates LaTeX code for a normal distribution table; it's a notebook because then I can display an HTML preview in a few lines of code.

Jupyter can't save interpreter state - that seems essentially impossible without adding state-saving and -loading code to every single library and dependency.

<comment><USER19> >>Anything graphical or interactive

but ipython is already well integrated with matplotlib in the --pylab mode, and is pretty interactive. Thats how I use it.

re:saving state - its technically a nightmare, I agree. I though it might be possible at OS level - just dumping the whole process. There still would be issues of what to do with open files, network connections, etc, but somehow it seems that in many use cases that would be enough.

<comment><USER10> Yeah, I used matplotlib like that for years (at times all day every day), but the notebook interface makes it much easier to keep track of lots of figures and where they actually came from, so I switched to that 4-5 years ago. The figures are also much more lasting, as they save in the notebook right next to the code, so it's a lot easier to go back and make sense of old work, especially with markdown cells documenting things right there too.

I think even saving state by dumping the whole process is unfeasible. What happens if some dependency gets upgraded, e.g. for a critical security hole? The problems seem unavoidable, so I think we're stuck.

</comment></comment></comment></comment><comment><USER20> Thoroughly deserved - Jupyter is helping push science forwards.

</comment><comment><USER21> A huge congrats from Azure Notebooks which is entirely built around Jupyter. We actually started way before that by connecting the visual studio REPL to Jupyter and the whole experience (technically and people) has been delightful.

</comment><comment><USER22> Wow this is awesome! It got the Award quickly also I think Juypter hasn't been around as long as other recipients.

<comment><USER13> As we say in the blog post, the Jupyter Name has been around only since 2014, but the work started in 2001. So 17 years is a good check of time !

<comment><USER23> Though the part that people mostly identify with the project, the Jupyter Notebook was introduced only in 2011. I remember that vividly as I was writing my thesises at that time and spending quite some time playing around with it :)

<comment><USER13> Though the current notebook is the 6th prototype, so there've been quite some work before actually being made "public". Hope the notebook didn't distract you too much from your PhD !

<comment><USER24> If open source didn't distract from ph.d's, we wouldn't have had ipython! ;)

</comment></comment></comment></comment></comment><comment><USER25> I'm excited for this award, it's certainly well deserved.

In my heart, I wish the "data community" would show more care for security (and, related, privacy), with deeper focus on features that simplify access control, and guidelines on how to enforce "reasonable defaults".

I fear that Jupyter in many companies is becoming the next Jenkins, with unconstrained access to all data vs all infra, and this will lead to more and more incidents and leaks.

I very much hope that recognitions like this one will foster not only better tools and support, but also best practice and security considerations.

But, back to the focus of the post, congrats on this success!

<comment><USER13> Thanks for your comment. One of the next focus area, where we are looking for funding and help, is making sure the right restrictions and permissions are in place.

We probably will put that in the context of GDPR/HIPPA/FERPA and follow these guidelines to make Jupyter "Ready" for these framework. We can't say that Jupyter it itself compliant, as you need to see in which context it is deployed, but we want to make it as easy as possible for a team of researcher with low budget, or a companies with 1000+ user to make it easy to deploy a secure, auditable and safe Jupyter environement.

</comment></comment><comment><USER26> Well deserved. There are few softwares that totally amazed me when I first used them. (I used Jupyter for the first time last week).

I was like "Did they actually achieve this?"

<comment><USER27> I've been using Jupyter for some time now since it was strongly recommended/almost-required in my school's classes. One thing I think Jupyter team achieved is reliability of common interface. Like, regardless what my data is, white noise, music, image, matrix, human face... I know I can easily output it and get some sane representation.

</comment></comment><comment><USER28> I really like jupyter and I am looking forward to jupyter lab and where it takes computing.

It is really great for solving one off problems or learning.

The jupyter code itself while verbose is pretty extensible also. I just put together something that lets me connect to my spark kubernetes pod.

I think being able to customize jupyter and add new kernels (languages) is where it becomes really powerful and awesome.

<comment><USER29> Jupyterlab is here. I've been using it basically full-time since August, but it recently went to Beta, so it's ready for use.

The major downside is that it disables JS by default within notebooks, so if you're using Bokeh you'll have to install the jupyterlab extension, but the innovations around files, downloads, views on notebooks, etc. are worth the price of admission.

</comment></comment><comment><USER30> Tools like Jupyter (and Mathematica) don't really match my mental model. I'm fine using them as a REPL (in which case they're like a bloated ipython or ROOT prompt), but as soon as I go back and change something, I get confused about the internal state.

<comment><USER13> Internal state is often a problem, look at project like stichfix nodebook, and dataflow kernels, they make things a bit easier by re-executing cells.

On of the issue is you always have internal state as soon as you interact with a data source or sink. If you read/write from a API, then rest is stateful. Your file system is stateful... etc.

It's an interesting but hard problem, we'll be happy to have more help with.

[https://github.com/dataflownb/dfkernel](https://github.com/dataflownb/dfkernel) [https://multithreaded.stitchfix.com/blog/2017/07/26/nodebook...](https://multithreaded.stitchfix.com/blog/2017/07/26/nodebook/)

<comment><USER27> Problem with that is, sometimes some cell in the middle is computationally intensive and I don't wanna run it again. Just going back and changing one function shouldn't run the whole computation.

</comment></comment></comment><comment> oh-kumudo Congratulations! Jupyter is such a gem. It has become such a critical piece in the whole data analytics/ML landscape that I can't think of living without.

</comment><comment><USER31> This is great. Without Jupyter, I doubt I would have started learning data science a few years ago.

</comment><comment><USER32> Do you have a favorite tutorial or simple use case that helped you grok it?

</comment><comment><USER33> I just started using Jupyter a couple of days ago as I'm starting to learn ML. I'm really impressed by it. Would love something similar for Ruby.

<comment><USER13> Good news ! It exists ! It's called Jupyter ! Just install one of the non- python kernels[1], for example the Ruby one [2], and create a new Ruby Notebook !

1: [https://github.com/jupyter/jupyter/wiki/Jupyter- kernels](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels) 2: [https://github.com/SciRuby/iruby](https://github.com/SciRuby/iruby)

</comment></comment><comment><USER34> Wish they would add variable inspection to Jupiterlab!

</comment><comment><USER35> I am non programmer business guy who 'excels' a lot. Been teaching myself pandas through notebook. Awesome combination.

</comment><comment><USER36> Cannot imagine liking data science nearly as much as I do if Jupyter didn't make it so easy to quickly test new ideas. Well-deserved.

</comment><comment><USER37> The other "innovations" that ACM lists alongside were real innovations " Unix, TeX, S (R's predecessor), the Web, Mosaic, Java, INGRES " now they are handing out awards for "copy commercial software but make it free" projects. Its funny how copying is a bad thing in an essay but applauded in software (as long as it has the right license).</comment>


Julia – A Fresh Approach to Numerical Computing (SIAM Review, Feb 2017) [pdf] - <AUTHOR> http://julialang.org/publications/julia-fresh-approach-BEKS.pdf <comment><AUTHOR> From Evelyn Sanders review:

 _This issue 's Research Spotlights section contains two papers. The first article, "Julia: A Fresh Approach to Numerical Computing," by Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B. Shah, presents the software package Julia. It includes an overview of the language with a concentration on numerical computing, accompanied by a set of examples which can be downloaded and run while reading the article. It quickly moves beyond a simple tutorial of basic features, offering a detailed perspective on design philosophy in the context of other and perhaps more familiar programming languages, explaining the ways in which Julia manages to be both dynamic and fast. While the examples start simply, they progress quickly and include disparate subjects such as different approaches to Brownian motion, and best methods for parallelization. The article will be a nice introduction to readers with an interest in numerical computing who are currently unfamiliar with Julia, but it will also be of great interest to those already using the package who would like a deeper understanding of the design of Julia, thereby gaining a better facility to optimally use it._

Source: [http://epubs.siam.org/doi/abs/10.1137/17N974264](http://epubs.siam.org/doi/abs/10.1137/17N974264)</comment>


Actually, Slack really sucks - <AUTHOR> https://medium.com/@chrisjbatts/actually-slack-really-sucks-625802f1420a#.yz78a6rku <comment><USER2> At least for our company, Slack has been a net positive. Most issues around notifications are resolved by customizing your notification preferences per channel.

 _Now try this in Slack. Remember, it's one of 10 issues you'll have in a day, so you start with creating a group chat instead of a dedicated channel._

Stop. All of the problems that follow this statement are a result of creating a group chat instead of a channel. If you're creating a group chat, you're most likely doing it wrong. You should have long-running specialized channels by topic that includes the relevant people. For example, if you're discussing a bug in XYZ analytics, post in #xyz-analytics. If you're discussing something specialized and short-lived, create a channel anyway. For example, #xyz-1.1-release. This helps in allowing engineers to focus on channels that are relevant to them.

 _Go through all the channels with notifications showing, reading through each channel. Realise most of the content wasn't relevant to me. Sometimes miss content that was that had shot too far up the page overnight._

Unless that communication simply wasn't happening (in which case, Slack is a making a positive impact), that communication existed in some form elsewhere. It might have been email or in person, but either way, if the volume increased for you it's probably because you are simply privy to it now where you weren't before. Perhaps it's not relevant for you. That's fine, because at some point it will be, whereas you would have been excluded before. That means being able to head off bad decisions before they've had much time to gestate.

I've established a guideline for my group that all (non-sensitive) communication should happen in public, and that means on Slack. That alone has made a big difference in making sure people are in the loop and that bad decisions are visible to everyone. Slack isn't my ideal vehicle for this (I'd prefer something behind a firewall), but if it gets people talking out loud, then sure.

<comment><USER3> > I've established a guideline for my group that all (non-sensitive) > communication should happen in public, and that means on Slack. That alone > has made a big difference in making sure people are in the loop and that bad > decisions are visible to everyone.

This is one of the main challenges I'm seeing. Getting people off DM's and into public chats. I explained several times that DMs should only be for sensitive, personal conversation. Everything else should go into channels. But people seem to drift into DMs to "not disturb" others, or because it's only a question to one person etc. Our stats show around 40% public chat vs. 60% in DM (sometimes even 30/70).

I some times wish I could block DMs completely, but I don't think it's possible. Any tips?

<comment><USER4> Whenever someone sends me a DM, I usually reply with a "please use #tech" or whatever the relevant channel is.

Our company onboarding guide includes the line: "All communication on slack _must_ be in public channels. All emails must be cc'ed to a mailing list." Of course there are sane exceptions to this, but everyone follows this generally, and we get >90% public communication over slack every month.

Our current stats are:

>95% public channels, 0% private channels, 4% DMs (>1M messages)

<comment><USER5> You probably know this, but any public channel a user is in can be relayed to a 3rd party (entirely) if that user subscribes to an app that was approved. All of those messages are also available for any of your admins to pull on demand. All of this said, I'm sure your org is ultra careful what things you talk about in Slack. I am just saying this in the event someone doesn't know.

</comment></comment></comment></comment><comment><USER6> Email is really useful and nobody cares.

\- If you need a way to discuss specific topics with specific people over time, you make a new mailing list.

\- If you want to have one single conversation within a specific group and topic, and follow it logically, just reply to the thread. You can forward the thread and the conversation remains, and can continue.

\- You can ignore all conversations that aren't addressed to you, or go back and read them, one thread at a time.

\- You can archive all of this, hierarchically, per mailing list, and search or browse it later.

Now - if you want to have a LIVE conversation, nothing's going to beat a chat room. It really does not matter what kind of chat room. But sometimes you join the chat late - it's good to see the old conversation and catch up. And after the conversation, you might want a log of it later. Somehow, most chat services in the world seem to miss this.

IRC has kludges that support this. Jabber was supposed to fix this (I think?), in a platform-independent way. Yet nobody wants to use Jabber. (Probably because most implementations suck or are incomplete)

Here's my question:

If an entire ecosystem of software has thrived for a generation using existing tools (IRC, mailing lists, and now Jabber), why the hell is every corporation in the world dead-set on running away from them screaming? THEY'VE WORKED FOR 20+ YEARS! _edit_ answering my own question: they don't have support contracts.

</comment><comment><USER7> I've found that slack takes up way more time for me too.

I'm usually in 50 IRC channels at once. My workflow when I open irc is to check my hilight window for pings, and then glance at the channel list for backscroll chatter in channels I follow closely (if bored, also look at chatter in other channels). I then respond to things, and switch back to the hilight window. I occasionally glance at this window to check for more pings and chatter in said channels. Each check is a quick glance, and can be done without context-switching.

I am in a bunch of slack teams, but I only really care about one channel in each. It takes much longer to handle these, since I need to open a tab for each, and if I've been pinged go look for the ping.

IRC sucks, but at least everything is in one place. I'd be far more productive with slack if there was a unified app for all teams with a dedicated notifications panel.

<comment><USER8> Have you tried the slack desktop app? It is much better than having to keep a bunch of different tabs open. I find it easier to handle multiple teams that way.

<comment><USER7> I will, thanks

</comment></comment></comment><comment><USER9> We (founders of ConferenceBadge.com) never enjoyed Slack nor other chat apps because they encourage immediate attention, or you just end up missing out. And we are a team of 4… can't imagine using it in larger organizations.

We never found any tool that would completely satisfy our collaboration needs, so we created Missive: [https://missiveapp.com](https://missiveapp.com)

It's built on the idea that conversations should be scoped and archivable, just like emails. It's both a full-featured email client and chat app. We can share emails and comment within threads, as well as create new chats from scratch and invite only the right people to discuss specific matters. When a topic is over, we just archive or snooze it like emails.

I have written in details about our vision: [https://medium.com/missive- app/building-the-team-communicati...](https://medium.com/missive-app/building- the-team-communication-app-of-the-future-ec5418517738)

<comment><USER10> The way you describe it reminds me of Google Wave. That was way too complex for most people though.

</comment></comment><comment><USER11> "It's kinda like having 10 Facebooks to simultaneously look through, then make sense of. Actually, no, its worse. Facebook isn't about returning back to, its about relaxed consumption of content. Slack on the other hand is meant to be about getting things done and being there to respond."

Love it!

<comment><USER12> yeah right to the point

</comment></comment><comment><USER13> Slack's identity management is really painful.

Why do I need to create a new account for each slack team I want to join? I am the same person.

Joining a slack team should be an entitlement that gets granted to my identity -not an entirely separate account.

</comment><comment><USER14> Slack and the other various IRC-reinventions start with the assumption that more communication is always better. I think it's time we challenged that assumption in the face of mounting evidence to the contrary.

</comment><comment><USER15> The author makes a good point that the reliability of Slack on intermittent connections is indeed pretty horrible. It's nearly impossible to send or receive messages, as I experienced while in foreign countries.

</comment><comment><USER16> My main gripe with slack is the abuses. At my company its mostly a water cooler or a way to break the headphones contract.

As well people using @channel gets tiresome when you're on vacation and slack is on your phone.

Basically slack is noisy in my situation, i wonder if for many?

<comment><USER17> I just set myself as "away" when I'm on vacation. @channel won't send you a notification when you're away, it'll by default only notify you when somebody directly messages or tags you.

If you're focusing on something you can put slack in "Do Not Disturb" mode for a desired amount of time.

</comment></comment><comment><USER18> This is surprisingly common with any chat app. I'm sure Whatsapp users are familiar with groups (with family, friends or coworkers) where everyone just sits and blabs on and on and it becomes a nuisance (especially with large group sizes).

Also because chat is so informal, everyone posts needlessly (shoot from the hip with no consideration for whether you're wasting someones' time)

One solution is the judicious use of @ (in Hipchat at least). People should also consider chat to be informal mail rather than chat when you're in the office.

</comment><comment><USER19> I've found that Slack is a big time suck as well.

@mention which generates an email is the only way to pull me in a thread. Meaning that my email client is still the main source of communication.

Slack is good for non-work related threads with my co-workers. We have a channel dedicated just to miscellaneous non-sense which can be rather entertaining. What is nice about the channel is that it isn't a long running email thread that needs to be muted and I only check it out when I want to. However, it could just as well be an IRC channel...

</comment><comment><USER20> I see similar criticisms of Slack a lot, and I think a solid portion of these are general gripes with using one chat system instead of Email + other solutions.

My biggest gripe with using Slack is that teams are managed individually. I can't have one Slack account from which I can view all of my teams and interact with them. Rather, I have to create separate accounts for each one, as if each team is using a completely different service. Then I have to remember a multitude of emails, usernames, and passwords.

</comment><comment><USER21> I work remotely, and I use Slack to keep an open dialog with my team. I used HipChat before that. They both have their upsides for this use case, but I think that it's basically a great tool for what I'm doing. I can easily see how wide use of Slack at a large company could get overwhelming quickly, but that's not so much Slack's fault (HipChat would do the same thing) as much as a general probelm of communication.

</comment><comment><USER22> Torn on this one. I'm sympathetic because of how the OP complains that he's been pulled into Slack because of its popularity. I don't think being virally popular is a demerit -- sometimes popular things are popular because they are good -- but the trade off comes when bystanders are forced into the hivemind if they want to do things that previously didn't require that hot-app-du-jour. And I've only used Slack as a social outlet, not in the workplace, so I can't comment how it compares to Hipchat or good old fashioned Gchat in overall trade offs in the workplace.

But this is where the OP totally lost me:

> _In the olden days, you'd send around a group email. As things progressed, > relevant people would be added to the email chain and irrelevant people > removed. It wasn't on trend and it might have been clunky but it served its > purpose. If a person needed information from one specific person, they could > contact them individually in the chain. Everyone could input ideas and get > the issue solved. When the issue was out of the way, the whole thing could > be shut down with a press of the delete key._

Those are some seriously rose-tinted lenses. Again, I haven't used Slack for work, but unless Slack is significantly clunkier than HipChat/Gchat, and the Slack client sends an electrical shock upon each message sent/received, I'm having a hard time believing that Slack is overall less pleasant than the email-chain-discussions of the past.

Ignoring the fact that the OP describes the most idyllic discussion-by-email- chain situation in this history of the universe when arguing why Slack chat is so inferior, the main flaw in his argument is that maybe his company has cultural communication problems overall that are exacerbated by features of Slack that are helpful to others?

Here's his main complaint:

 _Now try this in Slack. Remember, it's one of 10 issues you'll have in a day, so you start with creating a group chat instead of a dedicated channel. Now add someone. Yup. You've now just started an entirely different group chat. All the context of the conversation has gone and you've found yourself starting over, having to re-explain the whole situation for each new person added. You've also suddenly got a shitload of group chat windows open. Oh, and they aren't named like channels, so which one of those four group chat windows you've now got open was it again? Oh shit. You got it right, but someone else didn't. They've responded in the chat you were all using previously before you realised someone needed adding._

Sure, it's annoying to have to bring someone up to speed when inviting them into an ongoing conversation. Has the OP considered how fucking annoying it is to the person you've just _forcefully_ invited into your conversation to read through several paragraphs/pages of pre-existing chatter to get themselves up to speed? It sounds like there are issues in Slack's interface that could be cleaned up, but the problem of new people being added into a group chat without being able to see the history is a _feature_ that compels a few good practices:

1\. Don't start a group chat for something that could be sussed out in a channel.

2\. Don't start an important group chat without thinking of all the relevant stakeholders who might have input on the issue. Is it possible in Slack to invite someone into a group chat even if they're AFAIK (which would allow them to see the history of the conversation without having had to actively participate)?

3\. For those situations in which you need to invite someone ad-hoc, make sure you can sum up the issue in a couple of sentences (e.g. "Hey Jane, Bob tells me you know about this [URL to Github issue]"). For situations in which the new person needs to know all of what you've discussed before and behind her back before you decided to rudely invite her to your chat, see Point 2.

<comment><USER23> Actually, maybe you should try Slack in earnest with a project team before assuming the OP criticisms are wacked.

I tried it with my team about a year ago, and all of the OP criticisms resonate with me. We don't use Slack anymore.

<comment><USER22> I think what turned me off was the headline. If it had been more well-scoped, e.g. "Slack sucks balls when it comes to doing real work", I'd be less complainy. I've enjoyed Slack for conversation in medium-ish interest groups/clubs (100 <= x < 500), in which the channels aren't overwhelming and there's almost never a need to do private chat with more than one other person.

Everything in the context of work loses its allure, such as "unlimited" vacation time.

</comment></comment></comment><comment><USER24> Slack is so buggy - why is that not an issue. The android app is just the pits.

I prefer [https://flock.co](https://flock.co) (disclosure: was once part of the team that built it) it is fast, not buggy and makes different choices than slack for almost all the same problems.

</comment><comment><USER25> Shameless plug regarding Slack being distraction source: I am trying to scratch my own itch here so I'm working on a Slack app that should help with that, www.collabq.com. Would love to hear your opinions!</comment>


Star Maker - <AUTHOR> https://en.wikipedia.org/wiki/Star_Maker <comment><USER2> I'm happy to this posted on Hacker News. This book is one of my favorite books and ahead of its time.

The idea that stars could be living entities, that the universe itself could be a fully conscious living entity, and that the creation of the universe might be... (well I won't spoil it for you!).

Please read this if you get a chance to!</comment>


Windows Phone 8's Kid Corner Available on Android - <AUTHOR> http://blog.kytephone.com/2012/10/windows-phone-8s-kid-corner-available.html

<comment><USER2> this is a cool feature for those who have little kids and the only way to shut them up is to give them your phone</comment>


Boosting Email Engagement and Conversion with Send Time Optimization - <AUTHOR> https://www.sparkpost.com/blog/boosting-email-engagement-conversion-send-time-optimization/ <comment><AUTHOR> I had a lot of fun writing this post. Data munging was done with Apache Spark and analysis with Python pandas. While I can't provide the code, hopefully the post includes enough recipe detail to follow along.</comment>


Straight-line Expense Recognition of Leases - <AUTHOR> http://www.hhcpa.com/blogs/audit-accounting/straight-line-expense-recognition-of-leases/ <comment><AUTHOR> I came across this discussion and finds it very helpful. I do have some questions as I am working on similar leases with yearly rent increases and lease incentives.

The situation is: The 1st six months are free, the 7th to 12th month, there is a partial rent abatement. We also receive moving and tenant improvement allowances.

Lease is for 7 years and I straight lined lease expense over the life of the lease. Monthly lease expense excluding Amortized lease incentive is $35,640.

Minimum lease payment in year 1 is $38,800 Total lease incentive approx. $271,000 which is amortized over the life of the lease and equals To $2,988 per month.

Since my 1st 6 months are free, what would be my entry in the first 6 months? Also, would my recurring entry be:

Dr. Rent expense $35,640 Dr. Deferred asset/lease Incentive $2,988 Dr. Deferred asset. $672 ( excess payment over expense) Cr cash. $38,800

To record the incentive: Dr. Other receivable. $271k Dr. ST/ LT deferred liability $271k

I need your help to get some perspective on these entries.

Tnx</comment>


Using AI for music source separation: Demucs by FB Research - <AUTHOR> https://tech.fb.com/one-track-minds-using-ai-for-music-source-separation/ <comment><AUTHOR> "Demucs works by detecting complex patterns in sound waves, building a high- level understanding of which waveform patterns belong to each instrument or voice, and then neatly separating them." Audio examples and source code (Github) links are provided in the article.</comment>


Decrypt message, win 1 free year of killer web hosting - <AUTHOR> http://blog.phpfog.com/2010/12/16/php-fog-contest/

<comment><USER2> Answer has been posted - [http://blog.phpfog.com/2010/12/16/we-have-a-winner- of-the-fr...](http://blog.phpfog.com/2010/12/16/we-have-a-winner-of-the-free- php-paas-hosting-contest/) (hello world)

</comment><comment><USER3> I like that they use nginx as an asset host and not also as a reverse proxy in front of Apache.

Xerxes and Slowloris still rule supreme.

</comment><comment><USER4> User Popcorn: you are being censored. All your comments are dead on arrival.

<comment><USER5> You keep saying this. I wonder if it's helping much. There's probably a reason banned people don't know they are banned.</comment></comment>


Ask HN: What has progressed in your field in 2010? - <AUTHOR>

I want to hear about the new libraries that you've found that you can't live without. The technologies that have matured the point of being accessible. The languages you feel hit a critical mass of community support, libraries, and documentation.<p>Were any new breakthrough algorithms discovered? Did any one thing change how you program? Some tool that you think will change your life for the better going forward? <comment><USER2> For me, it's Node.JS. Most useful new tool since jQuery. I wrote a domain name generator with it a couple months ago:

<http://impossibility.org/>

It handled simultaneous Reddit and HN front page traffic without any problem. I'm pretty sure server load never passed 10%. The script ran for more than a month straight and performed 120k+ searches without crashing.

Considering the first Node.JS release was only 18 months old, it's amazingly stable and scalable.

<comment><USER3> Thanks for posting this.

I've heard of Node.js before, but had no clue what it was about. Your comment made me curious enough to go and look it up.

Ended up watching this video[0], and now it's all starting to make sense.

[0] <http://www.yuiblog.com/blog/2010/05/20/video-dahl/>

</comment></comment><comment><USER4> Well, suppose I'll start, though not sure this quite fits. I work for a university, and this year we completed a move of a very sizable portion of our business processes from proprietary software to open source software. The biggest breakthrough was moving some stuff to Drupal (<http://drupal.org>). Drupal was basically a snowball that picked up steam as it rolled downhill and is now being deployed across campus. 2011 will see it (along with other open source software) taking over the vast majority of our university's apps _. The best part has been that most of the effort has come from the ground up with a strong community emphasis.

_ (With the caveat that Oracle seems to have bought the administration's souls, so it will still be used as the database of choice for many apps for years to come, but eh they own MySQL now too).

<comment><USER5> I'm working on a big project for Emory. Would love to hear your feedback if you're up for it. No email in bio so please check mine. thanks!

</comment></comment><comment><USER6> For me, this year, it was using GPUs for numerical computing. I know that has been growing for years, but it seems like this is the year when it finally exploded: I work for in geophysics and the AGU (American Geophysical Union) conference last month had dozens of papers on using GPUs.

It's easier than it looks to get started - go to AMD or Nvidia and download their software, which includes demonstration code. Then hack the demos.

I wrote a summary of the basic ideas used in OpenCL at <http://acooke.org/cute/APractical0.html> (but there are also many more online).

</comment><comment><USER7> Automotive infotainment underwent a major shift to Linux from WinCE. There is even GENIVI alliance (<http://www.genivi.org/>) to speed up the adoption.

</comment><comment><USER8> for me it's javascript. many awesome libraries are being created (backbone, CoffeeScript, knockoutjs). I'm also excited about few new game engines for js (craftyjs - component based approach). I also like the approach behind couchdb/couchapp (2 layers apps).

</comment><comment><USER9> I ended up writing a fair bit of screen scraping this year. By far the best way I have found so far is using <http://simplehtmldom.sourceforge.net/>. I have tried yahoo pipes but scraping HTML with regex is a bad idea. I've also tried dapper but didn't find it reliable enough.</comment>


Khan Academy Competitor? Alison.com founder on the future of Online Ed - <AUTHOR> http://www.wiredacademic.com/2011/09/khan-academy-competitor-mike-feerick-of-alison-com-talks-about-future-of-online-education/

<comment><USER2> The whole site feels really cheap. I closed it at first because I thought I typed the URL in incorrectly and had landed on a parked domain.

The best thing about Khan Academy is its personality, which makes the learning very accessible. It's a no bullshit content delivery system, but this feels like it's straight out of Generic Corporate Designer™.

Also, if you want a clickable url: <http://alison.com>

<comment><USER3> I agree with you on the site's looks, very uninviting... And that damn animated button in the right corner drives me crazy.

</comment></comment><comment><USER4> I went to the site and it was virtually empty. A split second later I realized I had AdBlock Plus on. I disabled it and what I've seen looked like a spam site. I think the comparison with Khan Academy is extremely unjustified here.

<comment><USER5> I agree the comparison is unjustified. What makes Khan Academy so good is the quality of Khan's explanations and teaching on the videos.

<comment><USER6> I actually want to see an educational process/technique that produces content, even half as good as Khan Academy, _repeatably_.

My question has always been - will Khan Academy lose relevance after Sal Khan ? Or is there a recipe that allows the creation of more Khan Academys.

<comment><USER7> _My_ question has always been - will Khan Academy lose relevance outside Sal Khan? He can't be an expert in everything. In order to scale, I'd expect that he'd have to be able to pick people who are great teachers and also domain experts.

<comment><USER8> And he's not an expert in 90% of the things he teaches. He's just quite good at understanding things quickly and explaining it to others, like Feynman was. Being called Khan Academy is apropro, it's mostly about his teaching and he is irreplaceable. Fortunately, video lives forever and most of the subjects are not transient.

<comment><USER7> Feynman only taught physics. Sal Khan teaches maths, astrophysics, economics, history, etc etc...

The potential for misinformation from someone who is teaching a subject that they are not an expert in, and have just "understood it quickly" is quite worrying. he has been criticised on this before, see <http://www.nas.org/polArticles.cfm?Doc_Id=2029>

Economics is a lot like maths, and there is crossover so it's no surprise that he is good at both. But History is a different kind of subject.

If he is really irreplaceable, then there are severe limits on it. I hope and expect that even if Sal Khan is a trailblazer, others can do follow his example.

<comment><USER8> I've learned about a lot more than physics from Feynman. I also see little evidence that classes are taught by experts in their subject matter except at the university level, and even then only rarely.

But let's post a link to Khan's comments/rebuttals to the claims of the NAS article you cited.

<http://news.ycombinator.com/item?id=2633796>

<http://news.ycombinator.com/item?id=2634324>

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Non-free content? A scrolling marquee and pulsating Join Now buttons? And calling your well established and much-loved competition 'buzz-heavy'.. yeah right. This isn't going anywhere.

</comment><comment><USER10> If anyone is interested in a less corporate, more fun version of Alison, I started Curious Reef a while back. It's a social learning website.

For an example of someone actually learning something, here's a post by someone who just finished the MIT 6.00 Intro to Comp Sci class on Curious Reef: [http://curiousreef.com/class/mit- opencourseware-600-introduc...](http://curiousreef.com/class/mit- opencourseware-600-introduction/forum/topic/just-finished-the-class-100/)

</comment><comment><USER11> It looks corporate and soulless - not fun at all.

</comment><comment><USER12> This is the polar opposite of what the Khan Academy is all about: reducing barriers, open transfer of knowledge, transparency, no middle men.

</comment><comment><USER8> This article seems to be an ad for this guy's product, which is non-accredited college degrees from an online school. There's a lot of organizations doing online degrees, this is not very unique.

The comparisons to Khan are bizarre.

</comment><comment><USER13> I am happy to see innovation happening in how people get 'an education'. As usual, some of the most interesting stuff is being done by people outside the established providers (cf. _The Innovator's Dilemma_ ).

</comment><comment><USER14> It's interesting - Khan Academy has such massive following, but from a technology standpoint I've seen much cooler stuff... www.smart.fm comes to mind.

<comment><USER15> Sometimes, clear and intuitive explanations can go a lot farther than technology.

</comment></comment><comment><USER16> how can they win UNESCO award. Besides design, they are only promoting there diploma courses(paid) everywhere.

</comment><comment><USER17> Khan Academy provides an easy way to download content for later offline viewing. This site, however, seems to try to do the exact opposite!

Using flash to restrict consumption and bugging users with registration offers is not a better way to learn.

</comment><comment><USER18> Competing in what? They don't offer the same courses!

Alison has stuff like "personal development", "touch typing" and even adobe products and a bit of Mathematics and Science, while Khan Academy is mostly Math and Economics in detail.</comment>


A Python tutorial on Bayesian modeling techniques - <AUTHOR> https://github.com/markdregan/Bayesian-Modelling-in-Python <comment><USER2> I am the author of this tutorial. If you are interesting in contributing a section to this tutorial, please get in touch. Some suggested topics: survival analysis, mixture models, classification, time series models... Twitter @markdregan

<comment><USER3> Looks pretty nice, good job. I think survival analysis is a very underrated tool. I'm working in UX now and there's a lot of test setups were survival analysis makes a lot of sense but isn't used (mothly because people don't know it).

<comment><USER4> Would you like to expand on that?

</comment></comment></comment><comment><USER5> It's a good initiative!

However, I think the introduction could be improved by briefly describing the "why/what" of Bayesian modeling before you get into the first Hangouts example.

<comment><USER2> Good idea. It does jump in pretty quick. I'll update this during next revision.

</comment></comment><comment><USER6> Great initiative!

I have a few suggestions, maybe i missed it, but a prerequisite section would be useful both for knowledge and platform, software etc.

I am new to python and believe this tutorial would be great for me. However in the case of novice-users as myself, lots of time is spent getting the environment right rather than understanding the code.

For example, after downloading and installing anaconda, jupyter and seaborn, i stumble on error message "C:\Anaconda3\lib\site- packages\ipykernel\\__main__.py:89: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....)"

And here i am stuck, my next step, had it not been this post, would be to investigate syntax changes in python.

Maybe I that's not a correct way to address that problem however that is mainly my point. If the tutorial is targeted to beginners as me, a few more pointers to common errors setting the environment up would be helpful!

Thank you for otherwise great tutorial and keep up the good work!

</comment><comment><USER7> This is great work. I don't have any substantive comments (need to read it in depth for that). I did miss the lack of "next" links, though - not sure if there is a Jupyter-native way to do that.

I like the matplotlib style created for this too.

<comment><USER2> Thank you. Next links would be nice and keep the user within the nbviewer mode (which formats the notebooks correctly). I will add these.

</comment></comment><comment><USER8> Nice work, thanks for taking the time to put this together. Bookmarked.

</comment><comment><USER9> I don't have any Google Hangout chat messages to run the first example of using jupyter. I know that you are not going to share your data, but it should be handy if some fake conversations could be included. People like me like to first install the applications and then run it to see whether it works as claimed. I installed the conda distribution and the jupyter notebook works correctly. (I installed conda in ubuntu and then seaborn, PyMC3 and panda (PyMC3 and seaborn with pip since conda install 2.3 of PyMC3). It works.

I should say that the first step is to clone:

cd where_you_want_the_data_to_be_copied git clone ....

# and now start jupyter notebook with

jupyter notebook

# go to File/open/ and select the first section.

I see that I can edit the markdown. I translated the introduction to section 0, here it goes. Thanks for this tutorial. The graphics are nice.

### Sección 0: Introducción Bienvenido a "Bayesian Modelling in Python" \- un tutorial para personas interesadas en técnica de estadística bayesiana con Python. La lista de secciones del tutorial se encuentra en la página web del projecto [homepage]([https://github.com/markdregan/Hangout-with- PyMC3](https://github.com/markdregan/Hangout-with-PyMC3)).

La estadística es un tema que en mis años de universidad nunca me gustó . Las técnicas frecuentistas que nos enseñaron (p-values, etc.) parecían rebuscadas y en última instancia di la espalda a este tema en el que no estaba interesado.

Esto cambió cuando descubrí la estadística Bayesiana - una rama de la estadística bastante diferente a la estadística frecuentista que se suele enseñar en la mayoría de las universidades. Mi aprendizaje se inspiró en numerosas publicaciones, blogs y videos. A los que se inician en la estadística bayesiana les recomendaría fervientemente los siguientes:

\- [Doing Bayesian Data Analysis]([http://www.amazon.com/Doing-Bayesian- Analysis-Second-Edition...](http://www.amazon.com/Doing-Bayesian-Analysis- Second-Edition/dp/0124058884/ref=dp_ob_title_bk)) by John Kruschke \- [Python port]([https://github.com/aloctavodia/Doing_Bayesian_data_analysis](https://github.com/aloctavodia/Doing_Bayesian_data_analysis)) of John Kruschke's examples by Osvaldo Martin \- [Bayesian Methods for Hackers]([https://github.com/CamDavidsonPilon/Probabilistic- Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming- and-Bayesian-Methods-for-Hackers)) fue para mí una gran fuente de inspiración para aprender estadística bayesiana. En reconocimiento de la gran influencia que ejerció en mí, he adoptado el mismo estilo visual que se usa en BMH. \- [While My MCMC Gently Samples]([http://twiecki.github.io/](http://twiecki.github.io/)) blog de Thomas Wiecki \- [Healthy Algorithms]([http://healthyalgorithms.com/tag/pymc/](http://healthyalgorithms.com/tag/pymc/)) blog de Abraham Flaxman \- [Scipy Tutorial 2014]([https://github.com/fonnesbeck/scipy2014_tutorial](https://github.com/fonnesbeck/scipy2014_tutorial)) de Chris Fonnesbeck

He creado este tutorial con la esperanza de que otros lo encontrarán útil y que les servirá para aprender técnicas bayesianas de la misma forma que me ayudaron a mí. Cualquier aportación de la comunidad corrección/comentario/contribución será bienvenida.</comment>