The Future of Notebooks: Lessons from JupyterCon - <AUTHOR> http://willcrichton.net/notes/lessons-from-jupytercon/ <comment><USER2> I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering.

So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE.

All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the problem of execution order that some people here mention). Then I may move those functions into a normal Python module and build up complex classes, add unit tests. Or if the goal is communicating with other people the results of running that code, I'll add explanatory text and mathematical formulas, break it up into sections, include references to papers and web links.

I think what people miss here is many software development tasks, especially in data science and machine learning, are essentially exploratory data analysis. And the outputs are graphs, tables of data etc, designed to be read by humans. Jupyter's ease of use for iteration, and the ability to interleave documentation, formulas, graphs and tables with the code is a big win.

In my daily work, I have my IDE and Jupyter open side by side. The Jupyter notebooks are version controlled like my .py modules.

Over time, I expect editors like VSCode to edit Jupyter notebooks natively. And also more code editing to move from IDEs into the Jupyter (via integrating the Monaco editor). We will all benefit from that crosspollination, whether we are principally software engineers or data scientists.

<comment><USER3> > And also more code editing to move from IDEs into the Jupyter

Jupyter, and all REPLS in general, rely on a concrete top-level execution context to provide useful feeedback. Much of the code written outside of data science doesn't really have that, which is why you don't see notebooks being used for software development very often.

<comment><USER4> I frequently use IPython or REPL.it for software development. It's not my main editor, and I don't write large chunks of my code there, but for rapidly prototyping a small subset of some task I find them invaluable.

But you are right about notebooks - I don't find them particularly useful, except as a better REPL, and no-one I know uses them for "normal" (non Data Science) software development.

<comment><USER5> I think my favorite non "data science" use case is web scraping. Ad-hoc nature of the task lends itself to using a notebook. A lot of times I don't want the full suite and boilerplate Scrapy gives you so I write it out in a notebook with Parsel & Requests. Once done export and clean up.

</comment></comment></comment></comment><comment><USER6> I understand why they became popular, but as a software engineer considering how they work, I am just full of disappointment. we're going to spend the next ten years re-inventing every single software engineering best practice for jupyter's weirdo environment.

<comment><USER7> This is definitely one of my concerns too. Ad hoc code inside these notebooks is almost completely unmanageable from any reasonable software maintenance perspective, and refactoring code out of them is prohibitively difficult as well. I really want something to emerge that combines the best of both worlds of an IDE and notebook development, but there isn't anything close currently.

<comment><USER8> What? What are you using Jupyter notebooks for where you want maintenance? They should be records of data analysis/ procedures, not code that runs in production or something.

<comment><USER9> Maybe read the article. They are experimenting with putting the notebooks directly into production ala bash script. I don't think this is a great idea either.

<comment><USER8> Yeah, I read it after, silly of me to comment first.

</comment></comment></comment></comment></comment><comment><USER10> As the speaker for the scheduling notebook talk that got referenced here, I can probably give a few insights about how we're using notebooks. A lot of the risks and concerns brought up here were talked about in the session. The slides ended up on [https://conferences.oreilly.com/jupyter/jup- ny/public/schedu...](https://conferences.oreilly.com/jupyter/jup- ny/public/schedule/detail/68348) (and hopefully the talks themselves will get posted soon).

In particular I referenced how we treat and emphasize notebooks as an integration tool which acts as a good place to combine actions with documentation, visuals, and output logs. There's a section on Integrating Notebook which outlines how we approach this. There's also a strong emphasis on pushing complexity and shared code into the repositories housing notebooks. Effectively you end up with a lot of same best-practices found in non-notebook development, and the same abuses that lead to unmaintainable code -- which sometimes is needed in the short term.

So far we've had a lot of success with notebooks in production as parameterizable templates, or as a way to easily produce scheduled reports or machine learning experiments. Many users just provide the parameters while supporting teams provide the tested templates. Other users like being able to simply schedule their iterated work without needing to translate to another medium. One of the biggest wins though is gaining a shared interface for debugging, experimenting, and reusing code by having notebooks as the output artifacts of execution (even if it's just executing some other code elsewhere on behalf of the user). Papermill made a lot of this possible by separating input notebooks from output notebooks, and by being able to inject runtime values into those output notebooks.

<comment><USER10> We also are doing a blog series on how we're approaching notebooks at [https://medium.com/netflix-techblog/notebook- innovation-591e...](https://medium.com/netflix-techblog/notebook- innovation-591ee3221233) (the scheduling post is the second in the series). Also I'd recommend listening to talks from the conference with an open mind. There were a lot of great discussions and positive energy around notebooks.

Hopefully these help describe a few of the new patterns and tools available in the notebook space.

</comment></comment><comment><USER11> It is a disaster that more than half of the research papers do not provide data and the code, required either to reproduce research (if it is a theoretical one) or reproduce results analysis. Wider adoption of JupyterLab (and alike) will help to solve this problem. Using PDF and paper for distributing research results feels like XIX century these days.

</comment><comment><USER12> Most comments here express disbelief and disappointment in Jupyter from the software engineering point of view.

What exactly is wrong with it? I use Jupyter daily and find no other Python environment more productive, be it scripts or IPython or IDEs. Granted, I work in scientific computing and use Python for data wrangling and stats. I find immense value in interactivity and iteration speed.

<comment><USER13> Maybe I can provide a bit of perspective on this, as I have lots of conflicting feelings about Jupyter.

When I'm doing some bit of data wrangling or just exploratory work with data I quite like it - at least at first. As you pointed out, it's really easy to extremely quickly iterate on things and start to get an idea of what's in the data, what techniques work, and which don't. It's great.

Until it isn't. I'm probably "using it wrong" but all the suggestions I've seen for doing it "right" start cutting into the iteration time. If I'm in Jupyter it's because I want to flail around real fast and see what happens. And that causes all sorts of problems. Which cells do I need to rerun together? Which cells should I _not_ run again. What the heck is actually in all these variables right now anyway, because I don't remember which order I've run (and rerun) all the cells in. And what was that one approach or parameter that really worked well that one time? I don't remember.

These sorts of things aren't an issue with a non-Jupyter approach. And because I'm taking my time anyway I'm probably being diligent with source control, and all that. Even pulling code out of a notebook into a more stable workflow is a pain, because you have no assurances you can just copy/paste a cell and have it work. In my experience, it never will.

It's one of those things that has a pretty heavy costs and benefits. And unfortunately they aren't really possible to separate.

Not coincidentally, I feel the same about dynamic languages and even less usefully typed static languages. Jupyter takes an already fairly extreme position on the stable(safe)/easy continuum and really ramps it up to a point where it's hard to wrangle time and work done easy into some sort of stability. It's great, and it's awful.

<comment><USER14> I think notebooks could be hugely improved by saving a snapshot of state after each cell, and having the linear order of cells only move forward in time. Accidental variable reuse (especially from a cell that is in the "future" or maybe no longer exists) is a huge source of bugs.

<comment><USER13> From other comments, it sounds like something called Cocalc might be of interest to you. I haven't even looked at it yet but it sounds like it does at least some of those things.

</comment></comment></comment></comment><comment><USER15> People complaining that notebooks are not good for software engineering are missing one important point – notebooks are very often used by people who are not software engineers and/or not doing software engineering work.

(The point is otherwise valid, I have seen software engineering in notebooks and it was horrific.)

</comment><comment><USER16> "jupyter is the new bash"

As much as I love Jupyter this is a bridge too far even for me. There's nothing about Jupyter that makes sense for non-interactive scripting. Although, I suppose it would help you do block chain calculations... which would also be a terrible idea.

<comment><USER17> Inline documentation and charting (which for data tasks is a honking huge deal). It's literate programming with an actual use-case. Very into this idea.

<comment><USER3> None of that is what people use bash for, however.

<comment><USER17> You'd be surprised. It's certainly not _all_ of what people use bash for, but there are a lot of sketchy ETL (and sampling, and small-scale analytics) jobs out there held together with bash and awk and hopes and dreams, and replacing those with something like this is a substantial win.

</comment></comment></comment></comment><comment><USER18> If Jupyter is not your cup of tea you might like Cauldron, the unnotebook - [http://www.unnotebook.com](http://www.unnotebook.com)

Interview with the author in May, 2017 - [https://www.linkedin.com/pulse/why- cauldron-might-right-data...](https://www.linkedin.com/pulse/why-cauldron- might-right-data-analysis-environment-you-dan-mayhew)

</comment><comment><USER19> As a pretty heavy notebook user (for computer vision and machine learning), I have to say that all of these new features are nice but they don't address the real problems with the current ecosystem. Namely, there's no good way to transition exploratory notebook code to python modules. We need linting and refactoring tools _in_ the notebook and something better than `aimport` for moving code into separate `.py` files.

</comment><comment><USER20> I despise notebooks being used in production for the reasons given early in the article: you can execute code in any order so authors usually end up with spaghetti and long pages without clear flows. I would rather scientists use and learn the tools that have been developed over decades on collaborative code writing using version control. It helps integrate their solutions too.

<comment><USER21> This is a code style problem not a notebook problem: the same people would probably write spaghetti without clear control flow if writing a Python batch script instead of a Notebook.

<comment><USER20> I agree, but tools like version control do help a ton to manage and mitigate these kinds of risks. Actually, it could be super super cool if someone figured out how to version control notebooks, including tracking of code execution flow.

</comment></comment></comment><comment><USER22> To everyone frustrated with notebooks, I urge you to check out nextjournal.com:

\- Notebooks are automatically versioned

\- You can reference and reuse parts of other notebooks immutably through something they call "Transclusions": [https://nextjournal.com/nextjournal/transclusions](https://nextjournal.com/nextjournal/transclusions)

\- The technology-stacks underlying each notebook are immutable, meaning that notebooks work on any machine (no "hidden" dependencies).

\- You can explicitly reference other results instead of relying on global state, making execution order irrelevant.

It's currently in private beta, but people are starting to use it in production. (EDIT: You can sign up using the code curryon2018)

(Disclaimer: friends of mine are building this and I used to work on it in the past.)

</comment><comment><USER23> I love jupyter notebooks, but unfortunately the code editor is unusable to me. I cannot stand the bizarre parenthesis auto-completion, and the autoindentation settings; and there's no easy way to remove them.

<comment><USER24> I feel the same way. One workaround is that some editors have a plugin. Emacs has a mode called ein that is fantastic. As you poke around getting the snippets to work, you can cut/paste into a different file/buffer. After you start the Jupiter notebook copy the login token, run: ein:notebooklist-login then ein:notebooklist-open and away you go.

<comment><USER25> Just be careful to use [https://github.com/millejoh/emacs-ipython- notebook](https://github.com/millejoh/emacs-ipython-notebook) instead of the unmaintained [https://github.com/tkf/emacs-ipython- notebook](https://github.com/tkf/emacs-ipython-notebook). Somehow the latter comes up as my first search result.

</comment></comment></comment><comment> ur-whale "On the education side of things, Jupyter is quickly gaining adoption in universities around America, particularly for data science courses. Conversely, data science is increasingly becoming students' first exposure to programming and computer science"

Am I the only one this two sentences worries deeply?

Or is it just a rehash of the twenty year old "my first exposure to programming was Excel" (which, from what I'm reading is what the article is talking about : a glorified spreadsheet).

</comment><comment><USER26> I can't recommend enough using Atom with the Hydrogen plugin. A Jupyter kernel runs in the back and you can execute a line or multiple lines of code at once just like the web notebooks. But you're editing a plain code file, not an .ipynb, which is easy to check into git, move to inside a library, and get full IDE tooling on.

</comment><comment><USER27> When you consider that it is impossible to reuse notebooks in each other... Or unit test them... Or that it is not especially easy to version control them in any sort of branch/merge workflow... Jupyter Notebooks are much closer to Excel spreadsheets than they are to what most people would consider actual programs.

<comment><USER21> >impossible to reuse notebooks in each other... Or unit test them...

It's quite easy to unit test notebooks. Whenever you write a function you want to test, write some unit tests (or even just asserts) in the same cell it's defined, then any time the cell is run to define the function, the tests are run too. Likely you'll be doing some manual tests when you write a block of code cum function, and it's not hard to copy the input crafted from those manual tests to a unit test.

<comment><USER27> _write some unit tests (or even just asserts) in the same cell it 's defined_

As many people will do this as presently write tests for their Excel macros or formulas.

</comment></comment></comment><comment><USER28> It would seem that notebooks were specifically tailored for my use cases (being in academia and doing data analysis), in practice I found that I drift away from notebooks every time I try them and stick to a good old console (ipython or bash) instead. Here are a few things that irked me:

-It's browser based. Yuck. This means you can wait upwards of ten seconds for a session to load up every time you want to start one. Also, generally speaking, I deeply dislike browser-based applications that have nothing to do with the internet. It just feels so clunky. My workflow gets interrupted as I have to play between windows _and_ betwen tabs, and whenever I want to switch from a task to another I have to remind myself 'well this is my web browsing instance that also happens to be my notebook', it's not natural and gets in the way. By contrast, a terminal advertises itself as exactly what you want to use it for. It's also very snappy.

-Following up on browser-based awkwardness, my god is this thing slow. Initializing kernels. Interrupting kernels. Restarting kernels. God save you if you want to import more than a couple libraries. Wait you made a mistake and want to stop and restart that one cell? Guess you just have to go through the whole kernel thing again I guess. So you launched something and it turns out it eats way more memory than expected? Well, 'sudo pkill -9 firefox' it is, do not pass go, do not collect autosave. It just does not play nice and lags behind every thought or action you want to take. Again, contrast with a plain terminal that just does the job and does it _now_.

-Speaking of exiting, these things don't seem to be able to do so gracefully. If something hangs up you just have to kill everything if you want to resume working, instead of good old ctrl-C. The fact that killing everything _also_ tends to kill the web browser you used to do actual web browsing is just icing on the cake.

-Setting things up can be dodgy. Should your notebook fail to start, error messages are extremely cryptic. Documentation for troubleshooting is lackluster. No one ever seems to have any two identical issues on Stackoverflow, which is also plagued with WorksOnMyMachine (tm) syndrome. If you ever have to mess up with both Python 2 and 3, you may be in for a good time.

-The notebook format in which sessions are saved is supposed to be human readable and editable. In practice I have never seen a single human read or edit any of these by hand, for the very simple reason they are horrible to read or edit by hand. Again, why do I have to load a file in a cryptic format in my web browser when I could just %paste my snippets off of my editor or even just use my IDE's builtin capacities (code runner on vscode, F9 in spyder, etc.)? Plus, compatibility issues between versions can make your notebooks suddenly unusable (and hard to salvage due to the aforementioned format issue). This doesn't happen with snippets pasted onto a terminal.

-IDE integration has been sloppy in my experience. I've been told this may be subject to change, however.

Note that all this criticism doesn't mean I don't respect the work of the people behind this - they've made a fantastic tool that just doesn't happens to fit my use cases, for now. Every once in a while I try to work with notebooks, find they're still unusuable, shrug and go back to my terminals. As the experience improves I may make the switch one day, who knows.

<comment><USER29> I agree that probably working on bash feels more snappy but Jupyter is not that clunky as you mentioned. My laptop is a mediocre one and I can run the Jupyter, navigate to the code and open the editor in total like 3 seconds.

In my experience, kernels work just fine unless you are doing something you shouldn't. My kernel problems occur at points where I have a memory issues. Though I agree with you that you don't understand what's wrong once the kernel dies.

<comment><USER28> >In my experience, kernels work just fine unless you are doing something you shouldn't.

Here's the thing. I'm sure many of the complaints I voiced can be chalked up to lack of experience. But even if it weren't for the issues about documentation and troubleshooting, I'd still have yet another tool to manage, learn to use, learn best practices, etc. Instead of just getting things done with my tool it just gets in the way and tells me there are things I should and shouldn't do. It takes time to learn about it.

And now, I'm not sure the investment is worth the effort. Despite its warts, everyone uses bash and it's really easy to learn about it and translate that knowledge into many applications. And, it _does the job_ when I ask it to. Same for Python, IDE/editor usage, LaTeX, and all the other toys in your typical academia toolbox. I'm all for learning. But time is short, deadlines loom close, and the good enough is often the better's worst enemy. And when it turns out the better isn't actually better, welp.

</comment></comment></comment><comment><USER30> Been a bpython lover now I mainly use ipython. Used jupyter + ipython kernel once a while but for 99% my use cases, ipython is good enough(testing out code, checking out help info,etc).

For anything you want to have graph(e.g. matlab), notebook shall serve you well.

</comment><comment><USER31> I like notebooks. I like them for data work. I like them for prototyping things that many may consider software engineering. I still think this presentation has a lot of good points about where they could improve.

</comment><comment><USER32> I wrote an importers for notebooks that laods and runs the cell based on keywords I'm the markdown.

</comment><comment><USER33> from Joel Grus, "I don't like Notebooks"

[https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...](https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUhkKGvjtV- dkAIsUXP-AL4ffI/)

</comment><comment><USER34> I found this most exciting:

> While Jupyter notebooks have traditionally been a humans-only entrypoint > into a program, researchers and companies alike are increasingly using > notebooks for automation.

It's part of a slow journey back the power of the Lispms and the Smalltalk environments.

</comment><comment><USER35> "jupyter is the new bash" has to be the most underwhelming hyperbole of the decade

<comment><USER36> Are you kidding? The shell is a massively important part of modern computers, and making it more expressive would be a huge win. You might be familiar with TermKit[0] which also tried this.

[0]: [https://github.com/unconed/TermKit](https://github.com/unconed/TermKit)

<comment><USER35> ${SHELL} is\ horrible\ and\ dated

</comment></comment></comment><comment><USER37> Even in its present state, Jupyter is strictly better than plain REPL, especially if you need visualization. It's like Python in general: it was not designed to write 10-100KLOC programs. Python was designed for scripting. Jupyter was designed for interactive data exploration, and to create a record of results which you can view without re-executing the cells. Is it error prone? Yes. But then so is REPL.

<comment><USER2> > Python was designed for scripting.

It's far more than scripting, and for much bigger programs than 10k LOC... My company's Python codebase is 35m LOC/500k modules, with 25k commits per week and contributions from 2.5k developers per month.

To my mind, Python has the opposite problem. I've been a Python programmer since 2000. Every few years, I think I should devote more time to other languages (first Java, then R, then Haskell, then JS, ...). But the Python ecosystem just keeps getting stronger. My current focus is data science and machine learning. In these domains, there are no good reasons to drop Python.

<comment><USER37> JS was also designed for scripting

</comment></comment></comment><comment><USER38> You should never use notebooks - period.

My TL;DR: • Confusing for beginners • Encourage bad practices • Poor editor

<comment><USER12> Why not?

<comment><USER38> My TL;DR: • Confusing for beginners • Encourage bad practices • Poor editor

</comment></comment></comment><comment><USER39> As many already said, I think it is a great tool for prototyping and data exploration but when it comes to moving code to production, for me it makes very little sense to use it.

Netflix said that if the job breaks they can enter the notebook with the data and see what is wrong. For me it feels like they did development with 0 safe guards and if it breaks they check why. Instead of logging problems and dealing with edge cases in the code beforehand

<comment><USER40> I'm not sure exactly what Netflix means by "if the job breaks", but I've used jupyter notebook in production to do ML before.

If you consider the notebook as a way to augment your logs with plots, it might make a bit more sense.

Running a jupyter notebook is a nice way to generate a HTML report for a job. For a typical ML pipeline, you first plot some stats about the input data, then train some model, plot some training loss, a confusion matrix, some example of predictions, etc...

If some job gives a strange result (maybe that's what they mean by break), having the notebook rendered as an HTML page with all the plot is a very effective way to do a first round of diagnostics. You can also start the notebook with the same parameters and 'run' through your report, which is a nice way to do interactive debugging.

Also in this case, the notebook itself was quite small in terms of lines of code. All the functions were implemented in modules, so it's really like the notebook is your 20 lines 'main' function. So you need some discipline among your team.</comment></comment>


[STackoverflow]: What's your most controversial programming opinion? - <AUTHOR> http://stackoverflow.com/questions/406760/whats-your-most-controversial-programming-opinion

<comment><USER2> Nice list ... here's the top ten in order ...

\- Programmers who don't code in their spare time for fun will never become as good as those that do.

\- The only "best practice" you should be using all the time is "Use Your Brain".

\- "Googling it" is okay!

\- Most comments in code are in fact a pernicious form of code duplication.

\- XML is highly overrated.

\- Not all programmers are created equal.

\- I fail to understand why people think that Java is absolutely the best "first" programming language to be taught in universities.

\- If you only know one language, no matter how well you know it, you're not a great programmer.

\- Performance does matter.

\- Print statements are a valid way to debug code.</comment>


Google promises a unified social and mobile game platform - <AUTHOR> http://venturebeat.com/2012/03/06/google-promises-a-unified-social-and-mobile-game-platform/

<comment><USER2> There are some references in the Google+ app for Android to an unpublished app named "com.google.android.apps.oneup" that seems to be related to gaming.

I'm really interested to see where Google is heading with their gaming strategy.

</comment><comment><USER3> My first and only thought is "Google will fuck this up"

<comment><USER4> Google has an incredible amount of talent under its roof and has established a brilliant apparatus for drawing more talent in.

The problem is that Google has the wrong people making major decisions, and if it doesn't find a way to fix that, its next 50 years will be pure mediocrity. Talent doesn't matter if the direction is nukular buttfail.

One thing I like about Google is the ideology of being non-editorial. This served it very well in web search, where being non-editorial is necessary (because of the size of the problem) and stately, but it's an unmitigated disaster in game selection. Farmville was good for Zynga but bad for Facebook; it clogged the channels with spam, contributed to social network fatigue, and marred Facebook's reputation.

To get games right, you _have_ to be editorial. The _only_ way a general- purpose social network is going to beat Facebook in the next 5 years is if it generates an _excellent_ games brand. In-house game development wouldn't work at Google, so the solution is to form relationships with top indie developers, _not_ to promote (and give preferential treatment, such as early inclusion) whatever mainstream publishers like Zynga throw at them.

If any Google execs are reading this, that'll be $17,500 for the advice. A bargain.

<comment><USER5> I believe they are being editorial with games already, via the Staff Picks on the Android Market. They certainly are openly editorial with Google Music. So now that they're unifying all that, I guess we can only expect more of it.

<comment><USER4> Even if Google becomes "editorial" with games, there's no reason to believe that the people in editorial roles will be at all competent. I would bet against it. Google, without a doubt, has hundreds of people inside the company who would be awesome in such a role... but they'll never be picked.

Google's founders are brilliant, and the line engineers are great, but between those two sets there are layers upon layers of stewing, necrotic incompetence, a result of this being that some really horrible ideas (such as a performance review system that has everyone-- not just managers-- dropping everything for 2 weeks every fall to write reviews) see implementation while good ones die in obscurity.</comment></comment></comment></comment>


Algorithmic Approaches to Playing Minesweeper (2015) - <AUTHOR> https://dash.harvard.edu/handle/1/14398552 <comment><USER2> I did some work on a minesweeper solver [1], I'm surprised my work is not mentioned. The best version of their algorithm wins on expert 32.9% of the time, whereas my AI achieves about 50% win rate. My AI uses the connected components decomposition that they did, but also some endgame tactics that significantly improved win rate (that they didn't consider).

[1]: [https://luckytoilet.wordpress.com/2012/12/23/2125/](https://luckytoilet.wordpress.com/2012/12/23/2125/)

<comment><USER3> > Each of these 11 configurations should be equally likely to be the actual > position.

This is not true. For calculating the odds of a configuration you need to take into account the non-border squares and the number of the remaining mines too.

Example (3 remaining mines):

        1111
        *22*
        xxxx
        xxxx

The * represents the marked mines, 'x' represents the unmarked squares.

There are three border configurations:

        1. mine, empty, empty, mine
        2. empty, mine, empty, empty
        3. empty, empty, mine, empty

But actually the first one is actually is 4 global configurations, as the remaining one mine can be anywhere on the remaining squares. The second and third are 6 and 6 global configurations _respectively_ as the remaining two mines can be anywhere on the remaining 4 squares (4 choose 2).

The probabilities of a border squares to be empty are 3/4, 5/8, 5/8 and 3/4 respectively as opposed to 2/3, 2/3, 2/3 and 2/3 that your rule gives. The probability for a non-border square to be empty is 9/16, slightly more than 1/2.

One can argue that when the number of the remaining unmarked squares is large compared to the border squares this gives little difference. But it's not true, this gives significant differences even when this is the case.

[1] [http://nothings.org/games/minesweeper/](http://nothings.org/games/minesweeper/) "Counting unencountered mines"

<comment><USER2> That's a fair objection. You can view my rule as a heuristic that's easy to compute and gives good guesses in most situations, rather than calculating the mathematically correct probability.

My AI does actually account for this, but only in the endgame when there are a small number of squares left -- then it considers the global configuration instead of just local. This works out because the endgame is where this problem is the most significant, it's computationally expensive unless there's a small number of squares remaining.

<comment><USER3> Then I suggest an improved heuristics for mid-game.

You can actually calculate the number of global configurations for each border configuration, they are just (n choose k) type calculations. It's somewhat more sophisticated when there are multiple disjoint border regions.

Alternatively for a mid-game heuristic you can approximate the initial uniform distribution of the fixed amount of mines on the playing field with another distribution: Each square having an independent probability of having a mine or not. This could potentially simplify the calculation but it wouldn't be accurate when there are not so many mines left.

<comment><USER4> _"It 's somewhat more sophisticated when there are multiple disjoint border regions."_

I don't see why that would be the case. Can you explain?

</comment></comment></comment></comment></comment><comment><USER5> Well, it has been a while since the last time, but a ritual is a ritual and it has to be followed.

Whenever someone says "minesweeper", I have to post my implementation.

Here goes: [http://www.ronilan.com/bugsweeper/](http://www.ronilan.com/bugsweeper/)

Few notes:

1\. This is dated January 2014. It is a job interview homework.

2\. The HN community has detected a bug and indicated that it should be fixed. That was done March 2017.

3\. There are currently no known bugs.

4\. The goal of the game is to find the unknowns.

5\. No puns intended.

6\. Code: [http://www.ronilan.com/bugsweeper/js/bugsweeper.js](http://www.ronilan.com/bugsweeper/js/bugsweeper.js)

<comment><USER6> Like it - did you get the job?

<comment><USER5> Obviously not.

But it is totally understood. You can't expect a leading tech startup like Thumbtack to lower the bar for a 40+ yo self taught programmer like myself. They only hire the best!

</comment></comment></comment><comment><USER7> I can't not plug Peter Divianszky's implementation at [https://hackage.haskell.org/package/minesweeper](https://hackage.haskell.org/package/minesweeper) which has the awesome property that the board is generated as you play, in such a way that you never have to guess (in the sense that if a field is not forced by currently shown clues to have a mine, then it won't have one when you click on it).

</comment><comment><USER8> For anyone who wants to play Minesweeper without the hassle of trying to track down a known-good *.exe, I recommend Minesweeper X [0]. The board size isn't limited plus a ton of other features. It also works perfectly under WINE with Winetricks.

[0] [http://www.curtisbright.com/msx/](http://www.curtisbright.com/msx/)

<comment><USER9> I'll join you and show off mine as well! Runs in terminals, and also has a rougelike adventure mode where you navigate the board with a character.

[https://github.com/accatyyc/terminal- mines](https://github.com/accatyyc/terminal-mines)

<comment><USER8> Minesweeper X isn't mine (just a happy user), but yours is cool too, thanks!

</comment></comment></comment><comment><USER10> This paper is wrong in section 4.1:

>The first click of a Minesweeper game deserves special attention since it is rather unique. >Namely, the opener is always a guess because the player starts with a covered board. >In addition, the challenges associated with the opening move are unavoidable and are shared among all solvers. Thus, finding an optimal policy for dealing with the initial click will benefit every approach

The first click of a Minesweeper game is _always_ safe[1]

[1][http://www.minesweeper.info/wiki/Strategy#First_Click](http://www.minesweeper.info/wiki/Strategy#First_Click)

<comment><USER11> I think what the author meant is: It's a guess where the best position is for the first click.

You can't fail with the first move but you can have different outcomes and it's advisable to choose a position where the expected information gain is optimal.

</comment></comment><comment><USER12> Ah, let me throw my take on Minesweeper into the ring as well: [http://magnushoff.com/minesweeper/](http://magnushoff.com/minesweeper/)

My final variant is like the original Minesweeper except that when you are provably stuck, the game will help you to move on.

</comment><comment><USER13> For a nice minesweeper variant, try Mamono Sweeper.

It has the same mechanic as minesweeper, but instead of bombs you are sweeping for monsters, which have a level (1 to 9).

A clue in a cell is the sum of the monster's levels surrounding it.

You can kill monsters of level 1, and eventually you level up and are able to kill bigger monsters.

</comment><comment><USER14> I've long wanted to mak an "always possible" variant of minesweeper. This is equivalent to only presenting the player with a board that a "perfect" algorithm can solve.

</comment><comment><USER15> it's funny that there's a whole section (4.1) on picking a starting move that increases the probability that that first cell is a 0-cell when in fact the game guarantees that your first click will do that.

<comment><USER16> It guarantees that the first cell is not a mine. You could still just get a number there.</comment></comment>


I Know What You Downloaded on BitTorrent… - <AUTHOR> http://torrentfreak.com/i-know-what-you-downloaded-on-bittorrent-111210/?utm_source=dlvr.it&utm_medium=twitter <comment><USER2> Interesting removal terms. They pretty much force you to admit that you downloaded, whatever they say you downloaded in order to have your data removed. Immediately after having confirmed your real identity via Facebook, of course. They could sell this information to MPAA and RIAA for gold.

The Details:

By submitting a request to have your download activity removed from our database, you are acknowledging that the activity was, in fact, carried out by yourself. This means that you are only submitting a request to have the details of your own personal activity deleted. Any unrecognized activity, such as files you did not download or do not remember downloading, are not — I repeat, are not to be included in your removal request. Why is this imperative? Well, we actually don't have to explain ourselves...sorry.

The important part is that you understand these terms and conditions before hitting that beautiful button that will erase your criminal back ground, at least for now. Wait, you did remember to read these terms before making the decision to submit a removal request, right? Of course you did, everyone reads the fine print.

Other Important Things to Consider: We make no guarantees that your information will not appear on any other databases. We may have erased your bad behavior but, keep in mind that your data on this site is aggregated public domain. So, if by chance, another sadistic group of people decides to open a similar web site, we have no control over what they do with your information. Furthermore, if you continue to involve yourself in activity like this, your future download history will, without a doubt, appear in our database again and we may not be as nice about it next time.

If any part of these terms is still unclear, please visit your local elementary school and ask to repeat grades 3 through 5.

<comment><USER3> > They could sell this information to MPAA and RIAA for gold.

Maybe in the US, but in many other countries they would be breaking several privacy laws. In fact, even the collecting the initial minimal information (IP's + downloads) already breaks privacy laws in most of Europe. Commercially, this information has no value.

<comment><USER4> The lack of seriousness on the site greatly compromises the value of the data. As a captcha they they ask you enter the _entire_ text of the USA Constitution, and give a mailing address in Antarctica. There is no evidence that large portions of the data isn't made up, no sane company would want to touch this. The agreement above sound more like an attempt to prevent people from contacting them in the first place (probably as they have to remove data manually), than a prelude to extortion.

<comment><USER5> > The lack of seriousness on the site greatly compromises the value of the > data.

I think that's the point. They want to make a point about privacy, but they don't want anyone to take the specific data seriously.

</comment></comment></comment></comment><comment><USER6> It's a joke people. They claim that the scraper is real though:

"Don't take it seriously

The privacy policy, the contact us page — it's all a joke. We came up with the idea of building a crawler like this and keeping the maintenance price under $300 a month. There was only one way to prove our theory worked — to implement it in practice. So we did. Now, we find ourselves with a big crawler. We knew what it did but we didn't know how to use it. So we decided to make a joke out of it. That's the beauty of jokes — you can make them out of anything.

However, if you have a better idea — don't hesitate to contact us."

</comment><comment><USER7> I don't use BitTorrent very often (because I have incoming connections for that torrent in my router's state table for years after I do so), but there were a few things I wanted to download today so I just spun up an EC2 instance, installed Transmission, and did it from there. A 38GiB Blu-Ray image took about two hours. The next time I want to do this, I'll have a completely new IP address.

Is this a terrible idea?

<comment><USER8> any ideas why, transmission on a mac is crawling with the same torrent that's lightning fast on vuze via windows? i've tried a few torrents using trasmission, and they would either crawl or not DL at all.. Vuze seems to be more and more intrusive (in fact some malware detectors continually report malicious activity coming from vuze), but somehow they figured out how to make torrents fly.. it's not uncommon for me to get 1mbps downloads..

<comment><USER9> Check you haven't got the Speed Limit on; then make sure the slow torrent is set to ignore any speed limits; then check how many peers it can share with (default is 60); then check ports are okay.

</comment></comment></comment><comment><USER10> "Hi. We have no records on you."

Just got that for the past 2 or 3 connections I've used in the past few weeks. It might not be very good, but at least it seems inclined to falsely testify for one's innocence.

</comment><comment><USER11> I am surprised about the high quality taste other people have who share this VPN. Csikszentmihalyi, Ebook Libraries, Stephen Merchant - proud of my fellow anonymists.

<comment><USER12> _proud of my fellow anonymists_

Except, of course, that they didn't pay the authors for any of that high- quality content you like so much.

<comment><USER13> You can't actually know that.

<comment><USER12> I would, however, bet a beer.

</comment></comment></comment></comment><comment><USER14> And so the next step in the cat-and-mouse game is predictable: A whole bunch of forged requests.

Bonus points if the forged IPs are from RIAA sites, DHS, White House, FBI etc.

<comment> Tuna-Fish Done already. Most of the major trackers add random ips to the lists they publish.

</comment></comment><comment><USER15> This service is BS. I just went there and it gave me a random torrent, "Jennifer Lopez Papi" (or something like that). I wouldn't get anything of JLo's, even if they gave it to me free.

<comment><USER16> Somebody did from the IP address you seem to be using.

Either you are behind a masquerading NAT, or perhaps your ISP rotates IP addresses and somebody else was using your address the other day, or just maybe your WiFi router isn't as secure as you hoped, or...

The service neatly highlights the caveats of matching people to IP addresses.

<comment><USER17> I have a static IP, and I'm pretty sure that no-one at my home has downloaded the tv episode this service mentions. It's of course not entirely impossible that someone has hacked my wifi router, but I think that is less probable than it is for some random web service to show some made-up data.

<comment><USER18> Some trackers inject fake peers.

[https://torrentfreak.com/the-pirate-bay-tricks-anti- pirates-...](https://torrentfreak.com/the-pirate-bay-tricks-anti-pirates-with- fake-peers-081020/)

</comment></comment></comment></comment><comment><USER19> They claim they can track 20% of the public torrents people download, and I suspect it's a very specific 20% corresponding to certain trackers and certain torrent clients. I wonder what they are.

<comment><USER20> There's no way for any torrent client to hide itself since all this site needs to do is to have a program join the swarm of all torrents it can find and log all the ip's of the peers in that swarm, it doesn't matter what torrent client you use.

However if it's a closed tracker and they do not have access they can't join the swarm and log the ip's. I'd assume that at this point they are only logging ip's on open trackers but there's no technical reason why they couldn't join a private tracker (unless they don't accept more users) and log ip's there aswell.

<comment><USER21> Private trackers are a lot more difficult. I bet it would raise some eyebrows if a user on a private tracker were to suddenly announce all torrents. On public trackers they can just use multiple IPs, on DHT they are trackers themselves and don't even need to ask others, but on private trackers their crawler will have a hard time getting many accounts to hide its activity.

</comment></comment></comment><comment><USER22> I download around 2-3 torrents a week normally using public trackers and this site says "Hi. We have no records on you.". Another note, I use peerblock while downloading not really because I believe in the protection but because its interesting to see the names of organizations which care what i'm doing. Perhaps its actually doing something.

</comment><comment><USER23> Since I don't use torrents at all (unless if software uses it for updates but I am not aware of anything I currently use that does), it might not be too interesting.

However, the next time I share WiFi with someone, this may be quite useful.

</comment><comment><USER24> The last lecture of my P2P Systems and Overlay Networks class was about Anonymity. I should probably go over the slides once again.

EDIT: This was a paper about file sharing anonymity, [http://www.cs.cornell.edu/people/egs/papers/herbivore- esigop...](http://www.cs.cornell.edu/people/egs/papers/herbivore-esigops.pdf)

</comment><comment><USER25> Either my roommate has very different music tastes than I thought he did, or They've overestimated my ISPs lease time.

<comment><USER26> No kidding. I live in a one-bedroom in a college student apartment complex and none of these listed downloads are mine. But from visiting that website, I just learned that the Walking Dead is on it's second season.

</comment></comment><comment><USER27> Weird, I downloaded 4-5 episodes of an unlicensed series and only one of them got tracked. I'm fairly sure all torrent were from the same source and used the same trackers, but I could be wrong.

Not to mention I use torrents extensively but they only had one torrent on my "file".

</comment><comment><USER28> I can see this causing a lot of household problems.

</comment><comment><USER29> Can't we get updated definitions for peerblock which will blacklist their crawling ip?

</comment><comment><USER30> Looks fake to me:

<http://www.youhavedownloaded.com/?q=0.0.0.0>

<http://www.youhavedownloaded.com/?q=255.255.255.255>

<http://www.youhavedownloaded.com/?q=192.168.1.1>

<http://www.youhavedownloaded.com/?q=192.168.2.1>

<http://www.youhavedownloaded.com/?q=192.168.5.1>

[...]

Has this correctly identified anyone's downloading?

</comment><comment><USER31> i hope this doesn't prompt more people to start using bittorrent through tor.

<comment><USER32> why?

<comment><USER31> because it makes it so difficult to provide exit nodes in the US for people that actually need them. DMCA complaints will quickly take down an exit node because of tor users sending bittorrent traffic out of it.

<comment><USER7> Just like DMCA complaints have shut down other internet service providers, like Comcast? Ultimately, in the US, ISPs are not responsible for their users' activity. "Common carrier" and all that. When you open an exit node, you become an ISP.

<comment><USER31> i'm aware of DMCA's safe-harbor provision and i've fought with colo providers over it. when you're not an ISP the size of comcast (or any decent-sized "real" ISP), you're always going to be subject to another company's AUP.

i've had an exit node shut down by a colo provider simply because they didn't want to deal with the overhead of processing the DMCA requests. they knew what tor was and they knew i wasn't the one downloading content, but they just didn't want to deal with it, so it was either turn the exit node into a relay or have my server shut off.

[https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISP...](https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs#Badexperience)

</comment></comment></comment></comment></comment><comment><USER33> The only thing it has for me is something called "Worms.Crazy.Golf" - something I have never downloaded. I also have a static IP.

This site seems completely bogus.

<comment><USER34> apparently some trackers have fake peers that might use your IP address. regardless of if that is or isn't the case, it lends you some deniability.

</comment></comment><comment><USER35> In India, most ISP's don't provide you with a static IP address. So the addresses are shared/recycled. When I opened the website I got a list of torrents that I have not downloaded.

Basically every couple of days this list for me will change and won't be accurate. I'm sure many other ISP's do the same. So this list of downloads isn't entirely accurate.

</comment><comment><USER36> I knew this was a ruse when I read the bio's at the bottom of the page. I had a good laugh at this one:

"Suren Ter (me) I'm a producer of the site. Like a movie producer, I made the site."

Then several sentences later:

"Me? I don't do code, I don't do research, I don't do design — I do sites. Drop me a message if you'd like."

</comment><comment><USER37> They are probably not taking dynamic IPs in mind. But the idea is interesting.

</comment><comment><USER38> I am guilty of downloading one episode of Pawn Stars after my DVR messed up the recording. My DVR re-recorded it the next day. If this is illegal, then I'll just get a VPN and cancel my satellite service.

</comment><comment><USER39> And that's why I'm behind a VPN. Found one torrent which I assume someone else who had this VPN's temporary IP-adress downloaded earlier, though.

</comment><comment><USER40> For those on dynamic IP the data is so mixed up. Atleast a date or timestamp of the download would make it more clear.

</comment><comment><USER41> Anyone have any idea how they collect the data (what IP is dowloading what torrent)?

<comment><USER7> An integral part of the torrenting process is getting a list of IP addresses from the tracker so you can connect to each IP and start downloading the data. You can log those IP addresses to a database instead of downloading that data.

</comment></comment><comment><USER42> Also, doesn't work if your service provider provides dynamic IPs.

</comment><comment><USER43> I have static IP and it knew exactly what i have downloaded; i guess it is using some of the most popular trackers from 'the pirate bay'.

</comment><comment><USER44> usenet & ssl + nzbmatrix = no more problems.

<comment><USER7> Your Usenet provider certainly knows what you're downloading. The reason you don't care is because someone sued some Usenet provider in the early 90s, and Usenet providers were deemed "common carriers", not liable for the content they carry. Since _downloading_ copyrighted content is apparently not illegal, it all works out.

<comment><USER45> Also nice because SSL doesn't tend to get preempted by traffic shaping so you usually get close to the max download rate the ISP says you're paying for.

</comment></comment></comment><comment><USER46> for me it says, "Well, you are in the clear. But look what others do"...AnalTeenAngels.com - Abbie (584.32 MB)

should i check it out? must be good!

[turns out to be a gay flick, FML]</comment>


Preserving the Virtual Memory Abstraction - <AUTHOR> https://www.sigarch.org/preserving-the-virtual-memory-abstraction/ <comment><USER2> I've been hearing this from multiple places. Seems obvious after some talks and HN articles.

</comment><comment><USER3> interesting. i knee this was an issue for accelerators but haven't really thought about regular cores being bottlenecked by the TLB.</comment>


The iPad Pro Is Not a Computer - <AUTHOR> https://www.eva.nmccann.net/blog/the-ipad-pro-is-not-a-computer <comment><USER2> I agree in part with the author, but dissent in that the iPad Pro can be a computer for certain functions. It will never be the desktop or laptop we use now, but for industrial and marketing jobs it's proven to be a considerable computer.

<comment><AUTHOR> I absolutely agree it is considerable computer. At the end I mentioned the iPad Pro is the the best computing device I've ever used.

What I didn't articulate clearly enough is there are tasks where an iPad Pro is far better than a traditional desktop or laptop.

Thanks for the thoughtful comment.</comment></comment>


Re: Making Debian Responsible For Its Actions - <AUTHOR> http://sheddingbikes.com/posts/1285693551.html

<comment><USER2> This is a good, levelheaded response. But it makes it seem as if the process of becoming a sponsored maintainer was straightforward.

It's not.

A number of months ago, I tried getting trimage (a gui image optimizer, <http://trimage.org>) into debian. Since I already had a working Ubuntu PPA, I figured it wouldn't be too hard. Packaging instructions for debian are available at various official websites, but contradictionary and they all seem to presume you have packaged for Debian before. Jargon like RFP (request for packaging) and ITP (intent to package) is thrown around frequently without explanation.

Luckily, there is debian-mentors, meant to help upstream developers with becoming a sponsored maintainer. Most of the debian-mentors site links to outdated packaging instructions, using which will mean you will be ridiculed on the way-too-hostile mailinglist. Trying to do 'the right thing' is almost impossible, as most of the feedback will depend on the personal opinion of the person responding. I suspect this is because they, too, don't know which packaging instructions to use or point to.

Trying to get my application into debian has been nothing but a massive, timesinking frustration, and I wish that wasn't the case.

<comment><USER3> I've also brought up a counter question that people seem to not be able to answer: How do you get a bad Debian Developer fired?

It seems once they get in they're in for life. Yet, some of the worst DDs who crap on programmers seem to somehow get in charge of projects they actually hate. If they hate it, and they're breaking the software because of this, then why can't I, the original author, get them fired.

At an even simpler level, why can't you report bad DDs and have them reprimanded or taken off the project? That alone would bring out some better responses.

<comment><USER4> Why do you need to get them fired?

Just remove the community from an open source community and things will start happening FAST.

Since, most people reading this thread are truly influential can't you guys talk with canonical and fork Debian under the Ubuntu project? You could model a community around how HN runs and vote things organically into existence while keeping a high level of quality. Instead of a rant match between a bunch of egos.

If you think about it the HN model is _perfect_ for such stuff. If and only if you attract the right audience, but your work already reaches them!

Have you ever considered doing that? Or is there something wrong in my suggestion?

<comment><USER5> Have you seen how many packages there are in Debian/Ubuntu and how many Debian Developers there are? Forking Ubuntu entirely away from Debian would be a _massive_ task and would need a huge injection of skilled people. (Also internet voting for things leads to a system that encourages gaming the votes, see digg ;)

<comment><USER4> What I meant was inviting the best of those Debian devs to start afresh with the same code and create something even better under a new system.

You could always make voting tiered and restricted like stack overflow to avoid this... Moreover, if you attract just the right kind of people then the community will start moderating itself and keep the fluff at bay.

Surely, we need to try something like this out.

<comment><USER5> what are you trying to achieve with the voting? and what is the incentive for volunteers to put hours into packaging work if they're being motivated by votes rather than their personal interests?

<comment><USER4> A real democratic community where muscle flexing doesn't work and people can't exclude PHP from a package because it is "impure". If their big idea gets accepted they can collaborate with others to make it.

Wouldn't that be a better way to do things?

<comment><USER5> It would seem to me that the outcome would depend a great deal on who was voting.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> This significantly increases my respect for Zed Shaw.

Yes, he wrote an offensive post crying for blood, but after he had a chance to cool down, he used his own fame and audience to exhibit a well-reasoned response from someone who disagreed. That's a very adult thing to do. In some ways, it's actually better than an apology.

<comment><USER7> He reacted in the right way, but on the whole my impression is negative. I'm a bit disappointed, because most of Zed's previous rants were humorous, amusing and well researched. But in this case he didn't put any effort neither in research nor writing. He'd stumbled upon a dependency bug and, instead of reporting it, he for some reason concluded that it means that Debian is evil and everybody should boycott it. Weird.

PS. Excuse me, there was one amusing paragraph in the rant. The one in which he equaled Debian, the most independent and free software obsessed distribution out there, to Microsoft.

<comment><USER8> He didn't just stumble upon an overlooked dependency bug — the Debian maintainers _intentionally_ broke their Ruby package years ago because of the legal problems they imagine to exist with using OpenSSL.

I don't think it's so out of bounds to compare Debian, the most copyright- license-obsessed distribution out there, to Microsoft.

<comment><USER9> Maybe it's a name problem. It was said _ruby-full_ provides the full Ruby language, while _ruby_ provides the export-restricted version Debian has to distribute in order not to have police officers knocking down doors. In Python's case (sorry, I am more familiar with Python) there are _python_ and _python-minimal_ packages. They may or may not mean the same as _ruby_ and _ruby-full_ , but their names communicate a similar intention.

<comment><USER8> While "police officers knocking down doors" over crypto export restrictions do not exist, that's not what Debian's wankfest over OpenSSL is about.

They think that the advertising clause in the OpenSSL license makes it impossible to use with any GPL programs — despite the fact that the OpenSSL developers think it's fine as a platform library and tons of upstream developers of GPL software choose to link against it. In Debian's world, they'll only allow you to use it if you get every contributor to agree to a special exemption clause appended to the GPL.

Through their absolute fealty to the most draconian interpretation of the letter of the law, Debian does more to further the cause of software patents and non-free licensing than any other organization. They choose to demonstrate how idiotic such ideas about IP are by implementing them to their fullest — _by being idiots_.

<comment><USER10> > Debian does more to further the cause of software patents and non-free > licensing than _any other organization_.

Wow. Really?

> They choose to demonstrate how idiotic such ideas about IP are by > implementing them to their fullest — by being idiots.

From the guidelines:

> Be civil. Don't say things you wouldn't say in a face to face conversation.

> When disagreeing, please reply to the argument instead of calling names. > E.g. "That is an idiotic thing to say; 1 + 1 is 2, not 3" can be shortened > to "1 + 1 is 2, not 3."

I for one do not care at all for your rude name calling.

For what it's worth, I'm not particularly fond of debian-legal either, but it's entirely possible to express that disagreement without calling everyone who does not see the world as you do "idiots".

<comment><USER8> So who else is out there doing the work of enforcing American software patents on the users of the world? Even the BSA doesn't do that! Apple isn't out there enforcing their patent on the truetype hinting bytecode interpreter, but Debian refuses to ship a full version of freetype. The DVD-CCA gave up years ago, but Debian still won't ship a DeCSS implementation. Fraunhofer/Thomson/MPEG-LA only really care about companies shipping hardware, but Debian cares more about FUD and promoting OGG than actual use.

None of this shit matters _at all_ when shipping source code, and binaries only matter if both parties are in the US. But the Debian project is out there with jackboots on, doing what they can to take useful freely-licensed software out of distribution. What other organization is doing anything like that?

I'm sorry that you find yourself so wrapped up in such an organization, I did not mean to insult you directly, but rather the ideology y'all are applying to make my world a worse place to live. And I would be far more acerbic in a face-to-face conversation.

You didn't understand my line of argument centering around idiocy (should have used fewer anaphora) — my point was that the Debian people know that they're being idiots, and that it's the purpose of their actions in this squabble. We all know that software patents are stupid, but Debian's approach is to show the world just how stupid software patents are by fully respecting them, implementing the letter of the law until it is repealed. I think that's a useless and infuriating course of action.

<comment><USER10> With regards to the legal stuff, I simply don't know. One of the things that bugged me about debian-legal were all the people convinced that they did know it all. I'm not a lawyer and neither are most people there. Neither are you or the OpenSSL guys either, though, as far as I know. My firm conviction is that most "open source legal debates" are essentially the blind arguing with the blind about various painters. Although, of course, many real lawyers are also so vague and non-commital in what they say that they're not much use either, so I tend to just want to stay as far away as possible from that kind of discussion.

That said, one thing I don't agree with you on is that "well, they probably won't notice / bother us, so it's ok to skirt the letter of the law". I think that's not really the right attitude either. Debian makes some strong commitments to shipping free software that people can use and redistribute as they see fit, so I think they have to be cautious at times.

I am not a part of Debian any more. I moved on to being simply a user (and bug reporter) of Ubuntu, which I think gets some things right that Debian didn't. I just don't like the "you fucking idiots" style of debate.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> I am Debian Developer with upload rights.

If HN people want to get involved with Debian, packaging and so on email me and I'll do my best to help you.

Probably the best way to fix whatever you believe is wrong is by getting involved.

</comment><comment><USER12> Coincidentally, I just happened to see a link on Reddit leading to a post by the same Ersek: [http://groups.google.com/group/comp.lang.c/msg/22e3473f80eec...](http://groups.google.com/group/comp.lang.c/msg/22e3473f80eec512)

Like his message to Zed, it's clear, concise, and seemingly accurate. It also includes a wonderful and well commented example program in expert idiomatic C showing signal handling. I hadn't even known comp.lang.c was a going concern!

It's exactly the sort of quality code I'd expect from someone who writes such good prose. While I'm sure there are some examples of good prose writers who can't code, are there any definitive examples of good C programmers who can't write? Or is this just a "true Scotsman" problem?

</comment><comment><USER13> I still think my point at <http://news.ycombinator.com/item?id=1735344> is important. If Debian didn't throw away unit tests, a lot of these integration errors could get caught automatically.

<comment><USER3> Actually, I _really_ agree with you here. In fact, it got me thinking about sort of a protection clause in licenses along these lines. Where you can say, "This software is BSD licensed, providing you follow the guidelines in the PACKAGING file."

Then, any OS distribution would be required to follow your wishes or not include your software. If you want unit tests, they include them. If you want all of it, or parts of it, they do it. If they don't want to or can't then they don't adopt your project.

I'm currently just figuring out how to word the license that way so it'd still work.

<comment><USER14> Sounds like you want a license where you get to dictate exactly how any downstream user reuses your software, down to the details of the packaging, and prohibit modifications/forks/removals/etc.? But that's not really an open- source license at all, then.

The whole point of open-source software is that I can take a thing, decide that I only want 50% of it, rip out the other 50%, add on a bunch of new stuff, totally change some of the remaining stuff, use it for a new purpose, package it in a new way, undo a decision the maintainer made that I disagree with, etc. If you don't like people having that freedom, open-source just doesn't seem like the way to go. It sounds like you want a "freeware for verbatim distribution" proprietary-but-no-cost license.

<comment><USER10> > dictate exactly how any downstream user reuses your software

I agree with your post. It's also worth pointing out that if someone abandons open source licensing, one thing I'd put in my licenses - _way_ before we talk about fiddling with the path layout and such - is that the software can't be used by people I find reprehensible: neonazis, repressive governments, and so on and so forth.

Of course, I prefer freedom to that kind of licensing, so I just live with the knowledge that horrible people may use things I wrote.

<comment><USER15> The people you harm by restricting neo-nazis from using your software aren't neo-nazis, but everyone else who has to pay for lawyer time to review your custom license before they can use it. Or what's more likely: aren't going to use it at all once they see what sort of license you have.

<comment><USER10> Not to mention that actual neonazis will probably not give a great deal of thought to violating said license, meaning you'd have to have a way to monitor usage, and then go to court to stop them. All in all, a hassle that's not worth it.

</comment></comment></comment></comment></comment></comment><comment><USER16> This is a reply to <http://sheddingbikes.com/posts/1285659877.html> which was discussed here <http://news.ycombinator.com/item?id=1734936>

</comment><comment><USER5> I guess this discussion is dead, with essentially no useful outcomes :(</comment>


Shanghai Hacker News Meetup - <AUTHOR> http://devo.ps/blog/2013/02/20/shanghai-hacker-news-meetup.html

<comment><USER2> Hey Shanghai, what's the feeling on the ground about Mandiant's APT1 report from yesterday?

<http://www.mandiant.com/apt1>

<comment><AUTHOR> My personal opinion on it is the following:

1\. What's the news here? Isn't every nation on earth doing it, or trying to do it?

2\. Western nations would be ill-advised to not buckle up and take online warfare seriously. The police in particular should be way better prepared.

Overall, I think it's fun to expose it, but beyond reporting I fail to see what the value is.

<comment><USER2> I think what people seem to be saying is that China is one of the only countries so blatantly using it for economic espionage against private corporations. This is fundamentally different than what other countries are rumored to be doing.

</comment></comment></comment><comment><USER3> Awesome! Great to see such kind of event happening in Shanghai :D

<comment><USER4> There is a good little community here in Shanghai, including these upcoming meetups on Meta Programming and AngularJS:

[http://www.meetup.com/Shanghai-Agile-Scrum-User- Group/events...](http://www.meetup.com/Shanghai-Agile-Scrum-User- Group/events/101905672/) [http://www.meetup.com/Shanghai-Agile-Scrum-User- Group/events...](http://www.meetup.com/Shanghai-Agile-Scrum-User- Group/events/103126982/)

</comment></comment><comment><USER5> Anyone interested in a Beijing meetup?

<comment><AUTHOR> We'd be interested in throwing one next month when we're in town.

</comment></comment><comment><USER6> Anybody in Hong Kong?

<comment><USER7> Yes

</comment></comment><comment><USER8> Wonder if Unit 61398 will have anybody there...

</comment><comment><USER9> Hmmm... if I get the time I may pop by.

</comment><comment><USER10> Kunming? :)

<comment><USER7> Sounds optimistic! Is there much of a startup scene in Kunming?

<comment><USER10> Not bad for the size. Less tech-oriented, lots of F&B/tourism, some arts/creative media, NGOs.

<comment><USER7> Interesting, maybe you should put an email in your about so ppl can contact you next time they're in Kunming :)

<comment><USER10> I live in Bangkok.</comment></comment></comment></comment></comment>


We're creating the best Selenium recorder ever. Pls share ur thoughts and requests - <AUTHOR> Hi there!
Those of you using Selenium to test your web apps, perhaps you could help us with some input.
We are building an open-source Selenium-recorder for web testers and developers who write their own tests. The recorder will automate a lot of manual work and would be able to generate Selenium-powered tests for various programming languages. We want to use this recorder ourselfs, we know what we need it for. However the world of web development is large and diverse and we want to reach as many of you as we can to create something that not only us, but also the community would benefit from.
So what features would you like to see in the alpha / first public release? Please, tell us as much as you can:
- which Selenium bindings are you interested in the most (JavaScript, C#, Java, Python, Clojure)? - would you prefer to have integration with some popular test frameworks out of the box? Which ones? - are you using Page Object Design Pattern in your tests? Do you think we should add a support for it? In the alpha or could that wait? - are you using Selenium recorders already? Could you share which and what are their strong/weak points? We don't know of a good one, but may be we missed it.
Thanks!! <comment><USER2> Hi, great and ambitious project! If you want to consider I, as Java developer, use to test my apps with selenide (selenide.org) a wrapper for selenium which makes UT a lot easier to code.

<comment><AUTHOR> Thanks a lot!! Fully agree here :) We were really excited when we came to this idea and realized we have all we need to actually do it. We will certainly have a look at selenide!

</comment></comment><comment><USER3> I'm interested in C# with built-in Page Pattern support. Excited to see the first release. What is the target platform?

<comment><AUTHOR> Thanks a lot! C# is definitely on our list. We want to target the most popular binding first and then add support for all major selenium bindings out there. Right now we're targeting Chrome as we're using it most ourselves :) and as it also is the most popular browser at the moment. Please follow us on twitter, so you're up to date on the progress. But I will certainly post here when we have a release.

Thanks again!</comment></comment>


Story of XMLHTTP (2007) - <AUTHOR> http://www.alexhopmann.com/story-of-xmlhttp/ <comment><USER2> I thought this was well-known? Microsoft invented it for OWA but didn't realize what they had. A few years later other people realized how transformative it was and evangelized it. Other browsers added compatible implementations. The rest is history.

It is amusing to see the take in Avalon since we now know how that turned out. His complaints about HTML/JS/DOM are spot-on but it turns out the solution was to improve HTML, JS, and the DOM.

<comment><USER3> We were doing XMLHTTP like communication at a startup back in the early 2000's, by making use of a Java Applet and the communication JavaScritp/Applets.

I imagine we weren't the only ones doing that before XMLHTTP was born.

Regarding Avalon, I have been doing quite some consulting projects (WPF and UWP) in the last three years, and at least on the markets I move on, there are plenty of greenfield projects. :)

<comment><USER4> Yeah, using hidden iframes worked as well

</comment></comment></comment><comment><USER5> XmlHttpRequest is an awesome innovation from the point of view of the software development community as a whole.

But from a Microsoft shareholder's point of view it was a disaster. XmlHttpRequest and ContentEditableDiv (known back then as DesignMode) ushered in Web 2.0 and Web 2.0 ended Microsoft's dominance of the software industry by making Windows irrelevant.

<comment><USER6> > by making Windows irrelevant

Yeah, like it's not the most popular OS out there :) What Web2.0 did is that it removed the need of certain type of desktop software.

BTW if Microsoft didn't invent XmlHttpRequest, somebody else had done it.

<comment><USER7> > What Web2.0 did is that it removed the need of certain type of desktop > software.

The commonly held belief at the time was that Microsoft's dominance rested in large part on that desktop software. Think how hard it would be _today_ if you wanted to start a new mobile OS (let's call it "windows phone") years after entire software ecosystems have been established for iPhone and Android.

It's safe to say that Microsoft is not the shining star they once were. If we were to pick a single moment in history that lead to their undoing, I think this would be it.

> BTW if Microsoft didn't invent XmlHttpRequest, somebody else had done it.

"Somebody else" didn't have the market share IE did. Something that was incompatible with IE at the time was a non-starter. What's remarkable about this story is that when Microsoft saw the browser coming in the Netscape years, their entire strategy was to head that shit off by ensuring that their own browser become market dominant so that they could keep the web in a place that wouldn't hurt MS.

After all that effort, the anti-trust suit, etc, etc... and they'd gotten themselves in the place where they controlled the future of the web, it was some random developer at Microsoft itself that invented the very thing which was their undoing.

<comment><USER8> > If we were to pick a single moment in history that lead to their undoing

I always find these sorts of comments interesting.

Microsoft is a very different business than say other failures who you could point to a single event (Kodak, Word Perfect, Lotus Software, Myspace)

Remember, Microsoft have over $100 BILLION in cash[0].

You could perhaps cherry pick certain parts of the business. But I remember the stories of how they made more money from Android than Windows Phone[1].

Microsoft has a long, LONG way to fall before I would say they have failed.

[0] [https://www.bloomberg.com/news/articles/2016-06-13/why- micro...](https://www.bloomberg.com/news/articles/2016-06-13/why-microsoft- with-100-billion-is-borrowing-to-buy-linkedin) [1] [http://gizmodo.com/5806227/did-you-know-microsoft-makes- five...](http://gizmodo.com/5806227/did-you-know-microsoft-makes-five-times- more-money-from-android-than-from-windows-phone)

</comment></comment></comment></comment><comment><USER9> The sentense about GMail using XHR before Googe Suggest is wrong. GMail and Google Maps used iFrames, but not XHR in the first few years, and started to use XHR only in 2006+.

</comment><comment><USER10> _This was the good-old-days when critical features were crammed in just days before a release_

 _We did a quick security review but at the time no one really understood the seriousness or types of vulnerabilities that anything shipping could cause_

Explains the horrendous level of quality of all Microsoft products at that time! I find the honesty in these statements very refreshing.

<comment><USER11> Seems like a bit of a contrast between that and this:

[http://moishelettvin.blogspot.ca/2006/11/windows-shutdown- cr...](http://moishelettvin.blogspot.ca/2006/11/windows-shutdown- crapfest.html)

</comment></comment><comment><USER12> The first time I used dynamically loaded XML in the browser was in 2000, but it wasn't XMLHttpRequest -- it was in Flash.

Flash 5 came out that year, and the player was ubiquitous at something like 98% market penetration (it was preloaded by both Microsoft and Apple).

That was the first release that included real ECMAScript-compatible ActionScript, and it also had APIs for asynchronously fetching XML data. IIRC implementing it on a server was somehow tricky, but it was my first taste of a true rich client in the browser.

</comment><comment><USER13> Thank god for smmry

Reduced to 5 sentence for those who don't want to read the whole post.

` How we actually shipped XMLHTTP It was one thing to have this cool component we could use to talk to the server, but we still weren't in any position to use it in the shipping product yet.

Which is the real explanation of where the name XMLHTTP comes from- the thing is mostly about HTTP and doesn't have any specific tie to XML other than that was the easiest excuse for shipping it so I needed to cram XML into the name.

The beta shipped and the OWA team was able to start running forward using the beta IE5, while Shawn fixed a number of bugs through the ship day of IE5.

Actually shipping the thing required help from many more people that I've mentioned so far.

The lesson to take out of this thing is to appreciate the importance of shipping and having the patience to let something succeed.</comment>


PHP Founder Rasmus Lerdorf Joins Group Payments Startup WePay (YC S09) - <AUTHOR> http://techcrunch.com/2010/04/27/php-founder-rasmus-lerdorf-joins-group-payments-startup-wepay/

<comment><USER2> Congratulations on the hire. Even though I dislike PHP and avoid primarily-PHP projects (which excludes me from something like 75% of all start-ups, but was fine at Yahoo where many teams used Java/C++/Perl for backend/middleware work), I can only say good things about his contributions at Yahoo which went beyond PHP work e.g., educating engineers on web security.

<comment><USER3> Funny thing. When you google your username the first result is the php function.

<comment><USER4> strlen(3) first appeared in Reaserch Unix v7, circa January 1979:

[http://ftp.math.utah.edu/pub///mirrors/minnie.tuhs.org/PDP-1...](http://ftp.math.utah.edu/pub///mirrors/minnie.tuhs.org/PDP-11/Trees/V7/usr/src/libc/gen/strlen.c)

</comment></comment></comment><comment><USER5> I've been hanging out with the WePay guys since shortly after they moved to Silicon Valley. They are really great guys. I also happened to meet Rasmus at WePay's launch party, and he impressed the socks off me with how laid back and unassuming he was. We talked for about half an hour about the history of PHP. I learned a bunch and wanted to hang out with the guy a lot more.

I've been using WePay to manage the payment of sponsorships at the <http://HackersandFounders.com> meetups, and I'm also using it to transfer money between members of my family.

I have no skin in their game aside from really liking the guys. If you're interested in working for a great startup that has amazing potential, you should really send WePay your resume.

</comment><comment><USER6> Very cool. Do you guys do most of your backend logic in PHP as well, or just the website?

<comment><USER7> when we first started, basically the entire application was php. over time we have introduced other various languages in different places

<comment><USER8> which ones? and at what layers?

</comment></comment></comment><comment><USER9> Congrats to both! I think it's important to realize while there are a lot of valid criticisms with PHP (likewise with other popular languages), there are still a great many intelligent developers and programmers in the PHP community. Once you wade through a lot of the cruft (and there's a lot of it), you'll hear from some really intelligent people who use PHP.

</comment><comment><USER10> I've seen Rasmus speak twice and talked to him once. He certainly is very interested in APIs. It seems a theme of his to hack a mashup together with an API that exposes information that wasn't previously available.

I would guess that this post is more about his passion for APIs than it is about PHP.

</comment><comment><USER11> If anyone's ever used PHP, this sounds like the _worst_ hire ever. I bet he's a smart guy, but you can gain some insight on his attitude towards development here -- <http://en.wikiquote.org/wiki/Rasmus_Lerdorf>

"I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say "yeah it works but you're leaking memory everywhere. Perhaps we should fix that." I'll just restart apache every 10 requests."

<comment><USER12> He sounds like one of the most valuable kinds of programmer: humble about his own abilities and more interested in solving problems than being precious about the elegance of his code. PHP itself hews to this same pragmatic philosophy, and it's the reason PHP powers a huge swath of the internet.

Personally, I find coding PHP is like chewing on tinfoil. Nevertheless, it has allowed countless developers to _solve problems quickly_ without having to learn a bunch of boilerplate or internalize elaborate data models.

<comment><USER13> There's a huge difference between actual pragmatism and shooting yourself in the foot in the name of pragmatism.

No, you shouldn't build your program out of some overengineered metahyperpolymorphic Proton and Electron base classes in the name of "elegance". But the problem with PHP is that it goes completely to the other extreme: it has no internal consistency and little foresight. Simplicity is a great desideratum, but it shouldn't come at the expense of sense.

<comment><USER14> In the context of a startup hire, think about it this way:

He has created something very popular and widely used for an extremely difficult to please market with a huge amount of competition, some of it from scary places. The domain is a hard one.

</comment></comment></comment></comment><comment><USER15> Awesome hire, congrats, Bill and Rich! Looking forward to your API.

</comment><comment><AUTHOR> congrats guys, that's huge. i hope you see a large influx of resumes.

</comment><comment><USER16> Nice find!</comment>


Who wants to give me a job? - <AUTHOR>

Hello Hackernews,
I'm rather good with HTML and CSS, I can implement Javascript and Jquery elements wherever I've tried. I'm getting better and better with PHP and am about to start the second year of Higher certificate of web technologies. Really however, I'm pretty much self taught without any commercial web design experience.
I've been working at my personal business site for about six months which you can find at www.wildhives.co.uk
Its taught me a great deal about SEO, google analytics, PHP and basic things such as HTML and CSS. I've recently gotten interested in bootstrap.
Its pretty much running itself now however and I'ld like to take a step into a different river, that of web design and development.
You can find a few further details about me here www.noelvock.com
You can question me further here.
Currently I'ld be interested in small roles, large projects, big roles and small projects.
Is there anyone who would be happy to have an apprentice under wing? <comment><USER2> Higher Certificate? Perhaps my knowledge of this is dated, but after you finish your second year can you not go onto the final-year of a university course? That was the case for HND's a while back.

If you can move on, and get a degree within the next two years then I'd probably stick with the course you're on, and to continue freelancing.

From what I've read, you seem capable, but your tool set seems rather dated, something I've noticed a lot from students that study for web-related certifications. Sadly, the mere mention of Dreamweaver is usually enough to scare people away from you, despite the actual knowledge you need to use any IDE.

<comment><AUTHOR> The degree course I could continue to do has some modules of questionable value and I could learn at a faster more specific rate alone. I used to be somewhat anti dreamweaver, and I still never use it in design view for instance, I use it as a text editor, but as was suggested in another reply to this post I've downloaded netbeans and I'm starting to use that. First impressions, netbeans seems very nice.

<comment><USER2> I know the feeling all too well. In my opinion, a lot of degree courses are not fit for purpose. I studied at an ex-poly that had over 15 different Computing related courses and the only ones of real value were Computer Science and Software Engineering.

I know for a fact that some universities will consider a transfer for your final year to another course. One guy I worked with was on a HND course in Computing and he managed to get into the second year of Computer Science at Bristol University, and one managed to get into the final year at an ex-poly, again in Computer Science. It might be worth considering if you think your grades will be up to scratch.

However, to be fair, most people on here will agree that your skills show far more than what degree you have. Frankly, the value in the degree isn't necessarily the knowledge you attain, but the whole experience, as well as working on projects of a certain scale and complexity. Once you've got a job behind you, the degree is nothing more than a slip of paper.

In regards to IDE's, as long as you're using something used in industry and are using something you are comfortable in there's no need to worry. When I was a student I worked a ton of internships and picked up C# and .NET that way, so my career went down the .NET route, although I've dabbled a bit in PHP and Python where I could. If you're itching to work, then I'd highly recommend sticking with the course, and working over the summer. There are always schemes out there for students, and the experience is invaluable later on.

</comment></comment></comment><comment><USER3> A quick question what kind of IDE do you use when developing?

Couple things about the sites, you might be - as you said - good with HTML and CSS, but you don't have very good design sense. The pages you've linked don't look visually appealing, at least to me.

Also you say on your page that your websites are "coded to W3C XHTML strict guidelines.", but noelvock.com uses HTML5 doctype (as everything should imo) and wildhives nets almost 30 validation errors.

It's a good start and at least you are putting yourself out there, but these would be the things you'd get burned for from my experience.

<comment><AUTHOR> I've had some good feedback about the Wild Hives site with regards design, but hey, preferences like that are largely subjective.

My portfolio site definitely needs some updating and with regards the validation errors, I could fix them, but its low in my list of things to do.

The ide I use is Dreamweaver. I'm also familiar with Photoshop and Fireworks.

Thanks for the feedback, its very welcome and hoping for further suggestions..

<comment><AUTHOR> Here's something I whipped together in 45 minutes

[http://www.noelvock.com/sgto](http://www.noelvock.com/sgto)

</comment></comment></comment><comment><USER4> There is an official "Who is hiring?"/"Freelance?" monthly post the first day of each month. They usually get to the first page. You should post there. Wait until September 1st.

To get an idea of how it works, see the August editions: [https://news.ycombinator.com/item?id=6139927](https://news.ycombinator.com/item?id=6139927) [https://news.ycombinator.com/item?id=6139937](https://news.ycombinator.com/item?id=6139937)

<comment><AUTHOR> Wasn't aware of that, thanks.</comment></comment>