Ask HN: How can I deploy a private iOS app? - <AUTHOR> Thinking of writing an app for me and some friends. I do not want this to go on the app store at all, and I've never written an iOS app before. How can I do this for the lowest price possible? I'm going to use react native to make cross platform easier if that makes a difference. I also don't own an iOS device right now so I'm really blind to the whole process atm. <comment><USER2> [http://www.techrepublic.com/blog/smartphones/deploying- inter...](http://www.techrepublic.com/blog/smartphones/deploying-internal- apps-on-an-iphone/)

</comment><comment><USER3> HockeyApp or TestFlight should do.</comment>


JupyterCon: I don't like Notebooks [slides] - <AUTHOR> https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUhkKGvjtV-dkAIsUXP-AL4ffI <comment><USER2> I seriously cannot agree more.

Jupyter notebooks were fun to use for a bit, then I hit the inevitable wall of "ok, now let's turn this into a real, properly built script, but now everything is breaking for inexplicable reasons".

Notebooks are fine for early stage experimenting, but if you've got to the point where you start up you are relying on a notebook for anything, or your workflow consists of "start up notebook, run cells until I get to the one I'm working on" then you need to stop, and build it into something proper.

Experimentation is fine, but it should not come at the cost of writing things properly when the time comes, and they are not an excuse for not knowing some good software engineering. Serious alarm bells go off in my head when I read tweets like "Data science code doesn't need to follow the rules of good software engineering".

Edit: that remote notebook - is that honestly not one of the most terrifying things you've seen? That runs so counter to almost every good bit of software design and engineering practice about clarity, maintainability, good practice, security, etc that I can think of. It's the very definition of indecipherable, inscrutable hidden state and unknown side effects.

<comment><USER3> I don't even like it for initial experimentation and exploration. Much prefer the command line repl (the jupyter/ipython one). Faster feedback and easier navigation then you can just export the history when you're ready to capture and make permanent some workflow. It still requires going through and picking out the important bits, but reading the history like a story of commands is generally enough to pick out the important parts.

The only thing I would use notebooks for is demonstration / teaching.

<comment><USER4> If you work with image processing, notebooks are very handy as they can display images directly. Also to display tables nicely with pandas, or for any data visualization actually.

<comment><USER5> Doesn't IPython repl running in a QTconsole terminal do that ?

<comment><USER4> Yeah but it's less nice, and if you're running on a distant server you have to deal with X forwarding

</comment></comment></comment></comment></comment><comment><USER6> RMarkdown have all the advantages of notebook and without most of its problems.

\- code and document mixed. You have full markdown syntax, chunks, titles, table of contents. You can also convert RMarkdown into a script with comment if you need.

\- To render a RMarkdown, it will run in a separate environment, start from scratch. It's assumed you should make it reproducible, this is good for report/sharing.

\- I also use RMarkdown to write code, and run code/code chunks interactively in a session. I can write plan, notes, references, TODO in document, test code in chunks, execute code chunks in any order. Basically you just code and document. In the end you can turn it into a report, or refactor the tested code into functions and scripts. I kept the original RMarkdown as design document, which have all the original notes and previous version of code.

\- It's plain text so version control is fully supported.

RMarkdown started to support python, but it may be preliminary for now.

</comment><comment><USER7> I absolutely love Jupyter as a computer science student who uses it a lot to fiddle with new concepts, create visualizations, or write markdown reports with annotated code. I do agree with some points, though.

The problems with state felt sort of overstated (heehee), but it obviously is an issue. I always wonder why the menu option Kernel -> Restart & Run All is not a first class citizen, with a big red button at the top of the window. It's my main way of interacting with the notebook for anything that takes under a minute to run. Running cells one by one throughout the netbook isn't very useful. You either want to rerun the cell you're working on, or clear the state just in case and run everything.

Module versioning is another thing that should be a no-brainer.

I also wish the notebooks diffed better when it comes to SCM, without all the JSON artifacts.

Proper integration with existing Python tools/IDEs would be wonderful.

Also yeah, kernel sharing is one of the most horrific ideas I've heard recently.

<comment><USER8> > I also wish the notebooks diffed better when it comes to SCM, without all > the JSON artifacts.

This is one of my biggest qualms as well. I built a DevOps tool [1] that uses Notebooks & not being able to diff and review Notebooks was a pain. I have decided to solve it with a GitHub marketplace app. Wrote more about it here: [https://medium.freecodecamp.org/how-to-handle-version- contro...](https://medium.freecodecamp.org/how-to-handle-version-control-and- reproducibility-with-jupyter-notebook-e1fbc0b8f922)

[1] [https://nurtch.com](https://nurtch.com)

<comment><USER9> Might want to use the full url [0] when sharing your work - the one you linked gives me problems with the ssl cert.

[0] [https://www.nurtch.com/](https://www.nurtch.com/)

</comment></comment></comment><comment><USER10> Notebooks were not created as a way to implement and organize software. You can do that with code files.

Jupyter Notebook is a presentation software, for demonstrating something to yourself or others. There is hardly anything comparable to build interactive demonstrations. You'd have to implement a (multi-paged?) GUI application, or a web application. Just plain html output may cut it for certain use cases, but still more painful.

And I've done more stupid stuff than that. For example I have a demo recording and showing EEG data. And a snake game implemented as a widget with Bokeh output. And live-update graphs from FlightGear. Lately I've started experimenting with controlling my 3D printer from the notebook.

<comment><USER11> Exactly - it's a classic case of a tool being used for a different purpose ("IDE") than its original intention ("presentation software") and getting criticized for not holding up to misguided expectations.

His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.

<comment><USER12> Although the OP criticizes missing IDE features but that is not his main points.

<comment><USER11> But his main points aren't really showstoppers if you're using it as presentation software.

</comment></comment></comment></comment><comment><USER13> The thing that bothers me about notebooks is that they don't seem to be getting better. Splitting the data from the code so that they play nice with revision control should already be implemented by default! Allowing notebooks to also be libraries should be doable. Testing mechanisms on notebooks should be standardised etc.

Is anyone working on bridging the gap between notebook sketches and production ready code?

</comment><comment><USER14> When I first found notebooks I was amazed at their utility. The ability to quickly display algorithms and processes to a wider (potentially layman) audience is amazing.

To be able to have a scratchpad and play around with techniques while keeping a sort-of record is also amazing.

I see it like I see excel, it's a fantastic tool for data exploration and some visualisation. It's not something that should be used in any final workflow or production system.

Like excel, it can be badly mis-used but they unlock ways of working that simply weren't possible before it.

The criticisms about hidden state are fair, I think it would be better if previous step data was more explicitly wrapped in the following cells so you could choose to use the "wrapped package" of the previous data or choose to use a re-evaluating version.

I think it works best when most cells have few side-effects, even if that means repeating previous calculations.

</comment><comment><USER15> I have tought Python with notebooks to scientists and we always emphasize the pipeline:

Play with your code in the notebook, when you know what the right way to split it is, turn cells into functions. When you have a decent body of functions turn them into a module.

Notebooks excel as ad hoc interfaces to libraries and modules. They don't replace them.

I'm not sure the problems with scientists' code are enabled by notebooks to a significant degree. We used to exclusively use PyCharm and I was still debugging spaghetti scripts with massive global state interacting in weird ways.

Notebooks give people more foot-guns with hidden state. I think they are neutral when it comes to modularity. And I think they are a positive for encouraging documentation.

<comment><USER16> This seems like a decent progression for easing beginners into the basics of good programming practices.

It's certainly not a Jupyter problem _per se._ I went through a similar progression when I learned BASIC in high school, in 1981. Projects grow to a size where they become unmanageable without some structure.

In my view, the importance of documentation is huge. I've been programming for a long time. I don't write _software._ I use programming as a problem solving tool. Jupyter has made a profound difference in my ability to pick up work that I did a month ago, or years ago, and figure out what I did, mistakes and all. For me it's more about reproducible problem solving than software development.

<comment><USER17> Notebooks for me were something I found after iPython Qt console, when I was getting frustrated at how hard it was to replay a set of scripts to get back to my state.

I always view them as an execute in order abstraction. I suspect what we really need to improve them is a subset of python which only deals in immutable data structures somehow - and then let notebooks branch off that immutable state but still be "execute forward" only.

</comment></comment></comment><comment><USER18> The code written by a couple of "data scientists" I was working with is the worst code I have ever seen. They don't care, they just want to have an experimental results. The problem starts when their experimental "code" needs to be used on production or they are asked to describe how it works. Why cannot we just get good programmers and train them as data scientists?

<comment><USER19> _Why cannot we just get good programmers and train them as data scientists?_

For the same reason we can't just get good programmers and train then in biology or chemistry or structural engineering. Sure they exist, as do data scientists that are really good programmers, it's just that they're more rare and in very high demand.

Often much easier to find a domain expert and a programmer and have the programmer rework the code done by the domain expert. In fact that used to be my job for a while (working with physicists), and it was actually quite fun.

<comment><USER20> > Often much easier to find a domain expert and a programmer and have the > programmer rework the code done by the domain expert. In fact that used to > be my job for a while (working with physicists), and it was actually quite > fun.

It's basically what people are now calling 'Research Software Engineers'

</comment></comment></comment><comment><USER21> Had a project once where the "super helpful notebook" ended up being nicknamed "The Wall of Madness". Tons of weird out of order errors, #DONT RUN BELOW THIS LINE WITHOUT TALKING TO JIM, etc.

</comment><comment><USER22> My experience with notebooks matches the one presented here. For me the most disappointing thing is that notebooks are JSON, and not just marked up .pys. The cons of this decision outweighs the pros for me.

One thing though--one of the best books I've read, Trefethen's ATAP, was written as a collection of .m files, which when run would produce a pdf of each chapter. The .m files were filled with small formatting details that were simply omitted from the generated book. The slides suggest something equivalent is not possible with notebooks. That's unfortunate.

</comment><comment><USER23> I feel like I haven't really dealt with this out of order execution madness this guy and some others are talking about here, usually I end up with like 2-5 big cells that I use, typically just a "init everything cell" a "run simulation/training cell", and then some plotting cells and other "utility cells" that I use to poke at things.

I've been moving functionality to modules when I can too which helps minimize the amount of code actually in the notebooks, and I also will break up code into different notebooks (occasionally saving/loading specific variables between kernels) when it makes sense to. Maybe all this is helping a lot, have you all needed notebooks with dozens of cells as this presentation mentions?

</comment><comment><USER24> One thing not mentioned was that version controlling notebooks is horrible - try diffing a big json mess that mixes input and output.

<comment><USER25> I would encourage everyone to use [https://github.com/kynan/nbstripout](https://github.com/kynan/nbstripout) before commiting.

</comment></comment><comment><USER26> I'm glad these concerns are becoming more widespread, since they are somewhat subtle and when we originally shipped RunKit many people didn't know why we didn't "just write a js backend for Jupyter". The reality is that solving these problems are a huge engineering challenge and we spent the entire first year of development at RunKit on unifying the "module" and "REPL" environments. Our litmus test was that notebooks would be logical to work with once they could literally be required by other packages as if they were just modules with no modifications. The solution we came up with was VM-level time traveling: if you modify a previous cell, you should rewind the entire state of the machine (including undoing changes made to the file system, spawned processes, etc) and "pick up from there". In RunKit, if cell 3 deletes a file, you can still read the file if you modify cell 2, since we snapshot the entire computer, and thus you don't have to "pretend" you're modifying cell order like with Jupyter. In this way you really do get the best of both worlds: a notebook never has out of order cells or is displaying its contents in an unintuitive way because if behaves "as if" you had just rerun the entire notebook from the start on every change - _but_ with the feel and speed of iteratively appending cells. Additionally, since this isn't done at the "language level", computer-level side effects don't become "out of sync" with your notebook - "oops I dropped a table in the database" - don't worry just rerun the cell with corrected contents, it'll always run with the same state it started with. You can read more about this in a blog post we wrote when we initially released (even though it reads like a direct response to slide 25 of this presentation): [https://blog.runkit.com/2015/09/10/time-traveling-in- node-js...](https://blog.runkit.com/2015/09/10/time-traveling-in-node-js- notebooks/)

</comment><comment><USER27> Fascinating presentation! Joel is such a meme factory :-) Has anyone done a "rejected memes" section at the end of a technical talk before?

On topic: I applaud the effort to set things straight, but I'm afraid that ship has sailed. Hacky "Kaggle notebook solutions" _are_ now Data Science. That's what the term evolved to mean, and all the rest of the impedance highlighted in this presentation follows.

If your work involves creating well-designed, well-factored, tested, reusable ML software that is meant to be integrated, picked apart, extended and applied over time in practice (as opposed to be submitted once to an oracle in order to claim "SOTA"), you better come up with another term. And obviously Notebooks are not a good fit, beyond as a tool for documentation & reporting (which Joel correctly calls out as genuinely useful).

</comment><comment><USER28> See a previous post of mine ([https://news.ycombinator.com/item?id=17840216](https://news.ycombinator.com/item?id=17840216)) for my general (frustrating) experience with notebooks.

I'll add positives notes, though:

-They're excellent for beginner tutorials that explain an abstract concept or a library's use cases

-They're very well suited for academic peer-review. It's basically a way to say: here's what we obtained, here are the exact steps we did to obtain it, you can obtain it as well if you do exactly as we did. You'd think this reproducibility requirement would be common sense but in practice you're usually thankful if the data is available, or the software is released and works.

</comment><comment><USER29> I love notebooks! They are an excellent tool, if used judiciously.

They are really practical for situations where you want to play around (er even outright work with) principally not with code, but that code's output.

You see, I give lots of trainings.

Notebooks offer me an excellent way to mix commands, their output, and explanations into a single document with little effort. I'm able to show my students exactly what happens (including the literal messages), going step-by- step.

They are wonderful to create exercises.

However, for my use case, the notebook _is_ the output (perhaps rendered as PDF).

Rules I've adopted for my own training notebooks:

      * the first lines are to print the versions of all things I'm using, e.g. "git --version" for git trainings
      * I use "restart&run all" frequently
      * obviously, notebooks are version-controlled, including their output
      * before checking in, prove that "restart&run all" provides exactly the desired result

Having said all that, I'd never use a notebook to write actual programs. It feels weirdly impractical, to the point that I was wondering of the presentation was actually presenting reality, or a strawman (I'm not doubting the veracity of the description, I just had a hard time accepting it as real).

<comment><USER30> What do you use to version-control your notebook?

<comment><USER29> Just git.

The tool is just a detail. My point was rather that they are very much "code", and deserve to be treated as carefully as any other.

My notebooks also get refactored on occasion.

</comment></comment></comment><comment> a-dub Agree 100%! The best use of notebooks seems to be making demos of very simple things that demonstrate the idea of notebooks. My gripes:

1) It's claustrophobic! Trying to work in notebooks always felt like trying to do a math problem with too little paper when you're used to big empty sheets.

2) Readability is weird! I want to see all the code and then see the plots, not little crazy crunched up snippets with little crazy crunched up plots that _might_ allow interactivity.

3) I might actually want to do something that involves looking at more than one plot at a time! Seriously? This always struck me as ridiculous. I have screen real estate, I want to use it! ESPECIALLY for interactive data analysis.

4) They're not as portable as they should be! There's always some drama when you open up someone else's notebook locally.

5) They encourage people to write weird ass code! Scientists already have a tendency to be messy if they're not CS types, this just makes it worse.

Alternative:

vim + tmux + ipython REPL + vim/tmux slime for shipping stuff from the editor to the REPL + matplotlib in QT mode

It's not as good as MATLAB but it gets close.

</comment><comment><USER31> What if notebooks re-executed all cells in order as you type? That would solve the ordering and hidden state problem. To speed that up you could take a snapshot of the program state at each cell and re-execute from the snapshot of the cell preceding the cell you're modifying.

The advantage of a notebook over a repl is that the code you typed stays there and can be re-run and modified later. Re-executing all cells in order ensures that that actually works.

You can do away with the cell concept, and instead have some way to annotate which lines display their output. Then the distinction between an editor and a notebook almost disappears. An editor plus a way to annotate which lines' outputs are displayed, plus a way to type rich text, becomes a notebook.

Even better, you could allow users to display the output of lines _inside functions_ , and have a way to select which concrete call is actually displayed. Sean McDirmid has already implemented such an editor. This removes the incentive to avoid abstraction, because it allows you to display outputs even if you put code inside a function or class. It's even better than a repl in this regard, and it doubles as a powerful debugger that can navigate through the execution.

The navigation works similar to an IDE's go to definition, except that when you go to definition on a call, it sets the concrete execution context of the call. For instance,

        function foo(x)
          y = x+2
          return 3*y
        end

        foo(5)
        foo(6)

If you click on the foo(5) call then it jumps to the definition of foo and sets x=5 and displays the outputs of expressions you've annotated, such as y = x+2. A similar mechanism allows you to pick the iteration of a loop. It even works fine in the presence of lambdas, allowing you to debug through callbacks (unlike conventional debuggers).

<comment><USER32> This causes issues when working with large data sets or running expensive operations, which is an advantage of modifying a single step independently from the rest.

<comment><USER33> As an example, loading up a dataset I was working with into RAM from my NVME drive took about 9 minutes each time. Rerunning that step on each script run is productivity-prohibitive as I am just trying to explore and clean the data.

</comment></comment></comment><comment><USER34> I wish we didn't have a culture of being bitterly angry in our talks.

<comment><USER35> I agree. Found the "I know a few things" spiel distasteful, and the general tone to be patronizing and rude. Attitudes likes this are negatively impactful and points for or against notebooks can be made without the toxicity.

<comment><USER36> > _Found the "I know a few things" spiel distasteful_

And if he hadn't included it, all the comments would be "yeah, and who is he to talk".

</comment></comment></comment><comment><USER37> Agree with most of the things in this presentation. BUT. I generally use Jupyter Notebooks for these things: \- Play around with some libs and charts \- Draw charts for my papers or whatever \- In short: draft stuff

And, for serious stuff, of course, python in text files with modules, tests (pytest) etc.

</comment><comment><USER38> Great presentation, does anyone know if it was recorded?

The strength that kept me coming back to notebooks was their power at iterating on a problem but I was continuously frustrated at the difficulty of extracting my solution / tracking it in git / collaborating with colleagues etc. Also I didn't enjoy the editor experience from a UX point of view.

I've since started using hydrogen [1], a plugin for Atom which (via a Jupyter kernel) seems to get what I wanted from both worlds - it's just a python file but I get most of the notebook fun!

[1] [https://github.com/nteract/hydrogen](https://github.com/nteract/hydrogen)

</comment><comment><USER39> Ironic this pops up today when a colleague has this scenario happen...

BA: We found this duplicate field error in our database, do you know about this.

DS: I told you about this 2 years ago.

BA: Oh so you have a script to produce all the duplicate records?

DS: No I did this 2 years ago I don't know what version of Python or any of the libraries I used for the notebook.

In my experience a lot of data scientists like this work alone on business problems - if they were in a more collaborative environment they'd probably run into the problems that Joel is discussing a lot more and be more open to build libraries.

</comment><comment><USER40> I'll be honest and admit that I have barely used a Notebook, but I can see they have their place. They are very popular with scientists, where they are writing experimental code, much of which will be thrown away. For stuff that needs to be used in production then they seem like the wrong tool.

I had actually thought that they would be great for learning until I read this.

(I am a bit disappointed that the presentation doens't actually explain the reason for out of sequence execution, which seems to be one of the central complaints).

</comment><comment><USER41> Notebooks may be suitable for scratching things, but not much else. The problems noted by the OP are very serious, specially in the scientific world, where many do not have proper software engineering skills. The real issue is that newcomers do not know better, and so they don't realize the damage they're inflicting upon themselves and others before it's too late (irreproducible code, hidden states, dep. management, etc).

Even the fast.ai library, which is a wonder, has broken notebooks. For those who try to follow the course at home, trying to run the notebooks is frustrating, as things are out of order and so errors pop up all the time. Jeremy is a wonderful teacher, but compare following a Fast.ai course video, which uses notebooks, to following a python video from, e.g., Raymond Hettinger [2], which uses sphinx and a shell. While the documentation style and ugly shell don't look nearly as cool, they are so much clearer and better structured.

Notebooks become popular because they fill one gap that was left uncovered. As the scientific community moves away from Matlab into Python and R, reading code - pushed, amongst others, by the popularity of Github - becomes a day-to- day activity. Matlab scripts were easily explorable because users would load them, set breakpoints here and there, and look at results interactively - exactly what notebooks aim to provide. The difference is that what used to be breakpoints now become cells, comments now turn into Markdown and figures are inlined to add an extra layer of convenience. Yet all the awful problems of sloppy Matlab development are now masked, marketed as something fancy and start to pollute the Python dev. environment. Reproducibility and testing are gone, dependency management (which is not required for Matlab) breaks down completely, documentation is non-existent, and sharing becomes heavily constrained.

Notebooks may be suitable for scratching things, but not much else, at least nothing serious. Hopefully the slideshow above gets the attention it deserves. And in all cases, kudos for the Jupyter dev team for fighting the good fight, even with the drawbacks of their experiment.

[1] [http://course.fast.ai/lessons/lesson3.html](http://course.fast.ai/lessons/lesson3.html) [2] [https://www.youtube.com/watch?v=9zinZmE3Ogk](https://www.youtube.com/watch?v=9zinZmE3Ogk)

</comment><comment><USER42> I love the notebook concept, it is the best way to show to other people pieces of working code and their results.

The jupyter implementation of notebooks has two serious problems, however:

1\. The notebook source is not natively stored as a text file that you can easily edit with a text editor. You are forced to use the javascript interface.

2\. The editor component has a lot of strange quirks to the point of being unusable.

</comment><comment><USER43> I use notebooks a lot but when I've got a piece of code that I re-use more than once I pull it out into a module or two.

My notebooks are usually set up as: 1) a cell to load external libraries 2) a cell to load any of my own modules 3) a cell to set model parameters 4) a cell containing the functions that I use to load data (to be honest these are not always encapsulated in functions in the early stages of analysis) 5) cells containing functions that do stages of analysis that I'm happy with (but unique to this analysis and therefore not turned into modules). 6) A "main" cell that runs the cells in [5] and gets everything ready for 7) working cell[s] where I mess around with new analysis

While I'm working I only execute from [7] onwards unless I need to reload my data.

This is pretty easy to turn into production if needed because when the analysis is done, you can clean up the working cells and incorporate them into your main function.

</comment><comment><USER44> Some of this is interactive development vs static development. A lot of the IDE features and stuff you get in "static" development, you could imagine getting into notebooks one day. The thing I don't know any good way to deal with is cached state during development.

If I have to get data that takes 15 minutes to query, another 10 to preprocess, and then maybe a few more steps before it's ready to put into my algorithm and start poking around, notebooks shine. In an IDE you could (and I often do) use have cached datasets partway through the process saved to disk, as mentioned in OP. But that's a hack too. And it still takes a minute or so to load and process anyways if cached.

So the options are

1) Shitty practices, but I can write try new changes on your data _immediately_.

2) Better practices, wrapped in caching hacks, but I have to wait minutes between every single change.

</comment><comment><USER45> I use notebooks to test initial model performance, add some meaningful stats, descriptions of parts of computation performed and reasoning behind it, links to external (arxiv/github) papers/sources with detailed method descriptions, visualization of preliminary results of initial computation or hyperparameter search; that is a vital piece of information I provide to my clients who can then make an informed decision which way(s) they should move forward or abort completely. Then a production-ready code is developed outside notebook and productionalized including appropriate services (SaaS, serverless etc.). I found this workflow pretty good in convincing clients as they can play with results early in their decision process.

</comment><comment><USER46> I think notebooks are neat, and can be a useful tool, especially for learning. But until the last few days I had no clue people were trying to use them for actual software development. To me this is a mistake, and adds a lot of tooling to what was and should be fairly simple process of opening up a text editor (of your choice, mine will be emacs). I am also fairly shocked there is a conference around this idea. I feel that this is a fad and distraction :/. It is a shame so much effort is being spent on this idea as a tool for building software. I think its focus should be on becoming a good aid in teaching people concepts. The last thing I want required to write software is a web browser.

<comment><USER47> > I feel that this is a fad and distraction

LMAO. Using a notebook with executable code is literally the modern version of Knuth's literate programming.

While I personally don't do much of it, I loved Peter Norvig's python notebooks[1]. Go through one of them and tell me it's not a good aid in teaching people concepts.

[1]: [https://github.com/norvig/pytudes/](https://github.com/norvig/pytudes/)

</comment></comment><comment><USER48> Joel is definitely not alone, I agree wholeheartedly with everything mentioned in the slides.

I'd like to highlight that Notebooks do have use in pre-production while your are exploring data subset and trying out your initial hypothesis. But please, don't throw away decades of software development best practices and reliability for mere development convenience.

We are providing machine learning tooling and our CTO published a blog post talking about the same exact issues we keep encountering with dozens of customers. ([https://blog.valohai.com/leveling-up-your-ml-code-from- noteb...](https://blog.valohai.com/leveling-up-your-ml-code-from-notebook))

</comment><comment><USER49> Notebooks are a fantastic teaching tool despite some valid criticisms in the slides.

I teach Python to adult beginners including those with heavy Excel background.

Beginners catch on quickly that you need to run all the cells once to get reproducible results.

Worst comes to worse you just restart the kernel.

With Anaconda I can start teaching immediately over all 3 platforms(Win,MacOS and there are even Linux Python beginners) without worrying about differing versions or troubleshooting installations.

My students just git clone/pull my projects and start following around and experimenting themselves.

What would be the alternative, give them black on white REPL?

That said for personal projects that I do not intend to share I reach for my trusty text editor(VS Code is pretty good these days).

</comment><comment><USER50> For the longest time I was looking for a "highlight and run" environment for python. I've got Spyder IDE connected via ssh forwarded ports to a python kernel on a remote server.

This avoids much of the annoyances that the slides point out about Notebook.

</comment><comment><USER51> This was a well crafted and engaging slide deck. I enjoyed this thoroughly. I hope the issues presented are addressed by the Jupyter team. We should always promote good software development habits regardless of the target audience.

</comment><comment><USER52> Hmm, agreed. But before you share a notebook, restart kernel and rerun all cells, fix all errors, and then share... Solves many of his gripes.

I use notebooks all the time, but I put repetitive code in a module, try do put all functions at the start etc. It is much easier to share data science experiments, but indeed notebooks can be used and presented in horrible ways.

So... now we write a presentation on how we can horribly abuse VSCode + an iPython console and call it even?

</comment><comment><USER53> This was great, bravo! While I am not necessarily the audience intended. I've been following some Notebook development. I have always thought of it as convoluted and pedantic. I don't mean to distract from some very solid and well intentioned work. Just that it added a layer of complexity and turned out not to be as portable as advertised.

</comment><comment><USER54> I only use notebooks in R (via knitr). Unless you choose to cache specific blocks by flagging it, it re-runs the whole notebook from scratch every time... So personally I haven't had any of these problems before. Further, the rendering is static, so you can't really choose to run things out of order either.

<comment><USER55> If you use notebooks inside R-Studio (like I imagine 99% would do) you click ctr+enter inside a code block and that block and only that block is executed, and below, the output of that block is rendered.

<comment><USER54> Absolutely don't :)

Emacs + ESS all the way .

</comment></comment></comment><comment><USER56> When I used notebooks the first time my mind automatically assumed the cells would work as a sort of data-flow variables and auto-update when a previous one changed.

Maybe a "this are potentially out of date" marker on out of order cells would help with the problem.

</comment><comment><USER57> I love notebooks, but I agree with you. I like to use notebooks to instrument and debug modular models. Top down design, which is what I was taught years ago when I was learning C/C++ seems harmful when paired with out of order execution.

</comment><comment><USER58> Previous discussion under a submission from the conference where this was presented. [https://news.ycombinator.com/item?id=17839188](https://news.ycombinator.com/item?id=17839188)

</comment><comment><USER59> Has anyone mentioned [https://observablehq.com/](https://observablehq.com/)? I hate notebooks too, but I like this one a little.

</comment><comment><USER60> Although I am a huge fan of Jupyter, Joel made some serious points there. First, we should think about embedding used package versions in our notebook to enhance usability.

</comment><comment><USER55> They are not perfect, but they are still great

</comment><comment><USER61> So, as a sysadmin who is on the verge of deploying jupyter massively (and support it), what should I say to my management ?

</comment><comment><USER12> Yes. Reproducibility is a real problem. I saw my friends who study ML found to reproduce notebooks on github.

</comment><comment><USER62> This person is my hero.

Just... roasted Jupyter notebooks.

</comment><comment><USER63> Serious question, I don't want to attack this presentation but in general am I one of the few that finds meme annoying in slides?

<comment><USER64> I disagree. The memes give me memory cues when I think about the talk later. I have a visual memory so maybe this is why?

Also they are lighthearted and fun and life is short so why not have some fun even when being serious?

<comment><USER34> They only work in that capacity if you already watch a lot of memes, so they effectively act as a shibboleth (in-group out-group separator) for separating people who spend a lot of time on reddit from those that don't.

<comment><USER64> I'm not a Reddit user nor do I keep up with memes so I hadn't seen most of these memes until looking at this slidedeck.

</comment></comment></comment></comment><comment><USER65> The problem here is not notebooks, it's Python. Notebooks don't have tons of hidden state, _Python does_. Fix the language and notebooks are incredible.

</comment><comment><USER66> I also prefer a Desktop, but I can't take it with me.</comment>


MacRuby 0.6 Released - <AUTHOR> http://lists.macosforge.org/pipermail/macruby-devel/2010-April/004722.html

<comment><USER2> > In MacRuby, all Ruby classes and objects are actually Objective-C classes > and objects. There is no need to create costly proxies, convert objects, and > cache instances. A Ruby object can be cast (toll-free) at the C level as an > Objective-C object. The Ruby VM can also handle incoming Objective-C objects > without conversion.

> In MacRuby, the primitive Ruby classes (e.g., String, Array, and Hash) have > been re-implemented on top of their Cocoa equivalents (respectively, > NSString, NSArray, and NSDictionary). As an example, all strings in MacRuby > are Cocoa strings, so they can be passed directly to underlying C or > Objective-C APIs. It is also possible to call any method of the String > interface on any Cocoa string, subclass Objective-C methods, etc.

Interesting. Anyone used macruby, and would like to share experiences.

1\. How is the learning curve for a ruby programmer ?

2\. Are using the String and other core classes just the same as ruby, or more difficult / cumbersome.

3\. Can you reuse code from pure ruby apps, or does this re-implementation cause issues/problems.

<comment><USER3> _1\. How is the learning curve for a ruby programmer ?_

None. MacRuby is an implementation of Ruby 1.9. A very fast Ruby implementation, but, not a completely new language thats only similar. HotCocoa is a gem. A very useful gem that should be the first thing you install, but, a gem.

 _2\. Are using the String and other core classes just the same as ruby, or more difficult / cumbersome._

Just as easily.

 _3\. Can you reuse code from pure ruby apps, or does this re-implementation cause issues/problems._

Sure. Go for it. It is Ruby, after all. Just with an optional gem to access another framework, in this case, Cocoa.

</comment></comment><comment><USER2> Could someone tell me what I would do using macruby. I am a ruby programmer on a Mac. What kind of apps could i write ?

Are these essentially GUI apps which otherwise i would write using Objective C or Cocoa (i assume its a standalone language) or QT, GTK etc.

So does this allow me to write apps that have the Mac L&F ?

<comment><USER4> You can write native Cocoa apps with MacRuby that have Mac Look and Feel, with better than acceptable performance.

There are clean, high-level Ruby wrappings of UIKit classes, see HotCocoa: <http://macruby.com/hotcocoa.html>, although it's not clear to me how extensive this part of MacRuby is.

<comment><USER3> Just to nitpick, its AppKit, not UIKit. UIKit is on the iPhone, where, sadly, MacRuby isn't available.

</comment></comment></comment><comment><USER4> I'm excited by MacRuby's architecture and performance, but can anyone comment on how extensive and usable the HotCocoa bindings are?

</comment><comment><AUTHOR> Ehm, sorry. I didn't notice the "upcoming". So it's not yet released but soon to be. (Is there any way to modify the title?)

</comment><comment><USER5> Nice to see it progressing, but sad that we will probably never see it compiling iPhone apps.

<comment><USER6> Why do you think we'll never see it compiling iPhone apps? Neither the technical reasons - i.e. garbage collection in the iPhone Objective-C runtime, not the political reasons - the infamous 3.1.1, I presume, it doesn't do so already seem to me insurmountable. Especially since it is an Apple run project and one of its main goals is to be a first class Obj-C equivalent.

<comment><USER7> It's important to remember that when NeXTStep and Objective-C were first invented, they were invented explicitly from the inspiration of Smalltalk. Apple today is a descendant of NeXT. I'd speculate that not only that they've taken notice of Ruby, but have decided that since Ruby is the spiritual descendant of Smalltalk, it's worth trying to make Ruby _the_ first-class applications programming language for OS X platforms.</comment></comment></comment>


Open Source .NET libraries that make your life easier - <AUTHOR> http://thomasvm.github.io/blog/2015/03/17/open-source-net-libraries-that-make-your-life-easier/ <comment><USER4> To add to the author's list:

* I seem to install Json.net in every .NET project I create (json serializer - thanks James Newton-King!)

* NLog for logging. It's fantastic.

* Dapper is also excellent (Micro ORM) (and anything written by Marc Gravell is generally top-notch - see protobuf-net, miniprofiler, StackExchange.Redis)

* Automapper, to avoid writing boilerplate code to convert between similarly shaped types (E.G. DTOs to DB models).

* RestSharp makes talking to RESTful apis easier.

* Microsoft's Unity is a great IOC framework, although most people seem to use NInject.

<comment><USER5> How does Automapper compare to Entity Framework?

We use Elmah instead of Nlog, which is also fine. Looks like NLog is a bit more flexible than Elmah is. But Elmah is KISS which is nice.

<comment><USER6> I think you're asking about Dapper (ORM) which could be compared to Entity Framework. The two major things to think about if comparing would be simplicity and speed. Dapper is simple to setup and use. It uses standard SQL and it's very fast. On the other hand, this is not a fair comparison. EF is has many more features and options. Use the right tool for the right job depending on requirements. Elmah and NLog are not apples to apples either. Elmah make dealing with "Unhandled" exceptions very simple. Nlog is for application logging (Trace, Info) and exception management (A try catch scenario).

<comment><USER7> In fact it is not uncommon for me to use both Dapper and EF on the same project. Usually I'll use Dapper for the query/reporting side and EF for the domain model/update side. (Used to use nHibernate, but it seems to have really stagnated unfortunately)

<comment><USER8> Even then, for query _building_ I found EF (or any LINQ provider for SQL) to be easier solution than string concatenation. Well, at least for relatively simple queries with pagination and stuff. For filters, Dynamic.Linq would be helpful here.

On the other side, if you need to work extensively with stored procedures or you have complicated or custom SQL to run, Dapper (or Insight.Database - [https://github.com/jonwagner/Insight.Database](https://github.com/jonwagner/Insight.Database), a library which I found really nice to use) will be your best friend.

<comment><USER7> It's true. What I _really_ would like is a micro-orm like Dapper with support for using LINQ to dynamically build queries. But I suppose this is essentially EF with AsNoTracking, which I have yet to play around with and test performance wise.

<comment><USER8> AsNoTracking makes the materialization indeed faster, but EF always has a cost associated with it, both for initial use (set up of the context etc, which is noticeable) as well as every other query after that (expression tree parsing etc, not as noticeable especially for recurring queries).

I hope it's one of the pain points they will address and solve during their rewrite to EF7 at one point.

That being said, my opinion is that unless you have _really_ strict performance and/or latency requirements, EF should be good enough. At least for simpler queries and in 'longer-running' applications :)

<comment><USER9> My testing showed that AsNoTracking is faster than dapper. But indeed the tests were purely focused on materialization and not query parsing: linq query parsing takes a lot of time, so with lots of predicates etc. you'll see much slower performance with EF compared to other ORMs with query systems other than Linq (except NHibernate, that's slow regardless)

See: [https://github.com/FransBouma/RawDataAccessBencher](https://github.com/FransBouma/RawDataAccessBencher)

</comment></comment></comment></comment></comment></comment></comment></comment><comment> xyz-x Some that I build (and use):

* albacore for cross platform builds with ruby [https://github.com/Albacore/albacore/](https://github.com/Albacore/albacore/) C++.Net, VB, C#, F#

* suave.io [http://suave.io](http://suave.io) F#

* logary for structured logging and metrics and health-checks [http://logary.github.io/](http://logary.github.io/) C# and F#

* Chiron for F# JSON parsing

* logibit.hawk for F# API Hawk verification [https://github.com/logibit/logibit.hawk/](https://github.com/logibit/logibit.hawk/)

* Http.fs instead of RestSharp for F#

* FsSql for ADO.Net interaction

</comment><comment><USER10> Some additional ones I haven't seen mentioned so far:

* FluentValidation: excellent validation library, keeps validation rules grouped together and easy to keep track of - [https://github.com/JeremySkinner/FluentValidation](https://github.com/JeremySkinner/FluentValidation)

* AutoFixture: great for the "Arrange" phase of unit tests. Populates object graphs with test data, highly customizable. There's a bit of a learning curve. Pairs nicely with xUnit but can be used with any TDD library - [https://github.com/AutoFixture/AutoFixture](https://github.com/AutoFixture/AutoFixture)

* Shouldly: a different approach to unit test assertions using "Should" that I find more readable. Also had friendlier error messages when assertions failed - [https://github.com/shouldly/shouldly](https://github.com/shouldly/shouldly)

Shameless plug of my own library:

* Regextra: aims to solve some common regex related scenarios. Features a Passphrase Regex Builder to generate a pattern for passphrase criteria, and also supports named templates and a few utility methods - [https://github.com/amageed/Regextra](https://github.com/amageed/Regextra)

<comment><USER11> I've been considering abandoning AutoFixture for FsCheck.

Partially because that's what AutoFixture's maintainer seems to have done recently. Mostly because FsCheck adds a number of additional features such as automatically reducing failing test cases down to the the simplest failing example it can find that make it very powerful for more complex cases.

One downside, though, is that the C# interfaces are somewhat neglected. It's really a lot nicer to use from F#.

[https://fsharp.github.io/FsCheck/](https://fsharp.github.io/FsCheck/)

</comment></comment><comment><USER12> In addition to the great ones already listed on the link and in the comments here, I'd like to list 2 of my favorites:

* Simple.Data: [https://github.com/markrendle/Simple.Data](https://github.com/markrendle/Simple.Data) , the smallest and simplest ORM you'll ever use.

* Hyperletter: [https://github.com/Jiddler/Hyperletter](https://github.com/Jiddler/Hyperletter) , a very easy way to do inter-process communication. Great for writing distributed software over http, but can be just as useful locally.

</comment><comment><USER13> Check out Akka.Net - [http://getakka.net/](http://getakka.net/)

This is a port of the Akka framework from Java to the .Net platform. It's an actor model framework which works incredibly well for building highly concurrent and distributed applications. They just hit version 1.0 and are out of beta and are backed by a commercial company Petabridge.

Highly recommended.

</comment><comment><USER14> I used to use NLog for logging until I've discovered Serilog ( [http://serilog.net/](http://serilog.net/) ) - simple .NET logging with fully- structured events.

<comment><USER15> Serilog to me is a much more modern logging solution. Allows us to pipe logs to table storage/sqlserver/loggly/logentries/docdb/azure event hub ... its the backbone of our app - really just awesome

<comment><USER6> would you be interested in sharing your setup or pointers toward a good solution with Serilog? I found that something other than a basic setup requires a good chuck of effort to get going. For example, configuration (all code) and IOC.

</comment></comment></comment><comment><USER16> Humanizer Fantastic string, number, and datetime manipulation library [http://www.nuget.org/packages/Humanizer/](http://www.nuget.org/packages/Humanizer/)

Units.Net I love this for measurement and weight conversions [http://www.nuget.org/packages/UnitsNet/](http://www.nuget.org/packages/UnitsNet/)

And, of course, a shameless plug of my own library Fibber An indiscriminate data generator that will generate random data for all properties in a given class based on the property's type vs. its name [https://github.com/Schandlich/Fibber](https://github.com/Schandlich/Fibber) [http://www.nuget.org/packages/Fibber/](http://www.nuget.org/packages/Fibber/)

<comment><USER17> Fibber looks great, how does it compare to AutoFixture?

<comment><USER16> Thanks!

I can honestly say I had not heard about AutoFixture until today. Haha.

They look similar. The only difference I can really see is that I prefer my syntax a little better for type fixtures:

.For<string>("whatever"); vs fixture.TypeMappings[typeof(string)] = s => "whatever";

I also allow a seed to be set for ensure "consistent random data".

But there are some features of AutoFixture that I definitely want to look at.

I created Fibber more for mocking data sources when I was creating Web Api services; and AutoPoco was just too much configuration. Unit testing was not the main purpose of the library.

</comment></comment></comment><comment><USER18> Surprised no one has mentioned Nancy: [https://github.com/NancyFx/Nancy](https://github.com/NancyFx/Nancy) I can't imagine building APIs without it now.

<comment><USER19> I find NancyFX really interesting, but I can't really see a reason to stray away from Web API2 besides Mono support. Any specific reasons you prefer Nancy?

<comment><USER20> I've been wondering this, too. I've been using MVC since version 2 and am very comfortable with it and Web API, but a startup I've been doing side-work for is using Nancy. Nancy does seem lighter weight, and includes TinyIoC for dependency injection out-of-the-box, but I too don't see a real killer use case for Nancy.

</comment></comment></comment><comment><USER21> Here's another one that will make your life easier: [https://github.com/adamcaudill/libsodium- net](https://github.com/adamcaudill/libsodium-net)

Now you have no excuses for rolling your own broken crypto.

</comment><comment><USER22> This (article + HN comments) will be extremely handy for me in the future (I'm a .NET dev).

Knowing that'll need some of those libraries later I decided to compile into a list here: [https://github.com/tallesl/.NET- libraries](https://github.com/tallesl/.NET-libraries)

</comment><comment><USER23> With .NET going open source, I'd like to see what is out there for web services. Are there any linux-compatible .NET web servers/routers that I can use that have a similar feature set and performance to the Spray/Scala stack? C# or F#, but preferably F#. I've tried a couple (suave, nancy), but some simple benchmarks showed they were about half as scalable as spray.

<comment><USER24> Were those simple benchmarks running on Mono? .Net is certainly _going_ open- source, but it hasn't arrived yet. It might be worth rerunning those benchmarks once .Net proper is available for *nix.

<comment><USER23> Yes they were on the latest version of Mono, which in my experience outside of web programming is just as fast as the JVM. I think the gap has more to do with the libraries than the VM, but I'd be willing to wait it out to see. I'm not migrating soon anyway...although it would be nice to know how far out it is.

<comment><USER8> Lately I checked the Perlin noise performance test that was linked to on reddit ([http://www.reddit.com/r/programming/comments/31mzu1/go_vs_ru...](http://www.reddit.com/r/programming/comments/31mzu1/go_vs_rust_vs_d_vs_crystal_etc_perlin_noise/)).

Summary:

Benchmark: C# (Mono) - 1052.041381 ms, Java - 597.9874 ms

My machine: C# (Windows) - ~475 ms, Java - ~552 ms

Even considering that Mono is using maximum precision available (so instead of floats it's using doubles), when I swapped all floats to doubles it still ran in ~525 ms on my box.

So while Mono might be 'getting there', sometimes it's still lagging behind. It's just something to keep in mind :)

</comment></comment></comment></comment><comment><USER25> TopShelf looks interesting, I have written quite a few Windows services over the years and there have always been lots of gaps in the MS offering.

<comment><USER26> I've just rolled out a service using TopShelf having not used it previously. It really does simplify things. The best bit for me is being able to just hit F5 and having the thing run as a console application. No need to manually attach to processes for debugging.

<comment><USER27> This technique lets you debug your service as if it were a console app - <USER2> pass in -a on the command line.

In your Main method:

                if (args.Length > 0 && args[0] == "-a")
                {
                    // Running as an application
                    MyService ms = new MyService();
                    ms.DoStart();
                    Console.WriteLine("Press Enter to Stop");
                    Console.ReadLine();
                    ms.DoStop();
                } else {
                    // Running as a service
                    ServiceBase[] servicesToRun = { new MyService() };
                    ServiceBase.Run(servicesToRun);
                }

And in the MyService class that derives from ServiceBase:

            protected override void OnStart(string[] args)
            {
                DoStart();
            }

            protected override void OnStop()
            {
                DoStop();
            }

            internal void DoStart()
            {
                // Start your service thread here
            }

            internal void DoStop()
            {
                // Tell your service thread to stop here
            }

<comment><USER28> I use something similar to debug a WCF project:

      private static void Main()
      {
        var serviceHandler = new ServiceHandler();

        if (Environment.UserInteractive)
        {
          serviceHandler.OnStart(null);
          Console.WriteLine("Service started! Press <ENTER> to terminate service.");
          Console.ReadLine();
          serviceHandler.OnStop();
          Console.WriteLine("Service stopped!");
          return;
        }

        Run(serviceHandler);
      }

</comment></comment></comment></comment><comment><USER29> I've recently discovered Linq2Db. Works great for CRUD in conjunction with Dapper. [https://github.com/linq2db/linq2db](https://github.com/linq2db/linq2db) It also comes with handy T4 templates to generate models.

<comment><USER30> I've been using it for long time now. Very fast, light, and stable. Thanks Igor!!

</comment></comment><comment><USER31> For those interested, here are the licenses each project is available under: (courtesy Evairfairy)

Hangfire \</comment><comment>--

LGPL3 [https://tldrlegal.com/license/gnu-lesser-general-public- lice...](https://tldrlegal.com/license/gnu-lesser-general-public- license-v3-\(lgpl-3\))

Postal \</comment><comment>

MIT [https://tldrlegal.com/license/mit- license#summary](https://tldrlegal.com/license/mit-license#summary)

Formo \-----

MIT [https://tldrlegal.com/license/mit- license#summary](https://tldrlegal.com/license/mit-license#summary)

CsvHelper \</comment><comment>---

MS-PL [https://tldrlegal.com/license/microsoft-public- license-(ms-p...](https://tldrlegal.com/license/microsoft-public-license-\(ms- pl\)) Apache [https://tldrlegal.com/license/apache- license-2.0-(apache-2.0...](https://tldrlegal.com/license/apache- license-2.0-\(apache-2.0\))

Topshelf \</comment><comment>--

Apache [https://tldrlegal.com/license/apache- license-2.0-(apache-2.0...](https://tldrlegal.com/license/apache- license-2.0-\(apache-2.0\))

</comment><comment><USER9> See: [https://github.com/Microsoft/dotnet](https://github.com/Microsoft/dotnet)

This is an extensive list of (non-MS) OSS libs and tools for .NET , complete with descriptions. The URL might suggest this is the .NET source, it's a set of text documents with URLs to OSS libs/tools.

<comment><USER32> Really nice list, especially under [https://github.com/Microsoft/dotnet/blob/master/dotnet- devel...](https://github.com/Microsoft/dotnet/blob/master/dotnet-developer- projects.md)

</comment></comment><comment><USER33> It's been years since I worked in C#, and now I want to try out most of these libraries. I'm genuinely impressed by the ecosystem out there.

</comment><comment><USER34> One of my recent favorite libraries lately is DynamicData [1] which is Rx for collections. It allows you to use LINQ operators to create lots of different collections from a single "observable cache". An example from the github page:

        var list = new ObservableCollectionExtended<TradeProxy>();
        var myoperation = somedynamicdatasource
                            .Filter(trade=>trade.Status == TradeStatus.Live)
                            .Transform(trade => new TradeProxy(trade))
                            .Sort(SortExpressionComparer<TradeProxy>.Descending(t => t.Timestamp))
                            .ObserveOnDispatcher()
                            .Bind(list)
                            .DisposeMany()
                            .Subscribe()

This takes the collection at _somedynamicdatasource_ , transforms the contents to TradeProxy objects, sorts it, and binds it to _list_. Any time that an object is added or removed in _somedynamicdatasource_ , _list_ will be updated to match. This library has over 50 operators including dynamic filters (changing the filter causes the bound collection to be reevaluated), boolean operators (only update the resulting collection if the object is in both source collections or only one), etc.

[1] [https://github.com/RolandPheasant/DynamicData](https://github.com/RolandPheasant/DynamicData)

</comment><comment><USER35> I'd like to add one that's great for turning the web into a query-able database: HtmlAgilityPack. HtmlAgilityPack + Linq is amazing.

<comment><USER8> For working with HTML (parsing, extracting), I'd like to add HtmlTypeProvider from F# Data as one of the options. :)

</comment></comment><comment><USER36> I have to say that for someone starting to transition to .net, this is a great post. Posts like these are what I expect from the Ruby/Python/Node/PHP communities but the lack of them and lack of good learning opportunities had made me hesitant to use .net in the past. Thanks a lot, you've just made me feel a whole lot better about the .net community as a whole.

<comment><USER37> Please visit regularly asp.net\community , codeplex.com . There is ton of stuff there

</comment></comment><comment><USER17> Fasterflect, must-have reflection library [https://fasterflect.codeplex.com](https://fasterflect.codeplex.com)

Insight.Database, in my opinion much better than Dapper [https://github.com/jonwagner/Insight.Database](https://github.com/jonwagner/Insight.Database)

</comment><comment><USER38> One of my must haves that I'd definitely recommend is T4MVC[0]

        T4MVC is a T4 template for ASP.NET MVC apps that creates strongly typed helpers that eliminate the use of literal strings in many places.

[0]: [https://github.com/T4MVC/T4MVC](https://github.com/T4MVC/T4MVC)

<comment><USER39> Wow. I have suffered so much without it, unnecessarily! How can this not be built into the ASP.NET MVC framework?

</comment></comment><comment><USER40> Few more:

* ServiceStack - full-featured Services Framework [https://github.com/ServiceStack/ServiceStack](https://github.com/ServiceStack/ServiceStack)

* ServiceStack.OrmLite - Fast, Simple, Typed ORM for .NET [https://github.com/ServiceStack/ServiceStack.OrmLite](https://github.com/ServiceStack/ServiceStack.OrmLite)

* BCrypt.Net - A .Net port of jBCrypt implemented in C# [http://bcrypt.codeplex.com/](http://bcrypt.codeplex.com/)

* Stripe.net - .net api for Stripe [https://github.com/jaymedavis/stripe.net](https://github.com/jaymedavis/stripe.net)

* TuesPechkin - .NET Wrapper for the wkhtmltopdf library (to generate PDFs) [https://github.com/tuespetre/TuesPechkin](https://github.com/tuespetre/TuesPechkin)

</comment><comment><USER41> My personal list:

* RestfulRouting [http://restfulrouting.com/](http://restfulrouting.com/) \- must have

* Turbolinks for .net [https://github.com/kazimanzurrashid/aspnetmvcturbolinks](https://github.com/kazimanzurrashid/aspnetmvcturbolinks)

* Flash messages for .net [https://github.com/khalidabuhakmeh/MvcFlash](https://github.com/khalidabuhakmeh/MvcFlash)

* Fixie, unit testing [https://github.com/fixie/fixie](https://github.com/fixie/fixie)

* Entity Framework testing tool [http://effort.codeplex.com/](http://effort.codeplex.com/)

* Automapper, AutoFixture, FluentValidation - mentioned already

* StructureMap - <USER3>

</comment><comment><USER42> Does anyone have recommendations for frameworks or libraries for implementing CQRS and distributed processing of large sets of data through DDD?

I've found some different options including Lokad, NServiceBus, MassTransit, MPAPI, but would love to hear what, if any, tools HN users are utilizing.

<comment><USER43> I started use Circus. New and in active development. Greg Young mentioned at GOTO talk last year ([https://www.youtube.com/watch?v=8JKjvY4etTY&t=2712](https://www.youtube.com/watch?v=8JKjvY4etTY&t=2712))

[https://github.com/d60/Cirqus/](https://github.com/d60/Cirqus/)

<comment><USER42> Thank you. I hadn't heard of this one, so thanks for linking this and the video.

</comment></comment></comment><comment><USER18> One more I'll add to the mix that has come in handy several times: CsQuery jQuery port. [https://github.com/jamietre/CsQuery](https://github.com/jamietre/CsQuery)

</comment><comment><USER44> Some that I've been enjoying lately:

\- SpecsFor: Some nice BDD wrappers around NUnit and Moq [http://specsfor.com/](http://specsfor.com/) \- Glimpse: A great diagnostic console tool for ASP.NET applications [http://getglimpse.com/](http://getglimpse.com/) \- FluentMigrator: Awesome DB migration framework [https://github.com/schambers/fluentmigrator](https://github.com/schambers/fluentmigrator)

</comment><comment><USER45> re: Formo

Last summer and into the fall, I had plans for Formo 2.0, which included a lot more flexibility and customization capabilities (json config, http context config, environment variables, etc). When I learned about aspnet5's configuration I essentially stopped development on Formo. The big reason being that they're building in such a way that I imagined Formo would be used. (Which makes me quite happy)

For that reason, the project is largely "finished", but it's still stable and alive.

</comment><comment><USER46> Great list. I found BizArk.Core powerful console application entry point library very useful. It provides command line arguments parsing based on class and properties atributes [https://bizark.codeplex.com/wikipage?title=Console%20applica...](https://bizark.codeplex.com/wikipage?title=Console%20application%20with%20command- line%20parsing%20and%20validation&referringTitle=Documentation)

</comment><comment><USER47> Great list, but could use more F# love!

Lots of cool/useful open source projects are always popping up in the F# community - [http://fsharp.org/community/projects/](http://fsharp.org/community/projects/)

Favs of mine from that list: Visual F# Power Tools, FSharp.Data.SqlClient, FsEye, Fantomas, R type provider

</comment><comment><USER48> Reading through the list I realized that we have implemented every single one of these things for ourselves over years of coding.

</comment><comment><USER49> You guys should check these ones:

* Hangfire([http://hangfire.io/](http://hangfire.io/)): for background jobs.

* Polly([https://github.com/michael-wolfenden/Polly](https://github.com/michael-wolfenden/Polly)): For exception handling policies.

Edit: formatting.

<comment><USER33> Hangfire is the first library in the post.

<comment><USER49> My fault, I just read the comments here :)

</comment></comment></comment><comment><USER50> I can't believe no one mentioned EPPlus for manipulating excel spreadsheets.

FastJSON is my favorite non JSON.NET Json serializer.

</comment><comment><USER51> As an alternative to CsvHelper I can recommend FileHelpers by Marcos Meli: [http://filehelpers.sourceforge.net/](http://filehelpers.sourceforge.net/)

Don't be put off by the dated SourceForge location. It is pure awesome for getting data in and out of CSV format.

<comment><USER27> How well does it handle the oddball cases? Not that you could call CSV a rigorously-defined spec, but we have a vendor that sends us a file where they sometimes don't escape double-quotes correctly.

<comment><USER51> That is where I've found it's strengths to be. I've used it to process affiliate feed data. The quality is pretty poor and it has been pretty robust.

</comment></comment></comment><comment><USER52> About Object Mappers

* AutoMapper is a great mapper, but too slow [http://automapper.org/](http://automapper.org/)

* TinyMapper is extremely quick [http://tinymapper.net/](http://tinymapper.net/)

</comment><comment><USER53> A collection of awesome .NET libraries, tools, frameworks and software (curated by community) [https://github.com/quozd/awesome- dotnet](https://github.com/quozd/awesome-dotnet)

</comment><comment><USER54> Hangfire looks great. I'm currently running all my jobs through Azure Scheduler. Would anyone know what the pros & cons are between these two job schedulers? Are there certain requirements or architectures that favor one or the other?

<comment><USER55> Perhaps this is obvious but if you aren't running your own server, your web app (and Hangfire along with it) can stop running during a period of inactivity. Obviously this is not a problem with Azure scheduler.

I use Hangfire and then simply have Azure Scheduler ping my site regularly.

</comment></comment><comment><USER56> Will check out top shelf and love the list.

</comment><comment><USER57> very handy!</comment>


Daring Fireball: On Attribution and Credit - <AUTHOR> http://daringfireball.net/2011/07/attribution_and_credit <comment><USER2> Gruber is at his worst in this piece.

First, as he notes, this happens a LOT (link, but not explicity attribution). And as he even notes, he does it himself -- and he does it regularly. The difference between him and Ina? Apparently, Ina's readers don't follow links (how does Gruber know that -- he doesn't). And as he notes, many sites do far worse -- no link, no attribution. But apparently this isn't about the crime so much as the criminal (allthingsd).

And of course he conveniently picks an Apple story to become self-righteous. I could have found hundreds of non-Apple stories over the past five years where this happened. But clearly none of them are important. Only Apple writers count?

He then goes on and claims that the correction was even wrong:

 _"Enthusiast site" is pejorative. Enthusiast implies that MacStories is produced by zealous hobbyists._

This is the oddest read of that term I've ever seen. When people say it is an "Apple enthusiast site", that means it is a site read by Apple enthusiasts. And MacStories is that. As is DaringFireball. Engadget is a gadget enthusiast site. Sports Illustrated is a magazine for sports enthusiasts. There's nothing pejorative about it. A "fanboi" site, sure that would be pejorative, but not this.

And Gruber, effectively put his attack dogs on Ina, and for what, for not explicitly calling out the name of this site. For a while her comments were filled with hateful attacks against her as a person, and many saying they had come from Gruber's site. They were removed, but they were the types of personal attacks you'd expect from a 1960s civil rights protest, but not because someone didn't add explicit attribution (because clearly running the story with a web link isn't enough).

One of his last lines is this, "Defensiveness is never flattering." Gruber should have considered starting with that line.

<comment><USER3> As a journalist who has ben taught to always attribute quotations and sources, I agree with Gruber. Ina's attribution to "an enthusiast site" is disingenuous, like those non-apology apologies that say "I am sorry you felt insulted" instead of "I am sorry I insulted you". His criticism of Fried for referencing the URL instead of the name of the blog is also spot on.

Gruber also is right to make a distinction between writing that pretty much "consume(s) the story", and for that reason requires full attribution, as it will be read by many as a piece of text with no links, and his own teasers that give nothing away but a link to the real story, so attribution is less important.

His "order of preference" listing of how to give attribution is an object lesson in digital journalism. I have forwarded this piece to my friends who teach in J-schools.

<comment><USER2> No the real lesson is don't do stories sourced by Apple fan sites. This was a great example of a story that wasn't worth covering to begin with, and Apple zealots will tear into you for not showing proper deference to the Apple gods.

The one thing the tech industry could do without are so many pointless Apple stories.

<comment><USER3> I can agree with that. They could write less Apple stories.

But if they do write Apple stories, they should recognise the people whose work they draw from as colleagues in journalism, not dismiss them as "enthusiasts".

</comment></comment></comment></comment><comment><USER4> Wow this may be the first Gruber article I've actually liked. Professional blogging has devolved almost exclusively into rewriting someone else's article, cramming their story onto your site full of your links to your stuff and usurping as much traffic, SEO and ad impressions as you can get for it.

And it's really nice to see it called out. All of these blogs would be much more enjoyable to read if they weren't afraid of losing ad impressions by linking to something useful instead of a tag page or past fluff piece.

</comment><comment><USER5> Gruber went overboard in his criticism of the wording of the attribution.

"Never attribute to malice that which is adequately explained by stupidity." - Robert J. Hanlon

</comment><comment><USER6> It's always boring when a blogger writes a blog post about his own blog ('Hey guys, been meaning to blog more, but I've been busy with midterms...'). This is basically the same kind of thing, writ large, and also boring.

For my part at least, I know I don't read Daring Fireball every workday in order to read about the minutiae of operating a high-traffic tech blog and the etiquette of attribution and linking.

(I read it for the high-grade industrial snark, of course!)

</comment><comment><USER7> You can agree or disagree with this piece, like or dislike Gruber but you have to accept one thing: He has personality and he has things to say.</comment>


NLP toolkit for Node.js - <AUTHOR> https://github.com/Amberlamps/nlp-toolkit <comment><AUTHOR> NLP means dealing with huge amount of data. This module implements most common NLP tasks using streams to keep memory usage low.</comment>


CDNlion launching to disrupt the market - <AUTHOR> http://cdnlion.com/we-are-on-the-line/

<comment><USER2> "CDNlion™ operates the largest network of data centers in the CDN market." \- are you running your own servers or is everything build on top of an existing CDN Marketplace?</comment>


A seed-stage accelerator for women in STEM - <AUTHOR> https://wefunder.com/xx <comment><USER2> Founder of Wefunder here.

50% of Wefunder's team of 22 are women. So we're passionate about helping more female founders, particularly inspiring very early-stage ones that this path is possible.

We're open to any ideas on how we can make this better!

<comment><USER3> Hi! I'd love to connect with you. I work with non-technical founders, mostly women, to help them sell the right product before investing too much in developers building the wrong one. There are a few people I can share this opportunity with and can think of a bunch of places on Facebook to get the word out.

<comment><USER2> Awesome. Jiwon is organizing it - you can email her at firstname.lastname@example.com. Thank you!</comment></comment></comment>


I discovered you can hack most Twitter giveaways to win them - <AUTHOR> https://levels.io/giveaway <comment><USER2> Great breakdown. Would be interesting to see over the next month what the churn is on your new followers from the giveaway.</comment>


Ask HN: S-expression based vector graphics editor? - <AUTHOR> I saw a link on HN a while back which showed a demo of an online s-expression based vector graphics editor.
It had a text panel, with data in an s-expression format and a graphics panel and you could change the code to change the graphics.
Can't find this again with the any of the search tools. Anyone remember it or know of something similar? <comment><USER2> You're talking about sketch-n-sketch [https://ravichugh.github.io/sketch-n- sketch/releases/index.h...](https://ravichugh.github.io/sketch-n- sketch/releases/index.html)

<comment><AUTHOR> Yes, this looks like it. Thanks very much!</comment></comment>


The App Developer's Guide to In-App Purchases - <AUTHOR> http://blog.apptopia.com/guide-to-in-app-purchase-monetization/

<comment><USER2> Looks really interesting. From my experience monetizing with ads can be a real grind. IAP has a lot of complexity as well though. Curious to hear how many people are using a service to manage IAP and how many are doing it all in house.</comment>


Ask HN: What are the innovations in the world of Web Porn?  - <AUTHOR>

When I first used internet in 1997, www was very much different than what it is now. Ordinary man like me need not waste words explaining how the web has evolved.
However some trends are common. In those days websites provided content and as users we just consumed it. Today we talk of collaboration, social nature of web sites where everyone is contributor and consumer.
But has there been any research, survey in the world of web porn? Has that industry changed in some way? OR is it same thumbnail based galleries?
arent they innovating ? Arent they investing in developing technical competence of innovation ?
Can someone just suggest any good quality write up on this topic ? <comment><USER2> Wired did a big feature on the porn industry a few years back and what they are doing to combat piracy - google "site:wired.com porn"</comment>


Want to Know What's Happening in a Building? Listen in at the Breaker Box - <AUTHOR> http://spectrum.ieee.org/view-from-the-valley/energy/the-smarter-grid/want-to-know-whats-happening-in-a-building-listen-in-at-the-breaker-box <comment><USER2> In the Chicago suburbs the local electric utility (ComEd) has been installing smart meters on residences over the last couple of years. I got mine about 18 months ago.

The meter is Zigbee-capable, so I went out and got a utility-approved gateway to read the meter (the meter allows access only to approved MAC addreses) The Rainforest Automation RAVEn cost me about $50 (it might be discontinued now)

[http://rainforestautomation.com/wp- content/uploads/2014/02/r...](http://rainforestautomation.com/wp- content/uploads/2014/02/raven_datasheet_1.8.pdf)

There are a number of things you can query from the meter, but mainly you can read the current demand every 4 seconds down to about 10 watts resolution (the Verdigris device is claiming 5-Watt resolution). It's all in a tidy XML format with a published API spec.

Long story short, you can play with some of this stuff on your own right now in certain parts of the world.

<comment><USER3> What was the approval process to get the Raven device whitelisted for access to the meter?

(I own a Condo in Schaumburg)

</comment></comment><comment><USER4> > "We shrunk the sensors and got high fidelity signals," Chung says, "by > figuring out novel ways to fit more powerful sensors into tight spaces." > Some of that had to do with the arrangement of the magnetic material, he > says, but can't say more until patents are filed.

Which is interesting. I guess the main component here is manufacturing cost, since very small, very high capability probes exist since the ~60s. E.g. Tektronix has had small probes with 50-100 MHz [which is orders of magnitude wider than you'd need for this] bandwidth since then.

<comment><USER5> It's cost, size, and regulatory. Existing current probes that are magnetically coupled and thus don't require an electrical connection to the mains (which would be a safety/regulatory nightmare) are big: [http://www.tek.com/current- probe](http://www.tek.com/current-probe)

<comment><USER4> A small probe, like the P6021 (which I own [0]) is pretty small. The transformer itself is tiny; there is only a small PCB in the handle with some frequency compensation. (However it is said that these are difficult to manufacture - more so for their Hall-enhanced partners - due to the precisely ground halves of the core).

The bigger probes that are meant to go on bare, live bus bars and such are probably mostly air to give the operator the required safe handling distance... probably not an issue for a clip-on probe that goes on insulated wires?

[0] Technically a P6021 sold to Philips who bundled it up with a matching amplifier of their own design.

<comment><USER5> That probe is pretty small, but keep in mind that many of these need to be tucked into existing circuit breaker panels while maintaining certain clearances for regulatory reasons. They also need to be easy to install for safety and product adoption reasons.

I think we are saying similar things though. I worked on this project with Verdigris (though I was not personally responsible for the design of these probes) and, to your original point, the challenge is fitting these at a reasonable cost while achieving the desired accuracy. I guess I'm so used to low manufacturing costs being a given that I tend to focus more on the other aspects of the problem.

</comment></comment></comment></comment><comment><USER6> This sounds a lot like [https://sense.com/](https://sense.com/)

There was an episode of This Old House about it: [https://www.youtube.com/watch?v=HZft4qoK7Ds](https://www.youtube.com/watch?v=HZft4qoK7Ds)

<comment><USER7> It's very cool, was brought hip to it from TOH.

I've watched a few reviews and videos on it; definitely considering it.

I especially like the diagnostics part of the equation; which is still an algorithm quite in beta.

Beware, their AdSense is strong. It'll follow you forever.

</comment></comment><comment><USER8> Nice. I would love to see something like this for water too, although Without magnetic signals to sample, I'm not sure how it could be done. My girlfriend had a water bill twice everyone else's in the building and the problem ended up being a toilet that was leaking silently.

<comment><USER9> Sounds super similar to some of the research out of (The University of Washington UbiComp Lab)[[http://ubicomplab.cs.washington.edu/publications/](http://ubicomplab.cs.washington.edu/publications/)] back in 2010. There was a project led by Shwetak Patel, that ultimately turned into (Zensi)[[https://www.crunchbase.com/organization/zensi](https://www.crunchbase.com/organization/zensi)].

Basic premise was to use some DSP to detect electrical noise emitted via various appliances and extract usage patterns from this. The same approach could then be used for water and gas systems by detecting pressure, rather than voltage fluctuations.

From what I can tell this seems to have now just been consumed into the depths of Belkin.

<comment><USER10> Amazingly, I can actually speak to this. I'm Ph.D. student in Shwetak Patel's lab, and yes, this looks to be extremely similar to the ElectriSense [1] work done by Sidhant Gupta (now at Microsoft Research).

The water analogue was Eric Larson's Ph.D. work, and was much more difficult from a DSP perspective due to two separate issues:

* When water is flowing without turbulence (laminar flow) it doesn't leave much of a signature on flow or pressure sensors

* Any kind of signature that does exist is often changed significantly depending on the network of pipes that stand between the single point sensor and the source of the water event.

For these two reasons it took a while for us to work out a high enough accuracy for the water stuff (in the end we used a water pressure sensor plugged in close to your water heater as a fairly central point that most people can access easily) and that was published as HydroSense [2]. We like our *Sense names. :P

HydroSense was indeed bought up by Belkin, and they are pushing it forward in a way that we just cannot do in academia, so that's a good thing. ElectriSense I don't remember exactly what the IP status of it is, but I do know at least one student was working on some interesting extensions of it that may be published in the next year.

[1] [http://ubicomplab.cs.washington.edu/publications/electrisens...](http://ubicomplab.cs.washington.edu/publications/electrisense/) [2] [http://ubicomplab.cs.washington.edu/publications/a-longitudi...](http://ubicomplab.cs.washington.edu/publications/a-longitudinal/)

</comment></comment></comment><comment><USER11> > "They then set about rewriting an algorithm they'd been researching at work, > one designed to accelerate packet separation in chips, to apply it to this > problem."

IANAL but are they not worried that Broadcom will sue with statements like that?

</comment><comment><USER12> Eventually the necessary monitoring circuitry will be embedded into every circuit breaker. Of course, there will still be a need for analytics; the raw information from the breaker won't be sufficient.

</comment><comment><USER13> Uggh. Privacy nightmare.

<comment><USER14> Ditto that.</comment></comment>


Microsoft Standard for Progress Bar Display - <AUTHOR> http://thedailywtf.com/Articles/Meaningless-Progression.aspx

<comment><USER2> _This is a Microsoft STANDARD... I'm sure of it!_

The misleading title is a bigger wtf.</comment>


Fermi Pasta Ulam: the paradox that launched scientific computing - <AUTHOR> http://puzlet.com/m/b00fz

<comment><USER2> Setting n=12 for mass/springs shows some pretty nice dynamics in animated d3 graph. Loving the new in-code links, too.

</comment><comment><USER3> fascinating, thanks chaps</comment>


Ask HN: What do you look for in a co-founder? - <AUTHOR>

What traits and skills do you look for in a co-founder when building that initial team? <comment><USER2> For me, the most important trait I look for in a co-founder is trust. If that element is absent, I think building a long-term partnership would be too difficult, no matter how wonderful they may be in other areas. They also need to be reliable, as you need someone who's going to stick it out through the highs and lows. Apart from that, I look for an overall high level of intelligence and someone who compliments my skill set (e.g. someone with a killer sense of design).

<comment><USER3> As you say, trust (or, more specifically trustworthiness). This ends up being a lot in terms of integrity and authenticity.

Conviction -- particularly courage of their convictions.

Understands the difference between fact and opinion. Also known as the Criteria of Reality.

Argues honestly. Especially when combined with conviction and criteria of reality, this allows for the flexibility to to learn and adapt rapidly.

</comment></comment><comment><USER4> Relentless Resourcefulness ;)

</comment><comment><USER5> Undeterred motivation.

</comment><comment><USER6> \- intelligent, capable, agreeable, etc (obvious)

\- complements my skill set

\- understands, and unafraid of, risk

\- if a programmer, compatible with my philosophy (ie not a super- perfectionist who will spend 3 weeks on the "perfect" deployment system, or someone who will argue with me about text editors or some crap)

\- someone who has travelled and has some perspective of the world, ideally having lived overseas .. maybe this is a personal thing but is important to me

\- understands what he/she is getting into and will not bail out halfway before we even have a chance to succeed because they can't live frugally (or whatever)

\- single, or is willing to act like they're single

\- committed, as in they have resigned or will soon

\- I know they can work independently and finish things, ie in the (recent) past they have decided to do something (anything!) and actually done it and can show me. Can be anything at all. This is unbelievably rare, you know

\- Just have that _je ne sais quoi_ air of "this person is serious, they are going to do this, I am comfortable getting on board with this person" feeling which, on the flip side, would be the kind of ineffable quality which makes investors invest in someone

There are probably more but those spring to mind.</comment>


Porsche unveils production version of its Taycan electric sports car - <AUTHOR> https://www.engadget.com/2019/09/04/porsche-taycan-unveil/ <comment><USER2> The presentation was a bit, well, underwhelming.

Still the Taycan is for sure an absolutely amazing car. The first time in a few years there is something that I would call a dream car.</comment>