How to Grow Software Architecture out of Jupyter Notebooks - <AUTHOR> https://github.com/guillaume-chevalier/How-to-Grow-Neat-Software-Architecture-out-of-Jupyter-Notebooks <comment><USER2> So I'm software developer for 10 years that started using Jupyter Notebooks the last year. I absolutely love that the REPL that ipython gives you. Do a query that takes really long, store it in some variable, spend the next hour or two working on that dataset in memory, changing code, iterating, all the while never having to re-execute that query or load data because it's just a REPL.

How can one get the same type of developer experience in Python without using Jupyter notebooks? What I'm talking about, to people that maybe do traditional development in lets say Java or C++ is, imagine running your code in a debugger, setting a breakpoint, when it hits that breakpoint, you see "ah here's the problem", you fix the code, and have it re-execute all the while not having stopped your program at all. No re-compiling, and then having to re-execute from the beginning. It's like once you've done things the Juypter way, how can you possibly want to back to writing code in a traditional sense, it's just too slow of a process.

How do you get that same experience without using Jupyter? I tried the VSCode plugin [1] that tries to make things the same as a Juptyer Notebook, but it's no where near as smooth an experience and feels clunky.

[1] [https://marketplace.visualstudio.com/items?itemName=donjayam...](https://marketplace.visualstudio.com/items?itemName=donjayamanne.jupyter)

<comment><USER3> One thing that I would love in python is some sort of snapshotting,

Like "this is my global state" - the entire state of the python process, all objects, etc.

Then I change a variable and that creates a new state. We have this now.

What I want is a rewind button, so I can go back to the previous state (quickly preferably) like a time travelling debugger.

Someone build this, I'll pay for it (and I'm sure others would too)

I know there'd be some problems with external changes (API calls altering external system, writing to files, etc) that a time travelling debugger might not be able to reverse, but even so, I would live with this limitation happily if such a tool existed

<comment><USER4> A basic way of getting that on Unix systems is forking - you get a copy of the process with the same memory state, and likely relatively small interfaces would suffice to manage which process is currently receiving input from e.g. a REPL. (Sockets, open files, ... are more complex, but I guess many REPL workflows could do without that). Sadly you can't easily save much memory for the snapshots from copy-on-write (which otherwise is a neat trick with forking to keep old state) due to the way Python manages memory, at least not without modifying the interpreter.

<comment><USER5> If anyone wants to walk through these ideas more slowly I write about them in [http://ballingt.com/interactive-interpreter- undo](http://ballingt.com/interactive-interpreter-undo) and implement this in [https://github.com/thomasballinger/rlundo](https://github.com/thomasballinger/rlundo) but as you say, eventually decided I had to implement an interpreter for which serialization of state was more efficient and all external resources had amenable interfaces [http://dalsegno.ballingt.com/](http://dalsegno.ballingt.com/)

<comment><USER3> Will you marry me?

this is really cool, I'm going to give deeper into the code tonight. Awesome work

</comment></comment></comment></comment></comment><comment><USER6> My first thought when I saw the title: why would you want to do that? I feel like the only justification it gave (faster feedback) is way too insignificant compared to the downsides of developing inside a notebook. Joel Grus has a presentation that he gave at the 2018 JupyterCon about this which I found very enlightening and convincing. It is about why _not_ to use notebooks apart from the very beginning of any serious project. The slides are available here: [https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...](https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUhkKGvjtV- dkAIsUXP-AL4ffI/edit#slide=id.g3d168d2fd3_0_255)

Of course I still use notebooks for experimenting (usually it is open beside my IDE), I just think that developing a whole software architecture inside notebooks is not something notebooks are for.

<comment><USER7> If you don't mind elaborating a bit, how deep do you have to get into a project before you get past the point of experimenting?

Is there value in moving away from notebooks all together while you are still learning?

Do these comments apply to platforms such as MyBinder or Jupyterhub?

Finally,(other than Grus, he is excellent) are there any resources, people, or websites you would recommend for maintaining a sterile workflow or for general knowledge?

Thanks.

<comment><USER6> Sorry, I'm afraid I'm much less experienced than you think I am. My comment's value is entirely in sharing the slides, I only put some of my thoughts there so that my comment isn't just a link to a google drive file.

</comment></comment></comment><comment><USER8> I'm a backend python software engineer in my day job, working on production code in a standard git, text editor, .py files workflow. I occasionally use Jupyter notebooks (e.g. yesterday for the AdventOfCode challenges). I am ashamed to say that despite the beautiful front-end that works so well, I absolutely hate using Jupyter notebooks.

\- The mutable state with global variables from all cells in scope drives me crazy. I just want to run an ephemeral script repeatedly so I can be sure what the state is during execution.

\- The process of starting the server and moving code from version-controlled .py files into notebook cells to become part of Untitled(25).ipynb, which can't be sanely version controlled, drives me crazy.

\- Not being able to use my normal text editor drives me crazy.

Instead of building up lines of well-tested python functions in a disciplined line-by-line fashion, periodically committed to git with readable diffs, I end up with a chaotic mess of code fragments in various cells with no confidence regarding what will happen when some subset of the cells are executed, and none of it in git let alone with a sane commit history.

I've tried so many times over the last 10 years, and I feel bad because it's such an amazing project, but I really dislike the experience of using Jupyter instead of standard python development tools.

The reason I do it is for graphics. This isn't a Jupyter gripe other than the diverted attention, but why the fuck can't I just import matplotlib in a normal python file? (under macOS it throws something about "frameworks" which no-one cares to understand. I think there's some incantation that makes it work but seriously, this is ridiculous). And maybe draw graphics to a GUI graphics widget from the (i)python shell like R does.

(No need to reply "Because you haven't written it"! This is deliberately a rant; I contribute to open source projects.)

<comment><USER9> Never tried macOS, but matplotlib works fine under Windows and Linux. Maybe you could save plots to images on disks and prevent them to show? I once ran code that used PLT on a server and I needed to use something like `matplotlib.use('Agg')` to prevent the code from crashing because of lacking graphical output.

See this SO answer: [https://stackoverflow.com/a/34583288/2476920](https://stackoverflow.com/a/34583288/2476920)

Personally, I love to have notebook cells to be able to code without re- running everything. Especially in the case of deep learning, training a model is long. Jupyter is very good for creating and debugging code that: A) needs a trained model loaded for it to work but you want to skip the part of saving/loading the model, or B) code that saves-then-load a model.

If the "mutable state with global variables" drives you crazy, you may want to avoid reusing the same variable names from one cell to another, and reset the notebook more often. Also, avoid side effects (such as writing/loading from disks) and try to have what's called pure functions (e.g.: avoid singletons or service locators, pass references instead). If your code is clean and does not do too much side effects, you should be able to work fine in notebooks without having headaches.

EDIT: typo.

<comment><USER9> Also, you should be able to use your favorite editor for the code outside notebooks (over time, more and more of the code will be outside of your notebook). You might often work in the editor, and at other times in the notebook depending on the nature of the work. As the project advances, notebooks will become less and less important, they only kickstart projects.

<comment><USER8> Thanks. Yes when I'm being organized I manage to get the notebook able to `reload` a module that I'm working on in my editor, which is probably the most important step towards reducing code in the notebook.

But in general I wonder whether this is what I'm looking for: [https://github.com/daleroberts/itermplot](https://github.com/daleroberts/itermplot)

</comment></comment></comment></comment><comment><USER10> Unlike most IDEs which are desktop based, Jupyter Notebook system is fundamentally network based with a standard HTML5 front end sending messages back and forth to the kernel.

So fundamentally Jupyter is not a document based application that most of us software developers grow up with. You have to think in terms of messaging and the cells are just the input/output terminals. The rendering of cells is determined by the runtime container that parse the .ipynb (JSON) string.

Here is an contrived example to explain it:

[https://nbviewer.jupyter.org/gist/ontouchstart/ea1631f69e507...](https://nbviewer.jupyter.org/gist/ontouchstart/ea1631f69e507a81a9d9ec56b79e4d11)

There is a lot going on depends on the container.

<comment><USER11> That's good insight. Since the 50's, coding has been understood as creating text objects that contain the entire specification of a program in a programming language.

Input data, parameters & configurations are considered external to the code object. It's the same with other activities like debugging, exploratory programming, debugging and building tests, which are considered peripheral to the "true coding" activity of altering the source code that will run on the production environment. The single successful exception to that model is the spreadsheet, were data and code are indissolubly merged in the same document; and, despite its success as a development tool for non-programmers, professional developers usually despise it.

Could it be that Jupyter and notebooks are making developers comfortable with a new approach, where the distinction between "coding" and "running the code" is blurred, and where partial execution and mixing code with data are the norm?

Coders are accustomed to using the REPL for this approach, but in the end only the code that is "commited" by copying it to the source file is considered the "final" version. Yet, with notebooks, code that has been written within an interactive session can be stored as the official definition for a part of a program, without having to transfer it to a separate object.

<comment><USER10> At semantic level, coding is still about creating objects and managing their relationships with the environment. An .ipynb file itself just an object tree. Since Jupyter people made the wise decision of using easy to parse JSON format instead of more efficient binary format, the platform is opened up to a big ecosystem that people can make tools for specific need. Like the document centric desktop application revolution in the 80s that gave us MS Office and Adobe Photoshop, the open, data centric web application revolution we are seeing in this decade will fundamentally change how we consume and produce software. It is a natural evolution of computing system production from hardware panels to punch card, to text code instructions to object graph management in network environments. (The buzzword is "the cloud".)

<comment><USER11> Yeah, you're right that coding is creating computational objects.

I was pointing out that "coding" is traditionally understood to refer only to the sequences of instructions in source and object code that remains unmodified during runtime execution, but there are other views that extend the definition to all the other objects involved in running the software in its environment.

In this view, coding is about defining automated behaviors that can execute on their own without a human leading by hand every step. I'm thinking in particular about End-User Development approaches, which allow non-developers to create such automations without ever learning the syntax of a formal language.

Under this approach, the abstraction level may be lower, since the actual instances of data used are as important to understanding the system as the code that processes them.

<comment><USER10> Another type of coding is describing the desired state and let the runtime environment achieve the transition.

Unix Makefile is one of the early examples. So are HTML/CSS etc.

More recent examples are dataflow graph in tensorflow

[https://www.tensorflow.org/guide/graphs](https://www.tensorflow.org/guide/graphs)

and kubernetes

[https://github.com/kubernetes/kubernetes/blob/release-1.3/do...](https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/README.md)

A new idea just occurred to me during a technical lunch meeting with a bunch of friends today: since Jupyter Notebook itself is nothing but a object graph, it might be possible to use a "computed" .ipynb for this kind of "coding", i.e., describing desired state.

That would be TDD in another sense.

<comment><USER11> Yeah, I happen to work with declarative statements + problem solver for a living :-)

In this context, a precise textual description helps to exactly define what you want to achieve, but a graphical view helps to inspect how many different cases are there in your data, and how available context are connected.

Representing state machines with graphs is a relatively common thing in rapid prototyping apps, so I've thinking of building a prototyping app that would also allow embedding real code and data in them (similar to Visual GUI builders, but allowing much more code to be defined visually). There are several tools attempting to achieve that, but all fail at the abstraction problem of defining new instances for your objects (i.e. you can't create new instances of arbitrary visually defined objects, only for things in tables).

<comment><USER10> I really enjoy this long and insightful discussion. Let's see how deep we can go (in terms of the thread tree and thought itself).

Declarative constraints driven problem solver was pioneered by

[https://en.m.wikipedia.org/wiki/Sketchpad](https://en.m.wikipedia.org/wiki/Sketchpad)

and also used in places like Apple's

[https://en.wikipedia.org/wiki/Interface_Builder](https://en.wikipedia.org/wiki/Interface_Builder)

It is interesting that you use "texture description" for byte stream based artifact that we call code, which is runtime independent.

On the other hand, the term "graph" in your second paragraph has double meanings: visual representation for human eye and brain to parse, and mathematical graph that has can be serialized in JSON, XML or whatever byte stream.

I think many people are excited about GUI revolution pioneered by Doug Engelbart, Ivan Sutherland and Alan Kay for different reasons. Some people are excited about the implementations: bitmap display, mouse, etc., some people are excited about the principles and abstraction: MVC, constraints, data flow, messaging, etc, etc.

With Jupyter notebook system, which is based on three state machines: a persistent kernel process, a DOM session in the browser, and a message queue, it is the first time that we might have a nice abstraction to capture a graph in __space and time __.

That is at least why I am getting excited. The other features are just bells and whistles.

<comment><USER11> Earlier today, I took a note to myself saying this:

The next-generation development environment will be a combination of:

* a wiki (outlining content + version control),

* a spreadsheet (functional reactive coding + lightweight schema definition + easy working on collections), and

* a mockup wireframe builder (visual layout + visual state machines + rapid prototyping).

Pluggable connectors to constraint solvers and machine learning would be a plus.

I'm not sure that Jupyter fully captures all parts of the computation graph; most of all, restoring a previous state after closing and reopening it is cumbersome (if I'm not mistaken you need to run each individual cell by hand in the right order). There are theoretical models that could fix that, though, like those allowing for time travel debugging. And the visuals are simple and flexible enough to support many use cases. So yes, web notebooks are a good basis for a knowledge-building programmable platform.

<comment><USER10> Jupyter system has three different subsystems that communicate via message passing: browser state, ZeroMQ and kernel process. Each system has its own leaky abstraction. That is why the version controls and time travel debugging in the traditional sense become impossible. We shoud embrace this challenge and think beyond document based IDEs.

For example, you can think of rendered notebooks as caching. I have even played with the concept of using rendered notebook as the input/data source of another notebook. This looks like a radical idea. But it is exactly how [https://nbviewer.jupyter.org/](https://nbviewer.jupyter.org/) works.

<comment><USER11> > the concept of using rendered notebook as the input/data source of another > notebook

Isn't that essentially what a spreadsheet does? Each table contains the results of a previous computation, intertwined with input and configuration data, and you just keep chaining more and more programs through a single unifying metaphor (the cell coordinate system plus function expressions). This model also gets you reactive recomputation for free.

It's a powerful system, but quite old; I'd say the only radical thing is convincing classic developers that there's a better model than their beloved imperative, one-step-at-a-time idea of what running a program means.

<comment><USER10> Not exactly the same model as spreadsheets. In spreadsheets, a cell is a value in memory. The sheet is a graph of states.

In Jupyter Notebook, the cells are nothing but messages. And the saved notebook files are JSON serialized byte stream of the message and metadata. It doesn't keep the states, just data, a digital artifact.

<comment><USER11> Yet spreadsheet cells may also be formulas, which are akin to method calls / message passing / function application. The value of a cell with a formula is the memoized result of its latest function call execution. I don't see an essential difference with a notebook cell, other than the layout - grid vs linear. (Although I might be missing something; I don't have broad experience with Jupyter's computation model).

Plus, spreadsheets may also be distributed.

<comment><USER10> Good points.

I don't care much about the visual layout of Jupyter notebook, so in this sense, it is not much different than spreadsheets.

However, when I say Jupyter cells are nothing but messages to/from the kernel, it is profoundly different than spread sheets where cells can be message as you described. The same with distributed runtime states. Spreadsheets may also be distributed, but Jupyter notebook runtime is always distributed. You might have a half system like nbviewer that doesn't have a kernel running, but that is not a full Jupyter system. (I am distinguishing Jupyter runtime from the JSON byte stream artifact that we call a notebook file)

<comment><USER11> So, if I understand it right, Jupyter follows an agent-based model of computation, right? As opposed to spreadsheets, which follow a functional reactive paradigm.

Yes, these are different computation models; although formally it is possible to transform mathematically one into the other, and vice-versa (in the same way that you can transform any computation model to a Turing machine or lambda calculus).

It doesn't have any practical concerns, but it means that you can translate insights gained from one model to the other.

<comment><USER10> This thread is getting too deep and too narrow (visually) to read on HN. So I will summarize and stop here.

1\. It is a good observation that Jupyter runtime follows an agent-based computation model instead of the MVC of smalltalk, or functional reactive Spreadsheets. Theoretically they are equivalent in the Turing sense. But this model opens huge opportunities that allow it to mimic how human brain and human societies work. It is all about messages, context containers and just- in-time computation.

2\. This paradigm change also affects how we grow (instead of building) software architecture, which is the topic of the root of this HN submission. In the build/develop paradigm, software engineers focused on requirements, specification, foundation, frameworks, ..., i.e., the house building metaphor ([https://en.wikipedia.org/wiki/Design_Patterns](https://en.wikipedia.org/wiki/Design_Patterns) ) But if you are growing a software project, there is no architecture. Then you should use agriculture metaphors: ecosystem, environment, energy cycle, water cycle, etc. It is about seeding, weeding, environment control, harvesting and sustainability. Push it a little bit further, programmers in this paradigm are more like software farmers than data plumbers in a data warehouse or code assemblers in a factory.

It was a nice discussion in a virtual public square. I enjoyed it a lot. I hope you did as well.

<comment><USER11> I did.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> Web notebooks are the new system shell, it's just that most developers haven't realized it yet.

The original shell was an evolution over the teletype. Developers would type their code on a typewriter machine connected to the CPU over a wire (maybe involving punched tape or cards); and the CPU spouted back the results of the commands executed.

Later, when video screens were adopted, they emulated the same linear approach of the terminal on the new device, because developers tend to be an extremely conservative bunch when it comes to the approach taken by their tools of the trade. It took several decades to exploit the advantages of interactive sessions that the new media allowed, creating a new family of IDEs with live debugging, property inspectors and Intellisense/contextual help.

We are at a similar turning point with respect to classic development tools. The notebook is still used mostly as an improved version of the bash shell "with graphics", but there's a lot of research to do on how to enhance the model with new tools to take advantage of the new medium (including literate programming, web-based collaboration, or instant feedback of command execution on persistent objects).

People are exploring the possibilities of this approach, with Bret Victor possibly being the most influential. I'm sure the personal experiences of thousands of developers with their home-made workflows, involving lots of heterogeneous tools used in creative ways, could be studied to design a more general environment that will change how large software projects are built.

<comment><USER12> I think [https://datalore.io/](https://datalore.io/) is further away than Jupyter in respect to being a better shell for developers. It has interactive recomputation, collaboration built-in, integrated version control, etc.

</comment></comment><comment><USER13> I made a module to offer some more options for some of these problems recently - particularly useful when you have code that you _want_ to keep organized in a notebook for continued interactive experimentation and/or for explaining a prototype process — but you also want to reuse that code in an external script or program. Let's you call any notebook as a function. Keyword arguments can be passed and there's a simple way to receive these values within the notebook to allow the notebook to be parametrizable when invoked externally.

[https://github.com/breathe/NotebookScripter](https://github.com/breathe/NotebookScripter)

<comment><USER14> Is this similar to papermill?

[https://github.com/nteract/papermill](https://github.com/nteract/papermill)

<comment><USER13> I hadn't come across papermill before - looks quite full featured. Seems to be oriented a bit towards "scripting Notebooks" with the ability to generate output notebook files with populated output cells. I think it must have to emulate some of the logic of the Notebook ui to do that.

NotebookScripter is much more minimal and doesn't support generating/updating .ipynb files. It has just one api allowing one to treat a notebook file as a function that can be called from another python program.

I'm not entirely sure whether papermill can run the target notebook in the calling process — it looks like it spins up a Jupyter kernel and communicates with it via message passing. NotebookScripter creates an ipython interpreter context in process and execs the notebook code in process.

</comment></comment></comment><comment><USER15> > it might be a good idea to skip unit tests

barring some barriers (e.g. use caching where appropriate) I've found writing unit tests is a neat clean replacement for notebooks.

<comment><USER16> A thousand times this. Jupyter notebooks are an antipattern. They encourage bad habits and yield copy-pasted, hacked-together, unmaintainable code. The REPL is a tool for exploration, not for full-blown development.

> barring some barriers (e.g. use caching where appropriate)

Fixtures.

<comment><USER15> Yes thanks for the vocab. Notebooks are like a single mutable global namespace fixture. Definitely a regression in terms of engineering practice, but Python slow startup and reloading semantics are also barriers, and I'd like to see a nailgun (JVM startup accelerator) for Python someday.

<comment><USER16> > nailgun (JVM startup accelerator) for Python someday

It's not an exact corollary but pypy sort of fills that niche. Unfortunately, it's two major versions behind and it's not a 100% drop-in replacement.

</comment></comment></comment></comment><comment><USER17> Gradually pulling stable code out of the notebook into a tested library has worked well for me.

</comment><comment><USER9> Author here.

Whoa, thank you all for the nice comments, I didn't expect to make such a buzz here nor today. I'm glad to see the reactions - even the bad ones, it seems aligned with what I thought. Yes, notebooks are very useful for the faster coding cycle, but they become easily heavy (I'd love to see a better multiline edit and a better autocompletion in Jupiter).

Seems like I already posted my article 2 months ago, but renamed the GitHub repo since then, which may explain why someone else (jedwhite) could submit my article again: [https://news.ycombinator.com/item?id=18339703](https://news.ycombinator.com/item?id=18339703)

I didn't submit it twice to HN. Well, nice to see that in a parallel world my post did the 1st page on HN! :-)

But could this mean that my HN account is like "shadow banned" or something? Strange to see that all my own submissions on HN haven't got much attention for months. Or maybe it's just the random factor... Well, thanks!

</comment><comment><USER18> It seems weird that notebooks don't have built-in unit testing. All you'd have to do is compare the current output to the saved "known good" output and show the diffs. It could be a checkbox next to each item in the notebook.

<comment><USER19> nbval does something similar to what you want.

[https://github.com/computationalmodelling/nbval](https://github.com/computationalmodelling/nbval)

</comment></comment><comment><USER20> I argue that this misses the mark. Jupyter Notebooks is much more an evolution of Excel then it is of other development tools.

</comment><comment><USER21> TL;DR

Did not get any insights here except links to books.

What I prefer to do is to use Jupyter for all R&D tasks then `productize` modules/parts that proved to be working. When wrapping solutions as a product we go through typical engineering process: TDD, CI/CD, Docker, code review etc. In any case, we end up with having a bunch of machine instructions and there is no point in breaking best practices of engineering craft, they have been established for years. No need to invent another wheel.

</comment><comment><USER22> I no longer code anything.

I've been working remote at a big company now.

I write requirements of a function then i send it to my subcontractors on freelancer website.

While i am waiting, i play with my kids, kiss my wife and take her out for shopping.

I get back the functions, i paste them in the codebase... change a thing or two.

I am writing 3000 lines of code with this approach daily.

I used to write 200 lines of code alone.

One key thing, my passion in programming is killed entirely because always I've to work according to what plans management devise. Management changes direction, randomly kills project, changes requirements - this makes me feel like a slave who has to dance to statisfy the masters, no matter how many landmines they place on the dance floor.

So, now i just create outline and let cheap labor fill the blanks.

My value proposition isn't code but because i can understand the problems at hand and create plans to tackle the problem.

Choice of problem isn't in my hand tho area of problem is definitely in my hand.

I am never going back to open office plan, sebum/ear wax filled noise cancelling headphones.

<comment><USER23> Are you an employee or a contractor? If the former, are you not violating IP by subcontracting?

<comment><USER24> one price, one person All-tasks Contractor.. I farm out/purchase all basic graphics, use tons of open source libraries and farm out most tedious stuff. My employers know, as long as we deliver they do not care.</comment></comment></comment>


Zerg demo – Xen instance spawned for each web request - <AUTHOR> http://zerg.erlangonxen.org/ <comment><USER2> This is really neat stuff. I do not understand though why LING is not open- source already. If you are sincere in your claims for intentions of making it so, why would you wait until it is "mature" before doing so? Wouldn't it mature a lot faster if more people picked it up and tried to use it, submitted defects and even patches?

<comment><USER3> I'm afraid, only few people are familiar with internals of Erlang VM, especially, if that VM is totally different from BEAM. Thus, there's no reason to opensource it, at least yet. Though, build.erlangonxen.org is open, contains the most recent stable version, and free to use.

</comment></comment><comment><USER4> You know, the idea sure is impressive, but getting over capacity errors sure doesn't show it well :)

<comment><USER5> On the contrary I like it. They have finite resources and it answers immediately with the correct response.

<comment> St-Clock This. With typical web applications, your request would wait until something times out and cascades back to the client (webserver, app server, db, etc.). This can mean waiting for 30 seconds or more depending on your configuration.

It's not an easy problem to solve elegantly too.

<comment><USER6> It actually is quite easy if you set a finite limit on connections and show an appropriate message to those connections you're actively refusing. If you're allowing as many connections as your server/network/database can allow, it might take a bit longer to determine a failure status.

</comment></comment></comment></comment><comment><USER5> The demo about the vm+server startup time is interesting, but I am glad to know about the way they use Nginx to proxy the request twice to do the provisioning of the vm+server.

Edit: I wonder if the numbers would be similar if the example was done with HaLVM (<https://github.com/GaloisInc/HaLVM>).

<comment><USER7> Application which handles original request (called 'spawner') asks Xen to start a new server, and when Xen reports back, returns 'X-Accel-Redirect' to nginx, which then nginx serves.

</comment></comment><comment><USER8> For people who are interested but unable to get a connection, here's a saved page: [http://f.cl.ly/items/07293W450C3F0r472u1O/Zerg%20demo%20-%20...](http://f.cl.ly/items/07293W450C3F0r472u1O/Zerg%20demo%20-%20spawned%20for%20each%20web%20request.html)

I tweaked the background image URLs so they'd be absolute instead of relative, but otherwise it's unmodified.

<comment><USER3> Thanks. We used libvirt for this demo, unfortunately it sets its own limits.

</comment></comment><comment><USER9> I'm seeing 1.5 to 1.6 secs. Is that due to my crappy internet connection or is that simply what it is?

Because 1.5 secs per request, and more for somewhat more complex operations, seems long to me. As instance startup time it is impressive though! But they are designed to die after each request, right? Or is that just this example?

<comment><USER5> It is just an example to show how quickly they can provision a new vm+server. They are not advocating for it to be a good way to serve requests.

<comment><USER3> libvirt is the limit in this case...

</comment></comment></comment><comment><USER10> one thing I haven't been able to wrap my head around is why "on xen"? couldn't this be done "on qemu" as well, or even 'more betterly'?

<comment><USER3> Current implementation runs in Xen PV domain and uses Xen's hypercalls. There are no strong dependencies, though, Ling easily can be ported for other hypervisors. Xen is the priority, since it's the plaform for all major public clouds.

</comment></comment><comment><USER11> Can someone shine some light on why and how this is useful? What are some examples?

<comment><USER12> The folks at ZeroVM have a list of motivations[1] for this type of execution and hosting model. The only problem is that it's hard to tell how active the project is from their main page alone.

EDIT: Their github commit is fairly active for a solo developer[2].

[1] <http://zerovm.org/motivation/>

[2] <https://github.com/zerovm/zerovm/commits/master>

<comment><USER13> we are heads down coding and haven't got time yet to update ZeroVM website. If you want to contribute contact us now please, if you to use ZeroVM hypervisor stay tuned...

</comment></comment></comment><comment><USER14> So what's being shown off here? Clearly it's impressive that a web server VM is loaded so quickly, but what's the key? Is it a carefully configured Xen VM? Or is it the use of Erlang?

<comment><USER15> This is an OS-less web server VM, basically an modified Erlang VM to run directly on "almost" bare-metal

<comment><USER3> Actually, we didn't modify Ericsson's Erlang VM, we have written our own.

<comment><USER16> Impressive, is that what Zerg is about? where can I find more information about this?

<comment><USER17> <http://en.wikipedia.org/wiki/Zergling#Zerg> or

<http://knowyourmeme.com/memes/zerg-rush>

<comment><USER3> Zerg is just a demo that shows how elastic and scalable could be OSless VMs even over current cloud infrastructure

</comment></comment></comment></comment></comment></comment><comment><USER18> If you run Perl in CGI it creates a new VM for every single request. How is this different?

<comment><USER19> CGI creates a new _OS process_ for every request. Zerg creates a new _Xen VM_ for every request.

These are very different things.

</comment></comment><comment><USER16> It keeps saying "Over capacity", I want to know what this is about though.

<comment><USER20> About being able to quickly scale to capacity. :)

<comment><USER3> Sorry, gents, we don't have real datacenter there, just a single host, and libvirt sets its own limitations.

</comment></comment></comment><comment><USER21> zerg rush is imba (get the joke? ha... sorry)

</comment><comment><USER22> Borked.</comment>


Ask HN: How is religion affecting (if at all) the common hacker's life? - <AUTHOR>

Happy Easter to all Christians around!
I have always wondered how/if religion affects the hacker's decisions, and life in general. Religions have all kinds of "requests", like going to church every sunday, kneeling at sunset or visiting Mecca once in your life. Hackers tend to dedicate their life for long periods 100% to just coding and developing their product, thus obviously not respecting them.
Also, how about decisions and beliefs in general? The world has become very connected, allowing us to learn almost everything about anything, making us more self-aware. Are your beliefs affecting the decisions you make?
Last but not least, have you ever considered creating a startup that targets religious people? Are you working right now on such a startup ? <comment><USER2> Well, I don't do it much anymore, but in the past I've wasted innumerable hours arguing online about religion, or in debates that are irrevocably tainted by religion (like abortion, evolution/intelligent design, etc.). So that's a negative result. But I have been an atheist for decades, so there's no need to respect any religious edicts.

I've long gotten the sense that hackers have a better-than-average grasp of rationality, which means fewer of them ascribe to any major religion (though being more vaguely "spiritual", meditating, etc. isn't ruled out). There's just such a huge amount of obvious nonsense wrapped up in major religions that if you have the habit of taking things apart to see how they work, you'll eventually apply that to this _thing_ that your parents told you should be a central part of your life, and religion doesn't stand up well to analysis that doesn't assume its truth.

Hackers still usually have families, though -- sometimes families who are very dedicated to religion -- which means there are quite a lot who still go through the motions (at least when visiting home, for example) just so their Grandmother doesn't have a heart-attack because she thinks her grandchild is going to hell.

I personally just sort my way through the conflicts as patiently as possible (I have a family that's full of pretty serious Catholics), because I can't stomach faking it, and generally I'm not harassed about it, much.

For religious hackers -- I suspect their religion's requirements aren't too heavy. Generally we're more productive when we take breaks now and then, so feeling obligated to stop and think of something else for a bit is probably a good thing.

Whether they make a difference in decisions otherwise -- possibly, but probably not as you might think. I grew up in a wealthy area, and Christmas mass meant a parking lot full of expensive cars and pews full of fur coats. If they actually listened to the sermons, they'd hear they had the same chance to pass into heaven as a camel through a needle's eye, but that didn't affect how they used their money as far as I could see.

On the contrary, among people who are genuinely interested in making the world a better place, I suspect an atheist would be more likely to carefully analyze complicated situations before deciding what path might lead to better results, and test the outcomes... whereas anyone with strong religious beliefs would believe they already knew what was required. Think of how many AIDS cases could have been avoided if so many Christian missionaries weren't opposed to birth control....

I've long hoped that greater interconnectedness will lead to less belief in the supernatural globally, but that hasn't been the case so far -- I don't have the study link, but from what I've read people are more likely to seek out like-minded sites/people, to reinforce their beliefs, _not_ sample widely to find out if other approaches might be more valid.

Create a startup targeted at religious people? Not seriously. I've toyed with the idea of creating sites to explain gently & non-aggressively what atheism is and why some people choose to be non-believers, or to offer support to people trapped in highly-religious communities and families who have lost their faith -- but idea #1 would likely be fairly useless (see previous paragraph), and some sites already exist for idea #2.

<comment><USER3> You make some excellent and interesting points, but don't discount the number of smart, competent hacker/technologists who are also ardent believers in specific religions. I believe they are a significant if quiet minority.

<comment><USER2> Yup, not discounting them -- it's just been noticeable to me that the percentages of believers in the general population (where non-believers are a minority that's completely discounted and/or actively discriminated against in many places) is quite different from the ratios in hacker circles.

I also don't seek out believers to critique them; everyone goes through different paths in their life, and may or may not have reason to question any number of beliefs they've inherited or taken up along the way for whatever reason.

I'm fairly driven to dig down through almost everything, but I also have a friend who comfortably believes all kinds of crazy things with hardly a second thought, because "life is more interesting that way". Dunno what direction he'd go if stricken with a life-threatening illness (I really, really hope he wouldn't treat it with prayer circles, chelated mercury, toxin cleansers, homeopathy, etc..) but his approach _hasn't_ caused any problems I can point to, so I have no argument.

Back to hackers -- the topic of religion doesn't generally come up except possibly in random smalltalk, and never in my career has it become a point of conflict (i.e., company-mandated prayer groups or bible study or something like that). So if there's a quiet religious minority, and an equally-quiet non-religious majority, that seems like it would suit everyone just fine.

</comment></comment></comment><comment><USER4> For the same reasons that you mentioned about religious duties I have preferred to follow a simple and yet the most important religion which I call as "Humanity". It is very simple to follow, it is global and if the whole world starts to follow this same religion I am sure we can get rid of many of the problems that we face today.

<comment><USER5> I agree wholeheartedly with ankurdhama. All religions have some good things to take away: great parables on how to live a kind and just life, a sense of community and sharing, etc. However, often times these wonderful things that we should be taking away from the multitude of religions are marred by some people's ignorant, narrow views: the necessity of justifying that their views are right (hence, the only view), intolerance, a crutch to explain something that might be outside of our scope of knowledge at the moment, etc. to name a few.

I feel as though modern religion has become a paradox of useful teachings that are twisted into justifiable hypocrisy.

That is not to malign those who practice a religion while also treating their fellow man with respect and acceptance, but pointing out those who use religion as a dark alley in which they can hide their inadequacies, hate, etc.

</comment></comment><comment><USER6> It doesn't, mostly. I'm an (agnostic) atheist, and so are my parents. The rest of the family is either atheist, apatheist or just very "soft" Catholics (my grandmother, for example, doesn't believe in the afterlife!).

While I'm not religious, I mostly like religion as it's practiced here in Portugal and Spain - people seem to focus mostly on what I consider the good parts (community reunions, solidarity, processions) and less on the enforcement of Christians morals.

</comment><comment><USER7> Religion is a major, and painful, component of my life. I feel as though the longer I live in America, the further I am dragged into the muck of being a "world citizen", an atheistic, "rational" capitalist slave.

So we'll see what else I say when it's not Pesach.

<comment><USER2> It can get rough, or even very rough in some parts of the US (let alone some other parts of the world... a few months ago I was following the case of a Saudi Arabian who posted a few _tweets_ expressing atheist views; he had to flee the country, but Malaysia actually deported him back to SA to face a death sentence).

No need to be a capitalist slave, though. A life spent chasing gods or dollars isn't generally a life well-spent in either case; but of course there are more options.

<comment><USER7> No, no, you're not getting it. I _want_ to be religious. I _like_ being religious. What I don't like is living in a culture that is only conducive to capitalism, _nothing_ else. Even the dominant forms of religion in America are capitalist.</comment></comment></comment>


Ask HN: What are the quick sales ideas for an eCommerce store in 2 weeks? - <AUTHOR>

I have just started working for an eCommerce startup and they want to get quick sales for 2 weeks. They have OK budget, so I want to know what are the options?
I have suggested two.
1) Facebook Advertising. 2) BuySellAds Banner ads on relevant websites.
Can you help me out? <comment><USER2> Do they have an email list? If so, then emailing existing customers with a time-bounded promotion is the single easiest way to juice sales numbers. If not, remember to tell them "If we start building an email list now, you will not need me to pull a rabbit out of the hat in Q4 2015."

Otherwise: where are your prospective customers presently in their purchasing cycle? The channel you use depends on where they are.

Also, smack your employers and tell them that there is latency involved in doing e.g. creative development and negotiating private placements (which are, n.b., one thing you really want to be doing), and that two weeks before the planned sale is _way the heck too late_ to use many appropriate techniques. There is a reason why traditional retailers start going into war footing for Christmas _in early summer_.

<comment><USER3> Sounds like great advice. But what do you mean with "private placements" in this case?

<comment><USER2> "Hey owner of SomeSite. I love your website. In particular $PROVE_YOU_ARE_HUMAN_HERE. We at $COMPANY sell $FOO and are interested in getting in front of your audience. We'd like to place an ad on your site, and will pay monthly for this. I am envisioning something similar to the attached mockup. [What would you charge for this, on a monthly basis? / Would you be willing to do this ad placement for $X00 a month?]"

<comment><USER3> Ah, I see. Great e-mail template!

</comment></comment></comment></comment><comment><USER4> My top 5 are:

1\. Advertise on Google (adwords)

2\. Advertise on Bing (which is the same thing as advertising on Yahoo at this point)

3\. Advertise on Facebook

4\. Advertise on Twitter

5\. Open an affiliate program (CJ, Shareasale, etc)

While that's happening: Run split tests and optimize every part of your conversion funnel. Up those conversion rates, lower those bounce rates and abandoned shopping cart rates. Every little bit helps.

</comment><comment><USER5> What niche is the store in? Holiday/seasonal store?

<comment><AUTHOR> It's DIY designer leather handbags and clutches store!</comment></comment>


Wolfram Alpha will teach the masses about precision and recall. - <AUTHOR>

Google tends to work so well on most queries that people don't think about such issues.
But with Wolfram Alpha, the issues of precision and recall will become important issues discussed by everyone.
And there will be a huge interest in tracking precision and recall over time in various domains as Wolfram Alpha improves.
There's probably a startup opportunity here. <comment><USER2> What do you mean by 'the issues of precision and recall' ?

<comment><USER3> One imagines:

Precision = correct results / total results (E.g., what fraction of results are correct.)

Recall = total results / total queries. (E.g., what fraction of queries are answered.)</comment></comment>


Ask HN: What is the 'hacker news' for politics?  - <AUTHOR>

I was reading a series of nonsensical, ranting comments on blogs like politico, sites like NYT, and WSJ, guardian, etc... and it just struck me - why is there no "hacker news" for politics - good debate, differing opinions, a general attitude of respect for differences. I'm most interested in US and UK politics but as I live in South America would appreciate a site that covers other countries. And if it doesn't exist, is it a startup idea worth building? <comment><USER2> There's a couple key issues with it: current comment voting techniques don't do a good job of differentiating validity from user sentiment (how many times have you had someone need to rebuke the community for using the arrows to denote disagreement?)

Secondly, you will always to deal with trolls. If you have mechanisms in place to make people spend less time responding to trolls, you'll retain the sanity of the sane users and diminish incentives from trolls.

Thirdly, you have to rethink the way stories are voted on.

Thankfully, we're building that. If you sign up on forrus.org, I'll let you know when we roll these features out which should be in a month. We're actively just inviting people who are enthusiastic about these ideas because we want to grow the community slowly but with lots of quality.

</comment><comment><USER3> Political discussions tend to derail into ideological rants far more often than technical matters. They also tend to turn into echo chambers, attracting like-minded individuals who circle-jerk their own opinions.

That said, I'm a fan of Debate and Discussion subforum of SomethingAwful, which is remarkably intelligent and reasonable (although admittedly slanting heavily towards liberals and libertarians).

<comment><USER4> That's where the comment voting would help, but I guess it's still not a perfect system. I really just want a place where I can have political discussions/debates with people who form their opinions based on facts instead of being strictly democrat/republican because that's the way their parents brought them up.

</comment></comment><comment><USER5> It doesn't make sense for the same reasons that a 'Hacker news' for religion doesn't. People attracted in such topics are usually too invested emotionally in their personal beliefs to entertain the idea they might be wrong.

<comment><USER6> This is why any successful political community needs to be largely made up of people with the same beliefs, just like almost everyone at Hacker News believes the same thing about startups and are too emotionally invested to change their beliefs.

</comment></comment><comment><USER4> My brother and I had this exact idea and started to build it, but we've both been too busy with our normal jobs to be able to do anything with it. We even have a pretty decent domain name (I think).

</comment><comment><USER7> Many of the comments on online Economist articles are excellent.

</comment><comment><USER8> createdebate.com is the closest thing I've seen. But it isn't that good. I think a good political debate site is definitely worth building.

</comment><comment><USER9> reddit.com/r/politics

<comment><USER10> That is not a good site for politics, unless you like the echo chamber. r/politics is dominated by one group, another group has their own subreddit etc.</comment></comment>


Gmail emails gone missing - <AUTHOR> http://uladzislau.com/gmail-emails-missing/ <comment><USER2> I've noticed recently my Gmail spam folder is 90% legitimate email (stuff I should Unsubscribe from, but not viagra ads, Canadian pharmacy, or stock tips). That leads me to believe most of it is getting filtered before it even reaches my spam folder.</comment>


Exploring Flutter for Cross-Platform Mobile Development - <AUTHOR> https://sethlopez.me/article/exploring-flutter-for-cross-platform-mobile-development/ <comment><USER2> Definitely interested in seeing how Flutter progresses, IMO it has the optimal architecture for developing native, x-plat iOS/Android UI ([https://flutter.io/technical-overview/](https://flutter.io/technical- overview/)):

       - High perf Skia based UI for pixel perfect rendering and high-level Material design widgets
       - React inspired, productive development model
       - Fast dev/iteration cycles with hot reloading
       - Productive and high-performance Dart language, natively compiled (AOT on iOS)
       - Enable native interop with underlying iOS/Android APIs
       - Actively developed by Google

Overall I think it has a superior architecture to React Native where it will enable higher-perf native iOS/Android Apps in a single code-base but still enables a productive development model with Instant UI updates and hot reloading. I've done Java and Kotlin Android Apps as well as Obj-C and Swift iOS Apps but I find React Native's dev model a lot more productive.

Unfortunately I've run into a few issues with React Native that I've had to workaround which I've submitted repros to months ago but received no response from the React Native team except in the last couple of days where they've closed it without even looking at it because it didn't receive comments/activity from other devs. In the last 2 days React Native has closed 773 other issues because they consider it low priority:

[https://github.com/facebook/react- native/issues?q=label%3AIc...](https://github.com/facebook/react- native/issues?q=label%3AIcebox+is%3Aclosed)

This gives me low confidence that React Native will be a high quality platform with current low-priority issues lingering indefinitely so I welcome competition from Google with Flutter and will be anxiously looking forward to trying it out when it gets out of alpha.

<comment><USER3> I have very low hopes for React Native too.

It is sad but the name Facebook on any engineering project makes me think twice before considering adopting it.

Flutter looks like it could be really interesting framework.

Especially since it should allow to write apps for iOS, Android AND Fuchsia.

I am quite happy with kotlin on Android & Swift on iOS (and their respective native frameworks) but if Google publishes Fuchsia as Android's replacement with Flutter as its app framework, it could be an interesting switch :

Rewrite your app in Fuchsia. It is going to take a while of course BUT you are going to be able to publish the new on both Android & Fuchsia (in order to still support 'legacy' for a while)

<comment><USER4> Ahem, GWT anyone?

<comment><USER5> GWT had its purpose. You need to remember that it was released before jquery was a thing. And also to realize it was the first (was it?) framework to push the large single page apps that worked in all browsers. It was created so Google could make the AdWords and Adsense UIs.

Today's AdWords UI uses AngularDart. So they've moved away from GWT. But 11 years ago the JavaScript landscape was very different.

[I work for Google, but I was doing GWT dev for 4 years before joining]

</comment></comment></comment></comment><comment><USER6> I had high hopes for flutter but gave up after a while. It feels like their design meetings were spent arguing about grammar and splitting hairs, rather than thinking about ergonomics and how people would use it.

In Flutter, everything is a nested pile of objects with too many APIs to keep track of. Take this example: [https://github.com/flutter/flutter/blob/master/examples/stoc...](https://github.com/flutter/flutter/blob/master/examples/stocks/lib/stock_row.dart#L41-L56)

Why do I need to care if something takes a `child: (single object)` argument or a `children: [LIST of objects]`?

Flutter would be better with JSX: JSX hides how the puzzle pieces fit together. I don't care if it takes a child or children; just make everything connect the same way.

React Native's Flexbox also beats how Flutter did things. Why do I need to memorize which objects take which styling arguments? You want to center items on the screen? Re-nest everything inside a Center object! You want a column or a row of elements? Use a Column/Row object!

For a framework that's trying to bill itself as a great tool for prototyping, it feels like I'm sifting through a mountain of minutiae. I was able to guess my way through a React Native app and be right 99% of the time. With flutter, my luckiest guess would lead me to an abstract base class... Then I'd have to dig around to figure out what the hell I need to use to make a view scrollable. Seriously:

[https://github.com/flutter/flutter/blob/c6b0f833af9e431df1e6...](https://github.com/flutter/flutter/blob/c6b0f833af9e431df1e67f15e8b51a76e8bc7d71/packages/flutter/lib/src/widgets/scroll_view.dart#L42)

Why?

<comment><USER7> > _Why do I need to care if something takes a `child: (single object)` > argument or a `children: [LIST of objects]`?_

The IDE tells you what's needed and the latest version of the plugin automatically adds a list literal and places the cursor inside it:

[https://groups.google.com/forum/#!topic/flutter- dev/LXafJQqS...](https://groups.google.com/forum/#!topic/flutter- dev/LXafJQqSK4M)

Personally, I think it makes sense to differentiate between exactly one and zero or more.

> _Why do I need to memorize which objects take which styling arguments?_

The IDE should do that for you.

<comment><USER8> >The IDE should do that for you.

Fail. IDE dependence is an anti-pattern and a programmer-smell.

<comment><USER7> Either the IDE/editor is able to assist you or you have to consult the documentation regularly.

That's just how it is.

JS' standard library, for example, is really tiny but I haven't memorized all of it. If the editor can't clue me in, I have to check the docs.

<comment><USER8> Nothing wrong with doing any of it, as long as its a dependency you can manage while also maintaining productivity... but such is the life of a Javascript developer.

</comment></comment></comment></comment></comment><comment><USER9> Nice write-up. I've been popping my head on over to their project every few months to check out their progress. Something that's pretty cool is that they're using Flutter to write Fuchsia's UI [0].

My favorite thing about Flutter is that it looks like they took some heavy inspiration from React. If anyone reading this isn't familiarized with React, or they don't really "get it", I'd highly suggest reading the Removing User Interface Complexity, or Why React is Awesome [1].

One of the big problems with implementing a UI toolkit is having to re- implement everything relating to accessibility. Although it's totally understandable that they're still focusing on the core.

Something I'd be interested in seeing is how Dart and Flutter might affect battery life. I'd expect the stock UIs to be pretty well optimized by now, but I have no idea how Dart stacks up in performance.

[0] [https://github.com/fuchsia-mirror/sysui](https://github.com/fuchsia- mirror/sysui)

[1] [http://jlongster.com/Removing-User-Interface- Complexity,-or-...](http://jlongster.com/Removing-User-Interface- Complexity,-or-Why-React-is-Awesome)

<comment><USER10> [disclaimer: I work on the Flutter project]

One of the reasons we picked Dart was that is compiles to ARM (native) code. Dart has been pretty efficient for the project so far, but we haven't yet looked very deeply at battery life. Something to keep an eye on!

<comment><USER11> Do you use GC or RC on iOS?

<comment><USER12> GC

</comment></comment></comment></comment><comment><USER13> _> Though Flutter doesn't use the native UI widgets, it's a cross-platform framework and has widgets for both Android and iOS. [...] On iOS, the biggest thing that stuck out to me was the scrolling. It didn't quite feel right because of the physics, but I'm sure that would just require a little tweaking by the Flutter team._

Count me a skeptic. This is the same approach taken by Java's Swing (now JavaFX) toolkit and apparently it has exactly the same issues. Swing never felt quite right even after decades of tweaking.

<comment><USER14> AFAIK, Qt also uses the same approach (it does not use native widgets), yet it does not get the same bad reviews like Swing & JavaFX.

On the opposite, I think people generally like it.

I wonder why?

<comment><USER13> Perhaps because Qt occupies a couple of niches that don't have a particularly entrenched native look and feel or are not very widely used.

Home entertainment systems, in-vehicle entertainment systems, kiosks, medical devices, avionics, Linux desktop apps, printer displays, navigation systems, industrial equipment controls, these are the sort of things that Qt is mostly used for.

Do you use a Qt desktop or mobile app on a day to day basis? The only Qt app I have ever used is QtCreator. It's a good development environment, but the look and feel is definitely a bit weird.

<comment> Const-me Microsoft also did that in WPF.

For Windows PCs, it was already a widely used native look and feel. Also wide selection of native Win32 controls, mostly implemented in ComCtl32.dll.

Yet in WPF I can easily replicate native look and feels, if I want so.

<comment><USER15> That's because WPF like winforms used native windows controls under the hood. Calls would be marshaled via pinvoke.

<comment> Const-me Simplest way to verify, launch visual studio 2015, press Tools/Spy++, in Spy++ find the visual studio's main window, you'll see that Win32 thinks visual studio is a single window with no children, with class starting with "HwndWrapper[DefaultDomain;"

Unlike that, Spy++ itself uses Win32 (wrapped by MFC but it doesn't matter) for GUI, you'll see these SysTreeView32, msctls_statusbar32 and other native controls there.

<comment><USER15> I stand corrected - thank you. I don't have a windows machine anymore but as you said spy++ can tell you all you need to know. Hilarious to still see hungarian (hwnd - handle to a window).

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> Nice article. The lack of native iOS looking components would be a deal breaker for me.

I wonder how stuff like navigation is built? If that's all in dart I'd be interested in seeing how the back stack looks in the hierarchy explorer. I.e are precious screens rendered still.

<comment><USER10> [disclaimer: I work on the Flutter project]

We're adding more Cupertino (iOS-flavor) widgets. A few that we already have: spinner, toggle slider, button, alert. Check out the library ([https://docs.flutter.io/flutter/cupertino/cupertino- library....](https://docs.flutter.io/flutter/cupertino/cupertino- library.html)) and let us know which ones you'd like to see next! [https://flutter.io/issues](https://flutter.io/issues)

Thanks!

<comment><USER17> Will your custom widgets work with all iOS accessibility features?

Accessibility is often forgotten when something gets reimplemented.

<comment><USER12> The goal is to work with all iOS accessibility features. You can see our open accessibility issues here:

[https://github.com/flutter/flutter/issues?q=is%3Aopen+is%3Ai...](https://github.com/flutter/flutter/issues?q=is%3Aopen+is%3Aissue+label%3A%22prod%3A+accessibility%22)

Notice that there are several open issues relating specifically to iOS accessibility features.

Accessibility is not complete yet, but it is definitely not forgotten.

</comment></comment></comment></comment><comment><USER18> > Though Flutter doesn't use the native UI widgets, it's a cross-platform > framework and has widgets for both Android and iOS

Hot-reloading and good performance are very attractive parts of Flutter, but they really should have reconsidered the decision to make their own UI widgets. When you use the native UI elements, you get that native look-and- feel for free, and you don't have to dump man hours into replicating that behavior. They could have a "UI backend" which calls out to the native UI elements for each platform. The great thing is that since they use these UI widgets natively on Fuchsia, they can use their existing code as just another backend on that platform without having to throw the work away.

</comment><comment><USER11> That chat app demo they coded at the I/O 2017 presentation in under 400 LOC was pretty ridiculous (in a good way) considering all of the functionality it included. The demo code was run on an iOS and Android device and each used its respective look and feel.

</comment><comment> Const-me Awesome stuff. And the license appears to be OK. The only thing that worries me is their language choice, for some people that would be too much new stuff in a single project i.e. too risky.

Finally, people outside Microsoft started to realize modern GPUs and high- resolution screens need a purposely-built GUI framework, designed for GPU rendering from the ground up. Especially on mobile platforms. MS did that in 2006 with the first release of WPF, and with WinRT/XAML they continue to pursue the approach.

</comment><comment><USER19> This may be off-topic but I'm wondering if there is any chance to see cross platform and cross language toolkits any time soon. Something based on WebAssembly so that we don't have to worry or bash about programming languages. Dart is cool but should we all learn Dart now?

<comment> grey-area Here is what developers want:

        Platform independence
        Choice of tools/language
        Control of customer data
        Hardware independence

Here is what corporate platform owners (Google, Apple, Microsoft, Amazon etc) want:

        Platform lockin
        Minimal support costs (one language, one SDK)
        Control of customer data
        Control of the ecosystem
        Hardware independence
        A vig on every transaction

There are very few intersections between the interests of developers and the interests of platform owners (be that Apple with iOS or MacOS, Google with Android, or MS with Windows). This is why the web will win IMO, it's one of the few platforms focussed on what customers and developers want, and not owned by a single corporation (though Google has come close).

<comment><USER20> > focussed on what customers and developers want

I would hardly agree with the modern web being focused on what "customers" want. How many web pages today hijack scrolling, fill the screen with ads that lie on top of the content and don't scroll with the page, and so on? How much bloat is there? How many MB of unnecessary JS frameworks must be downloaded, often on a metered data plan, to support all of that? The web is a horrible mess these days. It may be a rip-roarin' good time for (some) developers, but it's absolutely awful for users.

<comment> grey-area Compared to app platforms, I think it is more open and more focussed on readers and devs. The problems you list are annoyances of some websites, not even the majority. For example they don't apply to the site you are currently on not hard limitations.

In contrast iOS forbids other payment systems (see deliberately limited kindle app) and browser engines. Android forces manufacturers to prominently feature lots of google products.

</comment></comment></comment></comment><comment><USER21> How does flutter compare to weex? (Native VueJS)

</comment><comment><USER22> From a quick look at the sample code and a scrub through the video, it sounds a _lot_ like JavaFX, especially when combined with the Kotlin TornadoFX extensions.

Relatively recent build of a custom widget toolkit? Check.

GPU accelerated 60fps compositing, render and animation layer? Check.

IntelliJ as the IDE? Certainly. Although JFX is perfectly usable from any Java IDE?

Cross platform including across iOS and Android? Check. (except JFX also does Mac/Win/Linux).

Fairly standard layout and box packing model. Check.

Material design? Via a third party company called Gluon, check.

Functional reactive design? Check.

Async/await? Yes, Kotlin has coroutines that integrate with JavaFX.

Hot reload? Actually yes, JVM+TornadoFX has some support for this, although it's not as slick as what Flutter can do ... the view you're working on will be sometimes be thrown out and rebuilt. But if you change your CSS or the behaviour of e.g. event handlers, then you can do hot reload.

There are some differences I see. Dart focuses more on AOT compilation. Another is immutable widgets in Flutter. That one I'm not convinced about at all.

> You can respond to events, like user interaction, by telling the framework > to replace a widget in the hierarchy with another widget. The framework then > compares the new and old widgets and efficiently updates the user interface.

No explanation for this odd design is provided. I suppose they believe it is self-evidently superior, but that seems like a lot of overhead for very little to me. After all, GUI is pretty much THE standard definition of a big pile of mutable state, and trying to pretend its not by just generating more garbage than a normal GCd toolkit would seems a little strange.

<comment><USER23> Flutter has stateful widgets too, and they can be arbitrarily nested with stateless widgets.

Stateless widgets are useful for static boilerplate that only changes when moving to a different screen, or when swapping out an entire subview. Supporting arbitrary mutations is extra coding that's unnecessary when it's not going to change anyway. Why implement it when you don't have to?

Note that even with entirely stateful widgets, if the whole layout changes, the whole subtree generally to be thrown out which also creates garbage.

</comment></comment><comment><USER24> I also played with Flutter recently. Timeline: heard Kotlin announcement at I/O, went through some Kotlin exercises, wanted to compare with Flutter/Dart.

Dart feels somewhere in between JavaScript on the one hand and a more-static OO, GC'd lang w/type inference (like Kotlin!) on the other. You can probably look closely at some sample app code and start doing some basic stuff quickly if you've worked much with JS and something more static. (Further study seems worth it if you plan to do a lot, heh!)

As others note the UI model seems Reacty--you write "builder" methods that recreate a widget tree when things change, and something behind the scenes sorts out an efficient way to update the screen with just what really changed. I'm not _hugely_ worried about performance: your UI rebuilds should be separated from your animations, and anyway, building your virtual widget hierarchy ideally shouldn't be too CPU intensive in the first place.

Hot reload is pretty great. I can't actually compare with "real Android" dev, but changes to my little app showed up in under a second in an Android phone, emulated or real. There were a couple surprising things about the basic libs, e.g. Flutter master only recently added a convenience object to bundle together a radio/checkbox and its associated label-stuff (RadioListTile).

The Flutter Gallery app is available on Google Play and its source is in the Flutter git tree. You can see a lot of the Material widgets implemented, including rich list types (e.g. tiles w/photos), pull-from-the-side drawers, top-of-screen tabs you can swipe through, bottom-of-the-screen nav bars etc. Even on iOS Google seems to follow Material guidelines a lot (or at least, the Daring Fireball guy complained that they do; I don't have iOS to check), so maybe it's the easiest fit if you're prepared to do the same. Someone who works on Flutter mentioned elsewhere in these comments that they're working on components that look more like the iOS-native ones, though.

Although Android Studio is _based on_ IntelliJ, you need to get actual IntelliJ if you want to use the plugin (Studio's component versions don't match the ones that the Flutter plugin works with, I think). Also, if you have Studio 3.0 canary installed (like to futz w/Kotlin, heh!), you need to either configure Flutter to look for the stable Studio 2.3's copy of Gradle (flutter config --gradle-dir=...) or just make sure 2.3 is located where the flutter tools look by default (~/android-studio for me on Linux). People working on Flutter helped some of us through this at [https://github.com/flutter/flutter/issues/10236#issuecomment...](https://github.com/flutter/flutter/issues/10236#issuecomment-303029027)

You get a lot of IDE-ish luxuries (as OP notes): Control-Space to offer identifiers, methods, or params available; autoformatting with dartfmt (right- click menu); lots of quick feedback when you mess something up.

Hixie (Ian Hickson) of the HTML5 spec works on Flutter which is kinda neat (he did RadioListTile just now! and there's a milestone on GitHub named 'Make Hixie Proud' haha :D). Outside of the tech specifics, Dart's an interesting creature in that it seems like it's got some key customers in Google (AdWords, so, like, the part that makes money) but comparatively little pickup outside. On Flutter GitHub you see people paying attention to outside-adopter issues (or even passerby issues such as my Gradle-version thing recently). There's apparently lots of tooling available publicly, e.g. a package manager (pub), dartfmt, IDE plugins, a playground (dartpad.dartlang.org) etc. Curious to see if there's any more pickup on the outside.

<comment><USER23> You can also use the Flutter plugin with the free, community edition of IntelliJ or with Webstorm. (But WebStorm does not have any Java or Android support.)

Also, the Dart plugin for VS Code is starting to add Flutter support. In that case you'd use the command line for hot reload.

\- a Googler working on Flutter plugin.

<comment><USER24> Yes--I should have mentioned I was using free IntelliJ!

Also great re: VS Code!

</comment></comment></comment><comment><USER25> Cross-platform development has been the holy grail for some 30+ years.

Alas, I stopped reading the moment I read it is being developed by Google despite the open-source nature.

Being developed by Google it means the chances of a project being discontinued are quite high: [https://en.wikipedia.org/wiki/Category:Discontinued_Google_s...](https://en.wikipedia.org/wiki/Category:Discontinued_Google_services)

While other big corporate entities have their own share of shut projects (no project is forever) the moment this project stops getting official Google support it will wither and die.

Maybe I am reading this wrong and we could have a Open Office/Libre Office situation.

<comment><USER11> >Being developed by Google it means the chances of a project being discontinued are quite high

Do you also avoid Microsoft products and services because of the chances of a project being discontinued are quite high?

[https://en.wikipedia.org/wiki/Category:Discontinued_Microsof...](https://en.wikipedia.org/wiki/Category:Discontinued_Microsoft_software)

You can also add these to the list:

      Wunderlist
      Sunset
      Games for Windows
      Silverlight
      Zune
      Kin
      XNA
      PlaysForSure
      Flight Sim
      Expression Suite
      SteadyState
      Windows RT
      Windows Phone 7
      Forefront
      Front Page
      Money

Or how about Apple?

[https://en.wikipedia.org/wiki/List_of_products_discontinued_...](https://en.wikipedia.org/wiki/List_of_products_discontinued_by_Apple_Inc).

<comment><USER25> I did say that other corporate entities discontinue products all the time.

I suppose the difference with Google products is that they usually seem "cool" and very promising.

With Microsoft I have no such expectations.

</comment></comment></comment><comment><USER3> > If there were dropped frames anywhere, I sure didn't see them

I would be curious to know what terminal the author uses. On my N6P, I can see a lot of dropped frames in the flutter demo app.

<comment><USER26> Author here. I have a Nexus 6. I hadn't tried the Google Play app until today so my observations were based on the sample app that I had built and another Flutter app that's in the play store called Newsvoice. I was under the impression that the Flutter sample app was just a github repo, and I didn't go through the trouble of trying it out.

The Flutter app in the Google Play store does lag for me during the first run of most large animations (nav drawer open/close, shared transitions, etc.). The second time those animations are run, everything is smooth. There must be some sort of caching in place for animations.

As far as I could tell, everything else seemed to be fine. Scrolling performance was good, as were smaller animations like checkboxes and switches.

</comment></comment><comment><USER27> This feels like a bad idea. Let's write a platform on Dart which is esoteric in terms of usage and community and hope for the best? Seriously? The whole reason to do a framework is to make it big so it has a tons of community support. I just don't understand the point although it looks elegant.

</comment><comment><USER28> Codename One ([https://www.codenameone.com/](https://www.codenameone.com/)) runs circles around fluttr.

<comment><USER29> > 180M DEPLOYED APPS

This is quite a claim...

Edit: It's customary to include a disclaimer you work for the product your spruiking FYI.

<comment><USER30> They must be counting every single test app as being "deployed".

<comment><USER28> It's installed apps on devices == deployed. That's a reasonable number at those scales.</comment></comment></comment></comment>


End-to-end encryption - <AUTHOR> https://blog.whatsapp.com/10000618/end-to-end-encryption <comment><USER2> Not a fan of WhatsApp but I think OWS has done an excellent job of actually affecting the real world by helping WhatsApp sort their stuff out, rather than trying to get the world to move to OWS Signal app directly.</comment>


Salt Stack - fast remote execution manager / provisioner - <AUTHOR> http://saltstack.org/

<comment><USER2> I want to like this but having to use a template syestem to build a yaml file seems entirely backwards to me.

Ref: [http://salt.readthedocs.org/en/latest/topics/tutorials/state...](http://salt.readthedocs.org/en/latest/topics/tutorials/states_pt3.html)

<comment><USER3> Templating is definitely not required - states are run through Jinja by default, but for most simple setups you won't need to template anything.

</comment></comment><comment><AUTHOR> After using Puppet and Chef for small production deployments and local vagrant setups, salt looks promising in terms of performance and transparency.</comment>


These tiny bubbles could save the planet - <AUTHOR> http://www.bloomberg.com/news/articles/2015-03-16/these-tiny-bubbles-may-save-the-planet

<comment><USER2> Because that crazy huge amount of gas would be good in sealed containers / underground? This is not a solution, this is a delay.

<comment><AUTHOR> It was underground already until we dug it up.

<comment><USER2> But most of it was not gas.

And large co2 pockets are dangerous.</comment></comment></comment>


Thank You for Inviting Me to Build Your App in Exchange for Equity,Kind Stranger - <AUTHOR> https://hackernoon.com/thank-you-for-inviting-me-to-build-your-app-in-exchange-for-equity-kind-stranger-a3ec1644cb0a <comment><USER2> > but I know your idea is going to be much, much better than the > transformative data science tool I'm building internally at the ol' Goog.

Maybe that tool is not that important too, but I understand that it beats working for free against the odds of eventually turning equity into cash :-)

</comment><comment><USER3> I don't think that sounded like a good plan, but good luck I hope it works out.</comment>


World's Most Powerful Man? Meet David Beers from S&P... - <AUTHOR> http://chaselanc.com/post/8672903978 It turns out the world's most powerful man is not a world leader, not an elected official and not a billionaire businessman.  If you want to find out who can crash international markets and provoke the loss of trillions of dollars by erasing one letter (A), look no further than David Beers, Global Head of Sovereign Ratings at S&P.  The aftermath of S&P's downgrade to the U.S. credit rating was scary.  On Monday, the S&P 500 and Dow Jones Industrial Average declined 6.7% and 5.6%, respectively.  All stocks in the S&P 500 retreated for the first time since at least 1996 – and the index's 10 main groups fell more than 5.3%.  So who is this David Beers guy?  While most have never heard his name before, he is well known by virtually every finance minister in the world.  He is a Wall Street veteran and graduated from the London School of Economics.  Beyond that, David Beers background and biography is a mystery to most.<p>But what's more of a mystery to most people is the power and relevance of the S&P, or more broadly, the collective group of rating agencies.  It's difficult to fathom how these companies have the authority to shake the global economy like this.  Their role is very similar to a credit bureau that hands out scores to households and individuals seeking loans.   Their reputation for assessing risk has been shaky on more than one occasion.  Could this be a new blackeye for S&P?  To be certain, it's going to be tough for Beers & Co. to adequately justify their recent downgrade of the U.S.  Much of the rating report submitted by S&P focused on the political dysfunction of the U.S. government as opposed to its true ability to repay its debt.  From S&P's perspective, the question really isn't whether the U.S. can repay its debts, it's whether the U.S. is willing to repay.  The political logjam from the recent debt crisis highlighted the fact that the U.S. certainly has the ability to make payments but there are many politicians who are unwilling to write the check.  And this is what the S&P is most troubled by.  While Warren Buffett thinks the U.S. deserves a "quadruple A" rating, the S&P is only halfway there, and that is making all the difference in the markets today. <comment><USER2> That's what you see on the surface; what you don't see is the family that owns more than half of the world's wealth: <http://en.wikipedia.org/wiki/Rothschild_family>. They basically run everything in the background for a pair of centuries. It's easy to doubt this, but google this.

<comment><USER3> While interesting to read about wealthy family similar to Swedens Wallenberg Family (<http://en.wikipedia.org/wiki/Wallenberg_family>) that also has similar claims about having sway over 1/3 of Swedens GDP (seems unlikely). Your link say nothing about them owning more than half of the world's, except for the conspiracy bit at the bottom. The conspiracy section is for what people believe about the family and not about the family itself.

<comment><USER2> Wallenberg didn't finance two parts of a war, and are not into federal reserve banking is similar fashion - not even comparable. Read up!</comment></comment></comment>


Show HN: My first painting app - <AUTHOR> https://itunes.apple.com/us/app/craytin-painting-sketching/id900605668?mt=8

<comment><USER2> I see that this is very similar to Paper by 53 in several aspects... Is that what you are trying to compete with? In that case, I wonder the logic of an upfront price instead of Paper's strategy which is to give you something to try out... At $4.99 a pop, it is much harder for someone to take the plunge.

<comment><AUTHOR> I do take your point. In fact I'm in the process of writing a small update with In App Purchases. With this update, it'll be free.</comment></comment>


Why Russian Government Should Forget about the Blockchain Technology - <AUTHOR> http://forklog.net/why-russian-government-should-forget-about-the-blockchain-technology/ <comment><USER2> So Russia is not ready, and debates like the blocksize limit debate would favour the powerful.

Well it's not a very advanced technology, it's really simple and off the shelf, and as far as I could see the blocksize debate did favour the powerful players anyway - the 1mb block was supported by the few big miners which is why it is staying that way stubbornly for so long.

On the plus side we have completely transparency in financial transactions, as well as having a network of trust which is an indelible record, automated trust if you will.

Some of the advantages touted:

"Basing on what I heard, blockchain means the following to me: any person may enter a database, extract all required documents to buy a land plot, form the package in an hour's time, send it to relevant entities, and once it is confirmed, get an answer in an hour. This is certainly a revolution."

"The same thing is taxation. Any taxpayer will be able to trace their particular payments. Generally, they say that taxation system becomes personalized, that you'll be able to pay taxes for particular needs, and trace what your payment is spent on. It is a completely different concept. We will have to realize many things, we will have to completely alter our way of thinking."</comment>


The next big space telescope could spot Earth-like oxygen levels on exoplanets - <AUTHOR> https://www.technologyreview.com/s/615011/a-telescope-in-the-works-has-a-new-way-to-spot-oxygen-on-exoplanets/ <comment><USER2> I really hope the launch and deployment of JWST goes smoothly. It's costly, being in development for long time, and almost no way to fix(L2 is far away) if something goes wrong.

<comment><AUTHOR> Yes, such unique projects are really a case of make or break! To be fair, the rate of success in the past is high enough to make us confident they will succeed again... barring unforeseen or unpredictable circumstances in the mechanical start up?</comment></comment>


Chicken Breeds - <AUTHOR> https://funpoultry.com/chicken-breed/large-fowl/croad-langshan-chicken/ <comment><AUTHOR> croad langshan black asian breeds</comment>


Show HN: WiFi Time Tracker -automatic time tracking app based on WiFi connection - <AUTHOR> https://play.google.com/store/apps/details?id=com.igor.wifi_time_tracker <comment><AUTHOR> Hi, developer here.

I developed a simple time tracking app that may help you analyze and visualize how much time exactly do you spend e.g. at work each day, or on the commute to work or any other places where you connect to the wifi often.

I created it so I could compare how much time I spend on different routes to work every morning, so I could find the best one and needed something with a good precision but couldn't find anything on the play store already.

The app is implemented using the Flutter framework and considering it is my first app ever, it was quite fun to implement it (especially considering no graphical skills whatsoever).

It's ads-free as I don't plan on monetizing it.</comment>


Show HN: Worldwide battle of virtual missile attack and evasion - <AUTHOR> http://www.friendswithguns.com

<comment><USER2> Why doesn't it work on my IPad 2? Don't I have geolocation API on here?</comment>


Postgres-XC, read and write scalability in Postgres - <AUTHOR> http://postgres-xc.sourceforge.net/

<comment><AUTHOR> I've submitted this in part just to see if any on here has used this and has any experience that they can share.

Particularly what the observed overhead with real-world use, and whether you encountered any gotchas when rolling it out that you wish you knew of in advance.</comment>