How fast can we make interpreted Python? - <AUTHOR> http://www.phi-node.com/2013/06/how-fast-can-we-make-interpreted-python.html <comment><USER2> There are a couple of things you want to do (some of which overlap with the article):

1) Use a register-based VM (with a sliding and growing register file) instead of a stack-based VM. In theory you can make a stack-based VM fast with lots of macroinstructions that fuse smaller operations together, but it isn't worth it.

2) Use inline caching for method calls, property accesses, and primitive operations that do type checks. In an interpreter you can modify the instruction stream even on platforms that disallow modification of executable code. I know this isn't the origin of the technique in bytecode interpreters, but here's a paper describing it in case it's not obvious:

[http://www.lirmm.fr/~ducour/Doc- objets/ECOOP10/papers/6183/6...](http://www.lirmm.fr/~ducour/Doc- objets/ECOOP10/papers/6183/61830429.pdf)

3) Pick your value encoding carefully. You almost always want fast immediate integers. On 64-bit platforms it is quite common these days to repurpose some of the NaN range in IEEE doubles for type tags to enable storing doubles in immediate values.

4) Write your interpreter in assembly. Compilers generate terrible code for interpreters, even (especially?) with the use of computed goto / labels-as- values extensions. The register allocators of traditional compilers are designed to optimize loops by moving spill code outside of them and to reduce the impact of function calls. They will not be able to realistically allocate registers across different instruction bodies, and they won't be able to make the correct tradeoff about how much work to push into the slow path of instruction bodies.

5) Rearrange your instruction bodies based on execution / transition frequencies to improve instruction cache performance.

6) Pay close attention to the boundaries between your interpreter and the runtime libraries / the FFI. You don't want to take a bigger hit than you need to every time you call out to native code.

<comment><AUTHOR> >Rearrange your instruction bodies based on execution / transition frequencies to improve instruction cache performance.

Do you mean...group all the frequent operations together so they overlap on cache lines? It's hard to tell how much this would help, have you tried it?

<comment><USER2> When I was working on WebKit we would rearrange instruction bodies to influence the generated code based on opcode statistics, but back then the interpreter was using computed goto, so there wasn't quite a direct connection between the placement of the input code and the generated code. It's unlikely that any two instruction implementations will overlap on cache lines, since they are all typically larger than a cache line, but more temporal coherency throughout code execution will improve performance, especially on CPUs with smaller caches.

You can do it automatically by gathering statistics on frequent instruction pairs. In practice greedy algorithms for code scheduling work fairly well, assuming you have meaningful statistics.

</comment></comment></comment><comment><USER3> It seems to me over the past ten years I've heard this story so many times: "Python sucks. Let's do the obvious thing that makes it faster." Then, a month or two later, "I did the obvious thing and it's sometimes faster but often slower, net no gain or possible loss." to which the response is obviously "No sale."

I say this merely as an interesting observation. I've come to consider this a _de facto_ counterargument to the claim that languages aren't slow, only implementations are. It may be theoretically true, but in practice, as nice as Python may be to use, it has proved a very difficult language to speed up. (PyPy has taken a very good run at it, but it sure wasn't a case of "I'll just do this easy, obvious thing." PyPy seems to have hit Python's performance with multiple PhD-thesis level attacks, and it's still certainly not C in the general case.)

<comment><AUTHOR> >It may be theoretically true, but in practice, as nice as Python may be to use, it has proved a very difficult language to speed up.

I disagree with you. Python isn't much harder to speed up than Lua and in some ways it's better behaved than JavaScript. Still, both of those languages enjoy implementations significantly faster than CPython. Really, it's not the semantics of the language which hold back Python's performance but rather the fact that extension modules are extremely tightly coupled with a particular interpreter implementation. Lua has a clean interface with C, JavaScript implementations generally force the outside world to use doubly indirect handles on objects. CPython, on the other hand, is shameless in flaunting its internals for the whole world to see.

The only reason that PyPy has taken "multiple PhD-thesis level attacks" to near completion is because their approach is insanely ambitious. They didn't write a JIT. Instead they wrote a toolkit for partially evaluating interpreters on source files and generate native code by tracing an interpreter while it itself runs a program. It's nuts! It's amazing that PyPy works and the amount of effort is totally unsurprising.

Had they gone a more traditional route, the whole thing could have been done in a year or two. They would, however, still face resistance from a Python community that wants to neither give up nor rewrite their PyObject-laced libraries.

<comment><USER4> I think you underestimate the complexity of Python language.

Note that PyPy is not the only project that did that - remember psyco? There are reasons why after 3 years Armin said "I give up, let's do PyPy". It's not the "well behaved" part, this can be worked around, Python is simply more complex than Javascript or Lua and by complex I mean just bigger. All the extension modules that everyone naturally expects to be fast (even just the stdlib), descriptor protocol, crazy frame access semantics. That does make it very labour intensive to do the right thing. Look what happened to Unladen Swallow - they did not get anywhere really within a year. Several of PyPy optimizations that took forever to do are really new stuff, whether you do JIT by hand or generate it automatically.

<comment><USER5> The last time I talked with the Unladen Swallow guys (a couple years ago), they were pretty clear that one of their main stumbling blocks was supporting the Python/C API, and wanting to have complete compatibility with C extension modules. While we can't really know how hard the task would've been if they'd lifted that requirement - it was baked into their design from an early stage - when I'd floated the idea of doing a from-the-ground-up LLVM-based implementation of Python, they seemed significantly more optimistic. It wouldn't be all that useful for most people, but it would've worked fine for my use case.

Alas, it's doubtful that a Python implementation that sacrifices C extensions would get all that far with mainstream adopters, as so many useful libraries are done as C extensions.

<comment><USER4> They were optimistic at the beginning too (even with C extensions). How would it fill your usecase in a way that pypy does not?

<comment><USER5> We were looking for something easily embeddable, but all host modules were provided by the application, so there was no need for outside C extensions. And the set of libraries that was importable was restricted and coding styleguides banned advanced language features like metaprogramming, so we could afford to cut corners on corner cases of the language. "Decent" performance (i.e. more like Java than CPython) was a requirement, as was multithreading support and lack of a GIL, and RAM usage was also at a premium (which was probably the largest argument against PyPy...also, this was a couple years ago, when PyPy was not as mature).

</comment></comment></comment></comment></comment></comment><comment><USER6> If you haven't already, LuaJIT's source code (and Mike Pall when asked) is a treasure trove of speedup ideas.

One idea that stood out to me (and which I first saw in LuaJIT, and as far as I know originated with Pall) is: when rewriting loop code, unroll at least 2 iterations of the loop. (The first executes and conditionally continues into the second; the second loops onto itself). So far, just extra work.

However, any kind of constant folding algorithm is now immediately elevated into a "code hoisting out of loop" algorithm at no extra cost - e.g., SSA form gets that kind of code motion.

I'm not sure Python can make much use of that, because it is nearly impossible to guarantee idempotence of operations - but in case you can somehow make that guarantee, that can be very significant for e.g. function name lookups.

A possible way to use that is to have the loop opcode have two branch targets: "namespaces modified" (which goes to the first iteration, which reloads values) and "namespaces unmodified" (which loops at the 2nd iteration, relying on the constant folding and not looking up in dicts again). This could make calls like "a.b.c.d.e.f" require 0 lookups in most iterations of most loops -- but would also require a global "namespace modified" flag.

<comment><USER4> We have this optimization in PyPy (so yes, Python can do it). There is even a paper [0]. LuaJIT is a great source of inspiration but Python is just a huge language, which makes it much harder.

[http://www.maths.lth.se/matematiklth/vision/publdb/reports/p...](http://www.maths.lth.se/matematiklth/vision/publdb/reports/pdf/ardo- bolz-etal-dls-12.pdf)

</comment></comment><comment><USER7> I don't think its correct to say the CPython is slow. What you can more accurately say about CPython is that the performance is highly variable. Some things are very fast, while others are comparatively slow.

The slow things tend to be the sort of numerical loops that you see in micro- benchmarks. It's no coincidence that the version of Python in the linked article saw its greatest speed up in a numerical loop, but only modest improvement elsewhere. It's exactly this sort of simple repetitive operation where interpreter overhead matters the most.

Language features that encapsulate complex functionality tend to be harder to speed up in CPython because the VM operates at a fairly high level. In effect you're just kicking off a large subroutine that is written in C, and you're really executing native code until that operation is complete. You're not going to improve very much on that no matter how much you try.

What this means is that speed will depend heavily on the type of application program being written, and also on how much the programmer takes advantage of the unique language features. It also makes realistic cross language benchmarks difficult because the right way to do something in Python may not have a direct equivalent in another language. The result tends to be "lowest common denominator" benchmarks, which are exactly the sort of algorithms which CPython does worst at.

<comment><USER8> it really is a slow interpreter. Python comes with a pystone benchmark, and CPython is invariably the slowest of all interpreters. This being said, if you take a look at it's implementation, then it's immediately obvious why. The interpreter is a basically a simple C switch, with no optimization whatsoever. Simply threading the interpreter would make it about a factor 2 faster (at least that's what the experts claim you gain by threading)

<comment><USER7> Pystone isn't a performance benchmark, or at least it isn't a useful one. It's more of a regression test to see if anything has changed between versions. It's not useful as a performance benchmark because it doesn't weight the results according to how much the individual features matter in real life. There are three versions of Python besides CPython that are in commercial use. Two are much slower than CPython (up to three times slower), and Pypy is (currently) faster in some applications and slower in others.

The CPython interpreter is _not_ a simple switch. It uses computed gotos if you compile it with gcc. Microsoft VC doesn't have language support needed for writing fast interpreters, so the Python source is written in a way that will default to using a switch if you compile it with MS VC. So, on every platform except for one, it's a computed goto.

Modern CPU performance is very negatively affected by branch prediction failure and cache effects. A lot of the existing literature that you may see on interpreter performance is obsolete because it doesn't take those factors into account, but rather assumes that all code paths are equal. Threading worked well with older CPUs, not so well with newer ones.

I am current working on an interpreter that recognises a subset of Python for use as a library in complex mathematical algorithms. As part of this I have bench marked multiple different interpreter designs for it and also compared it to native ('C') code. It is possible to get a much faster interpreter, provided you limit it to doing very simple things repetitively. These simple things also happen to be the sorts of things which are popular with benchmark writers (because they're easy to write cross language benchmarks for), but which CPython does not do well in.

A sub-interpreter which targets these types of problems should give improved performance in this area. Rewriting the entire Python interpreter though would probably have little value, as the characteristics of opening a file or doing set operations, or handling exceptions are entirely different from adding two numbers together.

There is no such thing as a single speed "knob" which you can crank up or down to improve performance. There are many, many, features in modern programming languages, all of which have their own characteristics. Picking out a benchmark which happens to exercise one or a few of them will tell you nothing about how a real world application will perform unless it corresponds to the actual bottlenecks in your application. For that, you need to know the application domain and the language inside and out.

One thing about Python developers is that they tend to be very pragmatic. When someone comes to them with an idea, they say "show me the numbers in a real life situation". More often than not, the theoretical advantage of the approach being espoused evaporates when subjected to that type of analysis.

<comment><USER8> [http://hg.python.org/cpython/file/16fe29689f3f/Python/ceval....](http://hg.python.org/cpython/file/16fe29689f3f/Python/ceval.c#l1334)

looks like a switch to me.

Anyway, I've let them tell me the CPython interpreter is very simple on purpose to allow it to function as a standard 'definition' of the language behaviour. A simple jit does wonders, as does a less brain dead gc. Superinstructions, threading, ... are all possible. But you're absolutely right: It's really difficult to predict how much each improvement would contribute.

<comment><USER7> Have a look at the lines starting at line 821 in the very file you referenced. I have quoted a bit of it here:

"Computed GOTOs, or the-optimization-commonly-but-improperly-known- as-"threaded code" using gcc's labels-as-values extension (...) At the time of this writing, the "threaded code" version is up to 15-20% faster than the normal "switch" version, depending on the compiler and the CPU architecture."

They also have an explanation of the branch prediction effect which I mentioned earlier.

They have both methods (switch and computed goto) since some compilers don't support computed gotos, and some people want to use alternative compilers (e.g. Microsoft VC).

In my own interpreter, I tried both switch and computed gotos, as well as another method called "replicated switch". I auto-generate the interpreter source code (using a simple script) so that I could change methods easily for comparison. In my own testing, computed gotos were about 50% faster than a simple switch, but keep in mind that is strictly doing numerical type code. More complex operations would water that down somewhat, as less of the execution time would be due to dispatch overhead.

Computed gotos aren't really any more complex than a switch once you understand the format, and as I said above you can convert between the two with a simple script. What does get complex is doing Python level static or run time code optimization to try to predict types or remove redundant operations from loops. CPython doesn't do that, while Pypy does this extensively. It's these types of compiler and run-time re-compile optimizations which make the big difference.

Overall, my interpreter is currently about 5.5 times faster than CPython with the specific simple benchmark program I tested. However, keep in mind it only does (and only ever will do) a narrow subset of the full Python language. Performance is never the result of a single technique. It's the result of many small improvements each of which address a specific problem.

<comment><USER8> so the conclusion really is: CPython is way slower than it should be. Question: if the subset is small, isn't it better to use something like 'shed skin' ? [http://code.google.com/p/shedskin/](http://code.google.com/p/shedskin/)

I once looked at it, and it does a fairly literal translation. The only problem is that it changes semantics of the primitive types. For example a python integer becomes a C++ int. (and overflow semantics change)

</comment></comment></comment></comment></comment></comment><comment><USER9> As a web developer, I've often heard neckbeards bickering about Python's performance, but haven't had a real point-of-reference to understand how bad it can be until recently.

I've started working on a side project that processes geo data in AppEngine. My dataset includes many long lists of numbers (lats, longs, altitudes, timestamps, etc.). A 700 route dataset is about 25MB in a sqlite database, but trying to access any significant portion of it quickly maxes out the 4GB of RAM available on either of my dev machines (which is more than I could reasonably expect to be provisioned in the cloud). I mentioned this as a potential bug to the relevant Googler at I/O this year and he basically said "that's not us, that's Python."

It's mindboggling how quickly you can burn through your RAM in CPython. Hopefully you can prove something that will eventually make its way back into CPython and lift everyone's boats. Unfortunately, even if Falcon helped on my dev machine, I can't imagine it being taken up on cloud platforms like AppEngine.

<comment><USER10> A dataset with "many long lists of numbers" sounds like an ideal use case for NumPy, have you tried using that?

<comment><USER11> GAE allows only pure Python. No binary modules like NumPy.

<comment><USER12> Numpy is supported [https://developers.google.com/appengine/docs/python/tools/li...](https://developers.google.com/appengine/docs/python/tools/libraries27)

</comment></comment></comment></comment><comment><USER13> Can we make function calls cheaper?

From my observations in pretty much any unoptimized Python (CPython interpreted) code function calls is nearly always a bottleneck. And speed is directly bound by the number of function calls being performed, not by ponderous data structures.

<comment><AUTHOR> The ponderousness of these data structures isn't just about memory consumption or having to use boxed numbers. As far as performance goes, PyObjects infect everything in the interpreter. For example, when you're calling a Python function, after a long run-around in ceval, PyObject_Call, the function object's function_call method, you'll finally get back to ceval which creates a frame via a lengthy call to PyFrame_New. The whole process is a mess of allocating, deconstructing, increfing, decrefing, and tag-checking.

</comment></comment><comment><USER4> The title is very misleading - it should be called "How fast can we make CPython", because it talks about compatibility with PyObject* and stuff.

While I would argue that we can't make interpreted Python particularly fast, the _actual_ topic in question is much harder than that.

</comment><comment><USER14> While this is an undeniably cool project from the tech side, I think it's usually a better idea to rewrite bottlenecks of the kind that this helps with as a c extension. It's a fairly easy process (MUCH easier then in Java for example) and only a small amount of code needs to be in c itself but you can get huge performance increases without changing the Cython environment. I've always thought that was one of python/ruby's greatest strength's is the easy C integration.

<comment><USER15> I think you mean the "CPython environment"? "Cython" is something else.

</comment></comment><comment><USER15> No mention of the somewhat confusingly named Cython [1]? It addresses the same issues in a different way.

[1] [http://en.wikipedia.org/wiki/Cython](http://en.wikipedia.org/wiki/Cython)

</comment><comment><USER16> Python is typically used to build applications that are IO bound. Squeezing more performance out of the interpreter is not going to translate to any real gains for most Python users these days.

<comment><USER17> That's circular thinking.

Because Python is slow, Python is not used in scenarios where speed is crucial. That much is true.

However, if Python was faster, it would be used in those scenarios, so more people would be using it for speed critical code so it would provide real gains for a great many Python programmers.

This is exactly what happened with JavaScript: before V8 JavaScript was in exactly the same position as Python. Not many people were writing large programs in JavaScript because JavaScript was too slow. V8 sped up JavaScript 10x+ and people started writing much larger apps that do require that speed. If JavaScript speed suddenly dropped to pre-V8 speeds, we would all find the most popular web apps unusably slow.

<comment><USER18> It's probably worth noting that Google already attempted a V8-like transformation for Python with Unladen Swallow, and that that attempt mostly failed. Perhaps it was just prioritized differently, and that's why V8 was successful and Unladen Swallow wasn't.

</comment></comment></comment><comment><USER19> Interesting approach. One criticism: The paper mentions that compile times max out at 1.1ms "for the most complex function" in the benchmark (AES), and therefore it is sufficient to just compile everything. However, those benchmarks seem too small to justify that conclusion.

</comment><comment><USER20> The answer is very simple:

1) Know what ought to be done - do it and send the patches.

2) Need "speed" \- write that part in C.)

<comment><USER21>
        2) Need "speed" - write that part in C.)

That's not so easy. Interfacing Python and C code is also incredibly hard, and no one true way exists.

<comment><USER22> >That's not so easy. Interfacing Python and C code is also incredibly hard, and no one true way exists.

Can you elaborate on this. I've worked on python C extensions (just minor updates and fixes, I've never been the one to write significant chunks of it), and it seems like interfacing python with C is pretty straight forward.

<comment><USER23> If you use the CPython C API it's easy - but you also bind yourself closely to CPython. If you know performance is going to be important it's probably better to bite the bullet and use PyPy - which means you have to use the somewhat cruder cffi to interface with C code.

</comment></comment></comment></comment><comment><USER24> This is bikeshedding, but "The only hard problems in computer science are cache invalidation and naming things" \- there's already a quite popular, multi-paradigm programming language named Falcon[1].

1\. [http://www.falconpl.org/](http://www.falconpl.org/)

<comment><USER25> Sorry, but the two hard problems in programming are actually cache invalidation, naming things and off by one errors.

</comment></comment><comment><USER26> Haha oh my god Terry A. Davis commented on it!

SQUAWK SQUAWK SQUAWK</comment>


The AMD Fusion is not my ideal machine. - <AUTHOR> http://ghosting.posterous.com/the-amd-fusion-is-not-my-ideal-machine

<comment><USER2> I have two of these machines. One is a quad-core laptop, and the other is a dual-core E-350 server.

In both cases, the driver provided by AMD was horrific. The drivers that came with LinuxMint and Ubuntu (not the non-free drivers, the default ones) were fantastic and very fast.

I am very happy and run both machines on dual 24" monitors @ 1920x1200 and they scream.

<comment><AUTHOR> Sadly in my case, default driver does not work with HDMI and it does not recognize my 24" monitor. The ATI official driver does recognize and it does use 1900x1200 resolution but only with large black border.</comment></comment>


What is your favorite IDE for development? - <AUTHOR>

More importantly why do you choose it and do you have a different favorite for each language that you use? <comment><USER2> TextWrangler when I'm at work on my Mac, I preferred the simplicity and the fact I can't leave a script without first saving, which forces me to rethink whether or not I really want to come back later to this code base or if that variable I left out would be better added now.

I use Notepad++ at home for Web Development and MS Visual Studio 2010 Express for C++ for the compile and test features.

</comment><comment><USER3> I find it interesting that Visual Studio hasn't been mentioned yet even though it is likely one of the most popular in the world.

I'm going to say Visual Studio. It is polished, it is well designed, and it makes me highly productive. I've used many other IDEs in my day but am yet to find something as efficient for RAD as C# with Visual Studio.

I will say its C++ implementation is only "alright."

</comment><comment><USER4> Sublime Text 2 for most of my ruby and python hacking - it's just a great text editor!

Visual Studio when I'm on M$ projects - I can't see the point of not using it - it's a good dev environment, great debugger and is tooled especially for everything Microsoft.

XCode if I'm working on iOS - same reasons as Visual Studio above - it's the right tool for the job.

</comment><comment><USER5> I use Sublime Text for c++, I like it's simple and clear design, it fast and makes me more productive.

For java I use Eclipse, it has useful set of features, especially I like it's refactroing tools. The biggest downside is it's too slow for big projects.

</comment><comment><USER6> Intellij - Java and Python and probably others Emacs - everything else

Intellij, in my opinion has no peer. I bought a license for it, and am happy to say that it has paid for itself many times over now. I am no longer afraid of refactoring stuff, never forget to checkin a file, and it just works

Emacs, if not for anything else, should be used to look at how software should be designed(for use). A lot of stuff in emacs is very well put, and it almost holds a huge structure by a wire frame, like a beautiful bridge.

</comment><comment><USER7> Editor: Sublime Text 2 IDE: IntelliJ IDEA / Related JetBrains products

Would love to check out Vim at some point, however refactoring sugar makes me stick with IDEA.

<comment><USER8> I second this setup. Coming from a Java background IntelliJ stood out over Eclipse for my eyes. I recently purchased PyCharm for my own side project. IDE struggles here a little, because of typing and var declaration( or lack there of). However, refactoring and the line by line debugger helped speed up my Django development. I wish it integrated other MVC web stacks as well.

</comment></comment><comment><USER9> Vim. I enjoy the flexibility, macros, and great snippets. I am also able to code new features and tools for myself to use making future coding quicker.

<comment><USER10> Vim within tmux/GNU Screen. Snipmate (or Ulti-snips) are great. Great set of plugins, try pathogen (for bundle management).

<comment><USER11> I prefer vundle.

</comment></comment></comment><comment><USER12> I love Vim using the spf13-vim distribution. It turns Vim into the best parts of an IDE and keeps it feeling like Vim.

</comment><comment><USER13> It has some flaws but I've really started enjoying xcode. I am forced to use it for iOS but its getting better.

</comment><comment><USER14> When working on Aardwolf which is mostly pure ANSI C (GCC) nothing quite beats VIM with cscope extensions.

For quick editing a webpage or some CSS just plain old notepad++.

For day job stuff, either xcode or Visual Studio depending on the App.

</comment><comment><USER15> for me its gotta be emacs. I love the keybindings (that also work in bash), also customizing to no end in lisp is fun stuff. So that's what I use whenever I'm programming in c or x86. For java its eclipse and I like everything about it from the debugger to the interface.

</comment><comment><USER16> Started with Notepad++ and Eclipse, moved to Netbeans, just recently switched to Textmate2

</comment><comment><USER17> I am trying to do as much of my code as I can in emacs, and I am using Eclipse for java.

</comment><comment><USER18> Sublime Text 2 for Ruby on Rails/Django. Coda 2 for HTML/CSS/JS.

</comment><comment><USER19> InteliJ

</comment><comment><USER20> notepad++, vs20xx, komodo edit on nix and ms

</comment><comment><USER21> webstorm.</comment>


Kids and Time - <AUTHOR> https://www.jefftk.com/p/kids-and-time <comment><USER2> I see a lot of new parents struggling to fit time in for code in those early days. The odd Ruby tweet mixed in with spit-up talk.

It took me a good 3 1/2 years to finally get into the right routine, both physically and mentally. Our kid's great (another one on the way in Sept) but we really struggled with a consistent bed time routine that was healthy for all sides for a long time.

We're in a good place now but I imagine, just like us, many parents fail to realize that every child has different needs and sometimes, you've gotta ride it out quite differently.

As an aside, the pandemic sure has helped us get into a more rigid routine. We have the odd late night for our kid but otherwise, the house is clean (as can be) and kid's in bed by 8pm. A silver lining in all this mess.

<comment><USER3> For what it's worth, two kids is not 2x the work. More like 4x. You will need to step up your game :).

<comment><USER4> _> two kids is not 2x the work. More like 4x_

Different families seem to have pretty different experiences here. In my case having two kids has felt like maybe 1.5x the work, decreasing as they get older. At this point (4y and 6y), I think one child would likely be more work than two, because they spend so much time playing with each other.

Probably depends a lot on what kids you happen to have, how they get along, your parenting style, and what resources you have.

<comment><USER3> > Probably depends a lot on what kids you happen to have

I agree. If my son had been born first, I'd think I was the greatest parent in the history of parenting. But my daughter came along first, and taught me otherwise.

</comment></comment></comment></comment><comment><USER5> I think this is why it's no coincidence that people often become people managers around the same time they have kids. Not only are you consistently managing people at home and work, it tends to be a more steady 9 to 5 job with less late night passionate coding or such required.

Or that could just be my experience.

<comment><USER4> For what it's worth, that hasn't been my experience. I've been working ~9-5:30 as a programmer for my whole career (2008-present) with essentially no late night coding. I think that's been very typical at the places I've worked (defense contractor, mature startup, Google, startup, Google).

I do think the fraction of managers with kids is higher, but I suspect this is just explained by managers generally being older?

<comment><USER5> Perhaps just different personality types. I have never worked well technically productive in an office. I view that as planning, reviewing, conversational time. I just love working on things at night. My brain feels optimized at that time. But kids stopped allowing me that window, which wasn't necessarily 9pm-2am or something. More like 3pm-midnight.

<comment><USER6> For me it's some combination of the darkness and less "other stuff happening". Otherwise I just want to go outside and otherwise get distracted by the hustle and bustle of humanity.

</comment></comment></comment></comment><comment><USER7> We have twins in elementary/primary school--anything I say is couched on that reality. We found that having them sleep through the night, go to the bathroom in middle of night by themselves, and wake up themselves, freed some energy. They also are in bed asleep before 8:30, which frees some time.

The primary issue for us in terms of "free time" is the quality of the free time. It happens when our circadian rhythms are in the downhill stage. And it happens in shorter chunks than the "ideal" chunk for creative and flow-driven work.

The pandemic & having them at home has exacerbated these issues.

PS: If you have young twins or are about to have them, it is true that feeling normal happens when they are 5, and not before. This is much later than if you are having one kid at a time, on average.

<comment><USER8> > it is true that feeling normal happens when they are 5, and not before.

As the father of a 4 year old and twin 2 year olds, curious what you mean by this.

Feeling normal in what way?

<comment><USER7> Would love to answer-- can you tell me more about your curiosity?

</comment></comment></comment><comment><USER9> With a 3 and 1 year old me and my wife are or on cusp of earlier bedtimes - will be a magic moment as our free time will shoot back up - we have told ourselves we will not just use it to talk about the kids or watch netflix ...lets see!

</comment><comment><USER10> cf Barber, "Women's Work: The First 20,000 Years"

Part of her thesis is that weaving and textiles have been stereotypical occupations for the distaff gender because they can be done in a single location and are interruptible, making them very suitable for child-rearing.

<comment><USER11> Speaking as an (amateur) archeologist, weaving and textiles are traditionally female jobs in most societies because it requires fine work by small, dexterous hands. I'd say a connection to child-rearing is reaching beyond the obvious.

<comment><USER12> Men can't have small dexterous soft hands?

<comment><USER13> In the same way that some women can be stronger than average men, but female soldiers are very rare historically.

<comment><USER14> That is why males cant play in orchestras ... male hands being bigger and stronger does not imply them having less dexterity

</comment></comment></comment></comment></comment><comment><USER15> I'd like to see his partner's figures too!

<comment><USER4> We've done time-tracking a few times as a couple to figure out whether we're splitting things fairly:

* 2011: [https://www.jefftk.com/p/time-division](https://www.jefftk.com/p/time-division) (no kids yet)

* 2015: [https://www.jefftk.com/p/time-division-ii](https://www.jefftk.com/p/time-division-ii) (18m)

* 2017: [https://www.jefftk.com/p/december-2017-time-tracking](https://www.jefftk.com/p/december-2017-time-tracking) (3y and 1y)

I think the increase in time has likely affected both of us similarly, but we haven't timed another week.

</comment></comment><comment><USER16> > Overall I'm enjoying having more time for projects, though I do miss some > things about very little kids.

I might have an idea to solve that one!</comment>


NYTimes: Warren Dares Facebook with Intentionally False Political Ad - <AUTHOR> Warren Dares Facebook With Intentionally False Political Ad https://nyti.ms/2q6uszb <comment><USER2> Clickable Link: [https://nyti.ms/2q6uszb](https://nyti.ms/2q6uszb)</comment>


Android has won: now what? - <AUTHOR> http://www.h-online.com/open/features/Android-has-won-now-what-1828041.html <comment><USER2> It's interesting that we still look at platforms through the lens of Microsoft's Windows. That was a platform which achieved market dominance and generated huge revenues for its creators by way of significant licensing costs. They then also sold content (typically software - Office and various back end tools) to end users at high cost and ate a huge portion of the overall software market revenue. We view high market share as winning because usually it brings high revenues and that means high margins.

Today we have something that looks a bit like the PC market did in the 80s - lots of competing platforms, some doing well, others not. Android has not yet achieved the market dominance that DOS/Windows did (i.e. 90+ percent) and it certainly isn't generating anywhere near the kind of revenue for its creators, Google, that DOS/Windows/Office/etc did for Microsoft. Most of the revenue instead appears to be going to Samsung, but even that is only a minority of the revenue for the whole market - by far the largest share is going to Apple.

So how do we define Android has having "won"? It has no overwhelming market share, its future looks uncertain (as mentioned in the linked article) and it's only making any serious money for a company that didn't even create it. This puts the claims of winning under serious strain, but they just won't go away.

We should want lots of competing platforms. Competition is good for us. Trying to decide that someone has "won" just so we can be the one who called it most decisively, is stupid and short sighted (but understandable, given the desperation of journalists, who like Microsoft, find themselves having to shout louder and louder to be noticed at all). We should resist another dark age like the one we endured under Windows, before post-PC computing made Microsoft charmingly irrelevant.

Sure it's nice to grind an axe against nasty nasty Apple, or poke fun at Android selling lots of shitty cheap phones made from 2 year old OSes, or to laugh at Ballmer telling us Windows Phone is going to do really well.

It would be nicer if we could recognise that we have multiple really good choices available to us and that is a good thing.

Unfortunately we are selfish, tribal creatures, so our wallets and our social graphs will vote this into a monopoly, just like last time.

<comment><USER3> Agree with your analysis, but I'm not so sure it will end up with a monopoly. Stickiness amongst iPhone users is very high. I think something like an 70/20 android/iPhone split, with apple taking most of the highest-value users, would be a stable equilibrium - if there is such a thing in such a young market.

</comment></comment><comment><USER4> What is the point of this article? Android has sold more phones almost since the beginning. So what?

The article announces victory on tablets based on predictions. So far the iPad continues to dominate in reality, though. Again, so what?

The only point of this article seems to be "If you view technology like sports and Android is your team, let's gloat in our victory!" Which is dumb on so many levels.

EDIT:

Looks like OP is just a shill account for this really low quality site.

<comment><USER5> Does the article have to have a point for us to talk about some cool technology that we all love? What's the point of your post?

Besides that, I think the point was clearly made in the title. What's wrong with viewing tech competitors as sports teams? Do you have a problem with sports?

<comment><USER4> > Does the article have to have a point for us to talk about some cool > technology that we all love?

Yes there needs to be a point to have a discussion. That's kind of the definition.

> What's the point of your post?

That this post was pointless spam. And further that the OP should be banned for being a shill for a poor quality site.

> What's wrong with viewing tech competitors as sports teams?

It's childish and counter-productive. You should not have loyalty to a company based purely on its brand (like a sports team), you should choose products based on what suits your needs and tastes best. Maybe for a period of time there will be one company that keeps doing that perfectly, but you should continue to examine the competition and give them a fair shake. Unlike sports teams.

That should be pretty obvious. I can't believe I had to explain it on here.

<comment><USER5> > _Yes there needs to be a point to have a discussion. That's kind of the > definition._

That's one definition of discussion. Here's one for talking - "Verb. Speak in order to give information or express ideas or feelings."

The article made many points. However, maybe next time they should notify you so you can judge them first.

>

> _That this post was pointless spam. And further that the OP should be banned > for being a shill for a poor quality site._

Well your honor, it seems like the other root commenters on this post were able to find something to talk about from the article.

I wouldn't call it spam. As a matter of fact, I learned something new while reading it. I won't bother telling you what it is because you'd probably attack me with more of your vitriol.

>

> _It's childish and counter-productive. You should not have loyalty to a > company based purely on its brand (like a sports team), you should choose > products based on what suits your needs and tastes best._

Should? According to you. I hold the view that people should do what they want within reason. You however seem to want to censor anything that you don't agree with.

>

> _That should be pretty obvious. I can't believe I had to explain it on > here._

You sound like a know it all asshole bully who got offended because someone stepped on his toe. I can't believe I had to call you out here.

</comment></comment></comment></comment><comment><USER6> Unix style fragmentation* is inevitable. We already have version fragmentation.

"To make matters worse, the big new players in the Unix market promptly committed major strategic blunders. One was to seek advantage by product differentiation — a tactic which resulted in the interfaces of different Unixes diverging. This threw away cross-platform compatibility and fragmented the Unix market." - <http://www.faqs.org/docs/artu/ch02s01.html>

<comment><USER7> The product differentiation did offer some major benefits, though.

It's what allowed IRIX to be used to perform graphics-related tasks that basically couldn't be done before, or on other systems.

It's what allowed SunOS and later Solaris to become extremely powerful workstation operating systems.

It's what allowed AIX and HP-UX to excel as server operating systems.

Only within the past few years (a couple of decades after the so-called "UNIX wars") have we seen Linux getting to the point of being a single system that can cover such vastly different areas sufficiently. Even then, the experience still isn't as smooth as it was on more focused systems back in the 1980s.

</comment></comment><comment><USER8> There are several things wrong with this analysis. First among them is the supposition that Android is rudderless without Andy Rubin.

Android has won, on mobile handsets, in an explosive way, in the past three years. It is time now to consolidate that victory.

Android has a whole other area to conquer, in tablets, in the workplace, displacing PCs instead of feature phones and failed smartphone platforms. What Google product already dominates in PCs? Chrome browser.

If Google simply knocks the rough edges off the process of becoming a Google Logo and/or Nexus OEM, Android is going to continue to mop up the remaining handset and tablet OEMs and expand into PC OEMs entering the tablet business.

Maintaining the blistering rate of Android versions with meaningful innovation will not be easy, but there are still plenty of units in the handset and tablet world to conquer.

</comment><comment><USER9> "That's because few outside the main cities can afford high-end iPhones, and most opt for low-cost Androids instead."

A statement like that absolutely needs a citation.

</comment><comment><USER10> Now what?

They should put their work-force into building high end browsers for iOS and Android and get of our backs with their native app frameworks ;)

<comment><USER11> Yes, let's bring the mobile phone experience 10 to 20 years backwards, letting only what can be done with the hodge-podge that is HTML5 be available.

</comment></comment><comment><USER12> What the hell? Android has won? Have you ever looked at statistics for device usage?

Here's a neat little graph of about 200M unique users over the last year, mostly from the US and UK.

Either we have a very odd sample (unlikely, broad basket of eCommerce sites), or, well, Android sure as shit ain't dominant.

<http://imgur.com/6eQpsOC>

Oh, and people buy iOS for Christmas, not Android...

<http://imgur.com/AbhnF08>

<comment><USER11> > _Either we have a very odd sample (unlikely, broad basket of eCommerce > sites), or, well, Android sure as shit ain't dominant._

It is. Just with the wrong crowd: mostly people that don't care much about smartphones and got a low tier Android phone for free with their contracts.

So it has won on volume, but not on the people that matter (both financially, ie those buying with higher margins, and technologically, ie the most tech savvy).

Which also explains why over 50% of Android phones run 1 or 2 years old Android versions.</comment></comment>


The Worst Internet in America - <AUTHOR> https://fivethirtyeight.com/features/the-worst-internet-in-america/ <comment><USER3> I used to live in Saguache County as a kid. We left in 2000 but this article does not surprise me. it is hard to adequately get across how desolate it was up there but i'll try.

We had 40 acres of land and were considered city slickers because 40 acres was enough for 1-2 cows. We were friends with people with 25k acres.

The ground was sand. 10 foot down was sand. You could water plants 3 times a day and they would still run out of water. (My mom fixed this with lots of silca gel for her garden.)

We lived on a county road and the postal service would not deliver mail closer that 2 miles away. We never did put up a mailbox and instead just had a PO box in Moffat which was about 8 miles of gravel roads away.

Annual rainfall is less than the Sahara dessert and temps would hit -40F in the winter. Some days would see 70F difference between day and night.

Walmart was 40 miles away (Alamosa in one direction and Salida in the other.)

In parts there are no trees for miles. We had ~30 trees by our house. The next ones were at another house 1/4 mile away and I believe the next were 2 miles away.

Our phone system used an antenna that looked like the old tv arials and was solar with a battery on it. (96/97 era.) We had that for about a year before the phone company decided they needed the system more in alaska and would lay phone line for free rather than charging us $6k to run the lines half a mile. Then they waited a year to come get out system. The battery on it would routinely run out in the middle of a call.

<comment><USER4> Wow that sounds hard. Have you ever considered founding a utilities company yourself and lay those lines for you and your neighbors? I have no idea how easy or hard that would have been.

<comment><USER5> You need access to capital to do that, and if you have access to capital you probably aren't in a place that considers 2 cows as wealthy.

<comment><USER6> They were "city slickers" because they _only_ had _two_ cows (or, more likely, only had enough land for two) where as everybody else had many, many more.

City slicker isn't an insult that means "rich" by the way, it means from the city and hence not willing do a hard day's work, dresses too nice and is not trustworthy.

<comment><USER3> For us it was not really an insult. It was poking fun at us but it was not meant in a bad way. Not to say it could not be meant as one though.

<comment><USER6> I know what you mean, my family calls me a city slicker and that's how they intend it as well.

I hope ...

</comment></comment></comment></comment></comment></comment><comment><USER7> That page shows the worst use of the internet in the world. 15.88 MB in size according to Firefox. And unintuitive gimmick navigation to boot. How does doing something like that help convey the message?

<comment><USER8> This seems like an extraneous tetchy comment but I don't think it is. Extant slow internet connections would be a lot more usable if websites weren't full of video/audio adverts, trackers, and JS being used for stuff (displaying text/images or doing some awful and slow scrolling) that browsers do well natively.

Try using the internet (beyond HN or gmail or pinboard) on a mediocre internet connection without any adblocker or noscript-equivalent. It gets nasty very fast.

To be fair, this isn't a justification for not building out better last-mile links -- just like, the miserable state of residential internet connections in the US is made a lot worse by awful bloated websites. [https://danluu.com/web- bloat/](https://danluu.com/web-bloat/) and [http://idlewords.com/talks/website_obesity.htm](http://idlewords.com/talks/website_obesity.htm) are good reads on this.

<comment><USER9> I may be wrong, but wouldn't the full "suite" of privacy tools, like uBlock, Privacy Badger, etc actually speed up these connections based on this?

</comment></comment></comment><comment><USER10> That page has a bad case of terrible website design. When I scroll down, I expect bits of image not to zoom in from the left. I nearly closed the page before even getting to the headline. They've also somehow made space bar not scroll as far as it does on other websites (in Safari), so every time I scroll down I spend a second searching for where I ended up.

At least there's only two locations of awful design during the adventure that is scrolling. Once you reach the headline, there's only one really unintuitive bit further down the page (a map of the USA), and that bit is short.

<comment><USER11> I could not read it on my iPad in landscape mode. Worked fine in portrait orientation. I don't know why.

<comment><USER12> I switched to Reader. Looked like landscape could work if you were patient enough, but it was painful.

</comment></comment></comment><comment><USER13> > if a household wants a download speed of 12 Mbps with an upload speed of 2 > Mbps, they can expect to pay a whopping $90.

This is the worst internet in america? Wow. That is amazing! I'm incredibly impressed at how fast the worst place for internet in america is.

In highschool (Class of 2011) I dealt with 5 mbps just because my parents didn't want to pay for more. It was tolerable as long as you didn't want to watch youtube in HD. Honestly, I have little sympathy for people with 12mbps.

<comment><USER14> As others have pointed out websites have gotten much larger for no good reason. The article we all just read is apparently over 18MB. My internet also used to be much slower than what people in that town can have. However, the internet was optimised for that.

<comment><USER13> Huge article even by today's standards. So the article would load in ~10 seconds under 12mbps, and thats assuming they don't have asynchronous progressive loading of larger images and such. - which i think they do.

Not really a big inconvenience IMO.

</comment></comment></comment><comment><USER15> Serious question from someone who flinches at living more than ten minutes' walk from a train station and won't rent anything that doesn't have a crackerjack shower with painfully hot water and enough pressure to bruise, and an internet connection that sees less than a few minutes degraded service a year (although for speed, I actually don't demand that much - I'm happy with a couple of megabytes per second); why do people live there?

That's a genuine question; not "why" as said to imply that there is no reason. There must be reasons. I just don't know what they are.

Not generalities, please. I'm interested in first-hand information from people who actually live or lived there. Did you move to such a place? Why? What's the economy based on? Does it generate enough money to be self-sustaining, or does it exist through inertia? In areas with such low density of people, what jobs exist? Is it all farming or other such? How many people living there might actually like to move, but can't?

<comment><USER16> Farming and ranching. Case closed.

Also, the rural population has quite a different culture than what you are used to. They don't _crave_ internet. Some of them likely ask the same thing about your culture. Why do you like living so close to each other? Everything is so fast and overwhelming.

<comment><USER15> "Farming and ranching. Case closed."

Is it? Are you a farmer or rancher? Why's it necessary to live so remotely? Is it not possible for a town to support farming and ranching? Is it necessary for everyone involved to live so remotely? Given the huge areas involved, living on the farm/ranch makes you local to one small part of it but still remote from all the rest, so why the need to live there?

"Why do you like living so close to each other?"

I don't, but it's the cost of having the things I do want. I want fast, reliable, cheap internet. I want my other utilities to be reliable as well. I want to be able to walk ten minutes to a mainline train station - it gives me a lot of opportunities very easily, needing nothing more than train fare. The density of people creates a lot of other opportunities; things that are wildly uneconomical in low-density communities become viable. Range of jobs, range of businesses for me to patronise, leisure opportunities. It's really about all those opportunities, and the people that make them. When I want to learn something, I just find a local class. If there isn't a local class, it's an hour from my door on foot and by train to central London (and I don't even live in a county adjacent to London), and there the opportunities increase by another order of magnitude.

We don't (necessarily) like living close to each other; we like all the opportunities that are created by that - economic and self-development - <USER2> we like the many things that become economical in doing so (cheap, reliable utilities and infrastructure and niche businesses and all the rest that I mentioned above).

<comment><USER17> Both Farming and Ranching require a sizable amount of land to be viable. It's much more viable to live on the land you farm (or ranch on), than it is to live in town. Beyond that, cities create higher land values that tend to make farming or ranching non-viable.

That said, even rural towns are very underserved in the US.

</comment></comment></comment></comment><comment><USER18> I live in Melbourne, AU.. 4km from city centre, in a brand new high-rise building. I pay AUD$80/mth for 8mbps down and 0.5mbps up over shitty copper that dies every time it rains.

Pings are so bad lots of services like Netflix, YouTube, etc think I can't connect and just give me "Check your internet" errors when I load, until I refresh multiple times and get it to load.

Funnily enough, our rural areas have better net than us here. I own a farm in the absolute middle of nowhere (region population of around 500) and it gets solid 24/4 and some farms nearby who face a better direction for fixed wireless get 50/8 (or something like that).

<comment><USER19> My buddy outside Valley City ND (calling it a city was wildly optimistic) gets 1 gigabyte up 256mb down for USD$90. As you've said, it's a bit more than a rural / urban problem.

</comment></comment><comment><USER20> My mother lives in rural Oklahoma, and her Internet connection (2-3Mbps/1Mbps for $60 a month) is a WiMax antenna on the roof of her house, pointed at another antenna on the water tower of the next town over.

It's not great, but it's the only option she has other than satellite providers, and works well enough for Facebook, email, and online shopping.

<comment><USER21> My parents had dial up until about 2 years ago. On a good day, they'd get 28.8, but typically it was more like 14.4 kbps! When it rained and water got in the phone lines, they got nothing at all. And yes, they'd try to get the phone company to fix the lines often to no avail. They are only 20 minutes from town but it's too far to get DSL or cable or anything like that and the terrain is too hilly for fixed wireless which did become available in their county four or five years ago.

They finally broke down and got satellite internet a couple years ago but it's slow and expensive.

<comment><USER20> When Mom was stuck with using a modem, I bought her a (literal) stack of USR Courier v.Everything modems (cheap, since a lot of ISPs in Austin were ditching dialup service around that time).

She had lightning hits and storms so often that I told her "if you try to dial up and the modem isn't working, just unplug it, toss it in the trash, and plug in the next one off the stack." Had to do this even with surge protection inline for both power and phone line.

Only in the past couple of years have I been able to get an LTE phone signal at her house (only while standing on the porch, not inside the building) and that's from a tower a few miles away...

</comment></comment></comment><comment><USER22> "For around $30 a month, New York City internet providers offer basic packages of 100 Mbps service."

I'd love to know where in NYC this service exists. I currently pay $60/month for 50Mbps and often get half that speed in practice. My office Internet service is even worse, it goes out 3-5 times a day and I've had maybe a dozen techs come out to look at both spaces. They won't even let me use my own modem because it's business Internet, which they charge 3x the price for.

I've looked for alternatives, but have never had any other option than over- priced terrible service from Time Warner/Spectrum for the past decade.

</comment><comment><USER23> The infomap "Broadband is still foreign to much of the U.S." shows high speed internet largely confined to urban areas, as the article discusses. But large swathes of North Dakota are deep green. What's the reason for that?

<comment><USER24> Oil money, North Dakota is the second largest producer of oil in the US. There was no big announced project, just a large number of grants and tax cuts to promote infrastructure investment. And, from the looks of it, a depressing amount of the area without access are the reservations.

<comment><USER25> > a depressing amount of the area without access are the reservations.

I'm from North Dakota and went to high school on the Standing Rock reservation. They have their own institutions and legal system, so it's not surprising that they would have different coverage.

<comment><USER19> Standing Rock also has to deal with two state governments.

</comment></comment></comment></comment><comment><USER26> As a Canadian, this article sums up pretty well the internet situation in most of the country, even in "urban" areas. Due to the "Canada is different" mentality here, foreign, more efficient companies are forbidden from making business here, leaving the market to a few local operators that have no incentive to make the service any better. This leads to the country being years later in terms of broadband technology compared to other western countries. There is no excuse for that. Canadian tech is always pretty much a crippled version of the same product from the United States at a higher price. Even in hardware, the same MacBook is 1,799.00 $ in the US but 2,399.00 $ in Canada.

Saguache County is in the same situation thanks to it's remote location, but I bet you it'll get fixed far sooner than Canada's nonsense situation in term of internet access. I hope the ambitious infrastructure plan the US has adopted will somewhat cover the installation of better rural internet access.

</comment><comment><USER27> And that's why web developer should stop using JavaScript without reason and use pure and clean HTML, CSS and well compressed images.

</comment><comment><USER28> Assuming the page is from America, it seems quite close to the worst considering I had to scroll through ~3 pages equivalent of weirdly horizontally scrolling images while all but the sky of the images was obscured by a cookie warning spanning 1/3 of the page (uncovering an equally large subscription request after being dismissed).

</comment><comment><USER29> Most of these articles about broadband access conflate Internet access with broadband speeds. Yes, Internet access is basically a necessity in modern life. However, 25Mbps download (which is the definition used) is absolutely not.

<comment><USER30> As I understand it, while the talk is about speed, the real issue is reliability: in rural America if you try to get broadband, some of the time you'll get a few megabits per second, which is enough to get by, but a good deal of the time, your nominally broadband connection won't be delivering any data at all.

</comment></comment><comment><USER31> This is an interesting market for the upcoming broadband satellite internet by Boeing, SpaceX, etc. They will provide low latency because they are in low earth orbit.

<comment><USER32> Interesting, got any links about that? And do you know if it for the whole world or only some parts of it?

</comment></comment><comment><USER33> I applaud the comparison with electricity. This is almost exactly how I look at it. I would compare Internet providers with utilities providing water, roads, sewage and even the post.

To think at one time we had telegraph data lines that were little more than a mud battery and a pole. The wire would run up the pole, go though a relay switched by the previous pole and continue to the next pole where it would switch its own relay. The mud battery provided very little power, just enough to switch a relay but the technology was good enough to bridge almost unlimited distance.

Without comparison with other services I would be tempted to argue my ISP provides really great services. Its 75 Euro for 300 Mbit and the package comes with a phone line and hundreds of channels of something called television.

However, when compared honestly with similar services their service is terrible. People working there told me the entire network is ready to provide 1 Gbit to everyone and the employees don't know why they didn't make the switch. I can easily guess why, I'm not willing to pay much more than 75 Euro, 3 times the speed wont bring in 3 times the money. What it will do is reveal glitches in the network that would require fixing. If I remember correctly the people who created the ISP were very passionate about the technology but ended up selling the business. Not sure if that was related but if I was passionate about my business but couldn't upgrade because of a hunger for money I would get bored fast.

Using more of this rather ignorant guess work I started to ponder how an ideal ISP should be implemented. What would it really cost? Which type of contract is best for the network rather than the user or the provider?

Surprisingly, in some areas where big money hungry ISPs refused to deliver services people had to rethink and do it themselves.

Like this one here!

[http://www.buergerbreitbandnetz.de/fernsehen-ueber- glasfaser...](http://www.buergerbreitbandnetz.de/fernsehen-ueber-glasfaser- fuer-nur-995-e-im-monat/)

The "Glasfaser – für nur 9,95 € im Monat" translates to fiber optic for 10 Euro (?!?)

My German almost isn't but from reading around I gathered they needed 65% adoption but got 94%, each end user had to pay 1000 Euro in advance. 900 in the form of a lone and 100 worth of shares.

To guess some numbers with that... 7 and a half years worth of 10 Euro/month does seem to add up to 900 Euro while profit made after that should increase the value of the investment. IOW, if it takes 20 years to repay the debt it would work out just fine for the investor/customer.

Some quote from speedtest: "With an average Q2-Q3 2016 download speed comparable to that in Bulgaria and Moldova, Germany's fixed broadband is slower than you might expect from Europe's largest economy. At 40.38 Mbps, Germany ranks 29th in the world for average fixed broadband download speed and 72nd for average upload."

A different page mentions: "Expect a 35€ bill on average for a broad-brand connection"

Most important to keep in mind, the big ISPs refused to service the area. It makes a great proof of concept. (Assuming buergerbreitbandnetz delivers 1 Gbps) in this worse case scenario 10/35 buys 1000/40.38 times the bandwidth. Or 29% of the cost for 2476% of the service. Or the 2 figured combined their service is 8438% better than the conventional urban offering.

Rural UK ISP [https://b4rn.org.uk](https://b4rn.org.uk) happened when big business "could" only provide crappy services at crazy rates. B4RN charges 30 pounds for 1 Gbps.

speedtest writes: "according to the Connected Nations 2015 report from the UK's communications regulator, Office of Communications (Ofcom), 48% of rural areas don't have access to broadband speeds of 10Mbps or higher."

Of course our germans and these brits had money to make their own ISP happen.

My Spanish and my Greek are even worse than my German but from reading around, apparently Guifi is the worlds largest Mesh network in Greece that originally started in Spain.

[http://guifi.net/en/what_is_guifinet](http://guifi.net/en/what_is_guifinet)

Trying to make sense of this document...

[https://translate.googleusercontent.com/translate_c?depth=1&...](https://translate.googleusercontent.com/translate_c?depth=1&hl=en&rurl=translate.google.com&sl=auto&sp=nmt4&tl=en&u=https://guifi.net/files/20170528InformacioAddicionalDenunciaGeneralitat_Final_signed.pdf&usg=ALkJrhjuH1IO9ufyAP6zEHXxdF1Xho6Hww)

...it seems they are actively fighting a tax per meter that guberment is trying to inflict on their fiber optic cables. If I understood correctly they are mainly offended by established businesses putting their cable in before the tax was invented. It seems big business did more than seal the ground after digging in their cables.

How different this is from the mundane consumer perspective where we all pay some corp as much as we can for as little as we would accept. We live in a world that is a bit like: when one asks why a carton of milk costs 200 Dollars the answer would be that if they charge 300 USD for it they wouldn't sell enough milk to justify the higher price.

Our entire western society works like this (I almost called it a civilization) one delivers 12 Dollars worth of labor that is sold for 40 Dollars, etc, and eventually it becomes 1000 dollars as part of a mortgage or infinite dollars as part of someones rent.

If that is the mentality or indoctrination that comes over these roads, in these letters and though this internet tube it might just be not worth having any of it.

[https://upload.wikimedia.org/wikipedia/en/archive/4/42/20150...](https://upload.wikimedia.org/wikipedia/en/archive/4/42/20150103143744%21Woodbridge_old_barn_raising.jpg)

[https://www.youtube.com/watch?v=y1CPO4R8o5M](https://www.youtube.com/watch?v=y1CPO4R8o5M)

<comment><USER31> The German page you found is for a deal where existing customers can get IP-TV for 10€/month. The 50MBit/s plans start at 26€/month or so.

<comment><USER33> Ah, thanks.

</comment></comment></comment><comment><USER34> Sounds doubtful.

Worst Internet is not on the Navajo res (UT, AZ)? Not in Hanksville, UT or a bunch of other small towns in Southern Utah? I am not even talking about ranches in places like Robbers Roost canyon complex.

Nothing in Colorado is really remote and unpopulated. Roads and people are everywhere. Maybe San Juans qualify for remote wilderness somewhat.

</comment><comment><USER35> I would do anything for your so called worst speeds. Your worst is better than my best speeds.

</comment><comment><USER36> How is SpaceX low orbit satellites ISP project progressing?

</comment><comment><USER37> Looking at the photos, I'm surprised there's Internet connectivity at all in those places, other than 3G or maybe WiMAX.

I think it's not realistic to complain about this. The costs of bringing infrastructure to a place like this are simply too high.

<comment><USER38> We brought electricity and phone service to these places. What makes internet different?

<comment><USER3> Lived there once. (More info in my other reply.) They wanted 15k to bring electric lines half a mile. $6k for phone. We never did have electric outside of solar/wind/genarator. We had phone service via a device they installed that had a solar panel and an antenna that looked like this on it. [https://cdn.instructables.com/FNY/J38K/GXQPMTQV/FNYJ38KGXQPM...](https://cdn.instructables.com/FNY/J38K/GXQPMTQV/FNYJ38KGXQPMTQV.LARGE.jpg)

After a year or so they decided they needed it more in Alaska and they laid the line for free. Then waited a year to collect the equipment they so desperately needed in Alaska.

<comment><USER38> It's interesting that it's so expensive to get electric service these days.

I wonder how things actually looked in the heyday of the rural electrification board.

My parents live in rural Australia (similarly 40 miles to the nearest supermarket) so I have some idea but obviously it's very different geography and a different a situation - they have electric and a rotten phone line (certainly can't reach 56K), but get their internet through a rooftop antenna connecting to 3G.

<comment><USER39> Probably similar: [https://www.minnpost.com/mnopedia/2017/02/power-farmer- minne...](https://www.minnpost.com/mnopedia/2017/02/power-farmer-minnesota- and-rural-electrification-administration). The article quotes $600 per mile in 1939. That's $10,500 per mile adjusted for inflation, but the inflation metric isn't a good one here. Inflation is calculated using consumer prices. The inputs for electric (or telecom) infrastructure are mainly labor and materials, which are relatively more expensive than they were back in 1940. For example, if you scale $600 by the change in minimum wage over that time, you get $20,500, about what's quoted.

<comment><USER40> It is my understanding that the REA charged a flat fee per home and ate the cost to the extent it exceeded the fee. They also setup generating stations and gave loans to power companies (and later rural telephone companies).

A lot of that assistance isn't available anymore.</comment></comment></comment></comment></comment></comment>


Google Glass 2.0 - <AUTHOR> http://www.wsj.com/video/google-glass-20-first-look/7D225564-0A67-41D7-97A9-CF4C19CF3A4C.html <comment><AUTHOR> I doubt this will satisfy the luddites, but we can hope...</comment>


Show HN: Jumble – Essays on the go #PaulInYourPocket - <AUTHOR> https://itunes.apple.com/us/app/jumble-find-startup-essay/id1150939197?ls=1&mt=8 <comment><AUTHOR> Hi HN,

Like the rest of the startup community, we love a good Paul Graham essay. However, when it comes to choosing one it's always a bit of a hassle. So we made Jumble!

Jumble gives you a Paul Graham essay to read with 1 tap and then lets you shuffle your way through more essays on the fly.

If you've got any feedback/comments, we'd love to hear them.

Cheers, Jack & Sam</comment>


Reid Hoffman and Brian Chesky on Handcrafting and Scaling Airbnb [pdf] - <AUTHOR> https://mastersofscale.com/wp-content/uploads/2017/05/moshandcraftedtranscript.pdf <comment><USER2> If it isn't clear, this is the transcript of the first episode of a podcast called "Masters of Scale". [https://mastersofscale.com/](https://mastersofscale.com/)

</comment><comment><USER3> This is a tremendous podcast. The guests and subjects are A+. It is a the best new podcast around (for me anyway). (Only point of criticism would be, there are too many little jingles and maybe one too many commercials).

<comment><USER4> Funny, I listened to this recently and had exactly the same reaction. The cheesy audio jingles aside, the idea of contemplating your customer's experience in terms of stars ("What's a 1-star experience? What's a 6-star experience?") was a really neat way to think about product vision.

I also loved the idea that you can't just ask users "what would make this better?", but rather "what could we do that would make you tell all your friends about this?"

Overall, great examples of how when humanism comes first and tech comes second, the results can be magical.

</comment></comment><comment><USER5> As much as I like/respect/admire both Reid Hoffman and Brian Chesky, I find this podcast... terrible. So much that I had to stop listening to it after a few minutes.

I really hate the weird sounds and interruptions. It feels like a podcast designed to appeal to the masses, and to "overdo" every aspect of what a podcast should be.

I hope this is read as "constructive feedback", because that's the intention.

I listed to 10-12 podcasts a week, and I have never experienced this "repellence" to a single podcast, like this one. I believe this is the style of other podcasts of this series, if I recall correctly.

Am I the only one, or is there anybody else that shares the same feelings about it?

<comment><USER6> Honestly sounds like the work of a Fiver gigster to me. Super tacky.

<comment><USER7> It isn't. Audio production was done by respected pros.

</comment></comment></comment><comment><USER8> Masters of Scale podcast RSS feed: [https://rss.art19.com/masters-of- scale](https://rss.art19.com/masters-of-scale)</comment>


Show HN: ImgPush – Minimalist Self-Hosted Image Upload Service for Your App - <AUTHOR> https://github.com/hauxir/imgpush <comment><AUTHOR> Wrote this for my own app to host user avatars, hopefully useful to someone else.

<comment><USER2> Would be nice to have a license as well.

<comment><AUTHOR> done!

</comment></comment></comment><comment><USER3> > imgpush requires docker

I find that extremely hard to believe.

<comment><AUTHOR> You are technically correct but it's easier to get it up and running using docker as it contains all the dependencies needed to run it and works on every operating system with very little overhead.

Simply easier to tell people to use docker rather then writing a separate guide for every OS don't you think?</comment></comment>


Myungwan Kim 9d (top go professional) reviews AlphaGo vs. Fan Hui [Video] - <AUTHOR> https://www.youtube.com/watch?v=NHRHUHW6HQE&feature=youtu.be <comment><AUTHOR> A top professional reviews some elements of the five-game series, and explains that in the engine's state as of when the games were played there is little chance the engine could take a single game against Lee Sedol in March.

Who knows what will happen with the extra half a year of preparatory time with this engine: but to make so much progress in 2015 starting from scratch leaves the match to happen in March one to look forward to.</comment>


Google Launches Android Studio And New Features For Developer Console - <AUTHOR> http://techcrunch.com/2013/05/15/google-launches-android-studio-a-development-tool-for-apps/

<comment><USER2> This looks awesome! I'm really excited that Google is starting to embrace IntelliJ over Eclipse (personal preference).

<comment><USER3> Same here! Eclipse feels sluggish after some time. I wish the company I work for moved their IDE product from Eclipse to IntelliJ.

</comment></comment><comment><USER4> Where can I download this? I can't find a download link anywhere.

<comment><USER5> can be downloaded here

<http://developer.android.com/sdk/installing/studio.html>

<comment><USER4> Thanks!

</comment></comment></comment><comment><USER6> I can't get it to launch on my laptop... It keeps timing out, but runs fine on my desktop. Still, I am excited for this thing to become stable.

</comment><comment><USER7> this does look promising, will be tempted to begin writing android apps to give this a go. Google does seem to be doing a lot of work to get devs to ship apps in android. I really liked the upgrades to the dev console and the new features for optimizing like suggestions for tablet support, translations etc. Good job google</comment>


Want to Make a Diamond in Just 10 Weeks? Use a Microwave - <AUTHOR> http://www.bloomberg.com/news/articles/2015-08-27/want-to-make-a-diamond-in-just-10-weeks-use-a-microwave

<comment><USER2> Do anyone have more technical details of the microwave? How much power does it use?</comment>


About to be interviewed on an NPR station about Bitcoin. Advice? - <AUTHOR>

The interview will be at 12:30 on http://news.stlpublicradio.org/content/cityscape I am not an expert in this at all, just a guy who wants to see ideas like this move forward. Any help or support would be deeply appreciated. Particularly around how quickly this technology will permeate regular culture. <comment><USER2> Bitcoin has a seen a fairly significant attraction and interest lately. So much so that the US government has had to have panels and discussions spanning several topics and implications pertain to bitcoins. It has already seen some adoption in regular culture as shops are beginning to accept bitcoins as a form of payment for goods and services.

Bitcoins value has been growing exponentially over the past few years and who knows when that will stop. I personally think this is due to people using it as an speculative market with the hopes of making some 'easy' money (I say easy because there are various ways of getting money or gaining bitcoins with little effort). Since bitcoin's monetary value is so highly speculative currently, its widespread adoption as a currency may take a while until its value is somewhat stable.</comment>


Ruffle for Dogs - Machine Learning for dogs - <AUTHOR> https://rufflefordogs.com/ <comment><AUTHOR> We're working on our new website, let us know things that you would like to see!</comment>


Fiberbots: Design of a multi-agent, fiber composite digital fabrication system - <AUTHOR> https://www.media.mit.edu/projects/fiberbots/overview/ <comment><USER2> Here's an industrial scale version already in use in China.[1] This process is called filament winding. The MIT group is just doing it with a slightly different mechanism than usual.

[1] [https://www.frpmachine.com/frpmachine/vertical-frp-tank- wind...](https://www.frpmachine.com/frpmachine/vertical-frp-tank-winding- machine.html)

</comment><comment><USER3> Took me a minute to understand the mechanism behind these devices. Interested to see how the idea could be applied to practical applications.

</comment><comment><USER4> Cool technology, but I don't see yet how this could be superior to other construction methods in any practical way.

</comment><comment><USER5> Could some one explain what 'digital' means in this context?

<comment><USER4> The design is expressed in data and executed with minimal Human intervention.

Much like 2D printing, 3D printing etc.

<comment><USER5> I thought that, but 3d printing isn't digital printing, CNC manufacture isn't digital manufacture.

3d printing, or additive manufacture seem to be more descriptive, as less likely to be confused with actual digital things?

<comment><USER4> There is an increase in the ratio of work performed in the digital domain, rather than through physical processing, often making things more efficient, easier or cheaper.

Like with the transition from an analog printing process (including shooting, developing etc photos) to the digital one.

<comment><USER5> Yes I understand. But take my CNC example, you design on a computer, a computer cuts out the pieces. If it's mass production, a robot could put the pieces together. This doesn't digitise anymore of the process.

I can understand the use of 'digital' to refer to a novel use of computing, such as your digital photography example. But that's a transitory use, digital photography is now just photography. But again though this doesn't seem to go any further than a CNC or 3d printer.

So I'm here confused as to why they've used that word, because it doesn't seem to add anything, which suggests it was probably the wrong word (or I'm misunderstanding something).

<comment><USER6> The process of choosing the name of a commercial product will (should) always contain some degree of marketing. Another, quite funny example:

[https://en.m.wikipedia.org/wiki/Dynamic_programming#History](https://en.m.wikipedia.org/wiki/Dynamic_programming#History)

More generally, it is quite difficult (out of chemistry at least) to maintain names that describe things to the point that you can "reason" about the chosen symbol. The space is too crowded, etimologies lose their stronghold on meaning when context is added to words, etc. One person chooses with more or less foresight the name of something, and from then on it is just inertia, fashion, and meme evolution that drive names. It is better to not think a lot about why the name of X is Y, out of curiosity, lest you end up frustrated with how random it is.

<comment><USER5> Perhaps you're right :sigh:</comment></comment></comment></comment></comment></comment></comment>


Ask HN: Large File Hosting? - <AUTHOR>

We have a large file (a 1GB Raspberry Pi image) that we want to users to download.  Sourceforge files keep getting corrupted for some customers.  Is there a more reliable site for hosting large downloads? <comment><USER2> Torrents?

<comment><USER3> Along those lines... [http://docs.aws.amazon.com/AmazonS3/latest/dev/S3Torrent.htm...](http://docs.aws.amazon.com/AmazonS3/latest/dev/S3Torrent.html)</comment></comment>


Chip Scale Atomic Clock - <AUTHOR> https://www.microsemi.com/product-directory/clocks-frequency-references/3824-chip-scale-atomic-clock-csac <comment><USER2> Can a clock nerd please ELI5 how this chip differs from a frequency standard like even the old Agilent (then Symmetricon) 5071A, not to speak of more modern variants? For bog-standard businesses that want a on-prem physical backup time standard for IT in case NTP and GPS are not available, would even this chip be overkill?

<comment><USER3> It's cheap, small, and low-power. It's not a metrology standard like the 5071A. Something like this might be used as a more stable reference oscillator for a radio transceiver. Cell towers typically use rubidium frequency standards, this would likely be part of a package to replace those. They'll also likely show up in high-end test equipment (oscilloscopes, spectrum analyzers, etc).

Note that this thing doesn't provide date and time. It provides a 10MHz square wave output and a 1 pulse per second output. You could use it to make a real time clock, but that's not the intended purpose.

</comment></comment><comment><USER4> $1,500 at release in 2011: [http://www.insidegnss.com/node/2446](http://www.insidegnss.com/node/2446)

Not sure if this is an updated version or if there's another reason for this to be relevant at the moment.

<comment><USER5> Looks like it's exactly the same product. Symmetricom, the company that originally designed the CSAC, was acquired by Microsemi in 2013.

</comment></comment><comment><USER6> I've been trying to find out if using one of these in a GNSS receiver would help with accuracy.

Anyone here know?

Clearly the clock on a receiver is well disciplined, but does that mean a better oscillator like this wouldn't help?

<comment><USER7> I suspect compared to rtk its not really worth the hassle. GPS has been specifically designed to not require a highly disciplined clock for the receiver.

<comment><USER8> Yea but time degredation of TCOX etc after losing a signal is a thing

</comment></comment></comment><comment><AUTHOR> Appears the dev kit is $1k at Digikey now.

I just saw someone mention it on FB and I didn't realize such a thing even existed -- it's still expensive (5-10x a good TXCO), but pretty awesome.

<comment><USER9> Note that the dev kit just has a socket; you have to buy the actual clock module separately.

<comment><USER10> The dev kit looks ridiculously overpriced, but this seems to be the norm even with other electronics --- the "evaluation board", which is often not much more than the device itself mounted on a PCB with some other cheap passives, costs more than 10-100x the actual part.

In this case, however, I couldn't find anywhere selling the actual clock module from a quick search, but there are mentions that it's $1500.

</comment></comment></comment><comment><USER11> This has existed for a little while now, but doesn't appear to be available in volumes greater than 'sample' (it is not uncommon for semi-conductor manufacturers to jump the gun a lot on something being actually available).

I haven't tried to order one in a month or two though, so that may have changed.

</comment><comment><USER12> Is this the kind of thing that makes Google's Spanner database possible?

<comment><USER13> One of the things. They use GPS clocks as well.

<comment><USER9> Yeah, even if you have atomic clocks in every datacenter, you need an external reference to synchronize them with each other.

(If you try to calculate the offsets between two clocks by using something like NTP to measure round-trip times over the internet, you're implicitly assuming that packet delays in both directions are symmetrical.)

<comment><USER14> How about syncing them at one physical place, moving while powered using a battery, then hot-deploy in target datacenter?

<comment><USER15> Pretty sure relativity limits the usefulness of this. [http://www.astronomy.ohio- state.edu/~pogge/Ast162/Unit5/gps....](http://www.astronomy.ohio- state.edu/~pogge/Ast162/Unit5/gps.html)

<comment><USER10> It's rather amazing that anyone with a couple of atomic clocks can see the effects of relativity themselves:

[http://leapsecond.com/great2005/](http://leapsecond.com/great2005/)

</comment></comment></comment></comment></comment></comment><comment><USER16> Very cool, didn't know these existed.

Still, it's quite the chip at 35 g mass (compare to an iPhone 8 at 148 g). More like a module or something.

</comment><comment><USER17> What is Microsemi's relation to Microsoft?

Similar logo (well, at least colors are similar to Windows logo), similar font, similar name.

<comment><USER18> Not sure why this was down voted, but this was my first thought too.</comment></comment>


A Photo that can steal your online credentials. - <AUTHOR> http://www.infoworld.com/article/08/08/01/A_photo_that_can_steal_your_online_credentials_1.html

<comment><USER2> If browsers would only stick to browsing then browsing would be safer for the vast majority of people who will never grasp that enabling the execution of remote code always means potentially handing over your data and processor to someone who will benefit by screwing you and everyone you know.

I think the right interface that makes browsing safer by default while allowing users to run code in a separate third-party environment could be part of the answer to <http://ycombinator.com/ideas.html> challenges #2 and #18.</comment>


Escape from Mercator Maps - <AUTHOR> https://mapzen.com/blog/escape-from-mercator <comment><USER2> Did not know about Tangram, seems awesome!

I have a (dumb) question. Why do you use Leaflet? Isn't Tangram the same as Leaflet, or am I missing something here?

A fully WebGL mapping solution is better for performance, but won't using Leaflet HTML5 Canvas defeat this purpose?

It seems to me that, because of this, Mapbox GL will have a better performance than Tangram.

<comment><USER3> Tangram is currently exposed as a Leaflet plugin, but manages its own WebGL Canvas element (which is inserted into Leaflet's DOM hierarchy) and handles all rendering on its own. Leaflet is essentially just serving as the UI/input layer here (map position and viewport info is forwarded to Tangram), and can also be a convenient way to add SVG paths or markers on top, or raster tile layers underneath. It hasn't shown any noticeable performance impact in benchmarks though (this is the same arrangement Mapbox GL uses for its own Leaflet plugin). In the future we'll also expose the Tangram API more directly for use outside of Leaflet though, this was just the quickest way to let people use it via a library they might already be familiar with.

<comment><USER2> Got it, thanks for the explanation!

</comment></comment></comment><comment><USER4> This is how maps should work. They all seem entirely responsive on my 18 month old mobile.</comment>


How can you make 2 and 2 = 5? - <AUTHOR> http://www.quora.com/How-can-you-make-2-%2B-2-5/answer/Natalia-Nezvanova?share=1

<comment><USER2> You use very large values of 2.</comment>


Physicists want to rebuild quantum theory from scratch - <AUTHOR> https://www.wired.com/story/physicists-want-to-rebuild-quantum-theory-from-scratch/ <comment><USER2> All I ask is the headline not be "Physicists want..." but rather "Some physicists want..." A couple of people I've never heard of don't speak of all physicists, in fact most

>[...] just shrug and say we have to live with the fact that quantum mechanics is weird.

In fact, I don't think it's weird, I think it's reality.

<comment><USER3> > In fact, I don't think it's weird, I think it's reality.

At the risk of sounding snarky, I think it's useful to point out that we call it the Standard Model and not the Standard Reality.

Model ≠ Reality, yadda yadda.

We have an immensely useful model of fundamental physics as long as the energies are low enough. However, I feel that we lose a lot by letting that utility stop us from asking epistemic questions of the model itself.

Historically, math has gained a lot by taking such questions seriously, especially in the areas of logic and computation.

<comment><USER4> > Model ≠ Reality, yadda yadda.

I think im a fan of what is described as quantum bayesianism in the article, that quantum mechanics is basically a probabiltiy theory, and this probability theory is required to express our _expectations_ from measurements (quantum mechanics encodes our beliefs about what will be observed given some knowledge about a system) and this is fundamentally a "retreat" because we dont know how to describe the underlying reality. even though calculating the probabilitoes may be very precise.

but i am havig trouble distinguising quantum bayesianism from the standard "model vs reality" fare as the motovation seems very similar

<comment><USER2> The thing I say is "what underlying reality?" If you can't measure it, how can you make a theory out of it or research it (or publish papers on it unless you're a string theorist)? If something is not measurable, then how can you claim it exists or not?

Whenever people ask me these "deep questions" after I tell them I'm a physicist, I tell them that. Science is all about the reality you can measure, not about meaning and what lies "deeper".

<comment><USER4> I take it you are a fan of the copenhagen interpretation of quantum mechanics. its the right answer for making calculations. but how do come up with a new theory? you have to tell a story or envision an underlying reason to justify it. new theories dont come out of nowhere.

this is also like saying the universe in the time of newton consisted of only newtons laws, which is untrue. if the universe _is_ science, then you have to say the universe is changing as science changes. unless the universe really is just made of published journal papers.

</comment></comment></comment></comment></comment><comment><USER5> Problem is, all these redefinitions of quantum mechanics via "quantum logic" or "quantum probability theory", while looking more fundamental and formal than the usual theory based on Schrödinger's equation and Born's rule, do not make it any more intuitive and do not allow anyone to say, "aha! now I get it!". This is because these new foundations themselves are just as unintuitive as quantum mechanics...

[Edit] On the other hand, I find the famous derivation of Schrödinger's equation by Feynman illuminating: [http://fermatslibrary.com/s/feynmans- derivation-of-the-schro...](http://fermatslibrary.com/s/feynmans-derivation- of-the-schrodinger-equation).

</comment><comment><USER6> It will be interesting to see if theoretical physics and computer science converge at all over the next century. Intuitively there's much to suggest that our understanding of computational limits on things (not to mention the idea that the world itself is a simulation of some kind) has analogs in fundamental physical limits (speed of light, quantum noise).

<comment><USER7> I would be interested if anyone knows about the connection between information theoretic entropy and physical entropy.

I think thermodynamics and statistical physics are really interesting. In Parallel Distributed Processing they discuss an extension of the entropy metaphor to learning and neural networks

[https://mitpress.mit.edu/books/parallel-distributed- processi...](https://mitpress.mit.edu/books/parallel-distributed-processing)

<comment><USER8> > I would be interested if anyone knows about the connection between > information theoretic entropy and physical entropy.

I'm pretty sure they're the same, with minor differences in notation and units, unless of course I misunderstand what you mean by "physical entropy". Entropy in physics only got a precise definition once physicists understood that the concept really comes from information theory.

<comment><USER7> erm, I think boltzman figured out entropy for thermo like 100 years before shannon's theory of communication

could be totally wrong though

entropy, complexity, and evolution are words that come to mind

<comment><USER9> Yes, Shannon explicitly stated in Mathematical Theory of Communication that the general form was the same as that in statistical mechanics:

"The form of H will be recognized as that of entropy as defined in certain formulations of statistical mechanics where p_i is the probability of a system being in cell i of its phase space. H is then, for example, the H in Boltzmann's famous H theorem."

where the general form was H = -K\sum p_i * log(p_i)

IIRC, this form is the only kind with the following two properties:

\- chain rule -- H(X,Y) = H(X) + H(Y|X)

\- maximized for the uniform distribution.

<comment><USER7> Could you explain KL divergence and/or cross entropy to me?

from wikipedia: "In mathematical statistics, the Kullback–Leibler divergence is a measure of how one probability distribution diverges from a second expected probability distribution"

Kullback-Leibler divergence is used a lot in the reinforcement learning setting

"In information theory, the cross entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set"

Cross Entropy is used as a cost function in neural networks rather than least squares [http://neuralnetworksanddeeplearning.com/chap3.html#the_cros...](http://neuralnetworksanddeeplearning.com/chap3.html#the_cross- entropy_cost_function)

</comment></comment></comment></comment></comment></comment><comment><USER10> I found this article's explanation of the basic state of physics to be one of the best I've read. I know squat about the field, and so can't say if it was accurate or not, but the layman's explanation of the core problems of quantum theory, the probability and whatnot, was easy to grasp, and made sense.

</comment><comment><USER11> Information theory will hopefully give us deeper insight into the why. If we derive QM from information theory it might come at it from another angle and provide different insights. I had read about people trying this in the past, but not for a while...maybe it proved fruitless or uninteresting.

</comment><comment><USER12> I'm surprised time and time again by the removal of causal ordering from quantum mechanics. It seems to be throwing away so much.

</comment><comment><USER13> So this is a project to create the "Maxwell equations of Quantum Physics"?

</comment><comment><USER14> What if quantum mechanics are the causal basis for spacetime. Then to the observer it might appear as random, because spacetime is a prerequisite for observation. Also for causation, this must imply time moves sideways in the quantum domain. Clearly!

I clearly had one to many to drink. My head hurts.</comment>


Terminal Emulator for Atom Editor - <AUTHOR> https://github.com/f/atom-term2 <comment><USER2> Seems there is another terminal emulator for Atom called Termrk. [https://github.com/romgrk/termrk](https://github.com/romgrk/termrk) Both Term2 and Termrk still work on the latest Atom build.

</comment><comment><USER3> legitimately considering moving to Atom for this. I use sublime text (i like it) but there is no terminal support. I have checked every package out and nothing really beats just using a hotkey on iTerm. If this works well, I would be all over it.

</comment><comment><USER4> I don't get it.

If you think Atom is a better window manager than whatever window manager you're running Atom under, why not just switch to a different window manager?

<comment><USER5> OS X users don't have that option unfortunately.</comment></comment>


Count trailing zeros, revisited - <AUTHOR>

I don't know if this is old and silly, but I came across common lisp code (here) to count trailing zeroes, an application I actually needed.
;; mysterious program depends on value of 2^n mod 37.. since 37 is ;; relatively prime to 2^n for n=0 to 32.
;; reference https://graphics.stanford.edu/~seander/bithacks.html
;; it's only 4 "real" lines if you strip off the declarations ;; plus the data. I haven't done any timings.
(defvar ctzmap(make-array 37 :element-type 'fixnum 			      :initial-contents
  '(32 0 1 26 2 23 27 0 3 16 24 30 28 11 0 13 4
  7 17 0 25 22 31 15 29 10 12 6 0 21 14 9 5
    20 8 19 18)))
;; if z is too big, just look at the trailing 32 bits.. if they're all zero,
;; shift z and ;; keep looking
(defun ctz(z) ;; works in blocks of 32 bits

    (declare (integer z) 	     (type (simple-array fixnum 37) ctzmap) 	     (optimize (speed 3) (safety 0)))

  (let* ((zz (logand z #.(1-(expt 2 32) ))) 	 (sofar(aref ctzmap (the fixnum (mod (logand zz (- zz)) 37)))))

    (declare (integer zz) (fixnum sofar))

    (if (= sofar 32) (+ sofar (the fixnum (ctz (ash z -32)))) sofar))))

 Cheers Richard Fateman   firstname.lastname@example.com.  ask if you want more than 2000 chars. (UC Berkeley, retired CS Professor) <comment><USER2> You've probably see this as well: [https://graphics.stanford.edu/~seander/bithacks.html](https://graphics.stanford.edu/~seander/bithacks.html)</comment>