Show HN: Covid-19 Patient Review Tool for Hospitals Open Sourced - <AUTHOR> https://github.com/smarterdx/covid-19 <comment><AUTHOR> Co-founder of SmarterDx here.

SmarterDx is releasing a tool that will help doctors and hospitals more rapidly review the relevant data for their COVID patients. We hope this will enable better triage, more rapid review and the ability to identify trends. We are working with our hospital partners to get this implemented and make improvements based on their feedback. However, two of our co-founders are physicians in major NYC hospitals and so have extremely limited bandwidth right now. We'd love some support from the community in improving this project. This is being implemented in NYC hospitals that are operating way beyond their max and could use the support.</comment>


Is Exercise Bad for Your Teeth? - <AUTHOR> http://well.blogs.nytimes.com/2014/09/24/is-exercise-bad-for-your-teeth/?_php=true&_type=blogs&_r=0

<comment><USER2> Nope.The last three paragraphs summarize: Small limited test, only _maybe_ if you train more than nine hours a week.

But interesting that the composition of your saliva changes when you're in an endurance situation. Why?

<comment><USER3> It is probably related to the lactic acid buildup during exercise.

<comment><USER4> …and general changes in water-availability: sweating, accelerated respiration- evaporation. And redirection of blood-flow. So I'd say hardly more surprising than the idea that saliva volume and composition might change while eating.

</comment></comment></comment><comment><USER5> My uncle who is a dentist (also same name as me) told me YEARS ago that most of my cavities were caused by gatorade. Gatorade I was consuming while playing soccer would dry up in my mouth and cause bacteria to eat away at the enamel. I stopped drinking gatorade and other sports drinks, and haven't had a cavity since.

<comment><USER6> i bought gatorade once. And it is still perplexing to me how people can drink that and similar synthetic stuff which looks and feels like chemistry plant waste water. If not teeth, it would definitely hit your gastroenterological tract just like "Drano". The power of "magic potions" advertisement is astonishing - "Because electrolytes". Which reminds of the quote from the "Idiocracy" ( [https://www.youtube.com/watch?v=g5Q-yNNu- tM](https://www.youtube.com/watch?v=g5Q-yNNu-tM) ) :

[cabinet has been debating putting water on the plants instead of Brawndo]

Pvt. Joe Bowers: What _are_ these electrolytes? Do you even know?

Secretary of State: They're... what they use to make Brawndo!

Pvt. Joe Bowers: But _why_ do they use them to make Brawndo?

Secretary of Defense: [raises hand after a pause] Because Brawndo's got electrolytes.

<comment><USER5> That was a great, and sadly under-watched movie.

</comment></comment></comment><comment> guard-of-terra I'm waiting for genetic engineering to arrive in order to fix teeth problem for once instead of caring about it. Some people have perfect teeth, I mean - let's just copy the good parts.

This and eyes - another offenders.

<comment><USER7> The interesting thing about eyesight is that the stereotype of the bookworm with glasses is actually true. Reading close-up does actually change the shape of your eyes, necessitating corrective lenses to fix this change.

<comment><USER8> Most stereotypes are actually true, most of the time.</comment></comment></comment>


The Long Chase - <AUTHOR> http://www.lightspeedmagazine.com/fiction/the-long-chase/ <comment><USER2> Site appears to be down. Archived at:

[https://web.archive.org/web/20160313135722/http://www.lights...](https://web.archive.org/web/20160313135722/http://www.lightspeedmagazine.com/fiction/the- long-chase/)

</comment><comment><USER3> If you liked this, you may like "House of Suns" by Alastair Reynolds. There's one chase scene that takes about 60,000 years.

<comment><USER4> "The Forever War" was my first introduction to this form of time stretching. I actively search out authors who incorporate it into their storytelling, so I was _thrilled_ when I stumbled across Alastair Reynolds' writing.

I think I have all of his books now.

<comment><USER5> Vinge's "A Deepness in the Sky".

</comment></comment></comment><comment><USER6> Fantastic!

</comment><comment><USER7> Excellent!</comment>


HPC is dying, and MPI is killing it - <AUTHOR> http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/ <comment><USER2> I came into the essay with suspicion. A map-reduce system like Hadoop isn't a good fit for HPC problems, and I thought it would argue that MPI is old => it's stuck in the past.

Instead, and to my joy, it was a well-reasoned essay with good, solid points.

My only quibble is that Charm++ is not "a framework for particle simulation methods". While the molecular dynamics program NAMD has been using it for 20 years, which is why I know of Charm++, it wasn't designed specifically for particle simulation methods, nor is restricted to that topic. Quoting from [http://charm.cs.illinois.edu/newPapers/08-09/paper.pdf](http://charm.cs.illinois.edu/newPapers/08-09/paper.pdf) :

> NAMD, from our oldest collaboration, is a program for biomolecular modeling > [2]; OpenAtom is a Car-Parinello MD program used for simulation of > electronic structure (in nanomaterials, as well as biophysics) [3]; ChaNGa, > an astronomy code [13]; and RocStar, a code for simulating solid-propellant > rockets, such as those in the space shuttle solid rocket booster

<comment><AUTHOR> Thanks! I'm probably mischaracterizing Charm++ a bit, because I'm most familiar with it in particle context (OpenAtom, ChaNGa, NAMD). I guess it's probably particularly used in that context just because it's so good with very fine-grained distributions of work units. I'll edit that line in the article.

<comment> semi-extrinsic Another nitpick: 128 cores equals ~ 4 nodes only if your problem is not bound by memory bandwidth. If it is, 128 cores equals ~ 16 cores, and then the interconnect matters a lot.

Great writeup though. I do think we need to get more people into the mindset that MPI won't be the standard in 10 years, otherwise it will still be the standard in 10 years.

<comment><AUTHOR> "won't be the standard in 10 years, otherwise it will still be the standard in 10 years." \-- nicely said.

</comment></comment></comment></comment><comment><USER3> MPI is Message Passing Interface, for those who don't know. Author never explicitly says what MPI stands for.

[https://en.wikipedia.org/wiki/Message_Passing_Interface](https://en.wikipedia.org/wiki/Message_Passing_Interface)

<comment><USER4> "MPI, the Message Passing Interface, began as a needed standardization above a dizzying array of..."

The capitalization and apposition makes it pretty explicit.

<comment><USER3> I triple checked before posting this, looks like someone edited the blog ;)

</comment></comment></comment><comment><USER5> I've heard there is a new effort being led by Torsten Hoefler to modernize MPI and address a number of the issues mentioned in this article.

[http://htor.inf.ethz.ch/](http://htor.inf.ethz.ch/)

I was at a talk of his last year and there are a number of fault tolerant MP algorithms being drawn in. MPI hasn't been updated in ages, I don't think that necessarily means we need to ditch it, it just means the standard needs to be modernized. I don't feel very strongly about this since working with MPI is a huge pain in the ass and it seems like the challenge of modernizing it is just gargantuan.

Also I'm not familiar with spark, but isn't Chapel a decade old at this point and barely works at all? I tried their compiler last summer and it took 5 minutes to compile hello world, hopefully its improving.

<comment><AUTHOR> Certainly Prof Hoefler has done a lot of work driving the design of updated remote-memory access for of MPI-3, and any further progress would be welcomed; but I don't think any modernizing of MPI can fix the basic problem. At the end of the day, it's just too low level for application developers, while being too high-level for tool developers. There are parts of MPI which don't share this problem so much - the collective operations, and especially MPI-IO; but the disconnect between what people need to build either tools or scientific applications and what MPI provides just seems too great.

For Chapel, it depends on what you count; it very heavily borrows from ZPL, which is much older, but Chapel itself was only released in 2009. It is already competitive with MPI in performance in simple cases, while operating at a much higher level of abstraction. Whether Chapel, or Spark, are the right answers in the long term, I don't know; but there's a tonne of other options out there that are worth exploring.

<comment><USER5> Again I'm not sure if I agree or disagree with this. My hatred of MPI is only outweighed by the fact that I can use it... and my code works.

I think a large part of the inertia behind MPI is legacy code. Often the most complex part of HPC scientific codes is the parallel portion and the abstractions required to perform them (halo decomposition etc). I can't imagine there are too many grad students out there who are eager to re-write a scientific code in a new language that is unproven and requires developing a skill set that is not yet useful in industry (who in industry has ever heard of Chapel or Spark??). Not to mention that re-writing legacy codes means you're delaying from getting results. Its just a terrible situation to be in.

<comment><AUTHOR> Oh, sure. I don't think anyone should start rewriting old codes; but as new projects start, I think we have a lot more options out there than we did 10 years ago, and it's worth looking closely at them before starting, rather than defaulting to something. _Especially_ since, once you start, you're probably pretty much locked into whatever you chose for a decade or so.

<comment><USER6> So, say you wanted to write a weather model, or engineering fluid mechanics model. Which options (besides MPI) you would look at?

<comment><AUTHOR> Chapel has been used for incompressible moving-grid fluid dynamics, so it's certainly feasible. For that problem the result was ~33% the lines of code of the MPI version. There is a performance hit, but the issues are largely understood; if (say) a meteorological centre were to put its weight behind it, a lot of things could get done.

It's also pretty easy to see how UPC or co-array fortran (which is part of the standard now, so isn't going anywhere any time soon) would work. They'd fall closer to MPI in complexity and performance.

You couldn't plausibly do big 3d simulations in Spark today; that's way outside of what it was designed for. Now _analysing_ the results, esp of a suite of runs, that might be interesting.

</comment></comment></comment></comment></comment></comment><comment><USER7> The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI.

I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

<comment><USER8> I spent a few months learning (modern) Fortran a year or two ago. My chief obstacle was the difficulty involved in finding modern tutorials. I don't want to have to read tutorials written in 1994 whose focus is getting people used to F77 up to speed. I've yet to find a tutorial that approached teaching F08 as if it was a new language, which is what I feel is needed.

Even in F08 there's a lot of backwards-compatibility cruft still left in the language, too. The IO model still provides very little abstraction and is based on tape drives. You can/have to "rewind" files. There are obscure "unit descriptors" that manifest themselves as integer literals in most code posted online which makes it a chore to learn from. As far as I can tell there is no functionality that approximates the behaviour of C++'s streams.

It's fast as hell, and the GNU compiler is mature and well-developed, but Fortran remains a horrid language for doing any sort of interactive programming. It's best used if you just give it some arguments, let it run free, and then have it return some object or value that a more sane language can then interpret and present to the user for a decision.

There is little reason to learn a language where the only sane choice for doing input/output involves calling your Fortran module from a python script and letting the python handle i/o.

<comment><USER9> "horrid for interactive" is ironic, since interactive and HPC are pretty much disjoint. (well, viz...) From an HPC perspective, Fortran IO should be performed by HDF5...

<comment><USER8> I think you're reinforcing my point.

Fortran might be used more widely, like C++ is, if it wasn't so awful for doing things other than shitting out numbers at insane speeds.

</comment></comment></comment></comment><comment><USER10> Yes, "high performance computing" is dying. There's no commercial market for it. Check the list of the top 500 supercomputers in the world.[1] The top 10 are all Government operations. In the top 25, there are a few oil companies, mostly running big arrays of Intel Xeons.

CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops special supercomputing CPUs any more. The market is tiny. Old supercomputer guys reminisce about the glory days when IBM, Cray, Control Data, and UNIVAC devoted their best R&D efforts to supercomputers. That ended 30 years ago.

Supercomputers have poor price-performance. Grosch's Law [2] stopped working a long time ago. Maximum price/performance today is achieved with racks of midrange CPUs, which is why that's what every commercial data center has. Now everybody has to deal with clusters of machines. So cluster interconnection has become mainstream, not the province of supercomputing.

[1] [http://www.top500.org/list/2014/11/](http://www.top500.org/list/2014/11/) [2] [http://en.wikipedia.org/wiki/Grosch%27s_law](http://en.wikipedia.org/wiki/Grosch%27s_law)

<comment><USER11> Did you RTFA? It's about MPI:

"MPI is a language-independent communications protocol used to program parallel computers."

Runs fine on commodity clusters.

<comment><USER12> >Runs fine on commodity clusters.

Kind of.... For simple, low communication jobs this is true. But when you start trying to find the eigenvectors of a large sparse matrix, communication becomes your bottleneck, at which point MPI on commodity clusters (those without a really fancy interconnect) "works", but not fast enough to be useful.

<comment><USER13> I don't think "really fancy interconnects" makes a cluster not commodity, since the original post in this thread is about supercomputer processors. You can put Infiniband in any system with a PCI-e 3.0 bus.

</comment></comment></comment></comment><comment><USER6> in High Performance Computing, there is (1) 3-dimensional simulations (weather, fluid dynamics, structural mechanics, all kinds of physics simulations, like magnetic storms in space or nuclear reactors etc.) and then there is (2) everything else, like data mining, machine learning, genomics etc.

Some of the sparse matrix computations in structural mechanics and in some machine learning algorithms have some overlap. But mostly, group 2 has little reason to be interested in what group 1 is doing.

Now, group 2 obviously has more modern tools than the 3d-simulation community, because machine learning came to common use much later that numerical fluid mechanics.

But do 3d-simulation people also have much reason to be interested in what the machine learning people are doing?

The "machine learning / big data" people are probably not doing anything that makes a weather prediction model to run faster? Or are they?

<comment><USER14> They are doing it (interesting things) for lower capex and lower development costs. On opex, good for operations, bad for power consumption (relatively).

In terms of absolute performance HPC is absolutely faster. In terms of bangs for bucks, Big Data is hands down faster. Also in terms of accessibility Big Data is hugely easier - I can build you a 100 core big data system for $300k

<comment><USER6> But your Big Data system is only good for Big Data. If I want to run a weather prediction model, it is not going to help.

My point is, the big data and the physics simulation people probably do not have a lot of common interests - besides using large amounts of computing power.

<comment><USER5> There are big data people running on supercomputers too. I know there are people writing custom asynch job managers to handle big data type problems because the top supercomputers have low memory latency.

Also I think the dichotomy you're looking for is IO bound vs CPU bound problems. Although certainly there are a plethora of different kinds of IO bound problems (asynch vs synch or disk bound vs memory bound vs cache bound).

</comment></comment></comment></comment><comment><USER15> The article briefly mentions Erlang with its focus on message passing, and the focus on being fault tolerant should be another benefit. I could find a few mentions of Erlang used for simulations, such as here[1], but I am curious whether there is much actual usage in scientific computing, or whether there are some problems in practice.

[1] [https://books.google.com/books?id=p0h9vAb1m7IC&pg=PA365#v=on...](https://books.google.com/books?id=p0h9vAb1m7IC&pg=PA365#v=onepage&q&f=false)

<comment><USER7> Erlang itself is not good for the type of numerical computation typical done in HPC. It is amazing as a backend, and there are several ways it can call code written in other languages through the use of ports or NIFS, but if you try to do massive number-crunching using its own libraries then you're going to be unhappy with the results.

<comment><USER15> How about in simulations where a large number of cellular automata are interacting with each other, but individually only carrying out simple computations?

<comment><USER7> You certainly can do that. In fact, I believe I saw an example of that in a tutorial somewhere talking about what Erlang/Elixir would be good for (I include Elixir because its the same VM controlling processes underneath, but a "nicer" syntax on top). Now, would it be the best language for that? Depends. If you're looking just for speed and massive computations, then no. In the end, C++ pretty much rules everything in that regard (except for maybe Fortran in some instances). But, I would say that it might be more fun to set it up in Erlang/Elixir. And you could pretty easily expand the whole thing just by adding more processors and simply telling it to spawn more processes (the Erlang VM is pretty awesome). I would almost say that an Erlang version of it would feel more life-like. You could probably experiment with it on the fly more easily, too. Kill off a process here or there and see what happens, etc.

<comment><USER15> I actually started going through _Programming Elixir_ this past week because I was thinking it would eventually be a fun way to explore models through ad hoc substitution of rules and exogenous conditions. There are a lot of models available in NetLogo, but it doesn't look like it would be very powerful in terms of running larger scale simulations, and the scope of the language's usefulness is pretty limited compared to that of a general purpose language.

While I have been using C++/Rcpp to extend R as an occasional time saver (for analysis rather than simulations), it's only been little snippets written badly.

Anyway, since its so easy to come up with a too long list of technologies to learn, then only scratch the surface, hearing that Erlang/Elixir isn't the completely wrong tool is helpful.

<comment><USER7> As an example, here's the game of life written in Elixir:

[https://github.com/spikegrobstein/elixir- conway](https://github.com/spikegrobstein/elixir-conway)

How much better that is than a C++ version, I don't know. But if you want to get better at C++ and give it a try, forget what you know and read this book: [http://www.amazon.com/Programming-Principles-Practice- Using-...](http://www.amazon.com/Programming-Principles-Practice- Using-2nd/dp/0321992784/)

</comment></comment></comment></comment></comment></comment><comment><USER16> I am trying to get into distributed computing so this article is particularly interesting to me. I may be mistaken so please excuse my naivety if my points are off marks. I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized, for example infiniband or bus between CPUs. One use of MPI is to manage distributed memory tasks between different physical CPUs while threads run on multiple cores of same CPU. Spark, I believe, doesn't handle cases like this well because JVM hides low level details. I have read papers that propose to layer MPI over RDMA rather than expose a flat memory model, which came as a surprise to me but it shows the flexibility of MPI. One thing unclear to me what performance we can expect from MPI when we use commodity network gears, and how it compares to Spark. The article is absolutely correct MPI leaves robustness to user and that is today an oversight.

Modern Hadoop ecosystem is designed for different workload from MPI's. It emphasizes co-localizing date and computation, seamless robustness,and trades off raw power for simple programmingmodels. MapReduce turns out too simple, so Spark implements graph execution, which is nothing new to HPC. As far I know Spark's authors don't believe it is ready for distributed numerical linear algebra yet. But a counterpoint is that I am seeing machine learning libraries using Spark, so perhaps things are improving.

One thing I have learnt today is that MPI isn't gaining popularity. I just have a hard time picturing a JVM language in overall control in HPC where precise control of memory is paramount to performance.

<comment><AUTHOR> I agree that languages that rely on tracing GC seem like they're fundamentally at a disadvantage when it comes to pushing the envelope of single-node performance; the best article I've read arguing this was actually in the context of mobile games, rather than HPC, but I can't for the life of me find the article now.

I don't know if Spark itself is the right way forward; but it's an example of a very productive high-level language for certain forms of distributed memory computing. And some of these issues - like the JVM - aren't fundamental to Spark's approach; there's no inherent reason why something similar couldn't be built based on C++ or the like.

<comment><USER17> I'd be very wary of criticising Spark based on the technologies it's built on.

In particular, a decent garbage collector will give you performance dependent on the number of live objects (typically low) and not on the number of allocations and deallocations, as you might see in a non-garbage collected language. This gives great allocation performance and reduces overheads.

The disadvantages can be (potentially long) GC pauses and higher overall memory requirements, but in practice this isn't usually a problem for non- interactive systems.

Of course, if you do have a device with low memory and low tolerance for GC pauses (i.e. mobile gaming) there might be a problem.

The main disadvantage seems to be less predictable performance; which could be a problem in domains which require good timing performance, but that's not really Spark's problem.

A GC'd language is also generally easier to program in, since one doesn't have to (in general) worry about memory management, so it's generally a lot easier to program very large systems with lots of moving parts.

</comment></comment></comment><comment><USER18> Too bad he didn't talk about GPGPU killing MPI too or not. I don't know enough to say.

I'm not familiar with the HPC space but I thought a lot of new work, at least in machine learning, was migrating to GPGPU instead of traditional CPUs. The compute per $ or per watt payoff is too large to ignore.

<comment><USER19> I wouldn't say GPUs are killing off MPI. You still need some way to pass data between nodes/GPUs (most of these datasets can't fit within a single GPU). What you are seeing though, is less and less use of the CPU. If code developers decide to use GPUs, they try and move their data onto the GPU and keep it there for as long as possible (data movement across PCIe is a killer for performance). ORNL's new machine Sumiit [1] will have 1/4 the nodes of their current machine, Titan, but multiple GPUs per node. Thus emphasizing the move away from CPUs and onto GPUs. Again though, there are still 3K nodes and you need some efficient way to pass data between those nodes.

[1] [https://www.olcf.ornl.gov/summit/](https://www.olcf.ornl.gov/summit/)

</comment></comment><comment><USER20> This made me feel old, as I remember the days when we got to learn PVM at distributed computing classes and MPI was presented as something that some people were kind of working on.

<comment><USER2> If it makes you feel better, I started with PVM 2.

</comment></comment><comment><USER21> I used MPI (Message Passing Interface...) back in the day.

But it was a pain, especially since our code was a mix of c (which was easy to mpi) and ada (not so easy). Its pretty low level stuff (I think we used Open MPI). All the nodes need to have MPI set up and configured, fine if you have a team willing to do it but these days....

mpiexec -n 10 myprocess

I think we liked it because the processes would be put to sleep by the mpi daemon until a message arrived. You can sleep and wait for a message with sockets now I think. Its been a while since I've used the unix IPC (Interprocess communication).

I don't think I'll miss it.

</comment><comment><USER22> This is a good article.

I built a tiny cluster in my basement (a prototype) and looked at MPI and decided that it was way to complicated, so I just built something that pushes the essential bits, pretty much without abstraction, to the nodes and was done. The cluster is a very specific solution, so I felt justified in not looking at MPI. And now the decision feels even more justified.

</comment><comment><USER23> John: You make some solid points. Most though seem to more support the idea that more research investment/emphasis is badly needed for HPC programming models. From an application perspective, one sees such a dominant reliance on MPI+X primarily because typically the value proposition just isn't there yet for alternatives (at least in the areas where I work, where we have undertaken new developments recently and done fairly exhaustive evaluations of available options). Though the coding can be somewhat tedious and rigid, in the end these shortcomings have been outweighed by the typical practical considerations -- low risk, extremely portable, tight control over locality and thus performance, etc. It's obviously not all or nothing - as you say we could choose something even lower level and possibly get better performance, but when seen from the perspective of the near and mid-term application goals, it's hard to make a different choice unless explicitly tasked with doing so.

</comment><comment><USER24> Thank goodness someone said it. I get tired of using ancient software that's older than me for relatively simply tasks that should have long ago been coded into a higher level of abstraction. I programmed a pair-correlation function calculation routine using MPI once -- yech.

Fortran, MPI, even C to an extent -- can we please move on? I don't understand why the scientific community is so reluctant to embrace change. It seriously doesn't take that long to learn a new language or a platform like Github (yeah, that's still considered "new" in the scientific community), and the time investment more than pays itself back many times over.

<comment><USER25> > I don't understand why the scientific community is so reluctant to embrace > change.

Let's assume the opposite were true, and it was _fast_ to embrace change. How much time would be spent on this change -- relearning, rewriting, refighting old bugs -- vs. actual work done?

 _Change is overhead_. You do as little of it as necessary, and only when _not changing_ starts costing a lot. Which means you change, but slowly.

As to Fortran, it will go away when something better comes along, and then it will do so slowly, for the aforementioned reasons.

</comment></comment><comment><USER26> The lion in the room is that the DOE National Laboratories have a huge amount of code tied up in MPI and they continue to spend millions of dollars both on hardware and software to support this infrastructure. If you look at the top 500 list:

[http://www.top500.org/lists/2014/11/](http://www.top500.org/lists/2014/11/)

Four out the ten computers are owned by DOE. That's a pretty significant investment, so they're going to be reluctant to change over to a different system. And, to be clear, a different software setup could be used on these systems, but they were almost certainly purchased with the idea that their existing MPI codes would work well on them. Hell, MPICH was partially authored by Argonne:

[http://www.mcs.anl.gov/project/mpich-high-performance- portab...](http://www.mcs.anl.gov/project/mpich-high-performance-portable- implementation-mpi)

so they've a vested interest in seeing this community stay consistent.

Now, on the technical merits, is it possible to do better? Of course. That being said, part of the reason that DOE invested so heavily in this infrastructure is that they often solve physics based problems based on PDE formulations. Here, we're basically using either a finite element, finite difference, or finite volume based method and it turns out that there's quite a bit of experience writing these codes with MPI. Certainly, GPUs have made a big impact on things like finite difference codes, but you still have to distribute data for these problems across a cluster of computers because they require too much memory to store locally. Right now, this can be done in a moderately straight forward way with MPI. Well, more specifically, people end up using DOE libraries like PETSc or Trilinos to do this for them and they're based on MPI. It's not perfect, but it works and scales well. Thus far, I've not seen anything that improves upon this enough to convince these teams to abandon their MPI infrastructure.

Again, this is not to say that this setup is perfect. I also believe that this setup has caused a certain amount of stagnation (read huge amount) in the HPC community and that's bad. However, in order to convince DOE that there's something better than MPI, someone has to put together some scalable codes that vastly outperform (or are vastly easier to use, code, or maintain) the problems that they care about. Very specifically, these are PDE discretizations of continuum mechanics based problems using either finite different, finite element, or finite volume methods in 3D. The 1-D diffusion problem in the article is nice, but 3-D is a pain in the ass, everyone knows it, and you can not get even a casual glance shy of 3-D problems. That sucks and is not fair, but that's the reality of the community.

By the way, the oil industry basically mirrors the sentiment of DOE as well. They're huge consumers of the same technology and the same sort of problems. If someone is curious, check out reverse time migration or full wave inversion. There are billions of dollars tied up in these two problems and they have a huge amount of MPI code. If someone can solve these problems better using a new technology, there's a huge amount of money in it. So far, no one has done it because that's a huge investment and hard.

</comment><comment><USER27> The sad thing is that MPI-3 doesn't have wide acceptance yet. A lot of organizations mainly use MPI-1/2\. While using RHEL 6.x. Shoutout to red hat for maintaining ancient software

</comment><comment><USER28> W-why are they using that picture at the start when their source is an essay on exactly the way they're misusing it?

</comment><comment><USER29> This happened with Smalltalk.

I was a Smalltalk coder. I thought it was the best thing since sliced bread. It has always been clear to me that Smalltalk is far superior to Java.

I left the company after a little while, to do C++ graphics. I later heard that my former employer rewrote their Smalltalk application in Java.

Now no one uses Smalltalk anymore. While Objective-C is based on Smalltalk, Smalltalk was far easier to use, however lots of people use Objective-C. No one uses Smalltalk.

How could it have been different? My friend Kurt Thames once said that "Smalltalk is the way object-oriented programming SHOULD be done." I have always agreed with that.

But when new methods (!) of OOP arose, all the Smalltalk crowd did was gripe about how Smalltalk was far better than Java or Objective-C.

<comment><USER30> What "new methods" are you thinking of?

What let Java get ahead of Smalltalk for me personally, as someone getting into programming in 1996, was that i could write it in the text editor i already had, compile it with a compiler i could get for free, and then post the source code on Geocities (actually, Xoom - remember that?) to share with others.

Whereas when i tried to get into Smalltalk, the first thing i had to do was learn my way around this wacky environment with its strange class browser and ultra-retro window manager, and get my head around the fact that my source code wasn't anywhere particular, and yet was everywhere, and that if i wanted to share your code, i had to somehow "file out", and then hope that my internet friends could successfully "file in" to their own potentially modified images. Once i'd got hold of the tools at all, that is.

Which is not to say that the Smalltalk environment was not better than Notepad/DOS box/javac, because of course it was. It just didn't lend itself to adoption and spread nearly as well. It was a tool for masters, with affordance for apprentices.

Also, Java had pretty good networking right in the standard library, and networking was really exciting in 1996.

<comment><USER29> I'd say that your ability to write Java source in any text editor at all was, all by itself, a new method of doing Object-Oriented Programming.

We used Visual Smalltalk Enterprise. It had the cool feature that, at the end of the workday, I could make what amounted to a core dump, then the following morning I would load my core dump into a running program, and there would be all my open windows with the cursors in the right places in the source documents and so on.

That was quite cool I really enjoyed it, however that environment was profoundly non-portable. I expect that much of the success of Java as opposed to Smalltalk was the simple ability one had to post a tarball full of source code on one's FTP site.

<comment><USER30> That wasn't new, though. C++ had that. Eiffel had that. Every programming language that isn't Smalltalk has that.

<comment><USER29> I'm not dead certain but I think smalltalk may have come before C++. For sure it was in widespread use before C++ was in widespread use.

<comment><USER2> Smalltalk started in '71 and was used for research purposes in '72\. Smalltalk '80 was the definitive version for most. This means that Smalltalk slightly predates C. By comparison, C++ didn't start until 1983.

You wrote "your ability to write Java source in any text editor at all was, all by itself, a new method of doing Object-Oriented Programming".

Simula, considered the first OO language, is text based in the same way that Java is, and not image based like Smalltalk. Since Smalltalk is inspired by Simula, I do not believe one can say using any text editor is a new method of doing OO programming.</comment></comment></comment></comment></comment></comment>


Dynamic Programming for Technical Interviews - <AUTHOR> https://blogarithms.github.io/articles/2019-03/cracking-dp-part-one <comment><USER2> > problems on DP are pretty standard in most product-company-based hiring > challenges

This is sad and a little surprising to me. I've always thought of DP problems as being obviously terrible interview questions, even among the different types of algo questions (which are already heavily criticized). Candidates who have learned DP in school usually find them really easy, and candidates who haven't usually don't even have a chance at solving them, so really they just test whether the person has learned DP, rather than other algo problems where you can at least try to claim that it's a proxy for "general problem solving ability". And DP comes up almost never in the real world (and when it does come up, you're often still better off taking a heuristic/inexact approach), so testing if the candidate knows DP is also almost completely useless.

If you're an interviewer at a company that asks DP questions, please reconsider. There are almost certainly alternatives that are more fair and higher-signal.

<comment><USER3> "and candidates who haven't usually don't even have a chance at solving them"

I am one of those candidates, and I don't know why it is called Dynamic programming. To me a vey naive understanding of DP is this - it's just a simple cache mechanism to store intermediary results so that you don't have to recompute them and waste resources.

In the real world we always think about and do such optimizations, be it I/O, disk access or db access. I would love to understand how DP is any different.

<comment><USER4> In my "real world" we normally don't care about things like big-O complexity. We worry about doing dumb things and not taking more time than we have available. I'm not saying big-O is useless or CS wizards are never helpful. It's just that you need one or two of them for a large team of normies, IME.

I have a problem with this notion that knowledge of algorithms is required to be a good engineer though. Case in point: Senior algorithm nerd on my project is going nuts over algorithmic complexity and delaying an early delivery to another team so they can begin using our new feature. In our case, n is fairly small no matter what, so 2n vs n^2 really doesn't matter to us. The code he's optimizing isn't even the long pole in the tent. It calls other routines which take 2/3 of the total time anyway. We could just deliver now and improve later when we have real world data on how people want to use our feature, but nope, we're wasting time on endless rewrites in search of perfection which may not matter if the other team can't deploy our stuff in time.

<comment><USER5> >> Senior algorithm nerd on my project is going nuts over algorithmic complexity

This is me, but luckily where I work I have people who can keep me in check because we generally do design reviews before anything big is built.

However, I have been in situations at previous companies where big(o) was ignored to take short cuts up front, because the "data was small" and suddenly scaling to even just 100 users starts to break things because of poor design decisions when it gets into production.

I guess the lesson here is more the importance of design reviews. Also n^2 is HUGE even for small data if there is IO or api calls involved. Any public api you provide, that is n^2 is not a good idea because you never know who may end up using it for what.

<comment><USER4> > if there is IO or api calls involved

Right. In my case, the operation was an extra memory comparison. For something already in the cache.

Sure, constraints can change and your assumptions about n<10k may prove unwise, but that's our call to make as engineers. YAGNI. If you know n is never going to grow, then why waste time on it? We're not paid to write pristine code. We're paid to solve problems while hopefully not creating new ones. Pragmatism and all that.

</comment></comment></comment></comment></comment><comment><USER6> This is a comment to demonstrate the differences between DP and memoized recursion to people in the sibling comments.

When I was learning DP vs Memoization I thought Floyd-Washall algorithm to find shortest path length between all pairs of nodes in a graph is a good example of DP that wouldn't work the same way with memoization.

In FW algorithm, because of the order of filling up the table and discarding old values on the go, we are able to run the algorithm only using O(n^2) memory, but if we were to run it in a memoized recursive fashion, I would guess you will have to do with O(n^3) memory.

Another example is when a recurrence in DP only depends on the previous row (if we are going row by row) and you can only keep around O(n) memory swapping two rows representing the current row and the previous one. In a recursive black-boxy memoization method you will have to do with a full O(n^2) memory usage.

Finally, there is a technique to do DP on a table using O(n) memory vs O(n^2) and recovering the optimal path by recursively splitting the table into halves and only storing O(1) rows at a time. This technique is more complicated to explain in an HN comment tho.

Update: forgot the simplest example: fibonacci numbers. In the top-down approach, you would memoize results based on the index of the fib number, so you would need an array of results caching O(n) values. But if you build it up bottom-up, you can get away with only using two variables: previous and current and the do something like:

        prev, cur = cur, prev + cur

<comment><USER7> That is interesting but I still don't totally get how DP is different from simple memoization.

Your fib example can be expressed as corecursion, and in fact, its an example often used to explain it.

      val fibsViaUnfold =
          unfold((res0, res1)) { case (f0, f1) => Some((f0, (f1, f0 + f1))) }

        fibsViaUnfold.take(7).toList shouldBe List(0, 1, 1, 2, 3, 5, 8)

That's scala, but should work anywhere where we can produce a lazy stream of values.

Here is a python version from wikipedia

      def nth_factorial(k):
        n, f = 0, 1
        while n < k:
            n, f = n + 1, f * (n + 1)
        yield f

Is DP a techique to arrive at the corecursive solution?

<comment><USER8> There's a "simple memoization" version of fibonacci that takes more space than `fibsViaUnfold`. I think the term "simple memoization" (and equivalents used in these comments) are a bit too imprecise to be useful.

</comment></comment></comment><comment><USER9> Dynamic Programming and memoization are definitely related techniques, but they are emphatically _not_ the same. Memoization is a black-box approach that can be applied to a generic recursive top-down, depth-first algorithm. Dynamic Programming is about rewriting the recursive top-down algorithm in a bottom- up, breadth-first manner.

Shriram Krishnamurthi explains it best:

[https://blog.racket-lang.org/2012/08/dynamic-programming- ver...](https://blog.racket-lang.org/2012/08/dynamic-programming-versus- memoization.html)

<comment><USER10> This is one definition, but I don't think it's the common one. The more common definition is that dynamic programming refers to solving a complicated problem by breaking it up into simpler overlapping subproblems that can be solved independently.

Solving it with recursion/memoization vs. bottom-up is merely an implementation detail, while DP refers to a class of algorithms.

EDIT: Corrected definition of DP.

<comment><USER11> > The more common definition is that dynamic programming refers to solving a > complicated problem by breaking it up into simpler subproblems that can be > solved independently

I dont think that's sufficient? I thought DP also implies you actually reuse the answers from subproblems.

From [https://en.m.wikipedia.org/wiki/Dynamic_programming](https://en.m.wikipedia.org/wiki/Dynamic_programming)

"There are two key attributes that a problem must have in order for dynamic programming to be applicable: optimal substructure and overlapping sub- problems. If a problem can be solved by combining optimal solutions to non- overlapping sub-problems, the strategy is called "divide and conquer" instead"

<comment><USER10> Yeah, you're right. The subproblems must overlap.

</comment></comment></comment></comment><comment><USER12> I have always felt that dynamic programming is only confusing because of the name. The concept of caching previously calculated values in order to save time by avoiding recalculation (at the cost of using more space) is intuitive.

What am I missing?

<comment><USER13> Richard Bellman coined the name, and according to legend it was because the phrase 'dynamic programming' was so anodyne that not even the most officious bureaucrat could object.

In Bellman's own words[0]:

"An interesting question is, 'Where did the name, dynamic programming, come from?' The 1950s were not good years for mathematical research. We had a very interesting gentleman in Washington named Wilson. He was Secretary of Defense, and he actually had a pathological fear and hatred of the word, research. I'm not using the term lightly; I'm using it precisely. His face would suffuse, he would turn red, and he would get violent if people used the term, research, in his presence. You can imagine how he felt, then, about the term, mathematical. The RAND Corporation was employed by the Air Force, and the Air Force had Wilson as its boss, essentially. Hence, I felt I had to do something to shield Wilson and the Air Force from the fact that I was really doing mathematics inside the RAND Corporation. What title, what name, could I choose? In the first place I was interested in planning, in decision making, in thinking. But planning, is not a good word for various reasons. I decided therefore to use the word, 'programming.' I wanted to get across the idea that this was dynamic, this was multistage, this was time-varying—I thought, let's kill two birds with one stone. Let's take a word that has an absolutely precise meaning, namely dynamic, in the classical physical sense. It also has a very interesting property as an adjective, and that is it's impossible to use the word, dynamic, in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning. It's impossible. Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities."

\-----

0\. From [http://arcanesentiment.blogspot.com/2010/04/why-dynamic- prog...](http://arcanesentiment.blogspot.com/2010/04/why-dynamic- programming.html)

<comment><USER14> A link to the original paper that your source quotes: [https://web.archive.org/web/20060209011347/http://www.eng.ta...](https://web.archive.org/web/20060209011347/http://www.eng.tau.ac.il/~ami/cd/or50/1526-5463-2002-50-01-0048.pdf). "Richard Bellman on the Birth of Dynamic Programming" by Stuart Dreyfus.

</comment></comment></comment><comment><USER15> After spending a lot of time interviewing candidates. I have pretty much come to the conclusion that DP problems dont give good signals about candidates problem solving skills. In my experience, less 5% of candidates can genuinely crack problems using DP. Most of them either give me memorized solution or just plain give up.

If you are interviewing for a regular CRUD job aka web application. There are soo many other problems which can give much more refined signal about candidates skill. Please for love of God, dont ask DP. Unless you actually use it at work.

</comment><comment><USER16> In the first problem illustrated, I wanted to also get the list of coins. I made this attempt, but this hardly seems elegant. Any comments to make it better?

      n=10
      denom = [1,3,4]
      dp = [-1 for i in range(n+1)]
      dpl = [[] for i in range(n+1)]
      def f(n):
          if dp[n]!= -1:
              return dp[n]
          ans = 10**10
          if n<=0:
              return 0
          mini=n
          for i in denom:
              if (n-i)>=0:
                  new = f(n-i)+1
                  if new <= ans:
                      mini=i
                      ans = new
          dpl[n].append(mini)
          if mini!=n:
              dpl[n] = [item for sublist in [dpl[n], dpl[n-mini]] for item in sublist]

          dp[n]=ans

          return ans

      if __name__ == "__main__":
          ans = f(n)
          print(ans, dpl)

<comment><USER17> To be more elegant, you can remove the "dp" array. If you want to keep track of the full list, then you only need to keep track of "dpl". Here is code that I wrote and it works:

      n = 10
      denom = [1, 3, 4]
      dpl = [[] for i in range(n+1)]
      def f(n):
          if dpl[n]:
              return dpl[n]

          if n <= 0:
              return []

          ans = list(range(n))  # this is the max size possible
          for i in denom:
              if n-i >= 0:
                  new = f(n-i) + [i]  # append i to the end of the array
                  if len(new) <= len(ans):
                      ans = new

          dpl[n] = ans
          return ans

      if __name__ == "__main__":
          sol = f(n)
          print(sol)

</comment></comment><comment><USER18> So "DP" is just recursion with memoization? Or an I missing another piece?

Edit: apparently I'm not the only one thinking this: [https://news.ycombinator.com/item?id=19395862](https://news.ycombinator.com/item?id=19395862)

<comment><USER19> I think many people struggle with understanding "Dynamic Programming." Hopefully the following clears it up for you.

Dynamic Programming can be done using Memoization (top-down; aka recursively) or Tabular method (bottom-up). So what's the difference?

When you see top-down, it means to start from the root of the problem and recursively descend to the base case. As you pop up the stack, you will either calculate and store the result or look up the value in the cache. e.g., in the Fibonacci sequence, check to see if fib(4) was already calculated. No? Calculated and store it so the next time you come across this, you can use the result and not worry about processing fib(1), fib(2), fib(3), etc.

When you see bottom-up, think about filling out a table from the upper left corner column by column then row by row. To speed performance, you'll look at prior values, the value in the column above vs. column to the left vs. or column diagonally to the left. I know this sounds a bit strange, but if you solve the following problems, you'll see a repeating pattern.

Edit Distance 0/1 Knapsack Rod Cutting Longest Common Subsequence Longest Path in Matrix Coin Change

I've been writing about this in detail. Eventually I'll publish my writing to help others. I solve ~20 problems using Memoization and the Tabular method. As I solve each problem, I compare the solutions with prior problems showing the pattern. What I want to do is help people spot "patterns" vs. memorizing algorithms that are very problem specific.

<comment><USER20> Thanks for helping clear this up.

It seems that many dynamic programming solutions can be arrived at by starting with a recursive formulation, adding memoization, and then optimizing the cache based on specialized knowledge of the problem. For example:

1\. Q: How can you compute Fibonacci number f(n) recursively? A: f(n) = f(n-1) + f(n-2)

2\. Q: How can you memoize the results of your recursive function(s) to dramatically reduce the number of calls? A: Make a cache keyed on n that stores f(n).

3\. Q: Given specialized understanding of the problem, how can you minimize the size of the cache? A: Notice that you don't need to keep all O(n) slots; you only need to keep two ints.

Can every dynamic programming solution be explained this way? Or is there a good example of a dynamic programming problem for which you really need to make a leap that can't be sensibly reached through this sequence of three questions?

</comment></comment></comment><comment><USER21> The problem with DP problems (for me) is there seem to be a really large set of unique DP problems and coin change, knapsack, and grid DP problems like number of paths are only a small subset of them. What's more is that the rest of the problems can't be easily based on the approaches used for these...there might be dozens of problem classes to understand!

<comment><USER22> That might be the only dynamic thing about the otherwise terribly-named set of problems.

</comment></comment><comment><USER23> I've found this blog post to be the best of the bunch [http://blog.ezyang.com/2010/11/dp-zoo- tour/](http://blog.ezyang.com/2010/11/dp-zoo-tour/)

<comment><USER24> This way of illustrating categories of DP problems seems really intuitive to me. Thanks for sharing.

</comment></comment><comment><USER25> Thirty years after it was first highlighted, we still don't ask a juggler to actually juggle before hiring them.

Admittedly we have moved on from simply talking about the balls to getting them to arrange the balls in a particular order. Still not juggling though.

(For those that don't get the reference. It's a chapter in Peopleware)

</comment><comment><USER26> DP is clearly more related to maths than programming. If you are looking for a mathematician it make sense... But if you are looking for a good real world developer it's completely wrong!

</comment><comment><USER27> One of the techniques as described in CLRS is to first find the subproblem graph.

Consider Fibonacci sequence: f(0) = 0; f(1) = 1; f(n) = f(n-1) + f(n-2)

If we solve it naively, complexity will be O(1.6^n).

Now we can solve it in DP using two ways:

1\. Top Down: Instead of recursively computing the same subproblem, just store the value of this computation and look it up when needed. That's it.

2\. Bottom Up: One cannot come up with bottom up representation directly, unless we identify its recursive pattern and subproblems. Once we have this recursive pattern, just plot a graph for some values and we can identify overlapping subproblem and its overall graph.

Constructing graph for fib(5), we can see solutions from fib(4) and fib(3) are needed. Therefore, we need to find fib(3) and fib(4) before even solving for fib(5). Once we identify this graph, we can do bottom up, where we solve base solution (trivial or first node in graph) and construct our solution based in it.

Therefore, easy approach is to solve it top-down. Once it is done, we can identify subproblem graph and construct bottom up solution.

<comment><USER21> This is all well and good for fibonacci but it feels like the difficulty of the problem becomes exponentially greater when you start running into more complicated optimization strategies. Text justification is an example:

[0] [http://courses.csail.mit.edu/6.006/fall09/lecture_notes/lect...](http://courses.csail.mit.edu/6.006/fall09/lecture_notes/lecture20.pdf)

</comment></comment><comment><USER28> Coin changing - my understanding is that if each denomination of coin is at least twice that of the next smaller, then greedy is optimal. Note to self - see if you can prove it.

That is the case for current US coinage, 1, 5, 10, 25, 50, 100, cents.

</comment><comment><USER29> In the knapsack example they say...

> "No global variables should be modifed in the function"

Am I wrong or does the author immediately go on to modify the global variable 'dp'

p.s. @author typo in that sentence 'modifed'

<comment><AUTHOR> Thanks for pointing out the typo! I'll fix that right away.

And in regards to the "global variable", my variable 'dp' is my cache array. It's where I'm storing my precomputed results.

If I don't modify it, it's not DP anymore, it's plain recursion. :)

I guess I should've mentioned that the actual memoization table is an exception.

Anyway, thanks for reading my article! :)

<comment><USER29> Totally makes sense I just think it's slightly confusing in that context. Anyway, awesome article I enjoyed the read!

<comment><AUTHOR> Thank you! I'll make sure to go over some non-classical problems in my next article on DP to add more value :)

</comment></comment></comment></comment><comment><USER30> I never much enjoyed dynamic programming and I do think it's a poor choice for timed interview questions, but I did become more interested in it when I realized there are patterns to the cache strategies that can be used to group problems. For example like the usual matrix raster fill approach for e.g. 0-1 knapsack, or sepatately using two cursors to fill the upper triangle of a matrix like for longest common subsequence and for optimal order of associative computation (like matrix multiplication).

</comment><comment><USER31> I spent 2-3 weeks going through all DP problems on leetcode after work. I got that Tetris-effect where I was starting to hallucinate/dream DP problems and solutions as I would fall asleep at night.

I did this specifically for interviewing. As I've said before, these kind of interviews screen for unusual geniuses or people with the motivation to spend many hours studying, which are two types of acceptable hires.

</comment><comment><USER32> >"I'll show you how to do DP"

Hey it's the same 3 example problems that are in every textbook, GeeksforGeeks, Leetcode, etc.

<comment><AUTHOR> Thanks for taking the time to read my post. :) Like I mentioned, this article only lays the groundwork. The next article I intend to write on is DP+Strings (for example, finding the longest substring of a string which is a subsequence of another, etc)

I'm starting with classical problems and I'll soon diverge into non-classical problems, as I've mentioned in my article too.

In any case, hope my other articles on my blog added more value to you in comparison!

<comment><USER32> Yeah, I was being snarky up above, but will definitely look forward to the future articles.

</comment></comment></comment><comment><USER33> The knapsack problem here is of particular interest since despite the optimization problem being NP-hard the solution can in fact be found in O(n * W). This feels similar to how the theoretical best comparison sort is O( n log n) but radix can do this in O( n).

<comment><USER27> Knapsack still is a NP-Hard Problem. Even though DP solution looks linear, it has pseudo-polynomial time complexity[1][2]

[1][https://en.wikipedia.org/wiki/Knapsack_problem#Computational...](https://en.wikipedia.org/wiki/Knapsack_problem#Computational_complexity)

[2] [https://stackoverflow.com/questions/4538581/why-is-the- knaps...](https://stackoverflow.com/questions/4538581/why-is-the-knapsack- problem-pseudo-polynomial)

</comment></comment><comment><USER20> Is dynamic programming the same as memoized recursion, or does it include techniques beyond that?

<comment><USER34> Consider Fibonacci. Memoized recursion uses O(n) memory, because you don't garbage-collect anything. Bottom-up dynamic programming is O(1) memory, because you only need to remember the largest 2 subvalues you've computed.

</comment></comment><comment><USER35> So, in all of these examples, the author basically makes a set of all possible solution into a tree (where moving from ancestor to child is that tree correlates to making a possible decision) and then recursively traverses this tree to find the best option?

</comment><comment><USER36> This is poorly written. I was hoping hacker news would have better scrutiny on articles.

</comment><comment><USER37> Note that the memory requirements in the Coin Change solution is O(N), and that is not optimal. You should be able to get away with O(max(denom) - min(denom)) (you don't need to cache everything).

</comment><comment><USER38> Only company I know of asking for "Dynamic Programming" is LinkedIn, is there any others?

<comment><USER39> Google ?

</comment></comment><comment><USER40> Hi, can you point the github source project of this blog? thanks.

</comment><comment><USER41> I agree with other's opinions that asking a Dynamic Programming question is a weak signal for a candidate's competencies in general software development work like frontend, backend, mobile applications, and the like.

However, I would like to counter a common opinion that eventually follows in similar threads and some of my social circles: "Algorithms is an undergraduate course in which students learn specialized solutions to esoteric math problems, all of which they ultimately forget when they spend real time working in the industry, so the knowledge shouldn't be relevant in an interview."

It is fair that if you don't exercise what you learned, you will gradually forget it, but I believe its still important for a candidate to cherish algorithm design and analysis because I consider it a great toolbox of the trade.

For all the concepts and the techniques I learned from my undergraduate course in data structures and algorithms, I utilized them as the basis of my Software Engineering Toolbox.

What is my Software Engineering Toolbox? It is a collection of algorithm design concepts and techniques that I can employ anytime I am faced with a novel problem or a problem whose standard Stack Overflow solutions are inadequate.

The Software Engineering Toolbox contains the following: Arrays, Linked List, Stack, Queue, Hash Table, Binary Search Tree, Priority Queue, Set, Trie, Sorting, Binary Search, Divide-and-Conquer, Backtracking, Dynamic Programming, Range Query Trees, Graph Algorithms, Bit Mask Optimizations, Square Root Bucket Optimizations, and Multiple Pointer Optimizations.

First, I rarely implement my own data structures from scratch; all the programming languages that I use provide great standard libraries. Yet, I always remind myself the use of these data structures, because you would be surprised with the amount of people you can meet who tries to answer a problem that boils down to a set membership with a HashMap<Integer, Boolean> when they can just use a HashSet<Integer> or with the amount of people who manually treat an Array as a Stack or a Queue when those data structures are readily available.

Second, I rarely implement my own sort or search functions from scratch; again, all the programming languages that I use provide great optimized functions. I treat Sorting and Binary Search as techniques that lend themselves to optimizing the locality of a data set such that you can easily answer basic statistics, find the bucket for a token in a ring, or merge data sets. These are simple techniques developers should readily know to exist when optimizing their code.

Third, why do I have Divide-and-Conquer and Backtracking in my toolbox? I believe that no matter what problem you face, you should be able to bruteforce it. You can't always tell someone that you can't implement something because you didn't find a Stack Overflow answer or you didn't deduce a collage of standard library functions or third-party libraries to solve your problem. Using these techniques, you can at least arrive at a pretty weak solution which is still a solution. To actually address Divide-and-Conquer and Backtracking in relation to bruteforcing, these techniques allow you to easily traverse through a search space to filter for a certain combination or permutation of items that satisfy a customer's constraints. Furthermore, Backtracking is a relatively easy to medium difficulty technique that is the basis for a lot of the Graph Algorithms people keep balking at!

Fourth, Dynamic Programming. To be honest, I rarely utilize it, but I appreciate it because the common subproblem types of 1D, 2D, Range, and Sub- Trees taught me how to order subproblems successively to solve other problems, which applies beyond DP. I discourage people from trying to pattern match Dynamic Programming problems and solutions, and I encourage them to truly digest CLRS and understand its 4 rules for Dynamic Programming to consider possible dependencies and structures for various combinations and permutations of the problem parameters to identify what the optimal substructure really is.

Finally, the remaining things in my toolbox are included in my toolbox because they are useful in my work experience with real-time network anomaly detection and streaming analytics. For example, topological sorting distributed tracing events into a rooted tree that I encode into a bit vector using a left-child right sibling binary tree. Not everyone will do this, but with my toolbox I never worry much about facing new frontiers of problems or being tasked to create libraries and tools for myself and others to use instead of being at whims of someone else on the Internet.

Overall, I hope people can look back at their courses in algorithm design and analysis and say, "Yeah, the problems and the solutions were really weird, but the techniques hidden away within them are actually GENERALIZABLE and are a fundamental basis to build new things and solve complex problems!"

Nonetheless, I don't want anyone who is weak in algorithms design and analysis to feel discourage. Play to your own strengths whatever they maybe, or you can always strengthen them; it's never too late.

Finally, my Software Engineering Toolbox has way more stuff like actual "engineering" stuff like automatic formatters, linters, fuzzers, automation, tests, mocks, coverage, "Infrastructure as Code", and blah blah blah. :")

I would like to close by saying that a good engineer knows the right tools for the job. :)</comment>


Gamers make faster decisions than nongamers, just as accurate - <AUTHOR> http://arstechnica.com/science/news/2010/09/gamers-make-faster-decisions-than-nongamers-are-just-as-accurate.ars <comment><USER2> Regardless of the subject, this article is superbly written and a great example of how to cover psychology articles as a reporter.

Instead of simply stating the most headling grabbing implication of the research and leaving it at that, it went in depth to show the "what if" questions researchers controlled for rather than leaving us to guess or hunt down a copy of the real research paper (the reader is lazy, were not going to do it). Hopefully we get more submissions likes this when it comes to psychology articles and less 'scientist found X, the end'

<comment><USER3> I was genuinely _thrilled_ that the article brought up the points about causality vs. correlation.

Then I was sad, because it reminded me how rare it is. Almost every news article about research I've read the past 20 years has been of the form "Scientists have found that X causes Y, and here is the correlation along with some uneducated speculation."

<comment><USER4> <http://xkcd.com/552/>

</comment></comment></comment><comment><USER5> When I am in the zone playing COD for instance, I can run into a room, see three people and determine an optimum shooting strategy in under a second. For instance, if one guy is turned I can shoot him last and instead shoot the closest then the second determine the best place to duck to reload if needed. All the while, I process sound from everywhere else, keep a running map in my head (I'm very good with spatial orientation) and can check off where explosions have taken place or possible spawn points coming up.

I can also tell the difference between a player with an 80ms ping and a 30ms ping.

This all comes with a caveat. I have been playing for ~8 years off and on so I have experience and confidence in my abilities. Also, to achieve such a state to determine split second actions, my heart rate is well above a sedentary resting state. Once I figured this out, I stopped playing as much as I can only assume that the adrenaline rushes from gaming while sitting can only be bad for my health.

Two cents..

<comment><USER6> From my years of top-level competitive FPS gaming I eventually determined the same thing.

A high level of play is greatly facilitated by a high heart rate and an adrenaline rush.

The best gamers, by far, are the ones who combine years of practice with amphetamines.

It works for the airforce and it works for gaming.

I decided that the constant self-induced adrenaline rushes were bad for my health and so I stopped playing those kinds of games.

<comment><USER7> I used to get distracted in tense Quake 3 duels because I would become aware of my heart racing.

</comment></comment></comment><comment><USER8> The fact that I'm a huge gamer helped me decide instantly that this finding is correct.

<comment><USER9> And will also help many more people on many more social sites decide that it needs to be upvoted, while the ad for Halo Reach at the bottom of the article will ensure that the publisher gets paid handsomely for their work.

Nice little trifecta: article topic, emotional impact, and commercial-event tie-in. Very nicely done.

</comment></comment><comment><USER10> Couldn't the same be said of people who play sports or jazz or any other thing that involves making fast decisions?

It's like there's this desperate need of justifying playing video games with every new study that says 'look! I am not actually wasting my time!'

Personally I think no silly justification is needed as there's nothing wrong with enjoying entertainment.

<comment><USER11> I think the fact that this is comparable to sports or jazz is exactly the point. While I imagine that you'd have to go back prettaayyyy far to find the time when sports were demonized as anything from an utter sin to a slightly pitiable time-waster, jazz certainly endured those characterizations when it was emerging into popular culture.

The same is true of any number of other media that now enjoy nigh-immeasurable popular esteem: the novel, for example. If we'd had the same educational- industrial apparatus then as we do now, surely some enterprising PhD candidate would have staked his career on studies illuminating the benefits of reading novels.

EDIT: I forgot to mention this - the real question that remains is just how valuable this effect of gaming is in the non-gaming world. And boy is that a doozy of a question!

<comment><USER12> ability in sport = martial prowess / hunting skills / good health in general / good genes in general -- It has always been more or less universally respected.

Jazz players always had subculture specific social status, same with hip-hop pioneers, early rock musicians etc.

<comment><USER13> _Jazz players always had subculture specific social status, same with hip-hop pioneers, early rock musicians etc._

As have gamers.

<comment><USER12> "social status" that doesn't get you laid =//= social status that does

</comment></comment></comment></comment></comment><comment><USER14> According to The Invisible Gorilla (very interesting book btw) there have been other experiments done with gamers that supposedly found improved cognitive abilities resulting from intense FPS gameplay. However, the authors of the book did more digging and concluded that some of those experiments on gamers were never independently confirmed by other experimenters. Here's an article on the subject: [http://theinvisiblegorilla.com/blog/2010/04/20/the-limits- of...](http://theinvisiblegorilla.com/blog/2010/04/20/the-limits-of-cognitive- training/)

<comment><USER15> That sentence is not entirely clear to me.

Did it imply that the experiments were never repeated or that the experiments were repeated and the results were negative?

<comment><USER14> results were negative.

</comment></comment></comment><comment><USER16> Seems logical. After hours and hours of poker I find it easier to figure out my odds in all sorts of situations. The same concept could be applied to video games.

<comment><USER17> Maybe it isn't really the same concept. While playing poker, you usually take your time to calculate each move, but the _reaction_ time is much, much less in video games (faster decisions).

<comment><USER18> That might be true if you're playing live or mainly for fun, but when you're playing 12 or more tables, poker starts to look a lot like those video games in terms of decision speeds. Throw in the need to remember how you've been playing on 12 different tables in addition to how 100 other players have been playing, and your attempt to trivialize poker playing looks somewhat silly.

</comment></comment></comment><comment><USER19> The history of probability and statistic show us that a gambler can distinguish between two states whose probability is very similar (1/36 or so), so if a gambler has more information that a nongamer then he can use that information to be more accurate and make a faster decision.

<comment><USER20> Do you have more information about those findings?

</comment></comment><comment><USER21> I started playing blitz chess online recently instead of normal chess (I play 2 mins with 7 second increment, rather than 15 mins with 30 sec increment). I'm not sure playing chess makes you better at anything else but I think it let's you learn a few things about yourself. For instance, I don't think it's made that much difference to my game, suggesting most of my thinking time had been going to waste previously. Also tells you how much failure is down to stupid mistakes vs. the other player being smarter.

</comment><comment><USER22> My guess is it's all about trained muscles that make eyes do microscopic movements.

Eyes can see only things that move, and to see a static picture there are special muscles that make retina move all the time the eyes are open (that's why blinking or closing eyes for short time gives them rest and helps to relax).

People that have these muscles more developed and move faster are able to notice and comprehend more visual information in shorter times.

Playing games and being all the time on the watch out for every pixel on the screen makes these muscles to work out and become stronger and faster.

Yep, it's just a workout.

<comment><USER23> Nope. FTA:

> The same thing happened when the test was switched to a similar task based > on tonal differences, indicating the success of gamers wasn't simply the > result of their focus on visual cues.

<comment><USER22> Why can't it be a combination of factors?

<comment><USER23> I think that it probably is a combination of factors. However, I was responding to your initial comment:

> "My guess is it's all about trained muscles [...]"

[all about] != [combination of factors]

</comment></comment></comment></comment><comment><USER3> Interesting and convincing. I would love to see similar studies for different kinds of games.

For example, after playing Braid for 4 hours straight, I had trouble performing normal day-to-day tasks naturally for about half an hour.

<comment><USER20> A weaker version of Braid-syndrome can happen if you have a game with quicksave/quickload and use it extensively.

</comment></comment><comment><USER24> _Both were shown a screen that had a set of randomly moving dots, and asked to determine whether there was any coherent motion, meaning that, despite the apparent randomness, the dots had a tendency to head in a single direction_

This article should have been titled, "Gamers make faster decisions than nongamers at video games." Duh.

<comment><USER25> Maybe you should read the entire article.

<comment><USER24> Am I missing something? All the tests were based on interfacing with a screen.

<comment><USER26> From the friendly article:

"The same thing happened when the test was switched to a similar task based on tonal differences, indicating the success of gamers wasn't simply the result of their focus on visual cues."

</comment></comment></comment></comment><comment><USER4> It's sad that I first heard this was true when the whole high school shootings were happening. The shooters were "trained" via video games to shoot for the head, one shot, and then move on to the next target, whereas a novice would keep shooting all over multiple times to make sure they got it right. So, it backs up the claim above, and yet, still doesn't make me want to let me kids play video games.

<comment><USER27> Boy Scouts would teach knife skills and knot-tying — both useful skills for kidnapping and torture — but I doubt you have similar misgivings about that. Making somebody competent at efficiency and rational decision-making under pressure doesn't make them a killer. Video games are not strongly correlated with homicide, that one data point 12 years ago notwithstanding.

(Incidentally, I'm not the downvote. I don't think your logic works here, but it's an interesting correlation.)</comment></comment>