Sleep deprivation is increasing our risk of serious illness - open-source-ux https://www.theguardian.com/lifeandstyle/2017/sep/24/why-lack-of-sleep-health-worst-enemy-matthew-walker-why-we-sleep <comment><AUTHOR> I haven't used an alarm in probably a decade, outside of super early flights. I consistently wake up when I need to. I have no problems with insomnia. I don't need caffeine to feel awake and in fact haven't drank any in about three years. I wake up feeling refreshed and happy and looking forward to the day.

I say this because I used to have pretty bad insomnia. Weeks of almost no sleep. Let me tell you: when sleep is not going right for you, nothing is going right. Sleep does wonders for depression, too. So I looked at everything I was doing wrong and changed it all. I changed my diet, my work habits, everything. I don't do a lot of critical thinking an hour or two before bed (no work, no computer). I've trained my body to know how to sleep. I go to bed at around the same time every night, and I listen to my body. Barring those with actual medical issues, for the majority, it is a choice.

<comment><USER2> > _I was doing wrong and changed it all. I changed my diet, my work habits, > everything..._

I did all of those things too and it did nothing for either my insomnia or depression. There are a surprising number of us out there with, as you say, "actual medical issues".

<comment><USER3> He found his cure so he thinks it will work for everyone. It never works that way.

<comment><USER4> Or he may have "found" his cure but it was not one of the variables he mentioned. With that caveat, here's what I changed to fix my 25+ years of insomnia.

1) Always wear socks and semi-tight fitting pants and t-shirt.

2) I increased (yes increased) my calories by ~20% but I stop eating by 6PM.

3) If you suffer from PTSD or wake up screaming when someone tries to wake you, either A) get a dog or B) Do heavy weight training and learn a viscous form of martial arts / street fighting as it will make you feel safer.

4) Avoid toxic people. This one was the hardest.

</comment></comment></comment></comment><comment><USER5> There is massive problem in the culture about value of sleep. The culture _implicitly_ defines mentality where working hard without much sleep is somewhat heroic. Therefore according to the culture (again _implicitly_ ) those who sleep a lot are lazy.

Hard working heroes who sacrifice sleep from time to time displayed in popular films. Take for example three films _Micro Men_ [1] and _The Martian_ [2] and _The Social Network_ [3].

In first two films, hard working heroes don't "waste" time on "useless/unproductive" sleep and instead drink lots caffeine.

In corporate culture it's often the same. Since 2006 till 2008 I worked as a technical support engineer and I often had to work at night. Sometimes, I took two hours of sleep (in an office's couch) from 4 AM till 6 AM. After these two hours of sleep I would be energized and work really productive till 9 AM. But the company decided to impose strict ban on sleeping in office and instead put coffee machine. As I expected my performance fell significantly no matter how much coffee I took. So the company put stupid discipline over actual performance.

In 2014, I had insomnia so I couldn't perform well at work (again no matter how much coffee I took). I went to corporate doctor (a doctor assigned by company). She looked at me and said that I don't have bruises under my eyes, so I'm totally fine. More than that she actually didn't take any my complaints seriously and instead assumed that I'm lazy.

Nowadays, I mostly sleep well and I almost don't drink caffeine. Sometimes, my insomnia came back. Since I participate in algorithm contests and I know very well that _no matter how hard I try to work hard without enough sleep I do not perform well in algorithm contests_.

The mentality that working hard is opposite to good sleep is most irrational, harmful and widespread yet very implicit thing in the culture.

[1] [https://en.wikipedia.org/wiki/Micro_Men](https://en.wikipedia.org/wiki/Micro_Men)

[2] [https://en.wikipedia.org/wiki/The_Martian_(film)](https://en.wikipedia.org/wiki/The_Martian_\(film\))

[3] [https://en.wikipedia.org/wiki/The_Social_Network](https://en.wikipedia.org/wiki/The_Social_Network)

<comment><USER6> I wonder if there is a reverse effect of taking caffeine when tired/sleep deprived? Some days when I'd hardly slept the night before I would try to rely on caffeine to make it through the day but actually found myself ready to fall asleep after the drink.

In any case I definitely agree, get sleep instead of coffee.

<comment><USER7> So weird, same thing happens to me - I can drink a huge can of energy drink, or a strong coffee or anything else with a lot more caffeine in it than usual and within 20 minutes I'm ready for a nap.

Not medical advice, I'm not a doctor, go speak to a doctor, this isn't a substitute for sleep etc etc. But in my experience, whenever I've found myself in a situation where I'm not able to get much sleep but still need to function for an extended period of time, Modafinil or Armodafinil has been an absolute life saver. No lingering caffeine like coffee as well, once it wears off it's gone and you can sleep normally (roughly 12-14 hours).

Overall downside - it's not super well researched, especially long-term effects, might cause brain problems, or cancer, who knows? I also often hear of people coming to rely on it as a substitute for sleep - it's not addictive per se, but what it allows you to do definitely can be.

</comment></comment></comment><comment><USER8> This article says:

> _It's his conviction that we are in the midst of a "catastrophic sleep-loss > epidemic", the consequences of which are far graver than any of us could > imagine... sleep deprivation... constitutes anything less than seven hours_

I thought we found out that we sleep the same as our ancestors. This article -- [https://ca.news.yahoo.com/nightmare-sleep-no-more- plentiful-...](https://ca.news.yahoo.com/nightmare-sleep-no-more-plentiful- primitive-cultures-183037441.html) \-- says

> _Even without electricity or other modern trappings, they [the Tsimane > people of Bolivia, Hadza people of Tanzania and San people of Namibia] > logged an average of 6 hours and 25 minutes of sleep daily, a figure near > the low end of industrialized society averages._

<comment><USER9> How long do those peoples live? If their expectancy is less than or similar to modern society's after controlling for infant mortality and preventable deaths before say 18, then maybe the lack of sleep is hurting them too.

</comment></comment><comment><USER10> Even if it's true that less sleep results in less life, it's still a wash.

Say you slept 10 hours a day and lived to 100. In other words, you have 14 wakings hours times 100 years. 1400 * 365 hours, in other words. Now compare this to sleeping 6 hours a day and living to 80. In other words, 18 waking hours. 1440 * 365 in a year, in this case.

This gets more complicated if you abandon the hours and rather frame it as a percentage of time you spend with the people you care about. Now your sleep is a function of their sleep and the math gets too convoluted, but I'm sure you get the point.

Whether one is better than the other is more of a philosophical question, but just food for thought.

IMHO people shouldn't worry about these kinds of things. In general, trying to optimize the quality of life you have when you're awake will generally lead to the optimal outcome in general. In this case, eating well, exercising and having meaningful relationships will result in a higher quality sleep due to lower stress and also make your waking hours also more enjoyable. Ultimately, that's the point, right?

EDIT: Hah, forgot to multiply by 365, but my point stands.

<comment><USER11> >Say you slept 10 hours a day and lived to 100. In other words, you have 14 wakings hours times 100 years. 1400 hours, in other words. Now compare this to sleeping 6 hours a day and living to 80. In other words, 18 waking hours. 1440 in a year, in this case.

True, but what's the quality of life of those zombie-walking hours?

<comment><USER10> That's not the point though. You could replace 6 and 10 with 8 and 12.

</comment></comment></comment><comment><USER12> I'm unemployed. I sleep 10 hours per day.

When I was employed, I slept 6 hours per day.

The difference in mood is massive.

However, I'm not sure if the working world will change. It's a zero sum game of who is willing to torture themselves the most, which is sometimes correlated with how much work you produce.

At my past job, it wasn't enough to just produce. You had to /look like/ you were producing. This meant working 8-6. This meant skipping lunch. This meant always sitting at your desk (no walking meetings).

Obvioisly not every employer is like this. But working is a pretty decent way to kill yourself. It happens to most people eventually.

<comment><USER13> >The difference in mood is massive.

What are two adjectives that describe your previous mood, and two that describe your current one?

<comment><USER12> 6 hours' sleep = zombified, confused, stressed, steeped in dread

10 hours' sleep = alert, optimistic, sharp, happy

And the only variable that changed was lengthOfSleep !

<comment><USER14> I thought another variable that changed was your employment status. That's a huge one, no?

<comment><USER12> Correct, becomeUnemployed() was what altered lengthOfSleep :)

I certainly could have tried to sleep more while I was working, but there really wasn't a way for me to personally achieve that. My boss and company were psycho. Lots of unpaid forced overtime, else fired. Lots of meetings and political games to play, else fired.

I wish my skills were desired elsewhere but it's been difficult to find a new job. So, plenty of time for me to sleep.

I definitely am exercising the same amount as I was while working. Sleep changed everything.

EDIT: How on earth is this answer downvotable?

What have I said that is even remotely wrong or controversial?

<comment><USER15> the job stress elimination could be the first order cause rather than amouny of sleep. thus pedants must downvote. it is their way. our way?

</comment></comment></comment></comment></comment></comment><comment><USER16> One thing that never seems to be discussed in these type of articles is that for most people time awake is significantly more valuable than time spent asleep. An extended lifespan is of little use to me if those extra hours are spent unconscious.

If I spend 10% extra of my time awake (2.4 hours) but shorten my life by 5%, that seems like a worthwhile trade-off. (That's just an example; I've no idea if those figures are in any way plausible, but I'd be interested in any research that tried to figure out the optimum level of sleep for maximum life- awake time.)

<comment><USER17> This makes no sense. It's not just that you might, for example, die at 70 instead of 75 if you don't sleep. The point is that your quality of life during those 70 years would be horrible, for all the points listed in the article.

<comment><USER4> But you could use caloric restriction to get some of those years back. You're probably near rock-bottom grumpiness/misery anyway so the cost is minimal.

</comment></comment></comment><comment><USER18> > An adult sleeping only 6.75 hours a night would be predicted to live only to > their early 60s without medical intervention.

I wonder if this controls for other lifestyle factors. Eating a lot before bed makes me sleepier and I could go 7-8 hours a night. Small dinner several hours before bed results in about 6 hours. No alarm clock, that's just how much I need. I find it hard to believe that I'm shortening my lifespan by eating less at night.

<comment><USER19> Maybe sleeping long enough (7-8 hours) requires a certain (higher) amount of energy, which would be supported by a big dinner before bed? Not sure if there are any studies on this.

</comment></comment><comment><USER20> This reminds me of a delightful article that helped me fall back in love with sleep (and therefore find it easier to fall asleep): [https://aeon.co/essays/the-cure-for-insomnia-is-to-fall- in-l...](https://aeon.co/essays/the-cure-for-insomnia-is-to-fall-in-love-with- sleep-again)

</comment><comment><USER21> > the number of people who can survive on five hours of sleep or less without > any impairment, expressed as a percent of the population and rounded to a > whole number, is zero.

Pretty deceiving claim, the author starts with "the number of people", but gives a rounded down percentage instead

the point would be perfectly clear as "only 0.X% of people ..."

</comment><comment><USER22> "Deep sleep – the part when we begin to dream"

While I'm glad they don't use the popular but incorrect REM sleep == dream sleep, I don't think this is correct either. I have a circadian rhythm disorder and had severe insomnia before that and so have observed the process of getting to sleep in slow motion many times. At least in many cases for me, dreaming preceeds sleep by a bit as gently waundering thoughts gradually turn into dreaming (this might be considered stage 1 sleep on polysomnograph, but certainly not deep sleep). I have a poor visual memory and when I notice visual recall when close to sleep then that is a very good sign that sleep is about to happen. It turns into dreaming a little bit after that and then I fall asleep, although I don't think it always follows this progression (and I don't think I always get visual recall before sleep).

<comment><USER23> I was under the impression that rem was the state when you dream, and you can sometimes enter it before the other stages, can you enlighten me to the truth with a decent source? Thanks =D

<comment><USER22> I think the best article on this is Mark Solms 2000 "Dreaming and REM sleep are controlled by different brain mechanisms"[0]. Solms and a couple of coauthors have an article this year, "Dreaming in Neurological Disorders"[1], which is also great although I don't think it directly answers your question.

My sense is that it is still being debated if there are differences in the types of dreams that happen in different stages or not. I couldn't find it last time I looked, but some research somewhat recently found that medically extinguishing REM sleep did not produce a detectable change in dream reports. Other studies seem to show that there may be differences.

[0] [http://neuro.bstu.by/ai/To- dom/My_research/Papers-2.1-done/C...](http://neuro.bstu.by/ai/To- dom/My_research/Papers-2.1-done/Cognitive-S/Consciousness/For- icnnai-2010/2/Ref/download.pdf)

[1] [https://twin.sci- hub.ac/cd77f4d5f6111801eee90b66a3ac13ee/sol...](https://twin.sci- hub.ac/cd77f4d5f6111801eee90b66a3ac13ee/solms2017.pdf)

<comment><USER23> Awesome material, thanks!

</comment></comment></comment></comment><comment><USER24> Two thoughts on how sleep deprivation directly impacts hackers and people working in the tech industry:

1\. Glorifying low sleep through all-nighters, perpetual crunch time, etc., has become part and parcel of startup life for generations [0]. (I've always wondered if it's a carryover from the particularly collegiate nature of software culture [1]). There's been some backlash against founders and the like boasting about operating on low sleep, as part of the general backlash against poor work-life balances in tech. Hackers need to push back against the suboptimal performance and system degradation that chronic sleep deprivation creates.

2\. The tech industry, known for its whimsical benefits and presumptions to think differently and be unconventional, could explore allowing employees to take nap breaks. So far it looks like Google and Cisco are among the handful of large tech companies to allow employees to nap [2], but one wonders how many employees with poor sleep hygiene actually do take advantage of it, and whenever those articles pop up it always seems like ads for sleep pods. Yet siestas and naps are something that have been allowed in cultures around the world, even in work-heavy East Asia [3].

[0] [https://www.folklore.org/StoryView.py?story=90_Hours_A_Week_...](https://www.folklore.org/StoryView.py?story=90_Hours_A_Week_And_Loving_It.txt)

[1] [https://news.ycombinator.com/item?id=11299145](https://news.ycombinator.com/item?id=11299145)

[2] [http://fortune.com/2011/08/18/why-companies-are-cozying- up-t...](http://fortune.com/2011/08/18/why-companies-are-cozying-up-to- napping-at-work/)

[3] [https://www.theguardian.com/world/2014/aug/18/japanese- firms...](https://www.theguardian.com/world/2014/aug/18/japanese-firms- encourage-workers-sleep-on-job) [http://mashable.com/2017/07/12/capsule-beds- beijing/#ohlpi6K...](http://mashable.com/2017/07/12/capsule-beds- beijing/#ohlpi6Knaaqr)

</comment><comment><USER25> It doesn't mention taking naps.

But damn, they make life pretty great!

<comment><USER26> It does, and mentions that they are not long enough to reach deep sleep. The next sentence says that we need 4-5 cycles to get enough deep sleep.

</comment></comment><comment><USER27> for the last few years, i've been getting by with 6 hours/night on weekdays and 10 on weekends, but lately i've found myself waking up too early on several weekends, and i'm a bit worried my body is getting used to "six hours are all you get". need to work up the self discipline to close the book at least an hour earlier every night.

<comment><USER23> I read somewhere recently that the 8 hours a night number is wrong, and it depends on your age, lifestyle, and genetics. 6 was at the lower end of what they said was fine, though unfortunately I don't remember the source or if it was reputable.

</comment></comment><comment><USER28> The quality of your sleep is an important factor too. If you suspect you may have obstructive sleep apnea then go get it checked out.

</comment><comment><USER29> Wasn't the consensus that 7h sleep was the best for lifespan and more or less than that lead du decreased life span?

<comment><USER30> There was never a conclusion that certain sleep lengths _lead to_ decreased life span, only a correlation. The causal link has been suggested to be the reverse - that sick people tend to stay in bed longer because they're sick. I believe I saw a recent article that even said that the popular takeaway that avoiding "oversleep" is healthy is the opposite of reality, because extended sleep is actually a healing mechanism.</comment></comment>


Driving in the US is making a big comeback - <AUTHOR> http://www.vox.com/2015/11/25/9800614/peak-car-driving-rebound <comment><USER2> Whether driving in the US is making a comeback or not, the lower price of fuel makes this an excellent time to raise the fuel tax, which hasn't seen an increase in 25 years [0].

Given the decrepit state of the US infrastructure [1], the funds will need to be gathered soon if we wish to avoid the deleterious effects of neglect.

[0][http://www.taxhistory.org/thp/readings.nsf/ArtWeb/5DDB791947...](http://www.taxhistory.org/thp/readings.nsf/ArtWeb/5DDB79194769C2BF852574D5003C28D5?OpenDocument) [1][http://www.infrastructurereportcard.org/](http://www.infrastructurereportcard.org/)

<comment><USER3> Thankfully that has about 0% chance of getting through Congress. The last thing working people need is a tax increase on essential goods.

<comment><USER4> Where to your propose that the money for infrastructure repair/maintenance come from?

<comment><USER3> The government is collecting plenty of revenue already. If the need for increased infrastructure spending is pressing, there's plenty of waste and pork-barrel to cut to make room.

<comment><USER2> But by cutting spending elsewhere and putting it towards roads, we're still hiding the cost of driving and subsidizing the use of cars. We've already bailed out the Highway Trust Fund for at least $65B. It's a hole we'll keep digging -- a cycle of demand inducement that begets budget shortfalls that begets general tax hikes -- unless we re-align the tax with usage.

I'm asking for people who use the roads to pay their fair share; you're asking the government to collect money from everyone so the road-users can keep believing roads are self-funding. When the users are forced to confront the true cost, they'll demand better solutions.

In short: we should both cut wasteful spending _and_ re-align the gas tax with infrastructure expenditures, because the bailout from the general fund is pork itself!

<comment><USER3> Okay. Then cut taxes by an equivalent amount elsewhere. My point is that I don't think that it's appropriate for total government revenue to move upwards at this point. I think that the government can do everything it needs to do with less money than it's currently spending. Therefore, having the government confiscate more of the people's wealth is immoral.

</comment></comment></comment></comment></comment></comment><comment><USER5> Or it is not. [http://www.citylab.com/commute/2015/11/peak-car-us- driving-a...](http://www.citylab.com/commute/2015/11/peak-car-us-driving- america-vmt-vox/417795/)

<comment><USER6> Isn't that addressed in there?

</comment></comment><comment><USER7> Public transportation needs to improve. Especially railroads. They are simply lacking in a lot of places.

<comment><USER8> Urban sprawl cities designed with cars in mind (eg. Dallas, Atlanta, etc.) will not be easy to convert over to public transportation.

<comment><USER9> This is why electric self-driving cars are the future. Door to door service using existing road infrastructure much more efficiently. Most public transit systems are so poorly run many will not survive.

<comment><USER10> Exactly. Passenger rail (as much as I love it) is about to be outdated. Expensive rail projects that amortize costs over >20 years are at risk of never paying off, because in that timeframe self-driving cars will take over as the preferred method of transport, because they:

* utilize existing infrastructure (roads)

* get you 100% of the way from point A to B vs having to travel additional distance from stations

* can be gradually introduced with purchasers being those who directly benefit vs big projects dealing with nimbys, budgets, and right-of-way issues where costs-benefits aren't as aligned

* are a drop-in replacement that offer significant improvements in both safety and driver time

* open new business models of cars as service vs as product

* allow commuting from farther away, reducing housing shortages

* offer lower labor costs than buses, so cities may control their own fleets or subsidize private fleets for low income riders

... among many more benefits. Dense urban cores will still find value in subways that do not take up outdoor space, but for the most part, mid-century cities are going to be doing a lot of ripping up excess roads/lanes and parking lots, and making things both denser and greener.

<comment><USER11> This really depends on the city. Go to any Asian city where people have loads of scooters and motor bikes. These take up way less space than a car. You'll see that the roads cannot handle the traffic at all. From my impression US is very car centric. Once you do away with that a lot of assumptions you have now don't apply. I like my public transport and cycling.

<comment><USER9> No problem: self-driving electric scooters and motor bikes - not too far behind cars.

[http://www.notimpossiblenow.com/the-latest/google-self- drivi...](http://www.notimpossiblenow.com/the-latest/google-self-driving- motorcycles)

</comment></comment></comment></comment></comment></comment><comment><USER6> There is no doubt that cheaper gasoline is helping. But right now in the US we seem blind to the actual strength of the economy. Car sales are about to have one of their best years _ever_. We had such a bad downturn coupled with bad economic news from Europe then China. Add to that a Republican primary that thrives off a negative economic message and you can see powerful forces at work to drive home a message that the economy here is still pretty bad or hanging in the balance. Data like this is "surprising" mainly in that context.

<comment><USER12> Surging economic indicators and cash in working people's pockets (or jobs for that matter) don't exactly correlate.

</comment></comment><comment><USER13> Before you make your population adjustments, compare the 1997 workforce population to today's. It hasn't grown with population.

Also, the workforce is less well compensated than it was. I recall from my days in high school and college in the 90s a retail manager at the mall could afford a decent used car on their salary. Now, the car costs double and the salary is 10-20% more.

<comment><USER14> A retail manager at a mall today can afford a 2012 Camry for $12,000 to $13,000 at 40-50k miles (or a few other competing vehicles). They're handing out record low rates on car loans, to pretty much anybody with an income.

~$268 per month, four year car payment

$40,000 salary / $2,400 per month disposable income

[http://www.payscale.com/research/US/Job=Retail_Manager/Salar...](http://www.payscale.com/research/US/Job=Retail_Manager/Salary)

</comment></comment><comment><USER15> How much of this is simple demographics? As the Baby Boomers started retiring, driving saw a downturn. Now, as the Millenials are starting to hit their thirties and starting families, driving is ticking back up again.

</comment><comment><USER16> I suspect that stimulus spending on "shovel ready" transportation infrastructure also plays a role. Many of those projects are coming online...at least on the roads I use and are making travel easier. Also, cars have gotten better.

<comment><USER17> Or...the less partisan and probably answer: The economy has just gotten better and people can afford to drive more.

<comment><USER18> Better economy, cheaper gasoline, but most importantly... numbers that weren't adjusted for population growth.

<comment><USER19> The second chart is population-adjusted and still shows a sharp uptick, it's just not an all-time high.

</comment></comment></comment></comment><comment><USER20> How many of those are Uber drivers I wonder... ;)</comment>


Parallel Map in Elixir - <AUTHOR> http://www.selectedintelligence.com/post/116327140769/parallel-map-in-elixir <comment><USER2> Elixir is great. It is built on an awesome industrial grade, battle tested VM with really unique features like lightweight but memory safe processes. Has a friendly syntax (although I personally like Erlang's syntax).

Another great thing about Elixir is the community -- it is helpful, friendly, oriented for learning and newcomers. Also Jose Valim is really an incredible guy.

<comment><USER3> I'm really surprised at how _fast_ the Elixir folks are moving. They're building all kinds of stuff that took Erlang a long time to get. First and foremost the Phoenix web framework.

<comment><USER4> It helps that Erlang already did the heavy lifting for us Elixir folks (or what heavy lifting _wasn 't_ already done was then done by Valim and friends in Elixir, Plug, Ecto, etc.). A pretty hefty amount of Elixir code (including most frameworks, like Phoenix, Sugar, Weber, etc.) leverages the same old Erlang/OTP, just with some funny-looking (in some cases better, in some cases perhaps worse, depending on perspective) syntax and Plug. Even the dominant web server in the Elixir realm - Cowboy - is actually entirely written in Erlang. Compatibility with existing Erlang code (and vice versa) makes for fewer wheels being reinvented.

It also helps that Elixir came about right around the time when folks were starting to realize that this old telecom language called "Erlang" might actually be precisely what we need for modern web development, hence why web frameworks ended up evolving pretty quickly relative to Erlang (which also saw a growth in available web frameworks at around the same time; Chicago Boss is a particularly-excellent example).

</comment></comment></comment><comment><USER5> Here is another cool version of parallel map in Erlang that uses a fold instead of two parallel maps.

      parallel_map(Fun, List) ->
          Last = lists:foldl(fun(Value, Parent) ->
              spawn(fun() ->
                  MappedValue = Fun(Value),
                  receive
                      Rest -> Parent ! [MappedValue|Rest]
                  end
              end)
          end, self(), List),
          Last ! [],
          receive
              Result -> Result
          end.

It essentially sets up a chain of processes which pass their results to their parent when they finish their mapping operation.

<comment><USER6> This implementation does rely on a lot of memory copying going on there. Rather than copying the original element to the new process, then the new mapped element back to the original process (giving you the original list's worth of memory copied twice), you'll be copying each list element once to every worker you spawn, and then you'll copy the parts of the list worked on to the next worker, on and on, until the full list is finally copied back to the caller.

So instead of copying 2n elements over the processes, you'll be copying roughly 2nlogn (or is it n(n+1)/2 ?) elements around.

<comment><USER5> One thing to note is that the the parallel map consisting of two mapping operations has a different hidden overhead. If a message is received by a process that does not pattern match with any clauses in the receive block, the message is stored in a queue. When a new receive block is entered, all messages in the queue are pattern matched against the new receive block. In the worst case scenario, the worker process mapping the elements will finish in list order, so a great many pattern matches will be tried.

The solution to this problem is just to store messages as they are received inside of a map data structure, so that there is no overhead for receives. This requires indexing the list which makes the code a lot more inelegant.

Edit: Given n processors and an input of size n, I believe the time complexities are: Two map solution: O(n^2)+O(f) Fold solution: O(n^2)+O(f) Two map with map data structure: O(n)+O(f)

Where O(f) is the asymptotic upper bound of the function being mapped

Edit 2: This has me thinking about what will be the most efficient way to assign an element from the list to worker processes. In Erlang, it's clear that there isn't a way to avoid the O(n) overhead since the original process must reconstruct the list in order using cons.

In an imperative language this isn't necessary. Perhaps the original process can recursively assign indicies by spawning two children worker processes which are given a range of indicies to work on (who then create their own assignment processes). I believe the overhead is then only O(log n)...

<comment><USER7> Just to comment, you're worrying about big O time complexities and that...really isn't what is going to dominate anything that is sufficiently complex to warrant a pmap rather than just a map. The constants on your computations will almost assuredly dwarf it.

The only O(n) operations are, yes, a completely degenerate case of when messages get sent back (which is -incredibly- unlikely; with Erlang's task scheduling you're likely only going to ever have a max message queue length of a few items, so it's more likely a constant factor. To get the degenerate case you would need it to finish in -reverse- list order, that is, would need to finish with the last one first, then the next to last one, etc), and when reversing the list(s) built up from the map at the end (as under the covers I'm pretty sure map is written to be tail recursive), which while technically O(n), is still incredibly fast.

<comment><USER5> I agree that worrying about the time complexity of the non-parallel portions of pmap is unlikely to be an issue for most use cases. It's still interesting to think about the tradeoffs though.

Hitting the degenerate case depends on the function you're computing in question. It's quite possible that the tasks will complete in the given order. I think you're giving too much credit to Erlang's task scheduler.

Also I'm not sure how one would even implement a tail recursive map function on a singly linked list. The cons operation can only add elements to the front of the list. I looked up how the map operation is implemented in Erlang. It isn't tail recursive:

[https://github.com/erlang/otp/blob/172e812c491680fbb175f56f7...](https://github.com/erlang/otp/blob/172e812c491680fbb175f56f7604d4098cdc9de4/lib/stdlib/src/lists.erl#L1236)

I'm interested to know how you'd implement a tail-recursive version of map (continuations aren't allowed).

<comment><USER7> Interesting that it's not; I would have expected it to be. Given that, though, it's spawning tasks in list order, then receiving in list order; if the tasks complete in the order you spawned them, the first thing in the queue/arriving is always the item you're receiving on. That's the ideal case; more likely they'd be nearly in the order you spawned them, in which case you'd only have a few items to check through before you found the one you're receiving on.

I'm not saying the task scheduler is perfect, but I'd be really, really weirded out if it gave priority to the final process spawned, and worked its way backwards, which would be necessary for the degenerate case (that is, we spawned off items 1,2,3,4 in that order, but they completed 4,3,2,1. I would expect them to finish in close to 1,2,3,4 order, which would leave it at O(1) on each receive).

I'd implement a tail recursive map as -

      map(F, L) -> map(F, L, []).

      map(_, [], Acc) -> lists:reverse(Acc);
      map(F, [H|T], Acc) -> map(F, T, [F(H) | Acc]).

</comment></comment></comment></comment></comment></comment><comment><USER8> Here is a parallel map in Haskell

        pmap f xs = map f xs `using` parList rdeepseq

The cool thing about this is that `map` is just the regular map function. However, as Haskell is lazy, `map f xs` returns a thunk and is not immediately evaluated. `using` allows you to evaluate the resultant thunk using any "evaluation strategy", and `parList rdeepseq` describes a strategy for evaluating lists in parallel!

Also, like the Erlang VM, GHC has fast, lightweight "green threads".

Edit: Another way to write it would be

        pmap _ [] = []
        pmap f (x:xs) = runEval $ do
                             x' <- rpar (f x)
                             xs' <- rpar (pmap f xs)
                             return $ (x':xs')

</comment><comment><USER9> Is this piece of code idiomatic Elixir/Erlang?

        receive do { ^pid, result } -> result end

Without looking up what the "receive" procedure does, it suggests that the current ("main"?) process is going to block waiting for an incoming message that matches the pattern. Done in a loop as I presume the Enum.map function does, it looks like the main process might be blocked waiting for the matching message while some internal mailbox fills up. In the worst case I would expect this to have an O(n^2) running time unless there is something fancy going on when pattern matching against a full mailbox.

Please correct me if I'm wrong. This looks elegant, but I would expect it to be considerably less performant than, say, the equivalent Go program which would issue a select on a channel of incoming messages, thus only blocking until _any_ message is received, rather than a specific message in series.

<comment><USER10> Not the case here, my friend.

From the horse's mouth:

"Each process has its own input queue for messages it receives. New messages received are put at the end of the queue. When a process executes a receive, the first message in the queue is matched against the first pattern in the receive, if this matches, the message is removed from the queue and the actions corresponding to the the pattern are executed.

However, if the first pattern does not match, the second pattern is tested, if this matches the message is removed from the queue and the actions corresponding to the second pattern are executed. If the second pattern does not match the third is tried and so on until there are no more pattern to test. If there are no more patterns to test, the first message is kept in the queue and we try the second message instead. If this matches any pattern, the appropriate actions are executed and the second message is removed from the queue (keeping the first message and any other messages in the queue). If the second message does not match we try the third message and so on until we reach the end of the queue. If we reach the end of the queue, the process blocks (stops execution) and waits until a new message is received and this procedure is repeated."

The Erlang VM accomplishes this by providing computation credits to each process, which are spent to perform actions ensuring the complete, non- blocking, parallel system that guzzles information at a startling rate all over the globe as we type this :)

<comment><USER11> Please correct me, but as best I can tell, the Enum.map will, sequentially, start the first receive, and that receive will operate against every incoming message until ^pid matches, adding its result as the first member of the collection while the unmatched messages accumulate in the mailbox; then it will do the second receive, which will scan the mailbox for its ^pid, and either find it or wait until the matching message arrives, adding its result as the second member of the collection; then it will do the third receive, etc. Assuming a random distribution of message arrivals, the last message to arrive will on average be the n/2 message, so for (on average) n/2 messages, the n/2 length mailbox (full of the (n/2) through n messages) will need to be scanned n/2 times, e.g., O(n*n). There might be some way to reduce the mailbox scanning time to O(1), but my shallow dive in the source code didn't show it.

<comment><USER12> Since the reason for the 'waiting till the one with the right order to come in' behaviour is the use of ^pid, presumably you could get rid of mailbox scanning time by getting rid of the caret (matching any message), so the mailbox never has >1 message.

This means you get results filling up the array in the order they come rather than the original order, but if order's important, you can always pass around an index and sort by that after the fact.

</comment></comment></comment></comment><comment><USER3> Erlang actually has this in the rpc module, in the standard library:

[http://erlang.org/doc/man/rpc.html#pmap-3](http://erlang.org/doc/man/rpc.html#pmap-3)

So you could call that from Elixir if you wanted.

<comment><USER13> One difference is that the Enum library works on anything that implements Enumerable. rpc.pmap only works on lists.

</comment></comment><comment><USER14> They don't handle errors very well.

</comment><comment><USER15> When choosing to apply parallelism, it should never be about the length of the collection but only the computational intensity of the function that's being applied. There's a considerable overhead in spawning processes and collecting the result from them

<comment><USER16> There's a considerable overhead unless they are _lightweight_ processes.

</comment></comment><comment><USER17> Seems silly to use the pipe to feed the collection into the first map call. Otherwise pretty nifty, I've been enjoying learning Elixir the last few months.

</comment><comment><USER18> Any recomendations on Elixir learning resources?

<comment><AUTHOR> Dave Thomas' book is an easy read: [https://pragprog.com/book/elixir/programming- elixir](https://pragprog.com/book/elixir/programming-elixir)

</comment></comment><comment> Ono-Sendai Looks somewhat ugly and also slow. (each element is processed in a separate process)

<comment><USER19> These are Erlang VM processes which are actually quite fast.

Re ugly: well, eye of the beholder I suppose.

<comment> Ono-Sendai Here's a map in my functional language (winter). It's not parallel yet, that shouldn't affect the syntax though. (map is already manifestly parallel)

def square(float x) float : x*x def main(array<float, 256> a) array<float, 256> : map(square, a)

<comment><USER20> Now make it efficiently parallel and show us the implementation. Then we could compare it with the Elixir snippet, which does just that.

<comment><USER21> Not just efficiently parallel, but easily distributable to multiple nodes.

<comment><AUTHOR> Exactly. With a tiny change you can be spawning those processes to any number of other nodes.</comment></comment></comment></comment></comment></comment>


MP3 is dead, long live AAC - <AUTHOR> https://www.engadget.com/2017/05/12/mp3-is-dead-long-live-aac/ <comment><AUTHOR> > The Fraunhofer Institute for Integrated Circuits, announced that it had > terminated licensing for certain MP3-related patents

What's the implication? My understanding is that MP3 is now patent-free. The wording seems to indicate that there are still valid patents on MP3.

<comment><USER2> They have terminated their licensing program because there is nothing to license anymore. The patents have run out.</comment></comment>


Prisoners' code word caught by software that eavesdrops on calls - <AUTHOR> https://www.newscientist.com/article/mg23030762-200-prisoners-code-word-caught-by-software-that-eavesdrops-on-calls/ <comment><USER2> I thought monitoring prisoners' phone calls was illegal?! Do authorities not care about the law at all these days?

<comment><USER3> IIRC, except for calls with attorneys and certain other exceptions, monitoring prisoners phone calls (like monitoring their visits) is a perfectly legal security measure.

That said, its kind of odd (to the point of being nearly implausible) that human monitors on the calls didn't make the connection between requests by prisoners for a "three-way" followed by the person referred to in that request being dialed in to the call by the person that was directly called until after the fact machine analysis identified "three-way" was a code word.

First, because using "three-way" (or more fully, "three-way call" for the call, "three-way calling" for the feature [0]) to refer to a call into which a third-party is added isn't obscure prison code, its standard usage, and second because even if human monitors were initially clueless as to the particular term, it defies reason to ask us to believe that they didn't notice the additional person being added to the call after the request.

This is also quite obviously a stealth ad (probably a press release being uncritically accepted and lightly rewritten and published as a news article) for Intelligent Voice, whose software is being promoted in the piece: there are no prison sources cited, the only sources cited are the company's CEO and one of their software developers.

[0] I mean, really, google "three-way call".</comment></comment>


Tiny JavaScript DOM selection library for modern browsers and IE5-8 - <AUTHOR> https://github.com/timcameronryan/cross-browser-selection

<comment><USER2> This is exactly what I need. Looks much more compact, and simpler, than <http://code.google.com/p/ierange/>

Edit: Well, that's embarrassing. It looks like you wrote ierange too. Kudos.</comment>


Ask HN: Is there any software like this (online marketing for business owners)? - <AUTHOR>

I have been speaking to a few small business owners and I have come to the conclusion that advertising online is a big pain point for them. Most of them are not tech savvy enough to be able to run adwords or facebook ads - let alone be able to measure conversion rates or A/B test them. They also do not have the budget to employ a company (or a professional) to do the same. These are hands-on business owners who want to do this, but just don't know where to start.<p>Is there any software or service that solves this pain point? <comment><USER2> I think that the premise of Hubspot is similar but also that it's way overpowered for SMBs.

FWIW I've seen exactly the same confusion that you describe in small businesses. I worked with the Yellow Pages for a while and went out in the field with their sales guys. They confirmed that most SMEs only think about advertising when the Yellow Pages sales girl walks through the door each year.

I would guess that a plug 'n play - put money in, get customers out for SMBs would do incredibly well. I'm not sure what it would take to deliver it but even if there is a service that's out there that does this, most SMBs do not know about it.

<comment><AUTHOR> Thanks. Hubspot looks pretty cool. I think you are right though - its an overkill for small b/s. The tool needs to be dead simple to use (even if it means limited options)

<comment><USER3> (disclosure: I worked at HubSpot.)

HubSpot is a pretty powerful tool but might be a little much for a small brick and mortar business. I think BuySellAds.com is a great (and simple) alternative to digging through Facebook or Google ads.

</comment></comment></comment><comment><USER4> There's Visual Website Optimizer and it's competitors. I've made good money in the past by setting VWO up on client websites and handling their testing for them. They aren't interested how it's accomplished or whether it's a software or service solution, only that before I started they sold 10x widgets a day and now they sell 15x. When x is worth several hundred dollars it makes them more than happy to pay good money for it to be done.

I suspect this is true for all but the tiniest of companies.

</comment><comment><USER5> At least in Europe, there are many companies helping people with Google ads... Some are dishonest though (like "I'm from Google", "We have a free slot on Google's first page", "your ad will show on each and every search request", etc...).

They're targetting professionals: I've come to know them when I was a photographer. And as far as I can tell, they're not very competent...

<comment><USER6> the reality is that if you know how to buy traffic online (meaning you can get high ROI) then you don't sell your consulting "services".

</comment></comment><comment><USER7> Small business owners can learn how to do this, through a step-by-step approach, and at a nominal cost. [http://affiliatemarketinguniversity.info/signup](http://affiliatemarketinguniversity.info/signup) It says affiliate marketing, but it's all about internet marketing. They can sign up for free and see if they like it, and they will get unlimited free coaching.

</comment><comment><USER8> I've been working on a startup called SMPLE.co (www.smple.co) that also tries to address some of this problem. Email us if you want to learn more - happy to share.

</comment><comment><USER9> Yes, the service is called an agency.

Software won't solve this. if you don't know how to use the Adwords editor, how can you use any software? makes no sense.

</comment><comment><USER10> There are numerous "pay for leads" companies. SMBs want the outcome, and are willing to pay for that. They have little time for DIY.

</comment><comment><USER11> Locbox(www.locbox.com) is a startup trying to address exactly this market.</comment>


Ask YC: Silicon Valley vs. the world? - <AUTHOR>

Hi,
I was just wondering what do you think are the  advantages of starting a web startup in Silicon Valley rather than somewhere else?
Thanks! <comment><USER2> Let's not do this again, please.

<http://searchyc.com/why+silicon+valley>

<http://news.ycombinator.com/item?id=208068>

<http://news.ycombinator.com/item?id=42933>

<http://news.ycombinator.com/item?id=174298>

<http://news.ycombinator.com/item?id=182746>

<http://news.ycombinator.com/item?id=174298>

<http://news.ycombinator.com/item?id=116600>

[http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#t...](http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#toc19)

[http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#t...](http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#toc20)

[http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#t...](http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#toc21)

[http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#t...](http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive#toc22)

And that's just what I found in a couple of minutes. There have been at least twice that many.

<comment><USER3> Also, <http://news.ycombinator.com/item?id=224110> .

<comment><USER2> upmodded for gratuitous use of recursion.

</comment></comment></comment><comment><USER4> Please have a look also at the Ask YC Archive from The Startup Wiki over here: <http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive>

</comment><comment><AUTHOR> Thanks</comment>


More Small Businesses able to offer healthcare - <AUTHOR> http://blogs.forbes.com/rickungar/2011/01/06/more-small-businesses-offering-health-care-to-employees-thanks-to-obamacare/ <comment><USER2> I'm not sure that article convinces me yet. Here is the experience of my employer, which offered 3 different healthcare plans for employees. Healthcare plan costs skyrocketed this year, to the tune of 30%-50% (not necessarily due to the healthcare bill, but a damaging hit nonetheless). As a result, the company was forced to eliminate the two most expensive plans that it did offer and consolidate down to the lowest cost one. In turn, many people were forced onto a plan they had never been on before.

He surveyed a couple different health insurance providers, but not a complete sampling by any means. Those providers he surveyed could very well have been the ones offering the cheaper plans. So I have to ask the question, could we be seeing a consolidation in the market rather than just an explosion of coverage?

<comment><USER3> The article seems to focus more on small businesses (fewer than 50 employees) who have not historically offered health care plans but are now starting to offer them because of the tax cut in the new law. The numbers appear to support the conclusion that at least this part of the law is working.

</comment></comment><comment><USER4> For those of us who offered healthcare to all employees prior to these "reforms", these new "incentives" offer me nothing but additional expenses and tax burdens.

Don't get me started on crow-baring the "1099 everyone" BS into it. I know its been discussed at length around here. Le sigh.

<comment><USER5> If you're already providing coverage, what are the additional expenses and burdens (aside from W2 reporting)?

<comment><USER4> I'm assuming W-2 reporting will be the easiest part, though it will be a pain.

The 1099 thing is a big hidden expense. My businesses are run as lean as they possibly can be so I'll have to hire someone to go and find all the tax ID's for all my vendors. Then I have to pay my accountant to generate all of them for me. I'd imagine cost of staying on top of this will be in the thousands / year.

The other thing is the Medicare tax. In a couple years I'll be paying an additional 3.8% to Medicare taxes on each dollar I make over $250k. This feels like a slap in the face given I'm already paying tons of moola for health care costs, which are now skyrocketing because of these "reforms." In general, those who are most likely to be affected by this new tax are the ones that are already paying (or will be forced to start paying) for others' health insurance.

Le sigh x2.

</comment></comment></comment><comment><USER6> Please don't call it "Obamacare".

<comment><USER7> > Please don't call it "Obamacare".

I agree it's stupid, but let me ask you since I've been wondering - what's the difference between using "Obamacare" derisively vs. "Bush tax cuts" derisively?

It seems to me that both are trying to paint a specific set of legislation as solely due to the president's prerogative. IE, I don't see people upset at the term Obamacare try to get people to call the tax cuts "the '01/'03 tax cuts" which is much less politically loaded and much less emotional...

(I do think both are stupid, and ignores the culpability of Congress in decisionmaking)

<comment><USER8> I really hope this doesn't turn into a political battle thread.

However, "Bush tax cuts" is an accurate term for the tax cuts provided by the Bush administration. It has no negative or positive (besides "tax cuts") meaning inherent in its wordings alone. "Obamacare" is a dumbed down slang for "Obama health reform", mostly because Obama, nor his administration, is providing care. It inherently does not describe what it represents, reform* not care.

When using either term derisively, then that is a whole separate ballgame. I will admit though that "Bush tax cuts" has turned into a phrase that one may use to speak ill about president Bush, but then the anger still should be directed towards that session of congress, not Bush himself.

I also agree that both are stupid.

* although I personally like the healthcare reform bill, one should always remember that just like evolution, the definition of reform does not suggest that the change it is describing has moved in a positive direction. There is no de-reform or devolution. Change is change, how positive or negative it is can only be valued by each individual separately.</comment></comment></comment>


Before and After pictures of Japan - <AUTHOR> http://www.abc.net.au/news/events/japan-quake-2011/beforeafter.htm

<comment><USER2> The "after" pictures are in a different tint or colour scheme which means they are biased (or rather, you are biased by the colours like the missing green). Bad comparison.

<comment><USER3> Why are the images tinted though? They did the same interface for some flooding that happened in Australia, and the 'after' images were also tinted similarly.

<comment><USER4> They aren't tinted. It's all about white balance and levels.

My guess is that satellite images are usually heavily edited to give them a natural appearance (and also to make them consistent with other satellite images) but right after a disaster that editing step is probably skipped due to time constraints. I would imagine that the post-disaster images are much closer to the raw satellite output. I turned on grayscale on my Mac – that helps a bit in that it eliminates the color differences but contrast and levels still do not match.

The destruction is still very obvious. Pay, or example, close attention to bridges, buildings and trees.

</comment></comment></comment><comment><USER5> I could be spatially challenged enough to not really see it, as the imagery is lacking perspective, but these pictures aren't much after seeing the video footage of the waves rolling in. _That_ was a much starker (re-)realization of their power, similar to 2004.

Perhaps images from a helicopter or high ground would be more evocative.

</comment><comment><USER6> I, I cannot write my accurate feeling to the article. It is amazing as web interface, but these images almost make me cry...

<comment><USER7> This interface is so good it's freaky. It's too good. It makes me feel almost responsible for the power of that wave — even as every rational cell in my body begs me that this is not so.

</comment></comment><comment><USER8> Third picture from the bottom, a new building magically appears in the after picture

<comment><USER9> Old satellite imagery in the "before" shot, meaning that building is fairly brand new if a bit recently weathered.

</comment></comment><comment><USER10> Apparently the earthquake moved the whole island by about 8 meters!

I wonder how that's going to affect GPS maps. 8 meters is enough to put you in a different street on your map.

Edit: Oops. 2.4 meters, about 8 feet.

<comment><USER11> 8 feet - a little less then 3 meters. But impressive nonetheless.

</comment></comment><comment><USER12> As a tool, these visual diffs are amazing. I'd love to see this feature built into the Google Maps satellite view.

</comment><comment><USER13> Very nice link thanks.With those pictures, we can realize how terrific was the impact of this tsunami..

</comment><comment><USER14> Please don't use hover for performing actions; your page won't work on touchscreen browsers.</comment>


When a $2.1M Drug Could Cure Your Child's Fatal Disease - <AUTHOR> https://www.bloomberg.com/news/features/2019-08-20/how-parents-get-zolgensma-the-world-s-most-expensive-drug <comment><USER2> I'm sure this will not be a popular opinion, but it's really too bad that more chemists don't have the altruistic drive to reverse engineer such things and open source them, so to speak. Maybe open source is the wrong term. Put the formulas and procedures on the dark web and dumb down the procedures enough for desperate amateur chemists to tinker with.</comment>


Data Science Past and Future - <AUTHOR> https://blog.dominodatalab.com/data-science-past-future/ <comment><USER2> This is a great insightful talk; starts with interesting historical perspective. Great video, great slidedeck. Why no comments?

<comment><USER3> Agree, this is a gem and I wish more talks like this made it to the front page. As a long-time HN lurker, this post finally got me to create an account and start commenting.

I really appreciate the practical perspective of the talk and focus on what's happening in industry. I've seen a lot of data science/ML posts on the latest research and how this can lead to automating X in Y years. While it's great the research is advancing, there's not as much focus on how the industry is evolving to capture the value of that research.

I love the range of topics Paco covers in his talk, from explaining how we got to today's data science environment to what's differentiating big players from the aspirers to making predictions on future M&A activity. The deck is also full of useful links and resources - I especially enjoyed Cassie Kozyrkov's talk on "The Missing Piece".

Great talk, and thanks for sharing.</comment></comment>


Some light reading on lock-free programming - <AUTHOR> http://blogs.msdn.com/b/oldnewthing/archive/2014/11/27/10576382.aspx

<comment><USER2> As I understand it, these techniques are oriented towards producer-consumer queues where there is only 1 producer and only 1 consumer. My software does IO in this manner. What is confusing is that these schemes don't scale beyond multiple cores for obvious reasons (1 consumer, 1 producer).

I understand that I can reduce the few ms latency when acquiring a std::mutex, and perhaps this is valuable for high frequency trading, but I am not sure if this will improve performance in a palpable manner.

Does anybody have any good uses cases for lock-free programming where the goal is to scale beyond a single producer-consumer scheme?

<comment><USER3> Lock free techniques can actually hurt performance compared to mutexes. Their advantage is predictability. A lock will often be fast and sometimes be really slow when it has to wait for some other thread. A lock less algorithm runs at about the same speed no matter what. This is important for realtime tasks such as audio recording and playback.

<comment><USER4> My understanding was that a lock free algorithm could still wait for an unbounded time, and what you're referring to are called wait free algorithms.

<comment><USER3> You're right, but much depends on how much performance you need and how critical your realtime performance is. A lock-free algorithm may retry, but it will likely only retry once or twice. Your running time on any given operation may occasionally double or triple, but it won't hit you with six order of magnitudes like might happen with a mutex if it's contended and the thread that owns a lock takes a page fault or something.

I'd probably want to go wait-free if I was writing software for a Mars lander, but lock-free is good enough for consumer audio tasks, where mutexes generally aren't.</comment></comment></comment></comment>


MapBBCode Share: a simple service to share custom maps and convert geodata - <AUTHOR> http://share.mapbbcode.org/

<comment><USER2> Ditto: [http://geojson.io/](http://geojson.io/), [https://github.com/mapbox/geojson.io](https://github.com/mapbox/geojson.io).

There's a few npm modules for the command line, so you can do the likes of `geojsonio < some.geojson` or `togeojson my.{kml,gpx,…} | geojsonio`

geojson.io uses gist.github.com as a data store: [http://geojson.io/#id=gist:anonymous/92dd314a02c072d658f4&ma...](http://geojson.io/#id=gist:anonymous/92dd314a02c072d658f4&map=4/-78.38/-48.6041)

<comment><AUTHOR> I know of that website, it is made by a fellow OSMer, and while it's perfect for creating and editing GeoJSON, it lacks in simplicity of sharing maps (well, it depends on an external site for that). For example, MapBBCode Share maps can be embedded into posts on any forum or blog with MapBBCode plugin installed. Both sites were made with the same technology stack, though, namely Leaflet + Leaflet.draw.

And of course MapBBCode Share maps can be loaded in geojson.io: for example, [http://geojson.io/#data=data:text/x-url,http%3A%2F%2Fshare.m...](http://geojson.io/#data=data:text/x-url,http%3A%2F%2Fshare.mapbbcode.org%2Fzdjzk%3Fformat%3Dgeojson) (replace zdjzk with map id).

</comment></comment><comment><USER3> Love that it uses OpenID for the login!

If you are/were using MyOpenID, you can switch your delegation to IndieAuth since they are shutting down. It's an easy switch: [http://aaronparecki.com/articles/2013/09/15/1/indieauth- now-...](http://aaronparecki.com/articles/2013/09/15/1/indieauth-now-supports- openid-delegation)

<comment><AUTHOR> Oh, thanks for reminding. I was told this before, but forgot.

</comment></comment><comment><USER4> I like it. What are the file formats that it can it import? I tried a .gpx file and it worked nicely.

<comment><AUTHOR> Click the "Export" button, there will be a list. Basically, almost all of those: [https://github.com/MapBBCode/share.mapbbcode.org/tree/master...](https://github.com/MapBBCode/share.mapbbcode.org/tree/master/formats) (except html, iframe, nmea).

</comment></comment><comment><USER5> Nice service indeed, I like it more than anything I tried before.</comment>


JPEG XL could let you pack twice as many photos into your phone - <AUTHOR> https://www.cnet.com/news/jpeg-xl-could-let-you-pack-twice-as-many-photos-into-your-phone/ <comment><USER2> It's somewhat odd to suddenly see lots of excitement in the 'continuous-tone still images' space, and I can't help but think that on some level, JPEG the group to wants to retain relevance at a time when the world is finally looking poised to move on from using its oldest work.

A still image file containing one frame of a video codec has been long theorized, pushed very aggressively since 2010 by Google as WebP (i.e. VP8-in- RIFF), and as a fully-functional thought experiment by ffmpeg's Fabrice Bellard since 2014 in BPG (HEVC-with-custom-header), the latter of which jumped the gun on HEIF before it was done.

Now that HEIF is here, we have a better container encapsulating an advanced codec, and suddenly we have hardware support with premium phones generating files in this format. The barrier is HEVC's hefty licensing pool, but a parallel effort exists in AVIF [1], which takes a frame of the royalty-free, HEVC-competitor AV1 video codec, and stuffs it into a HEIF container.

It seems a bit superfluous for JPEG to barge into this space _right now_ ; I'm confident they'll do technically solid, and even innovative work, but proliferating an incompatible implementation right when it looks like there's a realistic chance of a newer still-image codec replacing one that dates back to 1992 is a bit counterproductive.

But perhaps I should be more optimistic. Their calls-for-proposal document [2] lays out recent competitors, admits their technical merits, but concludes that they're not in widespread use. Then, it lays out that their vision to produce a state-of-the-art format that will achieve widespread adoption, helped by a "royalty-free baseline".

[1] [https://news.ycombinator.com/item?id=16803363](https://news.ycombinator.com/item?id=16803363) [2] [https://jpeg.org/downloads/jpegxl/jpegxl- draft_cfp.pdf](https://jpeg.org/downloads/jpegxl/jpegxl-draft_cfp.pdf)

<comment><USER3> >A still image file containing one frame of a video codec has been long theorized

The I-frame encoding in MPEG-1 was basically identical to baseline JPEG. The compressed frames can be losslessly converted to JPEG and vice-versa. So, in a sense, we were already there.

Reusing the I-frame compression from a video codec is the only strategy that makes sense for a lossy still image codec. Video codecs are where the major innovation is happening, with wide participation from every player in the industry, and a large part of the innovation has been around improving I-frame efficiency. They're being hardware optimized and baked into software implementations. And even when patents are involved, still image codecs are usually able to piggyback on the video codec licenses that are already on the device.

<comment><USER4> It is interesting that H.262 ( MPEG 2 ) > H.263 ( Divx ) > H.264 ( AVC ) > H.265 (HEVC )

That is 4 generation a video codec improvement, and yet as an image format, MPEG 1 based Jpeg is still doing very well compared to latest HEIF. I mean four generation of improvement and we only got 50% reduction at similar quality. Why is it still image don't scale down as well as video?

<comment><USER2> Because many of the gains of successive video codecs comes from finding more efficient ways of encoding _motion_ , which is something you clearly can't exploit with stills.

</comment></comment></comment></comment><comment><USER5> A lot of people have already (unknowingly) produced a ton of HEIF photos. It's the default on newer versions of iOS I believe.

<comment><USER6> It's a great surprise to realize you've been shooting HEIFs for months and then can't open them on your MBP because you're "only" on El Capitan.

<comment><USER7> That shouldn't be a surprise, because in order for that to happen you'd have to disable the compatible export option in iOS that's on by default. Unless you used some 3rd party image transfer that wasn't as thoughtful?

<comment><USER8> The default Formats setting for the built-in camera app is "High efficiency", which saves as HEIF/HEIVC. You have to change it to "Most compatible" to use JPEG/H.264.

<comment><USER7> The default export option is 'automatic' which converts to jpeg when exporting to anything other than new Apple OS devices. The format for taking the photos is irrelevant.

<comment><USER8> That's in the Photos app settings. I have no idea if Google Photos does automatic conversion too.

I think someone mentioned before in this thread the syncing between Android and iPhone using Google Photos and finding the format incompatible on Android.

</comment></comment></comment></comment></comment></comment><comment><USER9> Isn't part of the issue the network effect, not the file format?

(We can develop a whizzbang new compression algorithm, but unless Windows, macOS, iOS, Android, and all the flavors of Linux add support it can't go beyond the device)

<comment><USER10> I'd say so. That was one of the things that killed JPEG2000. Even if people bought encoders no one could view it anyway so it was useless.

EDIT: Comr to think of it this was the problem with APNG trying to replace GIF. Everything supports GIF, no one supported APNG (relative to GIF) so it never really gained traction.

Even though 75% of users have a browser that supports it the format has no mindshare so no one uses it.

<comment><USER4> It is interesting the JPEG CFP pdf note JPEG2000 as being widely successful, when it is barely used outside certain domain. And it names JPEG2000 being royalty free, but on Wikipedia they mention they problem of patents. And it is only free for baseline ( Whatever that means )

<comment><USER11> DICOM, a medical imaging format (meta-format? The spec is a beast) supports JPEG2000 compression. There are a _lot_ of DICOM files kicking around, though annoying not every device can write files with JPEG2000 compression, nor can every reader actually interpret them.

I also had a no-name digital camera that inexplicably supported JPEG2000.

Other than that, I'm don't think I've seen it much in the wild.

<comment><USER12> It's supported in Safari (c.f. JPEG XR in Edge and WebP in Chrome).

It is also used for digital cinema distribution. It was chosen for that role specifically because it was not in widespread use.

</comment></comment></comment></comment></comment><comment><USER13> Who remembers when ~~JPEG2000~~JPEG XR was going to be the next big thing?

<comment><USER14> FWIW Patents killed those, not lack of quality or interest.

<comment><USER12> There were patent concerns when they were first released, but AFAIK they were all eventually resolved. The issue really was lack of (perceived) quality. Despite their amazing PSNR scores, wavelets _look_ terrible at low bitrates, in ways that DCT-based codecs do not. JPEG XR has a few more features than JPEG, but not really much better quality [1].

[1] [https://web.archive.org/web/20160404051240/http://people.moz...](https://web.archive.org/web/20160404051240/http://people.mozilla.org/~josh/lossy_compressed_image_study_july_2014/)

</comment></comment></comment><comment><USER4> HEIF already does similar compression to this JPEG XL now, which is at least 2 - 3 years away.

We need something even better, compressing 150Kb Jpeg, a common image size on the internet, to 50Kb with similar or better quality.

I hope we do better with next gen video codec VVC.

<comment><USER15> Heif is now the default camera format on iPhones, ensuring that it is the winner of the next gen format race before it even started.

Everything will have to support heif, or lose every iPhone owner as a user.

<comment><USER4> Well, that is assuming everyone will support HEVC, which Google seems to be the major force opposing it. Youtube won't use HEVC.

<comment><USER15> It's irrelevant what YouTube uses behind the scenes, since they don't (officially) offer a download service. It always plays in the browser or the app.

You can _upload_ HEVC to YouTube just fine, and it works. So in the only way that counts, they do support it.

<comment><USER4> But they won't "encode" HEVC. They are only offering AVC, VP9 and very likely AV1. With any 4K or 2K+ content only available in VP9 / AV1.

The battle between AV1 and HEVC is not set. And so is the battle between HEIF and AVIF. HEVC have the upper hand right now. But that doesn't mean it has already won. iPhone does allow auto convert to Jpeg for compatibility reason.

So if you cant guarantee HEVC licensing issues revolved, and not having Google on board, HEIC will just be another internal format being used by Apple and Apple only. No Chrome, Firefox, Opera, Android support. ( Although Microsoft seems to be on board )

</comment></comment></comment></comment></comment><comment><USER16> What's the patent situation for JPEG XL?

<comment><USER2> _" This new JPEG activity aims to develop a new image coding standard that provides state-of-the-art image compression performance, and that addresses shortcomings in current standards. To encourage widespread adoption, an important goal for this standard is to support a royalty-free baseline."_ [1]

[1] [https://jpeg.org/downloads/jpegxl/jpegxl- draft_cfp.pdf](https://jpeg.org/downloads/jpegxl/jpegxl-draft_cfp.pdf)

<comment><USER4> It turns out JPEG2000 is also royalty free baseline. So that statement changes nothing.

</comment></comment></comment><comment><USER17> How badly patent encumbered is this one?

</comment><comment><USER18> How's it compare to a Guetzli'd JPEG?

<comment><USER19> JPEG XL is a new standard. Guetzli is a (extreme) compression approach for the existing standard.

<comment><USER18> I'm aware. I'm asking how image quality to file size compares.

</comment></comment></comment><comment><USER20> How about JPEG2000? WebP? BPG?

<comment><USER21> * JPEG 2000 is basically dead in the water. It does not offer substantial advantages over JPEG for common use cases.

* WebP is a good alternative.

* BPG has potential licensing problems.

<comment><USER22> HEIF, same idea as BPG (use HEVC intra-frames as images) but by the HEVC group, and already supported natively by Windows, OSX, iOS and Android.

<comment><USER21> HEIF is already successful and in widespread usage.

<comment><USER23> Only by virtue of Apple adopted it for its hundreds of millions of devices, but you could make the same case for anything Apple alone adopts. That alone doesn't make it a standard.

I for one hope the web will end up adopting the Netflix-backed, AV1-based, and royalty-free AVIF:

[https://aomediacodec.github.io/av1-avif/](https://aomediacodec.github.io/av1-avif/)

<comment><USER10> Same argument for Google's VP9. It has TONS of users because of YouTube.

People complain _that's_ a standard and Apple needs to support it.

No difference.

But as others have said, MS is supporting HEIF on Windows too.</comment></comment></comment></comment></comment></comment>


Bruce Schneier: you can secure America's e-voting systems tomorrow, use paper - <AUTHOR> https://www.theregister.co.uk/2019/08/09/schneier_voting_security/ <comment><USER2> Paper is secure because we are familiar with it and because people are bad at conceptualizing intangible things.

However,I don't think we should give up on secure electronic voting because it can solve issues with paper voting,especially with new democracies. In some countries,no matter who wins one side claims it's rigged and you have mass violence every few years. I don't have a solution but I think R&D investment should continue.

</comment><comment><USER3> Canada runs all their elections with this technology and with very few controversies even in hotly contested ridings.</comment>


Are There Cell-Scale Anatomical Coordinate Systems? - <AUTHOR>

For to accurately and repeatably describe positions, you need to agree on some kind of coordinate system.
Anatomical coordinate systems are tricky, because you're people are deformable and different.
If I wanted to be able to say "drive down this artery 1000 cells, and then go to this cell in the wall." Is there an existing coordinate system that  could do this? Are there software libraries? <comment><USER2> Yes and no. Yes, there are general tissue-level anatomical structures that have consistent, predictable topologies. No, there's not a standard way to map these onto a cell scale coordinate system easily. When you are talking about cell-level anatomy, you can almost think of things as being slightly disordered, with the disorder increasing the smaller in scale you go. Most tissues have constant turnover of cells, further complicating the concept of a coordinate system. Some examples where there is a consistent (almost) unchanging anatomy would be in neurons and stem cell germ layers.For germ layers, simple 3D coordinates would work. If you could figure out a coordinate system that works for neurons, you would get a Nobel prize. Sorry I couldn't help you more. What exactly are you hoping to do with said library?

<comment><AUTHOR> I'm trying to imagine how you'd tell a cell-scale machine "go to that cell ... that one there", rather than "drive along this chemical gradient", or "bounce around until you match this receptor."

<comment><USER2> Well that's an even more complicated problem. That would depend on what type of cell you wanted to target and in what tissue. Most tissues are quite heterogeneous and constantly remodeling. Even if you could get the coordinates of a particular cell before it moved, targeting the specific cell (and not the others that are likely to be present in the vicinity)is a problem unto itself. Don't diss diffusion and biorecognition, they're quite effective. In my mind, any robot designed on the scale to target individual cells would have to mostly rely on circulation, diffusion, and chemistry to target a particular cell type. The mechanics are far from deterministic on the nano scale. I highly recommend this book if you're considering a project like this: [http://www.garlandscience.com/product/isbn/0815341636](http://www.garlandscience.com/product/isbn/0815341636). If this is for a visualization or a game, I would just fake the coordinate system.

<comment><AUTHOR> Thanks!

</comment></comment></comment></comment><comment><USER3> > For to accurately and repeatably describe positions

IMO, for most use cases, you don't need to describe positions; you provide something analogous to a SQL where clause.

Most querying in cell biology appears to content-based rather than coordinate- based. "drive down this artery 1000 cells" is not useful because your target might be 1010 cells down next week. Instead, you might say "drive down this artery until you find a cell with this particular receptor on the cell wall." Rather than keeping track of how many cells you've passed, you randomly bounce around the artery until you bind to a compatible receptor.

<comment><AUTHOR> Good points. I'm coming from a pretty speculative direction where I'd want to enforce invariants, like: repair this tissue in-situ so that its thickness is always between x and y. Or make sure the density of this kind of cell in this region is at least A, and no more than B.

Basically, 3d modelling for tissue. How do you specify the reference model?</comment></comment>


The first news article ever written about Apple - <AUTHOR> http://www.businessinsider.com/first-article-ever-written-about-apple-kilobaud-sheila-clarke-craven-2015-5

<comment><USER2> Side note. Does anyone else think that the original author (photo in article) seems to have stayed the sane age for 30 years, and even looks younger now than in 1977?

<comment><USER3> Funny, I didn't see your comment, read the article and thought the same thing. She looks a lot younger now. Maybe it is just the advances in cameras that makes her appearance that much better. If that's the case I can stop going to the gym now because I don't have anything to worry about in 30 years.

</comment></comment><comment><USER4> Lol at the headline "Why so many computer languages?" Some things _certainly_ haven't changed.

<comment><USER5> The issue is available online, if you want to read it: [https://archive.org/stream/kilobaudmagazine-1977-02/Kilobaud...](https://archive.org/stream/kilobaudmagazine-1977-02/Kilobaud_1977_February#page/n27/mode/2up)

</comment></comment><comment><USER6> The article list reads like what Hacker News would have been in 1977.

Why so many computer languages?

The hobbyists operating system

The 8080 you may have missed

...

</comment><comment><USER7> Quote from the KiloBaud article:

"What You Get for $666.66" heh ... really Steve? ... :-)

<comment><USER8> Blame Wozniak.

According to his autobiography, he was unaware the numbers even meant anything, he just liked them.

</comment></comment><comment><USER9> and today we use more processing power in our phones than likely all the early Apple computers put together.

What is most telling was Steve's focus on the user and the idea that the hardware underneath doesn't matter but the experience the user has should be consistent

</comment><comment><USER10> Don't forget you will need a keyboard, screen, AAAND "two transformers"!

<comment><USER11> Well, Jobs said he intended to have a newly designed single transformer out sometime soon (rolls eyes).

<comment><USER12> I love the part about how if the keyboard you've scrounged up uses the wrong logic convention, you can just solder a 7404 on the board in the spot provided.

Truly a different era.</comment></comment></comment>


Hoplon: A Simpler Way to Program the Web - <AUTHOR> http://hoplon.io/

<comment><USER2> Looks like a great project.

Is it just me, or does it seem like cljs is making a big stir these last few weeks? After a few years with NodeJS, I've always wanted to do some more purely functional programming; I've seen a lot of buzz about Clojure/cljs and have started considering it for my next project.

<comment><USER3> ClojureScript specially has taken off. This project, then you have the extreamly cool (but a bit hard to understand) pedestal ([https://github.com/pedestal/pedestal](https://github.com/pedestal/pedestal)), the Om ([https://github.com/swannodette/om](https://github.com/swannodette/om)) library has made a splash even in the JS community.

A lot of exitment around core.asnyc (witch allows CSP like Go) on the web.

<comment><AUTHOR> I'm really looking forward to pedestal. From what I understand the team is vacationing, but that there are plans to integrate core.async. I do wish that its development were a bit more public, though.

<comment><USER3> I would probebly not bother with following the devlopment to closly, I will wait until they feel like it is realy for early testing.

Look at this talk, [http://www.infoq.com/presentations/pedestal- clojure](http://www.infoq.com/presentations/pedestal-clojure), I think it gives the most up to date ideas about pedastal and the next version.

</comment></comment></comment></comment><comment><USER4> > Reload this page with JavaScript disabled and see how the content was > "prerendered" at compile time.

To their credit, the page is not blank, unlike a lot of "web" authors out there. Unfortunately there are several links that point to invalid and nonexistent fragments like #/home/ instead of actual elements of that or some other document, which really means the framework is happy to make a siloed javascript app and does not ensure you are responsibly making all your content available as part of the world-wide web at some stable URL where third parties can reference it.

tl,dr: this encourages single-page apps which will be the death of the web.

<comment><USER5> > single-page apps will be the death of the web

What en earth makes you think that? Yeah, the current way of doing SPAs without fallback for non-js clients is horrible and the alternative of a two- way site too much work.

But the solution is obvious: if what you serve is of any meaning to anyone else, you also have some sort of content consumption API. Sooner or later we'll standardize some RESTy way of doing it for apps, and for blogs and magazines RSS/Atom feeds are already good enough content-consumption pseudo- APIs.

 _The API will be the Javascript-less version of all sites, and all sites will have an API._ Then people will make UIs for the content consumption parts of these (pseudo-)APIs, and these UIs, even if they run themselves in browsers will become the new "browsers" and gain their own scripting level features, while "old browser apps" will become today's equivalent of desktop apps.

It's not "death" of the web, just "endless, infinite pain" for us developers working in its perpetual shifting front-end... _it 's way worse than death! :)_

<comment><USER6> > Sooner or later we'll standardize some RESTy way of doing it for apps

We used to have that, and it was called HTTP and semantic HTML.

<comment><USER7> No we didn't.

Nobody used "semantic HTML" to do anything serious with cross-webpage interoperability (where nobody means less than 1% of web developers/pages). As for TBL's "semantic web" that was dead on arrival.

</comment></comment></comment></comment><comment><USER8> I feel like despite the great things these projects are trying to achieve, in the end they are just hacks. I think its time we try to shift the web standards to meet modern needs rather than trying to build around the limitatns right now. The web is the future, and there's no point putting off the inevitable.

<comment><USER9> At the same time, developers need to ship using the tools that are currently available. I think a multi-pronged approach is inevitable. Of course, I say this as someone who avoids this sort of tool as a general rule, since it seems risky to use in production, so I definitely see what you are saying. On the other hand, this sort of thinking would probably have been an argument against jQuery several years ago, and despite the fact that it was a 'hack' over the awful js standard api, it became a de facto standard through brute force of popularity.

</comment></comment><comment><USER10> It's a pitty they do not show how client server communication is done. Eg how, after a click on a button, do i query a db and display its result in the page.

<comment><USER11> There's an examples Folder on GitHub with a client/server chat demo. That really helps figuring the RPC part out

<comment><USER12> link : [https://github.com/tailrecursion/hoplon- demos/tree/master/ca...](https://github.com/tailrecursion/hoplon- demos/tree/master/castra-chat)

</comment></comment></comment><comment><USER13> Does this use react from Facebook? React is supposed to be the last missing piece for clojurescript.

<comment><USER14> Doesn't look like it. Additionally it appears to be conceptually heavy. It's hard to convince Clojure users to use anything but small libraries, even Noir (a Sinatra-like framework) was abandoned for this reason. This might be really good but the amount of buy-in required is a hard sell to the community.

<comment><USER15> it has its own impl of rp that doesnt seem as advanced as react. I think there have been some performance issues as well.

<comment><USER16> Hoplon's reactive model is quite different from React's. We think it's more advanced :). There are no "performance issues" reported lately (there was some confusion with an old proof-of-concept demo, but any recent version of Hoplon will provide acceptable performance).

Hoplon isn't optimized for "perf" in rendering things on the screen. Hoplon is designed to uncompromisingly optimize developer performance. I know that when my users are presented with a choice between more rendering perf and added features in the application it's a no-brainer. They just need good performance, but great features.

Hoplon presents a model in which web designers and programmers can coexist without friction, and it provides the underlying foundation for developing truly powerful abstractions (like components, for example) that are composable in ways that are impossible in other systems.

</comment></comment></comment></comment><comment><USER17> So what is the performance benchmark for Clojure / ClojureScript? I've seen a lot of buzz for Clojure as of late, but given how bloated the web is at the moment, until I see Clojure (or Haskell) make leaps-and-bounds better performance in return for the pain of learning the language (and the barrier to functional programming), how can we hope to mass-adopt this technology?

<comment><USER18> Performance isn't everything and in many cases not that important. Clojure and Haskell make leaps-and-bounds better performance than Python and Ruby which both have mass adoption.

</comment></comment><comment><USER19> Lost me at " Use a spread�sheet-like dataflow pro�gram�ming en�vi�ron�ment to man�age client state."

</comment><comment> jw- For those interested in the learning more about single language web applications the Links project ([http://groups.inf.ed.ac.uk/links/](http://groups.inf.ed.ac.uk/links/)) has been going for some time with quite a few research publications.

</comment><comment><USER20> I like the name.

</comment><comment><USER21> No.

</comment><comment><USER22> The funny thing is that even with "Single-page applications, not documents" you still have a document... your "single-page".

Besides, that's NOT dataflow programming (see old papers for Fabrik, Prograph, InterCons, etc).</comment>