Om Malik: Silicon Valley's Talent Crunch - <AUTHOR> http://gigaom.com/2010/10/13/silicon-valley-talent-crunch/

<comment><USER2> I think another factor Om didn't mention, which is also at play, is that an increasing number of talented engineers and designers are striking off on their own and building their own startups. It's not so much they're being poached by other companies as they are poaching themselves. There's so much cheap infrastructure, frameworks and open source packages today that I think the balance of power in software startups is shifting more towards talent inputs, and away from mere cash inputs.

<comment><USER3> Yeah, I think top engineers have realized that even if they don't succeed as a business, companies like Facebook will often buy them for their talent with more upside than they would have gotten being hired through the normal means.

But at some point that will stop being true, so now more than ever I think it's important to be trying to build real, sustainable businesses.

<comment><USER2> yep, plus when you work for somebody else in a traditional job you get put in a narrow slot. just an engineer. just a designer. whereas if you start your own business or (sometimes) work at a startup you can be an engineer, graphic designer, UI, product, writer, marketing, sales, biz dev, bookkeeper, etc. plus pick your own tech, tools, office, hours, dress code, etc. And even when you "lose" at a startup or self-employment venture you still win in many ways because of all the skills and experience you exercised and how much better your resume can look.

</comment></comment></comment><comment><USER4> Why is no one mentioning the somewhat obvious aspect that people might not want to live in CA?

At some point, all of the available in state talent is going to be employed. Now you have to look outside. Much of your senior talent are not going to be able to up and move. There's a big difference between working insane startup worloads with a company in your home town vice doing the same AND doing it a new city after having moved your family across the country.

<comment><USER5> _people might not want to live in CA_

And suddenly a lot of people can't. The financial crisis has toughened up immigration everywhere.

</comment></comment><comment><USER6> Personally, I am at a crossroads. I have suitable spare time to start pursuing ideas I think will provide the basis for a good idea or even work on the side with an interesting startup. I'm into boring backend things like pipes, data, security and infrastructure (software not hardware). However, owning a house and living in the South Bay puts geographic limits on where I am willing to engage with companies on a daily basis.

In addition to people starting their own thing, I've also noticed that a number of people are trying to focus a bit more on the work / life balance. Trying to get out more and enjoy nature/work out/etc. instead of working 20 hours a day. And, in some ways, this is impacting their decisions between startups and large companies.

</comment><comment><USER2> > More startups competing for fewer talent resources will mean that the cost > of doing business is going to go up, which in this era of on-demand > infrastructure from the likes of Amazon Web Services means SALARIES, which > are essentially the single biggest component of any startups's spending.

Reading that sentence alone, and then thinking about how much big corporate CEO's are paid, and banksters and Wall Street types, hedge fund managers, etc. and as a software engineer I tell ya my heart just bleeds for them. What a horrible tragedy, having salaries for talent go up. So irrational! :) Heck, at least the engineers and designers are actually building something, and adding to society, which is more than many of those other types can say.

</comment><comment><USER7> Maybe now tech companies will wake up and start to re-think their ageist hiring policies.

<comment><USER4> I don't know that it's ageism. Yeah, I'm sure that's a valid concern but it's not "this person is too old for our company culture".

It's more "can this person commit in the same way that at unattached 25 year old could commit?".

Typically that answer is "no" but it's not fair to make that assumption for them and it is quite honestly a stupid assumption to make.

<comment><USER7> If, as you say, _"typically that answer is 'no'"_ , then why do you also say it's a stupid assumption to make? I agree with the second part, not with the first.

<comment><USER4> Even if the odds are that the answer is "no", it's still stupid to make ANY assumption. There's a truism in the whole ass-you-me thing.

Instead of assuming, you should find out first if that's the case, and secondly why.

In my case, it's not a lack of ability but a refactoring of my priorities around my family and young children.

Maybe there are some allowances that can be made. If the person is really the best fit, then you should do whatever is reasonable to get them. You won't even get that far in the discussion if you stereotype based on age.

There are tactful and entirely legal ways to ask those questions without actually asking them:

i.e. "We have a pretty intense timeline right now. Are you able to put in X hours a day for the next X months?"

Honestly, I tend to make some bad assumptions as well. In my past experience, that type of "intensity" isn't because of trying to rush to market or because of pressure from external forces so much as poor planning internally. I have some pretty vivid memories of one company where the reason for the long hours were solely based on bad management decisions around over-catering to customers. Things like "Sure we can have that done in a week" combined with "We don't have time to do proper testing" which resulted in late nights massaging broken data back into the database or performing releases only to realize 2 hours into the process (don't ask why it took 2 hours), that the build was bad.

I've learned to watch for those signs and ask questions up front to suss that information from potential employers.

I think it's generally accepted that, regardless of age, after 16 hours of non-stop work people make stupid mistakes. I like to say "If it has to be done in 10 minutes, can we spend 20 minutes making sure we do it right?" If you can't spare an additional 10 minutes for even a tiny amount of discovery, that's an antipattern.

</comment></comment></comment></comment><comment><USER8> There is a direct relationship between uncertainty about the economy and the problem of finding/retaining talent for startups and small businesses.

The current U.S. government's "tax the rich" attitude can't be helping small business, either. It just adds to the uncertainty.

Some have been running for cover, but some of us are running against the crowd.</comment>


Issues with the API? - <AUTHOR>

Are there issues with the API at the moment?
MiniHack (iOS client) is showing deleted for every comment:<p>http://m.imgur.com/CP2lIrr <comment><USER2> Well the link text was removed and the link to the comment or story became just the timestamp recently, so if MiniHack doesn't use the official API that would explain it.

</comment><comment><AUTHOR> Looks like it was just MiniHack in the process of being updated. New version came out today and seems to be working again.</comment>


How Money Messes With Your Mind - <AUTHOR> http://tierneylab.blogs.nytimes.com/2009/07/20/how-money-messes-with-your-mind/ <comment><USER2> So, basically they found... that money makes people feel good. Duh.

Dan Ariely has done a lot of (better) research showing all the weird effects money has on people. For instance, people experience more pain relief if they are told that the pill they took cost $5.00, as opposed to $.50. Unlike this study, at least it's relevant to life. (Seriously, they're not going to give $100 bills to kids in hospitals to count.)

Incidentally, it can have also have negative effects if used as a motivator. They asked people to a simple computer task, and people did a lot better at it if they were asked to do it "because it would help the researchers" than if they received $5.00. They did the worst if they were only paid $.50.

(He has two very good, albeit not very relevant, Ted talks: <http://www.ted.com/speakers/dan_ariely.html> )

</comment><comment><USER3> Wow! Psychology at its finest. Especially the last paragraph.

Have they ever studied what happens if they just start treating people with dignity?

<comment><USER4> Indeed. Why don't we just go the whole hog and begin making up these little rituals where money truly becomes like a deity in it's own religion. Eventually, we could convert buildings to places of worship in the primary financial centers of the world, people will come to count their god and stock brokers could be like high priests.

</comment></comment><comment><USER5> "Other ideas are to have men count money before they go out hitting on women, because they are going to be rejected a lot."

That's funny. Same with the giving kids money to count idea. Wouldn't this make people feel more depressed, more in pain in the long run when they realize it's not theirs or some might be like "I can never make that much" etc.

Reading "The power of now" by Eckhart Tolle will totally make you reject these ideas and you will most likely be told to the way out of pain is not with money, rather to be present and accept the now.

</comment><comment><USER6> "Money won't solve all your problems but it allows you to arrive at them in style." - Tony Robbins</comment>


Show HN: WebGL/ThreeJS game with source - <AUTHOR> https://www.chromeexperiments.com/experiment/saving-nemo

<comment><USER2> Awesome, thanks for sharing.

Didn't work at first on my PC, but I definitely appreciate your sharing the code.

Edit: worked after a refresh.

</comment><comment><USER3> I like it, it's fun and simple. Good use of mouse controls, too.

I'm guessing this also works on tablets?

</comment><comment><USER4> I lost my cursor, had to restart FireFox.</comment>


The Problem with Stallman - <AUTHOR> https://medium.com/@omarh/the-problem-with-gnu-5941ba869be2 <comment><USER2> > I used to have this Arch Linux distro.

I think you'll strike a better balance with Ubuntu than Arch.

> I trust American capitalism to do a well-enough job.

Stallman's right about one thing: general purpose computing is under threat. What if buying a CPU that can boot without a signed bootloader from its manufacturer becomes so uncommon that it's outlawed? It doesn't sound terribly far-fetched to me. For now I have the freedom to buy x86 chips and opt-out of Trusted Platform. But I can foresee a day when that's no longer the case.

> I just don't want to spend 3 hours looking up what broke my Wi-Fi drivers > this time.

Yeah, I hear you. I've been using a linux laptop for my regular machine for 4-5 years now and sometimes there are some hiccups.

<comment><AUTHOR> I've spent some time with Stallman in real life. There is very little variance between Ubuntu and OS X to him. They both have free kernels (Linux, XNU) that come with nonfree binary blobs (device drivers) and nonfree userlands (Ubuntu apt-get main channel is not entirely free, OS X is obviously nonfree). To Stallman, once you give convenience the time of day, you might as well be Bill Gates.

</comment></comment><comment><USER3> The story seems to have been deleted.</comment>


Ask HN: What was the cool flipping credit card form posted recently? - <AUTHOR>

About a month ago, someone posted a project of a credit card form that you could flip over to enter the security code. I forgot to save the link to that project. Does anyone remember this, and can you share the link? <comment><USER2> Skeuocard

[http://kenkeiter.com/skeuocard/](http://kenkeiter.com/skeuocard/)

<comment><AUTHOR> That's it! Thanks! :)

<comment><USER3> And the HN thread: [https://news.ycombinator.com/item?id=6143604](https://news.ycombinator.com/item?id=6143604)

</comment></comment></comment><comment><USER4> (Bank) Simple released a new payment form that was kinda like that a week or two ago.</comment>


Is Greed Good? by John Stossel [6 part YouTube video] - <AUTHOR>

Part 1: http://www.youtube.com/watch?v=J5RPJkYU2Js

Part 2: http://www.youtube.com/watch?v=1FMD76SGLOg

Part 3: http://www.youtube.com/watch?v=7QNo_OyF3Uc

Part 4: http://www.youtube.com/watch?v=01XeCoTWKtg

 Part 5:  http://www.youtube.com/watch?v=srodAPzdQdk

Part 6: http://www.youtube.com/watch?v=oXQgl0eUzF4

<comment><USER2> Greed, for lack of a better word, is good. Greed is right. Greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit.

<comment><USER3> I know oliver stone was being ironic when he wrote that, but he was spot on accurate.

A great movie...

</comment></comment><comment><USER4> I think if you're blessed enough to be entrusted with great wealth, you should pass it on to others so they too can enjoy the benefit.

If I ever had money I'd want to give it away in unique ways that specifically help people - whether it's education, healthcare facilities, churches, etc... Seems like it'd be more fun than just buying $1 billion in Ramen and shipping it somewhere.

<comment><USER3> The best way to help people is not to give them money, but to use that money to build businesses, either by investing in them directly, or via public markets, etc.

You just give it away and it becomes a curse... for instance, a lot of the poverty in africa is due to aid killing local businesses-- there are no tailors anymore because the western world ships them shirts en masse, and so they just sell those...

Frankly, I'm of the opinion that the number one cause of poverty, world wide, are misdirected and ignorant attempts to "help people".

Certainly, social security, for instance, is the number one cause of poverty among the elderly in the US.

</comment></comment><comment><USER5> What's with the political threads? This and Ron Paul... what happen today? I wish I could downvote these threads....

<comment><USER2> Actually, if you watch the clips, its more about economics than politics, although one could make the stretch to politics / morals.

<comment><USER6> Everything stossel does is political.

<comment><USER7> Really? I haven't kept up with the guy. Does he make endorsements of various parties and candidates? Is he a prominent member of some political party or a spokesman for one?

<comment><USER3> No, he doesn't. He generally reports on things in a pretty factual way... the only politics is in his choice of topics to cover. He's a libertarian.

</comment></comment></comment></comment></comment><comment><USER8> haha i think i watched this in my highschool econ class.</comment>


Behind the Fiendish Complexities of Airfare Pricing - <AUTHOR> http://www.wired.com/culture/lifestyle/news/2007/07/mileage_software

<comment><USER2> Those guys have built a business around an NP-hard problem. Is anyone here working on solving hard computational problems and thinking of building a business around it?

<comment><USER3> I think the bigger story is less about computationally hard problems, but about

1) finding a problem being solved the wrong way by the world and setting out to solve it the right way

2) the pains of dealing with all of the proprietary (and often annoying) deals and details in a well-established industry.

PS: It's easy to miss, but the interesting link here is [http://www.demarcken.org/carl/papers/ITA-software-travel- com...](http://www.demarcken.org/carl/papers/ITA-software-travel- complexity/img0.html)

</comment></comment><comment><USER4> Is there a way to access the airfares data as an individual?

<comment><USER5> According to [http://www.demarcken.org/carl/papers/ITA-software-travel- com...](http://www.demarcken.org/carl/papers/ITA-software-travel- complexity/img1.html) :

"The search engines run on databases of flights, prices, and seat availability, provided electronically over private networks by the 800 or so airlines of the world. The data is not directly available to the general public and access often must be negotiated with individual airlines [...] A large portion of the flight, price and seat availability data, called published data, is used by all the major search engines, but a significant amount of private data is restricted."

</comment></comment><comment><USER6> Anyone else see the toilet advert? Very disturbing...</comment>


Kevin Rose Moves on from Silicon Valley to Watches - <AUTHOR> http://bits.blogs.nytimes.com/2015/06/11/kevin-rose-moves-on-from-silicon-valley-to-watches/ <comment><USER2> Always had mixed feelings about kevin; i liked digg, back in the day, before they mucked it up. And thebroken was pretty cool too, i definitely enjoyed that. Otherwise... not much worthwhile. He always came off more as a personality than as a actual entrepreneur.

But it's easy for me to judge up here on my mighty throne. He's doing what he wants to do, making money doing it, and overall seems to be doing well. I could learn a thing or two.

<comment><USER3> Kevin Rose is a god when you look at the products he was an early investor in. [https://angel.co/kevin](https://angel.co/kevin)

SimpleGeo, Facebook, Gowalla, Formspring, Yobongo, BufferBox, LE TOTE, Cluster, Wealthfront, Foodzie, The Social+Capital Partnership, Buttercoin, AngelList, Greylock Partners, SV Angel, FOBO, Clever, Fab.com, Tasty Labs, Cozy, Dailybooth, Foursquare, Digg, Nextdoor, Treehouse, 19 more...

<comment><USER4> Not sure I'd include Fab or Digg (among others) on that list...

<comment><USER5> If he was an angel in Fab he might have easily gotten a pretty good multiple on his investment if he took money off the table in early VC rounds.

I have no inside knowledge on this but I'd assume Digg is where he got the funds to make all these investments.

<comment><USER3> Rumor was he sold his shares in the last round. ([https://gigaom.com/2008/09/24/digg-raises-28-million-in- seri...](https://gigaom.com/2008/09/24/digg-raises-28-million-in-series-c- funding/))

</comment></comment></comment></comment></comment><comment><USER6> At this point I'm sure he's done well enough as an angel investor and with Google Ventures that he doesn't have to do the whole founder thing any longer. So I respect that he tries these little projects and then gravitates toward what users respond to.

It's a nice counterbalance to the obligatory "we must be changing the world" emptiness that is so often spouted. Some people just want to work on cool projects with people they like. That's OK.

<comment><USER7> I've always felt that Kevin Rose was the Damien Lindelof of Silicon Valley. Not that I know what it's like to be either, but I feel like everything they do ends up getting finished by someone else.

</comment></comment><comment><USER8> $3.6 million in VC funding for a news site about wrist-watches? I wonder what that pitch was, it must have been good.

<comment><USER9> """ Hi, I co-founded Digg and I like watches. Money please. """

Should be sufficient.

<comment><USER10> Digg raised 39 MILLION dollars in VC funding and exited with a 500 THOUSAND dollar sale to Betaworks. I get that SV operates in it's own odd ball reality of what defines success and that's what makes it annoying to outsiders knowing that you are probably correct.

<comment><USER11> Digg was fantastic. It just committed suicide with Digg version 4 - [http://searchengineland.com/digg-v4-how-to-successfully- kill...](http://searchengineland.com/digg-v4-how-to-successfully-kill-a- community-50450)

A bunch of people, including me, just kept commenting _" Roll back!!!"_ But for some reason that's beyond my understanding, they said they could not rollback the upgrade. And that was the end.

That said, I actually still use Digg in its current form today because I find it very useful. I just wish it had the on-site comments, but I understand the moderation nightmare task they'd be taking on if they enabled comments like the old Digg.

<comment><USER12> Digg almost killed Slashdot. Digg then killed itself with v4 and we were left with a less polished alternative Reddit. Reddit has been overrun by 4chan/mainstream. For some years, HN is the new /. with the most insightful comments. Though, I miss the comfort functions of the old /. comment system like collapsing threads (and tags like funny/insightful/etc).

<comment><USER13> For collapsing comment threads & a whole lot more, try Hacker News Enhancement Suite: [https://chrome.google.com/webstore/detail/hacker-news- enhanc...](https://chrome.google.com/webstore/detail/hacker-news- enhancement-s/bappiabcodbpphnojdiaddhnilfnjmpm)

</comment></comment></comment></comment></comment></comment><comment><USER14> Kevin Rose seems to be very good at maintaining his "brand" as well as monetizing short term niches. Most things he's involved in don't seem to last much longer. Not saying there's anything wrong with that, he's certainly been successful in that regards. It just seems he doesn't (or doesn't want to) see further than that.

I certainly admire his seeming willingness/fearlessness to jump into new things. I imagine if I had that same willingness/fearlessness, I'd be more successful.

</comment><comment><USER15> Kevin has come across publicly as a bit of a brat, but when he asked his friends to leave their jobs and join Milk, he promised them a certain return, even if it flopped and had to give them his own money.

I've always respected that.

</comment><comment><USER16> Kevin's Medium post regarding that subject: [https://medium.com/@kevinrose/hodinkee-watchville-join- force...](https://medium.com/@kevinrose/hodinkee-watchville-join- forces-131f556f2d42)

</comment><comment><USER17> It seems like the Kevin Rose of 5 or so years ago would have been excited to work on watch apps. The Kevin Rose of today is excited about fetishizing archaic jewelry.

It makes me sad because he's shown time and time again that he's full of good ideas, but he got sucked in to an "expensive niché market". Lusting over watches isn't really a problem that needs solving.

<comment><USER18> well its likely that interest in smart watches might lead people to getting into "archaic" watches as many as much art as function.

it is a problem to those involved in the collecting, selling, and trading, of watches. the issue here is that you may not see it as worthwhile market but there those who do and considering the sums involve niche markets like this could very well be profitable.

the surest way to not succeed is to close doors before you go through them. there are many areas where small markets exist that are just waiting to be served. establish a model that works at this size and expand outward. everyone's first hit cannot be facebook sized and some never will have such a hit but you can create success and personal satisfaction if you set goals and reach them, just don't set goals where your comparing yourself to others achievements.

<comment><USER17> I'm not questioning its profitability. I'm questioning its societal usefulness.

Smart watches will help with health, safety, communication, logistics, etc. Old mechanical watches were innovative feats of craftsmanship and engineering of their time. We should carry on that tradition instead of staying in the past.

<comment> k-mcgrady And Diggnation, Oink, and Tiny were useful to society? Not everything people in tech do has to help society. In fact, despite what startup founders claim, very little of what they do will have any lasting impact. He's doing something he's interested in. Why should it matter if it benefits society?

</comment></comment></comment></comment><comment><USER19> Interesting. How easy it to successfully monetize these niche markets? I mean 30 years ago this just would have been the equivalent of a watch magazine. Now it's an app/news site. But at least the magazine had people paying for copies and subscribers to keep them afloat. No one on the internet wants to pay anything.

<comment> k-mcgrady I think people are fine paying for niche things because otherwise they wouldn't exist. Plus with niches the people are generally enthusiasts and spend money in the space anyway so spending an extra few bucks for information they want and can't reliably get elsewhere isn't a big deal.

</comment></comment><comment><USER20> I kept on checking to see if I was on The Onion or ClickHole while reading that article.

</comment><comment><USER21> If Hodinkee is the rising juggernaut of modern horologists, Kevin has one shot to solidify this as newly appointed CEO. I'm excited for him.

</comment><comment><USER22> "1.3 million app sessions"

Does sessions = how many times the app has been opened?

<comment><USER23> Yes.

</comment></comment><comment><USER24> Does "Hodinkee" strike anyone else as a stupid name?

<comment> k-mcgrady Q. Where does the name Hodinkee come from?

A. I Googled "wristwatch" in about 25 different languages and picked the goofiest. Hodinkee is derived from a Czech word, "hodinky," which means wristwatch. It's a silly name, but it works because we're talking about incredibly expensive things that could be perceived as highly pretentious. It really cuts us down to size and says, "It's just watches."

Source: [http://www.nytimes.com/2014/03/27/fashion/hodinkee-watch- che...](http://www.nytimes.com/2014/03/27/fashion/hodinkee-watch- cheerleader.html)</comment></comment>


Volvo to Use Google Assistant, Play Store and Maps in Next-Gen Infotainment - <AUTHOR> https://www.media.volvocars.com/us/en-us/media/pressreleases/228639/volvo-cars-to-embed-google-assistant-google-play-store-and-google-maps-in-next-generation-infotainme <comment><USER2> Finally a car manufacturer realizes it can't out-compete a software company at making good software.

The reason infotainment sucks so hard is that car companies want to "own the experience" and also want to cut costs, so they underinvest and create poor experiences.

<comment><USER3> Nope. There's a current mismatch between automotive hardware life cycles and consumer software life cycles.

This Volvo hardware is going to be a minimum of 3 years old before it ships and will have to function for 10 years+.

How good is your 10 year old phone? How does it match up to the average consumer experience today? How's the speakers, camera, audio, OTA software, and display compare to what's available today?

Not owning the software ecosystem is a good first step but will be severely hampered by not aligning hardware development cycle expectations.

<comment><USER4> This, Android things, for example, is only offering guaranteed updates for 3 years which is not nearly enough support time... and people usually Volvos for 15-20+ years... Even Microsoft understands this better.

<comment><USER5> Is there a reason why head-unit of the infotainment system cannot be easily upgraded?

<comment><USER6> Everything in a car has to be certified for safety. Infotainment systems integrate deeply with cars - they consume and expose a lot of information that isn't guaranteed to be expressed over standard protocols, and most of them aren't even in consistent sizes or shapes.

<comment><USER7> Yet I can buy a double DIN head unit running android auto and just whack it in my 2012 corolla.

I think the major problems here aren't safety or engineering, they are purely economic.

<comment><USER8> There is a difference between can and should. Third party entertainment systems should not be a problem. In reality they are because of bad designs.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> If you want Next-Gen infotainment with Assistant and Maps, you need to buy a new shiny Volvo car... or, you know, just use your Android phone and hack a Raspberry Pi and install Crankshaft on it.

Crankshaft does all of the above, plus it doesn't talk to the cloud, doesn't require you to be connected to "the cloud," doesn't collect your data, doesn't ask you to buy a new expensive car, doesn't nudge you to subscribe to any service, and has a stallmanism approach to software freedom.

Disclaimer: I made it.

<comment><USER10> You might want to consider adding a link to it then :)

<comment><USER11> [https://getcrankshaft.com/](https://getcrankshaft.com/)

<comment><USER12> From your own site:

"Can I trust it to work?

It is alpha-level software, so no. It's not an official or even related to or certified by Google or Android. It's totally a hobby. But we hope you give it a try and have fun with it when it's safe to do so."

It's nowhere near the commercial applicability car manufacturers require. Nobody really cares about private users because thats not where the money is..

</comment></comment></comment></comment><comment><USER13> The switch to Google Maps will definitely be nice since their current maps are crap. Not sure about the other pieces but it probably can't get worse than it currently is. I have the newest XC90 and the on-screen stuff is absolutely horrible. I run into many bugs every day and the response from support has been abysmal.

<comment><USER14> My wife got an XC90 about 2 years ago and I have to agree. Even changing the channel on the radio is bad - I press next channel, next channel, and it takes me back to the first channel repeatedly.

The car looks and handles great, but we won't be getting another Volvo just because the electronics are so bad.

</comment></comment><comment><USER15> Considering the lifecycle of a car vs an Android device, they better just build in a touch screen w/ a HDMI + USB connection, and let the compute brains get swapped out inexpensively ...

<comment><USER16> already solved for, just support android auto or carplay and let the consumer decide.

<comment><USER17> Hopefully either of those will still work with latest gadgets 10 years from today.

Think of all the car owners with iPod connectivity crying in their new 5 years old cars all over the world. By the way, today's iPhones do not support iPod connectivity. And I'm talking about protocol, not about hardware port. Which is a hassle to replace in quite a few vehicles too.

<comment><USER18> My car has a 2005 Kenwood head unit with a 30 pin connector. It has worked fine with every iPhone including my iPhone 7, using a 30-pin to Lightning connector. What support is missing?

<comment><USER17> Thanks, I'll have to try an adapter. Does it support basic functionality like play/pause or browsing songs as well?

</comment></comment></comment></comment></comment><comment><USER19> It will be interesting to see how useful this will be vs. vanilla Android Auto & CarPlay.

I've had CarPlay for the last couple years, and I use it exclusively now. Even for things like radio, I've started using the Sirius app via CarPlay instead of the native radio interface. I don't use built-in navigation either, I just use my phone's on the same head unit.

What kind of benefits can the deliver through a tighter integration? And will they be able to keep it up-to-date as easily I keep my phone up-to-date? Will be interesting to watch.

<comment><USER20> The level of customization an OEM can do is pretty remarkable. This is Android Auto in an Audi Q8 and it looks nothing like Vanilla Android Auto or Carplay for that matter.

[https://www.youtube.com/watch?v=2Xf5hVqZHPY](https://www.youtube.com/watch?v=2Xf5hVqZHPY)

<comment><USER21> I know this is not what you trying to point to in your comment, but I just watched that video and I'm really surprised at how bad the system is.

Yes it looks completely different, but delay between user action and response is horrific, sometimes over a second and sometimes doesn't even do anything. It's like those cheap Chinese after market systems, that runs poor hardware and just barely works. And all of this in a new 2017 car, imagine this after 5 or 10 years, will be totally useless... Hopefully Volvo will do a better job with usability of the system.

<comment><USER20> Can you cite where in the video where these long delays were?

<comment><USER21> ~20sec music play and then followed pause. Just not sure what to say about that. Even the guy narrating gets confused and almost clicks for the second time to start the music. Then followed menu selection, not as slow, but still noticeable slowness in response. Later one when he clicks other buttons action follow much more rapidly, which indicates to me a weak processor.

</comment></comment></comment></comment></comment><comment><USER22> I feel like all I really want is Chromecast/AirPlay and otherwise the car should mostly be a dumb terminal.

Install the Volvo app on my phone, pair, done.

don't really want random apps running in my car.

<comment><USER23> Same, I'd much rather it relies on my phone's computation power and act as a display to show the map and blast the music in the speakers.

I don't care if the connection is cable, bluetooth wifi etc.

<comment><USER24> Yeah - I was thinking 'if only they had some simple way that you could upgrade the internal hardware every few years' aka buy a new phone.

Plus you can stick the phone/tomtom on the windscreen. I find that rather than being something in my view that's annoying I end up using it as a heads up display, so I don't have to look down at my speedometer any more as it's right there on the windscreen.

</comment></comment></comment><comment><USER25> Why can I trust a multi-national company to track movements and locations over time of a personal vehicle ?

<comment><USER26> I genuinely don't understand why cars suddenly have to have tablets built into them.

How is this better than just sticking your phone on the dashboard?

Is driving so tedious that drivers are crying out to be infotained?

<comment><USER27>
      I genuinely don't understand why cars suddenly
      have to have tablets built into them.

Back in the 1990s, in-dash sat nav was a premium feature only available on high-end cars. The first all-in-one TomTom wasn't introduced until 2004, and before that people in sales were willing to pay a big premium for sat nav, as a business expense.

Hence, car companies _think_ an executive/luxury car needs to have a screen in the dashboard because that's just how it's been for as long as they can remember. And they think putting premium car features in midrange cars makes them more appealing to buyers.

</comment></comment></comment><comment><USER28> What's Volvo's plan regarding OS and security updates?

If I'm going to have a car connected at all time to a LTE network, I want the software on it to be secure.

</comment><comment><USER29> Looking at how bad car companies are at security it makes sense to have someone that knows how it is done

[https://www.bleepingcomputer.com/news/security/volkswagen- an...](https://www.bleepingcomputer.com/news/security/volkswagen-and-audi- cars-vulnerable-to-remote-hacking/) Volkswagen and Audi Cars Vulnerable to Remote Hacking

</comment><comment><USER30> With the rate at which Google deprecates software this seems like a bad idea.

</comment><comment><USER20> Here's a demo of the Volvo system

[https://www.youtube.com/watch?v=2BuRMdjij4g](https://www.youtube.com/watch?v=2BuRMdjij4g)

Here's Audi's system

[https://www.youtube.com/watch?v=2Xf5hVqZHPY](https://www.youtube.com/watch?v=2Xf5hVqZHPY)

</comment><comment><USER31> Man I was really hoping they would bootstrap something awesome. Since they were acquired by Geely I was thinking that they would go there own path and pave a way for AI cars in China's future.

Maybe they are just evaluating it to get an idea of what to copy in the future?

</comment><comment><USER29> I need Google maps more then most as have poor sense of direction. But so dangerous looking at phone. Love to just have Google maps built into the car.

<comment><USER32> How is looking at them on the car any safer?

<comment><USER33> Unrestrained Objects can cause or worsen accidents (e.g. phone under the brake pedal). Plus there's better integration with steering wheel controls and a closer microphone for voice control.

Phones themselves aren't a problem if used safely, such as running Android Auto (full screen app version) and having it mounted so it can hear voice commands.

The problem is that people try to jerry-rig a poor man's version using Google Maps app, a phone sitting loose in the cup holder, and no voice commands ("let me pick it up briefly to pick a route").

It is the difference between ideal usage and real world usage. Built into the vehicle gives you closer to ideal usage every single time.

<comment><USER34> Just about everyone I know has a car mount. Used to be they required all sorts of extensive setup/install. Now however you have that mounts that you can get for 2 for $8 on Amazon.

</comment></comment></comment></comment><comment><USER35> I own a Volvo but not a mobile device? Any "advanced" tech baked into a vehicle makes zero sense to me. Tech runs in to 2-3 yr leap cycles. An auto can last you (+ a 2nd or 3rd owner) 10+ years.

Why would a manufacturer intentionally hurt the Blue Book value of their vehicles?

</comment><comment><USER36> So, can I replace these or are they embedded-embedded?

The Google Assistant seems to have custom code, so it would be understandable, if I can't replace that, but then it's hopefully at least possible to turn it off.

If not, I'm not buying a Volvo in my lifetime.

</comment><comment><USER37> I was thinking about getting a volvo, not a google car, so this is a deal breaker for me.

<comment><USER38> Can you follow up on that? What if the Volvo, other than Google's involvement, was otherwise the best vehicle for you? Would you get an otherwise worse car, just because of the G stamp on it?

<comment><USER39> Probably yes. I don't think cars need 'bundled' software any more than computers do.

Open standards such as the DIN dash mounts for in car entertainment systems do just fine. I've never seen an integrated audio system that I like, usually they're junk to begin with and they are obsolete long before the car is and hard - or sometimes even impossible - to get rid of.

</comment></comment></comment><comment><USER40> Interesting considering that Volvo are Chinese owned. Since their parent company are at the front of EV development and since the 5 year plan of China is 80% of the global EV market it looks to me like we will have lots of Waymo cars in due course. Maybe they have decided to go with mere Carplay for now with a view to future Waymo tech being what the real deal is to be.

In car tech is set on autopilot features which stymies development of anything that does not share that future. You can't invest $$$ in the next dashboard gizmos if you know that autopilot is going to 'change everything'. So it makes sense to just hnd it all over to Google now.

</comment><comment><USER41> Nice. Stand by for roadside billboards linked into your Google account which flashes custom advertising as your car nears.

Google - "You are our product"</comment>


My Trip Down the Crypto Rabbit Hole in Search of the DAO Hacker - <AUTHOR> https://www.bloomberg.com/news/articles/2020-09-16/a-trip-down-the-crypto-rabbit-hole-in-search-of-the-dao-hacker <comment><USER3> There's no such thing as the 'DAO Hacker'. The smart contract was executed exactly as written. Sad how when the going gets tough many cryptocurrency enthusiasts crawl back to statist inventions and conventions, but people should have seen the hard fork coming as Ethereum is a premined ponzi to begin with and the initial backers weren't going to let their payoff slip away so easily.

<comment><USER4> >Sad how when the going gets tough many cryptocurrency enthusiasts crawl back to statist inventions and conventions

I remember when the crypto market recently tanked and the fine folks on r/cryptocurrency were demanding the SEC shut down Bitmex and arrest its CEO.

Seems that at any given time, rugged crypto individualists are 10 percentage point losses from begging Big Daddy Government to step in.

<comment><USER5> There's a subset of rugged crypto individualists that don't want intervention at any cost - usually the ones that started using cryptocurrencies before the "investment" hype.

</comment></comment></comment><comment><USER6> I was involved in identifying and locating the attack vector of the DAO Hack, and Matthew and I connected pretty early in his research on it. I like him; he had a strong interest in figuring out what actually happened.

This should be a good read; real-life cyber-ninjas battling things out, and because of the anonymity available to those hacking smart contracts, the entire aftermath went down in an atmosphere of intense paranoia; who could be trusted with security findings? Who was everyone on this skype call? (Yes, it was 2016).

At any rate, this was a really fun and actually pretty important moment in crypto history, I'm glad to see a book about it out and published.

</comment><comment><USER7> [https://archive.is/xtB4d](https://archive.is/xtB4d)

</comment><comment><USER8> A very childishly written article with no real insight. The real question in my mind is why on earth Bloomberg funded the writer's travel to Switzerland and Japan for this?

</comment><comment><USER9> TL;DR: The 'hacker' wasn't found.

</comment><comment><USER10> after gay talese all journalists think they should describe how people dress when writing long stories - <USER2>

<comment><USER11> The first sentence appears written for the Bulwer Lytton contest.

<comment><USER10> that's funny thanks for showing it :P</comment></comment></comment>


Introducing Chrome for Android - <AUTHOR> http://chrome.blogspot.com/2012/02/introducing-chrome-for-android.html

<comment><USER2> Too bad 99% of Android devices can't run this, according to Google -- [http://developer.android.com/resources/dashboard/platform- ve...](http://developer.android.com/resources/dashboard/platform- versions.html)

</comment><comment><USER3> More discussion over here: <http://news.ycombinator.com/item?id=3562785></comment>


My students made zines, and so can you(rs) - <AUTHOR> http://composition.al/blog/2019/06/29/my-students-made-zines-and-so-can-yours/ <comment><AUTHOR> This post makes me discover "zines". This is an efficient way to share some knowledge on a tool like strace [[https://jvns.ca/debugging- zine.pdf](https://jvns.ca/debugging-zine.pdf)]. Having this applied to academic papers and ideas is really a cute idea.

<comment><USER2> These have been popular for a long time in various alt/underground/subculture communities either for disseminating news/information or publishing literature and other printable media in a low-budget way (eg. interviews with artists, a "how not to starve on tour" guide for punk bands, etc). Interesting and slightly jarring to see them used in a "high tech" context.

<comment><USER3> [https://www.youtube.com/watch?v=_N- HhIYmHeo#t=50](https://www.youtube.com/watch?v=_N-HhIYmHeo#t=50)

</comment></comment></comment><comment><USER4> Professor Kuper, thank you so much for pushing the envelope on new teaching ideas at the school. With the over enrollment in the program, personal interaction between the professors and the students is sometimes lost. I commend you for helping to engage the students on a more personal level beyond grading scripts and crowd graded assignments. I hope you continue giving creative assignments in your classes to help combat the assumption that Computer Science is a field where creativity is not valued.

</comment><comment><USER5> If you don't have a long-arm stapler for binding, you can take two normal swing-arm staplers and use the anvil from one to strike against with the other. Not as neat but can do in a pinch.

Also I mush prefer PDFs that aren't formatted for print: printing booklet order is trivial, but undoing the formatting into single-page is a nightmare

</comment><comment><USER6> Reminds me of the cartoon guides to xxx series by Larry Gonick. Strangely fun, even for non cartoony topics like calculus and genetics…</comment>


Pelosi Rejects 'blank Check To Wall Street' - <AUTHOR> http://www.cbsnews.com/stories/2008/09/21/politics/politico/thecrypt/main4463510.shtml

<comment><USER2> piss off with this stuff here</comment>


Classic bot-programming game Robo Rally has not aged well - <AUTHOR> https://arstechnica.com/gaming/2017/03/board-game-review-robo-rally-is-bot-programming-chaos/ <comment><AUTHOR> Damn! I've just bought it a few months ago at a flea market and haven't got around to actually playing it.

I still hope the reviewer is wrong.</comment>


Microsoft Build 2016 live keynote - <AUTHOR> https://channel9.msdn.com/LiveEmbedPlayer/Build2016?rnd=1459350446233 <comment><USER2> "Fastest Windows adoption ever"

Yeah, I know a bunch of people that had the 10 upgrade forced upon them. Making it a default "Windows Update" is certainly one way to get your adoption stats up!

<comment><USER3> It's more likely that they accidentally opted in, various stories about this sort of thing have been debunked to accidental opt-ins.

<comment><USER4> So it's a somewhat lighter dark pattern.

<comment><USER5> Enlighten me: Someone clicked in typical "just ok everything"-mode on "update to Windows 10" and ... surprise ... it updated to Windows 10. Dark pattern? I don't see it.

<comment><USER6> People like latest and greatest things, so they see a pop-up "Hey, Windows 10 is here! Click here to upgrade!". And the person is like "SWEET! GIVE IT TO ME!". After 1 hour of upgrading, user is back in Windows 10, "Damnit, some of my shit ain't working! Goddamn automatic upgrade!"

</comment></comment></comment></comment></comment><comment><USER7> "The bash shell is coming to Windows"

Whoa.

<comment><USER8> Some choice quotes from: [http://blog.dustinkirkland.com/2016/03/ubuntu-on- windows.htm...](http://blog.dustinkirkland.com/2016/03/ubuntu-on-windows.html)

"Nope! This isn't a container either. It's native Ubuntu binaries running directly in Windows."

"Oh, and it's totally shit hot! The sysbench utility is showing nearly equivalent cpu, memory, and io performance."

Full access to all of Ubuntu user space inc. apt, ssh, rsync, find, grep, awk, sed, sort, xargs, md5sum, gpg, curl, wget, apache, mysql, python, perl, ruby, php, gcc, tar, vim, emacs, diff, patch...

And most of the tens of thousands binary packages available in the Ubuntu archives.

<comment><USER9> Is there any way this is bad? I am honestly having a hard time not getting thrilled over it. I want to be cynical and worried, but hot damn this seems awesome. Getting that alphabet soup of utilities to be useful natively on Windows without Cygwin or `git bash` is a pain (and usually lags out of date; nevermind the startup lag too).

Just having curl, md5sum, and apt alone is a thrill and a half!

<comment><USER10> Totally agree. I use git for windows with optional unix tools. But having this built in is quite amazing.

</comment></comment></comment></comment><comment><USER11> After seeing some really impressive stuff, I cannot wait to test this stuff on my own! I mean seriously, a retail Xbox One as a devkit? That is so amazing I can't even comprehend it!

<comment><USER12> Is that finally ready to go (I haven't been able to check out the links)? That's been basically known to people who follow game dev news that it was an upcoming feature since a little before the console launched, it's just been nothing but a promise up to this point.

If it's finally released, awesome. That's a major reason why I decided to get an Xbox One over PS4. I made games with XNA on the previous system, so I'm excited to finally get to start developing on the Xbox One.

<comment><USER13> Really hoping XNA gets a reboot, even if it became a completely different project as long as it accomplishes the same goals.

<comment><USER12> Looks like Unity is the engine Microsoft picked this go around to support Xbox One development. Maybe XNA will be resurrected at some point, but I doubt it will be anytime soon.

XNA was nice and fun to program with, but I think Unity is generally a better game engine, it's still C#/.NET development, the base version of Unity is pretty open and free now, and you're not locked into "The Microsoft Platform" with it.

So overall I'm pretty satisfied. I could probably even quickly port some of the XNA code over with minimal changes (not the graphics, unfortunately. Assumed a fixed size 2D viewport for all my layout and animation code, which was a mistake).

</comment></comment></comment></comment><comment><USER14> Any retail Xbox one as a devkit, yikes that's awesome.

I remember the days of the tens-of-thousands-of-$ devkits. Probably won't have the extra memory but that only really matters if you're building something right up against the memory size barrier.

<comment><USER15> Only UWP Apps, access to just 1GB of ram.

</comment></comment><comment><USER16> Is any media site doing a "live blog" of this? Similar to what Mac Rumours do for Apple?

edit: Looks like The Verge has something here:

[https://live.theverge.com/microsoft-build-2016-live- blog/](https://live.theverge.com/microsoft-build-2016-live-blog/)

<comment><USER17> [http://live.arstechnica.com/microsoft- build-2016-day-1-keyno...](http://live.arstechnica.com/microsoft- build-2016-day-1-keynote/)

</comment></comment><comment><USER18> Siri, Cortana, Google Now and others..Software agents are finally here. When I got my first notifications from Google Now some years ago it was the first time my phone actually felt smart.

If Microsoft and others can figure out good ways to open these to the developers I bet this will be pretty big thing.

<comment><USER19> Don't forget Tay:

[http://uk.businessinsider.com/microsoft-deletes-racist- genoc...](http://uk.businessinsider.com/microsoft-deletes-racist-genocidal- tweets-from-ai-chatbot-tay-2016-3)

</comment></comment><comment><USER20> So what's the benefit of converting Win32 to UWP other than publishing to Windows store?

<comment><USER21> You can run your app on xbox one and windows phone. Access to some new APIs.

</comment></comment><comment><USER22> The guy in the Fedora is trying way too hard.

<comment><USER23> Which is something I really don't get, at all. This all looks and sounds way much _nerd_ -ier just because they try so desperately to look cool.

<comment><USER24> Pretty sure that's just that guys personality.

Why would you extrapolate his persona to all of Microsoft?

<comment><USER23> Actually, I was talking about this kind of _too-deliberate-to-be-true_ and _trying-too-hard-to-look-cool_ atmosphere at Microsoft events, not about Bryan Roper's style in particular.

</comment></comment></comment></comment><comment><USER25> Turn any Xbox One into a dev kit O.o

<comment><USER15> They have been saying that for over two years.

</comment></comment><comment><USER26> "...Age of Empires 2 running as a modern desktop app..."

I could not care less. If I'm a developer who has a game in Steam, why would I want to do this? To divide my user base?

<comment><USER16> Why would it divide your user base? Isn't it being used as a glorified package manager? Everything is still Win32 under the hood.

<comment><USER26> Well, since the game is no longer married to Steam at that point, my users would be losing Steam's achievements, mods and matchmaking. Am I not understanding that right?

<comment><USER27> They showed off installing mods from Steamworks in Age of Empires II during the demo.

<comment><USER26> Those were mods from SteamWorks? Sorry, I didn't catch that part. I still genuinely wonder why I would want to convert my app into a "modern desktop app".

<comment><USER28>
        Sandboxed security.
        Easier install/uninstall.
        Backup/restore.
        Live tiles and notifications
        Downloadable from the Windows Store (which means it will show up in Cortana search results when searching for it, even if it's not installed)
        Ability to use excellent new WinRT API's
        Deep linking
        Share to other apps easily (e.g., post to Facebook)

etc.

<comment><USER26> Thanks.

</comment></comment></comment></comment></comment></comment></comment><comment><USER29> Really interested in modern desktop apps and the desktop app converter. Are these actually available yet?

</comment><comment><USER20> Win32 to UWP converter Facebook, Instagram and messenger apps coming

</comment><comment><USER10> I think hololens is quite mind blowing though. I can totally see myself exporting class dependency graphs and exploring them in 3D.

</comment><comment><USER30> Any updates on Xamarin? Are they going to make it more affordable?

<comment><USER31> Xamarin session is on March 31 at 2:00PM PDT.

</comment></comment><comment><USER32> facebook advertisement and customer metric integration as if the native telemetry wasn't enough now you can add more to your apps.

</comment><comment><USER33> Will Cortana come with a thorough uninstaller?

</comment><comment><USER34> Aww yiss... I'm loving it!

</comment><comment><USER10> "Windows is a fully open ecosystem". Uhhhhh right :S

<comment><USER35> Except that it really is becoming that way.

<comment><USER36> Sounds amazing. Where do I find the Windows source on Github?

<comment><USER25> They never said "Open source", Just "Open"

<comment><USER10> yeah I understood that as open source. My bad.

</comment></comment></comment></comment></comment><comment><USER37> Hey MS - next time please stream on YT - got big lags

<comment><USER38> They should probably use Pied Piper, middle-out is unmatched

</comment></comment><comment><USER39> SUA lives again!

</comment><comment><USER40> What about the horrible performance issues with Universal Windows Apps?

[http://www.howtogeek.com/243012/why-you-shouldnt-buy-rise- of...](http://www.howtogeek.com/243012/why-you-shouldnt-buy-rise-of-the-tomb- raider-and-other-pc-games-from-the-windows-store/)

In a nutshell...

         No SLI or CrossFire
         VSync is Always On
         Always Borderless Fullscreen Mode
         No Modding
         No .exe File (and No Steam Controller)
         No Overlays
         Mouse Macros Won't Work
         No Clear Refund Policy
         Only for Windows 10

And that applies to anything running as a "Universal App". This seems very anti-customer, and a great reason to buy elsewhere (or, well, encourage piracy through bad policies/technical limitations). What is Microsoft doing to ameliorate these significant issues?

Edit: Really now? Some of those issues were addressed during the video(still no transcript), yet many others still stand. So one takes a karma hit for asking tough questions?

<comment><USER27> They covered a lot of those -- SLI and VSync will get fixed in a May update, they showed off mods for Age of Empires II, they talked about overlay support.

<comment><USER40> > They covered a lot of those -- SLI and VSync will get fixed in a May update, > they showed off mods for Age of Empires II, they talked about overlay > support.

Unfortunately, I've been around the block and then some. I've heard the empty platitudes of promises for X and Y features, yet later on it's considered inconvenient to implement. But as of right now, SLI and VSync are broken.

I did see the AoE2 mods. Having not played that game, I took those mods to not actually modify the base executable. In other words, it appears you can load in-game mods, but absolutely verboten to touch the 'executable'... because there is none in any usable form.

<comment><USER41> I'm not sure I'd be comfortable running mods that binary-patched the exe, anyway. Outside of a few old, old, abandonware-type games that never had bugs fixed, I'm pretty sure the overwhelming majority of game mods just swap out assets, data, and some scripted logic.

<comment><USER27> Yes, for the majority of users, removing the ability to arbitrarily change an .exe is a massive feature of UWP over conventional Windows, as it closes off a lot of attack vectors for malicious software.</comment></comment></comment></comment></comment>


Twitter DM Email Bot Script - <AUTHOR> http://bootstrapd.me/?p=550

<comment><AUTHOR> I like plain-text emails, so I wrote an email parsing script to reformat Twitter's emails into plaintext, and then allow me to reply to the email in order to reply to the DM. The source code for the script (PHP) is provided at the bottom of the post.

</comment><comment><USER2> Very cool. Just FYI: the combination of "bot script" in the title and a .me domain is likely keeping some people from looking at this submission; it makes it seem a little shady.

<comment><AUTHOR> Hmm, thanks for the insight. That's a bummer since I'm using that domain/site to chronicle my startup progress. In hindsight 'bot script' was probably not the greatest combination, either.</comment></comment>


Live Chat with anyone on Facebook (even non friends) - <AUTHOR> Messed around a bit and i found something really interesting. You can chat live with anyone at facebook even people you are not connected.
You can even see when the other party is typing and read your message.
First find his facebook id. You can do it at http://findmyfacebookid.com/ or any other service.
http://www.facebook.com/othelis ---> id= 699313289
Open you browser's console at facebook and type
Chat.openTab(id) eg. Chat.openTab(699313289)
You can live chat  with strangers and non friends now and get notified when they read your message! <comment><USER2> I have two Facebook accounts that are complete strangers to each other.

When I sent a message from one to the other one, message didn't pop up. It was in "Others" section of my account inbox.

So it's basically a normal message that you can send to anyone now. Even Zuck

</comment><comment><USER3> Facebook IM Roulette

Chat.openTab(Math.floor(Math.random() * 100000000))

<comment><USER4> Chat.openTab(~~(Math.random() * 1e8))

</comment></comment><comment><USER5> Interesting...

Chat with Mark Zuckerberg:

      javascript:Chat.openTab(4)

<comment><USER6> haha, pls do it and report back

</comment></comment><comment> why-el This is not surprising because Facebook treats chat and messages as the same. They both end up in the same thread. So as you can send a message to anybody, you can "chat" with anybody. Whatever you wrote will be received in their inbox as a regular message.

I am not sure what happens with people who choose not to receive messages from the public though.

</comment><comment><USER7> Ouch.

I can't help but think that somewhere at Facebook there's a huge fire-drill going on right now.

<comment><USER8> I am wondering why the OP decided to post it here, rather than try and get a bug bounty (edit: assuming it's a genuine issue, seems like maybe it isn't). It would probably be worth a fair amount...certainly a few thousand at least. Facebook are normally pretty happy for you to disclose after they fix it, so you can still get the PR...

<comment><USER7> After I commented, I flagged the post. I'm not comfortable with this being on HN. FB should have a chance to fix it.

On the plus side, the entire thing may just be a misunderstanding. Other commenters are pointing out that it doesn't really live message them if you're not friends, just regular messages them -- which you could do anyway. It's just the UI appears that you are live messaging.

We'll see. But you're right -- very odd to post that here with a reward being easily available.

<comment><AUTHOR> Confirming it's the default behavior. After some googling this call is known on the internet. It's just a shortcut to the original(quite hidden) behavior. Check comments below with "FacebookChatRoulette". It's interesting how easy and automated can be to send a msg to anyone. So that raises some concerns. Definetly not a bug ..

</comment></comment></comment></comment><comment><USER9> This is no different than navigating to that person's page and clicking on the 'Message' button. It's not a vulnerability.

<comment><AUTHOR> You can see when is typing or if he/she read your message

<comment><USER10> After you send a message, you can select open in chat from the actions drop down.

</comment></comment></comment><comment><USER11> You can also get someone's Facebook ID via this URL: <http://graph.facebook.com/user_name_here>

</comment><comment><USER12> This isn't news. Chat and messaging are one and the same. You can message anyone, and you can migrate conversations arbitrarily between the chat and message UI, or even both at the same time. If you message someone who you don't have good reason to message, facebook will classify it as spam and send it to their "other" box.

</comment><comment><USER13> Facebook id number 1, 2 and 3 do not exists.

<comment><USER2> Because Mark deleted them after he made sure his POSTs are working

</comment></comment><comment><USER14> [http://www.businessinsider.com/the-first-20-people-to- sign-u...](http://www.businessinsider.com/the-first-20-people-to-sign-up-for- facebook-2011-5?op=1)

Here are some fun people you can chat with. I felt kinda bad bugging Zuck with a message but how could I resist?

</comment><comment><USER15> <http://findmyfacebookid.com/> is a way to find real human users of facebook and currently using facebook.

<comment><AUTHOR> <http://findmyfacebookid.com> is just a mean to get fb id. <http://graph.facebook.com/user_name_here> is the official way to to that

</comment></comment><comment><USER16> Noooo Chat and Messaging are the same thing, and you've always been able to message strangers. Just go to their profile and hit "message".

<comment><AUTHOR> can you see when they receive the message and when there are typing? Also it appears on the main chat window?

</comment></comment><comment><USER17> I think it has already been fixed. It's not working for me anymore. Too bad, I was having fun.

<comment><USER18> Go to the users page and click 'message'. It's the exact same thing.

</comment></comment><comment><USER19> Holy crap, it really works

<comment><AUTHOR> yeap!

<comment><USER19> But don't work like it should. The chat only appears to the addressee if he opens a chat box to chat with you ..

<comment><AUTHOR> In my case it opens instantly..

</comment></comment></comment></comment><comment><USER20> ZukerBerg would be getting more money for getting hits for this bullshit too....why would he want to debug it???

</comment><comment><USER21> Oops it worked for me...!</comment>


Safely load untrusted YAML in Perl - <AUTHOR> http://blogs.perl.org/users/tinita/2018/02/safely-load-untrusted-yaml-in-perl.html <comment><USER2> Oh my, it misses the point completely.

The problem is the automatic call of methods of arbitrary classes, controlled by the external YAML, such as DESTROY. unblessing after loading does not help at all. You need to disallow blessing during load, or provide a whitelist, such as python does.

cperl's YAML library does contain the security fixes for some years already: [https://github.com/perl11/cperl/blob/master/cpan/YAML- LibYAM...](https://github.com/perl11/cperl/blob/master/cpan/YAML- LibYAML/lib/YAML/XS.pod), the cpan version maintained by this person is still vulnerable, besides being incompatible to itself, and unusable for cpan.

[https://github.com/rurban/yaml-libyaml- pm/tree/safe](https://github.com/rurban/yaml-libyaml-pm/tree/safe) contains the whitelist variant, as in python.

<comment><USER3> "the cpan version maintained by this person is still vulnerable"

In our github discussion you admitted that it is not vulnerable if you disable loading objects, which is possible since version 0.69. (Because of backwards compatibility this is not (yet) the default setting.)

Besides that, feel free to incorporate our other fixes, like loading many regexes, which I fixed in 0.68. I see your version still has this bug.

Seems like you think we open source developers are supposed to work against each other.</comment></comment>


The Hum - <AUTHOR> http://en.wikipedia.org/wiki/The_Hum

<comment><USER2> I heard this for months and only at home in North East England. I ruled out the nearby wind turbines as I heard it when they weren't on.

Never did get to the bottom of it as it disappeared as suddenly as it came.</comment>


How quickly can you remove spaces from a string? - <AUTHOR> http://lemire.me/blog/2017/01/20/how-quickly-can-you-remove-spaces-from-a-string/ <comment><USER3> Fun story: When I implemented the syntax highlighting feature in Sequel Pro, I needed a way to determine the string length of UTF-8 encoded strings. I didn't find a function available on macOS that worked directly on a char*, so I googled and found a really simple UTF-8 strlen function. It was easy to understand and very fast. I think it was this: [http://canonical.org/~kragen/strlen- utf8.html](http://canonical.org/~kragen/strlen-utf8.html)

I committed my syntax highlighting code, and a few days later someone had replaced the simple UTF8 strlen function with the really long vectorised version from this page: [http://www.daemonology.net/blog/2008-06-05-faster- utf8-strle...](http://www.daemonology.net/blog/2008-06-05-faster- utf8-strlen.html)

But the funny thing is, that the supposedly fast vectorised strlen was optimised for very long strings. The benchmarks shows results for megabytes of texts. But we measured the length of tokens, usually only a few characters long, so in most cases the new function was actually slower!

I was a very junior dev, didn't want to piss off anybody, and the strlen wasn't in the hot path anyway, so I didn't say anything. But I was a bit sad, that my easy-to-read code was replaced by such a monstrosity.

What's my point? Before you go and use these functions in your code, profile your code to see if it would actually affect performance.

</comment><comment><USER4> Somehow the article fails to mention that the speed will depend heavily on the input string: both its size and distribution of whitespace characters.

Even assuming the question is for the limit of very long strings, the distribution makes a huge difference. Natural English has spaces on average every 5.1 characters [1], so using multi-character tests to speed up the case of runs of 8 or more characters without a space will probably slow it down, not speed it up!

[1][https://arxiv.org/pdf/1208.6109](https://arxiv.org/pdf/1208.6109)

<comment><USER5> _using multi-character tests to speed up the case of runs of 8 or more characters without a space will probably slow it down, not speed it up!_

Yes, although there is the option of combining the vectorized comparisons with a branchless approach. I modified his code to do this, and got what seemed to be a flat .40 cycles per character independent of input, which is about twice the speed Daniel illustrated on the input with 3% spaces. I'm sure it can be made even faster (what would the limiting factor be?), but I think this shows that a multi-character approach is faster on all input than would be possible with any single-character approach?

<comment><USER6> Could you share your code?

<comment><USER5> I'm working with the author, and it should be in his GitHub repository soon. But if you want to reproduce before then, you should be able to just change the line "if (mask16 == 0)" to "if (0)" in sse4_despace(), which is in include/despace.h.

The idea is that rather than testing whether any spaces were found and taking a "shortcut", we always do a lookup on mask16, shuffle the bytes according to the result to remove the spaces, and advance the output pointer by 16 minus the number of spaces found. This costs 2-3 cycles for completely spaceless input, but saves the ~15 cycle branch misprediction penalty each time an unexpected space is found.

</comment></comment></comment></comment><comment><USER7> What strikes me is how many sites won't accept a login name, credit card number, phone number, or other field with leading or trailing spaces. This for something where the speed of the code doesn't matter at all. I can't think of any explanation other than incredible laziness or incompetence for not stripping off spaces.

You might not notice this if you use a password manager or browser autofill, but it's a lot of sites, including companies like major airlines for example.

Never mind that -- it's just a miracle when you can enter a credit card number formatted as 4123 4567 8901 2345 rather than squished together.

<comment><USER8> You hit the issue in a roundabout way:

The very premise of the article 'how quickly can we remove whitespaces' is rooted in the intellectual foundations of CS. As a culture, we are are obsessive about 'performance'.

This is because back in the day, it's always been important - and even today 'under the hood' it's always important. And of course there are situations in which it's still important (complex algs, limitations of mobile devices).

But in reality - these things are never the issue.

The 'issue' is the pragmatic application of basic algorithms to do a number of basic things elegantly, which together form the foundation of a good user experience.

Yes - the issue of 'no spaces' in card numbers etc. is a clumsy thing, and it's laziness by developers.

Also - things like 'performance' are objectively measurable, you can get cool data for it etc..

A 'bad experience' is sometimes difficult to define.

<comment> open-source-ux _As a culture, we are are obsessive about 'performance'._

My purely anecdotal impression is quite the opposite. Speed of delivery and convenience for the _developer_ (not the end user) seem to be the norm.

Frameworks, scripting languages, browser-based desktop and web apps: none of these have the characteristics of being small, nimble, lightweight, or performant. They certainly make life easier for the developer. Whether users get a 'good experience' out of the end result is open to debate.

</comment></comment></comment><comment><USER9> _This code will work on all UTF-8 encoded strings… which is the bulk of the strings found on the Internet if you consider that UTF-8 is a superset of ASCII._

Note that the first set of code (and possibly the rest), only work as the space, newline, and carriage return are the 7 bit ASCII set is included in UTF-8. However, the extended 8-bit ASCII set is not, but is often included when people speak of ASCII. So for example, if the request was to remove all "Copyright Sign" symbols, which is U00A9, it would not work correctly. The UTF-8 encoding for this symbol is 0xC2 0xA9, but the code only works on individual bytes, so it would remove the A9 byte, leaving a C2 byte and then whatever byte came next. Additionally, it would hit other UTF-8 characters like the "Greek Capital Letter Omega" (Ω which is encoded in UTF-8 as 0xCE 0xA9)

tl;dr Only works for the 7-bit ASCII set, but not the common extended 8-bit ASCII sets

<comment><USER10> It's sort of a semantic argument, but I've never once heard anyone refer to "ASCII" to refer to anything other than the 7-bit standard. There is no "extended 8-bit ASCII set". That's a blanket term (though in casual use, "code page" is more typical) for any of the literally dozens of 8-bit character encodings that overlap with ASCII in the bottom half of the encoding space.

Basically the blog post was correct and precise: it removes ASCII characters from UTF-8. There is no elaboration needed.

Again, though, the post was about using SIMD primitives to optimize what looks like a scalar problem.

<comment><USER11> You would be surprised. Referring to any of the most common 8-bit encodings as "extended ASCII" or even just "ASCII" has been very common in my experience.

<comment><USER12> Yeah. Most people (well, back in the DOS days anyway) who don't know what a "7-bit encoding" is would refer to code pages as "ASCII". I'm sure I did that hundreds of times myself (I didn't know better until I was what, 20?) and I've seen it thousands of times, not exaggerating. Just look at how many "[extended] ASCII games" there are.

<comment><USER13> I know a lot of DOS people used the term "extended ASCII" to mean specific 8-bit encodings whose first 7 bits were ASCII, but what "extended ASCII" meant depended quite a lot on the context. I came from a different background and it was very clear to me that ASCII was a 7 bit encoding that was almost always stored in an 8 bit byte (I can only think of a few instances in my whole life where I saw bitstreams of ASCII). So that last bit you could do whatever you wanted with.

Having said that, I clearly remember arguing with people that encodings with the eighth bit set were _not_ part of ASCII. As you say, there were many people who didn't understand. My guess is that the GP never interacted with those people. Especially if you were around pre-DOS I think it would be easy to do.

</comment></comment></comment></comment></comment><comment><USER14> 6.5MB for the tables to drive this faster de-spacer? No thank you. It would blow out the L1 cache and make your program slower.

[https://raw.githubusercontent.com/lemire/despacer/master/inc...](https://raw.githubusercontent.com/lemire/despacer/master/include/despacer_tables.h)

<comment><USER15> My pet peeve: using table lookups so a benchmark show faster when in reality your L1 cache is going to be stomped on. Not only will you be waiting on the L1 cache to populate but you also evict all the useful data.

<comment><USER16> I've seen this elsewhere too. For relatively mundane task (I think it was Morton code conversion), a giant lookup table was constructed and it gave a nice 2x or 4x performance improvement.

But no application will ever be doing only string despacing or Morton codes, so the "fast" lookup table algorithm will make everything else slower by evicting good cache lines. And once something else runs and evicts the lookup tables, the next run will be slow again.

</comment></comment></comment><comment><USER17> Cached text-only version: [http://webcache.googleusercontent.com/search?q=cache%3Ahttp%...](http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Flemire.me%2Fblog%2F2017%2F01%2F20%2Fhow- quickly-can-you-remove-spaces-from-a-string%2F&strip=1)

</comment><comment><USER18> This reminded me of spray-json's JsonParser. It has a bit of Scala code to seek past whitespace:

      @tailrec private def ws(): Unit =
        // fast test whether cursorChar is one of " \n\r\t"
        if (((1L << cursorChar) & ((cursorChar - 64) >> 31) & 0x100002600L) != 0L) { advance(); ws() }

[https://github.com/spray/spray- json/blob/765c83248e0bbe867dd...](https://github.com/spray/spray- json/blob/765c83248e0bbe867ddc9d479b4fe79493569a54/src/main/scala/spray/json/JsonParser.scala#L186)

<comment><USER19> The compiler needs to be extremely clever on @tailrec for not blowing up the L1 cache on this code.

To me, this looks like a perfect example of a developer thinking they're very smart while they're actually writing code that's worse that its naive counterpart version (a good summary of Scala in my experience).

<comment><USER15> Can you please explain why it would blow up the L1 cache?

</comment></comment></comment><comment><USER20> So using 128bits instructions would imply you had 'words' which were over 16 characters long on average, right?

The average (english) word is ~5 characters long, so most of the time, you'd be forced to check anyway.

<comment><USER21> Not all strings are made of English text

<comment><USER22> you don't even have to go to non english, just utf-8 has stuff like mathematic spaces and spaces of different em sizes. [http://perldoc.perl.org/perlrecharclass.html](http://perldoc.perl.org/perlrecharclass.html) has a reasonably good list of non ascii spaces, though I'm not sure where to dig through for locale specific lists.

That said, while it may not be a solution for every case, it's a solution for the common case and a starting point for other cases, and thus pretty nifty and potentially useful.

</comment></comment></comment><comment><USER23> I feel like you have to at _least_ include non-breaking spaces if you're going to say you're removing spaces from UTF-8 strings.

</comment><comment><USER24> Is there a clever way to remove all the space characters?

[https://www.cs.tut.fi/~jkorpela/chars/spaces.html](https://www.cs.tut.fi/~jkorpela/chars/spaces.html)

<comment><USER25> If you use the naive algorithm at the top of the page but iterate over UTF-8 code points rather than bytes -- which is straightforward, BTW -- you _will_ get some cleverness automatically in the compiler's implementation of the switch statement. I wrote a switch to handle the Unicode "is it space?" function, with the values from the table you linked, and compiled with "clang -Os". You know what it did?

It generated a friggin' binary search tree! Some of the leaves were a sequence of straight-line comparisons, because that's more compact, but the higher levels were all a bunch of "if (c > 0x167F) { ... }" sort of code. At one point it subtracts 8192 from something and then compares with 12, and I _think_ this is because the x86 instruction encoding is shorter and the compiler knows that the register won't be needed again along either of the code paths from that point.

Compilers are amazing sometimes.

<comment><USER26> Compiling switches into binary (and even sometimes trinary) trees is an old technique that's been around since at least the days of 16-bit DOS... they will usually choose between a single-level lookup, double-level lookup, or tree depending on sparsity and options used.

Subtraction and addition implicitly sets the flags, so you can generate very small code with inc/dec (1-byte instructions), like this:

        dec ax
        jz ax_was_zero
        dec ax
        jz ax_was_one
        ...

</comment></comment></comment><comment><USER27> Maybe the asm to do scan for byte in memory is an alternative. Repne scab.

It won't find all characters in a single scan. But maybe do 3 passes over a buffer which fits in cache.

</comment><comment><USER28> I wonder how much of a difference using a separate destination buffer would make. Apart from some memory/cache/invalidation/magic thing that I'm not certain might occur, it would allow one to use the extended instructions used for scanning.

</comment><comment><USER29>
      awk '
      {
        gsub(/[[:blank:]\015]/, "");
        printf("%s", $0);
      }' input | tee output

stripping out LF isn't necessary because AWK does it on every record automatically. For even more speed, the code can be translated into ANSI C and compiled with awka[1] using an optimizing C compiler.

[1] [http://awka.sourceforge.net/index.html](http://awka.sourceforge.net/index.html)

</comment><comment><USER16> I am surprised to see any speedup in this! I'd expect something trivial like this to be completely memory bound with the CPU sitting almost idle waiting for bytes coming in from memory.

Looking at the benchmark code, this is using rdtsc to read the CPU time stamp counter. That does not take waiting for memory into account, does it?

I wonder if there's a difference when measured in wall clock time. It's still somewhat beneficial to have the CPU work efficiently to give an opportunity for hyperthreading to take place when waiting for memory.

If you really wanted to make something like this faster, you should focus on cache utilization and make use of prefetching instructions. x86 has pretty bad prefetching instructions and pretty good speculative fetching, so don't expect massive speedups but on ARM or Aarch64, you have a finer grained control over cache prefetching (L1 and L2 separately) and you could see _much_ bigger differences.

As for benchmarking this kind of problems: you obviously want to measure real world performance, so you need wall clock time as well as time stamp counter, but I'd look for optimization clues in "perf stat" and other CPU perf counters, with an emphasis on cache misses and branch mispredictions.

The figure you should be staring at is the total _throughput_ of the algorithm, measured in gigabytes per second. You should be getting figures close to the memory bandwidth available (25-50 GB/s depending on CPU and memory).

edit: I measured the wall clock time with clock_gettime before/after all the repeats (using a megabyte sized buffer) and there is indeed no significant difference, here's my results:

        memcpy(tmpbuffer,buffer,N):  0.122945 cycles / ops 1495907352 nsec (1.495907 sec)
        countspaces(buffer, N):  3.657322 cycles / ops 1544915395 nsec (1.544915 sec)
        despace(buffer, N):  6.521193 cycles / ops 1621204460 nsec (1.621204 sec)
        faster_despace(buffer, N):  1.721657 cycles / ops 1500507217 nsec (1.500507 sec)
        despace64(buffer, N):  3.595031 cycles / ops 1544993649 nsec (1.544994 sec)
        despace_to(buffer, N, tmpbuffer):  6.307885 cycles / ops 1615101563 nsec (1.615102 sec)
        avx2_countspaces(buffer, N):  0.190992 cycles / ops 1460961459 nsec (1.460961 sec)
        avx2_despace(buffer, N):  5.750583 cycles / ops 1615971010 nsec (1.615971 sec)
        sse4_despace(buffer, N):  0.985002 cycles / ops 1482901389 nsec (1.482901 sec)
        sse4_despace_branchless(buffer, N):  0.338737 cycles / ops 1460874704 nsec (1.460875 sec)
        sse4_despace_trail(buffer, N):  1.950657 cycles / ops 1502268447 nsec (1.502268 sec)
        sse42_despace_branchless(buffer, N):  0.562246 cycles / ops 1468638389 nsec (1.468638 sec)
        sse42_despace_branchless_lookup(buffer, N):  0.624913 cycles / ops 1472445127 nsec (1.472445 sec)
        sse42_despace_to(buffer, N,tmpbuffer):  1.747046 cycles / ops 1507705780 nsec (1.507706 sec)

Here's the diff to the original: [http://pasteall.org/208511](http://pasteall.org/208511)

edit2: surprisingly, Clang is about 10% slower than GCC in my experiments.

<comment><USER5> Thanks for looking into this! I know that Daniel appreciates feedback, but rarely reads HN, so an email to him or blog comment with your results might be helpful.

 _Looking at the benchmark code, this is using rdtsc to read the CPU time stamp counter. That does not take waiting for memory into account, does it?_

On modern Intel processors, the "time stamp counter" is monotonically increasing at a constant rate, so it does take memory latency into account. On many Linux systems (including the one used here) clock_gettime() uses the same underlying clock source, so there should be no difference in accuracy for long measurements. The CPUID-RDTSC/RDSTCP-CPUID pattern used here has the advantage of a somewhat lower and significantly more predictable overhead, which helps when measuring shorter events.

 _I 'd expect something trivial like this to be completely memory bound with the CPU sitting almost idle waiting for bytes coming in from memory._

If I remember the numbers right, the Skylake processor this is running on can read about 64B per cycle if the source is L1, about 24B per cycle if the source is L3, and about 6B per cycle from main memory. Using the existing RDTSC framework, I get equal speeds at L3 size, and still get sub .4B/cycle coming from main memory.

 _I measured the wall clock time with clock_gettime before /after all the repeats (using a megabyte sized buffer) and there is indeed no significant difference_

I agree that testing on larger buffers would be informative (as would testing different ratios of whitespace) but I don't think your approach is capturing what you think it is. The macro being used for time measurement uses a different random input for each iteration (look at 'pre'), and the unoptimized time of initializing this dominates the clock time. So while I think your test is worthwhile, I think you need a better way to perform the measurement. I think you'll see that RDTSC maps exactly to wall time in this case, but surprises are definitely possible.

 _I 'd look for optimization clues in "perf stat" and other CPU perf counters, with an emphasis on cache misses and branch mispredictions._

Yes, although as with wall time, one needs to be sure to measure only on the section of code that one is optimizing. Perf makes this difficult, so something like "likwid" with an API that allows profiling fragments of code would be required. I haven't done that yet for this, but at the faster speeds (sse4_despace_branchless) I don't think there are any cache misses or branch mispredictions in the code of interest.

 _surprisingly, Clang is about 10% slower than GCC in my experiments_

My surprise was the opposite. Clang was 25% faster than GCC and ICC on the L1- and L3-sized vectorized branchless (.25 cycles/byte versus .33 cycles/byte, although the code I'm running is not quite what's on Github). Most of this benefit seems to be because clang unrolls 2x to reduce loop overhead.

<comment><USER16> I dropped a comment on his blog and a link to this discussion...

And how sloppy of me! I didn't notice the `pre;` in the code consuming most of the time (to be fair it's just 4 characters and I didn't put too much time to it). When I move that outside of the loop, before the timer, I get results that show improvement with the optimized version.

And indeed it looks like rdtsc is giving similar figures to clock_gettime now. I falsely presumed that it counts retired instructions.

I'm still surprised to see a speedup, and how badly the original version is performing.

My guess is that the conditional stores are poison to the CPU pipelines. The 64 bit version gets most of the performance out of it already, I presume it's because of the more efficient memory usage pattern.

I can't edit my earlier post any more, I'd correct it if I could.

Clang:

        memcpy(tmpbuffer,buffer,N):  0.000000 cycles / ops 65232 nsec (0.000065 sec)
        countspaces(buffer, N):  0.000000 cycles / ops 65176 nsec (0.000065 sec)
        despace(buffer, N):  4.547286 cycles / ops 7654310198 nsec (7.654310 sec)
        faster_despace(buffer, N):  1.582721 cycles / ops 2651677500 nsec (2.651678 sec)
        despace64(buffer, N):  0.583952 cycles / ops 1025215835 nsec (1.025216 sec)
        despace_to(buffer, N, tmpbuffer):  0.000000 cycles / ops 63847 nsec (0.000064 sec)
        avx2_countspaces(buffer, N):  0.000000 cycles / ops 63697 nsec (0.000064 sec)
        avx2_despace(buffer, N):  0.307253 cycles / ops 602528061 nsec (0.602528 sec)
        sse4_despace(buffer, N):  0.310504 cycles / ops 534542967 nsec (0.534543 sec)
        sse4_despace_branchless(buffer, N):  0.353851 cycles / ops 594652080 nsec (0.594652 sec)
        sse4_despace_trail(buffer, N):  0.314221 cycles / ops 562439990 nsec (0.562440 sec)
        sse42_despace_branchless(buffer, N):  0.608811 cycles / ops 1020576734 nsec (1.020577 sec)
        sse42_despace_branchless_lookup(buffer, N):  0.608494 cycles / ops 1020062750 nsec (1.020063 sec)
        sse42_despace_to(buffer, N,tmpbuffer):  1.779908 cycles / ops 2983955982 nsec (2.983956 sec)

GCC:

        memcpy(tmpbuffer,buffer,N):  0.285702 cycles / ops 489599835 nsec (0.489600 sec)
        countspaces(buffer, N):  0.000000 cycles / ops 63995 nsec (0.000064 sec)
        despace(buffer, N):  4.751018 cycles / ops 8014809122 nsec (8.014809 sec)
        faster_despace(buffer, N):  1.718575 cycles / ops 2898082416 nsec (2.898082 sec)
        despace64(buffer, N):  0.883421 cycles / ops 1560479651 nsec (1.560480 sec)
        despace_to(buffer, N, tmpbuffer):  6.424313 cycles / ops 10892716258 nsec (10.892716 sec)
        avx2_countspaces(buffer, N):  0.031227 cycles / ops 52369789 nsec (0.052370 sec)
        avx2_despace(buffer, N):  0.315633 cycles / ops 627793484 nsec (0.627793 sec)
        sse4_despace(buffer, N):  0.319739 cycles / ops 554173689 nsec (0.554174 sec)
        sse4_despace_branchless(buffer, N):  0.366240 cycles / ops 615638316 nsec (0.615638 sec)
        sse4_despace_trail(buffer, N):  0.318973 cycles / ops 572262343 nsec (0.572262 sec)
        sse42_despace_branchless(buffer, N):  0.638114 cycles / ops 1070772787 nsec (1.070773 sec)
        sse42_despace_branchless_lookup(buffer, N):  0.637988 cycles / ops 1069642163 nsec (1.069642 sec)
        sse42_despace_to(buffer, N,tmpbuffer):  1.768081 cycles / ops 2963488990 nsec (2.963489 sec)

</comment></comment></comment><comment><USER30> A variant of this problem yields further interesting possibilities: if you're trying to remove control characters like CR and LF, but replacing them with whitespace would be acceptable. That way you can work on it in-place, without needing to copy memory or allocate or anything like that.

</comment><comment><USER31> The "optimized" functions have a bug when the number of remaining bytes is less than the block size.

<comment><USER5> Certainly possible, but are you looking at the excerpts in his blog post, or the actual code that he links to? [https://github.com/lemire/despacer/blob/master/include/despa...](https://github.com/lemire/despacer/blob/master/include/despacer.h)

<comment><USER31> I'm looking at the excepts in his post, which are broken.

</comment></comment></comment><comment><USER32> Someone file a compiler bug? 14x difference between the readable code and the optimized code is a lot. The first code is extremely straightforward, you shouldn't have deal with that SIMD mess manually.

<comment><USER11> Failure to optimize is never a bug, by definition. Automatic vectorization of sequential code is a very difficult problem in general. Especially when your code still depends on testing single bytes in the input and vectorizing _that_ is only possible using some very clever bit twiddling.

</comment></comment><comment><USER33> Adding lookup tables to the naïve implementation gave me a ~3x speedup with virtually no extra effort. Branch penalties are a real killer.

<comment><USER34> You have to be careful with lookup tables. They can be impressive each at a time when microbenchmarking a single loop, but in the aggregate hurt performance by crowding other useful data out of limited cache space.

</comment></comment><comment><USER35> I guess you could write CUDA code to do this on a GPU, but then the question becomes why? :/

<comment><USER36> Ok, here it is : [https://gist.github.com/antonmks/17e0c711d41fb07d1b6f3ada3f5...](https://gist.github.com/antonmks/17e0c711d41fb07d1b6f3ada3f5f29ee)

0.25 cycles/byte using GTX 1080.

</comment></comment><comment><USER37> Does anyone know how to do such a benchmark?

<comment><USER38> Check out the linked Github repo.

</comment></comment><comment><USER39> Optimizing by hand is rarely the fastest thing to do nowadays. I wonder how the original naive approach fairs with all optimizations turned on for gcc and with realistic input?

<comment><USER32> The optimization flags used should be good enough.

[https://github.com/lemire/despacer/blob/master/Makefile](https://github.com/lemire/despacer/blob/master/Makefile)

CFLAGS = -fPIC -std=c99 -O3 -march=native

</comment></comment><comment><USER40> ASCII/1 byte characters? I stopped reading then.

UNICODE or GTFO.

Should be embarassingly parallel for a contiguous array.

<comment><USER41> this is in no way embarrassingly parallel at all! in fact, it's entirely non trivial to parallelise, writing the output is an inhertintly a serial operation.

<comment><USER42> Could you not do something like chunk the input and operate on the pieces in parallel? I'd think you could use something like a rope (à la text editors) to support the concurrent writes.

(I may or may not have forgotten the differences between concurrency and parallelism.)

<comment><USER41> yeah, you could split the input into chunks, but then you're left with the problem of recombining the outputs from each portion into one string - <USER2> operation must be serial and would likely outweigh the benefits of the initial parallelism in most cases.

</comment></comment></comment></comment><comment><USER43> Going to start using this as an interview question!

<comment><USER44> Please don't. Especially for a typical software engineering position.

</comment></comment><comment><USER45> Isn't this the very kind of thing Regular Expressions were created to solve?

* Remove spaces - myString.replace(/\u0020+/g, "");

* Remove common line terminators - myString.replace(/(\r|\n)+/g, "");

* Remove all white space - myString.replace(/\s+/g, "");

<comment><USER46> the post is not so much about most convenient way to remove spaces, but about efficiency of such an operation.

<comment><USER45> Is regex execution considered efficient? I have always been impressed with its speed.

<comment><USER11> Even if the regex library compiled the expression down to the very simplest state machine possible, it would still be _at most_ as efficient as the first handwritten code in the article, handling one byte at a time. I'm fairly sure no regex engine in existence is able to detect "simple enough" regexes and optimize them into code that handles 64 or 128 bit blocks at a time.

<comment><USER47> They exist. In fact, if you hand Rust's regex engine the pattern ` |\r|\n`, then it will compile it down to a very simple alternation of three literal bytes. It will never touch the actual regex engine at all. Once the regex engine knows its a simple alternation of three bytes, it will actually use a variant of memchr called memchr3[1], which looks almost exactly like the OP's "portable" approach.

So it's not quite the fastest possible because it doesn't use SIMD, but this is mostly because we don't have good explicit SIMD support on stable Rust yet. Once we do, this immediately becomes a strong candidate to optimize to something like the OP's fastest version (and also probably generalizing to AVX2 instructions as well).

N.B. My sibling comment linked to my blog post on ripgrep, which uses Rust's regex engine. ;-)

[1] - [https://github.com/BurntSushi/rust- memchr/blob/master/src/li...](https://github.com/BurntSushi/rust- memchr/blob/master/src/lib.rs#L347)

<comment><USER11> Awesome, thank you!

</comment></comment></comment></comment></comment></comment><comment><USER48> Not claiming to be the fastest, but here are two Python solutions suggested by Dave Beazley [0], plus one I extended using compiled regular expressions:

        # string replacement
        s = ' hello  world \n'
        s.replace(' ', '')
        # 'helloworld'

        # regular expression
        import re
        re.sub('\s+', '', s)
        # 'helloworld'

        # compiled regular expression
        pat = re.compile('\s+')
        pat.sub('', s)
        # 'helloworld'

[0] [https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwiDwpPQytHRAhVJWCwKHSTmD5wQFggaMAA&url=https%3A%2F%2Fwww.safaribooksonline.com%2Flibrary%2Fview%2Fpython- cookbook-3rd%2F9781449357337%2Fch02s11.html&usg=AFQjCNEZqL6SV76UmE4ojyJ5poOgeCexEQ&sig2=I9kGVDVg5DyLxZ3pCJ1FIw&bvm=bv.144224172,d.bGg)

<comment><USER49> my first python thought was

      s = ' hello  world \n'
      s = ''.join(s.split())

again probably not fast, but simple

<comment><USER50> Scala: val nospaces = "a string with spaces" filter { _ != ' ' }</comment></comment></comment>