Apple's iPhone 5s And iPhone 5c Sell 9M Units Over Opening Weekend - <AUTHOR> http://techcrunch.com/2013/09/23/apples-iphone-5s-and-iphone-5c-sell-9m-units-over-opening-weekend-topping-5m-for-iphone-5-last-year/ <comment><USER3> I was in NY for vacation during the opening Friday and decided to head into a queue and line up for the 5s, suspecting that it may sell out quickly while also leveraging the pricing difference between Europe and USA. This was the first time that I lined up for any Apple launch (execept for the yearly WWDC waiting line of course), and it was overall a fun experience, not only because my girlfriend joined me there and we had breakfast while waiting, also, it was sunny.

The 5s has been my first new iPhone since I got an iPhone 4 in 2010 and for me, coming from an older device that didn't even have Siri, I am extremely happy with it. It may be a moot update if you already haven an iPhone 5 (or maybe even a 4s) but to me it feels like an entirely different device (not only because I also updated to iOS7 at the same time, of course).

I really like the finger sensor, as it makes login a lot faster. Of course, there're security issues here, but for me it is just a convenience method, if people are able to break through this sensor, they will also be able to use other methods to gain access to my data. I also look forward to apps that leverage the M7 as I tend to do a fair bit of walking and running and like to see at the end of the day how well I performed. I really like Siri, again, this is my first time having access to it, and I already use it for writing text messages, checking the weather, or creating schedules / appointments. In general, the speed difference between my old iPhone 4 and this one is staggering. This, of course, was to be expected, as the 4 was crazy slow at times, but it still makes me happy.

All in all, for me, the 5s is a great update and I hope that the 'most forward thinking phone yet' comment from Apple is close to the truth, as I again, plan to use it for at least 3-4 years before updating again.

<comment><USER4> The 5S is my first cell phone. I love being able to make calls from outside of my house and quickly and easily write "text" messages to my friends. I use to have to wait until I was at work or at home to make a phone call, but now I can do it from anywhere! I also love these app things, there is one for everything.

I also can't believe I can just connect to the internet from anywhere, and there are not even any tubes connected to my phone. How is that possible?

<comment><USER5> You intend this to be ironic, but a proper perspective would be that:

1) 10 years ago this was way bulkier, far less convenient and more often than not without an intertube connection,

2) 20 years ago this was absolute bare-bones and only for upper middle class and rich people,

3) 30 years ago it was science fiction.

Now, a 20-something would of course take all this for granted. But technology will have some surprises for him down the road too, and he'll learn to put things in perspective when 2030 tech is 100 times more awesome than today's.

<comment><USER6> Not sure about bulkier 10 years ago, the nokia 8310 was all the rage back then and they were so tiny that people used to lose them down cracks in sofas. Not to mention better battery.

<comment><USER5> Sure, but that was a barebones model. I had the equivalent of today's iPhone style things (a Sony-Ericsson P910 IIRC), and it was like 3 iPhones in bulk.

</comment></comment></comment></comment></comment><comment><USER7> The article fails to mention that this significantly surpasses the most commonly repeated estimate of 6M (made by Piper Jaffray).

Source: [http://news.cnet.com/8301-13579_3-57603553-37/apple- expected...](http://news.cnet.com/8301-13579_3-57603553-37/apple-expected-to- sell-5-6m-iphone-5s-5cs-this-weekend/)

<comment><USER8> FTA: Analysts had predicted between 5 and 8 million launch weekend sales across both devices

That is a link to another earlier article on Apple Insider discussing such predictions:

"In fact, Kuo said in a note to investors on Wednesday that Apple will likely ship 6 to 8 million new iPhones over the devices' first weekend of sales. By comparison, Apple racked up 5 million sales when the iPhone 5 debuted last year. Earlier in the day, Piper Jaffray analyst Gene Munster forecast 5 to 6 million unit sales, a more conservative number based on the prior model's in- store launch sales."

<comment><USER9> Notice that the higher numbers are shipped, not sold.

The iPhone 5S seems to be selling briskly, but how many 5c are still in the channel after 3 days?

</comment></comment></comment><comment><USER10> Remember all the naysaying? One story on HN claimed Apple had a "Windows 95 problem". Yet it seems that every time Apple reveals a product that tech blogs are disappointed with, it goes on to set sales records.

<comment><USER11> First of all, this is an international phone. Second. there are TWO phones. It's likely that sales figures are very similar to last year's marked-down 4S and 5 sales combined, but if you count the delayed international launch last year and combine the sales after all markets were opened to the net sales this year, there's likely little to no difference.

<comment><USER12> I agree, Apple is doomed.

</comment></comment></comment><comment><USER13> > _Apple's iPhone 5s And iPhone 5c Sell 9M Units Over Opening Weekend, Topping > 5M For iPhone 5 Last Year_

Not a fair comparison, given that 5S + 5C is two models.

It would be fairer to compare it to the sales of the 5 + 4S last year - <USER2> were probably not 9 million, but certainly more than 5 million.

<comment><USER9> As mentioned on the other thread, pointing to the official announcement, the iPhone 5 launched in China on a different date, so it had 2 million sales on its launch there.

So to get the real comparison you need to add first weekend sales of the 4S in standard launch countries and China to that 7 million figure.

<comment><USER13> That makes the 9 million figure a whole lot less impressive. From 7+ million to 9 million... but it's not that unthinkable that there could have been 2 million iPhone 4S's sold when the iPhone 5 launched, so that it could even be a step down from the previous year.

<comment><USER14> Right, 9 million is pretty unimpressive, especially when you consider that Blackberry managed to sell over 3 million last quarter. Major Fail Apple.

<comment><USER13> It's 9 million over one weekend, not over a quarter. The Blackberry number is irrelevant in that perspective.

<comment><USER14> Sorry, I was trying to be cute. Major Fail me.

</comment></comment></comment></comment></comment></comment><comment><USER15> So I had a few separate experiences with this over the weekend:

1) A friend stood in line for a few hours, got to the front, found out they didn't have the color he wanted and left.

2) A colleague in another company got one and she immediately remarked, "it looks just like Android!".

3) A neighbor returned theirs because they couldn't read all the low contrast stuff on the tiny screen.

4) Another neighbor has kept theirs, but complains all the colors and things are too bright and too loud.

5) Another neighbor went in to buy his, got some time to play with it, decided he didn't like it and went and bought a couple Galaxy S4s for him and his wife instead.

6) Coworker said he like it, thought the control panel and "slide away the background apps" was great. Hated that the slide away the emails is now the wrong way.

<comment><USER16> 1) A friend stood in line for a few hours, got to the front, found out they didn't have the color he wanted and left.

– At the Apple store I was at, Apple staff greeted people entering the queue, asked them what they intended to purchase and gave them a card (reservation)

2) A colleague in another company got one and she immediately remarked, "it looks just like Android!".

– You overheard a conversation in another company? Good ears. Extra credit for "she".

3) A neighbor returned theirs because they couldn't read all the low contrast stuff on the tiny screen.

– "Hey neighbour! I couldn't help noticing you aren't mowing your lawn with a phone?!" "The screen was too tiny neighbourino!"

4) Another neighbor has kept theirs, but complains all the colors and things are too bright and too loud.

– "IT'S TOO LOUD, HOW DO YOU SHUT IT OFF?? I'M BLIND!!!"

5) Another neighbor went in to buy his, got some time to play with it, decided he didn't like it and went and bought a couple Galaxy S4s for him and his wife instead.

– 'Hey Wifey, wanna line up in this 2 hour queue? Dum, de dum [two hours in the queue pass]. Oh hey, we're at the front of the queue! Nah this sucks. Wanna go buy some Samsung Galaxies? YES!'

6) Coworker said he like it, thought the control panel and "slide away the background apps" was great. Hated that the slide away the emails is now the wrong way.

– Your astro-turfing obviously disinterests you, as does your job and it shows dude :[ Quit your job.

<comment><USER15> > – At the Apple store I was at, Apple staff greeted people entering the > queue, asked them what they intended to purchase and gave them a card > (reservation)

Didn't do that at either of the two Apple stores I'm aware of (there are 5 in my area), or the three AT&T stores my friends went to.

> – You overheard a conversation in another company? Good ears. Extra credit > for "she".

Grownup companies sometimes do things together, not every other company is your competitor.

> – "Hey neighbour! I couldn't help noticing you aren't mowing your lawn with > a phone?!" "The screen was too tiny neighbourino!"

My neighborhood has a very active community message board and I know many of the member personally from neighborhood events.

> – "IT'S TOO LOUD, HOW DO YOU SHUT IT OFF?? I'M BLIND!!!"

Here are the exact quotes

Neighbor 1: "I'm not a fan. Hate what they did to ical. The new icons look like they are for small children (that i can get over). And the bright white is obnoxious. What the hell did they do to safari?? I'm sad ... But im sure i will eventually get used to it. "

Neighbor 2: "I'm not "upgrading" yet. Hate the font."

Neighbor 1: "Everything is bright white and hard for me read...feeling old."

Neighbor 3: "So far I actually like it. The calendar took me a minute to get used to as well. At first I didn't like it until I realized you get a 5 day shot when you turn it on its side. I can't figure out how to do a list view yet - is there one?"

Neighbor 1: "I have light sensitivity issues (Astigmatism in both eyes - and I also have cataracts - God I sound old!) so the bright white is just harsh for me and not having huge contrasting colors is also hard on my eyes. Yep I agree the one swipe to have controls is nice and I do like what they did with the photos."

Neighbor 4: "Steve Jobs is probably rolling over in his grave right now..."

Neighbor 5: "I downloaded on my iPhone the other night, not a fan. But I downloaded on my iPad earlier today, and I don't mind it at all. But it is way too bright."

Neighbor 1: "Thank you for saying its too bright ... I was beginning to wonder if it was me."

> – 'Hey Wifey, wanna line up in this 2 hour queue? Dum, de dum [two hours in > the queue pass]. Oh hey, we're at the front of the queue! Nah this sucks. > Wanna go buy some Samsung Galaxies? YES!'

Talked to him in person (he's next door). And that was pretty much the conversation.

> – Your astro-turfing obviously disinterests you, as does your job and it > shows dude :[ Quit your job.

What does my job have to do with this?

<comment><USER17> Minimally, the first point is wrong. Official Apple Retail stores all do this the same way. Reservation cards were provided. It's possible that the person while in line was informed they didn't have the color they wanted and left, but highly, highly unlikely (unless they entered later in the day) that they made it into the store and had to walk out.

FWIW, I did see a few people leave because the color they wanted was not available, either to try a carrier store or order online and wait.

Also, 9M in sales and the huge upgrade numbers pretty much speak for themselves. YMMV, but the anecdotes from your circle are not reflective of what happened.

<comment><USER15> You're probably right about the apple brand stores. My personal network didn't relay it that way though. Time will tell if these are sustainable sales figures. They are certainly impressive no matter what. But with a year between phones, I don't think a weekend hit sales model is the best way to do things.

</comment></comment></comment></comment></comment><comment><USER18> most frustratingly this is another datapoint that won't matter much in the current slew of punditry aimed at talking Apple down.

just go back to the HN post of the 5s launch event and it's full of doomsayers.

seems like apple is going against the very fabric that unites neckbeards around the world - and is crazy successful. just like carmakers that stopped catering to the tinkerers. here's to the crazy ones.

next week? we'll return to the usual apple is doomed because innovation and open blabla.

<comment><USER18> and to illustrate my point, HN post, a mere 12 days ago:

Apple's Windows 95 problem. [http://blog.raavel.com/2013/09/11/apple- problem/](http://blog.raavel.com/2013/09/11/apple-problem/)

"Apple recently unveiled its new iPhone 5s and 5c models, to a collective "meh". The stock sold off and is poised to continue moving lower. The market is telling Apple that it is not innovative enough and that its need to do much more, and within that lies the crux of Apple's problem."

But yeah, a lot of comments, falling over each other in their collective agreement. will that same random blog now turn 180 and consider it's failure?

<comment><USER19> _Will that same random blog now turn 180 and consider it 's failure?_

No.

(Queue a half dozen responses about how Apple actually _has_ failed by selling more phones than both Nokia and Blackberry last quarter in a single weekend. Sometimes, I think people who think Apple is a failure are actually it's biggest supporters, because really, who but those most invested in the company's success are pushing for it to do better? You'd only call them a failure if you thought they _could_ do better, right? So...tell me who the Apple fanboys are, again?)

</comment></comment></comment><comment><USER20> Clearly they are going out of business. Not to mention only 200 Million devices running iOS7 so far.

<comment><USER21> Is anyone actually claiming that Apple are going out of business? This kind of reflexive defensiveness doesn't really help anything.

<comment><USER22> Um, _excuse_ me, don't you _realize_ that Android has been installed on approximately 8 million _times_ the number of devices that iOS has been installed on? Think of how many feature phones, Arduinos, and toasters have all of the same capabilities Apple's iterative, piddly phone offering. Probably more features, now that I think about it, since, like, I don't have to _jailbreak_ my toaster to get all of the features I want.

<comment><USER21> Mocking Android fanboys in a forum overwhelmingly dominated by Apple adulation? I hate to use a Reddit/4chan meme, but _so brave_...

<comment><USER23> Funny, I thought HN was dominated by Android fanboys. Perception is everything eh?

<comment><USER24> Quote from almost a decade ago by a Linux fangirl - "I love MS products, but I cannot stand their policies"

My perception is that here the situation is similar.

</comment></comment></comment></comment></comment></comment><comment><USER25> This video sums it all up: [http://m.youtube.com/watch?v=oprUI6nupfc](http://m.youtube.com/watch?v=oprUI6nupfc)

<comment><USER26> I couldn't watch it in one go... this is staged right ?

<comment><USER25> Check this also. It's old but personally knowing fan boys, I know they would act exactly like this- [http://m.youtube.com/watch?v=rdIWKytq_q4](http://m.youtube.com/watch?v=rdIWKytq_q4)

<comment><USER26> I saw that when it came online, you can't prove the latest kimmel video's honesty by using an older one. He also revealed posting fake youtube fail videos so now everything he shows is questionnable.

<comment><USER21> _now everything he shows is questionnable_

Everything he ever showed is questionable, he's an entertainer not an investigative news reporter. However, pretty much all of these segments have been real in the past, so I'm not sure why that would change now. They obviously cherry pick respondents.

<comment><USER26> Fair enough. But how do you know the previous ones weren't staged ? Maybe they weren't, it was a believable prank to compare a device with itself and see the average person make up fantasy differences. But here it's an ipad mini of twice the surface, and screen tasting ... I mean come on.

<comment><USER21> What do you mean by staged? In a sense they all are- I'm sure they ask the question to a lot of people who say "that's not an iPhone, that's an iPad Mini". But they edit those ones out. They're not intending to portray an accurate image of the American consumer.

<comment><USER26> Staged as in the girl licking is acting through a script.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER27> Nothing moves the stock like a successful product. I am glad they did not bow to market pressure and announce more dividends or stock buybacks. Stock buybacks are just dead money. They should instead use that money on their business and not to satisfy investors over the short term.

<comment><USER28> > Stock buybacks are just dead money

That is an oversimplification. Apple has more money than it could possibly hope to reinvest in its core business. When that is the case, and when the stock is undervalued as is the case with Apple, stock buybacks are a great use of a company's cash.

<comment><USER27> The point behind stock buyback is simply to assert some confidence in the stock. Which means the market does not have confidence in it in the first place and probably has negative momentum. Most cases I have seen the market is correct. So even if the stock is bought back the increase in the price will be temporary as the negative momentum will surely vaporize the stock increase. It does not help anybody but investors trying to make a profit in the short term. I have not seen a single stock buyback work. I would rather they throw money at anything but this.

<comment><USER28> > I have not seen a single stock buyback work

I haven't seen a single penguin in the wild, but that doesn't mean they don't exist ;)

Berkeshire Hathaway and Teledyne are two companies I know off that have had great success with stock buybacks.

Read this [http://observer.com/2003/04/the-brain-behind-teledyne-a- grea...](http://observer.com/2003/04/the-brain-behind-teledyne-a-great- american-capitalist/) for a fascinating account of Teledyne's story.

</comment></comment></comment></comment><comment><USER29> This is the glorious triumph of consumerism. Steve Jobs would be proud.

</comment><comment><USER30> 5S is much more in demand than the 5C. Pretty good, I say.

<comment><USER31> I'm honestly not too sure about that. Knowing the split between the two would be very interesting.

I would imagine that they can make tons of 5c phones while the 5s may be more problematic. So the balance between the two may be more even than it seems. (Also, I would be really interested whether the 5c or 5s is the more high margin device. I could imagine that at least for now the 5c has a higher margin than the 5s – well, at least the 16GB 5s.)

<comment><USER32> Apple's pricier iPhone 5s reportedly outsells more affordable iPhone 5c by wide margin

[http://appleinsider.com/articles/13/09/23/iphone-5s-3-times-...](http://appleinsider.com/articles/13/09/23/iphone-5s-3-times- more-popular-than-iphone-5c-on-launch-weekend---report)

<comment><USER33> "This makes sense since those who feel the need to buy a new device the very weekend it launches are most likely the power users who want the highest-end phone experience."

</comment></comment></comment></comment><comment><USER34> When I click this link I automatically download a file called widgets- tweet_button.html.torrent

<comment><USER35> [https://gist.github.com/gregclermont/6669056](https://gist.github.com/gregclermont/6669056) \- It's being worked on. Doesn't look to be malicious at current.

</comment></comment><comment><USER36> 9 million sold. 47 were available in gold.

<comment><USER37> Realistically, I'd estimate somewhere in the ballpark of 4,000, which is only slightly less absurd than your joke. From what I've heard, Apple stores typically only had ten or so gold units each available on launch day. That's what it was at the store where I was, and I've heard the same from others. Multiply that by the 413 Apple stores worldwide and... that's not very many. Pretty crazy.

<comment><USER38> No, but Apple probably sold most of their phones via their web site and not from retail stores.

Otherwise each retail store would have sold 20000 phones by the reverse of your math.

So 4000 is probably not a very accurate estimate either...

<comment><USER37> Even if 90% of sales were online, that's still only 40,000. An order of magnitude or so of accuracy is all I'm shooting for anyway.

<comment><USER38> I am sorry, but I think you are assuming a little bit too much here.

You are guestimating that 4000 gold phones were sold in the stores based on a 10 per store rumored number you've heard and 400 stores. Then you guess that retail is only 10% (another guess) and so multiply you original guestimate by 10 to get 40000 gold phones.

It's quite hopeless guesswork. We don't know the distribution of models in retail or online. We don't know how much is sold online compared to retail. We don't know how much was sold outside Apple's retail stores (by e.g. AT&T). We don't even know the geographical distribution (i.e. higher gold-ratio in China than US).

If we assume a factor 3 to 1 sales of iPhone 5S vs 5C based on Localitics'report (1), Apple has sold between 6-7 million iPhone 5S. Your 40.000 number isn't even 1% of that, so I think you are an order of magnitude off still.

[1] [http://appleinsider.com/articles/13/09/23/iphone-5s-3-times-...](http://appleinsider.com/articles/13/09/23/iphone-5s-3-times- more-popular-than-iphone-5c-on-launch-weekend---report)

<comment><USER37> How are you assuming any less than me?

Of course mine is a complete guess. I make no claims otherwise. But so is yours. So kindly cut the attitude.

<comment><USER38> Sorry. No atitude!

Just pointing out that it is impossible to guess, and that your _last_ number constitutes less than 1% of the total estimated iPhone 5Ss sold.

<comment><USER37> OK, I probably misread the tone in your first sentence.

It's easy to guess! Nearly impossible to guess accurately, though, I agree.

You're right about it being less than 1% of the total, although I disagree that this makes the estimate absurd. I think it's entirely possible that the gold 5S constitutes less than 1% of production or sales so far. I certainly could be wrong.

<comment><USER39> The under supply can't be too bad though - I just checked on eBay, and 16gb gold iPhones are going for around $1000. You can buy a TMobile contract free for $650 online through the Apple Store. That's only a 50% markup...

<comment><USER37> Man, does eBay's search ever suck. I specify gold, 5S, unlocked, and it still gives me all sorts of irrelevant listings.

Looks like the typical price is around there, though. I do see some sold for well over $2,000, though. I wonder if that means the $1,000 listings are underpriced, or if those high-end listings just got lucky and found a few suckers.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER40> I wouldn't be so quick to start doing the Apple victory dance. A lot of people waited to upgrade because the iPhone 5 was an iterative device and I have several anecdotal cases where people I knew said they would wait for a 5S.

Apple has painted itself somewhat into a corner now. Similar to Windows Service Packs, people are waiting for the second version of Apple products to get it 'right'.

This isn't to say they got it 'right' THIS time, either. It just is what it is.

<comment> k-mcgrady >> "A lot of people waited to upgrade because the iPhone 5 was an iterative device"

What?? The 'S' is the iterative product. The 5 was a new design and new screen size and overall a much bigger update.

>> "people are waiting for the second version of Apple products to get it 'right'."

What was wrong with the 5 that the 5S fixes?

<comment><USER40> Based on Apple's existing precedent to 'revolutionize' phones to the average consumer, I'd consider all the devices in the 5 generation to be iterative.

<comment><USER41> Let's look at that.

2007: iPhone.

2008: iPhone 3G. Same phone but with 3G.

2009: iPhone 3GS. Same as 3G but a bit faster.

2010: iPhone 4. New case, retina display, faster etc.

2011: iPhone 4S. Faster, better camera and Siri.

2012: iPhone 5. Faster, new case, bigger screen.

2013: iPhone 4s. Faster, Touch ID, better camera.

It's hard to say any of those releases were revolutionary except the original iPhone. All since has been steady improvement. And that's a good thing.

<comment><USER42> yes, incremental innovation is still innovation.

The fetishization of "big bang innovation" is really bizzare. Especially considering most of HN's startups are not really big bang innovation.

</comment></comment></comment></comment></comment><comment><USER43> And... AAPL is +6.9% pre-market.

<comment><USER44> +20.17 @ 9:59 AM

</comment></comment><comment><USER11> It would be wise to post the official Apple press release: [http://www.apple.com/pr/library/2013/09/23First-Weekend- iPho...](http://www.apple.com/pr/library/2013/09/23First-Weekend-iPhone-Sales- Top-Nine-Million-Sets-New-Record.html)

</comment><comment><USER45> We need a country breakdown considering this launch included China (unlike previous launches). That said, I'm assuming 30-40% in China and the rest elsewhere still puts it as a great launch.

<comment><USER9> Including China sales the increase was 29% (7m -> 9m).

33% China would be 50% increase in China (2m -> 3m) and up 20% (5m-6m) elsewhere.

40% China would be a near doubling in China (2->3.6), and up 8% (5m->5.4m) elsewhere

For reference, the smartphone market is growing at about 50% a year, in Q2 Apple's sales numbers rose 20% year-on-year and their share dropped over 3 percentage points year-on-year (16.6% -> 13.1%).

(These launch numbers assume that no-one bought a price-dropped 4S on the launch weekend last year)

</comment></comment><comment><USER46> Could Some one tell me how these people game the stock market by releasing news that are um, false or unrealistic.

I mean Analyst and Media, wanting to trash Apple for headlines has become insane.

</comment><comment><USER47> you'd think no one needed to line up at all seeing how the oversold the most common estimate by 3 million phones.

</comment><comment><USER48> It goes to show you just how hungry people were for change on the platform. Next up has to be screen size which would result in a similar massive weekend for the 6.

<comment><USER36> This was just their minor release cycle, so yeah it will be crazy to think how the 6 will sell.

<comment><USER37> I wouldn't be so sure. The 4S sold over four million units on its opening weekend, which is pretty close to the 5's five million, but way more than the 4's 1.7 million.

Personally, I'm deliberately on the "S" upgrade cycle. I started off with a 3GS, then got a 4S, and now have a 5S. I figure I'll let other people deal with the bleeding edge, and I can get the model with all the refinements afterwards. I'd be willing to bet that a lot of other people think the same way. I don't doubt that the 6 will sell really well, probably better than the 5S, but it may not sell _vastly_ better than the 5S.

<comment><USER49> I think it's also a good idea to wait a month or so after release. What's the rush? Let the first production runs go through and work out the kinks. So far I have heard some reports of screen issues on some 5s models. With a new and unusual part like the fingerprint sensor I'd rather wait a bit.

<comment><USER37> Personally, my rush was that I needed a device with an A7 in it for testing purposes. If it wasn't for that, I'd be right there with you and would have just ordered online and waited.

<comment><USER49> That makes sense. I'm hoping the 64 bit simulator is good enough for now.

<comment><USER37> It will be for almost everybody, as long as you have someone _somewhere_ with a 5S to use for a sanity check before you ship.

In my case, there was some code dealing with low-level stack walking and such that really needed the actual hardware in hand. Not a common case, to be sure.

</comment></comment></comment></comment></comment></comment></comment><comment><USER50> There you go, more proof that most people don't give a shit about surveillance, privacy and the lot.

Here it is, the one device that can create the biggest fingerprint database in the world sold 9 million units in a couple of days. NSA must be jubilating.

Very "forward thinking" indeed.

<comment><USER51> You missed the part where the fingerprint is stored so it's not even accessible to the OS and are never stored online?

Oh, but I guess the NSA has a back door and will manage to collect them all without anyone noticing, right? Just like they record all the sound from your phone mic and upload it?

I'm all against unlawful spying by the NSA, but give me a break.

<comment><USER52> >You missed the part where the fingerprint is stored so it's not even accessible to the OS and are never stored online?

I guess you missed the past 14 weeks of revelations. Either that or I admire your dedication to believing general company claims in the face of all evidence that this is not the status quo.

<comment><USER51> I didn't miss them, and I'd like you to link to the revelation that tha NSA routinely collects data stored locally on iPhones with Apples consent and involvement.</comment></comment></comment></comment>


Cuba Eliminates Mother-To-Child Transmission of HIV and Aids - <AUTHOR> https://www.youtube.com/watch?v=WKS01RV-jRo <comment><USER2> Is there any peer reviewed paper proving this?

(Cuba is claiming they have finally cured cancer every 4 months.)</comment>


Kite – Programming Copilot - <AUTHOR> https://kite.com <comment><USER3> First of all, this is a cool looking tool. Useful, well-organized, easily- accessed information is always a wonderful thing.

I think a model like Dash would work a lot better: Rather than opting in my code to be sent up to Kite's servers, I opt in to the packages I'm interested in having indexes for. In some cases, as with node, there are ways to see what packages my project depends on and then those bits of indexed data could be sent to me.

My computer has a 512GB SSD. I could devote 10GB to indexes of libraries and my own code without blinking. The argument that it's too much data and therefore belongs in the cloud doesn't seem to hold up.

Also, there are cases where I'm not online... this is one of the great things about Dash. I have fast access to library docs anywhere I happen to be.

<comment><USER4> Yes but how much CPU and memory are you willing to spare? Parsing is quite CPU-intensive, hence the CPU drag you often see when IDEs start indexing, and type inference involves a lot of unpredictable lookups all over the index, so much of it needs to be in memory to get reasonable performance (yes this is still true if we're talking SSD).

To see why: when you type "x.foo()" we need run type inference on the complete data flow chain that produced the value "x", so that we know which particular "foo" you're using. Throughout this analysis we may also need to know a lot about the python libraries you're using, since you may be passing values into and out of arbitrary third party libraries. If each of the steps in this chain triggered an SSD read then you'd often have a multi-second lag between hitting a key and seeing the result.

<comment><USER5> My editor had no problem indexing all of Chromium (fairly large project). It also indexed external libraries. It indexes new code as you write it so type a foo function, next time you type foo you get help immediately. It added standard libraries by default and you can add any other library (like I have it indexing Unity3D's mono libraries).

I didn't notice more than a 100-200ms delay in seeing the result (which happened in other threads so no effect on my editing). About the same I'd expect with a round trip over the the internet. It shows both help at the cursor as well as definitions and references in another pane in that time.

It doesn't look as slick as Kite but it also seems to suggest it's possible to do this all locally. If nothing else you at least have some context (the language) so you don't have to search all data, only data relevant to that language. You even know where in the language I am so you know when to search ids and which subset of ids to search.

On top of that you're basically going to have me sending a gig of source to you to index something like Chromium which will take hours on my crappy connection.

Let me be clear, I think kite looks amazing and I'd be happy to pay for it if it was local. Maybe you download the DB to my machine. I'm not nearly as comfortable with you reading my terminal though. I'm sure you can turn that feature off but that's a feature I liked. Turing features off = less interesting

<comment><USER6> What editor?

<comment><USER7> I'm not an OP, but Eclipse does quite well indexing fairly large projects.

</comment></comment></comment></comment></comment><comment><USER8> Privacy question - what will Kite do with the source code stored in its databases when/if Kite shuts down or is acquired? How will you respect the licenses of the files you upload?

(paraphrased) "You trust Github/Slack, why not also trust us?" Perhaps because Github and Slack's monitization models are well known, and Kite's are not.

It's a great concept for a tool, but I could never get it to fly at work. I couldn't even begin to imagine trying to convince a technically savvy cofounder (let alone their lawyer) with "let me use this tool which uploads all of our code to their server."

<comment><USER4> Alex from kite here. Re privacy: we totally agree that it's a legit concern. when we started working on this we realized if we wanted to index tens of thousands of libraries, we wouldn't be able to ship the entire index along with the client. Hence the cloud-based architecture. We've thought a lot about privacy and written up our thoughts here: www.kite.com/privacy. The short answer is: we don't index anything on your computer that you don't explicitly ask us to, and our plan is to earn trust the hard (i.e. only) way: transparency, published policies, and a track record of good decision making.

<comment><USER8> I did read the privacy document, which does not address the acquisition/shutdown aspect, which is fairly important; Oracle (for example) may not have the same views on the privacy of the acquired data as you do.

Also, do you have plans to support deletion of indexed data?

<comment><USER9> I think this is a great point. Does anyone know if it's possible to bulletproof against what an acquirer might want to do with the data? Is there a way, for instance, to shift the ownership away from the company gathering the data such that if ownerhship of the company changes, ownership of the data does not?

<comment><USER10> Lawyer here. Not my area of speciality but off the top of my head (and after thinking about it for all of 30 seconds) that strikes me as a surprisingly hard thing to do. Bankruptcy courts have extremely broad powers to administer the assets of debtors including disavowing contracts. There may be some way to do a structure where the data is escrowed with a 3rd party and the subject company is just holding the data as some sort of fiduciary, but I'm not sure anything like that has been tested. I would want to consult a bankruptcy expert to really figure something like this out.

<comment><USER11> What if Kite can't legally use the data for alternative purposes? I don't think disavowing a contract via a bankruptcy would let a company _sell assets they don 't own_.

So Kite should be able to avoid this fear by _asking only for limited license_. For example, a license can expire after 1 year, or be untransferable (or perhaps expire at bankruptcy?).

Facebook does this to some extent: "This IP License ends when you delete your IP content or your account unless your content has been shared with others, and they have not deleted it."

Here's what Heroku does: "Heroku claims no ownership or control over any Content or Application. You retain copyright and any other rights you already hold in the Content and/or Application, and you are responsible for protecting those rights, as appropriate. By submitting, posting or displaying the Content on or through the Heroku Services you give Heroku a worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute such Content for the sole purpose of enabling Heroku to provide you with the Heroku Services. [...]"

(IANAL)

</comment></comment></comment></comment></comment></comment><comment><USER12> First, this is amazing. I want.

Second, I do a lot of code readings that include not shareable code, I would imagine a lot of developers from a lot of companies would be in the same position. From [https://kite.com/privacy/](https://kite.com/privacy/):

* all the python files in your authorized directories are sent over the network

* everything you type in these files are sent as well

* all the terminal commands you are going to type (ouch ouch)

> Q: How does Kite secure this network traffic?

> A: As you would expect, all traffic goes over https.

Yeah unfortunately that is not enough, a MITM can also use https. How do you authenticate the server? Do you have certificate pinning?

> What information does Kite keep around on its servers?

Pretty much all the info I previously talked about, that you are now worrying about, is kept there, in clear (correct me if I'm wrong).

This is a big no-no at this point.

> Many developers have already chosen to trust their code to services such as > Github and Bitbucket

Many developers also do not trust Github/Bitbucket with there code (and they should not) and do not store secrets there. And who would want github to have access all the terminal commands they type? This sounds like a nightmare.

At this point I don't see why anyone would use that, if not in a VM, with extreme care on what commands are typed in the terminal and what code is used with Kite.

</comment><comment><USER13> I was really excited, then very disappointed to see the code I type is sent to your servers. This immediately prevents it from being used by many businesses, including my employers.

Super bummer :(

Please get the on-prem sorted out pronto!

<comment><USER14> seriously, I don't think anyone is after your code.

<comment><USER15> I'm not concerned about people stealing my code. I'm concerned about losing my job, which is the consequence of transmitting proprietary code over the network to an unknown 3rd party.

<comment><USER16> Also plain legal implication. Our code could contain some protected information (transient debugging code mostly) and it is illegal to transmit it to a third party first because of DNA our company has signed, and then with regards to data protection act (credit card information, name, address).

At the very least, the service would need to be validated quite extensively.

</comment></comment></comment></comment><comment><USER17> Hi everyone, we couldn't be more excited to tell you about Kite and get your feedback!

We think connecting programming environments to a smart backend will improve programming in a lot of ways, and this is just the first step. We'd love to hear your thoughts on where we should go from here!

<comment><USER18> This looks great! Loved the left-pad joke in there.

Could you elaborate on when it'll be available? Will you make it or parts open source?

<comment><USER19> Also the hostname of "TayandYou".

<comment><USER20> And the Batman v Superman. lol

</comment></comment></comment></comment><comment><USER21> While I really like the idea, there's a couple of no gos here:

        * Only supports python (as of now, I know you said you'll do others)
        * No Linux support (yet)
        * Uploads code to your servers
        * Wants me to sign up to a crappy newsletter

Really liked all the jokes in the video though. But I'm not convinced (yet), sorry.

And there's the issue that's I'd want to test it before recommending buying it to my employer. But I don't do enterprise Java for fun, so there's completely different languages and libraries between the open source projects where I could upload the code (but don't want to because fuck everything cloud, also I won't pay for it, because I'm not making any money) and the for work projects (where I personally don't give a shit, but all the libraries/frameworks the we/the customer bought are proprietary). And maybe even the customer wouldn't like his code being uploaded somewhere, but they usually don't really care about the code.

</comment><comment><USER22> I don't want undermine the value of this product, but this seems like something that I've (and many others) sort of already solved by using products like Dash [1] and Alfred [2] together. You can easily and quickly find all info on docs while working right in Sublime Text, or while also using an IDE to read into functions throughout your project. And maybe I'm missing more what this product does, but that's just what I noticed based on the video.

[1] [https://kapeli.com/dash](https://kapeli.com/dash) [2] [https://www.alfredapp.com/](https://www.alfredapp.com/)

<comment><USER23> Interesting. How are you connecting Dash and Alfred to arrive at something similar?

<comment><USER24> I have used both the Alfred workflow [https://github.com/Kapeli/Dash-Alfred- Workflow](https://github.com/Kapeli/Dash-Alfred-Workflow) and the Vim connection [https://github.com/rizzatti/dash.vim](https://github.com/rizzatti/dash.vim) in the past. They are useful and provide very fast symbol lookup.

<comment><USER25> Or, if you use SublimeText then DashDoc comes really handy.

[1] [https://github.com/farcaller/DashDoc](https://github.com/farcaller/DashDoc)

</comment></comment></comment></comment><comment><USER26> So if there was an open source version of this, I'd use it. I would never trust it otherwise. Large corps will never trust it either, IT sec policies would roast you for using this. Instead I see an opportunity for IDE's to step up this space and provide this built in, without copying and _keeping_ your source code.

"What information does Kite keep around on its servers? Usage information about which results you click on in the sidebar. Contents of all Python files in enabled directories."

<comment><USER27> Exactly, I really don't want to have to tie myself to a single company for my tools, and I definitely don't want to be handing over all the code I type to that same company. Having a company sponsored key logger just to remove googling for documentation/basic autocomplete is not something I'm interested in doing.

</comment></comment><comment><USER28> How does Kite protect against accidental API keys and passwords in source code (copy/paste) being shipped to them?

</comment><comment><USER29> I was about to ask some questions about exactly what gets sent to Kite and what you do with it. Your privacy page[0] answered my thoughts and concerns quite nicely.

[0] [https://kite.com/privacy](https://kite.com/privacy)

</comment><comment><USER30> I was exited until:

        What platforms does Kite run on?

        For now, Kite only works on OS X, but we'll support Linux and Windows soon, too.

bummer :(

<comment><USER17> I started slinging code in Visual Basic 5 back in the day, and ran Windows until recently. I hope to be able to jump into building the Windows client soon : )

Fortunately all of our UI is html and javascript, so no windows controls needed : )

<comment><USER31> +1 for Electron sounds like you have a knowledge gap and are missing an easy cross platform opportunity.

</comment></comment></comment><comment><USER32> This looks very slick. However, as others have said, uploading all source code to your servers is a pretty serious issue when working on anything other than open source projects.

Also, when asking about editors while signing up, you listed "XCode". That's not the correct spelling, it should be "Xcode" (lowercase c). Also, you should allow for selecting multiple languages/editors. I selected Swift/Xcode because that's my day job, but I also use Swift/Vim in some cases, and I use Vim for languages other than Swift/Obj-C. In fact, I'm guessing Kite won't be nearly as useful for Swift/Xcode as it would be for other languages, because Xcode already provides a lot of this functionality (e.g. intelligent code completion and quick help for any API).

<comment><USER33> I wouldn't say Xcode is providing this... kite looks to be going much further in providing examples, alternatives, etc with a diff screen real estate philosophy. I welcome it.

<comment><USER32> Yeah Xcode doesn't do everything Kite does, but it does enough that Kite isn't nearly so compelling there. And Xcode has the benefit of being 100% accurate with its code completion and quick help, whereas it remains to be seen how accurate Kite will be on that front.

</comment></comment></comment><comment><USER34> This is a great idea. I particularly think the trade-off between a separate tile vs. a traditional overlay is interesting.

While we lose things like auto complete, it might be less intrusive for the times when we don't need help. It will be interesting to see if we see a new generation of smart IDEs.

</comment><comment><USER35> While I realise that this could be helpful with editors.I do not see much use with IDEs.and when it comes to languages like Android/iOS/WebApps ?People tend to use IDEs.(I am not making my opinion the general opinion here). And when people google things,Documentations are often the last resort.You tend to google the exception or the specific condition and end up on StackOverflow where some one facing a similar problem solved it by using something which was mentioned somewhere deep in the documentation. Also if documentation and a few examples could work in most real world apps,then we really wouldn't need sites like Stack Overflow.

Simple use case: the `man` and `help` command are useful when you have not been able to solve for your particular use case.Else I would still prefer doing a google search which would tell me how to scp a file with the syntax and placeholders instead of reading the documentation.

The examples would help in this regard.But then again I am not sure how much.Normally documentation list pretty straight forward examples which any editor/IDE with intellisense (even Sublime provides a bit of prediction with plugins) ,should be able to provide.

This also goes against the tendency of comparing.Normally while trying to solve a problem,I tend to open 2~3 similar 'answers' which help me drill down to my particular problem.Whereas here I would be restricted to only a single solution.I love the idea for the technology involved.But I am not completely convinced with the help that it could provide.

</comment><comment><USER36> This looks really interesting. One question I have is how will you handle hotkeys? If I'm focused in my editor, I don't really want to have to move my mouse to select options suggested by Kite.

Excellent demo. Concise, simple overview of features but with enough little details (loved that rm -rf warning!) to keep me interested, and really well- edited.

<comment><USER37> I certainly hope there is a way to use Kite comfortably with minimal/no mouse usage.

<comment><USER38> Hi, Jay from Kite here. Agreed. We are definitely heading in the direction of keyboard shortcuts to access key features in Kite. For example, we have Cmd+; for switching between your editor and Kite. And more are coming up. We want you to get the most relevant info without leaving your keyboard!

</comment></comment></comment><comment><USER39> This looks really cool. Bothers me having my code sent over, because i can't use it for a variety of use cases, but for my personal work i may give it a try. I've got two main desires though:

1\. In editor pane. I use Vim in a full screen term, and really want want to deal with managing OS windows to allow a Kite window side by side with iTerm2. Perhaps an overlay would be solve this? That way it works for bash/etc?

2\. Rust support. I'm learning rust lately, and this sort of tool could _really_ benefit me with a learner-error-prone language like Rust. Based on the signup form though, Rust does not appear available.

<comment><USER4> 1\. Yeah, I agree. It's hard, and it's something we continue to iterate on. How would you feel about a global keyboard shortcut that would show/hide the overlay that was a separate application?

2\. I love Rust. We _will_ get to it even though it's (sadly) not at the top of our list right now.

</comment></comment><comment><USER40> Probably not the problem you're trying to solve but I'd like a better typing assistant as well as a co-pilot. For example, on the first line, I'd like to type "r" then be prompted for "import requests" Anywhere where the video had to be sped up is ripe for an advanced typing assistant. Typing entire lines like this seem unnecessary:

"from left_pad import left_pad"

I guess a super autosense along with Copilot would make for more exact coding.

<comment><USER4> Absolutely. We're going to be working on this for many years, and this is the kind of thing we're aiming at.

</comment></comment><comment><USER41> All "my BigTech Co won't let me use this at work" and other privacy concerns aside, this is an amazing tool for budding programmers or experienced ones using an unfamiliar language. Can't wait to try it out!

</comment><comment><USER42> Well, definitely this is a great project but at the moment, for what i was able to see in the demo video, the main features are very similar to code completion and API exploring as i can currently perform with my ide (i develop mainly java based app so IntellijIdea or Eclipse give me quite the same help).

Don't want to be rude but it doesn't seem like this tool may currently enhance my productivity.

<comment><USER4> The main difference versus IDEs is that everything we show is informed by all the public code we've collected from the web. So e.g. there are a ton of arguments to matplotlib.plot and IDEs can show you them all ranked alphabetically, whereas we can show you common patterns of how people actually use matplotlib.plot in practice, which is often far more useful. (We'll show you the docs, too.)

Another example is if you type "load('abc.json')" without having imported json: there are hundreds of python packages that define a function called "load", but "json" and "simplejson" are by _far_ the most widely used, so we can suggest that you "from json import load". That's something you can't do unless you have a good model of a lot of real-world code.

<comment><USER43> Any comments on how your product and plans compare to [https://sourcegraph.com/](https://sourcegraph.com/)?

They seem to have a similar offering with an on-premise version, though lacking the smooth editor integration.

</comment></comment></comment><comment><USER44> Agreed that this looks awesome. Seeing code completion and context sensitive help for running terminal commands (e.g. netstat), not just programming languages, was what made me hit the sign up button.

<comment><USER45> You might want to check out the Fish shell. I've been using it for about a year and it supports completion for most commands by parsing their man pages. It's pretty nice.

<comment><USER44> I tried Fish a while back, and liked it very much. Trouble is, I can't roll Fish out across a work server estate.

Well, I could. But then I probably wouldn't be at work much longer. ;-)

</comment></comment></comment><comment><USER46> > Left padding is so complicated that I've pulled it out into its own file, as > you can see here.

Am I just really ignorant, or is this satire?

<comment><USER34> Its satire directed towards the recent npm package fiasco (there was a 'leftpad' library by the original author who pulled his packages)[0].

The authors make a few other witty jokes in the video as well.

[0][http://blog.npmjs.org/post/141577284765/kik-left-pad-and- npm](http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm)

</comment></comment><comment><USER24> Often I work in a full screen editor environment like terminal Vim with Tmux. Are you considering support for a command line client that could run in a split pane or window alongside my terminal editor?

If you implemented such functionality the suggestion experience could be more seamless for myself and those with similar workflows.

<comment><USER17> Supporting the variety of dev setups will definitely be one of our challenges. We've tried to start addressing this with 2 column mode (not shown in the demo video, but happens when Kite is in landscape mode; good for multimon), snapping to the side of your screen, and left/right-handed orientations.

This will be a long term effort. We have a lot more work to do here, along the lines of assisting with Z-index management, attaching to windows, etc.

Overall there's a lot to do! We will get there and are signed up for the long term.

</comment></comment><comment><USER47> This is similar to the Developer Assistant plugin for Visual Studio made by Microsoft: [https://blogs.msdn.microsoft.com/onecode/p/devassistant/](https://blogs.msdn.microsoft.com/onecode/p/devassistant/)

</comment><comment><USER48> (scribbles with mouse, clicks on things)

"When I open my terminal...."

I am evidently not the target market.

Do professionals actually identify with this stuff? Am I really that out of touch when I think that that much mousing around is CRAZY slow? And who switches back and forth from GUI editors to a terminal?

<comment><USER49> Noah from Kite here. We're prototyping various keyboard shortcut schemes; we're excited to ship whatever we settle on.

</comment></comment><comment><USER50> The title is a bit misleading. This only works for Python and OSX.

<comment><USER38> Jay from Kite here. We're on our way to supporting more languages and operating systems. Stay tuned! Pro tip: when you sign up on our website, you can also specify which language you use so we can keep you in the loop when we get to it!

<comment><USER50> You folks have a big team, and have been working on this for two years now. Promising another language in the future doesn't mean the title is right.

<comment><USER51> It can't be used right now anyways, so if it currently doesn't support your favorite language, what does it matter? Who knows what it will support once we get an alpha/beta/release version?

Do you also tell the people that show you a prototype of their product that they are misleading you, because they claim to build an extended product out of that prototype?

</comment></comment></comment></comment><comment><USER52> Hmmm. Regarding one member of their team: "Prior to Kite, he was a tech lead at Homejoy, driving customer acquisition and retention." Not to spread tooooo much guilt by association, but didn't Homejoy just get roasted around here not too long ago for what thy did with the data on their servers? [1] Doesn't really make me feel the warm fuzzies for what they might do with all that data if they shut down.

[1] [https://medium.com/@johnsalzarulo/didn-t-homejoy-shut- down-e...](https://medium.com/@johnsalzarulo/didn-t-homejoy-shut- down-e8d7a2dfb485#.97zeolsll)

</comment><comment><USER53> So it's watching my bash terminal? Does it send my passwords on over to your servers when I ssh somewhere without keys?

<comment><USER4> No! If you can't see it (as visible chars) on your terminal then Kite is not seeing it.

<comment><USER53> It sees all the visible chars? Not just what I type? So every time I ls all of my filenames and dirs get sent?

That might be worse.

</comment></comment></comment><comment><USER54> Is it really a launch if very few people have access to it in a private beta?

All this noise on Twitter/FB/HN/etc. doesn't help when we don't get a chance to play around with it. Just saying.

</comment><comment><USER55> I thought I was signing up for Kite not for Kite's email list. Pretty misleading copy. Nowhere does it say "Coming soon" or anything to that effect.

Also, there seems to be no confirmation email.

<comment><USER4> Sorry if there was any confusion, but just FYI we're sending out the first invites today, so we're definitely "in motion".

</comment></comment><comment><USER56> I dream about a tool like this every time I need to look something up, which only happens about a hundred times every day. I NEED THIS IN MY LIFE.

But reading the kite.com/privacy doc is absolutely gutting. They copy and keep all your code, permanently. That's fine for an open-source project, but it's a deal breaker for anything else. So thanks for the brilliant idea, but I'll wait for it to be implemented in a way compatible with my everyday workflow.

</comment><comment><USER57> This is awesome! I was going to build something similar, just for the terminal, but extending it to code editors as well makes total sense. And what an all-star team!

</comment><comment><USER58> Nice tool, very nice demo video.

Unusable since I won't let my code be indexed by a third party. Also unusable since I won't let a third party have access to my shell. Imagine having your bash history stored centrally - a prime target for LEO/security services/black hats/data mining.

However - the code assistant is nice and I would love to have it running locally with a shared intelligent index - OSS based.

</comment><comment><USER24> Do you support code linting or code smells in addition to completions?

</comment><comment><USER59> Wow, this looks pretty amazing - well done with the demo. I'm super excited to try it out, how soon will it be ready?

<comment><USER17> We have some folks outside of the company using Kite already, and will be scaling that up as quickly as we can!

</comment></comment><comment><USER60> So, this looks pretty damn awesome, and I want to try it out yesterday! Seems like a lot of people are worried about privacy, and I guess I understand that. But I want this for when I'm learning a new language, and I honestly don't much care what happens with that code.

Also I think people are way too worried about their code. It's vanishingly rare that _the code_ is where the value in a business is. "But if I had Google's source code, I could run my own search engine and put them out of business!" Really? Could you? Because last I checked, Google has more engineers working on search than any other company on the planet. The stuff they do next week is probably going to improve their code more than what you would do by yourself for the entire year. And that's just the code, there's far more to search than algorithms.

<comment><USER61> >Also I think people are way too worried about their code.

I don't think anyone in this thread is worried about _their_ code. The problem is that most of us work somewhere where we write code that isn't _ours_ , and as such continued employment is contingent on us following the policies of the company. This generally include not sharing proprietary code.

<comment><USER60> I don't know everyone in the thread, so I can't comment specifically to them. You're right that getting fired over using this thing would be suboptimal. :)

I'm still eager to try it out!

</comment></comment></comment><comment><USER62> I have long wished nodejs provides such helpful info, e.g. when I use the nodejs APIs I can get hints on the syntax and even some concise demo for usage, instead of opening a webpage and read online each time. Both python and PHP provide man pages for all their APIs etc, but no such thing for nodejs yet.

</comment><comment><USER63> This looks fantastic. It would be great to get this working in Light Table.

Another commenter asked about hotkeys and I'd like to pile on to that sentiment – I've recently started using Vim (and Vimium) and it's really disappointing not being able to navigate by keyboard.

</comment><comment><USER64> I think this looks great, but it reminded me of Lightable. Also rather than using Sublime text as a comparison, isn't PyCharm a better comparison. In my opinion Pycharm already provides a subset of functionality, in addition to several other features.

<comment><USER63> [https://github.com/LightTable/LightTable/issues/2188](https://github.com/LightTable/LightTable/issues/2188)

</comment></comment><comment><USER65> Great video :-) Loved the left pad joke and duly noted down your panama account information.

</comment><comment><USER66> Autocomplete on steroids. This looks very cool and the demo vid was very well done!

</comment><comment><USER67> Ok, this convinced me. At first I was skeptical, but looking at how it works compared to existing methods that need a ton of configuration, I believe it is at least worth a try.

More importantly, I believe it appeals to newcomers. We are getting into a world where learning the proper way of coding requires a lot more than the famous 10.000 hours, mostly because there are hundreds of standards. This tool might become a much nicer introduction for newbies, compared to digging through pages of documentations in search for the one example that satisfy your use-case (I'm talking to you, Python requests).

</comment><comment><USER68> Looks great!

The only thing is I don't like the thought of having to share my screen real estate. Would it be possible to have some kind of navigable popup box within the editor?

<comment><USER4> Yeah this is something we're still iterating on. Though it wasn't in the demo video, there is actually a global keyboard shortcut to show/hide the sidebar without defocussing your editor. We're going to be experimenting more.

</comment></comment><comment><USER69> Looks like an amazing tool. Have to use it before i can tell more. Showing the documentation and example is cool and all, a very useful feature would be the search and analysis of the error messages upon program compilation/run. Most likely from stackOverflow.

This will make things so much easier. Reminds me of how2 [https://github.com/santinic/how2](https://github.com/santinic/how2)

</comment><comment><USER70> Great idea! But doesn't it work by intercepting keystrokes? What prevents Kite from reading keystrokes in other applications like a browser?

</comment><comment><USER71> Super cool concept. One thought: when I'm using my laptop I have limited screen space. It would be awesome to be able to use Kite on my phone's screen, with my phone clipped to my laptop using something like this: [https://tenonedesign.com/mountie.php](https://tenonedesign.com/mountie.php)

</comment><comment><USER72> I don't understand why this needs to store all the typed code on their server.

Couldn't this data just be deleted once the lookup has run?

<comment><USER63> I imagine they're using all the code they get to update their 'what is popular' examples for all the subsequent lookups; or that's what they want to be able to do.

</comment></comment><comment><USER73> left_pad.py - <USER2>

</comment><comment><USER74> This is amazing. Documentation pages that list methods alphabetically, or by some other random order, drive me crazy! Just being able to show me function calls by frequency and relevance would make Kite worth it. And there's so much more! Very excited about this.

</comment><comment><USER75> This is really cool. It reminds me of OmniSharp which provides Intellisense to any IDE. It just runs as a background process listening on HTTP so adding intelligent autocompletion to a text editor is fairly easy. I like this approach a lot.

</comment><comment><USER76> This is nice. Could we get a general description of what tech it is built on top of?

</comment><comment><USER77> For the early sign up it be cool if you could specify that you write in multiple languages and across multiple platforms to get invites as soon as possible. This looks like an amazing tool for a more hands off teaching style.

</comment><comment><USER78> What about support for VS Code?

<comment><USER79> Yes hopefully there will be integration for VS Code and Visual Studio. Though Microsoft already has a similar plugin, but this is purdier.

</comment></comment><comment><USER80> Many organizations already upload their source code to GitHub. I could get behind a tool like this that could scan GitHub repositories and build an on- premise index.

</comment><comment><USER81> Could I suggest some kind of per directory .kiteignore file?

</comment><comment><USER82> Who wants to build an open-source cross-platform clone ?

</comment><comment><USER83> The juxtaposition between seeing this on the frontpage today, and the VIM 8.0 announcement on the frontpage yesterday, is hilarious.

</comment><comment><USER84> What's with the invite? Is this a service or something? What about this requires it to be proprietary and hosted elsewhere?

</comment><comment><USER85> Justin kan mentioned the upcoming launch of this on snapchat the other day and I was intrigued. This does look awesome!

</comment><comment><USER86> Maybe I'm behind the curve, but I still use TextMate, is there any chance a plug-in is in the works for it?

</comment><comment><USER87> That's great, very exciting to have it internet based and to have it become smarter and smarter over time.

</comment><comment><USER88> Does it search Stack overflow along?

</comment><comment><USER89> Hopefully there will be _a bit_ less traffic on Stackoverflow after using that. :-)

</comment><comment><USER90> This looks great - can't wait to try it. Can I beg for an invite here?

</comment><comment><USER91> _No nerds were harmed in the making of this product._

So no questions asked on IRC?

</comment><comment><USER92> At age 13, I completed my PhD at Google while having an internship at MIT, Stanford, and Berkeley. By age 15 I won the Nobel Prize while working at my tenth SV startup. My name is Franklin, and I drink wine on the weekends.

</comment><comment><USER93> Very interesting. But closed. Not open.

</comment><comment><USER94> Loved the humor in the intro video :D

</comment><comment><USER95> This isn't that big a deal IMO. For a start, any non-trivial complex problem will probably not be included in kite, and the trivial stuff is generally memorised or easily googled.

This is really only spoon feeding.

<comment><USER63> I guess it depends on what a 'big deal' is exactly, but if it saves me all the Googling I do now then it'll be _really useful_ (to me).

I don't expect them to really provide an _intelligent_ artificial pair programmer, so you're right that this won't help with "any non-trivial complex problem", but that's okay with me.

I don't mind being spoon fed stuff I'm not interested in acquiring 'the hard way'. Do you also disdain autocompletion in your code editors?

<comment><USER95> Funnily enough, I've never used auto-completion. I've even gone through the trouble of deactivating it in Atom, but disdain is a strong word.

Also, it just occurred to me that Kite is basically a keylogger. Every keystroke is sent to their servers. That requires a lot of trust.

<comment><USER96> > Funnily enough, I've never used auto-completion. I've even gone through the > trouble of deactivating it in Atom,

Super 1337 bro

</comment></comment></comment></comment><comment><USER97> I hope it supports TextMate...

</comment><comment><USER14> I'm very impressed!

</comment><comment><USER98> awesome. intellisense on steroids.

</comment><comment><USER99> He should've put left_pad.py in a folder with an __init__.py to make it a proper package ;)

</comment><comment><USER100> This is really cool, and I wish you luck. It has great potential for educational purposes as well! I'd like to download it and try it when it's launched for my personal projects; however, I won't be able to use this at work (no way I'd get buy in to upload our source code lol). It'd be great to see local indexing on the roadmap for the future.

</comment><comment><USER101> This would be even more amazing if it was free or cheap.

It would be cool if they also open sourced it

<comment><USER40> What we be really cool is if you wrote an open source version. I'm sure if you started it, you'd find people to help once you reach minimal functionality.

</comment></comment><comment><USER102> This kind of tool destroys ones ability to program long sustainable production code. For a novice programmer this has tremendous negative effect on the learning curve. For an experienced programmer this tool is useless, because an experienced programmer will __NEVER __rely on "popularity" of some code- snippet out there in the wild. Programming is a very intense and deep practice and it is certainly not crafted using this kind of tools. This tool helps people write poor quality code for customers. Makes me wonder, what Knuth would say on this?

<comment><USER103> IntelliJ already has most of what this tool offers (but for java as opposed to python). It has auto-complete, quick access to documentation, quick fix suggestions (such as missing imports, etc.) and many many others not available in Kite at the moment. You wouldn't say IntelliJ "destroys ones ability to program long sustainable production code" would you? Because if you would, you'd be absolutely wrong.

<comment><USER102> IntelliJ certainly doesn't do this. IntelliJ and Kite have functionalities in common, which are fine. The other parts of Kite that IntelliJ hasn't, are problematic.</comment></comment></comment>


When Bloom filters don't bloom - <AUTHOR> https://blog.cloudflare.com/when-bloom-filters-dont-bloom/ <comment><USER2> In the final program `mmuniq` I did a couple of, I think, interesting hacks.

[https://github.com/cloudflare/cloudflare- blog/blob/master/20...](https://github.com/cloudflare/cloudflare- blog/blob/master/2020-02-mmuniq/mmuniq.c)

First, I used a hash function using aesni (aesenc) instruction set. See this:

[https://gist.github.com/majek/96dd615ed6c8aa64f60aac14e3f6ab...](https://gist.github.com/majek/96dd615ed6c8aa64f60aac14e3f6ab5a)

While I have little proof it's a good hash, it seems enough, and is _slightly_ (5-10%?) faster than siphash24 in this context.

Then I mixed counting hash with finding new lines \n. This allows me to do only one user-data load into XMM registers.

Most importantly, to offset the RAM latency cost, I'm doing 64 prefetches as I parse the input, and only after this I actually touch the hash table. The memory latency is still the biggest time sync, but at least this seem to speed up the program 2x or more. Hash table without this batching+prefetch is 6-8 seconds. With batching goes down below 3s.

I suspect linear probing / open addressing of the hash table may have some penalty. While it plays nicely with the cache prefetch, it generally leads to longer chains. This means we need to keep the hash table sparse, not loaded above 0.6-0.75. See this

[https://en.wikipedia.org/wiki/File:Hash_table_average_insert...](https://en.wikipedia.org/wiki/File:Hash_table_average_insertion_time.png)

from [https://en.wikipedia.org/wiki/Hash_table](https://en.wikipedia.org/wiki/Hash_table)

<comment><USER3> I'm a bit confused, you are now storing the IPv4 addresses in a hash table using a 64-bit hash?

Why not just use the 32-bit address as a key, and grow the 'blocks' so if two addresses are just a couple of digits apart, promote it to a /24 block etc.

<comment><USER2> Apologies, maybe I oversimplified the original problem. I'm dealing with IP's (both v4 and v6), subnets, ranges (which may or may not align to subnets). These map to one or more datacenter numbers.

I could indeed define data model, parse the data thoroughly, optimize in- memory data structure, and so on. That requires rigid data structure, knowing access pattern and understanding the problem space. I'm not there yet. Instead, I created this generic tool which works with any text files, and fell into a rabbit hole of over-optimizing it. That's it.

<comment><USER4> FWIW, you should be able to represent individual IPs, ranges, and subsets all in CIDR notation, tho for ranges you may need multiple CIRD entries to reflect the whole range.

CIDR for ipv4 consists of the 32 bit address and a 32 bit mask, so with some bit packing you can uniquely represent them in 64 bits without hashing.

The problem you'll run into there is doing a "contains" check on an origin IP for a list of CIDRs, but you'll need to do that currently since you're dealing with subnets, I assume.

<comment><USER5> 32 bit mask is _way_ too generous, you only need 5 bit masklen. It all doesn't matter though since they have v6 addresses and ranges.

<comment><USER3> Would save a lot of space to just have separate lists for each mask length etc.

</comment></comment></comment></comment></comment></comment><comment><USER6> A blocked bloom filter works on cache blocks <[https://www.tutorialspoint.com/blocked-bloom- filter>](https://www.tutorialspoint.com/blocked-bloom-filter>). It takes more memory but far fewer hits to RAM, so much better caching behaviour. It should solve your problem.

Edit: it will use more RAM than cuckoo filters.

<comment><USER7> FYI, that closing angle bracket breaks the link.

<comment><USER8> Working link: [https://www.tutorialspoint.com/blocked-bloom- filter](https://www.tutorialspoint.com/blocked-bloom-filter)

</comment></comment></comment><comment><USER9> >time (cat logs.txt | sort | uniq > /dev/null)

If I skip two useless pipes, and use "sort -u logs.txt > /dev/null" instead, I'm already twice as fast as original (it seems that piping to sort effectively prevents parallelization).

<comment><USER10> For me sort|uniq is _nine_ times slower than

      LANG=C sort -u

but `sort -u` on its own is only marginally faster than sort|uniq.

I can't remember exactly what LANG=C does, but I think the it makes sort not need to do some fancy unicode stuff? If the person writing the article just needs to uniqify IP addresses they should use it.

<comment><USER2> Oh boy. This is drastic. Thanks for great advice:

      marek:~$ time (cat logs-popcount-org.txt | sort -u | wc -l)
      39057531

      real 2m37.387s
      user 2m35.626s
      sys 0m2.937s

      marek:~$ time (cat logs-popcount-org.txt | LANG=C sort -u -S6G | wc -l)
      39057531

      real 0m12.908s
      user 0m42.826s
      sys 0m3.586s

</comment></comment></comment><comment><USER11> Even though I agree with another commenter that it's surprising that the author used a bloom filter instead of the hash map as a baseline, this article still is an excellent small walkthrough of quick ad-hoc low-level performance profiling.

</comment><comment><USER12> I went down the Bloom Filter rabbit hole for a project a while back and then wondered if I could actually fit the entire set space in memory. The set was IDs up to 2^32, so basically a giant bit array. I believe the IPV4 universe of this article is actually the same size as in my project. I coded up my project using Java bit arrays and got things working decently well, using a big heap. Then I found out there are a bunch of _compressed_ bit array libraries such as EWAH and Roaring Bitmap. When I substituted Roaring for the stock Java Bitset implementation, I saw space and computation improve by many orders of magnitude. Roaring uses a bunch of tricks to achieve this, but it mostly comes down to encoding large runs of 1s or 0s. Obviously, not as tiny as bloom filters, but still pretty small for most modern machines if you have a sparse set. [https://www.roaringbitmap.org/](https://www.roaringbitmap.org/)

<comment><USER13> And it's not probabilistic, it gives strict answer whether an element is in the set or not.

</comment></comment><comment><USER14> Bloom filters are ideal candidates for answering the question 'Is x _not_ in the set?'. If the answer is yes, nothing, including the item you're asking about, ever hashed to that location. If the answer is no, all you know is that something hashed there, possibly your item or not.

<comment><USER15> Which works perfectly for this case: if the answer is yes, add it to output, otherwise skip.

<comment><USER16> It's not that simple. If the correctness of your program relies on the "not in set" test being accurate, you're going to need to make the filter huge, and slow.

Probabilistic data structures are about trading off correctness and performance. If you try to push the correctness up to near perfect, they'll quickly stop making sense and you should just use an actual perfect algorithm instead, as the author did.

Bloom filters are great for early outs, where you can save a chunk of computation on a definite negative, but still be correct in case of false positive.

<comment><USER15> Not sure I follow. For detecting uniques all you care about is the definite negatives? Either way, the author addresses precision in his post.

<comment><USER16> False positives mean you're throwing away genuinely unique items that hash the same. If you use a 256 bit Bloom filter to process a billion items, you'll get exactly 256 results that are indeed unique.

The rate of false positives you require out of the data structure is key. If your program is correct with a 20% false positive rate, you're golden. If the goal is more or less 0, look elsewhere.

The author addresses precision, but not in a way that questions whether a Bloom filter is indeed the right tool for the job.

<comment><USER15> But you don't care about false positives, only the definite negatives! I was trying to address the fact that the top comment on the utility of a bloom filter is covered in the post.

<comment><USER16> If you don't care about the false positives, you don't need a Bloom filter. Just reject everything.

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> I'm not sure why the author started with bloom filters instead of a hash table, to be honest. The workload seems ideal for a hash table. Interesting to see how big the performance gap was though, I wouldn't have expected such a difference. It probably comes down to the fact that the bloom filter has to spread its data across many locations so if the set is particularly large it's always going to lose to a hash table due to hitting main memory more times (in scenarios where you can use either one, at least).

I suppose they started with a bloom filter because they intended to use one when consuming the data, i.e. checking incoming requests against a 'malicious IP' set?

<comment><USER18> I don't know why the author didn't stick to their shell script, which took all of 2 minutes for a manual data cleanup step. Like, go grab a coffee.

<comment><USER2> 2 minutes was for 40M items. I had 1B items to sort/uniq.

</comment></comment></comment><comment><USER19> > For example, source IPs belonging to a legitimate Italian ISP should not > arrive in a Brazilian datacenter.

This is an assumption based on a very naive understanding how packets get delivered on the Internet. I for one wouldn't enjoy being blocked from CloudFlare sites just because of poor routing or peering.

[https://en.wikipedia.org/wiki/Hot-potato_and_cold- potato_rou...](https://en.wikipedia.org/wiki/Hot-potato_and_cold- potato_routing)

<comment><USER20> Do you understand how rude you're being? This is an article about Bloom filters, not Internet routing.

</comment></comment><comment><USER21> Why not just use an array of 2^32 bits -- a half gigabyte -- and leave off hashing altogether?

All it would cost is the excess runtime, which we should not mind giving up unless we smoke.

If necessary, you could have two or more. 256 of them would fit in 128G, which lots of servers have without even needing it all.

<comment><USER22> 2^32 == 4GiB

<comment><USER23> 2^32 bytes is 4 GiB, but 2^32 bits is 512 MiB. In the GP's hypothetical array, I believe you would index (array[idx >> 3] >> (idx & 7)) != 0

</comment></comment></comment><comment><USER24> > Check out this excellent visualization by Thomas Hurst showing how > parameters influence each other:

> [https://hur.st/bloomfilter/](https://hur.st/bloomfilter/)

 _Blush_. However will I cope with all this extra traffic?

One common way to improve bloom filter cache performance is to divide them into blocks - elsewhere is mentioned doing it to the cache line, but it would be interesting to see how much performance would be gained with a more naive approach, for instance, splitting the filter into 4KiB pages.

I've done this for disk-backed filters, but never looked to see if it improved performance generally.

</comment><comment><USER25> Another solution is to change the order things are checked in. With optimum k, each hash takes about 1/2 of the possibilities out. If he'd batch it so the memory accesses are roughly ordered, it would be much faster.

EDIT:

So, for instance, if k = 19, that means there are 19 hash functions. If he goes through all the inputs, and checks: are there any hashes here falling within the first 1/19 of the memory space? If not, keep it. If so, check whether any are unset. If not, keep it. If they are, zero the pointer to the input in the array. After this is done, he should be rid of roughly 32% (0.5 * (1-(18/19)^19)) of candidates. The second pass throws out 33% of candidates, and so on and so forth.

He could even keep an absurdly large value for _k_ and _n_ : if it is 128G, then k = 19'053, meaning he can use an even finer increment. He'd have to spill the filter to disk, but the access patterns will be great.

</comment><comment><USER26> seems like it would be worth comparing to the old "awk '!a[$0] { a[$0]=1; print }'". I would assume that such arrays are implemented internally using hash tables. probably not as efficient as a C implementation, but the used parts of the interpreter should fit in I-cache, so it should be within a few times as fast.

<comment><USER2> Here you go:

      marek:~$ time (cat logs-popcount-org.txt | awk '!a[$0] {a[$0]=1; print }'|wc -l)
      39057531

      real 0m41.236s
      user 0m38.179s
      sys 0m5.447s

So: sort: 2m, awk 41 seconds. Also, awk used 6.1G of RAM at peak.

<comment><USER26> yeah, a generic, probably pointer-heavy hash table is definitely gonna be worse on memory. I'm surprised that it's _that_ much worse on time though, I expected it to be closer. I guess probably the cache misses are worse with such a large table though.

<comment><USER2> Ok, I'll bite again:

      marek:~$ cat logs-popcount-org.txt | perf stat -d awk '!a[$0] { a[$0]=1; print }' > /dev/null

       Performance counter stats for 'awk !a[$0] { a[$0]=1; print }':

               40,318.47 msec task-clock:u
                       0      context-switches:u
                       0      cpu-migrations:u
               1,670,649      page-faults:u
         112,979,634,215      cycles:u
          93,441,976,758      instructions:u
          18,990,099,679      branches:u
             208,386,137      branch-misses:u
          26,093,832,363      L1-dcache-loads:u
             708,880,979      L1-dcache-load-misses:u
             464,332,790      LLC-loads:u
             245,913,835      LLC-load-misses:u

            40.337768657 seconds time elapsed

            36.851718000 seconds user
             3.468126000 seconds sys

Compare this to the optimized approach which has 57M LLC-load-misses, and 7M instructions.

<comment><USER21> I would welcome seeing a comparison in your environment to using the simple 1/2 GB array of bits, with no hashing or storage of IP addresses. (Extra points for hugetlb mapping.)

</comment></comment></comment></comment></comment><comment><USER27> The latency problem must be due to (unnecessary data) dependencies and not taking advantage of superscalar architecture. Modern CPUs support 32 or more in-flight memory operations.

After hiding latency, the next bottleneck would be instructions so vector scatter/gather can alleviate this problem.

<comment><USER28> 32 per core? Most modern Intel only supports 10 or 12 outstanding (line) accesses per core. AMD is a bit better. Ice Lake is significantly better.

<comment><USER27> I think it's per memory controller. Probably per socket.

</comment></comment></comment><comment><USER29> On a bit unrelated note, if you want to handle very large Bloom filters (billions of entries with low false positive rates) there is an open source Java library that can help you to do that: [https://github.com/nixer-io/nixer- spring-plugin/tree/master/...](https://github.com/nixer-io/nixer-spring- plugin/tree/master/bloom-tool).

There is also a command line utility that accompanies the library: [https://github.com/nixer-io/nixer-spring- plugin/tree/master/...](https://github.com/nixer-io/nixer-spring- plugin/tree/master/bloom-tool).

</comment><comment><USER30> Another potentially useful data structure that has good asymptotic complexity, but probably also poor cache locality is the Van Emde Boas tree [0]. I've never seen one in practice, but they sure make for excellent p-set problems!

[0] [https://en.wikipedia.org/wiki/Van_Emde_Boas_tree](https://en.wikipedia.org/wiki/Van_Emde_Boas_tree)

</comment><comment><USER31> There is a very interesting talk [1] by Chandler Carruth that I stumbled upon last weekend that is very much related to this

[1] [https://www.youtube.com/watch?v=nXaxk27zwlk&t=4678s](https://www.youtube.com/watch?v=nXaxk27zwlk&t=4678s)

</comment><comment> mark-r The generic sort command is easy, but in this case maybe a custom radix sort would have been faster?

</comment><comment><USER32> Relevant, maybe: [https://medium.com/adobetech/filtering-duplicates-on-the- com...](https://medium.com/adobetech/filtering-duplicates-on-the-command- line-30x-faster-than-sort-uniq-96ca5f7b4277)

</comment><comment><USER33> I am surprised, isn't cloudflare a Go shop?

<comment><AUTHOR> It would be ridiculous to do everything in one language. Different tools for different situations. We use Go, Rust, C++, Python, Lua, JavaScript, ...</comment></comment>