Legit-Looking iPhone Lightning Cables Will Hijack Your Computer - <AUTHOR> https://www.vice.com/en_us/article/evj4qw/these-iphone-lightning-cables-will-hack-your-computer <comment><USER2> Reposting from 4 months ago ([https://news.ycombinator.com/item?id=19609745](https://news.ycombinator.com/item?id=19609745)):

It's a severe discredit to the major operating system vendors that plugging in a USB stick can still compromise a system.

If a USB device identifies itself as a keyboard, the system shouldn't accept its keystrokes until either that keyboard has typed the user's login password, or the user uses a different input device to authorize it. If it identifies itself as a storage device, the filesystem driver should be hardened. If it identifies itself as an obscure 90s printer with a buggy driver written in C, it should prompt the user to confirm the device type before it loads the driver.

It's 2019. Why the f* haven't Windows, MacOS and Linux all implemented these basic precautions?

<comment><USER3> > _If a USB device identifies itself as a keyboard, the system shouldn 't > accept its keystrokes until either that keyboard has typed the user's login > password, or the user uses a different input device to authorize it._

An interesting solution, it would definitely prompt the user to understand what the device is trying to do.

But I'm sure that it's extremely hard to prevent something malicious, once it has physical access to a port on your computer...

<comment><USER4> "An interesting solution, it would definitely prompt the user to understand what the device is trying to do."

But then it could simply wait for the user to enter the password (1), then read it by sniffing the traffic from the keyboard and store it internally for later use, since it's all in clear ad it cannot be encrypted before entering the machine unless most (all) USB consumer hardware get some heavy modifications.

1- very simple algorithm: store in the internal flash memory whatever the user enters between connecting the keyboard and hitting the 2nd enter key; if it's mostly the same words, then it's very likely an user/password pair.

"But I'm sure that it's extremely hard to prevent something malicious, once it has physical access to a port on your computer... "

Very true. Malicious plug in hardware was just a matter of time; we badly need some active protection for these things, or it would be a mess. This is the perfect weapon in the hands of people with a thing for vandalism, I hope mainstream media _won 't_ cover that story.

<comment><USER2> You're assuming the malicious device is a keyboard, or is on the signal path between the motherboard and a keyboard. That's not the common case, nor is it the case here. No one types their password into an iPhone cable, because the cable has no keys to type with.

</comment></comment></comment></comment><comment><USER5> > _One idea is to take this malicious tool, dubbed O.MG Cable, and swap it for > a target 's legitimate one. MG suggested you may even give the malicious > version as a gift to the target_

Even more frightening, people selling them as seemingly legitimate cables on Amazon? People will pay you and you get a new botnet.

How many could you sell before it's discovered?

How can I, as a consumer, even tell? Amazon will even allow you to sell your malcable under the Apple brand.

<comment><USER6> Your attack would need to be targeted since you can't connect to your cable over-the-internet, only over the wifi interface, limiting you to that range.

<comment><USER7> If you were mailing the cables to random people, you wouldn't use wifi, it's true. You'd just want the fake keyboard to just use a terminal to download and install a trojan.

If you can fire off a successful "curl | bash" on an internet-connected machine, wireless isn't needed.

Of course, without wifi you've only got a USB Rubber Ducky clone [1] whereas with wifi, you've got an NSA COTTONMOUTH clone [2] which I imagine is much more likely to get your talk accepted at DEFCON :)

[1] [https://shop.hak5.org/collections/physical- access/products/u...](https://shop.hak5.org/collections/physical- access/products/usb-rubber-ducky-deluxe) [2] [https://en.wikipedia.org/wiki/NSA_ANT_catalog](https://en.wikipedia.org/wiki/NSA_ANT_catalog)

<comment><USER6> At that point, just pre-load the cable with a flash drive and copy the malware onboard.

<comment><USER8> Seeing now why counterfeits are a _serious_ problem for resellers!

</comment></comment></comment></comment></comment><comment><USER9> Really need a setting "never trust any device ever". I've never once had a use case with my phone to do anything but charge. Really hate when I plug in my phone to charge in a car and the car takes over my UI. All bad ideas. If I want to move photos I use the network.

<comment><USER10> This device shows up as a keyboard - should keyboards never be trusted ever? How would that work?

<comment><USER11> It would be nice of my OS had an option to disallow any and all USB devices. Plug something in? Ask whether I want to allow it. I guess this would get annoying after a bit. But still, I only use a couple of USB devices on a daily basis, but I click on boatloads of cookie warnings every day.

<comment><USER12> > _It would be nice of my OS had an option to disallow any and all USB > devices_

Any desktop computer would have to be redesigned to add a "allow new device" button since they have no other input.

Even on many laptops, the internal keyboard and mouse are USB devices, when you install a new OS, do you have to accept trust to those as well? Or how will you stop an external device from spoofing them with the same vendor/device ID?

<comment><USER13> How about, trusted peripherals should speak DTLS over their USB/Thunderbolt PHY, and the OS should keep a certificate store for recognizing them?

This sounds like something that creates a chicken-and-egg problem of there not already being any such DTLS-speaking USB devices... but how about if vendors just create a little USB dongle that wraps whatever's plugged into it in "authentication" using DTLS? Ship the dongle with the laptop; tell people that if they want to install a new OS, they have to plug a USB keyboard in through the dongle.

<comment><USER14> > This sounds like something that creates a chicken-and-egg problem of there > not already being any such DTLS-speaking USB devices...

Or only allow completely unauthenticated devices as a fallback when there is _no_ other available authenticated device.

A computer not having any keyboard is a rare case. Most of the time you have what is built-in (and should be authenticated) or what came with the computer (and should be authenticated).

Allowing unauthenticated keyboards only on detection of _no_ authenticated ones probably covers 99.9% of all use cases and increases security dramatically.

<comment><USER15> Aren't we trying to prevent an attacker with physical access? They could simply unplug everything first.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> I've not found many details about how this is actually working — there's some info on his D̴̹̭͂ë̷̗́̃̿̓̾͜ṃ̸͔͚̗̙̪̎̄̋ȏ̸̝̤̱͜n̶͇͇͙̻̩͑͑S̴̳̩̮̥͚̥̚ė̸̟̃͋͂͝e̷̪̲̪̰̣̿̀͠d̵̡̂͗ cable here [1], but apparently the O.MG cable is "a very different piece of hardware that does a whole lot more."

Does anyone have any insight into how this attack works? My guess is that it acts like a hub that exposes both the iPhone lightning connector and a keyboard/mouse. And then the keyboard/mouse is controllable via some near- range wireless like WiFi or bluetooth? I suppose it could even scan for open networks and try to join to allow a more remote exploit. Anyone find more information anywhere?

1\. [https://github.com/O-MG/DemonSeed](https://github.com/O-MG/DemonSeed)

<comment><USER17> I'm guessing the cable has an esp8266 on board which you can get cheaply and is only a few mm2. It has WiFi and WiFi direct support and is powerful enough to run a webserver. Probably there are plenty chips that do the job, but the esp8266 (and its successor esp32) is very popular for custom hardware due to being cheap and easy to program

<comment><USER18> How would the board in the cable know what WiFi to connect to, and how would it run shell commands via USB?

<comment><USER19> If you know the wifi environment of your intended victim, you could pre- configure it with the network in question.

</comment></comment></comment></comment><comment><USER20> I have this USB-C looking like this[0] (not the same one though). The thing is whenever it is plugged into my MacBook Pro, the hub starts to overheat, even when there is nothing connected to it. I once tried plugging it into the MBP adapter and charging my phone through the USB port on it, and it did not heat at all.

I am suspecting it is running some program in the background (a miner maybe). Is there a way I can check if such a program is running?

[0] [https://www.amazon.com/Purgo-Adapter-2018-2016-Delivery- Thun...](https://www.amazon.com/Purgo-Adapter-2018-2016-Delivery- Thunderbolt/dp/B07K5ZR6HS/ref=sr_1_17?keywords=usb+c+hub&qid=1565706339&s=gateway&sr=8-17)

<comment><USER21> You could install Little Snitch on your Mac to see if it phones home.

IMO more likely that it's shoddy hardware; either way it's munching your battery, so I'd send it to the recyclers and find something more reputable.

</comment></comment><comment><USER22> > Now MG wants to get the cables produced as a legitimate security tool

Can someone explain how these could be considered a "legitimate" security tool? What legitimate use would require the cable to look like a genuine Apple cable? (I'm honestly asking.)

<comment><USER23> onsite pentesting for example. You want to train your employees to be aware of random cables and usb drives laying around, this is a good test to ensure your training worked.

</comment></comment><comment><USER24> Ask your dentist to take an x-ray of the cable you may be concerned about. We're all digital and it only takes a second. If your guy is cool, he'll do it.

<comment><USER25> Where do you find these "cool doctors"? I once tried to bribe six different doctors in my area with $3000 in exchange for agreeing to allow me to get an exploratory MRI and they all said no.

<comment><USER26> An exploratory MRI has inherit risk (even if minuscule) to your life, and thus their livelyhood. A very big difference from asking for an x-ray of a cable.

<comment><USER27> > _An exploratory MRI has inherit risk (even if minuscule) to your life_

As far as I've ever heard, an MRI without contrast has no risk itself, and any risk comes from acting on the data.

<comment><USER28> That is not true. It doesn't involve ionizing radiation, so not a dose risk like CT. But look up PNS and SAR (peripheral nerve stimulation and specific absorption rate), for example. This is mostly handled well for standard pulse sequences of course, but not "zero risk".

Beyond that, there is a reliance that you do not have any implants etc., even some tattoos. And you tell the truth about it. From the clinics point of view too risky.

<comment><USER29> I don't think that's what they're especially worried about; those are fairly minor.

Instead, think about interacting with someone who a) is so convinced that they need an exploratory MRI but b) can't convince a doctor of that need. I'd be afraid that either I'll be stuck dealing with someone perseverating over a totally normal anatomical variation (and everyone has a few). If they get sick later, I might also get dragged into a debate over whether I should have noticed something on that scan, done a different scan, or whatever, possibly with big legal implications.

This is why our techs will happily scan a fruit or something, but don't run an ad-hoc clinic.

<comment><USER28> I agree that in the parent comment case, there is no reason to risk a review or lawsuit which is probably mainly why a clinician wouldn't do it; I alluded that that in another comment.

This one was specifically a comment about "zero risk" on MRI, it's not true. Low risk, sure. But people have been hurt.

I also suspect any clinician is going to look askance at a low risk action that isn't necessary, but the potential liability is the kicker here.

<comment><USER29> It's pretty close.

Nothing is totally risk free, but compared to most medical procedures--and most activities of daily living--MRIs are a walk in the park. For a subject with no implanted devices, I would bet the drive to the scan center is _much_ more dangerous. I just flipped through MAUDE ([https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/d...](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/detail.cfm?mdrfoi__id=8367269&pc=LNH)) and I couldn't find any adverse events that were more severe than a small burn or blister.

<comment><USER28> Agree it's low risk, I was being pedantic.

There have been deaths of course, also, but not due to normal operation.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER30> Get a USB condom! [https://shop.syncstop.com/products/usb- condom?variant=354300...](https://shop.syncstop.com/products/usb- condom?variant=35430087052)

<comment><USER31> How do you know you can trust that? Couldn't it be doing the same thing as the fake lightning cable?

<comment><USER32> They block (or just dont have) the USB data pins - so all it can do is draw power.

</comment></comment></comment><comment><USER33> Is there a piece of cheap validation hardware where you can plug in both ends of your cable and a little display will tell you what kind of cable it is and if it is legitimate?

<comment><USER34> Maybe just have something like the USB killer [1] to "sterilize" cables. Zap the cable with a high voltage/high energy pulse, beyond what normal on-die ESD protection could handle. A bunch of copper and plastic won't get damaged (unless you really get crazy and it arcs over and carbonizes or something), but it will probably burn out any covert semiconductors in the cable. It's hard to absorb high energy pulses in small packages.

[1] [https://usbkill.com/](https://usbkill.com/)

<comment><USER35> Type-C cables have an E-Marker IC inside them by design, so this would probably render the cable non-functional or worse-functional unfortunately.

</comment></comment></comment><comment><USER36> > up to 300 feet

which means 50 feet, which is still impressive in that it's a useful distance. I remember the earlier version being more like 5 feet, which sounds pitiful but is still enough. In fact no wifi at all (0 ft) is enough to plant software (CMD-space Terminal RET curl | bash && exit) if you take your chances that the target is inattentive.

I learned of the earlier version here on HN but I can't find the link now. It was maybe 4 months ago?

Given that the attack is that it's a USB keyboard, nothing to do with the lightning aspect, except that the victim is likely to need a lightning cable at some point, any USB dongle will do.

Given the attack methodology for this specific device, of being in visual distance of the victim, just use an unpaired apple keyboard. Macs will automatically pair to them, so you just need to turn it on when the victim looks away (a brief 2-second overlay appears on the screen upon connecting). You could force this by creating a distraction: drop a glass. No dependence then on the victim using the cable.

</comment><comment><USER37> There's an interview with MG talking about these cables on the Amp Hour podcast this week [https://theamphour.com/the-amp-hour-454-mike- grover/](https://theamphour.com/the-amp-hour-454-mike-grover/)

</comment><comment><USER32> I saw Kevin Mitnik (FBIs most wanted hacker in the 1990s) at a conference plug one of these into laptop with a fully patched version of Windows 10 and one of the very common security suite of apps.

The laptop was completely compromised in seconds.

From a remote laptop, he had complete access to the target machines full filesystem, started the webcam and turned on the microphone without any notifications to the target user and connected a bluetooth hard drive remotely.

And this was using a rogue cable that he just bought off ebay.

I was honestly shocked at how easy it would be to compromise someones machine. I'll never look at a USB cable the same way.

</comment><comment><USER38> I still don't get why iPhones don't use USB-C. OOTB, there's issues between a brand new iPhone and a brand new MacBook Pro.

<comment><USER39> I think it's just an accident of timing and history.

The old 30-pin connector (inherited from the iPod) had various issues so I think Apple was eager to replace it. The lightning connector was their solution. It predates USB-C by a few years, so that wasn't an option at the time (I guess it might have been on Apple's radar by the time the lightning cable was introduced, but if so, they must have made the call not to wait.)

Since USB-C has made its way to some iPads, my guess is Apple is in the process of phasing out lightning connectors entirely.

<comment><AUTHOR> Instead I've heard that Apple essentially designed the USB-C standard after the successful design of Lightning, and gave the standard to USB-IF. (Apple is a member of the USB-IF.)

<comment><USER40> I've heard here from someone who claimed to have been involved that Apple's contributions were relatively late and minor, but I don't have any idea how to judge the accuracy of either assertion.

And it's possible both viewpoints have merit depending on what aspects of the standard are considered significant.

</comment></comment></comment></comment><comment><USER41> Any thoughts on how someone could validate a set of cables to ensure no trojans exist?

</comment><comment><USER42> I don't see how Hak5 can create exact replicas of Apple Lightning cables with hacking tools embedded in them and NOT have Apples dream litigation team blasting down their doors

It's pretty amazing how technology has gotten so small we can hide a wifi chip and keyboard emulator into the end of a USB port plug.

<comment><USER43> No, they are modified genuine cables.

<comment><USER44> The demo was a modified cable, but the article said Hak5 was interested in manufacturing them.</comment></comment></comment>


Stop Trying to Save the World - <AUTHOR> https://newrepublic.com/article/120178/problem-international-development-and-plan-fix-it <comment><USER2> Highly relevant book: Seeing Like a State.</comment>


Ask HN: what do you do with ideas you have no way of implementing yourself? - <AUTHOR>

Most likely, you have a lot of ideas all the time. Some are simple and you do something over a week-end, some are more complicated and you gather a couple of people, make a prototype and start a company, but some are just completely outside your reach.
It could be lack of time before an opportunity is closing, lack of capital (hard R&D necessary), lack of expertise… (though it all goes back to the same thing: time and money) but what do you do with these ideas that are too far out?
Find the right people? Put the idea out there and hope it gets picked up? Patent it and become a troll?
Also, do you have examples of things that you didn't know anything about but still pushed through to success?
Of course, one of the reasons I'm asking is that I've had a couple of ideas that I'd like to see come to life but don't quite have the means: one is a small hack for which the opportunity is likely to close before I could get to something by myself, and the other is a pretty big R&D thing that involves chemistry, optics and many other things that I know nothing about and can't just do from my couch. :)
So I'm curious about you do in these cases… <comment><USER2> Just keep an ear out for someone who can do something about it. A lot of good projects happen when a few people come together with similar ideas in their back pockets.

</comment><comment><USER3> I keep my eye out for people who are interested in starting a company and offer to fund them. It is interesting how rare it is to find anyone who really wants to start a company. A lot of people say they want to but when given the chance, wont take the leap.

</comment><comment><USER4> I tend to do one of two things: Keep it alive on some back burner forever and ever and ever while feeling like some loser who never accomplishes anything or do a quick google, discover someone else is already doing it and has put way more into it than I ever would and ditch it as a likely time-waster.

<comment><USER5> Just because someone else is doing it shouldn't be a reason to ditch it. In fact, it could validate your idea as it proves there is a market and if your idea is better and your vision greater you could have a success on your hands (just look at Wufoo for a recent example).

On the other hand if your enthusiasm for the idea isn't high enough then you should go ahead and ditch it.

<comment><USER4> <flake>

Yeah, well, the most recent thing I dreamed up as some possible money-maker was a numerology app to do the math part for you since I get annoyed by having to do it by hand, only it turns out there are already numerology apps and they have interpretation and stuff. Then I decided I probably didn't really _want_ to do a numerology app, I just had fantasies of making a quick buck because my finances have been very stressful of late. What I really want to do is what I have been working on all along only the "getting well part" that my personal expertise is rooted in is still taking too much effin time/money/effort and continues to delay the "make some cool simulation to teach folks what I did/my mental model so they have some shot at replicating it" part. The actual "getting well" part is critical to doing this because it is what my knowledge is rooted in and it's unique knowledge but, hey, it's really annoying to be sick for 45 years as a means to have a unique product. Bleh.

Then after that I plan on getting rich with a comic.

<everyone laughs>

</flake>

<comment><USER6> You're not stupid for creating something that already exists. I've "copied" sites before and have made decent money from them, even if the market was crowded or the idea was unoriginal. At the very least, its a learning experience.

The truth is your not going to come up with a brilliant idea and get rich, so STFU and start building something.

<comment><USER4> I think you have misunderstood me. I was poking fun at my belief in numerology, not suggesting I'm stupid or unoriginal. Most folks on boards like this one roll their eyes at things like numerology.

And I am working on stuff, though I am not at the "building something" (as in "writing code") stage. There is other groundwork that is being laid.

Take care.

<comment><USER6> Oh alright then, not sure I saw the self-deprecation as humor, but to each his own! Carry on good sir.

</comment></comment></comment></comment></comment></comment><comment><USER7> well, i think that a main aspect is definitely dedication. if you have to handle a full time job while doing this project, so be it it, you gotta live. but then every other second should be on this project. if that doesnt happen, then the project wont happen. another aspect, i think, is an awesome partner. going with PGs' philosophy of you almost always need a partner to succeed. but good luck =)

</comment><comment><USER8> Never give up, keep on trying, your aha moment will come.

<comment><USER9> I agree. Don't give up and try and bring anyone and everyone along for the ride. This doesn't mean make them a partner, but share your experience and don't be afraid to ask others for help. This will give you a network of other people with similar interests and yourself a reputation that will help down the lines. It will also give you a perspective on if you want to work with these people in the future or on your next project.

<comment><USER8> What's the best way to gain traction for new startup?

<comment><USER9> I think it really depends on your startup (SEO, advertising, bloggers, etc)

I saw this the other day: <http://news.ycombinator.com/item?id=2461271>

Also here is another great resource! [http://blog.traindom.com/places-where- to-submit-your-startup...](http://blog.traindom.com/places-where-to-submit- your-startup-for-coverage/)</comment></comment></comment></comment>


Share HN: When I first talked with PG - <AUTHOR>

October .... 1996. I was working as tech correspondent at The Economist and had managed to push the story of ecommerce being exciting not because of Amazon but because these guys at Viaweb were helping you set up stores yourself. So I am on the phone to PG going over this and that late one night--because of timezones. And I'm asking all these questions, trying to figure out how to get it across in five hundred words in The Economist. Really excited about revealing this cool company in The Economist. Next day, I am writing up the story and my assistant flags me down, interrupts again, and again 'It's urgent'. Turns out my dad is really ill (he's been fine ever since). Leave the story half written on my Atex terminal, off to the hospital, with a promise I'll call in to finish it, don't get back to my desk for a month and story never runs.
ViaWeb and PG a success nonetheless :) <comment><USER2> So... You did an interview with a microcelebrity, the story never ran, and you've been kicking yourself ever since?

Not sure this is news, dude.

<comment><AUTHOR> well hardly every since. the thing that prompted it was a HN post on how to get in touch with PG; and I have really strong memories of the Viaweb story, mostly around my dad's heart attack. Check here: <http://bit.ly/5e0XxJ>

Bear in mind, this was in the days well before blogging when getting any story out involved fighting for column cm with other journalist, and a (necessarily) sceptical business editor. I know, it seems antiquated now!

And I am not sure I said it was news. And yes, you may get semantic about this being news.ycombinator, but i'll leave you to do the detailed analysis of what qualifies since you have the media studies degree.

<comment><USER2> Ok, ok. =) And yes, I started as a print journalist, too. We worked in column inches, but still, I feel your pain.

</comment></comment></comment><comment><USER3> who fucking cares?

<comment><USER4> I found it interesting, for one. Teaches something about narrowly missed opportunities and their long-run effects.</comment></comment>


Verified accounts for everyone: Twitter announces new application process - <AUTHOR> http://thenextweb.com/insider/2016/07/19/verified-accounts-everyone-twitter-announces-new-application-process/ <comment><USER2> They also used it as a "weapon", removing verification from people who had unpopular views.</comment>


How much did AlphaGo Zero cost? (2018) - <AUTHOR> https://www.yuzeh.com/data/agz-cost.html <comment><USER2> Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish.

Lots of people contribute what I imagine are amounts of CPU Power/money to the Leela Chess Zero project[1].

Would love to see Alpha Chess vs Leela Chess.

[1] [https://training.lczero.org/](https://training.lczero.org/)

[edit] I've caused terrible confusion by melding Leela Go and Leela Chess when Leela Chess was originally forked from Leela Go and that's basically when similarities end.

Edited for a bit more clarity.

<comment><USER3> AFAIK Garry Kasparov to this day does computer&human vs. computer&human chess research, and it's far from a solved problem.

<comment><USER4> Is human&computer better than computer only?

<comment><USER5> Not really, humans barely provide insight (if anything), which chess engines don't already consider. Deep Blue could evaluate 200 million different moves... per second. And that's from 1997.

The few and rare times an engine gets funky is usually in end-game positions where the engine can't seem to find a sacrifice to win the game and will output a current position as drawn. These cases are few and I very much doubt that a human would be able to find these moves in an actual match.

Now if you're talking about the way the chess engine learns, it can learn in two different ways: without human help (learning completely on its own giving it nothing but the rules which is how AlphaGo works), or with human aid (through chess theory accumulated over centuries of human matches that these engines have built in as part of their evaluations). Things get very interesting.

I'd recommend you to look up a few games between AlphaGo and Stockfish, which embody these two different philosophies and battle it to the teeth and bones. The matches are brilliant. I would say though that it seems like AlphaGo (learning the game entirely through scratch without human help) has seemed to triumph more times than Stockfish and with the nature of these systems, I'd suspect it to continue that trend.

<comment><USER6> I'm not sure it's right to characterise Deep Blue or Stockfish as repositories of human chess theory. Fundamentally they were all based on a relatively simplistic function for calculating the value of a board position combined with the ability to evaluate more board positions further into the future than any human possibly could (plus a database of opening moves). That approach seems thoroughly non-human, and represents a victory of tactical accuracy over chess theory or strategy.

However I agree that the games between AlphaGo and Stockfish are really interesting. It strikes me that the AlphaGo version of chess looks a lot more human; it seems to place value on strategic ideas (activity, tempo, freedom of movement) that any human player would recognise.

<comment><USER5> I think you're right, I meant to say that chess engines usually have book openings built into them which derive off of human chess theory but you're absolutely right in that they don't play in a human form.

It's kind of crazy how AlphaZero has managed the success it has. Stockfish calculates roughly 60 million moves per second and AlphaZero calculates at only 60 thousand per second. Three orders of magnitude less yet its brilliance is mesmerizing, tearing Stockfish apart in certain matches.

</comment></comment></comment></comment></comment></comment><comment><USER7> Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it's a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would pay Google to use them. I understand it cost a lot, but I'm not convinced by the headline figure of $36M.

<comment><USER8> The article actually addresses that. The precise number is not the point but the ballpark is:

"In terms of actual cost to DeepMind (a subsidiary of Google's parent company) to run the experiment, there are other factors that need to be taken into account, such as researcher salaries, or that the quoted TPU rate probably includes a healthy amount of margin. But for someone outside Google, this number is a good ballpark estimate of how much it would cost to replicate this experiment."

<comment><USER7> KataGo and Leela Zero and all the other AIs certainly didn't cost that much (the people running them wouldn't have had that much money and resources) and are probably stronger than Alpha Go Zero. I don't think it's at all fair to say _this number is a good ballpark estimate of how much it would cost to replicate this experiment_. It's wrong as a calculation of Google's costs, it's wrong per the title _How much did AlphaGo Zero cost_ , and it's also wrong as an estimate of the cost of replication.

<comment><USER9> > KataGo and Leela Zero and all the other AIs certainly didn't cost that much

And you base that claim on what exactly? Leela Go is trained by the community, which donates self-play resources. Just because you outsource your cost to volunteers doesn't mean it's free!

In order to get to a realistic estimation, you'd need to get the average cost for electricity, hardware cost (proportionate to use), and of course opportunity costs.

Since you cannot do that, I'd argue that you have no clue what the true training cost of these projects compared to on-demand/cloud costs really are.

<comment><USER7> Go is a small community, and there's just no chance they managed to donate $36M worth of anything, whatever the electricity and hardware costs, that's just too much money. KataGo's page says it was developed using resources donated by Jane Street, the company that its developer worked for; the general magnitude of numbers is also way off: sure they're a quantitative trading firm, but it's implausible that they'd donate $36M to develop a go AI.

<comment><USER9> The cost estimate was based on what what you and I would have to pay if we were to train AlphaGo Zero in 40 days on the given hardware using the reported number of games and resources.

Just replace the self-play TPU resources with commodity hardware or even just cheaper GPU compute providers and you'd reduce cost 10-fold by just not using TPUs. Same goes for the number of self-play games.

That still doesn't change the estimate itself. IF the other projects would've used Google TPUs, they'd well have been around the same cost as the estimate.

I really don't understand what you're trying to argue against here.

</comment></comment></comment></comment></comment></comment><comment> tech-historian Achieving a new breakthrough in computing is often very expensive. Deep Blue is estimated to cost IBM over $100 million over a decade [1].

And in comparison to large tech company R&D budgets, the amount cited in the article is a drop in the bucket. Consider the fact that Google spent $26 billion in R&D budget in 2019 alone [2]. Microsoft spent almost $17 billion [3].

[1] [https://www.extremetech.com/computing/76552-project-deep- bli...](https://www.extremetech.com/computing/76552-project-deep-blitz-chess- pc-takes-on-deep-blue/2)

[2] [https://www.statista.com/statistics/507858/alphabet- google-r...](https://www.statista.com/statistics/507858/alphabet-google-rd- costs/)

[3] [https://www.statista.com/statistics/267806/expenditure-on- re...](https://www.statista.com/statistics/267806/expenditure-on-research-and- development-by-the-microsoft-corporation/)

<comment><USER10> Note that everything in software development is R&D. Building (not operating) Gmail and Android and Office and Azure are R&D.

<comment><USER11> Right. That's the Development part of Research and Development.

<comment><USER12> And they get tax breaks, so we are motivated to class as much as possible as R&D.

</comment></comment></comment></comment><comment><USER13> Another way of thinking about how efficient the brain is: By the article's numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion.

A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours.

In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of room for improvement.

<comment><USER14> This comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.

<comment><USER15> That's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.

<comment><USER16> I believe that there's a reasonable (but unprovable) assumption that any games which humans usually play - because the rulesets are learnable and interesting for humans - implicitly rely on priors of human brains and behavior.

The space of possible games is huge (infinite?), but only a tiny subset of these games could reasonably become a popular game for humans.

E.g. it's not an arbitrary random coincidence that the scoring rules for each grid intersection in go are the same (I mean, it could vary in an arbitrary pattern), it ensures that the ruleset is small enough so that humans can learn it.

It's not an arbitrary random coincidence that the playing of go involves pattern recognition on some level, since that's what we're good at and find interesting in many games.

It's not an arbitrary random coincidence that in Mario game after jumping the sprite falls back down eventually; that's reusing the priors from real world physics.

<comment><USER15> Games are designed to be fun and playable by humans, this doesn't seem surprising

</comment></comment></comment></comment></comment><comment><USER17> I might be wrong, but I think this cost calculation is way off:

Their running cost estimate of a single TPU in a machine with 4 "TPUs" is based off the price of a cloud TPU v2-8, but a v2-8 is actually 4 ASICS on 1 board.

Also, because of the date of publication being around the time v2s were announced, and the fact that the TPU is only used for inference and GPU is used for training, I think self play was likely done on TPU v1s, which use 5x less power per ASIC and so are likely much cheaper

I also think the way they calculated the number of TPUs required is wrong, it looks like they assume 1 machine with 4 TPUs makes 1 move in 0.4 seconds, but since making 1 move only requires a forwards pass through a moderately sized CNN with 19x19(tiny) input, 1 TPU should be able to make thousands of moves in parallel per second.

<comment><USER18> Making one move requires 1600 MCTS playouts to explore the game tree, so it's a 1600-1 correspondence of "forward pass" and "move played".

</comment></comment><comment><USER19> Alpha Go Zero*, which was trained from scratch, without human games.

I've also heard rumors that AlphaStar ([https://deepmind.com/blog/article/alphastar-mastering- real-t...](https://deepmind.com/blog/article/alphastar-mastering-real-time- strategy-game-starcraft-ii)) was essentially put on hold because it was too expensive to improve/train. The bot wasn't able to beat StarCraft champions and _only_ got to a grandmaster level.

<comment><USER20> Alpha Go Zero used a combination of deep learning within a classical AI Monte Carlo simulation framework. Without a similarly effective framework its not surprising that Alpha Star wasn't able to achieve similar success. I've watched a lot of Alpha Star replays and the lapses in forethought glare through pretty regularly, though other aspects of its judgement (not just reflexes!) seem frankly super human.

<comment><USER21> Why was MCTS (or some search variant) not used in alphastar ?

(Sure, u need to somehow roll forward and rollback the StarCraft world, but for Atari using MCTS was shown to be an order of magnitude more efficient )

I have also seen comments that the search width is too large, or maybe academic purity consideration?

</comment></comment></comment><comment><USER22> The amount was removed from the submission title, which sucks if you're like me and don't like to visit yet another possibly JS-heavy site and drain your battery.

For others: It's $36M.

<comment><USER23> Are you working off one solar panel on the way to Mars? First time I'm hearing battery drain as a reason to not visit a js site.

<comment><USER9> Phones. F'in phones man :D

</comment></comment></comment><comment><USER24> I'll quibble with a little bit of this.

"AlphaGo Zero showed the world that it is possible to build systems to teach themselves to do complicated tasks."

It didn't do any such thing. The game of go has a huge number of potential moves and outcomes, but the rules themselves are trivial, the board position can be measured in a handful of bytes and gameplay always and only progresses in one direction. And judging a good vs bad outcome is just a matter of comparing two numbers.

Go is challenging and interesting for humans, but it's not remotely as "complicated" as driving a car or translating a language.

</comment><comment><USER25> It's a shame that this 'Next big thing' is the complete opposite of the internet. Instead of opening up the world for anyone to create things, letting small companies compete with large, it is only going to concentrate power with the richest companies and leave small companies unable to get involved.

<comment><USER26> "Well sure, AlphaZero looks impressive but I predict that within one hundred years, deep learning systems will be twice as powerful, ten thousand times larger, and so expensive that only the five richest kings of Europe will own them."

</comment></comment><comment><USER27> I wonder if the code and network weights will ever see the light of day. I wonder what the eventual value proposition of working on this sort of stuff is. I suppose they are just going to try to apply the algorithms to better things.

I've been interested in the application of AlphaZero to chess. It's sad that this many resources were devoted to something which we can't even use to play chess as of now. Leela (the open source reengineer) is really strong, but the crushing results presented in the AlphaZero paper never materialized. And this article just shows how hard they are to replicate.

<comment><USER28> >>" I wonder what the eventual value proposition of working on this sort of stuff is."

It seems to me that, if you only take it as a marketing operation, it has been already very valuable.

<comment><USER29> I'd really love to know how big the marketing impact on IBM Watson from this was. Somehow they managed to put the idea that "Watson is the best available AI" so well in the heads of then general population that 50%+ of my non-tech friends somehow thought that Watson beat the Go pros when I was talking with them about the topic when it was in the news.

</comment></comment></comment><comment><USER30> > Each move during self-play uses about 0.4 seconds of computer thinking time.

> Over 72 hours, 4.9 million matches were played.

One of this claim must be incorrect or misinterpreted, I highly doubt they used so many TPU's as the article claims. That would be not only impractical but also it would raise a lot of other issues like networking, disk speed... etc...

My statement is not against this article, if anyone can confirm they used so many TPUs in parallel feel free to post it

<comment><USER31> 72 hours are 259200 seconds.

Playing 4.9 million matches of ~100 plies each at 0.4 seconds per ply is 196000000 seconds.

That's < 1000 TPUs. Sounds big but not too-large-for-google big. But other comments here say that the 0.4 second number is also wrong (and in fact significantly lower).

</comment></comment><comment><USER32> The (synchronous) 0.4s per move number is misleading (and wrong), that's not what the paper is saying. The "footnote 1" of the article is wrong.

</comment><comment><USER33> Our main compute doesn't go towards machine learning, but we do rely heavily on GPU power. I recently had to come up with the figures for us to invest in an expansion of our compute power, and it turned out that buying the machines ourselves would be cheaper than renting them from Google in 3-4 months.

We don't run on those fancy V100 cards though, just regular old gaming cards suffice, and I suppose if we bought the "industrial" nvidia versions it would a take a bit longer to recoup, but still definitely within the year.

Anyway what I'm saying is that it's probably possible to to this a lot cheaper than 36M, though maybe not in such a short time. Our startup is extremely cash intensive, and I bet machine learning companies are as well (I suppose machine learning experts aren't cheap ;)), so if we can put in some work and safe a big portion off our hardware costs that really goes the distance.

<comment><USER34> How Google started out: Run servers from a shack, using commodity PC hardware

How modern startups start out: Spend $50,000/month to run hundreds of microservices on a managed Kubernetes cluster

<comment><USER35> Which of the modern startup founders have gotten into the Stanford PhD program as teenagers? Those people are extremely rare and there is far too much VC floating around to focus only on such talent. Most founders, while still outstanding, are not from that class of person. Also, due to the larger sums of money being thrown around, people can just do those things now and spend their time growing their business instead of optimizing it. In the VC funded environment, optimizing businesses lose out to competitors which grow the fastest.

</comment></comment></comment><comment><USER36> Now, consider that this is the cost of the _final_ model reported in the paper. This doesn't account for all the iterations of trying out e.g. different model architectures, hyperparameter sweeps, etc. The true cost of the experimentation is likely at least an order of magnitude higher.

</comment><comment><USER37> Does Sarbanes Oxley apply to zero rating ML costs? Alpha go might have unfair kyu ranking, if Google don't have to "pay" to acquire rank. (95% joking)

</comment><comment><USER38> > The power consumption of the experiment is equivalent to 12,760 human brains > running continuously.

Given the experiment lasts for just days, this actually sounds pretty impressive I think.

Many humans studied the game for a big portion of their lives in order to get Go knowledge where it is.

</comment><comment><USER39> I'd like to see an AI play Monopoly (the board game) against CEOs of large companies.

</comment><comment><USER40> I wonder if there's some kind of software that takes an advantage of an AI to teach non-beginner players Go. E.g. you could play against the bot and then the AI would translate your mistakes into what you can improve upon.

</comment><comment><USER41> This is a lot of money.

However, if you want to reliably make an AI the best in the world at a range of complicated tasks, can you reasonably expect this to be cheap?

</comment><comment><USER42> Wondering when researchers will switch from "race to the moon" mode to looking at better optimization techniques instead of just throwing money at the problem.

I know some companies are doing that, but I think looking at AlphaGo or AGZ and making it go faster should be an interesting problem in itself.

<comment><USER43> KataGo optimized AlphaZero and achieved 50x compute reduction.

[https://arxiv.org/abs/1902.10565](https://arxiv.org/abs/1902.10565)

</comment></comment><comment><USER44> The interesting part to me, rather than cost, is Energy usage.

>The power consumption of the experiment is equivalent to 12,760 human brains running continuously.

But the problem is this "brains" unit on AlphaZero doesn't seems to take into account of GPU, CPU and Memory involved. It only took the TPU numbers.

Then there is another problem.

> a TPU consumes about 40 watts,[1]

The TPU referred to was a first Gen TPU built on 28nm running at 40W, more like a proof of concept. Currently Google is with Cloud TPU v3 [2], The latest-generation Cloud TPU v3 Pods are liquid-cooled for maximum performance. And each TPU v3 is actually a four chip module. [3]. If a single chip is 100W that is 400W per TPU.

Edit: Turns out Wiki list TPU v3 as 250W. [4]. Not sure if that is 250W per chip or 250W for 4 Chips.

That is on the assumption they are very high powered and hence would require liquid cooling. Although that might not always be the case.

So adding CPU, GPU, Memory, and TPU figures. That original estimate of 12,760 human brains may be off by a factor of 10 if not more.

Still pretty impressive. Considering we now only get about 1.8x improvement with each generation node. We would get about 19x by 2030. ( Assuming the same algorithm ). Which means AI is good, but human brain on its own is still very much magical in its efficiency :)

Correct me If I am wrong on the numbers.

My other questions is, that was how much energy it used to learn Go. But what about energy it used during the Game?

How would AlphaGo Zero perform if it was limited to 20W?

[1] [https://cloud.google.com/blog/products/gcp/an-in-depth- look-...](https://cloud.google.com/blog/products/gcp/an-in-depth-look-at- googles-first-tensor-processing-unit-tpu)

[2] [https://cloud.google.com/blog/products/ai-machine- learning/g...](https://cloud.google.com/blog/products/ai-machine- learning/googles-scalable-supercomputers-for-machine-learning-cloud-tpu-pods- are-now-publicly-available-in-beta)

[3] [https://techcrunch.com/2019/05/07/googles-newest-cloud- tpu-p...](https://techcrunch.com/2019/05/07/googles-newest-cloud-tpu-pods- feature-over-1000-tpus/)

[4] [https://en.wikipedia.org/wiki/Tensor_processing_unit](https://en.wikipedia.org/wiki/Tensor_processing_unit)

</comment><comment><USER45> I would be more interested in how much LCZero cost?

</comment><comment><USER46> the article doesn't seem to consider cost of hyperparameter optimization prior to the final training...

</comment><comment><USER47> Ahem, "one time cost"

</comment><comment><USER48> A: $400M—to acquire DeepMind

</comment><comment><USER49> You need 35$m to beat the best Stockfish engine which can work on a small computer. Who won?

</comment><comment><USER34> It is estimated to be 36 million for _someone else_ to train AlphaGo Zero, assuming they use Google TPU instances and pay the sticker price.

Google isn't operating with that cost, unless we assume that they are prioritizing AlphaGo to the point where they lose such customers 100% of the time.

It's way more likely that AlphaGo is trained on spare time, the cost for the hardware is sunk anyway, so only the cost for upkeep is real.

<comment><USER50> > It's way more likely that AlphaGo is trained on spare time, the cost for the > hardware is sunk anyway, so only the cost for upkeep is real.

Not quite, power is quite expensive and basically all modern computers use far less power at idle than going full bore saturated with multiply-add instructions and perfect memory streaming.

Having said that, I agree that there is a substantial cost efficiency gain if they can schedule it during periods of inactivity.

</comment></comment><comment><USER51> > This accomplishment is truly remarkable in that it shows that we can develop > systems that teach themselves to do non-trivial tasks from a blank slate, > and eventually become better than humans at doing the task.

"non-trivial" is a bit of a red herring here. Playing go is pretty trivial compared to something like walking or scratching your face. Winning go may be non-trivial compared to those in some ways but it is very trivial in comparison in other ways.

<comment><USER16> Playing go _at that level_ is non-trivial compared to walking because (a) most humans can walk, but not even the best human go masters can play go at that level; (b) we had algorithms that allow bipedal robots to walk long before we had algorithms for playing go at that level.

<comment><USER52> Do we have algorithms that allow bipedal robots to walk _at human level_? Or run at, let's say, 10th grade standard student level?

<comment><USER16> Bipedal walking and running on uneven terrain has been demonstrated years ago - Boston Dynamics has a lot of nice videos; e.g. here's a four year old video on walking [https://www.youtube.com/watch?v=rVlhMGQgDkY](https://www.youtube.com/watch?v=rVlhMGQgDkY) , here's two year old videos on running [https://www.youtube.com/watch?v=vjSohj- Iclc](https://www.youtube.com/watch?v=vjSohj-Iclc) and [https://www.youtube.com/watch?v=LikxFZZO2sk](https://www.youtube.com/watch?v=LikxFZZO2sk) \- perhaps it's debatable if it's "full human level" but it seems sufficient to me for most purposes. IMHO any improvements to that would be mostly for sensors (seeing what terrain is there even in poor conditions e.g. fog/rain/snow) and modelling environment (understanding which objects might break or slide if you step on them), not walking/running as such.

The main problem for bipedal robots that makes them still impractical is the hardware expense (wheels are simpler and cheaper) and the power supply required, so for most use cases it's more efficient to use something other than a bipedal robot and there's limited business application and future revenue in scaling up research demos of bipedal walking to practicality, so most people who are working on walking algorithms are doing so in simulated virtual environments (where we have algorithms that can learn walking and running "from scratch" through experimentation) and not building very expensive hardware.

<comment><USER53> > it seems sufficient to me for most purposes.

Current self driving car technology is sufficient for most purposes, except to actually drive on roads. So for those walking robots, can they run or even walk through a crowd without hitting people? A normal 15 year old human can do it, and that is the level you need to be to release it among people.

<comment><USER54> I'll be impressed when they can finish an orienteering course.

</comment></comment></comment></comment></comment></comment><comment><USER55> sorry if it is off topic but I want to learn alpha zero from beginning , I do have little understanding of Machine & deep learning including vision recognition. Unfortunately I don't able to understand how monto Carlo tree is used for decision making. where I can start, what shall I learn so that I can learn alpha go (or OpenAI Five - Dota 2 bit).

thanks

</comment><comment><USER56> Misleading numbers, and wrong calculations. The TPU and CPU cost them almost nothing as they use and build them anyway, and renting them out for this PR stunt just cost them the missed rental time, if customers would really pay that much. Maybe around 20.000. Energy cost? I don't see much additional costs as those machines run all the time, regardless if improving the model or doing something else.

I bet the much higher cost was the PR team, including the film team, press support, TV team, travels, inviting the expert Go players, building the stage, and such. Estimated 100.000.

Not counting the man hours, they were just doing their normal job.

<comment><USER57> "renting them out for this PR stunt just cost them the missed rental time"

Because renting them out generates no revenue, right?

"Maybe around 20.000"

At least the article used a formula for the calculation. You just picked a number at random.

<comment><USER56> You need actual customers paying that much, not hypothetical ones. That was my educated guess.

And the title was "His much did it cost" not how much it would cost.</comment></comment></comment>


The CPU C-States Power Saving Modes - <AUTHOR> http://www.hardwaresecrets.com/everything-you-need-to-know-about-the-cpu-c-states-power-saving-modes/

<comment><USER2> something interesting with APIC and power-states was with tick-less kernel starting 2.6.18 and only win8 in NT.

[http://lwn.net/Articles/223185/](http://lwn.net/Articles/223185/)

</comment><comment><USER3> From 2008. Are there any newer states in addition those listed?

<comment><USER4> Haswell introduced C7-C10, as described in a slide here: [http://www.techpowerup.com/175374/haswell-ult-processors- cou...](http://www.techpowerup.com/175374/haswell-ult-processors-could- use-24-mhz-bclk-new-c-states-and-mcm-to-cut-power-draw.html)</comment></comment>


Is There a Fourth Neutrino Out There in the Universe? - <AUTHOR> https://medium.com/starts-with-a-bang/is-there-really-a-fourth-neutrino-out-there-in-the-universe-6fa8ff7f1e78 <comment><USER2> I thought Joachim Kopp's article at [https://physics.aps.org/articles/v11/122](https://physics.aps.org/articles/v11/122) was an even better explanation in some ways. It was buried as a link in the middle of the medium.com article.

<comment><AUTHOR> I found Kopp's article easier to follow after reading Siegel's, myself. YMMV

<comment><USER3> YNMO

<comment><USER4> > _YNMO_

What?

<comment><USER3> Your Neutrino May Oscillate — physics humour

<comment><USER4> I'll believe you, but a google search for that phrase brings up nothing.

<comment><USER5> Well, neutrino detection _is_ hard...

Right. I'll show myself out.

<comment><USER3> _> I'll show myself out._

We hadn't even confirmed you'd arrived.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I wonder if solar neutrino detection could be done on a satellite heading towards the sun. Seems like the signals may vary over distance from the sun (guessing here) perhaps providing some insight.

<comment><USER7> The problem with that at present is our only way to detect neutrinos is by using loads of mass, usually ice, water, or xenon. It's going to be tough to launch a glacier or a vast tank of xenon to make up the body of a detector. We'd need some novel form of detection that isn't dependent on the occasional reaction of a neutrino as it passes theough a huge body of something. Sadly, that isn't on the horizon and would ironically require a better understanding of neutrino interactions before it could be done, if it could be done.

<comment><USER8> Even worse, most neutrino detectors rely on being under thick masses of rock to filter out things like cosmic rays, which would otherwise completely flood the detectors.

<comment><USER9> So what you're saying is we need to capture an icy asteroid and stick a probe in it?

<comment><USER10> The ice would need to be very clear, surrounded by a grid of detectors, and all of that surrounded by a minimum of several hundreds of metres of rock on all sides.

The reason it needs to be clear is the detector watches for flashes of Cherenkov radiation.

<comment><USER11> cough .... IceCube .... cough

<comment><USER12> IceCube uses the entire earth as a shield. Basically only neutrinos going up through the detector is considered.

<comment><USER11> I guess that depends on how you define 'up' (are you looking at the entire planet or standing on the pole)

As I understand it these days they grab all samples and are now able to distinguish 'downward' coming neutrinos from cosmic rays in software

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER13> Please note: Unrelated to article content.

[https://i.imgur.com/htxpRh7.jpg](https://i.imgur.com/htxpRh7.jpg)

Medium's viewport is now about an inch tall on my phone, unless I carefully click 3 different 1sqmm X's. That's... Gross.

<comment><USER14> Yeah. I have the sticky header bookmarklet saved first in my bookmarks so I can hit it whenever I browse Medium on mobile.

<comment><USER15> This sounds useful, would you mind sharing?

<comment><USER16> javascript:(function()%7B(function () %7Bvar i%2C elements %3D document.querySelectorAll('body *')%3Bfor (i %3D 0%3B i < elements.length%3B i%2B%2B) %7Bif (getComputedStyle(elements%5Bi%5D).position %3D%3D%3D 'fixed') %7Belements%5Bi%5D.parentNode.removeChild(elements%5Bi%5D)%3B%7D%7D%7D)()%7D)()</comment></comment></comment></comment>


Full-body reading - <AUTHOR> https://aeon.co/essays/how-a-medieval-mystic-was-the-first-creator-of-fanfiction <comment><USER2> So, the point of the article seems to be that "Full Body reading" is a way of reading in which one is so immersed in the story that one can vividly imagine being involved in an intimate way in the story; that itself creates other stories (fan-fiction). Is that what the author is talking about or did I miss something?

On a side note, complete immersion is what makes books like Harry Potter and LOTR so gratifying to read (at least for me). They are believable enough that you just slide into that world.

<comment><USER3> I think you nailed it, but I'd add that it was also about how 'literary criticism', at least the kind taught in the school(s) she went to, seems to try to avoid all that 'emotional' stuff for a purely intellectual analysis of the text.

So she felt like she 'wasn't supposed' to do that 'complete immersion' think that you (and I!) like so much.

</comment></comment><comment><USER3> I think this shows, to me, why I like Michael Dirda's book reviews for the Washington Post so much.

He understands the criticism ("mind" part, I guess) of literature, and can help me understand that side of things.

But he _loves_ a good book, and can really communicate that "full body" sense of how wonderful reading can be, when you really lose yourself in a good book.

Gee, maybe I should tell him that I appreciate his work :-)

</comment><comment><USER4> In _The Bestseller Code_ , Archer and Jockers touch on this to explain why literary criticism is out of touch with popular reading. They mention _Uses of Literature_ by Rita Felski as as a "manifesto" of sorts about how criticism could take it more seriously.

I haven't completed it, but the sample from Amazon looks promising.

</comment><comment><USER5> Some take-aways and wonderings:

Margery Kempe (circa 1550) was long dismissed/diagnosed with various mental illnesses. Nowadays, (with maybe more seeking than ever for psychidelic experiences) it seems she had something interesting to say.

If you like to read and are heading to grad school, be careful not to lose the joy.

Probably people of all descriptions could enjoy a shift away from how hetero romances dominate mainstream storytelling.

On the plus side, it sounds like there's starting to be more of a place at the serious/critical table for people interested in exploring the bodily delights of reading.

How would it upend "religion" if more people went to the bible and other "serious" texts looking more for visceral, bodily feeling rather than doctrine/dry facts?

<comment><USER6> "people of all descriptions could enjoy a shift away from how hetero romances dominate mainstream storytelling"

I wouldn't ever expect this to happen. The super majority are heterosexual and gravitate to the relateable and familiar. Nothing short of everyone turning homosexual will change this fact.</comment></comment>


The CTO Journey at a Small Startup - <AUTHOR> https://zapier.com/engineering/startup-cto/ <comment><USER3> C suite member still doing coding 80% of the time is extremely unlikely in my experience. You may still be able to steal away some time on nights and weekends to be able to stay sharp but day to day you're much more concerned with worrying about the future of the business, interacting with customers, and often acting as a service provider to other parts of the management team (vetting out partnerships/acquisitions etc) from a technology perspective the same way in house counsel and CFOs do for legal and financial matters.

What this seems to be calling CTO is more akin to a most senior engineer/fellow/hacker. I've seen it called Chief Engineer before. That's the person the the CTO should be able to hold their own in a conversation with but being that person would seem be unlikely for an exec team member as the business grows.

*Titles are more to less meaningless unless there is internal conflict or you're interacting with someone external, ignore that bit and think in terms of roles

<comment><USER4> > C suite member still doing coding 80% of the time is extremely unlikely in > my experience.

That depends on what you mean by "small startup". In a VC-backed Silicon Valley startup where you're hiring fast and aiming for massive growth in as short a time as possible? You're probably right. Small startup that's self- funded or close to it with a tiny team, like what happens in most of the rest of the world? Yeah, you're probably rolling up your sleeves and getting your hands dirty for quite a while to come, because the code that you're selling has to come from _somewhere_.

Different startups have different constraints. It sounds like you're used to the SV-style startup where the constraint is time, rather than money. A lot of businesses have their most senior technical staff write code because they don't have the cash to hire more developers but their product still needs to be built.

<comment><USER5> I think the confusion is semantics. Most people think of a startup as a growth business. A startup is different from a small business. Whereas a small business may aim for $10M in revenue per year in 5 years (and maintain that), a startup may aim for $1B in revenue per year in 5 years.

So, if you're the CTO in your second year of business and still writing most of the code, you're probably a small business.

That said, in my experience, if you're a CTO and not able to at least understand the source code, you're a liability. I've worked for CTOs like that, and they were unable to manage risk or estimate beyond random guesses. And, it's unlikely that such a person can forecast or expand business safely without at least some trusted advisors.

<comment><USER6> Do most people really think that?

I think Paul Graham uses the term "scalable". But I have seen talking points from lots of big VCs and other figures in business who seem to think it's more about age, profitability, and management structure that define a startup.

To focus on the scalability point, I tend to agree that many only think of scalable businesses as startups, but I think the predicted numbers don't have to be in the billions to be considered a scalable growth business.

If you are a one man shop building an innovative product and you raised seed money from friends and family and hire 2 employees and expect to start making $1m a year as a 3 man shop in a few years, I think most people would call that a startup.

On the other hand, if you raise funds and build just another restaurant on the corner but with no innovation in either product or scale, then I think you are a small business.

<comment><USER5> You made me curious, so I did some quick investigation on the etymology of the word. :) (sorry, insomnia)

Using Google NGram viewer, I looked up "startup" and quickly noticed that the word (in modern usage) seemed to be short for "startup costs". Interestingly, "startup costs" seemed to only come into use after the 60s and 70s and was often connected to financial instruments. So, I also plotted the word "options" and was surprised that the shape of that curve was the same as the curve for "startup". They just had different magnitudes.

I also learned that stock options became tradable instruments in the 70s. And, also, startups began being formed in the 70s. That is, they seemed to go hand in hand. In particular "startup" was used to describe speculative research and development ventures with high capital requirements.

More recently though, the word "startup" became being associated with small teams, minimum viable products, and micro seed funding. So, that could be what you mean.

</comment></comment></comment></comment></comment><comment><USER7> I was a consultant for a bunch of startups for 4-5 years, selling my services as a "CTO for hire" to them, and was full-time CTO in multiple startups to boot.

I had a general line/rule of thumb. At < 6 engineers, you had to write code regularly as the team was so small it couldn't carry the weight of a member of the tech team who didn't commit regularly.

At 12+ engineers, you didn't have time to in order to do the other work (management, prioritisation, reports, strategic thinking, etc.), well enough.

At 6-12 engineers (where I spent most of my time), I didn't have time to write code, but had to in order to keep the company moving. Cue 60-100 hour weeks for 10 years. Yeah.

I went to quite a few CTO events, and in all honesty, it was a surprise to many of them that I both knew how to code and that actually spent any time doing it. I thought it was insane that there are CTOs - many of them - <USER2> aren't interested in the practice of creating technology at a hands-on level, but I could also understand how that happened: in my location (London), it's quite normal for non-tech CTOs to pick up from founding CTOs after a few years.

It's a weird situation to be in, and eventually a couple of years ago I decided to evaluate what I wanted and wrote a list of what I liked and didn't like about my job as a CTO.

I realised all the things I enjoyed were actually the responsibilities of a senior engineer, and all the things I didn't like were the management and board duties of being a CTO. Slept on it for a week, resigned, applied for senior roles, and generally am much happier (2 years on).

It's worth really thinking about what you want from the role. If you're a co- founder, you can shape it, but you have responsibilities to your investors, wider board, exec team, managers and developers. Most importantly, your have responsibilities to yourself.

Choose your own adventure when it comes to being a CTO, but choose wisely and carefully.

<comment><USER8> What is a non-tech CTO?

<comment><USER9> One previous CTO of mine frequently told us he hadn't written a line of code since the late 1970s, and even then he hadn't done much. He didn't know what an SSH key was, didn't know the difference between FTP and HTTP, and hadn't heard of MySQL (this was mid 2000s).

He had two excellent abilities: convincing the board to give more money to IT, and hiring good people to spend the money wisely.

<comment><USER10> That's key to mgmt though it can be hard to hire people when you don't have the skills to judge.

<comment><USER11> Its hard to hire pepple you do have the skills to judge.

</comment></comment></comment></comment></comment><comment><USER12> Primary goal of CTO's is to make sure that the technology serves the business. As C-suite, you are also involved in setting the company strategy and are responsible for building the technical organization to own it.

Outside of executing the core strategy you can take an opportunistic approach to:

1/ create more strategic options for the company

2/ cut costs by automating or re-engineering business processes

3/ deliver an unfair technical advantage over competitors

4/ improve reliability of service

5/ introduce more technology in the rest of the business (sales, marketing, operations, ..)

Startup CTO's tend to combine many different roles as there are more roles than people to fulfill them. Generally startup CTO's wind up also doing product management, engineering management (people, culture), recruitment, SCRUM master, IT, support, BI, architecting and programming.

What you actually wind up doing depends on the needs of the business and the available talent in the company to delegate these roles to.

</comment><comment><USER13> The split between VPE and CTO is that a VPE is _operational_ and _internal- facing_ whereas a CTO is _strategic_ and _external-facing_. Neither role involves coding if your company has a dozen engineers; senior individual contributors usually have a title like Chief Scientist or Distinguished Engineer.

A CTO role is essentially that of a Technical Product Manager. What distinguishes it from a traditional TPM is that, to do the role well, you also take on the aspect of being the technical "moral authority" for the company, setting the _de facto_ engineering culture, and creating a compelling vision that goes beyond product management.

<comment><USER14> I've seen it where they hire almost 80% junior programmers and have the CTO set the tone and pace and learning environment. It worked pretty well actually, but the core technology wasn't groundbreaking, more of a relationship based business doing complex CRUD.

<comment><USER15> all the rules go out the window if you are just building a CRUD app. i have seen non technical founders outsource everything to third world countries and still be successful.

</comment></comment></comment><comment><USER16> Seems like I read a different take on CTOs / VPEs every week or two. I've been a VPE several times, and as a cofounder. I, personally, think that a company should never have a CTO until you split the role into two; a CTO & a VPE working alongside each other. In that situation, the VPE manages the team, makes sure things get done, etc. While the CTO is working with other "C-staff" planning the longer (year+ out) roadmap of the company, with a focus on Engineering.

I have struggled with these roles and names, similar to Bryan Helmig, the author of this post. I have consulted with some former bosses, that now lead engineering teams at some of the larger companies here in the bay area, and have come to the conclusion that most CTOs are really VPEs, just using the wrong title, but in the end, the title does not really matter. Since it is extremely flexible in definition.

<comment><USER17> C-levels are, first and foremost, performers. They should not be invested with real authority; they should just be the set of pretty faces that get trotted out to the media, run conference calls, give presentations, and sit on sales meetings. They are media, PR, and sales operatives.

A position in the C-suite is a nightmare for anyone who wants to do serious work, and there is no reason that people who have a primarily external focus should be invested with the authority to destroy the internals.

<comment><USER18> Weird, as a COO this doesn't accurately describe me or my peers in the slightest bit. Sounds like you had a bitter experience somewhere along the way with a bad team.

<comment><USER17> Nah, I disagree. I'm not really bitter and I haven't met a C-suite that openly acknowledged their role as performers (though they are probably aware of it). There are a lot of people with strong non-performance skills who wind up in these roles (and I have met several such persons) because they're recognized as the most important leadership roles, and most people aren't going to say "no" to that.

But by that same token, the super-high visibility of the C-suite comes with a lot of constraints that hinder the freedom of people who are not primarily interested in the dog and pony show, because there is an expectation that CxOs will have a presence at various functions, meetings, and events that are not really related to their ostensible job functions, and because every public action of a CxO is subject to intense external scrutiny, including social media, etc.

Could a CxO come on HN and state his true opinions about things? Of course they would _say_ they can, because the illusion loses its power when the curtain is drawn back, and it'd be a big story that "CxO of ImportantCo admits he's a big fat phony".

If their employer is small enough that nobody cares, then I'm sure it works fine because no one really cares. But if they get bigger, then it doesn't work fine anymore, and their history will be combed through by competitors and media alike in search of ammunition/scalps.

Really just need to get actors or media personalities for those roles so that real people can continue to be real and guide things intelligently, rather than constantly maintaining their persona and having to pull away from serious matters to attend irrelevant functions and presentations (schmoozing).

You can't really give the title "CxO" to people and not expect this kind of intrusive, restrictive, and distracting load to land on them, so it's better to compartmentalize the real tasks away from such titles.

</comment></comment></comment></comment><comment><USER19> CTO as a startup cofounder and CTO of a large (100+) company are such different roles that they should really have different names.

If you're CTO because you co-founded the company or you joined at an early stage and you just happened to be the most senior engineer at the time, you really have no idea what being CTO of a large company is like.

<comment><USER15> and thr same can be said of a CEO, COO, etc. Job titles are really meaningless in startups so who cares whether someone calls themselves a CTO or not?

</comment></comment><comment><USER20> I view the cto as the person who makes sure the 6 developers aren't using 6 different frameworks. I've worked at places where that was the case and the CEO knew nothing about coding and didn't offer much more advice than "you guys work it out". The problem is you sometimes get a dev who wants to use "insert esoteric bleeding edge tool here" and then leaves a huge mess when they leave the company 3 months later

<comment><USER21> I am literally leaving a failed company just like that, except the CTO is sticking around, and he is the one using the bleeding edge tools without collaborating with the rest of the team.

</comment></comment><comment><USER22> I don't see value in this article. The author asks a bunch of other people in high tech positions what they believe CTO should be. Unfortunately the plural of anecdote is not data.

For example "Don't innovate in the management structure." Sure if you're an average person, then don't. But some companies have (Valve, young Google) and shown outstanding results.

I'm very skeptical of reducing the CTO title to rules like this. This is garbage conversation fodder, it appeals to our weaker human side to create a facade of self-improvement but none of us are gonna remember this article in 2 months.

</comment><comment><USER23> Hello,

I'm having the same considerations and am writing about challenges CTOs of startups of different sizes face on a daily basis on [http://cto.pizza](http://cto.pizza)

The concept is simple: we talk about your growth, team and tech challenges over a pizza. Let me know of any of you might be interested in grabbing one.

I'm based in Paris but we could figure something out over Skype or something if you have interesting stories to share!

</comment><comment><USER24> Good article, but why people are calling Software Engineers Hackers these days?

</comment><comment><USER25> Pretty smart article, good introspection, matches my experience and observation. Thanks for this</comment>